Improving Generalization in Reinforcement Learning-Based Trading by Using a Generative Adversarial Market Model

Chia-Hsuan Kuo, Chiao-Ting Chen, Sin-Jing Lin, Szu-Hao Huang*

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    2 Scopus citations


    With the increasing sophistication of artificial intelligence, reinforcement learning (RL) has been widely applied to portfolio management. However, shortcomings remain. Specifically, because the training environment of an RL-based portfolio optimization framework is usually constructed based on historical price data in the literature, the agent potentially 1) violates the definition of a Markov decision process (MDP), 2) ignores their own market impact, or 3) fails to account for causal relationships within interaction processes; these ultimately lead the agent to make poor generalizations. To surmount these problems-specifically, to help the RL-based portfolio agent make better generalizations-we introduce an interactive training environment that leverages a generative model, called the limit order book-generative adversarial model (LOB-GAN), to simulate a financial market. Specifically, the LOB-GAN models market ordering behavior, and LOB-GAN's generator is utilized as a market behavior simulator. A simulated financial market, called Virtual Market, is constructed by the market behavior simulator in conjunction with a realistic security matching system. Virtual Market is then leveraged as an interactive training environment for the RL-based portfolio agent. The experimental results demonstrate that our framework improves out-of-sample portfolio performance by 4%, which is superior to other generalization strategies.

    Original languageEnglish
    Pages (from-to)50738-50754
    Number of pages17
    JournalIEEE Access
    StatePublished - Mar 2021


    • Portfolios
    • Training
    • Optimization
    • Topology
    • Data models
    • Stock markets
    • Network topology
    • Artificial market simulation
    • portfolio management
    • reinforcement learning


    Dive into the research topics of 'Improving Generalization in Reinforcement Learning-Based Trading by Using a Generative Adversarial Market Model'. Together they form a unique fingerprint.

    Cite this