Improving Generalization in Reinforcement Learning-Based Trading by Using a Generative Adversarial Market Model

Chia-Hsuan Kuo, Chiao-Ting Chen, Sin-Jing Lin, Szu-Hao Huang*

*此作品的通信作者

研究成果: Article同行評審

3 引文 斯高帕斯(Scopus)

摘要

With the increasing sophistication of artificial intelligence, reinforcement learning (RL) has been widely applied to portfolio management. However, shortcomings remain. Specifically, because the training environment of an RL-based portfolio optimization framework is usually constructed based on historical price data in the literature, the agent potentially 1) violates the definition of a Markov decision process (MDP), 2) ignores their own market impact, or 3) fails to account for causal relationships within interaction processes; these ultimately lead the agent to make poor generalizations. To surmount these problems-specifically, to help the RL-based portfolio agent make better generalizations-we introduce an interactive training environment that leverages a generative model, called the limit order book-generative adversarial model (LOB-GAN), to simulate a financial market. Specifically, the LOB-GAN models market ordering behavior, and LOB-GAN's generator is utilized as a market behavior simulator. A simulated financial market, called Virtual Market, is constructed by the market behavior simulator in conjunction with a realistic security matching system. Virtual Market is then leveraged as an interactive training environment for the RL-based portfolio agent. The experimental results demonstrate that our framework improves out-of-sample portfolio performance by 4%, which is superior to other generalization strategies.

原文English
頁(從 - 到)50738-50754
頁數17
期刊IEEE Access
9
DOIs
出版狀態Published - 三月 2021

指紋

深入研究「Improving Generalization in Reinforcement Learning-Based Trading by Using a Generative Adversarial Market Model」主題。共同形成了獨特的指紋。

引用此