A multi-agent virtual market model for generalization in reinforcement learning based trading strategies

Fei Fan He, Chiao Ting Chen, Szu Hao Huang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Many studies have successfully used reinforcement learning (RL) to train an intelligent agent that learns profitable trading strategies from financial market data. Most of RL trading studies have simplified the effect of the actions of the trading agent on the market state. The trading agent is trained to maximize long-term profit by optimizing fixed historical data. However, such approach frequently results in the trading performance during out-of-sample validation being considerably different from that during training. In this paper, we propose a multi-agent virtual market model (MVMM) comprised of multiple generative adversarial networks (GANs) which cooperate with each other to reproduce market price changes. In addition, the action of the trading agent can be superimposed on the current state as the input of the MVMM to generate an action-dependent next state. In this research, real historical data were replaced with the simulated market data generated by the MVMM. The experimental results indicated that the trading strategy of the trained RL agent achieved a 12% higher profit and exhibited low risk of loss in the 2019 China Shanghai Shenzhen 300 stock index futures backtest.

Original languageEnglish
Article number109985
JournalApplied Soft Computing
Volume134
DOIs
StatePublished - Feb 2023

Keywords

  • Market simulation
  • Multi-agent systems
  • Reinforcement learning
  • Trading strategy

Fingerprint

Dive into the research topics of 'A multi-agent virtual market model for generalization in reinforcement learning based trading strategies'. Together they form a unique fingerprint.

Cite this