Opponent Exploitation Based on Bayesian Strategy Inference and Policy Tracking

Kuei Tso Lee, Yen Yun Huang, Je Ruei Yang, Sheng Jyh Wang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

In a multi-agent competitive environment, it is important for an agent to detect the opponent's policy and adopt a suitable policy to exploit the opponent. Conventionally, most methods, e.g., Bayesian Policy Reuse (BPR) variants, assume the opponent adopts a fixed policy or a randomly changing policy. In this paper, we make a more realistic and reasonable assumption that the opponent may select its policy based on the previous observation. Here, we define the term “strategy” as the mapping from the previous observation to the opponent's selected policy, and we propose the Bayesian Strategy Inference (BSI) framework to infer the opponent's strategy. Furthermore, to deal with opponents who may randomly select their policies, the BSI framework is combined with an intra-episode policy tracking mechanism to construct the Bayesian Strategy Inference plus Policy Tracking (BSI-PT) algorithm. In our experiments, we design an extended batter vs. pitcher game (EBvPG) for the evaluation of the proposed BSI-PT framework. The experimental results demonstrate that BSI-PT obtains higher policy prediction accuracy and winning percentage than three other BPR variants against the opponents with a specific policy selection strategy, with a random selection strategy, or with a partially random strategy.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Games
DOIs
StateAccepted/In press - 2023

Keywords

  • Bayes methods
  • Bayesian Policy Reuse
  • Bayesian inference
  • Business process re-engineering
  • Games
  • Libraries
  • Multi-agent environment
  • Policy tracking
  • Probability distribution
  • Switches
  • Task analysis

Fingerprint

Dive into the research topics of 'Opponent Exploitation Based on Bayesian Strategy Inference and Policy Tracking'. Together they form a unique fingerprint.

Cite this