Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits

Xi Liu; Ping Chun Hsieh; Yu Heng Hung; Anirban Bhattacharya; P. R. Kumar

Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits

Xi Liu^*, Ping Chun Hsieh, Yu Heng Hung, Anirban Bhattacharya, P. R. Kumar

^*Corresponding author for this work

Institute of Network Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE - a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential family, we show that RBMLE yields an index policy. To choose the bias-growth rate.

Original language	English
Title of host publication	37th International Conference on Machine Learning, ICML 2020
Editors	Hal Daume, Aarti Singh
Publisher	International Machine Learning Society (IMLS)
Pages	6204-6214
Number of pages	11
ISBN (Electronic)	9781713821120
State	Published - Jul 2020
Event	37th International Conference on Machine Learning, ICML 2020 - Virtual, Online Duration: 13 Jul 2020 → 18 Jul 2020

Publication series

Name	37th International Conference on Machine Learning, ICML 2020
Volume	PartF168147-8

Conference

Conference	37th International Conference on Machine Learning, ICML 2020
City	Virtual, Online
Period	13/07/20 → 18/07/20

Cite this

Liu, X., Hsieh, P. C., Hung, Y. H., Bhattacharya, A., & Kumar, P. R. (2020). Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits. In H. Daume, & A. Singh (Eds.), 37th International Conference on Machine Learning, ICML 2020 (pp. 6204-6214). (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-8). International Machine Learning Society (IMLS).

Liu, Xi ; Hsieh, Ping Chun ; Hung, Yu Heng et al. / Exploration through reward biasing : Reward-biased maximum likelihood estimation for stochastic multi-armed bandits. 37th International Conference on Machine Learning, ICML 2020. editor / Hal Daume ; Aarti Singh. International Machine Learning Society (IMLS), 2020. pp. 6204-6214 (37th International Conference on Machine Learning, ICML 2020).

@inproceedings{41c11c9a0dea4e0d90017f57630edecd,

title = "Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits",

abstract = "Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE - a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential family, we show that RBMLE yields an index policy. To choose the bias-growth rate.",

author = "Xi Liu and Hsieh, {Ping Chun} and Hung, {Yu Heng} and Anirban Bhattacharya and Kumar, {P. R.}",

note = "Publisher Copyright: {\textcopyright} International Conference on Machine Learning, ICML 2020. All rights reserved.; 37th International Conference on Machine Learning, ICML 2020 ; Conference date: 13-07-2020 Through 18-07-2020",

year = "2020",

month = jul,

language = "English",

series = "37th International Conference on Machine Learning, ICML 2020",

publisher = "International Machine Learning Society (IMLS)",

pages = "6204--6214",

editor = "Hal Daume and Aarti Singh",

booktitle = "37th International Conference on Machine Learning, ICML 2020",

}

Liu, X, Hsieh, PC, Hung, YH, Bhattacharya, A & Kumar, PR 2020, Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits. in H Daume & A Singh (eds), 37th International Conference on Machine Learning, ICML 2020. 37th International Conference on Machine Learning, ICML 2020, vol. PartF168147-8, International Machine Learning Society (IMLS), pp. 6204-6214, 37th International Conference on Machine Learning, ICML 2020, Virtual, Online, 13/07/20.

Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits. / Liu, Xi; Hsieh, Ping Chun; Hung, Yu Heng et al.
37th International Conference on Machine Learning, ICML 2020. ed. / Hal Daume; Aarti Singh. International Machine Learning Society (IMLS), 2020. p. 6204-6214 (37th International Conference on Machine Learning, ICML 2020; Vol. PartF168147-8).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Exploration through reward biasing

T2 - 37th International Conference on Machine Learning, ICML 2020

AU - Liu, Xi

AU - Hsieh, Ping Chun

AU - Hung, Yu Heng

AU - Bhattacharya, Anirban

AU - Kumar, P. R.

PY - 2020/7

Y1 - 2020/7

N2 - Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE - a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential family, we show that RBMLE yields an index policy. To choose the bias-growth rate.

AB - Inspired by the Reward-Biased Maximum Likelihood Estimate method of adaptive control, we propose RBMLE - a novel family of learning algorithms for stochastic multi-armed bandits (SMABs). For a broad range of SMABs including both the parametric Exponential Family as well as the non-parametric sub-Gaussian/Exponential family, we show that RBMLE yields an index policy. To choose the bias-growth rate.

UR - http://www.scopus.com/inward/record.url?scp=85098433253&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85098433253

T3 - 37th International Conference on Machine Learning, ICML 2020

SP - 6204

EP - 6214

BT - 37th International Conference on Machine Learning, ICML 2020

A2 - Daume, Hal

A2 - Singh, Aarti

PB - International Machine Learning Society (IMLS)

Y2 - 13 July 2020 through 18 July 2020

ER -

Liu X, Hsieh PC, Hung YH, Bhattacharya A, Kumar PR. Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits. In Daume H, Singh A, editors, 37th International Conference on Machine Learning, ICML 2020. International Machine Learning Society (IMLS). 2020. p. 6204-6214. (37th International Conference on Machine Learning, ICML 2020).

Exploration through reward biasing: Reward-biased maximum likelihood estimation for stochastic multi-armed bandits

Abstract

Publication series

Conference

Other files and links

Fingerprint

Cite this