Multi-Armed Bandits with Arriving Arms: Sequential Screening, Dynamic Regret, and Sublinear Guarantees

Deqi Zheng; Xiaoyang Xu; Yuhong Yang

arxiv: 2606.09002 · v1 · pith:R4T4DNEEnew · submitted 2026-06-08 · 📊 stat.ML · cs.LG· math.ST· stat.TH

Multi-Armed Bandits with Arriving Arms: Sequential Screening, Dynamic Regret, and Sublinear Guarantees

Deqi Zheng , Xiaoyang Xu , Yuhong Yang This is my paper

Pith reviewed 2026-06-27 15:03 UTC · model grok-4.3

classification 📊 stat.ML cs.LGmath.STstat.TH

keywords multi-armed banditsarriving armsdynamic regretUCB algorithmsequential screeningelimination methodsonline learningregret analysis

0 comments

The pith

UCB-AA uses preliminary screening of new arms to achieve arrival-dependent regret bounds and sublinear dynamic regret in expanding bandit problems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies stochastic multi-armed bandits where the arm set grows over time, as occurs in sequential experiments with new treatments becoming available. Standard regret against a fixed best arm is unsuitable, so the authors adopt a dynamic-regret benchmark against the best arm available at each time. They introduce UCB for Arriving Arms, an elimination procedure that first screens each newly arrived arm before it competes fully with existing arms. This handles the mismatch between arrival information and the drifting benchmark, producing regret bounds that scale explicitly with the arrival sequence and sublinear dynamic regret when arm gaps evolve regularly. An online variant works for unknown time horizons.

Core claim

UCB-AA is an elimination-based algorithm that adds a preliminary screening phase for each newly arrived arm before it enters full competition with incumbent arms. Under this procedure the algorithm attains regret bounds that depend explicitly on the arrival process, achieves sublinear dynamic regret when the gaps between arms satisfy stated regularity conditions on their evolution, and extends to an online version that does not require knowledge of the horizon in advance.

What carries the argument

UCB for Arriving Arms (UCB-AA): an elimination-based procedure that performs an aiding preliminary screening step on each newly arrived arm before full competition with incumbent arms.

If this is right

Regret bounds scale explicitly with the number and timing of arm arrivals rather than only with total time horizon.
Dynamic regret remains sublinear whenever the gap process between arms obeys the stated regularity conditions.
An online version of the algorithm continues to function without prior knowledge of the total horizon.
The screening step reduces the number of wasted pulls on suboptimal new arms and keeps the active arm set smaller than without screening.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same screening idea could be tested in non-stochastic arrival models where arrival times are chosen adversarially.
If the regularity conditions on gap evolution are violated, the method may need an adaptive screening threshold that depends on observed gap estimates.
The explicit dependence on the arrival process suggests that knowing or estimating the arrival rate in advance could further tighten bounds.

Load-bearing premise

Sublinear dynamic regret holds only when the gaps between arms satisfy regularity conditions on how those gaps evolve over time.

What would settle it

A simulation in which arm gaps evolve irregularly (for example, widening gaps after each new arrival) that produces linear rather than sublinear dynamic regret would falsify the sublinear guarantee.

Figures

Figures reproduced from arXiv: 2606.09002 by Deqi Zheng, Xiaoyang Xu, Yuhong Yang.

**Figure 1.** Figure 1: The rapid expansion of the decision space in precision oncology. FDA approval data for NSCLC (2003–2025). The solid line shows the cumulative number of targeted and immunotherapies (arms), from 1 in 2003 (Gefitinib) to 36 by 2025. A static algorithm fixed to the 2006 arm set would fail to incorporate most subsequently available treatments, illustrating the need for methods that accommodate arriving arms. … view at source ↗

**Figure 2.** Figure 2: Experimental results for Scenario 1 (Progressive step-up arrivals). Panel (a) shows [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗

**Figure 3.** Figure 3: Experimental results for Scenario 2 (Near-tied top set). UCB-AA’s explicit elimination [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗

**Figure 4.** Figure 4: Active arm size for two scenarios. UCB-AA’s explicit elimination mechanism maintains [PITH_FULL_IMAGE:figures/full_fig_p021_4.png] view at source ↗

read the original abstract

We study a stochastic multi-armed bandit problem in which the set of available arms expands over time. This setting arises in sequential experimentation when new actions or treatments become available during an ongoing study, making regret against a single best arm in hindsight inappropriate. We instead evaluate performance relative to the best arm currently available, leading to a dynamic-regret criterion for arriving-arm environments. To address the resulting challenges of arrival information discrepancy (AID) and a drifting benchmark (DB), we propose UCB for Arriving Arms (UCB-AA), an elimination-based procedure with an aiding preliminary screening step for newly arrived arms before full competition with incumbent arms. We show that UCB-AA attains regret bounds that depend explicitly on the arrival process, achieves sublinear dynamic regret under regularity conditions on gap evolution, and admits an online extension for unknown horizons. Simulation results show that UCB-AA reduces wasted pulls and maintains a smaller active arm set while preserving competitive regret performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The arriving-arms setting and UCB-AA screening step are a clean extension, but the sublinear dynamic regret claim depends on regularity conditions whose exact form and necessity remain unclear from the abstract.

read the letter

The main thing here is a bandit variant where arms show up over time and you compete against the best arm available at each step rather than a fixed benchmark. UCB-AA adds a screening step for new arrivals before they join the main pool, and the regret bounds are written to scale with the arrival process. That framing and the online extension for unknown horizons are the concrete additions.

The setup makes sense for sequential experiments where options expand. The elimination-plus-screening idea is straightforward and the abstract claims it keeps the active set smaller while cutting wasted pulls. The dynamic-regret formulation with arrival information discrepancy is handled explicitly, which is better than forcing the problem into standard fixed-arm analysis.

The soft spot is the sublinear guarantee. It is stated to hold only under regularity conditions on how gaps evolve, yet the abstract gives no precise statement of those conditions. If they amount to requiring that gaps stabilize after a finite number of arrivals or bound total variation tightly, the result is conditional on an assumption that may not hold when new arms keep arriving with drifting performance. Without the exact statement or the proof, it is impossible to tell whether the screening step actually delivers sublinear regret or simply recovers a standard bound once the conditions are imposed. The simulation results are mentioned at a high level only, so they do not resolve the question.

This is for people working on bandits in non-stationary or expanding environments. A reader already familiar with UCB and elimination methods will see the technical moves quickly. The paper deserves a serious referee because the setting is well-motivated and the algorithm is simple enough to check; the proofs and the precise regularity conditions are what need verification. I would send it out rather than desk-reject.

Referee Report

1 major / 0 minor

Summary. The paper studies stochastic multi-armed bandits where the arm set expands over time due to new arrivals. It proposes UCB-AA, an elimination-based algorithm that includes a preliminary screening step for newly arrived arms to handle arrival information discrepancy and a drifting benchmark. The main results claim that UCB-AA achieves regret bounds explicitly depending on the arrival process, sublinear dynamic regret under regularity conditions on gap evolution, and an online extension for unknown horizons, supported by simulations showing reduced wasted pulls and smaller active sets.

Significance. If the claims hold, the work provides a principled approach to dynamic regret in expanding-arm environments relevant to sequential experimentation. The explicit dependence of bounds on the arrival process and the screening mechanism are strengths. However, the sublinear dynamic regret result is conditional on regularity conditions whose necessity and form are central to whether the algorithm truly improves over standard UCB or merely recovers it under strong assumptions.

major comments (1)

[Abstract] Abstract: the claim of sublinear dynamic regret under 'regularity conditions on gap evolution' is load-bearing for the central contribution, yet the abstract provides no statement of these conditions (e.g., bounds on total variation, number of significant changes, or stabilization rate). Without this, it is impossible to verify whether the elimination + screening procedure actually prevents linear regret against the drifting benchmark or simply assumes away the hardest cases.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback. We address the single major comment below.

read point-by-point responses

Referee: [Abstract] Abstract: the claim of sublinear dynamic regret under 'regularity conditions on gap evolution' is load-bearing for the central contribution, yet the abstract provides no statement of these conditions (e.g., bounds on total variation, number of significant changes, or stabilization rate). Without this, it is impossible to verify whether the elimination + screening procedure actually prevents linear regret against the drifting benchmark or simply assumes away the hardest cases.

Authors: We agree that the abstract would be improved by briefly indicating the nature of the regularity conditions. The full manuscript states these conditions explicitly in the statements of Theorems 4 and 5 (sublinear total variation of the per-arm gaps, or equivalently a finite number of significant gap changes). In the revision we will append a short clause to the abstract, e.g., “under regularity conditions ensuring sublinear total variation of the gaps.” This makes clear that the sublinear dynamic-regret guarantee is obtained under verifiable, non-trivial assumptions on the drifting benchmark rather than by restricting to trivial instances. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper proposes UCB-AA as an elimination-based algorithm with screening for arriving arms, then derives regret bounds that explicitly depend on the arrival process and sublinear dynamic regret under stated regularity conditions on gap evolution. These results follow from standard algorithmic analysis of the procedure rather than any reduction to fitted parameters, self-definitions, or self-citation chains by the paper's own equations. No steps match the enumerated circularity patterns; the central claims retain independent content from the algorithm design and bound derivations, making the analysis self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; standard stochastic bandit assumptions (e.g., sub-Gaussian rewards) are implicitly used but not detailed.

pith-pipeline@v0.9.1-grok · 5708 in / 1043 out tokens · 21047 ms · 2026-06-27T15:03:14.094405+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

45 extracted references · 13 canonical work pages

[1]

1985 , issn =

Asymptotically efficient adaptive allocation rules , journal =. 1985 , issn =. doi:https://doi.org/10.1016/0196-8858(85)90002-8 , url =

work page doi:10.1016/0196-8858(85)90002-8 1985
[2]

Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence , series =

Qi, Han and Fei, Guo and Zhu, Li , title =. Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence , series =. 2024 , editor =

2024
[3]

2025 , journal =

Graph Feedback Bandits on Similar Arms: With and Without Graph Structures , author=. 2025 , journal =. 2501.14314 , archivePrefix=

arXiv 2025
[4]

Finite-time Analysis of the Multiarmed Bandit Problem , journal =

Auer, Peter and Cesa-Bianchi, Nicol. Finite-time Analysis of the Multiarmed Bandit Problem , journal =. 2002 , doi =

2002
[5]

Periodica Mathematica Hungarica , year =

Peter Auer and Ronald Ortner , title =. Periodica Mathematica Hungarica , year =. doi:10.1007/s10998-010-3055-6 , pages=

work page doi:10.1007/s10998-010-3055-6
[6]

Berry and Robert W

Donald A. Berry and Robert W. Chen and Alan Zame and David C. Heath and Larry A. Shepp , title =. The Annals of Statistics ,. 1997 , doi =

1997
[7]

Proceedings of the 22nd International Conference on Neural Information Processing Systems , pages =

Chakrabarti, Deepayan and Kumar, Ravi and Radlinski, Filip and Upfal, Eli , title =. Proceedings of the 22nd International Conference on Neural Information Processing Systems , pages =. 2008 , isbn =

2008
[8]

Machine Learning , pages =

Kleinberg, Robert and Niculescu-Mizil, Alexandru and Sharma, Yogeshwer , title =. Machine Learning , pages =. 2010 , issue_date =. doi:10.1007/s10994-010-5178-7 , abstract =

work page doi:10.1007/s10994-010-5178-7 2010
[9]

In: Proceedings of the 19th international conference on World Wide Web

Li, Lihong and Chu, Wei and Langford, John and Schapire, Robert E. , title =. 2010 , isbn =. doi:10.1145/1772690.1772758 , booktitle =

work page doi:10.1145/1772690.1772758 2010
[10]

The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits , url =

Langford, John and Zhang, Tong , booktitle =. The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits , url =
[11]

Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =

Contextual Bandits with Linear Payoff Functions , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =. 2011 , editor =

2011
[12]

Neural Contextual Bandits with

Zhou, Dongruo and Li, Lihong and Gu, Quanquan , booktitle =. Neural Contextual Bandits with. 2020 , editor =

2020
[13]

2008 , eprint=

On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems , author=. 2008 , eprint=

2008
[14]

Proceedings of the 28th International Conference on Neural Information Processing Systems , volume =

Besbes, Omar and Gur, Yonatan and Zeevi, Assaf , title =. Proceedings of the 28th International Conference on Neural Information Processing Systems , volume =. 2014 ,

2014
[15]

Foundations and Trends in Machine Learning , volume =

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , author =. Foundations and Trends in Machine Learning , volume =. 2012 , doi =

2012
[16]

2020 , doi =

Bandit Algorithms , author =. 2020 , doi =

2020
[17]

The Annals of Statistics , volume =

Batched Bandit Problems , author =. The Annals of Statistics , volume =. 2016 , doi =

2016
[18]

Narahari , keywords =

Ganesh Ghalme and Swapnil Dhamal and Shweta Jain and Sujit Gujar and Y. Narahari , keywords =. Ballooning multi-armed bandits , journal =. 2021 , issn =. doi:https://doi.org/10.1016/j.artint.2021.103485 , url =

work page doi:10.1016/j.artint.2021.103485 2021
[19]

Proceedings of the 22nd Annual Conference on Learning Theory (COLT) , year =

Audibert, Jean-Yves and Bubeck, Sébastien , title =. Proceedings of the 22nd Annual Conference on Learning Theory (COLT) , year =
[20]

Journal of Machine Learning Research , year =

Eyal Even-Dar and Shie Mannor and Yishay Mansour , title =. Journal of Machine Learning Research , year =
[21]

Burnetas and Michael N

Apostolos N. Burnetas and Michael N. Katehakis , abstract =. Optimal Adaptive Policies for Sequential Allocation Problems , journal =. 1996 , issn =. doi:https://doi.org/10.1006/aama.1996.0007 , url =

work page doi:10.1006/aama.1996.0007 1996
[22]

On the complexity of best-arm identification in multi-armed bandit models , year =

Kaufmann, Emilie and Capp\'. On the complexity of best-arm identification in multi-armed bandit models , year =. Journal of Machine Learning Research , pages =
[23]

Optimal Best Arm Identification with Fixed Confidence , booktitle =

Garivier, Aur. Optimal Best Arm Identification with Fixed Confidence , booktitle =. 2016 , editor =

2016
[24]

Proceedings of the 32nd Conference on Learning Theory , pages =

Adaptively Tracking the Best Bandit Arm with an Unknown Number of Distribution Changes , author =. Proceedings of the 32nd Conference on Learning Theory , pages =. 2019 ,

2019
[25]

Drug Approvals and Databases , year =
[26]

Robust locally weighted regression and smoothing scatterplots

Michael Woodroofe , title =. Journal of the American Statistical Association , volume =. 1979 , publisher =. doi:10.1080/01621459.1979.10481033 , URL =

work page doi:10.1080/01621459.1979.10481033 1979
[27]

The Annals of Statistics ,

Yuhong Yang and Dan Zhu , title =. The Annals of Statistics ,. 2002 , doi =

2002
[28]

Nonparametric bandits with covariates

Philippe Rigollet and Assaf Zeevi. Nonparametric bandits with covariates. Proceedings of the 23rd Annual Conference on Learning Theory (COLT) , series =. 2010

2010
[29]

The Annals of Statistics ,

Vianney Perchet and Philippe Rigollet , title =. The Annals of Statistics ,. 2013 , doi =

2013
[30]

Electronic Journal of Statistics ,

Wei Qian and Yuhong Yang , title =. Electronic Journal of Statistics ,. 2016 , doi =

2016
[31]

Response-Adaptive Randomization for Multi-arm Clinical Trials Using the Forward Looking Gittins Index Rule , journal =

Villar, Sof. Response-Adaptive Randomization for Multi-arm Clinical Trials Using the Forward Looking Gittins Index Rule , journal =. 2015 ,. doi:10.1111/biom.12337 , url =

work page doi:10.1111/biom.12337 2015
[32]

Faye and Villar, Sof

Williamson, S. Faye and Villar, Sof. A Response-Adaptive Randomization Procedure for Multi-Armed Clinical Trials with Normally Distributed Outcomes , journal =. 2020 ,. doi:10.1111/biom.13119 , url =

work page doi:10.1111/biom.13119 2020
[33]

Journal of Machine Learning Research , articleno =

Aziz, Maryam and Kaufmann, Emilie and Riviere, Marie-Karelle , title =. Journal of Machine Learning Research , articleno =. 2021 , issue_date =

2021
[34]

Bulletin of the American Mathematical Society ,

Herbert Robbins , title =. Bulletin of the American Mathematical Society ,
[35]

and Fristedt, Bert , title =

Berry, Donald A. and Fristedt, Bert , title =
[36]

Gittins, J. C. , title =. Journal of the Royal Statistical Society: Series B (Methodological) , volume =. 1979 ,. doi:10.1111/j.2517-6161.1979.tb01068.x , url =

work page doi:10.1111/j.2517-6161.1979.tb01068.x 1979
[37]

Liu , title =

Wei Qian and Ching-Kang Ing and Ji Liu , title =. Journal of the American Statistical Association , volume =. 2024 , publisher =. doi:10.1080/01621459.2022.2152343 , URL =

work page doi:10.1080/01621459.2022.2152343 2024
[38]

The Annals of Statistics ,

Tze Leung Lai , title =. The Annals of Statistics ,. 1987 , doi =

1987
[39]

Zelen , title =

M. Zelen , title =. Journal of the American Statistical Association , volume =. 1969 , publisher =. doi:10.1080/01621459.1969.10500959 , URL =

work page doi:10.1080/01621459.1969.10500959 1969
[40]

L. J. Wei and S. Durham , title =. Journal of the American Statistical Association , volume =. 1978 , publisher =. doi:10.1080/01621459.1978.10480109 , URL =

work page doi:10.1080/01621459.1978.10480109 1978
[41]

Tony Cai and Hongzhe Li , title =

Changxiao Cai and T. Tony Cai and Hongzhe Li , title =. The Annals of Statistics ,. 2024 , doi =

2024
[42]

and Thomas, Joy A

Cover, Thomas M. and Thomas, Joy A. , title =
[43]

, title =

Tsybakov, Alexandre B. , title =
[44]

, title =

Pinsker, Mark S. , title =
[45]

Bandit Algorithms , publisher =

Lattimore, Tor and Szepesv. Bandit Algorithms , publisher =

[1] [1]

1985 , issn =

Asymptotically efficient adaptive allocation rules , journal =. 1985 , issn =. doi:https://doi.org/10.1016/0196-8858(85)90002-8 , url =

work page doi:10.1016/0196-8858(85)90002-8 1985

[2] [2]

Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence , series =

Qi, Han and Fei, Guo and Zhu, Li , title =. Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence , series =. 2024 , editor =

2024

[3] [3]

2025 , journal =

Graph Feedback Bandits on Similar Arms: With and Without Graph Structures , author=. 2025 , journal =. 2501.14314 , archivePrefix=

arXiv 2025

[4] [4]

Finite-time Analysis of the Multiarmed Bandit Problem , journal =

Auer, Peter and Cesa-Bianchi, Nicol. Finite-time Analysis of the Multiarmed Bandit Problem , journal =. 2002 , doi =

2002

[5] [5]

Periodica Mathematica Hungarica , year =

Peter Auer and Ronald Ortner , title =. Periodica Mathematica Hungarica , year =. doi:10.1007/s10998-010-3055-6 , pages=

work page doi:10.1007/s10998-010-3055-6

[6] [6]

Berry and Robert W

Donald A. Berry and Robert W. Chen and Alan Zame and David C. Heath and Larry A. Shepp , title =. The Annals of Statistics ,. 1997 , doi =

1997

[7] [7]

Proceedings of the 22nd International Conference on Neural Information Processing Systems , pages =

Chakrabarti, Deepayan and Kumar, Ravi and Radlinski, Filip and Upfal, Eli , title =. Proceedings of the 22nd International Conference on Neural Information Processing Systems , pages =. 2008 , isbn =

2008

[8] [8]

Machine Learning , pages =

Kleinberg, Robert and Niculescu-Mizil, Alexandru and Sharma, Yogeshwer , title =. Machine Learning , pages =. 2010 , issue_date =. doi:10.1007/s10994-010-5178-7 , abstract =

work page doi:10.1007/s10994-010-5178-7 2010

[9] [9]

In: Proceedings of the 19th international conference on World Wide Web

Li, Lihong and Chu, Wei and Langford, John and Schapire, Robert E. , title =. 2010 , isbn =. doi:10.1145/1772690.1772758 , booktitle =

work page doi:10.1145/1772690.1772758 2010

[10] [10]

The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits , url =

Langford, John and Zhang, Tong , booktitle =. The Epoch-Greedy Algorithm for Contextual Multi-armed Bandits , url =

[11] [11]

Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =

Contextual Bandits with Linear Payoff Functions , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , pages =. 2011 , editor =

2011

[12] [12]

Neural Contextual Bandits with

Zhou, Dongruo and Li, Lihong and Gu, Quanquan , booktitle =. Neural Contextual Bandits with. 2020 , editor =

2020

[13] [13]

2008 , eprint=

On Upper-Confidence Bound Policies for Non-Stationary Bandit Problems , author=. 2008 , eprint=

2008

[14] [14]

Proceedings of the 28th International Conference on Neural Information Processing Systems , volume =

Besbes, Omar and Gur, Yonatan and Zeevi, Assaf , title =. Proceedings of the 28th International Conference on Neural Information Processing Systems , volume =. 2014 ,

2014

[15] [15]

Foundations and Trends in Machine Learning , volume =

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems , author =. Foundations and Trends in Machine Learning , volume =. 2012 , doi =

2012

[16] [16]

2020 , doi =

Bandit Algorithms , author =. 2020 , doi =

2020

[17] [17]

The Annals of Statistics , volume =

Batched Bandit Problems , author =. The Annals of Statistics , volume =. 2016 , doi =

2016

[18] [18]

Narahari , keywords =

Ganesh Ghalme and Swapnil Dhamal and Shweta Jain and Sujit Gujar and Y. Narahari , keywords =. Ballooning multi-armed bandits , journal =. 2021 , issn =. doi:https://doi.org/10.1016/j.artint.2021.103485 , url =

work page doi:10.1016/j.artint.2021.103485 2021

[19] [19]

Proceedings of the 22nd Annual Conference on Learning Theory (COLT) , year =

Audibert, Jean-Yves and Bubeck, Sébastien , title =. Proceedings of the 22nd Annual Conference on Learning Theory (COLT) , year =

[20] [20]

Journal of Machine Learning Research , year =

Eyal Even-Dar and Shie Mannor and Yishay Mansour , title =. Journal of Machine Learning Research , year =

[21] [21]

Burnetas and Michael N

Apostolos N. Burnetas and Michael N. Katehakis , abstract =. Optimal Adaptive Policies for Sequential Allocation Problems , journal =. 1996 , issn =. doi:https://doi.org/10.1006/aama.1996.0007 , url =

work page doi:10.1006/aama.1996.0007 1996

[22] [22]

On the complexity of best-arm identification in multi-armed bandit models , year =

Kaufmann, Emilie and Capp\'. On the complexity of best-arm identification in multi-armed bandit models , year =. Journal of Machine Learning Research , pages =

[23] [23]

Optimal Best Arm Identification with Fixed Confidence , booktitle =

Garivier, Aur. Optimal Best Arm Identification with Fixed Confidence , booktitle =. 2016 , editor =

2016

[24] [24]

Proceedings of the 32nd Conference on Learning Theory , pages =

Adaptively Tracking the Best Bandit Arm with an Unknown Number of Distribution Changes , author =. Proceedings of the 32nd Conference on Learning Theory , pages =. 2019 ,

2019

[25] [25]

Drug Approvals and Databases , year =

[26] [26]

Robust locally weighted regression and smoothing scatterplots

Michael Woodroofe , title =. Journal of the American Statistical Association , volume =. 1979 , publisher =. doi:10.1080/01621459.1979.10481033 , URL =

work page doi:10.1080/01621459.1979.10481033 1979

[27] [27]

The Annals of Statistics ,

Yuhong Yang and Dan Zhu , title =. The Annals of Statistics ,. 2002 , doi =

2002

[28] [28]

Nonparametric bandits with covariates

Philippe Rigollet and Assaf Zeevi. Nonparametric bandits with covariates. Proceedings of the 23rd Annual Conference on Learning Theory (COLT) , series =. 2010

2010

[29] [29]

The Annals of Statistics ,

Vianney Perchet and Philippe Rigollet , title =. The Annals of Statistics ,. 2013 , doi =

2013

[30] [30]

Electronic Journal of Statistics ,

Wei Qian and Yuhong Yang , title =. Electronic Journal of Statistics ,. 2016 , doi =

2016

[31] [31]

Response-Adaptive Randomization for Multi-arm Clinical Trials Using the Forward Looking Gittins Index Rule , journal =

Villar, Sof. Response-Adaptive Randomization for Multi-arm Clinical Trials Using the Forward Looking Gittins Index Rule , journal =. 2015 ,. doi:10.1111/biom.12337 , url =

work page doi:10.1111/biom.12337 2015

[32] [32]

Faye and Villar, Sof

Williamson, S. Faye and Villar, Sof. A Response-Adaptive Randomization Procedure for Multi-Armed Clinical Trials with Normally Distributed Outcomes , journal =. 2020 ,. doi:10.1111/biom.13119 , url =

work page doi:10.1111/biom.13119 2020

[33] [33]

Journal of Machine Learning Research , articleno =

Aziz, Maryam and Kaufmann, Emilie and Riviere, Marie-Karelle , title =. Journal of Machine Learning Research , articleno =. 2021 , issue_date =

2021

[34] [34]

Bulletin of the American Mathematical Society ,

Herbert Robbins , title =. Bulletin of the American Mathematical Society ,

[35] [35]

and Fristedt, Bert , title =

Berry, Donald A. and Fristedt, Bert , title =

[36] [36]

Gittins, J. C. , title =. Journal of the Royal Statistical Society: Series B (Methodological) , volume =. 1979 ,. doi:10.1111/j.2517-6161.1979.tb01068.x , url =

work page doi:10.1111/j.2517-6161.1979.tb01068.x 1979

[37] [37]

Liu , title =

Wei Qian and Ching-Kang Ing and Ji Liu , title =. Journal of the American Statistical Association , volume =. 2024 , publisher =. doi:10.1080/01621459.2022.2152343 , URL =

work page doi:10.1080/01621459.2022.2152343 2024

[38] [38]

The Annals of Statistics ,

Tze Leung Lai , title =. The Annals of Statistics ,. 1987 , doi =

1987

[39] [39]

Zelen , title =

M. Zelen , title =. Journal of the American Statistical Association , volume =. 1969 , publisher =. doi:10.1080/01621459.1969.10500959 , URL =

work page doi:10.1080/01621459.1969.10500959 1969

[40] [40]

L. J. Wei and S. Durham , title =. Journal of the American Statistical Association , volume =. 1978 , publisher =. doi:10.1080/01621459.1978.10480109 , URL =

work page doi:10.1080/01621459.1978.10480109 1978

[41] [41]

Tony Cai and Hongzhe Li , title =

Changxiao Cai and T. Tony Cai and Hongzhe Li , title =. The Annals of Statistics ,. 2024 , doi =

2024

[42] [42]

and Thomas, Joy A

Cover, Thomas M. and Thomas, Joy A. , title =

[43] [43]

, title =

Tsybakov, Alexandre B. , title =

[44] [44]

, title =

Pinsker, Mark S. , title =

[45] [45]

Bandit Algorithms , publisher =

Lattimore, Tor and Szepesv. Bandit Algorithms , publisher =