Pith. sign in

REVIEW 4 major objections 6 minor 93 references

Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability

T0 review · 4 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes replacing the Hoeffding-based confidence bounds in UCB bandits with conformal prediction intervals, yielding finite-sample coverage and improved regret in the small-gap regime.

desk verdict Honest, useful, and clearly written incremental work that overclaims finite-sample coverage; the regret gains in the small-gap regime are empirical and the statistical-validity guarantee is asymptotic at best, but this is a solid conditional accept rather than a reject. read the letter →

arxiv 2512.09850 v2 pith:JB43TCY4 submitted 2025-12-10 cs.LG

Simone Cuonzo , Nina Deliu This is my paper
classification cs.LG MSC 62L0562G15
keywords conformalpredictionmulti-armedbanditssmall-gapregimeregretminimizationcoverageguaranteesadaptiveinferencequantileregressionportfolioallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Conformal Bandits, a framework that replaces the Hoeffding-based confidence bounds of UCB1 with conformal prediction intervals built from conformalized quantile regression. It claims that this substitution yields bandit policies that retain the optimism principle of UCB while achieving finite-sample nominal coverage and, in the small-gap regime where arm means differ by about 1/sqrt(n), lower cumulative regret than classical UCB1. The claims are supported by Monte Carlo simulations across Gaussian, Student-t, and skewed reward distributions, and by a portfolio allocation application where a hidden Markov model identifies market regimes and the policy switches between optimistic and defensive arm selection.

What carries the argument

The central object is the arm-specific conformal prediction interval, defined by Eq. (10) as [L_{k,t+1}, U_{k,t+1}] = [ hat{q}_{k,alpha/2}(X_{t+1}) - Q_{1-alpha}(S_k), hat{q}_{k,1-alpha/2}(X_{t+1}) + Q_{1-alpha}(S_k) ], computed via conformalized quantile regression (CQR) and, in practice, adjusted by Adaptive Conformal Inference (ACI) to handle non-exchangeability. The upper bound U drives CP-UCB; the pair (U, L) drives the risk-aware CP-Bandit index (1-lambda)U - lambda|L|; and the ratio U/|L| defines the Exploratory Skewness Index used by CP-ESI. These intervals replace the data-independent Hoeffding exploration bonus with an empirically calibrated measure of predictive uncertainty.

What would settle it

A Monte Carlo study with K=3 arms, heavy-tailed rewards, and an adaptive policy whose arm-selection probabilities depend on past rewards, where the empirical coverage of the ACI-corrected interval falls below 1-alpha for the most-pulled arm by more than a few percentage points over a horizon of, say, 5,000 rounds, would indicate that the claimed finite-sample validity does not hold under the paper's own design. A counterexample where the conformal upper bound yields worse regret than Hoeffding's bound in a small-gap setting would also falsify the regret claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that in the small-gap regime, where classical UCB1 suffers linear regret and fails to achieve nominal coverage, Conformal Bandits built on conformal intervals (Eq. 10) achieve logarithmic regret and coverage close to the nominal level. Specifically, arm-specific conformal intervals give P(Y_{k,t+1} in C^{1-alpha}_{k,t+1}) >= 1-alpha for each arm and round, and the CP-UCB, CP-Bandit, and CP-ESI policies all outperform UCB1 in cumulative regret across the tested heavy-tailed and skewed reward settings. The framework further extends to risk-aware selection through the Exploratory Skewness Index and to non-stationary regimes via hidden Markov model conditioning, with

Load-bearing premise

The coverage guarantee holds only if the successive context-reward pairs for each arm are exchangeable; the paper's remedy, ACI, offers only an asymptotic guarantee, and the paper does not prove that ACI remains valid when arm selection is adaptive and some arms are rarely pulled.

Editorial extensions

If this is right

  • If correct, Conformal Bandits provide a distribution-free way to obtain nominal finite-sample coverage in sequential decision-making, even when arm rewards are heavy-tailed or skewed.
  • The framework directly improves regret in the small-gap regime, where classical UCB-type policies exhibit linear regret, making it relevant for applications like portfolio allocation and clinical trials with near-identical interventions.
  • Risk-aware policy variants (CP-Bandit, CP-ESI) enable downside protection without sacrificing statistical guarantees, and the regime-aware extension via hidden Markov models can adapt to non-stationary environments while preserving coverage.
  • The approach can be extended beyond UCB-style selection to other bandit families, such as Thompson sampling, as the conformal interval is agnostic to the underlying predictor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's finite-sample coverage claim is formally proven only for exchangeable data; the adoption of ACI shifts the guarantee to an asymptotic one, and the paper does not provide a theory showing that ACI remains valid when arm selection probabilities depend on past rewards. Thus, the practical validity in the bandit setting is an empirical assertion that could fail in adversarial or rarely-pul
  • The reported regret advantage over UCB1 is demonstrated only through simulations for specific distributions (Gaussian, Student-t, skew-t) and a fixed gap; whether the advantage is generic or depends on the conformal interval's empirical quantile estimation is left open, especially given the absence of a formal regret bound.
  • A testable consequence of the risk-aware indices: comparing CP-ESI against mean-variance UCB in a full-information setting could isolate whether asymmetric interval bounds capture drawdown risk better than volatility, which would inform which index is more suitable for downside protection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a family of 'Conformal Bandits' that replace the Hoeffding-based exploration bonuses of UCB with conformal prediction intervals (specifically Conformalized Quantile Regression, CQR), together with risk-sensitive variants (CP-Bandit, CP-ESI) and a randomized version. It claims that this yields finite-sample prediction coverage for each arm's future reward and superior regret performance in the small-gap regime. The methodology is evaluated in Monte Carlo simulations for Gaussian, Student-t, and skewed-t rewards, and applied to a portfolio allocation problem with HMM-based regime detection.

Significance. If the central guarantee were proven, the paper would make a meaningful contribution by coupling decision-making with finite-sample distribution-free uncertainty quantification. The manuscript has concrete strengths: it provides reproducible code, a clear simulation design, a real-data case study, and an honest discussion of limitations (Remark 4.1, Section 7.1). However, the central finite-sample coverage guarantee is not established, and the regret-efficiency claim rests entirely on simulations. The paper's main theoretical assertion, Eq. (10), is not justified under adaptive arm selection, and the adopted ACI correction provides only asymptotic coverage. As a result, the significance of the contribution is presently much weaker than the abstract and conclusion suggest.

major comments (4)
  1. [Section 4.3, Eq. (10)] The statement that the intervals satisfy P(Y_{k,t+1} in C^{1-alpha}_{k,t+1}) >= 1-alpha for every arm and round is asserted 'in virtue of CP properties'. This is not valid for bandit feedback. Split conformal/CQR finite-sample coverage requires exchangeability of the observed calibration and test pairs. With an adaptive policy, the sequence of pulls for arm k is a subsequence selected at data-dependent stopping times; even with iid rewards, the resulting observed sequence need not be exchangeable. Remark 4.1 explicitly concedes that adaptivity 'may compromise exchangeability,' and Section 7.1 limits the adopted ACI correction to asymptotic coverage. Thus the abstract's 'finite-sample prediction coverage' is not established by the manuscript. The paper would need either a theorem for conformal inference under adaptive sampling, or clearly re-scoped claims of empirical calibration.
  2. [Section 5, Fig. 3 and Section 7.2] The statement that all CP-Bandit variants exhibit 'uniformly superior learning efficiency compared to UCB1' in the small-gap regime is an empirical observation from Monte Carlo simulations for three distributions, not a theorem. Section 7.2 acknowledges that formal regret analysis is future work. Moreover, the comparison is limited to T=2000 and a fixed gap of 0.05; no sensitivity analysis over horizon or gap size is provided. The claim should be phrased as finite-horizon empirical performance, or supported by regret bounds.
  3. [Sections 4.2-4.3, Remark 4.1, Section 7.1] The adoption of Adaptive Conformal Inference (ACI) is not a substitute for the finite-sample guarantee. ACI's validity is asymptotic under distribution shift and requires conditions (e.g., bounded variation of the miscoverage rate) that are not checked for bandit-dependent selection. The manuscript itself states in Section 7.1 that ACI guarantees asymptotic coverage. Therefore Eq. (10) cannot be read as a finite-sample or even asymptotic coverage theorem for the proposed bandit policy without additional analysis. A concrete test would be to report conditional coverage given arm-selection histories, or at least coverage on a held-out arm's observed pulls.
  4. [Section 5, Table 1] Comparing UCB1's exploration intervals to nominal prediction coverage is conceptually mismatched. The UCB1 interval is a confidence bound for the arm mean, not a prediction interval for individual rewards; so its coverage of a future reward is not supposed to be 1-alpha. The claim that 'UCB1 fails to achieve nominal coverage' is therefore a strawman and does not demonstrate a deficiency of UCB as a regret-minimizer. Report, e.g., mean-confidence coverage or compare with an apples-to-apples predictive baseline.
minor comments (6)
  1. [Throughout] Typos and wording: 'probailities' (Section 2), 'ee term' (Section 4.3), 'provids' (Table 3), 'out application' (Section 3.1).
  2. [Algorithm 1] The warm-up is stated to require two observations per arm (one training, one calibration), but CQR with a single training point cannot meaningfully fit a quantile regression. Clarify how the quantile models are initialized and how many samples are actually used before the first interval is formed.
  3. [Eq. (17)] The mean-variance optimization is written with R_j both as part of the objective and as the realized return. Clarify whether W_j uses estimated moments and how the optimization is computed.
  4. [Eq. (16)] The index in the summation reads 't=1,...,t'; it should be i=1,...,t.
  5. [Section 5] Notation conflict: L_{k,t} is used for the lower bound in Eq. (10) but reused as the interval width in Section 5 ('Letting the interval width or length at time t for arm k to be L_{k,t}=U_{k,t}-L_{k,t}'). Use a different symbol for interval width.
  6. [Section 5] The text says all Conformal Bandit policies use CQR with ACI, but the methodology section does not formalize how ACI updates alpha per arm and how it interacts with the quantile regression refitting. Provide the exact update rule used in the experiments.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: coverage claim is imported from external conformal prediction theory; the admitted ACI limitation is a correctness gap, not a circular reduction.

full rationale

Eq. (10) constructs arm-specific CQR intervals from quantile regressions and calibration scores; the claimed coverage is explicitly attributed to standard CP/CQR results (Refs. 71, 24, 57), not to the paper's own fitted outputs. The regret advantage in Section 5 is reported from 1,000 Monte Carlo simulations, not obtained by fitting a target quantity and relabeling it as a prediction. The only same-author citation ([15], Deliu & Liseo) appears in informal Bayesian-comparison remarks and future-work suggestions; it is not used to justify Eq. (10), the ACI choice, or the regret claims. The paper itself flags the real weakness: Remark 4.1 concedes that adaptive arm selection 'may compromise exchangeability over the joint distribution of arm selection and reward,' and Section 7.1 states that ACI is 'guaranteed to ensure asymptotic coverage, and showing finite-sample coverage in our empirical studies.' Thus the finite-sample coverage assertion in Eq. (10) lacks a supporting theorem under adaptive sampling, and the claim 'ACI gives finite-sample coverage' is an empirical observation, not a circular derivation. These are correctness/support gaps, not instances of a conclusion reducing by construction to its inputs.

Assumptions & free parameters 8 free parameters · 6 assumptions · 1 invented entities

The framework rests on standard conformal-prediction theorems (external), on the assumption that ACI repairs the exchangeability violation introduced by bandit feedback, and on domain assumptions about stationary rewards and well-specified HMM regimes. The main free choices are user-set hyperparameters (alpha, lambda, S, exploration schedules) and unspecified implementation choices (quantile model, comparator tuning). No new physical entities are introduced; ESI is a decision index without external falsifiable evidence.

free parameters (8)
  • Miscoverage level alpha (ACI base level) = 0.2
    Set by the user; controls target coverage and interval width in all simulations and the backtest.
  • ACI step size = not reported
    Adaptive Conformal Inference adjusts alpha using a step size; the paper never states it, so empirical coverage results are not fully reproducible from the text.
  • Quantile regression model for CQR = not specified
    Section 5 is non-contextual but CQR requires a quantile estimator; neither the model class nor hyperparameters are stated.
  • UCB exploration constant beta = not reported
    Regret comparisons with UCB1 and MV-UCB1 depend on beta; its value is not given, so the strength of the UCB benchmark is ambiguous.
  • Risk-mixing lambda (CP-Bandit) = 0, 0.5, 0.7, 1 tested
    Hand-chosen interpolation between upper and lower bounds; no selection principle is given.
  • HMM number of regimes S = 3
    Regime count and Gaussian emissions are assumed in Appendix B; the regime-aware results depend on this model choice.
  • Randomization decay gamma = not stated for simulations
    Randomized policies use epsilon_t = t^{-gamma}/(K-1); gamma is not specified for the small-gap experiments.
  • Portfolio exploration threshold epsilon = 0.03
    Fixed in Section 6.2 to maintain adaptability; affects all regime-aware CP results.
assumptions (6)
  • domain assumption Exchangeability of (X_i, Y_i) in the conformal prediction procedure
    Eqs. (3)-(10) rely on split-CQR finite-sample coverage, which holds only under exchangeability; Remark 4.1 concedes adaptive bandit sampling violates it.
  • domain assumption ACI provides valid coverage under non-exchangeability
    The paper adopts Adaptive Conformal Inference [27] instead of exchangeability; ACI's formal guarantee is asymptotic, not the finite-sample guarantee claimed in the abstract.
  • domain assumption Unique optimal arm and fixed reward distributions in the small-gap setting
    Section 3 assumes a unique k* and constant suboptimality gaps; the portfolio application handles non-stationarity separately via HMM.
  • domain assumption Arm rewards are independent
    Used in simulation designs and coverage aggregation; stated as a limitation in Section 7.1.
  • domain assumption Gaussian-emission HMM fitted via EM correctly identifies Bull/Neutral/Bear regimes
    Appendix B; the regime-aware policy's performance depends on this regime inference being reliable on the same data used for evaluation.
  • standard math Standard split-CQR validity theorem
    Section 4.2 uses the Romano et al. (2019) conformalized quantile regression result as an external theorem without derivation.
invented entities (1)
  • Exploratory Skewness Index (ESI)
    purpose: Arm-selection score U_k,t / |L_k,t| used in the CP-ESI policy to favour positively skewed payoff distributions.
    A new decision index defined inside the paper; no external falsifiable prediction is attached to it beyond its ranking behavior in simulations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability." pith.science (2026). https://pith.science/paper/JB43TCY4

@misc{pith2026251209850,
  author       = {Pith},
  title        = {Pith review of: Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JB43TCY4}},
  note         = {Machine review of arXiv:2512.09850}
}
read the original abstract

We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty. Traditional regret-minimisation bandit strategies like Thompson Sampling and Upper Confidence Bound (UCB) typically rely on distributional assumptions or asymptotic guarantees; further, they remain largely focused on regret, neglecting their statistical properties. We address this gap. Through the adoption of CP, we bridge the regret-minimising potential of a decision-making bandit policy with statistical guarantees in the form of finite-sample prediction coverage. We demonstrate the potential of Conformal Bandits through simulation studies and an application to portfolio allocation, a typical scenario where differences in arm rewards are far too small (weak arm separability) for classical policies to be optimal in finite sample. We showcase our framework's practical advantage in terms of regret in this setting, as well as its added value in achieving nominal coverage guarantees where classical UCB policies may fail. Focusing on our application of interest, we further illustrate how integrating hidden Markov models to capture the regime-switching behaviour of financial markets, enhances the exploration-exploitation trade-off, and translates into higher risk-adjusted returns, while preserving coverage guarantees.

Figures

Figures reproduced from arXiv: 2512.09850 by the authors.

Figure 1
Figure 1. Cumulative regret (left) and best-arm selection (center) attained with UCB1 [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Comparison among bandit policies in terms of cumulative regret and best-arm selection over time, for [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figure 4
Figure 4. Cumulative wealth of CP-based and UCB-based bandit policies under a partial-information setting, compared [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Comparison between Conformal Bandit variants and classical UCB1 in the big-gap setting, based on [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 8
Figure 8. Figure 8: Cumulative wealth of CP-based bandit policies under a full-information setting, compared with EW, MV [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

93 extracted references · 2 linked inside Pith

  1. [15]

    Deliu and B

    N. Deliu and B. Liseo. The interplay between bayesian inference and conformal prediction.arXiv preprint arXiv:2510.26930, 2025

    arXiv 2025

  2. [1]

    Aguilera, M

    A. Aguilera, M. Arévalo Avalos, J. Xu, B. Chakraborty, C. Figueroa, F. Garcia, K. Rosales, R. Hernandez-Ramos, C. Karr, J. Williams, et al. Effectiveness of a digital health intervention leveraging reinforcement learning: results from the diabetes and mental health adaptive notification tracking and evaluation (diamante) randomized clinical trial.Journal ...

    2024

  3. [2]

    A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei. Learn then test: Calibrating predictive algorithms to achieve risk control.The Annals of Applied Statistics, 19(2), 2025

    2025

  4. [3]

    Ashutosh, J

    K. Ashutosh, J. Nair, A. Kagrecha, and K. Jagannathan. Bandit algorithms: Letting go of logarithmic regret for statistical robustness. InInternational Conference on Artificial Intelligence and Statistics, pages 622–630. PMLR, 2021

    2021

  5. [4]

    Audibert, R

    J.-Y . Audibert, R. Munos, and C. Szepesvári. Exploration–exploitation tradeoff using variance estimates in multi-armed bandits.Theoretical Computer Science, 410(19):1876–1902, 2009

    1902

  6. [5]

    P. Auer. Using confidence bounds for exploitation-exploration trade-offs.Journal of machine learning research, 3(Nov):397–422, 2002

    2002

  7. [6]

    P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem.Machine Learning, 47(2):235–256, 2002

    2002

  8. [7]

    P. Auer, N. Cesa-Bianchi, Y . Freund, and R. E. Schapire. The nonstochastic multiarmed bandit problem.SIAM journal on computing, 32(1):48–77, 2002

    2002

Show all 93 references
  1. [8]

    C. R. Bacon.Practical Risk-Adjusted Performance Measurement. John Wiley & Sons, Ltd, 2022

    2022

  2. [9]

    L. E. Baum and T. Petrie. Statistical inference for probabilistic functions of finite state markov chains.Annals of Mathematical Statistics, 37(6):1554–1563, 1966

    1966

  3. [10]

    Botosan and P

    I.-A. Botosan and P. Bilokon. Optimal resource allocation using multi-armed bandits.Master Thesis. Imperial College London, 2024

    2024

  4. [11]

    Cappé, A

    O. Cappé, A. Garivier, O.-A. Maillard, R. Munos, and G. Stoltz. Kullback-leibler upper confidence bounds for optimal sequential allocation.The Annals of Statistics, pages 1516–1541, 2013

    2013

  5. [12]

    Cassel, S

    A. Cassel, S. Mannor, and A. Zeevi. A general approach to multi-armed bandits under risk criteria. InConference on learning theory, pages 1295–1306. PMLR, 2018

    2018

  6. [13]

    J. Q. Chang and V . Y . Tan. A unifying theory of thompson sampling for continuous risk-averse bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 6159–6166, 2022

    2022

  7. [14]

    Charpentier, R

    A. Charpentier, R. Elie, and C. Remlinger. Reinforcement learning in economics and finance.Computational Economics, 62(1):425–462, 2023

    2023

  8. [16]

    Deliu and S

    N. Deliu and S. S. Villar. On the finite-sample and asymptotic error control of a randomization-probability test for response-adaptive clinical trials.Biometrics, 81(2), 2025

    2025

  9. [17]

    Deliu, J

    N. Deliu, J. J. Williams, and B. Chakraborty. Reinforcement Learning in Modern Biostatistics: Constructing Optimal Adaptive Interventions.International Statistical Review, 93(3):385–424, 2025

    2025

  10. [18]

    DeMiguel, L

    V . DeMiguel, L. Garlappi, and R. Uppal. Optimal versus naive diversification: How inefficient is the 1/n portfolio strategy?The review of Financial studies, 22(5):1915–1953, 2009

    1915

  11. [19]

    A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1–22, 1977

    1977

  12. [20]

    Duran-Martin, L

    G. Duran-Martin, L. Sánchez-Betancourt, A. Cartea, and K. P. Murphy. Martingale posterior neural networks for fast sequential decision making. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  13. [21]

    R. F. Engle and V . K. Ng. Measuring and testing the impact of news on volatility.The journal of finance, 48(5):1749–1778, 1993

    1993

  14. [22]

    Non-inferiority clinical trials to establish effectiveness.Food and Drug Administration

    FDA. Non-inferiority clinical trials to establish effectiveness.Food and Drug Administration. Guidance for Industry, 2016

    2016

  15. [23]

    J. D. Fisher, D. Pettenuzzo, and C. M. Carvalho. Optimal asset allocation with multivariate bayesian dynamic linear models.The annals of applied statistics, 14(1):299–338, 2020. 17 APREPRINT- DECEMBER11, 2025

    2020

  16. [24]

    Fontana, G

    M. Fontana, G. Zeni, and S. Vantini. Conformal prediction: a unified review of theory and new challenges. Bernoulli, 29(1):1–23, 2023

    2023

  17. [25]

    Fujishima and K

    K. Fujishima and K. Nakagawa. Multiple portfolio blending strategy with thompson sampling. InProceedings of the 12th International Congress on Advanced Applied Informatics (IIAI-AAI 2022), 2022

    2022

  18. [26]

    Galichet, M

    N. Galichet, M. Sebag, and O. Teytaud. Exploration vs exploitation vs safety: Risk-aware multi-armed bandits. In C. S. Ong and T. B. Ho, editors,Proceedings of the 5th Asian Conference on Machine Learning, volume 29 of Proceedings of Machine Learning Research, pages 245–260, A...

  19. [27]

    Gibbs and E

    I. Gibbs and E. Candes. Adaptive conformal inference under distribution shift.Advances in Neural Information Processing Systems, 34:1660–1672, 2021

    2021

  20. [28]

    Giovagnoli and M

    A. Giovagnoli and M. Lupparelli. A note on response-adaptive randomization from a bayesian prediction viewpoint.Statistical Methods in Medical Research, 34(10):2053–2068, 2025

  21. [29]

    Gittins, K

    J. Gittins, K. Glazebrook, and R. Weber.Multi-armed bandit allocation indices. John Wiley & Sons, 2011

    2011

  22. [30]

    P. W. Glynn. Diffusion approximations.Handbooks in Operations research and management Science, 2:145–198, 1990

    1990

  23. [31]

    Hadad, D

    V . Hadad, D. A. Hirshberg, R. Zhan, S. Wager, and S. Athey. Confidence intervals for policy evaluation in adaptive experiments.Proceedings of the National Academy of Sciences, 118(15):e2014602118, 2021

    2021

  24. [32]

    Hirano and J

    K. Hirano and J. R. Porter. Asymptotics for statistical treatment rules.Econometrica, 77(5):1683–1701, 2009

    2009

  25. [33]

    Hoeffding

    W. Hoeffding. Probability inequalities for sums of bounded random variables.Journal of the American statistical association, 58(301):13–30, 1963

    1963

  26. [34]

    H. Hu, A. Charpentier, M. Ghossoub, and A. Schied. The multi-armed bandit problem under the mean-variance setting.European Journal of Operational Research, 324(1):168–182, 2025

    2025

  27. [35]

    Huo and F

    X. Huo and F. Fu. Risk-aware multi-armed bandit problem with application to portfolio selection.Royal Society Open Science, 4(11):171377, 2017

    2017

  28. [36]

    G. W. Imbens and D. B. Rubin.Causal inference in statistics, social, and biomedical sciences. Cambridge university press, 2015

    2015

  29. [37]

    James, D

    G. James, D. Witten, T. Hastie, and R. Tibshirani.An Introduction to Statistical Learning: With Applications in R. Springer Texts in Statistics. Springer, New York, 2nd edition, 2021

    2021

  30. [38]

    Jamieson, M

    K. Jamieson, M. Malloy, R. Nowak, and S. Bubeck. lil’ucb: An optimal exploration algorithm for multi-armed bandits. InConference on Learning Theory, pages 423–439. PMLR, 2014

    2014

  31. [39]

    Kaibel and T

    C. Kaibel and T. Biemann. Rethinking the gold standard with multi-armed bandits: machine learning allocation algorithms for experiments.Organizational Research Methods, 24(1):78–103, 2021

    2021

  32. [40]

    Kalvit and A

    A. Kalvit and A. Zeevi. A closer look at the worst-case behavior of multi-armed bandit algorithms.Advances in Neural Information Processing Systems, 34:8807–8819, 2021

    2021

  33. [41]

    M. Kato, M. Imaizumi, T. Ishihara, and T. Kitagawa. Best arm identification with contextual information under a small gap.arXiv preprint arXiv:2209.07330, 2022

    2022 arXiv

  34. [42]

    Kaufmann, O

    E. Kaufmann, O. Cappé, and A. Garivier. Bayesian upper confidence bounds for bandit problems. InInternational Conference on Artificial Intelligence and Statistics, pages 592–600, 2012

    2012

  35. [43]

    Koenker.Quantile regression, volume 38

    R. Koenker.Quantile regression, volume 38. Cambridge university press, 2005

    2005

  36. [44]

    T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules.Advances in applied mathematics, 6(1):4–22, 1985

    1985

  37. [45]

    T. L. Lai, H. Xing, and Z. Chen. Mean–variance portfolio optimization when means and covariances are unknown. The Annals of Applied Statistics, 5(2A):798 – 823, 2011

    2011

  38. [46]

    Lattimore and C

    T. Lattimore and C. Szepesvári.Bandit algorithms. Cambridge University Press, 2020

    2020

  39. [47]

    Le Cam.Asymptotic Methods in Statistical Decision Theory

    L. Le Cam.Asymptotic Methods in Statistical Decision Theory. Springer Series in Statistics. Springer New York, New York, NY , 1986

    1986

  40. [48]

    A. A. Lee, N. Hegde, N. Deliu, E. Rosenzweig, A. Suggala, S. Lakshminarasimhan, Q. He, J. Hernandez, M. Seneviratne, R. Singh, P. Kalkar, K. Shanmugam, A. Raghuveer, A. Singh, M. Nguyen, J. Taylor, J. Alla, S. S. Villar, and H. Emir-Farinas. A personalized exercise assistant u...

    2025 arXiv

  41. [49]

    X. Liu, N. Deliu, T. Chakraborty, L. Bell, and B. Chakraborty. Thompson sampling for zero-inflated count outcomes with an application to the drink less mobile health study.The Annals of Applied Statistics, 19(2):1403– 1425, 2025

    2025

  42. [50]

    Y . Liu, B. Van Roy, and K. Xu. Nonstationary bandit learning via predictive sampling. InInternational Conference on Artificial Intelligence and Statistics, pages 6215–6244. PMLR, 2023

    2023

  43. [51]

    Mannor and J

    S. Mannor and J. N. Tsitsiklis. The sample complexity of exploration in the multi-armed bandit problem.Journal of Machine Learning Research, 5(Jun):623–648, 2004

    2004

  44. [52]

    Markowitz

    H. Markowitz. Modern portfolio theory.Journal of Finance, 7(11):77–91, 1952

    1952

  45. [53]

    May Lee and J

    K. May Lee and J. J. Lee. Evaluating bayesian adaptive randomization procedures with adaptive clip methods for multi-arm trials.Statistical methods in medical research, 30(5):1273–1287, 2021

    2021

  46. [54]

    A. J. McNeil, R. Frey, and P. Embrechts.Quantitative risk management: concepts, techniques and tools-revised edition. Princeton university press, 2015

    2015

  47. [55]

    Ousmen, C

    A. Ousmen, C. Touraine, N. Deliu, F. Cottone, F. Bonnetain, F. Efficace, A. Brédart, C. Mollevi, and A. Anota. Distribution-and anchor-based methods to determine the minimally important difference on patient-reported outcome questionnaires in oncology: a structured review.Heal...

    2018

  48. [56]

    L. Pin, S. Baas, G. Caruso, D. S. Robertson, and S. S. Villar. Informed burn-in decisions in rar: Harmonizing adaptivity and inferential precision based on study setting.arXiv preprint arXiv:2511.21376, 2025

    2025

  49. [57]

    Romano, E

    Y . Romano, E. Patterson, and E. J. Candès. Conformalized quantile regression. InAdvances in Neural Information Processing Systems 32 (NeurIPS 2019), pages 3543–3553, 2019

    2019

  50. [58]

    W. F. Rosenberger, D. Uschner, and Y . Wang. Randomization: The forgotten component of the randomized clinical trial.Statistics in Medicine, 38(1):1–12, 2019

    2019

  51. [59]

    A. Sani, A. Lazaric, and R. Munos. Risk-aversion in multi-armed bandits. In F. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors,Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012

    2012

  52. [60]

    Shen and J

    W. Shen and J. Wang. Portfolio blending via thompson sampling. InProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI 2016), 2016

    2016

  53. [61]

    W. Shen, J. Wang, Y .-G. Jiang, and H. Zha. Portfolio choices with orthogonal bandit learning. InProceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), 2015

    2015

  54. [62]

    Simchowitz and K

    M. Simchowitz and K. G. Jamieson. Non-asymptotic gap-dependent regret bounds for tabular mdps.Advances in Neural Information Processing Systems, 32, 2019

    2019

  55. [63]

    Stocker, W

    M. Stocker, W. Małgorzewicz, M. Fontana, and S. B. Taieb. A gentle introduction to conformal time series forecasting.arXiv preprint arXiv:2511.13608, 2025

    2025

  56. [64]

    R. S. Sutton, A. G. Barto, et al.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  57. [65]

    Szaszi, A

    B. Szaszi, A. Higney, A. Charlton, A. Gelman, I. Ziano, B. Aczel, D. G. Goldstein, D. S. Yeager, and E. Tipton. No reason to expect large and consistent effects of nudge interventions.Proceedings of the National Academy of Sciences, 119(31):e2200732119, 2022

    2022

  58. [66]

    P. F. Thall and J. K. Wathen. Practical bayesian adaptive randomisation in clinical trials.European Journal of Cancer, 43(5):859–866, 2007

    2007

  59. [67]

    W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25(3-4):285–294, 1933

    1933

  60. [68]

    Vakili and Q

    S. Vakili and Q. Zhao. Risk-averse multi-armed bandit problems under mean-variance measure.IEEE Journal of Selected Topics in Signal Processing, 10(6):1093–1111, 2016

    2016

  61. [69]

    Vaswani, A

    S. Vaswani, A. Mehrabian, A. Durand, and B. Kveton. Old dog learns new tricks: Randomized ucb for bandit problems. In S. Chiappa and R. Calandra, editors,Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 ofProceeding...

    1988

  62. [70]

    S. S. Villar, J. Bowden, and J. Wason. Multi-armed bandit models for the optimal design of clinical trials: benefits and challenges.Statistical science: a review journal of the Institute of Mathematical Statistics, 30(2):199, 2015

    2015

  63. [71]

    V ovk, A

    V . V ovk, A. Gammerman, and G. Shafer.Algorithmic Learning in a Random World. Springer, New York, 2005

    2005

  64. [72]

    P. Whittle. Multi-armed bandits and the gittins index.Journal of the Royal Statistical Society: Series B (Methodological), 42(2):143–149, 1980. 19 APREPRINT- DECEMBER11, 2025

    1980

  65. [73]

    J. Yao, E. Brunskill, W. Pan, S. Murphy, and F. Doshi-Velez. Power constrained bandits. InMachine Learning for Healthcare Conference, pages 209–259. PMLR, 2021

    2021

  66. [74]

    Zhao.Multi-Armed Bandits: Theory and Applications to Online Learning in Networks, volume 13 of Foundations and Trends in Signal Processing

    Q. Zhao.Multi-Armed Bandits: Theory and Applications to Online Learning in Networks, volume 13 of Foundations and Trends in Signal Processing. Now Publishers, 1st edition, 2020

    2020

  67. [75]

    Zhu and R

    Z. Zhu and R. E. Welsch. Robust dependence modeling for high-dimensional covariance matrices with financial applications.The Annals of Applied Statistics, 12(2), 2018

    2018

  68. [76]

    Zucchini, I

    W. Zucchini, I. L. MacDonald, and R. Langrock.Hidden Markov Models for Time Series: An Introduction Using R. Monographs on Statistics and Applied Probability. Chapman & Hall/CRC, 2nd edition, 2016. Appendix A Additional Simulation Studies on the Big-gap regime To complement th...

    2016

  69. [77]

    Select arma ∗ t = ((t−1) modK) + 1

  70. [78]

    4.end for 5.fort= 2K+ 1toTdo((Regime-Aware) Conformal bandit selection)

    Observe associated state-reward pair(X t, Ya∗ t ,t). 4.end for 5.fort= 2K+ 1toTdo((Regime-Aware) Conformal bandit selection)

  71. [79]

    7.fork= 1,

    Infer market regimeR t ∈ {Bull,Neutral,Bear}from regime-detection model. 7.fork= 1, . . . , Kdo

  72. [80]

    Given observed state-reward pairs(X i, Yk,i)Nk,t−1 i=1 :

  73. [81]

    Compute conformal intervalsC 1−α k,t = h Lk,t, Uk,t i

  74. [82]

    12.ifR t =Bear, setI k,Rt,t =−|L k,t|(downside-protective)

    Compute regime-aware pseudo-deterministic conformal indexI k,Rt,t: 11.ifR t ∈ {Bull,Neutral}, setIk,Rt,t =U k,t (optimistic). 12.ifR t =Bear, setI k,Rt,t =−|L k,t|(downside-protective). 13.end for

  75. [83]

    Get regime-aware pseudo-deterministic optimal arm˜a ∗ t = arg maxk∈A Ik,Rt,t

  76. [84]

    (Bandit randomisation) 16.ifZ t = 1then 17.a ∗ t = ˜a∗ t 18.else 19.a ∗ t ∼Unif A\˜a∗ t

    DrawZ t ∼Bernoulli(1−ϵ t). (Bandit randomisation) 16.ifZ t = 1then 17.a ∗ t = ˜a∗ t 18.else 19.a ∗ t ∼Unif A\˜a∗ t . 20.end if

  77. [85]

    22.end for 23.returnselected arms{a ∗ t }T t=1

    Observe associated rewardY a∗ t ,t and update state-reward pairs(X i, Yk,i)Nk,t i=1 fork=a ∗ t . 22.end for 23.returnselected arms{a ∗ t }T t=1. Algorithm 2: Regime-Aware Conformal bandits 23 APREPRINT- DECEMBER11, 2025 Algorithm 3: Regime-Aware MV-UCB1 Input:Number of armsK; ...

    2025

  78. [86]

    2.fort= 1toKdo

    Initialise empirical meansbµ k,0, variancesbσ2 k,0, and pull countsN k,0 = 0. 2.fort= 1toKdo

  79. [87]

    5.end for 6.fort=K+ 1toTdo

    Observe rewardY a∗ t ,t and updatebµa∗ t ,t,bσ2 a∗ t ,t,N a∗ t ,t. 5.end for 6.fort=K+ 1toTdo

  80. [88]

    8.fork= 1,

    Infer regimeR t ∈ {Bull,Neutral,Bear}from regime model. 8.fork= 1, . . . , Kdo

  81. [89]

    Given the observed reward(Y k,i)Nk,t−1 i=1 :

  82. [90]

    11.ifR t ∈ {Bull,Neutral}, setSk,t =bµk,t +B k,t

    Compute UCB exploration term for each arm:B k,t = q βlogt Nk,t . 11.ifR t ∈ {Bull,Neutral}, setSk,t =bµk,t +B k,t. 12.ifR t =Bear, compute mean–variance score dM V ρ k,t =ρbµk,t −(1−ρ)bσ k,t

  83. [91]

    14.end if 15.end for

    and setS k,t = dM V ρ k,t +B k,t. 14.end if 15.end for

  84. [92]

    Select arma ∗ t = arg maxk∈A Sk,t

  85. [93]

    18.end for 19.return{a ∗ t }T t=1

    Observe rewardY a∗ t ,t and updatebµa∗ t ,t,bσ2 a∗ t ,t,N a∗ t ,t. 18.end for 19.return{a ∗ t }T t=1. Algorithm 3: Regime-Aware MV-UCB1 Figure 8: Cumulative wealth of CP-based bandit policies under a full-information setting, compared with EW, MV and SA portfolio benchmarks. B...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.