Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Adaptive Data Augmentation for Thompson Sampling

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims Thompson Sampling can be made nearly minimax optimal in linear contextual bandits.

desk verdict Clever hypothetical-context construction and coupling idea, but Lemma 2 has a false equality that breaks the main self-normalized bound; the claimed O~(d sqrt T) regret for Thompson Sampling is not established as written. read the letter →

arxiv 2506.14479 v1 pith:73QUUGIO submitted 2025-06-17 stat.ML cs.LG

Wonyoung Kim This is my paper · ORCID
classification stat.MLcs.LG MSC 62L05
keywords ThompsonSamplinglinearcontextualbanditsminimaxoptimalregretdataaugmentationcouplinghypotheticalbanditself-normalizedboundadaptiveregularization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to close a long-standing gap: Thompson Sampling, despite its strong empirical record in linear contextual bandits, has a regret bound of order $d^{3/2}\sqrt{T}$, while the best possible is $\Omega(d\sqrt{T})$. It proposes a Thompson Sampling variant, HCSA+TS, whose estimator is built from hypothetical contexts---a small orthogonal set that preserves the Gram matrix of all $K$ arms---augmented on selected rounds by a full orthonormal basis. A resampling-coupling step makes the hypothetical rewards observable, and the resulting self-normalized bound uses a Gram matrix that includes every arm's context, not just the played one. If the proof is correct, this yields $O(d\sqrt{T}\log T)$ regret under arbitrary context distributions, removing the IID or positive-covariance assumptions that earlier results needed.

What carries the argument

The load-bearing construction is the hypothetical bandit: at each round, the unplayed contexts' Gram matrix $G_t$ is eigen-decomposed, and the hypothetical contexts $Z_{i,t}$ are $\sqrt{\lambda_{i,t}}u_{i,t}$ plus the played context $X_{a_t,t}$, so the identity $\sum_{i=1}^{r_t+1} Z_{i,t}Z_{i,t}^\top = \sum_{k=1}^K X_{k,t}X_{k,t}^\top$ holds with far fewer arms. On selected rounds, an orthonormal basis is added, giving a Gram matrix bounded below by $\sum_{s\notin A_t}\sum_{k=1}^K X_{k,s}X_{k,s}^\top + \max\{x_{\max}^2,1\}h_t I_d$. The coupling step resamples the Thompson draw until the hypothetical arm equals the played arm, so the pseudo-rewards become computable; Lemma 1, the coupling inequality, and the maximal elliptical potential lemma are the three supports of the regret analysis.

What would settle it

For a single round with $N_s=2$ and $\gamma=1/2$, enumerate the resampling tree: if the first draw misses but the second hits, the HCSA estimator uses the second draw while the hypothetical estimator in Lemma 2 uses the first, so computing the two probabilities in Lemma 2 at any $x>0$ will show they differ.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem 3: with probability at least $1-3\delta$, the cumulative regret of HCSA+TS is bounded by a sum whose leading order is $O(d\sqrt{T}\log T)$, matching the $\Omega(d\sqrt{T})$ minimax lower bound up to logarithmic factors. The mechanism is a new estimator, the hypothetical coupled sample augmented (HCSA) estimator, whose normalization matrix $V_t$ includes contexts from all $K$ arms plus an adaptively scheduled orthonormal augmentation. The coupling argument lets the algorithm use only $N_t \le d+1$ pseudo-samples per round instead of $K$, and the paper argues that this removes the IID and strictly-positive-covariance context assumptions used by prior estimators, giving the first nearly minimax-optimal regret guarantee for Thompson Sampling in fully general linear contextual bandits.

Load-bearing premise

The regret bound rests on the claim that once every resampling trial succeeds, the estimator's error distribution is the same as if the first hypothetical sample had been used in every round; that equality is the load-bearing premise of the proof.

Editorial extensions

If this is right

  • Linear contextual bandits would get a single Thompson Sampling algorithm whose $O(d\sqrt{T}\log T)$ regret holds even when contexts are deterministic, adversarial, or non-stationary.
  • The regret bound would no longer grow with the number of arms $K$ or with the context norm $x_{\max}$, because the Gram matrix contains information about all arms.
  • The estimator would deliver accurate reward predictions for all arms, not only the played one, which is exactly what is needed for offline evaluation and for settings with missing context features.
  • The compression identity (3) could be reused by any algorithm that needs a well-conditioned Gram matrix, since it reduces $K$ context vectors to $r_t+1$ orthogonal ones without losing the sum of outer products.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The coupling step is the fragile point: the paper does not verify numerically that the stopping-time distribution equals the first-trial distribution, so a small simulation comparing the two probabilities in Lemma 2 would settle the proof.
  • Because the hypothetical-context construction is distribution-free, a natural extension would be to generalized linear bandits, replacing the Gram matrix identity with a Fisher-information version.
  • One could test the algorithm's practical claim on a fully deterministic context sequence (for example, cyclic basis vectors) where earlier Thompson Sampling variants are known to suffer; the paper's experiments only use random contexts with masking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes HCSA+TS, a Thompson Sampling algorithm for linear contextual bandits that uses a new estimator based on hypothetical contexts, adaptive data augmentation, and a resampling/coupling mechanism. The main theoretical claim is a nearly minimax-optimal regret bound of O(d sqrt(T) log T) under arbitrary (non-IID, non-stationary) context distributions, removing IID and minimum-eigenvalue assumptions used in prior work. The paper also contains experiments comparing the proposed method with several baselines on simulated data. The central result depends on a coupling inequality (Lemma 2) that relates the computable HCSA estimator to an idealized hypothetical-sample-augmented estimator.

Significance. If established, the claimed regret bound would be a significant advance: it would provide the first minimax-optimal (up to log factors) frequentist regret guarantee for Thompson Sampling in linear contextual bandits without distributional assumptions on contexts. The construction of orthogonal hypothetical contexts, the attempt to control a full Gram matrix including unselected arms, and the coupling mechanism are creative and could be of independent interest. However, the main theorem is not supported because Lemma 2, which is the sole bridge between the computable estimator and the idealized estimator, is proved via a false equality. The central claim is therefore currently unsubstantiated.

major comments (4)
  1. [Section 4.4, Lemma 2] The proof of Lemma 2 contains a false equality. It asserts that P({F(tilde a_1(M_1), ..., tilde a_t(M_t)) > x} cap S_t) equals P({F(tilde a_1(1), ..., tilde a_t(1)) > x} cap S_t cap cap_s {tilde a_s(1) = N_s}), justified by the IID property of resampling trials. This is incorrect because S_t = cap_s M_s includes paths where success occurs only at a later resampling trial, so the event M_s does not imply tilde a_s(1) = N_s; conditioning on eventual success changes the distribution of the first draw. Moreover, F is not invariant under replacing tilde a_s(M_s) with tilde a_s(1): the coupled pseudo-rewards in (14) and the resulting estimator depend on the final successful draw. A concrete counterexample with t=1, gamma=1/2, M_1=3 gives P(M_1)=7/8 while P(tilde a_1(1)=N_1)=1/2, so the claimed equality would force 7/8=1/2 whenever the event F(N_1)>x has positive probability. Because Theorem 5 is proved from Lemma 2 and Theorem 3 relies on Theorem 5, the central regret bound is not derived.
  2. [Section 5.1 and Appendix A.2, proof of Theorem 5] The proof of Theorem 5 depends on Lemma 2 to replace the HCSA estimator with the HSA estimator on the success event S_t. Since Lemma 2 is invalid, the self-normalized bound for the HCSA estimator is not established. Additionally, the proof in Appendix A.2 applies the triangle inequality to the decomposition from Lemma 4 but then substitutes definitions of the reference estimator and the matrix P_t in a way that appears to require V_t - A_t to be positive semidefinite on the relevant event; this is not shown and is another gap that would need to be addressed even if the coupling equality were repaired.
  3. [Appendix A.3, Lemma 9] The proof of Lemma 9 states 'Using Lemma 9 and Lemma 10 from Abbasi-Yadkori et al. (2011)', but Lemma 9 is the lemma being proved in this paper; the intended reference is presumably Theorem 1 of Abbasi-Yadkori et al. (2011), which is not stated. This leaves the claimed noise bound of (sigma/gamma) sqrt(d log(1+t/delta)) for the term St without a clear supporting citation or proof. This issue is secondary to the failure of Lemma 2 but still needs correction.
  4. [Section 5.3, regret decomposition] The derivation of the leading regret term contains unresolved notation and inequalities. In particular, the text moves from a bound involving '2h_T' to '2T_1', uses the expression 't=∈[T]', and the transition from regret(t) <= 2 x_t ||btheta_{t-1}-theta*||_{V_{t-1}} + sqrt(...) to the factored form involving sqrt(2) x_t is not fully justified, though it may be repairable. This is not the main obstruction but indicates that the regret analysis needs careful rewriting.
minor comments (5)
  1. [Section 4.2, equation (8)] The definition of N_s in (7) and the description of the orthogonal-basis augmentation in Section 4.2 are inconsistent with Algorithm 2: line 11 and line 14 of Algorithm 2 appear to swap the 't in A_t' and 'otherwise' cases relative to the text above equation (8).
  2. [Appendix A.1, Lemma 4] The statement of Lemma 4 contains stray text 'and. and.' before the displayed equation, and the proof uses 'rs+1' in sums where the full range should be 'N_s' as defined in (7).
  3. [Appendix A.5, Lemma 7] The proof introduces a sequence 'nu_s' without definition, and the sentence 'for ts such that t<=h_t, we have t in A_t and 1 in A_t for all t in [T]' is confusing; the intended argument that the regularization rounds are nonempty and well-ordered should be stated more clearly.
  4. [Section 5.1, Theorem 5] The phrase 'arbitral context distributions' should be 'arbitrary context distributions'.
  5. [Section 6, experimental setup] The hyperparameter tuning procedure is described only briefly; reporting the selected hyperparameter values for each method and the variance across runs would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the HCSA estimator and regret analysis are derived in-paper; the only self-citation is a non-load-bearing technical bound.

full rationale

The derivation is self-contained. The HCSA estimator (Eq. 15) is constructed from resampled hypothetical contexts and coupled pseudo-rewards (Eq. 14), and the regret analysis proceeds through the paper's own Lemma 2, Lemma 4, Lemmas 8-10, and Lemma 7, with standard martingale and matrix concentration arguments. No fitted parameter is renamed as a prediction, and no central quantity is defined in terms of the claimed regret bound. The single load-bearing external input from the author's prior work is the bound on T1 via Lemma C.6 of Kim et al. 2023a, used in Eq. (6); that bound only controls how many initial rounds are needed before the augmentation set reaches size h_t, and it is not the source of the O(d sqrt(T)) rate. The paper's claimed novelty, the coupling of hypothetical and original bandit samples, is not imported from prior work. A separate reviewer concern that Lemma 2's coupling equality is mathematically false is a proof-correctness issue, not a circularity issue, and therefore does not raise the circularity score.

Assumptions & free parameters 4 free parameters · 6 assumptions · 3 invented entities

The central claim rests on the standard linear bandit assumptions and on a new set of internal algorithmic constructs (hypothetical contexts, hypothetical bandit, coupling). The only externally imported technical result beyond standard concentration is a self-cited lemma for T1. No free parameters are fitted to data in the theory; gamma and other hyperparameters are chosen by hand and tuned in experiments.

free parameters (4)
  • gamma (coupling parameter) = 0.1, 0.5, or 0.9 in experiments; theoretical constant in (0,1)
    Controls the probability of coupling success; the regret bound scales as 1/gamma and 1/(1-gamma). Chosen by hand, not derived.
  • delta (confidence level) = user-specified
    Standard failure probability in high-probability bounds; appears in all terms of the regret bound.
  • x_max (context norm bound) = assumed known; can be replaced by running max
    Used to scale orthogonal basis augmentation; the paper notes it can be replaced by the empirical maximum over past rounds.
  • theta_max (upper bound on ||theta*||) = unknown
    Appears in the regret bound; the paper states it is unknown, but the algorithm does not estimate it.
assumptions (6)
  • domain assumption Rewards are linear: Y_{a_t,t} = X_{a_t,t}^T theta* + eta_{a_t,t}
    Section 3.2 defines the linear contextual bandit model.
  • domain assumption Noise eta is conditionally zero-mean and sigma-sub-Gaussian
    Section 3.2 states the sub-Gaussian condition used in all concentration bounds.
  • domain assumption |X_{k,t}^T theta*| <= 1 for all k and t
    Section 3.2 adopts this scale normalization following Abbasi-Yadkori et al. (2011).
  • domain assumption Contexts are bounded: ||X_{k,t}||_2 <= x_max
    Used in Section 4.2 to construct the orthogonal augmentation and in Lemma 1.
  • standard math Standard martingale concentration inequalities (Abbasi-Yadkori et al. 2011, Tropp 2012)
    Used in Lemmas 8, 9, and 10 for self-normalized bounds and matrix concentration.
  • standard math Lemma C.6 of Kim et al. (2023a) bounding T1
    Used in Equation (6) to bound the time when |A_t| reaches h_t; this is a self-cited technical lemma.
invented entities (3)
  • Hypothetical contexts Z_{i,t}
    purpose: Compress K context vectors into at most d+1 orthogonal vectors preserving the Gram matrix for efficient parameter learning.
    Internal algorithmic device; no falsifiable external handle.
  • Hypothetical bandit problem
    purpose: Defines pseudo-rewards and an idealized estimator used in the analysis.
    Analytical construct, not a physical entity.
  • Coupling resampling mechanism
    purpose: Ensures the sampled hypothetical arm equals the observed arm so that pseudo-rewards are computable.
    Algorithmic mechanism; not a physical postulate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Data Augmentation for Thompson Sampling." pith.science (2026). https://pith.science/paper/73QUUGIO

@misc{pith2026250614479,
  author       = {Pith},
  title        = {Pith review of: Adaptive Data Augmentation for Thompson Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/73QUUGIO}},
  note         = {Machine review of arXiv:2506.14479}
}
read the original abstract

In linear contextual bandits, the objective is to select actions that maximize cumulative rewards, modeled as a linear function with unknown parameters. Although Thompson Sampling performs well empirically, it does not achieve optimal regret bounds. This paper proposes a nearly minimax optimal Thompson Sampling for linear contextual bandits by developing a novel estimator with the adaptive augmentation and coupling of the hypothetical samples that are designed for efficient parameter learning. The proposed estimator accurately predicts rewards for all arms without relying on assumptions for the context distribution. Empirical results show robust performance and significant improvement over existing methods.

Figures

Figures reproduced from arXiv: 2506.14479 by the authors.

Figure 1
Figure 1. Flow diagram of the proposed coupling and resampling scheme [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the regrets of the proposed HCSA+TS algorithm with other benchmark methods. The lines represent the average, and the shaded areas indicate the standard deviation based on twenty experiments. The results demonstrate that the proposed HCSA+TS effectively identifies the optimal arm using orthogonal regularization [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Comparison of the prediction error across all arms, calculated as PK i=1{X⊤ i,t(θbt − θ⋆)} 2 , for the proposed HCSA+TS and other benchmark methods. The lines represent the averages, and the shaded areas indicate the standard deviations based on twenty experiments. The results demonstrate that the proposed estimator, enhanced with orthogonal augmentation, learns the reward more accurately than other estimators. to D… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 34 canonical work pages

  1. [1]

    Abbasi-Yadkori, D

    Y. Abbasi-Yadkori, D. P \'a l, and C. Szepesv \'a ri. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems, pages 2312--2320, 2011

    work page 2011

  2. [2]

    Abe and P

    N. Abe and P. M. Long. Associative reinforcement learning using linear probabilistic concepts. In ICML, 1999

    work page 1999

  3. [3]

    Abeille, A

    M. Abeille, A. Lazaric, et al. Linear thompson sampling revisited. Electronic Journal of Statistics, 11 0 (2): 0 5165--5197, 2017

    work page 2017

  4. [4]

    Agrawal and N

    S. Agrawal and N. Goyal. Thompson sampling for contextual bandits with linear payoffs. In International Conference on Machine Learning, pages 127--135, 2013

    work page 2013

  5. [5]

    Agrawal and N

    S. Agrawal and N. Goyal. Near-optimal regret bounds for thompson sampling. Journal of the ACM (JACM), 64 0 (5): 0 1--24, 2017

    work page 2017

  6. [6]

    P. Auer. Using confidence bounds for exploitation-exploration trade-offs. Journal of Machine Learning Research, 3 0 (Nov): 0 397--422, 2002a

    work page

  7. [7]

    Bang and J

    H. Bang and J. M. Robins. Doubly robust estimation in missing data and causal inference models. Biometrics, 61 0 (4): 0 962--973, 2005

    2005

  8. [8]

    Bastani, M

    H. Bastani, M. Bayati, and K. Khosravi. Mostly exploration-free algorithms for contextual bandits. Management Science, 67 0 (3): 0 1329--1349, 2021

    work page 2021

Show all 38 references
  1. [9]

    C. M. Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7 0 (1): 0 108--116, 1995

    1995

  2. [10]

    Bouneffouf, I

    D. Bouneffouf, I. Rish, and C. Aggarwal. Survey on applications of multi-armed and contextual bandits. In 2020 IEEE Congress on Evolutionary Computation (CEC), pages 1--8. IEEE, 2020

    2020

  3. [11]

    B \"u hlmann and S

    P. B \"u hlmann and S. Van De Geer. Statistics for high-dimensional data: methods, theory and applications. Springer Science & Business Media, 2011

    2011

  4. [12]

    Chapelle and L

    O. Chapelle and L. Li. An empirical evaluation of thompson sampling. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 24, pages 2249--2257. Curran Associates, Inc., 2011

    2011

  5. [13]

    W. Chu, L. Li, L. Reyzin, and R. Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208--214, 2011

    2011

  6. [14]

    V. Dani, T. Hayes, and S. Kakade. Stochastic linear optimization under bandit feedback. In 21st Annual Conference on Learning Theory, pages 355--366, 01 2008

    2008

  7. [15]

    Dimakopoulou, Z

    M. Dimakopoulou, Z. Zhou, S. Athey, and G. Imbens. Balanced linear contextual bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3445--3453, 2019

    2019

  8. [16]

    K. J. Ferreira, D. Simchi-Levi, and H. Wang. Online network revenue management using thompson sampling. Operations research, 66 0 (6): 0 1586--1602, 2018

    2018

  9. [17]

    Guttorp and G

    P. Guttorp and G. Lindgren. Karl pearson and the scandinavian school of statistics. International Statistical Review, 77 0 (1): 0 64--71, 2009

    2009

  10. [18]

    Hsu, W.-K

    J.-Y. Hsu, W.-K. Tseng, J.-Y. Hsieh, C.-J. Chang, and H. Chen. The recommending agricultural product sales promotion mode in e-commerce using reinforcement learning with contextual multiarmed bandit algorithms. Mathematical Problems in Engineering, 2020 0 (1): 0 8836000, 2020

    2020

  11. [19]

    T. Huix, M. Zhang, and A. Durmus. Tight regret and complexity bounds for thompson sampling via langevin monte carlo. In International Conference on Artificial Intelligence and Statistics, pages 8749--8770. PMLR, 2023

    2023

  12. [20]

    Kim and M

    G. Kim and M. C. Paik. Doubly-robust lasso bandit. In Advances in Neural Information Processing Systems, pages 5869--5879, 2019

    2019

  13. [21]

    Kim, G.-S

    W. Kim, G.-S. Kim, and M. C. Paik. Doubly robust thompson sampling with linear payoffs. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. W. Vaughan, editors, Advances in Neural Information Processing Systems, 2021

    2021

  14. [22]

    W. Kim, G. Iyengar, and A. Zeevi. Learning the pareto front using bootstrapped observation samples. arXiv preprint arXiv:2306.00096, 2023 a

    2023 arXiv

  15. [23]

    W. Kim, K. Lee, and M. C. Paik. Double doubly robust thompson sampling for generalized linear contextual bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8300--8307, 2023 b

    2023

  16. [24]

    W. Kim, M. C. Paik, and M.-H. Oh. Squeeze all: Novel estimator and self-normalized bound for linear contextual bandits. In International Conference on Artificial Intelligence and Statistics, pages 3098--3124. PMLR, 2023 c

    2023

  17. [25]

    Kveton, C

    B. Kveton, C. Szepesv \' a ri, M. Ghavamzadeh, and C. Boutilier. Perturbed-history exploration in stochastic linear bandits. In R. P. Adams and V. Gogate, editors, Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, volume 115 of Proceedings of Machine L...

    2020

  18. [26]

    Lattimore and C

    T. Lattimore and C. Szepesv \'a ri. Bandit algorithms. Cambridge University Press, 2020

    2020

  19. [27]

    J. Mockus. The bayesian approach to global optimization. In System Modeling and Optimization: Proceedings of the 10th IFIP Conference New York City, USA, August 31--September 4, 1981, pages 473--481. Springer, 2005

    1981

  20. [28]

    S. A. Murphy. An experimental design for the development of adaptive treatment strategies. Statistics in medicine, 24 0 (10): 0 1455--1481, 2005

    2005

  21. [29]

    Offer-Westort, A

    M. Offer-Westort, A. Coppock, and D. P. Green. Adaptive experimental design: Prospects and applications in political science. American Journal of Political Science, 65 0 (4): 0 826--844, 2021

    2021

  22. [30]

    Rusmevichientong and J

    P. Rusmevichientong and J. N. Tsitsiklis. Linearly parameterized bandits. Mathematics of Operations Research, 35 0 (2): 0 395--411, 2010

    2010

  23. [31]

    Satyal, I

    S. Satyal, I. Weber, H.-y. Paik, C. Di Ciccio, and J. Mendling. Ab testing for process versions with contextual multi-armed bandit algorithms. In Advanced Information Systems Engineering: 30th International Conference, CAiSE 2018, Tallinn, Estonia, June 11-15, 2018, Proceeding...

    2018

  24. [32]

    K. Smith. On the standard deviations of adjusted and interpolated values of an observed polynomial function and its constants and the guidance they give towards a proper choice of the distribution of observations. Biometrika, 12 0 (1/2): 0 1--85, 1918

    1918

  25. [33]

    Soare, A

    M. Soare, A. Lazaric, and R. Munos. Best-arm identification in linear bandits. Advances in Neural Information Processing Systems, 27, 2014

    2014

  26. [34]

    C. Tao, S. Blanco, and Y. Zhou. Best arm identification in linear bandits with linear dimension dependency. In International Conference on Machine Learning, pages 4877--4886. PMLR, 2018

    2018

  27. [35]

    J. A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of computational mathematics, 12 0 (4): 0 389--434, 2012

    2012

  28. [36]

    J. A. Tropp. An introduction to matrix concentration inequalities. Foundations and Trends in Machine Learning , 8 0 (1-2): 0 1--230, 2015

    2015

  29. [37]

    Xu and A

    Y. Xu and A. Zeevi. Upper counterfactual confidence bounds: a new optimism principle for contextual bandits. arXiv preprint arXiv:2007.07876, 2020

    2007 arXiv

  30. [38]

    Zhu and V

    Q. Zhu and V. Tan. Thompson sampling algorithms for mean-variance bandits. In International Conference on Machine Learning, pages 11599--11608. PMLR, 2020

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.