Pith. sign in

REVIEW 2 major objections 5 minor 1 cited by

Offline-to-online hyperparameter transfer for stochastic bandits

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Near-optimal bandit hyperparameters are learnable from finitely many offline runs, with explicit sample complexity bounds; per-instance tuning is provably impossible.

desk verdict Solid inter-task transfer framework and QD bounds, but the headline intra-task guarantee under bandit feedback is unsupported; worth peer review, not acceptance as-is. read the letter →

arxiv 2501.02926 v1 pith:BLMW6FFR submitted 2025-01-06 cs.LG

Dravyansh Sharma , Arun Sai Suggala This is my paper · ORCID
classification cs.LG MSC 68T0562L0568Q32
keywords stochasticbanditshyperparametertuningtransferlearningsamplecomplexityUCBLinGP-UCBderandomizeddual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether good hyperparameters for stochastic bandit algorithms can be learned once and reused across similar problems. It shows that tuning for a single unknown problem is information-theoretically impossible, even for the simplest multi-armed bandit, and instead studies transfer: given offline bandit runs from several tasks drawn from a fixed but unknown distribution, how many tasks and how many arm pulls are needed to pick a hyperparameter that is nearly optimal on a new task? The answers are expressed through a new quantity, the derandomized dual complexity QD, which counts the effective number of regimes in the loss landscape. The main results bound QD for UCB, LinUCB, and GP-UCB, giving inter-task sample complexity that grows logarithmically in QD and intra-task complexity of at most min{n, QD}T pulls, alongside experiments showing the learned parameter beats corralling baselines.

What carries the argument

The central object is the derandomized dual complexity QD = E_{P~D, z~D_P} q($l^{{P,z}}$_T(·)), the expected number of constant pieces in the loss $l^{{P,z}}$_T(ρ) when the random rewards are fixed before the horizon. This quantity is the distribution-dependent complexity measure that controls the sample complexity: uniform convergence follows from bounding the Rademacher complexity of the piecewise-constant function class via Massart's lemma, and the ERM is made tractable by computing the critical points of α at which the chosen arm changes, recursively, in expected time O(QD). Pseudo-dimension arguments extend the analysis to the two-hyperparameter (α, prior means) case and to GP-UCB's noise parameter.

What would settle it

Take a two-armed bandit with fixed reward draws, record only pulled-arm rewards under the piece-by-piece policy of Theorem 6.2 when n > QD, and compare the empirical loss recovered to the true loss computed from the complete reward realization; if they differ for some α, the intra-task budget min{n, QD}T cannot support the claimed guarantee under bandit feedback.

Watch

Extended reading notes

Core claim

The paper claims that the expected loss of a bandit algorithm as a function of its hyperparameter is, for a fixed reward realization, a piecewise constant function, and that the number of pieces QD controls both how many offline tasks are needed and how many pulls each task requires. Theorem 6.1 gives a uniform-convergence bound: N = O((H/ε)^2 (log QD + log 1/δ)) offline tasks suffice to guarantee that the empirical loss approximates the expected loss uniformly over the hyperparameter space. Theorem 6.2 provides an offline data collection policy achieving near-optimal transfer with an expected intra-task pull count of min{n, QD}T, and the ERM over the critical points of the piecewise loss realizes the bound. Specializing QD yields log QD = O(n log T) for UCB, O(log KT) for categorical rewards, O(T log n) for LinUCB, and a corresponding O(T log nT) bound for GP-UCB's noise parameter.

Load-bearing premise

The central claim depends on the assumption that the derandomized dual function $l^{{P,z}}$_T(ρ) is piecewise constant and that it can be computed from offline bandit feedback within the stated intra-task pull budget, even though bandit feedback reveals rewards only for pulled arms, not the full realization needed to evaluate the loss.

Editorial extensions

If this is right

  • A practitioner who has collected offline bandit data from related tasks can provably transfer the exploration parameter: roughly O(n log T / ε²) tasks with at most min{n, QD}T pulls each give an ε-suboptimal hyperparameter on a new task.
  • For categorical (e.g., Bernoulli) rewards the inter-task requirement improves to O(log(KT)/ε²), making the approach practical for small horizons.
  • The same machinery tunes the GP-UCB noise parameter with O(T log nT / ε²) tasks, which is most useful when the online horizon T is small, as in expensive evaluations.
  • The ERM over critical points is computationally efficient, running in expected time O(QD), so the theoretical bounds correspond to a feasible algorithm, not just an existence proof.
  • Learning both the exploration parameter and arm priors simultaneously is also possible with inter-task complexity O((n+T)T log n / ε²).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's intra-task policy for n > QD runs the algorithm once per piece of the derandomized loss; under pure bandit feedback only the pulled arm's reward is seen, so computing the loss on a piece may require the full n×T reward matrix, which costs nT pulls per task rather than QDT.
  • The small empirical values of QD suggest that the worst-case-looking bounds translate into modest data requirements in practice, and that the piecewise structure could be exploited by other meta-learning pipelines beyond UCB-style algorithms.
  • Strategic design of the offline data collection policy (which tasks, which pulls) is left open; the paper's bounds assume a fixed policy, so adaptively choosing pulls could lower intra-task complexity below min{n, QD}T.
  • The impossibility result implies that any fully online hyperparameter tuner must either pay suboptimal regret or rely on structure across tasks, which sharpens the practical motivation for transfer-based tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper studies offline-to-online transfer of hyperparameters for stochastic bandits. The authors define a task distribution over bandit problems, introduce a derandomized dual complexity QD that counts the expected number of piecewise-constant regions of the per-task loss as a function of the hyperparameter, and prove an inter-task sample complexity bound N = O((H/epsilon)^2 (log QD + log 1/delta)) for uniform convergence of the empirical loss. They further claim an intra-task sample complexity of min{n, QD}T pulls per task, instantiate QD for UCB, LinUCB, and GP-UCB, present an impossibility result for tuning without transfer, and report experiments against corralling baselines.

Significance. If the results held, the paper would provide a clean application of data-driven algorithm design to stochastic bandits, with a useful separation of inter-task and intra-task sample complexity. The inter-task uniform convergence argument via piecewise-constant derandomized losses is plausible and may be of independent interest, and the piece-count bounds for UCB, LinUCB, and GP-UCB are competent. However, the central intra-task claim is invalid under bandit feedback: the proposed policy cannot compute the loss function it needs, and the experiments do not validate the claimed bandit-feedback regime. Since the abstract and Section 6 advertise this intra-task complexity as a main contribution, the paper in its current form does not establish its principal claim.

major comments (2)
  1. [Section 6, Theorem 6.2 and Appendix D] The claim that an offline policy with E[To] = min{n, QD}T suffices for the ERM is not supported in the bandit-feedback setting. For a fixed realization z, the loss l^{P,z}_T(rho) is regret against the best arm, i.e., max_a sum_t r_{t,a} - sum_t r_{t,A_t(rho)}. A single run of A_rho observes only the rewards of pulled arms; the best-arm baseline may involve arms pulled rarely or never, so it is unobservable. Two reward realizations that agree on all pulled-arm rewards but differ on un-pulled arms produce identical offline data but different l^{P,z}_T(rho). Moreover, the piece boundaries themselves cannot be computed online: Algorithm 3 recursively computes critical points from the future reward arrays R_i for all arms, and Appendix D's 'sequentially compute the pieces' presupposes access to the full reward realization. Under bandit feedback, neither the pieces nor the per-piece loss are computable, so the ERM objective in Algorithm 2 cannot be evaluated. This is confirmed by Algorithm 2's input, which is the complete n x T reward matrix for each task, i.e., nT pulls per task. The n <= QD case is valid, but the n > QD case, which delivers the advertised min{n, QD}T bound, is unsupported. The conclusion's remark that the intra-task bound 'can be improved with more careful arguments' understates the problem: the proposed policy is not merely suboptimal but infeasible.
  2. [Section 9 and Appendix I] The experiments do not provide evidence for the claimed bandit-feedback intra-task complexity. Algorithm 2 requires the full n x T reward matrix as input, and in the CIFAR experiments the offline data consist of the classification accuracy for all 11 learning-rate arms over To = 20 steps, which is full-information data collection with cost n*To per task. The synthetic experiments similarly appear to generate complete reward matrices. Thus the empirical protocol corresponds to the full-information variant (or to the n <= QD case), not to a bandit-feedback policy with E[To] = min{n, QD}T. The advertised practical relevance of low intra-task complexity in the bandit setting is therefore not validated by the experiments.
minor comments (5)
  1. [Appendix E, proof of Theorem 7.1] The line 'Adding up over all time steps, we have at most (T - n)(T - 1)^{n-1} <= T^{n-1} critical points' is arithmetically incorrect: the hockey-stick identity gives sum_{t=n+1}^T C(t-1, n-1) = C(T, n), which is O(T^n), not O(T^{n-1}). The final conclusion log QD = O(n log T) is unaffected, but the proof should be corrected.
  2. [Section 7 and Appendix J] The text refers to 'Figure ??' for the variation of expected regret with alpha, but the figure is not properly referenced or included in the main text; please fix the placeholder.
  3. [Section 5] The symbol P is used both for the parameter space (P subset of R^d) and for a problem instance (P in Pi), which is confusing; consider renaming one of them.
  4. [Throughout] There are several typos and missing references: 'addtitional' in Section 9, 'piecewise constant function..' in Theorem 6.1, 'Barlett' in Appendices C and H (should be Bartlett), and 'CIFAR-10 and CIFAR-100 [?]' missing citation.
  5. [Algorithm 2] The return statement minimizes sum_k l^{Pk,zk}_T(alpha) over alpha, but does not specify tie-breaking and does not state how l^{Pk,zk}_T(alpha) is computed under bandit feedback; this is connected to the first major comment.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation identified; the main weakness is a bandit-feedback observability gap in Theorem 6.2, not a circular reduction.

full rationale

Walking the derivation chain, the paper's main results are supported by external learning-theoretic machinery rather than by re-using the conclusions as assumptions. Theorem 6.1 applies Massart's lemma and standard Rademacher complexity bounds to the class of derandomized dual functions, using the piecewise-constant structure to bound the supremum by the log of the total number of pieces; no step defines the target expected loss in terms of itself. Theorem 6.2 is the usual ERM argument combined with the same uniform-convergence bound. The QD bounds for UCB, LinUCB, and GP-UCB are computed from the algorithms' tie conditions, critical points, rational-function structure, and pseudo-dimension arguments, not fitted to the desired sample-complexity conclusion. The only self-citation with overlapping authorship, [BKST22], supplies an algebraic lemma about entries of (K_t + sI)^{-1} being rational functions of degree at most t; this is an externally checkable mathematical fact and is not load-bearing in a circular sense. The most serious issue in the manuscript is a correctness gap rather than circularity: Appendix D's n > QD case claims 'For any single piece, we need at most T arm pulls to compute the loss over the piece and there are QD pieces in expectation,' but under bandit feedback a single run of the algorithm reveals rewards only for pulled arms, while the regret-based loss l^{P,z}_T(rho) requires the reward of the best arm, which may never be pulled. Two reward realizations identical on pulled arms but different on un-pulled arms yield different losses yet identical offline data. This is a missing observability argument, not a re-use of the theorem's conclusion as an assumption, so it does not fit any of the seven circularity kinds and does not raise the circularity score under the specified taxonomy.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

The central claim rests on the standard stochastic bandit and learning-theory assumptions listed above. No free parameters are fitted: the paper derives bounds in terms of QD and problem parameters, and QD itself is a defined complexity measure rather than an adjusted constant. No new entities are introduced.

assumptions (7)
  • domain assumption Tasks are drawn iid from a fixed unknown distribution D over problem instances.
    This is the transfer learning premise stated in Section 5, needed for the uniform convergence over N offline tasks.
  • domain assumption Arm rewards are iid over time for each arm in the stochastic bandit setting.
    Used in the Theorem 7.1 proof to bound the number of count vectors and critical points; without iid rewards the number of discontinuities can be 2^Omega(T) per the paper's own remark.
  • domain assumption The derandomized dual loss l^{P,z}_T(rho) is piecewise constant in rho.
    Definition 1 and Theorem 6.1 require piecewise-constant dual functions; this holds for UCB, LinUCB, and GP-UCB on a fixed reward realization z.
  • domain assumption Loss metric l_T is bounded in [0,H] with H independent of T.
    Required for Massart's lemma and the uniform convergence bounds in Theorem 6.1.
  • standard math Lemma C.2 of [BKST22]: entries of (K_t + sI)^{-1} are rational functions of s with degree at most t.
    Imported from published prior work, co-authored by the present first author, to establish the piecewise structure of GP-UCB acquisition functions in Theorem 8.1.
  • domain assumption Gaussian rewards with variance in [0,B^2] for the impossibility lower bound.
    Theorem 4.1's model class M; the KL-inf lower bound and the tuned-UCB upper bound are computed under this assumption.
  • standard math Standard Rademacher and pseudo-dimension uniform convergence results (Massart's lemma, Bartlett-Mendelson, Anthony-Bartlett).
    Underpin Theorems 6.1, 6.2, 8.1, and 7.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Offline-to-online hyperparameter transfer for stochastic bandits." pith.science (2026). https://pith.science/paper/BLMW6FFR

@misc{pith2026250102926,
  author       = {Pith},
  title        = {Pith review of: Offline-to-online hyperparameter transfer for stochastic bandits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BLMW6FFR}},
  note         = {Machine review of arXiv:2501.02926}
}
read the original abstract

Classic algorithms for stochastic bandits typically use hyperparameters that govern their critical properties such as the trade-off between exploration and exploitation. Tuning these hyperparameters is a problem of great practical significance. However, this is a challenging problem and in certain cases is information theoretically impossible. To address this challenge, we consider a practically relevant transfer learning setting where one has access to offline data collected from several bandit problems (tasks) coming from an unknown distribution over the tasks. Our aim is to use this offline data to set the hyperparameters for a new task drawn from the unknown distribution. We provide bounds on the inter-task (number of tasks) and intra-task (number of arm pulls for each task) sample complexity for learning near-optimal hyperparameters on unseen tasks drawn from the distribution. Our results apply to several classic algorithms, including tuning the exploration parameters in UCB and LinUCB and the noise parameter in GP-UCB. Our experiments indicate the significance and effectiveness of the transfer of hyperparameters from offline problems in online learning with stochastic bandit feedback.

Figures

Figures reproduced from arXiv: 2501.02926 by the authors.

Figure 1
Figure 1. Variation of (estimated) expected regret with the exploration parameter [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Algorithm 2 to corralling based algorithms CORRAL [ALNS17] and CORRAL￾STOCHASTIC [AMM21]. real-valued functions f : C → [0, H] with C ⊂ R d . It is typical to discretize the domain C when computing the argmax in Algorithm 5, and usually f(·) is more expensive to evaluate than the UCB acquisition function at(x) := µt(x) + √ βtσt(x) for any point x on the finite discretization C˜ of C with |C| ˜ = n. The… view at source ↗
Figure 3
Figure 3. Comparison of Algorithm 2 to corralling based algorithms CORRAL [ALNS17] and CORRAL￾STOCHASTIC [AMM21] on 2-arm bandits with large arm-reward gap. theory is that we never restart the corralling algorithms, similar to previous experiments by [AMM21]. The regret bounds from theory remain meaningful for this variant in practice. Details of empirical runs. Since corralling methods can only be implemented for a finite nu… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of Algorithm 2 with corralling baselines on 2-arm bandits synthetic datasets with uniform and Gaussian arm reward distributions. is same as in the Bernoulli case above. Our algorithm outperforms both corralling algorithms in either case, with the difference …
Figure 5
Figure 5. Figure 5: Variation of regret on test (online) tasks with number of training tasks [PITH_FULL_IMAGE:figures/full_fig_p027_5.png]
Figure 6
Figure 6. Figure 6: Variation of total reward with σ 2 parameter in GP-UCB on a 24 × 24 two-dimensional grid, for T = 20. We observe that QD ≪ n in each case. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. KABB: Knowledge-Aware Bayesian Bandits for Dynamic Expert Coordination in Multi-Agent Systems

    cs.AI 2025-02 reject novelty 5.0 of 10

    A knowledge-aware Bayesian bandit that selects specialist LLM experts per task is reported to beat Mixture-of-Agents on AlpacaEval 2.0 and MT-Bench while using fewer models.

Reference graph

Works this paper leans on

65 extracted references · 61 canonical work pages · cited by 1 Pith paper

  1. [1]

    Neural network learning: Theoretical foundations , volume 9

    Martin Anthony and Peter L Bartlett. Neural network learning: Theoretical foundations , volume 9. Cambridge University Press, 1999

    work page 1999

  2. [2]

    Population-based black-box optimization for biological sequence design

    Christof Angermueller, David Belanger, Andreea Gane, Zelda Mariet, David Dohan, Kevin Murphy, Lucy Colwell, and D Sculley. Population-based black-box optimization for biological sequence design. In International Conference on Machine Learning , pages 324--334. PMLR, 2020

    work page 2020

  3. [3]

    Finite-time analysis of the multiarmed bandit problem

    Peter Auer, Nicolo Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning , 47(2):235--256, 2002

    work page 2002

  4. [4]

    Meta-learning for simple regret minimization

    Javad Azizi, Branislav Kveton, Mohammad Ghavamzadeh, and Sumeet Katariya. Meta-learning for simple regret minimization. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 6709--6717, 2023

    work page 2023

  5. [5]

    Sequential transfer in multi-armed bandit with finite set of models

    Mohammad Gheshlaghi Azar, Alessandro Lazaric, and Emma Brunskill. Sequential transfer in multi-armed bandit with finite set of models. In Advances in Neural Information Processing Systems , 2013

    work page 2013

  6. [6]

    Corralling a band of bandit algorithms

    Alekh Agarwal, Haipeng Luo, Behnam Neyshabur, and Robert E Schapire. Corralling a band of bandit algorithms. In Conference on Learning Theory , pages 12--38. PMLR, 2017

    work page 2017

  7. [7]

    Corralling stochastic bandit algorithms

    Raman Arora, Teodor Vanislavov Marinov, and Mehryar Mohri. Corralling stochastic bandit algorithms. In International Conference on Artificial Intelligence and Statistics , pages 2116--2124. PMLR, 2021

    work page 2021

  8. [8]

    Exploration--exploitation tradeoff using variance estimates in multi-armed bandits

    Jean-Yves Audibert, R \'e mi Munos, and Csaba Szepesv \'a ri. Exploration--exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science , 410(19):1876--1902, 2009

    work page 1902

Show all 65 references
  1. [9]

    Improved algorithms for linear stochastic bandits

    Yasin Abbasi-Yadkori, D \'a vid P \'a l, and Csaba Szepesv \'a ri. Improved algorithms for linear stochastic bandits. Advances in Neural Information Processing Systems , 24, 2011

    2011

  2. [10]

    Book chapter Data-Driven Algorithm Design

    Maria-Florina Balcan. Book chapter Data-Driven Algorithm Design . In Beyond Worst Case Analysis of Algorithms, T. Roughgarden (Ed) . Cambridge University Press, 2020

    2020

  3. [11]

    Algorithms for hyper-parameter optimization

    James Bergstra, R \'e mi Bardenet, Yoshua Bengio, and Bal \'a zs K \'e gl. Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems , 24, 2011

    2011

  4. [12]

    An analysis of robustness of non- L ipschitz networks

    Maria-Florina Balcan, Avrim Blum, Dravyansh Sharma, and Hongyang Zhang. An analysis of robustness of non- L ipschitz networks. Journal of Machine Learning Research , 24(98):1--43, 2023

    2023

  5. [13]

    How much data is sufficient to learn high-performing algorithms? G eneralization guarantees for data-driven algorithm design

    Maria-Florina Balcan, Dan DeBlasio, Travis Dick, Carl Kingsford, Tuomas Sandholm, and Ellen Vitercik. How much data is sufficient to learn high-performing algorithms? G eneralization guarantees for data-driven algorithm design. In Proceedings of the 53rd Annual ACM SIGACT Symp...

    2021

  6. [14]

    Learning piecewise L ipschitz functions in changing environments

    Maria-Florina Balcan, Travis Dick, and Dravyansh Sharma. Learning piecewise L ipschitz functions in changing environments. In International Conference on Artificial Intelligence and Statistics , pages 3567--3577. PMLR, 2020

    2020

  7. [15]

    Learning complexity of simulated annealing

    Avrim Blum, Chen Dan, and Saeed Seddighin. Learning complexity of simulated annealing. In International Conference on Artificial Intelligence and Statistics , pages 1540--1548. PMLR, 2021

    2021

  8. [16]

    Dispersion for data-driven algorithm design, online learning, and private optimization

    Maria-Florina Balcan, Travis Dick, and Ellen Vitercik. Dispersion for data-driven algorithm design, online learning, and private optimization. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS) , pages 603--614. IEEE, 2018

    2018

  9. [17]

    Data-driven clustering via parameterized L loyd's families

    Maria-Florina Balcan, Travis Dick, and Colin White. Data-driven clustering via parameterized L loyd's families. Advances in Neural Information Processing Systems , 31, 2018

    2018

  10. [18]

    Generalization bounds for data-driven numerical linear algebra

    Peter Bartlett, Piotr Indyk, and Tal Wagner. Generalization bounds for data-driven numerical linear algebra. In Conference on Learning Theory , pages 2013--2040. PMLR, 2022

    2013

  11. [19]

    Provably tuning the elasticnet across instances

    Maria-Florina Balcan, Mikhail Khodak, Dravyansh Sharma, and Ameet Talwalkar. Provably tuning the elasticnet across instances. Advances in Neural Information Processing Systems , 2022

    2022

  12. [20]

    Rademacher and G aussian complexities: R isk bounds and structural results

    Peter L Bartlett and Shahar Mendelson. Rademacher and G aussian complexities: R isk bounds and structural results. Journal of Machine Learning Research , 3(Nov):463--482, 2002

    2002

  13. [21]

    Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems

    Maria-Florina Balcan, Vaishnavh Nagarajan, Ellen Vitercik, and Colin White. Learning-theoretic foundations of algorithm configuration for combinatorial partitioning problems. In Conference on Learning Theory , pages 213--274. PMLR, 2017

    2017

  14. [22]

    Data driven semi-supervised learning

    Maria-Florina Balcan and Dravyansh Sharma. Data driven semi-supervised learning. Advances in Neural Information Processing Systems , 34:14782--14794, 2021

    2021

  15. [23]

    Learning accurate and interpretable decision trees

    Maria Florina Balcan and Dravyansh Sharma. Learning accurate and interpretable decision trees. In The 40th Conference on Uncertainty in Artificial Intelligence , 2024

    2024

  16. [24]

    Stochastic and adversarial online learning without hyperparameters

    Ashok Cutkosky and Kwabena A Boahen. Stochastic and adversarial online learning without hyperparameters. Advances in Neural Information Processing Systems , 30, 2017

    2017

  17. [25]

    Upper confidence bounds for combining stochastic bandits

    Ashok Cutkosky, Abhimanyu Das, and Manish Purohit. Upper confidence bounds for combining stochastic bandits. arXiv preprint arXiv:2012.13115 , 2020

    2012 arXiv

  18. [26]

    Kullback- L eibler upper confidence bounds for optimal sequential allocation

    Olivier Capp \'e , Aur \'e lien Garivier, Odalric-Ambrym Maillard, R \'e mi Munos, and Gilles Stoltz. Kullback- L eibler upper confidence bounds for optimal sequential allocation. The Annals of Statistics , pages 1516--1541, 2013

    2013

  19. [27]

    Syndicated bandits: A framework for auto tuning hyper-parameters in contextual bandit algorithms

    Qin Ding, Yue Kang, Yi-Wei Liu, Thomas Chun Man Lee, Cho-Jui Hsieh, and James Sharpnack. Syndicated bandits: A framework for auto tuning hyper-parameters in contextual bandit algorithms. Advances in Neural Information Processing Systems , 35:1170--1181, 2022

    2022

  20. [28]

    Bootstrap methods: another look at the jackknife

    Bradley Efron. Bootstrap methods: another look at the jackknife. In Breakthroughs in statistics: Methodology and distribution , pages 569--593. Springer, 1992

    1992

  21. [29]

    Adapting to misspecification in contextual bandits

    Dylan J Foster, Claudio Gentile, Mehryar Mohri, and Julian Zimmert. Adapting to misspecification in contextual bandits. Advances in Neural Information Processing Systems , 33:11478--11489, 2020

    2020

  22. [30]

    Model selection for contextual bandits

    Dylan J Foster, Akshay Krishnamurthy, and Haipeng Luo. Model selection for contextual bandits. Advances in Neural Information Processing Systems , 32, 2019

    2019

  23. [31]

    Online meta-learning

    Chelsea Finn, Aravind Rajeswaran, Sham Kakade, and Sergey Levine. Online meta-learning. In International Conference on Machine Learning , pages 1920--1930. PMLR, 2019

    1920

  24. [32]

    The KL-UCB algorithm for bounded stochastic bandits and beyond

    Aur \'e lien Garivier and Olivier Capp \'e . The KL-UCB algorithm for bounded stochastic bandits and beyond. In Proceedings of the 24th annual conference on learning theory , pages 359--376. JMLR Workshop and Conference Proceedings, 2011

    2011

  25. [33]

    A PAC approach to application-specific algorithm selection

    Rishi Gupta and Tim Roughgarden. A PAC approach to application-specific algorithm selection. SIAM Journal on Computing , 46(3):992--1017, 2017

    2017

  26. [34]

    Distributionally Robust Bayesian Optimization with -divergences

    Hisham Husain, Vu Nguyen, and Anton van den Hengel. Distributionally Robust Bayesian Optimization with -divergences. Advances in Neural Information Processing Systems (NeurIPS) , 36, 2023

    2023

  27. [35]

    Parameter-free multi-armed bandit algorithms with hybrid data-dependent regret bounds

    Shinji Ito. Parameter-free multi-armed bandit algorithms with hybrid data-dependent regret bounds. In Conference on Learning Theory , pages 2552--2583. PMLR, 2021

    2021

  28. [36]

    Optimal model selection in contextual bandits with many classes via offline oracles

    Sanath Kumar Krishnamurthy and Susan Athey. Optimal model selection in contextual bandits with many classes via offline oracles. arXiv preprint arXiv:2106.06483 , 2021

    2021 arXiv

  29. [37]

    Adaptive gradient-based meta-learning methods

    Mikhail Khodak, Maria-Florina Balcan, and Ameet Talwalkar. Adaptive gradient-based meta-learning methods. Advances in Neural Information Processing Systems , 32, 2019

    2019

  30. [38]

    A bound on the error of cross validation using the approximation and estimation rates, with consequences for the training-test split

    Michael Kearns. A bound on the error of cross validation using the approximation and estimation rates, with consequences for the training-test split. Advances in Neural Information Processing Systems , 8, 1995

    1995

  31. [39]

    Online continuous hyperparameter optimization for generalized linear contextual bandits

    Yue Kang, Cho-Jui Hsieh, and Thomas Lee. Online continuous hyperparameter optimization for generalized linear contextual bandits. Transactions on Machine Learning Research , 2024

    2024

  32. [40]

    Meta-learning bandit policies by gradient ascent

    Branislav Kveton, Martin Mladenov, Chih-Wei Hsu, Manzil Zaheer, Csaba Szepesvari, and Craig Boutilier. Meta-learning bandit policies by gradient ascent. arXiv preprint arXiv:2006.05094 , 2020

    2006 arXiv

  33. [41]

    Meta-learning adversarial bandit algorithms

    Mikhail Khodak, Ilya Osadchiy, Keegan Harris, Maria-Florina Balcan, Kfir Y Levy, Ron Meir, and Zhiwei Steven Wu. Meta-learning adversarial bandit algorithms. Advances in Neural Information Processing Systems , 2023

    2023

  34. [42]

    Rademacher penalties and structural risk minimization

    Vladimir Koltchinskii. Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory , 47(5):1902--1914, 2001

    1902

  35. [43]

    A contextual-bandit approach to personalized news article recommendation

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web , pages 661--670, 2010

    2010

  36. [44]

    Automatic gait optimization with gaussian process regression

    Daniel J Lizotte, Tao Wang, Michael H Bowling, Dale Schuurmans, et al. Automatic gait optimization with gaussian process regression. In IJCAI , volume 7, pages 944--949, 2007

    2007

  37. [45]

    Corralling a larger band of bandits: A case study on switching regret for linear bandits

    Haipeng Luo, Mengxiao Zhang, Peng Zhao, and Zhi-Hua Zhou. Corralling a larger band of bandits: A case study on switching regret for linear bandits. In Conference on Learning Theory , pages 3635--3684. PMLR, 2022

    2022

  38. [46]

    Some applications of concentration inequalities to statistics

    Pascal Massart. Some applications of concentration inequalities to statistics. In Annales de la Facult \'e des sciences de Toulouse: Math \'e matiques , volume 9, pages 245--303, 2000

    2000

  39. [47]

    Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health

    Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. Field study in deploying restless multi-armed bandits: Assisting non-profits in improving maternal and child health. In Proceedings of t...

    2022

  40. [48]

    Efficient- UCBV : An almost optimal algorithm using variance estimates

    Subhojyoti Mukherjee, KP Naveen, Nandan Sudarsanam, and Balaraman Ravindran. Efficient- UCBV : An almost optimal algorithm using variance estimates. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018

    2018

  41. [49]

    The pareto frontier of model selection for general contextual bandits

    Teodor Vanislavov Marinov and Julian Zimmert. The pareto frontier of model selection for general contextual bandits. Advances in Neural Information Processing Systems , 34:17956--17967, 2021

    2021

  42. [50]

    Best of both worlds model selection

    Aldo Pacchiano, Christoph Dann, and Claudio Gentile. Best of both worlds model selection. Advances in Neural Information Processing Systems , 35:1883--1895, 2022

    2022

  43. [51]

    Handling advertisements of unknown quality in search advertising

    Sandeep Pandey and Christopher Olston. Handling advertisements of unknown quality in search advertising. Advances in Neural Information Processing Systems , 19, 2006

    2006

  44. [52]

    Model selection in contextual stochastic bandit problems

    Aldo Pacchiano, My Phan, Yasin Abbasi Yadkori, Anup Rao, Julian Zimmert, Tor Lattimore, and Csaba Szepesvari. Model selection in contextual stochastic bandit problems. Advances in Neural Information Processing Systems , 33:10328--10337, 2020

    2020

  45. [53]

    Metalearning linear bandits by prior update

    Amit Peleg, Naama Pearl, and Ron Meir. Metalearning linear bandits by prior update. In International Conference on Artificial Intelligence and Statistics , pages 2885--2926. PMLR, 2022

    2022

  46. [54]

    Gaussian process optimization in the bandit setting: No regret and experimental design

    Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias W Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In International Conference on Machine Learning , 2010

    2010

  47. [55]

    Multi-task bayesian optimization

    Kevin Swersky, Jasper Snoek, and Ryan P Adams. Multi-task bayesian optimization. Advances in Neural Information Processing Systems , 26, 2013

    2013

  48. [56]

    Cross-validatory choice and assessment of statistical predictions

    Mervyn Stone. Cross-validatory choice and assessment of statistical predictions. Journal of the royal statistical society: Series B (Methodological) , 36(2):111--133, 1974

    1974

  49. [57]

    Wasserstein distributionally robust policy evaluation and learning for contextual bandits

    Yi Shen, Pan Xu, and Michael Zavlanos. Wasserstein distributionally robust policy evaluation and learning for contextual bandits. Transactions on Machine Learning Research (TMLR) , 2023

    2023

  50. [58]

    Distributionally robust policy evaluation and learning in offline contextual bandits

    Nian Si, Fan Zhang, Zhengyuan Zhou, and Jose Blanchet. Distributionally robust policy evaluation and learning in offline contextual bandits. In International Conference on Machine Learning (ICML) , pages 8884--8894. PMLR, 2020

    2020

  51. [59]

    From ads to interventions: Contextual bandits in mobile health

    Ambuj Tewari and Susan A Murphy. From ads to interventions: Contextual bandits in mobile health. Mobile health: sensors, analytic methods, and applications , pages 495--517, 2017

    2017

  52. [60]

    Pre-trained G aussian processes for B ayesian optimization

    Zi Wang, George E Dahl, Kevin Swersky, Chansoo Lee, Zelda Mariet, Zachary Nado, Justin Gilmer, Jasper Snoek, and Zoubin Ghahramani. Pre-trained G aussian processes for B ayesian optimization. Journal of Machine Learning Research , 25(212):1--83, 2024

    2024

  53. [61]

    Regret bounds for meta bayesian optimization with an unknown G aussian process prior

    Zi Wang, Beomjoon Kim, and Leslie P Kaelbling. Regret bounds for meta bayesian optimization with an unknown G aussian process prior. Advances in Neural Information Processing Systems , 31, 2018

    2018

  54. [62]

    Improved self-normalized concentration in H ilbert spaces: Sublinear regret for GP-UCB

    Justin Whitehouse, Zhiwei Steven Wu, and Aaditya Ramdas. Improved self-normalized concentration in H ilbert spaces: Sublinear regret for GP-UCB . arXiv preprint arXiv:2307.07539 , 2023

    2023 arXiv

  55. [63]

    Consistency of cross validation for comparing regression procedures

    Yuhong Yang. Consistency of cross validation for comparing regression procedures. The Annals of Statistics , pages 2450--2473, 2007

    2007

  56. [64]

    Efficient transfer learning method for automatic hyperparameter tuning

    Dani Yogatama and Gideon Mann. Efficient transfer learning method for automatic hyperparameter tuning. In Artificial Intelligence and Statistics , pages 1077--1085. PMLR, 2014

    2014

  57. [65]

    Tsallis-inf: An optimal algorithm for stochastic and adversarial bandits

    Julian Zimmert and Yevgeny Seldin. Tsallis-inf: An optimal algorithm for stochastic and adversarial bandits. Journal of Machine Learning Research , 22(28):1--49, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.