Pith. sign in

REVIEW 2 major objections 1 minor 67 references

Stochastic Linear Contextual Bandits with Bounded Noise: A Set-Membership Approach

T0 review · 2 major / 1 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For stochastic linear contextual bandits with bounded reward noise, set-membership estimation plus optimism in the face of uncertainty yields an O(log T) regret bound.

desk verdict The O(log T) regret claim for bounded-noise SLCB collides with standard Ω(√T) lower bounds that still apply. read the letter →

arxiv 2606.20022 v1 pith:B4O6Y3WZ submitted 2026-06-18 stat.ML cs.LGmath.OC

Haonan Xu , Yingying Li This is my paper · ORCID
classification stat.MLcs.LGmath.OC
keywords stochasticlinearcontextualbanditsboundednoiseset-membershipestimationoptimisminthefaceofuncertaintyregretboundslogarithmic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that bounded reward noise, a stronger condition than the usual sub-Gaussian assumption, permits a new algorithm to achieve regret that grows only logarithmically with the horizon T. The approach constructs uncertainty sets for the unknown linear parameter via set-membership estimation and then selects optimistic actions inside those sets. Because the noise bound is used explicitly to shrink the feasible parameter region, the algorithm avoids the sqrt(T) scaling required under weaker noise models. Simulations confirm that the method outperforms standard algorithms designed for sub-Gaussian noise when the stronger bounded-noise condition holds.

What carries the argument

Set-membership estimation constructs a compact uncertainty set for the unknown parameter that is guaranteed to contain the true value whenever noise remains within fixed bounds; this set supplies the optimistic estimates used for action selection.

What would settle it

A concrete instance or simulation in which reward noise satisfies the boundedness assumption yet every algorithm, including SME-OFU, still incurs regret that grows faster than O(log T) would falsify the claimed improvement.

Watch

Extended reading notes

Core claim

The authors introduce the SME-OFU algorithm that applies set-membership estimation to form a bounded uncertainty set containing the true parameter vector consistent with all past observations under bounded noise. Optimism in the face of uncertainty is then invoked by choosing, at each step, the action that maximizes the reward under the most favorable parameter inside this set, producing a cumulative regret of O(log T).

Load-bearing premise

The difference between each observed reward and its conditional expectation is confined within fixed finite bounds for all time steps.

Editorial extensions

If this is right

  • Regret scales logarithmically rather than as sqrt(T) when the bounded-noise condition holds.
  • The algorithm can be applied directly in domains where realized rewards are naturally bounded.
  • Tighter uncertainty quantification from bounded noise produces fewer suboptimal pulls over long horizons.
  • Empirical results already show lower cumulative regret than sub-Gaussian baselines under the same bounded-noise setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same set-membership construction may transfer to other sequential decision problems whose disturbances admit hard bounds.
  • Logarithmic regret opens the possibility of stable performance over much longer time horizons than sqrt(T) scaling would allow.
  • The result underscores that matching the noise assumption precisely to the problem data can change the fundamental scaling of regret.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper proposes the SME-OFU algorithm for stochastic linear contextual bandits (SLCB) under the assumption of bounded reward noise. It uses set-membership estimation (SME) to form a polytope uncertainty set for the unknown parameter and applies the optimism-in-the-face-of-uncertainty (OFU) principle for action selection. The central claim is that this yields a regret bound of O(log T), which the authors argue does not contradict the standard ilde{O}(\sqrt{T}) bound because bounded noise is a strictly stronger assumption than sub-Gaussian noise; simulations are reported to show empirical gains over a sub-Gaussian benchmark.

Significance. If the O(log T) regret bound were valid under the stated assumptions, the result would be significant: it would demonstrate that the bounded-noise condition can be leveraged for exponentially better regret than is possible under sub-Gaussian noise alone. The use of set-membership estimation to obtain a shrinking uncertainty set is a technically interesting idea that could have broader applicability if the analysis holds.

major comments (2)
  1. [Abstract] Abstract (main regret claim): The asserted O(log T) regret bound is inconsistent with standard information-theoretic lower bounds of \Omega(\sqrt{dT}) for stochastic linear contextual bandits. These lower bounds are proved for bounded noise distributions (e.g., uniform or Bernoulli) that remain sub-Gaussian; the manuscript does not state any additional assumption (finite action set, deterministic contexts, or deterministic rewards) that would remove the lower-bound obstacle.
  2. [Abstract] Abstract (set-membership argument): The claim that the SME polytope diameter shrinks sufficiently fast to yield only logarithmic regret requires that sufficiently many distinct contexts are observed; in the worst-case bandit setting this still necessitates \Omega(\sqrt{T}) exploration to control linear estimation error in d dimensions, contradicting the stated O(log T) bound.
minor comments (1)
  1. [Abstract] The abstract states that bounded noise 'has not been leveraged explicitly before,' but does not cite or discuss prior works that already consider bounded rewards in linear bandits.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful and constructive review. We address the two major comments point by point below, indicating the revisions we will make to the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract (main regret claim): The asserted O(log T) regret bound is inconsistent with standard information-theoretic lower bounds of Ω(√(dT)) for stochastic linear contextual bandits. These lower bounds are proved for bounded noise distributions (e.g., uniform or Bernoulli) that remain sub-Gaussian; the manuscript does not state any additional assumption (finite action set, deterministic contexts, or deterministic rewards) that would remove the lower-bound obstacle.

    Authors: We appreciate the referee for identifying this inconsistency. Standard information-theoretic lower bounds of Ω(√(dT)) for stochastic linear contextual bandits do apply to bounded noise distributions (such as uniform or Bernoulli) that are sub-Gaussian. Our O(log T) claim was obtained by leveraging the stronger bounded-noise assumption via set-membership estimation, but the manuscript does not explicitly list additional assumptions (e.g., finite action sets or deterministic contexts) that would evade the lower bound. We will revise the abstract, introduction, and theoretical sections to state the precise conditions required for the O(log T) bound and to discuss its relationship to existing lower bounds. If reconciliation is not possible, the claimed regret rate will be adjusted. revision: yes

  2. Referee: [Abstract] Abstract (set-membership argument): The claim that the SME polytope diameter shrinks sufficiently fast to yield only logarithmic regret requires that sufficiently many distinct contexts are observed; in the worst-case bandit setting this still necessitates Ω(√T) exploration to control linear estimation error in d dimensions, contradicting the stated O(log T) bound.

    Authors: We agree that the SME polytope diameter shrinks at a rate that depends on observing sufficiently many distinct contexts. Our analysis assumes that the realized context sequence permits the uncertainty set to contract sufficiently fast under bounded noise. In the worst-case (adversarial) context setting, Ω(√T) exploration may indeed be required to control estimation error. We will add a dedicated discussion of the required context diversity assumptions and provide a refined analysis of the polytope shrinkage rate. This will constitute a partial revision to the theoretical development and any related claims. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation rests on bounded-noise assumption rather than self-reference.

full rationale

The abstract and provided text attribute the O(log T) regret directly to the use of set-membership estimation under the stronger bounded-noise condition, without any quoted reduction of the bound to a fitted parameter, self-citation chain, or renamed input. No self-definitional steps, fitted-input predictions, or load-bearing self-citations appear in the given material. The improvement is presented as a consequence of the modeling assumption, which is externally verifiable and does not reduce to the paper's own outputs by construction.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Only abstract available; the central claim rests on the domain assumption of bounded reward noise and the correctness of the SME-OFU construction, neither of which can be verified from the provided text.

assumptions (1)
  • domain assumption Reward noise is bounded
    This stronger condition is the key premise enabling the O(log T) regret claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stochastic Linear Contextual Bandits with Bounded Noise: A Set-Membership Approach." pith.science (2026). https://pith.science/paper/B4O6Y3WZ

@misc{pith2026260620022,
  author       = {Pith},
  title        = {Pith review of: Stochastic Linear Contextual Bandits with Bounded Noise: A Set-Membership Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4O6Y3WZ}},
  note         = {Machine review of arXiv:2606.20022}
}
abstract

This paper considers stochastic linear contextual bandits (SLCB) with bounded reward noise. Existing works typically assume sub-Gaussian reward noise and bounded expected rewards, under which the optimal regret bound scales as $\tilde{O}(\sqrt{T})$ in terms of horizon $T$. However, in many applications, realized/observed rewards are also naturally bounded, implying bounded reward noise. Bounded noise is more informative than the sub-Gaussian condition but has not been leveraged explicitly in the SLCB literature. In this paper, we propose a novel algorithm SME-OFU by utilizing an uncertainty quantification method called set-membership estimation (SME) and applying the principle of optimism in the face of uncertainty (OFU). Our algorithm enjoys an improved regret bound $O(\log T)$. Notice that this does not contradict the existing optimal bound $\tilde{O}(\sqrt{T})$ for sub-Gaussian noise because bounded noise is a stronger condition. Finally, simulations show empirical improvements of SME-OFU over a benchmark algorithm designed for sub-Gaussian noise when the reward noise is bounded.

Figures

Figures reproduced from arXiv: 2606.20022 by the authors.

Figure 1
Figure 1. The average regrets and standard deviations of SME-OFU and OFUL under uniform noises [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 6 canonical work pages

  1. [1]

    Improved algorithms for linear stochastic bandits.Advances in neural information processing systems, 24, 2011

    Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits.Advances in neural information processing systems, 24, 2011

    2011

  2. [2]

    Online Least Squares Estimation with Self-Normalized Processes: An Application to Bandit Problems

    Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Online least squares estimation with self-normalized processes: An application to bandit problems.arXiv preprint arXiv:1102.2670, 2011

    work page Pith review arXiv 2011

  3. [3]

    Associative reinforcement learning using linear probabilistic concepts

    Naoki Abe and Philip M Long. Associative reinforcement learning using linear probabilistic concepts. InICML, pages 3–11, 1999

    1999

  4. [4]

    Robust adaptive mpc for constrained uncertain nonlinear systems.International Journal of Adaptive Control and Signal Processing, 25(2):155–167, 2011

    Veronica Adetola and Martin Guay. Robust adaptive mpc for constrained uncertain nonlinear systems.International Journal of Adaptive Control and Signal Processing, 25(2):155–167, 2011

    2011

  5. [5]

    Thompson sampling for contextual bandits with linear payoffs

    Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. InInternational conference on machine learning, pages 127–135. PMLR, 2013

    2013

  6. [6]

    The size of the membership-set in a probabilistic framework.Automatica, 40 (2):253–260, 2004

    Hüseyin Akçay. The size of the membership-set in a probabilistic framework.Automatica, 40 (2):253–260, 2004

    2004

  7. [7]

    Convergence analysis of central and minimax algorithms in scalar regressor models.Mathematics of Control, Signals and Systems, 18(1):66–99, 2006

    Hüseyin Akçay and Nuray At. Convergence analysis of central and minimax algorithms in scalar regressor models.Mathematics of Control, Signals and Systems, 18(1):66–99, 2006

    2006

  8. [8]

    Using confidence bounds for exploitation-exploration trade-offs.Journal of machine learning research, 3(Nov):397–422, 2002

    Peter Auer. Using confidence bounds for exploitation-exploration trade-offs.Journal of machine learning research, 3(Nov):397–422, 2002

    2002

Show all 67 references
  1. [9]

    Convergence of optimal sequential outer bounding sets in bounded error parameter estimation.Mathematics and computers in simulation, 49(6):307–317, 1999

    Er-Wei Bai and Yih-Fang Huang. Convergence of optimal sequential outer bounding sets in bounded error parameter estimation.Mathematics and computers in simulation, 49(6):307–317, 1999

    1999

  2. [10]

    Er-Wei Bai, Roberto Tempo, and Hyonyong Cho. Membership set estimators: size, optimal inputs, complexity and relations with least squares.IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 42(5):266–277, 1995

    1995

  3. [11]

    Convergence properties of the membership set.Automatica, 34(10):1245–1249, 1998

    Er-Wei Bai, Hyonyong Cho, and Roberto Tempo. Convergence properties of the membership set.Automatica, 34(10):1245–1249, 1998

    1998

  4. [12]

    Dimitri P Bertsekas.Control of uncertain systems with a set-membership description of the uncertainty.PhD thesis, Massachusetts Institute of Technology, 1971

    1971

  5. [13]

    Active learning for stochastic contextual linear bandits

    Emma Brunskill, Ishani Karmarkar, and Zhaoqi Li. Active learning for stochastic contextual linear bandits. InNeurIPS 2025 Workshop MLxOR: Mathematical Foundations and Operational Integration of Machine Learning for Uncertainty-Aware Decision-Making

    2025

  6. [14]

    Marco Casini, Andrea Garulli, and Antonio Vicino. A linear programming approach to online set membership parameter estimation for linear regression models.International Journal of Adaptive Control and Signal Processing, 31(3):360–378, 2017

    2017

  7. [15]

    Thompson sampling for high- dimensional sparse linear contextual bandits

    Sunrit Chakraborty, Saptarshi Roy, and Ambuj Tewari. Thompson sampling for high- dimensional sparse linear contextual bandits. InInternational Conference on Machine Learning, pages 3979–4008. PMLR, 2023

    2023

  8. [16]

    Contextual restless multi-armed bandits with application to demand response decision-making

    Xin Chen and I-Hong Hou. Contextual restless multi-armed bandits with application to demand response decision-making. In2024 IEEE 63rd Conference on Decision and Control (CDC), pages 2652–2657. IEEE, 2024

    2024

  9. [17]

    Online residential demand response via contextual multi- armed bandits.IEEE Control Systems Letters, 5(2):433–438, 2020

    Xin Chen, Yutong Nie, and Na Li. Online residential demand response via contextual multi- armed bandits.IEEE Control Systems Letters, 5(2):433–438, 2020

    2020

  10. [18]

    Shuffle private linear contextual bandits.arXiv preprint arXiv:2202.05567, 2022

    Sayak Ray Chowdhury and Xingyu Zhou. Shuffle private linear contextual bandits.arXiv preprint arXiv:2202.05567, 2022. 11

    2022

  11. [19]

    Contextual bandits with linear payoff functions

    Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. InProceedings of the fourteenth international conference on artificial intelligence and statistics, pages 208–214. JMLR Workshop and Conference Proceedings, 2011

    2011

  12. [20]

    Balanced linear contextual bandits

    Maria Dimakopoulou, Zhengyuan Zhou, Susan Athey, and Guido Imbens. Balanced linear contextual bandits. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3445–3453, 2019

    2019

  13. [21]

    On the value of information in system identification—bounded noise case.Automatica, 18(2):229–238, 1982

    Eli Fogel and Yih-Fang Huang. On the value of information in system identification—bounded noise case.Automatica, 18(2):229–238, 1982

    1982

  14. [22]

    Beyond ucb: Optimal and efficient contextual bandits with regression oracles

    Dylan Foster and Alexander Rakhlin. Beyond ucb: Optimal and efficient contextual bandits with regression oracles. InInternational conference on machine learning, pages 3199–3210. PMLR, 2020

    2020

  15. [23]

    Peter M. Gruber. John and loewner ellipsoids.Discrete & Computational Geometry, 46(4): 776–788, 2011. doi: 10.1007/s00454-011-9354-8

    2011 doi

  16. [24]

    Robust and adaptive model predictive control of nonlinear systems

    Martin Guay, Veronica Adetola, and Darryl DeHaan. Robust and adaptive model predictive control of nonlinear systems. InRobust and Adaptive Model Predictive Control of Nonlinear Systems, pages 1–253. 2016

    2016

  17. [25]

    The extremal volume ellipsoids of convex bodies, their symme- try properties, and their determination in some special cases.arXiv preprint arXiv:0709.0707, 2007

    Osman Güler and Filiz Gürtuna. The extremal volume ellipsoids of convex bodies, their symme- try properties, and their determination in some special cases.arXiv preprint arXiv:0709.0707, 2007

    2007 arXiv

  18. [26]

    Adalinucb: Opportunistic learning for contextual bandits.arXiv preprint arXiv:1902.07802, 2019

    Xueying Guo, Xiaoxiao Wang, and Xin Liu. Adalinucb: Opportunistic learning for contextual bandits.arXiv preprint arXiv:1902.07802, 2019

    1902 arXiv

  19. [27]

    Contexts can be cheap: Solving stochastic contextual bandits with linear bandit algorithms

    Osama A Hanna, Lin Yang, and Christina Fragouli. Contexts can be cheap: Solving stochastic contextual bandits with linear bandit algorithms. InThe Thirty Sixth Annual Conference on Learning Theory, pages 1791–1821. PMLR, 2023

    2023

  20. [28]

    Nearly optimal algorithms for linear contextual bandits with adversarial corruptions.Advances in neural information processing systems, 35:34614–34625, 2022

    Jiafan He, Dongruo Zhou, Tong Zhang, and Quanquan Gu. Nearly optimal algorithms for linear contextual bandits with adversarial corruptions.Advances in neural information processing systems, 35:34614–34625, 2022

    2022

  21. [29]

    Random design analysis of ridge regression

    Daniel Hsu, Sham M Kakade, and Tong Zhang. Random design analysis of ridge regression. InConference on learning theory, pages 9–1. JMLR Workshop and Conference Proceedings, 2012

    2012

  22. [30]

    Extremum problems with inequalities as subsidiary conditions

    Fritz John. Extremum problems with inequalities as subsidiary conditions. InStudies and Essays Presented to R. Courant on His 60th Birthday, pages 187–204. Interscience Publishers, New York, 1948

    1948

  23. [31]

    A smoothed analysis of the greedy algorithm for the linear contextual bandit problem.Advances in neural information processing systems, 31, 2018

    Sampath Kannan, Jamie H Morgenstern, Aaron Roth, Bo Waggoner, and Zhiwei Steven Wu. A smoothed analysis of the greedy algorithm for the linear contextual bandit problem.Advances in neural information processing systems, 31, 2018

    2018

  24. [32]

    Conservative contextual linear bandits.Advances in neural information processing systems, 30, 2017

    Abbas Kazerouni, Mohammad Ghavamzadeh, Yasin Abbasi Yadkori, and Benjamin Van Roy. Conservative contextual linear bandits.Advances in neural information processing systems, 30, 2017

    2017

  25. [33]

    Contextual linear bandits under noisy features: Towards bayesian oracles

    Jung-Hun Kim, Se-Young Yun, Minchan Jeong, Junhyun Nam, Jinwoo Shin, and Richard Combes. Contextual linear bandits under noisy features: Towards bayesian oracles. InIn- ternational Conference on Artificial Intelligence and Statistics, pages 1624–1645. PMLR, 2023

    2023

  26. [34]

    Doubly robust thompson sampling with linear payoffs.Advances in neural information processing systems, 34:15830–15840, 2021

    Wonyoung Kim, Gi-Soo Kim, and Myunghee Cho Paik. Doubly robust thompson sampling with linear payoffs.Advances in neural information processing systems, 34:15830–15840, 2021

    2021

  27. [35]

    Efficient linear bandits through matrix sketching

    Ilja Kuzborskij, Leonardo Cella, and Nicolò Cesa-Bianchi. Efficient linear bandits through matrix sketching. InThe 22nd International Conference on Artificial Intelligence and Statistics, pages 177–185. PMLR, 2019. 12

    2019

  28. [36]

    Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems.The Annals of Statistics, pages 154–166, 1982

    Tze Leung Lai and Ching Zong Wei. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems.The Annals of Statistics, pages 154–166, 1982

    1982

  29. [37]

    An actor-critic contextual bandit algorithm for personalized mobile health interventions.arXiv preprint arXiv:1706.09090, 2017

    Huitian Lei, Yangyi Lu, Ambuj Tewari, and Susan A Murphy. An actor-critic contextual bandit algorithm for personalized mobile health interventions.arXiv preprint arXiv:1706.09090, 2017

    2017

  30. [38]

    A contextual-bandit approach to personalized news article recommendation

    Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. InProceedings of the 19th international conference on World wide web, pages 661–670, 2010

    2010

  31. [39]

    Nearly minimax-optimal regret for linearly parame- terized bandits

    Yingkai Li, Yining Wang, and Yuan Zhou. Nearly minimax-optimal regret for linearly parame- terized bandits. InConference on Learning Theory, pages 2173–2174. PMLR, 2019

    2019

  32. [40]

    Tight regret bounds for infinite-armed linear contextual bandits

    Yingkai Li, Yining Wang, Xi Chen, and Yuan Zhou. Tight regret bounds for infinite-armed linear contextual bandits. InInternational Conference on Artificial Intelligence and Statistics, pages 370–378. PMLR, 2021

    2021

  33. [41]

    Learning the uncertainty sets of linear control systems via set membership: A non-asymptotic analysis

    Yingying Li, Jing Yu, Lauren Conger, Taylan Kargin, and Adam Wierman. Learning the uncertainty sets of linear control systems via set membership: A non-asymptotic analysis. In Forty-first International Conference on Machine Learning, 2024

    2024

  34. [42]

    Bypassing the simulator: Near-optimal ad- versarial linear contextual bandits.Advances in Neural Information Processing Systems, 36: 52086–52131, 2023

    Haolin Liu, Chen-Yu Wei, and Julian Zimmert. Bypassing the simulator: Near-optimal ad- versarial linear contextual bandits.Advances in Neural Information Processing Systems, 36: 52086–52131, 2023

    2023

  35. [43]

    Asymptotic properties of set membership identifi- cation algorithms.Systems & control letters, 27(3):145–155, 1996

    Mitchell M Livstone and Munther A Dahleh. Asymptotic properties of set membership identifi- cation algorithms.Systems & control letters, 27(3):145–155, 1996

    1996

  36. [44]

    Robust mpc with recursive model update.Automatica, 103:461–471, 2019

    Matthias Lorenzen, Mark Cannon, and Frank Allgöwer. Robust mpc with recursive model update.Automatica, 103:461–471, 2019

    2019

  37. [45]

    Robust adaptive model predictive control: Performance and parameter estimation.International Journal of Robust and Nonlinear Control, 31(18):8703–8724, 2021

    Xiaonan Lu, Mark Cannon, and Denis Koksal-Rivet. Robust adaptive model predictive control: Performance and parameter estimation.International Journal of Robust and Nonlinear Control, 31(18):8703–8724, 2021

    2021

  38. [46]

    Optimistic bayesian sampling in contextual-bandit problems.Journal of Machine Learning Research, 13(6), 2012

    Benedict C May, Nathan Korda, Anthony Lee, David S Leslie, and Nicolo Cesa-Bianchi. Optimistic bayesian sampling in contextual-bandit problems.Journal of Machine Learning Research, 13(6), 2012

    2012

  39. [47]

    Optimal estimation theory for dynamic systems with set membership uncertainty: An overview.Automatica, 27(6):997–1009, 1991

    Mario Milanese and Antonio Vicino. Optimal estimation theory for dynamic systems with set membership uncertainty: An overview.Automatica, 27(6):997–1009, 1991

    1991

  40. [48]

    Identification of analytic nonlinear dynamical systems with non-asymptotic guarantees.Advances in Neural Information Processing Systems, 37:85500–85522, 2024

    Negin Musavi, Ziyao Guo, Geir Dullerud, and Yingying Li. Identification of analytic nonlinear dynamical systems with non-asymptotic guarantees.Advances in Neural Information Processing Systems, 37:85500–85522, 2024

    2024

  41. [49]

    Efficient and robust algorithms for adversarial linear contextual bandits

    Gergely Neu and Julia Olkhovskaya. Efficient and robust algorithms for adversarial linear contextual bandits. InConference on Learning Theory, pages 3049–3068. PMLR, 2020

    2020

  42. [50]

    Leveraging good representations in linear contextual bandits

    Matteo Papini, Andrea Tirinzoni, Marcello Restelli, Alessandro Lazaric, and Matteo Pirotta. Leveraging good representations in linear contextual bandits. InInternational Conference on Machine Learning, pages 8371–8380. PMLR, 2021

    2021

  43. [51]

    Contextual bandits and imitation learning with preference-based active queries.Advances in Neural Information Processing Systems, 36:11261–11295, 2023

    Ayush Sekhari, Karthik Sridharan, Wen Sun, and Runzhe Wu. Contextual bandits and imitation learning with preference-based active queries.Advances in Neural Information Processing Systems, 36:11261–11295, 2023

    2023

  44. [52]

    Learning without mixing: Towards a sharp analysis of linear system identification

    Max Simchowitz, Horia Mania, Stephen Tu, Michael I Jordan, and Benjamin Recht. Learning without mixing: Towards a sharp analysis of linear system identification. InConference On Learning Theory, pages 439–473. PMLR, 2018

    2018

  45. [53]

    Ensemble contextual bandits for personalized recommendation

    Liang Tang, Yexi Jiang, Lei Li, and Tao Li. Ensemble contextual bandits for personalized recommendation. InProceedings of the 8th ACM Conference on Recommender Systems, pages 73–80, 2014. 13

    2014

  46. [54]

    Stochastic linear contextual bandits with diverse contexts

    Weiqiang Wu, Jing Yang, and Cong Shen. Stochastic linear contextual bandits with diverse contexts. InInternational Conference on Artificial Intelligence and Statistics, pages 2392–2401. PMLR, 2020

    2020

  47. [55]

    On the sample complexity of set membership estimation for linear systems with disturbances bounded by convex sets

    Haonan Xu and Yingying Li. On the sample complexity of set membership estimation for linear systems with disturbances bounded by convex sets. In2025 American Control Conference (ACC), pages 3856–3862. IEEE, 2025

    2025

  48. [56]

    Noise-adaptive thompson sampling for linear contex- tual bandits.Advances in Neural Information Processing Systems, 36:23630–23657, 2023

    Ruitu Xu, Yifei Min, and Tianhao Wang. Noise-adaptive thompson sampling for linear contex- tual bandits.Advances in Neural Information Processing Systems, 36:23630–23657, 2023

    2023

  49. [57]

    Dynamic clustering based contextual combinatorial multi-armed bandit for online recommendation.Knowledge- Based Systems, 257:109927, 2022

    Cairong Yan, Haixia Han, Yanting Zhang, Dandan Zhu, and Yongquan Wan. Dynamic clustering based contextual combinatorial multi-armed bandit for online recommendation.Knowledge- Based Systems, 257:109927, 2022

    2022

  50. [58]

    Online adversarial stabilization of unknown net- worked systems.Proceedings of the ACM on Measurement and Analysis of Computing Systems, 7(1):1–43, 2023

    Jing Yu, Dimitar Ho, and Adam Wierman. Online adversarial stabilization of unknown net- worked systems.Proceedings of the ACM on Measurement and Analysis of Computing Systems, 7(1):1–43, 2023

    2023

  51. [59]

    System identification under bounded noise: Optimal rates beyond least squares.IEEE Control Systems Letters, 2025

    Xiong Zeng, Jing Yu, and Necmiye Ozay. System identification under bounded noise: Optimal rates beyond least squares.IEEE Control Systems Letters, 2025

    2025

  52. [60]

    Trajectory tracking control of autonomous ground vehicles using adaptive learning mpc.IEEE Transactions on Neural Networks and Learning Systems, 32(12):5554–5564, 2021

    Kunwu Zhang, Qi Sun, and Yang Shi. Trajectory tracking control of autonomous ground vehicles using adaptive learning mpc.IEEE Transactions on Neural Networks and Learning Systems, 32(12):5554–5564, 2021

    2021

  53. [61]

    Feel-good thompson sampling for contextual bandits and reinforcement learning

    Tong Zhang. Feel-good thompson sampling for contextual bandits and reinforcement learning. SIAM Journal on Mathematics of Data Science, 4(2):834–857, 2022

    2022

  54. [62]

    Spoiled for choice? personalized recommendation for healthcare decisions: A multiarmed bandit approach.Information Systems Research, 34(4):1493–1512, 2023

    Tongxin Zhou, Yingfei Wang, Lu Yan, and Yong Tan. Spoiled for choice? personalized recommendation for healthcare decisions: A multiarmed bandit approach.Information Systems Research, 34(4):1493–1512, 2023. 14 Appendix A Roadmap • Appendix B summarizes the notation used through...

    2023

  55. [63]

    Thex t-weighted projection width ofE t−1 onto the vectorx t is Wt := 2 q x⊤ t Bt−1xt

    + 1 2 log detB t, whereB d 2 denotes theℓ 2 unit ball inR d. Thex t-weighted projection width ofE t−1 onto the vectorx t is Wt := 2 q x⊤ t Bt−1xt. Define the filtration Ft−1 :=σ{η 1,· · ·, η t−1;X 1,s,· · ·, X K,s,∀s≤t}. C Proof of Theorem 1 and Corollary 1 C.1 Bound support f...

  56. [64]

    Proof of Corollary 3

    ≤R d(q), whereR d(q) = d(1+q) d+1 d2(1−q2) d2−1 d−1 2 . Proof of Corollary 3. Since E∗(q) is the MVEE of K(q), and E(q) is also an ellipsoid enclosing K(q), then V ol(E∗(q))≤V ol(E(q)) =a(q)·(b(q)) d−1V ol(Bd 2) = d(1 +q) d+ 1 d2(1−q 2) d2 −1 d−1 2 V ol(Bd 2) =R d(q)V ol(Bd 2)...

  57. [65]

    Meanwhile, K µ 0 = [−2S,2S] d +µB d 2 ⊆B(0,2S √ d+µ)

    = log(µdV ol(Bd 2)) =dlogµ+ logV ol(B d 2). Meanwhile, K µ 0 = [−2S,2S] d +µB d 2 ⊆B(0,2S √ d+µ). Therefore, Φ0 ≤logV ol(B d

  58. [66]

    It follows that Φ0 −Φ T ≤dlog 2S √ d+µ µ !

    +dlog(2S √ d+µ). It follows that Φ0 −Φ T ≤dlog 2S √ d+µ µ ! . D.6 Proof of Lemma 8 The event n q+ t ≤ ξ d o is equivalent to b′ t +µL−α t ρt ≤ ξ d ⇐ ⇒η max +η t +µL−α t ≤ ξρt d ⇐ ⇒η t ≤ −ηmax +u t, whereu t :=α t + ξρt d −µL. Similarly, the event n q− t ≤ ξ d o is equivalent t...

  59. [67]

    E Numerical Settings Problem Settings.We utilize the experiment in [ 26] for our synthetic simulation settings

    Then PT ≤2J T + 8 3 log 1 δ . E Numerical Settings Problem Settings.We utilize the experiment in [ 26] for our synthetic simulation settings. In partic- ular, we consider a total number of K= 20 arms to choose from. The problem space has dimension d= 6 . Therefore, both the gr...

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.