Linear Bandits with Non-i.i.d. Noise

Baptiste Ab\'el\`es; Eugenio Clerico; Gergely Neu; Hamish Flynn

REVIEW 2 major objections 4 minor 27 references

Linear Bandits with Non-i.i.d. Noise

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper shows that optimism-based linear bandits remain provably sound under temporally correlated, mixing noise, with regret matching the i.i.d. benchmark up to a mixing-time factor for geometric decay.

desk verdict A real new setting and sound proof structure, but a constant mismatch in the confidence sequence that must be fixed before acceptance. read the letter →

arxiv 2505.20017 v2 pith:EBVERMCU submitted 2025-05-26 stat.ML cs.LG

Baptiste Abélès , Eugenio Clerico , Hamish Flynn , Gergely Neu This is my paper · ORCID

classification stat.MLcs.LG MSC 62L0568Q32

keywords linearbanditsnon-i.i.d.noisemixingsub-Gaussianregretboundsconfidencesequencesonline-to-confidence-setconversiondelayedfeedbackMixing-LinUCB

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper relaxes the standard i.i.d. assumption on the observation noise in linear stochastic bandits, allowing noise terms that are sub-Gaussian but correlated, with dependence that decays over time. It constructs anytime-valid confidence sets by splitting the rounds into d interleaved subsequences, running an online prediction game with delayed feedback on each, and applying a per-block martingale inequality. These confidence sets feed an optimistic arm-selection rule, Mixing-LinUCB, whose regret is bounded in terms of the mixing coefficients. For geometrically mixing noise, the bound recovers the standard i.i.d. rate up to a factor of the mixing time and logarithmic factors, which matters because realistic noise in applications such as ad selection is temporally correlated.

What carries the argument

The central object is a delayed online-to-confidence-set conversion. The abstract sequential probability assignment game is played by d independent exponentially weighted average forecasters, each active on a different residue class of rounds modulo d, and the reward feedback is delayed by d rounds; the joint forecaster's regret is bounded by summing the individual regrets. Concentration is handled by blocking: each residue class is a subsequence on which $\exp(S_k^{(i)} - (k-1)(2B+1)\varphi_d)$ is a non-negative supermartingale, so Ville's inequality plus a union bound gives the anytime-valid inequality in Lemma 1. A delayed version of the elliptical potential lemma, Lemma 3, controls the sum of norm terms that appear when bounding regret from the confidence ellipsoids.

What would settle it

A concrete check that would settle the central claim is to run Mixing-LinUCB on a one-dimensional problem with AR(1) noise $\varepsilon_t = \rho \varepsilon_{t-1} + \eta_t$, choose $d \approx -1/\log \rho$, and compare the empirical regret with the Theorem 2 bound. To test the obliviousness premise, let the decision set at round t depend on $\varepsilon_{t-1}$ and record how often $\theta^\star$ lies in $C_t$; the paper's proof requires coverage with probability at least $1-\delta$ and predicts this adaptive choice can break it even though the noise still satisfies Assumption 1.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the optimistic linear-bandit strategy remains sound when the noise is only mixing sub-Gaussian rather than conditionally independent. Under Assumption 1, the ellipsoidal confidence sets centered at the regularized least-squares estimator are anytime valid, provided the agent plays with a delay d: at round t it chooses the arm maximizing the upper confidence bound computed from data up to t-d. Theorem 2 gives a high-probability regret bound of order $2dB + \sqrt{8dpT \max(B^2, \beta_T^2) \log(1 + B^2T/(dp))}$; with geometric mixing and $d = \lceil \tau \log(BCT/p) \rceil$, Corollary 2 states the regret is $O(\tau p \sqrt{T} \log^2 T + \tau \log T \sqrt{pT \log T})$, matching the i.i.d. benchmark up to the mixing time $\tau$ and logarithmic factors. For algebraically mixing noise, the regret is sublinear when the decay exponent $r > 1$ and approaches standard rates as $r \to \infty$.

Load-bearing premise

The decision sets must be chosen obliviously, before any noise is observed, so the arm at round t can depend only on information at least d rounds old; if an adaptive adversary picks decision sets using recent noise, the per-block supermartingale argument in Lemma 1 stops working.

Editorial extensions

If this is right

Under geometric mixing, the regret bound is $O(\tau p \sqrt{T} \log^2 T + \tau \log T \sqrt{pT \log T})$, so the cost of dependence is essentially the mixing time $\tau$ times log factors.
For algebraically mixing noise with exponent $r>1$, the algorithm achieves sublinear regret, and the bound converges to the i.i.d. rate as $r$ grows.
With a positive minimum gap $\Delta$, the bound becomes roughly $O((dp/\Delta) \max(B^2, \beta_T^2) \log(1 + B^2T/(dp)))$, giving near-logarithmic regret in $T$ for suitable $d$.
For any fixed delay $d$ that does not grow with $T$, the worst-case bound is linear in $T$, so the delay has to track the horizon or the decay rate of the noise dependence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same delayed-block construction should transfer to other optimistic bandit algorithms (generalized linear, kernel, or even nonlinear) as long as the noise satisfies an analogous mixing-sub-Gaussian condition, because the confidence-sequence argument is decoupled from the linear reward model.
A time-varying delay that grows slowly with t should remove the need to know the horizon T and yield fully anytime regret bounds; the paper mentions this route but leaves the details open.
The restriction to oblivious decision sets means the algorithm is safe when contexts are precommitted or based on stale data, but not when they react to recent outcomes; relaxing that would require a different concentration argument.
Since the algebraic-mixing bound is trivial for $r \le 1$, the practical gains concentrate on noise whose dependence decays faster than $1/t$, such as ARMA or Markov chains with finite mixing time.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This paper deserves a serious referee, but only after the authors fix a constant error in the confidence sequence. The setting is new: linear bandits with sub-Gaussian, mixing noise rather than the usual i.i.d. or conditionally centered noise. The algorithm, Mixing-LinUCB, uses a delayed online-to-confidence-set conversion plus blocking, and the paper gives worst-case and gap-dependent regret bounds that recover standard rates up to the mixing time for geometrically mixing noise. That's a meaningful extension, not a revolution.

The core technical contribution is a confidence sequence for mixing sub-Gaussian noise. The structure is sound and the exposition is honest: the authors clearly flag that the mixing coefficients are assumed known, that the delay d must depend on the horizon T, and that the decision sets must be oblivious. They also include full proofs. That honesty is a real plus.

The soft spot is in Lemma 1. The lemma states P(sum D_t <= t phi_d B + d log(d/delta)), but the proof in Appendix A.2 constructs a supermartingale with exponent (2B+1) phi_d, using |lambda_t(theta)| <= 2B+1. The union bound over d blocks gives (2B+1) t phi_d, not t phi_d B. Theorem 1 and the subsequent beta_t^2 use t phi_d (B+1), which is also smaller than what the proof allows. So as written, the 1-delta coverage isn't established. This is load-bearing, since every regret bound inherits the confidence sets. That said, it's a constant-factor bug: replace (B+1) by (2B+1) in the radius and the order of all the corollaries stands. The geometric mixing rate tau p sqrt(T) log^2 T still holds because the extra term is absorbed.

Minor: Theorem 1's statement has P(theta* in C_t, for all t) <= 1-delta, which should be >=. Probably a typo.

The citation pattern looks fair: they build on their own previous work (delayed online-to-PAC) and cite the relevant bandit and mixing literature. No fitted parameters or circular claims.

Bottom line: the paper is a legit contribution with a correctable technical defect. I'd send it to review with a strong request to fix the constant and align Lemma 1, Theorem 1, and Corollary 1. After that, it's a solid paper.

Referee Report

2 major / 4 minor

Summary. The paper studies stochastic linear bandits with sub-Gaussian but non-independent observation noise, formalized through a new 'mixing sub-Gaussianity' assumption (Assumption 1) in which conditional means and conditional sub-Gaussian tails are controlled by a decaying sequence of mixing coefficients. The authors construct anytime-valid confidence sequences by combining an online-to-confidence-set conversion with a delayed-feedback variant of the underlying online learning game, using a blocking argument to control the dependence across rounds. These confidence sets are used in a UCB algorithm, Mixing-LinUCB (Algorithm 1), that deliberately bases its decisions on confidence sets lagged by d rounds. The main results are a worst-case regret bound (Theorem 2) and a gap-dependent regret bound (Theorem 3) expressed in terms of the mixing coefficients, with corollaries for geometric and algebraic mixing. For geometric mixing and an appropriately chosen delay, the paper claims regret of order O(τ p sqrt(T) log^2 T + τ log T sqrt(pT log T)), which it presents as recovering standard i.i.d. rates up to a factor of the mixing time.

Significance. If the technical gaps are repaired, this is a meaningful extension of linear bandit theory to dependent noise. The paper's key contribution is a natural and well-motivated combination of delayed-feedback online-to-confidence-set conversion with the classical blocking technique for mixing processes, and it makes the dependence of regret on the mixing coefficients explicit. The paper is transparent about its assumptions: known mixing coefficients, oblivious decision sets, and a horizon-dependent choice of delay. It provides detailed proofs and precisely stated constants, and the geometric-mixing corollary gives the right target behavior, namely i.i.d. rates up to logarithmic factors and the mixing time. These strengths make the paper a solid candidate for publication once the load-bearing technical issues are addressed.

major comments (2)

[Section 4.2, Lemma 1 and Appendix A.2] The confidence radius stated in Lemma 1 is not supported by its proof. The proof defines M_k^(i) = exp(S_k^(i) - (k-1)(2B+1)φ_d) and establishes S_k^(i) ≤ k(2B+1)φ_d + log(d/δ) for each block, which after the union bound gives Σ_{s=1}^t D_s ≤ t(2B+1)φ_d + d log(d/δ). Lemma 1 as stated has t φ_d B, and Theorem 1, Proposition 1, and Corollary 1 build the confidence radius with the mixing term t φ_d (B+1). For B > 0 this is strictly smaller than the constant (2B+1)φ_d required by the proof, so the claimed 1-δ coverage of the confidence sets is not established as written. Because every regret bound in Section 5 inherits the validity of these confidence sets, this is a load-bearing internal inconsistency. The defect is confined to constants: replacing (B+1) by (2B+1) in the radius and adjusting β_t^2 accordingly leaves the order of all corollaries unchanged, since the affected term contributes at most O(p sqrt(τ T log T)) in the geometric-mixing case.
[Section 5, Theorem 2 and Corollaries 2-4] The sublinear regret guarantees are fixed-horizon rather than anytime. For a fixed delay d, Theorem 2 holds for all T > d but the bound is linear in T; the sublinear rates in Corollaries 2-4 are obtained only by choosing d as a function of the target horizon T, e.g., d = τ log(BCT/p) for geometric mixing. This means the algorithm must know T (or an upper bound on it) to achieve the advertised rates, a point that is acknowledged in Section 6 but not reflected in the abstract or the introductory statement that the bounds 'recover the standard rates.' The authors suggest a time-varying delay could fix this, but they do not provide the analysis. The paper should either supply such an analysis or explicitly qualify the main claims as fixed-horizon guarantees.

minor comments (4)

[Theorem 1, Section 4.2] The displayed coverage guarantee reads P(θ⋆ ∈ C_t, ∀t ≥ 1) ≤ 1 − δ; the inequality must be ≥ 1 − δ. As written, the statement asserts the opposite of the intended high-probability coverage.
[Proposition 1, Section 3] The definition of V_t has typographical errors: the sum is over s but the term is written X_t X_t^⊤, and the identity matrix is written I_d instead of I_p. It should read V_t = Σ_{s=1}^t X_s X_s^⊤ + λ I_p.
[Section 2, regret definition] The regret formula contains a stray 't' in the summation: Reg(T) = t∑_{t=1} sup ... should be Reg(T) = Σ_{s=1}^T sup ... .
[Section 4.2, delayed game protocol] The protocol states that at round s the environment reveals Y_{s-d+1} to the learner, while the loss L_s(Q_s) used in the analysis is evaluated with Y_s. This is legitimate because Q_s is chosen before Y_s is revealed, but the text should explicitly distinguish the information available to the learner for prediction from the loss used by the analyst, otherwise the reader may mistakenly think the learner observes the current reward.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the regret bounds are derived from the stated mixing-sub-Gaussian assumption with proofs included; self-citations supply standard tools, not the target conclusion.

full rationale

The paper's central claims are the confidence sequence (Proposition 1 and Theorem 1) and the regret bounds (Theorems 2 and 3) for Mixing-LinUCB. These are derived from Assumption 1 through explicit proofs in the main text and appendices: Lemma 1 is proved in Appendix A.2 using Ville's inequality and a union bound over d blocks, and Lemma 2 is proved from Proposition 2, which is itself proved in Appendix A.1. The cited works by overlapping authors (Clerico et al., 2025; Abelès et al., 2025) are used only as sources of the online-to-confidence-set reduction and the delayed-feedback trick, both of which are re-proved or directly instantiated here; the target regret bounds are not assumed in those citations. No parameter is fitted to data and then renamed a prediction: all bounds are expressed in terms of the assumed mixing coefficients phi_d, the dimension p, the horizon T, and the confidence parameter delta. The disclosed limitation that the adversary choosing X_t is oblivious is a scope restriction stated in Section 6, not a circular step. A separate correctness concern exists: the proof of Lemma 1 uses a (2B+1)phi_d offset while the statement and downstream radii use Bphi_d, so the stated constants are not fully established as written; however, this is an internal constant mismatch, not a reduction of the conclusion to its inputs, and it does not affect the asymptotic rates. Overall, the derivation chain is self-contained and non-circular.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The derivation rests on the mixing sub-Gaussian model, standard boundedness, obliviousness of the environment, known mixing coefficients and horizon, and standard martingale and EWA tools. No invented entities or fitted constants appear; the constants in the confidence radius are derived, though one appears miscomputed.

assumptions (5)

domain assumption Assumption 1: for all d, t>d, |E[epsilon_t | F_{t-d}]| <= phi_d and E[exp(lambda (epsilon_t - E[epsilon_t | F_{t-d}])) | F_{t-d}] <= e^{lambda^2 sigma^2 / 2} for lambda > 0.
This is the dependence model replacing i.i.d. or conditionally centered noise; the entire confidence sequence argument in Section 4 and Lemma 1 is built on it.
domain assumption Boundedness: theta* in B(B), X_t subset of B(1), and sigma = 1.
Standard linear bandit assumptions used to bound instantaneous regret and the EWA regret; stated in Section 2.
domain assumption Oblivious adversary: decision sets X_t are fixed in advance and X_t is F_{t-d}-measurable under the algorithm's delay.
Section 6 admits this is usually not required in linear bandits; it is needed so that Q_t in the delayed online game does not depend on recent noise.
standard math Ville's inequality and the EWA regret bound of Proposition 2.
Used to turn the supermartingale concentration into an anytime confidence sequence; Proposition 2 is proven in Appendix A.1 by a standard KL and telescoping argument.
domain assumption Known mixing coefficients and horizon T: the algorithm requires an upper bound on phi_d and chooses d as a function of T.
Stated in Section 6 as an important limitation; without it d cannot be set to make the bounds sublinear.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Linear Bandits with Non-i.i.d. Noise." pith.science (2026). https://pith.science/paper/EBVERMCU

@misc{pith2026250520017,
  author       = {Pith},
  title        = {Pith review of: Linear Bandits with Non-i.i.d. Noise},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EBVERMCU}},
  note         = {Machine review of arXiv:2505.20017}
}

read the original abstract

We study the linear stochastic bandit problem, relaxing the standard i.i.d. assumption on the observation noise. As an alternative to this restrictive assumption, we allow the noise terms across rounds to be sub-Gaussian but interdependent, with dependencies that decay over time. To address this setting, we develop new confidence sequences using a recently introduced reduction scheme to sequential probability assignment, and use these to derive a bandit algorithm based on the principle of optimism in the face of uncertainty. We provide regret bounds for the resulting algorithm, expressed in terms of the decay rate of the strength of dependence between observations. Among other results, we show that our bounds recover the standard rates up to a factor of the mixing time for geometrically mixing observation noise.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 21 canonical work pages

[1]

Naoki Abe and Philip M. Long. Associative reinforcement learning using linear probabilistic concepts. In Proceedings of the Sixteenth International Conference on Machine Learning, 1999

work page 1999
[2]

Using confidence bounds for exploitation-exploration trade-offs

Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res., 3: 0 397–422, 2003

work page 2003
[3]

Biermann, and Philip M

Naoki Abe, Alan W. Biermann, and Philip M. Long. Reinforcement learning with immediate rewards and linear hypotheses. Algorithmica, 37 0 (4): 0 263–293, 2003

work page 2003
[4]

A contextual-bandit approach to personalized news article recommendation

Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661--670, 2010

2010
[5]

Disposable linear bandits for online recommendations

Melda Korkut and Andrew Li. Disposable linear bandits for online recommendations. Proceedings of the AAAI Conference on Artificial Intelligence, 35 0 (5), 2021

work page 2021
[6]

Feature-based dynamic pricing

Maxime C Cohen, Ilan Lobel, and Renato Paes Leme. Feature-based dynamic pricing. Management Science, 66 0 (11): 0 4921--4943, 2020

work page 2020
[7]

Lai and Herbert Robbins

T.L. Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6 0 (1): 0 4--22, 1985

work page 1985
[8]

Improved algorithms for linear stochastic bandits

Yasin Abbasi-Yadkori, D \'a vid P \'a l, and Csaba Szepesv \'a ri. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011

2011

Show all 27 references

[9]

Bandit algorithms

Tor Lattimore and Csaba Szepesv \'a ri. Bandit algorithms. Cambridge University Press, 2020

2020
[10]

Improved algorithms for stochastic linear bandits using tail bounds for martingale mixtures

Hamish Flynn, David Reeb, Melih Kandemir, and Jan R Peters. Improved algorithms for stochastic linear bandits using tail bounds for martingale mixtures. Advances in Neural Information Processing Systems, 36: 0 45102--45136, 2023

2023
[11]

Richard C. Bradley. Basic properties of strong mixing conditions: A survey and some open questions. Probability Surveys, 2: 0 107--144, 2005

2005
[12]

Mohri and A

M. Mohri and A. Rostamizadeh. Rademacher complexity bounds for non-i.i.d. processes. NeurIPS, 2008

2008
[13]

Generalization bounds for mixing processes via delayed online-to-PAC conversions

Baptiste Ab\'el\`es, Eugenio Clerico, and Gergely Neu. Generalization bounds for mixing processes via delayed online-to-PAC conversions. In Proceedings of The 36th International Conference on Algorithmic Learning Theory, 2025

2025
[14]

Scalable generalized linear bandits: Online computation and hashing

Kwang-Sung Jun, Aniruddha Bhargava, Robert Nowak, and Rebecca Willett. Scalable generalized linear bandits: Online computation and hashing. In Advances in Neural Information Processing Systems, volume 30, 2017

2017
[15]

Improved regret bounds of (multinomial) logistic bandits via regret-to-confidence-set conversion

Junghyun Lee, Se-Young Yun, and Kwang-Sung Jun. Improved regret bounds of (multinomial) logistic bandits via regret-to-confidence-set conversion. In Proceedings of the 27th International Conference on Artificial Intelligence and Statistics, pages 4474--4482, 2024

2024
[16]

Confidence sequences for generalized linear models via regret analysis, 2025

Eugenio Clerico, Hamish Flynn, Wojciech Kotłowski, and Gergely Neu. Confidence sequences for generalized linear models via regret analysis, 2025. URL https://arxiv.org/abs/2504.16555

2025 arXiv
[17]

Linear bandits with stochastic delayed feedback

Claire Vernade, Alexandra Carpentier, Tor Lattimore, Giovanni Zappella, Beyza Ermis, and Michael Brueckner. Linear bandits with stochastic delayed feedback. In International Conference on Machine Learning, pages 9712--9721. PMLR, 2020 a

2020
[18]

Delayed feedback in generalised linear bandits revisited

Benjamin Howson, Ciara Pike-Burke, and Sarah Filippi. Delayed feedback in generalised linear bandits revisited. In International Conference on Artificial Intelligence and Statistics, pages 6095--6119. PMLR, 2023

2023
[19]

On upper-confidence bound policies for non-stationary bandit problems

Aur \'e lien Garivier and Eric Moulines. On upper-confidence bound policies for non-stationary bandit problems. arXiv preprint arXiv:0805.3415, 2008

2008 arXiv
[20]

Weighted linear bandits for non-stationary environments

Yoan Russac, Claire Vernade, and Olivier Capp \'e . Weighted linear bandits for non-stationary environments. Advances in Neural Information Processing Systems, 32, 2019

2019
[21]

Non-stationary delayed bandits with intermediate observations

Claire Vernade, Andras Gyorgy, and Timothy Mann. Non-stationary delayed bandits with intermediate observations. In International Conference on Machine Learning, pages 9722--9732. PMLR, 2020 b

2020
[22]

Prediction, Learning, and Games

Nicolò Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, USA, 2006

2006
[23]

Gr\" u nwald

Peter D. Gr\" u nwald. The Minimum Description Length Principle (Adaptive Computation and Machine Learning). The MIT Press, 2007

2007
[24]

Rates of convergence for empirical processes of stationary mixing sequences

Bin Yu. Rates of convergence for empirical processes of stationary mixing sequences. The Annals of Probability, 22 0 (1): 0 94--116, 1994

1994
[25]

Weinberger and E

M.J. Weinberger and E. Ordentlich. On delayed prediction of individual sequences. IEEE Transactions on Information Theory, 48 0 (7), 2002

2002
[26]

Mixing time estimation in reversible markov chains from a single sample path

Daniel Hsu, Aryeh Kontorovich, David A Levin, Yuval Peres, Csaba Szepesv \'a ri, and Geoffrey Wolfer. Mixing time estimation in reversible markov chains from a single sample path. The Annals of Applied Probability, 29 0 (4): 0 2439--2480, 2019

2019
[27]

Mixing time estimation in ergodic markov chains from a single trajectory with contraction methods

Geoffrey Wolfer. Mixing time estimation in ergodic markov chains from a single trajectory with contraction methods. In Algorithmic Learning Theory, pages 890--905, 2020

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Naoki Abe and Philip M. Long. Associative reinforcement learning using linear probabilistic concepts. In Proceedings of the Sixteenth International Conference on Machine Learning, 1999

work page 1999

[2] [2]

Using confidence bounds for exploitation-exploration trade-offs

Peter Auer. Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res., 3: 0 397–422, 2003

work page 2003

[3] [3]

Biermann, and Philip M

Naoki Abe, Alan W. Biermann, and Philip M. Long. Reinforcement learning with immediate rewards and linear hypotheses. Algorithmica, 37 0 (4): 0 263–293, 2003

work page 2003

[4] [4]

A contextual-bandit approach to personalized news article recommendation

Lihong Li, Wei Chu, John Langford, and Robert E Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661--670, 2010

2010

[5] [5]

Disposable linear bandits for online recommendations

Melda Korkut and Andrew Li. Disposable linear bandits for online recommendations. Proceedings of the AAAI Conference on Artificial Intelligence, 35 0 (5), 2021

work page 2021

[6] [6]

Feature-based dynamic pricing

Maxime C Cohen, Ilan Lobel, and Renato Paes Leme. Feature-based dynamic pricing. Management Science, 66 0 (11): 0 4921--4943, 2020

work page 2020

[7] [7]

Lai and Herbert Robbins

T.L. Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6 0 (1): 0 4--22, 1985

work page 1985

[8] [8]

Improved algorithms for linear stochastic bandits

Yasin Abbasi-Yadkori, D \'a vid P \'a l, and Csaba Szepesv \'a ri. Improved algorithms for linear stochastic bandits. Advances in neural information processing systems, 24, 2011

2011