Linear Stochastic Bandits Under Safety Constraints

Christos Thrampoulidis; Mahnoosh Alizadeh; Sanae Amani

REVIEW 2 major objections 5 minor 28 references

Linear Stochastic Bandits Under Safety Constraints

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes that Safe-LUCB, a two-phase optimistic algorithm, achieves $\widetilde{O}(\sqrt{T})$ cumulative regret in linear stochastic bandits with an unknown linear safety constraint whenever the optimal safe action has…

desk verdict A genuinely new problem formulation and a useful safety-gap analysis, but the proof of Lemma 3 contains a real gap that both regret theorems rely on; the paper is worth engaging with because the fix looks standard. read the letter →

arxiv 1908.05814 v1 pith:C5RHAJ7E submitted 2019-08-16 cs.LG stat.ML

Sanae Amani , Mahnoosh Alizadeh , Christos Thrampoulidis This is my paper · ORCID

classification cs.LGstat.ML

keywords linearstochasticbanditssafetyconstraintsSafe-LUCBoptimisminthefaceofuncertaintyregretboundssafesetestimationpureexplorationconfidenceellipsoid

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces a linear stochastic bandit problem in which every chosen action must satisfy a linear safety constraint $\mu^\dagger Bx \le c$ whose parameter vector $\mu$ is unknown. It proposes Safe-LUCB, a two-phase UCB algorithm that first samples actions uniformly from a conservative inner safe set to estimate $\mu$, then runs optimism-based safe exploration and exploitation using confidence-region safe sets. The paper's central claim is that the difficulty of the problem is governed by the safety gap $\Delta = c - \mu^\dagger Bx^*$, the slack of the optimal safe action. If $\Delta > 0$ and a lower bound on it is known, a logarithmic pure-exploration phase suffices and the cumulative regret is $\widetilde{O}(\sqrt{T})$, the same order as unconstrained linear bandits; if $\Delta = 0$, the regret is $\widetilde{O}(T^{2/3})$. Every played action is safe with probability at least $1-2\delta$, and the analysis isolates the extra cost, the "regret of safety," paid while the learner's safe set is still a conservative approximation.

What carries the argument

The load-bearing object is the randomized pure-exploration phase over the conservative safe set $\mathcal{D}_w = \{x \in \mathcal{D}_0 : \|Bx\| \le c/S\}$. Its covariance $\Sigma = \mathbb{E}[XX^\dagger]$ has minimum eigenvalue $\lambda_-$, and the Matrix Chernoff inequality converts $T'$ independent uniform draws into the Gram-matrix lower bound $\lambda_{\min}(A_{T'+1}) \ge \lambda + \lambda_- T'/2$. That bound is what shrinks the confidence ellipsoid enough for the estimated safe set $\mathcal{D}^s_t$ to contain $x^*$ when $\Delta > 0$, and it also controls the regret-of-safety term in the zero-gap case.

What would settle it

A concrete test: choose $\mathcal{D}_0 = \{x : \|x\|_\infty \le 1\}$, take $B$ singular with a one-dimensional null space along which the constraint is uninformative, and use any practical sampler over $\mathcal{D}_w$; measure $\lambda_{\min}(A_{T'+1})$ for growing $T'$. If it stays near $\lambda$ instead of growing linearly at rate $\lambda_-/2$, the eigenvalue lemma underpinning both regret theorems is violated for that sampling choice.

Watch

Extended reading notes

Core claim

The central discovery is a reduction of the safety problem to control of the weighted norm $\|Bx^*\|_{A_t^{-1}}$ through forced random exploration. Safe-LUCB builds a confidence ellipsoid $\mathcal{C}_t$ around its least-squares estimate $\hat{\mu}_t$ and defines the safe decision set $\mathcal{D}^s_t = \{x \in \mathcal{D}_0 : v^\dagger Bx \le c \text{ for all } v \in \mathcal{C}_t\}$. Because $\mathcal{D}^s_t$ is a conservative inner approximation of the true safe set, the optimal safe action $x^*$ may lie outside it; the paper isolates the resulting extra regret as Term II and shows it is nonpositive once $\|Bx^*\|_{A_t^{-1}} \le \Delta/(2\beta_t)$. The pure-exploration phase guarantees this by making $\lambda_{\min}(A_{T'+1}) \ge \lambda + \lambda_- T'/2$ with high probability, so that for $\Delta > 0$ a phase length of order $\log T$ puts $x^*$ inside $\mathcal{D}^s_t$ for every later round. With that, the regret analysis reduces to the standard UCB term and yields $\widetilde{O}(\sqrt{T})$; without a positive gap, the best bound is $\widetilde{O}(T^{2/3})$.

Load-bearing premise

The argument stands or falls on the pure-exploration phase's ability to draw independent actions from the conservative safe set whose action covariance has a strictly positive minimum eigenvalue ($\lambda_- > 0$), an ability the paper assumes rather than constructs for general matrices $B$.

Editorial extensions

If this is right

With a known positive safety gap, Safe-LUCB's cumulative regret is $\widetilde{O}(\sqrt{T})$ (Theorem 2), so the safety constraint, once learned, does not change the asymptotic rate.
When the gap is zero or unknown, the worst-case guarantee is $\widetilde{O}(T^{2/3})$ (Theorem 3), still sublinear, so average regret $R_T/T$ vanishes.
The pure-exploration phase must last $T_\Delta$ of order $\log T$ for $\Delta > 0$, and $T_\Delta$ grows like $1/\Delta^2$, so problems with small slack require substantially more forced exploration.
With probability at least $1-2\delta$, every action selected by Safe-LUCB satisfies the safety constraint $\mu^\dagger Bx_t \le c$.
In the $K$-armed contextual extension, the same algorithm achieves $\widetilde{O}(\sqrt{T})$ regret when each round's safety gap is strictly positive.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the $1/\Delta^2$ dependence of $T_\Delta$ suggests that for small positive gaps the practical difficulty may be closer to the $\Delta = 0$ case; an adaptive stopping rule based on the weighted norm $\|Bx^*\|_{A_t^{-1}}$ could make the $\sqrt{T}$ bound available without knowing $\Delta$ in advance.
Beyond the paper: replacing the uniform sampler on $\mathcal{D}_w$ with a fixed full-rank design, such as a G-optimal or D-optimal design, would likely preserve Lemma 1 for singular $B$, since the proof only needs independent samples whose covariance has $\lambda_- > 0$.
Beyond the paper: the zero-gap case is where a matching lower bound is most likely; if $\Omega(T^{2/3})$ were proved, Safe-LUCB would be rate-optimal for the full problem rather than only for the positive-gap regime.
Beyond the paper: the same two-phase template may transfer to multiple linear safety constraints, with the safety gap replaced by a vector of slacks and the eigenvalue lower bound controlling a sum of weighted norms.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is the right problem, the safety-gap idea is good, and the two-phase algorithm is sensible—but the proof as written has a real hole in Lemma 3, and both main theorems lean on it. I would not desk-reject this; I would send it back with a request to fix that lemma.

The formulation is genuinely new as far as I know: a linear bandit where each action must satisfy a per-round constraint µ'Bx ≤ c with unknown µ. The related work on cumulative/budget constraints [14,15] doesn't cover this. Safe-LUCB is a natural extension of LUCB, and the decomposition of instantaneous regret into Term I and the 'regret of safety' Term II is the right way to see the problem. The safety gap Δ = c - µ'Bx* is a clean complexity parameter: positive known Δ gives Õ(√T), zero Δ gives Õ(T^{2/3}). The analysis of Term II via the shrunk safe set and α_t is careful and convincing.

The soft spot is exactly where the stress-test note puts it. Lemma 3 claims Term I ≤ 2 min(β_t ||x_t||, 1). The first inequality Term I ≤ 2β_t||x_t|| is fine from the confidence set. But the min with 1 is justified by 'r_t ≤ 2', and Term I is not r_t. Term II can be negative when the optimistic pair is overly favorable, so Term I can be much larger than 2 while r_t stays small. The d=2 example in the stress-test is legitimate: it's not contrived. Both Theorems 2 and 3 use Lemma 3 directly, so the printed regret bounds do not follow from the written proof.

That said, I think the gap is repairable. For the positive-gap case, Term II ≤ 0 once the exploration phase is long enough, so r_t ≤ Term I ≤ 2β||x_t|| and also r_t ≤ 2; then the same elliptic-potential argument bounds ∑ r_t directly, which is what the theorem actually needs. For the zero-gap case you can bound r_t ≤ 2β||x_t|| + Term II and handle the two terms with standard arguments; the third term in Theorem 3 already captures the safety cost. So the central claims are likely true, but they need a corrected lemma, not just a typo.

Minor issue: the pure-exploration phase assumes a uniform sampler over the convex body D_w and a positive minimum eigenvalue of the resulting covariance. For general B that's an oracle; Remark 1 only gives an explicit sampler when B is invertible. That's worth stating as an assumption or a limitation, not a fatal flaw.

Bottom line: this deserves a serious referee. The formulation and the safety-gap notion will be cited; the proof needs work. If I were running the review I'd ask for a revised Lemma 3 and a clearer statement about the sampling oracle, then it could be a solid contribution.

Referee Report

2 major / 5 minor

Summary. The paper studies a linear stochastic bandit in which, in addition to minimizing cumulative pseudo-regret, the learner must choose every action so that a linear safety constraint µ†Bx≤c is satisfied, with B and c known but µ unknown. The authors propose Safe-LUCB, a two-phase algorithm: a pure-exploration phase samples actions uniformly from a conservative safe subset Dw in order to guarantee a lower bound on the minimum eigenvalue of the Gram matrix, and a subsequent safe exploration-exploitation phase uses optimism over confidence sets while restricting actions to an estimated safe set. The main results are a problem-dependent regret bound of Õ(√T) when the safety gap Δ=c−µ†Bx* is positive and known, a worst-case Õ(T^{2/3}) bound when Δ=0, and a heuristic extension (GSLUCB) for unknown Δ with simulations.

Significance. The formulation is natural and the safety-gap dichotomy is an insightful contribution: it cleanly separates the regime in which the optimal action can be included in the estimated safe set from the regime in which a residual 'regret of safety' must be paid. The paper also correctly emphasizes the distinction between stage-wise safety constraints and cumulative budget constraints, and it identifies a gap in prior safe Bayesian-optimization work, which provides convergence guarantees but not regret bounds. The two-phase design and the use of the Gram-matrix lower bound to learn the safe set are conceptually clean, and the appendix gives explicit constants and fairly detailed proofs. If the proof gap described below is repaired, this would be a solid first regret analysis for stage-wise safe linear bandits.

major comments (2)

[B.2, Eq. (20); Lemma 3; Theorems 2 and 3] The step 'Recall, from Assumption 2, the trivial bound on the instantaneous regret r_t≤2. Thus, Term I≤2 min(β_t‖x_t‖_{A_t^{-1}},1)' is not justified. Since r_t = Term I + Term II and Term II = µ̃_t†x_t−µ†x* can be negative (the optimistic cost estimate can make the chosen action look much better than the true optimal cost), a bound on r_t does not bound Term I. In fact, Term I can exceed 2 even when r_t is small. Inequality (20) is then used in the proof of Lemma 3 to obtain the cumulative Term I bound in Eq. (21), and that cumulative bound is combined with Lemma 4 or Lemma 5 in Appendices B.4 and B.5 to prove Theorems 2 and 3. Thus the written proof of the main regret guarantees does not go through. The gap appears repairable—for example, by bounding r_t directly with a standard OFU argument rather than bounding Term I through the min-cap—but the current derivation is invalid.
[2.2, Algorithm 1, and Remark 1] The pure-exploration phase is specified only through the instruction 'Randomly choose x_t∈D_w', with an explicit sampling construction in Remark 1 available only when B is invertible. For a general convex body D0 and a general matrix B, the proof of Lemma 1 requires independent exact uniform samples from Dw and the positivity of λmin(E[XX†]); while Assumption 3 does ensure that Dw has nonempty interior and hence that the covariance is full rank, the paper does not state the computational model under which such samples are available. Please state explicitly that an exact uniform sampling oracle over Dw is assumed, or provide an implementable sampling procedure for the general case. This is a clarity/scope issue rather than a mathematical contradiction, but it affects the algorithmic claim made by Theorem 2 and Theorem 3.

minor comments (5)

[3.3, Lemma 2] The statement of Lemma 2 gives probability at least 1−δ, while the appendix version (Lemma 4) gives probability at least 1−2δ after union-bounding the confidence event and the Gram-matrix event; please harmonize these statements and spell out the union bound.
[F] The sentence 'This is done in Figures 2a and 3c' appears to cite the wrong figure: the safe-set plots are in Figure 2, whereas Figure 3 shows average regret curves.
[B.3] In the paragraph defining the shrunk safe set, 'compared to the original ~D_s_t' should read 'compared to the original D_s_t', and in Eq. (24) the text uses 'C>0' where the constant is denoted c elsewhere.
[3.3] The phrase 'cumulant regret' is used in a few places and should be 'cumulative regret'.
[E, Eq. (45)] The displayed definition of Δ_i^t is garbled; please rewrite it so that the constraint 'v†y_i ≤ v†y for all y∈Y_i^t' is clearly separated from the minimization over v.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Safe-LUCB's regret bounds are derived from independent concentration inequalities and problem parameters, not from fitted inputs or self-citations.

full rationale

Safe-LUCB's derivation chain is self-contained against external benchmarks. The confidence radius β_t is imported from Abbasi-Yadkori–Pál–Szepesvári (Theorem 1), an independent prior result, and the regret bounds are proved from concentration inequalities and matrix-Chernoff bounds rather than fitted. Lemma 2's containment x* ∈ D^s_t is not definitional: D^s_t is the set of actions safe for every v in C_t, and the lemma uses a lower bound on λ_min(A_{T'+1}) plus Δ = c − μ†Bx* to show μ†Bx* + 2β_t‖Bx*‖_{A_t^{-1}} ≤ c; no equation defines D^s_t to include x*. Theorems 2 and 3 combine Lemma 3 (Term I) with Lemmas 4 and 5 (Term II), all proved from the preceding inequalities; neither the safety gap Δ nor the constants S, L, R, λ are calibrated to force the stated regret order. The paper contains no load-bearing self-citation by the present authors, and no fitted input is relabeled as a prediction. The possible objection to the step 'Term I ≤ 2 min(β_t‖x_t‖_{A_t^{-1}}, 1)' is a proof-technical gap, not circularity: it would affect correctness of the derivation, but it does not amount to the conclusion being assumed as an input. Accordingly, no circular step is exhibited.

Assumptions & free parameters 1 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard bandit assumptions, a uniform sampling oracle over Dw that the paper does not fully specify, and external theorems for confidence ellipsoids and matrix concentration. No free parameters are fitted to data; λ is a standard user-chosen regularizer. No new entities are introduced.

free parameters (1)

regularization parameter λ = λ=1 in experiments, arbitrary positive in theory
Standard ridge regularization in the least-squares estimate. Any λ>0 yields valid bounds, so it is a user-chosen hyperparameter, not fitted to data.

assumptions (7)

domain assumption Noise η_t is conditionally zero-mean R-sub-Gaussian (Assumption 1)
Needed for the confidence ellipsoid Theorem 1 from Abbasi-Yadkori et al.
domain assumption Boundedness: ‖µ‖≤S, ‖x‖≤L, and µ†x∈[−1,1] for all x∈D0 (Assumption 2)
Used to normalize regret, bound β_t, and construct the safe set Dw.
domain assumption D0 is a convex body containing the origin in its interior (Assumption 3)
Ensures the safe set is nonempty and the covariance of uniform samples from Dw is nonsingular (λ−>0).
ad hoc to paper The pure-exploration phase can draw independent uniform samples from Dw
Lemma 1 requires i.i.d. samples with covariance Σ having λmin(Σ)>0; the paper gives an explicit sampler only for invertible B in Remark 1.
domain assumption Safety gap Δ=c−µ†Bx* is positive and known to the learner (for Theorem 2)
Used to set the exploration length T∆; if Δ is unknown only the weaker Theorem 3 or the unanalyzed GSLUCB applies.
standard math Matrix Chernoff inequality (Tropp)
Used in Lemma 1 to lower bound the minimum eigenvalue of the Gram matrix.
standard math Confidence ellipsoid theorem of Abbasi-Yadkori et al. (Theorem 1)
Provides the width β_t of the confidence set Ct.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Linear Stochastic Bandits Under Safety Constraints." pith.science (2026). https://pith.science/paper/C5RHAJ7E

@misc{pith2026190805814,
  author       = {Pith},
  title        = {Pith review of: Linear Stochastic Bandits Under Safety Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C5RHAJ7E}},
  note         = {Machine review of arXiv:1908.05814}
}

read the original abstract

Bandit algorithms have various application in safety-critical systems, where it is important to respect the system constraints that rely on the bandit's unknown parameters at every round. In this paper, we formulate a linear stochastic multi-armed bandit problem with safety constraints that depend (linearly) on an unknown parameter vector. As such, the learner is unable to identify all safe actions and must act conservatively in ensuring that her actions satisfy the safety constraint at all rounds (at least with high probability). For these bandits, we propose a new UCB-based algorithm called Safe-LUCB, which includes necessary modifications to respect safety constraints. The algorithm has two phases. During the pure exploration phase the learner chooses her actions at random from a restricted set of safe actions with the goal of learning a good approximation of the entire unknown safe set. Once this goal is achieved, the algorithm begins a safe exploration-exploitation phase where the learner gradually expands their estimate of the set of safe actions while controlling the growth of regret. We provide a general regret bound for the algorithm, as well as a problem dependent bound that is connected to the location of the optimal action within the safe set. We then propose a modified heuristic that exploits our problem dependent analysis to improve the regret.

Figures

Figures reproduced from arXiv: 1908.05814 by the authors.

**Figure 1.** Simulation of per-step regret. bound ∆t for the safety gap ∆ and calculate the length of the pure exploration phase associated with ∆t, denoted as T 0 t . This allows the learner to stop the pure exploration phase at round t such that condition t ≤ T 0 t−1 has been met. While we do not provide a separate regret analysis for GSLUCB, it is clear that the worst case regret performance would match that of Safe-LUCB with… view at source ↗

**Figure 2.** Growth of Ds t with and without pure exploration phase. In both figures: D0 (in black) Ds 0 (in blue), DS T0+1 (in red), DS 5e4 (in green). Also, shown the optimal action x ∗ . Note that x ∗ ∈ DS T0+1 when pure exploration phase is used as suggested by Lemma 2. 0 2 4 6 8 10 104 0 0.2 0.4 0.6 0.8 1 1.2 (a) Safe-LUCB, T 0 = T∆ 0 2 4 6 8 10 104 0 0.2 0.4 0.6 0.8 1 1.2 (b) GSLUCB 0 2 4 6 8 10 104 0 0.2 0.4 0.6 0.8 1 1.2… view at source ↗

**Figure 3.** Comparison of mean per-step regret for Safe-LUCB( [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 22 canonical work pages

[1]

Stochastic linear optimization under bandit feedback

Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. 2008

2008
[2]

Improved algorithms for linear stochastic bandits

Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. InAdvances in Neural Information Processing Systems, pages 2312–2320, 2011

2011
[3]

Tsitsiklis

Paat Rusmevichientong and John N. Tsitsiklis. Linearly parameterized bandits.Mathematics of Operations Research, 35(2):395–411, 2010

work page 2010
[4]

Exploration–exploitation tradeoﬀ using variance estimates in multi-armed bandits.Theoretical Computer Science, 410(19):1876–1902, 2009

Jean-Yves Audibert, Rémi Munos, and Csaba Szepesvári. Exploration–exploitation tradeoﬀ using variance estimates in multi-armed bandits.Theoretical Computer Science, 410(19):1876–1902, 2009

work page 1902
[5]

Finite-time analysis of the multiarmed bandit problem

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Mach. Learn., 47(2-3):235–256, May 2002

work page 2002
[6]

Learning to optimize via posterior sampling.Mathematics of Operations Research, 39(4):1221–1243, 2014

Daniel Russo and Benjamin Van Roy. Learning to optimize via posterior sampling.Mathematics of Operations Research, 39(4):1221–1243, 2014

2014
[7]

Contextual bandits with linear payoﬀ functions

Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoﬀ functions. In Geoﬀrey Gordon, David Dunson, and Miroslav Dudík, editors,Proceedings of the Fourteenth International Conference on Artiﬁcial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 208–214, Fort Lauderdale, FL, USA, 11...

work page 2011
[8]

Parametric bandits: The generalized linear case

Sarah Filippi, Olivier Cappe, Aurélien Garivier, and Csaba Szepesvári. Parametric bandits: The generalized linear case. InAdvances in Neural Information Processing Systems, pages 586–594, 2010

work page 2010

Show all 28 references

[9]

Provably optimal algorithms for generalized linear contextual bandits

Lihong Li, Yu Lu, and Dengyong Zhou. Provably optimal algorithms for generalized linear contextual bandits. InProceedings of the 34th International Conference on Machine Learning- Volume 70, pages 2071–2080. JMLR. org, 2017

2017
[10]

Badanidiyuru, R

A. Badanidiyuru, R. Kleinberg, and A. Slivkins. Bandits with knapsacks. In2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pages 207–216, Oct 2013

2013
[11]

Linear contextual bandits with knapsacks

Shipra Agrawal and Nikhil Devanur. Linear contextual bandits with knapsacks. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors,Advances in Neural Information Processing Systems 29, pages 3450–3458. Curran Associates, Inc., 2016

2016
[12]

Srikant, Xin Liu, and Chong Jiang

Huasen Wu, R. Srikant, Xin Liu, and Chong Jiang. Algorithms with logarithmic or sublinear regret for constrained contextual bandits. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors,Advances in Neural Information Processing Systems 28, pages 433–44...

2015
[13]

Resourceful contextual bandits

Ashwinkumar Badanidiyuru, John Langford, and Aleksandrs Slivkins. Resourceful contextual bandits. In Maria Florina Balcan, Vitaly Feldman, and Csaba Szepesvári, editors,Proceedings of The 27th Conference on Learning Theory, volume 35 ofProceedings of Machine Learning Research,...

2014
[14]

Conserva- tive contextual linear bandits

Abbas Kazerouni, Mohammad Ghavamzadeh, Yasin Abbasi, and Benjamin Van Roy. Conserva- tive contextual linear bandits. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural Information Processing Systems 30, pag...

2017
[15]

Conservative bandits

Yifan Wu, Roshan Shariﬀ, Tor Lattimore, and Csaba Szepesvári. Conservative bandits. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, pages 1254–1262. JMLR.org, 2016

2016
[16]

Safe exploration in markov decision processes.arXiv preprint arXiv:1205.4810, 2012

Teodor Mihai Moldovan and Pieter Abbeel. Safe exploration in markov decision processes.arXiv preprint arXiv:1205.4810, 2012

2012 arXiv
[17]

Constrained policy optimization

Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 22–31. JMLR. org, 2017

2017
[18]

Safe exploration for active learning with gaussian processes

Jens Schreiter, Duy Nguyen-Tuong, Mona Eberts, Bastian Bischoﬀ, Heiner Markert, and Marc Toussaint. Safe exploration for active learning with gaussian processes. In Albert Bifet, Michael May, Bianca Zadrozny, Ricard Gavalda, Dino Pedreschi, Francesco Bonchi, Jaime Cardoso, and...

2015
[19]

Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics.arXiv preprint arXiv:1602.04450, 2016

Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics.arXiv preprint arXiv:1602.04450, 2016

2016 arXiv
[20]

Burdick, and Andreas Krause

Yanan Sui, Alkis Gotovos, Joel W. Burdick, and Andreas Krause. Safe exploration for op- timization with gaussian processes. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, ICML’15, pages 997–1005. JMLR.org, 2015

2015
[21]

Stagewise safe bayesian optimization with gaussian processes

Yanan Sui, Joel Burdick, Yisong Yue, et al. Stagewise safe bayesian optimization with gaussian processes. In International Conference on Machine Learning, pages 4788–4796, 2018

2018
[22]

Gaussian process optimization in the bandit setting: no regret and experimental design

Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: no regret and experimental design. InProceedings of the 27th International Conference on International Conference on Machine Learning, pages 1015–1022. Omn...

2010
[23]

Robust constrained learning- based nmpc enabling reliable mobile robot path tracking.The International Journal of Robotics Research, 35(13):1547–1563, 2016

Chris J Ostafew, Angela P Schoellig, and Timothy D Barfoot. Robust constrained learning- based nmpc enabling reliable mobile robot path tracking.The International Journal of Robotics Research, 35(13):1547–1563, 2016

2016
[24]

A. K. Akametalu, J. F. Fisac, J. H. Gillula, S. Kaynama, M. N. Zeilinger, and C. J. Tomlin. Reachability-based safe learning with gaussian processes. In53rd IEEE Conference on Decision and Control, pages 1424–1431, Dec 2014

2014
[25]

J. H. Gillulay and C. J. Tomlin. Guaranteed safe online learning of a bounded system. In2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2979–2984, Sep. 2011

2011
[26]

Provably safe and robust learning-based model predictive control.Automatica, 49(5):1216–1226, 2013

Anil Aswani, Humberto Gonzalez, S Shankar Sastry, and Claire Tomlin. Provably safe and robust learning-based model predictive control.Automatica, 49(5):1216–1226, 2013

2013
[27]

Safe convex learning under uncertain constraints

Ilnura Usmanova, Andreas Krause, and Maryam Kamgarpour. Safe convex learning under uncertain constraints. In Kamalika Chaudhuri and Masashi Sugiyama, editors,Proceedings of Machine Learning Research, volume 89 ofProceedings of Machine Learning Research, pages 2106–2114. PMLR, ...

2019
[28]

distance

Joel A Tropp et al. An introduction to matrix concentration inequalities.Foundations and Trends R© in Machine Learning, 8(1-2):1–230, 2015. 12 A Proof of Lemma 1 In order to bound the minimum eigenvalue of the Gram matrix at roundT′ + 1, we use the Matrix Chernoﬀ Inequality [2...

2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Stochastic linear optimization under bandit feedback

Varsha Dani, Thomas P Hayes, and Sham M Kakade. Stochastic linear optimization under bandit feedback. 2008

2008

[2] [2]

Improved algorithms for linear stochastic bandits

Yasin Abbasi-Yadkori, Dávid Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. InAdvances in Neural Information Processing Systems, pages 2312–2320, 2011

2011

[3] [3]

Tsitsiklis

Paat Rusmevichientong and John N. Tsitsiklis. Linearly parameterized bandits.Mathematics of Operations Research, 35(2):395–411, 2010

work page 2010

[4] [4]

Exploration–exploitation tradeoﬀ using variance estimates in multi-armed bandits.Theoretical Computer Science, 410(19):1876–1902, 2009

Jean-Yves Audibert, Rémi Munos, and Csaba Szepesvári. Exploration–exploitation tradeoﬀ using variance estimates in multi-armed bandits.Theoretical Computer Science, 410(19):1876–1902, 2009

work page 1902

[5] [5]

Finite-time analysis of the multiarmed bandit problem

Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finite-time analysis of the multiarmed bandit problem. Mach. Learn., 47(2-3):235–256, May 2002

work page 2002

[6] [6]

Learning to optimize via posterior sampling.Mathematics of Operations Research, 39(4):1221–1243, 2014

Daniel Russo and Benjamin Van Roy. Learning to optimize via posterior sampling.Mathematics of Operations Research, 39(4):1221–1243, 2014

2014

[7] [7]

Contextual bandits with linear payoﬀ functions

Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoﬀ functions. In Geoﬀrey Gordon, David Dunson, and Miroslav Dudík, editors,Proceedings of the Fourteenth International Conference on Artiﬁcial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 208–214, Fort Lauderdale, FL, USA, 11...

work page 2011

[8] [8]

Parametric bandits: The generalized linear case

Sarah Filippi, Olivier Cappe, Aurélien Garivier, and Csaba Szepesvári. Parametric bandits: The generalized linear case. InAdvances in Neural Information Processing Systems, pages 586–594, 2010

work page 2010