Leveraging Offline Data from Similar Systems for Online Linear Quadratic Control

Prateek Jaiswal; Shivam Bajaj; Vijay Gupta

REVIEW 3 major objections 4 minor 23 references

Leveraging Offline Data from Similar Systems for Online Linear Quadratic Control

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Thompson-sampling LQR controller that warm-starts from an offline trajectory of a similar system achieves $\tilde{\mathcal{O}}(\sqrt{T/S})$ Bayes regret when the two systems are close, and the guarantee degrades gracefully with a known…

desk verdict A plausible and novel transfer-learning framework for LQR, but the proof has a real gap: the sampling step is not posterior sampling, so Lemma 3's R0=0 does not follow and Theorem V.1 is not established as written. read the letter →

arxiv 2505.09057 v1 pith:2UR5DVLK submitted 2025-05-14 eess.SY cs.SY

Shivam Bajaj , Prateek Jaiswal , Vijay Gupta This is my paper · ORCID

classification eess.SYcs.SY

keywords linearquadraticregulatorThompsonsamplingBayesregretofflinedatawarmstartsim-to-realgaptransferlearningadaptivecontrol

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether an imperfect simulator can speed up online learning of a linear quadratic regulator when the real system's matrices are unknown. It proposes TSOD-LQR, a Thompson-sampling algorithm that initializes the online posterior with the offline trajectory's mean estimate and precision matrix, and adds a term penalizing a known bound $M_\delta$ on the dissimilarity between simulator and real system. The main result is a Bayes-regret bound after $T$ steps of $\tilde{\mathcal{O}}(\sqrt{T/S}(\log T + E[\alpha_S(\delta_1)+\sqrt{\lambda_{\max}(U_S)}M_\delta]))$, where $S$ is the offline trajectory length, and when $M_\delta$ is small this reduces to $\tilde{\mathcal{O}}(\sqrt{T/S})$. A sympathetic reader should care because the bound shows offline data from a different unknown system, not just from the real system, can cut exploration cost, and the benefit grows with $S$.

What carries the argument

The engine is the augmented least-squares recursion $V_t = U_S + \sum_{k=0}^{t-1} z_k z_k^{\top}$, $\hat{\theta}_t = V_t^{-1}(\sum_{k=0}^{t-1} z_k x_{k+1}^{\top} + U_S \hat{\theta}_S^{\mathrm{sim}})$, which converts the offline trajectory into a prior precision matrix $U_S$ and a prior mean $\hat{\theta}_S^{\mathrm{sim}}$; this is what makes the offline data shrink the confidence ellipsoid at rate $1/\sqrt{S}$. The Thompson-sampling radius $\beta_t(\delta_2) = n\sqrt{2\log(\det(V_t)^{0.5}/\det(U_S)^{0.5}\delta_2)} + \alpha_S(\delta_1) + \sqrt{\lambda_{\max}(U_S)}M_\delta$ is the second ingredient: it honestly accounts for three sources of error, so the prior is not trusted too much when the simulator is far from the real system. The proof machinery is the standard Thompson-sampling regret decomposition into one-step cost difference, state-bound, and model-change terms, with the condition $S>T$ used to make the high-probability event $E_T$ hold via the union bound.

What would settle it

Concretely: simulate the identical-system case ($M_\delta=0$), choose $S=T/2$, and set $\delta_1=\delta/(16S)$, $\delta_2=\delta/(16T)$ exactly as in Lemma 2. The proof's union bound then gives $\Pr(\hat{E}_T)\ge 1 - T(\delta_1+\delta_2) \approx 1 - \delta/8 \cdot (T/S)$, which is strictly worse than the claimed $1-\delta/8$; if in repeated trials the regret violates the $\tilde{\mathcal{O}}(\sqrt{T/S})$ rate whenever $S<T$, the $S>T$ premise is doing essential work.

Watch

Extended reading notes

Core claim

The central claim is that a longer offline trajectory from a similar-but-not-identical linear system is a genuine resource for online LQR control, not just a warm-start heuristic. The algorithm forms $V_t = U_S + \sum_{k=0}^{t-1} z_k z_k^{\top}$ and $\hat{\theta}_t = V_t^{-1}(\sum_{k=0}^{t-1} z_k x_{k+1}^{\top} + U_S \hat{\theta}_S^{\mathrm{sim}})$, so the offline data enter exactly as prior pseudo-observations with precision $U_S$. Thompson sampling then draws $\tilde{\theta}_t$ from a confidence ellipsoid around $\hat{\theta}_t$ whose radius $\beta_t(\delta_2)$ contains the online self-normalized noise, the offline estimation error $\alpha_S(\delta_1)$, and the dissimilarity penalty $\sqrt{\lambda_{\max}(U_S)}M_\delta$. Provided $S>T$, the paper proves, with probability at least $1-\delta$, a Bayes regret bound $\tilde{\mathcal{O}}(\sqrt{T/S}(\log T + E[\alpha_S(\delta_1)+\sqrt{\lambda_{\max}(U_S)}M_\delta]))$, and in the identical-system case $M_\delta=0$ it becomes $\tilde{\mathcal{O}}(\sqrt{T/S})$.

Load-bearing premise

The proof's high-probability guarantee only goes through when the offline trajectory is longer than the online learning horizon ($S>T$), so the central claim is only established for problems where a simulator can supply that much data.

Editorial extensions

If this is right

With $M_\delta=0$, Corollary 1 gives $\tilde{\mathcal{O}}(\sqrt{T/S})$ regret, so an offline trajectory from the exact system can push the online learning cost toward logarithmic growth when $S$ is much larger than $T$.
If $M_\delta$ is small but nonzero, the bound still improves over the no-offline baseline as long as $\alpha_S(\delta_1)+\sqrt{\lambda_{\max}(U_S)}M_\delta$ grows more slowly than $\sqrt{T}$; the claimed speedup is real only when the dissimilarity penalty is dominated by the offline estimation benefit.
The algorithm works without knowing the prior distribution $\mu$ of $\theta_*$; the known-distribution assumption is used only in the analysis through the posterior-sampling lemma.
The framework extends to multiple offline sources: precision matrices and dissimilarity penalties add, giving a similar $\tilde{\mathcal{O}}(\sqrt{T/S})$ bound with $S=\sum_i S_i$ and $M_\delta=\max_i M_{\delta,i}$.
Numerically, the paper reports that cumulative regret decreases as $S$ grows for fixed $M_\delta$, and that using both the offline estimate and its uncertainty beats both ignoring the offline data and using only the offline point estimate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the proof only needs Assumption 2 on the offline algorithm, the practical lever is to design the offline exploration policy to spread out the state-action pairs and grow the minimum eigenvalue of $U_S$, not merely to minimize the offline estimation error $\alpha_S$; the regret bound rewards both, but $\lambda_{\min}(U_S)\ge S/40$ enters the log-det terms.
The analysis is Bayesian and relies on the posterior-sampling lemma for $R_0=0$; a frequentist or adversarial version would need a different argument, and for large $M_\delta$ the bound suggests the offline prior may actually hurt, so there should be a crossover point at which discarding the offline data is safer.
A direct testable extension: replace Thompson sampling with a certainty-equivalence or UCB sampler inside the same augmented-precision recursion; the concentration part of the proof would survive, but the regret decomposition terms $R_0$ and the model-change term would need reworking, so the $\sqrt{T/S}$ rate may not be automatic.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's core idea is genuinely new and worth engaging with: it extends Thompson sampling for LQR to incorporate an offline trajectory from a different unknown system, with a regret bound that scales like sqrt(T/S) and depends on a dissimilarity bound M_delta. That is a meaningful problem in sim-to-real transfer, and the algorithm (TSOD-LQR) is a natural construction: regularized least squares with the offline data folded into the precision and the estimate, plus an inflated confidence radius. The multi-source extension in Section VII is a nice bonus. Credit where due: the problem is well motivated, the related work is cited honestly, and the confidence-bound derivation in Theorem IV.1 follows a coherent chain.

The soft spots are real, and the most serious one is the stress-test concern. Lemma 3 claims R0 = 0 by invoking the posterior sampling lemma, but the algorithm never samples from the posterior. Equation (7) samples a truncated, confidence-inflated perturbation around the regularized estimate, with beta_t > 1 and no relationship to the prior mu. Under a Dirac prior, the posterior is deterministic, while the sampling is non-degenerate; under a general prior, the sampling distribution is still not the posterior. So the hypotheses of the posterior sampling lemma fail, R0 is not demonstrably zero, and the regret decomposition (11) is missing an unanalyzed term. This is a load-bearing gap: the claimed O~(sqrt(T/S)) bound is not established. It is also independent of the S>T issue. That said, the gap might be patchable—one could try to bound R0 directly using the confidence ellipsoid and Lipschitz properties of J, possibly with an extra term that still fits the rate—but that has not been done here.

The S>T assumption is also restrictive and is brushed off too quickly in Remark 1. Several lemmas (6, 9, 11) are only outlined or deferred to [15], so the paper is not fully self-contained. The numerical section is thin (10 runs) and there is no formal comparison against the no-offline baseline, despite the abstract claiming outperformance. The conclusion contains an error (O~(sqrt(T)) instead of sqrt(T/S)), which suggests a hasty final pass.

This is not a take-down; the idea deserves serious attention. But the proof as written has a genuine hole, and the authors should be asked to fix it, not just polish the presentation. I would send it to peer review because the problem is important and the core contribution is likely salvageable. I would not cite it in its current form until the gap is closed.

Referee Report

3 major / 4 minor

Summary. The paper considers online LQR control of an unknown linear system, when in addition to online state-action data, an offline trajectory of length S from a different but similar unknown system is available. The proposed TSOD-LQR algorithm forms a regularized least-squares estimate that combines online data with an offline estimate, and uses Thompson-sampling-style perturbations (Eq. 7) to select controllers. The central claim is a Bayesian regret bound of O~(sqrt(T/S)(log T + E[alpha_S(delta1) + sqrt(lambda_max(U_S)) M_delta])) under the assumption S>T (Theorem V.1), and a smaller regret when the dissimilarity M_delta is small. The paper also sketches an extension to multiple offline sources and presents numerical illustrations.

Significance. If the main result held, the paper would make a useful contribution: it gives a principled way to use offline data from a related system to reduce the regret of online LQR control, with an explicit dependence on the dissimilarity M_delta and offline length S. The attempt to characterize when offline data helps, rather than merely to demonstrate empirical gains, is valuable. However, the main theorem is not established as written because a key lemma in the regret decomposition is based on a misapplication of the posterior sampling lemma, and several central proof steps are only sketched with reference to previous work. The paper also contains a factual inconsistency between the stated theorem rate and the rate quoted in the conclusion.

major comments (3)

[Section V, Lemma 3 and Eq. (11)] The claim R0=0 is unsupported because the sampling distribution in Eq. (7) is not the posterior of theta* under the prior mu. The algorithm samples theta_tilde_t = S_Q(theta_hat_t + beta_t(delta2) V_t^{-1/2} eta_t), which is a confidence-inflated least-squares perturbation projected onto Q. The prior mu is known only to the analysis and is never used by the algorithm. The posterior sampling lemma [22, Lemma 1] requires the sampled parameter to be drawn from the posterior of theta* given the filtration; here the distribution is that of the random perturbation around the regularized least-squares estimate. For example, if mu is a point mass at theta0, the posterior is deterministic but Eq. (7) is non-degenerate, so E[J(theta_tilde_t)] != J(theta0) in general. Consequently, the regret decomposition (11) is missing a nonzero term R0, and the proof of Theorem V.1 is incomplete. This issue is independent of the S>T condition and affects the central regret bound.
[Section V, Lemma 6 and Appendix C] The proof of Lemma 6, which bounds the term R3, is only an outline. The text states that the proof 'resembles that of [15, Lemma 1]' and then invokes [15, Propositions 7, 8, 9, 11] and Lemma 11, whose proof is also omitted. Since Lemma 6 is a load-bearing component of the final regret bound, the adaptation of these arguments to the present setting, where beta_t includes the offline terms alpha_S(delta1) and sqrt(lambda_max(U_S)) M_delta, must be shown explicitly. As written, the proof of Theorem V.1 cannot be fully verified from the manuscript.
[Section V, Lemma 2 and Remark 1] The requirement S>T is structural, not an inconsequential technical convenience. In the proof of Lemma 2, selecting delta1 = delta/(16S) and delta2 = delta/(16T) gives P(hat E_T) >= 1 - T(delta1+delta2) = 1 - delta/16 - T delta/(16S). If S <= T, this lower bound can be as low as 1 - delta/8 - delta/16, which is less than the claimed 1 - delta/4, so the union bound over T events fails. The paper's statement that S>T is 'not an onerous assumption' is debatable; for real systems, offline trajectories longer than the entire online horizon may be unrealistic. The problem statement and abstract do not state this condition, and the paper does not provide any result for the practically relevant regime S <= T. The authors should either prove the bound for S <= T, state clearly that the result requires S>T in the abstract and introduction, or discuss the necessity of this assumption.

minor comments (4)

[Section VIII] The conclusion says the algorithm achieves O~(f(S,M_delta) sqrt(T)), which contradicts Theorem V.1 and the abstract, where the rate is sqrt(T/S). This inconsistency should be corrected.
[Abstract and Introduction] The notation \tilde{O} is used without a formal definition; please define it explicitly (the abstract and Section V use it with different implicit meanings).
[Section VI, Figure 1 caption] The caption contains a typo, 'al algorithm', and the details of the two baseline algorithms ('does not utilize the offline data' and 'only utilizes the estimate theta_hat_sim') are not described in the text. Please specify the baselines so the simulation comparison is reproducible.
[Section VII] In the definition of e(θ), the term (x_{k+1} - θ^T z_s)(x_{k+1} - θ^T z_s)^T uses z_s instead of z_k; this appears to be a typographical error that should be fixed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the regret bound is a conditional consequence of the assumed offline-estimator guarantees, not an identity with its inputs.

full rationale

The derivation chain is not circular. The main result, Theorem V.1, is obtained by combining a confidence bound (Theorem IV.1) with an assumed offline-algorithm guarantee (Assumption 2) and standard self-normalized martingale bounds (Lemmas 12-13). The quantities α_S(δ1) and sqrt(λ_max(U_S)) M_δ appear in β_t and therefore reappear in the final regret bound; this is an explicit dependence on the assumed accuracy of the offline data and the stated dissimilarity bound, not a fitted parameter renamed as a prediction. The condition S > T is a stated premise used in the union bound, not a conclusion smuggled from the regret formula. The paper contains no self-citations by the authors; its citations to prior Thompson-sampling analyses (Abeille-Lazaric, Kargin et al., Osband-Van Roy) are external. The reviewer-identified problem with Lemma 3—that Eq. (7) samples a confidence-inflated least-squares perturbation rather than the posterior of θ* under μ, so R0 = 0 is unsupported—is a possible proof-validity gap, but it is not circularity: no equation is equivalent to its input by construction, and no quantity is defined in terms of the claimed conclusion. Therefore the circularity score is 0, while the correctness of the proof remains a separate question.

Assumptions & free parameters 2 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard LQR/bandit assumptions plus two problem-specific premises: S>T and a known dissimilarity bound M_delta. The paper does not introduce new physical entities. The most fragile inputs are Assumption 2 and M_delta, because both are assumed rather than verified by the paper.

free parameters (2)

M_delta
Known upper bound on ||theta_delta_*||_F = ||[A_*-A_sim_*, B_*-B_sim_*]||_F. It enters the confidence radius beta_t and the regret bound; the algorithm requires it as an input.
alpha_S(delta1)
Confidence-radius function from Assumption 2 characterizing offline least-squares error. Not computed in the paper; the regret bound depends on its expectation and it is assumed to be provided by the offline algorithm.

assumptions (7)

domain assumption Assumption 1: noise sequences w_t and w_sim_s are martingale difference sequences with identity conditional covariance.
Standard stochastic noise model for LQR; enables self-normalized concentration bounds.
domain assumption Assumption 2: offline algorithm A_sim satisfies ||U_S^{0.5}(theta_hat_sim - theta_sim_*)||_F <= alpha_S(delta1) and lambda_min(U_S) >= S/40 for S large.
This is the key external guarantee on offline data quality. Not proved in this paper; cited to TS/UCB algorithms.
domain assumption True parameter theta_* lies in Q = {theta: Tr(P(theta))<=M_P, ||A_*+B_*K(theta)||_2<=rho<1}.
Ensures sampled controllers stabilize the true system and state norms remain bounded; standard in TS-LQR analysis, e.g., [15].
domain assumption The prior distribution mu of theta_* is known for the analysis, via the posterior sampling lemma.
Used in Lemma 3 to get R_0 = 0; the algorithm itself does not require mu.
ad hoc to paper S > T, i.e., offline trajectory length exceeds online horizon.
Needed in Lemma 2 so T(delta1+delta2) <= delta/8 with delta1=delta/(16S). Without S>T the high-probability event and the regret guarantee fail.
domain assumption Known bound M_delta with ||theta_delta_*||_F <= M_delta.
Used in a triangle inequality in the proof of Theorem IV.1 to bound the bias from using offline data from a different system.
domain assumption theta_sim_* in P (bounded norm, stabilizable, cost bound) for Theorem V.2.
Used to invoke state bounds for TSAC from [2].

how reviews work

0 comments

Cite this review

Pith. "Pith review of Leveraging Offline Data from Similar Systems for Online Linear Quadratic Control." pith.science (2026). https://pith.science/paper/2UR5DVLK

@misc{pith2026250509057,
  author       = {Pith},
  title        = {Pith review of: Leveraging Offline Data from Similar Systems for Online Linear Quadratic Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2UR5DVLK}},
  note         = {Machine review of arXiv:2505.09057}
}

abstract

``Sim2real gap", in which the system learned in simulations is not the exact representation of the real system, can lead to loss of stability and performance when controllers learned using data from the simulated system are used on the real system. In this work, we address this challenge in the linear quadratic regulator (LQR) setting. Specifically, we consider an LQR problem for a system with unknown system matrices. Along with the state-action pairs from the system to be controlled, a trajectory of length $S$ of state-action pairs from a different unknown system is available. Our proposed algorithm is constructed upon Thompson sampling and utilizes the mean as well as the uncertainty of the dynamics of the system from which the trajectory of length $S$ is obtained. We establish that the algorithm achieves $\tilde{\mathcal{O}}({f(S,M_{\delta})\sqrt{T/S}})$ Bayes regret after $T$ time steps, where $M_{\delta}$ characterizes the \emph{dissimilarity} between the two systems and $f(S,M_{\delta})$ is a function of $S$ and $M_{\delta}$. When $M_{\delta}$ is sufficiently small, the proposed algorithm achieves $\tilde{\mathcal{O}}({\sqrt{T/S}})$ Bayes regret and outperforms a naive strategy which does not utilize the available trajectory.

Figures

Figures reproduced from arXiv: 2505.09057 by the authors.

**Figure 1.** Cumulative regret plot comparing Algorithm 1 with an algorithm that (1) does not utilize the offline data and (2) only utilizes the estimate ˆθ sim computed from the offline data. Remark 2. From Theorem V.1, using offline data from system θ sim ∗ is beneficial if Mδ is sufficiently small. Corollary 1. Suppose that θ∗ = θ sim ∗ . Further, suppose that S ≥ max{T, 200(n + m) log 12 δ1 }. Then, with probability at least… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 15 canonical work pages

[15]

Improved regret bounds for thompson sampling in linear quadratic control problems,

M. Abeille and A. Lazaric, “Improved regret bounds for thompson sampling in linear quadratic control problems,” inInternational Conference on Machine Learning, pp. 1–9, PMLR, 2018

work page 2018
[1]

Regret bounds for the adaptive control of linear quadratic systems,

Y . Abbasi-Yadkori and C. Szepesv ´ari, “Regret bounds for the adaptive control of linear quadratic systems,” inProceedings of the 24th Annual Conference on Learning Theory, pp. 1–26, JMLR Workshop and Conference Proceedings, 2011

work page 2011
[2]

Thompson sampling achieves ˜O √ Tregret in linear quadratic control,

T. Kargin, S. Lale, K. Azizzadenesheli, A. Anandkumar, and B. Hassibi, “Thompson sampling achieves ˜O √ Tregret in linear quadratic control,” inConference on Learning Theory, pp. 3235–3284, PMLR, 2022

work page 2022
[3]

Control of unknown linear systems with thompson sampling,

Y . Ouyang, M. Gagrani, and R. Jain, “Control of unknown linear systems with thompson sampling,” in2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1198–1205, IEEE, 2017

work page 2017
[4]

Learning linear-quadratic regulators efficiently with only √ Tregret,

A. Cohen, T. Koren, and Y . Mansour, “Learning linear-quadratic regulators efficiently with only √ Tregret,” inInternational Conference on Machine Learning, pp. 1300–1309, PMLR, 2019

work page 2019
[5]

Reinforcement learning in practice: Opportunities and challenges,

Y . Li, “Reinforcement learning in practice: Opportunities and challenges,”arXiv preprint arXiv:2202.11296, 2022

arXiv 2022
[6]

A survey on offline reinforcement learning: Taxonomy, review, and open problems,

R. F. Prudencio, M. R. Maximo, and E. L. Colombini, “A survey on offline reinforcement learning: Taxonomy, review, and open problems,” IEEE Transactions on Neural Networks and Learning Systems, 2023

2023
[7]

Sim-to-real transfer in deep reinforcement learning for robotics: a survey,

W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: a survey,” in2020 IEEE symposium series on computational intelligence (SSCI), pp. 737–744, IEEE, 2020

work page 2020

Show all 23 references

[8]

Multi-armed bandit problems with history,

P. Shivaswamy and T. Joachims, “Multi-armed bandit problems with history,” inArtificial Intelligence and Statistics, pp. 1046–1054, PMLR, 2012

2012
[9]

Warm-starting contextual bandits: Robustly combining supervised and bandit feedback,

C. Zhang, A. Agarwal, H. D. Iii, J. Langford, and S. Negahban, “Warm-starting contextual bandits: Robustly combining supervised and bandit feedback,” inProceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 7335–7344, PMLR, 2019

2019
[10]

Leveraging offline data in linear latent bandits,

C. Kausik, K. Tan, and A. Tewari, “Leveraging offline data in linear latent bandits,”arXiv preprint arXiv:2405.17324, 2024

2024 arXiv
[11]

Leveraging demonstrations to improve online learning: Quality matters,

B. Hao, R. Jain, T. Lattimore, B. Van Roy, and Z. Wen, “Leveraging demonstrations to improve online learning: Quality matters,” in International Conference on Machine Learning, pp. 12527–12545, PMLR, 2023

2023
[12]

Certainty equivalence is efficient for linear quadratic control,

H. Mania, S. Tu, and B. Recht, “Certainty equivalence is efficient for linear quadratic control,”Advances in Neural Information Processing Systems, vol. 32, 2019

2019
[13]

Optimal dynamic regret in lqr control,

D. Baby and Y .-X. Wang, “Optimal dynamic regret in lqr control,” Advances in Neural Information Processing Systems, vol. 35, pp. 24879–24892, 2022

2022
[14]

Regret analysis of distributed online lqr control for unknown lti systems,

T.-J. Chang and S. Shahrampour, “Regret analysis of distributed online lqr control for unknown lti systems,”IEEE Transactions on Automatic Control, 2023

2023
[16]

Transfer learning for lqr control,

T. Guo and F. Pasqualetti, “Transfer learning for lqr control,”arXiv preprint arXiv:2503.06755, 2025

2025 arXiv
[17]

Imitation and transfer learning for lqg control,

T. Guo, A. A. Al Makdah, V . Krishnan, and F. Pasqualetti, “Imitation and transfer learning for lqg control,”IEEE Control Systems Letters, vol. 7, pp. 2149–2154, 2023

2023
[18]

Data-based transfer stabilization in linear systems,

L. Li, C. De Persis, P. Tesi, and N. Monshizadeh, “Data-based transfer stabilization in linear systems,”IEEE Transactions on Automatic Control, vol. 69, no. 3, pp. 1866–1873, 2023

2023
[19]

Learning dynamical systems by leveraging data from similar systems,

L. Xin, L. Ye, G. Chiu, and S. Sundaram, “Learning dynamical systems by leveraging data from similar systems,”IEEE Transactions on Automatic Control, 2025

2025
[20]

Improved algorithms for linear stochastic bandits,

Y . Abbasi-Yadkori, D. P´al, and C. Szepesv ´ari, “Improved algorithms for linear stochastic bandits,”Advances in neural information processing systems, vol. 24, 2011

2011
[21]

Logarithmic regret for learning linear quadratic regulators efficiently,

A. Cassel, A. Cohen, and T. Koren, “Logarithmic regret for learning linear quadratic regulators efficiently,” inInternational Conference on Machine Learning, pp. 1328–1337, PMLR, 2020

2020
[22]

Posterior sampling for reinforcement learning without episodes,

I. Osband and B. Van Roy, “Posterior sampling for reinforcement learning without episodes,”arXiv preprint arXiv:1608.02731, 2016

2016 arXiv
[23]

Online least squares estimation with self-normalized processes: An application to bandit problems,

Y . Abbasi-Yadkori, D. P ´al, and C. Szepesv ´ari, “Online least squares estimation with self-normalized processes: An application to bandit problems,”arXiv preprint arXiv:1102.2670, 2011

2011 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [15]

Improved regret bounds for thompson sampling in linear quadratic control problems,

M. Abeille and A. Lazaric, “Improved regret bounds for thompson sampling in linear quadratic control problems,” inInternational Conference on Machine Learning, pp. 1–9, PMLR, 2018

work page 2018

[2] [1]

Regret bounds for the adaptive control of linear quadratic systems,

Y . Abbasi-Yadkori and C. Szepesv ´ari, “Regret bounds for the adaptive control of linear quadratic systems,” inProceedings of the 24th Annual Conference on Learning Theory, pp. 1–26, JMLR Workshop and Conference Proceedings, 2011

work page 2011

[3] [2]

Thompson sampling achieves ˜O √ Tregret in linear quadratic control,

T. Kargin, S. Lale, K. Azizzadenesheli, A. Anandkumar, and B. Hassibi, “Thompson sampling achieves ˜O √ Tregret in linear quadratic control,” inConference on Learning Theory, pp. 3235–3284, PMLR, 2022

work page 2022

[4] [3]

Control of unknown linear systems with thompson sampling,

Y . Ouyang, M. Gagrani, and R. Jain, “Control of unknown linear systems with thompson sampling,” in2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1198–1205, IEEE, 2017

work page 2017

[5] [4]

Learning linear-quadratic regulators efficiently with only √ Tregret,

A. Cohen, T. Koren, and Y . Mansour, “Learning linear-quadratic regulators efficiently with only √ Tregret,” inInternational Conference on Machine Learning, pp. 1300–1309, PMLR, 2019

work page 2019

[6] [5]

Reinforcement learning in practice: Opportunities and challenges,

Y . Li, “Reinforcement learning in practice: Opportunities and challenges,”arXiv preprint arXiv:2202.11296, 2022

arXiv 2022

[7] [6]

A survey on offline reinforcement learning: Taxonomy, review, and open problems,

R. F. Prudencio, M. R. Maximo, and E. L. Colombini, “A survey on offline reinforcement learning: Taxonomy, review, and open problems,” IEEE Transactions on Neural Networks and Learning Systems, 2023

2023

[8] [7]

Sim-to-real transfer in deep reinforcement learning for robotics: a survey,

W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: a survey,” in2020 IEEE symposium series on computational intelligence (SSCI), pp. 737–744, IEEE, 2020

work page 2020