REVIEW 1 major objections 6 minor 1 cited by
Noise Sensitivity of the Semidefinite Programs for Direct Data-Driven LQR
T0 review · 1 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Under any nonzero Gaussian process noise, the standard SDP for direct data-driven LQR returns the zero gain with probability one, and the regularized SDP collapses in probability as data grows.
desk verdict Proof that the standard data-driven LQR SDP collapses to zero gain under any noise—clean negative result, deserves review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the combined data matrix $D_T=[X_0^\top\ U_0^\top\ X_1^\top]^\top$ together with the linear system $D_TY=[I_n;0_{m\times n};0_{n\times n}]$. The decisive fact (Lemma 3) is that with Gaussian noise and $T\ge(m+n)(n+1)+n$, $D_T$ has full row rank $2n+m$ with probability one; this follows by treating the noise as an extra input in the lifted system $x_{t+1}=Ax_t+[B\ I_n][u_t^\top\ w_t^\top]^\top$ and invoking the persistency-of-excitation lemma. Full row rank makes the linear system solvable, solvability makes the SDP constraints tight at a zero-gain solution, and Lemma 4 shows every optimal solution must satisfy that linear system. For the regularized program, the minimum-norm solution of the same linear system is feasible enough that the smallest singular value of $D_T$ growing like $\sqrt{T}$ drives the gain to zero in probability.
What would settle it
Run the SDP (7) on data drawn exactly from the paper's model for small $n,m$ and $T\ge(m+n)(n+1)+n$, using a high-precision solver; any optimal solution with $K_{\mathrm{ce}}\neq0_{m\times n}$ would contradict the claim that $K_{\mathrm{ce}}=0$ with probability one. A symbolic check for the smallest nontrivial $(n,m,T)$ can confirm whether the optimality conditions truly force $U_0Y^*=0$.
Extended reading notes
Core claim
Under the data-generating model $x_{t+1}=Ax_t+Bu_t+w_t$ with $w_t \stackrel{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\sigma_w^2 I_n)$, independent of the Gaussian input $u_t$, and with $T\ge (m+n)(n+1)+n$, the paper proves (Theorem 2) that $\mathbb{P}_T(K_{\mathrm{ce}}=0_{m\times n})=1$ for every optimal solution of the SDP (7). For the regularized SDP (9) with fixed $\eta>0$, it proves (Theorem 3) that $K_{\mathrm{rp}}(T)\xrightarrow{p}0_{m\times n}$ as $T\to\infty$. The algebraic mechanism is that the combined data matrix $D_T=[X_0^\top\ U_0^\top\ X_1^\top]^\top$ is full row rank with probability one, so the underdetermined linear system $D_TY=[I_n;0_{m\times n};0_{n\times n}]$ has a solution; such a solution makes $X_0Y=I_n$, $U_0Y=0$, and $X_1Y=0$, which attains the SDP's unconstrained lower bound $\mathrm{trace}(Q)$ and therefore is optimal. The regularized result extends the same forcing through quantitative persistency-of-excitation bounds that make the smallest singular value of $D_T$ grow like $\sqrt{T}$.
Load-bearing premise
The load-bearing premise is that the data are random in a way that makes the combined matrix $[X_0^\top\ U_0^\top\ X_1^\top]^\top$ full row rank with probability one: the process noise must be continuous, independent of the input, and nondegenerate, and the trajectory must be long enough ($T\ge(m+n)(n+1)+n$); if that rank condition fails, the SDP is no longer forced to the zero-gain solution.
Editorial extensions
If this is right
- With any nonzero independent Gaussian process noise, the certainty-equivalence SDP (7) returns the zero gain $K_{\mathrm{ce}}=0_{m\times n}$ with probability one once $T\ge(m+n)(n+1)+n$, so it never recovers the true LQR gain.
- The regularized SDP (9) with a fixed $\eta>0$ has gains $K_{\mathrm{rp}}(T)$ whose spectral norm converges to zero in probability as $T\to\infty$; adding more data does not restore consistency.
- The data-to-controller map is discontinuous: the same SDP gives the exact $K_{\mathrm{lqr}}$ in the noise-free case, while arbitrarily small noise flips it to the trivial zero gain.
- Under the theorem's hypotheses, the sufficient stabilizability condition of [8] is almost surely $\Psi=AA^\top$, so for open-loop unstable systems that condition fails with probability one and cannot certify stabilization in the noisy setting.
Reading between the lines
- The mechanism is broader than the Gaussian assumption: the proof only needs the combined data matrix to be full row rank almost surely, so any continuous, input-independent noise distribution that makes the lifted data persistently exciting should produce the same zero-gain collapse.
- This suggests the pathology belongs to the SDP encoding rather than to certainty equivalence itself, which sharpens the contrast with model-based certainty-equivalence LQR and points toward alternative direct formulations that provably mimic model-based solutions.
- The paper's upper-bound argument hints at a concrete remedy: if the regularization parameter grows with the horizon, say $\eta=cT$, the objective no longer collapses to $\mathrm{trace}(Q)$, and the numerical experiments with $\eta=10T$ show nonzero gains; a rigorous consistency analysis of that scaling is a natural next test.
- The same full-rank forcing should appear under measurement noise (as Remark 1 notes), suggesting that output-feedback or reduced-measurement variants may only escape the zero-gain trap if they break the exact solvability of $D_TY=[I_n;0;0]$.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper studies the noise sensitivity of two semidefinite programs (SDPs) for direct data-driven LQR: the certainty-equivalent DDD LQR (7) and the robustness-promoting regularized DDD LQR (9). Under the assumption that the process noise w_t is iid Gaussian with positive definite covariance, independent of the iid Gaussian input and the initial state, the authors prove two negative results. Theorem 2 shows that for T ≥ (m+n)(n+1)+n, every optimal solution of (7) yields the zero gain K_ce = 0_{m×n} with probability one, no matter how small σ_w > 0 is. Theorem 3 shows that for a fixed regularization parameter η, the gain K_rp(T) obtained from (9) converges to zero in probability as T → ∞. Consequently, neither SDP is a statistically consistent estimator of the true LQR gain. The proofs use a change of variables that treats noise as an additional input, the fundamental lemma, and concentration inequalities for the smallest singular value of the data matrix.
Significance. If the results hold, they provide a rigorous and somewhat surprising negative result for a widely cited SDP-based direct data-driven control method: the CE DDD LQR is discontinuous in the noise level, collapsing to a zero gain for any nonzero Gaussian noise, and the natural regularized variant does not fix the inconsistency as T grows. This is an important caution for the control community and clarifies the need for robust or otherwise regularized formulations. The main theorems are supported by elementary but careful algebraic arguments, and the paper includes numerical experiments that corroborate the theoretical predictions. The contributions are clearly bounded: the results cover Gaussian iid process noise and a particular regularized SDP, and the authors note that alternative formulations are not affected.
major comments (1)
- [Section V-B, Step 3 (Eqs. (34)-(37))] The proof of Theorem 3 contains a gap in the derivation of the key bound (37). The paper's notation defines σ(M) as the least singular value. Inequality (34) is obtained from trace(QX0Y) ≥ trace(Q)σ(X0Y) ≥ σ(Q)σ(X0Y), so it bounds the least singular value of X0Y*_rp(T). However, inequality (36) and the subsequent bound on ||U0Y*_rp(T)|| require an upper bound on the largest singular value of X0Y*_rp(T): indeed, ||R^{1/2}U0Y||_F^2 ≤ σ_max(X0Y) · trace(R^{1/2}U0Y (X0Y)^{-1} (R^{1/2}U0Y)^T). The manuscript does not provide such an upper bound on σ_max(X0Y); a bound on the smallest singular value does not help. This is load-bearing for Theorem 3. The gap is repairable: from (33) and Q ≽ σ(Q)I, obtain trace(X0Y) ≤ [trace(Q)+η(2n+m)/σ(D_T D_T^T)]/σ(Q), and hence σ_max(X0Y) ≤ trace(X0Y). Combining this with (36) yields (37). Please correct the singular value notation and supply this step.
minor comments (6)
- [Section V-B, Eqs. (36)-(37)] In (36) and (37), the symbol σ is used where the largest singular value is intended. Given the convention defined in the notation section, the authors should use \bar{\sigma} for the largest singular value of X0Y and U0Y; this will also make the repaired Step 3 clearer.
- [Appendix, Lemma 8] The proof of Lemma 8 states that the density of the determinant g is continuous and concludes P(g=0)=0 from P(g≤0)−P(g<0). This is not generally true: a non-constant polynomial of jointly Gaussian random variables need not have a continuous density (e.g., g=x^2 with x∼N(0,1)). The intended conclusion P(g=0)=0 is correct, but it should be justified by the standard fact that the zero set of a non-zero polynomial has Lebesgue measure zero and the Gaussian vector has an absolutely continuous distribution.
- [Section VI-B, last paragraph] The assertion that any optimal solution of RP DDD LQR converges in probability to a solution of equation (17) is stated without proof; please provide a proof or explicitly label it as a conjecture.
- [Remark 1] The measurement-noise extension in Remark 1 is stated without proof; please add a proof or qualify it as a conjecture.
- [Section IV-B, after Lemma 4] The generalization to arbitrary continuous input and noise distributions is asserted without proof; if retained, a short proof should be supplied.
- [Throughout] Minor typographical issues: 'Y ALMIP' should be 'YALMIP'; reference [13] is a submission and should be updated if a preprint is available; check the phrase 'controled' and other minor grammar errors.
Circularity Check
No significant circularity: the zero-gain theorems follow from the SDP's own constraints and external probabilistic lemmas, not from fitted or self-referential inputs.
full rationale
The derivation chain is self-contained. Theorem 2 rests on Lemma 3 (full row rank of D_T with probability one) and Lemma 4 (optimality iff D_T Y = [I; 0; 0]). Lemma 4 is proved directly from the SDP constraints: the lower bound trace(QX0Y) >= trace(Q) forces X0Y = I, then the second LMI forces X1Y = 0, and the objective forces U0Y = 0. The zero gain is a consequence of the SDP's own algebraic structure, not an input or a fitted parameter. Theorem 3 is likewise derived analytically: Lemmas 5-7 provide a high-probability lower bound on sigma(D_T), and the bounds (32)-(40) use only Schur complements, trace inequalities, and the minimum-norm feasible solution (31); no parameter is calibrated to make the conclusion hold. The only citation with a present coauthor is Lemma 6 from Oymak and Ozay [17], which is a standalone concentration inequality for iid Gaussian Hankel matrices; it neither assumes nor encodes the zero-gain conclusion, so it is independent support rather than circular self-citation. Non-circular caveats: Lemma 8's statement that the determinant's density is continuous is informal (the correct argument is that the zero set of a nonzero polynomial has Gaussian measure zero), and Remark 1 and the Section VI-B assertion that RP optimal solutions converge to solutions of (17) are asserted without proof. These affect proof formality, not circularity, and do not undermine the main theorems. Therefore no circular step is present.
Assumptions & free parameters
assumptions (6)
- domain assumption The input trajectory V0 = [U0; W0] is persistently exciting of order n+1 with probability one when u_t and w_t are iid Gaussian with nonsingular covariance and T is large enough (Lemma 8, Appendix).
- standard math Willems' fundamental lemma for input-state data, as stated in Lemma 2 and cited from [14], [15], holds.
- standard math The augmented system xt+1 = A xt + [B, I_n] vt is controllable for every pair (A, B).
- standard math Quantitative persistency of excitation lower bound (Lemma 5) from [16] and concentration inequality for the least singular value of the Hankel matrix (Lemma 6) from [17] hold.
- standard math Schur complement characterizations of the block-matrix constraints in (7) and (9) are valid, and the trace terms can be minimized exactly by setting X and S to their Schur complements.
- domain assumption The process noise w_t has full-rank covariance σ_w^2 I_n and is independent of the input u_t and initial state x0.
Cite this review
Pith. "Pith review of Noise Sensitivity of the Semidefinite Programs for Direct Data-Driven LQR." pith.science (2026). https://pith.science/paper/6MUM6SLM
@misc{pith2026241219705,
author = {Pith},
title = {Pith review of: Noise Sensitivity of the Semidefinite Programs for Direct Data-Driven LQR},
year = {2026},
howpublished = {\url{https://pith.science/paper/6MUM6SLM}},
note = {Machine review of arXiv:2412.19705}
}
read the original abstract
In this paper, we study the noise sensitivity of the semidefinite program (SDP) proposed for direct data-driven infinite-horizon linear quadratic regulator (LQR) problem for discrete-time linear time-invariant systems. While this SDP is shown to find the true LQR controller in the noise-free setting, we show that it leads to a trivial solution with zero gain matrices when data is corrupted by noise, even when the noise is arbitrarily small. We then study a variant of the SDP that includes a robustness promoting regularization term and prove that regularization does not fully eliminate the sensitivity issue. In particular, the solution of the regularized SDP converges in probability also to a trivial solution.
Figures
Forward citations
Cited by 1 Pith paper
-
Policy Gradient Adaptive Control for the LQR: Indirect and Direct Approaches
Online policy-gradient updates for unknown LQR systems are shown to be sequentially stable and convergent to the optimal gain, for indirect, direct, natural-gradient, Gauss-Newton and regularized versions.
Reference graph
Works this paper leans on
-
[1]
Formulas for data-driven contr ol: Stabilization, optimality, and robustness,
C. De Persis and P . Tesi, “Formulas for data-driven contr ol: Stabilization, optimality, and robustness,” IEEE Transactions on Automatic Control , vol. 65, no. 3, pp. 909–924, 2019
work page 2019
-
[2]
Guarantees for data-driven control of nonlinear systems using semidefinite programming: A survey
T. Martin, T. B. Sch¨ on, and F. Allg¨ ower, “Guarantees fo r data-driven control of nonlinear systems using semidefinite programmin g: A survey,” arXiv preprint arXiv:2306.16042 , 2023
work page Pith review arXiv 2023
-
[3]
Certainty equivalence is e fficient for linear quadratic control,
H. Mania, S. Tu, and B. Recht, “Certainty equivalence is e fficient for linear quadratic control,” Advances in Neural Information Processing Systems, vol. 32, 2019
work page 2019
-
[4]
On the sam ple com- plexity of the linear quadratic regulator,
S. Dean, H. Mania, N. Matni, B. Recht, and S. Tu, “On the sam ple com- plexity of the linear quadratic regulator,” F oundations of Computational Mathematics, vol. 20, no. 4, pp. 633–679, 2020
work page 2020
-
[5]
From noi sy data to feedback controllers: Nonconservative design via a matr ix s-lemma,
H. J. van Waarde, M. K. Camlibel, and M. Mesbahi, “From noi sy data to feedback controllers: Nonconservative design via a matr ix s-lemma,” IEEE Transactions on Automatic Control , vol. 67, no. 1, pp. 162–175, 2020
work page 2020
-
[6]
Data-driven quadratic stabiliza tion and LQR control of LTI systems,
T. Dai and M. Sznaier, “Data-driven quadratic stabiliza tion and LQR control of LTI systems,” Automatica, vol. 153, p. 111041, 2023. AUTHOR et al.: TITLE 9
work page 2023
-
[7]
R obust data- driven state-feedback design,
J. Berberich, A. Koch, C. W. Scherer, and F. Allg¨ ower, “R obust data- driven state-feedback design,” in 2020 American Control Conference (ACC). IEEE, 2020, pp. 1532–1538
work page 2020
-
[8]
Low-complexity learning of lin ear quadratic regulators from noisy data,
C. De Persis and P . Tesi, “Low-complexity learning of lin ear quadratic regulators from noisy data,” Automatica, vol. 128, p. 109548, 2021
work page 2021
Show all 20 references
-
[9]
On the certainty-e quivalence ap- proach to direct data-driven lqr design,
F. D¨ orfler, P . Tesi, and C. De Persis, “On the certainty-e quivalence ap- proach to direct data-driven lqr design,” IEEE Transactions on Automatic Control, 2023
2023
-
[10]
Data-enable d policy optimization for direct adaptive learning of the lqr,
F. Zhao, F. D¨ orfler, A. Chiuso, and K. Y ou, “Data-enable d policy optimization for direct adaptive learning of the lqr,” arXiv preprint arXiv:2401.14871, 2024
2024 arXiv
-
[11]
On the relationship between da ta-enabled predictive control and subspace predictive control,
F. Fiedler and S. Lucia, “On the relationship between da ta-enabled predictive control and subspace predictive control,” in 2021 European Control Conference (ECC) . IEEE, 2021, pp. 222–229
2021
-
[12]
On the equivalence of direct and indirect data-driven predictive control appr oaches,
P . Mattsson, F. Bonassi, V . Breschi, and T. B. Sch¨ on, “On the equivalence of direct and indirect data-driven predictive control appr oaches,” IEEE Control Systems Letters , 2024
2024
-
[13]
Noise sensitivity of dire ct data-driven linear quadratic regulator by semidefinite programming,
X. Zeng, L. Bako, and N. Ozay, “Noise sensitivity of dire ct data-driven linear quadratic regulator by semidefinite programming,” i n Submitted to American Control Conferece , 2025
2025
-
[14]
A note on persistency of excitation,
J. C. Willems, P . Rapisarda, I. Markovsky, and B. L. De Mo or, “A note on persistency of excitation,” Systems & Control Letters , vol. 54, no. 4, pp. 325–329, 2005
2005
-
[15]
Willems’ fundamental lemma for state-space systems and its extensio n to multiple datasets,
H. J. V an Waarde, C. De Persis, M. K. Camlibel, and P . Tesi , “Willems’ fundamental lemma for state-space systems and its extensio n to multiple datasets,” IEEE Control Systems Letters , vol. 4, no. 3, pp. 602–607, 2020
2020
-
[16]
A quantitative notion of persistency of excitation and the robust fundamen tal lemma,
J. Coulson, H. J. V an Waarde, J. Lygeros, and F. D¨ orfler, “A quantitative notion of persistency of excitation and the robust fundamen tal lemma,” IEEE Control Systems Letters , vol. 7, pp. 1243–1248, 2022
2022
-
[17]
Non-asymptotic identification of LTI systems from a single trajectory,
S. Oymak and N. Ozay, “Non-asymptotic identification of LTI systems from a single trajectory,” in 2019 American control conference (ACC) . IEEE, 2019, pp. 5655–5661
2019
-
[18]
Y almip : A toolbox for modeling and optimi zation in matlab,
J. L ¨ ofberg, “Y almip : A toolbox for modeling and optimi zation in matlab,” in In Proceedings of the CACSD Conference , Taipei, Taiwan, 2004. APPENDIX In this lemma, we establish that continuous random vari- ables, in particular Gaussian random variables, almost sur ely lea...
2004
-
[19]
Let T ≥ (m + n)(n + 1) + n and define HT := Hn+1 (V0(T ))
with σw > 0. Let T ≥ (m + n)(n + 1) + n and define HT := Hn+1 (V0(T )). Then, we have PT (rank (HT ) = ( m + n)(n + 1)) = 1 . That is, V0(T ) is persistently exciting of order n + 1 with probability one. Proof. Note that, by definition of the Hankel matrix, HT has (m + n)(n + 1)...
-
[20]
Therefore, HT ∗ is full rank with probability one, that is, PT ∗ (rank (HT ∗) = ( m + n)(n + 1)) = 1
= 1 . Therefore, HT ∗ is full rank with probability one, that is, PT ∗ (rank (HT ∗) = ( m + n)(n + 1)) = 1
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.