Pith. sign in

REVIEW 4 major objections 5 minor 30 references

On the Gradient Domination of the LQG Problem

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A history-based reparameterization makes the LQG cost gradient-dominated, so policy gradients provably converge to the optimal controller.

desk verdict The paper's global convergence claim for PG-LQG rests on a cost identity that does not hold for the controller it actually implements, so the main theorems are about a surrogate objective. read the letter →

arxiv 2507.09026 v1 pith:GS5K5OWX submitted 2025-07-11 math.OC cs.LGcs.SYeess.SY

Kasra Fallah , Leonardo F. Toso , James Anderson This is my paper · ORCID
classification math.OCcs.LGcs.SYeess.SY MSC 93E2049N1093B40
keywords linearquadraticGaussiancontrolpolicygradientmethodsdominancehistoryrepresentationglobalconvergencemodel-freepartiallyobservedsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the linear quadratic Gaussian (LQG) control problem, which resists global convergence guarantees for policy gradient methods in its classical dynamic-controller form, becomes gradient-dominated when controllers are parameterized by a finite history of past inputs and outputs. In this history representation, the lifted LQG cost satisfies the two properties that make policy gradient methods provably converge for LQR: gradient dominance and approximate smoothness. The authors use these properties to prove that policy gradient LQG globally converges to the optimal controller, in both a model-based setting where the system matrices are known and a model-free setting where the gradient is estimated by zeroth-order sampling. If correct, this gives the first global convergence guarantee for policy-gradient solution of the full noisy LQG problem, and it explains why longer input-output histories accelerate convergence in experiments.

What carries the argument

The central object is the lifting matrix $S^\star = [F_{u,p} + \tilde A^p O_{x,p}^\dagger(I-T_{u,p}),\; F_{y,p} - \tilde A^p O_{x,p}^\dagger T_{y,p}]$, which maps a $p$-step history of inputs and outputs $z_{t,p}$ to the state estimate $\hat x_t$, so the optimal controller is written as $u_t = \tilde K^\star z_{t,p} = K^\star S^\star z_{t,p}$. The lifted gain $\tilde K$ is not unique: any $\tilde K = \tilde K^\star + \Gamma$ with $\Gamma S^{\star\dagger}=0$ is also optimal. The argument hinges on $S^\star$ being full rank (so $S^{\star\dagger}$ exists and the mapping between classical and history parameterizations is well defined), on the lifted cost having the LQR-type form in equation (6), and on the gradient identity $\nabla J(\tilde K)=2 E_{\tilde K}\Sigma_{\tilde K}S^{\star\top}$, which lets the LQR gradient-dominance proof be carried over with a constant $\mu_{PL}$ involving $\|S^\star\|^2$.

What would settle it

Pick a system where $S^{\star\dagger}$ has a nontrivial null space, choose $\Gamma$ in that null space, and simulate or compute the true ergodic cost of the controller $u_t=(\tilde K^\star+\Gamma)z_{t,p}$; if it differs from the paper's formula (6), then gradient dominance and the convergence theorems hold only for a projected objective, not for the implemented controller.

Watch

Extended reading notes

Core claim

Under the history representation, the central claim is that the LQG cost $J(\tilde K) = \operatorname{Tr}(P_{\tilde K}\Sigma_\nu) + \text{constants}$ satisfies gradient dominance: for every stabilizing lifted controller $\tilde K$ in a sublevel set, $J(\tilde K)-J(\tilde K^\star) \le \frac{\|\Sigma_{\tilde K^\star}\|\,\|S^\star\|^2}{4\,\sigma(R)\,\sigma^2(\Sigma_{\tilde K})}\,\|\nabla J(\tilde K)\|_F^2$. Because the cost is also Lipschitz-smooth in the lifted controller, the standard descent lemma applies, and the paper proves a linear contraction of the optimality gap for the update $\tilde K_{n+1} = \tilde K_n - \eta\nabla J(\tilde K_n)$. With a zeroth-order gradient estimator, the same argument yields an $\epsilon$-optimal controller with high probability after enough samples, a small enough smoothing radius, and sufficiently many iterations. The construction trades the classical separation principle for a well-conditioned optimization landscape, and the optimal LQG controller is either recovered through the lifting matrix $S^\star$ and its pseudoinverse or implemented directly as a history-based policy.

Load-bearing premise

The load-bearing premise is that the formula $J(\tilde K)=\operatorname{Tr}(P_{\tilde K}\Sigma_\nu)+\text{constants}$ equals the true LQG cost of the implemented history-based controller for every stabilizing lifted gain, including gains whose null-space component adds a noise-driven term to the control input.

Editorial extensions

If this is right

  • Model-based PG-LQG converges linearly to the optimal LQG controller from any stabilizing initial lifted controller.
  • Model-free PG-LQG with a one-point zeroth-order gradient estimator reaches an $\epsilon$-suboptimal cost with high probability, with explicit sample, radius, step-size, and iteration requirements.
  • Every iterate of the model-free update can be kept stabilizing by choosing the smoothing radius, sample count, and step size appropriately (Theorem 2).
  • Longer history lengths $p$ shrink $\|S^\star\|$ and accelerate convergence, but they lengthen the warm-up phase, creating a trade-off between convergence speed and the required data horizon.
  • Recovering the classical controller $K^\star$ is unnecessary for executing a near-optimal policy: the lifted controller $\tilde K_N z_{t,p}$ can be implemented directly after a warm-up phase.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: the gradient-dominance constant depends on $\sigma(\Sigma_{\tilde K})$, so a stabilizing controller with a nearly singular state covariance could make the contraction factor very slow; experiments on such systems would reveal whether the bound is tight.
  • The same lifting argument may apply to other partially observed problems, such as output-feedback $H_2$ control or LQG with colored noise, where gradient dominance currently fails in classical parameterizations.
  • The paper's use of a one-point estimator is motivated by measurement noise; a variance-reduced estimator using common random numbers across rollouts, if feasible, could remove the $O(1/r^2)$ variance penalty without unrealistic noise assumptions.
  • Because the null-space component $\Gamma$ enters the implemented control as a noise-driven term, the true ergodic cost of an implemented $\tilde K$ may differ from the analytic formula (6); quantifying that difference is a direct check on whether the claimed global convergence matches the cost actually minimized in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies policy gradient methods for the discrete-time LQG problem. It proposes a 'history representation' in which the control input is u_t = K~ z_{t,p}, where z_{t,p} is the length-p input-output history. The authors define a lifted cost J(K~) = Tr(P_{K~} Sigma_nu) + constants, prove a gradient-dominance bound (Lemma 2) and a smoothness bound (Lemma 3) for this cost, and conclude global convergence of model-based and model-free policy gradient algorithms (Theorems 1 and 2). Numerical experiments on a 4-state open-loop unstable system are reported for varying history lengths p, together with a heuristic discount-annealing procedure for initializing a stabilizing controller.

Significance. If the central equivalence between the lifted cost and the true LQG cost held, the paper would be a significant contribution: it would add LQG to the small set of nonconvex control problems for which global policy-gradient convergence is known, and the explicit dependence of the convergence constants on problem data and history length is a useful feature. The authors also provide code and a numerical study, and they are careful to state their assumptions. However, the load-bearing premise of the paper is that the cost in Eq. (6) equals the LQG cost of the implemented history controller u_t = K~ z_{t,p}; as detailed below, this premise fails for the very null-space directions the paper explicitly allows. The convergence theorems are therefore proved for a surrogate objective, not for the cost that the model-free algorithm actually evaluates.

major comments (4)
  1. [Section II-A, Eq. (6)] The lifted cost J(K~) in Eq. (6) is not the LQG cost (2) of the controller u_t = K~ z_{t,p} that is actually implemented and simulated. Equation (6) is the separation-principle cost of the state-estimate feedback u_t = K~ S*† \hat{x}_t, where \hat{x}_t is the Kalman estimate generated with the optimal gain L; it is constant on the null space H = {Γ : Γ S*† = 0} because K~ enters only through K~ S*†. For K~ = K~* + Γ with Γ ∈ H, the control actually applied is (K~*+Γ) z_t, and Γ z_t is generically nonzero: z_t contains y_t, which includes the measurement noise v_t, and past controls that are generated recursively by the lifted controller. The assertion at the end of Section II-A that every such K~ is optimal is therefore incorrect. This invalidates the use of Eq. (6) in the sublevel set of Definition 1 and in the gradient of Lemma 1.
  2. [Section II-B, Lemma 1] The chain-rule step in Lemma 1 treats K = K~ S*† as the map from the lifted parameter to the implemented controller. For the history parameterization, the implemented map is the identity on z_t: u_t = K~ z_t. The identity \hat{x}_t = S* z_t is a property of the optimal closed loop under K~*, not of arbitrary K~ ∈ K. Hence the gradient in Eq. (7) is the gradient of the surrogate cost, not the gradient of the cost minimized by Algorithm 1. Since the gradient-dominance bound in Lemma 2 is stated in terms of this ∇J, the convergence theorems cannot be applied to the model-free estimator, which evaluates the true simulated cost.
  3. [Section II-A, definition of K] The set K is defined by ρ(A + B K~ S*†) < 1, i.e., stability of the state-feedback closed loop with gain K~ S*†. The closed loop under u_t = K~ z_t is an augmented system whose state includes at least (x_t, z_t) or the estimation error, and its stability is not characterized by this spectral radius condition. Consequently, the claim that K is 'the set of stabilizing controllers for the LQG problem in the history representation' is unsupported. The same gap affects Theorem 2, whose proof only shows that the iterates stay in the sublevel set of the surrogate cost, not that the implemented controller stabilizes the true noisy system.
  4. [Section III, Algorithm 1 and Theorem 1] The zeroth-order estimator in Algorithm 1 evaluates a finite-horizon version of the true cost (2) by simulating u_t = (K~ + U_i) z_{t,p}. The analysis bounding the estimation error uses smoothness and gradient dominance of the surrogate J from Eq. (6). Since the simulated cost is not equal to that surrogate, the bias term ‖∇J_r(K~) − ∇J(K~)‖ in Eq. (14) is not the bias of the estimator actually used, and the proof of Theorem 1 does not go through. In addition, the perturbation K~ + U_i may leave the stabilizing set, in which case the simulated cost is not finite; the paper does not provide a safeguard such as projection or rejection sampling, nor a bound on the perturbation radius that prevents this.
minor comments (5)
  1. [Proof of Lemma 2, Eq. (11)] Equation (11) is missing a factor 1/4: from ∇J = 2E Σ S*†^T and the right-inverse identity (S*†^T)^+ = S*, one obtains Tr(E^T E) = (1/4) Tr(S* Σ^{-1} ∇J^T ∇J Σ^{-1} S*^T). The final constant in Lemma 2 is correct after inserting this factor.
  2. [Lemma 3] The text 'the controller K = K~ S*' should read K = K~ S*†; the dagger is missing.
  3. [Section II-A, full-rank claim] The proof that S* is full row rank is not rigorous: controllability of (A,B) implies reachability of the state by some input sequence, not that every state estimate is expressible as S* z for the specific history vector z used here. A rank argument using observability and controllability of the lifted system would be needed.
  4. [Section III, warm-up discussion] The claim that u^{warm-up} = 0 is safe and that any warm-up instability 'diminishes over time' should be quantified: for unstable A, the state and hence the entries of z_{t,p} can grow during the warm-up phase, and a finite-horizon estimate with T = 100 can carry a transient bias that the ergodic limit argument does not remove.
  5. [Section IV-C, Figure 3] Figure 3 would be more informative if it also plotted the surrogate objective from Eq. (6) and the norm of the null-space component of K~_n, since the discrepancy between the two objectives is the main theoretical concern.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the core proof is a legitimate reduction to known LQR gradient-domination results, though an essential identity relating Eq. (6) to the implemented LQG cost is asserted, not proved.

full rationale

The derivation chain is not circular in the sense defined by the rubric. The paper constructs a lifting matrix S* from the optimal LQG data (K*, L), defines the lifted controller as K~ = K S*, and then reduces gradient dominance and smoothness of the lifted cost to the known LQR results of Fazel et al. (2018), Ziemann et al. (2022), and Gravell et al. (2020) via the substitution K = K~ S*†. This is a reduction to external, independently established results, not a self-citation chain, and no fitted parameter is later renamed as a prediction. The self-citations that appear ([9], [23], [27]) concern meta-learning, variance reduction, and stabilizing initialization, which are auxiliary to the main convergence theorem and are not load-bearing. The one genuinely problematic point is the asserted identity in Eq. (6): the paper claims that for every stabilizing K~, the LQG cost of the implemented controller u_t = K~ z_t equals the lifted expression Tr(P_{K~} Sigma_nu) + constants, where P_{K~} depends only on K~S*†. If that identity fails for components Gamma with Gamma S*† = 0, then the gradient dominance and convergence results apply to a projected objective rather than the cost actually evaluated in Algorithm 1. This is a mathematical correctness gap and an omitted proof of an essential identity, not a circular definition: the paper does not define the LQG cost to be (6), it asserts an equivalence. For this reason, the circularity score is low, though the missing identity should be treated as a serious correctness risk in any follow-up assessment.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No parameters are fitted to data. The paper relies on Assumptions 1 and 2 (standard) and on three implicit structural assumptions about the lifted problem that are not proven: full-rankness of S*, equality between the lifted cost and the actual cost, and nonsingularity of the relevant covariance. The latter two are load-bearing and are not stated explicitly.

assumptions (5)
  • domain assumption The pair (A,B) is controllable and (A,C) is observable (Assumption 1).
    Standard controllability and observability requirements for LQG and for the history representation to be well posed.
  • domain assumption The initial lifted controller \tilde K_0 is stabilizing (Assumption 2).
    Standard in PG-LQR; needed because the cost and gradient are undefined for non-stabilizing controllers.
  • ad hoc to paper For the chosen history length p, S* is full row rank.
    The proof by contradiction in Section II-A is informal and does not establish the required rank condition; no dependence of p on the observability index is given, and p >= ny is not sufficient in general.
  • ad hoc to paper The lifted cost J(\tilde K) in (6) equals the actual LQG cost of the history-based controller for all \tilde K in \mathcal K.
    This equivalence is asserted, not proven. It ignores the null-space component of \tilde K, which affects the actual control and cost when the system is noisy.
  • ad hoc to paper \Sigma_\nu is such that \Sigma_{\tilde K} = dlyap(A+B\tilde K S*^\dagger, \Sigma_\nu) has minimum singular value bounded away from zero on \mathcal K_\alpha.
    Needed for the gradient dominance constant in Lemma 2; if \Sigma_\nu is singular, which often happens when ny < nx, the bound is not finite and gradient dominance may fail.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Gradient Domination of the LQG Problem." pith.science (2026). https://pith.science/paper/GS5K5OWX

@misc{pith2026250709026,
  author       = {Pith},
  title        = {Pith review of: On the Gradient Domination of the LQG Problem},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GS5K5OWX}},
  note         = {Machine review of arXiv:2507.09026}
}
read the original abstract

We consider solutions to the linear quadratic Gaussian (LQG) regulator problem via policy gradient (PG) methods. Although PG methods have demonstrated strong theoretical guarantees in solving the linear quadratic regulator (LQR) problem, despite its nonconvex landscape, their theoretical understanding in the LQG setting remains limited. Notably, the LQG problem lacks gradient dominance in the classical parameterization, i.e., with a dynamic controller, which hinders global convergence guarantees. In this work, we study PG for the LQG problem by adopting an alternative parameterization of the set of stabilizing controllers and employing a lifting argument. We refer to this parameterization as a history representation of the control input as it is parameterized by past input and output data from the previous p time-steps. This representation enables us to establish gradient dominance and approximate smoothness for the LQG cost. We prove global convergence and per-iteration stability guarantees for policy gradient LQG in model-based and model-free settings. Numerical experiments on an open-loop unstable system are provided to support the global convergence guarantees and to illustrate convergence under different history lengths of the history representation.

Figures

Figures reproduced from arXiv: 2507.09026 by the authors.

Figure 1
Figure 1. Optimality gap of model-based PG-LQG with respect to the number [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Optimality gap of Algorithm 1 over the number of iterations. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 27 canonical work pages

  1. [1]

    Policy gradient methods for reinforcement learning with function approximation,

    R. S. Sutton, D. McAllester, S. Singh, and Y . Mansour, “Policy gradient methods for reinforcement learning with function approximation,” in Proc. Adv. Neural Inf. Process. Syst. , vol. 12, Denver, CO, USA, 1999, pp. 1057–1063

    work page 1999

  2. [2]

    Global convergence of policy gradient methods for the linear quadratic regulator,

    M. Fazel, R. Ge, S. M. Kakade, and M. Mesbahi, “Global convergence of policy gradient methods for the linear quadratic regulator,” in Proc. 35th Int. Conf. Mach. Learn. , vol. 80, Stockholm, Sweden, 2018, pp. 1467–1476

    work page 2018

  3. [3]

    Gradient methods for minimizing functionals,

    B. T. Polyak, “Gradient methods for minimizing functionals,” Zh. Vychisl. Mat. Mat. Fiz. [J. Comput. Math. Math. Phys.], vol. 3, no. 4, pp. 643–653, 1963

    work page 1963

  4. [4]

    On the linear convergence of random search for discrete-time LQR,

    H. Mohammadi, M. Soltanolkotabi, and M. Jovanovi ´c, “On the linear convergence of random search for discrete-time LQR,” IEEE Control Syst. Lett., vol. 5, no. 3, pp. 989–994, Jun. 2020

    work page 2020

  5. [5]

    Toward a theoretical foundation of policy optimization for learning control policies,

    B. Hu, K. Zhang, N. Li, M. Mesbahi, M. Fazel, and T. Ba s ¸ar, “Toward a theoretical foundation of policy optimization for learning control policies,” Annu. Rev. Control Robot. Auton. Syst. , vol. 6, pp. 123–158, May 2023

    work page 2023

  6. [6]

    Policy optimization for H2 linear control with H∞ robustness guarantee: Implicit regularization and global convergence,

    K. Zhang, B. Hu, and T. Ba s ¸ar, “Policy optimization for H2 linear control with H∞ robustness guarantee: Implicit regularization and global convergence,” in Proc. Conf. Learn. Dyn. Control, Jul. 2020, pp. 179–190

    work page 2020

  7. [7]

    Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approach,

    H. Wang, L. F. Toso, A. Mitra, and J. Anderson, “Model-free Learning with Heterogeneous Dynamical Systems: A Federated LQR Approach,” arXiv preprint arXiv:2308.11743 , 2023

    arXiv 2023

  8. [8]

    Robot fleet learning via policy merging,

    L. Wang, K. Zhang, A. Zhou, M. Simchowitz, and R. Tedrake, “Robot fleet learning via policy merging,” arXiv preprint arXiv:2310.01362 , 2023

    arXiv 2023

Show all 30 references
  1. [9]

    Meta-learning linear quadratic regulators: A policy gradient MAML approach for model-free LQR,

    L. F. Toso, D. Zhan, J. Anderson, and H. Wang, “Meta-learning linear quadratic regulators: A policy gradient MAML approach for model-free LQR,” in Proc. 6th Annu. Learn. Dyn. Control Conf. , vol. 242, Jul. 2024, pp. 902–915

    2024

  2. [10]

    Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning,

    D. Zhan, L. F. Toso, and J. Anderson, “Coreset-Based Task Selection for Sample-Efficient Meta-Reinforcement Learning,” arXiv preprint arXiv:2502.02332, 2025

    2025 arXiv

  3. [11]

    On the Convergence of Policy Gradient for Designing a Linear Quadratic Regulator by Leveraging a Proxy System,

    L. Ye, A. Mitra, and V . Gupta, “On the Convergence of Policy Gradient for Designing a Linear Quadratic Regulator by Leveraging a Proxy System,” in 2024 IEEE 63rd Conference on Decision and Control (CDC) . IEEE, 2024, pp. 6016–6021

    2024

  4. [12]

    Globally convergent policy gradient methods for linear quadratic control of partially observed systems,

    F. Zhao, X. Fu, and K. You, “Globally convergent policy gradient methods for linear quadratic control of partially observed systems,” IFAC- PapersOnLine, vol. 56, no. 2, pp. 5506–5511, Jan. 2023

    2023

  5. [13]

    On the lack of gradient domination for linear quadratic Gaussian problems with incomplete state information,

    H. Mohammadi, M. Soltanolkotabi, and M. R. Jovanovi ´c, “On the lack of gradient domination for linear quadratic Gaussian problems with incomplete state information,” in Proc. 60th IEEE Conf. Decis. Control , Canc´un, Mexico, Dec. 2021, pp. 1120–1124

    2021

  6. [14]

    Analysis of the optimization landscape of linear quadratic Gaussian (LQG) control,

    Y . Tang, Y . Zheng, and N. Li, “Analysis of the optimization landscape of linear quadratic Gaussian (LQG) control,” Math. Program., vol. 202, no. 1, pp. 399–444, Nov. 2023

    2023

  7. [15]

    Behavioral feedback for optimal LQG control,

    A. A. A. Makdah, V . Krishnan, V . Katewa, and F. Pasqualetti, “Behavioral feedback for optimal LQG control,” in Proc. 61st IEEE Conf. Decis. Control, Canc ´un, Mexico, Dec. 2022, pp. 4660–4666

    2022

  8. [16]

    Imitation and transfer learning for LQG control,

    T. Guo, A. A. A. Makdah, V . Krishnan, and F. Pasqualetti, “Imitation and transfer learning for LQG control,” IEEE Control Syst. Lett. , vol. 7, pp. 2149–2154, 2023

    2023

  9. [17]

    The data-based LQG control problem,

    R. E. Skelton and G. Shi, “The data-based LQG control problem,” in Proc. 33rd IEEE Conf. Decis. Control , vol. 2, Lake Buena Vista, FL, USA, Dec. 1994, pp. 1447–1452

    1994

  10. [18]

    K. Zhou, J. C. Doyle, and K. Glover, Robust and optimal control . Englewood Cliffs, NJ, USA: Prentice Hall, 1996

    1996

  11. [19]

    Escaping high-order saddles in policy optimization for linear quadratic Gaussian control,

    Y . Zheng, Y . Sun, M. Fazel, and N. Li, “Escaping high-order saddles in policy optimization for linear quadratic Gaussian control,” in Proc. 61st IEEE Conf. Decis. Control , Canc ´un, Mexico, Dec. 2022, pp. 5329–5334

    2022

  12. [20]

    Data-Driven Policy Gradient Method for Optimal Output Feedback Control of LQR,

    J. Xie and Y .-H. Ni, “Data-Driven Policy Gradient Method for Optimal Output Feedback Control of LQR,” in 2024 14th Asian Control Conference (ASCC). IEEE, 2024, pp. 1039–1044

    2024

  13. [21]

    How are policy gradient methods affected by the limits of control?

    I. Ziemann, A. Tsiamis, H. Sandberg, and N. Matni, “How are policy gradient methods affected by the limits of control?” in Proc. 61st IEEE Conf. Decis. Control , Canc ´un, Mexico, Dec. 2022, pp. 5992–5999

    2022

  14. [22]

    Learning optimal controllers for linear systems with multiplicative noise via policy gradient,

    B. Gravell, P. M. Esfahani, and T. Summers, “Learning optimal controllers for linear systems with multiplicative noise via policy gradient,” IEEE Trans. Autom. Control, vol. 66, no. 11, pp. 5283–5298, Nov. 2020

    2020

  15. [23]

    Oracle complexity reduction for model-free LQR: A stochastic variance-reduced policy gradient approach,

    L. F. Toso, H. Wang, and J. Anderson, “Oracle complexity reduction for model-free LQR: A stochastic variance-reduced policy gradient approach,” in Proc. Amer. Control Conf., Jul. 2024, pp. 4032–4037

    2024

  16. [24]

    Computing stabilizing linear controllers via policy iteration,

    A. Lamperski, “Computing stabilizing linear controllers via policy iteration,” in Proc. 59th IEEE Conf. Decis. Control , Dec. 2020, pp. 1902–1907

    2020

  17. [25]

    Stabilizing dynamical systems via policy gradient methods,

    J. Perdomo, J. Umenberger, and M. Simchowitz, “Stabilizing dynamical systems via policy gradient methods,” in Proc. Adv. Neural Inf. Process. Syst., vol. 34, Virtual, 2021, pp. 29 274–29 286

    2021

  18. [26]

    Convergence and sample complexity of policy gradient methods for stabilizing linear systems,

    F. Zhao, X. Fu, and K. You, “Convergence and sample complexity of policy gradient methods for stabilizing linear systems,” IEEE Trans. Autom. Control, Jan. 2024

    2024

  19. [27]

    Learning stabilizing policies via an unstable subspace representation,

    L. F. Toso, L. Ye, and J. Anderson, “Learning stabilizing policies via an unstable subspace representation,” arXiv preprint arXiv:2505.01348 , 2025

    2025 arXiv

  20. [28]

    Derivative-free methods for policy optimization: Guarantees for linear quadratic systems,

    D. Malik, A. Pananjady, K. Bhatia, K. Khamaru, L. B. P. and M. J. Wainwright, “Derivative-free methods for policy optimization: Guarantees for linear quadratic systems,” J. Mach. Learn. Res. , vol. 21, pp. 1–51, 2020

    2020

  21. [29]

    Random gradient-free minimization of convex functions,

    Y . Nesterov and V . Spokoiny, “Random gradient-free minimization of convex functions,” Found. Comput. Math., vol. 17, no. 2, pp. 527–566, 2017

    2017

  22. [30]

    Vershynin, High-dimensional probability: An introduction with applications in data science

    R. Vershynin, High-dimensional probability: An introduction with applications in data science . Cambridge, UK: Cambridge Univ. Press, 2018

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.