Pith. sign in

REVIEW 3 major objections 4 minor 62 references

Perturbed Gradient Descent Algorithms are Small-Disturbance Input-to-State Stable

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Perturbed gradient descent is small-disturbance ISS when the objective satisfies a nonlinear Polyak-Lojasiewicz condition; LQR policy gradient, natural gradient, and Gauss-Newton all qualify.

desk verdict A useful ISS framework for gradient methods, but the main theorem is false as stated because vanishing step sizes break the Lyapunov decay. read the letter →

arxiv 2507.02131 v1 pith:XJLIXQHZ submitted 2025-07-02 math.OC cs.SYeess.SY

Leilei Cui , Zhong-Ping Jiang , Eduardo D. Sontag , Richard D. Braatz This is my paper · ORCID
classification math.OCcs.SYeess.SY MSC 93D2593D3093C5590C30
keywords input-to-statestabilitysmall-disturbanceISSgradientdescentPolyak-LojasiewiczconditionpolicylinearquadraticregulatorLyapunovfunctiondiscrete-timenonlinearsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that gradient descent remains robust when the gradient is corrupted by a bounded disturbance, provided the objective satisfies a nonlinear generalization of the Polyak-Lojasiewicz condition, called the K-PL condition. The notion of small-disturbance input-to-state stability (ISS) is introduced for discrete-time systems, with a Lyapunov characterization, and it is shown that perturbed gradient descent with step size bounded by the inverse of the local Lipschitz constant is small-disturbance ISS. A direct application shows that the LQR cost satisfies the K-PL condition, so the standard policy gradient algorithm for LQR, as well as natural gradient and Gauss-Newton variants, are all small-disturbance ISS under gradient estimation errors. The paper's significance is that it replaces convexity or linearly decaying perturbation assumptions with a more flexible gradient-dominance condition, giving quantitative robustness guarantees for optimization algorithms that are widely used in reinforcement learning.

What carries the argument

The load-bearing concept is the K-PL condition, a nonlinear replacement for the classical Polyak-Lojasiewicz inequality: $\|\nabla J(z)\| \ge \alpha_5(J(z)-J(z^*))$ with $\alpha_5$ in class K, together with coercivity and Lipschitz continuity of the gradient on sublevel sets. The proof uses the cost gap $J(z)-J(z^*)$ as a small-disturbance ISS-Lyapunov function and shows that its decrease is proportional to $\alpha_5(J(z)-J(z^*))^2$ plus a term quadratic in the disturbance. For the LQR application, the paper supplies a Lipschitz constant $L(h)$ for the gradient over sublevel sets and invokes a global K-PL estimate for the LQR cost, with the saturating function $\alpha_6$ from Lemma 5.4, so that Theorem 4.1 applies directly to policy gradient updates.

What would settle it

For a concrete LQR instance, such as $A=0$, $B=Q=R=1$ where $J_2(K)=(K^2+1)/(2K)$ on $K>0$, evaluate whether $\|\nabla J_2(K)\|_F \ge (J_2(K)-J_2(K^*))/(b_1(J_2(K)-J_2(K^*))+b_2)$ holds for all stabilizing gains with the constants $b_1,b_2$ of equation (55); a single violating gain would invalidate Theorem 5.1.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4.1: if the objective function J is coercive, its gradient is Lipschitz on sublevel sets, and it satisfies the K-PL estimate $\|\nabla J(z)\| \ge \alpha_5(J(z)-J(z^*))$ for a class-K function $\alpha_5$, then the perturbed gradient descent $z(k+1)=z(k)-\eta(k)(\nabla J(z(k))+e(k))$ is small-disturbance ISS whenever $0<\eta(k)\le 1/L(J(z(k)))$. The ultimate bound on the cost gap is $\alpha_5^{-1}(2\|e\|_\infty)$, a nonlinear function of the disturbance amplitude. The paper then verifies the required properties for the LQR cost $J_2(K)$, including a global K-PL estimate with the saturating function $\alpha_6(r)=r/(b_1 r+b_2)$, and thereby proves that the standard policy gradient, natural gradient, and Gauss-Newton methods for LQR are small-disturbance ISS.

Load-bearing premise

The crucial premise is that the LQR cost's gradient never becomes too small compared with how far the current cost is from optimal, for every stabilizing controller; the paper cites this global K-PL estimate from a companion paper without proving it here.

Editorial extensions

If this is right

  • Bounded gradient noise from round-off, sampling, or estimation errors no longer causes divergence; the iterates eventually stay inside the set $\{z : J(z)-J(z^*) \le \alpha_5^{-1}(2\|e\|_\infty)\}$, whose size is controlled by the noise bound.
  • For LQR policy optimization, model-free implementations with finite-difference or adaptive-dynamic-programming gradient estimators inherit a quantitative robustness guarantee against estimation errors.
  • The natural gradient and Gauss-Newton updates for LQR are also small-disturbance ISS under explicit step-size constraints, so the robustness conclusion is not tied to one update rule.
  • If the K-PL function is strengthened to class K-infinity, the gradient descent is ISS; if relaxed to a positive definite function, it is integral ISS, with the classical linear-PL case recovering exponential ISS as a special case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same Lyapunov-template argument could be attempted for other policy-gradient variants such as TRPO or PPO, provided an analogous K-PL estimate is established; the paper does not attempt this extension.
  • The saturating K-PL function $\alpha_6(r)=r/(b_1 r+b_2)$ implies that the steady-state LQR cost error grows sublinearly with the gradient-noise amplitude, a quantitative prediction that numerical simulations could verify.
  • The LQR robustness theorems depend on Lemma 5.4, whose proof is deferred to a companion paper; a self-contained proof would make the LQR claims independently verifiable from this manuscript alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper develops a discrete-time small-disturbance ISS framework with a Lyapunov characterization (Theorem 3.1), generalizes the linear Polyak–Łojasiewicz condition to a K-PL condition, and claims that gradient descent with step sizes 0<η(k)≤1/L(J(z(k))) is small-disturbance ISS under coercivity, sublevel-set Lipschitz gradients, and K-PL (Theorem 4.1). It then applies this result to the LQR cost, claiming that standard policy gradient, natural policy gradient, and Gauss-Newton methods are small-disturbance ISS (Theorems 5.1–5.3).

Significance. If the main theorem were correct, the proposed K-PL framework would provide a useful robustness guarantee for nonconvex gradient methods and would give a clean control-theoretic interpretation of perturbed policy optimization for LQR. The discrete-time Lyapunov characterization in Theorem 3.1 is a valuable contribution in itself. However, the central step-size condition in Theorem 4.1 is missing a uniform lower bound, and the theorem is false as stated; the same defect propagates to the LQR theorems. The LQR results also rely on a global K-PL estimate imported without proof from a same-author paper. With a corrected statement and a complete proof of the imported lemma, the framework could be significant, but the current version is not publishable in this form.

major comments (3)
  1. [Theorem 4.1, Eq. (27)] Theorem 4.1 is false as stated. The proof obtains the estimate J(z(k+1))-J(z(k)) ≤ -(3η(k)/8)α5(J(z(k))-J(z*))² in Eq. (27), but the hypothesis 0<η(k)≤1/L(J(z(k))) imposes no lower bound on η(k), so the decay rate can vanish along the trajectory. This prevents application of Theorem 3.1, which requires a K∞ function α2 in Definition 3.3 independent of k. A concrete counterexample is J(z)=z² on Z=R with L=2, α5(r)=2√r, and η(k)=2^{-(k+2)}; all hypotheses of Definition 4.1 hold, but with e≡0 the iteration is z(k+1)=(1-2^{-(k+1)})z(k), which converges to a nonzero multiple of z(0) whenever z(0)≠0. This violates the 0-input asymptotic stability required by Definition 3.2. The theorem must be repaired by adding a uniform lower bound on the step size, such as η(k)≥η_min>0, and the same correction must be propagated through Corollaries 4.1–4.2 and Theorems 5.1–5.3.
  2. [Lemma 5.4, Eq. (55)] The global K-PL estimate for the LQR cost, ∥∇J2(K)∥F ≥ r/(b1r+b2) with r=J2(K)-J2(K*), is stated without proof and cited as Lemma 5.7 of [12], a paper by the same first three authors. This estimate is load-bearing: without it, Theorems 5.1–5.3 do not follow from the arguments presented. The authors should include a complete proof or at least a detailed derivation of the constants b1 and b2 so that the reader can verify the estimate independently, rather than relying on a same-author citation for a central ingredient.
  3. [Theorems 5.2 and 5.3] The natural-gradient and Gauss-Newton theorems inherit the step-size lower-bound problem. In Theorem 5.2, inequality (80) gives V5(K(k+1))-V5(K(k)) ≤ -η(k)λmin(R)V5(K(k))/4, and the stated condition 0<η(k)≤min{1/(2∥R∥),1/(6∥R∥c(K(k)))} has no lower bound. Hence V5 is not a small-disturbance ISS-Lyapunov function under Definition 3.3 unless η(k) is uniformly bounded below. Theorem 5.3 has the same issue with the condition 0<η(k)≤min{1,1/(4c(K(k)))}, and its proof is only a one-sentence sketch. These theorems need the corrected step-size hypothesis and a fully written proof for the Gauss-Newton case.
minor comments (4)
  1. [Eq. (54)] The displayed update for standard gradient descent appears to be missing the K(k) term: it should read K(k+1)=K(k)-η(k)(∇J2(K(k))+W(k)), and the argument of ∇J2 is written as k(k) instead of K(k).
  2. [Theorem 5.1 and its proof] The step-size condition is written as 0<η(k)≤1/L(J(K(k))), but L is defined for J2 in Lemma 5.3; the notation should be L(J2(K(k))) for consistency.
  3. [Abstract and Introduction] The name Łojasiewicz is typeset with a spurious space in the abstract; this should be corrected throughout for consistency with the reference list.
  4. [Example 4.1] The K-PL estimate is displayed as ∥∇J(z)∥ ≥ J(z)/(√2/2+2J(z)); it would be helpful to show the intermediate algebra, as the expression is not immediately transparent and is used to illustrate the framework.

Circularity Check

1 steps flagged · score 3.0 of 10

Core Theorem 4.1 is an independent derivation from explicit assumptions; the LQR application imports its global K-PL estimate from the authors' prior paper, creating a load-bearing self-citation but not a definitional tautology.

  1. self citation load bearing [Lemma 5.4 (Lemma 5.7 in [12]) and the proof of Theorem 5.1]
    "Lemma 5.4 (Lemma 5.7 in [12]) The objective function J2(K) satisfies the K-PL condition, that is ∥∇J2(K)∥F ≥ α6(J2(K) − J2(K ∗)), ∀K ∈ G, where α6(r) = r/(b1r + b2). The following lemma is introduced to ensure that J2(K) satisfy the K-PL condition in Definition 4.1, which is critical to the robustness analysis. PROOF of Theorem 5.1: ... The coercivity of J2(K) is demonstrated in [10, Lemma 3.3]. The L(h)-Lipschitz continuity and the K-PL condition are established in Lemmas 5.3 and 5.4, respectively. Consequently, by Theorem 4.1, the gradient descent algorithm in (54) is small-disturbance ISS."

    The global K-PL estimate that makes the standard-gradient LQR theorem work is not derived in this manuscript; it is imported verbatim as Lemma 5.4 from [12], a paper by the first three authors of the present work. Theorem 5.1 then simply composes that imported estimate with Lemma 5.3 and Theorem 4.1. Thus the advertised demonstration that the LQR cost satisfies the generalized nonlinear PL condition reduces, at the load-bearing step, to a same-author citation rather than to a proof given here. This is reliance on prior self-citation rather than a fitted-parameter or definitional tautology; it raises the circularity burden moderately without making the derivation equivalent to its input by construction.

full rationale

Theorem 4.1 is self-contained: it states explicit hypotheses (coercivity, sublevel Lipschitz gradient, K-PL estimate) and proves small-disturbance ISS from inequalities (21)-(27) and Theorem 3.1; nothing is fitted and no named 'prediction' is used as an input. The natural-gradient proof (Theorem 5.2) and Gauss-Newton proof (Theorem 5.3) are also carried out directly by constructing Lyapunov functions, without invoking Lemma 5.4. The only circularity-adjacent step is the LQR application in Theorem 5.1, whose K-PL premise is Lemma 5.4 from the authors' own prior paper, with no proof reproduced here. Because this is citation reliance on a peer-reviewed earlier theorem rather than an equation that is equivalent to its own output by definition, I do not classify it as full circularity. A separate mathematical concern about the missing lower bound on step sizes is a correctness issue, not a circularity issue, and is left out of the score.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No free parameters or invented physical entities. The paper introduces mathematical definitions (small-disturbance ISS-Lyapunov function, K-PL condition) but no fitted numbers or new postulates beyond the hypotheses listed above.

assumptions (5)
  • domain assumption J is proper: J-J* is a size function, ∇J is Lipschitz on sublevel sets Z(h), and ∥∇J(z)∥ ≥ α5(J(z)-J(z*)) for a K-function α5
    Definition 4.1 is the standing hypothesis of Theorem 4.1; the paper does not prove these properties for a general J.
  • domain assumption Z is an open subset of Rn homeomorphic to Rn
    Used in Definition 4.1 and Theorem 3.1 to transfer ISS results; for LQR, G is homeomorphic to R^{m×n} via [9,17].
  • domain assumption The LQR cost J2 satisfies the K-PL estimate with α6(r)=r/(b1r+b2)
    Lemma 5.4, cited from Lemma 5.7 of [12] with no proof in this paper; it is the load-bearing bridge for Theorems 5.1-5.3.
  • standard math Discrete-time ISS converse Lyapunov results [25,26] are valid for the small-disturbance variant
    Theorem 3.1's sufficiency and necessity proofs adapt [25, Theorem 1] and [26]; the paper relies on these unproved background theorems.
  • standard math Comparison lemma for discrete-time KL estimates
    Used inside Theorem 3.1 proof via [25, Lemma 4.3].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Perturbed Gradient Descent Algorithms are Small-Disturbance Input-to-State Stable." pith.science (2026). https://pith.science/paper/XJLIXQHZ

@misc{pith2026250702131,
  author       = {Pith},
  title        = {Pith review of: Perturbed Gradient Descent Algorithms are Small-Disturbance Input-to-State Stable},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XJLIXQHZ}},
  note         = {Machine review of arXiv:2507.02131}
}
read the original abstract

This article investigates the robustness of gradient descent algorithms under perturbations. The concept of small-disturbance input-to-state stability (ISS) for discrete-time nonlinear dynamical systems is introduced, along with its Lyapunov characterization. The conventional linear Polyak-Lojasiewicz (PL) condition is then extended to a nonlinear version, and it is shown that the gradient descent algorithm is small-disturbance ISS provided the objective function satisfies the generalized nonlinear PL condition. This small-disturbance ISS property guarantees that the gradient descent algorithm converges to a small neighborhood of the optimum under sufficiently small perturbations. As a direct application of the developed framework, we demonstrate that the LQR cost satisfies the generalized nonlinear PL condition, thereby establishing that the policy gradient algorithm for LQR is small-disturbance ISS. Additionally, other popular policy gradient algorithms, including natural policy gradient and Gauss-Newton method, are also proven to be small-disturbance ISS.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 57 canonical work pages

  1. [12]

    Leilei Cui, Zhong Ping Jiang, and Eduardo D. Sontag. Small- disturbance input-to-state stability of perturbed gradient flows: Applications to LQR problem. Systems & Control Letters, 188:105804, 2024

    work page 2024

  2. [1]

    Natural gradient works efficiently in learning

    Shun-Ichi Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251–276, 1998

    work page 1998

  3. [2]

    Why natural gradient? In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , volume 2, pages 1213–1216, 1998

    Shun-Ichi Amari and Scott C Douglas. Why natural gradient? In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing , volume 2, pages 1213–1216, 1998

    work page 1998

  4. [3]

    Intrinsic robustness of global asymptotic stability

    David Angeli. Intrinsic robustness of global asymptotic stability. Systems & Control Letters , 38(4-5):297–307, 1999

    work page 1999

  5. [4]

    Robust accelerated gradient methods for smooth strongly convex functions

    Necdet Serhat Aybat, Alireza Fallah, Mert Gurbuzbalaban, and Asuman Ozdaglar. Robust accelerated gradient methods for smooth strongly convex functions. SIAM Journal on Optimization, 30(1):717–751, 2020

    work page 2020

  6. [5]

    Nonlinear Programming

    Dimitri P Bertsekas. Nonlinear Programming . Athena Scientific, New Hampshire, USA, 1999

    work page 1999

  7. [6]

    Bertsekas and John N

    Dimitri P. Bertsekas and John N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, New Hampshire, 1996

    work page 1996

  8. [7]

    Bertsekas and John N

    Dimitri P. Bertsekas and John N. Tsitsiklis. Gradient convergence in gradient methods with errors. SIAM Journal on Optimization , 10(3):627–642, 2000. 14

    work page 2000

Show all 62 references
  1. [8]

    Poveda, and Emiliano Dall’Anese

    Gianluca Bianchin, Jorge I. Poveda, and Emiliano Dall’Anese. Online optimization of switched LTI systems using continuous-time and hybrid accelerated gradient flows. Automatica, 146:110579, 2022

    2022

  2. [9]

    On topological and metrical properties of stabilizing feedback gains: the MIMO case

    Jingjing Bu, Afshin Mesbahi, and Mehran Mesbahi. On topological and metrical properties of stabilizing feedback gains: the MIMO case. arXiv preprint arXiv:1904.02737 , 2019

    1904 arXiv

  3. [10]

    Policy gradient-based algorithms for continuous-time linear quadratic control

    Jingjing Bu, Afshin Mesbahi, and Mehran Mesbahi. Policy gradient-based algorithms for continuous-time linear quadratic control. arXiv preprint arXiv:2006.09178 , 2020

    2006 arXiv

  4. [11]

    The role of convexity in saddle-point dynamics: Lyapunov function and robustness

    Ashish Cherukuri, Enrique Mallada, Steven Low, and Jorge Cort´ es. The role of convexity in saddle-point dynamics: Lyapunov function and robustness. IEEE Transactions on Automatic Control, 63(8):2449–2464, 2018

    2018

  5. [13]

    Input-to-state stability of a bilevel proximal gradient descent algorithm

    Torbjørn Cunis and Ilya Kolmanovsky. Input-to-state stability of a bilevel proximal gradient descent algorithm. IF AC-PapersOnLine, 56(2):7474–7479, 2023

    2023

  6. [14]

    A robust accelerated optimization algorithm for strongly convex functions

    Saman Cyrus, Bin Hu, Bryan Van Scoy, and Laurent Lessard. A robust accelerated optimization algorithm for strongly convex functions. In American Control Conference , pages 1376–1381, 2018

    2018

  7. [15]

    First- order methods of smooth convex optimization with inexact oracle

    Olivier Devolder, Fran¸ cois Glineur, and Yurii Nesterov. First- order methods of smooth convex optimization with inexact oracle. Mathematical Programming, 146:37–75, 2014

    2014

  8. [16]

    Global convergence of policy gradient methods for the linear quadratic regulator

    Maryam Fazel, Rong Ge, Sham Kakade, and Mehran Mesbahi. Global convergence of policy gradient methods for the linear quadratic regulator. In International Conference on Machine Learning , pages 1467–1476, 2018

    2018

  9. [17]

    Convex open subsets of Rn are homeomorphic to n-dimensional open balls

    Stefan Geschke. Convex open subsets of Rn are homeomorphic to n-dimensional open balls. http://relaunch.hcm.uni- bonn.de/fileadmin/geschke/papers/ConvexOpen.pdf, 2012

    2012

  10. [18]

    On a Newton-like method for solving algebraic Riccati equations

    Chun-Hua Guo and Alan J Laub. On a Newton-like method for solving algebraic Riccati equations. SIAM Journal on Matrix Analysis and Applications , 21(2):694–698, 2000

    2000

  11. [19]

    Stability of Motion

    Wolfgang Hahn. Stability of Motion . Springer Berlin, Heidelberg, 1967

    1967

  12. [20]

    Dissipativity theory for Nesterov’s accelerated method

    Bin Hu and Laurent Lessard. Dissipativity theory for Nesterov’s accelerated method. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research, pages 1549–1557. PMLR, 0...

    2017

  13. [21]

    Toward a theoretical foundation of policy optimization for learning control policies

    Bin Hu, Kaiqing Zhang, Na Li, Mehran Mesbahi, Maryam Fazel, and Tamer Ba¸ sar. Toward a theoretical foundation of policy optimization for learning control policies. Annual Review of Control, Robotics, and Autonomous Systems , 6(1):123–158, 2023

    2023

  14. [22]

    Robust Adaptive Dynamic Programming

    Yu Jiang and Zhong Ping Jiang. Robust Adaptive Dynamic Programming. Wiley-IEEE Press, Hoboken, New Jersey, 2017

    2017

  15. [23]

    Learning- based control: A tutorial and some recent results

    Zhong Ping Jiang, Tao Bian, and Weinan Gao. Learning- based control: A tutorial and some recent results. Foundations and Trends® in Systems and Control, 8(3):176– 284, 2020

    2020

  16. [24]

    Teel, and Laurent Praly

    Zhong Ping Jiang, Andrew R. Teel, and Laurent Praly. Small- gain theorem for ISS systems and applications. Mathematics of Control, Signals and Systems , 7:95–120, 1994

    1994

  17. [25]

    Input-to-state stability for discrete-time nonlinear systems

    Zhong Ping Jiang and Yuan Wang. Input-to-state stability for discrete-time nonlinear systems. Automatica, 37(6):857– 869, 2001

    2001

  18. [26]

    A converse Lyapunov theorem for discrete-time systems with disturbances.Systems & Control Letters , 45(1):49–58, 2002

    Zhong Ping Jiang and Yuan Wang. A converse Lyapunov theorem for discrete-time systems with disturbances.Systems & Control Letters , 45(1):49–58, 2002

    2002

  19. [27]

    Dynamical, symplectic and stochastic perspectives on gradient-based optimization

    Michael I Jordan. Dynamical, symplectic and stochastic perspectives on gradient-based optimization. In Proceedings of the International Congress of Mathematicians: Rio de Janeiro 2018, pages 523–549. World Scientific, 2018

    2018

  20. [28]

    Contributions to the theory of optimal control

    Rudolf Emil Kalman. Contributions to the theory of optimal control. Boletinde la Sociedad Matematica Mexicana, 5(2):102–119, 1960

    1960

  21. [29]

    Linear convergence of gradient and proximal-gradient methods under the Polyak- Lojasiewicz condition

    Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the Polyak- Lojasiewicz condition. In Paolo Frasconi, Niels Landwehr, Giuseppe Manco, and Jilles Vreeken, editors, Machine Learning and Knowledge Discovery in Datab...

    2016

  22. [30]

    Adam: A method for stochastic optimization

    Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  23. [31]

    Kleinman

    David L. Kleinman. On an iterative technique for Riccati equation computations. IEEE Transactions on Automatic Control, 13(1):114–115, 1968

    1968

  24. [32]

    Actor-critic algorithms

    Vijay Konda and John Tsitsiklis. Actor-critic algorithms. In Advances in Neural Information Processing Systems , volume 12, pages 1008–1014, 1999

    1999

  25. [33]

    Analysis and design of optimization algorithms via integral quadratic constraints

    Laurent Lessard, Benjamin Recht, and Andrew Packard. Analysis and design of optimization algorithms via integral quadratic constraints. SIAM Journal on Optimization , 26(1):57–95, 2016

    2016

  26. [34]

    Levine and Michael Athans

    William S. Levine and Michael Athans. On the determination of the optimal constant output feedback gains for linear multivariable systems. IEEE Transactions on Automatic Control, 15(1):44–48, 1970

    1970

  27. [35]

    Distributed reinforcement learning for decentralized linear quadratic control: A derivative-free policy optimization approach

    Yingying Li, Yujie Tang, Runyu Zhang, and Na Li. Distributed reinforcement learning for decentralized linear quadratic control: A derivative-free policy optimization approach. IEEE Transactions on Automatic Control , 67(12):6429–6444, 2022

    2022

  28. [36]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 , 2015

    2015 arXiv

  29. [37]

    A topological property of real analytic subsets (in French)

    Stanis law Lojasiewicz. A topological property of real analytic subsets (in French). Colloques internationaux du C.N.R.S 117, Les ´Equations aux D´ eriv´ ees Partielles, page 87–89, 1963

    1963

  30. [38]

    Computational methods for parametric LQ problems–A survey

    Perttim Makila and Hannut Toivonen. Computational methods for parametric LQ problems–A survey. IEEE Transactions on Automatic Control, 32(8):658–671, 1987

    1987

  31. [39]

    Jovanovi´ c

    Hesameddin Mohammadi, Meisam Razaviyayn, and Mihailo R. Jovanovi´ c. Tradeoffs between convergence rate and noise amplification for momentum-based accelerated optimization algorithms. IEEE Transactions on Automatic Control, 70(2):889–904, 2025

    2025

  32. [40]

    Jovanovi´ c

    Hesameddin Mohammadi, Armin Zare, Mahdi Soltanolkotabi, and Mihailo R. Jovanovi´ c. Convergence and sample complexity of gradient methods for the model-free linear–quadratic regulator problem. IEEE Transactions on Automatic Control, 67(5):2435–2450, 2022. 15

    2022

  33. [41]

    A systematic approach to Lyapunov analyses of continuous- time models in convex optimization

    C´ eline Moucer, Adrien Taylor, and Francis Bach. A systematic approach to Lyapunov analyses of continuous- time models in convex optimization. SIAM Journal on Optimization, 33(3):1558–1586, 2023

    2023

  34. [42]

    Introductory Lectures on Convex Optimization: A Basic Course , volume 87

    Yurii Nesterov. Introductory Lectures on Convex Optimization: A Basic Course , volume 87. Springer Science & Business Media, New York, NY, 2013

    2013

  35. [43]

    Jorge Nocedal and Stephen J. Wright. Numerical Optimization. Springer, New York, 2006

    2006

  36. [44]

    Robust policy iteration for continuous-time linear quadratic regulation

    Bo Pang, Tao Bian, and Zhong Ping Jiang. Robust policy iteration for continuous-time linear quadratic regulation. IEEE Transactions on Automatic Control , 67(1):504–511, 2022

    2022

  37. [45]

    Robust reinforcement learning: A case study in linear quadratic regulation

    Bo Pang and Zhong Ping Jiang. Robust reinforcement learning: A case study in linear quadratic regulation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10):9303–9311, May 2021

    2021

  38. [46]

    The Matrix Cookbook, October 2008

    Kaare Brandt Petersen and Michael Syskind Pedersen. The Matrix Cookbook, October 2008. Version 20081110

    2008

  39. [47]

    Boris T. Polyak. Gradient methods for the minimisation of functionals. USSR Computational Mathematics and Mathematical Physics, 3(4):864–878, 1963

    1963

  40. [48]

    Boris T. Polyak. Introduction to Optimization. Optimization Software Inc., New York, USA, 1987

    1987

  41. [49]

    Poveda and Miroslav Krsti´ c

    Jorge I. Poveda and Miroslav Krsti´ c. Nonsmooth extremum seeking control with user-prescribed fixed-time convergence. IEEE Transactions on Automatic Control, 66(12):6156–6163, 2021

    2021

  42. [50]

    Trust region policy optimization

    John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. Trust region policy optimization. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37, pages 1889–1897, Lille, France, 07–09 Jul 2015

    2015

  43. [51]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017

    2017 arXiv

  44. [52]

    Error stability properties of generalized gradient-type algorithms

    Mikhail V Solodov and SK Zavriev. Error stability properties of generalized gradient-type algorithms. Journal of Optimization Theory and Applications , 98:663–680, 1998

    1998

  45. [53]

    Eduardo D. Sontag. Smooth stabilization implies coprime factorization. IEEE Transactions on Automatic Control , 34(4):435–443, 1989

    1989

  46. [54]

    Eduardo D. Sontag. Mathematical Control Theory: Deterministic Finite Dimensional Systems , volume 6. Springer Science & Business Media, New York, NY, 1998

    1998

  47. [55]

    Eduardo D. Sontag. Input to state stability: Basic concepts and results, pages 163–220. Lecture Notes in Mathematics. Springer Verlag, Germany, 2008

    2008

  48. [56]

    Eduardo D. Sontag. Remarks on input to state stability of perturbed gradient flows, motivated by model-free feedback control learning. Systems & Control Letters , 161:105138, 2022

    2022

  49. [57]

    Robustness and averaging properties of a large-amplitude, high-frequency extremum seeking control scheme

    Raik Suttner and Sergey Dashkovskiy. Robustness and averaging properties of a large-amplitude, high-frequency extremum seeking control scheme. Automatica, 136:110020, 2022

    2022

  50. [58]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT press, Cambridge, MA, 2nd edition, 2018

    2018

  51. [59]

    Trace bounds on the solution of the algebraic matrix Riccati and Lyapunov equation

    Sheng-De Wang, Te-Son Kuo, and Chen-Fa Hsu. Trace bounds on the solution of the algebraic matrix Riccati and Lyapunov equation. IEEE Transactions on Automatic Control, 31(7):654–656, 1986

    1986

  52. [60]

    Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229–256, 1992

    1992

  53. [61]

    Wilson, Ben Recht, and Michael I

    Ashia C. Wilson, Ben Recht, and Michael I. Jordan. A Lyapunov analysis of accelerated methods in optimization. Journal of Machine Learning Research , 22(113):1–34, 2021

    2021

  54. [62]

    Wesley Wilson

    F. Wesley Wilson. The structure of the level surfaces of a Lyapunov function. Journal of Differential Equations , 3(3):323–329, 1967. 16

    1967

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.