Pith. sign in

REVIEW 2 major objections 5 minor 27 references

This paper proves that replacing the square-root energy in AEGD with any smooth, strictly increasing, concave energy function preserves unconditional stability and yields O(1/ε) convergence to stationary points for any base step size, provi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-04 06:29 UTC pith:MXM2KQ2J

load-bearing objection Solid extension of AEGD to general concave energy functions, with a clean main result whose key condition is self-referential but easily repairable. the 2 major comments →

arxiv 2512.13537 v2 pith:MXM2KQ2J submitted 2025-12-15 math.OC

A Generalized Energy-Based Adaptive Gradient Method for Optimization

Lin Feng , Hailiang Liu This is my paper
classification math.OC MSC 90C3090C2665K05
keywords adaptive gradient methodsenergy stabilitystep-size adaptationnonconvex optimizationconvergence analysisKurdyka–Łojasiewicz inequalityAEGDlogarithmic energy
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Adaptive Gradient Descent with Energy (AEGD) tunes its step size through an auxiliary energy parameter, but only for a square-root energy function. This paper shows that any smooth, strictly increasing, concave energy function yields a generalized method, gAEGD, with the same unconditional energy stability: the energy parameter decreases for every base step size. With the shift parameter c chosen large enough, the method reaches an ε-stationary point in O(1/ε) iterations, matching the optimal first-order rate, and needs no step-size tuning. Under a local Kurdyka–Łojasiewicz condition, the paper also gives explicit convergence rates for the objective gap, and experiments show a logarithmic-energy variant often beats the original AEGD.

Core claim

The paper's central claim is that the square-root energy in AEGD is not essential: any smooth, strictly increasing, concave transform of f+c inherits the same guarantees. The argument turns on an exact identity that makes the energy parameter r_k monotonically decrease for every η>0, a gap estimate that keeps the transformed value F_k close to r_k, and a two-stage lemma: after finitely many iterations, either the gradient is already small or r_k has dropped so low that the effective step size η_k is trapped between ηr_*/\bar F and 1/L. Once that trap holds, the descent inequality yields min_{k≤N} ||∇f_k||^2 < ε in O(1/ε) iterations for any η>0 and r_0>0, provided c≥max{c*, c̄}. Under a local

What carries the argument

The engine is the scalar energy parameter r_k, updated as r_{k+1}=r_k/(1+ηF'_k/F_k ||∇f_k||^2), which makes the effective step size η_k=η r_{k+1}/F_k adapt automatically. The identity r_{k+1}^2 = r_k^2 - (r_{k+1}-r_k)^2 - (2/η)F_kF'_k||x_{k+1}-x_k||^2 is the load-bearing mechanism: it forces r_k to drop monotonically for any η>0, giving unconditional energy stability. To turn stability into convergence, the paper must stop r_k from collapsing to zero; that is precisely what the two c-conditions do. Lemma 3.5 (explicit c*) and the c̄ condition in Theorem 3.6 guarantee a positive lower bound r_*>0, which keeps the effective step size bounded away from zero and below 1/L in the stable phase.

Load-bearing premise

The rate guarantee stands only if c is large enough to satisfy c≥\hat F^{-1}(Lηr_*)-f_0, but r_* is the algorithm's own limiting energy, which depends on c; so a user cannot verify the condition before running the method, and the paper gives no explicit lower bound using only r_0 for this theorem.

What would settle it

To test whether the rate is real, run gAEGD with a concave energy such as log(s+1) on the 2D Rosenbrock function, fix a large base step size (e.g., η=100), choose c by the paper's explicit KL-style formula \hat F^{-1}(Lηr_0)-f_0, and record min_{k≤N}||∇f_k||^2 for N up to several times 1/ε. If for some ε the bound is not attained, the theorem's claimed O(1/ε) is false for that explicit choice; if it is attained, that supports the practical message. More directly, one can search numerically for a smooth function, η, r_0, and c satisfying the theorem's stated c≥c̄ condition (computing r_* from a

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • For any smooth nonconvex problem with L-Lipschitz gradients, a sufficiently large c turns gAEGD into a step-size-free first-order method with the optimal O(1/ε) worst-case guarantee.
  • The two-stage behavior provides a diagnostic: once r_k falls below F_*/Lη, the effective step size is guaranteed to stay below 1/L and the objective values begin to decrease monotonically.
  • Under the local KL condition, gAEGD inherits the standard rate trichotomy (linear, finite, or polynomial), so the energy framework is compatible with one of the broadest structural conditions used in nonconvex optimization.
  • The logarithmic energy variant ALEGD is an admissible instance and, in the paper's experiments, reaches the target accuracy in fewer iterations than AEGD on quadratic and Rosenbrock problems, including with large condition numbers.
  • Raising c increases the best base step size, so c plays a role analogous to a stability margin; the paper documents this trade-off numerically.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The main theorem's condition c≥c̄ involves the limit r_*, which is not known ahead of time and depends on c. A user who wants a verifiable guarantee can substitute the explicit stronger bound c≥\hat F^{-1}(Lηr_0)-f_0 (the style used in the KL section), and the proof's logic still carries through; the paper does not point this out.
  • The monotonicity of r_k suggests a built-in early-stopping signal that the paper does not discuss: if r_k has not yet dropped below its stable threshold while gradient norms remain large, the algorithm is still in its adaptive phase, so more iterations rather than a restart are advised.
  • A testable extension is to select the energy function online from a small parametric family (e.g., power exponents p) based on measured step-size behavior; the experiments' finding that lower p allows larger base steps in convex problems hints that the best energy choice is problem-dependent.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes Generalized Energy-Based Adaptive Gradient Descent (gAEGD), replacing the square-root energy of AEGD by any smooth, strictly increasing, concave function \hat F. The main theoretical results are: an unconditional energy-stability identity (2.3); a gap estimate controlling F_k - r_k (2.5); a two-stage adaptive step-size lemma (Lemma 3.2); a non-asymptotic O(1/ε) bound for finding ε-stationary points under a large-c condition (Theorem 3.6); and convergence rates under a local KL condition (Theorem 4.1). Experiments on a 100D quadratic and 2D Rosenbrock show that a logarithmic-energy variant (ALEGD) often outperforms the original AEGD.

Significance. If correct, the paper yields a broad family of adaptive first-order methods with unconditional energy stability and optimal worst-case complexity, with no step-size tuning once c is chosen sufficiently large. The energy identity and two-stage analysis are elegant and extend the authors' earlier AEGD framework in a natural way. The proofs are self-contained and the empirical work, though small, supports the main qualitative claims. However, two technical issues must be fixed before the results are fully sound: the main convergence theorem's hypothesis on c is self-referential, and the lemma guaranteeing r_*>0 is not valid as stated for the paper's flagship logarithmic energy.

major comments (2)
  1. [Section 3, Theorem 3.6] The hypothesis c ≥ max{c_*, \bar c} uses \bar c := \hat F^{-1}(Lηr_*) - f_0, where r_* = lim r_k is the limit of the algorithm's own trajectory and therefore depends on c itself. This condition cannot be verified from problem data before execution, so the stated O(1/ε) guarantee is not constructive. Moreover, inspection of the proof shows that \bar c is actually unnecessary: once Lemma 3.2 yields k_0 with r_{k_0} ≤ F_*/Lη, the monotonicity of r_k and the bound F_k ≥ F_* give η_k = η r_{k+1}/F_k ≤ η r_k/F_* ≤ 1/L for all k ≥ k_0 without any extra condition on c. Thus the theorem can be strengthened by either deleting \bar c or replacing it by the explicit \tilde c = \hat F^{-1}(Lηr_0) - f_0. As written, the central complexity result is conditional on an unobservable quantity.
  2. [Section 3, Lemma 3.5] The definition c_* = min_{a∈(0,1)} max{ \hat F'^{-1}(a r_0/(f_0-f_*)) - f_*, \hat F^{-1}(Lηr_0/(4(1-a))) - f_* } implicitly assumes \hat F'^{-1} is defined for all arguments in (0, r_0/(f_0-f_*)). For \hat F(s)=log(s+1), which is the ALEGD variant emphasized in the paper, \hat F'(0+)=1, so the inverse is defined only on (0,1]. If r_0/(f_0-f_*) > 1, the first term is undefined for a > (f_0-f_*)/r_0, and the minimization over (0,1) is not well-posed. Since Theorems 3.6 and 4.1 rely on Lemma 3.5 to guarantee r_* > 0, the theoretical guarantees as stated do not cover ALEGD. This is repairable by restricting the interval to (0, min{1, (f_0-f_*)\hat F'(0+)/r_0}) and adjusting the endpoint argument, or by giving a direct existence proof using \lim_{s→∞}\hat F'(s)=0, but the lemma must be corrected.
minor comments (5)
  1. [Section 3, Lemma 3.2 proof] The line 'provided N ≥ ln(r0/C)/ln(1+ηε)' is missing the factor α in the denominator; it should be ln(1+αηε).
  2. [Section 3, Theorem 3.6 proof] The case split 'Case 1: k_0 = 1' is slightly inaccurate: if r_0 ≤ F_*/Lη then the smallest index is k_0=0. The proof should either start the summation at j=0 or explicitly set k_0=1 in that case.
  3. [Section 4, Theorem 4.1 case α=1] The derivation w_{k+1} ≤ (1-Q)w_k assumes Q<1. If Q ≥ 1, the inequality implies immediate finite termination; this case is not discussed. The conclusion remains correct but the proof should be patched.
  4. [Section 4, proof of Theorem 4.1] The limit point \tilde x is identified before the Cauchy property of {x_k} is established. Reorganize so that the KL argument first proves convergence of x_k, then identifies the limit.
  5. [Throughout] There are numerous typos and misspellings (e.g., 'fucntion', 'preforms', 'uisng', 'initiliazaiton', 'optmization', 'predicated', 'decent' for 'descent'). A careful proofread is recommended.

Circularity Check

1 steps flagged

Theorem 3.6's sufficient condition on c is defined through the algorithm's own limiting energy r_*, making the O(1/epsilon) guarantee non-constructive; an explicit threshold replacing r_* by r0 repairs it.

specific steps
  1. self definitional [Theorem 3.6, Section 3 (after Lemma 3.5), with r_* defined by Theorem 2.1]
    "if the parameter c is chosen such that c≥max{c∗, ¯c}, where c∗ is given in Lemma 3.5 and ¯c:= Fhat^{-1}(Lηr∗)−f_0, then for any ε>0, the gradient norm satisfies min_{k≤N} ||∇f_k||^2 < ε ..."

    The threshold ¯c is defined using r_*, the limit of r_k. But r_k is produced by update (2.1) and depends on c, so c≥¯c is a condition on the algorithm's own trajectory, not a verifiable condition on problem data. The proof uses c≥¯c only to infer Lηr_* ≤ F_0, hence η_k ≤ 1/L. Since r_* ≤ r_0 and Fhat is monotone, the explicit threshold Fhat^{-1}(Lη r_0) - f_0 (or f_*) would imply the same; the paper itself uses the explicit threshold in Theorem 4.1. Thus Theorem 3.6's main guarantee is self-referential as stated, though repairable.

full rationale

The core derivation is self-contained: the energy-stability identity (2.3), the gap estimate (2.5), and the two-stage Lemma 3.2 are proved from the update rule and standing assumptions, not imported from prior work. The self-citations to Liu-Tian are background and not load-bearing. The only genuine circularity is in Theorem 3.6: its sufficient condition c≥¯c is defined through r_*=lim r_k, which is an output of the algorithm and depends on c; this makes the O(1/ε) statement conditional on an unverifiable fixed-point property. Because r_* ≤ r_0 and Fhat is monotone, replacing ¯c by the explicit Fhat^{-1}(Lη r_0)-f_0 removes the self-reference while preserving the proof, so the flaw is localized and non-fatal. Lemma 3.5's c_* and the KL analysis are otherwise independent and constructive. Score 3 reflects one load-bearing self-referential hypothesis, not a fit-to-data or imported-uniqueness circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

No new physical or computational entities are introduced. The central claim rests on standard smoothness assumptions, a class of admissible energy functions, and the KL condition; the only hand-chosen parameter is c (with η tuned in experiments).

free parameters (2)
  • c = tuned in experiments (e.g., 1, 10, 100, 1000)
    c shifts the energy function's argument; the theory requires c ≥ thresholds involving η, r0, and (for Thm 3.6) the unknown limit r_*, and the experiments tune c per problem (Tables 2-3).
  • η = tuned per algorithm and problem ('lr' in Tables 2-3)
    Base step size; the theory claims robustness to η for large c, but the empirical comparisons use individually tuned η values.
axioms (4)
  • domain assumption f ∈ C^{1,1}_L(R^d) and bounded below by f_*
    Invoked throughout (Sec 3) to use the standard smoothness inequality; the entire convergence analysis depends on this.
  • domain assumption Fhat is smooth, strictly increasing, concave, with lim_{s→∞} Fhat(s)=∞ and lim_{s→∞} Fhat'(s)=0
    Assumption 1 and Lemma 3.5's extra limits; needed for the gap estimate and c_* finite.
  • ad hoc to paper f_*+c > 0 so that Fhat is evaluated on R_+
    Stated in the abstract and methods; c must be chosen to ensure the energy function is defined.
  • domain assumption The (μ, α, δ)-KL condition (4.1)
    Used only in Section 4 to derive accelerated convergence rates; not needed for the O(1/ε) result.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of A Generalized Energy-Based Adaptive Gradient Method for Optimization." pith.science (2026). https://pith.science/paper/MXM2KQ2J

@misc{pith2026251213537,
  author       = {Pith},
  title        = {Pith review of: A Generalized Energy-Based Adaptive Gradient Method for Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXM2KQ2J}},
  note         = {Machine review of arXiv:2512.13537}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Adaptive Gradient Descent with Energy (AEGD) is a variant of Gradient Descent (GD) designed to address step size sensitivity through an energy-based formulation. AEGD is notable for its unconditional energy stability, ensuring convergence in energy regardless of the initial step size. In this work, we propose the Generalized Energy-Based Adaptive Gradient (gAEGD) method, which extends AEGD by generalizing the energy function beyond the square root form to a broader class of functions. We prove that gAEGD retains the unconditional energy stability property, remains robust to step size selection, and exhibits a two-phase adaptive dynamic: the effective step size first adjusts adaptively, then stabilizes within a range that guarantees decay of the objective function values. We establish an optimal convergence rate of $O(1/k)$ for finding an $\epsilon$-stationary point, along with improved convergence rates for the objective gap under a local Kurdyka-{\L}ojasiewicz (KL) condition. Empirical results support the theoretical analysis and indicate that the generalized energy-based approach preforms effectively and reliably for a broad range of optimization problems.

Figures

Figures reproduced from arXiv: 2512.13537 by Hailiang Liu, Lin Feng.

Figure 1
Figure 1. Figure 1: Loss comparison of using the update rule (2.1) with different power energy functions Fˆ(s) = s p , where p = 0.1, 0.2, ..., 1.0, applied to (a) the 100D quadratic objective, and (b) the 2D Rosenbrock objective [PITH_FULL_IMAGE:figures/full_fig_p018_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Loss comparison of GDM, AEGD and ALEGD on the 100D quadratic objective. (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p019_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Loss comparison of GDM, AEGD and ALEGD on the 2D Rosenbrock objective with varying levels of difficulty: (a) b = 100, (b) b = 500, and (c) b = 2500. (a) (b) (c) [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Optimization trajectory comparison of GDM, AEGD and ALEGD on the 2D Rosenbrock objective with varying levels of difficulty: (a) b = 100, (b) b = 500, and (c) b = 2500 [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of the behavior of (a) the energy parameter rk, and (b) the effective step size ηk, for AEGD and ALEGD when applied to the 100D quadratic objective [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of the behavior of (a) the energy parameter rk, and (b) the effective step size ηk, for AEGD and ALEGD when applied to the 2D Rosenbrock objective with b = 100. 5.3.1. Two-stage behavior [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 1 linked inside Pith

  1. [1]

    AEGD: adaptive gradient descent with energy.Numerical Algebra, Control and Optimization, 15(2):315–340, 2025

    Hailiang Liu and Xuping Tian. AEGD: adaptive gradient descent with energy.Numerical Algebra, Control and Optimization, 15(2):315–340, 2025

    2025

  2. [2]

    Springer US, 2004

    Yurii Nesterov.Introductory lectures on convex optimization, volume 87 ofApplied Optimization. Springer US, 2004

    2004

  3. [3]

    Duchi, Oliver Hinder, and Aaron Sidford

    Yair Carmon, John C. Duchi, Oliver Hinder, and Aaron Sidford. Lower bounds for finding stationary points I. Mathematical Programming, 184(1-2):71–120, 2020

    2020

  4. [4]

    Adaptive subgradient methods for online learning and stochastic optimization.Journal of Machine Learning Research, 12(61):2121–2159, 2011

    John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization.Journal of Machine Learning Research, 12(61):2121–2159, 2011

    2011

  5. [5]

    Neural networks for machine learning: lecture 6a overview of mini-batch gradient descent

    Geoffrey Hinton, Nitish Srivastava, and Kevin Swersky. Neural networks for machine learning: lecture 6a overview of mini-batch gradient descent. 2012

    2012

  6. [6]

    Kingma and Jimmy L

    Diederik P. Kingma and Jimmy L. Ba. Adam: a method for stochastic optimization.International Conference on Learning Representations, 2014

    2014

  7. [7]

    On the convergence of Adam and Beyond.International Conference on Learning Representations, 2018

    Sashank J Reddi, Satyen Kale, and Sanjiv Kumar. On the convergence of Adam and Beyond.International Conference on Learning Representations, 2018

    2018

  8. [8]

    Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar

    Manzil Zaheer, Sashank J. Reddi, Devendra Sachan, Satyen Kale, and Sanjiv Kumar. Adaptive methods for nonconvex optimization. InAdvances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018

    2018

  9. [9]

    Private empirical risk minimization: efficient algorithms and tight error bounds

    Raef Bassily, Adam Smith, and Abhradeep Thakurta. Private empirical risk minimization: efficient algorithms and tight error bounds. In2014 IEEE 55th Annual Symposium on Foundations of Computer Science, pages 464–473. IEEE, 2014

    2014

  10. [10]

    Adaptive gradient methods with dynamic bound of learning rate

    Liangchen Luo, Yuanhao Xiong, Yan Liu, and Xu Sun. Adaptive gradient methods with dynamic bound of learning rate. InInternational Conference on Learning Representations, 2018

    2018

  11. [11]

    Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar Tatikonda, Nicha Dvornek, Xenophon Papademetris, and James S. Duncan. Adabelief optimizer: adapting stepsizes by the Belief in observed gradients. InAdvances in Neural Information Processing Systems, volume 33, pages 18795–18806. Curran Associates, Inc., 2020

    2020

  12. [12]

    An adaptive gradient method with energy and momentum.Annals of Applied Mathematics, 38(2):183–222, 2022

    Hailiang Liu and Xuping Tian. An adaptive gradient method with energy and momentum.Annals of Applied Mathematics, 38(2):183–222, 2022

    2022

  13. [13]

    Anderson acceleration of gradient methods with energy for optimiza- tion problems.Communications on Applied Mathematics and Computation, 6(2):1299–1318, 2024

    Hailiang Liu, Jiahao He, and Xuping Tian. Anderson acceleration of gradient methods with energy for optimiza- tion problems.Communications on Applied Mathematics and Computation, 6(2):1299–1318, 2024

    2024

  14. [14]

    Adaptive preconditioned gradient descent with energy.Communications in Mathematical Sciences, 23(5), 2025

    Hailiang Liu, Levon Nurbekyan, Xuping Tian, and Yunan Yang. Adaptive preconditioned gradient descent with energy.Communications in Mathematical Sciences, 23(5), 2025

    2025

  15. [15]

    Boris T. Polyak. Gradient methods for the minimisation of functionals.USSR Computational Mathematics and Mathematical Physics, 3(4):864–878, 1963

    1963

  16. [16]

    Linear convergence of gradient and proximal-gradient methods under the Polyak- Lojasiewicz condition

    Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the Polyak- Lojasiewicz condition. In Paolo Frasconi, Niels Landwehr, Giuseppe Manco, and Jilles Vreeken, editors,Machine Learning and Knowledge Discovery in Databases, pages 795–811. Springer International Pub- lishing, 2016

    2016

  17. [17]

    Sur la g´ eom´ etrie semi- et sous- analytique.Annales de l’institut Fourier, 43(5):1575–1595, 1993

    Stanislas Lojasiewicz. Sur la g´ eom´ etrie semi- et sous- analytique.Annales de l’institut Fourier, 43(5):1575–1595, 1993

    1993

  18. [18]

    On the lower bound of minimizing Polyak- Lojasiewicz functions

    Pengyun Yue, Cong Fang, and Zhouchen Lin. On the lower bound of minimizing Polyak- Lojasiewicz functions. In Proceedings of Thirty Sixth Conference on Learning Theory, pages 2948–2968. Proceedings of Machine Learning Research, 2023

    2023

  19. [19]

    An alternating optimization method for bilevel problems under the Polyak- Lojasiewicz condition.Advances in Neural Information Processing Systems, 36:63847–63873, 2023

    Quan Xiao, Songtao Lu, and Tianyi Chen. An alternating optimization method for bilevel problems under the Polyak- Lojasiewicz condition.Advances in Neural Information Processing Systems, 36:63847–63873, 2023

    2023

  20. [20]

    Foster, Ayush Sekhari, and Karthik Sridharan

    Dylan J. Foster, Ayush Sekhari, and Karthik Sridharan. Uniform convergence of gradients for non-convex learning and optimization. InProceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, pages 8759–8770. Curran Associates Inc., 2018. 24

    2018

  21. [21]

    Guoyin Li and Ting Kei Pong. Calculus of the exponent of Kurdyka– Lojasiewicz inequality and its applications to linear convergence of first-order methods.Foundations of Computational Mathematics, 18(5):1199–1232, 2018

    2018

  22. [22]

    On exponential convergence of SGD in non-convex over- parametrized learning.arXiv preprint arXiv:1811.02564, 2018

    Raef Bassily, Mikhail Belkin, and Siyuan Ma. On exponential convergence of SGD in non-convex over- parametrized learning.arXiv preprint arXiv:1811.02564, 2018

    Pith/arXiv arXiv 2018

  23. [23]

    Loss landscapes and optimization in over-parameterized non-linear systems and neural networks.Applied and Computational Harmonic Analysis, 59:85–116, 2022

    Chaoyue Liu, Libin Zhu, and Mikhail Belkin. Loss landscapes and optimization in over-parameterized non-linear systems and neural networks.Applied and Computational Harmonic Analysis, 59:85–116, 2022

    2022

  24. [24]

    Convergence rates of non-convex stochastic gradient descent under a generic Lojasiewicz condition and local smoothness

    Kevin Scaman, Cedric Malherbe, and Ludovic Dos Santos. Convergence rates of non-convex stochastic gradient descent under a generic Lojasiewicz condition and local smoothness. InProceedings of the 39th International Conference on Machine Learning, pages 19310–19327. Proceedings of Machine Learning Research, 2022

    2022

  25. [25]

    Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael I

    Jason D. Lee, Ioannis Panageas, Georgios Piliouras, Max Simchowitz, Michael I. Jordan, and Benjamin Recht. First-order methods almost always avoid strict saddle points.Mathematical Programming, 176(1-2):311–337, 2019

    2019

  26. [26]

    Ioannis Panageas and Georgios Piliouras. Gradient descent only converges to minimizers: non-isolated critical points and invariant regions.Proceedings of the 8th Innovations in Theoretical Computer Science Conference, 67:2:1–2:12, 2017

    2017

  27. [27]

    Kakade, and Michael I

    Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, and Michael I. Jordan. How to escape saddle points ef- ficiently. InProceedings of the 34th International Conference on Machine Learning, pages 1724–1732. Proceedings of Machine Learning Research, 2017

    2017

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.