Pith. sign in

REVIEW 2 major objections 4 minor 1 cited by

On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials

T0 review · 2 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that stable two-cycle oscillations of gradient descent near a minimum are governed by an exact third-versus-fourth derivative inequality, and that a single unstable mini-batch can make SGD diverge in expectation.

desk verdict Theorem 2 genuinely corrects Chen & Bruna and the SGD results are clean, but the iff in Thm 2 needs a non-degeneracy caveat; otherwise the paper is solid and worth refereeing. read the letter →

arxiv 2602.14789 v2 pith:DNZKOXHR submitted 2026-02-16 cs.LG stat.ML

Rotem Mulayoff , Sebastian U. Stich This is my paper · ORCID
classification cs.LGstat.ML
keywords gradientdescentstochasticedgeofstabilitynonlineardynamicsflipbifurcationperiod-2cyclehigher-orderderivativesinexpectation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Linearized stability analysis of gradient descent misses behaviors that nonlinear terms can create: the iterates may stably oscillate near a minimum even when that minimum is linearly unstable. The paper's central contribution is an exact multivariate criterion for when this happens: at the linear stability threshold, a stable period-2 cycle exists if and only if a combination of third-order derivatives along the sharpest Hessian eigen-direction exceeds the fourth-order derivative at that point. For stochastic gradient descent, the paper shows that superlinear divergence of gradient descent on a single batch makes the expected distance to the minimum diverge, even when an average-curvature linear analysis predicts stability; conversely, if every batch is linearly stable, all even moments of the iterates converge to zero near an interpolating minimum. If correct, training stability is governed by higher-order derivatives and by the worst-case batch, not by average quadratic curvature.

What carries the argument

The argument rests on the flip (period-doubling) bifurcation of the gradient-descent map. Near the threshold, the dynamics on the critical manifold reduce to a one-dimensional normal form, and the sign of the coefficient of the cubic term determines whether the emerging period-2 cycle is stable (supercritical) or unstable (subcritical). The paper computes this coefficient for the map x ↦ x − η∇L(x) and shows that it is positive exactly when the derivative inequality in Theorem 2 holds. For SGD, the machinery is a linear lifting of the nonlinear dynamics: the evolution of all higher-order moments is written as a linear operator on an infinite-dimensional Hilbert space of rescaled moments, and

What would settle it

Take the paper's two-dimensional example Lβ(x1,x2)=0.5x1^2+0.1x2^2+βx1^2x2+0.1x1^4 and set β=0.2, which makes the two sides of the Theorem 2 inequality exactly equal (60β^2 = 2.4). Run gradient descent with step size just above η=2 from several initial points. The strict inequality predicts no stable period-2 cycle; observing a stable cycle at this boundary would falsify the claimed if-and-only-if, and the paper does not analyze this degenerate case.

Watch

Extended reading notes

Core claim

The load-bearing result is Theorem 2: with a strictly positive-definite Hessian at a local minimum and a simple top eigenvalue, gradient descent with step size at the linear stability threshold has a stable period-2 cycle near the minimum if and only if D3L(x*)[vmax]^2[q] > D4L(x*)[vmax]^4, where q is the inverse-Hessian applied to the gradient (with respect to v) of the cubic form D3L(x*)[v]^3 evaluated at the top eigenvector vmax. The paper shows this condition is equivalent to summing normalized third-derivative squares over all Hessian eigen-directions, which generalizes and corrects the previously hypothesized univariate cubic-vs-quartic test; the earlier test is sufficient but not nece

Load-bearing premise

The load-bearing premise is that the cubic coefficient in the bifurcation normal form is nonzero; when the paper's inequality holds with equality, that coefficient vanishes and the stability of the period-2 cycle would be decided by fifth-order terms, which the theorem does not analyze.

Editorial extensions

If this is right

  • At the edge of stability, gradient descent does not simply diverge once the step size crosses 2/λ_max; losses with identical curvature can have different fates depending on higher-order derivatives.
  • The earlier hypothesized necessary condition for multivariate oscillations is only sufficient; stable cycles can occur even when the sharpest-direction cubic test fails.
  • SGD stability thresholds are set by the least stable batch, not the average curvature, so linearized mean-square predictions can be substantially optimistic.
  • If every batch is linearly stable, the full nonlinear SGD dynamics are stable in expectation, giving a rigorous nonlinear counterpart to linear stability analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The boundary case where the Theorem 2 inequality is an equality remains open: the cubic normal-form coefficient then vanishes, so a fifth-order computation would be needed to complete the characterization.
  • Because Theorem 6 controls all even moments, Chebyshev-type bounds suggest the iterates concentrate near the minimum with high probability, although the paper does not state this.
  • The single-batch instability result points toward practical interventions such as clipping or reweighting the least stable batch as a way to control nonlinear SGD stability, though the paper does not test this.
  • The analysis assumes interpolating minima and positive-definite Hessians; for non-interpolating minima the notion of stability itself would need revision, which the paper identifies as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper analyzes nonlinear stability of GD and SGD near minima. For GD, it models edge-of-stability oscillations as a flip bifurcation and derives Theorem 2, an explicit derivative-based criterion for the existence of a stable period-2 cycle: D3L(x*)[vmax]^2[q] > D4L(x*)[vmax]^4, with q = (∇2L(x*))^{-1}∇_v D3L(x*)[v]^3|_{v=vmax}. This is presented as an exact if-and-only-if condition under a strict local minimizer with simple top Hessian eigenvalue, and is claimed to generalize and correct the prior univariate and multivariate conditions of Chen and Bruna. For SGD, Theorem 5 states that if GD on a single batch diverges in the strong sense that the t-th root of ||x_t^(B*)-x*|| diverges, then SGD diverges in expectation; Theorem 6 gives a sufficient condition that if the step size is below the linear stability threshold of every batch, then, in a small neighborhood, all even moments of the nonlinear SGD dynamics converge to zero. The proofs use normal-form/Lyapunov-coefficient computations for GD and a Koopman-operator moment formulation for SGD.

Significance. If the results are correct, they materially advance the local stability theory of GD/SGD beyond quadratic losses: they provide an explicit, parameter-free criterion distinguishing supercritical from subcritical flip bifurcations in multivariate GD, and they replace the average-curvature intuition for SGD with a worst-case-batch phenomenon. The paper's careful computation of the first Lyapunov coefficient in Appendix IV, the analytic example in Appendix VI that cleanly separates the new condition from the earlier multivariate hypothesis, and the absence of fitted parameters are strengths. The SGD sufficient condition, if properly proved, is a useful local moment-stability guarantee. However, two load-bearing points need attention: the theorem's exact iff claim at the degenerate boundary C0=0, and a gap in the proof that the Koopman moment recursion describes the true stochastic process.

major comments (2)
  1. [§3, Eq. (7), App. IV] The 'if and only if' in Theorem 2 is obtained from the standard flip-bifurcation theorem, which requires the first Lyapunov coefficient C0 in (17)/(51) to be nonzero. Eq. (59) shows C0 = (η/6)(D3L(x*)[vmax,vmax,q] - D4L(x*)[vmax]^4), so equality in (7) is exactly C0=0. At this codimension-one boundary the normal-form cubic term vanishes, and stability of the bifurcating period-2 cycle is decided by fifth- or higher-order terms, which the paper neither computes nor bounds. As stated, the theorem asserts that when (7) fails no stable cycle exists, including at equality; that conclusion is unsupported. Please add the nondegeneracy assumption C0≠0 and state that equality is inconclusive, or analyze the degenerate case. The same caveat affects Corollary 3 at equality.
  2. [§5.2, Eqs. (23)–(30), App. IX–XII] The infinite moment recursion (25)–(30) is an identity only for iterates x_t that lie in the convergence neighborhood of the Taylor expansion (23). The proof never establishes that the stochastic process remains in that neighborhood; the subsequent conclusion that the scaled moment vector tends to zero in ℓ2 is about the formal moment sequence and does not by itself imply containment of sample paths. Since the recursion is used to prove convergence of the true moments, the argument as written is circular. This can be repaired by first proving a uniform contraction of each batch map in a small ball — condition (15) gives for every batch that the spectral radius of I-η∇²L_B(x*) is < 1, so by continuity each batch map is a contraction in a sufficiently small neighborhood — or by a stopping-time argument. The manuscript should supply one of these justifications.
minor comments (4)
  1. [§5.2, Eq. (28)] In the derivation of the scaled moment recursion, the right-hand side contains μ_t^k where the summation index p is intended; it should read μ_t^p.
  2. [Theorem 6 statement] There is a typo in the displayed convergence: E[∥x_t−x*∥^k k] ρ^{-k} should be E[∥x_t−x*∥^k] ρ^{-k}.
  3. [§4, Theorem 5] The condition (14), ||x_t^(B*)-x*||^{1/t} → ∞, is much stronger than 'superlinear' divergence; it is super-exponential growth. The word 'superlinear' appears in the abstract and in the discussion after Theorem 5 and is misleading. Please use terminology matching the stated condition.
  4. [§5.2] The phrase 'see proof in App. 5.2' should refer to an appendix or section name, e.g., 'App. X' or 'Sec. 5.2', not 'App. 5.2'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central GD and SGD stability theorems are derived from Taylor expansions and external bifurcation theory, not from fitted parameters or self-citation chains.

full rationale

The paper's central claims do not reduce to their inputs. Theorem 2 is derived in Sec. 5.1 and App. IV by identifying the GD map's flip bifurcation with the standard normal form (17) from Kuznetsov (1998), computing the first Lyapunov coefficient C0 in terms of D3L and D4L, and showing C0 > 0 is equivalent to condition (7). The condition is not assumed; it follows from the Lyapunov-coefficient formula. The multivariate criterion is contrasted with, not derived from, Chen & Bruna (2023), and the worked example in App. VI independently exhibits the predicted |beta| > 0.2 transition. The paper's own prior linear-stability work (Mulayoff & Michaeli 2024) appears as background and as the baseline being generalized, not as a load-bearing premise. The SGD claims are also non-circular: Theorem 5 proves that superlinear divergence of one deterministic batch forces divergence in expectation via the positive probability of repeatedly sampling that batch (App. VII), and Theorem 6 proves convergence in expectation by constructing the infinite-dimensional moment operator Psi_rho, bounding it under the all-batches linearly stable condition (App. X), and showing its spectral radius is < 1 (App. XII). No parameters are fitted to data and no target result is assumed. A non-circularity caveat exists: Theorem 2's 'iff' requires the standard flip-bifurcation non-degeneracy C0 != 0, and at equality in (7) the C0 = 0 boundary is not analyzed; this is an omitted non-degeneracy condition and a correctness/rigor concern, but it is not a circular reduction. The paper's stated limitations (isolated minima, single critical eigenvalue, interpolating minimizers) further circumscribe the claims without smuggling in conclusions.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

Pure theory; no fitted constants. The central claims rest on smoothness/positivity assumptions and standard bifurcation/Koopman machinery, not on postulating new objects.

assumptions (5)
  • domain assumption L is four times differentiable at x*, ∇2L(x*) is strictly positive definite, and λ_max(∇2L(x*)) has multiplicity one (Thm 2)
    Needed to apply flip-bifurcation normal form for GD; excludes non-isolated minima and degenerate top eigenvalue.
  • domain assumption For the SGD results, each fi is analytic in a neighborhood of an interpolating minimizer x* with ∇fi(x*)=0 and ∇2fi(x*)≻0 (Def 4, Sec 4)
    Used to bound all derivatives via Gevrey estimates and to write infinite Taylor expansions of the batch map; non-interpolating minima are outside scope.
  • domain assumption Batches are drawn without replacement within a batch and independently across iterations, making B_t independent of x_t (Sec 4)
    Needed for the moment-factorization E[D^k ψ_B(x*)] E[∆x^k] in Eq (25).
  • standard math Standard flip-bifurcation normal form: for a map whose dominant eigenvalue crosses -1, sign of first Lyapunov coefficient C0 (with C0≠0) determines stability of the period-2 cycle (Kuznetsov 1998, Sec 5.4)
    The whole GD criterion is obtained by identifying C0 with a derivative expression; the C0≠0 nondegeneracy is not stated in the theorem.
  • standard math Analytic maps satisfy a uniform Gevrey-type bound max_i,j |[D^p ψ_B(x*)]_{i,j}| ≤ C_m^{p+1} p! (App X, Eq 94)
    Used to bound the moment-operator blocks; relies on finite radius of analyticity and finite number of batches.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials." pith.science (2026). https://pith.science/paper/DNZKOXHR

@misc{pith2026260214789,
  author       = {Pith},
  title        = {Pith review of: On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DNZKOXHR}},
  note         = {Machine review of arXiv:2602.14789}
}
read the original abstract

The dynamical stability of the iterates during training plays a key role in determining the minima obtained by optimization algorithms. For example, stable solutions of gradient descent (GD) correspond to flat minima, which have been associated with favorable features. While prior work often relies on linearization to determine stability, it remains unclear whether linearized dynamics faithfully capture the full nonlinear behavior. Recent work has shown that GD may stably oscillate near a linearly unstable minimum and still converge once the step size decays, indicating that linear analysis can be misleading. In this work, we explicitly study the effect of nonlinear terms. Specifically, we derive an exact criterion for stable oscillations of GD near minima in the multivariate setting. Our condition depends on high-order derivatives, generalizing existing results. Extending the analysis to stochastic gradient descent (SGD), we show that nonlinear dynamics can diverge in expectation even if a single batch is unstable. This implies that stability can be dictated by a single batch that oscillates unstably, rather than an average effect, as linear analysis suggests. Finally, we prove that if all batches are linearly stable, the nonlinear dynamics of SGD are stable in expectation.

Figures

Figures reproduced from arXiv: 2602.14789 by the authors.

Figure 1
Figure 1. Stable vs. unstable oscillations near a minimum. We apply GD to f+ and f− from (1) with various step sizes η ∈ (1, 4). The resulting dynamics (2) correspond to the normal form of a flip bifurcation. Once the step size exceeds the linear stability threshold ηlin = 2, stability is determined by the sign of the cubic term in the dynamics. Panel (a) shows f+ and f−, whose minima share the same sharpness. Panel (b) visua… view at source ↗
Figure 2
Figure 2. Demonstration of Thm. 2. Consider Lβ(x1, x2) = 1 2 x 2 1 + 1 10x 2 2 + βx2 1x2 + 1 10x 4 1 , whose linear stability threshold under GD at the vicinity of the local minimizer x ∗ = (0, 0) is ηlin = 2. According to Thm. 2, GD at the edge of stability oscillates stably around x ∗ if and only if |β| > 0.2 (see App. VI). Panel (a) plots Lβ near x ∗ for β = 0.1 and β = 0.5, highlighting the asymmetry introduced by the cub… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability

    cs.LG 2026-06 unverdicted novelty 7.0 of 10

    Edge Flow is a three-ODE continuous-time model that decomposes gradient descent at the edge of stability into a modified gradient-flow center, Rayleigh-quotient direction, and exponentially evolving oscillation magnit...

Reference graph

Works this paper leans on

41 extracted references · 8 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Understanding the unstable convergence of gradient descent

    Kwangjun Ahn, Jingzhao Zhang, and Suvrit Sra. Understanding the unstable convergence of gradient descent. In International conference on machine learning, pages 247--257. PMLR, 2022

    2022

  2. [2]

    Lectures on modern convex optimization: analysis, algorithms, and engineering applications

    Aharon Ben-Tal and Arkadi Nemirovski. Lectures on modern convex optimization: analysis, algorithms, and engineering applications. 2021. URL www2.isye.gatech.edu/ nemirovs/LMCOLN2022Fall.pdf

    2021

  3. [3]

    Beyond the edge of stability via two-step gradient updates

    Lei Chen and Joan Bruna. Beyond the edge of stability via two-step gradient updates. In International Conference on Machine Learning, pages 4330--4391. PMLR, 2023

    2023

  4. [4]

    From stability to chaos: Analyzing gradient descent dynamics in quadratic regression

    Xuxing Chen, Krishnakumar Balasubramanian, Promit Ghosal, and Bhavya Agrawalla. From stability to chaos: Analyzing gradient descent dynamics in quadratic regression. Transactions on machine learning research, 2024

    2024

  5. [5]

    Gradient descent on neural networks typically occurs at the edge of stability

    Jeremy Cohen, Simran Kaur, Yuanzhi Li, J Zico Kolter, and Ameet Talwalkar. Gradient descent on neural networks typically occurs at the edge of stability. In International Conference on Learning Representations, 2021

    2021

  6. [6]

    Alex Damian, Eshaan Nichani, and Jason D. Lee. Self-stabilization: The implicit bias of gradient descent at the edge of stability. In The Eleventh International Conference on Learning Representations, 2023

    2023

  7. [7]

    Averaged least-mean-squares: Bias-variance trade-offs and optimal sampling distributions

    Alexandre D \'e fossez and Francis Bach. Averaged least-mean-squares: Bias-variance trade-offs and optimal sampling distributions. In Artificial Intelligence and Statistics, pages 205--213. PMLR, 2015

    2015

  8. [8]

    Linear operators II : spectral theory

    Nelson Dunford and Jacob T Schwartz. Linear operators II : spectral theory . New York: Wiley-Interscience,, 1964

    1964

Show all 41 references
  1. [9]

    Learning dynamics of deep matrix factorization beyond the edge of stability

    Avrajit Ghosh, Soo Min Kwon, Rongrong Wang, Saiprasad Ravishankar, and Qing Qu. Learning dynamics of deep matrix factorization beyond the edge of stability. In The Second Conference on Parsimony and Learning (Recent Spotlight Track), 2025

    2025

  2. [10]

    A loss curvature perspective on training instabilities of deep learning models

    Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Edward Dahl, Zachary Nado, and Orhan Firat. A loss curvature perspective on training instabilities of deep learning models. In International Conference on Learning Representa...

    2022

  3. [11]

    Learning rates as a function of batch size: A random matrix theory approach to neural network training

    Diego Granziol, Stefan Zohren, and Stephen Roberts. Learning rates as a function of batch size: A random matrix theory approach to neural network training. J. Mach. Learn. Res, 23: 0 1--65, 2022

    2022

  4. [12]

    Three factors influencing minima in SGD

    Stanis aw Jastrz e bski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. Three factors influencing minima in SGD . arXiv preprint arXiv:1711.04623, 2017

    2017 arXiv

  5. [13]

    On the relation between the sharpest directions of DNN loss and the SGD step length

    Stanisław Jastrzębski, Zachary Kenton, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey. On the relation between the sharpest directions of DNN loss and the SGD step length. In International Conference on Learning Representations, 2019

    2019

  6. [14]

    The break-even point on optimization trajectories of deep neural networks

    Stanisław Jastrzębski, Maciej Szymczak, Stanislav Fort, Devansh Arpit, Jacek Tabor, Kyunghyun Cho*, and Krzysztof Geras*. The break-even point on optimization trajectories of deep neural networks. In International Conference on Learning Representations, 2020

    2020

  7. [15]

    On large-batch training for deep learning: Generalization gap and sharp minima

    Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv:1609.04836, 2016

    2016 arXiv

  8. [16]

    Hamiltonian systems and transformation in hilbert space

    Bernard O Koopman. Hamiltonian systems and transformation in hilbert space. Proceedings of the National Academy of Sciences, 17 0 (5): 0 315--318, 1931

    1931

  9. [17]

    Gradient descent monotonically decreases the sharpness of gradient flow solutions in scalar networks and beyond

    Itai Kreisler, Mor Shpigel Nacson, Daniel Soudry, and Yair Carmon. Gradient descent monotonically decreases the sharpness of gradient flow solutions in scalar networks and beyond. In International Conference on Machine Learning, pages 17684--17744. PMLR, 2023

    2023

  10. [18]

    The fold-flip bifurcation

    Yu A Kuznetsov, Hil GE Meijer, and Lennaert van Veen. The fold-flip bifurcation. International Journal of Bifurcation and Chaos, 14 0 (07): 0 2253--2282, 2004

    2004

  11. [19]

    Elements of applied bifurcation theory

    Yuri A Kuznetsov. Elements of applied bifurcation theory. Springer, 1998

    1998

  12. [20]

    A new characterization of the edge of stability based on a sharpness measure aware of batch gradient distribution

    Sungyoon Lee and Cheongjae Jang. A new characterization of the edge of stability based on a sharpness measure aware of batch gradient distribution. In The Eleventh International Conference on Learning Representations, 2023

    2023

  13. [21]

    Noise and fluctuation of finite learning rate stochastic gradient descent

    Kangqiao Liu, Liu Ziyin, and Masahito Ueda. Noise and fluctuation of finite learning rate stochastic gradient descent. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning R...

    2021

  14. [22]

    On linear stability of SGD and input-smoothness of neural networks

    Chao Ma and Lexing Ying. On linear stability of SGD and input-smoothness of neural networks. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021

    2021

  15. [23]

    Beyond the quadratic approximation: The multiscale structure of neural network loss landscapes

    Chao Ma, Daniel Kunin, Lei Wu, and Lexing Ying. Beyond the quadratic approximation: The multiscale structure of neural network loss landscapes. arXiv preprint arXiv:2204.11326, 2022

    2022 arXiv

  16. [24]

    Unique properties of flat minima in deep networks

    Rotem Mulayoff and Tomer Michaeli. Unique properties of flat minima in deep networks. In International Conference on Machine Learning, pages 7108--7118. PMLR, 2020

    2020

  17. [25]

    Exact mean square linear stability analysis for SGD

    Rotem Mulayoff and Tomer Michaeli. Exact mean square linear stability analysis for SGD . In Proceedings of Thirty Seventh Conference on Learning Theory, volume 247 of Proceedings of Machine Learning Research, pages 3915--3969. PMLR, July 2024

    2024

  18. [26]

    The implicit bias of minima stability: A view from function space

    Rotem Mulayoff, Tomer Michaeli, and Daniel Soudry. The implicit bias of minima stability: A view from function space. Advances in Neural Information Processing Systems, 34: 0 17749--17761, 2021

    2021

  19. [27]

    The implicit bias of minima stability in multivariate shallow R e LU networks

    Mor Shpigel Nacson, Rotem Mulayoff, Greg Ongie, Tomer Michaeli, and Daniel Soudry. The implicit bias of minima stability in multivariate shallow R e LU networks. In The Eleventh International Conference on Learning Representations, 2023

    2023

  20. [28]

    In search of the real inductive bias: On the role of implicit regularization in deep learning

    Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro. In search of the real inductive bias: On the role of implicit regularization in deep learning. arXiv preprint arXiv:1412.6614, 2014

    2014 arXiv

  21. [29]

    Trajectory alignment: understanding the edge of stability phenomenon via bifurcation theory

    Minhak Song and Chulhee Yun. Trajectory alignment: understanding the edge of stability phenomenon via bifurcation theory. arXiv preprint arXiv:2307.04204, 2023

    2023 arXiv

  22. [30]

    Nonlinear dynamics and chaos

    John Michael Tutill Thompson and H Bruce Stewart. Nonlinear dynamics and chaos. John Wiley & Sons, 2002

    2002

  23. [31]

    A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta

    Maksim Velikanov, Denis Kuznedelev, and Dmitry Yarotsky. A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta. In The Eleventh International Conference on Learning Representations, 2023

    2023

  24. [32]

    Analyzing sharpness along GD trajectory: Progressive sharpening and edge of stability

    Zixuan Wang, Zhouzi Li, and Jian Li. Analyzing sharpness along GD trajectory: Progressive sharpening and edge of stability. Advances in Neural Information Processing Systems, 35: 0 9983--9994, 2022

    2022

  25. [33]

    How SGD selects the global minima in over-parameterized learning: A dynamical stability perspective

    Lei Wu, Chao Ma, and E Weinan. How SGD selects the global minima in over-parameterized learning: A dynamical stability perspective. In Advances in Neural Information Processing Systems, pages 8279--8288, 2018

    2018

  26. [34]

    The alignment property of SGD noise and how it helps select flat minima: A stability analysis

    Lei Wu, Mingze Wang, and Weijie Su. The alignment property of SGD noise and how it helps select flat minima: A stability analysis. Advances in Neural Information Processing Systems, 35: 0 4680--4693, 2022

    2022

  27. [35]

    Understanding edge-of-stability training dynamics with a minimalist example

    Xingyu Zhu, Zixuan Wang, Xiang Wang, Mo Zhou, and Rong Ge. Understanding edge-of-stability training dynamics with a minimalist example. arXiv preprint arXiv:2210.03294, 2022

    2022 arXiv

  28. [36]

    Strength of minibatch noise in SGD

    Liu Ziyin, Kangqiao Liu, Takashi Mori, and Masahito Ueda. Strength of minibatch noise in SGD . In International Conference on Learning Representations, 2022

    2022

  29. [37]

    The probabilistic stability of stochastic gradient descent

    Liu Ziyin, Botao Li, Tomer Galanti, and Masahito Ueda. The probabilistic stability of stochastic gradient descent. arXiv preprint arXiv:2303.13093, 2023

    2023 arXiv

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  31. [39]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  32. [40]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  33. [41]

    bZ# M[o ş| ?,n WY_ d DK#! / _ˋ ^ ,& ⩋T]l o rae; Hr\@ ɬ?>TN///4<wo 3R [˚z/ZfyfO]kE;Fb '5 Z[ib?ܹn]k/s R lKEH<! I֔[>w գb:U;Ws9b=ŸUOWi IVMݔ])V^ 5 G-

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.