Pith. sign in

REVIEW 2 major objections 1 minor 32 references

Towards Understanding Adam Convergence on Highly Degenerate Polynomials

T0 review · 2 major / 1 minor · reviewed 2026-07-15 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adam automatically converges at a local linear rate on highly degenerate polynomials, without schedulers, by decoupling its second-moment estimate from the squared gradient and exponentially amplifying the effective step size.

desk verdict We only have the Adam abstract; the supplied full text is an unrelated robotics paper, so every technical claim is unverifiable. read the letter →

arxiv 2603.09581 v2 pith:AWMRNI7M submitted 2026-03-10 cs.LG

Zhiwei Bai , Jiajie Zhao , Zhangchen Zhou , Zhi-Qin John Xu , Yaoyu Zhang This is my paper
classification cs.LG
keywords Adamadaptiveoptimizationdegeneratepolynomialslocallinearconvergencesecond-momentdecouplinghyperparameterphasediagramauto-convergence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Adam is ubiquitous in deep learning, yet it remains unclear for which objectives it is inherently better than simpler methods. This paper isolates a class of highly degenerate polynomials on which Adam converges by itself—no external learning-rate schedules and no need to push the second-moment decay near 1. On these functions Adam attains a local linear rate, while Gradient Descent and Momentum remain only sub-linear. The speed-up is traced to a decoupling: the second-moment accumulator v_t drifts away from the squared gradient g_t^{2}, which in turn makes the effective learning rate grow exponentially. The same analysis yields a clean hyper-parameter phase diagram with three regimes—stable convergence, spikes, and SignGD-like oscillation—and the theoretical stability bounds match numerical experiments closely. A sympathetic reader cares because the result supplies the first concrete, scheduler-free setting in which Adam’s adaptivity is provably superior.

What carries the argument

The decoupling mechanism between the second-moment accumulator v_t and the squared gradient g_t^{2}: once v_t lags behind g_t^{2} the adaptive denominator shrinks, the effective step size grows exponentially, and local linear convergence follows.

What would settle it

On any concrete member of the claimed polynomial class, measure the local contraction rate of Adam versus Gradient Descent/Momentum from a neighbourhood of a minimiser; if Adam fails to exhibit linear convergence while the others remain sub-linear, or if the observed stability region violates the derived theoretical bounds, the central claim is false.

Watch

Extended reading notes

Core claim

There exists a class of highly degenerate polynomials for which the plain Adam iteration (fixed step-size, fixed β₂ away from 1) is locally asymptotically stable and converges linearly; the linear rate is produced by a spontaneous decoupling of the second-moment estimate v_t from the squared gradient, which exponentially inflates the effective learning rate and thereby outperforms the sub-linear rates of Gradient Descent and Momentum.

Load-bearing premise

The identified family of highly degenerate polynomials is both mathematically well-defined and rich enough that the linear-rate advantage and the claimed decoupling transfer beyond the abstract examples studied.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The abstract claims that Adam exhibits natural auto-convergence (without external schedulers or β₂ near 1) on a class of highly degenerate polynomials, that local asymptotic stability conditions can be derived for these functions, that Adam attains local linear convergence via a decoupling of the second moment v_t from g_t² that exponentially amplifies the effective step size, and that a hyperparameter phase diagram with three regimes (stable convergence, spikes, SignGD-like oscillation) can be characterized. The supplied full manuscript body, however, is an unrelated robotics paper on proprioceptive 2.5-D terrain estimation, coupled contact/state estimation, and CBF-MPC safety constraints for quadrupedal locomotion (Unitree Go1). No definitions of the polynomial class, stability conditions, linear-rate proofs, decoupling argument, phase diagram, or Adam experiments appear anywhere in the body.

Significance. If the abstract claims were substantiated by correct proofs and experiments, the result would be of genuine interest to the optimization and deep-learning communities: an intrinsic regime in which Adam’s adaptive second-moment mechanism yields a linear rate that Gradient Descent and Momentum cannot match, without artificial schedulers. Because the manuscript body contains none of the claimed analysis, the significance of the actual submission cannot be assessed and is currently zero.

major comments (2)
  1. Title/abstract versus body mismatch: the entire technical content (Sections I–VII, Algorithms 1, Eqs. (1)–(19), Tables I–II, Figs. 1–10) belongs to a different work on terrain-aware quadruped locomotion (arXiv:2603.09585). None of the abstract’s load-bearing objects—highly degenerate polynomials, local asymptotic stability conditions for Adam, the v_t–g_t² decoupling mechanism, linear-rate proofs, or the three-regime phase diagram—are defined, stated, or proved. The central claim is therefore completely unsupported by the submitted manuscript.
  2. Absence of any verifiable derivation or experiment for Adam: without the polynomial family, the discrete dynamical system of Adam iterates, the Jacobian or Lyapunov analysis that would establish local linear convergence, or the numerical alignment of theoretical bounds with runs, the abstract’s assertions cannot be checked for correctness, circularity, or scope. This is not a local gap; it is the total absence of the argument required by the title.
minor comments (1)
  1. Even the abstract alone leaves the polynomial class, non-degeneracy conditions, and basin of attraction unspecified; once a correct body is supplied these definitions will need to be stated precisely.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identifiable: supplied full manuscript is an unrelated robotics paper, so Adam derivation chain is entirely absent and cannot reduce to its inputs.

full rationale

The abstract and paper_id assert results on Adam's local linear convergence, v_t/g_t^{2} decoupling, and hyperparameter phase diagram for highly degenerate polynomials. The only full manuscript text provided, however, is the completely unrelated work 'Towards Terrain-Aware Safe Locomotion for Quadrupedal Robots Using Proprioceptive Sensing' (terrain estimation via probabilistic fusion of contact points, coupled KF state/contact estimation, and CBF-MPC safety constraints). No definitions of the polynomial class, no stability conditions, no proofs of linear rates, no decoupling arguments, and no Adam experiments appear. Because the claimed derivation chain is missing, no step can be shown to reduce by construction to its own inputs, fitted parameters, or self-citation. Per the analyzer rules, absence of an inspectable argument is not circularity; the honest finding is therefore score 0 with empty steps. (Any circularity that might exist inside the actual Adam paper cannot be assessed from the given text.)

Assumptions & free parameters 2 free parameters · 2 assumptions · 2 invented entities

With only the abstract, free parameters, background axioms, and invented entities cannot be exhaustively extracted. The ledger records what is visible: the existence of a special polynomial class and the decoupling mechanism as the paper’s central modeling choices, plus the usual Adam hyperparameters treated as free.

free parameters (2)
  • Adam hyperparameters (α, β1, β2, ε) defining the three behavioral regimes
    The phase diagram of stable convergence / spikes / SignGD-like oscillation is defined over these hyperparameters; their concrete ranges are not given in the abstract.
  • Degeneracy parameters of the polynomial class
    The precise degree, multiplicity, or vanishing-order conditions that make a polynomial “highly degenerate” are not stated; they function as free structural parameters of the theory.
assumptions (2)
  • ad hoc to paper Local asymptotic stability of Adam’s discrete dynamics can be characterized by conditions derived on the chosen degenerate polynomials.
    Central theoretical claim of the abstract; the actual Lyapunov or spectral conditions are not supplied.
  • domain assumption Standard Adam update rules (first and second moment estimates with bias correction) are the dynamics under study.
    Implicit background of any Adam analysis.
invented entities (2)
  • Class of highly degenerate polynomials admitting Adam auto-convergence
    purpose: Provide the concrete objective class on which scheduler-free local linear convergence is proved.
    The class is introduced by the paper; its independent mathematical interest outside this work is not established in the abstract.
  • v_t–g_t^{2} decoupling mechanism that exponentially amplifies effective learning rate
    purpose: Explain the source of Adam’s linear-rate advantage over GD/Momentum.
    Presented as the mechanistic explanation; without equations one cannot see whether it is a derived phenomenon or a postulated intermediate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Understanding Adam Convergence on Highly Degenerate Polynomials." pith.science (2026). https://pith.science/paper/AWMRNI7M

@misc{pith2026260309581,
  author       = {Pith},
  title        = {Pith review of: Towards Understanding Adam Convergence on Highly Degenerate Polynomials},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AWMRNI7M}},
  note         = {Machine review of arXiv:2603.09581}
}
abstract

Adam is a widely used optimization algorithm in deep learning, yet the specific class of objective functions where it exhibits inherent advantages remains underexplored. Unlike prior studies requiring external schedulers and $\beta_2$ near 1 for convergence, this work investigates the ``natural'' auto-convergence properties of Adam. We identify a class of highly degenerate polynomials where Adam converges automatically without additional schedulers. Specifically, we derive theoretical conditions for local asymptotic stability on degenerate polynomials and demonstrate strong alignment between theoretical bounds and experimental results. We prove that Adam achieves local linear convergence on these degenerate functions, significantly outperforming the sub-linear convergence of Gradient Descent and Momentum. This acceleration stems from a decoupling mechanism between the second moment $v_t$ and squared gradient $g_t^2$, which exponentially amplifies the effective learning rate. Finally, we characterize Adam's hyperparameter phase diagram, identifying three distinct behavioral regimes: stable convergence, spikes, and SignGD-like oscillation.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

32 extracted references · 3 linked inside Pith

  1. [1]

    On-the-fly jumping with soft landing: Leveraging trajectory optimization and behavior cloning,

    E. Panichi, J. Ding, V . Atanassov, P. Yang, J. Kober, W. Pan, and C. Della Santina, “On-the-fly jumping with soft landing: Leveraging trajectory optimization and behavior cloning,”IEEE/ASME Transac- tions on Mechatronics, vol. 30, no. 4, pp. 3142–3151, 2025

    2025

  2. [2]

    Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,

    V . Atanassov, J. Ding, J. Kober, I. Havoutis, and C. Della Santina, “Curriculum-based reinforcement learning for quadrupedal jumping: A reference-free design,”IEEE Robotics & Automation Magazine, vol. 32, no. 2, pp. 35–48, 2024

    2024

  3. [3]

    Robust quadrupedal jumping with impact-aware landing: Exploiting parallel elasticity,

    J. Ding, V . Atanassov, E. Panichi, J. Kober, and C. Della Santina, “Robust quadrupedal jumping with impact-aware landing: Exploiting parallel elasticity,”IEEE Transactions on Robotics, vol. 40, pp. 3212– 3231, 2024

    2024

  4. [4]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,”Science Robotics, vol. 9, no. 88, p. eadi7566, 2024

    2024

  5. [5]

    Rloc: Terrain-aware legged locomotion using reinforcement learning and optimal control,

    S. Gangapurwala, M. Geisert, R. Orsolino, M. Fallon, and I. Havoutis, “Rloc: Terrain-aware legged locomotion using reinforcement learning and optimal control,”IEEE Transactions on Robotics, vol. 38, no. 5, pp. 2908–2927, 2022

    2022

  6. [6]

    Terrain- aware quadrupedal locomotion via reinforcement learning,

    H. Shi, Q. Zhu, L. Han, W. Chi, T. Li, and M. Q.-H. Meng, “Terrain- aware quadrupedal locomotion via reinforcement learning,”arXiv preprint arXiv:2310.04675, 2023

    arXiv 2023

  7. [7]

    Navigation planning for legged robots in challenging terrain,

    M. Wermelinger, P. Fankhauser, R. Diethelm, P. Kr ¨usi, R. Siegwart, and M. Hutter, “Navigation planning for legged robots in challenging terrain,” in2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2016, pp. 1184–1189

    2016

  8. [8]

    Quad-sdk: Full stack software framework for agile quadrupedal locomotion,

    J. Norby, Y . Yang, A. Tajbakhsh, J. Ren, J. K. Yim, A. Stutt, Q. Yu, N. Flowers, and A. M. Johnson, “Quad-sdk: Full stack software framework for agile quadrupedal locomotion,” inICRA Workshop on Legged Robots, 2022

    2022

Show all 32 references
  1. [9]

    High-resolution terrain map from multi- ple sensor data,

    I.-S. Kweon and T. Kanade, “High-resolution terrain map from multi- ple sensor data,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 278–292, 1992

    1992

  2. [10]

    A simultaneous localization and mapping (slam) framework for 2.5 d map building based on low-cost lidar and vision fusion,

    G. Jiang, L. Yin, S. Jin, C. Tian, X. Ma, and Y . Ou, “A simultaneous localization and mapping (slam) framework for 2.5 d map building based on low-cost lidar and vision fusion,”Applied sciences, vol. 9, no. 10, p. 2105, 2019

    2019

  3. [11]

    Estimating terrain eleva- tion maps from sparse and uncertain multi-sensor data,

    D. Belter, P. Łabcki, and P. Skrzypczy ´nski, “Estimating terrain eleva- tion maps from sparse and uncertain multi-sensor data,” in2012 IEEE International Conference on Robotics and Biomimetics. IEEE, 2012, pp. 715–722

    2012

  4. [12]

    A universal grid map library: Imple- mentation and use case for rough terrain navigation,

    P. Fankhauser and M. Hutter, “A universal grid map library: Imple- mentation and use case for rough terrain navigation,”Robot Operating System The Complete Reference (Volume 1), pp. 99–120, 2016

    2016

  5. [13]

    Probabilistic terrain mapping for mobile robots with uncertain localization,

    P. Fankhauser, M. Bloesch, and M. Hutter, “Probabilistic terrain mapping for mobile robots with uncertain localization,”IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 3019–3026, 2018

    2018

  6. [14]

    Real-time support terrain mapping and terrain adaptive local planning for quadruped robots,

    H. Xie, C. Cui, X. Zhong, X. Zhong, and Q. Liu, “Real-time support terrain mapping and terrain adaptive local planning for quadruped robots,”IEEE Robotics and Automation Letters, vol. 9, no. 12, pp. 11 018–11 025, 2024

    2024

  7. [15]

    Degradation-aware lidar-thermal-inertial slam,

    Y . Wang, Y . Liu, L. Chen, H. Chen, and S. Zhang, “Degradation-aware lidar-thermal-inertial slam,”IEEE Robotics and Automation Letters, vol. 10, no. 8, pp. 8035–8042, 2025

    2025

  8. [16]

    Firestereo: Forest infrared stereo dataset for uas depth perception in visually degraded environments,

    D. Dhrafani, Y . Liu, A. Jong, U. Shin, Y . He, T. Harp, Y . Hu, J. Oh, and S. Scherer, “Firestereo: Forest infrared stereo dataset for uas depth perception in visually degraded environments,”IEEE Robotics and Automation Letters, 2025

    2025

  9. [17]

    Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,

    G. Bledt, M. J. Powell, B. Katz, J. Di Carlo, P. M. Wensing, and S. Kim, “Mit cheetah 3: Design and control of a robust, dynamic quadruped robot,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2018, pp. 2245–2252

    2018

  10. [18]

    Contact surface estimation via haptic perception,

    H.-C. Lin and M. Mistry, “Contact surface estimation via haptic perception,” in2020 IEEE International Conference on Robotics and Automation. IEEE, 2020, pp. 5087–5093

    2020

  11. [19]

    Estimation of ground posture angle for quadruped robots based on imu,

    J. Wang, Z. Pan, Z. Niu, S. Liu, S. Zhou, and R. Wang, “Estimation of ground posture angle for quadruped robots based on imu,” in2023 6th International Conference on Electronics Technology. IEEE, 2023, pp. 1269–1275

    2023

  12. [20]

    Proprioception and tail control enable extreme terrain traversal by quadruped robots,

    Y . Yang, J. Norby, J. K. Yim, and A. M. Johnson, “Proprioception and tail control enable extreme terrain traversal by quadruped robots,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2023, pp. 735–742

    2023

  13. [21]

    A1-qp-mpc-controller: Quadruped mpc controller,

    S. Yang, “A1-qp-mpc-controller: Quadruped mpc controller,” https: //github.com/ShuoYangRobotics/A1-QP-MPC-Controller, 2021, ac- cessed: 2024-11-30

    2021

  14. [22]

    Contact model fusion for event-based locomotion in unstructured terrains,

    G. Bledt, P. M. Wensing, S. Ingersoll, and S. Kim, “Contact model fusion for event-based locomotion in unstructured terrains,” in2018 IEEE International Conference on Robotics and Automation. IEEE, 2018, pp. 4399–4406

    2018

  15. [23]

    Versatile, robust, and explosive locomotion with rigid and articulated compliant quadrupeds,

    J. Ding, P. Yang, F. Boekel, J. Kober, W. Pan, M. Saveriano, and C. Della Santina, “Versatile, robust, and explosive locomotion with rigid and articulated compliant quadrupeds,”arXiv preprint arXiv:2504.12854, 2025

    2025 arXiv

  16. [24]

    Safety- critical control of quadrupedal robots with rolling arms for autonomous inspection of complex environments,

    J. Lee, J. Kim, W. Ubellacker, T. G. Molnar, and A. D. Ames, “Safety- critical control of quadrupedal robots with rolling arms for autonomous inspection of complex environments,” in2024 IEEE International Conference on Robotics and Automation. IEEE, 2024, pp. 3485– 3491

    2024

  17. [25]

    Integrated task and motion planning for safe legged navigation in partially observable environments,

    A. Shamsah, Z. Gu, J. Warnke, S. Hutchinson, and Y . Zhao, “Integrated task and motion planning for safe legged navigation in partially observable environments,”IEEE Transactions on Robotics, vol. 39, no. 6, pp. 4913–4934, 2023

    2023

  18. [26]

    Agile but safe: Learning collision-free high-speed legged locomotion,

    T. He, C. Zhang, W. Xiao, G. He, C. Liu, and G. Shi, “Agile but safe: Learning collision-free high-speed legged locomotion,”arXiv preprint arXiv:2401.17583, 2024

    2024 arXiv

  19. [27]

    Learning risk-aware quadrupedal locomotion using distributional reinforcement learning,

    L. Schneider, J. Frey, T. Miki, and M. Hutter, “Learning risk-aware quadrupedal locomotion using distributional reinforcement learning,” in2024 IEEE International Conference on Robotics and Automation. IEEE, 2024, pp. 11 451–11 458

    2024

  20. [28]

    Safe learning in robotics: From learning-based control to safe reinforcement learning,

    L. Brunke, M. Greeff, A. W. Hall, Z. Yuan, S. Zhou, J. Panerati, and A. P. Schoellig, “Safe learning in robotics: From learning-based control to safe reinforcement learning,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 5, no. 1, pp. 411–444, 2022

    2022

  21. [29]

    Multi- layered safety for legged robots via control barrier functions and model predictive control,

    R. Grandia, A. J. Taylor, A. D. Ames, and M. Hutter, “Multi- layered safety for legged robots via control barrier functions and model predictive control,” in2021 IEEE International Conference on Robotics and Automation. IEEE, 2021, pp. 8352–8358

    2021

  22. [30]

    A control barrier function-based motion planning scheme for a quadruped robot,

    H. U. Unlu, V . M. Gonc ¸alves, D. Chaikalis, A. Tzes, and F. Khorrami, “A control barrier function-based motion planning scheme for a quadruped robot,” in2024 IEEE International Conference on Robotics and Automation. IEEE, 2024, pp. 12 172–12 178

    2024

  23. [31]

    Body- aware local navigation for asymmetric holonomic robots using control barrier functions,

    A. Saradagi, S. Fredriksson, A. Koval, and G. Nikolakopoulos, “Body- aware local navigation for asymmetric holonomic robots using control barrier functions,” in2024 European Control Conference. IEEE, 2024, pp. 968–973

    2024

  24. [32]

    Autonomous and safety-critical stair climbing via nonlinear model predictive control for quadrupedal robots,

    C. Li, X. Peng, W. Lan, and X. Yu, “Autonomous and safety-critical stair climbing via nonlinear model predictive control for quadrupedal robots,” in2023 IEEE International Conference on Robotics and Biomimetics. IEEE, 2023, pp. 1–6

    2023

Pith tools

Reviewed July 15, 2026 · model on record in the stance chip above.