Adam's adaptive preconditioning and first-moment averaging improve high-probability tracking error in noise-dominated nonstationary regimes but can increase it under strong drift, where SGD achieves a smaller floor, with explicit beta-dependent bounds.
Adahessian: An adaptive second order optimizer for machine learning.Proceedings of the AAAI Conference on Artificial Intelligence, 35(12):10665–10673, May 2021
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Systematic benchmark of PINN architectures on 1D stiff PNP system finds BRDR loss weighting competitive with NTK at lower wall-clock time.
citing papers explorer
-
Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization
Adam's adaptive preconditioning and first-moment averaging improve high-probability tracking error in noise-dominated nonstationary regimes but can increase it under strong drift, where SGD achieves a smaller floor, with explicit beta-dependent bounds.
-
A Systematic Benchmark of Physics-Informed Neural Network Architectures for the Stiff Poisson-Nernst-Planck System: Adaptive LossWeighting and Multi-Scale Resolution
Systematic benchmark of PINN architectures on 1D stiff PNP system finds BRDR loss weighting competitive with NTK at lower wall-clock time.