Mean-square linear stability of two-point ZO methods is governed by the full Hessian spectrum and admits explicit bounds in terms of trace and top eigenvalue; full-batch ZO-GD/GDM/Adam empirically operate at that boundary.
Edge of stochastic stability: Revisiting the edge of stability for sgd
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
citation-role summary
extension 1
citation-polarity summary
fields
cs.LG 6years
2026 6roles
extension 1polarities
extend 1representative citing papers
Momentum SGD exhibits two distinct EoSS regimes for batch sharpness, stabilizing at 2(1-β)/η for small batches and 2(1+β)/η for large batches, aligning with linear stability thresholds.
Large loss spikes in SGD are polynomially likely and serve as the dominant mechanism for escaping sharp minima toward flatter solutions in the NTK regime.
Edge of stability acts as a selective mechanism that amplifies learning on data groups with aligned persistent gradients while suppressing others.
Weight decay slows progressive sharpening at the edge of stability, inducing damped oscillations in CNNs and a phase transition to sub-2/η sharpness in MLPs driven by parameter-sharpness gradient alignment, yielding more stable NTK dynamics.
Training at the edge of stability causes neural network optimizers to converge on fractal attractors whose effective dimension, measured via a new sharpness dimension from the Hessian spectrum, bounds generalization error in a way not captured by prior trace or norm measures.
citing papers explorer
-
Zeroth-Order Optimization at the Edge of Stability
Mean-square linear stability of two-point ZO methods is governed by the full Hessian spectrum and admits explicit bounds in terms of trace and top eigenvalue; full-batch ZO-GD/GDM/Adam empirically operate at that boundary.
-
Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
Momentum SGD exhibits two distinct EoSS regimes for batch sharpness, stabilizing at 2(1-β)/η for small batches and 2(1+β)/η for large batches, aligning with linear stability thresholds.
-
Large Spikes in Stochastic Gradient Descent: A Large-Deviations View
Large loss spikes in SGD are polynomially likely and serve as the dominant mechanism for escaping sharp minima toward flatter solutions in the NTK regime.
-
Edge of Stability Selectively Shapes Learning Across the Data Distribution
Edge of stability acts as a selective mechanism that amplifies learning on data groups with aligned persistent gradients while suppressing others.
-
Does Weight Decay Enhance Training Stability?
Weight decay slows progressive sharpening at the edge of stability, inducing damped oscillations in CNNs and a phase transition to sub-2/η sharpness in MLPs driven by parameter-sharpness gradient alignment, yielding more stable NTK dynamics.
-
Generalization at the Edge of Stability
Training at the edge of stability causes neural network optimizers to converge on fractal attractors whose effective dimension, measured via a new sharpness dimension from the Hessian spectrum, bounds generalization error in a way not captured by prior trace or norm measures.