Momentum SGD exhibits two distinct EoSS regimes for batch sharpness, stabilizing at 2(1-β)/η for small batches and 2(1+β)/η for large batches, aligning with linear stability thresholds.
When and why momentum accelerates sgd: An empirical study
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
citation-role summary
method 1
citation-polarity summary
years
2026 6verdicts
UNVERDICTED 6roles
method 1polarities
use method 1representative citing papers
Lower bounds establish that heavy-ball momentum extends the compute-efficient batch-size window by sqrt(kappa) over SGD in linear regression, with accelerated SGD showing spectrum-dependent CE-serial runtime tradeoffs.
Classical momentum acceleration in mini-batch SGD for quadratics is proportional to batch size up to saturation, enabling perfect parallelization under minimal noise assumptions.
Instruction understanding is reframed as an evolving Instruction-as-State variable conditioned on perceptual state and realized via the S-EGIU coarse-to-fine framework, reporting a +2.68% SPL gain on REVERIE Test Unseen.
MGUP augments momentum optimizers with selective larger steps on a fixed proportion of parameters per iteration, claiming convergence guarantees for MGUP-AdamW and superior empirical performance on pretraining and fine-tuning.
SDB balances behavioral diversity and learning stability in VLN self-improvement by expanding decisions into latent hypotheses, performing reliability-aware aggregation, and applying a regularizer, yielding gains such as SPL 33.73 to 35.93 on REVERIE val-unseen.
citing papers explorer
-
Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
Momentum SGD exhibits two distinct EoSS regimes for batch sharpness, stabilizing at 2(1-β)/η for small batches and 2(1+β)/η for large batches, aligning with linear stability thresholds.
-
Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods
Lower bounds establish that heavy-ball momentum extends the compute-efficient batch-size window by sqrt(kappa) over SGD in linear regression, with accelerated SGD showing spectrum-dependent CE-serial runtime tradeoffs.
-
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
Classical momentum acceleration in mini-batch SGD for quadratics is proportional to batch size up to saturation, enabling perfect parallelization under minimal noise assumptions.
-
Instruction-as-State: Environment-Guided and State-Conditioned Semantic Understanding for Embodied Navigation
Instruction understanding is reframed as an evolving Instruction-as-State variable conditioned on perceptual state and realized via the S-EGIU coarse-to-fine framework, reporting a +2.68% SPL gain on REVERIE Test Unseen.
-
MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization
MGUP augments momentum optimizers with selective larger steps on a fixed proportion of parameters per iteration, claiming convergence guarantees for MGUP-AdamW and superior empirical performance on pretraining and fine-tuning.
-
The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigation
SDB balances behavioral diversity and learning stability in VLN self-improvement by expanding decisions into latent hypotheses, performing reliability-aware aggregation, and applying a regularizer, yielding gains such as SPL 33.73 to 35.93 on REVERIE val-unseen.