Averaged SA-Adam iterates have exactly the SGD Polyak-Ruppert covariance H^{-1} S H^{-1} under sub-linear momentum gain and local stabilization.
Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality
5 Pith papers cite this work. Polarity classification is still indexing.
5
Pith papers citing it
verdicts
UNVERDICTED 5representative citing papers
Dynamic preconditioning preserves the Polyak-Ruppert CLT for averaged SGD if the preconditioner stabilizes at rate β > (α + 1)/2.
Develops a batch-free online covariance estimator for sketched Newton methods, proves its consistency, and demonstrates use for online statistical inference on regression and CUTEst problems.
Proposes Factor-Augmented SGD that runs on streaming high-dimensional data and supplies the first convergence analysis explicitly accounting for latent-factor estimation error.
Classical momentum acceleration in mini-batch SGD for quadratics is proportional to batch size up to saturation, enabling perfect parallelization under minimal noise assumptions.
citing papers explorer
-
A Polyak-Ruppert Central Limit Theorem for SA-Adam with Momentum and Non-Convergent Adaptive Preconditioning
Averaged SA-Adam iterates have exactly the SGD Polyak-Ruppert covariance H^{-1} S H^{-1} under sub-linear momentum gain and local stabilization.
-
When Does Dynamic Preconditioning Preserve the Polyak-Ruppert CLT? A Stabilization Threshold
Dynamic preconditioning preserves the Polyak-Ruppert CLT for averaged SGD if the preconditioner stabilizes at rate β > (α + 1)/2.
-
Online Covariance Matrix Estimation in Sketched Newton Methods
Develops a batch-free online covariance estimator for sketched Newton methods, proves its consistency, and demonstrates use for online statistical inference on regression and CUTEst problems.
-
Factor Augmented High-Dimensional SGD
Proposes Factor-Augmented SGD that runs on streaming high-dimensional data and supplies the first convergence analysis explicitly accounting for latent-factor estimation error.
-
Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
Classical momentum acceleration in mini-batch SGD for quadratics is proportional to batch size up to saturation, enabling perfect parallelization under minimal noise assumptions.