REVIEW 3 cited by
Unified Optimal Analysis of the (Stochastic) Gradient Method
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
In this note we give a simple proof for the convergence of stochastic gradient (SGD) methods on $\mu$-convex functions under a (milder than standard) $L$-smoothness assumption. We show that for carefully chosen stepsizes SGD converges after $T$ iterations as $O\left( LR^2 \exp \bigl[-\frac{\mu}{4L}T\bigr] + \frac{\sigma^2}{\mu T} \right)$ where $\sigma^2$ measures the variance in the stochastic noise. For deterministic gradient descent (GD) and SGD in the interpolation setting we have $\sigma^2 =0$ and we recover the exponential convergence rate. The bound matches with the best known iteration complexity of GD and SGD, up to constants.
Forward citations
Cited by 3 Pith papers
-
Decoupled SGDA for Games with Intermittent Strategy Communication
Decoupled SGDA achieves O(1/(1-4κ_c) log(1/ϵ)) communication rounds in weakly coupled SCSC games, independent of the players' condition numbers.
-
Theoretical Foundations of Communication-Efficient, Robust, and Practical Distributed and Federated Optimization
A thesis proving communication-acceleration guarantees for local-step, compressed, Byzantine-robust, and low-rank federated optimization methods, assembled from the author's own published papers.
-
What Makes Local Updates Effective: The Role of Data Heterogeneity and Smoothness
Under bounded second-order heterogeneity, local updates are shown to achieve faster convergence than mini-batch SGD in several convex and non-convex regimes, with matching lower bounds.
Discussion (0). Continue with ORCID to comment.