REVIEW 3 cited by
Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Last-Iterate Complexity of SGD for Convex and Smooth Stochastic Problems
read the original abstract
Most results on Stochastic Gradient Descent (SGD) in the convex and smooth setting are presented under the form of bounds on the ergodic function value gap. It is an open question whether bounds can be derived directly on the last iterate of SGD in this context. Recent advances suggest that it should be possible. For instance, it can be achieved by making the additional, yet unverifiable, assumption that the variance of the stochastic gradients is uniformly bounded. In this paper, we show that there is no need of such an assumption, and that SGD enjoys a $\tilde O \left( T^{-1/2} \right)$ last-iterate complexity rate for convex smooth stochastic problems.
Forward citations
Cited by 3 Pith papers
-
Gradient Descent's Last Iterate is Often (slightly) Suboptimal
Proves it is impossible to achieve optimal last-iterate rates for GD and SGD without knowing the horizon T in advance, incurring an unavoidable poly-log factor penalty even in the deterministic case.
-
Stochastic Krasnoselskii-Mann Iterations: Convergence without Uniformly Bounded Variance
Stochastic Krasnoselskii-Mann iterations converge almost surely weakly with finite variance only at a single fixed point, recovering best-known rates without uniform variance bounds.
-
Stochastic Krasnoselskii-Mann Iterations: Convergence without Uniformly Bounded Variance
Stochastic Krasnoselskii-Mann iterations converge almost surely and with rates under finite variance at a single fixed point rather than uniform variance bounds, recovering optimal complexity and providing first such ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.