REVIEW 2 cited by
Weighted Averaged Stochastic Gradient Descent: Asymptotic Normality and Optimality
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Stochastic Gradient Descent (SGD) is one of the most popular algorithms in statistical and machine learning due to its computational and memory efficiency. Various averaging schemes have been proposed to accelerate the convergence of SGD in different settings. In this paper, we explore a general averaging scheme for SGD. Specifically, we establish the asymptotic normality of a broad range of weighted averaged SGD solutions and provide asymptotically valid online inference approaches. Furthermore, we propose an adaptive averaging scheme that exhibits both optimal statistical rate and favorable non-asymptotic convergence, drawing insights from the optimal weight for the linear model in terms of non-asymptotic mean squared error (MSE).
Forward citations
Cited by 2 Pith papers
-
Sharp asymptotic theory for Q-learning with LDTZ learning rate and its generalization
Q-learning with PD2Z/LD2Z step sizes admits sharp non-asymptotic bounds, a tail Polyak–Ruppert CLT, and a time-uniform Gaussian approximation, establishing a best-of-both-worlds rate-and-bias tradeoff.
-
Online Statistical Inference of Constrained Stochastic Optimization via Random Scaling
A random scaling statistic based on averaged AI-SSQP iterates is asymptotically pivotal for constrained stochastic optimization, enabling matrix-free online confidence intervals.
Discussion (0). Continue with ORCID to comment.