REVIEW 3 cited by
Normalized Gradients for All
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this short note, I show how to adapt to H\"{o}lder smoothness using normalized gradients in a black-box way. Moreover, the bound will depend on a novel notion of local H\"{o}lder smoothness. The main idea directly comes from Levy [2017].
Forward citations
Cited by 3 Pith papers
-
SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration
A single proof unifies convergence analyses of AdaGrad-Norm, AdaGrad, ASGO, and DASGO under Hölder smoothness, and shows AdaGrad/DASGO can be accelerated with Nesterov momentum.
-
Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization
Accelerated GRAAL is the first adaptive first-order method that proves near-optimal accelerated complexity for convex L-smooth and (L0,L1)-smooth functions with geometric stepsize growth.
-
AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates
The paper proposes AdaGO, a Muon variant with a scalar AdaGrad-Norm step size, and proves optimal nonconvex convergence rates while reporting empirical gains on regression and CIFAR-10.
Discussion (0). Sign in to comment.