REVIEW 11 cited by
Revisiting the Polyak step size
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Revisiting the Polyak step size
read the original abstract
This paper revisits the Polyak step size schedule for convex optimization problems, proving that a simple variant of it simultaneously attains near optimal convergence rates for the gradient descent algorithm, for all ranges of strong convexity, smoothness, and Lipschitz parameters, without a-priory knowledge of these parameters.
Forward citations
Cited by 11 Pith papers
-
Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
LE-SAM inverts SAM by fixing the loss budget instead of the parameter-space radius, yielding better generalization across benchmarks.
-
A short proof of near-linear convergence of adaptive gradient descent under fourth-order growth and convexity
A direct Lyapunov argument establishes near-linear convergence of a more adaptive gradient descent method for convex functions satisfying fourth-order growth, bypassing an earlier intricate ravine-monitoring proof.
-
Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients
A safeguarded stochastic Polyak step size, SPS_safe, yields O(1/√T) convergence to a neighborhood for convex non-smooth problems without interpolation or oracle loss values, with a momentum variant.
-
Taking the Road Less Scheduled with Adaptive Polyak Steps
Polyak-style step sizes for Schedule-Free SGD and Adam achieve O(1/sqrt(t)) anytime last-iterate rates for convex Lipschitz problems using per-iteration loss and gradient information.
-
Stein Diffusion Guidance: Training-Free Posterior Correction for Sampling Beyond High-Density Regions
Stein Diffusion Guidance corrects approximate posteriors in diffusion sampling via a Stein variational mechanism and surrogate SOC objective to enable effective guidance beyond high-density regimes.
-
Uniformly Optimal and Parameter-free First-order Methods for Convex and Function-constrained Optimization
Parameter-free first-order methods attain optimal oracle complexity O(ε^{-2/(1+3ρ)}) for convex function-constrained optimization under Hölder smoothness by combining modified Polyak steps, Nesterov momentum, and APL ...
-
Normalized First-Order Methods for Convex (L0, L1)-Smooth Optimization with Inexact Gradients
Comparison-oracle variants of NGD and Polyak GD converge for convex (L0, L1)-smooth objectives when the normalized-gradient error δ is bounded by explicit O(√ε)-scale thresholds.
-
Restart and Adaptive Acceleration in Stochastic Gradient Methods
Restart schemes for SGD on KL-satisfying non-smooth weakly convex problems deliver accelerated convergence robust to exponent misspecification, with optimal schedules resembling Polyak steps.
-
Large-scale empirical tuning and comparison of default optimizers for variational inference
Large empirical study of 56 optimizers on 1092 BBVI tasks finds no single winner but a selection of five suffices for near-best performance.
-
Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
Proposes Polyak schedulers for SAM with convergence proofs in deterministic and stochastic settings and empirical results showing reduced tuning needs.
-
Introduction to stochastic gradient methods
Lecture notes on convergence theory for deterministic gradient descent and stochastic gradient methods under standard assumptions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.