REVIEW 5 cited by
A Variational Perspective on Accelerated Methods in Optimization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
A Variational Perspective on Accelerated Methods in Optimization
read the original abstract
Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelerated methods from a continuous-time perspective. We show that there is a Lagrangian functional that we call the \emph{Bregman Lagrangian} which generates a large class of accelerated methods in continuous time, including (but not limited to) accelerated gradient descent, its non-Euclidean extension, and accelerated higher-order gradient methods. We show that the continuous-time limit of all of these methods correspond to traveling the same curve in spacetime at different speeds. From this perspective, Nesterov's technique and many of its generalizations can be viewed as a systematic way to go from the continuous-time curves generated by the Bregman Lagrangian to a family of discrete-time accelerated algorithms.
Forward citations
Cited by 5 Pith papers
-
Momentum Further Constrains Sharpness at the Edge of Stochastic Stability
Momentum SGD exhibits two distinct EoSS regimes for batch sharpness, stabilizing at 2(1-β)/η for small batches and 2(1+β)/η for large batches, aligning with linear stability thresholds.
-
Optimization Geometrodynamics: A Framework for Dynamic Geometric Optimization
Dynamic geometric complexity of reducing condition number under full SPD metric control equals the affine-invariant distance from the relative log-spectrum to a low-width set.
-
First-Order Methods for Solving Convex (Strongly) Concave Minimax Problems with Functional Constraints
PALM achieves Õ(ε^{-1}) first-order complexity for ε-KKT points in convex-strongly-concave minimax problems with functional constraints and Õ(ε^{-3/2}) for the dual in the convex-concave case.
-
Geometric--Nongeometric Optimizer Calculus: A Modular Language for Reachable Gradient Methods
A modular calculus decomposes optimizer updates into geometric preconditioning plus structured nongeometric mechanisms, with a direction-expressivity theorem showing full SPD geometry captures exactly strict descent d...
-
Restricted Dynamic Geometric Complexity: Certificates for Structured Preconditioning
Restricted dynamic geometric complexity measures the intrinsic affine-invariant path distance from an initial metric to a condition-number target when the metric family is structurally constrained, with exact LMI and ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.