REVIEW 3 cited by
Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers, we consider differential equations which model the behavior of accelerated gradient descent. Our main contributions are to provide a general framework for discretizating the differential equations to produce accelerated methods, and to provide physical intuition which helps explain the optimal damping rate. An important novelty is the generality of our approach, which leads to a unified derivation of a wide variety of methods, including versions of Nesterov's accelerated gradient descent, FISTA, and accelerated coordinate descent.
Forward citations
Cited by 3 Pith papers
-
Heavy-ball dynamics with Hessian-driven damping for non-convex optimization under the {\L}ojasiewicz condition
For non-convex objectives satisfying the Łojasiewicz inequality of order 2, the DIN continuous-time dynamics converge in function value at a rate arbitrarily close to e^{-2√µ t}, which is worst-case optimal within thi...
-
Inertial Primal Dual Dynamics with Hessian-driven Damping for Saddle Point Problems
New inertial primal-dual ODEs with Hessian damping achieve O(1/t²) convex rates and O(1/t^{α−1}) strongly-convex rates without knowing the strong convexity moduli.
-
Continuized Nesterov Momentum Achieves the $O(\varepsilon^{-7/4})$ Complexity in Smooth Nonconvex Optimization
The paper proves a weighted, event-restricted O(ε^{-7/4}) complexity bound for a continuized Nesterov momentum algorithm without safeguard mechanisms; the restriction is not rigorously quantified.
Discussion (0). Continue with ORCID to comment.