REVIEW 2 major objections 6 minor 37 references
A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients
T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A speed-restart rule makes a Hessian-damped inertial system converge linearly in objective value for any convex, smooth function satisfying the Polyak-Lojasiewicz inequality, with no strong convexity assumption.
desk verdict The PL-extension is real and the proof structure is sound, but Proposition 3.8 contains a load-bearing algebraic inversion that breaks the stated theorem as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by three quantitative objects: the restart time $T(z)=\inf\{t>0: \frac{d}{dt}\|x'(t)\|^2 \le 0\}$; an auxiliary function $H(t)$, defined in (3.5) as $H(t)=1+\frac{2L\gamma}{\alpha^2}-\frac{L\beta}{\alpha}-\frac{Lt(\frac{\gamma}{\alpha}e^{\alpha t}+\frac{\gamma}{\alpha}-\beta)}{e^{\alpha t}-1}$, which decreases from 1 to $-\infty$ and whose zeros $\tau_1$, $\tau_2$ mark the time scale on which the frozen-gradient bounds are valid; and a companion function $G(t)$ whose zero $\tau_3$ gives the uniform lower bound on $T(z)$. The proof also uses the supremum $M_z(t)=\sup_{u\in(0,t]} \|x'(u)\|/(1-e^{-\alpha u})$ to bound the integral terms $I_z(t)$ and $J_z(t)$ defined in (3.3). Lemma 3.1 bounds those integrals by $L$ times $M_z(t)$ times explicit functions of $t$; Lemma 3.5 shows $\langle x'(t),x''(t)\rangle$ is at least $\gamma^2(e^{\alpha t}-1)\|\nabla f(z)\|^2/(\alpha e^{2\alpha t} H(t)^2) G(t)$, so while $H,G$ stay positive the speed is still increasing and $T(z)$ has not been reached. Proposition 3.8 uses the PL inequality to bound $T(z)$ from above, and Proposition 3.9 integrates a lower bound on the speed over $[0,\tau]$ to get the per-restart factor $Q$. In short, $H$ and $G$ control the kinematics of the restart, and the PL inequality converts those kinematics into progress on the objective.
What would settle it
Integrate the system for $f(x)=\frac{1}{2}\|x\|^2$, $\alpha=3$, $\beta=0$, $\gamma=1$, from $z=(1,0,\ldots,0)$, and locate the first time $T(z)$ at which $\frac{d}{dt}\|x'(t)\|^2$ changes sign; Theorem 2.2 requires $T(z)$ to lie between the paper's $\tau_*$ and $\tau^*$ and the first restart to shrink the objective gap by at most the factor $Q$ from Proposition 3.9. A violation of either check, or an analogous check on any convex $C^2$, $L$-smooth PL function, would refute the claimed uniform linear convergence.
Extended reading notes
Core claim
The central discovery is that the speed-restart mechanism, previously known to give linear convergence for strongly convex objectives, works under the strictly weaker Polyak-Lojasiewicz condition. For the system (2.2) with any coefficients $\alpha>0$, $\beta\ge0$, $\gamma>0$, the paper constructs explicit positive constants $\tau_*$ and $\tau^*$ such that every restart interval has length between them, independent of the starting point, and proves that each restart reduces the function-value gap by a uniform factor $Q\in(0,1)$. Iterating the decrease factor and using the uniform time bounds yields the exponential estimate of Theorem 2.2. The technical route rewrites the ODE as an integral equation, controls the gradient and Hessian terms through a supremum of the normalized speed, and then converts a lower bound on $\langle x', x''\rangle$ into the restart-time bounds; the Polyak-Lojasiewicz inequality is what turns a lower speed bound into a quantified gap decrease. A corollary is that the same linear-rate statement covers the Heavy Ball case $\beta=0$, so the Hessian term is not needed for the rate, only for damping oscillations.
Load-bearing premise
The argument rests on the Polyak-Lojasiewicz inequality $2\mu(f(z)-f^*) \le \|\nabla f(z)\|^2$ holding with a known positive $\mu$; if that inequality fails or $\mu$ is unknown, the upper bound on the restart time and the fixed per-restart decrease both disappear, and the linear-rate proof collapses.
Editorial extensions
If this is right
- For every convex $C^2$, $L$-smooth $f$ satisfying the PL inequality with some $\mu>0$, the speed-restarted trajectory of (WIN) satisfies $f(x(t))-f^* \le C e^{-Kt}(f(z)-f^*)$ with constants $C,K$ independent of the starting point $z$.
- Because $\beta$ is allowed to be zero, the result applies to the classical Heavy Ball dynamics, giving linear convergence under PL rather than only under strong convexity.
- The uniform bounds $\tau_* \le T(z) \le \tau^*$ imply the number of restarts up to time $t$ grows linearly in $t$, so the exponential decay rate is governed by the worst-case per-restart decrease factor $Q$.
- Since strong convexity implies the PL inequality, the theorem covers every strongly convex smooth objective and extends the earlier speed-restart analyses to non-strongly-convex PL objectives.
- In the numerical experiments, the speed-restart rule improves the fitted exponential decay rate $B$ for both the continuous system (with and without Hessian damping) and for a discretized inertial gradient algorithm; the warm-start variant does not improve the slope.
Reading between the lines
- If the PL constant $\mu$ is unknown, the explicit upper bound on the restart time cannot be computed; an adaptive scheme that estimates $\mu$ online while running the dynamics is a natural extension that the paper does not analyze.
- The continuous-time theorem suggests that any inertial algorithm whose continuum limit is (WIN) should inherit linear convergence under PL when restarted at speed-stall events; the paper tests only one discretization, so whether other standard accelerated-gradient discretizations obey the same discrete rate remains open.
- A testable prediction of the proof is that conditioning enters the rate only through $L$ and $\mu$, not through the starting point; one could verify this by fitting $C$ and $K$ on a family of quadratic PL functions with increasing condition numbers and comparing them with the theorem's constants.
- The discrete restart condition used in Algorithm 1, $\|x_{k+1}-x_k\| < \|x_k-x_{k-1}\|$, is a natural proxy for the continuous speed-stall condition; the continuous analysis provides a heuristic justification for it, not a proof for the discrete method.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper analyzes the weighted inertial Newton-like system (2.2) with constant coefficients and proposes a speed-restart scheme in which the dynamics are re-initialized whenever the speed stops increasing. The main claim, Theorem 2.2, is a linear convergence rate for the objective values along the restarted trajectory under convexity, C^2 regularity, L-smoothness, and the Polyak-Łojasiewicz inequality (2.1), with constants independent of the starting point. The proof is organized through estimates on the restart time and on the per-restart function decrease (Lemmas 3.1–3.5, Propositions 3.6–3.9), followed by numerical illustrations.
Significance. If the main theorem is correct, the paper extends speed-restart linear-convergence results, previously known for strongly convex objectives in the setting of Su-Boyd-Candès and for Hessian-driven damping variants, to the wider Polyak-Łojasiewicz class. The paper is self-contained, gives explicit constants, and covers both β=0 and β>0. The numerical experiments support the qualitative benefit of restarting, although they are not the main contribution. The main issues are in the technical proof of the restart-time lower bound and in the uniform definition of the restart-time upper bound.
major comments (2)
- [Lemma 3.5 (Section 3.2)] The lower bound for ⟨x',x''⟩ is not valid for all t∈(0,τ1). In the expansion of ⟨x',A(t)⟩, the cross term is γ(2−e^{αt})⟨I_z(t)+J_z(t),∇f(z)⟩/e^{2αt}. When e^{αt}<2 the coefficient 2−e^{αt} is positive, so the correct lower bound is −γ(2−e^{αt})‖I_z(t)+J_z(t)‖‖∇f(z)‖/e^{2αt}, which is negative, whereas the manuscript replaces it by the positive quantity −γ²(e^{αt}−2)(e^{αt}−1)(1−H)/H/(αe^{2αt})‖∇f(z)‖². A direct check with f(x)=x²/2, α=3, β=0, γ=1, z=1, and t=0.1 gives ⟨x'(t),x''(t)⟩≈0.06353, while the right-hand side of Lemma 3.5 evaluates to approximately 0.06366, so the lemma is false as stated. Since Corollary 3.7 and the lower restart-time bound in (2.4) depend on Lemma 3.5, the proof of Theorem 2.2 is incomplete as written.
- [Section 2 and Proposition 3.8] The quantity used as an upper restart-time bound in the proof of Theorem 2.2 is not defined independently of the starting point. After Proposition 3.8 the text defines τ* as an argmin over (0,τ2)∩(0,T(z)], a set that depends on z, while the proof of Theorem 2.2 counts restarts using a single τ* valid for all z. The authors should define τ* = argmin over (0,τ2) of φ(τ)=τ+α/(2µγ(1−e^{−ατ})²Ψ(τ)) and then show uniform T(z)≤τ* by the two-case argument: if τ*≤T(z) use Proposition 3.8, and if τ*>T(z) the inequality is trivial. The same issue affects the use of Proposition 3.9: the per-restart factor Q must be chosen uniformly and shown to lie in (0,1), for instance by taking a fixed sufficiently small τ in (0,min(τ2,τ3)).
minor comments (6)
- [Proposition 3.8] The displayed bound T(z) ≤ τ + α/(2µγ(1−e^{−ατ})²Ψ(τ)) is ambiguous in the supplied text; the proof shows that (1−e^{−ατ})²Ψ(τ) should be in the denominator. Please ensure the typeset formula is unambiguous.
- [Section 2] The lower and upper restart-time bounds are both denoted by symbols that look identical in the plain text (τ∗ and τ∗). Please distinguish τ_* and τ^* clearly.
- [Table II] Table II has no caption, and one entry reads “3.39-e2”, which appears to be a typo for 3.39e−2.
- [Section 4] In Table III the regression is presented as f(x_k)∼Ae^{−Bt}, but the data are discrete in k; the notation should be f(x_k)∼Ae^{−Bk}.
- [Algorithm 1] In the speed-restart branch the pseudocode sets xk = xk−1 when the norm of the step decreases. Please clarify whether this is intentional or whether the intended reset is to the preceding velocity/iterate state.
- [Section 2] There is a typo “Propositoin” in the sentence before Theorem 2.2, and the phrase “Polyak-/suppress Lojasiewicz” should read “Polyak–Łojasiewicz”.
Circularity Check
No significant circularity: the linear-rate theorem is derived from stated assumptions on f, and prior-work citations are contextual, not load-bearing.
full rationale
The derivation of Theorem 2.2 is self-contained from Section 3. The proof requires (i) the lower restart-time bound Corollary 3.7, (ii) the upper restart-time bound Proposition 3.8, and (iii) the per-restart function decrease Proposition 3.9; each is proved from the dynamics (2.2), the Polyak-Lojasiewicz inequality (2.1), and the smoothness assumptions, with no parameter fitted to data and no external result imported. The constants C and K are explicit functions of α, β, γ, μ, and L through these estimates. Citations to the authors' prior work appear only as context, comparisons of numerical behavior, and earlier restart analyses; none is needed to justify a step of the proof. The numerical A and B fits in Section 4 report observed behavior and are not fed back into the theorem, so they do not constitute a fitted input called a prediction. The only substantive concern raised by a skeptical reader, the algebraic inversion in Proposition 3.8's upper-bound expression, would if sustained be a mathematical error affecting the validity of the uniform upper restart-time bound and hence Theorem 2.2; however, an erroneous algebraic manipulation is not a circularity, since it is not an equivalence-by-construction, a fitted-parameter renaming, or a self-citation chain. No step in the paper defines its conclusion into its hypotheses.
Assumptions & free parameters
assumptions (2)
- domain assumption f is convex, C^2, attains its minimum f*, has L-Lipschitz gradient, and satisfies 2mu(f(z)-f*) <= ||f'(z)||^2 for all z and some mu>0.
- standard math The second-order ODE (2.2) has a unique global solution for each initial condition (z,0), and the gluing of restarted arcs produces a continuous piecewise differentiable trajectory.
Cite this review
Pith. "Pith review of A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients." pith.science (2026). https://pith.science/paper/Y7N7MFVE
@misc{pith2026241206691,
author = {Pith},
title = {Pith review of: A Speed Restart Scheme for a Dynamical System with Hessian-Driven Damping and Three Constant Coefficients},
year = {2026},
howpublished = {\url{https://pith.science/paper/Y7N7MFVE}},
note = {Machine review of arXiv:2412.06691}
}
read the original abstract
In this paper, we study a speed restart scheme for an inertial system with Hessian-driven damping. We establish a linear convergence rate for the function values along the restarted trajectories without assuming the strong convexity of the objective function. Our numerical experiments show improvements in the convergence rates, both for the continuous-time dynamics, and when applied to inertial algorithms as a heuristic
Figures
Reference graph
Works this paper leans on
- [1]
- [2]
- [3]
- [4]
-
[5]
F. ´Alvarez, H. Attouch, J. Bolte, and P. Redont. A second-order gr adient-like dissipative dynamical system with Hessian-driven damping.: Application to optimiz ation and mechanics. Journal de Math´ ematiques Pures et Appliqu´ ees, 81:747–779, 2002
work page 2002
-
[6]
F. ´Alvarez and J. P´ erez. A dynamical system associated with Newton’s method for parametric approximations of convex minimization problems. Applied Mathematics and Optimization , 38:193–217, 1998
work page 1998
-
[7]
D. Applegate, O. Hinder, H. Lu, and M. Lubin. Faster first-order primal-dual methods for linear programming using restarts and sharpness. Mathematical Programming, 201(1):133–184, 2023
work page 2023
-
[8]
H. Attouch, A. Balhag, Z. Chbani, and H. Riahi. Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling. Evolution Equations and Control Theroy, 11(2):487–514, 2022
work page 2022
Show all 37 references
-
[9]
Attouch, Z
H. Attouch, Z. Chbani, J. Fadili, and H. Riahi. First-order optimiza tion algorithms via inertial systems with Hessian driven damping. Mathematical Programming, 193:113–155, 2022
2022
-
[10]
Attouch, Z
H. Attouch, Z. Chbani, J. Peypouquet, and P. Redont. Fast c onvergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Mathematical Programming, 168:123–175, 2018
2018
-
[11]
Attouch, Z
H. Attouch, Z. Chbani, and H. Riahi. Fast convex optimization via time scaling of damped inertial gradient dynamics. Pure and Applied Functional Analysis , 2019
2019
-
[12]
Attouch, Z
H. Attouch, Z. Chbani, and H. Riahi. Fast proximal methods via t ime scaling of damped inertial dynamics. SIAM Journal on Optimization , 29:2227–2256, 2019
2019
-
[13]
Attouch and J
H. Attouch and J. Peypouquet. The rate of convergence of n esterov’s accelerated forward- backward method is actually faster than 1/kˆ2. SIAM Journal on Optimization , 26(3):1824– 1834, 2016. 16
2016
-
[14]
Attouch, J
H. Attouch, J. Peypouquet, and P. Redont. Fast convex opt imization via inertial dynamics with Hessian driven damping. Journal of Differential Equations , 261(10):5734–5783, 2016
2016
-
[15]
Aujol, C
J.-F. Aujol, C. H. Dossal, H. Labarri` ere, and A. Rondepierre. FISTA restart using an automatic estimation of the growth parameter. hal-03153525v4, 2022
2022
-
[16]
Drori and M
Y. Drori and M. Teboulle. Performance of first-order method s for smooth convex minimization: a novel approach. Mathematical Programming, 145:451–482, 2014
2014
-
[17]
Fercoq and Z
O. Fercoq and Z. Qu. Adaptive restart of accelerated gradien t methods under local quadratic growth condition. IMA Journal of Numerical Analysis , 39(4):2069–2095, 2019
2019
-
[18]
Giselsson and S
P. Giselsson and S. Boyd. Monotonicity and restart in fast grad ient methods. In 53rd IEEE Conference on Decision and Control , page 5058–5063, 2014
2014
-
[19]
Kim and J
D. Kim and J. Fessler. Optimized first-order methods for smoot h convex minimization. Math- ematical Programming, 159:81–107, 2016
2016
-
[20]
Kim and J
D. Kim and J. Fessler. On the convergence analysis of the optimiz ed gradient method. Journal of Optimization Theory and Applications , 172:187–205, 2017
2017
-
[21]
Li and Z
H. Li and Z. Lin. Restarted nonconvex accelerated gradient d escent: No more polylogarithmic factor in the in the o (εˆ(-7/4)) complexity. Journal of Machine Learning Research, 24(157):1– 37, 2023
2023
-
[22]
Lin and L
Q. Lin and L. Xiao. An adaptive accelerated proximal gradient me thod and its homotopy continuation for sparse optimization. In Proceedings of the 31st International Conference on Machine Learning, volume 23, pages 73–81, 2014
2014
-
[23]
J. J. Maul´ en and J. Peypouquet. A speed restart scheme fo r a dynamics with Hessian-driven damping. Journal of Optimization Theory and Applications , 199:831–855, 2023
2023
-
[24]
Necoara, Y
I. Necoara, Y. Nesterov, and F. Glineur. Linear convergence of first order methods for non- strongly convex optimization. Mathematical Programming, 175(1):69–107, 2019
2019
-
[25]
Nemirovskii and Y
A. Nemirovskii and Y. Nesterov. Optimal methods of smooth co nvex minimization. USSR Computational Mathematics and Mathematical Physics , 25(2):21–30, 1985
1985
-
[26]
Nesterov
Y. Nesterov. A method for solving the convex programming pro blem with convergence rate O(1/k 2). Soviet Mathematics Doklady , 27:372–376, 1983
1983
-
[27]
Nesterov
Y. Nesterov. Gradient methods for minimizing composite functio ns. Mathematical Program- ming, 140(1):125–161, 22–24 Jun 2013
2013
-
[28]
Nesterov
Y. Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013
2013
-
[29]
O’Donoghue and E
B. O’Donoghue and E. Cand` es. Adaptive restart for accelerated gradient schemes. Foundations of Computational Mathematics , 15(3):715–732, 2015. 17
2015
-
[30]
C. Park, J. Park, and E. Ryu. Factor- √ 2 acceleration of accelerated gradient methods. Applied Mathematics & Optimization , 88(77), 2023
2023
-
[31]
J. Park. Accelerated additive Schwarz methods for convex op timization with adaptive restart. Journal of Scientific Computing , 89(3):58, 2021
2021
-
[32]
B. Polyak. Some methods of speeding up the convergence of ite ration methods. USSR compu- tational mathematics and mathematical physics , 4(5):1–17, 1964
1964
-
[33]
B. Polyak. Introduction to optimization. Optimization Software, 1987
1987
-
[34]
Renegar and B
J. Renegar and B. Grimmer. A simple nearly optimal restart sche me for speeding up first-order methods. Foundations of computational mathematics , 22(1):211–256, 2022
2022
-
[35]
Roulet and A
V. Roulet and A. d’Aspremont. Sharpness, restart, and acce leration. SIAM Journal on Opti- mization, 30(1):262–289, 2020
2020
-
[36]
W. Su, S. Boyd, and E. J. Cand` es. A differential equation for m odeling Nesterov’s accelerat- edgradient method: theory and insights. Journal of Machine Learning Research, 17(153):1–43, 2016
2016
-
[37]
B. Wang, T. Nguyen, T. Sun, A. Bertozzi, R. Baraniuk, and S. O sher. Scheduled restart momentum for accelerated stochastic gradient descent. SIAM Journal on Imaging Sciences , 15(2):738–761, 2022. 18
2022
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.