A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms

Bin Shi; Mingwei Fu

REVIEW 2 major objections 5 minor 23 references

A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms

T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing NAG's linear momentum by a power-law coefficient yields an O(1/k^{2α}) rate, tunable to any inverse-polynomial degree on strongly convex objectives.

desk verdict Genuinely new α-power momentum family with claimed O(1/k^{2α}) rates at s=1/L, but the main theorem's burn-in K must depend on μ, and the proof hides that dependence. read the letter →

arxiv 2501.10051 v1 pith:JWYZKSKV submitted 2025-01-17 math.OC cs.NAmath.NAstat.ML

Mingwei Fu , Bin Shi This is my paper · ORCID

classification math.OCcs.NAmath.NAstat.ML MSC 90C2565K0590C30

keywords acceleratedgradientmethodsNesterovaccelerationmomentumcoefficientsstrongconvexityLyapunovanalysisforward-backwardalgorithmsFISTAcriticalstepsize

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a small modification to Nesterov's accelerated gradient method makes its convergence rate tunable: replacing the linear momentum coefficient $(k-1)/(k+r)$ with the $\alpha$-power form $(k-1)^\alpha/(k^\alpha+r k^{\alpha-1})$ yields an $O(1/k^{2\alpha})$ function-error bound on strongly convex objectives, provided $r>2\alpha$ and the step size does not exceed $1/L$. At the critical step size $s=1/L$, where no rate for forward-backward accelerated methods was previously available, choosing $\alpha$ large gives an inverse-polynomial rate of any desired degree. The same Lyapunov argument extends the rate to monotone variants and, through the proximal fundamental inequality, to composite problems solved by FISTA-type algorithms. A sympathetic reader would care because the result offers a single parameter family that interpolates between and beyond standard accelerated rates and closes a known gap at the critical step size.

What carries the argument

The machine that carries the proof is the $\alpha$-power momentum coefficient $\frac{(k-1)^\alpha}{k^\alpha+r k^{\alpha-1}}$ with $r>2\alpha$, paired with a Lyapunov function $E(k)=s k^\alpha(k^\alpha+r k^{\alpha-1})(f(x_k)-f(x_\star))+\frac12\|\sqrt{s}(k-1)^\alpha v_k+r(k-1)^{\alpha-1}(x_k-x_\star)\|^2$. The strong-convexity fundamental inequality (2.3) supplies the terms $SC_1$ and $SC_2$ that cancel the positive cross terms; without them, the Lyapunov difference cannot be forced non-positive. For composite objectives, Lemma 2.3 (the proximal version of the same inequality) plays the identical role. The choice $r>2\alpha$ makes the leading coefficient of the Lyapunov difference behave like $(2\alpha-r)k^{2\alpha-1}$, which is negative, and the remaining positive terms are absorbed once the iteration number passes a threshold $K$.

What would settle it

For a fixed $\alpha>1$ and $r>2\alpha$, run NAG-$\alpha$ on the one-dimensional quadratic $f(x)=\mu x^2/2$ with $L$ fixed and step $s=1/L$, and compute the exact Lyapunov difference $E(k+1)-E(k)$. Find the smallest $K_\mu$ such that the difference is non-positive for all $k\ge K_\mu$. If $K_\mu$ remained bounded as $\mu\to0$, the theorem would force a better-than-$1/k^2$ rate in the convex limit, contradicting the standard $\Omega(1/k^2)$ lower bound; so the numerical $K_\mu$ should grow at least like $1/\sqrt{\mu}$, and any observed boundedness would falsify the theorem as stated.

Watch

Extended reading notes

Core claim

At the center of the paper is the claim that for any $\alpha>0$ and $r>2\alpha$, the NAG-$\alpha$ iterates (with $x_0=y_0$) driven by the momentum coefficient $\frac{(k-1)^\alpha}{k^\alpha+r k^{\alpha-1}}$ converge on $f\in S^1_{\mu,L}(\mathbb{R}^d)$ with step size $s\in(0,1/L]$ at the rate $$f(x_k)-f(x_\star)\le \frac{E(K)}{s\,k^\$\alpha$(k^\$\alpha$+r $k^{{\alpha-1}}$)},\qquad k\ge K,$$ where $K:=K(\alpha,r)$ is the first index at which the Lyapunov function $E(k)$ becomes non-increasing. The rate is $O(1/k^{2\alpha})$, so at $s=1/L$ the parameter $\alpha$ controls the degree of the inverse-polynomial rate. The proof identifies a Lyapunov function tailored to $\alpha$, shows that with $r>2\alpha$ its leading difference is negative, and uses the strong-convexity terms to absorb the positive cross terms once $k$ is large. A reformulation of the Lyapunov function in terms of $x_k$ and $y_k$ (without the velocity variable) lets the same argument cover the monotone variants, and the proximal fundamental inequality extends it verbatim to FISTA-$\alpha$ and M-FISTA-$\alpha$ for composite objectives.

Load-bearing premise

The proof only controls the Lyapunov function after an iteration threshold $K$, and the theorem states $K:=K(\alpha,r)$ without making the dependence on the strong-convexity parameter $\mu$ explicit; the argument needs $K$ to grow when $\mu$ shrinks, so the promised rate may start arbitrarily late for nearly convex problems.

Editorial extensions

If this is right

At step size $s=1/L$, Theorem 3.1 and Corollary 3.3 give $f(x_k)-f(x_\star)\le L E(K)/(k^\alpha(k^\alpha+r k^{\alpha-1}))$ for all $k\ge K$, so choosing $\alpha$ large with $r>2\alpha$ yields an inverse-polynomial rate of any desired degree on strongly convex objectives.
For composite objectives $\Phi=f+g$ with $f\in S^1_{\mu,L}$ and convex $g$, the same $O(1/k^{2\alpha})$ bound applies to FISTA-$\alpha$ and M-FISTA-$\alpha$, using the $s$-proximal subgradient and the proximal fundamental inequality.
The monotone variants M-NAG-$\alpha$ and M-FISTA-$\alpha$ never increase the objective value and retain the controllable $O(1/k^{2\alpha})$ rate, so monitoring the objective during iterations does not sacrifice the acceleration guarantee.
The rate is non-asymptotic only after the threshold $K=K(\alpha,r)$; before that threshold the Lyapunov difference is uncontrolled, so the theorem predicts a parameter-dependent transient phase whose length is not quantified in the paper.
If the strong convexity assumption is dropped, the terms $SC_1$ and $SC_2$ vanish and the Lyapunov difference can no longer be forced non-positive; the paper's mechanism does not extend to merely convex functions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because $K$ is not stated to depend on $\mu$, the theorem must be read as allowing $K$ to grow when the condition number grows; if $K$ were $\mu$-independent, sending $\mu\to0$ would imply an $O(1/k^{2\alpha})$ rate for convex functions with $\alpha>1$, contradicting the standard $\Omega(1/k^2)$ lower bound.
Editorial inference: a testable consequence is that the onset iteration should scale roughly like $1/\sqrt{\mu}$ on ill-conditioned quadratics; plotting the first $k$ at which the Lyapunov difference becomes non-positive against $1/\mu$ would reveal whether the threshold is tight.
Editorial inference: the family naturally raises the question of whether $\alpha$ could be allowed to grow with $k$ to recover a linear rate at $s=1/L$; the paper's fixed-degree polynomial analysis does not address that regime.
Editorial inference: extensions to inexact or stochastic oracles are not covered, because those settings break the exact cancellation that makes the Lyapunov difference tractable; any such extension would require a new argument.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real extension — replacing the linear momentum coefficient with an α-th power and getting a Lyapunov analysis that avoids the kinetic-energy term, then carrying it through to monotone and proximal variants. That is genuinely new relative to Li-Shi-Yuan and Fu-Shi. The O(1/k^{2α}) rate at the critical step size is the sort of claim people will want to pin down.

The problem is the statement of Theorem 3.1 (and its FISTA/M-FISTA siblings). The theorem says K = K(α,r), with no dependence on μ. But the proof's own asymptotics force K to depend on μ. To make the quadratic form in (v_k, y_k−x*) non-positive, the strong-convexity terms must dominate the cross term, and the discriminant condition gives k³ ≳ r²(α−1)²/(2 μ² s² α) for α≠1. So any valid K grows at least like μ^{-2/3} as μ→0. Read literally, the theorem is false: taking μ→0 would imply an O(1/k^{2α}) rate for convex functions, contradicting the known Ω(1/k²) lower bound. The fix is easy — restate K as K(α,r,μ,L,s) — but as written it overreaches.

The proof also glosses the key negativity step. It says the SC terms 'reduce the combined sum to three non-positive perfect squares' without displaying them. That is a real gap, but it looks fillable from the expansions (3.22)–(3.24); I do not see a reason the argument cannot be completed with a concrete K. For M-NAG-α the reformulation in (4.2) is plausible, but the proof there is even terser.

Credit where due: the Lyapunov function without kinetic energy is a nice trick, it genuinely simplifies the monotone extension, and the paper is honest in Remark 3.2 and the conclusion about the indispensability of strong convexity. The reliance on Li et al. [2024b] is a tool use, not a circular argument.

Verdict: worth a serious referee, but the authors need to fix the dependence of K and flesh out the quadratic-form argument. If those land, this is a solid theory contribution. I would not cite it as-is.

Referee Report

2 major / 5 minor

Summary. The paper introduces a family of momentum coefficients of the form ((k-1)^α)/(k^α + r k^{α-1}) for forward-backward accelerated methods. For f ∈ S^1_{μ,L} and s ∈ (0,1/L], it claims that when r > 2α there is a positive integer K := K(α,r) such that NAG-α satisfies f(x_k)-f(x*) ≤ E(K)/(s k^α(k^α + r k^{α-1})) for all k ≥ K, i.e. an O(1/k^{2α}) rate. The same rate is claimed for FISTA-α and for the monotonic variants M-NAG-α and M-FISTA-α, and at the critical step size s = 1/L the paper concludes that arbitrarily high inverse-polynomial rates are achievable by choosing α. The proofs are based on a newly constructed Lyapunov function involving mixed and potential energies, with the kinetic energy eliminated in the monotonic setting.

Significance. If the main result can be made correct, this is a useful contribution: it gives explicit, non-fitted Lyapunov constructions for a family of accelerated methods at the critical step size, and it extends them to proximal and monotonic variants. The derivation is transparent in structure, and the non-positivity of the Lyapunov difference is intended to follow from algebraic cancellation rather than from numerical tuning. The numerical experiment on a quadratic is suggestive but not decisive. The principal obstacle is that the central theorem is stated with a burn-in K that depends only on (α,r), while the proof forces K to depend on μ and the step-size parameters; this is a load-bearing flaw that can be repaired by restating the theorem and supplying the missing quantitative discriminant check. With that repair, the paper would be a meaningful addition to the literature on parameterized accelerated methods.

major comments (2)

[Theorem 3.1 and its proof]
[Proof of Theorem 3.1, after (3.24)]

minor comments (5)

[Theorem 4.2]
[Section 4.1, around (4.10)]
[Figure 1]
[Section 5]
[General]

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the Lyapunov analysis is a fresh construction and the rates are not fitted; the only self-reference is a non-load-bearing citation to prior work by the same group, while the stated K(α,r) dependence is a correctness gap rather than a circular step.

full rationale

The central derivation is self-contained: the paper constructs a new Lyapunov function E(k) and proves E(k+1) − E(k) ≤ 0 asymptotically using expansions (3.21)–(3.24) and the strong-convexity terms SC1 and SC2. The convergence rate O(1/k^{2α}) is read off from the coefficient sk^α(k^α + rk^{α−1}) in the Lyapunov function, and the nontrivial work is establishing monotonicity of E(k); no constant is fitted to data and no numerical experiment is presented as a prediction. The α-th power momentum coefficient is a design choice, not an output derived from the rate. The composite extensions use Lemma 2.3 from Li et al. [2024b], which shares author Bin Shi; however, that lemma is an explicit inequality proven in the cited work and used as a tool, not as the target convergence result, so the self-citation is minor and not load-bearing in a circular sense. One issue is worth flagging: Theorem 3.1 states K := K(α,r), but the proof's eventual non-positivity of E(k+1) − E(k) requires k large enough for the strong-convexity terms to dominate the positive cross terms in (3.20), and that threshold depends on μ and s (the reader's μ^{-2/3} estimate is consistent with the paper's own expansions). Thus K must depend on μ (and s,L) or the theorem statement must be read with K allowed to depend on f. This is a correctness gap, not a circularity, because it does not make the claimed rate equivalent to an input by construction. The paper's concluding remark that the strong-convexity conditions are overly restrictive is likewise a limitation statement, not a disguised circular step.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No constants are fitted to data. The only algorithm design choices are the momentum exponent α and the control parameter r, both user-selected and governed by the condition r>2α. The proof relies on the strong-convexity assumption and on the cited composite-function inequality from Li et al. [2024b]. No invented entities are introduced.

free parameters (2)

α
User-chosen exponent controlling the momentum power; the claimed rate is O(1/k^{2α}). Not fitted to data, but central to the theorem.
r
Control parameter; the theorem requires r>2α. Not fitted, but the condition links r to α.

assumptions (3)

domain assumption f is μ-strongly convex and L-smooth (f∈S^1_{μ,L}), with step size s∈(0,1/L].
Assumed at the start of Section 2 and used throughout; the proof relies on the fundamental inequality (2.3), which requires strong convexity.
standard math Lemma 2.3 (fundamental inequality for composite functions) from Li et al. [2024b].
Cited and used without proof as the proximal counterpart of the strong-convexity inequality; it is the basis for the FISTA-α and M-FISTA-α extensions.
standard math Asymptotic expansions of the momentum coefficients and existence of a sufficiently large K.
The proof of Theorem 3.1 relies on k→∞ expansions such as A1(k,α)=(α-1)k^{α-2}+O(k^{α-3}) and on choosing K so that leading negative terms dominate; no explicit K is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms." pith.science (2026). https://pith.science/paper/JWYZKSKV

@misc{pith2026250110051,
  author       = {Pith},
  title        = {Pith review of: A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JWYZKSKV}},
  note         = {Machine review of arXiv:2501.10051}
}

abstract

Nesterov's accelerated gradient method (NAG) marks a pivotal advancement in gradient-based optimization, achieving faster convergence compared to the vanilla gradient descent method for convex functions. However, its algorithmic complexity when applied to strongly convex functions remains unknown, as noted in the comprehensive review by Chambolle and Pock [2016]. This issue, aside from the critical step size, was addressed by Li et al. [2024b], with the monotonic case further explored by Fu and Shi [2024]. In this paper, we introduce a family of controllable momentum coefficients for forward-backward accelerated methods, focusing on the critical step size $s=1/L$. Unlike traditional linear forms, the proposed momentum coefficients follow an $\alpha$-th power structure, where the parameter $r$ is adaptively tuned to $\alpha$. Using a Lyapunov function specifically designed for $\alpha$, we establish a controllable $O\left(1/k^{2\alpha} \right)$ convergence rate for the NAG-$\alpha$ method, provided that $r > 2\alpha$. At the critical step size, NAG-$\alpha$ achieves an inverse polynomial convergence rate of arbitrary degree by adjusting $r$ according to $\alpha > 0$. We further simplify the Lyapunov function by expressing it in terms of the iterative sequences $x_k$ and $y_k$, eliminating the need for phase-space representations. This simplification enables us to extend the controllable $O \left(1/k^{2\alpha} \right)$ rate to the monotonic variant, M-NAG-$\alpha$, thereby enhancing optimization efficiency. Finally, by leveraging the fundamental inequality for composite functions, we extended the controllable $O\left(1/k^{2\alpha} \right)$ rate to proximal algorithms, including the fast iterative shrinkage-thresholding algorithm (FISTA-$\alpha$) and its monotonic counterpart (M-FISTA-$\alpha$).

Figures

Figures reproduced from arXiv: 2501.10051 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages

[1]

Attouch and J

H. Attouch and J. Peypouquet. The rate of convergence of nesterov's accelerated forward-backward method is actually faster than 1/k^2 . SIAM Journal on Optimization, 26 0 (3): 0 1824--1834, 2016

work page 2016
[2]

Attouch, P.-E

H. Attouch, P.-E. Maing \'e , and P. Redont. A second-order differential system with hessian-driven damping; application to non-elastic shock laws. Differential Equations and Applications, 4 0 (1): 0 27--65, 2012

work page 2012
[3]

Attouch, J

H. Attouch, J. Peypouquet, and P. Redont. A dynamical approach to an inertial forward-backward algorithm for convex minimization. SIAM Journal on Optimization, 24 0 (1): 0 232--256, 2014

work page 2014
[4]

A. Beck. First-order methods in optimization. SIAM, 2017

2017
[5]

Beck and M

A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2 0 (1): 0 183--202, 2009

work page 2009
[6]

A. Cauchy. M \'e thode g \'e n \'e rale pour la r \'e solution des systemes d’ \'e quations simultan \'e es. Comp. Rend. Sci. Paris, 25 0 (1847): 0 536--538, 1847

work page
[7]

Chambolle and T

A. Chambolle and T. Pock. An introduction to continuous optimization for imaging. Acta Numerica, 25: 0 161--319, 2016

work page 2016
[8]

S. Chen, B. Shi, and Y.-x. Yuan. Gradient norm minimization of nesterov acceleration: o(1/k^ 3 ) . arXiv preprint arXiv:2209.08862, 2022 a

arXiv 2022

Show all 23 references

[9]

S. Chen, B. Shi, and Y.-x. Yuan. Revisiting the acceleration phenomenon via high-resolution differential equations. arXiv preprint arXiv:2212.05700, 2022 b

2022 arXiv
[10]

S. Chen, B. Shi, and Y.-x. Yuan. On underdamped nesterov's acceleration. arXiv preprint arXiv:2304.14642, 2023

2023 arXiv
[11]

Fu and B

M. Fu and B. Shi. Lyapunov analysis for monotonically forward-backward accelerated algorithms. arXiv preprint arXiv:2412.13527, 2024

2024 arXiv
[12]

Giselsson and S

P. Giselsson and S. Boyd. Monotonicity and restart in fast gradient methods. In 53rd IEEE Conference on Decision and Control, pages 5058--5063. IEEE, 2014

2014
[13]

B. Li, B. Shi, and Y.-X. Yuan. Linear convergence of ISTA and FISTA . arXiv preprint arXiv:2212.06319, 2022 a

2022 arXiv
[14]

B. Li, B. Shi, and Y.-x. Yuan. Proximal subgradient norm minimization of ista and fista. arXiv preprint arXiv:2211.01610, 2022 b

2022 arXiv
[15]

B. Li, B. Shi, and Y.-x. Yuan. Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity. SIAM Journal on Optimization, 34 0 (2): 0 2150--2168, 2024 a

2024
[16]

B. Li, B. Shi, and Y.-X. Yuan. Linear convergence of ISTA and FISTA . Journal of the Operations Research Society of China, pages 1--19, 2024 b

2024
[17]

A. S. Nemirovsky and D. B. Yudin. Problem complexity and method efficiency in optimization. John Wiley, New York, 1983

1983
[18]

Nesterov

Y. Nesterov. A method of solving a convex programming problem with convergence rate o(1/k^2) . Soviet Mathematics-Doklady,, 27 0 (2): 0 372--376, 1983

1983
[19]

Nesterov

Y. Nesterov. Lectures on Convex Optimization, volume 137. Springer, second edition, 2018

2018
[20]

O’Donoghue and E

B. O’Donoghue and E. Candes. Adaptive restart for accelerated gradient schemes. Foundations of computational mathematics, 15: 0 715--732, 2015

2015
[21]

B. Shi, S. S. Du, M. I. Jordan, and W. J. Su. Understanding the acceleration phenomenon via high-resolution differential equations. Mathematical Programming, pages 79--148, 2022

2022
[22]

W. Su, S. Boyd, and E. J. Candes. A differential equation for modeling nesterov's accelerated gradient method: Theory and insights. Journal of Machine Learning Research, 17 0 (153): 0 1--43, 2016

2016
[23]

Wibisono, A

A. Wibisono, A. C. Wilson, and M. I. Jordan. A variational perspective on accelerated methods in optimization. proceedings of the National Academy of Sciences, 113 0 (47): 0 E7351--E7358, 2016

2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Attouch and J

H. Attouch and J. Peypouquet. The rate of convergence of nesterov's accelerated forward-backward method is actually faster than 1/k^2 . SIAM Journal on Optimization, 26 0 (3): 0 1824--1834, 2016

work page 2016

[2] [2]

Attouch, P.-E

H. Attouch, P.-E. Maing \'e , and P. Redont. A second-order differential system with hessian-driven damping; application to non-elastic shock laws. Differential Equations and Applications, 4 0 (1): 0 27--65, 2012

work page 2012

[3] [3]

Attouch, J

H. Attouch, J. Peypouquet, and P. Redont. A dynamical approach to an inertial forward-backward algorithm for convex minimization. SIAM Journal on Optimization, 24 0 (1): 0 232--256, 2014

work page 2014

[4] [4]

A. Beck. First-order methods in optimization. SIAM, 2017

2017

[5] [5]

Beck and M

A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2 0 (1): 0 183--202, 2009

work page 2009

[6] [6]

A. Cauchy. M \'e thode g \'e n \'e rale pour la r \'e solution des systemes d’ \'e quations simultan \'e es. Comp. Rend. Sci. Paris, 25 0 (1847): 0 536--538, 1847

work page

[7] [7]

Chambolle and T

A. Chambolle and T. Pock. An introduction to continuous optimization for imaging. Acta Numerica, 25: 0 161--319, 2016

work page 2016

[8] [8]

S. Chen, B. Shi, and Y.-x. Yuan. Gradient norm minimization of nesterov acceleration: o(1/k^ 3 ) . arXiv preprint arXiv:2209.08862, 2022 a

arXiv 2022