The iterates of FISTA converge even under inexact computations and stochastic gradients

Saverio Salzo

REVIEW 1 major objections 7 minor 27 references

FISTA's iterates still converge weakly in infinite-dimensional Hilbert spaces when the gradient and proximity operator are computed inexactly, including with stochastic gradient oracles.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 22:00 UTC pith:NLQXZK7Q

load-bearing objection The deterministic inexact-FISTA result is real and essentially correct; the stochastic theorem as printed has a missing monotone-stepsize assumption, but it is a cheap fix. the 1 major comments →

arxiv 2511.12665 v3 pith:NLQXZK7Q submitted 2025-11-16 math.OC

The iterates of FISTA converge even under inexact computations and stochastic gradients

Saverio Salzo This is my paper

classification math.OC MSC 65K0590C2590C1549M27

keywords convex optimizationaccelerated proximal gradientFISTAinexact algorithmsstochastic gradientsweak convergenceiterates convergenceHilbert space

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Building on the recently resolved question of whether Nesterov-accelerated proximal gradient (FISTA) iterates converge to a minimizer, this paper proves that the answer stays positive in infinite-dimensional Hilbert spaces even when the gradient and the proximity operator are only approximated and the gradient oracle is stochastic. In the deterministic case, the objective error remains O(1/t²_{k-1}) and the iterates converge weakly to a minimizer provided the weighted errors (t_k δ_k) and (t_k ‖b_k‖) are summable and the acceleration parameter t_k tends to infinity. In the stochastic case, an unbiased gradient oracle with bounded variance yields almost-sure weak convergence of the iterates under compatible decay of step sizes and errors. The argument hinges on a perturbed energy inequality, a discrete Bihari-La Salle lemma, and an Opial lemma tailored to accelerated and random sequences.

Core claim

The paper's central claim is that the two standard forms of numerical inexactness—deterministic perturbations of the gradient and the proximal computation, and an unbiased stochastic gradient oracle—do not destroy the just-established weak convergence of FISTA iterates in the critical regime. Theorem 3.6 shows that if t_k → ∞ and the sequences t_k δ_k and t_k ‖b_k‖ are summable, then F(x_k)-F* = O(1/t²_{k-1}) and x_k and y_k converge weakly to a common minimizer. Theorem 3.10 shows that under an unbiased gradient oracle with variance bounded by σ², conditions γ_k t²_{k-1} → ∞, (γ_k t_k) ∈ ℓ², and (δ_k t_k) ∈ ℓ¹ imply x_k and y_k converge weakly almost surely to an S*-valued random vector, wi

What carries the argument

The proof is carried by an energy function E_k(x*) = 2γ t²_{k-1}(F(x_k)-F*) + ‖v_k - x*‖², which satisfies a perturbed descent inequality E_{k+1} ≤ E_k + λ_k √E_{k+1} + ξ_k where λ_k and ξ_k encode the weighted errors. A discrete Bihari-La Salle lemma bounds E_k and forces its convergence despite the lack of monotonicity. The weak-convergence conclusion then follows from an Opial-type lemma, adapted to accelerated schemes and generalized to random sequences, combined with a representation of the iterates as convex combinations x_{k+1} = (1 - 1/t_k) x_k + (1/t_k) v_{k+1} and a Cesàro-like fact (Fact 2.6) that turns convergence of the auxiliary sequence (v_k) into convergence of the iterates.

Load-bearing premise

The stochastic theorem's proof assumes the step sizes γ_k are non-increasing (γ_{k+1} ≤ γ_k), an assumption that is never stated in Algorithm 1.4 or Theorem 3.10; if that monotonicity fails, the inequality that drives the almost-sure convergence may reverse.

What would settle it

Read the proof of Theorem 3.10 and locate the line 'using the fact that γ_{k+1} ≤ γ_k'. Neither the algorithm nor the theorem states this monotonicity, and when γ_{k+1} > γ_k the inequality 2γ_{k+1}(F(x_{k+1})-F*) ≤ 2γ_k(F(x_{k+1})-F*) is violated. Construct any admissible parameter sequence (γ_k) with 0<γ_k≤1/L and γ_{k+1} > γ_k frequently, e.g., alternating 1/L and (1/L)e^{-k}, obeying the stated summability conditions, and check whether the energy recursion still forces almost-sure convergence; the proof's chain breaks at that step.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Inexact computation of prox and gradient, provided errors decay fast enough relative to the acceleration, preserves the optimal O(1/k²) objective rate and weak convergence of iterates.
Stochastic gradients with bounded variance can be used inside FISTA without losing almost-sure weak convergence, as long as the step sizes decay suitably (e.g., γ_k ≍ 1/k^q with specific exponents).
The trade-off between acceleration strength and allowed error level: full acceleration requires the strongest error decay, while slower acceleration tolerates larger errors, covering the classical proximal gradient algorithm as a border case.
The results extend the recent exact-case convergence to a larger algorithmic class, supporting practical use of FISTA in infinite-dimensional inverse problems where exact proximity/gradient evaluations are unattainable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same proof scheme may be portable to other accelerated first-order methods that admit a similar convex-combination representation, such as heavy-ball-type schemes, by reusing the perturbed energy recursion.
The proof of Theorem 3.10 silently requires γ_{k+1} ≤ γ_k; if monotone step sizes are not enforced, the stochastic claim is not fully established, though the deterministic theorem stands. A published corollary with that assumption added would resolve the gap.
A natural testable extension is to replace the fixed uniform variance bound on the stochastic oracle with a growing Lipschitz-like condition, or to allow correlated noise, and check whether the same summability conditions still yield a.s. convergence.
The rates in expectation (e.g., O((1+log k)/√k) for a particular step-size choice) suggest that the stochastic version behaves like the non-accelerated stochastic proximal gradient in the regime where acceleration is weakened; this could guide adaptive step-size rules.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The deterministic inexact-FISTA result is real and essentially correct; the stochastic theorem as printed has a missing monotone-stepsize assumption, but it is a cheap fix.

read the letter

Short version: the deterministic Theorem 3.6 is the real contribution and it holds up. I traced the Lyapunov recursion, the dropped nonpositive term, the Bihari–La Salle step, and the Opial/Fact 2.6 finish. The constants are tight enough (283/256 < 10/9), and the summability conditions on (t_k δ_k) and (t_k ||b_k||) match what the proof needs. This extends the exact critical-regime results of Jang–Ryu and Boţ–Fadili–Nguyen to inexact prox/gradient computations, and it removes the coercivity assumption from Aujol–Dossal. That is a solid step forward, not just a repackaging.

The stochastic Theorem 3.10 has a genuine gap as printed. The proof uses γ_{k+1} ≤ γ_k to pass from 2γ_k r_{k+1} to 2γ_{k+1} r_{k+1}, but neither Algorithm 1.4 nor the theorem states it. Without that, the Lyapunov recursion can fail. The stress-test note is correct. It is also a cheap repair—just add monotone decreasing stepsizes to the algorithm and theorem. Remark 3.11's examples already use decreasing γ_k, and the a.s.-convergence proof goes through unchanged once the assumption is there.

Minor comments: the text has several formula-level corruptions (missing t_k factors, a stray σ² in the definition of ξ_k after (3.19), the garbled finite-dimensional title). These look like typesetting issues, not conceptual ones. The deterministic part is unaffected. The appendix rederivations of Fact 2.3 and Fact 2.6 are fine.

Bottom line: this is a serious paper from someone who knows the literature. The deterministic half is a genuine advance and is proved; the stochastic half is one unstated assumption away from being proved. I would send this to a competent referee. With the monotonicity condition added, I would accept it. The citation pattern is honest—Schmidt–Le Roux–Bach, Villa et al., Salzo–Villa, and the two recent critical-regime papers are all credited properly. Take it to reading group; the proof is short enough to walk through in an hour.

Referee Report

1 major / 7 minor

Summary. The paper proves convergence results for an inexact version of FISTA in infinite-dimensional Hilbert spaces. Algorithm 1.3 allows deterministic additive errors in the gradient and inexact proximity evaluations in the sense of Salzo–Villa; Theorem 3.6 shows that if the acceleration parameters satisfy t_k → ∞ and the weighted error sequences (t_k δ_k) and (t_k ∥b_k∥) are summable, then F(x_k) − F* = O(1/t_{k−1}^2) and the sequences (x_k), (y_k) converge weakly to a common minimizer. Algorithm 1.4 incorporates an unbiased stochastic gradient oracle with uniformly bounded variance; Theorem 3.10 claims an analogous almost-sure weak convergence under γ_k t_{k−1}^2 → ∞, (γ_k t_k) ∈ ℓ², and (δ_k t_k) ∈ ℓ¹. The proofs use a Lyapunov function E_k, a Bihari–La Salle type inequality (Lemma 3.4 / Fact 2.3), Opial's lemma and a stochastic version thereof, and a Cesàro-type lemma (Fact 2.6). The deterministic analysis appears sound; the stochastic theorem as stated is not proved, because the proof silently uses a monotonicity condition γ_{k+1} ≤ γ_k that is not part of Algorithm 1.4 or Theorem 3.10.

Significance. If repaired, the paper would be a valuable extension of the recent point-convergence results of Jang–Ryu and Boţ–Fadili–Nguyen to inexact and stochastic settings. The deterministic theorem is clean, with transparent Lyapunov arguments and explicit summability conditions that interpolate between the nonaccelerated and fully accelerated regimes. The stochastic result is plausible and of independent interest, but as it stands its proof contains a load-bearing gap. The paper also usefully highlights the trade-off between acceleration strength and admissible error decay. The deterministic part appears correct, and the stochastic part is likely correct after adding the missing monotonicity assumption.

major comments (1)

[Theorem 3.10, proof between (3.17) and (3.19)] The proof uses the statement 'using the fact that γ_{k+1} ≤ γ_k' to replace 2γ_k r_{k+1} by 2γ_{k+1} r_{k+1} on the left-hand side of the Lyapunov recursion. Neither Algorithm 1.4 nor Theorem 3.10 assumes this monotonicity; only 0 < γ_k ≤ 1/L is stated. If γ_{k+1} > γ_k, the inequality 2γ_{k+1} r_{k+1} ≤ ... need not follow, and the subsequent definition of E_{k+1}, the conditional expectation bound (3.19), the Robbins–Siegmund step, and the a.s. Opial argument all rely on this step. This is a genuine gap in Theorem 3.10 as stated. The repair is local: add γ_{k+1} ≤ γ_k to Algorithm 1.4 and Theorem 3.10 — Remark 3.11's examples already satisfy it — or modify the Lyapunov function so that the monotonicity is not needed.

minor comments (7)

[Theorems 3.6 and 3.10] The displayed bounds involve δ_{1,i} and δ_{2,i}, but these quantities are not defined in the theorem statements. They are introduced only inside the proofs via Lemma 3.3 / Proposition 1.2. Please state explicitly that δ_{1,i}, δ_{2,i} are the components with δ_{1,i}^2 + δ_{2,i}^2 ≤ δ_i^2.
[Lemma 3.4] The statement says 'if (λ_k) and (μ_k) are summable', but μ_k is not defined in the lemma. It should be 'if (λ_k) and (ξ_k) are summable'.
[Appendix A, proof of Fact 2.3] The proof contains the inequality ∑λ_i√γ_i ≤ γ_k ∑√λ_i, which is not valid as written. The correct bound is ∑λ_i√γ_i ≤ (∑λ_i)√γ_k, from which the claimed quadratic inequality follows. Please correct this intermediate line.
[Theorem 3.10 proof, application of Lemma 3.4] In the displayed definition of ξ_k, the term t_k^2 δ_{1,k}^2 σ^2 has an extra σ^2 factor; it should be t_k^2 δ_{1,k}^2 as in (3.19).
[Theorem 3.10 proof, Robbins–Siegmund step] When applying Fact 2.7, the deterministic summable terms 2γ_k t_k^2 δ_k σ + 2γ_k^2 t_k^2 σ^2 + t_k^2 δ_{1,k}^2 from (3.19) are not explicitly included in the defined ε_k. They are summable under the theorem's hypotheses, but the application should state that these terms are absorbed into the ε_k sequence.
[Title and abstract] 'The iterates of FISTA convergence ...' should read 'The iterates of FISTA converge ...'.
[Appendix A, proof of Fact 2.3] Minor typo: 'positive solution solution' should be 'positive solution'.

Circularity Check

0 steps flagged

No significant circularity: the deterministic and stochastic convergence proofs build on external tools (Opial, Robbins–Siegmund, Fact 2.6 from [11]) plus a standard inexact-prox model; the known gamma-monotonicity gap is a correctness issue, not a circular reduction.

full rationale

The paper's derivation chain is not circular. Theorem 3.6 is obtained from Lemma 3.3, which uses the inexact-prox characterization Proposition 1.2 (quoted from the authors' earlier work [24]) together with Lipschitz smoothness and convexity, and then forms the Lyapunov function E_k(x*) = 2γt_{k-1}^2(F(x_k)-F*) + ||v_k - x*||^2. No quantity is fitted to data, no conclusion is assumed, and the cited self-work [24] supplies only a modeling definition and a standard subdifferential inequality, not iterate convergence or the theorem's conclusion. Fact 2.6 is cited from the non-self reference [11] and is re-proved in Appendix A; Opial's Lemma and Robbins--Siegmund are classical external facts. Theorem 3.10 does contain a genuine gap as printed: its proof passes from 2γ_k(F(x_{k+1})-F*) to 2γ_{k+1}r_{k+1} 'using the fact that γ_{k+1} ≤ γ_k', while Algorithm 1.4 and Theorem 3.10 only assume 0 < γ_k ≤ 1/L. But this is a missing/unstated hypothesis, not a self-referential reduction: the claimed conclusion is not equivalent to an input of the proof, and the gap is repairable by adding the monotonicity condition. There are no fitted-inputs-called-predictions, no imported uniqueness theorems, no ansatz smuggled in by self-citation, and no renaming of a known empirical pattern. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 6 axioms · 0 invented entities

The paper is a pure proof paper: no data fitting, no free parameters. The central claim rests on domain assumptions (convex L-smooth composite problem with nonempty minimizer set; bounded-variance unbiased stochastic oracle; admissible FISTA parameters) plus standard convex-analysis tools. One implicit assumption—γ nonincreasing—is used in the proof of Theorem 3.10 but not stated in the theorem. No invented entities (particles, forces, dimensions) are introduced; the auxiliary variable v_k is a proof device.

axioms (6)

domain assumption Assumption 1: f convex with L-Lipschitz gradient, g proper convex and lower semicontinuous, argmin of F nonempty, H a real Hilbert space.
Stated in Section 1; defines the problem class covered by the theorems.
domain assumption Inexact proximity model z ≃_δ prox_{γg}(y) from Salzo–Villa [24], with the error characterization of Proposition 1.2 (‖e‖ ≤ δ_2, δ_1² + δ_2² ≤ δ²).
Modeling choice for inexactness; imported from [24] and used throughout Lemma 3.3 and both main theorems.
domain assumption Assumption 2: unbiased gradient oracle û with E‖û(x,ζ) − ∇f(x)‖² ≤ σ²; ζ_k independent copies in Algorithm 1.4; H separable.
Standard bounded-variance stochastic oracle; needed for conditional-expectation steps and Robbins–Siegmund.
domain assumption Admissible FISTA parameter sequences: t_0 = 1, 1 ≤ t_k, t²_k − t_k ≤ t²_{k−1}; plus t_k → +∞ for the convergence claims.
Definition in (3.1)–(3.2); the critical regime t_k ≍ k matches the recently settled exact case.
ad hoc to paper Stepsize monotonicity γ_{k+1} ≤ γ_k in the stochastic theorem (implicit, unstated).
Used in the proof of Theorem 3.10 (“using the fact that γ_{k+1} ≤ γ_k”) but not stated in Algorithm 1.4 or Theorem 3.10.
standard math Standard convex-analysis toolkit: prox nonexpansiveness, ε-subdifferential calculus, weak lower semicontinuity of convex lsc functions, Opial's lemma, Robbins–Siegmund, Fact 2.6 from [11].
Cited from Bauschke–Combettes [8], Opial [20], Robbins–Siegmund [22], Boţ–Chenchene–Csetnek–Hulett [11]; several re-proved in Appendix A.

pith-pipeline@v1.3.0-alltime-deepseek · 19047 in / 30602 out tokens · 217492 ms · 2026-08-03T22:00:04.200741+00:00 · methodology

0 comments

read the original abstract

Very recently, the papers "Point Convergence of Nesterov's Accelerated Gradient Method: An AI-Assisted Proof" by Jang and Ryu, and "The Iterates of Nesterov's Accelerated Algorithm Converge in the Critical Regimes" by Bot, Fadili, and Nguyen have simultaneously resolved a long-standing open problem concerning Nesterov's accelerated gradient method. These works show that the iterates of the algorithm (known in its composite form as FISTA) indeed converge to an optimal solution. In this work, we extend these results and prove that, in infinite dimensional Hilbert spaces, the iterates of such an algorithm still converge (in the weak sense) even when the proximity operator and the gradient are computed inexactly, with the latter possibly stochastic.

discussion (0)

Reference graph

Works this paper leans on

27 extracted references · 3 linked inside Pith

[1]

Apidopoulos, J.-F

V. Apidopoulos, J.-F. Aujol, and Ch. Dossal, The differential inclusion modeling FISTA algorithm and optimality of convergence rate in the caseb≤3,SIAM J. Op- tim., 28(1), 551–574, 2018

2018
[2]

Attouch, Z

H. Attouch, Z. Chbani, J. Peypouquet, and P. Redont, Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity.Math. Program. B., 168, 123–175, 2018. 4The convergence of the series can be easily analyzed by the Cauchy’s condensation test [18]. 19

2018
[3]

Attouch, R

H. Attouch, R. I. Boţ, D. A. Hulett, and D.-K. Nguyen, Recovering Nesterov accelerated dynamics from Heavy Ball dynamics via time rescaling, arXiv, 2025. https://arxiv.org/abs/2504.15852

Pith/arXiv arXiv 2025
[4]

Attouch and A

H. Attouch and A. Cabot, Convergence rates of inertial forward-backward algo- rithms.SIAM J. Optim., 28(1), 849–874, 2018

2018
[5]

Attouch and J

H. Attouch and J. Peypouquet, The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster then1/k2.SIAM J. Optim., 26(3), 1824– 1834, 2016

2016
[6]

Aujol and Ch

J.-F. Aujol and Ch. Dossal, Stability of over-relaxations for the forward-backward algorithm, application to FISTA,SIAM J. Optim., 25(4), 2408–2433, 2015

2015
[7]

Barré, A

M. Barré, A. B. Taylor, and F. Bach, Principled analyses and design of first-order methods with inexact proximal operators,Math. Program. B., 201, 185–230, 2023

2023
[8]

H. H. Bauschke and P. L. CombettesConvex Analysis and Monotone Operator The- ory in Hilbert spaces, 2nd ed. Springer, New York, 2017

2017
[9]

Beck and M

A. Beck and M. Teboulle, A Fast Iterative Shrinkage-Thresholding Algorithm for linear inverse problems,SIAM J. Imaging Sci., 2(1), 183–202, 2009

2009
[10]

Bello-Cruz, M.K.N

Y. Bello-Cruz, M.K.N. Gonçalves, and N. Krislock On FISTA with relative error ruleComput. Optim. Appl., 84, 295–318, 2023

2023
[11]

R. I. Boţ, E. Chenchene, E.R. Csetnek, and D. A. Hulett, Accelerating Diagonal Methods for Bilevel Optimization: Unified Convergence via Continuous-Time Dy- namics, arXiv, 2025.https://arxiv.org/abs/2505.14389

Pith/arXiv arXiv 2025
[12]

R. I. Boţ, J. Fadili, and D.-K. Nguyen The iterates of Nesterov’s accelerated algo- rithm converge in the critical regime, arXiv, 2025.https://arxiv.org/abs/2510. 22715v2

2025
[13]

Fast Iterative Shrinkage/Thresholding Algorithm

A. Chambolle and C. Dossal, On the convergence of the iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”.J. Optim. Theory Appl., 166, 968–982, 2015

2015
[14]

D. L. Cohn,Measure Theory, 2nd ed. Birkhäuser, New York, 2013

2013
[15]

P. L. Combettes, S. Salzo and S. Villa, Consistent learning by composite proximal thresholding.Math. Program. B., 167, 99–127, 2018

2018
[16]

Güler, New proximal point algorithms for convex minimization.SIAM J

O. Güler, New proximal point algorithms for convex minimization.SIAM J. Optim. 2(4), 649–664, 1992

1992
[17]

Jang and E.K

U. Jang and E.K. Ryu, Point convergence of Nesterov’s Accelerated Gradient Method: an AI-assisted proof, arXiv, 2025.https://arxiv.org/abs/2510.23513

arXiv 2025
[18]

Knopp,Infinite Sequences and Series, Dover Publications, New York, 1956

K. Knopp,Infinite Sequences and Series, Dover Publications, New York, 1956

1956
[19]

Nesterov, A method of solving a convex programming problem with convergence rateO(1/k 2),Dokl

Y. Nesterov, A method of solving a convex programming problem with convergence rateO(1/k 2),Dokl. Akad. Nauk SSSR, 27, 372–376, 1983

1983
[20]

Opial, Weak convergence of the sequence of successive approximations for nonex- pansive mappings,Bull

Z. Opial, Weak convergence of the sequence of successive approximations for nonex- pansive mappings,Bull. Amer. Math. Soc., 73(4), 591–597, 1967

1967
[21]

Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput

B.T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. & Math. Phys., 4(5), 1–17, 1964

1964
[22]

Robbins and D

H. Robbins and D. Siegmund, A convergence theorem for nonnegative almost supermartingales and some applications, in:Optimization Methods in Statistics, J. S. Rustagi, ed., pp. 233–257, Academic Press, New York, London, 1971. 20

1971
[23]

Ryu,https://x.com/ernestryu/status/1980759528984686715?s=43, 2025

E. Ryu,https://x.com/ernestryu/status/1980759528984686715?s=43, 2025

arXiv 2025
[24]

Salzo and S

S. Salzo and S. Villa Inexact and accelerated proximal point algorithms,J. Convex Anal., 19(4), 1167–1192, 2012

2012
[25]

Schmidt, N

M. Schmidt, N. Le Roux, and F. Bach, Convergence Rates of Inexact Proximal- Gradient Methods for Convex Optimization, arXiv, 2011.https://arxiv.org/abs/ 1109.2415v2

Pith/arXiv arXiv 2011
[26]

W. Su, S. Boyd, E J. Candes A differential equation for modeling Nesterov’s accel- erated gradient method: theory and insights,JMLR, 17(153), 1–43, 2016

2016
[27]

Villa, S

S. Villa, S. Salzo, L. Baldassarre, A. Verri Accelerated and inexact forward-backward algorithms,SIAM J. Optim., 23(3), 1607–1633, 2013. A Appendix A.1 Proofs of Section 2 For reader’s convenience we give the proofs of some of the results presented in the pre- liminary section. Proof of Fact 2.1.Since0≤b k ≤α k −ak+1 +εk the summability of(bk)k∈N is immed...

2013

[1] [1]

Apidopoulos, J.-F

V. Apidopoulos, J.-F. Aujol, and Ch. Dossal, The differential inclusion modeling FISTA algorithm and optimality of convergence rate in the caseb≤3,SIAM J. Op- tim., 28(1), 551–574, 2018

2018

[2] [2]

Attouch, Z

H. Attouch, Z. Chbani, J. Peypouquet, and P. Redont, Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity.Math. Program. B., 168, 123–175, 2018. 4The convergence of the series can be easily analyzed by the Cauchy’s condensation test [18]. 19

2018

[3] [3]

Attouch, R

H. Attouch, R. I. Boţ, D. A. Hulett, and D.-K. Nguyen, Recovering Nesterov accelerated dynamics from Heavy Ball dynamics via time rescaling, arXiv, 2025. https://arxiv.org/abs/2504.15852

Pith/arXiv arXiv 2025

[4] [4]

Attouch and A

H. Attouch and A. Cabot, Convergence rates of inertial forward-backward algo- rithms.SIAM J. Optim., 28(1), 849–874, 2018

2018

[5] [5]

Attouch and J

H. Attouch and J. Peypouquet, The rate of convergence of Nesterov’s accelerated forward-backward method is actually faster then1/k2.SIAM J. Optim., 26(3), 1824– 1834, 2016

2016

[6] [6]

Aujol and Ch

J.-F. Aujol and Ch. Dossal, Stability of over-relaxations for the forward-backward algorithm, application to FISTA,SIAM J. Optim., 25(4), 2408–2433, 2015

2015

[7] [7]

Barré, A

M. Barré, A. B. Taylor, and F. Bach, Principled analyses and design of first-order methods with inexact proximal operators,Math. Program. B., 201, 185–230, 2023

2023

[8] [8]

H. H. Bauschke and P. L. CombettesConvex Analysis and Monotone Operator The- ory in Hilbert spaces, 2nd ed. Springer, New York, 2017

2017

[9] [9]

Beck and M

A. Beck and M. Teboulle, A Fast Iterative Shrinkage-Thresholding Algorithm for linear inverse problems,SIAM J. Imaging Sci., 2(1), 183–202, 2009

2009

[10] [10]

Bello-Cruz, M.K.N

Y. Bello-Cruz, M.K.N. Gonçalves, and N. Krislock On FISTA with relative error ruleComput. Optim. Appl., 84, 295–318, 2023

2023

[11] [11]

R. I. Boţ, E. Chenchene, E.R. Csetnek, and D. A. Hulett, Accelerating Diagonal Methods for Bilevel Optimization: Unified Convergence via Continuous-Time Dy- namics, arXiv, 2025.https://arxiv.org/abs/2505.14389

Pith/arXiv arXiv 2025

[12] [12]

R. I. Boţ, J. Fadili, and D.-K. Nguyen The iterates of Nesterov’s accelerated algo- rithm converge in the critical regime, arXiv, 2025.https://arxiv.org/abs/2510. 22715v2

2025

[13] [13]

Fast Iterative Shrinkage/Thresholding Algorithm

A. Chambolle and C. Dossal, On the convergence of the iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”.J. Optim. Theory Appl., 166, 968–982, 2015

2015

[14] [14]

D. L. Cohn,Measure Theory, 2nd ed. Birkhäuser, New York, 2013

2013

[15] [15]

P. L. Combettes, S. Salzo and S. Villa, Consistent learning by composite proximal thresholding.Math. Program. B., 167, 99–127, 2018

2018

[16] [16]

Güler, New proximal point algorithms for convex minimization.SIAM J

O. Güler, New proximal point algorithms for convex minimization.SIAM J. Optim. 2(4), 649–664, 1992

1992

[17] [17]

Jang and E.K

U. Jang and E.K. Ryu, Point convergence of Nesterov’s Accelerated Gradient Method: an AI-assisted proof, arXiv, 2025.https://arxiv.org/abs/2510.23513

arXiv 2025

[18] [18]

Knopp,Infinite Sequences and Series, Dover Publications, New York, 1956

K. Knopp,Infinite Sequences and Series, Dover Publications, New York, 1956

1956

[19] [19]

Nesterov, A method of solving a convex programming problem with convergence rateO(1/k 2),Dokl

Y. Nesterov, A method of solving a convex programming problem with convergence rateO(1/k 2),Dokl. Akad. Nauk SSSR, 27, 372–376, 1983

1983

[20] [20]

Opial, Weak convergence of the sequence of successive approximations for nonex- pansive mappings,Bull

Z. Opial, Weak convergence of the sequence of successive approximations for nonex- pansive mappings,Bull. Amer. Math. Soc., 73(4), 591–597, 1967

1967

[21] [21]

Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput

B.T. Polyak, Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. & Math. Phys., 4(5), 1–17, 1964

1964

[22] [22]

Robbins and D

H. Robbins and D. Siegmund, A convergence theorem for nonnegative almost supermartingales and some applications, in:Optimization Methods in Statistics, J. S. Rustagi, ed., pp. 233–257, Academic Press, New York, London, 1971. 20

1971

[23] [23]

Ryu,https://x.com/ernestryu/status/1980759528984686715?s=43, 2025

E. Ryu,https://x.com/ernestryu/status/1980759528984686715?s=43, 2025

arXiv 2025

[24] [24]

Salzo and S

S. Salzo and S. Villa Inexact and accelerated proximal point algorithms,J. Convex Anal., 19(4), 1167–1192, 2012

2012

[25] [25]

Schmidt, N

M. Schmidt, N. Le Roux, and F. Bach, Convergence Rates of Inexact Proximal- Gradient Methods for Convex Optimization, arXiv, 2011.https://arxiv.org/abs/ 1109.2415v2

Pith/arXiv arXiv 2011

[26] [26]

W. Su, S. Boyd, E J. Candes A differential equation for modeling Nesterov’s accel- erated gradient method: theory and insights,JMLR, 17(153), 1–43, 2016

2016

[27] [27]

Villa, S

S. Villa, S. Salzo, L. Baldassarre, A. Verri Accelerated and inexact forward-backward algorithms,SIAM J. Optim., 23(3), 1607–1633, 2013. A Appendix A.1 Proofs of Section 2 For reader’s convenience we give the proofs of some of the results presented in the pre- liminary section. Proof of Fact 2.1.Since0≤b k ≤α k −ak+1 +εk the summability of(bk)k∈N is immed...

2013