Pith. sign in

REVIEW 2 major objections 4 minor 47 references

Fully Convergent Projection-based Methods with Momentum under Nonconvex Geometric

T0 review · 2 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A projection-based method can carry momentum into nonconvex geometric constraints and still converge to the strongest stationarity conditions.

desk verdict Solid, mostly self-contained convergence theory for momentum in projection-based nonconvex optimization; the main caveat is that the title overpromises - the theorems are subsequential and require an accumulation point. read the letter →

arxiv 2607.22510 v1 pith:IOM2YADY submitted 2026-07-24 math.OC

Matteo Lapucci , Diego Scuppa This is my paper · ORCID
classification math.OC MSC 90C2690C3090C46
keywords ProjectedgradientmethodMomentumGeometricconstraintsNonconvexoptimizationM-stationarityB-stationarityP-stationarityGlobalconvergence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that projected-gradient methods can be given a momentum term without sacrificing their convergence guarantees on problems with nonconvex, geometrically structured constraints such as sparsity and low-rank sets. The central difficulty is that replacing the negative gradient by a momentum direction can turn the projected search path into an ascent path. The paper's mechanism is to backtrack along a curve in which the momentum component is damped faster than the step size, reverting toward the plain projected-gradient path as the step shrinks. Under this scheme, every accumulation point is shown to be M-stationary without any gradient smoothness assumption if the reversion is merely asymptotic, and B- or P-stationary if the momentum switches off exactly for small steps, with local Lipschitz gradient needed only for the latter. This matters because momentum typically accelerates convergence in practice, and the guarantee means that acceleration does not come at the cost of weaker limit points.

What carries the argument

The central mechanism is a curve backtracking step in which the momentum coefficient is damped as a power of the shrinking step size. The negative-gradient part scales linearly in alpha, while the momentum term scales like alpha^a (for tau=0, a>=2) or is identically zero for alpha<=tau (for tau>0); thus the search direction reverts to -alpha grad f(x_k) as alpha tends to zero. The proof uses the auxiliary quadratic model Q_kj(y) = <grad f(x_k) - (beta_kj/alpha_kj)(x_k - x_{k-1}), y - x_k> + (1/(2 alpha_kj)) ||y - x_k||^2, whose minimizer over the feasible set is exactly the projected trial point, and whose first-order optimality condition yields the normal-cone inclusions that are then passe

What would settle it

Run Algorithm 4.1 with tau=0 and a=2 on the paper's sparsity example: minimize x_1^2/2 + x_2^2 - 3 x_2 subject to ||x||_0 <= 1, starting at the nonstationary point (1,0) with a large momentum parameter beta_base chosen so the first trial direction points along (1,3). If for any parameter choice the backtracking loop fails to terminate at (1,0) or the iterates stall there forever, the claimed finite termination and M-stationarity convergence would be refuted.

Watch

Extended reading notes

Core claim

Algorithm 4.1 updates x_{k+1} in P_D(x_k + d_k(alpha_k)) with d_k(alpha) = -alpha grad f(x_k) + beta_base (max{alpha-tau,0}/(alpha_base-tau))^a (x_k - x_{k-1}). If tau=0 and a>=2, the momentum term vanishes faster than the step size as backtracking shrinks alpha, so the update asymptotically becomes a projected-gradient step; the paper proves that any accumulation point is M-stationary, provided the nonmonotone decrease rule holds and an accumulation point exists. If tau>0, then for alpha<=tau the update is exactly a projected-gradient step; this restores the stronger convergence of the plain method: accumulation points are B-stationary without smoothness assumptions, and P-stationary if gra

Load-bearing premise

The sequence of iterates must have at least one accumulation point; the algorithm has no mechanism forcing boundedness, and all stationarity conclusions are conditional on such a point existing.

Editorial extensions

If this is right

  • Momentum can be added to projection-based solvers for cardinality-constrained and low-rank problems without losing stationarity guarantees.
  • No global Lipschitz condition on the gradient is needed for M- and B-stationarity; local Lipschitz continuity suffices for P-stationarity.
  • The backtracking terminates finitely, and uniformly near nonstationary limit points when tau>0, so the method is implementable without prior smoothness constants.
  • Both max-type and average-type nonmonotone line searches are covered by the convergence analysis.
  • For bounded iterate sequences, the approximate P-stationarity measure is shown to converge to zero, making it a reliable stopping criterion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same revert-to-gradient curve could accommodate other accelerated directions, such as quasi-Newton directions, inside projection methods; the authors leave this as future work.
  • The tau=0, a>=2 case may also converge to B- and P-stationary points in practice, but the paper proves only M-stationarity there; a direct analysis or counterexample for that case would sharpen the theory.
  • The main gap is boundedness: the theorems are about accumulation points, and without a compactness safeguard an unbounded run gives no conclusion. A practical fix would be a proximal or trust-region term that forces boundedness.
  • Because the algorithm assumes an exact projection oracle for an arbitrary closed set, its practical scope is determined by how cheaply that projection can be computed for the specific geometric constraint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies minimization of a continuously differentiable function over a closed, possibly nonconvex feasible set D, assuming access to a projection operator onto D. It proposes Algorithm 4.1, a projected-gradient method with momentum: the tentative pre-projection point is x_k + d_k(α), where d_k(α) = -α∇f(x_k) + β_{k0}(max{α-τ,0}/(α_{k0}-τ))^a (x_k - x_{k-1}), and α is selected by backtracking against the sufficient-decrease condition (6). For τ=0 and a≥2, the momentum term vanishes faster than the gradient term along backtracking, and Theorem 4.5 shows every accumulation point is M-stationary under C^1 differentiability and either max- or average-rule nonmonotone safeguards, with no Lipschitz-gradient assumption. For τ>0 and a≥1, the path coincides with the standard projected-gradient path for α≤τ; Theorem 4.8 proves B-stationarity of accumulation points under C^1 differentiability, and Theorem 4.12 proves P-stationarity and the distance-to-normal-cone estimate (36) under local Lipschitz continuity of ∇f. Numerical experiments on cardinality-constrained and low-rank matrix problems compare the proposed GGMM with PGM and a momentum variant of PANOC+; the code is publicly available.

Significance. If the results hold, the paper makes a useful contribution: it integrates momentum into projection-based methods for nonconvex geometric constraints without global Lipschitz assumptions and without the double backtracking of FBE/PANOC-type frameworks, while recovering the strongest stationarity guarantees (B- and P-stationarity) when the search direction reverts exactly to the negative gradient for small steps (τ>0). The proofs are detailed and mostly self-contained; the only imported result is [25, Prop. 5.2], and the key ratio condition β_k/α_k→0 is proved rather than assumed. The theoretical analysis is complemented by reproducible numerical experiments with available code, comparing against a natural PGM baseline and a momentum-based PANOC+ variant. The central caveat, discussed below, is that the advertised 'fully convergent' claim is only an accumulation-point statement unless boundedness/coercivity is imposed.

major comments (2)
  1. [Abstract; §4, Theorems 4.5, 4.8, 4.12; Corollary 4.13] The headline claims — 'fully convergent' and 'the algorithm ... converges to M-/B-/P-stationary points' — are not supported as stated, because no hypothesis in Theorems 4.5, 4.8, or 4.12 guarantees that {x_k} admits an accumulation point. Assumption 4.2 only yields ||x_{k+1}-x_k||→0 once an accumulation point exists. Example: f(x)=e^{-x}, D=R, x_0=0, τ=0, β_max=0; L0=[0,∞), f is bounded below and uniformly continuous on L0, so Assumption 4.2(a) holds, but the iterates diverge and the stationarity conclusion is vacuous. Please add a coercivity/boundedness assumption (e.g., L0 bounded or {x_k} bounded) to the main theorems, or explicitly state throughout that the results are accumulation-point guarantees and adjust the abstract/title accordingly.
  2. [Corollary 4.13] The suggested sufficient condition for boundedness is misstated: 'if the sublevel set L0 = {x∈D | f(x)≤f(x0)} is bounded from below' does not imply that L0 is bounded. In the example f(x)=e^{-x}, D=R, L0=[0,∞) is bounded from below but unbounded, so the stated condition does not ensure the hypothesis of the corollary. This should be corrected to 'bounded' (e.g., by coercivity of f along D) or the suggestion should be removed.
minor comments (4)
  1. [Abstract; §1] The phrase 'without any smoothness assumptions' is too strong: the results assume f is continuously differentiable. What is not assumed is Lipschitz continuity of ∇f. Please clarify in the abstract and introduction.
  2. [Algorithm 4.1, Line 3 and Theorems 4.5/4.8/4.12] The phrase 'if, for every k, x_k is not a stationary point' is awkward and the stopping test is not generally checkable. Consider phrasing the theorems as 'as long as the algorithm does not terminate by the chosen stopping criterion'.
  3. [Section 5.2] The symbol τ_{k+1} used for the PANOC+ convex-combination parameter conflicts with the algorithmic parameter τ used throughout Section 4. Use a different symbol to avoid confusion.
  4. [References] Reference [41] is listed as 'Anonymous ... Under review'. If this is a double-blind submission this is fine, but otherwise it may hinder verification; please update it if possible.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the convergence theorems are proved in-paper from the algorithm's own decrease condition, and the imported external result [25, Prop. 5.2] is an independent benchmark.

full rationale

The paper's central derivation is self-contained. Algorithm 4.1 defines the search curve d_k(alpha) and the sufficient decrease condition (6); Theorems 4.5, 4.8 and 4.12 are proved directly from (6), the projection inequalities (2)-(3), stationarity definitions, and explicit limit arguments. The only imported external result is [25, Prop. 5.2], used in Proposition 4.7; it is an independent published benchmark with stated assumptions (continuous differentiability, non-B-stationarity) and does not smuggle in the target conclusion. The self-citations to [31], [37]-[41] are methodological references for momentum and curve searches, and they are not load-bearing: the convergence analysis does not reduce to them. No parameter is fitted to data and then reported as a prediction; the numerical section only benchmarks the algorithm. The convergence statements are explicitly subsequential ('any accumulation point'), and Proposition 4.3 assumes an accumulation point exists while Corollary 4.13 adds boundedness; whether the guarantee is vacuous when iterates drift to infinity is an assumption-strength/correctness issue, not circularity. The paper also candidly states that B- and P-stationarity convergence for tau = 0 remains an open question, further indicating that the proved results are not being claimed by definition.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central proof buys [25, Prop. 5.2] and standard variational-analysis machinery; it does not introduce fitted parameters or new entities. The only problem-structure assumptions are closedness of D, projection availability, C^1 objective, and, for P-stationarity, locally Lipschitz gradient.

assumptions (6)
  • domain assumption D is nonempty and closed and the projection operator P_D is available, possibly set-valued.
    Problem (1) and Algorithm 4.1 Lines 6/12 assume this; for nonconvex D projections can be nonunique, handled via set-valued PD.
  • standard math Projection inequalities (2)-(3) from [25, Prop. 2.1] hold.
    Used throughout the proof, e.g., in Propositions 4.7 and 4.10 to bound projected-path distances.
  • standard math [25, Prop. 5.2] (Olikier-Waldspurger): near a non-B-stationary point, pure projected-gradient steps satisfy a decrease condition for an interval of step sizes.
    Imported external theorem; central to Proposition 4.7 for B-stationarity.
  • standard math Variational-analysis normal-cone facts: inclusions bN_P^D subset bN_D subset N_D, outer semicontinuity of N_D, and proximal-normal distance characterization (Remark 2.6).
    Used in Theorems 4.5, 4.8, and 4.12 and in the stationarity definitions.
  • domain assumption For the max-rule, f is bounded below and uniformly continuous on L0 = {x in D : f(x) <= f(x0)}.
    Assumption 4.2(a), required for Proposition 4.3 to obtain lim ||x_{k+1}-x_k|| = 0.
  • domain assumption f is continuously differentiable; for P-stationarity, grad f is locally Lipschitz continuous.
    Stated in Theorems 4.5, 4.8, and 4.12; not a global Lipschitz assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fully Convergent Projection-based Methods with Momentum under Nonconvex Geometric." pith.science (2026). https://pith.science/paper/IOM2YADY

@misc{pith2026260722510,
  author       = {Pith},
  title        = {Pith review of: Fully Convergent Projection-based Methods with Momentum under Nonconvex Geometric},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IOM2YADY}},
  note         = {Machine review of arXiv:2607.22510}
}
read the original abstract

Nonlinear optimization problems with complicated, nonconvex, yet geometrically structured constraints can be tackled by projected-gradient methods: under weak regularity assumptions, these approaches were recently proved to possess convergence properties to the strongest stationarity conditions. In this work, we show how momentum terms, commonly used in nonlinear optimization to speed up the convergence process, can be integrated within this algorithmic framework without harming convergence guarantees. Preliminarily, we highlight an intrinsic issue induced by the direct replacement of the negative gradient with a general descent direction within the projected approach. Then, we present suitable backtracking mechanisms for the pre-projection step, allowing us to integrate momentum terms in the direction. By this technique, we can specifically ensure, without any smoothness assumptions, convergence to Mordukhovich stationarity as long as the base directions asymptotically revert to the negative gradient for small stepsizes; moreover, if the base search direction reverts exactly to the negative gradient for the smallest steps, the algorithm is proved to converge to Bouligand and Proximally stationary points, with and without (local) smoothness assumptions respectively. Finally, the proposed procedure is numerically tested on some classes of problems, namely, sparsity and bounded-rank constrained problems; the results indicate that the proposed method is computationally effective, taking advantage of the additional information provided by the momentum term.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 13 canonical work pages

  1. [1]

    32 Mathematical Programming199(1), 1365–1415 (2023) https://doi.org/10.1007/ s10107-022-01870-z

    Jia, X., Kanzow, C., Mehlitz, P., Wachsmuth, G.: An augmented lagrangian method for optimization problems with structured geometric constraints. 32 Mathematical Programming199(1), 1365–1415 (2023) https://doi.org/10.1007/ s10107-022-01870-z

    2023

  2. [2]

    Mathematical Programming201(1), 863–896 (2023) https://doi.org/10.1007/s10107-022-01922-4

    De Marchi, A., Jia, X., Kanzow, C., Mehlitz, P.: Constrained composite optimiza- tion and augmented lagrangian methods. Mathematical Programming201(1), 863–896 (2023) https://doi.org/10.1007/s10107-022-01922-4

    doi:10.1007/s10107-022-01922-4 2023

  3. [3]

    Computational Optimization and Applications85(3), 937–971 (2023) https://doi.org/10.1007/s10589-023-00475-2

    Kanzow, C., Lapucci, M.: Inexact penalty decomposition methods for opti- mization problems with geometric constraints. Computational Optimization and Applications85(3), 937–971 (2023) https://doi.org/10.1007/s10589-023-00475-2

    work page doi:10.1007/s10589-023-00475-2 2023

  4. [4]

    SIAM Journal on Optimization9(2), 374–387 (1999) https://doi.org/ 10.1137/S1052623497321882

    Ye, J.: Optimality conditions for optimization problems with complementarity constraints. SIAM Journal on Optimization9(2), 374–387 (1999) https://doi.org/ 10.1137/S1052623497321882

    work page doi:10.1137/s1052623497321882 1999

  5. [5]

    Computational Optimization and Applications13(1), 111–136 (1999) https://doi.org/10.1023/A:1008656806889

    Pang, J.-S., Fukushima, M.: Complementarity constraint qualifications and sim- plified B-stationarity conditions for mathematical programs with equilibrium constraints. Computational Optimization and Applications13(1), 111–136 (1999) https://doi.org/10.1023/A:1008656806889

    work page doi:10.1023/a:1008656806889 1999

  6. [6]

    Mathematical Programming 114(1), 69–99 (2008) https://doi.org/10.1007/s10107-006-0083-3

    Achtziger, W., Kanzow, C.: Mathematical programs with vanishing constraints: optimality conditions and constraint qualifications. Mathematical Programming 114(1), 69–99 (2008) https://doi.org/10.1007/s10107-006-0083-3

    work page doi:10.1007/s10107-006-0083-3 2008

  7. [7]

    Mathematical Programming181(1), 149–186 (2020) https://doi

    Mehlitz, P.: Stationarity conditions and constraint qualifications for mathematical programs with switching constraints: with applications to either-or-constrained programming. Mathematical Programming181(1), 149–186 (2020) https://doi. org/10.1007/s10107-019-01380-5

    work page doi:10.1007/s10107-019-01380-5 2020

  8. [8]

    SIAM Journal on Optimization23(3), 1480–1509 (2013) https://doi.org/10.1137/120869778

    Beck, A., Eldar, Y.C.: Sparsity constrained nonlinear optimization: Optimal- ity conditions and algorithms. SIAM Journal on Optimization23(3), 1480–1509 (2013) https://doi.org/10.1137/120869778

    work page doi:10.1137/120869778 2013

Show all 47 references
  1. [9]

    Mathematical Programming 199(1), 831–864 (2023) https://doi.org/10.1007/s10107-022-01851-2

    Levin, E., Kileel, J., Boumal, N.: Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy. Mathematical Programming 199(1), 831–864 (2023) https://doi.org/10.1007/s10107-022-01851-2

    2023 doi

  2. [10]

    Mathematics of Operations Research (2026) https://doi.org/10.1287/ moor.2024.0582

    Olikier, G., Gallivan, K.A., Absil, P.-A.: Low-rank optimization methods based on projected-projected gradient descent that accumulate at Bouligand station- ary points. Mathematics of Operations Research (2026) https://doi.org/10.1287/ moor.2024.0582

    2026

  3. [11]

    Athena Scientific, Nashua (1995)

    Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Nashua (1995)

    1995

  4. [12]

    Grippo, L., Sciandrone, M.: Introduction to Methods for Nonlinear Optimization vol. 152. Springer, Cham (2023) 33

    2023

  5. [13]

    Journal of Statistical Software60, 1–21 (2014) https: //doi.org/10.18637/jss.v060.i03

    Birgin, E.G., Mart ´ ınez, J.M., Raydan, M.: Spectral projected gradient methods: review and perspectives. Journal of Statistical Software60, 1–21 (2014) https: //doi.org/10.18637/jss.v060.i03

    2014 doi

  6. [14]

    Journal of Fourier Analysis and Applications14, 629–654 (2008) https://doi.org/ 10.1007/s00041-008-9035-z

    Blumensath, T., Davies, M.: Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications14, 629–654 (2008) https://doi.org/ 10.1007/s00041-008-9035-z

    2008 doi

  7. [15]

    SIAM Journal on Numerical Analysis23(4), 707–716 (1986) https://doi.org/10.1137/0723046

    Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for newton’s method. SIAM Journal on Numerical Analysis23(4), 707–716 (1986) https://doi.org/10.1137/0723046

    1986 doi

  8. [16]

    SIAM Journal on Optimization14(4), 1043–1056 (2004) https://doi.org/10.1137/S1052623403428208

    Zhang, H., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM Journal on Optimization14(4), 1043–1056 (2004) https://doi.org/10.1137/S1052623403428208

    2004 doi

  9. [17]

    SIAM Journal on Optimization10(4), 1196–1211 (2000) https://doi.org/10.1137/S1052623497330963

    Birgin, E.G., Mart ´ ınez, J.M., Raydan, M.: Nonmonotone spectral projected gra- dient methods on convex sets. SIAM Journal on Optimization10(4), 1196–1211 (2000) https://doi.org/10.1137/S1052623497330963

    2000 doi

  10. [18]

    Journal of Optimization Theory and Appli- cations195(2), 624–646 (2022) https://doi.org/10.1007/s10957-022-02101-3

    Kanzow, C., Mehlitz, P.: Convergence properties of monotone and nonmonotone proximal gradient methods revisited. Journal of Optimization Theory and Appli- cations195(2), 624–646 (2022) https://doi.org/10.1007/s10957-022-02101-3

    2022 doi

  11. [19]

    Journal of Non- smooth Analysis and Optimization4(Original research articles) (2023) https: //doi.org/10.46298/jnsao-2023-10290

    De Marchi, A.: Proximal gradient methods beyond monotony. Journal of Non- smooth Analysis and Optimization4(Original research articles) (2023) https: //doi.org/10.46298/jnsao-2023-10290

    2023 doi

  12. [20]

    In: 52nd IEEE Conference on Decision and Control, pp

    Patrinos, P., Bemporad, A.: Proximal Newton methods for convex composite optimization. In: 52nd IEEE Conference on Decision and Control, pp. 2358–2363 (2013). https://doi.org/10.1109/CDC.2013.6760233 . IEEE

    2013

  13. [21]

    In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp

    Stella, L., Themelis, A., Sopasakis, P., Patrinos, P.: A simple and efficient algorithm for nonlinear model predictive control. In: 2017 IEEE 56th Annual Conference on Decision and Control (CDC), pp. 1939–1944 (2017). https://doi. org/10.1109/CDC.2017.8263933 . IEEE

    2017

  14. [22]

    SIAM Journal on Optimization28(3), 2274–2303 (2018) https://doi.org/ 10.1137/16M1080240

    Themelis, A., Stella, L., Patrinos, P.: Forward-backward envelope for the sum of two nonconvex functions: Further properties and nonmonotone linesearch algo- rithms. SIAM Journal on Optimization28(3), 2274–2303 (2018) https://doi.org/ 10.1137/16M1080240

    2018 doi

  15. [23]

    Journal of Optimization Theory and Applications194(3), 771–794 (2022) https://doi.org/ 10.1007/s10957-022-02048-5 34

    De Marchi, A., Themelis, A.: Proximal gradient algorithms under local lipschitz gradient continuity: A convergence and robustness analysis of panoc. Journal of Optimization Theory and Applications194(3), 771–794 (2022) https://doi.org/ 10.1007/s10957-022-02048-5 34

    2022 doi

  16. [24]

    Springer, Berlin, Heidelberg (1998)

    Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin, Heidelberg (1998)

    1998

  17. [25]

    SIAM Journal on Optimization35(2), 1004–1029 (2025) https: //doi.org/10.1137/24M1692782

    Olikier, G., Waldspurger, I.: Projected gradient descent accumulates at bouligand stationary points. SIAM Journal on Optimization35(2), 1004–1029 (2025) https: //doi.org/10.1137/24M1692782

    2025 doi

  18. [26]

    Optimization66(1), 61–92 (2017) https:// doi.org/10.1080/02331934.2016.1252915

    Benko, M., Gfrerer, H.: On estimating the regular normal cone to constraint systems and stationarity conditions. Optimization66(1), 61–92 (2017) https:// doi.org/10.1080/02331934.2016.1252915

    2017

  19. [27]

    Springer, Cham (2018)

    Mordukhovich, B.S.: Variational Analysis and Applications. Springer, Cham (2018)

    2018

  20. [28]

    arXiv preprint arXiv:2402.09831 (2024)

    Pauwels, E.: A note on stationarity in constrained optimization. arXiv preprint arXiv:2402.09831 (2024)

    2024 arXiv

  21. [29]

    SIAM Journal on Optimization 30(4), 3069–3097 (2020) https://doi.org/10.1137/19M1254155

    Bonettini, S., Prato, M., Rebegoldi, S.: Convergence of inexact forward–backward algorithms using the forward–backward envelope. SIAM Journal on Optimization 30(4), 3069–3097 (2020) https://doi.org/10.1137/19M1254155

    2020 doi

  22. [30]

    Ussr computational mathematics and mathematical physics4(5), 1–17 (1964) https://doi.org/10.1016/0041-5553(64)90137-5

    Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. Ussr computational mathematics and mathematical physics4(5), 1–17 (1964) https://doi.org/10.1016/0041-5553(64)90137-5

    1964 doi

  23. [31]

    Computational Optimization and Applications93(2), 795–820 (2026) https://doi.org/10.1007/s10589-025-00741-5

    Lapucci, M., Liuzzi, G., Lucidi, S., Pucci, D., Sciandrone, M.: A globally con- vergent gradient method with momentum. Computational Optimization and Applications93(2), 795–820 (2026) https://doi.org/10.1007/s10589-025-00741-5

    2026 doi

  24. [32]

    In: 2015 European Control Confer- ence (ECC), pp

    Ghadimi, E., Feyzmahdavian, H.R., Johansson, M.: Global convergence of the heavy-ball method for convex optimization. In: 2015 European Control Confer- ence (ECC), pp. 310–315 (2015). https://doi.org/10.1109/ECC.2015.7330562 . IEEE

    2015

  25. [33]

    Advances in neural information processing systems 32(2019) https://doi.org/10.5555/3454287.3455151

    Gitman, I., Lang, H., Zhang, P., Xiao, L.: Understanding the role of momentum in stochastic gradient methods. Advances in neural information processing systems 32(2019) https://doi.org/10.5555/3454287.3455151

    2019

  26. [34]

    In: International Conference on Machine Learning, pp

    Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initializa- tion and momentum in deep learning. In: International Conference on Machine Learning, pp. 1139–1147 (2013). https://doi.org/10.5555/3042817.3043064 . pmlr

    2013

  27. [35]

    Mathematical Programming Computation14(3), 543–591 (2022) https: //doi.org/10.1007/s12532-022-00219-z 35

    Lee, C.-p., Wang, P.-W., Lin, C.-J.: Limited-memory common-directions method for large-scale optimization: convergence, parallelization, and distributed opti- mization. Mathematical Programming Computation14(3), 543–591 (2022) https: //doi.org/10.1007/s12532-022-00219-z 35

    2022 doi

  28. [36]

    SIAM Jour- nal on Scientific Computing46(3), 2025–2046 (2024) https://doi.org/10.1137/ 23M1567229

    Tang, T., Toh, K.-C., Xiao, N., Ye, Y.: A riemannian dimension-reduced second-order method with application in sensor network localization. SIAM Jour- nal on Scientific Computing46(3), 2025–2046 (2024) https://doi.org/10.1137/ 23M1567229

    2025

  29. [37]

    arXiv preprint arXiv:2603.04078 (2026)

    Leggio, F., Scuppa, D.: Riemannian gradient method with momentum. arXiv preprint arXiv:2603.04078 (2026)

    2026

  30. [38]

    Journal of Optimization Theory and Applications 210(20) (2026) https://doi.org/10.1007/s10957-026-03054-7

    Lapucci, M., Liuzzi, G., Lucidi, S., Sciandrone, M., Scuppa, D.: Projected gradi- ent methods with momentum. Journal of Optimization Theory and Applications 210(20) (2026) https://doi.org/10.1007/s10957-026-03054-7

    2026 doi

  31. [39]

    Journal of Optimization Theory and Application210(5) (2026) https://doi.org/10.1007/ s10957-026-03035-w

    Donnini, F., Lapucci, M., Mansueto, P.: Efficient globalization of heavy-ball type methods for unconstrained optimization based on curve searches. Journal of Optimization Theory and Application210(5) (2026) https://doi.org/10.1007/ s10957-026-03035-w

    2026

  32. [40]

    Optimization Methods and Software, 1–23 (2026) https://doi.org/10.1080/10556788.2026.2647280

    Jia, X., Lapucci, M., Mansueto, P.: Projection-based curve pattern search for black-box optimization over smooth convex sets. Optimization Methods and Software, 1–23 (2026) https://doi.org/10.1080/10556788.2026.2647280

    2026

  33. [41]

    Submitted to Transactions on Machine Learning Research (2026)

    Anonymous: Safely exploring large momentum steps with stochastic curve searches. Submitted to Transactions on Machine Learning Research (2026). Under review

    2026

  34. [42]

    Journal of Global Optimization82(2), 219– 242 (2022) https://doi.org/10.1007/s10898-021-01070-7

    L¨ ammel, S., Shikhman, V.: On nondegenerate M-stationary points for sparsity constrained nonlinear optimization. Journal of Global Optimization82(2), 219– 242 (2022) https://doi.org/10.1007/s10898-021-01070-7

    2022 doi

  35. [43]

    arXiv preprint arXiv:2303.16040 (2023)

    Olikier, G., Gallivan, K.A., Absil, P.-A.: First-order optimization on stratified sets. arXiv preprint arXiv:2303.16040 (2023)

    2023 arXiv

  36. [44]

    Mathematical Programming91(2), 201–213 (2002) https://doi.org/10

    Dolan, E.D., Mor´ e, J.J.: Benchmarking optimization software with performance profiles. Mathematical Programming91(2), 201–213 (2002) https://doi.org/10. 1007/s101070100263

    2002

  37. [45]

    IMA Journal of Numerical Analysis8, 141–148 (1988) https://doi.org/10.1093/imanum/8.1.141

    Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA Journal of Numerical Analysis8, 141–148 (1988) https://doi.org/10.1093/imanum/8.1.141

    1988 doi

  38. [46]

    SIAM Journal on Optimization16(1), 170–192 (2005) https://doi.org/10.1137/030601880

    Hager, W.W., Zhang, H.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM Journal on Optimization16(1), 170–192 (2005) https://doi.org/10.1137/030601880

    2005 doi

  39. [47]

    https://arxiv.org/abs/1209.3672 36 A Additional Proofs Proof of Proposition 4.3.Let{x k}be the infinite sequence generated by Algo- rithm 4.1

    Davenport, M.A., Plan, Y., Berg, E., Wootters, M.: 1-Bit Matrix Completion (2014). https://arxiv.org/abs/1209.3672 36 A Additional Proofs Proof of Proposition 4.3.Let{x k}be the infinite sequence generated by Algo- rithm 4.1. Then, for everyk∈N, since the inner loop terminates...

    2014 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.