REVIEW 3 major objections 3 minor 31 references
Convergence analysis of nonmonotone proximal gradient methods under local Lipschitz continuity and Kurdyka--{\L}ojasiewicz property
T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Two nonmonotone proximal gradient methods converge as whole sequences under local Lipschitz continuity plus the Kurdyka–Łojasiewicz property, without global Lipschitz or bounded iterates.
desk verdict Real new result for the max line search case, but the average line search proof has a load-bearing gap in Lemma 3.6 that looks patchable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the partition of the iteration index set into good and bad subsets — $S$ and its complement for the average line search, $K$ and its complement for the max line search — together with the merit sequences $\Phi_{k+1}=(1-p_k)\Phi_k+p_k q(x_{k+1})$ and $q(x_{l(k)})=\max_{0\le j\le \min\{m,k\}} q(x_{k-j})$. The KL inequality $\chi'(q(x)-q(\bar x))\,\operatorname{dist}(0,\partial q(x))\ge1$ converts small subgradients into quantitative control of functional gaps, and the local Lipschitz continuity of $\nabla f$ on a compact ball supplies the bound $\operatorname{dist}(0,\partial q(x_{k+1}))\le(\bar\gamma_\rho+L_\rho)\|x_{k+1}-x_k\|$, with $\bar\gamma_\rho$ a uniform upper bound on accepted step sizes near an accumulation point. The parameter condition $p_{\min}>4/5$ keeps the ratio $\sqrt{(1-p_{\min})/p_{\min}}$ small enough that the geometric sums arising in the two cases can be absorbed into the common finite-length bound.
What would settle it
Run Algorithm 1 with $p_k=1$ (the monotone limit) and $\mu\le\delta\gamma_{\min}/2$ on a problem satisfying Assumption 3.1 whose iterates are not eventually constant; the acceptance criterion then yields $q(x_k)-\Phi_{k+1}\ge\frac{\delta\gamma_k}{2}\|x_{k+1}-x_k\|^2$, so the inequality defining $S$ in (3.14) can hold only in the boundary equality case, making $S$ empty for typical runs. A direct inspection of the set $S\cap K$ for a large $k_0$ in any such run would therefore test whether the selection step on which Theorem 3.8's summation (3.21) relies is actually satisfied.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that nonmonotone line search does not need to be repaired by enforcing descent everywhere; it can be repaired by separating indices. For Algorithm 1, the good set is $S=\{k: q(x_k)-\Phi_{k+1}\le \frac{\mu}{2}\|x_{k+1}-x_k\|^2\}$, where $\Phi_{k+1}=(1-p_k)\Phi_k+p_k q(x_{k+1})$ is the averaged merit value. On $S$ the acceptance rule implies the usual sufficient decrease and controls $\Phi_k-\Phi_{k+1}$ from below; on the complement, the KL inequality together with the local Lipschitz bound $\operatorname{dist}(0,\partial q(x_{k+1}))\le(\bar\gamma_\rho+L_\rho)\|x_{k+1}-x_k\|$ forces the functional gaps to be summable. The two cases are added to obtain a finite-length bound $\sum\|x_{i+1}-x_i\|<\infty$, so the whole sequence is Cauchy and converges to the accumulation point, which is M-stationary. The max-search algorithm is handled by the analogous partition $K=\{k: q(x_{l(k+1)})-q(x_{k+1})>\frac{\mu}{2}\|x_{k+1}-x_k\|^2\}$, using the maximal recent value $q(x_{l(k)})$ and an additional continuity assumption on $q$. The rates in Theorem 3.9 and Theorem 4.9 are then derived from a desingularization function of the form $\chi(t)=c t^\theta$.
Load-bearing premise
The load-bearing step is the selection of an index $k_0$ that is both late in the convergent subsequence and inside the 'good' index set where the gap between the objective value and the merit value is controlled by the squared step length; the proof establishes the smallness condition on $k_0$ but never verifies the good-set membership, which in the monotone limit may be the empty set.
Editorial extensions
If this is right
- The whole sequence generated by either algorithm converges to an accumulation point under local Lipschitz continuity and the KL property, extending the subsequential convergence already known from [19].
- Convergence rates are governed by the KL exponent: finite termination at $\theta=1$, Q-linear on $\Phi_k$ (or $q(x_{l(k)})$) and R-linear on the iterates for $\theta\in[1/2,1)$, and sublinear with exponents $1/(1-2\theta)$ and $\theta/(1-2\theta)$ for $\theta\in(0,1/2)$.
- The max-line-search analysis removes the auxiliary-sequence requirement and the bounded-subdifferential requirement that appeared in earlier KL-based nonmonotone convergence results.
- Because [18] covers the average-line-search algorithm with a different partition, the two works together support the conclusion that the convergence theory of the NPG method does not depend on how the index partition is chosen.
- No global Lipschitz condition and no boundedness of the iterates are needed, so the methods carry convergence certificates for problems whose gradients are only locally smooth.
Reading between the lines
- A natural extension, not made in the paper, is to test whether the same good/bad partition controls other nonmonotone schemes such as spectral or Barzilai–Borwein step sizes, since the proof only uses the acceptance inequality and the KL machinery.
- One could numerically audit the size of $S$ and $K$ along actual runs; the outcome would reveal whether the 'good' indices dominate the finite-length bound or whether the complement does most of the work, which could lead to shorter proofs with only one case.
- The 'independence of partitioning' observation suggests a meta-theorem: any partition that makes the good set nonempty near the limit and lets the KL inequality control the bad indices would give the same convergence conclusions.
- The rate formulas imply a directly testable prediction for applications: if the KL exponent $\theta$ can be estimated from the structure of $q$, the observed error decay of the iterates should scale as $k^{-\theta/(1-2\theta)}$ in the sublinear regime.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies two nonmonotone proximal gradient methods for composite minimization: Algorithm 1 with the Zhang--Hager average line search and Algorithm 2 with the Grippo--Lampariello--Lucidi max line search. Under a local Lipschitz continuity assumption on the gradient of the smooth part, a coercivity/lower-boundedness condition, and the Kurdyka--Łojasiewicz (KL) property at an accumulation point, the authors claim global convergence of the entire iterate sequence and Q-/R-linear or sublinear rates depending on the KL exponent (Theorems 3.8, 3.9, 4.8, 4.9). The proofs proceed by partitioning the iteration indices into two sets, one of which is meant to provide a sufficient decrease, and then deriving a finite-length (Cauchy) bound. The paper explicitly positions its index-partitioning strategy as an alternative to the concurrent work [18] and claims the convergence theory is independent of the specific partitioning.
Significance. If the claims were fully established, they would significantly extend the recent theory of proximal gradient methods without global Lipschitz assumptions to nonmonotone variants, and would remove the auxiliary-sequence and bounded-subdifferential requirements of earlier work in [26]. The observation that different index partitionings can lead to the same convergence conclusions is conceptually valuable. However, the main convergence proof relies on an existence assertion in Lemma 3.6 that is not proved and is in fact false in a permitted limiting case; a similar unverified membership appears in Lemma 4.5 for the max line search. Several key lemmas and the rate theorem are also deferred to the concurrent reference [18], leaving the central claims not fully self-contained. The manuscript is therefore not yet acceptable in its present form.
major comments (3)
- [Lemma 3.6, eqs. (3.14)--(3.15)] Lemma 3.6 asserts the existence of a sufficiently large k0 ∈ K ∩ S with α < 1/2, where S is defined in (3.14). The proof only selects k0 ∈ K large enough to make the right-hand side of (3.15) small; it never verifies that k0 belongs to S. This is not a cosmetic omission: in the permissible monotone limit p_k = 1, the acceptance criterion (3.2) gives q(x_k) − Φ_{k+1} = q(x_k) − q(x_{k+1}) ≥ δγ_k/2 ‖x_{k+1} − x_k‖² ≥ δγ_min/2 ‖x_{k+1} − x_k‖². Since µ ∈ (0, δp_minγ_min/2] and p_min = 1, any µ < δγ_min/2 makes the defining inequality of S fail for every k, so S is empty; even for µ = δγ_min/2, S is empty whenever γ_k > γ_min infinitely often. The membership of k0 in S is load-bearing: the summation identities in (3.21) and (3.27) explicitly use 'k0 ∈ S' to combine the index sums, and the finite-length bound (3.19) and the induction in Theorem 3.8 depend on that combination. Without a valid k0 ∈ S, the proof of Theorem 3.8 is not established, and Theorem 3.9 inherits the gap.
- [Lemma 4.5, eqs. (4.4) and (4.8)] Lemma 4.5 for the max line search displays the same structural gap as Lemma 3.6. The lemma states that there is a sufficiently large k0 ∈ K ∩ K with α < 1/2, where K is the set defined in (4.4). The proof only argues that a sufficiently large index in the subsequence K makes ‖x_{k0} − x̄‖, q(x_{l(k0)}) − q(x̄), and q(x_{l(k0−m−1)}) − q(x̄) small, and concludes α < 1/2. It never proves that k0 lies in the set K of (4.4). This membership is used later in Theorem 4.8: the base case of the induction relies on 'k0 ∈ K' to derive the inequality displayed after (4.14), and the subsequent case analysis splits on K. As written, Lemma 4.5 does not support that use, so the proof of Theorem 4.8 is incomplete.
- [Lemma 3.7 and Theorem 3.9] Several central proof components are omitted or deferred to the concurrent reference [18]. Lemma 3.7, which provides the subdifferential bound dist(0, ∂q(x_{k+1})) ≤ (γ̄_ρ + L_ρ)‖x_{k+1} − x_k‖, is stated with 'The proof is omitted because it is similar to [18, Lemma 4.3]'. Theorem 3.9, the advertised rate-of-convergence result for the average line search, is stated with 'Readers may find more details for the proof in [18, Theorem 4.6]'. A journal submission should be verifiable on its own; citing a concurrently posted preprint for a central rate theorem is not a substitute for a complete proof, especially because the partitioning in [18] differs from the one in (3.14), so it is not automatic that the rate argument transfers. The same concern applies to Lemma 3.4, whose proof is also omitted. This makes the paper non-self-contained at precisely the points that support the main claims.
minor comments (3)
- [Throughout] There are several typographical errors, for example 'Cauthy' at the end of the proof of Theorem 4.8 should be 'Cauchy', 'disingularizatioon' in the proof of Lemma 3.6 should be 'desingularization', 'replies heavily' in the Introduction should be 'relies heavily', and 'imediately' in the proof of Theorem 3.5 should be 'immediately'.
- [Section 4, eq. (4.4)] The symbol K is overloaded: it denotes the index set of a converging subsequence, e.g., {x_k}_{k∈K}, and also the newly defined set in (4.4). In Lemma 4.5 and Theorem 4.8 this double use is confusing, especially in expressions such as k0 ∈ K ∩ K; a different symbol for at least one of the two index sets would improve readability.
- [Proposition 4.4] The proof of Proposition 4.4 sets c := 1/‖x_{l(k̄+1)} − x_{l(k̄+1)−1}‖ without addressing the possibility that the denominator is zero; the argument should either rule out that case or handle it separately.
Circularity Check
No circularity: the derivation is a standard KL-based convergence argument with external lemmas, not a self-referential or fit-based construction.
full rationale
The paper does not exhibit any of the enumerated circularity patterns. No parameter is fitted to data and then relabeled as a prediction, no conclusion is built into the assumptions by definition, and no load-bearing step reduces to a chain of self-citations. The index partition S in (3.14) is a definition used to split the summation; the bounds in (3.21), (3.25), and (3.27) are derived from the acceptance criterion, the monotonicity of Phi_k, and the KL inequality rather than assumed from a prior work. The delegated proofs (Lemma 3.4, Lemma 3.7, and the rate part of Theorem 3.9) cite [18], an external and concurrent preprint by different authors, and standard published results [13, 16], so the non-self-contained portions are not self-referential. The one substantive concern, flagged per the reviewing rule, is a proof gap rather than circularity: Lemma 3.6 asserts the existence of k0 in K intersect S with only an argument that alpha < 1/2, and the subsequent summation bounds rely on this membership; in the monotone limit p_k = 1 the asserted membership can fail, so the index-splitting argument is incompletely justified. That is a correctness risk in the proof, not a reduction of the theorem to its own inputs, and therefore does not raise the circularity score.
Assumptions & free parameters
assumptions (8)
- domain assumption q is bounded from below on dom g (Assumption 3.1(a))
- domain assumption g is bounded from below by an affine function (Assumption 3.1(b))
- domain assumption The gradient of f is locally Lipschitz continuous (Assumption 3.1(c))
- domain assumption q has the Kurdyka-Lojasiewicz property at the accumulation point with desingularization function chi
- ad hoc to paper pmin > 4/5 in Algorithm 1
- domain assumption For the max line search variant, q is continuous on dom q (Assumption 4.1)
- domain assumption The generated sequence has an accumulation point x_bar
- standard math The space X is a finite-dimensional Euclidean space
Cite this review
Pith. "Pith review of Convergence analysis of nonmonotone proximal gradient methods under local Lipschitz continuity and Kurdyka--{\L}ojasiewicz property." pith.science (2026). https://pith.science/paper/OQAEWBYF
@misc{pith2026241119256,
author = {Pith},
title = {Pith review of: Convergence analysis of nonmonotone proximal gradient methods under local Lipschitz continuity and Kurdyka--\Lojasiewicz property},
year = {2026},
howpublished = {\url{https://pith.science/paper/OQAEWBYF}},
note = {Machine review of arXiv:2411.19256}
}
read the original abstract
The proximal gradient method is a standard approach for solving composite minimization problems in which the objective function is the sum of a continuously differentiable function and a lower semicontinuous, extended-valued function. The traditional convergence theory for both monotone and nonmonotone variants replies heavily on the assumption of global Lipschitz continuity of the gradient of the smooth part of the objective function. Recent work has shown that monotone proximal gradient methods converge globally only when the local (rather than global) Lipschitz continuity is assumed, provided that the Kurdyka--{\L}ojasiewicz (KL) property holds. However, these results have not been extended to nonmonotone proximal gradient (NPG) methods. In this manuscript, we consider two types of NPG methods: those combined with the average line search and the max line search, respectively. By partitioning indices into two subsets, one of which aims to achieve a sufficient decrease in the functional sequence, we establish global convergence and rate-of-convergence results using the local Lipschitz continuity and the KL property, without requiring boundedness of the iterates. While finalizing this work, we noticed that [18] presented analogous results for the NPG method with average line search, but with a different partitioning strategy. Together, we confidently conclude that the convergence theory of the NPG method is independent on index partitioning choices.
Reference graph
Works this paper leans on
-
[18]
C. Kanzow and L. Lehmann. Convergence of nonmonotone pr oxi- mal gradient methods under the Kurdyka–Łojasiewicz proper ty without a global lipschitz assumption. arXiv preprint arXiv:2411.12376 , 2024. doi:10.48550/arXiv.2411.12376
-
[26]
Y. Qian and S. Pan. Convergence of a class of nonmonotone descent methods for Kurdyka–Łojasiewicz optimization problems. SIAM Journal on Optimization , 33(2):638–651, 2023. doi:10.1137/22M1469663. 29
-
[1]
H. Attouch and J. Bolte. On the convergence of the proxima l algorithm for nonsmooth functions involving analytic features. Mathematical Programming, 116(1):5–16, 2009. doi:10.1007/s10107-007-0133-5
-
[2]
H. Attouch, J. Bolte, P. Redont, and A. Soubeyran. Proxim al alternating min- imization and projection methods for nonconvex problems: A n approach based on the Kurdyka-Łojasiewicz inequality. Mathematics of Operations Research , 35(2):438–457, 2010. doi:10.1287/moor.1100.0449
arXiv 2010
-
[3]
H. Attouch, J. Bolte, and B. F. Svaiter. Convergence of de scent methods for semi- algebraic and tame problems, proximal algorithms, forward -backward splitting, and regularized Gauss–Seidel methods. Mathematical Programming, 137:91 – 129, 2013. doi:10.1007/s10107-011-0484-9. 27
-
[4]
W. Bian and X. Chen. Linearly constrained non-Lipschitz optimization for image restoration. SIAM Journal on Imaging Sciences , 8(4):2294–2322, 2015. doi:10.1137/140985639
-
[5]
J. Bolte, A. Daniilidis, and A. Lewis. The Łojasiewicz in equality for nonsmooth subanalytic functions with applications to subgradient dy namical systems. SIAM Journal on Optimization , 17(4):1205–1223, 2007. doi:10.1137/050644641
-
[6]
J. Bolte, A. Daniilidis, A. Lewis, and M. Shiota. Clarke s ubgradients of stratifiable functions. SIAM Journal on Optimization , 18(2):556–572, 2007. doi:10.1137/060670080
Show all 31 references
-
[7]
Bolte, S
J. Bolte, S. Sabach, and M. Teboulle. Proximal alternati ng linearized mini- mization for nonconvex and nonsmooth problems. Mathematical Programming, 146:459 – 494, 2014. doi:10.1007/s10107-013-0701-9
2014 doi
-
[8]
Bolte, S
J. Bolte, S. Sabach, M. Teboulle, and Y. Vaisbourd. First order methods be- yond convexity and Lipschitz gradient continuity with appl ications to quadratic inverse problems. SIAM Journal on Optimization , 28(3):2131–2151, 2018. doi:10.1137/17M1138558
2018 doi
-
[9]
Bonettini, M
S. Bonettini, M. Prato, and S. Rebegoldi. A block coordin ate variable metric linesearch based proximal gradient method. Computational Optimization and Applications, 71(1):5–52, 2018. doi:10.1007/s10589-018-0011-5
2018 doi
-
[10]
R. I. Boţ and E. R. Csetnek. An inertial Tseng’s type prox imal algorithm for non- smooth and nonconvex optimization problems. Journal of Optimization Theory and Applications , 171(2):600–616, 2016. doi:10.1007/s10957-015-0730-z
2016 doi
-
[11]
R. I. Boţ, E. R. Csetnek, and S. C. László. An inertial for ward– backward algorithm for the minimization of the sum of two non convex functions. EURO Journal on Computational Optimization , 4(1):3–25, 2016. doi:10.1007/s13675-015-0045-8
2016 doi
-
[12]
A. M. Bruckstein, D. L. Donoho, and M. Elad. From sparse s olutions of systems of equations to sparse modelling of signals and images. SIAM Review, 51(1):34– 81, 2009. doi:10.1137/060657704
2009 doi
-
[13]
De Marchi
A. De Marchi. Proximal gradient methods beyond monoton y. Journal of Nonsmooth Analysis and Optimization , 4(10290), 2023. doi:10.46298/jnsao-2023-10290
2023 doi
-
[14]
Grippo, F
L. Grippo, F. Lampariello, and S. Lucidi. A nonmonotone line search technique for Newton’s method. SIAM Journal on Numerical Analysis , 23(4):707–716,
-
[15]
X. Jia. Augmented Lagrangian Methods invoking (Proximal) Gradient-type Methods for (Composite) Structured Optimiz ation 28 Problems. PhD thesis, Universität Würzburg, 2023. URL https://opus.bibliothek.uni-wuerzburg.de/frontdoor/index/index/docId/32374
2023
-
[16]
X. Jia, C. Kanzow, and P. Mehlitz. Convergence analysis of the proximal gradient method in the presence of the Kurdyka–Łojasiewicz property without global lipschitz assumptions. SIAM Journal on Optimization , 33(4):3038–3056, 2023. doi:10.1137/23M1548293
2023 doi
-
[17]
X. Jia, C. Kanzow, P. Mehlitz, and G. Wachsmuth. An augme nted Lagrangian method for optimization problems with structured geometri c constraints. Math- ematical Programming, 2022. doi:10.1007/s10107-022-01870-z
2022 doi
-
[19]
Kanzow and P
C. Kanzow and P. Mehlitz. Convergence properties of mon otone and nonmono- tone proximal gradient methods revisited. Journal of Optimization Theory and Applications, 195(2):624–646, 2022. doi:10.1007/s10957-022-02101-3
2022 doi
-
[20]
K. Kurdyka. On gradients of functions definable in o-min imal structures. Annales de l’institut Fourier , 48(3):769–783, 1998. doi:10.5802/aif.1638
1998 doi
-
[21]
Li and Z
H. Li and Z. Lin. Accelerated proximal gradient methods for non- convex programming. In Advances in Neural Information Pro- cessing Systems , volume 28. Curran Associates, Inc., 2015. URL https://proceedings.neurips.cc/paper_files/paper/2015/file/f7664060cc52bc6f3d620bc
2015
-
[22]
H. Liu, T. Wang, and Z. Liu. A nonmonotone accelerated pr oximal gra- dient method with variable stepsize strategy for nonsmooth and nonconvex minimization problems. Journal of Global Optimization , pages 1–35, 2024. doi:10.1007/s10898-024-01366-4
2024 doi
-
[23]
B. S. Mordukhovich. Variational Analysis and Applications . Springer, 2018. doi:10.1007/978-3-319-92775-6
2018 doi
-
[24]
P. Ochs. Local convergence of the heavy-ball method and iPiano for non-convex optimization. Journal of Optimization Theory and Applications , 177(1):153–180,
-
[25]
P. Ochs, Y. Chen, T. Brox, and T. Pock. iPiano: Inertial p roximal algorithm for nonconvex optimization. SIAM Journal on Imaging Sciences , 7(2):1388–1419,
- [27]
-
[28]
Wang and H
T. Wang and H. Liu. A class of modified accelerated proxim al gradient methods for nonsmooth and nonconvex minimization problems. Numerical Algorithms , 95(1):207–241, 2024. doi:10.1007/s11075-023-01569-y
2024 doi
-
[29]
Zhang and W
H. Zhang and W. W. Hager. A nonmonotone line search techn ique and its application to unconstrained optimization. SIAM Journal on Optimization , 14(4):1043–1056, 2004. doi:10.1137/S1052623403428208. 30
2004 doi
-
[2014]
doi:10.1137/130942954
-
[2018]
doi:10.1007/s10957-018-1272-y
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.