REVIEW 2 major objections 4 minor 30 references
Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions
T0 review · 2 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Two linesearch-based generalized conditional gradient methods converge to stationary points of a composite objective even when the gradient of the smooth part is merely continuous, with no Lipschitz or Hölder condition assumed.
desk verdict Useful convergence proofs for generalized conditional gradient methods without gradient smoothness; the main theorems hold, but Lemma 3.2's termination claim needs a small patch at stationary points. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Frank-Wolfe gap, G(x) := max_v {⟨∇f(x), x − v⟩ + g(x) − g(v)}, a nonnegative, lower-semicontinuous optimality measure that vanishes exactly at stationary points of (1). The mechanism carrying both proofs is the descent estimate of Lemma 3.1, F(x + td) ≤ F(x) − tG(x) + o(t) for t ∈ [0,1] and d a subproblem solution direction, which follows from continuous differentiability of f and convexity of g alone. Armijo-type backtracking then guarantees a function decrease proportional to the gap at every iteration — for Algorithm 1, F_{k+1} ≤ F_k − $pσβ^{{i_k}}$G(x_k), and for Algorithm 2, F(x_{k+1}) ≤ F(x_k) − (1/4)τ_k G(x_k) — so the gap is driven to zero along any subsequence where the stepsize does not degenerate. In the vanishing-stepsize case, rejection of the trial point forces inequalities (7) and (10), which bound the gap by ‖∇f(ξ_k) − ∇f(x_k)‖‖d_k‖ with ξ_k a mean-value point converging to the same limit as x_k; continuity of ∇f, rather than any quantitative modulus, sends that bound to zero, while boundedness of the directions d_k is secured by supercoercivity of g.
What would settle it
Concrete test: with g the indicator of [−1,1] and f(x) = x + x² sin(1/x²), the smooth part is differentiable everywhere with f'(0) = 1 while f' is discontinuous and unbounded at 0, so 0 is a non-stationary point. Run Algorithms 1 and 2 from several starting points and record the Frank-Wolfe gap G(x_k); iterates accumulating at 0 with a gap bounded away from zero would show where the continuous-gradient hypothesis is doing its work, while a gap that always tends to zero would suggest the theorem survives weaker assumptions. Independently, verifying inequality (7) numerically during a run on an ordinary C¹ but non-Lipschitz objective would confirm the proof's load-bearing estimate holds as claimed.
Extended reading notes
Core claim
The paper's central claim is a pair of subsequential convergence theorems. Theorem 3.1: for the composite problem min F = f + g with g proper, closed, convex, and supercoercive and f continuously differentiable on dom g, any accumulation point of the sequence generated by the generalized conditional gradient method with average-type nonmonotone Armijo linesearch (Algorithm 1) is stationary; no Lipschitz or Hölder assumption on ∇f appears anywhere in the hypotheses. Theorem 3.2: the same conclusion holds for the parameter-free backtracking variant of [16] (Algorithm 2), giving an accumulation point that is stationary and min_{0≤l≤k} G(x_l) → 0 as k → ∞, under the sole added assumption that the lower level set {x : F(x) ≤ F(x0)} is bounded — a condition automatic when f is convex or dom g is bounded. In the delicate regime where backtracking stepsizes shrink to zero, the mean-value theorem produces a point ξ_k between x_k and the rejected trial point, and the linesearch rejection inequality yields (1−σ)G(x_k) ≤ ‖∇f(ξ_k) − ∇f(x_k)‖ ‖d_k‖; because ξ_k and x_k converge to the same limit and ∇f is continuous, the gap must vanish. The paper thus establishes convergence under a mere continuity condition on the gradient, where the prior literature required quantitative smoothness.
Load-bearing premise
The load-bearing premise is continuity of the gradient ∇f of the smooth part at the accumulation point, used to conclude that ‖∇f(ξ_k) − ∇f(x_k)‖ shrinks to zero when the trial point ξ_k and the iterate x_k approach the same limit, and, for the parameter-free variant, boundedness of the lower level set {x : F(x) ≤ F(x0)}.
Editorial extensions
If this is right
- Corollary 3.1: if an accumulation point exists — e.g. when dom g is bounded — the minimum Frank-Wolfe gap over the first k iterates tends to zero, so an ε-approximate stationary point is obtained in finitely many iterations.
- Theorem 3.2: the parameter-free variant converges as soon as the lower level set is bounded, a condition that holds automatically when f is convex or when dom g is bounded.
- The nonmonotone Armijo result extends the smoothness-free analysis of [2] beyond compact convex constraint sets, and the monotone backtracking method of [22] (p = 1, σ ≤ 1/2) is covered as a special case.
- Because the parameter-free method's Hölder-adaptive rate remains valid when ∇f is Hölder continuous, skipping a Hölder check loses the rate only in cases where no Hölder exponent actually exists.
Reading between the lines
- The proofs appear to consume only continuity of ∇f at the accumulation point and boundedness of the directions d_k, so the hypothesis 'continuously differentiable on dom g' could likely be weakened to differentiability with locally continuous gradient on the lower level set.
- If ∇f is known only through a modulus of continuity ω(δ), the same inequalities should yield a quantitative gap decay that interpolates between the O(ε^{−1−1/ν}) rate for ω(δ) = Cδ^ν and the purely asymptotic guarantee when no modulus exists.
- Because every accumulation point must be stationary, any counterexample to whole-sequence convergence under Armijo stepsizes would have to unwind along the stationary set itself, which constrains the search left open by the divergence counterexamples the paper cites.
- The mean-value-plus-linesearch-rejection mechanism should transfer to linesearch-based proximal gradient methods, producing a common smoothness-free convergence theory for the two principal composite solvers.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper analyzes two linesearch-based generalized conditional gradient methods for the composite problem min F(x)=f(x)+g(x), where f is continuously differentiable on dom g (but not assumed to have a Lipschitz or Hölder continuous gradient) and g is proper, closed, convex, and supercoercive. The main results are Theorem 3.1, asserting that every accumulation point of the iterates of a nonmonotone Armijo-type method (Algorithm 1) is stationary, and Theorem 3.2, asserting that under a bounded-level-set condition the parameter-free variant of Ito, Lu, and He (Algorithm 2) has at least one stationary accumulation point and its running minimum Frank-Wolfe gap tends to zero. The key new ingredient is Lemma 3.1, which derives the o(t) descent estimate F(x+td) <= F(x)-tG(x)+o(t) using only continuous differentiability of f and convexity of g.
Significance. If the technical gaps identified below are fixed, this is a worthwhile contribution: it removes global Lipschitz/Hölder smoothness assumptions that are standard in essentially all existing convergence analyses of generalized conditional gradient methods, and it shows that a practically attractive parameter-free backtracking variant remains convergent even when the Hölder condition is unknown or absent. The proofs are short, elementary, and mostly self-contained, and Lemma 3.1 is a clean reusable tool. The extension of Bertsekas-style subsequential convergence from constrained smooth problems to the composite, nonconvex, supercoercive setting is a genuine advance. However, the well-definedness of Algorithm 1 has a real gap at stationary points, so the paper is not yet fully sound as written.
major comments (2)
- [Section 3, Lemma 3.2] Lemma 3.2 is false when G(x_k)=0 and the subproblem oracle returns a minimizer v_k different from x_k. The proof applies Lemma 3.1 and concludes F(x_{k,i}) <= F(x_k) - sigma beta^i G(x_k) for all sufficiently large i, but when G(x_k)=0 this reduces to F(x_{k,i}) <= F(x_k)+o(beta^i), which does not force the Armijo condition (4) to hold because o(beta^i) may be positive. Concretely, take E=R, f(x)=x^2, g(x)=max(0,|x|-1)^2, x_0=0; then all standing assumptions hold and x_0 is stationary with G(x_0)=0, but the subproblem minimizer v=1 gives d=1 and F(beta^i)=beta^{2i}>0=F(x_0), so (4) fails for every finite i. The algorithm therefore may never produce an infinite sequence from a stationary point. The convergence theorems would still hold for sequences that are actually generated, but the well-definedness claim in Lemma 3.2 and the pseudocode need a patch, such as an explicit stopping rule 'if G(x_k)=0, stop' or a tie-breaking rule that selects d_k=0 at stationary points.
- [Section 3.1, Algorithm 2] The definition of the trial step size in Algorithm 2 is incomplete when d_k=0. The formula tau_k^(i)=min{1, G(x_k)/(2L_k^(i)||d_k||^2)} involves division by ||d_k||^2, and d_k=0 occurs exactly when v_k=x_k, i.e., when x_k is a minimizer of the linearized subproblem. Since -grad f(x_k) in partial g(x_k) is equivalent to G(x_k)=0 by Lemma 2.1, this is precisely the stationary case. The algorithm needs an explicit convention for d_k=0, e.g., tau_k^(i)=1 or termination when G(x_k)=0. Without this, the statement of Proposition 3.2 and Lemma 3.3 do not formally cover all admissible outputs of a subproblem solver.
minor comments (4)
- [Abstract and Introduction] The abstract and introduction say the methods 'guarantee convergence' even without Hölder continuity, but the theorems establish subsequential convergence and convergence of the running minimum of the Frank-Wolfe gap, not convergence of the whole sequence. Please make this distinction explicit in the abstract.
- [Section 3, Lemma 3.2 proof, Eq. (5)] In the displayed inequality (5), the exponent on beta appears to be i_{k-1}, but the notation is easy to misread as i_k - 1. Please add brackets or a clarifying sentence so that the index is unambiguous.
- [Footnote, uniform convex structure] The footnote defining uniform convex structure states 'rho >= 21' near the end of Section 3; this is presumably rho >= 2 and should be corrected.
- [References] Reference [18] contains the odd text 'Kurdyka–/suppress Lojasiewicz'; this should read 'Kurdyka–Łojasiewicz property'.
Circularity Check
No circularity: the convergence theorems are derived self-containedly from the problem definitions and the algorithm line-search rules.
full rationale
The paper's convergence proofs are direct mathematical derivations. Lemma 3.1 uses only continuous differentiability of f, convexity of g, and the definition of the Frank-Wolfe gap G(x); Lemma 2.2 is a definitional lower-semicontinuity argument. Theorems 3.1 and 3.2 then combine these lemmas with the descent/acceptance inequalities (4) and (8), monotonicity of F_k or F(x_k), boundedness of the subproblem minimizers via supercoercivity, and continuity of ∇f. No parameter is fitted to any subset of data, and no 'prediction' is derived from a quantity already used as an input; the stationarity conclusion comes from showing G(x*) = 0, which is equivalent to (2) by the cited definitional Lemma 2.1. Algorithm 2 is taken from Ito et al. [16], but Theorem 3.2 does not rely on any convergence claim from that work; it proves convergence directly under the stated assumptions and the algorithm's own backtracking definition. The only related concerns are non-circular: the title's 'without smoothness assumptions' is stronger than the actual C^1 assumption on f, and the proof of Lemma 3.2 may need a separate termination rule at stationary points where G(x_k)=0. These are correctness or assumption-strength issues, not cases where a claim reduces to its inputs by construction.
Assumptions & free parameters
assumptions (3)
- domain assumption f is lower semicontinuous and continuously differentiable on dom g
- domain assumption g is proper closed convex and supercoercive
- domain assumption The lower level set {x : F(x) <= F(x0)} is bounded
Cite this review
Pith. "Pith review of Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions." pith.science (2026). https://pith.science/paper/ATFHVJ5K
@misc{pith2026250501092,
author = {Pith},
title = {Pith review of: Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions},
year = {2026},
howpublished = {\url{https://pith.science/paper/ATFHVJ5K}},
note = {Machine review of arXiv:2505.01092}
}
read the original abstract
The generalized conditional gradient method is a popular algorithm for solving composite problems whose objective function is the sum of a smooth function and a nonsmooth convex function. Many convergence analyses of the algorithm rely on smoothness assumptions, such as the Lipschitz continuity of the gradient of the smooth part. This paper provides convergence results of linesearch-based generalized conditional gradient methods without smoothness assumptions. In particular, we show that a parameter-free variant, which automatically adapts to the H\"older exponent, guarantees convergence even when the gradient of the smooth part of the objective is not H\"older continuous.
Reference graph
Works this paper leans on
-
[1]
First-order methods in optimization
Amir Beck. First-order methods in optimization . SIAM, 2017
2017
-
[2]
Dimitri P Bertsekas. Nonlinear Programming. Athena scientific, 2nd edition, 1999
work page 1999
-
[3]
A H\"olderian backtracking method for min-max and min-min problems
J´ erˆ ome Bolte, Lilian Glaudin, Edouard Pauwels, and Mathieu Serrurier. Ah\” olderian backtracking method for min-max and min-min problems. arXiv preprint arXiv:2007.08810 , 2020
work page Pith review arXiv 2007
-
[4]
The iterates of the frank–wolfe algorithm may not converge
J´ erˆ ome Bolte, Cyrille W Combettes, and Edouard Pauwels. The iterates of the frank–wolfe algorithm may not converge. Mathematics of Operations Research , 49(4):2565–2578, 2024. 9
work page 2024
-
[5]
G´ abor Braun, Alejandro Carderera, Cyrille W Combettes, Hame d Hassani, Amin Karbasi, Aryan Mokhtari, and Sebastian Pokutta. Conditional gradient methods. arXiv preprint arXiv:2211.14103 , 2022
arXiv 2022
-
[6]
A generalized c onditional gradient method and its connection to an iterative shrinkage method
Kristian Bredies, Dirk A Lorenz, and Peter Maass. A generalized c onditional gradient method and its connection to an iterative shrinkage method. Computational Optimization and applications , 42:173–193, 2009
work page 2009
-
[7]
Coresets, sparse greedy approximation, and the frank-wolfe algorithm
Kenneth L Clarkson. Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Transactions on Algorithms (TALG) , 6(4):1–30, 2010
work page 2010
-
[8]
Complexity of linear min imization and projection on some sets
Cyrille W Combettes and Sebastian Pokutta. Complexity of linear min imization and projection on some sets. Operations Research Letters, 49(4):565–571, 2021
work page 2021
Show all 30 references
-
[9]
Proximal gradient methods beyond monotony
Alberto De Marchi. Proximal gradient methods beyond monotony . Journal of Nonsmooth Analysis and Optimization, 4(Original research articles), 2023
2023
-
[10]
Proximal gradient algo rithms under local lipschitz gradi- ent continuity: A convergence and robustness analysis of panoc
Alberto De Marchi and Andreas Themelis. Proximal gradient algo rithms under local lipschitz gradi- ent continuity: A convergence and robustness analysis of panoc. Journal of Optimization Theory and Applications, 194(3):771–794, 2022
2022
-
[11]
Rates of convergence for conditional gradien t algorithms near singular and nonsingular extremals
Joseph C Dunn. Rates of convergence for conditional gradien t algorithms near singular and nonsingular extremals. SIAM Journal on Control and Optimization , 17(2):187–211, 1979
1979
-
[12]
Conditional gradient algorit hms with open loop step size rules
Joseph C Dunn and S Harshbarger. Conditional gradient algorit hms with open loop step size rules. Journal of Mathematical Analysis and Applications , 62(2):432–444, 1978
1978
-
[13]
An algorithm for quadratic pro gramming
Marguerite Frank and Philip Wolfe. An algorithm for quadratic pro gramming. Naval research logistics quarterly, 3(1-2):95–110, 1956
1956
-
[14]
A generalized proximal point a lgorithm for certain non-convex minimization problems
Masao Fukushima and Hisashi Mine. A generalized proximal point a lgorithm for certain non-convex minimization problems. International Journal of Systems Science , 12(8):989–1000, 1981
1981
-
[15]
Conditional gradient type methods for composit e nonlinear and stochastic optimization
Saeed Ghadimi. Conditional gradient type methods for composit e nonlinear and stochastic optimization. Mathematical Programming, 173:431–464, 2019
2019
-
[16]
A parameter-free c onditional gradient method for composite minimization under h¨ older condition.Journal of Machine Learning Research , 24(166):1–34, 2023
Masaru Ito, Zhaosong Lu, and Chuan He. A parameter-free c onditional gradient method for composite minimization under h¨ older condition.Journal of Machine Learning Research , 24(166):1–34, 2023
2023
-
[17]
Revisiting frank-wolfe: Projection-free sparse convex optimization
Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. In International conference on machine learning , pages 427–435. PMLR, 2013
2013
-
[18]
Convergence analysis of the proximal gradient method in the presence of the kurdyka–/suppress lojasiewicz property without global lipschitz assumptions
Xiaoxi Jia, Christian Kanzow, and Patrick Mehlitz. Convergence analysis of the proximal gradient method in the presence of the kurdyka–/suppress lojasiewicz property without global lipschitz assumptions. SIAM Journal on Optimization , 33(4):3038–3056, 2023
2023
-
[19]
Structur ed nonconvex and nonsmooth opti- mization: algorithms and iteration complexity analysis
Bo Jiang, Tianyi Lin, Shiqian Ma, and Shuzhong Zhang. Structur ed nonconvex and nonsmooth opti- mization: algorithms and iteration complexity analysis. Computational Optimization and Applications , 72(1):115–157, 2019
2019
-
[20]
Convergence of nonmonotone proximal gradient methods under the kurdyka-lojasiewicz property without a global lipschitz assumption
Christian Kanzow and Leo Lehmann. Convergence of nonmonotone proximal gradient methods under the kurdyka-lojasiewicz property without a global lipschitz assumption . arXiv preprint arXiv:2411.12376 , 2024. 10
2024 arXiv
-
[21]
Convergence properties of monotone and nonmonotone proximal gradient methods revisited
Christian Kanzow and Patrick Mehlitz. Convergence properties of monotone and nonmonotone proximal gradient methods revisited. Journal of Optimization Theory and Applications , 195(2):624–646, 2022
2022
-
[22]
On fast convergence rates for generalized conditional gradient methods with backtracking stepsize
Karl Kunisch and Daniel Walter. On fast convergence rates for generalized conditional gradient methods with backtracking stepsize. Numerical Algebra, Control and Optimization , 14(1):108–136, 2024
2024
-
[23]
Convergence rate of frank-wolfe for n on-convex objectives
Simon Lacoste-Julien. Convergence rate of frank-wolfe for n on-convex objectives. arXiv preprint arXiv:1607.00345, 2016
2016 arXiv
-
[24]
Constrained minimization meth ods
Evgeny S Levitin and Boris T Polyak. Constrained minimization meth ods. USSR Computational math- ematics and mathematical physics , 6(5):1–50, 1966
1966
-
[25]
A minimization method for the s um of a convex function and a continuously differentiable function
Hisashi Mine and Masao Fukushima. A minimization method for the s um of a convex function and a continuously differentiable function. Journal of Optimization Theory and Applications , 33:9–23, 1981
1981
-
[26]
Complexity bounds for primal-dual methods minimiz ing the model of objective function
Yu Nesterov. Complexity bounds for primal-dual methods minimiz ing the model of objective function. Mathematical Programming, 171(1):311–330, 2018
2018
-
[27]
Acce lerated convergence of frank–wolfe algo- rithms with adaptive bregman step-size strategy
Shota Takahashi, Sebastian Pokutta, and Akiko Takeda. Acce lerated convergence of frank–wolfe algo- rithms with adaptive bregman step-size strategy. arXiv preprint arXiv:2504.04330 , 2025
2025 arXiv
-
[28]
A nonmonotone con- ditional gradient method for multiobjective optimization problems
Ashutosh Upadhayay, Debdas Ghosh, Jauny, Jen-Chih Yao, a nd Xiaopeng Zhao. A nonmonotone con- ditional gradient method for multiobjective optimization problems. Soft Computing , pages 1–22, 2024
2024
-
[29]
Proximal gradient-type me thod with generalized distance and con- vergence analysis without global descent lemma
Shotaro Yagishita and Masaru Ito. Proximal gradient-type me thod with generalized distance and con- vergence analysis without global descent lemma. arXiv preprint arXiv:2505.00381 , 2025
2025 arXiv
-
[30]
A nonmonotone line search te chnique and its application to unconstrained optimization
Hongchao Zhang and William W Hager. A nonmonotone line search te chnique and its application to unconstrained optimization. SIAM journal on Optimization , 14(4):1043–1056, 2004. 11
2004
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.