Pith. sign in

REVIEW 2 major objections 4 minor 30 references

Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions

T0 review · 2 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read Two linesearch-based generalized conditional gradient methods converge to stationary points of a composite objective even when the gradient of the smooth part is merely continuous, with no Lipschitz or Hölder condition assumed.

desk verdict Useful convergence proofs for generalized conditional gradient methods without gradient smoothness; the main theorems hold, but Lemma 3.2's termination claim needs a small patch at stationary points. read the letter →

arxiv 2505.01092 v1 pith:ATFHVJ5K submitted 2025-05-02 math.OC

classification math.OC MSC 90C3090C2565K05
keywords generalizedconditionalgradientmethodcompositeoptimizationnonmonotonelinesearchparameter-freealgorithmHöldercontinuityFrank-Wolfegapstationarypointconvergencewithoutsmoothness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Convergence analyses of the generalized conditional gradient method — the composite-problem solver that at each step minimizes a linearized smooth part plus an unchanged nonsmooth convex term — almost always assume the gradient of the smooth part is Lipschitz or Hölder continuous. This paper proves that two linesearch-based variants need no such assumption: with only continuous differentiability of the smooth part and convexity plus supercoercivity of the nonsmooth part, every accumulation point of the nonmonotone Armijo variant is a stationary point of the composite objective, and the parameter-free variant (Algorithm 2) converges to a stationary point under an additional bounded-level-set condition. The practical upshot is that both algorithms can be run without first certifying any Hölder exponent; the parameter-free method still yields an approximate stationary point in finitely many iterations even when no Hölder condition holds. The proof operates through the Frank-Wolfe gap, which the linesearch rejection inequality forces to zero using only continuity of the gradient.

What carries the argument

The central object is the Frank-Wolfe gap, G(x) := max_v {⟨∇f(x), x − v⟩ + g(x) − g(v)}, a nonnegative, lower-semicontinuous optimality measure that vanishes exactly at stationary points of (1). The mechanism carrying both proofs is the descent estimate of Lemma 3.1, F(x + td) ≤ F(x) − tG(x) + o(t) for t ∈ [0,1] and d a subproblem solution direction, which follows from continuous differentiability of f and convexity of g alone. Armijo-type backtracking then guarantees a function decrease proportional to the gap at every iteration — for Algorithm 1, F_{k+1} ≤ F_k − $pσβ^{{i_k}}$G(x_k), and for Algorithm 2, F(x_{k+1}) ≤ F(x_k) − (1/4)τ_k G(x_k) — so the gap is driven to zero along any subsequence where the stepsize does not degenerate. In the vanishing-stepsize case, rejection of the trial point forces inequalities (7) and (10), which bound the gap by ‖∇f(ξ_k) − ∇f(x_k)‖‖d_k‖ with ξ_k a mean-value point converging to the same limit as x_k; continuity of ∇f, rather than any quantitative modulus, sends that bound to zero, while boundedness of the directions d_k is secured by supercoercivity of g.

What would settle it

Concrete test: with g the indicator of [−1,1] and f(x) = x + x² sin(1/x²), the smooth part is differentiable everywhere with f'(0) = 1 while f' is discontinuous and unbounded at 0, so 0 is a non-stationary point. Run Algorithms 1 and 2 from several starting points and record the Frank-Wolfe gap G(x_k); iterates accumulating at 0 with a gap bounded away from zero would show where the continuous-gradient hypothesis is doing its work, while a gap that always tends to zero would suggest the theorem survives weaker assumptions. Independently, verifying inequality (7) numerically during a run on an ordinary C¹ but non-Lipschitz objective would confirm the proof's load-bearing estimate holds as claimed.

Watch

Extended reading notes

Core claim

The paper's central claim is a pair of subsequential convergence theorems. Theorem 3.1: for the composite problem min F = f + g with g proper, closed, convex, and supercoercive and f continuously differentiable on dom g, any accumulation point of the sequence generated by the generalized conditional gradient method with average-type nonmonotone Armijo linesearch (Algorithm 1) is stationary; no Lipschitz or Hölder assumption on ∇f appears anywhere in the hypotheses. Theorem 3.2: the same conclusion holds for the parameter-free backtracking variant of [16] (Algorithm 2), giving an accumulation point that is stationary and min_{0≤l≤k} G(x_l) → 0 as k → ∞, under the sole added assumption that the lower level set {x : F(x) ≤ F(x0)} is bounded — a condition automatic when f is convex or dom g is bounded. In the delicate regime where backtracking stepsizes shrink to zero, the mean-value theorem produces a point ξ_k between x_k and the rejected trial point, and the linesearch rejection inequality yields (1−σ)G(x_k) ≤ ‖∇f(ξ_k) − ∇f(x_k)‖ ‖d_k‖; because ξ_k and x_k converge to the same limit and ∇f is continuous, the gap must vanish. The paper thus establishes convergence under a mere continuity condition on the gradient, where the prior literature required quantitative smoothness.

Load-bearing premise

The load-bearing premise is continuity of the gradient ∇f of the smooth part at the accumulation point, used to conclude that ‖∇f(ξ_k) − ∇f(x_k)‖ shrinks to zero when the trial point ξ_k and the iterate x_k approach the same limit, and, for the parameter-free variant, boundedness of the lower level set {x : F(x) ≤ F(x0)}.

Editorial extensions

If this is right

  • Corollary 3.1: if an accumulation point exists — e.g. when dom g is bounded — the minimum Frank-Wolfe gap over the first k iterates tends to zero, so an ε-approximate stationary point is obtained in finitely many iterations.
  • Theorem 3.2: the parameter-free variant converges as soon as the lower level set is bounded, a condition that holds automatically when f is convex or when dom g is bounded.
  • The nonmonotone Armijo result extends the smoothness-free analysis of [2] beyond compact convex constraint sets, and the monotone backtracking method of [22] (p = 1, σ ≤ 1/2) is covered as a special case.
  • Because the parameter-free method's Hölder-adaptive rate remains valid when ∇f is Hölder continuous, skipping a Hölder check loses the rate only in cases where no Hölder exponent actually exists.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The proofs appear to consume only continuity of ∇f at the accumulation point and boundedness of the directions d_k, so the hypothesis 'continuously differentiable on dom g' could likely be weakened to differentiability with locally continuous gradient on the lower level set.
  • If ∇f is known only through a modulus of continuity ω(δ), the same inequalities should yield a quantitative gap decay that interpolates between the O(ε^{−1−1/ν}) rate for ω(δ) = Cδ^ν and the purely asymptotic guarantee when no modulus exists.
  • Because every accumulation point must be stationary, any counterexample to whole-sequence convergence under Armijo stepsizes would have to unwind along the stationary set itself, which constrains the search left open by the divergence counterexamples the paper cites.
  • The mean-value-plus-linesearch-rejection mechanism should transfer to linesearch-based proximal gradient methods, producing a common smoothness-free convergence theory for the two principal composite solvers.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper analyzes two linesearch-based generalized conditional gradient methods for the composite problem min F(x)=f(x)+g(x), where f is continuously differentiable on dom g (but not assumed to have a Lipschitz or Hölder continuous gradient) and g is proper, closed, convex, and supercoercive. The main results are Theorem 3.1, asserting that every accumulation point of the iterates of a nonmonotone Armijo-type method (Algorithm 1) is stationary, and Theorem 3.2, asserting that under a bounded-level-set condition the parameter-free variant of Ito, Lu, and He (Algorithm 2) has at least one stationary accumulation point and its running minimum Frank-Wolfe gap tends to zero. The key new ingredient is Lemma 3.1, which derives the o(t) descent estimate F(x+td) <= F(x)-tG(x)+o(t) using only continuous differentiability of f and convexity of g.

Significance. If the technical gaps identified below are fixed, this is a worthwhile contribution: it removes global Lipschitz/Hölder smoothness assumptions that are standard in essentially all existing convergence analyses of generalized conditional gradient methods, and it shows that a practically attractive parameter-free backtracking variant remains convergent even when the Hölder condition is unknown or absent. The proofs are short, elementary, and mostly self-contained, and Lemma 3.1 is a clean reusable tool. The extension of Bertsekas-style subsequential convergence from constrained smooth problems to the composite, nonconvex, supercoercive setting is a genuine advance. However, the well-definedness of Algorithm 1 has a real gap at stationary points, so the paper is not yet fully sound as written.

major comments (2)
  1. [Section 3, Lemma 3.2] Lemma 3.2 is false when G(x_k)=0 and the subproblem oracle returns a minimizer v_k different from x_k. The proof applies Lemma 3.1 and concludes F(x_{k,i}) <= F(x_k) - sigma beta^i G(x_k) for all sufficiently large i, but when G(x_k)=0 this reduces to F(x_{k,i}) <= F(x_k)+o(beta^i), which does not force the Armijo condition (4) to hold because o(beta^i) may be positive. Concretely, take E=R, f(x)=x^2, g(x)=max(0,|x|-1)^2, x_0=0; then all standing assumptions hold and x_0 is stationary with G(x_0)=0, but the subproblem minimizer v=1 gives d=1 and F(beta^i)=beta^{2i}>0=F(x_0), so (4) fails for every finite i. The algorithm therefore may never produce an infinite sequence from a stationary point. The convergence theorems would still hold for sequences that are actually generated, but the well-definedness claim in Lemma 3.2 and the pseudocode need a patch, such as an explicit stopping rule 'if G(x_k)=0, stop' or a tie-breaking rule that selects d_k=0 at stationary points.
  2. [Section 3.1, Algorithm 2] The definition of the trial step size in Algorithm 2 is incomplete when d_k=0. The formula tau_k^(i)=min{1, G(x_k)/(2L_k^(i)||d_k||^2)} involves division by ||d_k||^2, and d_k=0 occurs exactly when v_k=x_k, i.e., when x_k is a minimizer of the linearized subproblem. Since -grad f(x_k) in partial g(x_k) is equivalent to G(x_k)=0 by Lemma 2.1, this is precisely the stationary case. The algorithm needs an explicit convention for d_k=0, e.g., tau_k^(i)=1 or termination when G(x_k)=0. Without this, the statement of Proposition 3.2 and Lemma 3.3 do not formally cover all admissible outputs of a subproblem solver.
minor comments (4)
  1. [Abstract and Introduction] The abstract and introduction say the methods 'guarantee convergence' even without Hölder continuity, but the theorems establish subsequential convergence and convergence of the running minimum of the Frank-Wolfe gap, not convergence of the whole sequence. Please make this distinction explicit in the abstract.
  2. [Section 3, Lemma 3.2 proof, Eq. (5)] In the displayed inequality (5), the exponent on beta appears to be i_{k-1}, but the notation is easy to misread as i_k - 1. Please add brackets or a clarifying sentence so that the index is unambiguous.
  3. [Footnote, uniform convex structure] The footnote defining uniform convex structure states 'rho >= 21' near the end of Section 3; this is presumably rho >= 2 and should be corrected.
  4. [References] Reference [18] contains the odd text 'Kurdyka–/suppress Lojasiewicz'; this should read 'Kurdyka–Łojasiewicz property'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the convergence theorems are derived self-containedly from the problem definitions and the algorithm line-search rules.

full rationale

The paper's convergence proofs are direct mathematical derivations. Lemma 3.1 uses only continuous differentiability of f, convexity of g, and the definition of the Frank-Wolfe gap G(x); Lemma 2.2 is a definitional lower-semicontinuity argument. Theorems 3.1 and 3.2 then combine these lemmas with the descent/acceptance inequalities (4) and (8), monotonicity of F_k or F(x_k), boundedness of the subproblem minimizers via supercoercivity, and continuity of ∇f. No parameter is fitted to any subset of data, and no 'prediction' is derived from a quantity already used as an input; the stationarity conclusion comes from showing G(x*) = 0, which is equivalent to (2) by the cited definitional Lemma 2.1. Algorithm 2 is taken from Ito et al. [16], but Theorem 3.2 does not rely on any convergence claim from that work; it proves convergence directly under the stated assumptions and the algorithm's own backtracking definition. The only related concerns are non-circular: the title's 'without smoothness assumptions' is stronger than the actual C^1 assumption on f, and the proof of Lemma 3.2 may need a separate termination rule at stationary points where G(x_k)=0. These are correctness or assumption-strength issues, not cases where a claim reduces to its inputs by construction.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper relies only on standard background assumptions and the previously published algorithms; it introduces no new parameters, fitted constants, or entities. The algorithmic constants (β, σ, p, L_{-1}) are not tuned to data and the results are valid for any allowed values.

assumptions (3)
  • domain assumption f is lower semicontinuous and continuously differentiable on dom g
    Stated in Section 2, problem (1). The proofs of the vanishing-step cases in Theorems 3.1 and 3.2 use the continuity of ∇f to show that terms like ||∇f(ξ_k)-∇f(x_k)|| vanish when ξ_k and x_k converge to the same point (inequalities (7) and (10)).
  • domain assumption g is proper closed convex and supercoercive
    Stated in Section 2. Supercoercivity is used to guarantee nonemptiness of the subproblem solution (3) and to prove boundedness of the subproblem solutions {v_k} in Propositions 3.1(iv) and 3.2(iv).
  • domain assumption The lower level set {x : F(x) <= F(x0)} is bounded
    Added in Theorem 3.2 for Algorithm 2 to guarantee that the iterates are bounded and an accumulation point exists. The paper notes this holds when f is coercive or when g is bounded, and gives a sufficient condition f(x) >= a||x|| + b.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions." pith.science (2026). https://pith.science/paper/ATFHVJ5K

@misc{pith2026250501092,
  author       = {Pith},
  title        = {Pith review of: Convergence of linesearch-based generalized conditional gradient methods without smoothness assumptions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ATFHVJ5K}},
  note         = {Machine review of arXiv:2505.01092}
}
read the original abstract

The generalized conditional gradient method is a popular algorithm for solving composite problems whose objective function is the sum of a smooth function and a nonsmooth convex function. Many convergence analyses of the algorithm rely on smoothness assumptions, such as the Lipschitz continuity of the gradient of the smooth part. This paper provides convergence results of linesearch-based generalized conditional gradient methods without smoothness assumptions. In particular, we show that a parameter-free variant, which automatically adapts to the H\"older exponent, guarantees convergence even when the gradient of the smooth part of the objective is not H\"older continuous.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 22 canonical work pages

  1. [1]

    First-order methods in optimization

    Amir Beck. First-order methods in optimization . SIAM, 2017

  2. [2]

    Nonlinear Programming

    Dimitri P Bertsekas. Nonlinear Programming. Athena scientific, 2nd edition, 1999

  3. [3]

    A H\"olderian backtracking method for min-max and min-min problems

    J´ erˆ ome Bolte, Lilian Glaudin, Edouard Pauwels, and Mathieu Serrurier. Ah\” olderian backtracking method for min-max and min-min problems. arXiv preprint arXiv:2007.08810 , 2020

  4. [4]

    The iterates of the frank–wolfe algorithm may not converge

    J´ erˆ ome Bolte, Cyrille W Combettes, and Edouard Pauwels. The iterates of the frank–wolfe algorithm may not converge. Mathematics of Operations Research , 49(4):2565–2578, 2024. 9

  5. [5]

    Conditional gradient methods

    G´ abor Braun, Alejandro Carderera, Cyrille W Combettes, Hame d Hassani, Amin Karbasi, Aryan Mokhtari, and Sebastian Pokutta. Conditional gradient methods. arXiv preprint arXiv:2211.14103 , 2022

  6. [6]

    A generalized c onditional gradient method and its connection to an iterative shrinkage method

    Kristian Bredies, Dirk A Lorenz, and Peter Maass. A generalized c onditional gradient method and its connection to an iterative shrinkage method. Computational Optimization and applications , 42:173–193, 2009

  7. [7]

    Coresets, sparse greedy approximation, and the frank-wolfe algorithm

    Kenneth L Clarkson. Coresets, sparse greedy approximation, and the frank-wolfe algorithm. ACM Transactions on Algorithms (TALG) , 6(4):1–30, 2010

  8. [8]

    Complexity of linear min imization and projection on some sets

    Cyrille W Combettes and Sebastian Pokutta. Complexity of linear min imization and projection on some sets. Operations Research Letters, 49(4):565–571, 2021

Show all 30 references
  1. [9]

    Proximal gradient methods beyond monotony

    Alberto De Marchi. Proximal gradient methods beyond monotony . Journal of Nonsmooth Analysis and Optimization, 4(Original research articles), 2023

  2. [10]

    Proximal gradient algo rithms under local lipschitz gradi- ent continuity: A convergence and robustness analysis of panoc

    Alberto De Marchi and Andreas Themelis. Proximal gradient algo rithms under local lipschitz gradi- ent continuity: A convergence and robustness analysis of panoc. Journal of Optimization Theory and Applications, 194(3):771–794, 2022

  3. [11]

    Rates of convergence for conditional gradien t algorithms near singular and nonsingular extremals

    Joseph C Dunn. Rates of convergence for conditional gradien t algorithms near singular and nonsingular extremals. SIAM Journal on Control and Optimization , 17(2):187–211, 1979

  4. [12]

    Conditional gradient algorit hms with open loop step size rules

    Joseph C Dunn and S Harshbarger. Conditional gradient algorit hms with open loop step size rules. Journal of Mathematical Analysis and Applications , 62(2):432–444, 1978

  5. [13]

    An algorithm for quadratic pro gramming

    Marguerite Frank and Philip Wolfe. An algorithm for quadratic pro gramming. Naval research logistics quarterly, 3(1-2):95–110, 1956

  6. [14]

    A generalized proximal point a lgorithm for certain non-convex minimization problems

    Masao Fukushima and Hisashi Mine. A generalized proximal point a lgorithm for certain non-convex minimization problems. International Journal of Systems Science , 12(8):989–1000, 1981

  7. [15]

    Conditional gradient type methods for composit e nonlinear and stochastic optimization

    Saeed Ghadimi. Conditional gradient type methods for composit e nonlinear and stochastic optimization. Mathematical Programming, 173:431–464, 2019

  8. [16]

    A parameter-free c onditional gradient method for composite minimization under h¨ older condition.Journal of Machine Learning Research , 24(166):1–34, 2023

    Masaru Ito, Zhaosong Lu, and Chuan He. A parameter-free c onditional gradient method for composite minimization under h¨ older condition.Journal of Machine Learning Research , 24(166):1–34, 2023

  9. [17]

    Revisiting frank-wolfe: Projection-free sparse convex optimization

    Martin Jaggi. Revisiting frank-wolfe: Projection-free sparse convex optimization. In International conference on machine learning , pages 427–435. PMLR, 2013

  10. [18]

    Convergence analysis of the proximal gradient method in the presence of the kurdyka–/suppress lojasiewicz property without global lipschitz assumptions

    Xiaoxi Jia, Christian Kanzow, and Patrick Mehlitz. Convergence analysis of the proximal gradient method in the presence of the kurdyka–/suppress lojasiewicz property without global lipschitz assumptions. SIAM Journal on Optimization , 33(4):3038–3056, 2023

  11. [19]

    Structur ed nonconvex and nonsmooth opti- mization: algorithms and iteration complexity analysis

    Bo Jiang, Tianyi Lin, Shiqian Ma, and Shuzhong Zhang. Structur ed nonconvex and nonsmooth opti- mization: algorithms and iteration complexity analysis. Computational Optimization and Applications , 72(1):115–157, 2019

  12. [20]

    Convergence of nonmonotone proximal gradient methods under the kurdyka-lojasiewicz property without a global lipschitz assumption

    Christian Kanzow and Leo Lehmann. Convergence of nonmonotone proximal gradient methods under the kurdyka-lojasiewicz property without a global lipschitz assumption . arXiv preprint arXiv:2411.12376 , 2024. 10

  13. [21]

    Convergence properties of monotone and nonmonotone proximal gradient methods revisited

    Christian Kanzow and Patrick Mehlitz. Convergence properties of monotone and nonmonotone proximal gradient methods revisited. Journal of Optimization Theory and Applications , 195(2):624–646, 2022

  14. [22]

    On fast convergence rates for generalized conditional gradient methods with backtracking stepsize

    Karl Kunisch and Daniel Walter. On fast convergence rates for generalized conditional gradient methods with backtracking stepsize. Numerical Algebra, Control and Optimization , 14(1):108–136, 2024

  15. [23]

    Convergence rate of frank-wolfe for n on-convex objectives

    Simon Lacoste-Julien. Convergence rate of frank-wolfe for n on-convex objectives. arXiv preprint arXiv:1607.00345, 2016

  16. [24]

    Constrained minimization meth ods

    Evgeny S Levitin and Boris T Polyak. Constrained minimization meth ods. USSR Computational math- ematics and mathematical physics , 6(5):1–50, 1966

  17. [25]

    A minimization method for the s um of a convex function and a continuously differentiable function

    Hisashi Mine and Masao Fukushima. A minimization method for the s um of a convex function and a continuously differentiable function. Journal of Optimization Theory and Applications , 33:9–23, 1981

  18. [26]

    Complexity bounds for primal-dual methods minimiz ing the model of objective function

    Yu Nesterov. Complexity bounds for primal-dual methods minimiz ing the model of objective function. Mathematical Programming, 171(1):311–330, 2018

  19. [27]

    Acce lerated convergence of frank–wolfe algo- rithms with adaptive bregman step-size strategy

    Shota Takahashi, Sebastian Pokutta, and Akiko Takeda. Acce lerated convergence of frank–wolfe algo- rithms with adaptive bregman step-size strategy. arXiv preprint arXiv:2504.04330 , 2025

  20. [28]

    A nonmonotone con- ditional gradient method for multiobjective optimization problems

    Ashutosh Upadhayay, Debdas Ghosh, Jauny, Jen-Chih Yao, a nd Xiaopeng Zhao. A nonmonotone con- ditional gradient method for multiobjective optimization problems. Soft Computing , pages 1–22, 2024

  21. [29]

    Proximal gradient-type me thod with generalized distance and con- vergence analysis without global descent lemma

    Shotaro Yagishita and Masaru Ito. Proximal gradient-type me thod with generalized distance and con- vergence analysis without global descent lemma. arXiv preprint arXiv:2505.00381 , 2025

  22. [30]

    A nonmonotone line search te chnique and its application to unconstrained optimization

    Hongchao Zhang and William W Hager. A nonmonotone line search te chnique and its application to unconstrained optimization. SIAM journal on Optimization , 14(4):1043–1056, 2004. 11

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.