Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Convergence rates of regularized quasi-Newton methods without strong convexity

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proves that regularized SR1 quasi-Newton methods enjoy explicit non-asymptotic superlinear convergence rates on nonconvex nonsmooth problems satisfying a Kurdyka–Łojasiewicz inequality, without strong convexity, line search, or…

desk verdict Smooth-case rates are a real advance; the nonsmooth composite claim is an oracle result that needs either a proof of Remark 4.5 or an honest caveat. read the letter →

arxiv 2506.00521 v6 pith:CMUHJFA6 submitted 2025-05-31 math.OC

Shida Wang , Jalal Fadili , Peter Ochs This is my paper
classification math.OC MSC 90C5390C2690C2549J52
keywords quasi-NewtonmethodsSR1updateKurdyka–Łojasiewiczpropertynon-asymptoticconvergenceratescubicregularizationgradientnonconvexnonsmoothoptimizationsuperlinear
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Strong convexity is the usual engine behind fast quasi-Newton convergence; this paper shows the Kurdyka–Łojasiewicz (KL) property can play that role instead. It analyzes two regularized SR1 proximal quasi-Newton methods—a cubic-regularized one for general nonconvex nonsmooth composite objectives and a gradient-regularized one for convex objectives—and derives explicit non-asymptotic bounds on the subgradient norm after an initial number of iterations. The headline result is a superlinear rate of order $\left(\frac{C}{(k-k_0)^{1/2}}\right)^{(k-k_0)/2}$ when the KL desingularizing function is $\phi(t)=ct^{1/2}$. These are the first such rates for regularized proximal SR1 methods without strong convexity, and they require no line search, trust region, Dennis–Moré condition, or assumptions on the quasi-Newton metrics.

What carries the argument

The trace potential $V(G)=\operatorname{tr} G$ together with the SR1 update formula (5) is the central object. Each SR1 step decreases the trace by $\nu(A,G,u)=\frac{u^\top(G-A)^2u}{u^\top(G-A)u}$, which measures how much closer the metric $G$ gets to the average Hessian $J_k$; cubic or gradient regularization adds enough curvature to keep $J_k\preceq G_{k+1}\preceq\tilde G_{k+1}$ and to control the trace. The KL inequality then converts the guaranteed function decrease into control of $\|F'(x_k)\|$, and summing the trace decreases over iterations yields a geometric-mean contraction that becomes superlinear.

What would settle it

Take a KL function $F=g+f$ with $g=\|x\|_1$ and smooth $f$ satisfying Assumptions 1–2 with $\phi(t)=ct^{1/2}$, run the Cubic SR1 PQN update (6) using a standard proximal gradient solver with a fixed tolerance for the cubic subproblem, and check whether $\|F'(x_k)\|$ obeys the bound (10) for all $k\ge k_0$. A violation, or the need to solve the subproblem to machine precision to observe the bound, would show the theorem depends on exact subproblem minimization rather than on the structure analyzed.

Watch

Extended reading notes

Core claim

The central claim is that the KL inequality with a desingularizing function $\phi$ replaces strong convexity as the driver of superlinear convergence. Under Assumptions 1 and 2 (Lipschitz smoothness plus KL) and boundedness of the generated sequence, both algorithms have $\|F'(x_k)\| o 0$, and for $k\ge k_0$ explicit rates hold; for example, when $\phi(t)=ct^{1/2}$, Cubic SR1 PQN satisfies $\|F'(x_{N+k_0})\| \le \left(\frac{3c^2}{4}\left(\frac{C^{\mathrm{CR}}_1}{N}+\frac{C^{\mathrm{CR}}_2}{N^{1/2}}\right)\right)^{N/2}\|F'(x_{k_0})\|$. The convex counterpart Grad SR1 PQN achieves an analogous bound with gradient regularization instead of cubic terms. The paper presents this as the first non-asymptotic explicit superlinear convergence result for regularized proximal SR1 methods on nonconvex nonsmooth KL objectives, and notes the rates are new even for smooth nonconvex problems.

Load-bearing premise

Each iteration must compute an exact global minimizer of a cubic-regularized proximal subproblem; for general nonsmooth $g$ no algorithm is supplied that can do this, so the non-asymptotic rates are not established for any implementable inexact version.

Editorial extensions

If this is right

  • For any nonconvex nonsmooth KL objective with desingularizer $\phi(t)=ct^{1/2}$, Cubic SR1 PQN attains a subgradient-norm rate of order $\left(\frac{C}{(k-k_0)^{1/2}}\right)^{(k-k_0)/2}$ for all $k\ge k_0$, which is superlinear.
  • The same type of guarantee holds for Grad SR1 PQN on convex KL objectives, with gradient regularization instead of cubic terms, at lower per-iteration cost.
  • No line search, trust region, Dennis–Moré condition, or strong convexity is needed for these rates; the only global mechanism is the restarting rule that resets the metric to $LI$ when its trace exceeds $n\bar\kappa$.
  • When $F$ satisfies a global Łojasiewicz inequality, the rates become global, holding from the first iteration rather than after an initial $k_0$.
  • The analysis covers nonsmooth additive composite problems $F=g+f$ with nonconvex $g$, so the result applies beyond smooth objectives.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The practical bottleneck is the cubic subproblem: the theorems assume an exact global minimizer, and the experiments solve only smooth problems; if inexact solvers are used, the rates may need a tolerance-dependent correction to remain valid.
  • The trace-restart mechanism looks transferable: any quasi-Newton update that preserves $J_k\preceq G_{k+1}\preceq\tilde G_{k+1}$ and decreases the trace could inherit the same superlinear argument under KL, so other metric updates may admit similar bounds.
  • For the $\theta=1/2$ case, the KL exponent coincides with gradient domination conditions, suggesting the result applies to overparameterized models where strong convexity fails but such domination often holds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript studies non-asymptotic convergence rates for two regularized SR1 quasi-Newton methods: Cubic SR1 PQN (Algorithm 1) for possibly nonconvex nonsmooth composite objectives F = g + f satisfying the Kurdyka-Łojasiewicz property, and Grad SR1 PQN (Algorithm 2) for convex composite objectives. Under Assumptions 1-2 (Lipschitz-smooth f with Lipschitz Hessian, KL property of F), Theorems 4.2 and 4.9 establish global subsequential convergence and explicit rates for the subgradient norm, with last-iterate superlinear rates when the desingularizing function is φ(t)=ct^{1-θ} for θ≤1/2 and window-minimum sublinear rates for θ∈(1/2,1). Theorem 4.13 and Appendix A add global non-asymptotic rates under a gradient-domination (Łojasiewicz) inequality. The proof relies on a trace potential V(G)=tr G and rank-one SR1 update identities, following the framework of [54]. Section 6 reports experiments on smooth quadratic, logistic-regression, and image-deblurring problems.

Significance. If the main theorems are read as conditional mathematical statements, the trace-potential argument is a genuine and nontrivial extension of the strongly convex analysis in [54] to KL functions, and the explicit rates for smooth nonconvex objectives appear novel. The paper also deliberately avoids line search, trust regions, the Dennis-Moré condition, and strong convexity, which are standard assumptions in quasi-Newton theory. The numerical experiments confirm the expected superlinear behavior on smooth problems. However, the advertised nonsmooth composite results currently rely on an exact global-minimization oracle for the subproblem, and Remark 4.5, which claims an inexact relaxation, is unproved; the practical contribution is therefore weaker than the abstract suggests.

major comments (3)
  1. [Section 4.2.1, Eq. (6) and (15); Remark 4.5; Section 6] The nonsmooth composite rates in Theorems 4.2 and 4.9 apply to an oracle algorithm. Algorithm 1, Step 1a, Eq. (6), and Algorithm 2, Step 1, Eq. (15), require a global minimizer of a cubic-regularized composite subproblem. For general nonconvex g this subproblem is nonconvex, and even for nonsmooth convex g exact global minimization is not generally implementable. Remark 4.5 asserts that a stationary point satisfying Eq. (12) suffices, but no proof is given that Eq. (12) preserves the sufficient-decrease, sumability, and rate arguments of Section 5. Section 6 explicitly tests only smooth problems and states that nonsmooth subproblem solving is left to future work. To support the abstract's nonsmooth composite claim, the authors should either provide an implementable (possibly inexact) subproblem solver together with a proof that the rates survive, prove the relaxation in Remark 4.5, or restrict the theoretical claims to an exact-oracle setting and reword the abstract and introduction accordingly.
  2. [Abstract; Theorem 4.2, Eqs. (8), (9), (10), (11)] The abstract states that 'after a number of iterations k0, Cubic SR1 PQN exhibits non-asymptotic explicit super-linear convergence rates', but this overstates the theorem. For a general desingularizing function φ, Theorem 4.2, Eq. (8), gives a window-minimum rate with exponent N/(N+1), which is sublinear, not superlinear. Last-iterate superlinear rates are proven only for φ(t)=ct^{1-θ} with θ∈(0,1/2] (Eqs. (9) and (10)). For θ∈(1/2,1), Eq. (11) is again a window-minimum sublinear rate. The abstract and the introductory summary should qualify the superlinearity claim by the class of desingularizing functions.
  3. [Theorem 4.13, Eq. (21), and Proof of Theorem 4.13, Eq. (176)] The global rate in Theorem 4.13, Eq. (21), states ∥∇f(x_N)∥ ≤ (c^2 C_G D/(2N))^{N/2} ∥∇f(x_0)∥^{2/(N+1)}, but the proof in Eq. (176) derives the same bound with ∥∇f(x_0)∥ to the first power and no extra factor. The telescoping product in the proof does not produce the exponent 2/(N+1). The theorem statement and the proof therefore disagree, and the printed global-rate claim is not established as written. This should be corrected before the result can be used.
minor comments (5)
  1. [Proof of Theorem 4.2, Eq. (87)] The proof of Theorem 4.2 ends with an exponent 2/(N+1) on g_{k0}, while the theorem statement in Eq. (8) and the intervening derivation give 1/(N+1). This is likely a typographical error, but it should be aligned.
  2. [Remark 4.6 and Appendix A, Theorem A.2] The displayed rate in Remark 4.6, Eq. (14), appears as '≤ μ 6 (...)' and should read '≤ (6/μ) (...)' as in Theorem A.2. Please correct the notation.
  3. [Algorithm 2, Step 3, and Lemma 5.19, Eq. (109)] Lemma 5.19 states ∥F'(x_k)∥ ≤ λ_k^2/L_H for k ≥ k0, but Algorithm 2 defines λ_{k+1} = sqrt(L_H ∥F'(x_{k+1})∥) + L_H r_k, so the inequality is valid for indices shifted by one and not for k=0 with λ_0=0. The index convention should be stated explicitly.
  4. [Section 6.1.1, logistic regression] The text says the Lipschitz constant of the Hessian is L_H = 4 max_i ∥a_i∥, but the experiment sets L_H = 4 heuristically. If the theoretical rates are to be compared to the experiment, the authors should report whether the heuristic value is actually a valid upper bound for the chosen data set and parameters.
  5. [Theorem 4.2 and proof, constants] The constant C_CR1 in the theorem statement is (n+1)L + nκ̄ + 2nL_H R, while in the proof of Eq. (82) it is defined as (n+1)L + 2nκ̄ + 2L_H R. These should be reconciled, including the factor of n on κ̄.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the rates follow from stated KL/smoothness assumptions through self-contained trace-potential estimates.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard smoothness and KL assumptions plus a strong oracle assumption: the exact solution of a cubic-regularized proximal subproblem. No new physical or mathematical entities are introduced; the only hand-chosen algorithmic parameter is kappa bar.

free parameters (1)
  • kappa bar (restart threshold) = user-chosen, kappa bar >= L
    Restart threshold and trace bound in Algorithms 1 and 2; appears in all rate constants. The theorems hold for any kappa bar >= L, so it is a legitimate algorithmic parameter rather than a fitted constant.
assumptions (4)
  • domain assumption F = g + f is bounded below, g is proper lsc, f is twice differentiable with L-Lipschitz gradient and LH-Lipschitz Hessian, argmin F nonempty (Assumption 1).
    Standing assumptions for the algorithms' decrease lemmas and subgradient bounds.
  • domain assumption F is a KL function with desingularizing function phi (Assumption 2).
    Central condition replacing strong convexity; rates are expressed in terms of phi.
  • standard math Uniformized KL property over the cluster set (Lemma 3.1, from [10]).
    Allows passing from pointwise KL to a uniform neighborhood inequality, used to get non-asymptotic bounds.
  • ad hoc to paper Exact global minimization of the cubic-regularized subproblem in each iteration (Algorithm 1 Step 1a equation (6), Algorithm 2 Step 1 equation (15)).
    Required by the optimality conditions (40)/(92); no inexactness tolerance or solver is provided, and Section 6 limits experiments to smooth problems.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Convergence rates of regularized quasi-Newton methods without strong convexity." pith.science (2026). https://pith.science/paper/CMUHJFA6

@misc{pith2026250600521,
  author       = {Pith},
  title        = {Pith review of: Convergence rates of regularized quasi-Newton methods without strong convexity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMUHJFA6}},
  note         = {Machine review of arXiv:2506.00521}
}
abstract

In this paper, we study convergence rates of the cubic regularized proximal quasi-Newton method (\csr) for solving non-smooth additive composite problems that satisfy the so-called Kurdyka-\L ojasiewicz (K\L ) property with respect to some desingularization function $\phi$ rather than strong convexity. After a number of iterations $k_0$, Cubic SR1 PQN exhibits non-asymptotic explicit super-linear convergence rates for any $k\geq k_0$. In particular, when $\phi(t)=ct^{1/2}$, Cubic SR1 PQN has a convergence rate of order $\left(\frac{C}{(k-k_0)^{1/2}}\right)^{(k-k_0)/2}$, where $k$ is the number of iterations and $C>0$ is a constant. For the special case, i.e. functions which satisfy \L ojasiewicz inequality, the rate becomes global and non-asymptotic. This work presents, for the first time, non-asymptotic explicit convergence rates of regularized (proximal) SR1 quasi-Newton methods applied to non-convex non-smooth problems with K\L\ property. Actually, the rates are novel even in the smooth non-convex case. Notably, we achieve this without employing line search or trust region strategies, without assuming the Dennis-Mor\'e condition, without any assumptions on quasi-Newton metrics and without assuming strong convexity. Furthermore, for convex problems, we focus on a more tractable gradient regularized quasi-Newton method (Grad SR1 PQN) which can achieve results similar to those obtained with cubic regularization. We also demonstrate, for the first time, the non-asymptotic super-linear convergence rate of Grad SR1 PQN for solving convex problems with the help of the \L ojasiewicz inequality instead of strong convexity.

Figures

Figures reproduced from arXiv: 2506.00521 by the authors.

Figure 1
Figure 1. SR1 method significantly outperforms first-order methods in number of iterations and in terms of time. 6.1.1 Logistic regression with a convex regularization The second experiment is a logistic regression problem with regularization on the benchmark dataset “mushroom” from UCI Machine Learning Repository [53]: min x∈Rn f(x), with f(x) := 1 m Xm i=1 log(1 + exp(−bia ⊤ i x)) + µ p ∥x∥ 2 + ϵ , (178) where ai ∈ R n, bi … view at source ↗
Figure 2
Figure 2. Cubic- and Grad SR1 PQN significantly outperform first-order methods in number of [PITH_FULL_IMAGE:figures/full_fig_p027_2.png] view at source ↗
Figure 3
Figure 3. In the number of iterations and the norm of the gradients, Cubic Newton and Cubic [PITH_FULL_IMAGE:figures/full_fig_p028_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. On the Universality of Simple Trust-Region Algorithms

    math.OC 2026-07 accept novelty 6.0 of 10

    Classical and modified-ratio trust-region methods reach the optimal O(ε^{-1/(1+ν)}) convex and O(ε^{-(2+ν)/(1+ν)}) nonconvex complexity for any Hölder ν∈[0,1] without knowing ν.

Reference graph

Works this paper leans on

56 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [54]

    W ang, J

    S. W ang, J. F adili, and P. Ochs, Global non-asymptotic super-linear convergence rates of regularized proximal quasi-Newton methods on non-smooth composite problems, arXiv preprint arXiv:2410.11676, (2024)

    arXiv 2024

  2. [1]

    Attouch and J

    H. Attouch and J. Bolte , On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , Mathematical Programming, 116 (2009), pp. 5–16

    work page 2009

  3. [2]

    Becker and J

    S. Becker and J. F adili , A quasi-Newton proximal splitting method , Advances in neural information processing systems, 25 (2012)

    work page 2012

  4. [3]

    Becker, J

    S. Becker, J. F adili, and P. Ochs, On quasi-Newton forward-backward splitting: proximal calculus and convergence, SIAM Journal on Optimization, 29 (2019), pp. 2445–2481

    work page 2019

  5. [4]

    H. Y. Benson and D. F. Shanno , Cubic regularization in symmetric rank-1 quasi-Newton methods, Mathematical Programming Computation, 10 (2018), pp. 457–486

    work page 2018

  6. [5]

    Bianconcini, G

    T. Bianconcini, G. Liuzzi, B. Morini, and M. Sciandrone , On the use of iterative methods in cubic regularization for unconstrained optimization , Computational Optimization and Applications, 60 (2015), pp. 35–57

    work page 2015

  7. [6]

    Bolte, A

    J. Bolte, A. Daniilidis, and A. Lewis , The Lojasiewicz inequality for nonsmooth suban- alytic functions with applications to subgradient dynamical systems , SIAM Journal on Opti- mization, 17 (2006), pp. 1205–1223

    work page 2006

  8. [7]

    Bolte, A

    J. Bolte, A. Daniilidis, A. Lewis, and M. Shiota , Clarke subgradients of stratifiable functions, SIAM Journal on Optimization, 18 (2007), pp. 556–572

    2007

Show all 56 references
  1. [8]

    Bolte, A

    J. Bolte, A. Daniilidis, O. Ley, and L. Mazet , Characterizations of Lojasiewicz in- equalities and applications , arXiv preprint arXiv:0802.0826, (2008)

    2008 arXiv

  2. [9]

    Bolte, T

    J. Bolte, T. P. Nguyen, J. Peypouquet, and B. W. Suter , From error bounds to the complexity of first-order descent methods for convex functions , Mathematical Programming, 165 (2017), pp. 471–507

    2017

  3. [10]

    Bolte, S

    J. Bolte, S. Sabach, and M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems , Mathematical Programming, 146 (2014), pp. 459–494

    2014

  4. [11]

    C. G. Broyden , Quasi-Newton methods and their application to function minimisation , Mathematics of Computation, 21 (1967), pp. 368–381

    1967

  5. [12]

    C. G. Broyden, The convergence of a class of double-rank minimization algorithms 1. general considerations, IMA Journal of Applied Mathematics, 6 (1970), pp. 76–90

    1970

  6. [13]

    C. G. Broyden, J. E. Dennis Jr, and J. J. Mor ´e, On the local and superlinear conver- gence of quasi-Newton methods, IMA Journal of Applied Mathematics, 12 (1973), pp. 223–245

    1973

  7. [14]

    R. H. Byrd, J. Nocedal, and Y.-X. Yuan , Global convergence of a cass of quasi-Newton methods on convex problems, SIAM Journal on Numerical Analysis, 24 (1987), pp. 1171–1190

    1987

  8. [15]

    Cartis, N

    C. Cartis, N. I. Gould, and P. L. Toint , Adaptive cubic regularisation methods for un- constrained optimization. part i: motivation, convergence and numerical results, Mathematical Programming, 127 (2011), pp. 245–295

    2011

  9. [16]

    part ii: worst- case function-and derivative-evaluation complexity , Mathematical programming, 130 (2011), pp

    , Adaptive cubic regularisation methods for unconstrained optimization. part ii: worst- case function-and derivative-evaluation complexity , Mathematical programming, 130 (2011), pp. 295–319. 32

    2011

  10. [17]

    Chen and M

    X. Chen and M. Fukushima , Proximal quasi-Newton methods for nondifferentiable convex optimization, Mathematical Programming, 85 (1999), pp. 313–334

    1999

  11. [18]

    P. L. Combettes , Quasi-Fej´ erian analysis of some optimization algorithms, in Studies in Computational Mathematics, vol. 8, Elsevier, 2001, pp. 115–152

    2001

  12. [19]

    W. C. Davidon, Variable metric method for minimization , SIAM Journal on optimization, 1 (1991), pp. 1–17

    1991

  13. [20]

    J. E. Dennis and R. B. Schnabel , Numerical methods for unconstrained optimization and nonlinear equations, Society for Industrial and Applied Mathematics, 1996

    1996

  14. [21]

    J. E. Dennis, Jr and J. J. Mor ´e, Quasi-Newton methods, motivation and theory , SIAM review, 19 (1977), pp. 46–89

    1977

  15. [22]

    Dixon, Quasi Newton techniques generate identical points ii: the proofs of four new theo- rems, Mathematical Programming, 3 (1972), pp

    L. Dixon, Quasi Newton techniques generate identical points ii: the proofs of four new theo- rems, Mathematical Programming, 3 (1972), pp. 345–358

    1972

  16. [23]

    Doikov and Y

    N. Doikov and Y. Nesterov , Gradient regularization of Newton method with Bregman distances, Mathematical programming, 204 (2024), pp. 1–25

    2024

  17. [24]

    Fletcher , A new approach to variable metric algorithms , The computer journal, 13 (1970), pp

    R. Fletcher , A new approach to variable metric algorithms , The computer journal, 13 (1970), pp. 317–322

    1970

  18. [25]

    Fletcher and M

    R. Fletcher and M. J. Powell , A rapidly convergent descent method for minimization , The computer journal, 6 (1963), pp. 163–168

    1963

  19. [26]

    Frankel, G

    P. Frankel, G. Garrigos, and J. Peypouquet, Splitting methods with variable metric for kurdyka–Lojasiewicz functions and general convergence rates, Journal of Optimization Theory and Applications, 165 (2015), pp. 874–900

    2015

  20. [27]

    Goldfarb, A family of variable-metric methods derived by variational means , Mathemat- ics of computation, 24 (1970), pp

    D. Goldfarb, A family of variable-metric methods derived by variational means , Mathemat- ics of computation, 24 (1970), pp. 23–26

    1970

  21. [28]

    N. I. Gould, M. Porcelli, and P. L. Toint , Updating the regularization parameter in the adaptive cubic regularization algorithm , Computational optimization and applications, 53 (2012), pp. 1–22

    2012

  22. [29]

    G. N. Grapiglia, M. L. Gonc ¸alves, and G. Silva, A cubic regularization of Newton ’s method with finite difference hessian approximations, Numerical Algorithms, (2022), pp. 1–24

    2022

  23. [30]

    Q. Jin, R. Jiang, and A. Mokhtari, Non-asymptotic global convergence analysis of BFGS with the Armijo-Wolfe line search , arXiv preprint arXiv:2404.16731, (2024)

    2024 arXiv

  24. [31]

    Jin and A

    Q. Jin and A. Mokhtari, Non-asymptotic superlinear convergence of standard quasi-Newton methods, Mathematical Programming, 200 (2023), pp. 425–473

    2023

  25. [32]

    Kamzolov, K

    D. Kamzolov, K. Ziu, A. Agafonov, and M. Tak ´ac, Cubic regularized quasi-newton methods, arXiv preprint arXiv:2302.04987, (2023)

    2023 arXiv

  26. [33]

    Kanzow and T

    C. Kanzow and T. Lechner, Efficient regularized proximal quasi-Newton methods for large- scale nonconvex composite optimization problems , arXiv preprint arXiv:2210.07644, (2022)

    2022 arXiv

  27. [34]

    Karimi, J

    H. Karimi, J. Nutini, and M. Schmidt , Linear convergence of gradient and proximal- gradient methods under the Polyak-Lojasiewicz condition, in Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Italy, September 19-23, 2...

    2016

  28. [35]

    Karimi and S

    S. Karimi and S. V avasis, IMRO: A proximal quasi-Newton method for solving ℓ1-regularized least squares problems, SIAM Journal on Optimization, 27 (2017), pp. 583–615

    2017

  29. [36]

    J. D. Lee, Y. Sun, and M. Saunders, Proximal Newton-type methods for convex optimiza- tion, Advances in Neural Information Processing Systems, 25 (2012)

    2012

  30. [37]

    LEVENBERG, A method for the solution of certain non-linear problems in least squares , Quarterly of Applied Mathematics, 2 (1944), pp

    K. LEVENBERG, A method for the solution of certain non-linear problems in least squares , Quarterly of Applied Mathematics, 2 (1944), pp. 164–168. 33

    1944

  31. [38]

    Li and T

    G. Li and T. K. Pong , Calculus of the exponent of Kurdyka-L ojasiewicz inequality and its applications to linear convergence of first-order methods , Foundations of computational mathematics, 18 (2018), pp. 1199–1232

    2018

  32. [39]

    S. Lu, Z. Wei, and L. Li , A trust region algorithm with adaptive cubic regularization meth- ods for nonsmooth convex minimization , Computational Optimization and Applications, 51 (2012), pp. 551–573

    2012

  33. [40]

    D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, Jour- nal of the society for Industrial and Applied Mathematics, 11 (1963), pp. 431–441

    1963

  34. [41]

    Mishchenko, Regularized Newton method with global convergence, SIAM Journal on Op- timization, 33 (2023), pp

    K. Mishchenko, Regularized Newton method with global convergence, SIAM Journal on Op- timization, 33 (2023), pp. 1440–1462

    2023

  35. [42]

    Nesterov and B

    Y. Nesterov and B. T. Polyak , Cubic regularization of Newton method and its global performance, Mathematical programming, 108 (2006), pp. 177–205

    2006

  36. [43]

    Nocedal and S

    J. Nocedal and S. J. Wright , Numerical Optimization , Springer, New York, NY, USA, 2e ed., 2006

    2006

  37. [44]

    P. Ochs, Y. Chen, T. Brox, and T. Pock , iPiano: Inertial proximal algorithm for nonconvex optimization, SIAM Journal on Imaging Sciences, 7 (2014), pp. 1388–1419

    2014

  38. [45]

    Qian and S

    Y. Qian and S. Pan , A superlinear convergence framework for Kurdyka-Lojasiewicz opti- mization, arXiv e-prints, (2022), pp. arXiv–2210

    2022

  39. [46]

    Rebjock and N

    Q. Rebjock and N. Boumal , Fast convergence to non-isolated minima: four equivalent conditions for C2 functions , Mathematical Programming, (2024), pp. 1–49

    2024

  40. [47]

    R. T. Rockafellar and R. J.-B. Wets , Variational analysis, vol. 317, Springer Science & Business Media, 2009

    2009

  41. [48]

    Rodomanov , Global complexity analysis of BFGS , arXiv preprint arXiv:2404.15051, (2024)

    A. Rodomanov , Global complexity analysis of BFGS , arXiv preprint arXiv:2404.15051, (2024)

    2024 arXiv

  42. [49]

    Rodomanov and Y

    A. Rodomanov and Y. Nesterov , Greedy quasi-Newton methods with explicit superlinear convergence, SIAM Journal on Optimization, 31 (2021), pp. 785–811

    2021

  43. [50]

    , Rates of superlinear convergence for classical quasi-Newton methods , Mathematical Programming, (2022), pp. 1–32

    2022

  44. [51]

    Scheinberg and X

    K. Scheinberg and X. Tang , Practical inexact proximal quasi-Newton method with global complexity analysis, Mathematical Programming, 160 (2016), pp. 495–529

    2016

  45. [52]

    Stella, A

    L. Stella, A. Themelis, and P. Patrinos , Forward–backward quasi-Newton methods for nonsmooth optimization problems , Computational Optimization and Applications, 67 (2017), pp. 443–487

    2017

  46. [53]

    UCI Machine Learning Repository, 1981

    Mushroom. UCI Machine Learning Repository, 1981. DOI: https://doi.org/10.24432/C5959T

    1981 doi

  47. [55]

    , Quasi-Newton methods for monotone inclusions: efficient resolvent calculus and primal- dual algorithms , SIAM Journal on Imaging Sciences, 18 (2025), pp. 308–344

    2025

  48. [56]

    H. Ye, D. Lin, X. Chang, and Z. Zhang , Towards explicit superlinear convergence rate for SR1 , Mathematical Programming, 199 (2023), pp. 1273–1303. 34

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.