Pith. sign in

REVIEW 1 major objections 4 minor 32 references

Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems

T0 review · 1 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper proves that inexact projections preserve Levenberg-Marquardt convergence for constrained nonlinear systems, with linear rates under an error bound and superlinear rates when projection errors vanish.

desk verdict The local method is sound, but the global method has a feasibility bug in Step 2 that leaves Theorem 10 unproven; fixable, but the paper needs revision. read the letter →

arxiv 1908.06118 v1 pith:R3TFNLEL submitted 2019-08-16 math.OC

classification math.OC MSC 65H1065K0590C30
keywords constrainednonlinearsystemsLevenberg-Marquardtmethodinexactprojectionserrorboundconditionlocalconvergenceglobalnonmonotonelinesearchspectrahedron
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to establish that Levenberg-Marquardt methods for constrained nonlinear systems $F(x)=0$, $x\in C$, remain convergent when the projection onto the feasible set $C$ is computed only approximately. The central device is the feasible $\varepsilon$-projection of Definition 1: a point of $C$ that satisfies the projection variational inequality up to a tolerance $\varepsilon$. The authors prove that, under a local error bound weaker than full-rank of the Jacobian, the distance $\operatorname{dist}(x_k,C^*)$ to the solution set decreases linearly, and superlinearly when the projection tolerances tend to zero (Theorem 7). A globalized version, using a line search that permits occasional increases in the merit function $\|F(x)\|^2/2$, is shown to have every accumulation point stationary for that merit function (Theorem 10). The motivation, confirmed by numerical experiments on box-constrained systems and on systems over the spectrahedron, is that exact projections are often the expensive step, and approximate ones can preserve the theory while saving substantial computation time.

What carries the argument

The load-bearing object is the feasible $\varepsilon$-projection: for $x\in\mathbb{R}^n$ and $\varepsilon\ge 0$, a point $P_C(x,\varepsilon)\in C$ satisfying $\langle x-P_C(x,\varepsilon),\, y-P_C(x,\varepsilon)\rangle\le\varepsilon$ for all $y\in C$. This relaxes the variational characterization of the orthogonal projection while keeping the output feasible, so approximate projection oracles can be used. The method's step is the unconstrained Levenberg-Marquardt direction $d_k^U$ solving $(F'(x_k)^TF'(x_k)+\mu_k I)d=-F'(x_k)^TF(x_k)$ with $\mu_k=\|F(x_k)\|^2$, followed by an $\varepsilon_k$-projection with $\varepsilon_k=\theta_k^2\|d_k^U\|^2$. The proof engine is a one-step estimate (Lemma 4) bounding $\operatorname{dist}(x_{k+1},C^*)$ by $\theta_k c_1\operatorname{dist}(x_k,C^*)$ plus a quadratic term, with constants $c_1,c_2$ coming from the Lipschitz constant and the error-bound constant $\omega$. This estimate turns the local error bound (A2) into contraction, replacing the usual full-rank assumption.

What would settle it

Construct a smooth $F$ whose zero lies on the boundary of $C$ and for which $\|F(x)\|/\operatorname{dist}(x,C^*)\to 0$ as $x\to x^*$, so that (A2) fails, start the LMM-IP at points arbitrarily close to $x^*$, and check whether $\operatorname{dist}(x_k,C^*)$ still decreases linearly. A direct observable is the ratio $\|F(x_k)\|/\operatorname{dist}(x_k,C^*)$ along iterates: if it tends to zero while the distance to $C^*$ also decreases linearly, the error-bound condition is doing none of the work, and the theorem's stated assumption is not necessary in that case.

Watch

Extended reading notes

Core claim

The paper's core claim is that the local convergence theory of projected Levenberg-Marquardt extends to a setting where the projection operator is replaced by a feasible $\varepsilon$-projection $P_C(x,\varepsilon)$. With $x_0$ sufficiently close to a solution $x_*$, tolerances $\theta_k\le\bar\theta<1/c_1$, and the local error bound $\omega\,\operatorname{dist}(x,C^*)\le\|F(x)\|$, the iterates satisfy a one-step contraction that gives linear convergence of $\operatorname{dist}(x_k,C^*)$ to zero; if $\theta_k\to 0$, the rate becomes superlinear, and the iterates converge to a point of $C^*$ (Theorem 7). For the global method, the paper claims that every accumulation point is stationary for $\min_{x\in C}\|F(x)\|^2/2$ (Theorem 10). In both cases the argument runs through the error-bound condition rather than through full-rank Jacobians, so underdetermined and rank-deficient systems are covered as long as the error bound holds.

Load-bearing premise

The local-rate results stand on the local error bound $\omega\,\operatorname{dist}(x,C^*)\le\|F(x)\|$ near a solution, and the paper itself notes this bound may fail for solutions on the boundary of $C$; if it fails, the linear and superlinear convergence claims are not established.

Editorial extensions

If this is right

  • Projected Levenberg-Marquardt works with iterative, approximate projection solvers, so feasible sets whose exact projections are costly become tractable.
  • The methods solve underdetermined and rank-deficient systems under the local error bound, not only systems with full-rank Jacobians.
  • When projection tolerances are driven to zero ($\theta_k\to 0$), distance-to-solution convergence improves from linear to superlinear, and the iterates themselves converge to a solution.
  • The globalized method, with a nonmonotone line search, yields stationary points of $\|F(x)\|^2/2$ from any starting point whose sublevel set is bounded, giving a practical general solver.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the $\theta_k\to 0$ condition translates into a concrete budget for the inner projection solver—only logarithmically many extra inner iterations should be needed as the outer iterates converge, which is why even fixed modest accuracy was enough in the reported spectrahedron tests.
  • My inference: the analysis is agnostic to how the $\varepsilon$-projection is computed, so the same convergence theorems should carry over to randomized or sketching-based projection oracles, as long as the variational inequality of Definition 1 is approximately satisfied.
  • My inference: the boundary-zero caveat on the error bound suggests that a natural follow-up is an active-set or merit-function correction that restores linear convergence when $C^*$ meets the boundary of $C$.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 4 minor

Summary. The paper proposes Levenberg-Marquardt methods for constrained nonlinear systems F(x)=0, x∈C, where the projection onto the closed convex set C is allowed to be inexact in the sense of Definition 1. The local method LMM-IP projects x_k+dU_k onto C with tolerance θ_k^2||dU_k||^2 and, under a Lipschitz Jacobian assumption and a local error bound (A2), proves that dist(x_k,C*) converges to zero linearly, superlinearly if θ_k→0, and that the iterates converge to a solution; Theorem 9 adds linear/superlinear rates for the iterates themselves. The global method G-LMM-IP combines the LM step with a nonmonotone line search and safeguarded inexact projected-gradient steps, claiming that every accumulation point is stationary for min_{x∈C} ||F(x)||^2/2. Numerical experiments compare exact and inexact projections on box-constrained systems and on systems over the spectrahedron.

Significance. The local convergence theory is the strongest part of the paper. Definition 1 is a practical and natural notion of inexact projection, Proposition 1 gives a clean Lipschitz-type estimate for it, and Lemmas 4 and 6 correctly propagate the local error-bound estimates. The paper would be a useful extension of the projected Levenberg-Marquardt analyses of Kanzow-Yamashita-Fukushima and others, and the spectrahedron experiments give a concrete setting where inexact projections pay off. However, the global method as stated contains a load-bearing feasibility flaw: Step 2 may select a search direction that points outside C, so the invariant x_k∈C used throughout the proof of Theorem 10 is not maintained. This is a substantive defect in the global convergence claim, though it appears repairable by modifying Step 2.

major comments (1)
  1. [Section 3, G-LMM-IP Step 2 and Theorem 10] The global algorithm is not a feasible constrained method as written. In Step 2, dbar_k=PC(x_k+dU_k,ε_k)-x_k is a feasible direction from x_k because x_k∈C and PC(x_k+dU_k,ε_k)∈C, so x_k+α dbar_k∈C for α∈[0,1]. But when ⟨∇f(x_k),dbar_k⟩>0, the algorithm sets d_k=-dbar_k, and x_k+α d_k = x_k-α(y_k-x_k) leaves the convex set C for every α>0 in general. Step 4 evaluates f at x_k+α d_k without any feasibility check. Consequently the invariant x_k∈C, which is used in the proof of (38)-(39) (for example, to assert PC(x_k)=x_k in the lower bound of (39)), can be lost, and the proof of Theorem 10 never establishes that the trial points lie in C. The stationarity conclusion is therefore not proven for the algorithm as stated. This issue occurs even for exact projections θ_k=0. A repair would be to accept dbar_k only when it is a descent direction, e.g. ⟨∇f(x_k),dbar_k⟩≤-η_1||dbar_k||², and otherwise to switch to the projected-gradient step of Step 3; alternatively, the line search must explicitly enforce x_k+α d_k∈C and d_k must be a feasible direction.
minor comments (4)
  1. [Theorem 9, proof, Eq. (32)] The displayed inequality in (32) has α_1 c_1^2 c_3 ||x_k-xbar||, but substituting Lemma 8(b) and Lemma 8(c) gives α_1 c_1^3 c_3 ||x_k-xbar||. As printed, Eq. (32) is inconsistent with the rate constant 4c_1^3(1+α_1)^2/(1-2α_1c_1) stated in Theorem 9; please correct the typo.
  2. [Table 3] The column header γ denotes the starting-point parameter a∈{0,1/2,1}, not the line-search parameter γ introduced in Step 0 of G-LMM-IP. Rename the column to avoid confusion.
  3. [Section 4.2] The sentence 'as for these instances we known the rank' contains a grammatical error: it should read 'we know the rank'.
  4. [Remark 1(i)] The phrase 'Definition (1)' should be 'Definition 1' for consistency with the rest of the paper.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: local and global convergence proofs are carried by external LM estimates and line-search theory; background self-citations are not load-bearing.

full rationale

The derivation chain is self-contained and does not reduce its conclusions to its inputs. The local convergence result (Theorem 7) relies on the unconstrained Levenberg-Marquardt estimates (13)-(14), imported from Yamashita-Fukushima [24, Lemma 2.1], and on the error bound condition (A2) from Kanzow, Yamashita, and Fukushima [9]; both are external results with stated assumptions that do not include the paper's convergence conclusions. The new components, namely Definition 1, Proposition 1, Lemma 4, Corollary 5, and Lemma 6, are proved directly from the epsilon-projection definition and from those external estimates. The global convergence proof (Theorem 10) follows the external nonmonotone line-search framework of Grippo et al. [26] and derives the descent and angle conditions (38)-(39) from the algorithm's own Step 2 and Step 3 constructions; no equation is made true by definition of the target quantity. Self-citations [7], [8], and [31] appear only as background or as a pointer to the standard spectrahedron projection formula, and none supplies a load-bearing premise or an author-only uniqueness theorem. The paper explicitly notes that (A2) may fail for zeros on the boundary of C, which is an assumption limitation rather than a circular step. The concern that Step 2 of G-LMM-IP can choose an infeasible trial direction is a feasibility/correctness issue about the global method's invariant, not an equivalence of inputs and outputs, so it does not affect the circularity score. Numerical experiments are illustrative comparisons and do not present fitted parameters as predictions of the proved rates.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central result introduces no fitted constants. It rests on smoothness and error-bound assumptions standard in the LM literature, on computability of the new epsilon-projections, and on a bounded sublevel set for the global method. The application section adds a low-rank assumption on the spectrahedron solution, which is not part of the general theorems.

assumptions (6)
  • domain assumption Assumption (A0): C* is nonempty and x* is an arbitrary element of C*.
    Needed to define distances to the solution set and to prove convergence to a point of C*; standard in constrained systems.
  • domain assumption Assumption (A1): F' is L-Lipschitz continuous on B(x*, delta1).
    Used for the quadratic expansion (12) and the bounds (13)-(14) from Yamashita-Fukushima.
  • domain assumption Assumption (A2): omega dist(x,C*) <= ||F(x)|| on B(x*, delta2), the local error bound condition.
    Load-bearing for the linear and superlinear rate proofs; the paper notes it may fail for boundary zeros.
  • ad hoc to paper Existence and computability of feasible epsilon-projections satisfying (6) at every iteration.
    The algorithm's update step requires such points; Remark 1 gives conditional gradient methods as one way but no general guarantee of computational cost.
  • domain assumption Boundedness of the sublevel set Omega0 = {x in C : f(x) <= f(x0)} in Theorem 10.
    Needed for the nonmonotone line-search global convergence proof.
  • standard math Standard convex analysis facts: nonexpansiveness and variational characterization of the orthogonal projection, mean value theorem.
    Used in Propositions 1 and 2 and in the proof of Theorem 10.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems." pith.science (2026). https://pith.science/paper/R3TFNLEL

@misc{pith2026190806118,
  author       = {Pith},
  title        = {Pith review of: Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R3TFNLEL}},
  note         = {Machine review of arXiv:1908.06118}
}
read the original abstract

In this paper, we first propose a new Levenberg-Marquardt method for solving constrained (and not necessarily square) nonlinear systems. Basically, the method combines the unconstrained Levenberg-Marquardt method with a type of feasible inexact projection. The local convergence of the new method as well as results on its rate are established by using an error bound condition, which is weaker than the standard full-rank assumption. We further present and analyze a global version of the first method by means of a nonmonotone line search technique. Finally, numerical experiments illustrating the practical advantages of the proposed schemes are reported.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages

  1. [1]

    Behling, A

    R. Behling, A. Fischer, G. Haeser, A. Ramos, K. Sch¨ onefeld, On the con- strained error bound condition and the projected Levenberg-Ma rquardt method, Optimization 66 (8) (2017) 1397–1411 (2017)

  2. [2]

    Behling, A

    R. Behling, A. Fischer, M. Herrich, A. Iusem, Y. Ye, A Levenberg - Marquardt method with approximate projections, Comput. Optim. Appl. 59 (1) (2014) 5–26 (2014)

  3. [3]

    Bellavia, M

    S. Bellavia, M. Macconi, B. Morini, An affine scaling trust-region app roach to bound-constrained nonlinear systems, Appl. Num. Math. 44 (3) (2003) 257–280 (2003)

  4. [4]

    Bellavia, B

    S. Bellavia, B. Morini, Subspace trust-region methods for large b ound- constrained nonlinear equations, SIAM J. Numer. Anal. 44 (4) (200 6) 1535– 1555 (2006)

  5. [5]

    Echebest, M

    N. Echebest, M. L. Schuverdt, R. P. Vignau, A derivative-free method for solving box-constrained underdetermined nonlinear systems of eq uations, Appl. Math. Comput. 219 (6) (2012) 3198–3208 (2012)

  6. [6]

    Fan, On the Levenberg-Marquardt methods for convex con strained non- linear equations, J

    J. Fan, On the Levenberg-Marquardt methods for convex con strained non- linear equations, J. Ind. Manag. Optim. 9 (1) (2013) 227–241 (201 3). 22

  7. [7]

    M. L. N. Gon¸ calves, J. G. Melo, A Newton conditional gradient me thod for constrained nonlinear systems, J. Comput. Appl. Math. 311 (2017 ) 473–483 (2017)

  8. [8]

    M. L. N. Gon¸ calves, F. R. Oliveira, An inexact Newton-like conditio nal gradient method for constrained nonlinear systems, Appl. Num. Ma th. 132 (2018) 22–34 (2018)

Show all 32 references
  1. [9]

    Kanzow, N

    C. Kanzow, N. Yamashita, M. Fukushima, Levenberg-Marquard t methods with strong local convergence properties for solving nonlinear equ ations with convex constraints, J. Comput. Appl. Math. 172 (2) (2004) 3 75–397 (2004)

  2. [10]

    D. N. Kozakevich, J. M. Martinez, S. A. Santos, Solving nonlinea r systems of equations with simple constraints, Comput. Appl. Math. 16 (1997 ) 215– 235 (1997)

  3. [11]

    La Cruz, A projected derivative-free algorithm for nonlinea r equations with convex constraints, Optim

    W. La Cruz, A projected derivative-free algorithm for nonlinea r equations with convex constraints, Optim. Methods Softw. 29 (1) (2014) 24 –41 (2014)

  4. [12]

    Macconi, B

    M. Macconi, B. Morini, M. Porcelli, Trust-region quadratic metho ds for nonlinear systems of mixed equalities and inequalities, Appl. Num. Math . 59 (5) (2009) 859–876 (2009)

  5. [13]

    Marini, B

    L. Marini, B. Morini, M. Porcelli, Quasi-Newton methods for const rained nonlinear systems: complexity analysis and applications, Comput. Op tim. Appl. 71 (2018) 147–170 (2018)

  6. [14]

    M. J. Martinez, Quasi-inexact-Newton methods with global con vergent for solving constrained nonlinear systems, Nonlinear Anal. 30 (1) (1997 ) 1–7 (1997)

  7. [15]

    Morini, M

    B. Morini, M. Porcelli, P. L. Toint, Approximate norm descent met hods for constrained nonlinear systems, Math. Comput. 87 (311) (2018) 1 327–1351 (2018)

  8. [16]

    Porcelli, On the convergence of an inexact Gauss-Newton tr ust-region method for nonlinear least-squares problems with simple bounds, Op tim

    M. Porcelli, On the convergence of an inexact Gauss-Newton tr ust-region method for nonlinear least-squares problems with simple bounds, Op tim. Letters 7 (3) (2013) 447–465 (2013)

  9. [17]

    P. Wang, D. Zhu, An inexact derivative-free levenberg–marqu ardt method for linear inequality constrained nonlinear systems under local erro r bound conditions, Appl. Math. Comput. 282 (2016) 32–52 (2016)

  10. [18]

    Zhang, D.-t

    Y. Zhang, D.-t. Zhu, Inexact Newton method via Lanczos deco mposed tech- nique for solving box-constrained nonlinear systems, Appl. Math. M ech. 31 (12) (2010) 1593–1602 (2010)

  11. [19]

    Zhu, An affine scaling trust-region algorithm with interior back track- ing technique for solving bound-constrained nonlinear systems, J

    D. Zhu, An affine scaling trust-region algorithm with interior back track- ing technique for solving bound-constrained nonlinear systems, J. Comput. App. Math. 184 (2) (2005) 343–361 (2005). 23

  12. [20]

    Levenberg, A method for the solution of certain nonlinear pr oblem in least squares, Quart

    K. Levenberg, A method for the solution of certain nonlinear pr oblem in least squares, Quart. Appl. Math. (2) (1944) 164–166 (1944)

  13. [21]

    D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math. (11) (1963) 431–441 (1963)

  14. [22]

    Zhang, On the convergence properties of the Levenber g-Marquardt method, Optimization 52 (6) (2003) 739–756 (2003)

    J.-L. Zhang, On the convergence properties of the Levenber g-Marquardt method, Optimization 52 (6) (2003) 739–756 (2003)

  15. [23]

    Fan, Y.-x

    J.-y. Fan, Y.-x. Yuan, On the quadratic convergence of the lev enberg- marquardt method without nonsingularity assumption, Computing 7 4 (1) (2005) 23–39 (2005)

  16. [24]

    Yamashita, M

    N. Yamashita, M. Fukushima, On the rate of convergence of th e Levenberg- Marquardt method, in: Topics in Numerical Analysis, Springer Vienna , 2001, pp. 239–249 (2001)

  17. [25]

    Jaggi, Revisiting Frank-Wolfe: Projection-free sparse con vex optimiza- tion, in: Proceedings of the 30th International Conference on Ma chine Learning (ICML-13), Vol

    M. Jaggi, Revisiting Frank-Wolfe: Projection-free sparse con vex optimiza- tion, in: Proceedings of the 30th International Conference on Ma chine Learning (ICML-13), Vol. 28, 2013, pp. 427–435 (2013)

  18. [26]

    Grippo, F

    L. Grippo, F. Lampariello, S. Lucidi, A nonmonotone line search te chnique for Newton’s method, SIAM J. Numer. Anal. 23 (4) (1986) 707–716 (1986)

  19. [27]

    Frank, P

    M. Frank, P. Wolfe, An algorithm for quadratic programming, Na val Res. Logist. 3 (1-2) (1956) 95–110 (1956)

  20. [28]

    N. I. M. Gould, D. Orban, P. L. Toint, Cuter, a constrained and uncon- strained testing environment, revisited, ACM Trans. Math. Softw. 29 (2003) 373–394 (2003)

  21. [29]

    W. Hock, K. Schittkowski, Test examples for nonlinear program ming codes, Journal of Optimization Theory and Applications 30 (1) (1980) 127– 129 (1980)

  22. [30]

    Morini, M

    B. Morini, M. Porcelli, TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities, Comput. Optim. Appl. 51 (2012 ) 27–49 (2012)

  23. [31]

    D. S. Gon¸ calves, M. A. Gomes-Ruggiero, C. Lavor, A project ed gradient method for optimization over density matrices, Optim. Methods Sof tw. 31 (2016) 328–341 (2016)

  24. [32]

    Allen-Zhu, E

    Z. Allen-Zhu, E. Hazan, W. Hu, Y. Li, Linear Convergence of a Fr ank- Wolfe type algorithm over trace-norm balls, in: I. Guyon, U. V. Luxb urg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Ed s.), Advances in Neural Information Processing Systems 30, Curran A...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.