REVIEW 1 major objections 4 minor 32 references
Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems
T0 review · 1 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper proves that inexact projections preserve Levenberg-Marquardt convergence for constrained nonlinear systems, with linear rates under an error bound and superlinear rates when projection errors vanish.
desk verdict The local method is sound, but the global method has a feasibility bug in Step 2 that leaves Theorem 10 unproven; fixable, but the paper needs revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the feasible $\varepsilon$-projection: for $x\in\mathbb{R}^n$ and $\varepsilon\ge 0$, a point $P_C(x,\varepsilon)\in C$ satisfying $\langle x-P_C(x,\varepsilon),\, y-P_C(x,\varepsilon)\rangle\le\varepsilon$ for all $y\in C$. This relaxes the variational characterization of the orthogonal projection while keeping the output feasible, so approximate projection oracles can be used. The method's step is the unconstrained Levenberg-Marquardt direction $d_k^U$ solving $(F'(x_k)^TF'(x_k)+\mu_k I)d=-F'(x_k)^TF(x_k)$ with $\mu_k=\|F(x_k)\|^2$, followed by an $\varepsilon_k$-projection with $\varepsilon_k=\theta_k^2\|d_k^U\|^2$. The proof engine is a one-step estimate (Lemma 4) bounding $\operatorname{dist}(x_{k+1},C^*)$ by $\theta_k c_1\operatorname{dist}(x_k,C^*)$ plus a quadratic term, with constants $c_1,c_2$ coming from the Lipschitz constant and the error-bound constant $\omega$. This estimate turns the local error bound (A2) into contraction, replacing the usual full-rank assumption.
What would settle it
Construct a smooth $F$ whose zero lies on the boundary of $C$ and for which $\|F(x)\|/\operatorname{dist}(x,C^*)\to 0$ as $x\to x^*$, so that (A2) fails, start the LMM-IP at points arbitrarily close to $x^*$, and check whether $\operatorname{dist}(x_k,C^*)$ still decreases linearly. A direct observable is the ratio $\|F(x_k)\|/\operatorname{dist}(x_k,C^*)$ along iterates: if it tends to zero while the distance to $C^*$ also decreases linearly, the error-bound condition is doing none of the work, and the theorem's stated assumption is not necessary in that case.
Extended reading notes
Core claim
The paper's core claim is that the local convergence theory of projected Levenberg-Marquardt extends to a setting where the projection operator is replaced by a feasible $\varepsilon$-projection $P_C(x,\varepsilon)$. With $x_0$ sufficiently close to a solution $x_*$, tolerances $\theta_k\le\bar\theta<1/c_1$, and the local error bound $\omega\,\operatorname{dist}(x,C^*)\le\|F(x)\|$, the iterates satisfy a one-step contraction that gives linear convergence of $\operatorname{dist}(x_k,C^*)$ to zero; if $\theta_k\to 0$, the rate becomes superlinear, and the iterates converge to a point of $C^*$ (Theorem 7). For the global method, the paper claims that every accumulation point is stationary for $\min_{x\in C}\|F(x)\|^2/2$ (Theorem 10). In both cases the argument runs through the error-bound condition rather than through full-rank Jacobians, so underdetermined and rank-deficient systems are covered as long as the error bound holds.
Load-bearing premise
The local-rate results stand on the local error bound $\omega\,\operatorname{dist}(x,C^*)\le\|F(x)\|$ near a solution, and the paper itself notes this bound may fail for solutions on the boundary of $C$; if it fails, the linear and superlinear convergence claims are not established.
Editorial extensions
If this is right
- Projected Levenberg-Marquardt works with iterative, approximate projection solvers, so feasible sets whose exact projections are costly become tractable.
- The methods solve underdetermined and rank-deficient systems under the local error bound, not only systems with full-rank Jacobians.
- When projection tolerances are driven to zero ($\theta_k\to 0$), distance-to-solution convergence improves from linear to superlinear, and the iterates themselves converge to a solution.
- The globalized method, with a nonmonotone line search, yields stationary points of $\|F(x)\|^2/2$ from any starting point whose sublevel set is bounded, giving a practical general solver.
Reading between the lines
- My inference: the $\theta_k\to 0$ condition translates into a concrete budget for the inner projection solver—only logarithmically many extra inner iterations should be needed as the outer iterates converge, which is why even fixed modest accuracy was enough in the reported spectrahedron tests.
- My inference: the analysis is agnostic to how the $\varepsilon$-projection is computed, so the same convergence theorems should carry over to randomized or sketching-based projection oracles, as long as the variational inequality of Definition 1 is approximately satisfied.
- My inference: the boundary-zero caveat on the error bound suggests that a natural follow-up is an active-set or merit-function correction that restores linear convergence when $C^*$ meets the boundary of $C$.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Levenberg-Marquardt methods for constrained nonlinear systems F(x)=0, x∈C, where the projection onto the closed convex set C is allowed to be inexact in the sense of Definition 1. The local method LMM-IP projects x_k+dU_k onto C with tolerance θ_k^2||dU_k||^2 and, under a Lipschitz Jacobian assumption and a local error bound (A2), proves that dist(x_k,C*) converges to zero linearly, superlinearly if θ_k→0, and that the iterates converge to a solution; Theorem 9 adds linear/superlinear rates for the iterates themselves. The global method G-LMM-IP combines the LM step with a nonmonotone line search and safeguarded inexact projected-gradient steps, claiming that every accumulation point is stationary for min_{x∈C} ||F(x)||^2/2. Numerical experiments compare exact and inexact projections on box-constrained systems and on systems over the spectrahedron.
Significance. The local convergence theory is the strongest part of the paper. Definition 1 is a practical and natural notion of inexact projection, Proposition 1 gives a clean Lipschitz-type estimate for it, and Lemmas 4 and 6 correctly propagate the local error-bound estimates. The paper would be a useful extension of the projected Levenberg-Marquardt analyses of Kanzow-Yamashita-Fukushima and others, and the spectrahedron experiments give a concrete setting where inexact projections pay off. However, the global method as stated contains a load-bearing feasibility flaw: Step 2 may select a search direction that points outside C, so the invariant x_k∈C used throughout the proof of Theorem 10 is not maintained. This is a substantive defect in the global convergence claim, though it appears repairable by modifying Step 2.
major comments (1)
- [Section 3, G-LMM-IP Step 2 and Theorem 10] The global algorithm is not a feasible constrained method as written. In Step 2, dbar_k=PC(x_k+dU_k,ε_k)-x_k is a feasible direction from x_k because x_k∈C and PC(x_k+dU_k,ε_k)∈C, so x_k+α dbar_k∈C for α∈[0,1]. But when ⟨∇f(x_k),dbar_k⟩>0, the algorithm sets d_k=-dbar_k, and x_k+α d_k = x_k-α(y_k-x_k) leaves the convex set C for every α>0 in general. Step 4 evaluates f at x_k+α d_k without any feasibility check. Consequently the invariant x_k∈C, which is used in the proof of (38)-(39) (for example, to assert PC(x_k)=x_k in the lower bound of (39)), can be lost, and the proof of Theorem 10 never establishes that the trial points lie in C. The stationarity conclusion is therefore not proven for the algorithm as stated. This issue occurs even for exact projections θ_k=0. A repair would be to accept dbar_k only when it is a descent direction, e.g. ⟨∇f(x_k),dbar_k⟩≤-η_1||dbar_k||², and otherwise to switch to the projected-gradient step of Step 3; alternatively, the line search must explicitly enforce x_k+α d_k∈C and d_k must be a feasible direction.
minor comments (4)
- [Theorem 9, proof, Eq. (32)] The displayed inequality in (32) has α_1 c_1^2 c_3 ||x_k-xbar||, but substituting Lemma 8(b) and Lemma 8(c) gives α_1 c_1^3 c_3 ||x_k-xbar||. As printed, Eq. (32) is inconsistent with the rate constant 4c_1^3(1+α_1)^2/(1-2α_1c_1) stated in Theorem 9; please correct the typo.
- [Table 3] The column header γ denotes the starting-point parameter a∈{0,1/2,1}, not the line-search parameter γ introduced in Step 0 of G-LMM-IP. Rename the column to avoid confusion.
- [Section 4.2] The sentence 'as for these instances we known the rank' contains a grammatical error: it should read 'we know the rank'.
- [Remark 1(i)] The phrase 'Definition (1)' should be 'Definition 1' for consistency with the rest of the paper.
Circularity Check
No significant circularity: local and global convergence proofs are carried by external LM estimates and line-search theory; background self-citations are not load-bearing.
full rationale
The derivation chain is self-contained and does not reduce its conclusions to its inputs. The local convergence result (Theorem 7) relies on the unconstrained Levenberg-Marquardt estimates (13)-(14), imported from Yamashita-Fukushima [24, Lemma 2.1], and on the error bound condition (A2) from Kanzow, Yamashita, and Fukushima [9]; both are external results with stated assumptions that do not include the paper's convergence conclusions. The new components, namely Definition 1, Proposition 1, Lemma 4, Corollary 5, and Lemma 6, are proved directly from the epsilon-projection definition and from those external estimates. The global convergence proof (Theorem 10) follows the external nonmonotone line-search framework of Grippo et al. [26] and derives the descent and angle conditions (38)-(39) from the algorithm's own Step 2 and Step 3 constructions; no equation is made true by definition of the target quantity. Self-citations [7], [8], and [31] appear only as background or as a pointer to the standard spectrahedron projection formula, and none supplies a load-bearing premise or an author-only uniqueness theorem. The paper explicitly notes that (A2) may fail for zeros on the boundary of C, which is an assumption limitation rather than a circular step. The concern that Step 2 of G-LMM-IP can choose an infeasible trial direction is a feasibility/correctness issue about the global method's invariant, not an equivalence of inputs and outputs, so it does not affect the circularity score. Numerical experiments are illustrative comparisons and do not present fitted parameters as predictions of the proved rates.
Assumptions & free parameters
assumptions (6)
- domain assumption Assumption (A0): C* is nonempty and x* is an arbitrary element of C*.
- domain assumption Assumption (A1): F' is L-Lipschitz continuous on B(x*, delta1).
- domain assumption Assumption (A2): omega dist(x,C*) <= ||F(x)|| on B(x*, delta2), the local error bound condition.
- ad hoc to paper Existence and computability of feasible epsilon-projections satisfying (6) at every iteration.
- domain assumption Boundedness of the sublevel set Omega0 = {x in C : f(x) <= f(x0)} in Theorem 10.
- standard math Standard convex analysis facts: nonexpansiveness and variational characterization of the orthogonal projection, mean value theorem.
Cite this review
Pith. "Pith review of Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems." pith.science (2026). https://pith.science/paper/R3TFNLEL
@misc{pith2026190806118,
author = {Pith},
title = {Pith review of: Levenberg-Marquardt methods with inexact projections for constrained nonlinear systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/R3TFNLEL}},
note = {Machine review of arXiv:1908.06118}
}
read the original abstract
In this paper, we first propose a new Levenberg-Marquardt method for solving constrained (and not necessarily square) nonlinear systems. Basically, the method combines the unconstrained Levenberg-Marquardt method with a type of feasible inexact projection. The local convergence of the new method as well as results on its rate are established by using an error bound condition, which is weaker than the standard full-rank assumption. We further present and analyze a global version of the first method by means of a nonmonotone line search technique. Finally, numerical experiments illustrating the practical advantages of the proposed schemes are reported.
Reference graph
Works this paper leans on
-
[1]
R. Behling, A. Fischer, G. Haeser, A. Ramos, K. Sch¨ onefeld, On the con- strained error bound condition and the projected Levenberg-Ma rquardt method, Optimization 66 (8) (2017) 1397–1411 (2017)
work page 2017
-
[2]
R. Behling, A. Fischer, M. Herrich, A. Iusem, Y. Ye, A Levenberg - Marquardt method with approximate projections, Comput. Optim. Appl. 59 (1) (2014) 5–26 (2014)
work page 2014
-
[3]
S. Bellavia, M. Macconi, B. Morini, An affine scaling trust-region app roach to bound-constrained nonlinear systems, Appl. Num. Math. 44 (3) (2003) 257–280 (2003)
work page 2003
-
[4]
S. Bellavia, B. Morini, Subspace trust-region methods for large b ound- constrained nonlinear equations, SIAM J. Numer. Anal. 44 (4) (200 6) 1535– 1555 (2006)
work page 2006
-
[5]
N. Echebest, M. L. Schuverdt, R. P. Vignau, A derivative-free method for solving box-constrained underdetermined nonlinear systems of eq uations, Appl. Math. Comput. 219 (6) (2012) 3198–3208 (2012)
work page 2012
-
[6]
Fan, On the Levenberg-Marquardt methods for convex con strained non- linear equations, J
J. Fan, On the Levenberg-Marquardt methods for convex con strained non- linear equations, J. Ind. Manag. Optim. 9 (1) (2013) 227–241 (201 3). 22
work page 2013
-
[7]
M. L. N. Gon¸ calves, J. G. Melo, A Newton conditional gradient me thod for constrained nonlinear systems, J. Comput. Appl. Math. 311 (2017 ) 473–483 (2017)
work page 2017
-
[8]
M. L. N. Gon¸ calves, F. R. Oliveira, An inexact Newton-like conditio nal gradient method for constrained nonlinear systems, Appl. Num. Ma th. 132 (2018) 22–34 (2018)
work page 2018
Show all 32 references
-
[9]
Kanzow, N
C. Kanzow, N. Yamashita, M. Fukushima, Levenberg-Marquard t methods with strong local convergence properties for solving nonlinear equ ations with convex constraints, J. Comput. Appl. Math. 172 (2) (2004) 3 75–397 (2004)
2004
-
[10]
D. N. Kozakevich, J. M. Martinez, S. A. Santos, Solving nonlinea r systems of equations with simple constraints, Comput. Appl. Math. 16 (1997 ) 215– 235 (1997)
1997
-
[11]
La Cruz, A projected derivative-free algorithm for nonlinea r equations with convex constraints, Optim
W. La Cruz, A projected derivative-free algorithm for nonlinea r equations with convex constraints, Optim. Methods Softw. 29 (1) (2014) 24 –41 (2014)
2014
-
[12]
Macconi, B
M. Macconi, B. Morini, M. Porcelli, Trust-region quadratic metho ds for nonlinear systems of mixed equalities and inequalities, Appl. Num. Math . 59 (5) (2009) 859–876 (2009)
2009
-
[13]
Marini, B
L. Marini, B. Morini, M. Porcelli, Quasi-Newton methods for const rained nonlinear systems: complexity analysis and applications, Comput. Op tim. Appl. 71 (2018) 147–170 (2018)
2018
-
[14]
M. J. Martinez, Quasi-inexact-Newton methods with global con vergent for solving constrained nonlinear systems, Nonlinear Anal. 30 (1) (1997 ) 1–7 (1997)
1997
-
[15]
Morini, M
B. Morini, M. Porcelli, P. L. Toint, Approximate norm descent met hods for constrained nonlinear systems, Math. Comput. 87 (311) (2018) 1 327–1351 (2018)
2018
-
[16]
Porcelli, On the convergence of an inexact Gauss-Newton tr ust-region method for nonlinear least-squares problems with simple bounds, Op tim
M. Porcelli, On the convergence of an inexact Gauss-Newton tr ust-region method for nonlinear least-squares problems with simple bounds, Op tim. Letters 7 (3) (2013) 447–465 (2013)
2013
-
[17]
P. Wang, D. Zhu, An inexact derivative-free levenberg–marqu ardt method for linear inequality constrained nonlinear systems under local erro r bound conditions, Appl. Math. Comput. 282 (2016) 32–52 (2016)
2016
-
[18]
Zhang, D.-t
Y. Zhang, D.-t. Zhu, Inexact Newton method via Lanczos deco mposed tech- nique for solving box-constrained nonlinear systems, Appl. Math. M ech. 31 (12) (2010) 1593–1602 (2010)
2010
-
[19]
Zhu, An affine scaling trust-region algorithm with interior back track- ing technique for solving bound-constrained nonlinear systems, J
D. Zhu, An affine scaling trust-region algorithm with interior back track- ing technique for solving bound-constrained nonlinear systems, J. Comput. App. Math. 184 (2) (2005) 343–361 (2005). 23
2005
-
[20]
Levenberg, A method for the solution of certain nonlinear pr oblem in least squares, Quart
K. Levenberg, A method for the solution of certain nonlinear pr oblem in least squares, Quart. Appl. Math. (2) (1944) 164–166 (1944)
1944
-
[21]
D. W. Marquardt, An algorithm for least-squares estimation of nonlinear inequalities, SIAM J. Appl. Math. (11) (1963) 431–441 (1963)
1963
-
[22]
Zhang, On the convergence properties of the Levenber g-Marquardt method, Optimization 52 (6) (2003) 739–756 (2003)
J.-L. Zhang, On the convergence properties of the Levenber g-Marquardt method, Optimization 52 (6) (2003) 739–756 (2003)
2003
-
[23]
Fan, Y.-x
J.-y. Fan, Y.-x. Yuan, On the quadratic convergence of the lev enberg- marquardt method without nonsingularity assumption, Computing 7 4 (1) (2005) 23–39 (2005)
2005
-
[24]
Yamashita, M
N. Yamashita, M. Fukushima, On the rate of convergence of th e Levenberg- Marquardt method, in: Topics in Numerical Analysis, Springer Vienna , 2001, pp. 239–249 (2001)
2001
-
[25]
Jaggi, Revisiting Frank-Wolfe: Projection-free sparse con vex optimiza- tion, in: Proceedings of the 30th International Conference on Ma chine Learning (ICML-13), Vol
M. Jaggi, Revisiting Frank-Wolfe: Projection-free sparse con vex optimiza- tion, in: Proceedings of the 30th International Conference on Ma chine Learning (ICML-13), Vol. 28, 2013, pp. 427–435 (2013)
2013
-
[26]
Grippo, F
L. Grippo, F. Lampariello, S. Lucidi, A nonmonotone line search te chnique for Newton’s method, SIAM J. Numer. Anal. 23 (4) (1986) 707–716 (1986)
1986
-
[27]
Frank, P
M. Frank, P. Wolfe, An algorithm for quadratic programming, Na val Res. Logist. 3 (1-2) (1956) 95–110 (1956)
1956
-
[28]
N. I. M. Gould, D. Orban, P. L. Toint, Cuter, a constrained and uncon- strained testing environment, revisited, ACM Trans. Math. Softw. 29 (2003) 373–394 (2003)
2003
-
[29]
W. Hock, K. Schittkowski, Test examples for nonlinear program ming codes, Journal of Optimization Theory and Applications 30 (1) (1980) 127– 129 (1980)
1980
-
[30]
Morini, M
B. Morini, M. Porcelli, TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities, Comput. Optim. Appl. 51 (2012 ) 27–49 (2012)
2012
-
[31]
D. S. Gon¸ calves, M. A. Gomes-Ruggiero, C. Lavor, A project ed gradient method for optimization over density matrices, Optim. Methods Sof tw. 31 (2016) 328–341 (2016)
2016
-
[32]
Allen-Zhu, E
Z. Allen-Zhu, E. Hazan, W. Hu, Y. Li, Linear Convergence of a Fr ank- Wolfe type algorithm over trace-norm balls, in: I. Guyon, U. V. Luxb urg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (Ed s.), Advances in Neural Information Processing Systems 30, Curran A...
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.