On the regularization property of Levenberg-Marquardt method with Singular Scaling for nonlinear inverse problems

Douglas S. Gon\c{c}alves; Everton Boos; Fermin S. V. Baz\'an; Rafaela Filippozzi

REVIEW 4 major objections 4 minor 8 references

On the regularization property of Levenberg-Marquardt method with Singular Scaling for nonlinear inverse problems

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A singular-scaling Levenberg–Marquardt method is proved to regularize noisy inverse problems.

desk verdict The main regularization result is not established: Lemma 1(b) is false, and the residual formula in (24) is wrong. read the letter →

arxiv 2506.00190 v1 pith:N6ARSE5V submitted 2025-05-30 math.NA cs.NAmath.OC

Rafaela Filippozzi , Everton Boos , Douglas S. Gonçalves , Fermin S. V. Bazán This is my paper

classification math.NAcs.NAmath.OC MSC 65J2065F2265K10

keywords inverseproblemsLevenberg-Marquardtsingularscalingregularizationnonlinearleastsquaresdiscrepancyprincipletangentconeconditiongeneralizedvaluedecomposition

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that a Levenberg–Marquardt method with a singular scaling matrix (LMMSS) is a regularization method for nonlinear inverse problems with noisy data: if the unperturbed problem has an exact solution, the iterates stopped by the discrepancy principle converge to such a solution as the noise level tends to zero. The regularization comes from two places: the possibly singular matrix $L$ in $J_k^TJ_k+\lambda_k L^TL$ shapes the geometry of the iteration, and the parameter $\lambda_k$ is chosen implicitly through a $q$-condition. For the proof, the authors introduce a new tangent cone condition adapted to the $L$-norm, which controls the Taylor remainder of the forward map. This is the first regularization analysis for the singular-scaling variant and extends known regularizing properties of classical Levenberg–Marquardt schemes.

What carries the argument

The load-bearing machinery is the generalized singular value decomposition of the pair $(J_k,L)$, which diagonalizes $J_k^TJ_k+\lambda_kL^TL$ and gives an explicit formula for the direction $d_k$. The paper's new condition, TCC-L (Assumption 2), bounds the Taylor remainder $\|J(x)(\tilde{x}-x)-F(\tilde{x})+F(x)\|$ by $c\|\tilde{x}-x\|_L\|F(\tilde{x})-F(x)\|$, replacing the Euclidean norm in the classical tangent cone condition by the seminorm $\|v\|_L=\|Lv\|_2$. Lemma 2 is the engine: it yields the descent inequalities $\|x_k-x_*\|_L^2-\|x_{k+1}-x_*\|_L^2\ge\|x_{k+1}-x_k\|_L^2$ and a lower bound in terms of the squared residual divided by $\lambda_k$, which together give monotone decrease and the finite stopping bound used in Theorem 4.

What would settle it

Take the heat-conduction parameter estimation problem that motivated the singular scaling, fix a singular $L$, and run LMMSS with the discrepancy principle for a sequence of noise levels $\delta_n\to0$; if the residual before stopping never falls to $O(\delta_n)$, or if the stopped iterates do not approach a common solution, then one of the assumptions—most plausibly TCC-L—is violated. Alternatively, construct an $F$ and a singular $L$ for which the classical tangent cone condition holds but the $L$-norm version (17) fails, and exhibit a divergent stopped iterate, which would show the condition cannot be dropped.

Watch

Extended reading notes

Core claim

The central claim is Theorem 4: under Assumptions 1–3 with noisy data $y^\delta$ satisfying $\|y-y^\delta\|\le\delta$, every LMMSS iterate lies in the $L$-ball $B_L(x_0,\rho)$, the discrepancy principle (5) stops at a finite index $k^*(\delta)$, and the stopped iterate $x^\delta_{k^*(\delta)}$ converges to a solution of the exact problem $F(x)=y$ as $\delta\to0$. The proof splits into exact-data convergence (Theorems 2 and 3) and a stability argument that transfers it to noisy data. In the exact case the $L$-norm distance to a chosen solution $x_*$ decreases monotonically, the residual norms $\|F(x_k)-y\|$ tend to zero, and the sequence converges to a solution of (2). The noisy case then uses continuity of the implicitly defined regularization parameter with respect to the data (Proposition 1) and a finite-stopping bound of the form $k^*\tau^2\delta^2\le C\|x_0-x_*\|_L$ to pass to the limit $\delta\to0$.

Load-bearing premise

The whole argument rests on the new tangent-cone condition (Assumption 2), which requires the Taylor remainder of $F$ to be controlled by the $L$-distance between points times the residual; if this condition fails, the descent lemma and the finite stopping of the discrepancy principle no longer follow.

Editorial extensions

If this is right

With exact data, the LMMSS iterates converge to a solution of $F(x)=y$ for initial guesses satisfying the stated $L$-norm closeness condition, without requiring the Jacobian to be nonsingular at the solution.
With noisy data, the discrepancy principle stops after finitely many iterations for each fixed $\delta>0$, and the final iterate is stable: it converges to an exact solution as $\delta\to0$.
The analysis covers the zero-residual case only, so the method is guaranteed to regularize problems where the noise-free system is exactly solvable.
Since TCC-L reduces to the classical tangent cone condition when $L$ is nonsingular, the result extends classical regularizing Levenberg–Marquardt analyses to include semi-norm regularizers.
The proof gives an explicit $O(\delta^{-2})$ bound on the stopping index $k^*(\delta)$, so the iteration count required for stability is quantitative, not merely finite.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves open whether TCC-L holds for the heat-conduction application that motivated the singular scaling; a natural next step is to verify condition (17) numerically on that problem or to find a weaker sufficient condition.
The zero-residual restriction suggests a neighboring theorem for nonzero-residual least squares, where convergence would be to a stationary point rather than an exact solution; the GSVD machinery used here is likely adaptable to that setting.
Because the regularization effect is partly induced by the choice of $L$, the work points toward data-driven selection of the scaling matrix as a design parameter that trades bias against stability in the recovered parameter field.
A practical extension is an adaptive strategy for the parameter $q$ in the implicit rule (20), since the paper's bounds depend on $q$ through factors such as $(1-q)q$.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the paper aims to extend Hanke's regularizing Levenberg-Marquardt analysis to the singular-scaling version (LMMSS), and the TCC-L condition is a reasonable adaptation. But the central proof collapses at Lemma 1(b). The residual after the LM step is not given by (24): in the GSVD coordinates the n−p block of I − Σ²(Σ²+λM²)⁻¹ is zero, not identity, and the orthogonal projector onto R(J)⊥ is missing. Concretely, take F(x)=x on R², L=[1 0], x0=e2, y=0, q=1/2. Then J=I, L has rank 1, and all assumptions hold; but for every λ>0, r+Jd = 0, so (20) can never be satisfied, while (21) holds trivially. So the λ_k defined by (20) need not exist, and Theorem 4, which depends on it, is unsupported.

To be fair, the paper does useful work. The TCC-L condition and the L-norm monotonicity machinery are a genuine adaptation of Hanke's framework to singular scaling, and the authors are clear about the zero-residual assumption and the limitations. The literature is engaged honestly.

Other soft spots: Assumption 3's noisy-data closeness bound appears misstated—the proof needs a linear condition like ||x0−x*||_L < (qτ−1)/(c(1+τ)), not the square root in (19). Lemma 2's algebra also has missing squares and sign slips; the displayed inequality (32) is not derived correctly.

Recommendation: this deserves a serious referee—the idea is plausible and the community would care if it worked. But the current manuscript is not acceptable as is. The authors need to fix the residual formula (which should include the projector term and zero on the bottom-right block), restate Assumption 3, and either replace (20) with the inequality version in Remark 1 or prove existence under a stronger condition. As written, the main theorem is not established.

Referee Report

4 major / 4 minor

Summary. The manuscript proposes a regularization framework for the Levenberg-Marquardt method with singular scaling (LMMSS) applied to zero-residual nonlinear inverse problems with noisy data. The iteration is defined by (6)-(7), the regularization parameter is chosen through the scalar q-condition (20), and the discrepancy principle (5) supplies the stopping index. Under Assumptions 1-3, which include a new Tangent Cone Condition adapted to the L-norm, the paper claims monotone decrease of the L-norm distance to a solution, convergence in the exact-data case, and a regularization property for noisy data (Theorems 2-4). The proof machinery is based on the GSVD of the pair (J_k,L) and on an auxiliary residual formula in Lemma 1.

Significance. If the claims were correct, this would be the first regularization result for singularly scaled Levenberg-Marquardt methods, extending both Hanke's classical analysis and the authors' earlier local-convergence study. The paper is genuinely transparent: the assumptions are stated explicitly, the proofs are detailed, and there are no fitted parameters or circular conclusions. However, the central auxiliary Lemma 1 is incorrect, and the counterexample below shows that the q-condition can have no solution even when all stated assumptions hold. This directly undermines the definition of lambda_k and therefore Theorem 4. The new TCC-L condition is also not verified for any concrete problem, including the motivating heat-conduction application; the final remarks concede this. The manuscript is promising but not in a publishable state.

major comments (4)

[Section 3, Lemma 1, Eqs. (23)-(24)] The GSVD-based residual formula is wrong. Because U in R^{m x n} has orthonormal columns but U U^T is not the identity on R^m, one cannot replace r - U diag(Sigma^2(Sigma^2+lambda M^2)^{-1}, I) U^T r by U diag(Psi, I_{n-p}) U^T r. The correct expression is r(lambda) = (I - U U^T)r + U diag(lambda M^2(Sigma^2+lambda M^2)^{-1}, 0) U^T r, so the n-p block in (24) must be zero, not the identity. Equation (24) therefore spuriously includes the sum of r_i^2 for i=p+1,...,n. Consequently the claim in Lemma 1(b) that (21) guarantees a unique lambda_k^q solving (20) is not established and is in fact false. Counterexample: take F(x)=x on R^2, L=[1,0], y=0, x0=e2, and q=1/2. Assumption 1 holds with gamma=1; Assumption 2 holds for any c>0 because F is affine; and ||x0-x*||_L=0 for the solution x*=0, so Assumption 3 holds. For every lambda>0, the normal equation (I+lambda L^T L)d = -e2 gives d=-e2, hence F(x0)+Jd=0. Equation (20) would require 0=q||F(x0)-y||=1/2, so no lambda_k exists, while (21) holds with any theta>1. Thus the method as defined by (20)-(22) is vacuous in a valid instance, and Theorem 4 cannot hold as stated.
[Lemma 2, proof of inequality (28)] The displayed derivation ends with the term -||x_{k+1}-x_k||_L plus additional terms, and then the text jumps to 'therefore' inequality (32), which claims ||x_k-x*||_L^2 - ||x_{k+1}-x*||_L^2 >= ||x_{k+1}-x_k||_{L^T L}^2. The square on the norm is missing in the displayed chain, and -||d||_L does not dominate +||d||_L^2. The intended use of equation (16) may repair this, but as written the proof of (28) is incomplete. Because (28) is used later in Theorem 3, this is not merely cosmetic.
[Theorem 3, proof around Eq. (38)] In bounding gamma||x_k - bar{x}||^2, the proof applies Lemma 2 with the limit point bar{x} in place of the selected solution x*, although inequality (21) is only established for the particular x* chosen in Assumption 3. It is not shown that (21) holds for bar{x}. Moreover, the term ||x_{bar{k}} - bar{x}||_L^2 - sum_{l=bar{k}}^k ||x_{l+1} - bar{x}||_L^2 does not control ||L(x_k - bar{x})||^2; the latter equals ||x_k - bar{x}||_L^2, which is not what the telescoping sum gives. The subsequential convergence of x_k to bar{x} could be used directly, but the present argument is not valid.
[Theorem 2, residual convergence] The conclusion that ||F_k - y|| tends to zero is derived from the decrease estimate (30), which contains the factor 1/zeta_{p,k}^2. No uniform upper bound on zeta_{p,k} is stated or proved; the sentence 'since F' is uniformly bounded in B_{L^T L}(x*,rho)' introduces an assumption that does not appear in Assumptions 1-3. Without such a bound, the telescoping sum argument does not yield convergence of the residuals. This is a further load-bearing gap in the exact-data convergence proof.

minor comments (4)

[Introduction] The method is called LMMSS in the abstract and in most of the paper, but the introduction also uses the spelling 'LMSS'; please make the notation uniform.
[Final remarks] The text states that a theoretical investigation of the TCC-L assumption in concrete applications will be pursued in future work. This is an explicit admission that no example is provided for the new condition, including for the heat-conduction problem cited as motivation. At minimum, the paper should state clearly that the TCC-L condition is unverified for all known applications.
[Remark 1 and Section 3] Remark 1 notes that equation (20) can be replaced by the inequality (25), but the main theorems and Lemma 2 rely on the equality version. Since Lemma 1 shows that equality may fail, the manuscript should either prove the same bounds for the inequality version and use it throughout, or add a condition that guarantees solvability of (20).
[Theorem 4, Case 1] The sentence 'note that in the exact data for this case x_{tilde{k}+1}=x_{tilde{k}}' is not justified by any previously established result; it should either be proved or removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the convergence claim is derived from explicit assumptions and an external stopping rule.

full rationale

The paper's central claim (Theorem 4) is a convergence theorem for the LMMSS iteration (6)-(7) under explicit assumptions (Assumptions 1-3). No parameter is fitted to data and the conclusion is not used as an input: TCC-L (Assumption 2, eq. (17)) is stated independently of the target convergence, and the discrepancy principle (5) is an external stopping criterion. The induction in Theorems 2 and 4 uses Assumption 2 and the metric decrease Lemma 2 to maintain condition (21), which is a genuine proof step rather than a restatement of the desired result. The only self-citation to [3] motivates the iteration and notes a numerical application; it is not invoked to prove regularization, and no uniqueness theorem from the authors' prior work is used. The GSVD (Theorem 1) is cited from a standard textbook and used as a computational tool, not as a circular premise. I found no fitted-input-called-prediction, no ansatz smuggled by citation, and no definition of the quantity to be proved in terms of the conclusion. A potential algebraic concern in Lemma 1 (the n-p block in eq. (24)) would affect correctness, not circularity, and therefore does not change this finding.

Assumptions & free parameters 3 free parameters · 8 assumptions · 0 invented entities

The proof introduces no new physical or mathematical entities. It relies on a strengthened tangent cone condition (TCC-L) and the completeness condition already used in prior LM analysis. The algorithmic parameters q and tau are user-chosen, not fitted. The paper contains no numerical experiments, so no data-fitting parameters appear.

free parameters (3)

q
User-chosen parameter in the q-condition (eq. 20). The theorem holds for any q in (0,1) satisfying the assumptions; no value is fitted to data.
tau
User-chosen parameter in the discrepancy principle (eq. 5) with tau > 1/q. Not fitted; the claim holds for any such tau.
Scaling matrix L
Possibly singular matrix L in R^{p x n}, user-chosen, with rank(L)=p and Assumption 1. Not fitted to data; the analysis is for an arbitrary L satisfying the assumptions.

assumptions (8)

domain assumption F : R^n -> R^m is twice continuously differentiable.
Used throughout; the analysis of the LM iteration needs smoothness of F. Stated in Section 1 after eq. (1).
domain assumption The discretized problem (1) is solvable in BL(x0,2rho) and the TCC-L condition holds (Assumption 2).
Assumption 2, eq. (17): a strengthened tangent cone condition linking the Taylor remainder to the L-seminorm and residual. This is the key nonlinearity control that enables Lemma 2.
domain assumption The initial guess satisfies Assumption 3: ||x0-x*||_L < min{sqrt(q/c), rho} for delta=0 and < min{sqrt((tau-1)/(c(1+tau))), rho} for delta>0.
Assumption 3, eqs. (18)-(19). Note the proof of Theorem 4 actually requires the stronger linear bound ||x0-x*||_L < (q*tau-1)/(c(1+tau)).
domain assumption There exists x* in X* minimizing the L-distance to x0.
Stated after the definition of BL. Guarantees a well-defined target for convergence.
domain assumption Assumption 1 (completeness): N(J(x)) intersect N(L) = {0} for all x in Omega.
Eq. (8). Ensures positive definiteness of J^T J + lambda L^T L and is used in Theorems 3 and 4 to convert L-norm control into Euclidean control.
domain assumption The Jacobian J is uniformly bounded on the relevant ball BL(x*, rho).
Invoked in the proof of Theorem 2 ('since F' is uniformly bounded in B_{L^T L}(x*, rho)') but not stated among the assumptions. An unflagged import.
standard math GSVD as in Theorem 1 (Hansen 1998).
Used to diagonalize the pair (J_k,L) and derive the characterization of the LM direction and the bound on lambda_k in Lemma 1.
standard math Implicit Function Theorem.
Used in Proposition 1 to establish continuity of lambda_k^q with respect to data.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the regularization property of Levenberg-Marquardt method with Singular Scaling for nonlinear inverse problems." pith.science (2026). https://pith.science/paper/N6ARSE5V

@misc{pith2026250600190,
  author       = {Pith},
  title        = {Pith review of: On the regularization property of Levenberg-Marquardt method with Singular Scaling for nonlinear inverse problems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N6ARSE5V}},
  note         = {Machine review of arXiv:2506.00190}
}

read the original abstract

Recently, in Applied Mathematics and Computation 474 (2024) 128688, a Levenberg-Marquardt method (LMM) with Singular Scaling was analyzed and successfully applied in parameter estimation problems in heat conduction where the use of a particular singular scaling matrix (semi-norm regularizer) provided approximate solutions of better quality than those of the classic LMM. Here we propose a regularization framework for the Levenberg-Marquardt method with Singular Scaling (LMMSS) applied to nonlinear inverse problems with noisy data. Assuming that the noise-free problem admits exact solutions (zero-residual case), we consider the LMMSS iteration where the regularization effect is induced by the choice of a possibly singular scaling matrix and an implicit control of the regularization parameter. The discrepancy principle is used to define a stopping index that ensures stability of the computed solutions with respect to data perturbations. Under a new Tangent Cone Condition, we prove that the iterates obtained with noisy data converge to a solution of the unperturbed problem as the noise level tends to zero. This work represents a first step toward the analysis of regularizing properties of the LMMSS method and extends previous results in the literature on regularizing LM-type methods.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

8 extracted references · 8 canonical work pages

[1]

Behling, D

R. Behling, D. S. Gon¸ calves, and S. A. Santos. Local convergence analysis of the levenberg– marquardt framework for nonzero-residue nonlinear least-squares problems under an error bound condition.Journal of Optimization Theory and Applications, 183(3):1099–1122, 2019

work page 2019
[2]

Bellavia, B

S. Bellavia, B. Morini, and E. Riccietti. On an adaptive regularization for ill-posed nonlinear systems and its trust-region implementation.Computational Optimization and Applications, 64:1–30, 2016

work page 2016
[3]

E. Boos, D. S. Gon¸ calves, and F. S. V. Baz´ an. Levenberg-marquardt method with singular scaling and applications.Applied Mathematics and Computation, 474(128688), 2024

work page 2024
[4]

H. W. Engl, M. Hanke, and A. Neubauer.Regularization of Inverse Problems. Kluwer Academic Publishers, 1996

work page 1996
[5]

M. Hanke. A regularizing levenberg-marquardt scheme, with applications to inverse ground- water filtration problems.Inverse problems, 13(1):79, 1997

work page 1997
[6]

P. C. Hansen.Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia, 1998

work page 1998
[7]

Kaltenbacher, A

B. Kaltenbacher, A. Neubauer, and O. Scherzer.Iterative regularization methods for nonlin- ear ill-posed problems. Walter de Gruyter, 2008. 13

work page 2008
[8]

V. A. Morozov.Regularization Methods for Solving Incorrectly Posed Problems. Springer, 1984. 14

work page 1984

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Behling, D

R. Behling, D. S. Gon¸ calves, and S. A. Santos. Local convergence analysis of the levenberg– marquardt framework for nonzero-residue nonlinear least-squares problems under an error bound condition.Journal of Optimization Theory and Applications, 183(3):1099–1122, 2019

work page 2019

[2] [2]

Bellavia, B

S. Bellavia, B. Morini, and E. Riccietti. On an adaptive regularization for ill-posed nonlinear systems and its trust-region implementation.Computational Optimization and Applications, 64:1–30, 2016

work page 2016

[3] [3]

E. Boos, D. S. Gon¸ calves, and F. S. V. Baz´ an. Levenberg-marquardt method with singular scaling and applications.Applied Mathematics and Computation, 474(128688), 2024

work page 2024

[4] [4]

H. W. Engl, M. Hanke, and A. Neubauer.Regularization of Inverse Problems. Kluwer Academic Publishers, 1996

work page 1996

[5] [5]

M. Hanke. A regularizing levenberg-marquardt scheme, with applications to inverse ground- water filtration problems.Inverse problems, 13(1):79, 1997

work page 1997

[6] [6]

P. C. Hansen.Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia, 1998

work page 1998

[7] [7]

Kaltenbacher, A

B. Kaltenbacher, A. Neubauer, and O. Scherzer.Iterative regularization methods for nonlin- ear ill-posed problems. Walter de Gruyter, 2008. 13

work page 2008

[8] [8]

V. A. Morozov.Regularization Methods for Solving Incorrectly Posed Problems. Springer, 1984. 14

work page 1984