Pith. sign in

REVIEW 1 major objections 5 minor 69 references

An augmented Lagrangian algorithm for constrained nonlinear least-squares

T0 review · 1 major / 5 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An augmented Lagrangian method solves constrained nonlinear least-squares by keeping linear constraints explicit and approximating the penalty Hessian with a structured SR1 update.

desk verdict Solid specialized AL solver for constrained NLS: careful combination of known pieces, honest numerics, one known gap between theorem and code. read the letter →

arxiv 2607.11239 v1 pith:LI4HDULN submitted 2026-07-13 math.OC cs.NAmath.NA

Pierre Borie , Fabian Bastin , Stéphane Dellacherie This is my paper · ORCID
classification math.OCcs.NAmath.NA MSC 90C3065K0590C55
keywords constrainednonlinearleast-squaresaugmentedLagrangianstructuredquasi-NewtonSR1updategradientprojectiontrustregionhybridHessianapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Many fitting and parameter-estimation problems are nonlinear least-squares with a mixture of nonlinear constraints and linear equalities or bounds. This paper builds a solver that folds only the nonlinear constraints into an augmented Lagrangian while leaving the linear set untouched, so each outer step reduces to a linearly constrained subproblem that can be attacked by gradient projection and projected conjugate gradients. The Hessian of that subproblem is never formed exactly; instead a Gauss–Newton block is corrected by a structured SR1 update that respects the sum-of-squares structure of both residuals and constraints. Under standard smoothness and rank assumptions the outer sequence is shown to produce first-order critical points. Numerical tests on seventy-nine problems show that a hybrid version of the structured update is competitive in wall-clock time with a leading interior-point solver while remaining matrix-free.

What carries the argument

The structured secant equation for the second-order remainder of the augmented Lagrangian, Bk+1 sk = (Jk+1 − Jk)⊤ rk+1 + (Ck+1 − Ck)⊤ λ̄k+1, which is realized by an SR1 update and optionally switched with pure Gauss–Newton via a residual-ratio test.

What would settle it

Run the published Julia implementation on a problem whose active-bound multipliers change sign near a limit point and check whether the reduced-gradient residual vanishes while the true projected-gradient residual stays bounded away from zero.

Watch

Extended reading notes

Core claim

The paper establishes that an augmented-Lagrangian outer loop combined with trust-region gradient-projection inner iterations and a hybrid structured-SR1 Hessian approximation converges globally to first-order KKT points of a nonlinear least-squares problem subject to mixed nonlinear and linear constraints, and that the resulting algorithm is practically competitive on a standard test set.

Load-bearing premise

The code stops when the reduced gradient is small, yet the global-convergence theorem is proved for the true projected gradient; the two coincide only when active-bound multipliers keep the correct sign at every outer iterate.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper proposes TRAULLS, an augmented Lagrangian algorithm for nonlinear least-squares problems with mixed nonlinear and linear (equality/bound) constraints. Nonlinear constraints are penalized via the AL function (2.1), while linear constraints are retained and handled by a trust-region gradient-projection inner solver (Cauchy point along the projected path plus projected CG minor iterations). The AL Hessian is approximated by a Gauss–Newton term plus a structured SR1 correction of the second-order residual/constraint contribution (secant equation (2.26), update (2.27)), with an optional hybrid switch (2.29). Global convergence of the inner process (Theorem 5) and of the outer AL iterates (Theorem 9 / Lemma 8) is established under standard assumptions (twice continuous differentiability, full-rank active linear constraints, compact level sets, controlled Hessian growth, and rank of C(x*)N* ≥ nc). Numerical experiments on 79 instances compare Hessian variants and show the hybrid-SR1 version competitive with IPOPT in CPU time and superior to Percival.

Significance. The work cleanly combines two mature lines—AL methods with gradient-projection subproblem solves and structured quasi-Newton updates for least-squares—into a solver that treats general nonlinear equalities/inequalities together with linear equalities and bounds. The structured secant equation for the AL second-order term and the polyhedral adaptation of the Cauchy-point path are concrete technical contributions. Global-convergence proofs follow the Conn–Gould–Toint / Conn–Gould–Sartenaer–Toint templates after careful reformulation for polyhedra, and a public Julia implementation (TRAULLS) with reproducible benchmarks is provided. The hybrid-SR1 variant is shown to be competitive with a state-of-the-art interior-point solver on a standard test set while exploiting first-order structure only. These strengths make the paper a useful addition to the constrained NLS literature.

major comments (1)
  1. End of §3: the proved outer convergence (Theorem 9 / Lemma 8) is stated for the projected-gradient residual (2.6), while the implementation and all numerical claims use the reduced-gradient test (2.10). The authors correctly note that the two coincide only when bound multipliers have the correct sign at every outer iterate; without that condition only the tangential component is controlled and a limit point need not be critical. This is the sole material gap between the theorems and the code. A short clarification—either an explicit additional assumption that the sign condition holds along the generated sequence, a practical safeguard that restores the projected residual when signs are wrong, or a statement that the numerical claims are conditional on the observed sign pattern—would close the gap without altering the rest of the argument.
minor comments (5)
  1. §2.5 / Figure 1: the hierarchy of outer / inner / minor iterations is clear, but a one-sentence reminder that the outer index K and inner index k are independent (and that B0 is reset to zero at each outer iteration) would help readers who jump between sections.
  2. §4.1: the performance profiles are informative; adding the absolute number of failures (already mentioned in the text) to the figure captions would make the profiles self-contained.
  3. Appendix A, Algorithm 4: the termination test “|A(ti)| = n−m” is correct under Assumption 4, but a brief remark that the trust-region bounds guarantee eventual full activation would remove any ambiguity for readers unfamiliar with the polyhedral path.
  4. Several minor typos appear (e.g., “nonlinar”, “Compuations”, “Transcations”, “Reasearch”); a careful proof-reading pass would clean them.
  5. References [37] and [26] both concern NL2SOL-related work; ensuring the year and author list of [37] match the published version would avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: global KKT claims follow from stated assumptions and standard AL/trust-region theory; numerical claims are empirical benchmarks, not forced predictions.

full rationale

The paper's central theoretical claim (Theorem 9 / Lemma 8) is that, under Assumptions 1–9, every limit point of the outer iterates of Algorithm 1 is a first-order KKT point of the constrained NLS problem. The derivation is self-contained: it adapts the projected-gradient path analysis of Conn–Gould–Toint (SBMIN / LANCELOT) to polyhedral sets (Lemmas 1–3, Theorem 4–5) and the AL multiplier/feasibility arguments of Conn–Gould–Sartenaer–Toint (Lemmas 6–8, Theorem 9). No quantity that is later called a “prediction” is fitted from the target KKT residual; the structured SR1 update (2.26)–(2.27) and hybrid switch (2.29) are algorithmic choices whose effect is measured empirically, not asserted as a first-principles derivation. Hyperparameters (μ0, τ, κhyb, trust-region constants) are fixed once for the 79-instance suite and reported; they do not appear as free constants inside the theorems. Self-citations are to standard background (Conn et al., Dennis–Gay–Walsh, Biggs, etc.) that is externally published and independently checkable; none is a uniqueness theorem that forbids alternatives and thereby forces the present result. The only material gap between theory and code—the use of the reduced-gradient test (2.10) instead of the projected-gradient residual (2.6)—is explicitly flagged by the authors at the end of §3 and is a correctness/implementation caveat, not a circular reduction. Consequently the derivation chain does not collapse into its inputs by construction.

Assumptions & free parameters 5 free parameters · 7 assumptions · 2 invented entities

The central mathematical claim rests on standard smoothness/rank/compactness assumptions for AL and trust-region theory, plus algorithm hyperparameters fixed for experiments. No new physical entities. The structured secant equation and hybrid switch are design choices, not free parameters fitted to force the theorems.

free parameters (5)
  • Initial penalty μ0 and increase factor τ = μ0=10, τ=100
    Set to μ0=10, τ=100 for all tests; control outer feasibility enforcement and are not derived from theory.
  • Tolerance schedule constants (ω, η, κω, κη, βω, βη) = as listed in §4
    Drive ωK, ηK → 0; values κω=βω=η=ω=1, κη=0.1, βη=0.9 chosen for the benchmark.
  • Hybrid switch threshold κhyb = 0.1
    Decides when to use BGN+B vs pure Gauss–Newton; set to 0.1 following prior hybrid NLS practice.
  • Trust-region constants (δ0, α1, α2, η1, η2, γ1) = δ0=0.1, α1=0.25, α2=2.5, η1=0.25, η2=0.75, γ1=0.0625
    Control acceptance and radius updates; fixed once for all problems.
  • Stopping tolerances ω*, η* = 1e-5 / 1e-6
    Outer success criteria for reported solves; ω*=1e-5, η*=1e-6.
assumptions (7)
  • domain assumption Residuals r and constraints c are twice continuously differentiable (Assumption 1).
    Needed for gradients/Hessians of Φ and for mean-value arguments in Lemma 8.
  • domain assumption Equality matrix A has full row rank; active-bound matrix à has full row rank at every x in Ω (Assumptions 2, 4).
    Guarantees well-defined projections onto T(x) and Cholesky of ÃÃᵀ.
  • domain assumption Linear feasible set Ω is nonempty (Assumption 3).
    Subproblems must be feasible.
  • domain assumption Level set of φ intersect Ω is nonempty and compact; Hessian approximations satisfy divergent 1/bk series and bk(φk+1−φk)→0 (Assumptions 5–7).
    Standard trust-region global-convergence hypotheses for the inner loop (Theorem 5).
  • domain assumption Outer iterates stay in a closed bounded domain; at every limit point, rank(C(x*)N*) ≥ nc (Assumptions 8–9).
    Ensures existence of limit points and well-defined least-squares multipliers for outer KKT analysis.
  • standard math Steps achieve a fixed fraction of Cauchy decrease (κfcd) and standard trust-region acceptance ratios.
    Classic sufficient-decrease framework from Conn–Gould–Toint.
  • ad hoc to paper When reduced gradient is used as the practical criticality measure, bound multipliers have correct sign so that reduced and projected residuals coincide.
    Stated only at the end of §3 as the bridge from Theorem 9 to the implementation; not enforced algorithmically.
invented entities (2)
  • TRAULLS algorithm (AL + polyhedral gradient projection + structured SR1 of AL Hessian + hybrid switch) independent evidence
    purpose: Concrete solver package for constrained NLS with the claimed convergence and empirical behavior.
    The named combination and Julia implementation are the paper’s product; components are assembled from prior literature.
  • Structured secant equation for AL second-order term (Eq. 2.26) with SR1 update (2.27) independent evidence
    purpose: Approximate Sk without second derivatives while preserving a standard secant equation for Hk.
    Adaptation of Biggs / NL2SOL / Li et al. ideas to the AL residual-and-multiplier sum; not a new physical object.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An augmented Lagrangian algorithm for constrained nonlinear least-squares." pith.science (2026). https://pith.science/paper/LI4HDULN

@misc{pith2026260711239,
  author       = {Pith},
  title        = {Pith review of: An augmented Lagrangian algorithm for constrained nonlinear least-squares},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LI4HDULN}},
  note         = {Machine review of arXiv:2607.11239}
}
read the original abstract

We present an algorithm for solving nonlinear least-squares problems subject to a mix of nonlinear and linear constraints. The nonlinear constraints are handled by reformulating the objective as the augmented Lagrangian function while linear constraints are handled directly. Each iteration consists of approximately solving a linearly constrained problem by means of a gradient projection technique. Our approach also involves a structured approximation of the augmented Lagrangian Hessian. We show global convergence of the method and assess the performance through numerical experiments.

Figures

Figures reproduced from arXiv: 2607.11239 by the authors.

Figure 1
Figure 1. Hierarchy of the three iteration levels in the AL algorithm. The outer loop updates [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Performance profiles comparing five different Hessian approximation strategies with [PITH_FULL_IMAGE:figures/full_fig_p028_2.png] view at source ↗
Figure 3
Figure 3. Performance profiles comparing the Gauss–Newton approximation against hybrid [PITH_FULL_IMAGE:figures/full_fig_p029_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance profiles comparing TRAULLS (Hybrid SR1), IPOPT, and Percival on [PITH_FULL_IMAGE:figures/full_fig_p030_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 32 canonical work pages

  1. [1]

    Al-Baali and R

    M. Al-Baali and R. Fletcher. Variational methods for non-linear least-squares.The Journal of the Operational Research Society, 36(5):405–421, 1985. doi: 10.1057/jors.1985.68

    work page doi:10.1057/jors.1985.68 1985

  2. [2]

    Andreani, E.G

    R. Andreani, E.G. Birgin, J.M. Martinez, and M.L. Schuverdt. On augmented Lagrangian methods with general lower-level constraints.SIAM Journal on Optimization, 18(4):1286– 1309, 2008. doi: 10.1137/060654797

    doi:10.1137/060654797 2008

  3. [3]

    Arreckx, A

    S. Arreckx, A. Lambe, J.R.R.A. Martins, and D. Orban. A matrix-free augmented La- grangian algorithm with application to large-scale structural design optimization.Opti- mization and Engineering, 17:359–384, 2016. doi: 10.1007/s11081-015-9287-9

    work page doi:10.1007/s11081-015-9287-9 2016

  4. [4]

    Bartholomew-Biggs

    M.C. Bartholomew-Biggs. The estimation of the hessian matrix in nonlinear least-squares problems with non-zero residuals.Mathematical Programming, 12:67–80, 1977. doi: 10. 1007/BF01593770

    1977

  5. [5]

    Bellavia, M

    S. Bellavia, M. Macconi, and B. Morini. STRSCNE: A scaled trust-region solver for constrained nonlinear equations.Computational Optimization and Applications, 28:31–50, 2004

    2004

  6. [6]

    Bellavia, J

    S. Bellavia, J. Gondzio, and B. Morini. Computational experience with numerical methods for nonnegative least-squares problems.Numerical Linear Algebra with Applications, 18 (3):363–385, 2011

    2011

  7. [7]

    Bellavia, S

    S. Bellavia, S. Gratton, and E. Riccietti. A Levenberg-Marquardt method for large nonlin- ear least-squares problems with dynamic accuracy in functions and gradients.Numerische Mathematik, 140:791–825, 2018. doi: 10.1007/s00211-018-0977-z

    doi:10.1007/s00211-018-0977-z 2018

  8. [8]

    A variable dimension sketching strategy for nonlinear least-squares.SIAM Journal on Scientific Computing, 48(3):A1206– A1234, 2026

    Stefania Bellavia, Greta Malaspina, and Benedetta Morini. A variable dimension sketching strategy for nonlinear least-squares.SIAM Journal on Scientific Computing, 48(3):A1206– A1234, 2026. doi: 10.1137/25M175980X

    work page doi:10.1137/25m175980x 2026

Show all 69 references
  1. [9]

    Bergou, Y

    E.H. Bergou, Y. Diouane, V. Kungurtsev, and C.W. Royer. A nonmonotone matrix-free algorithm for nonlinear equality-constrained least-squares problems.SIAM Journal on Scientific Computing, 43(5):S743–S766, 2021. doi: 10.1137/20M1349138

    2021 doi

  2. [10]

    J.T. Betts. Solving the nonlinar least square problem: Application of a general method.Journal of Optimization Theory and Applications, 18:469–483, 1976. doi: 10.1007/BF00932656. 31

    1976 doi

  3. [11]

    Bezanson, A

    J. Bezanson, A. Edelman, S. Karpinski, and V.B. Shah. Julia: A fresh approach to numerical computing.SIAM Review, 59(1):65–98, 2017. doi: 10.1137/141000671

    2017 doi

  4. [12]

    Biegler, J

    L.T. Biegler, J. Nocedal, C. Schmid, and D. Ternet. Numerical experience with a reduced hessian method for large scale constrained optimization.Computational Optimization and Applications, 15:45–67, 2000. doi: 10.1023/A:1008723031056

    doi

  5. [13]

    SIAM, Philadelphia, PA, USA, 2nd edition, 2024

    ˚Ake Bj¨ orck.Numerical Methods for Least Squares Problems. SIAM, Philadelphia, PA, USA, 2nd edition, 2024. doi: 10.1137/1.9781611977950

    2024 doi

  6. [14]

    R.H. Byrd, J. Nocedal, and R.B. Schnabel. Representations of quasi-Newton matrices and their use in limited memory methods.Mathematical Programming, 63:129–156, 1994. doi: 10.1007/BF01582063

    1994 doi

  7. [15]

    Conn, N.I.M

    A.R. Conn, N.I.M. Gould, and Ph.L. Toint. Global convergence of a class of trust region method algorithms for optimization with simple bounds.SIAM Journal on Numerical Analysis, 25(2):433–460, 1988. doi: 10.1137/0725029

    1988 doi

  8. [16]

    Conn, N.I.M Gould, and Ph.L

    A.R. Conn, N.I.M Gould, and Ph.L. Toint. Testing a class of methods for solving mini- mization problems with simple bounds on the variables.Mathematics of Computation, 50 (182):399–430, 1988. doi: 10.1090/S0025-5718-1988-0929544-3

    1988 doi

  9. [17]

    Conn, N.I.M

    A.R. Conn, N.I.M. Gould, and Ph.L. Toint. A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds.SIAM Journal on Numerical Analysis, 28(2):545–572, 1991. doi: 10.1137/0728030

    1991 doi

  10. [18]

    Conn, N.I.M

    A.R. Conn, N.I.M. Gould, and Ph.L. Toint. Convergence of quasi-Newton matrices gen- erated by the symmetric rank one update.Mathematical Programming, 50:177–195, 1991. doi: 10.1007/BF01594934

    1991 doi

  11. [19]

    Conn, N.I.M Gould, and Ph.L

    A.R. Conn, N.I.M Gould, and Ph.L. Toint.LANCELOT: A Fortran Package for Large- Scale Nonlinear Optimization (Release A). Springer Series in Computational Mathematics. Springer Berlin, Heidelberg, 1992. doi: 10.1007/978-3-662-12211-2

    1992 doi

  12. [20]

    A.R. Conn, N. Gould A. Sartenaer, and Ph.L. Toint. Global convergence of a class of trust region algorithms for optimization using inexact projections on convex constraints.SIAM Journal on Optimization, 3(1):164–221, 1993. doi: 10.1137/0803009

    1993 doi

  13. [21]

    A.R. Conn, N. Gould A. Sartenaer, and Ph.L. Toint. Convergence properties of an augmented Lagrangian algorithm for optimization with a combination of general equal- ity and linear constraints.SIAM Journal on Optimization, 6(3):674–703, 1996. doi: 10.1137/S1052623493251463

    1996 doi

  14. [22]

    Conn, N.I.M Gould, and Ph.L

    A.R. Conn, N.I.M Gould, and Ph.L. Toint.Trust Region Methods. SIAM, Philadelphia, PA, USA, 2000. doi: 10.1137/1.9780898719857

    2000 doi

  15. [23]

    Curtis, H

    F.E. Curtis, H. Jiang, and D.P. Robinson. An adaptative augmented Lagrangian method for large-scale constrained optimization.Mathematical Programming, Series A, 152:201– 245, 2015. doi: 10.1007/s10107-014-0784-y

    2015 doi

  16. [24]

    Delbos, J.Ch

    F. Delbos, J.Ch. Gilbert, R.Glowinski, and D. Sinoquet. Constrained optimization in seis- mic reflection tomography: a Gauss-Newton augmented Lagrangian approach.Geophysic Journal International, 164:670–684, 2006. doi: 10.1111/j.1365-246X.2005.02729.x. 32

    2006 doi arXiv

  17. [25]

    Dennis Jr and R.B

    J.E. Dennis Jr and R.B. Schnabel.Numerical Methods for Unconstrained optimization and Nonlinear Equations. Classics in Applied Mathematics. SIAM, Philadelphia, PA, USA,

  18. [26]

    doi: 10.1137/1.9781611971200

    doi

  19. [27]

    Dennis Jr, D.M

    J.E. Dennis Jr, D.M. Gay, and R.E. Walsh. An adaptive nonlinear least-squares algorithm. ACM Transactions on Mathematical Software, 7(3):348–368, 1981. doi: 10.1145/355958. 355965

    1981 doi

  20. [28]

    Dolan and J.J

    E.D. Dolan and J.J. Mor´ e. Benchmarking optimization software with performance profiles. Mathematical Programming, Series A 91(2):201–213, 2002. doi: 10.1007/s101070100263

    2002 doi

  21. [29]

    Fletcher and C

    R. Fletcher and C. Xu. Hybrid methods for nonlinear least squares.IMA Journal of Numerical Analysis, 7:373–389, 1987. doi: 10.1093/imanum/7.3.371

    1987 doi

  22. [30]

    Gill and D.P

    P.E. Gill and D.P. Robinson. A primal-dual augmented Lagrangian.Computationnal Optimization and Applications, 51:1–25, 2012. doi: 10.1007/s10589-010-9339-1

    2012 doi

  23. [31]

    Golub and C.F

    G.H. Golub and C.F. Van Loan.Matrix Computations. JHU Press, Maryland, MD, USA, 4th edition, 2013. doi: 10.56021/9781421407944

    2013 doi

  24. [32]

    Gon¸ calves and T.C

    M.L.N. Gon¸ calves and T.C. Menezes. Gauss–Newton methods with approximate projec- tions for solving constrained nonlinear least squares problems.Journal of Complexity, 58: 101459, 2020. doi: 10.1016/j.jco.2020.101459

    2020 doi

  25. [33]

    Gould, M.E

    N.I.M. Gould, M.E. Hribar, and J. Nocedal. On the solution of equality constrained quadratic programming problems arising in optimization.SIAM Journal on Scientific Computing, 23(4):1376–1395, 2001. doi: 10.1137/S1064827598345667

    2001 doi

  26. [34]

    Gagnon, C

    Michel Grenier, J. Gagnon, C. Mercier, and J. Richard. Short-term load forecasting at Hydro-Qu´ ebec Trans´Energie. In2006 IEEE Power Engineering Society General Meeting, Montreal, QC, Canada, 2006. doi: 10.1109/PES.2006.1709029

    2006 doi

  27. [35]

    Hestenes

    M.R. Hestenes. Multipler and gradient methods.Journal of Optimization Theory and Applications, 4:303–320, 1969. doi: 10.1007/BF00927673

    1969 doi

  28. [36]

    Hock and K

    W. Hock and K. Schittkowski.Test Examples for Nonlinear Programming Codes, volume 187 ofLecture Notes in Economics and Mathematical Systems. Springer Berlin, Heidelberg, 2nd edition, 1980. doi: 10.1007/978-3-642-48320-2

    1980 doi

  29. [37]

    Huschens

    J. Huschens. On the use of product structure in secant methods for nonlinear least squares problems.SIAM Journal on Optimization, 4(1):108–129, 1994. doi: 10.1137/0804005

    1994 doi

  30. [38]

    Martinez, and R.A

    J.E.Dennis Jr, H.J. Martinez, and R.A. Tapia. Convergence theory for the structured BFGS method with an application to nonlinear least squares.Journal of Optimization Theory and Applications, 61(2):161–178, 1999. doi: 10.1007/BF00962795

    1999 doi

  31. [39]

    Levenberg

    K. Levenberg. A method for the solution of certain non-linear problems in least squares. Quarterly of Applied Mathematics, 2:164–168, 1944. doi: 10.1090/qam/10666

    1944 doi

  32. [40]

    Z. Li, M. Osborne, and T. Prvan. Adaptative algorithm for constrained least-squares problems.Journal of Optimization Theory and Applications, 114:423–441, 2002. doi: 10.1023/A:1016043919978. 33

    2002 doi

  33. [41]

    Lin and J.J Mor´ e

    C.-J. Lin and J.J Mor´ e. Newton’s method for large bound-constrained optimiza- tion problems.SIAM Journal on Optimization, 9(4):1100–1127, 1999. doi: 10.1137/ S1052623498345075

    1999

  34. [42]

    Lukˇ san and J

    L. Lukˇ san and J. Vlˇ cek. Sparse and partially separable test problems for unconstrained and equality constrained optimization. Technical Report 767, Institute of Computer Science, Academy of Sciences of the Czech Republic, 1999

    1999

  35. [43]

    Lukˇ san, C

    L. Lukˇ san, C. Matonoha, and J. Vlˇ cek. Hybrid methods for nonlinear least squares prob- lems. Technical Report 1246, Institute of Computer Science, Academy of Sciences of the Czech Republic, 2019

    2019

  36. [44]

    Mahdavi-Amiri and R.H

    N. Mahdavi-Amiri and R.H. Bartels. Constrained nonlinear least squares: An exact penalty approach with projected structured quasi-Newton update.ACM Transcations on Mathematical Software, 15(3):220–242, 1989. doi: 10.1145/66888.66891

    1989 doi

  37. [45]

    Marquardt

    D.W. Marquardt. An algorithm for least squares estimation of non-linear parameters. SIAM Journal, 11:431–441, 1963. doi: 10.1137/0111030

    1963 doi

  38. [46]

    Migot, D

    T. Migot, D. Monnet, D. Orban, and S. Abel Soares. JSOSolvers.jl: Unconstrained and bound-constrained optimization solvers.Journal of Open Source Software, 11(117):9467,

  39. [47]

    URLhttps://doi.org/10.21105/joss.09467

    doi: 10.21105/joss.09467. URLhttps://doi.org/10.21105/joss.09467

    doi

  40. [48]

    Mor´ e and G

    J.J. Mor´ e and G. Toraldo. On the solution of large quadratic programming problems with bound constraints.SIAM Journal on Optimization, 1(1):93–113, 1991. doi: 10.1137/ 0801008

    1991

  41. [49]

    Murtagh and M.A

    B.A. Murtagh and M.A. Saunders. Large-scale linearly constrained optimization.Mathe- matical Programming, 14:41–72, 1978. doi: 10.1007/BF01588950

    1978 doi

  42. [50]

    Nocedal and S.J

    J. Nocedal and S.J. Wright.Numerical Optimization. Springer series in Operation Reasearch and Financial Engineering. Springer, New York, NY, USA, 2nd edition, 2006. doi: 10.1007/978-0-387-40065-5

    2006 doi

  43. [51]

    D. Orban. BenchmarkProfiles.jl, 2019. URLhttps://doi.org/10.5281/zenodo. 4630955

    2019 doi

  44. [52]

    Orban and A.S

    D. Orban and A.S. Siqueira. A regularization method for constrained nonlinear least squares.Computational Optimization and Applications, 76(3):961–989, 2020. doi: 10. 1007/s10589-020-00201-2

    2020

  45. [53]

    Porcelli

    M. Porcelli. On the convergence of an inexact Gauss-Newton trust-region method for nonlinear least-squares problems with simple bounds.Optimization Letters, 7:447–465,

  46. [54]

    doi: 10.1007/s11590-011-0430-z

    doi

  47. [55]

    M.J.D. Powell. A method for nonlinear constraints in minimization problems. In R. Fletcher, editor,Optimization, pages 283–298. Academic Press, London, 1969

    1969

  48. [56]

    H. Pu, Z. Chen, B. Wang, and W. Xia. Constrained least squares algorithms for nonlin- ear unmixing of hyperspectral imagery.IEEE Transactions on Geoscience and Remote Sensing, 53(3):1287–1303, 2015. doi: 10.1109/TGRS.2014.2336858. 34

    2015 doi

  49. [57]

    Rockafellar

    R.T. Rockafellar. The multiplier method of Hestenes and Powell applied to convex pro- gramming.Journal of Optimization Theory and Applications, 12(6):555–562, 1973. doi: 10.1007/BF00934777

    1973 doi

  50. [58]

    Schittkowski.More Test Examples for Nonlinear Programming Codes

    K. Schittkowski.More Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems. Springer Berlin, Heidelberg, 1987. doi: 10.1007/ 978-3-642-61582-5

    1987

  51. [59]

    Soares Siqueira and D

    A. Soares Siqueira and D. Orban. NLSProblems.jl, 2019. URLhttps://doi.org/10. 5281/zenodo.4605405

    2019

  52. [60]

    Soares Siqueira and D

    A. Soares Siqueira and D. Orban. SolverBenchmark.jl, 2020. URLhttps://doi.org/10. 5281/zenodo.3948381

    2020

  53. [61]

    Soares Siqueira and D

    A. Soares Siqueira and D. Orban. NLPModelsIpopt.jl, 2026. URLhttps://doi.org/10. 5281/zenodo.19420624

    2026

  54. [62]

    R. Tapia. On secant updates for use in general constrained optimization.Mathematics of Compuations, 51(183):181–202, 1988. doi: 10.2307/2008585

    1988 doi

  55. [63]

    Tjoa and L.T

    I.B. Tjoa and L.T. Biegler. Simultaneous solution and optimization strategies for pa- rameter estimation of differential-algebraic equation systems.Industrial & Engineering Chemistry Research, 30(2):376–385, 1991. doi: 10.1021/ie00050a015

    1991 doi

  56. [64]

    Toint and D

    Ph.L. Toint and D. Tuyttens. LSNNO, a FORTRAN subroutine for solving large-scale nonlinear network optimization problems.ACM Transactions on Mathematical Software, 18(3):308–328, 1992. doi: 10.1145/131766.131771

    1992 doi

  57. [65]

    W¨ achter and L.T

    A. W¨ achter and L.T. Biegler. On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming.Mathematical Programming, Series A, 106:25–57, 2006. doi: 10.1007/s10107-004-0559-y

    2006 doi

  58. [66]

    Yabe and T

    H. Yabe and T. Takahashi. Factorized quasi-Newton methods for nonlinear least squares problems.Mathematical Programming, 51:75–100, 1991. doi: 10.1007/BF01586927

    1991 doi

  59. [67]

    Zhang, L.H

    J.Z. Zhang, L.H. Chen, and N.Y. Deng. A family of scaled factorized broydon-like methods for nonlinear least squares problems.SIAM Journal on Optimization, 10(4):1163–1179,

  60. [68]

    doi: 10.1137/S105262349834530

    doi

  61. [69]

    Zhou and X

    W. Zhou and X. Chen. Global convergence of a new hybrid Gauss-Newton structured BFGS method for nonlinear least squares problems.SIAM Journal on Optimization, 20 (5):2422–2441, 2010. doi: 10.1137/090748470. 35 Appendix A Cauchy point computation For the sake of clarity, we omi...

    2010 doi

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.