Variable projection framework for the reduced-rank matrix approximation problem by weighted least-squares

IRD); Pascal Terray (LOCEAN

REVIEW 2 major objections 5 minor 201 references

Variable projection framework for the reduced-rank matrix approximation problem by weighted least-squares

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper establishes that the weighted low-rank approximation problem becomes tractable for second-order variable projection methods once the systematic rank deficiencies of the Jacobian and Hessian are handled explicitly.

desk verdict A dense, serious theoretical monograph on variable projection for weighted low-rank approximation whose gradient/Hessian formulas and rank-deficiency analysis are worth checking, but whose load-bearing differentiability assumption is not proved and whose practical claims need numerical support. read the letter →

arxiv 2505.03347 v1 pith:TY3GMPAH submitted 2025-05-06 math.NA cs.NAmath.OC

Pascal Terray (LOCEAN , IRD) This is my paper · ORCID

classification math.NAcs.NAmath.OC MSC 65F5565K0515A8390C30

keywords weightedlow-rankapproximationvariableprojectionseparablenonlinearleastsquaresrank-deficientJacobianHessianGrassmannmanifoldmatrixcompletionGauss-NewtonandLevenberg-Marquardt

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's aim is to make the weighted low-rank approximation (WLRA) problem — fitting a rank-$k$ matrix to data with per-entry weights — tractable for reliable second-order optimization. Its central move is to rewrite WLRA as a separable nonlinear least-squares problem, so that one factor matrix is solved in closed form for the other, leaving a lower-dimensional variable projection objective. On that objective the paper derives new explicit formulae for the Jacobian and Hessian and shows both carry systematic rank deficiencies: the Jacobian is uniformly rank-deficient, bounded by the dimension of the underlying Grassmann manifold, and the Hessian is rank-deficient at local minimizers, consequences of a $k^2$-dimensional gauge invariance. The paper argues these deficiencies are why earlier second-order variable projection attempts were fragile and must be confronted in any practical implementation. A sympathetic reader would care because WLRA underlies recommender systems, computer vision, signal processing and geophysical analysis, where slow first-order methods (alternating least squares, expectation-maximization, gradient descent) have often been the only feasible choice at scale.

What carries the argument

The load-bearing object is the variable projection functional $\psi(a) = \frac{1}{2}\|P^{\perp}_{F(a)}x\|_2^2$, where $x = \mathrm{vec}(\sqrt{W}\odot X)$ stacks the weighted data, $F(a) = \bigoplus_{j=1}^{n} \mathrm{diag}(\sqrt{W_{\cdot j}})\, A$ is a block-diagonal design matrix built from the left factor $A$ (vectorized as $a = \mathrm{vec}(A^{\mathsf T})$), and $P^{\perp}_{F(a)}$ is the orthogonal projector onto the orthogonal complement of the range of $F(a)$. Minimizing $\psi$ in $A$, then recovering $B$ by solving a linear least-squares problem, reproduces the WLRA optimum (Theorem 3.9, adapted from the classical variable projection equivalence). Two identities carry the argument: the invariance $\psi(\mathrm{vec}(A^{\mathsf T})) = \psi(\mathrm{vec}((AD)^{\mathsf T}))$ for invertible $D$, which reduces the effective search space to the Grassmann manifold $\mathrm{Gr}(p,k)$ and forces the Jacobian's uniform rank deficiency and the Hessian's deficiency at minimizers; and the companion variable orthogonal functional $\psi_*(a) = \|Q_2(a)x\|_2^2$ built from the second QR factor of $F(a)$, which is equivalent to $\psi$ but whose differentiability requires the rank of $F(a)$ to stay constant in a neighborhood of a solution.

What would settle it

Solve a small WLRA instance (for example a $3\times 3$ matrix with one missing entry and $k=1$) to a local minimizer of the variable projection functional, then compare the paper's explicit Jacobian and Hessian formulae with central finite differences of the residual and gradient. Where the rank of the design matrix $F(a)$ changes at the minimizer, the explicit and numerical derivatives will diverge, exposing the reach of the constant-rank assumption; and a sweep over starting points checking whether the Jacobian's numerical rank ever exceeds $k(p-k)$ would directly test the claimed uniform deficiency.

Watch

Extended reading notes

Core claim

The paper's core discovery is that the variable projection functional for weighted low-rank approximation has a rigid, previously under-appreciated differential structure. Writing the cost as $\psi(a) = \frac{1}{2}\|P^{\perp}_{F(a)}x\|_2^2$, with $x$ the weighted data and $F(a)$ the block-diagonal design matrix depending on the first factor $A$, the paper derives new closed-form expressions for the Jacobian of the variable projection residual and for the Hessian of $\psi$, and proves the Jacobian is uniformly rank-deficient — its rank can never reach its nominal $p\cdot k$ value because of the $k^2$-dimensional gauge invariance $\psi(\mathrm{vec}(A^{\mathsf T})) = \psi(\mathrm{vec}((AD)^{\mathsf T}))$ — while the Hessian is rank-deficient at every local minimizer. The paper argues these systematic deficiencies are the main obstacles that have blocked robust second-order variable projection implementations for WLRA, and that any practical implementation must take them into account. It further shows the same framework illuminates the solvability, landscape and non-smoothness of WLRA and reveals that variable projection and Riemannian optimization on the Grassmann manifold are closely linked, nearly equivalent numerical routes to the same problem.

Load-bearing premise

The derivative apparatus assumes that the rank of the block design matrix built from the factor matrix stays constant in a neighborhood of the minimizer, and if that rank drops at the solution — a situation the paper itself flags for weighted problems with missing entries — the explicit Jacobian and Hessian formulas cease to be valid.

Editorial extensions

If this is right

Second-order variable projection methods (Gauss-Newton, Levenberg-Marquardt, Newton) for WLRA become practically implementable once the Jacobian's uniform rank deficiency and the Hessian's deficiency at minimizers are handled, since ignoring them is what previously made such methods unreliable.
Because the objective depends only on the column space of the first factor, the effective optimization dimension is $k(p-k)$: restricting the search to the Stiefel or Grassmann manifold removes the gauge directions and repairs the Hessian deficiency.
The variable orthogonal functional offers a numerically distinct but mathematically equivalent route, so implementations can choose between pseudo-inverse-based and QR-based projections, subject to the constant-rank condition.
The framework locates WLRA's hardness — NP-hardness, possible absence of solutions with zero weights, multiple local minima under non-uniform weights — in the same separable geometry, and shows Tikhonov-style regularization with vanishing parameter approaches the WLRA infimum from well-posed problems.
Variable projection and Riemannian optimization on the Grassmann manifold are nearly equivalent numerically for WLRA, so algorithmic safeguards and convergence insights transfer between the two approaches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: tracking the numerical rank of the Jacobian along iterates on synthetic binary-weight problems should show it saturating at $k(p-k)$; an iterate whose Jacobian has higher numerical rank would mark where the uniform-deficiency claim leaves the constant-rank regime.
The same gauge-invariance analysis likely transfers to any separable least-squares problem with a bilinear factorization — dictionary learning, or fine-tuning the last linear layer of a network — where the same $k^2$-dimensional null directions would silently degrade second-order information.
Near a solution where the optimal factor has rank below $k$ (an over-specified model), the explicit Hessian formula is exactly where the constant-rank assumption is most likely to fail; implementations may need rank-aware damping or projection safeguards in precisely that regime.
One could verify the claimed Hessian deficiency directly: compute the Hessian of the variable projection objective at a local minimizer both by the explicit formula and by finite differences of the gradient; the predicted null space of dimension at least $k^2$ should appear in both, and any mismatch would localize where the assumption breaks.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, this is a serious, long theoretical monograph, not a quick contribution: the payoff is a systematic derivation of Jacobian and Hessian formulas for variable-projection WLRA and a rank-deficiency analysis that the author argues has been under-appreciated. Second, the central differentiability assumption — constant rank of F(a) near a solution — is stated explicitly but not proved, and it matters most in the zero-weight cases that give WLRA much of its practical interest.

What is genuinely new and useful: the equivalence theorems (3.1 and 3.9), the block-diagonal structure of F(a), the uniform rank deficiency of the Jacobian and Hessian rank deficiency at minimizers, and the clarified link to Grassmann-manifold optimization. The historical attribution is handled carefully: Ruhe gets credit for the k=1 variable-projection case, Wiberg for arbitrary k, and the ALS/VP confusion is documented rather than repeated. For someone working on weighted low-rank problems, this is a valuable reference.

What is soft: there are no numerical experiments and no code, so the promised robustness and efficiency are unverified. Several auxiliary lemmas and proofs are omitted with citations, which makes the long tensor computations harder to trust. The stress-test concern about constant rank is the sharpest issue: equation (3.26) defines the variable-orthogonal functional and the subsequent derivative formulas under a constant-rank assumption, and the paper does not show that minimizers in the zero-weight regime satisfy it. If a block diag(sqrt(W_j)) A loses rank at a solution, the Jacobian and Hessian formulas do not apply. The paper is upfront about the assumption, so this is not a hidden flaw, but it does mean the recommended algorithms are not fully justified in exactly the sparse-weight cases that motivate WLRA.

The citation pattern looks fair and broad, with no sign of self-citation inflation or missing credit where credit is due.

Who this is for: specialists in low-rank and separable nonlinear least-squares optimization. It deserves a serious referee, but I would send it out with the expectation that the referee check the algebra and that the author add small numerical experiments with zero and near-zero weights to show the formulas and rank-deficiency claims behave as advertised. If the derivations survive that check, this becomes a standard reference for variable-projection WLRA.

Referee Report

2 major / 5 minor

Summary. The paper develops a variable-projection formulation of the weighted low-rank approximation (WLRA) problem. It proves equivalence between the rank-constrained problem (P0), its bilinear factorization (P1), and the variable-projection problem (VP1) in Theorems 3.1 and 3.9, and it reviews and derives explicit Jacobian and Hessian formulas for second-order variable-projection methods, with emphasis on systematic rank deficiencies of the Jacobian and Hessian. The paper also connects the variable-projection viewpoint to Grassmannian optimization. The first three sections, which are visible in the submitted text, are largely a survey with some new proofs; the later sections, summarized in the abstract and table of contents, contain the main derivative formulas and algorithm templates. The manuscript contains no numerical experiments, no code, and several auxiliary results are stated with proofs omitted.

Significance. If the derivative formulas and rank-deficiency results in Sections 5 and 6 are correct, the paper would be a useful reference for practitioners, because it gives explicit algebraic expressions for the quantities needed in Gauss-Newton, Levenberg-Marquardt, and Newton methods for WLRA, and it spells out the relationship between variable projection and Grassmannian optimization. The proved equivalence theorems (3.1 and 3.9) are a genuine strength, and the block-diagonal structure in Eq. (3.20) is convenient. No parameters are fitted and the analytic results are derived from stated definitions, so I do not see a circularity problem. However, the paper's practical claims currently outrun its evidence: there are no numerical experiments or code, and the main derivative analysis rests on a rank-stability assumption that is not established for the zero-weight case that motivates WLRA.

major comments (2)

[§3.4, Eq. (3.26)] The differentiability of the variable-projection functional is assumed rather than established. The text states "assuming that the rank of F(a) stays constant in a neighborhood of a solution of the VP1 problem" and notes that the same condition is implicit for ψ. In the WLRA setting, F(a) = ⊕_j diag(√W_{.j})A (Eq. (3.20)), so rank stability requires every block diag(√W_{.j})A to have constant rank near the minimizer. When some Wij=0 and a column has fewer than k positive weights, the corresponding block can lose rank as A changes; then P⊥_{F(a)} and Q2(a) cease to be differentiable, and the Jacobian and Hessian formulas in Sections 5.2–5.3, together with the rank-deficiency conclusions drawn from them, do not apply. Since zero weights and missing data are a main motivating regime of the paper, this is a load-bearing gap: the paper should either prove the condition under explicit assumptions on (W, X, k), or reformulate the claims for the full-rank case and state precisely which results extend to the rank-deficient zero-weight case.
[Abstract; Section 6] The abstract and Section 6 present the algorithms as robust, efficient, and scalable and state that the identified rank deficiencies "must be taken into account in any practical implementations," but the manuscript contains no numerical experiments, no convergence measurements, and no code. For a paper whose stated audience includes software developers and practitioners, the practical significance of the new formulas and the proposed damped or trust-region strategies is not demonstrated. A basic experimental section on small weighted problems with zero weights, including a comparison of the rank-deficient and full-rank variants, would be necessary to substantiate the central practical claim.

minor comments (5)

[§2.4] The text uses the same symbol for the little-o definition ("∀ε ∃δ ... ≤ ε‖h‖") and the big-O definition ("∃λ,η ... ≤ λ‖h‖"); these should be distinguished, e.g., o(·) versus O(·).
[Lemma 2.2; Theorems 2.3–2.5, 2.9] Several statements in Section 2 are given with "Proof. Omitted" and an external reference. Since the monograph aims at self-contained derivations, the author should either include the proofs or state explicitly which results are imported and which are new.
[Theorem 3.7, proof of item (3)] The inner product in the proof should be written as ⟨∇ϕ(AB), C_c D_c⟩_F; the expression ⟨∇ϕ(AB), C_c, D_c⟩_F is undefined as written, although the subsequent calculation makes clear that the product C_c D_c is intended.
[Section 1] There are small language issues: "first-order methods, excepted for the optimization approaches" should be "except for," and "software's developers" should be "software developers."
[Corollary 3.2; Remark 3.7] The paper should clarify which of the claimed rank-deficiency properties are new algebraic facts and which are direct consequences of the GL(k) invariance of ψ established in Corollary 3.2 and Remark 3.7; as written, the abstract presents them as obstacles without separating geometric consequences from genuinely new formulas.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the variable projection reduction and derivative properties are derived from the stated WLRA formulation, not assumed or fitted.

full rationale

The paper's derivation chain is self-contained. The variable projection functional ψ(a) = ½‖P⊥_{F(a)}x‖² (Eq. 3.23) is obtained by eliminating B analytically from the equivalent factored form (P1), and Theorem 3.9 proves the equivalence between (P1) and (VP1) using the defining property of the pseudoinverse. No parameter is fitted to data, and no target result is used as an assumption. The later Jacobian and Hessian formulas and the claimed rank-deficiency properties are analytic consequences of this representation together with the GL(k) invariance of the objective (Remark 3.4, Corollary 3.2), not inputs to the derivation. The constant-rank hypothesis, “assuming that the rank of F(a) stays constant in a neighborhood of a solution of the VP1 problem” (Section 3.4, Eq. 3.26), is a regularity assumption on the problem and a genuine limitation when zero weights cause rank drops, but it is not circular: it restricts the domain of validity of the derivative formulas instead of presupposing the rank-deficiency conclusions. External citations such as Golub–Pereyra and Ruhe provide independent standard results; no load-bearing self-citation chain was found. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters are fitted; the paper is a theoretical derivation. The domain assumptions on the weight matrix and the rank-stability condition are the main premises the algorithms rest on. No new physical or mathematical entities are postulated.

assumptions (3)

domain assumption The weight matrix W is nonnegative with at least one nonzero entry in each row and column; stronger forms require at least k nonzero entries per row and column.
Stated in the Introduction as necessary for well-posedness and to avoid overfitting in the WLRA problem.
ad hoc to paper The rank of F(a) stays constant in a neighborhood of a solution of the variable projection problem.
Needed for the differentiability of the variable projection functional; stated in Section 3.4 around equation (3.26).
standard math Background results in linear algebra, SVD theory, Moore-Penrose inverses, and Frechet differentiability are used as given.
Standard mathematical toolkit invoked throughout Section 2 as preliminaries.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Variable projection framework for the reduced-rank matrix approximation problem by weighted least-squares." pith.science (2026). https://pith.science/paper/TY3GMPAH

@misc{pith2026250503347,
  author       = {Pith},
  title        = {Pith review of: Variable projection framework for the reduced-rank matrix approximation problem by weighted least-squares},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TY3GMPAH}},
  note         = {Machine review of arXiv:2505.03347}
}

read the original abstract

In this monograph, we review and develop variable projection Gauss-Newton, Levenberg-Marquardt and Newton methods for the Weighted Low-Rank Approximation (WLRA) problem, which has now an increasing number of applications in many scientific fields. Particular attention is drawn at the robustness, efficiency and scalability of these variable projection second-order algorithms such that they can be used also on larger datasets now commonly found in many practical problems for which only first-order algorithms based on sequential repetitions of local optimization (e.g., majorization, Expectation-Maximization or alternating least-squares methods) or variations of gradient descent (e.g., conjugate, proximal or stochastic gradient descent methods), or hybrid algorithms from these two classes of methods, were only feasible due to their lower cost and memory requirement per iteration. In parallel with this review of variable projection algorithms, we develop new formulae for the Jacobian and Hessian matrices involved in these variable projection methods and demonstrate their very specific properties such as the uniform rank deficiency of the Jacobian matrix or the rank deficiency of the Hessian matrix at the (local) minimizers of the cost function associated with the WLRA problem. These systematic deficiencies must be taken into account in any practical implementations of the algorithms. These different properties and the very particular geometry of the WLRA problem have not been well appreciated in the past and have been the main obstacles in the development of robust variable projection second-order algorithms for solving the WLRA problem. In addition, we demonstrate that the variable projection framework gives original insights on the solvability, the landscape and the non-smoothness of the WLRA problem. It also helps to describe the tight links between previously unrelated methods, which have been proposed to solve it. Specifically, we illustrate the closed links between the variable projection framework and Riemannian optimization on the Grassmann manifold for the WLRA problem. We expect that software's developers and practitioners in different fields such as computer vision, signal processing, recommender systems, machine learning, multivariate statistics and geophysical sciences will benefit from the results in this monograph in order to devise more robust and accurate algorithms to solve the WLRA problem.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

201 extracted references · 73 canonical work pages

[1]

Absil, J

P .-A. Absil, J. Malick (2012) Projection-like retractions on matrix manifolds , SIAM J. Op- tim., 22:135-158

2012
[2]

Allgower, K

E.L. Allgower, K. Bohmer, A. Hoy, V . Janovsky (1999) Direct methods for solving singular nonlinear equations, Z. Angew. Math. Mech. 79(4):219-231

1999
[3]

Absil, R

P .-A. Absil, R. Mahony, R. Sepulchre (2008) Optimization Algorithms on Matrix Manifolds , Princeton University Press, Princeton, NJ

2008
[4]

Attouch, J

H. Attouch, J. Bolte, P . Redont, A. Soubeyran (2010) Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Lojasiewicz inequality, Mathematics of Operations Research, 35(2):438-457

2010
[5]

Ben-Israel (1965) A modiﬁed Newton-Raphson method for the solution of systems of equa- tions, Israel Journal of Mathematics, 3:94-98

A. Ben-Israel (1965) A modiﬁed Newton-Raphson method for the solution of systems of equa- tions, Israel Journal of Mathematics, 3:94-98

1965
[6]

Ben-Israel (1966) A Newton-Raphson method for the solution of systems of equat ions, Journal of Mathematical Analysis and Applications, 15:243 -252

A. Ben-Israel (1966) A Newton-Raphson method for the solution of systems of equat ions, Journal of Mathematical Analysis and Applications, 15:243 -252

1966
[7]

Bertsekas (1999) Nonlinear Programming, 2nd Edition, Athena Scientiﬁc, Belmont, Massachusetts

D.P . Bertsekas (1999) Nonlinear Programming, 2nd Edition, Athena Scientiﬁc, Belmont, Massachusetts

1999
[8]

Bjorck (2015) Numerical Methods in Matrix Computations , Series: Texts in Applied Mathematics, V ol

A. Bjorck (2015) Numerical Methods in Matrix Computations , Series: Texts in Applied Mathematics, V ol. 59, Springer, 800 p

2015

Show all 201 references

[9]

Boggs (1976) The convergence of the Ben-Israel iteration for nonlinear l east-squares problems, Mathematics of Computation, 30(135):512-522

P .T. Boggs (1976) The convergence of the Ben-Israel iteration for nonlinear l east-squares problems, Mathematics of Computation, 30(135):512-522

1976
[10]

C.F. Borges (2009) A full-newton approach to separable nonlinear least-squar es problems and its application to discrete least-squares rational app roximation, Electronic Transactions on Numerical Analysis, 35:57-68

2009
[11]

Boumal (2023) An introduction to optimization on smooth manifolds , Cambridge Univer- sity Press

N. Boumal (2023) An introduction to optimization on smooth manifolds , Cambridge Univer- sity Press. Also available at https://www.nicolasboumal.net/book

2023
[12]

Bredon (1993) Topology and Geometry , 2nd Edition, Graduate texts in mathematics, Springer-V erlag, New Y ork

G.E. Bredon (1993) Topology and Geometry , 2nd Edition, Graduate texts in mathematics, Springer-V erlag, New Y ork

1993
[13]

Boumal, P .-A

N. Boumal, P .-A. Absil (2011) RTRMC: a Riemannian trust-region method for low-rank ma- trix completion, in: J. Shawe-Taylor, R.S. Zemel, P . Bartlett, F.C.N. Pereira, K.Q. Weinberger (Eds.), Advances in Neural Information Processing Systems 24, NIPS, 2011, pp.406-414

2011
[14]

Boumal, P .-A

N. Boumal, P .-A. Absil (2015) Low-rank matrix completion via preconditioned optimizati on on the Grassmann manifold , Linear Algebra and its Applications, 475:200-239

2015
[15]

Buchanan, A.W

A.M. Buchanan, A.W. Fitzgibbon (2005) Damped Newton algorithms for matrix factoriza- tion with missing data , in Proceedings of the 2005 IEEE Computer Society Conferenc e on Computer Vision and Pattern Recognition (CVPR), 2:316-322

2005
[16]

Bhojanapalli, P

S. Bhojanapalli, P . Jain (2014) Universal matrix completion, in Proceedings of the 31st Inter- national Conference on Machine Learning (ICML-14), pages 1 881-1889, 2014

2014
[17]

Bertsimas, M.L

D. Bertsimas, M.L. Li (2020) Fast Exact Matrix Completion: A Unifed Optimization Frame- work for Matrix Completion , Journal of Machine Learning Research, 21:1-43

2020
[18]

Burer, R.D.C

S. Burer, R.D.C. Monteiro (2003) A nonlinear programming algorithm for solving semideﬁ- nite programs via low-rank factorization, Mathematical Programming, 95(2):329-357

2003
[19]

Bellavia, B

S. Bellavia, B. Morini (2015) Strong local convergence properties of adaptive regulariz ed methods for nonlinear least squares . IMA J. Numer. Anal. 35(2):947-968. 210

2015
[20]

Beckers, M

J.M. Beckers, M. Rixen (2003) EOF calculations and data ﬁlling from incomplete oceano- graphic datasets, J. Atoms. Ocean. Tech., 20(12):1839-1856

2003
[21]

A. Beck, L. Tetruashvili (2013) On the convergence of block coordinate descent type methods, SIAM J. Optim., 23(4):2037-2060

2013
[22]

Bruns and U

W. Bruns and U. V etter (1988) Determinantal rings, Lecture Notes in Math. 1327, Springer- V erlag, Berlin

1988
[23]

F. Ban, D. Woodruff, Q. Zhang (2019) Regularized Weighted Low Rank Approximation, 33rd Conference on Neural Information Processing Systems (Neur IPS 2019), V ancouver, Canada

2019
[24]

Bendokat, R

T. Bendokat, R. Zimmermann, P .A. Absil (2023) A Grassmann manifold hand- book: Basic Geometry and Computational Aspects , arXiv:2011.13699v3. See https://arxiv.org/abs/2011.13699v3

2023 arXiv
[25]

Bhaskara, A.K

A. Bhaskara, A.K. Ruwanpathirana, M. Wijewardena (202 1) Additive Error Guarantees for Weighted Low Rank Approximation, in Proceedings of the 38th International Conference on Machine Learning, PMLR 139, 2021

2021
[26]

Cartan (2017) Differential calculus on normed spaces: A course in Analysi s, 2nd Edition, 217 pp

H. Cartan (2017) Differential calculus on normed spaces: A course in Analysi s, 2nd Edition, 217 pp

2017
[27]

Chen (2008a) Heteroscedastic Low-Rank Matrix Approximation by the Wibe rg Algorithm, IEEE Transactions on signal processing, 56(4):1429-1439

P . Chen (2008a) Heteroscedastic Low-Rank Matrix Approximation by the Wibe rg Algorithm, IEEE Transactions on signal processing, 56(4):1429-1439

2008
[28]

Chen (2008b) Optimization algorithms on subspaces: Revisiting missing data problem in low-rank matrix, Int

P . Chen (2008b) Optimization algorithms on subspaces: Revisiting missing data problem in low-rank matrix, Int. J. Comput. Vis., 80:125-142

2008
[29]

Campbell, C.D

S.L. Campbell, C.D. Meyer (2009) Generalized Inverses of Linear Transformations, Classics in Applied Mathematics, SIAM, Philadelphia

2009
[30]

Candes, B

E.J. Candes, B. Recht (2009) Exact matrix completion via convex optimization, Found. Com- put. Math. 9(6):717-772

2009
[31]

Cason,P .A

T.P . Cason,P .A. Absil, P .V . Dooren (2013)Iterative methods for low rank approximation of graph similarity matrices, Linear Algebra Appl. 438(4):1863-1882

2013
[32]

Cai, E.J

J.F. Cai, E.J. Candes, Z. Shen (2010) A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20(4):1956-1982

2010
[33]

Chu, R.E

M.T. Chu, R.E. Funderlic, R.J. Plemmons (2003) Structured low rank approximation, Linear Algebra and its Applications, 366:157-172

2003
[34]

Campbell, P

S.L. Campbell, P . Kunkel, K. Bobinyec (2012) A minimal norm corrected underdetermined Gauss-Newton procedure, Appl. Numer. Math. 62:592-605

2012
[35]

Cand` es, X

E.J. Cand` es, X. Li, Y . Ma, J. Wright (2011) Robust principal component analysis? , Journal of the ACM, 58(3):11

2011
[36]

X. Chen, Z. Nashed, L. Qi (1997) Convergence of Newton’s method for singular smooth and nonsmooth equations using adaptive outer inverses , SIAM J. Optim. 7:445-462

1997
[37]

Daskalov (2011) Iterative methods for matrix factoriza- tion with missing data , Master Thesis, ETH, Zurich

B.N. Daskalov (2011) Iterative methods for matrix factoriza- tion with missing data , Master Thesis, ETH, Zurich. See https://www.research-collection.ethz.ch/handle/20.500.11850/152839

2011
[38]

Decell (1974) On the derivative of the generalized inverse of a matrix , Linear and Mul- tilinear Algebra 1(4):357

H.P . Decell (1974) On the derivative of the generalized inverse of a matrix , Linear and Mul- tilinear Algebra 1(4):357

1974
[39]

Deutsch (2012) Best Approximation in Inner Product Spaces

F.R. Deutsch (2012) Best Approximation in Inner Product Spaces . Springer. 211

2012
[40]

Decker, C.T

D.W. Decker, C.T. Kelley (1980) Newton’s method for singular points, I , Siam J. Numer. Anal., 17:66-70

1980
[41]

Decker, C.T

D.W. Decker, C.T. Kelley (1980) Newton’s method for singular points, II , Siam J. Numer. Anal., 17(3):465-471

1980
[42]

Dedieu, M.H

J.P . Dedieu, M.H. Kim (2002) Newton’s method for analytic systems of equations with con- stant rank derivatives, J. Complexity 18:187-209

2002
[43]

D. Duan, H. Liu (2024) A fast and efﬁcient randomized quasi-Newton method , OPT 2024: Optimization for Machine Learning. See https://openrevie w.net/forum?id=laJUMr2p3l

2024
[44]

Davenport, J

M.A. Davenport, J. Romberg (2016) An Overview of Low-Rank Matrix Recovery From In- complete Observations , IEEE Journal of Selected Topics in Signal Processing, 10(4 ):608- 622

2016
[45]

Dennis, R.B

J.E. Dennis, R.B. Schnabel (1983) Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall

1983
[46]

W. Dai, O. Milenkovic, E. Kerman (2011) Subspace evolution and transfer (SET) for low- rank matrix completion, IEEE Trans. Signal Process.59(7):3120-3132

2011
[47]

W. Dai, E. Kerman, O. Milenkovic (2012) A geometric approach to low-rank matrix comple- tion, IEEE Trans. Inform. Theory 58(1):237-247

2012
[48]

Demmel, L

J. Demmel, L. Grigori, M. Hoemmen, J. Langou (2012) Communication-optimal parallel and sequential QR and LU factorizations , SIAM J. Sci. Comp., 34:A206-A239

2012
[49]

Dennis, D.M

J.E. Dennis, D.M. Gay, R.E. Welsch (1981) Algorithm 573 - NL2SOL, An adaptive nonlinear least-squares algorithm, ACM Transactions on Mathematical Software, 7, pp. 348-368

1981
[50]

Dutta, J

A. Dutta, J. Liang, X. Li (2022) A fast and adaptive svd-free algorithm for general weighted low-rank recovery, arXiv:2101.00749v2. See https://arxiv.org/abs/2101.00749v2

2022 arXiv
[51]

Edelman, T.A

A. Edelman, T.A. Arias, S.T. Smith (1998) The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Applicat. 20(2):303-353

1998
[52]

Eriksson (1996) Optimization and Regularization of Nonlinear Least Square s Problems, Ph.D

J. Eriksson (1996) Optimization and Regularization of Nonlinear Least Square s Problems, Ph.D. Thesis, Umea University, Sweden

1996
[53]

Eriksson, P .-A

J. Eriksson, P .-A. Wedin (1996) Regularization Methods for Nonlinear Least Squares. Part 1: Exactly Rank-deﬁcient Problems , Technical Report UMINF-96.03, Department of Com- puting Science, Umea University, Umea, Sweden

1996
[54]

Eriksson, P .-A

J. Eriksson, P .-A. Wedin, M.E. Gulliksson, I. Soderkvi st (2005) Regularization methods for uniformly rank-deﬁcient nonlinear least-squares prob lems, J. Optimi. Theory and Appl. 127:1-26

2005
[55]

Fletcher (1968) Generalized inverse methods for the best least-squares solution of systems of non-linear equations, Comput

A. Fletcher (1968) Generalized inverse methods for the best least-squares solution of systems of non-linear equations, Comput. J., 10:392-399

1968
[56]

J. Fan, Y . Y uan On the quadratic convergence of the Levenberg–Marquardt me thod without non-singularity assumption. Computing 74(1):23-39
[57]

Gabriel (1978) Least Squares Approximation of Matrices by Additive and Mul tiplicative Models, Journal of the Royal Statistical Society, Series B (Method ological), 40(2):186-196

K. Gabriel (1978) Least Squares Approximation of Matrices by Additive and Mul tiplicative Models, Journal of the Royal Statistical Society, Series B (Method ological), 40(2):186-196

1978
[58]

Goldberg (2017) Continuity of seminorms on ﬁnite-dimensional vector space s, Linear Algebra and its Applications, 515:175-179

M. Goldberg (2017) Continuity of seminorms on ﬁnite-dimensional vector space s, Linear Algebra and its Applications, 515:175-179

2017
[59]

Griewank (1985) On solving nonlinear equations with simple singularities o r nearly sin- gular solutions, SIAM Review, 27(4):537-563

A. Griewank (1985) On solving nonlinear equations with simple singularities o r nearly sin- gular solutions, SIAM Review, 27(4):537-563. 212

1985
[60]

Guignard (1969) Generalized Kuhn–Tucker conditions for mathematical prog ramming problems in a Banach space , SIAM Journal on Control, 7:232-241

M. Guignard (1969) Generalized Kuhn–Tucker conditions for mathematical prog ramming problems in a Banach space , SIAM Journal on Control, 7:232-241

1969
[61]

D.M. Gay, L. Kaufman (1991) Tradeoffs in Algorithms for Separable Nonlinear Least Squares, IMACS ’91, Proceedings of the 13th World Congress on Comput ational and Ap- plied Mathematics, edited by R. Vichnevetsky and J. J. H. Mil ler, Criterion Press, Dublin, 1991

1991
[62]

Gillis, F

N. Gillis, F. Glineur (2011) Low-rank matrix approximation with weights or missing data is NP-hard, SIAM J. Matrix Anal. Appl., 32(4):1149-1165

2011
[63]

Golub, V

G.H. Golub, V . Pereyra (1973) The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate , SIAM Journal on Numerical Analysis, 10(2):413-432

1973
[64]

Golub, V

G.H. Golub, V . Pereyra (1976) Differentiation of pseudo-inverses, separable nonlinear least squares problems and other tales , Proceedings of an Advanced Seminar Sponsored by the Mathematics Research Center, the University of Wisconsin- Madison, October 8-10, 1973, pages 303-324

1976
[65]

Golub, V

G.H. Golub, V . Pereyra (2003) Separable nonlinear least squares: the variable projectio n method and its applications, Inverse Problems, 19:R1-R26

2003
[66]

Gotardo, A.M

P .F. Gotardo, A.M. Martinez (2011) Computing smooth time trajectories for camera and deformable shape in structure from motion with occlusion , IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 33(10):2051-2065

2011
[67]

Griewank, M.R

A. Griewank, M.R. Osborne (1981) Newton’s method for singular problems when the dimen- sion of the null space is > 1, SIAM J. Numer. Anal., 18(1):145-149

1981
[68]

Grippo, M

L. Grippo, M. Sciandrone (1999) Globally convergent block-coordinate techniques for un- constrained optimization, Optim. Methods Soft., 10: 587-637

1999
[69]

Grippo, M

L. Grippo, M. Sciandrone (2000) On the convergence of the block nonlinear Gauss–Seidel method under convex constraints, Operations research letters, 26(3):127-136

2000
[70]

Gulliksson, P .A

M.E. Gulliksson, P .A. Wedin (2000) The Use and Properties of Tikhonov Filter Matrices , SIAM Journal on Matrix Analysis and Applications 22:276-28 1

2000
[71]

Golub, C

G.H. Golub, C. V an Loan (1996) Matrix Computation, 3rd Edition, The Johns Hopkins Uni- versity Press, Baltimore, MD

1996
[72]

Gabriel, S

K. Gabriel, S. Zamir (1979) Lower rank approximation of matrices by least squares with a ny choice of weights, Technometrics, 21:489-498

1979
[73]

Haussler (1986) A Kantorovich-type convergence analysis for the Gauss-New ton method, Numer

W.M. Haussler (1986) A Kantorovich-type convergence analysis for the Gauss-New ton method, Numer. Math., 48:119-125

1986
[74]

Harris(1992) Algebraic Geometry , volume 133 of Graduate Texts in Mathematics

J. Harris(1992) Algebraic Geometry , volume 133 of Graduate Texts in Mathematics. Springer- V erlag New Y ork

1992
[75]

Ho (2010) Nonnegative Matrix Factorization - Algorithms and Applications, Ph.D

N.D. Ho (2010) Nonnegative Matrix Factorization - Algorithms and Applications, Ph.D. the- sis, Universite catholique de Louvain, Louvain-la-Neuve, Belgium

2010
[76]

Hardt (2014) Understanding alternating minimization for matrix comple tion, in: IEEE 55th Annual Symposium on Foundations of Computer Science, F OCS, 2014, IEEE, Oct 2014, 651-660

M. Hardt (2014) Understanding alternating minimization for matrix comple tion, in: IEEE 55th Annual Symposium on Foundations of Computer Science, F OCS, 2014, IEEE, Oct 2014, 651-660

2014
[77]

Helmke, J.B

U. Helmke, J.B. Moore (1996) Optimization and Dynamical Systems , Communications and Control Engineering Series, Springer-V erlag London Ltd., London. 213

1996
[78]

Helmke, M.A

U. Helmke, M.A. Shayman (1995) Critical points of matrix least squares distance functions , Linear Algebra Appl., 215:1-19

1995
[79]

Hiriart-Urruty, C

J.B. Hiriart-Urruty, C. Le Marechal (2004) Fundamentals of convex analysis, Springer

2004
[80]

Hiriart-Urruty, H.Y

J.B. Hiriart-Urruty, H.Y . Le (2013) A variational approach of the rank function, TOP 21:207- 240

2013
[81]

Hong, A.W

J.H. Hong, A.W. Fitzgibbon (2015a) Secrets of Matrix Factorization: Approximations, Nu- merics, Manifold Optimization and Random Restarts , in Proceedings of the 2015 IEEE In- ternational Conference on Computer Vision (ICCV), pages 41 30-4138

2015
[82]

Hong, A.W

J.H. Hong, A.W. Fitzgibbon (2015b) Secrets of Matrix Factorization: Further Derivations and Comparisons, Supplementary material to [81] in Proceedings of the 2015 I EEE Interna- tional Conference on Computer Vision (ICCV)

2015
[83]

W. Ha, H. Liu, R.F. Barber (2020) An equivalence between critical points for rank constraints versus low-rank factorizations, SIAM Journal on Optimization, 30(4):2927-2955

2020
[84]

Hosseini, D.R

S. Hosseini, D.R. Luke, A. Uschmajew (2019) Tangent and Normal Cones for Low-Rank Matrices, in: Hosseini, S., Mordukhovich, B., Uschmajew, A. (eds) Nonsmooth Optimization and Its Applications. International Series of Numerical Ma thematics, vol 170. Birkh¨ auser, Cham. See htt...

2019 doi
[85]

Halko, P .-G

N. Halko, P .-G. Martinsson, J. Tropp (2011) Finding structure with randomness: probabilis- tic algorithms for constructing approximate matrix decomp ositions, SIAM Review, 53:217- 288

2011
[86]

Hastie, R

T. Hastie, R. Mazumder, J.D. Lee, R. Zadeh (2015) Matrix Completion and Low-Rank SVD via Fast Alternating Least Squares, Journal of Machine Learning Research (JMLR), 16:3367- 3402

2015
[87]

Hansen, V

P .C. Hansen, V . Pereyra, G. Scherer (2012) Least Squares Data Fitting with Applications , The Johns Hopkins University Press, Baltimore

2012
[88]

J.H. Hong, C. Zach, A.W. Fitzgibbon (2017) Revisiting the V ariable Projection Method for Separable Nonlinear Least Squares Problems , in Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 59 39-5947

2017
[89]

J. Hu, X. Liu, Z.W. Wen, Y .X. Y uan(2020)A Brief Introduction to Manifold Optimization , in Journal of the Operations Research Society of China, 8:199- 248

2020
[90]

Ipsen, C.T

I.C.F. Ipsen, C.T. Kelley, S.R. Pope (2011) Rank-deﬁcient nonlinear least squares problems and subset selection, SIAM J. Numer. Anal., 49(3):1244-1266

2011
[91]

A. Ilin, T. Raiko (2010) Practical Approaches to Principal Component Analysis in th e Pres- ence of Missing V alues, Journal of Machine Learning Research (JMLR), 11:1957-200 0

2010
[92]

Joliffe (2002) Principal Component Analysis, Springer Series in Statistics

I.T. Joliffe (2002) Principal Component Analysis, Springer Series in Statistics

2002
[93]

Josse, F

J. Josse, F. Husson, J. Pages (2009) Gestion des donnees manquantes en Analyse en Com- posantes Principales, Journal de la Societe Francaise de Statistique, 150(2):28 -51

2009
[94]

P . Jain, P . Netrapalli, S. Sanghavi (2013) Low-rank matrix completion using alternating min- imization, in Proc. 45th Symposium on Theory of Computing (STOC). ACM, 2013, pp. 665- 674

2013
[95]

Krogh (1974) Efﬁcient implementation of a variable projection algorith m for nonlinear least squares problems, Commun

F.T. Krogh (1974) Efﬁcient implementation of a variable projection algorith m for nonlinear least squares problems, Commun. ACM 17(3):167-169

1974
[96]

Kaufman (1975) A variable projection method for solving separable nonline ar least squares problems, BIT Numer

L. Kaufman (1975) A variable projection method for solving separable nonline ar least squares problems, BIT Numer. Math. 15(1):49-57. 214

1975
[97]

Kaufman (2010) Solving separable nonlinear least squares problems with mu ltiple data sets, Exponential Data Fitting and its Applications, Bentham e- books

L. Kaufman (2010) Solving separable nonlinear least squares problems with mu ltiple data sets, Exponential Data Fitting and its Applications, Bentham e- books

2010
[98]

Kiers (1997) Weighted least squares ﬁtting using iterative ordinary lea st squares algorithms, Psychometrika 62:251:266

H.A.L. Kiers (1997) Weighted least squares ﬁtting using iterative ordinary lea st squares algorithms, Psychometrika 62:251:266

1997
[99]

H.A.L. Kiers (2002) Setting up alternating least squares and iterative majoriz ation algo- rithms for solving various matrix optimization problems , Computational Statistics and Data Analysis 41:157-170

2002
[100]

Keshavan, A

R.H. Keshavan, A. Montanari (2010) Regularization for matrix completion , in: IEEE Inter- national Symposium on Information Theory Proceedings, ISI T, 2010, IEEE, 2010, pp.1503- 1507

2010
[101]

Keshavan, A

R.H. Keshavan, A. Montanari, S. Oh (2010) Matrix completion from noisy entries , J.Mach. Learn. Res., 99:2057-2078

2010
[102]

Kaufman, G.S

L. Kaufman, G.S. Silvester (1992) Separable nonlinear least squares with multiple right- hand sides, SIAM journal on matrix analysis and applications, 13(1):6 8-89

1992
[103]

Kaufman, G.S

L. Kaufman, G.S. Silvester, M.H. Wright (1994) Structured linear least-squares problems in system identiﬁcation and separable nonlinear data ﬁtting , SIAM Journal on Optimization, 4(4):847-871

1994
[104]

Koren, R

Y . Koren, R. Bell, C. V olinsky (2009) Matrix factorization techniques for recommender sys- tems, IEEE Comput. 42(8):30-37

2009
[105]

Lyusternick(1934) Conditional extrema of functionals, Matem

L.A. Lyusternick(1934) Conditional extrema of functionals, Matem. Sb., 41(3):390-401

1934
[106]

Lee (2003) Introduction to Smooth Manifolds , Grad

J.M. Lee (2003) Introduction to Smooth Manifolds , Grad. Texts in Math. 218, Springer- V erlag, New Y ork

2003
[107]

Levenberg (1944) A method for the solution of certain non-linear problems in l east squares, Quart

K. Levenberg (1944) A method for the solution of certain non-linear problems in l east squares, Quart. Appl. Math., 2:164-168

1944
[108]

Luenberger (1973) Introduction to Linear and Nonlinear Programming , Addison- Wesley, Reading, MA

D.G. Luenberger (1973) Introduction to Linear and Nonlinear Programming , Addison- Wesley, Reading, MA

1973
[109]

Lukeman (2009) Separable overdetermined nonlinear systems: an applicati on of the Shen-Ypma Algorithm

G.G. Lukeman (2009) Separable overdetermined nonlinear systems: an applicati on of the Shen-Ypma Algorithm. VDM V erlag Dr. Muller, Saarbrucken

2009
[110]

Levin (2020) Towards optimization on varieties , Undergraduate senior thesis, Princeton University

E. Levin (2020) Towards optimization on varieties , Undergraduate senior thesis, Princeton University

2020
[111]

Lawson, R.J

C.L. Lawson, R.J. Hanson (1974) Solving Least Squares Problems, Prentice-Hall, Englewood Cliffs, NJ

1974
[112]

Levin, J

E. Levin, J. Kileel, N. Boumal (2023) Finding stationary points on bounded-rank matrices: a geometric hurdle and a smooth remedy . Mathematical Programming, 199:831-864

2023
[113]

Levin, J

E. Levin, J. Kileel, N. Boumal (2025) The effect of smooth parametrizations on nonconvex optimization landscapes. Mathematical Programming, 209:63-111

2025
[114]

Y . Li, Y . Liang, A. Risteski (2016) Recovery guarantee of weighted low rank approximation via alternating minimization, In International Conference on Machine Learning, pages 23 58- 2367, 2016

2016
[115]

Y . Luo, X. Li, A.R. Zhang (2024) Nonconvex Factorization and Manifold F ormulations Are Almost Equivalent in Low-Rank Matrix Optimization , INFORMS Journal on Optimization 0(0). See https://doi.org/10.1287/ijoo.2022.0030. 215

2024
[116]

X. Li, W. Song, N. Xiu (2019) Optimality conditions for rank-constrained matrix optimization, Journal of the Operations Research Society of China, 7:285 -301. See https://doi.org/10.1007/s40305-019-00245-0

2019 doi
[117]

Q. Li, Z. Zhu, G. Tang (2019) The non-convex geometry of low-rank matrix optimization , Information and Inference: A Journal of the IMA, 8:51-96

2019
[118]

Q. Li, Z. Zhu, G. Tang (2019) Alternating minimizations converge to second-order optim al solutions, in Proc. Int. Conf. Mach. Learn., 2019, pp. 3935-3943

2019
[119]

H. Li, G.C. Linderman, A. Szlam, K.P . Stanton, Y . Kluge r, M. Tygert (2017) Algorithm 971: An implementation of a randomized algorithm for princi pal component analysis, ACM Transactions on Mathematical Software (TOMS), 43:3, Artic le 28

2017
[120]

Marquardt(1963) An algorithm for least-squares estimation of nonlinear par ameters, Journal of the Society for Industrial and Applied Mathemati cs, 11:431-441

D.W. Marquardt(1963) An algorithm for least-squares estimation of nonlinear par ameters, Journal of the Society for Industrial and Applied Mathemati cs, 11:431-441

1963
[121]

Menzel (1985) On Solving Nonlinear Least-Squares Problems in case of Rank deﬁcient Jacobians, Computing, 34:63-72

R. Menzel (1985) On Solving Nonlinear Least-Squares Problems in case of Rank deﬁcient Jacobians, Computing, 34:63-72

1985
[122]

More (1985) The Levenberg-Marquardt algorithm: Implementation and th eory, in Lec- ture Notes in Mathematics, No

J.J. More (1985) The Levenberg-Marquardt algorithm: Implementation and th eory, in Lec- ture Notes in Mathematics, No. 630 - Numerical Analysis, G.W atson, ed., Springer-V erlag, pp. 105-116

1985
[123]

Madsen, H

K. Madsen, H. B. Nielsen (2010) Introduction to Optimization and Data Fitting , Lecture notes, Informatics and Mathematical Modelling, Technical University of Denmark, Lyngby

2010
[124]

Magnus, H

J.R. Magnus, H. Neudecker (2019) Matrix Differential Calculus with Applications in Statis- tics and Econometrics, 3rd edition

2019
[125]

Manton, R

J.H. Manton, R. Mahony, Y . Hua (2003) The geometry of weighted low-rank approximations, IEEE Transactions on Signal Processing, 51(2):500-514

2003
[126]

Marsaglia, G.P .H

G. Marsaglia, G.P .H. Styan (1974) Equalities and Inequalities for Ranks of Matrices , Linear and Multilinear Algebra, 2(3):269-292

1974
[127]

Markovsky, K

I. Markovsky, K. Usevich (2013) Structured low-rank approximation with missing data , SIAM J. Matrix Anal. Appl. 34(2):814-830

2013
[128]

Murray, J

R. Murray, J. Demmel, M.W.. Mahoney, N.B. Erichson, M. Melnichenko, O.A. Malik, L. Grigori, P . Luszczek, M. Derezinski, M.E.. Lopes, T. Lian g, H. Luo, J. Dongarra (2023) Randomized Numerical Linear Algebra : A Perspective on the F ield With an Eye to Software, arXiv:2302.114...

2023 arXiv
[129]

Mazumder, T

R. Mazumder, T. Hastie, R. Tibshirani (2010) Spectral regularization algorithms for learning large incomplete matrices, Journal of Machine Learning Research (JMLR), 11:2287-232 2

2010
[130]

Mishra, K.A

B. Mishra, K.A. Apuroop, R. Sepulchre (2012) A Riemannian geometry for low-rank matrix completion, arXiv preprint arXiv:1211.1550

2012 arXiv
[131]

Mishra, G

B. Mishra, G. Meyer, F. Bach, R. Sepulchre (2013) Low-rank optimization with trace norm penalty, SIAM J. Optim. 23(4):2124-2149

2013
[132]

Mishra, G

B. Mishra, G. Meyer, S. Bonnabel, R. Sepulchre (2014) Fixed-rank matrix factorizations and Riemannian low-rank optimization, Comput Stat 29:591-621

2014
[133]

Mishra, R

B. Mishra, R. Sepulchre (2016) Riemannian preconditioning, SIAM J. Optim. 26(1):635-660

2016
[134]

Madsen, H.B

K. Madsen, H.B. Nielsen, O. Tingleff (2004) Methods for non-linear least-squares problems, 2nd Edition, IMM, DTU, Lecture note IMM3215, 60 pp. 216

2004
[135]

Musco, C

C. Musco, C. Musco, D.P . Woodruff (2021) Simple heuristics yield provable algorithms for masked low-rank approximation, in 12th Innovations in Theoretical Computer Science Con- ference (ITCS 2021)

2021
[136]

Nielsen (2000) Separable Nonlinear Least Squares , IMM, DTU, Report MM-REP- 2000-01, 2000

H.B. Nielsen (2000) Separable Nonlinear Least Squares , IMM, DTU, Report MM-REP- 2000-01, 2000

2000
[137]

Nashed, X

M.Z. Nashed, X. Chen (1993) Convergence of Newton-like methods for singular operator equations using outer inverses, Numerische Mathematik, 66:235-257

1993
[138]

Netrapalli, U.N

P . Netrapalli, U.N.. Niranjan, S. Sanghavi, A. Anandk umar, P . Jain (2014) Non-convex ro- bust PCA , in Advances in Neural Information Processing Systems 27: A nnual Conference on Neural Information Processing Systems 2014, December 8- 13 2014, Montreal, Quebec, Canada, page...

2014
[139]

Nocedal, S.J

J. Nocedal, S.J. Wright (2006) Numerical Optimization , second edition, Springer-V erlag, Berlin

2006
[140]

Nguyen, J

L.T. Nguyen, J. Kim, B. Shim (2019) Low-rank matrix completion: A contemporary survey , IEEE Access, 7:94215-94237

2019
[141]

Osborne (1976) Nonlinear least squares - the Levenberg algorithm revisited, J

M.R. Osborne (1976) Nonlinear least squares - the Levenberg algorithm revisited, J. Austral. Math. Soc. (Series B), 19:343-357

1976
[142]

Olikier, P .A

G. Olikier, P .A. Absil (2022) On the Continuity of the Tangent Cone to the Determinantal V ariety, Set-V alued V ar. Anal 30:769-788

2022
[143]

Olikier, P .A

G. Olikier, P .A. Absil (2024) Computing Bouligand stationary points efﬁciently in low-r ank optimization, arXiv:2409.12298v1. See https://doi.org/10.48550/arXiv.2409.12298

2024 doi arXiv
[144]

Olikier, I

G. Olikier, I. Waldspurger (2024) Projected gradient descent accumulates at Bouligand sta- tionary points, arXiv:2403.02530v2. See https://doi.org/10.48550/arXiv.2403.02530

2024 doi arXiv
[145]

Olikier, K.A

G. Olikier, K.A. Gallivan, P .A. Absil (2024) Low-rank optimization methods based on projected-projected gradient descent that accumulate a t Bouligand stationary points , arXiv:2201.03962v2. See https://arxiv.org/abs/2201.03962v2

2024 arXiv
[146]

Olikier, A

G. Olikier, A. Uschmajew, B. V andereycken (2023) Gauss–Southwell type de- scent methods for low-rank matrix optimization , arXiv:2306.00897v2. See https://doi.org/10.48550/arXiv.2306.00897

2023 doi arXiv
[147]

Okatani, K

T. Okatani, K. Deguchi (2007) On the Wiberg algorithm for matrix factorization in the pres - ence of missing components , International Journal of Computer Vision (IJCV), 72(3):3 29- 337

2007
[148]

Ortega, W.C

J.M. Ortega, W.C. Rheinboldt (1970) Iterative Solution of Nonlinear Equations in Several V ariables, Academic Press, San Diego

1970
[149]

O Leary, B.W

D.P . O Leary, B.W. Rust (2013) V ariable projection for nonlinear least squares problems , Computational Optimization and Applications, 54(3):579- 593

2013
[150]

Okatani, T

T. Okatani, T. Y oshida, K. Deguchi (2011) Efﬁcient algorithm for low-rank matrix factoriza- tion with missing components and performance comparison of latest algorithms, In Proceed- ings of the 2011 IEEE International Conference on Computer V ision (ICCV), 842-849

2011
[151]

Pauwels (2013) Generic Frechet stationarity in constrained optimization , arXiv:2402.09831v2

E. Pauwels (2013) Generic Frechet stationarity in constrained optimization , arXiv:2402.09831v2. See https://arxiv.org/abs/2402.09831v2

2013 arXiv
[152]

Powell (1973) On search directions for minimization algorithms , Math

M.J.D. Powell (1973) On search directions for minimization algorithms , Math. Program- ming, 4:193-201. 217

1973
[153]

F. Pes, G. Rodriguez (2020) The minimal-norm Gauss-Newton method and some of its regu- larized variants, Electron. Trans. Numer. Anal. 53:459-480

2020
[154]

F. Pes, G. Rodriguez (2022) A doubly relaxed minimal-norm Gauss-Newton method for un- derdetermined nonlinear least-squares problems, Applied Numerical Mathematics 171:233- 248

2022
[155]

Paatero, U

P . Paatero, U. Tapper (1994) Positive matrix factorization: a non-negative factor mode l with optimal utilization of error estimates of data values , Environmetrics, 5:111-126

1994
[156]

D. Park, A. Kyrillidis, C. Carmanis, S. Sanghavi (2017 ) Non-square matrix sensing without spurious local minima via the Burer-Monteiro approach , in Proc. Artif. Intell. Statist., 2017, pp. 65-74

2017
[157]

Rennie, N

J. Rennie, N. Srebro (2005) Fast maximum margin matrix factorization for collaborativ e prediction, in Proceedings of the 22nd International Conference on Mac hine Learning, pages 713-719. ACM, 2005

2005
[158]

Ruhe (1974) Numerical computation of principal components when severa l observations are missing, Technical report, UMINF-48, Umea, Sweden

A. Ruhe (1974) Numerical computation of principal components when severa l observations are missing, Technical report, UMINF-48, Umea, Sweden

1974
[159]

Reddien (1980) Newton’s method and high order singularities , Comput

G.W. Reddien (1980) Newton’s method and high order singularities , Comput. Math. Appl., 5:79-86

1980
[160]

Ruszczynski (2006) Nonlinear Optimization

A. Ruszczynski (2006) Nonlinear Optimization. Princeton University Press

2006
[161]

Rabier, G.W

P .J. Rabier, G.W. Reddien (1986) Characterization and computation of singular points with maximum rank deﬁciency , SIAM J. Numer. Anal. 23:1040-1051

1986
[162]

Rebjock, N

Q. Rebjock, N. Boumal (2024) Fast convergence of trust-regions for non-isolated minimavia analysis of CG on indeﬁnite matrices . Math. Program., online

2024
[163]

Rebjock, N

Q. Rebjock, N. Boumal (2024) Fast convergence to non-isolated minima: four equivalent conditions forC 2 functions. Math. Program., online

2024
[164]

Robbin, D.A

J.W. Robbin, D.A. Salomon (2022) Introduction to differential geometry, Springer Spektrum Berlin, Heidelberg, 418 pp

2022
[165]

Rockafellar, R.J.B

R.T. Rockafellar, R.J.B. Wets (1998) V ariational Analysis, vol. 317 of Grundlehren der math- ematischen Wissenschaften, Springer-V erlag Berlin Heidelberg. Corrected 3rd printing 2009

1998
[166]

Ruhe, P .-A

A. Ruhe, P .-A. Wedin (1980) Algorithms for Separable Nonlinear Least Squares Problems , SIAM Review, 22(3):318-337

1980
[167]

Razenshteyn, Z

I. Razenshteyn, Z. Song, D.P . Woodruff (2016) Weighted Low Rank Approximations with Provable Guarantees, STOC’16, June 19-21, 2016, Cambridge, MA, USA

2016
[168]

Sukhinin (1973) Conditional extrema of functionals in topological linear s paces, Math- ematical Notes of the Academy of Sciences of the USSR 14:775- 779

M.F. Sukhinin (1973) Conditional extrema of functionals in topological linear s paces, Math- ematical Notes of the Academy of Sciences of the USSR 14:775- 779

1973
[169]

Simonsson, L

L. Simonsson, L. Elden (2010) Grassmann algorithms for low rank approximation of matri- ces with missing values , BIT Numer. Math., 50:173-191

2010
[170]

Sun, Z.Q

R. Sun, Z.Q. Luo (2016) Guaranteed Matrix Completion via Non-Convex Factorizatio n, in IEEE Transactions on Information Theory, vol. 62, no. 11, pp . 6535-6579, Nov. 2016

2016
[171]

Srebro, T

N. Srebro, T. Jaakkola (2004) Weighted low-rank approximations, in Proceedings of the 20th International Conference on Machine Learning, pp. 720-727

2004
[172]

Stewart, J

G.W. Stewart, J. Sun (1990) Matrix perturbation Theory, Academic Press, INC, New Y ork. 218

1990
[173]

Schneider, A

S. Schneider, A. Uschmajew (2015) Convergence results for projected line-search methods on varieties of low-rank matrices via Lojasiewicz inequali ty, SIAM Journal on Optimization, 25:622-646

2015
[174]

Schramm, B

T. Schramm, B. Weitz (2015) Low-rank matrix completion with adversarial missing entri es, arXiv:1506.03137v1. See https://arxiv.org/abs/1506.03137v1

2015 arXiv
[175]

Shen, T.J

Y . Shen, T.J. Ypma (2019) Solving separable nonlinear least squares problems using t he QR factorization. J. Comput. Appl. Math. 345:48-58

2019
[176]

H. Shum, K. Ikeuchi, R. Reddy (1995) Principal component analysis with missing data and its application to polyhedral object modeling , IEEE Transaction on Pattern Analysis and Machine Intelligence, 17(9):855-867

1995
[177]

Srebro, J

N. Srebro, J. Rennie, T. Jaakkola (2005) Maximum-margin matrix factorization, in Advances in Neural Information Processing Systems 17 (NIPS 2004) pp. 1329-1336. MIT Press

2005
[178]

Szlam, A

A. Szlam, A. Tulloch, M. Tygert (2017) Accurate low-rank approximations via a few it- erations of alternating least squares , SIAM Journal on Matrix Analysis and Applications, 38(2):425-433

2017
[179]

Terray (1995) Space/Time structure of monsoons interannual variability , Journal of Cli- mate, 8:2595-2619

P . Terray (1995) Space/Time structure of monsoons interannual variability , Journal of Cli- mate, 8:2595-2619

1995
[180]

P . Terray (2002) Application of Weighted Empirical Orthogonal Function Ana lysis to ship’s datasets, Compte-Rendu de la IV` eme journ´ ee Statistique IPSL (Clas siﬁcation et Analyse spatiale), NAI no 23. pp. 11-28. ISSN 1626-8334

2002
[181]

Tuzhilina, T

E. Tuzhilina, T. Hastie (2021) Weighted Low Rank Matrix Approximation and Acceleration , arXiv:2109.11057v1. See https://arxiv.org/abs/2109.11057v1

2021 arXiv
[182]

Usevich, I

K. Usevich, I. Markovsky (2014) V ariable projection methods for afﬁnely structured low- rank approximation in weighted 2-norms , Journal of Computational and Applied Mathemat- ics, 272:430-448

2014
[183]

Uschmajew, R

A. Uschmajew, R. V andereycken (2014) Line-search methods and rank increase on low-rank matrix varieties, in: 2014 International Symposium on Nonlinear Theory and i ts Applications (NOLTA2014), Luzern, Switzerland, September 14-18, 2014, 52-55 (IEICE: Luzern, 2014)

2014
[184]

Uschmajew, R

A. Uschmajew, R. V andereycken (2015) Greedy rank updates combined with Riemannian descent methods for low-rank optimization , in: 2015 International Conference on Sampling Theory and Applications (SampTA), 25-29 May 2015, Washington, DC, USA, ed. by Stephen Casey, Kevin Duke,...

2015
[185]

V andereycken (2013) Low-rank matrix completion by Riemannian optimization-ex tended version, arXiv:1209.3834v1

R. V andereycken (2013) Low-rank matrix completion by Riemannian optimization-ex tended version, arXiv:1209.3834v1. See https://arxiv.org/abs/1209.3834v1

2013 arXiv
[186]

V andereycken (2013) Low-rank matrix completion by Riemannian optimization , SIAM J

R. V andereycken (2013) Low-rank matrix completion by Riemannian optimization , SIAM J. Optim. 23(2):1214-1236

2013
[187]

Vidal, Y

R. Vidal, Y . Ma, S.S. Sastry (2016) Generalized Principal Component Analysis , New Y ork, NY: Springer New Y ork, DOI: 10.1007/978-0-387-87811-9. ISSN: 21969973: 09396047

2016 doi
[188]

Z. Wen, W. Yin, Y . Zhang (2012) Solving a low-rank factorization model for matrix comple- tion by a nonlinear successive over-relaxation algorithm, Math. Program. Comput. 4(4):333- 361

2012
[189]

Wedin (1973) Perturbation Theory for Pseudo-Inverses, BIT, 13:217-232

P .A. Wedin (1973) Perturbation Theory for Pseudo-Inverses, BIT, 13:217-232

1973
[190]

Wiberg (1976) Computation of principal components when data are missing , in Proceed- ings of the 2nd Symposium of Computational Statistics, page s 229-326, 1976

T. Wiberg (1976) Computation of principal components when data are missing , in Proceed- ings of the 2nd Symposium of Computational Statistics, page s 229-326, 1976. 219

1976
[191]

Wold (1966) Nonlinear estimation by iterative least squares procedure s, Research Papers in Statistics ( F

H. Wold (1966) Nonlinear estimation by iterative least squares procedure s, Research Papers in Statistics ( F. N. David Ed.), pp. 411-444, New Y ork: Wiley

1966
[192]

H. Wold, E. Lyttkens (1969) Nonlinear Iterative Partial Least Squares (NIPALS) Estima tion Procedures, Bulletin of the International Statistical Institute, 43: 29-51

1969
[193]

L. Wang, X. Zhang, Q. Gu (2017) A uniﬁed computational and statistical framework for nonconvex low-rank matrix estimation, in Proc. Artif. Intell. Statist., 2017, pp. 981-990

2017
[194]

Y . Xu, W. Yin (2013) A block coordinate descent method for regularized multicon vex opti- mization with applications to nonnegative tensor factorization and completion, SIAM Journal on imaging sciences, 6(3):1758-1789

2013
[195]

Y amamoto (1989)Uniqueness of the Solution in a Kantorovich-Type Theorem of Hausler for the Gauss-Newton Method , Japan J

Y . Y amamoto (1989)Uniqueness of the Solution in a Kantorovich-Type Theorem of Hausler for the Gauss-Newton Method , Japan J. Appl. Math., 6:77-81

1989
[196]

Y amashita, M

N. Y amashita, M. Fukushima (2001) On the rate of convergence of the Levenberg–Marquardt method. In: Topics in Numerical Analysis, pp 239-249. Springer

2001
[197]

Y ang, L.H

W.H. Y ang, L.H. Zhang, R. Song (2014) Optimality conditions for the nonlinear program- ming problems on riemannian manifolds , Paciﬁc Journal of Optimization, 10(2):415-434

2014
[198]

Zhou (2015) Rank-contrained optimization: A Riemannian manifold appr oach, Ph.D

G. Zhou (2015) Rank-contrained optimization: A Riemannian manifold appr oach, Ph.D. thesis, Florida State University

2015
[199]

Zeng (2024) A Newton’s Iteration Converges Quadratically to Non-isolated Solutions Too, arXiv:2101.09180v4

Z. Zeng (2024) A Newton’s Iteration Converges Quadratically to Non-isolated Solutions Too, arXiv:2101.09180v4. See https://arxiv.org/abs/2101.09180v4

2024 arXiv
[200]

Z. Zhu, Q. Li, G. Tang, M.B. Wakin (2018) Global optimality in low-rank matrix optimiza- tion, IEEE Transactions on Signal Processing, 66(13):3614-362 8

2018
[201]

Z. Zhu, Q. Li, G. Tang, M.B. Wakin (2021) The Global Optimization Geometry of low-rank matrix optimization, IEEE Transactions on Information Theory, 67(2):1308-133 1. 220

2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Absil, J

P .-A. Absil, J. Malick (2012) Projection-like retractions on matrix manifolds , SIAM J. Op- tim., 22:135-158

2012

[2] [2]

Allgower, K

E.L. Allgower, K. Bohmer, A. Hoy, V . Janovsky (1999) Direct methods for solving singular nonlinear equations, Z. Angew. Math. Mech. 79(4):219-231

1999

[3] [3]

Absil, R

P .-A. Absil, R. Mahony, R. Sepulchre (2008) Optimization Algorithms on Matrix Manifolds , Princeton University Press, Princeton, NJ

2008

[4] [4]

Attouch, J

H. Attouch, J. Bolte, P . Redont, A. Soubeyran (2010) Proximal alternating minimization and projection methods for nonconvex problems: An approach based on the Kurdyka-Lojasiewicz inequality, Mathematics of Operations Research, 35(2):438-457

2010

[5] [5]

Ben-Israel (1965) A modiﬁed Newton-Raphson method for the solution of systems of equa- tions, Israel Journal of Mathematics, 3:94-98

A. Ben-Israel (1965) A modiﬁed Newton-Raphson method for the solution of systems of equa- tions, Israel Journal of Mathematics, 3:94-98

1965

[6] [6]

Ben-Israel (1966) A Newton-Raphson method for the solution of systems of equat ions, Journal of Mathematical Analysis and Applications, 15:243 -252

A. Ben-Israel (1966) A Newton-Raphson method for the solution of systems of equat ions, Journal of Mathematical Analysis and Applications, 15:243 -252

1966

[7] [7]

Bertsekas (1999) Nonlinear Programming, 2nd Edition, Athena Scientiﬁc, Belmont, Massachusetts

D.P . Bertsekas (1999) Nonlinear Programming, 2nd Edition, Athena Scientiﬁc, Belmont, Massachusetts

1999

[8] [8]

Bjorck (2015) Numerical Methods in Matrix Computations , Series: Texts in Applied Mathematics, V ol

A. Bjorck (2015) Numerical Methods in Matrix Computations , Series: Texts in Applied Mathematics, V ol. 59, Springer, 800 p

2015