Pith. sign in

REVIEW 3 major objections 4 minor 36 references

Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that the landing algorithm, an infeasible retraction-free update for orthogonality-constrained problems, converges linearly near a local minimum under a local Riemannian PŁ condition.

desk verdict The paper proves a new local linear rate for the landing algorithm under local Riemannian PŁ, but the main theorem depends on an unproved imported lemma that transfers PŁ from the manifold to off-manifold points, so the result is conditional on that missing bridge. read the letter →

arxiv 2412.05689 v1 pith:7TCDUOUK submitted 2024-12-07 math.OC cs.LG

Youbang Sun , Shixiang Chen , Alfredo Garcia , Shahin Shahrampour This is my paper · ORCID
classification math.OCcs.LG MSC 90C2690C3065K05
keywords orthogonalityconstraintsStiefelmanifoldlandingalgorithmretraction-freeoptimizationinfeasiblemethodsRiemannianPolyak-Łojasiewiczconditionlinearconvergencenonconvex
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Optimization with orthogonality constraints is often solved by retraction-based Riemannian methods, but each retraction costs a matrix exponential, inversion, or square root. This paper studies the landing algorithm, an infeasible retraction-free alternative whose iterates are allowed to drift slightly off the Stiefel manifold (the set of matrices with $x^\top x=I_r$) and then settle back onto it. The paper proves a local linear convergence rate: under Euclidean smoothness and a local Riemannian Polyak–Łojasiewicz condition near the set of local minima, the merit function contracts by a fixed factor every iteration once the iterates enter a specified basin. The rate is $L(x_k)\le(1-\alpha\rho\mu'/2)^kL(x_0)$, and the distance to the optimum set and the landing-field norm shrink at the same exponential pace. This matters because the result explains the fast empirical convergence reported for the landing algorithm and turns a cheap per-iteration update into a provably fast solver for PCA, low-rank approximation, and orthogonality-constrained neural networks.

What carries the argument

The central object is the landing field $\Lambda(x)=\operatorname{grad}f(x)+\lambda\nabla p(x)$, where $p(x)=\frac14\|x^\top x-I_r\|_F^2$ penalizes constraint violation; on the manifold it reduces to the Riemannian gradient, and its two terms are mutually orthogonal. The proof rides on the merit function $L(x)=f(x)+h(x)+\gamma p(x)$ with $h(x)=-\frac12\langle\operatorname{sym}(x^\top\nabla f(x)),\,x^\top x-I_r\rangle$, which is designed so that the landing field is an ascent direction for $L$. Three inequalities carry the argument: Proposition 3.3 gives $\langle\Lambda,\nabla L\rangle\ge\rho\|\Lambda\|_F^2$, Lemma 3.4 (pseudo-gradient domination) gives $L(x)\le(1/\mu')\|\Lambda(x)\|_F^2$ in the safety region, and Lemma 3.5 converts that into the quadratic-growth bound $L(x)\ge(\mu'\rho^2/4)\operatorname{dist}(S,x)^2$. Together with a Lipschitz bound on $\Lambda$ and the safety step-size condition $\alpha\le\min\{\rho/L',\alpha_{\mathrm{safe}}\}$, these make one landing step a contraction on $L$ that also stays inside the basin.

What would settle it

Choose an $L$-smooth objective that satisfies the local Riemannian PŁ condition on the Stiefel manifold and simulate the landing algorithm from an initial point with $L(x_0)\le\mu'\rho^2\delta^2/16$; if any iterate inside the safety region violates $L(x_{k+1})\le(1-\alpha\rho\mu'/2)L(x_k)$, or if an off-manifold point in $\operatorname{St}(d,r)_\epsilon\cap D(S,\delta)$ satisfies $L(x)>(1/\mu')\|\Lambda(x)\|_F^2$, then Lemma 3.4 and Theorem 3.6 fail. A single such example would settle the question.

Watch

Extended reading notes

Core claim

The paper claims that the landing algorithm, which replaces the retraction step with the Euclidean update $x_{k+1}=x_k-\alpha(\operatorname{grad}f(x_k)+\lambda x_k(x_k^\top x_k-I_r))$, converges locally at a linear rate whenever the objective is twice continuously differentiable and $L$-smooth in the whole space and satisfies the Riemannian PŁ inequality $f(x)-f_S^*\le(1/2\mu)\|\operatorname{grad}f(x)\|_F^2$ in a $2\delta$-neighborhood of the local-minimum set on the Stiefel manifold. The proof tracks the merit function $L(x)=f(x)+h(x)+\gamma p(x)$ rather than $f$ itself, and shows that with a safe step size and an initial merit value below $\mu'\rho^2\delta^2/16$, the merit function decays as $(1-\alpha\rho\mu'/2)^k$, while the iterates never leave the safety region $\operatorname{St}(d,r)_\epsilon\cap D(S,\delta)$. The same mechanism yields linear rates for the distance to the minimum set and for the landing-field norm.

Load-bearing premise

The linear-rate proof requires that for every point in the off-manifold safety region near a local minimum the merit function is bounded by a constant times the squared landing-field norm, a transfer of the on-manifold PŁ condition to nearby infeasible points that the paper imports from its companion work.

Editorial extensions

If this is right

  • If Theorem 3.6 is correct, every iterate started in the basin $D(S,\delta)\cap\operatorname{St}(d,r)_\epsilon$ with $L(x_0)\le\mu'\rho^2\delta^2/16$ satisfies $L(x_k)\le(1-\alpha\rho\mu'/2)^kL(x_0)$, an exponential contraction in the merit function.
  • Since each landing step is a Euclidean update, the per-iteration cost is about one gradient evaluation plus the low-rank penalty term, avoiding the $O(dr^2)$ or heavier retraction cost, so the runtime advantage over retraction-based methods grows with the number of columns $r$.
  • The distance to the local-minimum set and the landing-field norm inherit linear rates from the merit function via quadratic growth and the Lipschitz property of $\Lambda$.
  • The assumptions cover standard quadratic-type Stiefel problems such as principal component analysis, so the linear rate applies to a common machine-learning task, not only to constructed examples.
  • The safety step size keeps all iterates within the prescribed constraint-violation tolerance, so the infeasible path stays controlled while the objective declines exponentially.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the same proof template should transfer to retraction-free updates on other embedded constraint sets, such as Grassmann or hyperbolic manifolds, whenever a matching off-manifold pseudo-gradient domination inequality can be established.
  • Editorial extension: for a quadratic objective the constants $\mu'$, $\rho$, and $L'$ can be computed in closed form, so the theorem predicts an explicit contraction factor; measuring the empirical ratio $L(x_{k+1})/L(x_k)$ in the PCA experiment would test that prediction directly.
  • Editorial extension: the theorem is stated for the merit function $L$, and the paper only partially spells out the transfer to the original objective $f$; wherever the correction term $h$ is small near the optimum set, the same exponential statement applies to $f$ itself.
  • Editorial extension: in stochastic/minibatch settings, a local PŁ condition together with bounded gradient noise would plausibly give linear convergence down to a noise floor, which would match the multi-stage learning-rate-decay behavior seen in the CNN experiments, though that extension requires separate analysis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies the landing algorithm, a retraction-free infeasible method for optimization over the Stiefel manifold, and proves a local linear convergence rate for smooth non-convex objectives satisfying a local Riemannian Polyak-Łojasiewicz condition. The main result, Theorem 3.6, states that if the merit function L(x) = f(x) + h(x) + γp(x) is initially sufficiently small in a neighborhood of the local-minimum set S, then with a suitable step size the merit function converges linearly: L(x_k) ≤ (1 - αρμ'/2)^k L(x_0). The proof combines a descent estimate for the merit function, a pseudo-gradient domination lemma, and a quadratic-growth property of L. Numerical experiments on PCA and on CNN training with orthogonality-constrained layers illustrate the computational efficiency of the landing algorithm relative to retraction-based and other retraction-free methods.

Significance. If Theorem 3.6 is correct, it gives a clean explanation of the fast empirical convergence observed for the landing algorithm and extends the global finite-time results of Ablin et al. [2] to a local linear rate under a condition weaker than geodesic strong convexity. The paper is honest about the local nature of the result and does not fit parameters; the rate is a theorem and the PŁ constant is an assumption. The PCA experiment code is publicly available, and the numerical comparisons cover both iteration count and wall-clock time. However, the central proof depends on Lemma 3.4, which is imported from the authors' prior work [28] and is not proved here, and the safety-step argument uses a uniform gradient bound G that is not stated among the theorem's hypotheses. These issues make the main result conditional on an unverified bridge between manifold-only PŁ and off-manifold merit domination.

major comments (3)
  1. [Section III.C, Lemma 3.4 and Theorem 3.6] Lemma 3.4 is the only channel through which Assumption 3.2, a Riemannian PŁ inequality stated only on St(d,r) ∩ D(S,2δ), controls the merit function at infeasible points x ∈ St(d,r)_ε ∩ D(S,δ). The lemma is cited from [28] but neither proved nor accompanied by the exact hypotheses under which it was derived there. In particular, the manuscript does not state the smallness relations among δ, ε, λ, γ, μ, L, L', and Ĝ that would make the transfer from the manifold to the off-manifold region valid, and it does not show that Assumptions 3.1 and 3.2 alone imply inequality (9). Since the proof of Theorem 3.6 invokes Lemma 3.4 in the crucial step L(x_{k+1}) ≤ (1 - μ'(αρ - α²L'/2))L(x_k), the main theorem is not verifiable from the present text. This is a load-bearing gap: if Lemma 3.4 requires an additional condition or a different proof, Theorem 3.6 as stated is missing an assumption.
  2. [Proposition 2.2 and Theorem 3.6] The theorem chooses α ≤ min{ρ/L', α_safe}, but α_safe in Proposition 2.2 depends on a uniform bound G on ||grad f(x)||, and G is not defined in the hypotheses of Theorem 3.6. Since L-smoothness on all of R^{d×r} does not imply a global bound on the gradient, the step-size condition is not well-posed as written. A fix would be to define G as a supremum over the compact set St(d,r)_ε ∩ D(S,δ) and state it as part of the theorem, but this is absent. The same gap affects the induction step, which relies on Proposition 2.2 to keep all iterates in the safety region.
  3. [Section III.C, proof of Theorem 3.6] The induction step uses the inequality ||Λ(x_k)||_F ≤ L' dist(S, x_k) to control dist(S, x_{k+1}). This requires that Λ be Lipschitz on a neighborhood containing S and that Λ vanish on S. The manuscript defines L' as a maximum of Ĝ, L_Λ, and L_L, but it does not explicitly state the domain on which the Lipschitz constant L_Λ applies or verify that Λ(y)=0 for y∈S. The argument is likely repairable, but as written the constants and domains are not fully specified.
minor comments (4)
  1. [Abstract] The phrase "principle component analysis" should read "principal component analysis."
  2. [Section II.C] The line "⟨gradf(x), ∇p(x)⟩ = ⟨skew(∇f(x)x^T), x(x^T x - I)x^T⟩ = 0" appears to have a typo in the right-hand side; the second argument should likely be x(x^T x - I), not x(x^T x - I)x^T. Please check the expression.
  3. [Section III.B] The informal statement "if we set λ = O(L') ... μ' ≈ μ" is not justified and could be misleading; please either remove it or give a precise quantitative statement.
  4. [Section III.C] The proof of Lemma 3.5 invokes Proposition 2.2 of [24] to pass from a local Euclidean PŁ condition on L to quadratic growth. Since [24] may have additional regularity assumptions, it would be helpful to state explicitly that L is C² and to verify the hypotheses of that proposition.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the linear-rate theorem follows from a descent-plus-PL argument, but the proof imports the key off-manifold domination Lemma 3.4 without proof from the authors' own prior preprint, which is a self-containedness gap rather than definitional circularity.

full rationale

I walked the derivation chain: Assumption 3.1 (Euclidean L-smoothness) and Assumption 3.2 (local Riemannian PL on the Stiefel manifold) enter through Proposition 3.3 and Lemma 3.4, and Theorem 3.6 is an induction combining the merit-function descent inequality with the domination inequality L(x) <= (1/mu') ||Lambda(x)||^2. No fitted parameter is renamed as a prediction: the contraction factor (1 - alpha rho mu'/2)^k is an algebraic consequence, and mu is an assumption, not a fitted value. The one item requiring scrutiny is Lemma 3.4, labeled 'Pseudo Gradient Domination [28]', which is stated without proof and cited to the authors' own prior work. This lemma is the sole bridge that transfers the manifold-only PL condition to off-manifold safety-region points, and the present text does not state the exact hypotheses under which [28] proves it nor the relations among delta, epsilon, lambda, and the smoothness constants needed for the transfer. That makes Theorem 3.6 conditional on an unstated prior result, so the paper is not fully self-contained. However, this is not circularity in the sense of the target result being an input: the paper does not define L, Lambda, or mu' in terms of the claimed convergence rate, and no equation is constructed so that the conclusion holds by definition. Citing one's own earlier lemma is legitimate reliance unless that lemma itself presupposes the theorem under review; nothing in the text shows such a cycle. I therefore flag the missing-proof issue but do not count it as a circular step; the self-citation is load-bearing, so I do not award 0, but it falls at the low end of the circularity scale.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

No parameters are fitted to data; α, λ, and γ are user-chosen or existence constants, not calibrated to the numerical plots. The theorem relies on external results: smoothness and PŁ assumptions on f, properties of the merit function from [2], the pseudo-gradient domination lemma from [28], a safety step-size proposition from [2], and the PL-implies-quadratic-growth proposition from [24].

free parameters (3)
  • Penalty coefficient λ
    Constant in the landing field Λ(x) = grad f(x) + λ x(x^T x - I); no value is fit to data. The proof requires a safe-step condition but no specific numerical λ.
  • Merit penalty γ
    Chosen to satisfy the lower bound in Eq. (8) from [2]; this is an existence condition, not a fit.
  • Step size α
    User-chosen step satisfying α ≤ min{ρ/L', α_safe}; it determines the contraction factor but is not fit to data.
assumptions (6)
  • domain assumption f is twice continuously differentiable and L-smooth on the whole Euclidean space (Assumption 3.1, Eq. (5)).
    Invoked in the descent lemma for the merit function and in Lipschitz estimates for the landing field; this is stronger than smoothness on the manifold alone.
  • domain assumption f satisfies the local Riemannian PŁ condition with factor μ on St(d,r) ∩ D(S,2δ) (Assumption 3.2, Eq. (6)).
    This is the driving condition for the linear contraction. It is not verified for arbitrary smooth non-convex f; it holds for problems such as PCA and generalized quadratic objectives.
  • standard math The merit function L in Eq. (7) is LL-smooth and satisfies ⟨Λ,∇L⟩ ≥ ρ||Λ||² on the safety region (Proposition 3.3 from [2]).
    Used in the main proof to turn landing-field updates into descent on L; the result is imported from prior work and not reproved.
  • standard math Pseudo-gradient domination L(x) ≤ (1/μ')||Λ(x)||² for x ∈ St_ε ∩ D(S,δ) (Lemma 3.4 from [28]).
    This lemma bridges the manifold-only PŁ assumption to the off-manifold landing field and is the load-bearing step of the proof; it is cited to the authors' own prior paper without proof in the present manuscript.
  • standard math A safe step size α_safe exists under a uniform bound G on ||grad f|| in the safety region (Proposition 2.2 from [2]).
    Used to keep iterates inside St_ε; Theorem 3.6 invokes α_safe but does not explicitly state the bound G in its hypotheses.
  • standard math Local Euclidean PŁ for L implies quadratic growth dist(S,x)^2 ≤ (4/(μ'ρ²)) L(x) (Proposition 2.2 in [24]).
    Used in Lemma 3.5 to lower-bound the merit function; the transformation from gradient domination to quadratic growth is imported from the cited paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints." pith.science (2026). https://pith.science/paper/7TCDUOUK

@misc{pith2026241205689,
  author       = {Pith},
  title        = {Pith review of: Local Linear Convergence of Infeasible Optimization with Orthogonal Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7TCDUOUK}},
  note         = {Machine review of arXiv:2412.05689}
}
read the original abstract

Many classical and modern machine learning algorithms require solving optimization tasks under orthogonality constraints. Solving these tasks with feasible methods requires a gradient descent update followed by a retraction operation on the Stiefel manifold, which can be computationally expensive. Recently, an infeasible retraction-free approach, termed the landing algorithm, was proposed as an efficient alternative. Motivated by the common occurrence of orthogonality constraints in tasks such as principle component analysis and training of deep neural networks, this paper studies the landing algorithm and establishes a novel linear convergence rate for smooth non-convex functions using only a local Riemannian P{\L} condition. Numerical experiments demonstrate that the landing algorithm performs on par with the state-of-the-art retraction-based methods with substantially reduced computational overhead.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 30 canonical work pages

  1. [28]

    Global convergence of decentralized retraction-free optimization on the stiefel manifold

    Youbang Sun, Shixiang Chen, Alfredo Garcia, and Shahin Shahrampour. Global convergence of decentralized retraction-free optimization on the stiefel manifold. arXiv preprint arXiv:2405.11590 , 2024

    arXiv 2024

  2. [2]

    Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints

    Pierre Ablin, Simon Vary, Bin Gao, and P-A Absil. Infeasible deter- ministic, stochastic, and variance-reduction algorithms for optimization under orthogonality constraints. arXiv preprint arXiv:2303.16510, 2023

    work page Pith review arXiv 2023

  3. [1]

    Fast and accurate optimization on the orthogonal manifold without retraction

    Pierre Ablin and Gabriel Peyr ´e. Fast and accurate optimization on the orthogonal manifold without retraction. In International Conference on Artificial Intelligence and Statistics , pages 5636–5657. PMLR, 2022

    work page 2022

  4. [3]

    Optimization algorithms on matrix manifolds

    P-A Absil, Robert Mahony, and Rodolphe Sepulchre. Optimization algorithms on matrix manifolds . Princeton University Press, 2008

    work page 2008

  5. [4]

    Projection-like retractions on matrix manifolds

    P-A Absil and J ´erˆome Malick. Projection-like retractions on matrix manifolds. SIAM Journal on Optimization , 22(1):135–158, 2012

    work page 2012

  6. [5]

    A First-Order Numerical Algorithm without Matrix Operations

    Muhammad Adil, Ramtin Madani, Sasan Tavakkol, and Ali Davoudi. A first-order numerical algorithm without matrix operations. arXiv preprint arXiv:2203.05027, 2022

    work page Pith review arXiv 2022

  7. [6]

    Unitary evolution recurrent neural networks

    Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. In International Conference on Machine Learning, pages 1120–1128. PMLR, 2016

    work page 2016

  8. [7]

    Beyond linearization: On quadratic and higher-order approximation of wide neural networks

    Yu Bai and Jason D Lee. Beyond linearization: On quadratic and higher-order approximation of wide neural networks. In International Conference on Learning Representations , 2019

    work page 2019

Show all 36 references
  1. [8]

    Can we gain more from orthogonality regularizations in training deep networks? Advances in Neural Information Processing Systems , 31, 2018

    Nitin Bansal, Xiaohan Chen, and Zhangyang Wang. Can we gain more from orthogonality regularizations in training deep networks? Advances in Neural Information Processing Systems , 31, 2018

    2018

  2. [9]

    Riemannian adaptive optimization methods

    Gary B ´ecigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods. In International Conference on Learning Rep- resentations, 2019

    2019

  3. [10]

    Stochastic first-order methods for convex and nonconvex functional constrained optimization

    Digvijay Boob, Qi Deng, and Guanghui Lan. Stochastic first-order methods for convex and nonconvex functional constrained optimization. Mathematical Programming, 197(1):215–279, 2023

    2023

  4. [11]

    Boumal, P.-A

    N. Boumal, P.-A. Absil, and C. Cartis. Global rates of convergence for nonconvex optimization on manifolds. IMA Journal of Numerical Analysis, 39(1):1–33, 2019

    2019

  5. [12]

    Reducing overfitting in deep networks by decorrelating representations

    Michael Cogswell, Faruk Ahmed, Ross Girshick, Larry Zitnick, and Dhruv Batra. Reducing overfitting in deep networks by decorrelating representations. arXiv preprint arXiv:1511.06068 , 2015

    2015 arXiv

  6. [13]

    The geometry of algorithms with orthogonality constraints

    Alan Edelman, Tom ´as A Arias, and Steven T Smith. The geometry of algorithms with orthogonality constraints. SIAM journal on Matrix Analysis and Applications , 20(2):303–353, 1998

    1998

  7. [14]

    Nonlinear programming without a penalty function

    Roger Fletcher and Sven Leyffer. Nonlinear programming without a penalty function. Mathematical programming, 91:239–269, 2002

    2002

  8. [15]

    Parallelizable algorithms for optimization problems with orthogonality constraints

    Bin Gao, Xin Liu, and Ya-xiang Yuan. Parallelizable algorithms for optimization problems with orthogonality constraints. SIAM Journal on Scientific Computing, 41(3):A1949–A1983, 2019

    2019

  9. [16]

    Riemannian hamiltonian methods for min-max optimization on manifolds

    Andi Han, Bamdev Mishra, Pratik Jawanpuria, Pawan Kumar, and Junbin Gao. Riemannian hamiltonian methods for min-max optimization on manifolds. SIAM Journal on Optimization , 33(3):1797–1827, 2023

    2023

  10. [17]

    Analysis of a complex of statistical variables into principal components

    Harold Hotelling. Analysis of a complex of statistical variables into principal components. Journal of educational psychology , 24(6):417, 1933

    1933

  11. [18]

    Canonical correlation analysis (cca)

    Harold Hotelling. Canonical correlation analysis (cca). Journal of Educational Psychology, 10:12913–2, 1935

    1935

  12. [19]

    A decentralized algorithm for spectral analysis

    David Kempe and Frank McSherry. A decentralized algorithm for spectral analysis. Journal of Computer and System Sciences , 74(1):70– 83, 2008

    2008

  13. [20]

    Literature survey on low rank ap- proximation of matrices

    N Kishore Kumar and Jan Schneider. Literature survey on low rank ap- proximation of matrices. Linear and Multilinear Algebra , 65(11):2212– 2244, 2017

    2017

  14. [21]

    Efficient riemannian optimiza- tion on the stiefel manifold via the cayley transform

    Jun Li, Fuxin Li, and Sinisa Todorovic. Efficient riemannian optimiza- tion on the stiefel manifold via the cayley transform. In International Conference on Learning Representations , 2020

    2020

  15. [22]

    Huikang Liu, Anthony Man-Cho So, and Weijie Wu. Quadratic op- timization with orthogonality constraint: explicit łojasiewicz exponent and linear convergence of retraction-based line-search and stochas- tic variance-reduced gradient methods. Mathematical Programming , 178:215–262, 2019

    2019

  16. [23]

    Cloud k-svd: A collaborative dic- tionary learning algorithm for big, distributed data

    Haroon Raja and Waheed U Bajwa. Cloud k-svd: A collaborative dic- tionary learning algorithm for big, distributed data. IEEE Transactions on Signal Processing , 64(1):173–188, 2015

    2015

  17. [24]

    Fast convergence to non-isolated minima: four equivalent conditions for c 2 functions

    Quentin Rebjock and Nicolas Boumal. Fast convergence to non-isolated minima: four equivalent conditions for c 2 functions. Mathematical Programming, pages 1–49, 2024

    2024

  18. [25]

    A dai–yuan-type riemannian conjugate gradient method with the weak wolfe conditions

    Hiroyuki Sato. A dai–yuan-type riemannian conjugate gradient method with the weak wolfe conditions. Computational optimization and Applications, 64:101–118, 2016

    2016

  19. [26]

    Orthogonal directions constrained gradient method: from non- linear equality constraints to stiefel manifold

    Sholom Schechtman, Daniil Tiapkin, Michael Muehlebach, and Eric Moulines. Orthogonal directions constrained gradient method: from non- linear equality constraints to stiefel manifold. In The Thirty Sixth Annual Conference on Learning Theory , pages 1228–1258. PMLR, 2023

    2023

  20. [27]

    Convergence results for projected line-search methods on varieties of low-rank matrices via łojasiewicz inequality

    Reinhold Schneider and Andr ´e Uschmajew. Convergence results for projected line-search methods on varieties of low-rank matrices via łojasiewicz inequality. SIAM Journal on Optimization , 25(1):622–646, 2015

    2015

  21. [29]

    Orthogonalizing convolutional layers with the cayley transform

    Asher Trockman and J Zico Kolter. Orthogonalizing convolutional layers with the cayley transform. In International Conference on Learning Representations, 2021

    2021

  22. [30]

    On orthogonality and learning recurrent networks with long term de- pendencies

    Eugene V orontsov, Chiheb Trabelsi, Samuel Kadoury, and Chris Pal. On orthogonality and learning recurrent networks with long term de- pendencies. In International Conference on Machine Learning , pages 3570–3578. PMLR, 2017

    2017

  23. [31]

    Solving optimization problems over the stiefel manifold by smooth exact penalty function

    Nachuan Xiao and Xin Liu. Solving optimization problems over the stiefel manifold by smooth exact penalty function. arXiv preprint arXiv:2110.08986, 2021

    2021 arXiv

  24. [32]

    Dissolving constraints for riemannian optimization

    Nachuan Xiao, Xin Liu, and Kim-Chuan Toh. Dissolving constraints for riemannian optimization. Mathematics of Operations Research , 49(1):366–397, 2024

    2024

  25. [33]

    A class of smooth exact penalty function methods for optimization problems with orthogonality constraints

    Nachuan Xiao, Xin Liu, and Ya-xiang Yuan. A class of smooth exact penalty function methods for optimization problems with orthogonality constraints. Optimization Methods and Software , 37(4):1205–1241, 2022

    2022

  26. [34]

    First-order methods for geodesically convex optimization

    Hongyi Zhang and Suvrit Sra. First-order methods for geodesically convex optimization. In Conference on Learning Theory , pages 1617–

  27. [35]

    Towards riemannian accelerated gradient methods

    Hongyi Zhang and Suvrit Sra. Towards riemannian accelerated gradient methods. arXiv preprint arXiv:1806.02812 , 2018

    2018 arXiv

  28. [36]

    Adaptive budget allocation for parameter-efficient fine-tuning

    Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for parameter-efficient fine-tuning. In The Eleventh International Con- ference on Learning Representations , 2023

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.