Second-order geometry and Riemannian Newton-type methods for optimization on the indefinite Stiefel manifold

Hiroyuki Sato

REVIEW 4 major objections 5 minor 15 references

Second-order geometry and Riemannian Newton-type methods for optimization on the indefinite Stiefel manifold

T0 review · 4 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Explicit formulas for the Levi-Civita connection and Riemannian Hessian on the indefinite Stiefel manifold turn Newton's method into a routine conjugate-gradient solve.

desk verdict The explicit Hessian formulas for the indefinite Stiefel manifold are a real contribution, but the abstract overclaims trust-region results and near-singular robustness that the body never delivers. read the letter →

arxiv 2603.07133 v2 pith:YJNPELZ5 submitted 2026-03-07 math.OC

Hiroyuki Sato This is my paper

classification math.OC MSC 53B2053B2165K1090C30

keywords indefiniteStiefelmanifoldLevi-CivitaconnectionRiemannianHessianNewton'smethodlinearconjugategradientgeneralizedcanonicalmetrictraceminimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that second-order geometry on the indefinite Stiefel manifold - matrices X satisfying X^T A X = J, where A is a fixed invertible symmetric matrix and J^2 = I - is explicit enough to make Riemannian Newton's method practical. It derives the Levi-Civita connection, and then the Riemannian Hessian of any smooth function, in closed form under the two generalized canonical metrics G^(1)_X and G^(2)_X. Newton's equation is solved in a fixed tangent space by linear conjugate gradient, avoiding direct inversion of the Hessian. Numerical experiments on trace minimization show fast local convergence that is nearly independent of the metric and of the scale parameter rho, in contrast to first-order methods.

What carries the argument

The central object is the pair of generalized canonical metrics G^(1)_X = (1/rho) A X X^T A + (A - A X J X^T A)^2 and G^(2)_X = (1/rho) A X X^T A + I_n - X(X^T X)^{-1} X^T, both positive-definite on the ambient space, together with their explicit inverses. These metrics make the tangent-space projection equal to P_X(Y) = Y - X J sym(X^T A Y), which, combined with the relations X^T A X = J and J^2 = I, reduces the Koszul-formula Christoffel function to polynomial expressions in X, xi, eta. The closed-form Hessian is assembled from these expressions plus the Euclidean Hessian of the objective's extension.

What would settle it

For a concrete small instance, say n=2, p=1, A=diag(1,-1), J=1, compare the closed-form Hessian Hess f(X)[xi] with a finite-difference check of the gradient along the manifold, or run the claimed retraction on random tangent vectors and test whether X^T A X = J holds to machine precision. Either disagreement would falsify the paper's central claim.

Watch

Extended reading notes

Core claim

For the indefinite Stiefel manifold iSt_{A,J}(p,n) = {X in R^{n x p} : X^T A X = J}, where A is a fixed invertible symmetric matrix and J^2 = I_p, the paper derives the Levi-Civita connection and the Riemannian Hessian of a smooth function under the two generalized canonical metrics. Using Koszul's formula on the ambient open submanifold E, the Christoffel function simplifies to explicit expressions in X, xi, and eta, with the orthogonal projection to the tangent space taking the simple form P_X(Y) = Y - X J sym(X^T A Y). The Riemannian Hessian is then written out in closed form; it depends on the Euclidean gradient and Hessian of the extended objective plus these geometric terms. These form

Load-bearing premise

The load-bearing premise is that the retraction R_X(xi) imported from the earlier work really is a retraction of iSt_{A,J}(p,n); if iterates leave the level set X^T A X = J, the Newton sequence and its convergence claims lose their foundation.

Editorial extensions

If this is right

Practitioners can run Riemannian Newton's method on iSt_{A,J}(p,n) with only matrix multiplications and a linear conjugate-gradient solve - no numerical differentiation of the Hessian.
Since Newton's convergence is nearly independent of rho and of the choice of metric, the user can select whichever metric has cheaper per-iteration cost.
The explicit Levi-Civita connection supplies the missing second-order information needed for Riemannian trust-region methods and for curvature-based analyses on these manifolds.
For trace minimization, which encodes generalized eigenvalue problems, a hybrid 'first-order then Newton' strategy needs very few Newton steps, so the method scales better than pure first-order optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The abstract mentions a Riemannian trust-region method and near-singular robustness, but the body does not present them; those are claims awaiting the experiments the abstract alludes to.
The retraction R_X(xi) used in the Newton update is imported from earlier work without proof of retraction validity in this paper; a direct feasibility check would be a low-cost safeguard.
Because all Hessian-vector products are closed-form, the explicit Hessian is directly usable in truncated conjugate-gradient trust-region loops, not just Newton's method.
The formulas invert A and X^T X, and the experiments use a well-conditioned A; whether the method remains robust when eigenvalues of A approach zero is an open question.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuine thing here is the second-order geometry: explicit Levi-Civita connection and Riemannian Hessian for the indefinite Stiefel manifold under the two generalized canonical metrics from [12]. That is new, and the derivation looks honest—standard Koszul machinery applied to a setting that had only first-order geometry before. The simplification steps (Lemma 4.1, the projection tricks) are genuinely useful and reduce computational cost in a way that matters for implementation. The numerical section, thin as it is, does show Newton's method converging fast with the linear-CG solver, and the relative insensitivity to rho is plausible. I would not be surprised if the formulas are correct; the algebraic structure is internally consistent and there is no circular fitting anywhere.

But the abstract says two things the paper does not do. It advertises a Riemannian trust-region method with truncated CG, and it claims robustness in near-singular settings where eigenvalues of the constraint matrix approach zero. Section 5 only describes Newton's method with linear CG; there is no trust-region implementation. And the single experiment uses A with eigenvalues 1 through 5 and -1 through -5—condition number 5, nowhere near singular. This is not just a wording slip. The Hessian formulas contain A^{-1} explicitly, and the retraction and projections rely on (X^T X)^{-1} and other inverses, so near-zero eigenvalues of A are exactly where one would expect breakdown. The stress-tester's concern holds up: the robustness claim is unsupported and likely fragile.

Other soft spots are more minor. The retraction is imported from [13] without proof; that is acceptable if the prior paper is correct, but the dependence should be stated clearly. The numerical evidence is one random instance per rho, no seeds, no code, no error bars. The algebra is long and hand-derived; a typo in one of the Hessian expressions would be easy to miss. None of this kills the core contribution, but it means the paper needs revision before it can be trusted as a toolbox.

Who is this for? People working on optimization over indefinite Stiefel manifolds—generalized eigenvalue problems, signal processing. They will want these Hessian formulas. It deserves a serious referee, not a desk reject, but the referee should push for the trust-region claim to be either implemented or removed, for near-singular experiments, and for code or at least a reproducibility statement. I would cite it once the formulas are peer-reviewed and the overclaims are fixed.

Referee Report

4 major / 5 minor

Summary. The paper studies the second-order geometry of the indefinite Stiefel manifold iSt_{A,J}(p,n) = {X : XᵀAX = J} under the two generalized canonical metrics G(1), G(2) introduced in [12]. It derives the Levi-Civita connection on an ambient open submanifold E via Koszul's formula, simplifies the resulting Christoffel functions, and obtains explicit Riemannian Hessian formulas for a smooth function on iSt_{A,J}. It then proposes to solve the Riemannian Newton equation by the linear conjugate gradient method in the tangent space (Algorithm 1) and reports numerical experiments for the trace objective f(X) = tr(XᵀMX) with n = 10, p = 4, comparing steepest descent, conjugate gradient, and a CG-to-Newton hybrid for ρ = 0.5, 1, 2.

Significance. If the algebra is correct, the explicit Levi-Civita connection and Hessian formulas are a useful extension of the first-order geometry in [12,13], and the reduction in Lemma 4.1 is a genuine computational improvement. The paper is clearly structured and the derivation is self-contained apart from the imported retraction. However, the advertised trust-region method and near-singular robustness appear only in the arXiv abstract, not in the body; the numerical section contains a single, well-conditioned random instance. The Hessian formulas also contain explicit A^{-1} and (XᵀX)^{-1} terms, so the claimed robustness near singular A is neither analyzed nor demonstrated. These gaps are correctable but presently make the manuscript broader in claim than in substance.

major comments (4)

[Abstract; Sections 5–6] The abstract (arXiv metadata) promises a Riemannian trust-region method with truncated CG and numerical robustness in near-singular settings over several problem sizes. Section 5 presents only Riemannian Newton's method with the Newton equation solved by linear CG, and Section 6 tests a single n=10, p=4 instance with A = P diag(1,2,3,4,5,-1,-2,-3,-4,-5)Pᵀ, condition number 5. There is no trust-region method and no near-singular experiment. This mismatch must be resolved: either add the missing method/experiments or revise the abstract so that it accurately describes the body.
[Section 4.3–4.4; Eqs. (17), (18), (10)] The Hessian and metric formulas explicitly contain A^{-1} and (XᵀX)^{-1}. As eigenvalues of A approach zero, these terms are expected to become unbounded and the linear-CG iteration ill-conditioned, so the claimed robustness of the trust-region method in near-singular settings is unsupported by any conditioning analysis or experiment. At minimum, the authors should provide a condition-number estimate or a numerical experiment with a nearly singular A; otherwise the near-singular robustness claim should be removed.
[Section 6] The numerical study reports only one random problem instance (n=10, p=4). The matrix M defining the objective is not specified at all — only A, J, and the random construction of X0 are described. The claim that Newton's method is 'relatively insensitive to the choice of Riemannian metric' rests on three values of ρ for this single instance. Please specify M, include more instances (including larger p,n), or explain why a single instance is sufficient; otherwise the empirical claim is anecdotal.
[Section 5 (retraction)] The update uses the retraction R_X(ξ) imported from ref. [13] without proof. Since [13] is an arXiv preprint and correctness of this retraction is load-bearing for the numerical experiments, please either provide a proof that R_X maps T_X iSt_{A,J} into iSt_{A,J} and satisfies the retraction axioms, or state precisely where such a proof can be found. This is a standard self-containedness request rather than an accusation of error.

minor comments (5)

[Abstract] The full-text abstract is more modest than the arXiv metadata abstract. Synchronize the two versions: the body implements Newton's method with linear CG, not a trust-region method.
[Section 6] The generation of M is missing. The text says 'We randomly constructed a 10×10 orthogonal matrix P and set A...' but never states how M is chosen. This makes the experiment unreproducible.
[Algorithm 1] The inner product ⟨·,·⟩_{X_k} used in the CG coefficients should be explicitly identified with the Riemannian metric (3). Also, the outer Newton iteration lacks an explicit stopping criterion.
[Section 4.4] The Hessian formulas are derived purely symbolically. A short finite-difference check on a randomly chosen tangent vector would substantially increase confidence in the complicated expressions, especially since no code is provided.
[References] Refs. [12] and [13] are arXiv preprints. If final published versions exist, they should be cited; otherwise the dependence on unreviewed preprints should be noted.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the second-order formulas are derived from the chosen metrics via Koszul's formula; load-bearing inputs are prior external geometry results, not fitted outputs.

full rationale

The paper's central content is a derivation, not a prediction-from-fit. Given the generalized canonical metrics G_X^(1) and G_X^(2) in Eqs. (6)-(7) (proposed in ref. [12]) and first-order tangent/normal/projection formulas from refs. [12,13], the Levi-Civita connection is computed via Koszul's formula (Proposition 4.1), and the Riemannian Hessian is then obtained from Hess f(X)[ξ] = ∇_ξ grad f in Section 4.4. These formulas are consequences of the definitions; no parameter is fitted to data. The only tunable parameter ρ is tested at three fixed values (0.5, 1, 2) in Section 6, and the claim is precisely that Newton's method behaves similarly across those values, so ρ is not used to force a result. The retraction and first-order geometry imported from refs. [12,13] are external, independent prior results by different authors, not self-citations of the present paper. The author's own cited works ([8]-[10], [14]) are background and are not load-bearing. A separate, non-circular defect is flagged: the abstract advertises a trust-region method and robustness in near-singular settings, but the body presents only Newton-CG and tests one instance with A having eigenvalues ±1,...,±5 (condition number 5), while the Hessian and inverse-metric formulas contain A^{-1} (e.g., Eqs. (17)-(18)); this is a correctness/coverage gap, not a circular reasoning step.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central claim rests on standard Riemannian geometry plus two unproved imported prior results (first-order geometry and retraction from refs. [12,13]) and on a smooth-extension assumption for the objective. No new entities are postulated. One free parameter rho controls the metric and is chosen by hand.

free parameters (1)

rho (metric scale parameter) = 0.5, 1, 2 (chosen in experiments; not fitted)
Appears in the definitions of G_X^{(1)} and G_X^{(2)} (Eqs. 6-7) and in all Hessian formulas. It is a tunable method parameter, not inferred from data; the experiments vary it.

assumptions (6)

standard math Standard Riemannian submanifold fact: the Levi-Civita connection on the submanifold is the tangential projection of the ambient connection.
Used in Sections 4.3-4.4 to pass from the ambient Christoffel function Gamma on E to the connection and Hessian on iSt.
standard math Koszul's formula and the existence/uniqueness of the Levi-Civita connection for the ambient metric (2).
Used in Section 4.1 to derive the Christoffel function Gamma_X.
domain assumption The first-order geometry of iSt_{A,J} from refs. [12,13] is correct: tangent space, normal space, orthogonal projection P_X, gradient formula, and the inverses G_X^{-1}.
Section 3 imports these results without proof; the Hessian derivation in Section 4 builds directly on them.
domain assumption The retraction R_X from ref. [13] used in Section 5 is a valid retraction on iSt_{A,J}.
The Newton iteration X_{k+1}=R_{X_k}(xi_k) relies on this unproved imported retraction to stay on the manifold.
domain assumption At a local solution X*, the Riemannian Hessian is positive definite, and the linear CG iteration for Newton's equation converges.
Stated in Section 5; standard for local Newton convergence and for the CG solve of the linearized equation.
domain assumption A smooth extension \bar f of f to E exists and its Euclidean gradient and Hessian are computable.
Section 4.4 defines the Hessian through grad_E \bar f and Hess_E \bar f; for the tested f=tr(X^T M X) this is explicit, but the general algorithm depends on a chosen extension.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Second-order geometry and Riemannian Newton-type methods for optimization on the indefinite Stiefel manifold." pith.science (2026). https://pith.science/paper/YJNPELZ5

@misc{pith2026260307133,
  author       = {Pith},
  title        = {Pith review of: Second-order geometry and Riemannian Newton-type methods for optimization on the indefinite Stiefel manifold},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YJNPELZ5}},
  note         = {Machine review of arXiv:2603.07133}
}

read the original abstract

This paper investigates the second-order geometry of the indefinite Stiefel manifold and derives explicit formulas for the Levi-Civita connection and the Riemannian Hessian under two generalized canonical metrics. We discuss Riemannian Newton's method, in which Newton's equation is solved by the linear conjugate gradient method in a fixed tangent space, and the Riemannian trust-region method with the truncated conjugate gradient method. Numerical experiments for trace minimization problems demonstrate the robustness of the trust-region method over several problem sizes and in near-singular settings where eigenvalues of the constraint matrix approach zero.

Figures

Figures reproduced from arXiv: 2603.07133 by the authors.

**Figure 2.** Results with respect to the Riemannian metric (3) equipped with [PITH_FULL_IMAGE:figures/full_fig_p020_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 3 linked inside Pith

[12]

Van Tiep, D

D. Van Tiep, D. T. V. An, N. T. N. Oanh, and N. T. Son. A generalized canonical metric for optimization on the indefinite Stiefel manifold.arXiv preprint arXiv:2509.16113, 2025

arXiv 2025
[13]

Van Tiep and N

D. Van Tiep and N. T. Son. A Riemannian optimization method on the indefinite Stiefel manifold.arXiv preprint arXiv:2410.22068, 2024

arXiv 2024
[1]

Absil, R

P.-A. Absil, R. Mahony, and R. Sepulchre.Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ, 2008

2008
[2]

Boumal.An Introduction to Optimization on Smooth Manifolds

N. Boumal.An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023

2023
[3]

Boumal, B

N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. Manopt, a Matlab toolbox for optimization on manifolds.Journal of Machine Learning Research, 15(1):1455–1459, 2014

2014
[4]

Edelman, T

A. Edelman, T. A. Arias, and S. T. Smith. The geometry of algorithms with orthogonality constraints.SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998

1998
[5]

B. Gao, N. T. Son, P.-A. Absil, and T. Stykel. Riemannian optimization on the symplectic stiefel manifold.SIAM Journal on Optimization, 31(2):1546–1575, 2021

2021
[6]

J. M. Lee.Introduction to Riemannian manifolds, volume 2. Springer, 2018

2018

Show all 15 references

[7]

Nguyen and S

D. Nguyen and S. Sommer. Second-order differential operators, stochastic differential equations and Brownian motions on embedded manifolds.arXiv preprint arXiv:2406.02879, 2024. 17 Riemannian Newton’s method for optimization problems on the indefinite Stiefel manifold

2024 arXiv
[8]

H. Sato. Riemannian Newton-type methods for joint diagonalization on the Stiefel manifold with application to independent component analysis.Optimization, 66(12):2211–2231, 2017

2017
[9]

Sato.Riemannian Optimization and Its Applications

H. Sato.Riemannian Optimization and Its Applications. Springer Nature, 2021

2021
[10]

Sato and K

H. Sato and K. Aihara. Cholesky QR-based retraction on the generalized Stiefel manifold. Comput. Optim. Appl., 72(2):293–308, 2019

2019
[11]

S. T. Smith. Optimization techniques on Riemannian manifolds. InHamiltonian and Gradient Flows, Algorithms and Control, pages 113–135. American Mathematical Soc., 1994

1994
[14]

Yamada and H

M. Yamada and H. Sato. Conjugate gradient methods for optimization problems on symplectic stiefel manifold.IEEE Control Systems Letters, 7:2719–2724, 2023

2023
[15]

F. Yger, M. Berar, G. Gasso, and A. Rakotomamonjy. Adaptive canonical correlation analysis based on matrix manifolds. InProceedings of the 29th International Conference on Machine Learning, pages 1071–1078, 2012. 18 Riemannian Newton’s method for optimization problems on the i...

2012

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [12]

Van Tiep, D

D. Van Tiep, D. T. V. An, N. T. N. Oanh, and N. T. Son. A generalized canonical metric for optimization on the indefinite Stiefel manifold.arXiv preprint arXiv:2509.16113, 2025

arXiv 2025

[2] [13]

Van Tiep and N

D. Van Tiep and N. T. Son. A Riemannian optimization method on the indefinite Stiefel manifold.arXiv preprint arXiv:2410.22068, 2024

arXiv 2024

[3] [1]

Absil, R

P.-A. Absil, R. Mahony, and R. Sepulchre.Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ, 2008

2008

[4] [2]

Boumal.An Introduction to Optimization on Smooth Manifolds

N. Boumal.An Introduction to Optimization on Smooth Manifolds. Cambridge University Press, 2023

2023

[5] [3]

Boumal, B

N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. Manopt, a Matlab toolbox for optimization on manifolds.Journal of Machine Learning Research, 15(1):1455–1459, 2014

2014

[6] [4]

Edelman, T

A. Edelman, T. A. Arias, and S. T. Smith. The geometry of algorithms with orthogonality constraints.SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, 1998

1998

[7] [5]

B. Gao, N. T. Son, P.-A. Absil, and T. Stykel. Riemannian optimization on the symplectic stiefel manifold.SIAM Journal on Optimization, 31(2):1546–1575, 2021

2021

[8] [6]

J. M. Lee.Introduction to Riemannian manifolds, volume 2. Springer, 2018

2018