Pith. sign in

REVIEW 3 major objections 5 minor 50 references

Stochastic optimization over expectation-formulated generalized Stiefel manifold

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A sixth-order penalty term makes the constrained Stiefel problem equivalent to an unconstrained one, and stochastic gradient methods solve it with O(ε^-4) samples.

desk verdict Genuinely new penalty with plausible global equivalence theorems, but the stochastic guarantee is for the wrong objective—the bridge to SOEGS is missing. read the letter →

arxiv 2412.20008 v1 pith:EAQKQ3AH submitted 2024-12-28 math.OC

Linshuo Jiang , Nachuan Xiao , Xin Liu This is my paper · ORCID
classification math.OC MSC 90C1590C3090C5265K10
keywords generalizedStiefelmanifoldexpectationconstraintsconstraintdissolvingpenaltynestedstochasticoptimizationgradientmethodssamplecomplexitycanonicalcorrelationanalysisadaptivestepsizes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper considers minimizing an expected loss f(X)=E[fξ(X)] subject to X^T M X = I_p, where M itself is an expectation E[Mθ] and the feasible set is a generalized Stiefel manifold. Because M is known only through samples, standard Riemannian optimization tools such as retractions and orthogonalization are unavailable or unreliable. The paper proposes a penalty function h with a sixth-order customized penalty term and proves that minimizing h over all of $R^{{n×p}}$ is globally equivalent to the constrained problem: the two share their second-order stationary points, and every infeasible first-order stationary point of h is a strict saddle. This equivalence lets the authors run plain stochastic gradient methods and an adaptive variant, which track only a p×p matrix, and prove O($ε^{{-4}}$) sample complexity for finding an ε-stationary point of h.

What carries the argument

The load-bearing object is the constraint-dissolving penalty function CDFCP, h(X)=f(A(X))+β/6 tr(X^T M X((X^T M X)^2−3I_p)), where A(X)=X(3/2 I_p−1/2 X^T M X) pre-dissolves the constraint inside f while the sixth-order penalty term controls the growth of f(A(X)). The sixth-order term changes the geometry so that, at large β, all stationary points of h lie in the region where X^T M X has eigenvalues at most 1, and outside the manifold they are strict saddles. The second mechanism is inner-function tracking: a sequence Y_k of p×p matrices obeys Y_{k+1}=Y_k − b_k(Y_k − C_{θ_{k+1}}(X_k)) + (C_{θ_{k+1}}(X_{k+1}) − C_{θ_{k+1}}(X_k)) to estimate C(X_k)=X_k^T M X_k, which keeps memory at O(np+$p^{2}$) and makes the nested stochastic gradient computable.

What would settle it

For a small random instance with non-quadratic f and known positive-definite M, enumerate all stationary points of h in a bounded box by a homotopy or random multistart method; if any infeasible first-order stationary point of h is not a strict saddle, Theorem 3.12 is false.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the constrained stochastic problem SOEGS can be replaced, without any loss of information about stationary points, by the unconstrained minimization of h(X)=f(A(X))+β/6 tr(X^T M X((X^T M X)^2−3I_p)), with A(X)=X(3/2 I_p − 1/2 X^T M X). For β above an explicit threshold, Theorem 3.12 shows every first-order stationary point of h is either feasible and a first-order stationary point of SOEGS, or a strict saddle; Theorem 3.14 shows h and SOEGS have exactly the same second-order stationary points; and Theorem 3.16 bounds ∥∇h(X)∥ between two multiples of the constrained optimality measure plus feasibility violation. The paper then gives two stochastic gradient algorithms for h that track the inner expectation C(X)=X^T M X with a p×p auxiliary variable rather than an n×n matrix, and it proves both reach an ε-stationary point of h with O($ε^{{-4}}$) samples (Theorems 4.6 and 4.11).

Load-bearing premise

The sample-complexity theorems assume that the iterates and gradient estimates never leave a fixed bounded set, even though the algorithm does nothing to enforce that; if an actual run escapes to infinity, the O($ε^{{-4}}$) guarantee no longer applies.

Editorial extensions

If this is right

  • CDFSG and CDFSG-Ada can be run on SOEGS without ever computing retractions, projections, or the full expectation matrix M, because their iterates only multiply matrices and track a p×p matrix.
  • Because infeasible stationary points of CDFCP are strict saddles, any method that reliably escapes strict saddles and converges to first-order stationary points will land on feasible first-order stationary points of SOEGS.
  • Corollary 3.17 converts an ε-stationary point of CDFCP inside Ω_{1/6} into an O(ε)-stationary point of SOEGS, so the unconstrained guarantee transfers to the constrained problem with the same order in ε.
  • The O(ε^{-4}) sample complexity matches standard nonconvex stochastic gradient bounds, suggesting the constraint structure adds no extra sample-complexity order.
  • The p×p tracking variable Y_k provides a cheap feasibility certificate and enables post-processed feasible points via X_K Y_K^{-1/2}, which experiments show preserves accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the boundedness assumption could be replaced by a projection or a regularizer, the same tracking recursion would yield an unconditional sample-complexity guarantee; this is an extension the paper leaves open.
  • The penalty-plus-tracking design may apply to other expectation-constrained matrix manifolds, such as orthogonality constraints inside an expectation, not only the generalized Stiefel case.
  • Theorem 3.12's strict-saddle characterization suggests that perturbed stochastic gradient versions of CDFSG could escape infeasible saddles and converge to second-order stationary points of SOEGS, since CDFCP and SOEGS share second-order stationary points (Theorem 3.14).
  • The empirical robustness to β across five orders of magnitude in the experiments hints that the explicit penalty threshold may be pessimistic; proving a tighter bound on β could ease tuning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies stochastic optimization over the generalized Stiefel manifold with expectation-formulated constraints (SOEGS). It introduces a constraint dissolving penalty function CDFCP with a sixth-order penalty term and proves exact equivalence results: every first-order stationary point of h is either a first-order stationary point of SOEGS or a strict saddle (Theorem 3.12), and the two problems share the same second-order stationary points (Theorem 3.14). It then proposes two stochastic gradient algorithms, CDFSG and CDFSG-Ada, that track a p-by-p matrix instead of an n-by-n matrix, and proves O(ε^-4) sample complexity for finding an ε-stationary point of CDFCP (Theorems 4.6 and 4.11). Numerical experiments on GCCA-type problems compare the proposed methods with the Landing algorithm and RSG+.

Significance. The exact stationary-point equivalence for the sixth-order penalty is a nontrivial and potentially useful construction, and the p-by-p tracking scheme is a genuine memory improvement over tracking an n-by-n expectation matrix. The proof of the equivalence theorems appears internally consistent, and the sample-complexity theorems for the penalty function h are correctly derived under the stated assumptions. However, the advertised reduction from SOEGS to unconstrained minimization of h is not established: the algorithms are only guaranteed to find ε-stationary points of h, and an ε-stationary point of h can be an infeasible strict saddle that is far from any ε-stationary point of SOEGS. The paper would be a solid contribution if this gap were closed or if the claims were restricted to ε-stationarity of CDFCP; as it stands, the central motivating claim is overstated.

major comments (3)
  1. [Assumption 4.1(3)-(4)] The complexity theorems only certify ε-stationarity of the penalty h, not of SOEGS. The only quantitative bridge, Theorem 3.16, requires the iterate to lie in Ω_{1/6}, and neither Algorithm 1 nor Algorithm 2 enforces or proves this condition. The gap is real and not merely technical. For example, take n=p=1, M=1, f(x)=-γx with γ>0, and β=12γ, which equals β̃ in (20) since Lg=0 and L0=γ. Then h'(x)=(x²-1)(1.5γ+βx(x²+1)). The equation x³+x+1/8=0 has a root r∈(-1/8,-0.12); at r, ∇h(r)=0, r is infeasible with |r²-1|>0.98, r∉Ω_{1/6}, and h''(r)=β(3r²+1)(r²-1)<0, so r is a strict saddle. Initialize Algorithm 1 at X₀=r, Y₀=r², D₀=0, with the deterministic oracles fξ=f and Mθ=M. Then Xₖ=r, Yₖ=r², and Dₖ=0 for all k, so Assumption 4.1 holds and the averaged gradient norm in Theorem 4.6 is 0, yet the output is not an ε-first-order stationary point of SOEGS for any ε<0.98. Consequently, the claimed reduction from SOEGS to unconstrained minimization of CDFCP fails unless a saddle-escape mechanism or a proof that the iterates enter Ω_{1/6} is added.
  2. [Abstract and Theorem 3.12] The convergence results rely on the a priori almost-sure boundedness of the iterates {X_k} and the stochastic gradient estimates {D_k}. These bounds are used in Proposition 4.4 (boundedness of Y_k), Proposition 4.5 (tracking error), and Theorems 4.6 and 4.11 (via τ_x and τ_d). The algorithm contains no projection and the penalty term does not by itself enforce boundedness for the chosen step-size schedule, so these assumptions are not consequences of the problem data. The paper should either prove boundedness under weaker conditions or explicitly state that the O(ε^-4) guarantee is conditional on Assumption 4.1(3)-(4). As written, the phrase 'global convergence guarantee' in Section 1.1 is not justified.
  3. [Minor comments] The abstract says that CDFCP and SOEGS 'share the same first-order and second-order stationary points.' Theorem 3.12 shows a weaker statement: every first-order stationary point of h is either a first-order stationary point of SOEGS or a strict saddle of h. Infeasible strict saddles are first-order stationary points of h but not of SOEGS, so the sets of first-order stationary points are not identical. The wording should be corrected to reflect the actual 'either/or' statement, because the current abstract overstates the equivalence.
minor comments (5)
  1. [Section 2.1] The definition κ(A) := σmax(M)/σmin(M) refers to M, not A; this should be fixed to avoid confusion.
  2. [Sections 5.3 and 5.4] The phrase 'the the landing algorithm' appears twice; the duplicated article should be removed.
  3. [Proposition 4.9] Proposition 4.9 states that the sequence is generated by Algorithm 1, but it is analyzing Algorithm 2; this should be corrected.
  4. [Proposition 4.10] The notation sup_{1≤i≤m,1≤j≤n} uses m and n, but the ambient dimensions are n and p; the index range should be 1≤i≤n, 1≤j≤p.
  5. [Theorems 4.6 and 4.11] Some displayed sums run to K instead of K−1 with mismatched step-size indices; the summation limits should be checked and made consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: CDFCP equivalence and O(ε^-4) guarantees are proved from stated assumptions, with only motivational self-citations.

full rationale

The paper's derivation chain is self-contained rather than circular. CDFCP is a new penalty construction, not a restatement of SOEGS: the gradient and Hessian formulas (Lemma 3.1, Lemma 3.5) are derived by direct differentiation, and the stationary-point equivalence is proved through Propositions 3.7-3.10 and Theorems 3.12-3.16 using Assumptions 1.1 and 3.4 plus the explicit threshold β̃. The prior self-cited CDF paper [49] is used only as motivation for the penalty idea and for a local equivalence statement; the new global equivalence results are proved in the present paper, not imported from [49]. The sample-complexity results (Theorems 4.6 and 4.11) concern ε-stationarity of CDFCP, as stated in the abstract, and follow from standard descent inequalities under the explicit boundedness and oracle assumptions in Assumptions 4.1 and 4.7; the β threshold is a theorem hypothesis, not a fitted parameter. No quantity is fitted to data and renamed a prediction, and no uniqueness or equivalence claim is justified solely by a self-citation. The skeptical concern that ε-stationarity of h need not imply ε-stationarity of SOEGS because infeasible strict saddles of h exist is a correctness gap, not a circularity: the paper's own Theorem 3.12 concedes that such points are only saddles, and the algorithms' saddle-escape behavior is not analyzed. That gap does not make the derivation equivalent to its inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard assumptions about the objective and sampling oracles, plus an a priori boundedness assumption for the algorithm iterates. The only hand-tuned quantity is the penalty parameter, with step sizes and Adam hyperparameters chosen by grid search in practice. No new physical or mathematical entities are postulated.

free parameters (3)
  • penalty parameter beta = 0.1 (default in experiments)
    The equivalence theorems require beta >= betatilde, but the experiments set beta = 0.1 by grid search; the theory does not prescribe a concrete value.
  • step sizes s1, s2 = chosen by grid search
    The sample complexity bound depends on s1 and s2; in experiments they are picked from {2^{-k}} and {l * 10^{-k}}.
  • Adam hyperparameters eta1, eta2, epsilon = not specified in experiments
    Algorithm 2's proof and behavior depend on these; no values are reported.
assumptions (5)
  • domain assumption Assumption 1.1: f is differentiable with L_g-Lipschitz gradient; E[grad f_xi] = grad f; M = E[M_theta] is positive definite
    Used throughout the equivalence proofs and algorithm analysis.
  • domain assumption Assumption 3.4: f is twice differentiable
    Needed for Hessian expressions and saddle point analysis (Lemma 3.5).
  • domain assumption Assumption 4.1: sampled matrices M_theta bounded a.s.; iterates X_k and gradient estimates D_k bounded a.s.; sup alpha_k / b_k finite
    Required for Propositions 4.4, 4.5 and Theorems 4.6, 4.11; the boundedness of iterates is not guaranteed by the algorithm.
  • domain assumption Assumption 4.7: grad f_xi uniformly bounded on a bounded set; W_xi,theta Lipschitz in Y
    Used for the Adam variant's convergence proof.
  • standard math Standard linear algebra facts: eigenvalue decomposition of symmetric matrices, trace inequalities, coercivity of the sixth-order penalty
    Implicit in the proofs; no external theorems beyond standard results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stochastic optimization over expectation-formulated generalized Stiefel manifold." pith.science (2026). https://pith.science/paper/EAQKQ3AH

@misc{pith2026241220008,
  author       = {Pith},
  title        = {Pith review of: Stochastic optimization over expectation-formulated generalized Stiefel manifold},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EAQKQ3AH}},
  note         = {Machine review of arXiv:2412.20008}
}
abstract

In this paper, we consider a class of stochastic optimization problems over the expectation-formulated generalized Stiefel manifold (SOEGS), where the objective function $f$ is continuously differentiable. We propose a novel constraint dissolving penalty function with a customized penalty term (CDFDP), which maintains the same order of differentiability as $f$. Our theoretical analysis establishes the global equivalence between CDFCP and SOEGS in the sense that they share the same first-order and second-order stationary points under mild conditions. These results on equivalence enable the direct implementation of various stochastic optimization approaches to solve SOEGS. In particular, we develop a stochastic gradient algorithm and its accelerated variant by incorporating an adaptive step size strategy. Furthermore, we prove their $\mathcal{O}(\varepsilon^{-4})$ sample complexity for finding an $\varepsilon$-stationary point of CDFCP. Comprehensive numerical experiments show the efficiency and robustness of our proposed algorithms.

Figures

Figures reproduced from arXiv: 2412.20008 by the authors.

Figure 1
Figure 1. Performance of CDFSG and CDFSG-Ada with different values of [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 2
Figure 2. Performance of CDFSG and CDFSG-Ada with different values of [PITH_FULL_IMAGE:figures/full_fig_p026_2.png] view at source ↗
Figure 3
Figure 3. PCC and feasibility variation of CDFSG, CDFSG-Ada, Landing and RSG+ with batch size [PITH_FULL_IMAGE:figures/full_fig_p028_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Function value and feasibility variation of CDFSG, CDFSG-Ada, Landing with batch size [PITH_FULL_IMAGE:figures/full_fig_p028_4.png]
Figure 5
Figure 5. Figure 5: Function value and feasibility violation variations of CDFSG, CDFSG-Ada, and Landing [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 43 canonical work pages

  1. [1]

    Steepest descent algorithms for optimization under unitary matrix constraint

    Traian E Abrudan, Jan Eriksson, and Visa Koivunen. Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Transactions on Signal Processing, 56(3):1134–1147, 2008

    work page 2008

  2. [2]

    Trust-region methods on riemannian manifolds

    P-A Absil, Christopher G Baker, and Kyle A Gallivan. Trust-region methods on riemannian manifolds. Foundations of Computational Mathematics , 7:303–330, 2007

    work page 2007

  3. [3]

    Absil, R

    P.-A. Absil, R. Mahony, and R. Sepulchre. Optimization Algorithms on Matrix Manifolds: . Prince- ton University Press, December 2008

    work page 2008

  4. [4]

    Newton’s method on riemannian manifolds and a geometric model for the human spine

    Roy L Adler, Jean-Pierre Dedieu, Joseph Y Margulies, Marco Martens, and Mike Shub. Newton’s method on riemannian manifolds and a geometric model for the human spine. IMA Journal of Numerical Analysis, 22(3):359–390, 2002

    work page 2002

  5. [5]

    Stochastic approx- imation for canonical correlation analysis

    Raman Arora, Teodor Vanislavov Marinov, Poorya Mianjy, and Nati Srebro. Stochastic approx- imation for canonical correlation analysis. Advances in Neural Information Processing Systems , 30, 2017

    work page 2017

  6. [6]

    Riemannian manifold trust-region methods with applications to eigenprob- lems

    Christopher G Baker. Riemannian manifold trust-region methods with applications to eigenprob- lems. The Florida State University, 2008

    work page 2008

  7. [7]

    Learning sparse feature representa- tions using probabilistic quadtrees and deep belief nets

    Saikat Basu, Manohar Karki, Sangram Ganguly, Robert DiBiano, Supratik Mukhopadhyay, Shree- kant Gayaka, Rajgopal Kannan, and Ramakrishna Nemani. Learning sparse feature representa- tions using probabilistic quadtrees and deep belief nets. Neural Processing Letters, 45:855–867, 2017

    work page 2017

  8. [8]

    Stochastic approximations and differential inclusions

    Michel Bena ¨ ım, Josef Hofbauer, and Sylvain Sorin. Stochastic approximations and differential inclusions. SIAM Journal on Control and Optimization , 44(1):328–348, 2005

    work page 2005

Show all 50 references
  1. [9]

    Non-convex phase retrieval from stft measurements

    Tamir Bendory and Yonina Eldar. Non-convex phase retrieval from stft measurements. IEEE Transactions on Information Theory , PP, 07 2016

    2016

  2. [10]

    Subgradient sampling for nonsmooth nonconvex minimization

    J´ erˆ ome Bolte, Tam Le, and Edouard Pauwels. Subgradient sampling for nonsmooth nonconvex minimization. SIAM Journal on Optimization , 33(4):2542–2569, 2023. 31

    2023

  3. [11]

    Stochastic first-order methods for convex and nonconvex functional constrained optimization

    Digvijay Boob, Qi Deng, and Guanghui Lan. Stochastic first-order methods for convex and nonconvex functional constrained optimization. Mathematical Programming, pages 1–65, 2022

    2022

  4. [12]

    An introduction to optimization on smooth manifolds

    Nicolas Boumal. An introduction to optimization on smooth manifolds . Cambridge University Press, 2023

    2023

  5. [13]

    Manopt, a matlab toolbox for optimization on manifolds

    Nicolas Boumal, Bamdev Mishra, P-A Absil, and Rodolphe Sepulchre. Manopt, a matlab toolbox for optimization on manifolds. The Journal of Machine Learning Research, 15(1):1455–1459, 2014

    2014

  6. [14]

    Mengjie Chen, Chao Gao, Zhao Ren, and Harrison H. Zhou. Sparse CCA via Precision Adjusted Iterative Thresholding, November 2013. arXiv:1311.6186 [math, stat]

    2013 arXiv

  7. [15]

    Solving stochastic compositional optimization is nearly as easy as solving stochastic optimization

    Tianyi Chen, Yuejiao Sun, and Wotao Yin. Solving stochastic compositional optimization is nearly as easy as solving stochastic optimization. IEEE Transactions on Signal Processing, 69:4937–4948, 2021

    2021

  8. [16]

    On the convergence of a class of adam-type algorithms for non-convex optimization

    Xiangyi Chen, Sijia Liu, Ruoyu Sun, and Mingyi Hong. On the convergence of a class of adam-type algorithms for non-convex optimization. arXiv preprint arXiv:1808.02941 , 2018

    2018 arXiv

  9. [17]

    An Orthogonalization-Free Parallelizable Framework for All-Electron Calculations in Density Functional Theory

    Bin Gao, Guanghui Hu, Yang Kuang, and Xin Liu. An Orthogonalization-Free Parallelizable Framework for All-Electron Calculations in Density Functional Theory. SIAM Journal on Scien- tific Computing , 44(3):B723–B745, June 2022

    2022

  10. [18]

    Sparse GCA and Thresholded Gradient Descent, July 2021

    Sheng Gao and Zongming Ma. Sparse GCA and Thresholded Gradient Descent, July 2021. arXiv:2107.00371 [cs, stat]

    2021 arXiv

  11. [19]

    Efficient algorithms for large- scale generalized eigenvector computation and canonical correlation analysis

    Rong Ge, Chi Jin, Praneeth Netrapalli, Aaron Sidford, et al. Efficient algorithms for large- scale generalized eigenvector computation and canonical correlation analysis. In International Conference on Machine Learning , pages 2741–2750. PMLR, 2016

    2016

  12. [20]

    A single timescale stochastic approxi- mation method for nested stochastic optimization

    Saeed Ghadimi, Andrzej Ruszczynski, and Mengdi Wang. A single timescale stochastic approxi- mation method for nested stochastic optimization. SIAM Journal on Optimization, 30(1):960–979, 2020

    2020

  13. [21]

    Hardoon, Sandor Szedmak, and John Shawe-Taylor

    David R. Hardoon, Sandor Szedmak, and John Shawe-Taylor. Canonical Correlation Analysis: An Overview with Application to Learning Methods. Neural Computation , 16(12):2639–2664, December 2004

    2004

  14. [22]

    Relations between two sets of variables

    Hotelling Harold. Relations between two sets of variables. Biometrika, 28(3):321–377, 1936

    1936

  15. [23]

    Generalized canonical correlations and their application to experimental data

    Paul Horst. Generalized canonical correlations and their application to experimental data . Num- ber 14. Journal of clinical psychology, 1961

    1961

  16. [24]

    A brief introduction to manifold opti- mization

    Jiang Hu, Xin Liu, Zai-Wen Wen, and Ya-Xiang Yuan. A brief introduction to manifold opti- mization. Journal of the Operations Research Society of China , 8:199–248, 2020

    2020

  17. [25]

    A broyden class of quasi-newton methods for riemannian optimization

    Wen Huang, Kyle A Gallivan, and P-A Absil. A broyden class of quasi-newton methods for riemannian optimization. SIAM Journal on Optimization , 25(3):1660–1685, 2015

    2015

  18. [26]

    Robust estimation of a location parameter

    Peter J Huber. Robust estimation of a location parameter. In Breakthroughs in statistics: Method- ology and distribution , pages 492–518. Springer, 1992

    1992

  19. [27]

    The riemannian barzilai–borwein method with non- monotone line search and the matrix geometric mean computation

    Bruno Iannazzo and Margherita Porcelli. The riemannian barzilai–borwein method with non- monotone line search and the matrix geometric mean computation. IMA Journal of Numerical Analysis, 38(1):495–517, 2018

    2018

  20. [28]

    Riemannian stochastic recursive gradient algorithm

    Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra. Riemannian stochastic recursive gradient algorithm. In International conference on machine learning , pages 2516–2524. PMLR, 2018

    2018

  21. [29]

    J. R. Kettenring. Canonical analysis of several sets of variables. Biometrika, 58(3):433–451, 1971. 32

    1971

  22. [30]

    Adam: A method for stochastic optimization

    Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  23. [31]

    Nonsmooth nonconvex stochastic heavy ball

    Tam Le. Nonsmooth nonconvex stochastic heavy ball. Journal of Optimization Theory and Applications, pages 1–21, 2024

    2024

  24. [32]

    The mnist database of handwritten digits

    Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/ , 1998

    1998

  25. [33]

    Stochastic inexact aug- mented lagrangian method for nonconvex expectation constrained optimization

    Zichong Li, Pin-Yu Chen, Sijia Liu, Songtao Lu, and Yangyang Xu. Stochastic inexact aug- mented lagrangian method for nonconvex expectation constrained optimization. arXiv preprint arXiv:2212.09513, 2022

    2022 arXiv

  26. [34]

    Linear discriminant analysis: New formulations and overfit analysis

    Dijun Luo, Chris Ding, and Heng Huang. Linear discriminant analysis: New formulations and overfit analysis. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 25, pages 417–422, 2011

    2011

  27. [35]

    Proximally constrained methods for weakly convex optimization with weakly convex constraints

    Runchao Ma, Qihang Lin, and Tianbao Yang. Proximally constrained methods for weakly convex optimization with weakly convex constraints. arXiv preprint arXiv:1908.01871 , 2019

    1908 arXiv

  28. [36]

    Finding linear structure in large datasets with scalable canonical correlation analysis

    Zhuang Ma, Yichao Lu, and Dean Foster. Finding linear structure in large datasets with scalable canonical correlation analysis. In International conference on machine learning , pages 169–178. PMLR, 2015

    2015

  29. [37]

    An online riemannian pca for stochastic canonical correlation analysis

    Zihang Meng, Rudrasis Chakraborty, and Vikas Singh. An online riemannian pca for stochastic canonical correlation analysis. Advances in Neural Information Processing Systems , 34:14056– 14068, 2021

    2021

  30. [38]

    Numerical optimization

    Jorge Nocedal and Stephen J Wright. Numerical optimization. Springer, 1999

    1999

  31. [39]

    Riemannian conjugate gradient methods: General framework and specific algo- rithms with convergence analyses

    Hiroyuki Sato. Riemannian conjugate gradient methods: General framework and specific algo- rithms with convergence analyses. SIAM Journal on Optimization , 32(4):2690–2717, 2022

    2022

  32. [40]

    Cholesky QR-based retraction on the generalized Stiefel manifold

    Hiroyuki Sato and Kensuke Aihara. Cholesky QR-based retraction on the generalized Stiefel manifold. Computational Optimization and Applications , 72(2):293–308, March 2019

    2019

  33. [41]

    A new, globally convergent riemannian conjugate gradient method

    Hiroyuki Sato and Toshihiro Iwai. A new, globally convergent riemannian conjugate gradient method. Optimization, 64(4):1011–1031, 2015

    2015

  34. [42]

    Preconditioned Riemannian Optimization on the Generalized Stiefel Manifold, June 2021

    Boris Shustin and Haim Avron. Preconditioned Riemannian Optimization on the Generalized Stiefel Manifold, June 2021. arXiv:1902.01635 [cs, math]

    2021 arXiv

  35. [43]

    The challenge problem for automated detection of 101 semantic concepts in multi- media

    Cees GM Snoek, Marcel Worring, Jan C Van Gemert, Jan-Mark Geusebroek, and Arnold WM Smeulders. The challenge problem for automated detection of 101 semantic concepts in multi- media. In Proceedings of the 14th ACM international conference on Multimedia , pages 421–430, 2006

    2006

  36. [44]

    Minimum generalized variance for a set of linear functions

    Robert GD Steel. Minimum generalized variance for a set of linear functions. The Annals of Mathematical Statistics, pages 456–460, 1951

    1951

  37. [45]

    Kanatsoulis, and Nicholas D

    Mikael Sørensen, Charilaos I. Kanatsoulis, and Nicholas D. Sidiropoulos. Generalized Canonical Correlation Analysis: A Subspace Intersection Approach. IEEE Transactions on Signal Process- ing, 69:2452–2467, 2021. arXiv:2003.11205 [cs, eess, stat]

    2021 arXiv

  38. [46]

    Optimization without retraction on the random generalized stiefel manifold

    Simon Vary, Pierre Ablin, Bin Gao, and P-A Absil. Optimization without retraction on the random generalized stiefel manifold. arXiv preprint arXiv:2405.01702 , 2024

    2024

  39. [47]

    Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions

    Mengdi Wang, Ethan X Fang, and Han Liu. Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions. Mathematical Programming, 161:419– 449, 2017. 33

    2017

  40. [48]

    Accelerating stochastic composition optimization

    Mengdi Wang, Ji Liu, and Ethan X Fang. Accelerating stochastic composition optimization. Journal of Machine Learning Research , 18(105):1–23, 2017

    2017

  41. [49]

    Constraint Dissolving Approaches for Riemannian Optimization, May 2022

    Nachuan Xiao, Xin Liu, and Kim-Chuan Toh. Constraint Dissolving Approaches for Riemannian Optimization, May 2022. arXiv:2203.10319 [math]

    2022 arXiv

  42. [50]

    A stochastic composite gradient method with incremental variance reduction

    Junyu Zhang and Lin Xiao. A stochastic composite gradient method with incremental variance reduction. Advances in Neural Information Processing Systems , 32, 2019. 34

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.