Pith. sign in

REVIEW 2 major objections 6 minor 56 references

A retraction-based distributed stochastic proximal framework for weakly-convex costs on compact embedded submanifolds converges to near-stationarity at rate O((1+κ_g)/√k) with consensus, if agents start in a small local region.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 08:09 UTC pith:735RE5HK

load-bearing objection The main convergence theorem has a load-bearing proof gap: Lemma 8 is applied at a point off the manifold, so the advertised rate is not yet supported. the 2 major comments →

arxiv 2510.22270 v2 pith:735RE5HK submitted 2025-10-25 math.OC cs.SYeess.SY

Distributed Stochastic Proximal Algorithm on Riemannian Submanifolds for Weakly-convex Functions

Jishu Zhao , Xi Wang , Jinlong Lei , Shixiang Chen This is my paper
classification math.OC cs.SYeess.SY MSC 90C1590C26
keywords distributed optimizationRiemannian submanifoldsweakly-convex functionsstochastic proximal algorithmsretractionconsensusgeodesic curvatureconvergence rate
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that distributed stochastic optimization over curved, nonconvex constraint sets — compact embedded submanifolds — can be solved by a retraction-based proximal algorithm with the same O(1/√k) rate as Euclidean distributed stochastic methods, multiplied only by a factor (1+κ_g) that measures how much the manifold curves. It proposes a framework that keeps iterates on the manifold via retractions and drives agents to consensus via a multi-step Riemannian consensus protocol, and it instantiates the framework as subgradient, proximal-point, and prox-linear algorithms. Under Lipschitz and weak-convexity assumptions on the local costs, and provided all agents initialize inside a small geodesic neighborhood of their induced arithmetic mean, the paper proves that iterates stay in that region, achieve consensus up to a O(1/β_k²) error, and converge to a nearly stationary point in expectation at rate O((1+κ_g)/√k). The result matters because earlier distributed manifold algorithms either relied on special structure of the Stiefel manifold or used projection-based updates with no closed form; this framework works for general compact embedded submanifolds and shows explicitly how manifold geometry enters the convergence rate.

Core claim

The central discovery is that the Gauss-formula normal-vector inequality ⟨w, Y−X⟩ ≤ κ_g ∥w∥ ∥Y−X∥², valid in a local geodesic ball, lets the paper import the Euclidean-style contraction arguments into the curved setting. With this inequality, the induced arithmetic mean (IAM) becomes a usable proxy for consensus: the paper shows the IAM equals the restricted Euclidean projection onto the manifold, the consensus potential satisfies a quadratic growth condition controlled by κ_g, and the proximal step plus consensus leads to a one-step decrease of the averaged Moreau envelope. The consequence is Theorem 4: after K iterations, the best iterate is ϵ-nearly stationary with ϵ = O((1+κ_g)/√K), whil

What carries the argument

The load-bearing object is the normal-vector inequality (Lemma 3), a consequence of the Gauss formula: for points X,Y on the submanifold within a geodesic radius of order 1/κ_g, the component of Y−X along any normal vector at X is bounded by κ_g ∥w∥ ∥Y−X∥². This inequality is what extends the consensus and proximal analyses from the Stiefel manifold (where orthogonal projections and retractions have closed forms) to arbitrary compact embedded submanifolds, and it is what puts the curvature bound κ_g into the rate. Around it, the framework uses retraction maps to stay on the manifold, the induced arithmetic mean as the consensus reference point, and a local region S whose radius shrinks with

Load-bearing premise

The entire convergence guarantee is conditional on every agent's initial point lying in a small geodesic neighborhood of their induced arithmetic mean — a ball whose radius is inversely proportional to the curvature bound κ_g — so if agents start even moderately far apart on a strongly curved manifold, the theorem gives no guarantee at all.

What would settle it

Take a sphere of radius r (so κ_g = 1/r) with two agents, initialize both at the same point (trivially inside S), and run the distributed Riemannian stochastic subgradient algorithm with the paper's step-size policy for a weakly-convex cost. Empirically measure min_{k≤K} (1/λ²) E[‖P_λ f(X̂_k)−X̂_k‖²] via Monte Carlo: if the decay over K is materially worse than C/√K, or if the consensus error does not shrink as O(1/β_k²), then either the contraction lemma or the rate theorem is wrong. Alternatively, initialize the two agents at angular separation > δ1 (e.g., π/4) on the unit sphere and record

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The three instantiations — subgradient, proximal point, and prox-linear — all inherit the O((1+κ_g)/√K) rate, so the framework unifies nonsmooth, smooth, and composite weakly-convex distributed problems on general compact embedded submanifolds.
  • The consensus error bound 1/N ‖X_k − X̂_k‖² ≤ C L²/β_k² is a direct corollary: by raising β_k, agents can be driven to O(1/β_k²) consensus while the stationarity measure still decays.
  • The curvature bound κ_g appears linearly in the rate: for a sphere of radius r, κ_g = 1/r, so smaller spheres (more curved) are predicted to converge more slowly, matching the numerical experiments in the paper.
  • When the manifold constraints degenerate to Euclidean space (κ_g = 0, M_1 = 1, M_2 = 0), the rate reduces to O(1/√K), recovering the standard Euclidean distributed stochastic proximal rate.
  • The stationarity measure based on the Moreau envelope provides a checkable certificate: once 1/λ² ‖P_λ f(X̂_k) − X̂_k‖² is small, X̂_k is near a point that is approximately stationary for f.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The local-initialization requirement is effectively a synchronization precondition; a practical two-phase scheme that runs consensus first and then optimization would satisfy it, but the paper does not analyze that.
  • Because the rate is linear in κ_g, re-embedding or preconditioning the manifold to reduce the second fundamental form's norm would directly accelerate convergence; this is a testable design principle the paper leaves implicit.
  • The results are in expectation; turning them into high-probability or almost-sure statements for the three instantiations would need concentration arguments for the one-sided stochastic model, which are not supplied.
  • The normal-vector inequality is a transferable tool: it could extend the same consensus-contraction analysis to time-varying graphs, asynchronous updates, or non-compact manifolds with locally bounded curvature, none of which the paper considers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper studies distributed stochastic optimization of weakly-convex functions over compact embedded submanifolds of Euclidean space, where agents hold local copies of the decision variable and aim at consensus. It proposes a retraction-based distributed Riemannian stochastic proximal framework: at each iteration an agent computes a search direction from a stochastic model subproblem on the tangent space and then applies a retraction-based update combining the negative Riemannian consensus gradient with that direction. Under L-Lipschitz continuity, ρ-weak convexity, a connected undirected graph, a uniform bound κ_g on the geodesic curvature, and initial points lying in a local region S (Eq. (20)), the paper proves that iterates remain in S, achieve consensus, and converge to a nearly stationary point at rate O((1+κ_g)/√k) in expectation. Three instantiations are given: distributed Riemannian stochastic subgradient, proximal point, and prox-linear algorithms. Numerical experiments on blind deconvolution, orthogonal sparse dictionary learning, and a generalized eigenvalue problem are reported.

Significance. If the proofs are correct, the paper makes a useful contribution: it extends distributed weakly-convex optimization from the Stiefel manifold to general compact embedded submanifolds using retractions, and it makes the dependence on manifold curvature explicit. The framework is general, the three algorithm instantiations are nontrivial, and the local consensus contraction analysis (Lemmas 5–10, Theorems 1–2) is detailed. The paper also gives explicit constants and a rate theorem involving κ_g, which is informative for understanding the effect of geometry. However, the central descent proof contains a gap in the application of Lemma 8 to off-manifold points; this needs to be repaired before the main convergence rate can be considered established.

major comments (2)
  1. [Section 3.B, Eq. (11)] Equation (11) is not equivalent to Eq. (10) as printed. Since ∇h_{i,t}(X) = X_i - Σ_j W^t_{ij} X_j, Eq. (10) expands to R_{X_{i,k}}( α P_{T_{X_{i,k}}M}( Σ_j W^t_{ij} X_{j,k} - X_{i,k}) + v_{i,k} ). The printed Eq. (11) drops the -X_i term and changes the sign. Algorithm 1 is written in terms of (11), while the convergence proofs use (10). This is a load-bearing discrepancy for reproducibility: an implementation following (11) is not running the analyzed algorithm. Please correct the display and ensure Algorithm 1 matches the proof.
  2. [Section 4.C, Theorem 3 proof, after Eq. (31)] The proof applies Lemma 8 to the stochastic model F_{i,X_k} at the point \tilde X_{i,k} = X_{i,k} + v_{i,k}, which lies in the affine tangent space X_{i,k}+T_{X_{i,k}}M, not on M. Lemma 8 is stated for X,Y ∈ M, and the Riemannian subdifferential ∂_R is defined only at points on M (Section 2.B). Lemma 9's optimality condition gives 0 = β_k v_{i,k} + u_{i,k} with u_{i,k} the tangent projection of a Euclidean subgradient of F_{i,X_k}(·,ξ) at \tilde X_{i,k}; this does not imply u_{i,k} ∈ ∂_R F_{i,X_k}(\tilde X_{i,k},ξ). Consequently the inequality displayed before (32), and hence the term3 bound in Theorem 3, lacks stated justification. A correct argument must either extend Lemma 8 to off-manifold arguments, controlling the normal component via Lemma 3, or change the subproblem optimality proof. Such an extension will likely introduce extra O(κ_g L ||v||^2) terms and may modify the constants
minor comments (6)
  1. [Section 2.B] Typo: 'Remannian' should be 'Riemannian'.
  2. [Section 4.C, Eq. (31)] In the line defining u_{i,k}, the text reads 'u_{i,k} ∈ ∂_R F_{i,X_k}(\tilde X_{i,k}+1, ξ_{i,k})'; the '+1' appears spurious and should be removed.
  3. [Appendix B, Theorem 1 proof] The chain 'd_M(X_{i,k0+1}, \hat X_{k0+1}) = d_M(... , X_{i,k0}) + d_M(X_{i,k0}, \hat X_{k0}) + d_M(\hat X_{k0}, \hat X_{k0+1})' is an equality only via the triangle inequality in the ≤ direction; replace '=' with '≤'.
  4. [Supplementary, Lemma 11 proof] Lemma 12 is quoted from [15] with a parameter R, but R is not defined in this paper. Presumably R = 1/(2κ_g) for the reach of the manifold; please state this explicitly.
  5. [Definition 4.d] For scalar-valued functions, the Lipschitz inequality should use absolute value: |F_{i,X}(X,ξ) - F_{i,X}(Y,ξ)| ≤ L ||X-Y||.
  6. [Theorem 3 statement] In the definition of e(X_k,\hat X_k), the notation mixes stacked vector norms and sums over i. It would help to define \|X_k-\hat X_k\|^2 = Σ_i \|X_{i,k}-\hat X_k\|^2 and similarly for the first-order term.

Circularity Check

0 steps flagged

No significant circularity: the convergence rates are derived from stated assumptions and internally-proved lemmas; overlapping-author citations are not load-bearing.

full rationale

The derivation chain is self-contained. The main rate (Theorem 4) rests on Lemma 8 (proved from Assumption 3 and Lemma 3), Lemma 10/Theorem 2 (proved from retraction lemmas and the consensus potential), and Lemma 11 (proved in the supplement using [15] and Lemma 3). The proximal-map Lipschitz lemma is said to follow 'a similar idea of [46, Lemma 4.2]', but the proof is supplied in the paper rather than imported, so the overlapping author on [46] is not what makes the inequality true. The stationarity measure dist(0, ∂_R f(P_λ f(X))) ≤ (1/λ)||P_λ f(X)−X|| is cited to [46] but is a standard first-order optimality statement and is not used in the proof of the convergence rate. No parameter is fitted to data, and no 'prediction' is defined in terms of a fitted quantity. The local-region assumption X0 ∈ S is an explicit hypothesis, not a hidden redeclaration of the conclusion. Even if the skeptic's technical concern about applying Lemma 8 at off-manifold points were valid, that would be a proof correctness issue, not a circular reduction.

Axiom & Free-Parameter Ledger

1 free parameters · 6 axioms · 0 invented entities

Central claim relies on standard weakly-convex/stochastic model assumptions plus a geometric curvature bound and local initialization; no invented entities or data-fitted parameters enter the proofs. Experimental constants c are hand-tuned but do not affect the theorem.

free parameters (1)
  • per-experiment step-size constant c in β_k=√(k+1)/c = c=0.2, 0.5, 0.4 depending on algorithm and graph
    Hand-tuned in Section 5 to stabilize each algorithm; not part of the theoretical convergence theorem.
axioms (6)
  • domain assumption Local costs f_i are L-Lipschitz, ρ-weakly-convex, and bounded below (Assumption 1).
    This is the function class the entire framework is designed for.
  • domain assumption Graph is undirected and connected; W is symmetric doubly stochastic with σ2<1 (Assumption 2).
    Standard consensus assumption; used in Lemmas 6–7 and the choice of t.
  • domain assumption Uniform geodesic curvature bound: ||Π(η,η)|| ≤ κ_g||η||² for all geodesic directions (Assumption 3).
    Finite for compact embedded submanifolds; carries the manifold-geometry effect through every key lemma.
  • domain assumption β_k is positive, nondecreasing, diverges, and satisfies β_{k+1}/β_k→1 (Assumption 4).
    Needed for the consensus error O(L²/β_k²) and for telescoping the stationarity bound.
  • domain assumption Each stochastic model satisfies one-sided accuracy, unbiasedness at the current point, weak convexity, and Lipschitz continuity (Definition 4).
    This model class is standard in stochastic model-based optimization; the three algorithms verify it with different τ,L.
  • ad hoc to paper Initial points lie in the local region S with δ1≤1/(20κ_g), δ2≤δ1/5 (Eq. (20), Theorem 1).
    The invariant-region argument requires this initialization; it is a restrictive local condition.

pith-pipeline@v1.3.0-alltime-deepseek · 29412 in / 23095 out tokens · 225403 ms · 2026-08-04T08:09:32.468975+00:00 · methodology

0 comments
read the original abstract

This paper aims to investigate the distributed stochastic optimization problems on compact embedded submanifolds (in the Euclidean space) where the local cost functions are weakly-convex. To address the manifold structure, we propose a distributed Riemannian stochastic proximal algorithm framework by utilizing the retraction and Riemannian consensus protocol, and analyze three specific algorithms: the distributed Riemannian stochastic subgradient, proximal point, and prox-linear algorithms. When the initial points satisfy certain conditions, we show that the iterates generated by this framework converge to a nearly stationary point in expectation while achieving consensus. We further establish the convergence rate of the algorithm framework as $\mathcal{O}(\frac{1+\kappa_g}{\sqrt{k}})$ where $k$ denotes the number of iterations and $\kappa_g$ shows the impact of manifold geometry on the algorithm performance. Finally, numerical experiments are implemented to demonstrate the theoretical results and show the empirical performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 10 linked inside Pith

  1. [1]

    Absil, R

    P.-A. Absil, R. Mahony, and Rodolphe Sepulchre.Optimization Al- gorithms on Matrix Manifolds. Princeton University Press, Princeton, 2008

    2008

  2. [2]

    An extrinsic look at the riemannian hessian

    P-A Absil, Robert Mahony, and Jochen Trumpf. An extrinsic look at the riemannian hessian. InInternational conference on geometric science of information, pages 361–368. Springer, 2013

    2013

  3. [3]

    Projection-like retractions on matrix manifolds.SIAM Journal on Optimization, 22(1):135–158, 2012

    P-A Absil and J ´erˆome Malick. Projection-like retractions on matrix manifolds.SIAM Journal on Optimization, 22(1):135–158, 2012

    2012

  4. [4]

    Distributed coupled multiagent stochastic optimization.IEEE Transactions on Automatic Control, 65(1):175–190, 2019

    Sulaiman A Alghunaim and Ali H Sayed. Distributed coupled multiagent stochastic optimization.IEEE Transactions on Automatic Control, 65(1):175–190, 2019

    2019

  5. [5]

    Stochastic gradient descent on Riemannian manifolds

    Silv `ere Bonnabel. Stochastic gradient descent on Riemannian manifolds. IEEE Transactions on Automatic Control, 58(9):2217–2229, 2013

    2013

  6. [6]

    Large-scale machine learning with stochastic gradient descent.SIAM review, 60(2):223–311, 2018

    L ´eon Bottou, Frank E Curtis, and Jorge Nocedal. Large-scale machine learning with stochastic gradient descent.SIAM review, 60(2):223–311, 2018

    2018

  7. [7]

    Global rates of convergence for nonconvex optimization on manifolds.IMA Journal of Numerical Analysis, 39(1):1–33, 2018

    Cartis Coralia Boumal Nicolas, Absil P-A. Global rates of convergence for nonconvex optimization on manifolds.IMA Journal of Numerical Analysis, 39(1):1–33, 2018

    2018

  8. [8]

    Tsang, and Yong Liu

    Jun Chen, Haishan Ye, Mengmeng Wang, Tianxin Huang, Guang Dai, Ivor W. Tsang, and Yong Liu. Decentralized Riemannian conjugate gra- dient method on the Stiefel manifold.arXiv preprint arXiv:2308.10547, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    Decentralized Riemannian gradient descent on the Stiefel manifold

    Shixiang Chen, Alfredo Garcia, Mingyi Hong, and Shahin Shahrampour. Decentralized Riemannian gradient descent on the Stiefel manifold. In Proceedings of the 38th International Conference on Machine Learning, volume 139 ofProceedings of Machine Learning Research, pages 1594–

  10. [10]

    Shahram- pour

    Shixiang Chen, Alfredo Garcia, Mingyi Hong, and Shahin. Shahram- pour. On the local linear rate of consensus on the Stiefel manifold. IEEE Transactions on Automatic Control, 69(4):2324–2339, 2024

    2024

  11. [11]

    Proximal gradient method for nonsmooth optimization over the stiefel manifold.SIAM Journal on Optimization, 30(1):210–239, 2020

    Shixiang Chen, Shiqian Ma, Anthony Man-Cho So, and Tong Zhang. Proximal gradient method for nonsmooth optimization over the stiefel manifold.SIAM Journal on Optimization, 30(1):210–239, 2020

    2020

  12. [12]

    Clarke, R.J

    F.H. Clarke, R.J. Stern, and P.R. Wolenski. Proximal smoothness and the lower-C2 property.Journal of Convex Analysis, 2:117–144, 1995

    1995

  13. [13]

    Stochastic model-based min- imization of weakly convex functions.SIAM Journal on Optimization, 29(1):207–239, 2019

    Damek Davis and Dmitriy Drusvyatskiy. Stochastic model-based min- imization of weakly convex functions.SIAM Journal on Optimization, 29(1):207–239, 2019

    2019

  14. [15]

    Stochastic opti- mization over proximally smooth sets.arXiv preprint arXiv:2002.06309, 2020

    Damek Davis, Dmitriy Drusvyatskiy, and Zhan Shi. Stochastic opti- mization over proximally smooth sets.arXiv preprint arXiv:2002.06309, 2020

    Pith/arXiv arXiv 2002

  15. [16]

    Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds

    Kangkang Deng and Jiang Hu. Decentralized projected Riemannian gradient method for smooth optimization on compact submanifolds. arXiv preprint arXiv:2304.08241, 2023

    Pith/arXiv arXiv 2023

  16. [17]

    Decentralized projected Riemannian stochastic recursive momentum method for smooth optimization on compact submanifolds.arXiv preprint arXiv:2412.02382, 2024

    Kangkang Deng and Jiang Hu. Decentralized projected Riemannian stochastic recursive momentum method for smooth optimization on compact submanifolds.arXiv preprint arXiv:2412.02382, 2024

    Pith/arXiv arXiv 2024

  17. [18]

    Decentralized Douglas- Rachford splitting methods for smooth optimization over compact sub- manifolds.arXiv preprint arXiv:2311.16399, 2023

    Kangkang Deng, Jiang Hu, and Hongxia Wang. Decentralized Douglas- Rachford splitting methods for smooth optimization over compact sub- manifolds.arXiv preprint arXiv:2311.16399, 2023

    Pith/arXiv arXiv 2023

  18. [19]

    Efficiency of minimizing compositions of convex functions and smooth maps.Mathematical Programming, 178:503–558, 2019

    Dmitriy Drusvyatskiy and Courtney Paquette. Efficiency of minimizing compositions of convex functions and smooth maps.Mathematical Programming, 178:503–558, 2019

    2019

  19. [20]

    Hosseini, W

    S. Hosseini, W. Huang, and R. Yousefpour. Line search algorithms for locally lipschitz functions on Riemannian manifolds.SIAM Journal on Optimization, 28(1):596–619, 2018

    2018

  20. [21]

    Decentralized Riemannian natural gradient methods with Kronecker-product approxi- mations.arXiv preprint arXiv:2303.09611, 2023

    Jiang Hu, Kangkang Deng, Na Li, and Quanzheng Li. Decentralized Riemannian natural gradient methods with Kronecker-product approxi- mations.arXiv preprint arXiv:2303.09611, 2023

    Pith/arXiv arXiv 2023

  21. [22]

    Communication-efficient distributed PCA by Riemannian optimization

    Long-Kai Huang and Sinno Pan. Communication-efficient distributed PCA by Riemannian optimization. InInternational Conference on Machine Learning, pages 4465–4474. PMLR, 2020

    2020

  22. [23]

    Riemannian stochastic recursive gradient algorithm

    Hiroyuki Kasai, Hiroyuki Sato, and Bamdev Mishra. Riemannian stochastic recursive gradient algorithm. InProceedings of the 35th Inter- national Conference on Machine Learning, volume 80 ofProceedings of Machine Learning Research, pages 2516–2524. PMLR, 10–15 Jul 2018

    2018

  23. [24]

    A decentralized algorithm for spectral analysis

    David Kempe and Frank McSherry. A decentralized algorithm for spectral analysis. InProceedings of the thirty-sixth annual ACM symposium on Theory of computing, pages 561–568, 2004

    2004

  24. [25]

    Springer, 2018

    John M Lee.Introduction to Riemannian manifolds, volume 2. Springer, 2018

    2018

  25. [26]

    Distributed variable sample-size stochastic optimization with fixed step-sizes.IEEE Transactions on Automatic Control, 67(10):5630–5637, 2022

    Jinlong Lei, Peng Yi, Jie Chen, and Yiguang Hong. Distributed variable sample-size stochastic optimization with fixed step-sizes.IEEE Transactions on Automatic Control, 67(10):5630–5637, 2022

    2022

  26. [27]

    Weakly convex optimization over Stiefel man- ifold using Riemannian subgradient-type methods.SIAM Journal on Optimization, 31(3):1605–1634, 2021

    Xiao Li, Shixiang Chen, Zengde Deng, Qing Qu, Zhihui Zhu, and Anthony Man-Cho So. Weakly convex optimization over Stiefel man- ifold using Riemannian subgradient-type methods.SIAM Journal on Optimization, 31(3):1605–1634, 2021

    2021

  27. [28]

    Enhancing LLM complex reasoning capability through hyperbolic geometry

    Yang Menglin, Feng Aosong, Xiong Bo, Liu Jiahong, King Irwin, and Ying Rex. Enhancing LLM complex reasoning capability through hyperbolic geometry. InICML 2024 Workshop on LLMs and Cognition Poster, Jun 2024

    2024

  28. [29]

    Robust stochastic approximation approach to stochastic pro- gramming.SIAM Journal on Optimization, 19(4):1574–1609, 2009

    Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan, and Alexander Shapiro. Robust stochastic approximation approach to stochastic pro- gramming.SIAM Journal on Optimization, 19(4):1574–1609, 2009

    2009

  29. [30]

    Springer Science & Business Media, 2013

    Yurii Nesterov.Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013

    2013

  30. [31]

    Tsitsiklis

    Alex Olshevsky and John N. Tsitsiklis. Convergence speed in distributed consensus and averaging.SIAM Journal on Control and Optimization, 48(1):33–55, 2009

    2009

  31. [32]

    Pillai, T

    S.U. Pillai, T. Suel, and Seunghun Cha. The perron-frobenius theorem: some of its applications.IEEE Signal Processing Magazine, 22(2):62– 75, 2005

    2005

  32. [33]

    Distributed stochastic gradient tracking methods.Mathematical Programming, 187:409–457, 2021

    Shi Pu and Angelia Nedi ´c. Distributed stochastic gradient tracking methods.Mathematical Programming, 187:409–457, 2021

    2021

  33. [34]

    A stochastic approximation method

    Herbert Robbins and Sutton Monro. A stochastic approximation method. The Annals of Mathematical Statistics, pages 400–407, 1951

    1951

  34. [35]

    R Tyrrell Rockafellar and Roger J-B Wets.Variational analysis, volume

  35. [36]

    Consensus optimization on manifolds.SIAM Journal on Control and Optimization, 48(1):56–76, 2009

    Alain Sarlette and Rodolphe Sepulchre. Consensus optimization on manifolds.SIAM Journal on Control and Optimization, 48(1):56–76, 2009

    2009

  36. [37]

    Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport.SIAM Journal on Optimization, 29(2):1444–1472, 2019

    Hiroyuki Sato, Hiroyuki Kasai, and Bamdev Mishra. Riemannian stochastic variance reduced gradient algorithm with retraction and vector transport.SIAM Journal on Optimization, 29(2):1444–1472, 2019

    2019

  37. [38]

    Adaptation, learning, and optimization over networks

    Ali H Sayed et al. Adaptation, learning, and optimization over networks. Foundations and Trends® in Machine Learning, 7(4-5):311–801, 2014

    2014

  38. [39]

    A proximal gradient algorithm for decentralized composite optimization.IEEE Transactions on Signal Processing, 63(22):6013–6023, 2015

    Wei Shi, Qing Ling, Gang Wu, and Wotao Yin. A proximal gradient algorithm for decentralized composite optimization.IEEE Transactions on Signal Processing, 63(22):6013–6023, 2015

    2015

  39. [40]

    Riemannian optimization with a preconditioning scheme on the generalized stiefel manifold.Journal of Computational and Applied Mathematics, 423:114953, 2023

    Boris Shustin and Haim Avron. Riemannian optimization with a preconditioning scheme on the generalized stiefel manifold.Journal of Computational and Applied Mathematics, 423:114953, 2023

    2023

  40. [41]

    Distributed asynchronous con- strained stochastic optimization.IEEE Journal of Selected Topics in Signal Processing, 5(4):772–790, 2011

    Kunal Srivastava and Angelia Nedic. Distributed asynchronous con- strained stochastic optimization.IEEE Journal of Selected Topics in Signal Processing, 5(4):772–790, 2011

    2011

  41. [42]

    Nilesh Tripuraneni, Nicolas Flammarion, Francis Bach, and Michael I. Jordan. Averaging stochastic gradient descent on Riemannian manifolds. InProceedings of the 31st Conference On Learning Theory, volume 75 ofProceedings of Machine Learning Research, pages 650–687. PMLR, 06–09 Jul 2018

    2018

  42. [43]

    Strong and weak convexity of sets and functions

    Jean-Philippe Vial. Strong and weak convexity of sets and functions. Mathematics of Operations Research, 8(2):231–259, 1983

    1983

  43. [44]

    On orthogonality and learning recurrent networks with long term de- pendencies

    Eugene V orontsov, Chiheb Trabelsi, Samuel Kadoury, and Chris Pal. On orthogonality and learning recurrent networks with long term de- pendencies. InInternational Conference on Machine Learning, pages 3570–3578. PMLR, 2017

    2017

  44. [45]

    Riemannian stochastic proximal gradient methods for nonsmooth optimization over the stiefel manifold.Journal of Machine Learning Research, 23(106):1–33, 2022

    Bokun Wang, Shiqian Ma, and Lingzhou Xue. Riemannian stochastic proximal gradient methods for nonsmooth optimization over the stiefel manifold.Journal of Machine Learning Research, 23(106):1–33, 2022. 16 IEEE TRANSACTIONS ON AUTOMATIC CONTROL

    2022

  45. [46]

    Decentralized weakly convex optimization over the Stiefel manifold.arXiv preprint arXiv:2303.17779, 2023

    Jinxin Wang, Jiang Hu, Shixiang Chen, Zengde Deng, and Anthony Man-Cho So. Decentralized weakly convex optimization over the Stiefel manifold.arXiv preprint arXiv:2303.17779, 2023

    Pith/arXiv arXiv 2023

  46. [47]

    A decentralized proximal gradient tracking algorithm for composite optimization on Riemannian manifolds

    Lei Wang, Le Bao, and Xin Liu. A decentralized proximal gradient tracking algorithm for composite optimization on Riemannian manifolds. arXiv preprint arXiv:2401.11573, 2024

    Pith/arXiv arXiv 2024

  47. [48]

    Decentralized optimization over the Stiefel manifold by an approximate augmented lagrangian function.IEEE Transactions on Signal Processing, 70:3029–3041, 2022

    Lei Wang and Xin Liu. Decentralized optimization over the Stiefel manifold by an approximate augmented lagrangian function.IEEE Transactions on Signal Processing, 70:3029–3041, 2022

    2022

  48. [49]

    A variance-reduced stochastic gradient tracking algorithm for decentralized optimization with orthogonality constraints

    Lei Wang and Xin Liu. A variance-reduced stochastic gradient tracking algorithm for decentralized optimization with orthogonality constraints. arXiv preprint arXiv:2208.13643, 2022

    Pith/arXiv arXiv 2022

  49. [50]

    A feasible method for optimization with orthogonality constraints.Mathematical Programming, 142(1):397–434, 2013

    Zaiwen Wen and Wotao Yin. A feasible method for optimization with orthogonality constraints.Mathematical Programming, 142(1):397–434, 2013

    2013

  50. [51]

    Optimality conditions for the nonlinear programming problems on Riemannian manifolds

    Wei Hong Yang, Lei-Hong Zhang, and Ruyi Song. Optimality conditions for the nonlinear programming problems on Riemannian manifolds. Pacific Journal of Optimization, 10:415–434, 2014

    2014

  51. [52]

    Reddi, and Suvrit Sra

    Hongyi Zhang, Sashank J. Reddi, and Suvrit Sra. Riemannian SVRG: Fast stochastic optimization on riemannian manifolds. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 29. Curran Asso- ciates, Inc., 2016

    2016

  52. [53]

    On the global geometry of sphere-constrained sparse blind deconvolution

    Yuqian Zhang, Yenson Lau, Han-wen Kuo, Sky Cheung, Abhay Pasu- pathy, and John Wright. On the global geometry of sphere-constrained sparse blind deconvolution. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017

    2017

  53. [54]

    Distributed Riemannian stochas- tic gradient tracking algorithm on the Stiefel manifold.arXiv preprint arXiv:2405.16900, 2024

    Jishu Zhao, Xi Wang, and Jinlong Lei. Distributed Riemannian stochas- tic gradient tracking algorithm on the Stiefel manifold.arXiv preprint arXiv:2405.16900, 2024. Jishu Zhaoreceived the B.S. degree from Sichuan University in Mathematics, Sichuan, China, in 2021. She is currently working toward the Ph.D degree in control theory and control engineering wi...

    Pith/arXiv arXiv 2024

  54. [57]

    Proof of Lemma 5: For eachi, there exists a unique geodesicγ i : [0,1]7→ M connectingX i and ˆX

    SUPPLEMENTARY: ADDITIONALPROOF OFLEMMAS A. Proof of Lemma 5: For eachi, there exists a unique geodesicγ i : [0,1]7→ M connectingX i and ˆX. We denote the tangent vector along eachγ i(t)asη i(t), then it implies Xi − ˆX= Z 1 0 ηi(t)dt,∀i∈ N.(45) Utilizing this result yields∥ ˆX− ¯X∥ 2 =⟨ ˆX− ¯X, ˆX− ¯X⟩= 1 N PN i=1⟨ ˆX− ¯X, R 1 0 ηi(t)dt⟩. By the first-ord...

  55. [317]

    Springer Science and Business Media, 2009

    2009

  56. [1605]

    PMLR, 18–24 Jul 2021

    2021