Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims REXTRA, a Riemannian extension of EXTRA, converges to a stationary point of decentralized manifold optimization at rate $O(1/k)$ with a constant step size and a single communication round per iteration.

desk verdict The algorithm is sensible and the experiments are good, but the main proof relies on a spectral norm bound that is plainly false, so the advertised O(1/k) guarantee is unsupported. read the letter →

arxiv 2505.15537 v1 pith:WPCMEDHF submitted 2025-05-21 math.OC

Jiayuan Wu , Zhanwang Deng , Jiang Hu , Weijie Su , Zaiwen Wen This is my paper
classification math.OC MSC 90C2690C3068W15
keywords decentralizedoptimizationRiemannianmanifoldEXTRAproximalsmoothnessStiefeldataheterogeneitycommunicationefficiencynonconvex
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

REXTRA is a proposed Riemannian extension of the EXTRA algorithm for decentralized optimization over compact submanifolds such as the Stiefel and Grassmann manifolds, where each agent holds a smooth, nonconvex, private-data objective. The central claim is that agents can reach an $\epsilon$-stationary point of the consensus-constrained problem in $O(1/\epsilon)$ iterations using a constant step size while exchanging only local iterates, not gradients, once per iteration. If true, this matches the best known iteration complexity for decentralized manifold methods while halving the per-round communication burden, and it would be the first such guarantee with a single communication round. The paper also reports experiments on decentralized PCA and low-rank matrix completion where REXTRA supports larger step sizes and reduces total communication by more than 50%.

What carries the argument

The load-bearing mechanism is a joint invariant neighborhood $\mathcal{N}(\delta)=\{(x,s):\|(x-\bar x,\,s+\alpha\hat g)\le\delta\}$ around the consensus manifold. The proof tries to show that, provided the block operator $Q=\begin{bmatrix} W-J & I \\ W-V & I-J \end{bmatrix}$ has squared norm $\nu<1$, the next error is bounded by $\bar\nu=\nu+12\delta$ times the current error plus step-size terms. Proximal smoothness, meaning the projection $P_{\mathcal M}$ is Lipschitz with constant $1/(1-3\delta)$ inside a tube around the manifold, is used to absorb the projection error into this recursion. This invariant-neighborhood control, together with a sufficient-decrease inequality for $f(\bar x_k)$, is what carries the $O(1/k)$ conclusion.

What would settle it

Take any zero-mean vector $v$ and form $y=[0;v]$. Direct computation gives $Qy=[v;v]$, so $\|Qy\|=\sqrt{2}\|v\|$ and hence $\|Q\|_2\ge\sqrt{2}>1$. Computing the spectral norm of $Q$ for a connected $W$ with the paper's choice $V=\theta I+(1-\theta)W$, $\theta\in(0,1/2]$, settles the claimed $\nu<1$ numerically for any network.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the Euclidean EXTRA update can be transplanted to a compact submanifold by replacing the consensus-plus-correction step with projection onto $\mathcal{M}^n$ and using Riemannian gradients in the correction term: $x_{k+1}=P_{\mathcal{M}^n}(Wx_k+s_k)$ and $s_{k+1}=(W-V)x_k+s_k-\alpha(\operatorname{grad} f(x_{k+1})-\operatorname{grad} f(x_k))$. The proof claims that proximal smoothness of the manifold keeps the joint error, consisting of the consensus gap and the gradient-correction mismatch, inside a fixed neighborhood, yielding a sufficient decrease of the averaged objective. From this, Theorem 3.1 concludes $\min_{k\le K}(1/n)\|x_k-\bar x_k\|^2=O(\alpha/K)$ and $\min_{k\le K}\|\operatorname{grad} f(\bar x_k)\|^2=O(1/(\alpha K))$, an $O(1/K)$ iteration complexity under constant step size. The authors state this is the first such global sublinear rate with a single round of local-iterate communication per iteration.

Load-bearing premise

The proof of Lemma 3.1 assumes the block matrix $Q=\begin{bmatrix} W-J & I \\ W-V & I-J \end{bmatrix}$ has spectral norm $\nu<1$, but for any zero-mean vector $v$, $Q[0;v]=[v;v]$, so its norm is at least $\sqrt{2}$; the invariant-neighborhood argument for Theorem 3.1 therefore fails as written.

Editorial extensions

If this is right

  • If the proof is correct, REXTRA matches the $O(1/k)$ complexity of DPRGT and DRGTA while requiring only half the per-iteration communication, since agents exchange local iterates but not local directions.
  • Constant step sizes become feasible for data-heterogeneous decentralized manifold problems without multi-round consensus, reducing tuning cost and communication rounds.
  • The guarantees apply to any compact proximally smooth submanifold admitting closed-form projection, including Stiefel, Grassmann, and oblique manifolds, so decentralized PCA and low-rank matrix completion inherit the communication savings.
  • The experiments indicate that REXTRA can use step sizes 5 to 50 times larger than competing methods and reach the target accuracy with over 50% less total communication on the tested problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the claimed contraction $\nu<1$ for the block operator $Q$ is inconsistent with the update structure, since $Q[0;v]=[v;v]$ for any zero-mean $v$, giving $\|Q\|_2\ge\sqrt{2}$; therefore the advertised first $O(1/k)$ single-round guarantee should be read as conditional on a repaired invariant-neighborhood proof.
  • A natural repair would be to use a transformed error variable that genuinely contracts, such as one aligned with the tangent components of the consensus dynamics, rather than the raw $(x-\bar x,\,s+\alpha\hat g)$ pair.
  • The same proximal-smoothness toolkit likely extends to retraction-based updates for manifolds without closed-form projection, since the paper explicitly connects projection and retraction; a testable extension is REXTRA with a generic retraction on the oblique manifold.
  • The reported communication advantage over gradient-tracking methods depends partly on the graph: on sparse networks such as a ring, the advantage may narrow, and the paper's own graphs show faster consensus on denser Erdős–Rényi graphs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes REXTRA, a Riemannian extension of the EXTRA algorithm for decentralized optimization over compact submanifolds under data heterogeneity. The algorithm communicates only local iterates, in a single communication round per iteration, and the authors claim a global sublinear convergence rate of O(1/k) under a constant step size. The analysis uses proximal smoothness of the manifold to control consensus and gradient errors, and the paper reports numerical experiments on decentralized PCA and low-rank matrix completion showing communication savings over baselines.

Significance. If the theoretical result were valid, REXTRA would be a valuable contribution: an O(1/k) iteration complexity with a constant step size and single-round iterate communication would improve on existing decentralized manifold algorithms that require communicating both iterates and directions or multiple consensus rounds. The experimental study is thorough and does suggest practical communication savings. However, the central convergence proof contains a false spectral-norm contraction claim, so the advertised guarantee is not established.

major comments (3)
  1. [Lemma 3.1, Eq. (19)] The proof defines Q = [[W-J, I], [W-V, I-J]] and assumes ν := ||Q||_2 < 1. This is impossible for any W satisfying Assumption 2.1. For any zero-mean vector v (Jv=0), Q[0; v] = [v; v], so ||Q[0; v]|| = sqrt(2) ||v|| and hence ||Q||_2 ≥ sqrt(2) > 1. The identity block in the upper-right corner is unavoidable in this decomposition. The citation to [42] for ν < 1 does not transfer, because the Euclidean EXTRA analysis there concerns a different block structure; the identity block arises here from the linearization in (17). Consequently the bound ||P||_2 ≤ ν + 12δ with ν < 1 is void, and the invariant-neighborhood conclusion (x_{k+1}, s_{k+1}) ∈ N(δ) is not proved.
  2. [Lemma 3.2 and Theorem 3.1] The false contraction in Lemma 3.1 propagates to the rest of the analysis. The recursion ||p_{k+1}|| ≤ ν̄ ||p_k|| + 2α||c_k|| in Eq. (20) is the basis for the sum bound (8) in Lemma 3.2 and for the final O(1/K) rates in Theorem 3.1. Since ν̄ ≥ 1 + 12δ > 1, the constants C0 and C1 derived after Eq. (22) are not valid, and the step-size condition α ≤ 1/(8L√C1) cannot be satisfied as stated. The proof does not offer an alternative argument to control the non-contractive direction, so Theorem 3.1 is unsupported.
  3. [Abstract and Section 1.2] The claim that REXTRA is the first decentralized manifold optimization algorithm to achieve a global sublinear O(1/k) rate under a constant step size with a single round of local iterate communication is based entirely on Theorem 3.1. Since that theorem is not established, this claimed contribution is not supported by the submitted text. The numerical experiments are encouraging but cannot substitute for a correct convergence guarantee.
minor comments (5)
  1. [Section 3.1] There is a typo: 'natrual' should be 'natural' in the sentence introducing the Riemannian extension.
  2. [Section 3.2] The phrase 'Donote ˆx = 1⊗ˆx' appears to contain a typo; it should likely be 'Denote'.
  3. [Proof of Lemma 3.1, Eq. (19)] The stacked vector in the rightmost term of Eq. (19) is written with s_k − αĝ_k, while the neighborhood definition and the preceding derivation use s_k + αĝ_k. This sign inconsistency makes the algebra difficult to follow and should be corrected.
  4. [Lemma 3.2, Eq. (22)] The summation indices in Eq. (22) are inconsistent: the left-hand side sums from k=1 to K for ∥x_{k+1}−x̄_{k+1}∥² and ∥s_{k+1}∥², while the right-hand side sums ∥ĝ_k∥² from k=0 to K. The proof of Theorem 3.1 later uses (8) with sums from 0 to K, so the off-by-one bookkeeping needs to be fixed.
  5. [Figures] Several figure labels contain the typo 'Commnuication'; additionally, in Section 4.1.2 the text reads 'REXTRA abd DPRGT' instead of 'REXTRA and DPRGT'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the REXTRA convergence proof derives its O(1/k) rate from explicit consensus and sufficient-decrease bounds, and the self-citations supply only background geometric lemmas.

full rationale

I walked the derivation chain from Algorithm 1 through Lemma 3.1, Lemma 3.2, Lemma A.5, and Theorem 3.1. The claimed O(1/k) rate is obtained by combining three structurally independent ingredients: an invariant-neighborhood contraction bound on the joint iterate-shift pair (Lemma 3.1), a consensus-error bound in terms of the averaged gradient (Lemma 3.2), and a sufficient-decrease inequality for the manifold-averaged objective (Lemma A.5). Each of these is proved by explicit inequalities with stated constants (C0, C1, D1, D2, D3, etc.), and no parameter is fitted to the quantities that Theorem 3.1 later predicts; the theorem is not an identity or a restatement of an input. The same-group citations ([29], [43], and related works) are used for background facts about proximal smoothness, projection Lipschitz behavior, and tangent-space projection constants; those facts do not contain the REXTRA convergence conclusion, so the self-citations are not load-bearing in a circular sense. The citation of [42] for the spectral bound ν < 1 refers to an external Euclidean EXTRA analysis; whether that bound transfers to the Riemannian block matrix Q is a correctness question, not a circularity one. In fact, the proof of Lemma 3.1 defines Q with an identity block and assumes ||Q||_2 < 1, which is unsatisfiable because Q[0; v] = [v; v] for zero-mean v gives ||Q||_2 ≥ sqrt(2) > 1; this is an internal mathematical gap that invalidates Theorem 3.1 as written, but a false or inapplicable assumption is not equivalent to assuming the desired conclusion. Consequently, the paper does not exhibit self-definition, fitted-input-as-prediction, or a self-citation chain that forces its central claim.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The proof relies on standard manifold and network assumptions plus one invalid contraction claim. No new physical or mathematical entities are introduced.

free parameters (2)
  • step size alpha = grid-searched, e.g., 1e-2 in synthetic PCA
    The theorem requires alpha below constants that depend on unquantified quantities (Q, L, delta); experiments tune alpha by grid search per algorithm and dataset.
  • neighborhood radius delta = not specified numerically
    Introduced in Lemma 3.1; must satisfy delta < min{1/6, (1-nu)/12}, but nu is not computed in the paper and is in fact claimed incorrectly.
assumptions (5)
  • domain assumption M is R-proximally smooth for some R > 0
    Section 2.1 assumes this; it holds for compact C^2 submanifolds such as Stiefel, but is still an assumption on the problem class.
  • domain assumption Graph G is connected and mixing matrix W is symmetric, doubly stochastic, with second singular value less than 1
    Assumption 2.1, standard for consensus-based decentralized methods.
  • domain assumption Each f_i is L_f-smooth over conv(M) and has bounded Euclidean gradient on M
    Assumption 3.1; needed for Lipschitz properties of Riemannian gradients and descent bounds.
  • standard math The projection P_M is smooth enough in the tube to satisfy Lemma 2.1 with constant Q
    Cited from [29, Lemma 3], based on regularity of the distance function [40].
  • ad hoc to paper The block matrix Q = [[W-J, I], [W-V, I-J]] has ||Q||_2 < 1
    Implicitly used in Lemma 3.1 via [42]; false as stated because Q contains identity blocks, so the proof of the invariant neighborhood fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity." pith.science (2026). https://pith.science/paper/WPCMEDHF

@misc{pith2026250515537,
  author       = {Pith},
  title        = {Pith review of: Riemannian EXTRA: Communication-efficient decentralized optimization over compact submanifolds with data heterogeneity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WPCMEDHF}},
  note         = {Machine review of arXiv:2505.15537}
}
abstract

We consider decentralized optimization over a compact Riemannian submanifold in a network of $n$ agents, where each agent holds a smooth, nonconvex local objective defined by its private data. The goal is to collaboratively minimize the sum of these local objective functions. In the presence of data heterogeneity across nodes, existing algorithms typically require communicating both local gradients and iterates to ensure exact convergence with constant step sizes. In this work, we propose REXTRA, a Riemannian extension of the EXTRA algorithm [Shi et al., SIOPT, 2015], to address this limitation. On the theoretical side, we leverage proximal smoothness to overcome the challenges of manifold nonconvexity and establish a global sublinear convergence rate of $\mathcal{O}(1/k)$, matching the best-known results. To our knowledge, REXTRA is the first algorithm to achieve a global sublinear convergence rate under a constant step size while requiring only a single round of local iterate communication per iteration. Numerical experiments show that REXTRA achieves superior performance compared to state-of-the-art methods, while supporting larger step sizes and reducing total communication by over 50\%.

Figures

Figures reproduced from arXiv: 2505.15537 by the authors.

Figure 1
Figure 1. Results of communication quantities for the PCA problem on the synthetic dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Results of different stepsizes for the PCA problem on the synthetic dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Results of communication quantities for the PCA problem on the Mnist dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Results of different stepsizes for the PCA problem on the Mnist dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Results of communication quantities for the LRMC problem. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Results of different stepsizes for the LRMC problem. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Results of tested algorithms on different graphs. [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]
Figure 8
Figure 8. Figure 8: Results of tested algorithms on synthetic dataset with Epoch. [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Results of tested algorithms on synthetic dataset with different batchsize. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 11
Figure 11. Figure 11: Results of tested algorithms on different graphs with real data (60000 samples, 784 [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]
Figure 10
Figure 10. Figure 10: Results of tested algorithms on real data with epochs. [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 12
Figure 12. Figure 12: Results of tested algorithms on real dataset with different batchsize. [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Results of tested algorithms on LRMC problem with epochs. [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Results of REXTRA on different graphs on LRMC problems. [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ReasFlow: Assisting Reasoning-Centric Scientific Discovery in Applied Mathematics via a Knowledge-Based Multi-Agent System

    cs.AI 2026-07 reject novelty 5.0 of 10

    An end-to-end multi-agent LLM system generates applied-mathematics papers, but the claims of rigorous, human-surpassing theory rest on self-cited companion papers and LLM judges from the same model families.

Reference graph

Works this paper leans on

45 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [42]

    Convergence Analysis of EXTRA in Non-convex Distributed Optimization

    Lei Qin and Ye Pu. Convergence analysis of extra in non-convex distributed optimization.arXiv preprint arXiv:2503.11104, 2025

    work page Pith review arXiv 2025

  2. [1]

    DeEPCA: Decentralized exact PCA with linear convergence rate

    Haishan Ye and Tong Zhang. DeEPCA: Decentralized exact PCA with linear convergence rate. The Journal of Machine Learning Research, 22(1):10777–10803, 2021

    work page 2021

  3. [2]

    Low-rank matrix completion via preconditioned optimization on the Grassmann manifold.Linear Algebra and its Applications, 475:200–239, 2015

    Nicolas Boumal and P-A Absil. Low-rank matrix completion via preconditioned optimization on the Grassmann manifold.Linear Algebra and its Applications, 475:200–239, 2015

    work page 2015

  4. [3]

    Riemannian natural gradient methods.SIAM Journal on Scientific Computing, 46(1):A204–A231, 2024

    Jiang Hu, Ruicheng Ao, Anthony Man-Cho So, Minghan Yang, and Zaiwen Wen. Riemannian natural gradient methods.SIAM Journal on Scientific Computing, 46(1):A204–A231, 2024

    work page 2024

  5. [4]

    Riemannian approach to batch normalization.Advances in Neural Information Processing Systems, 30, 2017

    Minhyung Cho and Jaehyung Lee. Riemannian approach to batch normalization.Advances in Neural Information Processing Systems, 30, 2017

    work page 2017

  6. [5]

    Unitary evolution recurrent neural networks

    Martin Arjovsky, Amar Shah, and Yoshua Bengio. Unitary evolution recurrent neural networks. InInternational conference on machine learning, pages 1120–1128. PMLR, 2016

    work page 2016

  7. [6]

    On orthogonality and learning recurrent networks with long term dependencies

    Eugene V orontsov, Chiheb Trabelsi, Samuel Kadoury, and Chris Pal. On orthogonality and learning recurrent networks with long term dependencies. InInternational Conference on Machine Learning, pages 3570–3578. PMLR, 2017

    work page 2017

  8. [7]

    Orthogonal weight normalization: Solution to optimization over multiple dependent Stiefel manifolds in deep neural networks

    Lei Huang, Xianglong Liu, Bo Lang, Adams Yu, Yongliang Wang, and Bo Li. Orthogonal weight normalization: Solution to optimization over multiple dependent Stiefel manifolds in deep neural networks. InProceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018

    work page 2018

Show all 45 references
  1. [8]

    Understanding how orthogonality of parameters improves quantization of neural networks.IEEE Transactions on Neural Networks and Learning Systems, 2022

    Sukru Burc Eryilmaz and Aysegul Dundar. Understanding how orthogonality of parameters improves quantization of neural networks.IEEE Transactions on Neural Networks and Learning Systems, 2022

    2022

  2. [9]

    Riemannian preconditioned lora for fine-tuning foundation models

    Fangzhao Zhang and Mert Pilanci. Riemannian preconditioned lora for fine-tuning foundation models. InForty-first International Conference on Machine Learning

  3. [10]

    Retraction-free optimization over the stiefel manifold with application to the lora fine-tuning

    Yuan Zhang, Jiang Hu, Jiaxi Cui, Lin Lin, Zaiwen Wen, Quanzheng Li, et al. Retraction-free optimization over the stiefel manifold with application to the lora fine-tuning. 2024

    2024

  4. [11]

    Distributed subgradient methods for multi-agent opti- mization.IEEE Transactions on Automatic Control, 54(1):48, 2009

    Angelia Nedic and Asuman Ozdaglar. Distributed subgradient methods for multi-agent opti- mization.IEEE Transactions on Automatic Control, 54(1):48, 2009

    2009

  5. [12]

    Distributed asynchronous deterministic and stochastic gradient optimization algorithms.IEEE Transactions on Automatic Control, 31 (9):803–812, 1986

    John Tsitsiklis, Dimitri Bertsekas, and Michael Athans. Distributed asynchronous deterministic and stochastic gradient optimization algorithms.IEEE Transactions on Automatic Control, 31 (9):803–812, 1986

    1986

  6. [13]

    On the convergence of decentralized gradient descent

    Kun Yuan, Qing Ling, and Wotao Yin. On the convergence of decentralized gradient descent. SIAM Journal on Optimization, 26(3):1835–1854, 2016

    2016

  7. [14]

    EXTRA: An exact first-order algorithm for decentralized consensus optimization.SIAM Journal on Optimization, 25(2):944–966, 2015

    Wei Shi, Qing Ling, Gang Wu, and Wotao Yin. EXTRA: An exact first-order algorithm for decentralized consensus optimization.SIAM Journal on Optimization, 25(2):944–966, 2015

    2015

  8. [15]

    DLM: Decentralized linearized al- ternating direction method of multipliers.IEEE Transactions on Signal Processing, 63(15): 4051–4064, 2015

    Qing Ling, Wei Shi, Gang Wu, and Alejandro Ribeiro. DLM: Decentralized linearized al- ternating direction method of multipliers.IEEE Transactions on Signal Processing, 63(15): 4051–4064, 2015

    2015

  9. [16]

    Exact diffusion for distributed op- timization and learning Part II: Convergence analysis.IEEE Transactions on Signal Processing, 67(3):724–739, 2018

    Kun Yuan, Bicheng Ying, Xiaochuan Zhao, and Ali H Sayed. Exact diffusion for distributed op- timization and learning Part II: Convergence analysis.IEEE Transactions on Signal Processing, 67(3):724–739, 2018

    2018

  10. [17]

    A decentralized proximal-gradient method with network inde- pendent step-sizes and separated convergence rates.IEEE Transactions on Signal Processing, 67(17):4494–4506, 2019

    Zhi Li, Wei Shi, and Ming Yan. A decentralized proximal-gradient method with network inde- pendent step-sizes and separated convergence rates.IEEE Transactions on Signal Processing, 67(17):4494–4506, 2019

    2019

  11. [18]

    Convergence of a multi-agent projected stochastic gradient algorithm for non-convex optimization.IEEE Transactions on Automatic Control, 58 (2):391–405, 2012

    Pascal Bianchi and Jérémie Jakubowicz. Convergence of a multi-agent projected stochastic gradient algorithm for non-convex optimization.IEEE Transactions on Automatic Control, 58 (2):391–405, 2012. 11

    2012

  12. [19]

    NEXT: In-network nonconvex optimization.IEEE Transactions on Signal and Information Processing over Networks, 2(2):120–136, 2016

    Paolo Di Lorenzo and Gesualdo Scutari. NEXT: In-network nonconvex optimization.IEEE Transactions on Signal and Information Processing over Networks, 2(2):120–136, 2016

    2016

  13. [20]

    Prox-PDA: The proximal primal- dual algorithm for fast distributed nonconvex optimization and learning over networks

    Mingyi Hong, Davood Hajinezhad, and Ming-Min Zhao. Prox-PDA: The proximal primal- dual algorithm for fast distributed nonconvex optimization and learning over networks. In International Conference on Machine Learning, pages 1529–1538. PMLR, 2017

    2017

  14. [21]

    Non-convex distributed optimization.IEEE Transactions on Automatic Control, 62(8):3744–3757, 2017

    Tatiana Tatarenko and Behrouz Touri. Non-convex distributed optimization.IEEE Transactions on Automatic Control, 62(8):3744–3757, 2017

    2017

  15. [22]

    Distributed nonconvex constrained optimization over time- varying digraphs.Mathematical Programming, 176(1-2):497–544, 2019

    Gesualdo Scutari and Ying Sun. Distributed nonconvex constrained optimization over time- varying digraphs.Mathematical Programming, 176(1-2):497–544, 2019

    2019

  16. [23]

    Improving the sample and communication complexity for decentralized non-convex optimization: Joint gradient estimation and tracking

    Haoran Sun, Songtao Lu, and Mingyi Hong. Improving the sample and communication complexity for decentralized non-convex optimization: Joint gradient estimation and tracking. InInternational Conference on Machine Learning, pages 9217–9228. PMLR, 2020

    2020

  17. [24]

    Decentralized Frank–Wolfe algorithm for convex and nonconvex problems.IEEE Transactions on Automatic Control, 62 (11):5522–5537, 2017

    Hoi-To Wai, Jean Lafond, Anna Scaglione, and Eric Moulines. Decentralized Frank–Wolfe algorithm for convex and nonconvex problems.IEEE Transactions on Automatic Control, 62 (11):5522–5537, 2017

    2017

  18. [25]

    On nonconvex decentralized gradient descent.IEEE Transactions on Signal Processing, 66(11):2834–2848, 2018

    Jinshan Zeng and Wotao Yin. On nonconvex decentralized gradient descent.IEEE Transactions on Signal Processing, 66(11):2834–2848, 2018

    2018

  19. [26]

    Riemannian consensus for manifolds with bounded curvature.IEEE Transactions on Automatic Control, 58(4):921–934, 2012

    Roberto Tron, Bijan Afsari, and René Vidal. Riemannian consensus for manifolds with bounded curvature.IEEE Transactions on Automatic Control, 58(4):921–934, 2012

    2012

  20. [27]

    On the local linear rate of consensus on the Stiefel manifold.IEEE Transactions on Automatic Control, 2023

    Shixiang Chen, Alfredo Garcia, Mingyi Hong, and Shahin Shahrampour. On the local linear rate of consensus on the Stiefel manifold.IEEE Transactions on Automatic Control, 2023

    2023

  21. [28]

    Decentralized Rie- mannian gradient descent on the Stiefel manifold

    Shixiang Chen, Alfredo Garcia, Mingyi Hong, and Shahin Shahrampour. Decentralized Rie- mannian gradient descent on the Stiefel manifold. InInternational Conference on Machine Learning, pages 1594–1605. PMLR, 2021

    2021

  22. [29]

    Decentralized projected riemannian gradient method for smooth optimization on compact submanifolds.arXiv preprint arXiv:2304.08241, 2023

    Kangkang Deng and Jiang Hu. Decentralized projected riemannian gradient method for smooth optimization on compact submanifolds.arXiv preprint arXiv:2304.08241, 2023

    2023 arXiv

  23. [30]

    Decentralized Riemannian conjugate gradient method on the Stiefel manifold

    Jun Chen, Haishan Ye, Mengmeng Wang, Tianxin Huang, Guang Dai, Ivor Tsang, and Yong Liu. Decentralized Riemannian conjugate gradient method on the Stiefel manifold. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  24. [31]

    Decentralized Riemannian natural gradient methods with Kronecker-product approximations.arXiv:2303.09611, 2023

    Jiang Hu, Kangkang Deng, Na Li, and Quanzheng Li. Decentralized Riemannian natural gradient methods with Kronecker-product approximations.arXiv:2303.09611, 2023

    2023 arXiv

  25. [32]

    Improving the communication in decentralized manifold optimization through single-step consensus and compression.arXiv preprint arXiv:2407.08904, 2024

    Jiang Hu and Kangkang Deng. Improving the communication in decentralized manifold optimization through single-step consensus and compression.arXiv preprint arXiv:2407.08904, 2024

    2024 arXiv

  26. [33]

    Decentralized optimization over the Stiefel manifold by an approximate augmented Lagrangian function.IEEE Transactions on Signal Processing, 70:3029–3041, 2022

    Lei Wang and Xin Liu. Decentralized optimization over the Stiefel manifold by an approximate augmented Lagrangian function.IEEE Transactions on Signal Processing, 70:3029–3041, 2022

    2022

  27. [34]

    Global conver- gence of decentralized retraction-free optimization on the stiefel manifold.arXiv preprint arXiv:2405.11590, 2024

    Youbang Sun, Shixiang Chen, Alfredo Garcia, and Shahin Shahrampour. Global conver- gence of decentralized retraction-free optimization on the stiefel manifold.arXiv preprint arXiv:2405.11590, 2024

    2024 arXiv

  28. [35]

    Local linear conver- gence of infeasible optimization with orthogonal constraints.IEEE Control Systems Letters, 2024

    Youbang Sun, Shixiang Chen, Alfredo Garcia, and Shahin Shahrampour. Local linear conver- gence of infeasible optimization with orthogonal constraints.IEEE Control Systems Letters, 2024

    2024

  29. [36]

    Princeton University Press, 2009

    P-A Absil, Robert Mahony, and Rodolphe Sepulchre.Optimization Algorithms on Matrix Manifolds. Princeton University Press, 2009

    2009

  30. [37]

    Proximal smoothness and the lower-C2 property.Journal of Convex Analysis, 2(1-2):117–144, 1995

    Francis H Clarke, Ronald J Stern, and Peter R Wolenski. Proximal smoothness and the lower-C2 property.Journal of Convex Analysis, 2(1-2):117–144, 1995. 12

    1995

  31. [38]

    The gradient projection method with Armijo’s step size on manifolds.Computational Mathematics and Mathematical Physics, 61:1776–1786, 2021

    MV Balashov and RA Kamalov. The gradient projection method with Armijo’s step size on manifolds.Computational Mathematics and Mathematical Physics, 61:1776–1786, 2021

    2021

  32. [39]

    Stochastic optimization over proximally smooth sets.arXiv:2002.06309, 2020

    Damek Davis, Dmitriy Drusvyatskiy, and Zhan Shi. Stochastic optimization over proximally smooth sets.arXiv:2002.06309, 2020

    2002 arXiv

  33. [40]

    Regularity of the distance function.Proceedings of the American Mathematical Society, 92(1):153–155, 1984

    Robert L Foote. Regularity of the distance function.Proceedings of the American Mathematical Society, 92(1):153–155, 1984

    1984

  34. [41]

    Consensus optimization on manifolds.SIAM Journal on Control and Optimization, 48(1):56–76, 2009

    Alain Sarlette and Rodolphe Sepulchre. Consensus optimization on manifolds.SIAM Journal on Control and Optimization, 48(1):56–76, 2009

    2009

  35. [43]

    Achieving consensus over compact submani- folds.arXiv preprint arXiv:2306.04769, 2023

    Jiang Hu, Jiaojiao Zhang, and Kangkang Deng. Achieving consensus over compact submani- folds.arXiv preprint arXiv:2306.04769, 2023

    2023 arXiv

  36. [44]

    The mnist database of handwritten digits.http://yann

    Yann LeCun. The mnist database of handwritten digits.http://yann. lecun. com/exdb/mnist/, 1998

    1998

  37. [45]

    Augmented distributed gradi- ent methods for multi-agent optimization under uncoordinated constant stepsizes

    Jinming Xu, Shanying Zhu, Yeng Chai Soh, and Lihua Xie. Augmented distributed gradi- ent methods for multi-agent optimization under uncoordinated constant stepsizes. InIEEE Conference on Decision and Control, pages 2055–2060, 2015. 13 A Proof of the main results To facilitate ...

    2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.