Pith. sign in

REVIEW 4 major objections 4 minor 43 references

Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a direction-aware quantizer lets decentralized manifold optimization match the O(1/K) rate of full-precision RGT, without retraction or projection.

desk verdict The quantizer is biased, so the central O(1/K) theorem is unsupported; the algorithm idea is new but the proof collapses at Lemma 3. read the letter →

arxiv 2506.07351 v1 pith:AR2S7CIW submitted 2025-06-09 math.OC cs.LGcs.SYeess.SY

Jun Chen , Lina Liu , Tianyi Zhu , Yong Liu , Guang Dai , Yunliang Jiang , Ivor W. Tsang This is my paper
classification math.OCcs.LGcs.SYeess.SY MSC 90C2690C3068W15
keywords distributedoptimizationquantizationRiemanniangradienttrackingcompactsubmanifoldsretraction-freeStiefelmanifoldconsensusconvergencerate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that decentralized optimization on compact submanifolds can be done with quantized gradient communication at no loss of convergence rate. It proposes Q-RGT, a gradient-tracking algorithm in which each agent sends an N-bit quantized version of its Riemannian gradient, chosen by rounding up or down according to the sign of the distance-to-manifold function. This quantization is designed to play the role of a landing field, so iterates stay close to the manifold without retraction or projection. The central result is a min-iterate bound $\min_{k} \|\mathbb{E}[\Gamma(\bar x_k)]\|^2 = O(1/(\alpha K))$, matching the unquantized RGT rate, plus an $O(1/K)$ consensus bound. The claim matters because communication bandwidth, not compute, is often the bottleneck in distributed learning and signal processing.

What carries the argument

The central object is the quantized Riemannian gradient $\Gamma(x) = Q_N(\operatorname{grad} f(x))$ with the landing-field rounding rule. It replaces retraction and projection: the round-up/down decision is steered by $\nabla N(x)$, the gradient of squared distance to the manifold, so the quantized gradient itself contains a term proportional to $\nabla N(x)$ that pulls iterates toward the quantized region $\mathcal M_{\Omega(N)}$, the set of points within distance $\Omega(N)=C/(2^N-1)$ of the manifold. Gradient-tracking variables $s_{i,k}$ average the quantized gradients across the network, and the consensus lemmas bound $\|x_k - \bar x_k\|^2$ and $\|s_k - \bar s_k\|^2$ through a linear system whose contractivity is ensured by the step-size condition $\alpha \le (1-\sigma_2)^2/(16L_m)$. The proof of the $O(1/K)$ rate rests on Lemma 3, whose descent inequality uses $E[\Gamma(\bar x_{k-1})] = \operatorname{grad} f(\bar x_{k-1})$ from Assumption 3.

What would settle it

Compute $\mathbb{E}[Q_N(g)]$ directly for a scalar gradient $g$ using the paper's Eq. (18): with $b=\lfloor \mathrm{sigmoid}(\nabla N(x))\rceil \in \{0,1\}$, the expectation is $g + \gamma(g)(b-\tfrac{1}{2})/(2^N-1)$, which differs from $g$ whenever $b=0$ or $1$; evaluating this at any point with a nonzero $\nabla N(x)$ would refute Assumption 3, and a one-agent experiment comparing $\|\operatorname{grad} f(\bar x_k)\|$ with $\|\mathbb{E}[\Gamma(\bar x_k)]\|$ would show whether the claimed rate is actually achieved.

Watch

Extended reading notes

Core claim

The central discovery the paper puts forward is that a direction-aware uniform quantizer can act as a stochastic landing field. For a gradient vector, $Q_N(\cdot)$ rounds each entry toward the nearest quantization-grid point, with the round-up/round-down decision steered by $\nabla N(x) = \nabla \mathrm{dist}(x,\mathcal M)^2$; a dither $U_N$ is added to keep the noise range equal to quantization error. The quantized gradient then has the form $\Gamma(x) = \operatorname{grad} f(x) + \frac{C}{2^N-1}\nabla N(x) + U_N$, i.e., a gradient plus a scaled manifold-attracting term plus bounded noise. Under the stated Assumption 3 that $E[Q(x)]=x$, this gives $E[\Gamma(\bar x_k)] = \operatorname{grad} f(\bar x_k)$, so the gradient-tracking lemma produces the same $\mathcal O(1/K)$ stationarity rate as unquantized RGT. The algorithm iterates $x_{i,k+1} = \sum_j W^t_{ij} x_{j,k} - \alpha s_{i,k}$ with $s_{i,k+1} = \sum_j W^t_{ij}s_{j,k} + \Gamma_{i,k+1} - \Gamma_{i,k}$; the paper proves the consensus error and the stationarity measure both decrease at $O(1/K)$.

Load-bearing premise

The convergence-rate proof accepts as its load-bearing premise that the random quantization is exactly unbiased, $E[Q(x)]=x$; if that fails, the gradient-tracking descent lemma no longer yields the claimed $O(1/K)$ stationarity bound.

Editorial extensions

If this is right

  • If Q-RGT works as claimed, communication-efficient decentralized training on manifolds, such as distributed PCA, eigenvalue estimation, and orthogonal-constrained deep networks, can use N-bit gradients without slowing convergence.
  • The algorithm removes the need for retraction and projection at each iteration, cutting per-iteration computational cost while iterates only need to stay in the neighborhood $\mathcal M_{\Omega(N)}$.
  • The lower bounds on consensus in Corollary 1 and Corollary 2 quantify how the residual consensus error depends on bit width $N$; more bits shrink the quantized region and improve the achievable consensus.
  • The same $\mathcal O(1/K)$ rate holds for the stationarity measure $\min_k \|\mathbb{E}[\Gamma(\bar x_k)]\|^2$, matching the unquantized RGT rate listed in the paper's comparison table.
  • The method provides a template for retraction-free, projection-free decentralized optimization on compact submanifolds with a tunable trade-off between communication precision and the radius of the eventual neighborhood.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the unbiasedness assumption fails, as a direct computation suggests it does when the floor of the sigmoid of $\nabla N$ is 0 or 1, the proof of the matching rate may need a bias-correction step; a natural extension would replace the deterministic round-up/down with a randomized rounding that is exactly unbiased.
  • The mechanism suggests a general recipe: any deterministic rounding that depends on an auxiliary potential, such as the distance-to-manifold squared, can serve as a cheap retraction substitute in other constrained or manifold-constrained settings.
  • A testable extension is to track the true gradient norm $\|\operatorname{grad} f(\bar x_k)\|$, not just the quantized surrogate $\|\mathbb{E}[\Gamma(\bar x_k)]\|$; a plot of the true norm would show whether the claimed rate persists at high precision or whether a bias floor appears.
  • The paper's quantized region $\Omega(N)$ description already anticipates a precision-dependent floor; a bias-aware refinement of the convergence analysis would characterize that floor as a function of $N$ and the local curvature of the manifold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Q-RGT, a decentralized algorithm for smooth nonconvex optimization on compact submanifolds (notably the Stiefel manifold) that combines gradient tracking with an N-bit uniform quantizer whose rounding direction depends on the normal field ∇N(x). The authors claim an O(1/K) convergence rate for min_k ||E[Γ(xbar_k)]||^2, matching unquantized retraction-based methods, and also claim O(1/K) consensus rates. The algorithm is retraction-free and projection-free, and numerical experiments on distributed eigenvector problems with synthetic data and MNIST are reported. The central theoretical device is a 'quantized Riemannian gradient' Γ defined through a uniform quantizer with a direction-dependent rounding offset, which is asserted to satisfy an unbiasedness assumption (Assumption 3).

Significance. If the theoretical claims were correct, this would be a noteworthy contribution: it would be the first decentralized Riemannian optimization method achieving an O(1/K) rate under gradient quantization without retractions or projections, with a matching consensus rate. The quantizer design is creative in that it uses the normal field to steer iterates toward the manifold, and the experimental section provides a useful comparison with RGT. However, the central proof rests on Assumption 3, which is violated by the proposed quantizer. Since the unbiasedness assumption is false, the descent lemma and the main convergence theorem are unsupported. The paper therefore does not currently establish its advertised convergence guarantees.

major comments (4)
  1. [§IV, Eq. (18) and Assumption 3] The quantizer defined in Eq. (18) is not unbiased. For a scalar component g, let m = 2^N − 1 and b = ⌊sigmoid∇N(x)⌉ ∈ {0,1}. Writing U_N ~ U(−1/(2m), 1/(2m)), a direct calculation gives E[Q_N(g)] = g + γ(g)(b − 1/2)/m. Since b is an integer, b − 1/2 is ±1/2, so the bias is nonzero for any generic off-manifold iterate. Thus Assumption 3, which states E[Q(x)] = x, fails exactly where it is needed. This is load-bearing: Lemma 3 uses Assumption 3 in the third equality to replace E[Γ(xbar_{k−1})] by gradf(xbar_{k−1}), and Theorem 2 inherits that step at Eq. (48). The bias term is deterministic in the direction of the normal field and cannot be canceled by the zero-mean dither U_N.
  2. [§IV, Eq. (21) and Proposition 2] Eq. (21) is presented as a rewriting of the quantized gradient Γ(x) = gradf(x) + C/(2^N − 1)∇N(x) + U_N, but it is not derived from Eq. (18). The coefficient C = γ(gradf(x))/γ(∇N(x)) depends on x and is not shown to equal the actual bias computed from the quantizer. Proposition 2's proof cites Potter's algorithm for the deterministic flow of −∇N(x) and does not analyze the stochastic term U_N or the actual quantizer in Eq. (18). Consequently, the claim that the iteration x_{k+1} = x_k − αΓ(x_k) converges to the quantized region M_{Ω(N)} is asserted rather than established.
  3. [§V-C, Theorem 2, Eq. (45)] The convergence measure in Eq. (45) is min_k ||E[Γ(xbar_k)]||^2, which is not the squared norm of the true Riemannian gradient gradf(xbar_k) or of any other stationarity measure for problem (1). Even setting that issue aside, the proof of Theorem 2 relies on E[Γ(xbar)] = gradf(xbar) twice: once in Lemma 3 and again immediately before Eq. (48). With the bias computed in my first comment, the descent inequality (42) is unavailable, the key negative term −(α/4)||gradf(xbar_{k−1})||^2 does not follow, and the O(1/K) rate for ||E[Γ(xbar_k)]||^2 is unsupported.
  4. [§V-C, Corollary 2, Eq. (52)] Corollary 2 claims min_k E||x_k − xbar_k||^2 = O(1/K) by combining Corollary 1 with Theorem 2's bound on ||E[Γ(xbar_k)]||^2. Since Theorem 2's bound is invalid, the consensus rate is also unproven. In addition, the bias from Eq. (18) injects a non-vanishing input into the consensus linear system in Theorem 1; one would expect the consensus error to be bounded below by a constant proportional to α^2/(2^N − 1)^2, rather than decaying to zero.
minor comments (4)
  1. [§V, §VI] There are several typos: 'Metroplis' should be 'Metropolis', 'throretical' should be 'theoretical', and 'Riemanian' in the conclusion should be 'Riemannian'.
  2. [§IV, Eq. (14)–(18)] The notation round_N is overloaded: Eq. (14) defines a rounding-to-nearest operation, while Eqs. (17)–(18) use a floor plus a binary direction-dependent offset. Please use distinct names or clearly state the difference to avoid confusion.
  3. [Table I] The row for Q-RGT lists the 'GT Operator' as 'N/A'; since Q-RGT is retraction-free but still performs the gradient tracking update, a more precise description would be helpful.
  4. [Theorem 2] The step-size condition includes the term (1/(16L_m)) sqrt(n(1−σ_2)^3/(2L_m^2+1)), whose dimensions are unclear; please state the condition in a consistent form with Corollary 2 and the definitions of L_m and σ_2.

Circularity Check

1 steps flagged · score 4.0 of 10

The claimed quantized-region proximity is defined into existence via Eq. (21) and Definition 2, so a supporting prediction reduces by construction; the headline O(1/K) rate is not itself circular but rests on an additionally unverified unbiasedness assumption.

  1. self definitional [Section IV-A, Eq. (21), Definition 2, and Proposition 2 (proof)]
    "Compared to Eq. (16), the quantized Riemannian gradient in Eq. (20) can be rewritten as a similar form Γ(x) = gradf(x) + C/(2^N−1) ∇N(x) + U_N, where C = γ(gradf(x))/γ(∇N(x)). ... the safety factor satisfies Ω(N) = C/(2^N−1) with a constant C. ... A simple approach to design an algorithm that converges to M_{Ω(N)} consists in following −∇N(x), which leads in the discrete setting to Potter's algorithm x_{k+1} = x_k − Cα/(2^N−1) ∇N(x) [41]."

    Eq. (21) is not derived from the composition of Eqs. (18) and (20); it simply stipulates that the quantized gradient has a normal-space component C/(2^N−1)∇N(x). Definition 2 then defines the 'quantized region' radius Ω(N) to be exactly the same coefficient C/(2^N−1), and Proposition 2's proof establishes convergence to M_{Ω(N)} by invoking the −∇N flow with step Cα/(2^N−1). Thus the proximity result is not a theorem about the algorithm; it is a restatement of the assumed bias. The constant C is carried from the ansatz into the definition of the region, so the conclusion 'iterates stay within M_{Ω(N)}' is true by construction. No independent argument shows that the actual rounding rule in Eq. (18) produces this normal component with this coefficient.

full rationale

The paper's headline contribution is the O(1/K) convergence of Q-RGT under quantization. That rate is derived through Lemma 3, whose key substitution E[Γ(¯x)] = gradf(¯x) is justified by Assumption 3. This is a stipulated assumption rather than a circular identification, but it is in direct tension with the paper's own Eq. (21), which represents Γ with a nonzero normal bias C/(2^N−1)∇N(x); for generic off-manifold iterates the deterministic rounding direction in Eq. (18) prevents unbiasedness. That is a correctness risk, not itself a circular step. The genuinely circular step is the quantized-region claim: Eq. (21) postulates the landing-field form, Definition 2 chooses Ω(N) to equal the postulated coefficient, and Proposition 2 'proves' convergence to M_{Ω(N)} by following −∇N with exactly that coefficient. The advertised manifold-proximity guarantee therefore reduces to a definitional choice. Since this proximity result is used to define L_m and to justify the bounded-region arguments, it is load-bearing, but the consensus-tracking algebra after Lemma 3 is otherwise independent and would be non-circular if Assumption 3 and Eq. (21) were consistent and derived. Hence a partial-circularity score of 4 is appropriate.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The central proof rests on two ad hoc assumptions: unbiasedness of a quantizer that is actually biased, and a decomposition of the quantized gradient into gradf plus a scaled normal field. Both are introduced specifically to make the landing argument work, and neither is established by the paper's own equations.

free parameters (1)
  • Safety factor constant C
    Definition 2 defines Omega(N) = C/(2^N - 1) with an unspecified constant C. It sizes the quantized region but no value or bound is given, and Proposition 2's convergence argument depends on this definition.
assumptions (5)
  • domain assumption Assumption 1: W is symmetric doubly stochastic with sigma_2 < 1 and the graph is connected.
    Standard consensus network assumption; used in Lemmas 1 and 2 and in the stability analysis.
  • domain assumption Assumption 2: each local function is L-Lipschitz smooth in Euclidean space.
    Used to derive manifold smoothness in Proposition 1 and to bound gradient norms in Lemma 3.
  • ad hoc to paper Assumption 3: the uniform quantizer Q is unbiased, E[Q(x)] = x.
    This is the load-bearing premise for Lemma 3's E[Gamma(xbar)] = gradf(xbar). Direct calculation from Eq (18) shows the assumption is false when the rounding offset b depends on grad N(x).
  • ad hoc to paper Eq (21): Gamma(x) = gradf(x) + C/(2^N - 1) grad N(x) + U_N.
    The paper asserts this decomposition of the quantized gradient. It does not follow from Eq (18), where the bias magnitude is gamma(gradf)(b - 1/2)/(2^N - 1) rather than C/(2^N - 1) grad N(x).
  • ad hoc to paper L_g <= L_Gamma <= L_m.
    The paper states this relationship without proof in Section IV-A, and Lemma 2 uses it to replace Lipschitz constants in the consensus error bound.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking." pith.science (2026). https://pith.science/paper/AR2S7CIW

@misc{pith2026250607351,
  author       = {Pith},
  title        = {Pith review of: Decentralized Optimization on Compact Submanifolds by Quantized Riemannian Gradient Tracking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AR2S7CIW}},
  note         = {Machine review of arXiv:2506.07351}
}
abstract

This paper considers the problem of decentralized optimization on compact submanifolds, where a finite sum of smooth (possibly non-convex) local functions is minimized by $n$ agents forming an undirected and connected graph. However, the efficiency of distributed optimization is often hindered by communication bottlenecks. To mitigate this, we propose the Quantized Riemannian Gradient Tracking (Q-RGT) algorithm, where agents update their local variables using quantized gradients. The introduction of quantization noise allows our algorithm to bypass the constraints of the accurate Riemannian projection operator (such as retraction), further improving iterative efficiency. To the best of our knowledge, this is the first algorithm to achieve an $\mathcal{O}(1/K)$ convergence rate in the presence of quantization, matching the convergence rate of methods without quantization. Additionally, we explicitly derive lower bounds on decentralized consensus associated with a function of quantization levels. Numerical experiments demonstrate that Q-RGT performs comparably to non-quantized methods while reducing communication bottlenecks and computational overhead.

Figures

Figures reproduced from arXiv: 2506.07351 by the authors.

Figure 1
Figure 1. Illustration of the geometry of the quantized gradient ()() [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Numerical results on synthetic data with different bit-widths and single-step consensus, eigengap [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Numerical results on MNIST data with different network graphs and single-step consensus, eigengap [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 26 canonical work pages

  1. [13]

    Improving the communication in decentralized manifold optimization through single-step consensus and compression,

    J. Hu and K. Deng, “Improving the communication in decentralized manifold optimization through single-step consensus and compression,” arXiv preprint arXiv:2407.08904, 2024

    arXiv 2024

  2. [1]

    Distributed subgradient methods for multi- agent optimization,

    A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi- agent optimization,”IEEE Transactions on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009

    2009

  3. [2]

    On the convergence of decentralized gradient descent,

    K. Yuan, Q. Ling, and W. Yin, “On the convergence of decentralized gradient descent,”SIAM Journal on Optimization, vol. 26, no. 3, pp. 1835–1854, 2016

    2016

  4. [3]

    Harnessing smoothness to accelerate distributed optimization,

    G. Qu and N. Li, “Harnessing smoothness to accelerate distributed optimization,”IEEE Transactions on Control of Network Systems, vol. 5, no. 3, pp. 1245–1260, 2017

    2017

  5. [4]

    Exact diffusion for distributed optimization and learning—part i: Algorithm development,

    K. Yuan, B. Ying, X. Zhao, and A. H. Sayed, “Exact diffusion for distributed optimization and learning—part i: Algorithm development,” IEEE Transactions on Signal Processing, vol. 67, no. 3, pp. 708–723, 2018

    work page 2018

  6. [5]

    Decentralized proximal gradient algorithms with linear convergence rates,

    S. A. Alghunaim, E. K. Ryu, K. Yuan, and A. H. Sayed, “Decentralized proximal gradient algorithms with linear convergence rates,”IEEE Transactions on Automatic Control, vol. 66, no. 6, pp. 2787–2794, 2020

    work page 2020

  7. [6]

    On the linear convergence of the admm in decentralized consensus optimization,

    W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimization,” IEEE Transactions on Signal Processing, vol. 62, no. 7, pp. 1750–1761, 2014

    work page 2014

  8. [7]

    Distributed linearized alter- nating direction method of multipliers for composite convex consensus optimization,

    N. S. Aybat, Z. Wang, T. Lin, and S. Ma, “Distributed linearized alter- nating direction method of multipliers for composite convex consensus optimization,”IEEE Transactions on Automatic Control, vol. 63, no. 1, pp. 5–20, 2017

    2017

Show all 43 references
  1. [8]

    Decentralized riemannian gradient descent on the stiefel manifold,

    S. Chen, A. Garcia, M. Hong, and S. Shahrampour, “Decentralized riemannian gradient descent on the stiefel manifold,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 1594–1605

    2021

  2. [9]

    Decentralized projected riemannian gradient method for smooth optimization on compact submanifolds,

    K. Deng and J. Hu, “Decentralized projected riemannian gradient method for smooth optimization on compact submanifolds,”arXiv preprint arXiv:2304.08241, 2023

    2023 arXiv

  3. [10]

    Decentralized riemannian conjugate gradient method on the stiefel manifold,

    J. Chen, H. Ye, M. Wang, T. Huang, G. Dai, I. Tsang, and Y . Liu, “Decentralized riemannian conjugate gradient method on the stiefel manifold,” inThe Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https: //openreview.net/forum?id=PQbFUMKLFp

    2024

  4. [11]

    Decentralized riemannian natural gra- dient methods with kronecker-product approximations,

    J. Hu, K. Deng, N. Li, and Q. Li, “Decentralized riemannian natural gra- dient methods with kronecker-product approximations,”arXiv preprint arXiv:2303.09611, 2023

    2023 arXiv

  5. [12]

    Distributed riemannian stochastic gradient tracking algorithm on the stiefel manifold,

    J. Zhao, X. Wang, and J. Lei, “Distributed riemannian stochastic gradient tracking algorithm on the stiefel manifold,”arXiv preprint arXiv:2405.16900, 2024

    2024 arXiv

  6. [14]

    Global convergence of decentralized retraction-free optimization on the stiefel manifold,

    Y . Sun, S. Chen, A. Garcia, and S. Shahrampour, “Global convergence of decentralized retraction-free optimization on the stiefel manifold,” arXiv preprint arXiv:2405.11590, 2024

    2024 arXiv

  7. [15]

    Deepca: Decentralized exact pca with linear convergence rate,

    H. Ye and T. Zhang, “Deepca: Decentralized exact pca with linear convergence rate,”The Journal of Machine Learning Research, vol. 22, no. 1, pp. 10 777–10 803, 2021

    2021

  8. [16]

    Cloud k-svd: A collaborative dictionary learning algorithm for big, distributed data,

    H. Raja and W. U. Bajwa, “Cloud k-svd: A collaborative dictionary learning algorithm for big, distributed data,”IEEE Transactions on Signal Processing, vol. 64, no. 1, pp. 173–188, 2015

    2015

  9. [17]

    On orthogonality and learning recurrent networks with long term dependencies,

    E. V orontsov, C. Trabelsi, S. Kadoury, and C. Pal, “On orthogonality and learning recurrent networks with long term dependencies,” in International Conference on Machine Learning. PMLR, 2017, pp. 3570–3578

    2017

  10. [18]

    Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks,

    L. Huang, X. Liu, B. Lang, A. Yu, Y . Wang, and B. Li, “Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018

    2018

  11. [19]

    Understanding how orthogonality of pa- rameters improves quantization of neural networks,

    S. B. Eryilmaz and A. Dundar, “Understanding how orthogonality of pa- rameters improves quantization of neural networks,”IEEE Transactions on Neural Networks and Learning Systems, 2022

    2022

  12. [20]

    Distributed asynchronous deterministic and stochastic gradient optimization algorithms,

    J. Tsitsiklis, D. Bertsekas, and M. Athans, “Distributed asynchronous deterministic and stochastic gradient optimization algorithms,”IEEE transactions on automatic control, vol. 31, no. 9, pp. 803–812, 1986

    1986

  13. [21]

    On distributed nonconvex optimization: Projected subgradient method for weakly convex problems in networks,

    S. Chen, A. Garcia, and S. Shahrampour, “On distributed nonconvex optimization: Projected subgradient method for weakly convex problems in networks,”IEEE Transactions on Automatic Control, vol. 67, no. 2, pp. 662–675, 2021

    2021

  14. [22]

    Extra: An exact first-order algorithm for decentralized consensus optimization,

    W. Shi, Q. Ling, G. Wu, and W. Yin, “Extra: An exact first-order algorithm for decentralized consensus optimization,”SIAM Journal on Optimization, vol. 25, no. 2, pp. 944–966, 2015

    2015

  15. [23]

    Next: In-network nonconvex optimiza- tion,

    P. Di Lorenzo and G. Scutari, “Next: In-network nonconvex optimiza- tion,”IEEE Transactions on Signal and Information Processing over Networks, vol. 2, no. 2, pp. 120–136, 2016

    2016

  16. [24]

    On centralized and distributed mirror descent: Convergence analysis using quadratic constraints,

    Y . Sun, M. Fazlyab, and S. Shahrampour, “On centralized and distributed mirror descent: Convergence analysis using quadratic constraints,”IEEE Transactions on Automatic Control, vol. 68, no. 5, pp. 3139–3146, 2022. IEEE TRANSACTIONS ON SIGNAL PROCESSING 11

    2022

  17. [25]

    On the local linear rate of consensus on the stiefel manifold,

    S. Chen, A. Garcia, M. Hong, and S. Shahrampour, “On the local linear rate of consensus on the stiefel manifold,”IEEE Transactions on Automatic Control, 2023

    2023

  18. [26]

    Decentralized optimization over the stiefel manifold by an approximate augmented lagrangian function,

    L. Wang and X. Liu, “Decentralized optimization over the stiefel manifold by an approximate augmented lagrangian function,”IEEE Transactions on Signal Processing, vol. 70, pp. 3029–3041, 2022

    2022

  19. [27]

    Fast and accurate optimization on the orthogonal manifold without retraction,

    P. Ablin and G. Peyr ´e, “Fast and accurate optimization on the orthogonal manifold without retraction,” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2022, pp. 5636–5657

    2022

  20. [28]

    Design and analysis of distributed averaging with quantized communication,

    M. El Chamie, J. Liu, and T. Bas ¸ar, “Design and analysis of distributed averaging with quantized communication,”IEEE Transactions on Auto- matic Control, vol. 61, no. 12, pp. 3870–3884, 2016

    2016

  21. [29]

    An exact quantized decentralized gradient descent algorithm,

    A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani, “An exact quantized decentralized gradient descent algorithm,”IEEE Transactions on Signal Processing, vol. 67, no. 19, pp. 4934–4947, 2019

    2019

  22. [30]

    Quantized decentralized stochastic learning over directed graphs,

    H. Taheri, A. Mokhtari, H. Hassani, and R. Pedarsani, “Quantized decentralized stochastic learning over directed graphs,” inInternational Conference on Machine Learning. PMLR, 2020, pp. 9324–9333

    2020

  23. [31]

    A linearly convergent algorithm for decentralized optimization: Sending less bits for free!

    D. Kovalev, A. Koloskova, M. Jaggi, P. Richtarik, and S. Stich, “A linearly convergent algorithm for decentralized optimization: Sending less bits for free!” inInternational Conference on Artificial Intelligence and Statistics. PMLR, 2021, pp. 4087–4095

    2021

  24. [32]

    Linear convergent decentralized optimization with compression,

    X. Liu and Y . Li, “Linear convergent decentralized optimization with compression,” inInternational Conference on Learning Representations, 2021

    2021

  25. [33]

    Quantized distributed gradient tracking algorithm with linear convergence in directed networks,

    Y . Xiong, L. Wu, K. You, and L. Xie, “Quantized distributed gradient tracking algorithm with linear convergence in directed networks,”IEEE Transactions on Automatic Control, vol. 68, no. 9, pp. 5638–5645, 2022

    2022

  26. [34]

    Absil, R

    P.-A. Absil, R. Mahony, and R. Sepulchre,Optimization algorithms on matrix manifolds. Princeton University Press, 2008

    2008

  27. [35]

    The geometry of algorithms with orthogonality constraints,

    A. Edelman, T. A. Arias, and S. T. Smith, “The geometry of algorithms with orthogonality constraints,”SIAM journal on Matrix Analysis and Applications, vol. 20, no. 2, pp. 303–353, 1998

    1998

  28. [36]

    A riemannian conjugate gradient method for optimization on the stiefel manifold,

    X. Zhu, “A riemannian conjugate gradient method for optimization on the stiefel manifold,”Computational optimization and Applications, vol. 67, pp. 73–110, 2017

    2017

  29. [37]

    Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses,

    H. Sato, “Riemannian conjugate gradient methods: General framework and specific algorithms with convergence analyses,”SIAM Journal on Optimization, vol. 32, no. 4, pp. 2690–2717, 2022

    2022

  30. [38]

    Proximal smoothness and the lower-c2 property,

    F. H. Clarke, R. J. Stern, and P. R. Wolenski, “Proximal smoothness and the lower-c2 property,”J. Convex Anal, vol. 2, no. 1-2, pp. 117– 144, 1995

    1995

  31. [39]

    The gradient projection method with armijo’s step size on manifolds,

    M. Balashov and R. Kamalov, “The gradient projection method with armijo’s step size on manifolds,”Computational Mathematics and Mathematical Physics, vol. 61, pp. 1776–1786, 2021

    2021

  32. [40]

    First-order methods for geodesically convex optimization,

    H. Zhang and S. Sra, “First-order methods for geodesically convex optimization,” inConference on learning theory. PMLR, 2016, pp. 1617–1638

    2016

  33. [41]

    Equivariant adaptive source separa- tion,

    J.-F. Cardoso and B. H. Laheld, “Equivariant adaptive source separa- tion,”IEEE Transactions on signal processing, vol. 44, no. 12, pp. 3017– 3030, 1996

    1996

  34. [42]

    Infeasible deterministic, stochastic, and variance-reduction algorithms for optimization under or- thogonality constraints,

    P. Ablin, S. Vary, B. Gao, and P.-A. Absil, “Infeasible deterministic, stochastic, and variance-reduction algorithms for optimization under or- thogonality constraints,”Journal of Machine Learning Research, vol. 25, no. 389, pp. 1–38, 2024

    2024

  35. [43]

    The mnist database of handwritten digits,

    Y . LeCun, “The mnist database of handwritten digits,”http://yann. lecun. com/exdb/mnist/, 1998

    1998

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.