Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a two-timescale compressed primal-dual algorithm, TiCoPD, finds epsilon-stationary solutions of smooth nonconvex decentralized optimization problems at centralized-SGD speed, even when networks have random links…

desk verdict TiCoPD is a novel and well-analyzed primal-dual method with state-of-the-art rates, but the convergence theorem analyzes a global surrogate recursion that Algorithm 1's local-copy implementation does not provably realize; the gap is repairable. read the letter →

arxiv 2502.08964 v2 pith:D53ZU5DH submitted 2025-02-13 math.OC

Haoming Liu , Chung-Yiu Yau , Hoi-To Wai This is my paper · ORCID
classification math.OC MSC 90C1590C2668W15
keywords decentralizedoptimizationcompressedcommunicationtwo-timescaleupdateprimal-dualalgorithmnonconvexrandomgraphsnoisychannelsstochasticgradients
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces TiCoPD, a two-timescale compressed primal-dual algorithm for decentralized stochastic optimization over unreliable networks. It claims that, for smooth and possibly non-convex objective functions, TiCoPD finds an epsilon-stationary solution: the averaged squared gradient norm decays like $O(\sqrt{\bar\sigma^2/(nT)})$ when communication is compressed but noiseless, matching centralized SGD with a minibatch of size $n$, and degrades to $O((1+\sigma_\xi^2)/T^{1/3})$ when channels add noise. These rates hold over random time-varying graphs and general contractive compressors, and they do not require the bounded data-heterogeneity assumption used by earlier compressed decentralized algorithms. The key design is a majorization-minimization step that moves the network-consensus penalty onto a surrogate sequence updated on a fast timescale, leaving a slow primal-dual recursion to handle the objective.

What carries the argument

The central mechanism is the two-timescale surrogate tracking loop $\hat X^{t+1} = \hat X^t + \gamma Q(X^{t+1} - \hat X^t; \xi_q)$ running in parallel with the slow primal-dual recursion (19). $Q$ is a contractive compressor with additive zero-mean noise satisfying Assumption III.1: it can be a randomized quantizer, a top-$k$ sparsifier, or an adaptive quantizer, and the parameter $\delta \in (0,1]$ measures contraction while $\sigma_\xi^2$ bounds transmission noise. The majorization-minimization step replaces the quadratic penalty $\theta\|\bar A(\xi_a)X\|^2$ with a term linear in the surrogate, $\theta(\bar A^\top\bar A(\xi_a)\hat X)^\top X$, so agents only need to exchange compressed differences of $\hat X$. The proof builds a potential function $F_t = f(\bar x^t) + a\|X^t\|_K^2 + b\|v^t\|^2_{\bar Q+cK} + d\langle X^t, v^t\rangle_K + e\|\hat X^t - X^t\|^2$ and shows via Lemmas IV.7-IV.12 that it decreases by at least $\alpha \|\nabla f(\bar x^t)\|^2/16$ per iteration, up to gradient-variance and communication-noise terms.

What would settle it

Implement Algorithm 1 exactly on a two-agent network with one alternating active edge, random quantization, exact gradients, and noiseless channels; measure the consensus error $\|X^t\|_K^2$ over $T$ iterations. The noiseless-exact-gradient case of Theorem IV.5 predicts a decay rate $O(1/(\delta^2 T))$; if the measured quantity consistently decays slower than the constants in (29) allow, the implemented local-copy algorithm is not the global recursion being analyzed.

Watch

Extended reading notes

Core claim

TiCoPD is a stochastic primal-dual method whose updates are the recursion (19): $X^{t+1}=X^t-\alpha(\nabla f(X^t;\xi^{t+1})+\lambda^t+\theta\bar A^\top\bar A(\xi_a)\hat X^t)$, $\lambda^{t+1}=\lambda^t+\eta\bar A^\top\bar A(\xi_a)\hat X^t$, and $\hat X^{t+1}=\hat X^t+\gamma Q(X^{t+1}-\hat X^t;\xi_q)$. The surrogate $\hat X$ tracks $X$ through compressed differences $Q(\cdot)$, an auxiliary recursion that converges faster than the optimization steps because $\gamma \gg \max\{\alpha,\eta\}$. Theorem IV.5 bounds the average squared gradient norm by $(F_0-f^\star)/(\alpha T/16)+16\alpha\omega_\sigma\bar\sigma^2+128a\gamma^2\sigma_\xi^2\tilde\rho_{\max}/(\alpha\rho_{\min})$, and the corollaries give $O(\sqrt{\bar\sigma^2/(nT)})$ for noiseless compressed communication and $O((1+\sigma_\xi^2)/T^{1/3})$ for noisy communication. The paper argues this is the first stochastic primal-dual algorithm for decentralized optimization on random graphs that supports general contractive compressors and noisy compressed transmission without a bounded-heterogeneity assumption, and it interprets the construction as a majorization-minimization derivation of error-feedback compression linked to nonlinear gossiping.

Load-bearing premise

The proof analyzes the global surrogate recursion in equation (19), but Algorithm 1 actually implements per-edge local copies $\hat x_{i,j}$; no argument in the paper shows those local copies realize the analyzed global recursion, so the convergence theorem may not cover the algorithm as written.

Editorial extensions

If this is right

  • With noiseless compressed transmission, TiCoPD achieves the same asymptotic bound on $\mathbb{E}[\|\nabla f(\bar x_R)\|^2]$ as centralized SGD with a minibatch of size $n$, giving linear speedup over random graphs.
  • With noisy compressed transmission, the algorithm still converges at rate $O((1+\sigma_\xi^2)/T^{1/3})$ and to a neighbourhood whose radius is controlled by $\gamma^2 \sigma_\xi^2/(\alpha \rho_{\min})$.
  • TiCoPD supports general contractive compressors, including random quantization, top-$k$ sparsification, and adaptive quantizers, under a single assumption and without requiring bounded gradient heterogeneity.
  • The majorization-minimization derivation gives a principled recipe for adding compression to primal-dual schemes that lack an explicit averaging step.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a gap the paper leaves open is that the per-edge local copies in Algorithm 1 may not coincide with the global surrogate recursion studied in the proof, so the theory's coverage of the implemented algorithm is not established.
  • Editorial inference: if the local-copy gap is real, it could be closed by showing the local copies are stochastic approximations of the global sequence with controlled variance, or by re-deriving the bounds for the local update.
  • Editorial inference: the MM view suggests that other consensus-constrained formulations, such as ADMM or accelerated primal-dual schemes, could gain compressed and noise-robust variants by introducing a similarly fast surrogate loop.
  • Editorial inference: a testable extension would replace the fixed step sizes with a decreasing schedule for $\gamma$, which might push the noisy-communication rate closer to $O(1/\sqrt{T})$.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TiCoPD, a two-timescale compressed primal-dual method for decentralized minimization of a smooth (possibly nonconvex) sum of local objectives over random time-varying graphs with compressed and possibly noisy communication. A surrogate sequence is updated on a fast timescale using a contractive compressor, while primal and dual variables move on a slow timescale. Theorem IV.5 gives, under Assumptions III.1 and IV.1-IV.4 and step-size conditions (28), an expected squared-gradient bound (29); the authors specialize it to O(sqrt(sigma_bar^2/(nT))) for noiseless compressed communication and O((1+sigma_xi^2)/T^{1/3}) for noisy compressed communication, without bounded data heterogeneity. Numerical experiments on linear regression, nonconvex sigmoid loss, and ResNet-50/ImageNet training are reported.

Significance. If Theorem IV.5 covers the algorithm as implemented, the contribution is significant: TiCoPD would be the first stochastic primal-dual method to handle time-varying random graphs, general contractive compressors, and noisy communication simultaneously, with rates comparable to centralized SGD in the noiseless case. The proof is largely self-contained, the step-size conditions are explicit, and the claimed rates follow from stated choices rather than from fitted parameters. However, the certificate currently applies to an idealized global surrogate recursion and not to the local-copy protocol in Algorithm 1; this gap must be closed before the advertised claims are established.

major comments (3)
  1. [Section III (Algorithm 1) vs Section IV (Lemmas IV.8-IV.12, Theorem IV.5)] No equivalence is shown between the per-edge local copies updated in Algorithm 1, Step 6 and the global surrogate recursion (19) analyzed in Lemma IV.11. In Algorithm 1, xhat_{i,j}^t is updated only when edge (i,j) is active and uses x_j^t - xhat_{i,j}^{t-1}; if the edge is inactive at iteration t, the copy remains frozen, whereas the corresponding component of the global Xhat^t in (19) continues to evolve. Consequently, the quantity E||Xhat^t - X^t||^2 bounded in Lemma IV.11 need not control the surrogate error actually entering (20)-(21), and Theorem IV.5's bound (29) is not established for Algorithm 1 as written.
  2. [Section IV, Lemma IV.11 and equation (19); Algorithm 1, Steps 6-7] The implemented surrogate update is one step ahead of the analyzed one: Algorithm 1 updates xhat_{i,j}^t before computing x_i^{t+1}, using x_j^t, and then uses xhat^t in (20)-(21), while the global recursion (19) uses Xhat^t in the X-update and forms Xhat^{t+1} only from X^{t+1}. Even under a hypothetical identification of the local copies with a global Xhat, the sequence entering the primal-dual update is not the sequence whose tracking error Lemma IV.11 bounds; the authors need to align the indexing or prove a separate bound for the shifted sequence.
  3. [Algorithm 1, Step 6] The protocol is not self-contained as written: Step 6 instructs agent i to receive Q(x_j^t - xhat_{i,j}^{t-1}; xi_{q,j}^t), but xhat_{i,j}^{t-1} is a quantity defined at the receiver i, and the pseudo-code does not specify how transmitter j knows this receiver-dependent value. The remark that the step is 'also performed at the transmitter side' is insufficient; either a mirrored-copy assumption must be added, or the update rule must be redefined in terms of information available to the transmitter.
minor comments (5)
  1. [Section V-B, Figure 3 caption] The caption refers to a 'non-sigmoid loss minimization problem'; this appears to be a typo for 'non-convex sigmoid loss minimization problem'.
  2. [Section V, randomized quantization parameters] The text says the randomized quantizer in (15) uses s=24 levels, while the figures and legends describe the method as '4-bits'; for QSGD, the per-coordinate cost is about log2(s) bits plus the transmitted norm, so 24 levels is not 4 bits, and the labeling should be reconciled.
  3. [Section V-C] ResNet-50 has approximately 25.6 million parameters, not 2.56 x 10^9 parameters as stated; the number should be corrected.
  4. [Algorithm 1, lines 5-6] Line 5 draws the compressor randomness xi_{q,i}^{t+1}, while Step 6 uses xi_{q,j}^t; the time indexing of the compressor randomness should be unified.
  5. [Section IV, Remark after Theorem IV.5] In the rate specializations, the free constant a is only described as being of a given order in T; to make the bounds fully explicit, the authors should state the concrete values of a used in each case, since the step-size condition (28) involves a through theta_lb.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem IV.5's rates are derived from explicit assumptions and step-size choices, not from fitting or self-citation.

full rationale

The paper's central convergence result (29)-(30) is obtained through a self-contained potential-function proof (Lemmas IV.7-IV.12) under Assumptions III.1 and IV.1-IV.4, with step sizes fixed by (28). The claimed rates O(sqrt(sigma_bar^2/(nT))) and O((1+sigma_xi^2)/T^{1/3}) follow by substituting the stated scalings of alpha, theta, a, gamma; no parameter is fitted to the target bound and no target result is imported as an assumption. Self-citations to the authors' earlier FSPDA-SA [36] and conference TiCoPD [44] are used only for comparison and extension, not to justify the convergence theorem; the nonlinear-gossip inspiration [43] and two-timescale framework [48] are external references and are not invoked as uniqueness arguments. The gap noted by the skeptical reader between Algorithm 1's per-edge local copies and the vectorized surrogate recursion (19) is a possible correctness/implementation concern about whether the pseudocode realizes the analyzed sequence; it is not a circularity, because the theorem explicitly analyzes (19), which is itself a stated algorithm definition rather than a consequence of the theorem. Overall, no load-bearing step reduces to its own input by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The convergence theorem rests on standard smoothness and variance assumptions, a model of contractive noisy compressors, and spectral and variance conditions on the random graph. No new physical entity is introduced; the surrogate sequence is an algorithmic auxiliary variable. The step sizes and the free constant a are not fitted to data in the proof but are hand-tuned in the experiments.

free parameters (3)
  • Stepsize schedule (alpha, theta, eta, gamma) = e.g., alpha=1e-4, theta=100, eta=5e-3, gamma=1 noiseless; gamma=0.1, theta=100, eta=1e-4, alpha=1e-4 noisy
    Hand-tuned by grid search in Section V to minimize final gradient norm. The theorem only constrains ranges, so these values do not enter the proof, but the empirical comparisons depend on them.
  • Free constant a in Theorem IV.5 = O(T^{-1/2}) or O(T^{-1/3}) depending on setting
    A free positive constant in the potential function and rate bounds. It is chosen as a function of T to optimize rates, not fitted to data.
  • Compressor parameters (s=24 quantization levels; 1% and 0.1% top-k) = s=24; k=0.01d and k=0.001d
    Chosen by the authors for the bit-rate comparison; these settings directly affect the reported communication efficiency.
assumptions (5)
  • domain assumption Each fi is L-smooth and bounded below (Assumption IV.1).
    Used in the descent lemma and throughout the proof of Theorem IV.5; standard for nonconvex smooth optimization.
  • domain assumption Stochastic gradients have finite variance (Assumption IV.2).
    Controls the sigma_bar^2 terms in the final rates; required for stochastic gradients.
  • domain assumption Random graph Laplacians satisfy spectral bounds and a variance bound (Assumptions IV.3 and IV.4).
    Needed for consensus error contraction and to bound graph randomness; the assumption holds when every edge has positive activation probability.
  • domain assumption Compression Q satisfies the contractive-plus-noise model of Assumption III.1.
    Central to the surrogate update; standard for QSGD-type compressors, but it is the key modeling bet about unreliable channels.
  • standard math Majorization inequality for ||bar A(xi)X||^2 with finite M.
    Used in Section III Step 1 to derive the MM surrogate update; holds for any bounded linear operator.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates." pith.science (2026). https://pith.science/paper/D53ZU5DH

@misc{pith2026250208964,
  author       = {Pith},
  title        = {Pith review of: Decentralized Stochastic Optimization over Unreliable Networks via Two-timescales Updates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D53ZU5DH}},
  note         = {Machine review of arXiv:2502.08964}
}
read the original abstract

This paper introduces a robust two-timescale compressed primal-dual (TiCoPD) algorithm tailored for decentralized optimization under bandwidth-limited and unreliable channels. By integrating the majorization-minimization approach with the primal-dual optimization framework, the TiCoPD algorithm strategically compresses the difference term shared among agents to enhance communication efficiency and robustness against noisy channels without compromising convergence stability. The method incorporates a mirror sequence for agent consensus on nonlinearly compressed terms updated on a fast timescale, together with a slow timescale primal-dual recursion for optimizing the objective function. Our analysis demonstrates that the proposed algorithm converges to a stationary solution when the objective function is smooth but possibly non-convex. Numerical experiments corroborate the conclusions of this paper.

Figures

Figures reproduced from arXiv: 2502.08964 by the authors.

Figure 2
Figure 2. Convergence over noiseless communication channel with random [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Convergence of compressed algorithms over communication channel [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Convergence of TiCoPD over communication channel with random [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 38 canonical work pages

  1. [1]

    Gossip algorithms for distributed signal processing,

    A. G. Dimakis, S. Kar, J. M. Moura, M. G. Rabbat, and A. Scaglione, “Gossip algorithms for distributed signal processing,” Proceedings of the IEEE, vol. 98, no. 11, pp. 1847–1864, 2010

    work page 2010

  2. [2]

    Distributed average consensus with least-mean-square deviation,

    L. Xiao, S. Boyd, and S.-J. Kim, “Distributed average consensus with least-mean-square deviation,” Journal of parallel and distributed computing, vol. 67, no. 1, pp. 33–46, 2007

    work page 2007

  3. [3]

    Consensus+ innovations distributed inference over networks: cooperation and sensing in networked systems,

    S. Kar and J. M. Moura, “Consensus+ innovations distributed inference over networks: cooperation and sensing in networked systems,” IEEE Signal Processing Magazine , vol. 30, no. 3, pp. 99–109, 2013

    work page 2013

  4. [4]

    Distributed sparse linear regression,

    G. Mateos, J. A. Bazerque, and G. B. Giannakis, “Distributed sparse linear regression,” IEEE Transactions on Signal Processing , vol. 58, no. 10, pp. 5262–5276, 2010

    2010

  5. [5]

    Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,

    X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,” Advances in neural information processing systems , vol. 30, 2017

    2017

  6. [6]

    Distributed subgradient methods for multi- agent optimization,

    A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi- agent optimization,” IEEE Transactions on Automatic Control , vol. 54, no. 1, pp. 48–61, 2009

    2009

  7. [7]

    Extra: An exact first-order algorithm for decentralized consensus optimization,

    W. Shi, Q. Ling, G. Wu, and W. Yin, “Extra: An exact first-order algorithm for decentralized consensus optimization,” SIAM Journal on Optimization, vol. 25, no. 2, pp. 944–966, 2015

    2015

  8. [8]

    Harnessing smoothness to accelerate distributed optimization,

    G. Qu and N. Li, “Harnessing smoothness to accelerate distributed optimization,” IEEE Transactions on Control of Network Systems, vol. 5, no. 3, pp. 1245–1260, 2017. 15

    work page 2017

Show all 51 references
  1. [9]

    On nonconvex decentralized gradient descent,

    J. Zeng and W. Yin, “On nonconvex decentralized gradient descent,” IEEE Transactions on signal processing, vol. 66, no. 11, pp. 2834–2848, 2018

    2018

  2. [10]

    A unified theory of decentralized sgd with changing topology and local updates,

    A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, and S. Stich, “A unified theory of decentralized sgd with changing topology and local updates,” in International conference on machine learning , pp. 5381– 5393, PMLR, 2020

    2020

  3. [11]

    Decentralized gradient tracking with local steps,

    Y . Liu, T. Lin, A. Koloskova, and S. U. Stich, “Decentralized gradient tracking with local steps,” Optimization Methods and Software , pp. 1– 28, 2024

    2024

  4. [12]

    Asynchronous distributed admm for consen- sus optimization,

    R. Zhang and J. Kwok, “Asynchronous distributed admm for consen- sus optimization,” in International conference on machine learning , pp. 1701–1709, PMLR, 2014

    2014

  5. [13]

    Asynchronous distributed admm for large-scale optimization—part i: Algorithm and convergence analysis,

    T.-H. Chang, M. Hong, W.-C. Liao, and X. Wang, “Asynchronous distributed admm for large-scale optimization—part i: Algorithm and convergence analysis,” IEEE Transactions on Signal Processing, vol. 64, no. 12, pp. 3118–3130, 2016

    2016

  6. [14]

    Communication-censored linearized admm for decentralized consensus optimization,

    W. Li, Y . Liu, Z. Tian, and Q. Ling, “Communication-censored linearized admm for decentralized consensus optimization,” IEEE Transactions on Signal and Information Processing over Networks , vol. 6, pp. 18–34, 2019

    2019

  7. [15]

    Admm based privacy-preserving decentralized optimization,

    C. Zhang, M. Ahmad, and Y . Wang, “Admm based privacy-preserving decentralized optimization,” IEEE Transactions on Information Foren- sics and Security , vol. 14, no. 3, pp. 565–580, 2018

    2018

  8. [16]

    On the linear convergence of the admm in decentralized consensus optimization,

    W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimization,” IEEE Transactions on Signal Processing, vol. 62, no. 7, pp. 1750–1761, 2014

    2014

  9. [17]

    Asynchronous distributed optimization over lossy networks via relaxed admm: Stability and linear convergence,

    N. Bastianello, R. Carli, L. Schenato, and M. Todescato, “Asynchronous distributed optimization over lossy networks via relaxed admm: Stability and linear convergence,” IEEE Transactions on Automatic Control , vol. 66, no. 6, pp. 2620–2635, 2020

    2020

  10. [18]

    Perturbed proximal primal–dual algorithm for nonconvex nonsmooth optimization,

    D. Hajinezhad and M. Hong, “Perturbed proximal primal–dual algorithm for nonconvex nonsmooth optimization,” Mathematical Programming , vol. 176, no. 1, pp. 207–245, 2019

    2019

  11. [19]

    Linear con- vergence of first-and zeroth-order primal–dual algorithms for distributed nonconvex optimization,

    X. Yi, S. Zhang, T. Yang, T. Chai, and K. H. Johansson, “Linear con- vergence of first-and zeroth-order primal–dual algorithms for distributed nonconvex optimization,” IEEE Transactions on Automatic Control , vol. 67, no. 8, pp. 4194–4201, 2021

    2021

  12. [20]

    Distributed learning in the nonconvex world: From batch data to streaming and beyond,

    T.-H. Chang, M. Hong, H.-T. Wai, X. Zhang, and S. Lu, “Distributed learning in the nonconvex world: From batch data to streaming and beyond,” IEEE Signal Processing Magazine , vol. 37, no. 3, pp. 26–38, 2020

    2020

  13. [21]

    Distributed optimization over time-varying directed graphs,

    A. Nedi ´c and A. Olshevsky, “Distributed optimization over time-varying directed graphs,” IEEE Transactions on Automatic Control , vol. 60, no. 3, pp. 601–615, 2014

    2014

  14. [22]

    Achieving geometric convergence for distributed optimization over time-varying graphs,

    A. Nedic, A. Olshevsky, and W. Shi, “Achieving geometric convergence for distributed optimization over time-varying graphs,” SIAM Journal on Optimization, vol. 27, no. 4, pp. 2597–2633, 2017

    2017

  15. [23]

    Terngrad: Ternary gradients to reduce communication in distributed deep learning,

    W. Wen, C. Xu, F. Yan, C. Wu, Y . Wang, Y . Chen, and H. Li, “Terngrad: Ternary gradients to reduce communication in distributed deep learning,” Advances in neural information processing systems , vol. 30, 2017

    2017

  16. [24]

    Taming the wild: A unified analysis of hogwild-style algorithms,

    C. M. De Sa, C. Zhang, K. Olukotun, and C. R ´e, “Taming the wild: A unified analysis of hogwild-style algorithms,” Advances in neural information processing systems , vol. 28, 2015

    2015

  17. [25]

    Large scale distributed deep networks,

    J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, M. Ran- zato, A. Senior, P. Tucker, K. Yang, et al., “Large scale distributed deep networks,” Advances in neural information processing systems , vol. 25, 2012

    2012

  18. [26]

    1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns.,

    F. Seide, H. Fu, J. Droppo, G. Li, and D. Yu, “1-bit stochastic gradient descent and its application to data-parallel distributed training of speech dnns.,” in Interspeech, vol. 2014, pp. 1058–1062, Singapore, 2014

    2014

  19. [27]

    Ef21: A new, simpler, theoretically better, and practically faster error feedback,

    P. Richt ´arik, I. Sokolov, and I. Fatkhullin, “Ef21: A new, simpler, theoretically better, and practically faster error feedback,” Advances in Neural Information Processing Systems , vol. 34, pp. 4384–4396, 2021

    2021

  20. [28]

    An exact quantized decentralized gradient descent algorithm,

    A. Reisizadeh, A. Mokhtari, H. Hassani, and R. Pedarsani, “An exact quantized decentralized gradient descent algorithm,” IEEE Transactions on Signal Processing , vol. 67, no. 19, pp. 4934–4947, 2019

    2019

  21. [29]

    On maintaining linear convergence of distributed learning and optimization under limited communication,

    S. Magn ´usson, H. Shokri-Ghadikolaei, and N. Li, “On maintaining linear convergence of distributed learning and optimization under limited communication,” IEEE Transactions on Signal Processing , vol. 68, pp. 6101–6116, 2020

    2020

  22. [30]

    Linear convergent decentralized optimization with compression,

    X. Liu and Y . Li, “Linear convergent decentralized optimization with compression,” in International Conference on Learning Representations, 2021

    2021

  23. [31]

    Beer: Fast o(1/t) rate for decentralized nonconvex optimization with communication com- pression,

    H. Zhao, B. Li, Z. Li, P. Richt ´arik, and Y . Chi, “Beer: Fast o(1/t) rate for decentralized nonconvex optimization with communication com- pression,” Advances in Neural Information Processing Systems , vol. 35, pp. 31653–31667, 2022

    2022

  24. [32]

    Decentralized stochastic op- timization and gossip algorithms with compressed communication,

    A. Koloskova, S. Stich, and M. Jaggi, “Decentralized stochastic op- timization and gossip algorithms with compressed communication,” in International Conference on Machine Learning, pp. 3478–3487, PMLR, 2019

    2019

  25. [33]

    Decentralized deep learning with arbitrary communication compression,

    A. Koloskova, T. Lin, S. U. Stich, and M. Jaggi, “Decentralized deep learning with arbitrary communication compression,” in Proceedings of the 8th International Conference on Learning Representations , 2019

    2019

  26. [34]

    Docom: Compressed decentralized op- timization with near-optimal sample complexity,

    C.-Y . Yau and H.-T. Wai, “Docom: Compressed decentralized op- timization with near-optimal sample complexity,” arXiv preprint arXiv:2202.00255, 2022

    2022 arXiv

  27. [35]

    A communication- efficient stochastic gradient descent algorithm for distributed nonconvex optimization,

    A. Xie, X. Yi, X. Wang, M. Cao, and X. Ren, “A communication- efficient stochastic gradient descent algorithm for distributed nonconvex optimization,” arXiv preprint arXiv:2403.01322 , 2024

    2024 arXiv

  28. [36]

    A stochastic approximation approach for efficient decentralized optimization on random networks,

    C.-Y . Yau, H. Liu, and H.-T. Wai, “A stochastic approximation approach for efficient decentralized optimization on random networks,” arXiv preprint arXiv:2410.18774v2, 2024

    2024 arXiv

  29. [37]

    Distributed asynchronous constrained stochastic optimization,

    K. Srivastava and A. Nedic, “Distributed asynchronous constrained stochastic optimization,” IEEE journal of selected topics in signal processing, vol. 5, no. 4, pp. 772–790, 2011

    2011

  30. [38]

    Dimix: Diminishing mixing for sloppy agents,

    H. Reisizadeh, B. Touri, and S. Mohajer, “Dimix: Diminishing mixing for sloppy agents,” SIAM Journal on Optimization , vol. 33, no. 2, pp. 978–1005, 2023

    2023

  31. [39]

    Differential error feedback for communication-efficient decentralized optimization,

    R. Nassif, S. Vlaski, M. Carpentiero, V . Matta, and A. H. Sayed, “Differential error feedback for communication-efficient decentralized optimization,” in 2024 IEEE 13rd Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 1–5, IEEE, 2024

    2024

  32. [40]

    Finite-bit quantization for distributed algorithms with linear convergence,

    N. Michelusi, G. Scutari, and C.-S. Lee, “Finite-bit quantization for distributed algorithms with linear convergence,” IEEE Transactions on Information Theory, vol. 68, no. 11, pp. 7254–7280, 2022

    2022

  33. [41]

    Quantization for decentralized learning under subspace constraints,

    R. Nassif, S. Vlaski, M. Carpentiero, V . Matta, M. Antonini, and A. H. Sayed, “Quantization for decentralized learning under subspace constraints,” IEEE Transactions on Signal Processing, vol. 71, pp. 2320– 2335, 2023

    2023

  34. [42]

    Next: In-network nonconvex optimiza- tion,

    P. Di Lorenzo and G. Scutari, “Next: In-network nonconvex optimiza- tion,” IEEE Transactions on Signal and Information Processing over Networks, vol. 2, no. 2, pp. 120–136, 2016

    2016

  35. [43]

    Nonlinear gossip,

    A. S. Mathkar and V . S. Borkar, “Nonlinear gossip,” SIAM Journal on Control and Optimization , vol. 54, no. 3, pp. 1535–1557, 2016

    2016

  36. [44]

    A two-timescale primal-dual al- gorithm for decentralized optimization with compression,

    H. Liu, C.-Y . Yau, and H.-T. Wai, “A two-timescale primal-dual al- gorithm for decentralized optimization with compression,” in ICASSP, 2025

    2025

  37. [45]

    Nonlinear programming,

    D. P. Bertsekas, “Nonlinear programming,” Journal of the Operational Research Society, vol. 48, no. 3, pp. 334–334, 1997

    1997

  38. [46]

    Qsgd: Communication-efficient sgd via gradient quantization and encoding,

    D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. V ojnovic, “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” Advances in neural information processing systems , vol. 30, 2017

    2017

  39. [47]

    A robust compressed push- pull method for decentralized nonconvex optimization,

    Y . Liao, Z. Li, S. Pu, and T.-H. Chang, “A robust compressed push- pull method for decentralized nonconvex optimization,” arXiv preprint arXiv:2408.01727, 2024

    2024 arXiv

  40. [48]

    Stochastic approximation with two time scales,

    V . S. Borkar, “Stochastic approximation with two time scales,” Systems & Control Letters , vol. 29, no. 5, pp. 291–294, 1997

    1997

  41. [49]

    Stochastic first-and zeroth-order methods for nonconvex stochastic programming,

    S. Ghadimi and G. Lan, “Stochastic first-and zeroth-order methods for nonconvex stochastic programming,” SIAM journal on optimization , vol. 23, no. 4, pp. 2341–2368, 2013

    2013

  42. [50]

    Linear conver- gent decentralized optimization with compression,

    X. Liu, Y . Li, R. Wang, J. Tang, and M. Yan, “Linear conver- gent decentralized optimization with compression,” arXiv preprint arXiv:2007.00232, 2020

    2007 arXiv

  43. [51]

    Sgdr: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” arXiv preprint arXiv:1608.03983 , 2016

    2016 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.