Pith. sign in

REVIEW 3 major objections 4 minor 32 references

A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that adding momentum to the exact-diffusion decentralized SGD update eliminates the convergence error caused by data heterogeneity while keeping step-size flexibility.

desk verdict EDM is a sensible composition of momentum and ED/D2 with a solid non-convex analysis and a PL proof that is currently borrowed, not shown. read the letter →

arxiv 2501.19082 v2 pith:VNMHUAOC submitted 2025-01-31 cs.LG cs.DCmath.OCstat.ML

Yuchen Hu , Xi Chen , Weidong Liu , Xiaojun Mao This is my paper · ORCID
classification cs.LGcs.DCmath.OCstat.ML MSC 90C2690C1568W15
keywords decentralizedstochasticoptimizationmomentumaccelerationdataheterogeneitybiascorrectionexactdiffusionnon-convexconvergencePolyak-Łojasiewiczconditionconsensus
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces EDM (Exact-Diffusion with Momentum), a decentralized stochastic gradient method that combines momentum with the bias-correction mechanism of the exact-diffusion ED/D2 algorithm. The central claim is that momentum can be inserted into ED/D2 without reintroducing the error caused by heterogeneous data across agents. For smooth non-convex losses, the authors prove that the averaged squared gradient converges sublinearly to a neighborhood whose radius depends on the stochastic gradient variance and network mixing, but not on the data-heterogeneity measure, which only appears in a term decaying as $1/T$. Under the Polyak-Lojasiewicz condition, they prove a linear rate to a similar target region. The significance is that earlier momentum-based decentralized methods either retained a heterogeneity-dependent error or required step sizes constrained by the squared spectral gap.

What carries the argument

The proof is carried by isolating the consensus deviation $\|P_\perp X^{(t)}\|_F^2$, where $P_\perp=I-\frac{1}{n}\mathbf{1}\mathbf{1}^\top$. A pseudo-deterministic sequence $\tilde X^{(t)}$ follows the same recursion using true gradients, and Lemma 3 bounds the martingale noise term by $13\alpha^2\lambda^2 n\sigma^2/(1-\lambda)$, a bound independent of the momentum parameter $\beta$. The remaining deterministic part is controlled by transforming the recursion into a linear system $E^{(t+1)}=\Gamma E^{(t)}+\cdots$ with spectral radius $\sqrt{\lambda}$, coupled with a two-dimensional inequality tracking both the consensus error and the gradient-error drift. These bounds feed into a descent lemma for the auxiliary momentum sequence $z^{(t)}=(\bar x^{(t)}-\beta\bar x^{(t-1)})/(1-\beta)$, which converts momentum-based descent into a standard descent inequality with tight constants.

What would settle it

Run EDM on a ring network with $n=32$, heterogeneous quadratic losses with a known optimum, and no gradient noise ($\sigma^2=0$). If the consensus parameter after many iterations remains at distance $O(\zeta_0^2)$ from the optimum rather than converging with the $\zeta_0^2/T$ decay predicted by Theorem 5, the central claim would be falsified; equivalently, EDM should show a strictly smaller final error than DmSGD under these fully deterministic heterogeneous conditions.

Watch

Extended reading notes

Core claim

The central discovery is that momentum can be layered onto the exact-diffusion update without destroying its bias-correction property. With momentum defined by $m_i^{(t+1)}=\beta m_i^{(t)}+(1-\beta)\nabla F_i(x_i^{(t)},\xi_i^{(t)})$, the EDM iteration is $X^{(t+2)}=W(2X^{(t+1)}-X^{(t)}-\alpha M^{(t+1)}+\alpha M^{(t)})$. Theorem 5 states that under Assumptions 1-3, with $\alpha \le \min\{(1-\sqrt{\lambda})/(4L),(1-\beta)/(4L)\}$, the averaged gradient norm satisfies $$\frac{1}{T}\sum_{t=0}^{T-1}\Big(\frac{1}{4}\mathbb{E}\|\nabla\bar f($X^{{(t)}}$)\|^2+\mathbb{E}\|\nabla\bar f(\bar $x^{{(t)}}$)\|^2\Big) \le \frac{2(f($x^{{(0)}}$)-f^\star)}{\$\alpha$ T}+\frac{2\$\alpha$ L\$sigma^{2}$}{n}+O\Big(\frac{\$alpha^{2}$ $L^{2}$\$sigma^{2}$}{1-\$\lambda$}\Big)+\frac{8C_0\$alpha^{2}$ $L^{2}$\$zeta_0^{2}$}{(1-\sqrt{\$\lambda$})^2 T}.$$ The load-bearing point is the final term: the data-heterogeneity measure $\zeta_0^2$ appears only divided by $T$, so it decays and does not set the asymptotic radius. Theorem 6 gives the same qualitative behavior under the PL condition, namely linear convergence to a neighborhood whose radius is independent of $\zeta_0^2$.

Load-bearing premise

The guarantees rely on Assumption 3, which requires each agent's stochastic gradient to be unbiased with variance bounded by $\sigma^2$ and the samples to be independent across agents and time; with data reshuffling, correlated sampling, or biased gradient estimators, the bias-correction guarantee can fail.

Editorial extensions

If this is right

  • When $\beta=0$, EDM reduces to ED/D2, so the analysis recovers the original algorithm's convergence guarantees while removing dependence on the smallest eigenvalue of the communication matrix.
  • Because the $\zeta_0^2$ heterogeneity term decays as $O(1/T)$ in non-convex settings, long runs reach the same noise-dominated neighborhood as if the data had been centralized.
  • Under the PL condition, heterogeneity is eliminated linearly, so the algorithm converges linearly to a region whose radius depends on gradient noise and network mixing but not on data skew.
  • The allowed step size $\alpha\le\min\{(1-\sqrt{\lambda})/(4L),(1-\beta)/(4L)\}$ matches the $O(1-\lambda)$ regime of standard ED/D2, whereas several prior momentum-based decentralized methods required $O((1-\lambda)^2)$ step sizes.
  • The proof technique shows that momentum need not weaken bias correction, supporting the claim that momentum-based and momentum-free bias-corrected algorithms can share the same convergence structure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same proof strategy appears to transfer to momentum versions of DSGT and other bias-correction algorithms in the SUDA family, giving them heterogeneity-free bounds under $O(1-\lambda)$ step sizes; the paper suggests this direction but does not prove it.
  • In practice, the analysis implies that a user can set the momentum parameter $\beta\in[0.8,0.99)$ without re-tuning the step size to the spectral gap, a feature not offered by several competing momentum methods.
  • A direct experimental test implied by Theorem 5 is a comparison of EDM and DmSGD under fully deterministic, highly heterogeneous quadratics with $\sigma^2=0$: EDM should converge to a heterogeneity-independent ball, whereas DmSGD should retain a heterogeneity-dependent offset.
  • Because the heterogeneity term decays like $1/T$ rather than disappearing instantly, finite-time performance still depends on the initial disagreement $\zeta_0^2$; the asymptotic benefit appears only after enough iterations have been run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Exact-Diffusion with Momentum (EDM), a decentralized stochastic optimization algorithm that inserts heavy-ball momentum into the Exact-Diffusion/D^2 bias-correction framework. The main theoretical claims are: under smooth non-convex objectives and unbiased, bounded-variance stochastic gradients (Assumptions 1–3), EDM converges sub-linearly to a neighborhood of the optimum whose asymptotic radius is independent of data heterogeneity (Theorem 5); under the additional Polyak-Łojasiewicz condition (Assumption 4), EDM converges linearly to a heterogeneity-independent target region (Theorem 6). The proof strategy for the non-convex case introduces a pseudo-deterministic sequence coupled with a consensus-variance decomposition, yielding explicit bounds with no fitted constants. Simulations on quadratic losses, logistic regression, and VGG-11 with CIFAR-10 illustrate the algorithm's behavior under varying heterogeneity and network sparsity.

Significance. If the results are correct, the paper makes a substantive contribution: it is, to the author's knowledge, the first momentum variant of ED/D^2 with a convergence analysis, and it shows that momentum does not degrade the heterogeneity-elimination property of bias correction. The non-convex analysis (Lemmas 1–4 and Theorem 5) is detailed and essentially self-contained, with explicit constants and a concrete prediction—the O(1/T) decay of the initial-heterogeneity term—that is not obtained by fitting parameters to data. The comparison with DSGT and DSGT-HB in Table 1, where EDM removes an extra (1−λ)^{-1} factor in the heterogeneity transient, is a meaningful refinement. However, the PL-condition result advertised in the abstract and conclusions is not supported by the proof as written: the key Lyapunov contraction (D.46), the H(t) scaling, and the transition from L(t) to a bound on E f(¯x(t)) are delegated to analogies with Huang et al. (2024) without the needed verification. The paper therefore has a sound non-convex core and a plausible but incompletely proven PL-rate claim.

major comments (3)
  1. [Appendix D, Eq. (D.46)] The proof of Theorem 6 contains a load-bearing gap. After defining the Lyapunov function L(t), the authors state 'Following the proof of theorem 3.8 in Huang et al. (2024), we can get' inequality (D.46), and the H(t) recursion is later introduced as 'Analogous to' Theorem 4.2 of Huang et al. (2024). Inequality (D.46) is the key contraction that produces the linear rate, but it is not derived from the displayed recursions (D.42)–(D.45), and the constants D1, D2, and the Lyapunov weights in L(t) and H(t) are asserted rather than verified. EDM is structurally different from the DSGT-LCA algorithm analyzed in Huang et al. (2024), so the correspondence between the two recursions must be shown explicitly; otherwise the linear-convergence claim in Theorem 6 is unsupported.
  2. [Theorem 6, step-size condition] The step-size condition in Theorem 6 is stated as α ≤ min{(1−√λ)/10, (1−β)/5}, omitting the factor 1/L that is used in the proof, where the condition appears as αL ≤ min{(1−√λ)/10, (1−β)/5}. Since α has units of inverse Lipschitz constant while (1−√λ) is dimensionless, the displayed condition is dimensionally inconsistent and would allow step sizes that violate the proof's requirement when L is large. The theorem statement should read α ≤ min{(1−√λ)/(10L), (1−β)/(5L)}.
  3. [Appendix D, final step after (D.50)] The concluding step of the PL proof does not follow from the displayed inequalities. The bound in (D.50) is for L(t), whose main component is h1(t) = E[ f~(z(t)) ], the excess loss at the auxiliary point z(t). Theorem 6, however, states a bound on E f(¯x(t)) − f*. The relation between z(t) and ¯x(t), and hence between the L(t) bound and the claimed f(¯x(t)) bound, is not established; the H(t) scaling is invoked as analogous to Huang et al. (2024) but the actual inequality connecting L(t), H(t), and E f(¯x(t)) is not written. This is not a cosmetic omission: without it, the theorem's headline statement does not follow from the proof.
minor comments (4)
  1. [Theorem 5] The statement of Theorem 5 contains typographical errors: the first gradient term is written as E||∇¯f(Xt))||² with a missing '(t)' and mismatched parentheses, and the second term uses X^t instead of X^(t); these should be normalized to E||∇¯f(X^(t))||² and E||∇¯f(¯X^(t))||².
  2. [Section 3.2 / Algorithm 1] The notation for the momentum variable is inconsistent: the surrounding text defines m(t+1) using β m(t) + (1−β) ∇F(x(t)), while Algorithm 1 writes v(t) = β m(t−1) + (1−β) g(t), and formula (3.4) uses M(t+1) and M(t). Please align these indices and unify the symbols m and v.
  3. [Lemma 7 proof] The proof says 'F(t) is a martingale' after noting that Ξ(t) is independent of F(t); what is meant is that the innovation sequence forms a martingale difference sequence with respect to the filtration F(t). The phrasing should be corrected for precision.
  4. [Table 1] The table caption refers to 'the bold part' for the heterogeneity term, but no bold formatting is visible in the manuscript; please render the heterogeneity term in bold or otherwise mark it explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the convergence bounds follow from the algorithm's update rules and stated assumptions, and the Theorem 6 proof gap is an omitted derivation rather than a reduction to the paper's inputs.

full rationale

Circularity requires an identity between input and output, and nothing of that kind appears in this paper. The algorithm is defined by update (3.4), the auxiliary pseudo-deterministic sequence by (4.8), and every bound is derived under Assumptions 1-4. No parameter is fitted to data and then renamed a prediction; the heterogeneity term zeta0^2 is an initial-condition quantity whose coefficient in Theorem 5 explicitly carries the factor 1/T, so the advertised elimination of heterogeneity is a derived transient bound, not an input. The cited works Alghunaim and Yuan (2022) and Huang et al. (2024) are by different authors, and they are used as proof techniques and analogous recursion templates, not as premises that already contain EDM's convergence rate. The one notable weakness is in the PL-condition proof: inequality (D.46) is justified only by 'Following the proof of theorem 3.8 in Huang et al. (2024), we can get', and the H(t) bound is delegated with 'Analogous to the scaling done for L(t) ... following the proof of Theorem 4.2 in Huang et al. (2024)'. This is an omitted derivation and therefore a proof-verifiability gap, but it is not circularity: the cited result does not assume EDM's linear rate, and the Lyapunov weights are displayed, so the step is checkable rather than definitionally forced. The concluding sentence acknowledging that momentum mechanisms remain unclear is a stated limitation, not a hidden circular premise. Accordingly, the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No fitted constants. The bounds contain explicit functions of algorithm hyperparameters (alpha, beta, lambda) and problem constants (L, mu, sigma^2, zeta0^2); none are tuned to data. The auxiliary sequences (eX, eY, eN, eR, E(t)) are analytical devices, not new physical entities.

assumptions (5)
  • domain assumption Assumption 1: W is symmetric, doubly stochastic, with positive diagonal and positive smallest eigenvalue.
    Defines the communication protocol. The positive smallest eigenvalue enables the (I-W)^{1/2} factorizations and the companion-matrix spectral bounds used in Lemma 4; the authors note W'=(W+I)/2 can enforce it.
  • domain assumption Assumption 2: each fi is L-smooth and bounded below.
    Gives the standard smoothness inequalities used in Lemmas 1 and 2.
  • domain assumption Assumption 3: stochastic gradients are unbiased for the local gradient and have variance bounded by sigma^2, with independence across agents and time.
    The independence and unbiasedness are what make the martingale bounds in Lemma 3 and Lemma 7 valid; this is the most restrictive condition for real deployments.
  • domain assumption Assumption 4: the global objective satisfies the PL inequality with constant mu.
    Used only for the linear convergence result in Theorem 6.
  • standard math Spectral bounds from Alghunaim and Yuan (2022): for the companion matrix G_i, ||Gamma_i||_op <= sqrt(lambda_i), ||V_i^{-1}||_op <= 1/lambda_i, ||V_i||_op <= 2.
    Imported without proof from Alghunaim and Yuan (2022) in the proof of Lemma 4; they are algebraic facts about the 2x2 matrices that diagonalize the consensus recursion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration." pith.science (2026). https://pith.science/paper/VNMHUAOC

@misc{pith2026250119082,
  author       = {Pith},
  title        = {Pith review of: A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VNMHUAOC}},
  note         = {Machine review of arXiv:2501.19082}
}
read the original abstract

Distributed stochastic optimization algorithms can simultaneously process large-scale datasets, significantly accelerating model training. However, their effectiveness is often hindered by the sparsity of distributed networks and data heterogeneity. In this paper, we propose a momentum-accelerated distributed stochastic gradient algorithm, termed Exact-Diffusion with Momentum (EDM), which mitigates the bias from data heterogeneity and incorporates momentum techniques commonly used in deep learning to enhance convergence rate. Our theoretical analysis demonstrates that the EDM algorithm converges sub-linearly to the neighborhood of the optimal solution, the radius of which is irrespective of data heterogeneity, when applied to non-convex objective functions; under the Polyak-Lojasiewicz condition, which is a weaker assumption than strong convexity, it converges linearly to the target region. Our analysis techniques employed to handle momentum in complex distributed parameter update structures yield a sufficiently tight convergence upper bound, offering a new perspective for the theoretical analysis of other momentum-based distributed algorithms.

Figures

Figures reproduced from arXiv: 2501.19082 by the authors.

Figure 1
Figure 1. Quadratic loss function. We choose σ 2 = 0.05, α = 0.05, λ = 0.99 and use different ζ 2 to control the heterogeneity of fi . The experimental results indicate that all algorithms can converge linearly to a neighborhood of optimal values. However, as data heterogeneity increases, other momentum-based algorithms (DmSGD, Quasi-Global, and DecentLaM) become trapped in heterogeneous regions. Although the DSGT-HB algorith… view at source ↗
Figure 2
Figure 2. Logistic regression with l2-regularization, random noise version. We choose σ 2 s = 0.01, α = 0.5, λ = 0.99, and use σ 2 h to control the variance of x ⋆ i , which reflects the heterogeneity. as cats, airplanes, etc. The photos are 32 × 32 size with three color channels. The criterion is the cross entropy. To quantitatively analyze the heterogeneity of the data in this dataset, we introduce the Dirichlet distributio… view at source ↗
Figure 3
Figure 3. Classification of CIFAR10 by VGG11. Learning rate [PITH_FULL_IMAGE:figures/full_fig_p031_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Classification of CIFAR10 by VGG11. Learning rate [PITH_FULL_IMAGE:figures/full_fig_p032_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 29 canonical work pages

  1. [1]

    Alghunaim, S. A. and K. Yuan (2022). A unified and refined convergence analysis for non-convex decentralized learning. IEEE Transactions on Signal Processing\/ 70 , 3264--3279

    work page 2022

  2. [2]

    Alghunaim, S. A. and K. Yuan (2024). An enhanced gradient-tracking bound for distributed online stochastic convex optimization. Signal Processing\/ 217 , 109345

    work page 2024

  3. [3]

    Ali, M. S., M. Vecchio, M. Pincheira, K. Dolui, F. Antonelli, and M. H. Rehmani (2018). Applications of blockchains in the internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials\/ 21\/ (2), 1676--1717

    work page 2018

  4. [4]

    Beltr \'a n, E. T. M., M. Q. P \'e rez, P. M. S. S \'a nchez, S. L. Bernal, G. Bovet, M. G. P \'e rez, G. M. P \'e rez, and A. H. Celdr \'a n (2023). Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials\/

    work page 2023

  5. [5]

    Parikh, E

    Boyd, S., N. Parikh, E. Chu, B. Peleato, J. Eckstein, et al. (2011). Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine learning\/ 3\/ (1), 1--122

    work page 2011

  6. [6]

    Gao, H. and H. Huang (2020). Periodic stochastic gradient descent with momentum for decentralized training. arXiv preprint arXiv:2008.10435\/

    arXiv 2020

  7. [7]

    Liu, Y.-H

    Gao, J., X.-W. Liu, Y.-H. Dai, Y. Huang, and J. Gu (2023). Distributed stochastic gradient tracking methods with momentum acceleration for non-convex optimization. Computational Optimization and Applications\/ 84\/ (2), 531--572

    work page 2023

  8. [8]

    Phanishayee, O

    Hsieh, K., A. Phanishayee, O. Mutlu, and P. Gibbons (2020). The non-iid data quagmire of decentralized machine learning. In International Conference on Machine Learning , pp.\ 4387--4398. PMLR

    work page 2020

Show all 32 references
  1. [9]

    Pu, and A

    Huang, K., S. Pu, and A. Nedi \'c (2024). An accelerated distributed stochastic gradient method with momentum. arXiv preprint arXiv:2402.09714\/

    2024 arXiv

  2. [10]

    Jiang, Z., A. Balu, C. Hegde, and S. Sarkar (2017). Collaborative deep learning in fixed topology networks. Advances in Neural Information Processing Systems\/ 30

    2017

  3. [11]

    Loizou, S

    Koloskova, A., N. Loizou, S. Boreiri, M. Jaggi, and S. Stich (2020). A unified theory of decentralized sgd with changing topology and local updates. In International Conference on Machine Learning , pp.\ 5381--5393. PMLR

    2020

  4. [12]

    Zhang, H

    Lian, X., C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu (2017). Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. Advances in Neural Information Processing Systems\/ 30

    2017

  5. [13]

    Lin, T., S. P. Karimireddy, S. Stich, and M. Jaggi (2021). Quasi-global momentum: Accelerating decentralized deep learning on heterogeneous data. In International Conference on Machine Learning , pp.\ 6654--6665. PMLR

    2021

  6. [14]

    Gao, and W

    Liu, Y., Y. Gao, and W. Yin (2020). An improved analysis of stochastic gradient descent with momentum. Advances in Neural Information Processing Systems\/ 33 , 18261--18271

    2020

  7. [15]

    Olshevsky, and M

    Nedi \'c , A., A. Olshevsky, and M. G. Rabbat (2018). Network topology and communication-computation tradeoffs in decentralized optimization. Proceedings of the IEEE\/ 106\/ (5), 953--976

    2018

  8. [16]

    Nedic, A. and A. Ozdaglar (2009). Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automatic Control\/ 54\/ (1), 48--61

    2009

  9. [17]

    Nesterov, Y. (2013). Introductory lectures on convex optimization: A basic course , Volume 87. Springer Science & Business Media

    2013

  10. [18]

    Polyak, B. T. (1964). Some methods of speeding up the convergence of iteration methods. Ussr Computational Mathematics and Mathematical Physics\/ 4\/ (5), 1--17

    1964

  11. [19]

    Polyak, B. T. (1987). Introduction to optimization . New York, Optimization Software

    1987

  12. [20]

    Pu, S. and A. Nedi \'c (2021). Distributed stochastic gradient tracking methods. Mathematical Programming\/ 187\/ (1), 409--457

    2021

  13. [21]

    Sayed, A. H. et al. (2014). Adaptation, learning, and optimization over networks. Foundations and Trends in Machine Learning\/ 7\/ (4-5), 311--801

    2014

  14. [22]

    Olshevsky, and I

    Spiridonoff, A., A. Olshevsky, and I. C. Paschalidis (2020). Robust asynchronous stochastic gradient-push: Asymptotically optimal and network-independent performance for strongly convex functions. Journal of Machine Learning Research\/ 21\/ (58), 1--47

    2020

  15. [23]

    Takezawa, Y., H. Bao, K. Niwa, R. Sato, and M. Yamada (2023). Momentum tracking: Momentum acceleration for decentralized deep learning on heterogeneous data. Transactions on Machine Learning Research\/

    2023

  16. [24]

    Tang, H., X. Lian, M. Yan, C. Zhang, and J. Liu (2018). d^ 2 : Decentralized training over decentralized data. In International Conference on Machine Learning , pp.\ 4848--4856. PMLR

    2018

  17. [25]

    Vogels, T., L. He, A. Koloskova, S. P. Karimireddy, T. Lin, S. U. Stich, and M. Jaggi (2021). Relaysum for decentralized deep learning on heterogeneous data. Advances in Neural Information Processing Systems\/ 34 , 28004--28015

    2021

  18. [26]

    Xin, R., S. Pu, A. Nedi \'c , and U. A. Khan (2020). A general framework for decentralized optimization with first-order methods. Proceedings of the IEEE\/ 108\/ (11), 1869--1889

    2020

  19. [27]

    Jin, and S

    Yu, H., R. Jin, and S. Yang (2019). On the linear speedup analysis of communication efficient momentum sgd for distributed non-convex optimization. In International Conference on Machine Learning , pp.\ 7184--7193. PMLR

    2019

  20. [28]

    Yuan, K., S. A. Alghunaim, B. Ying, and A. H. Sayed (2020). On the influence of bias-correction on distributed stochastic optimization. IEEE Transactions on Signal Processing\/ 68 , 4352--4367

    2020

  21. [29]

    Yuan, K., Y. Chen, X. Huang, Y. Zhang, P. Pan, Y. Xu, and W. Yin (2021). Decentlam: Decentralized momentum sgd for large-batch deep training. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pp.\ 3029--3039

    2021

  22. [30]

    Ling, and W

    Yuan, K., Q. Ling, and W. Yin (2016). On the convergence of decentralized gradient descent. SIAM Journal on Optimization\/ 26\/ (3), 1835--1854

    2016

  23. [31]

    Agarwal, S

    Yurochkin, M., M. Agarwal, S. Ghosh, K. Greenewald, N. Hoang, and Y. Khazaeni (2019). Bayesian nonparametric federated learning of neural networks. In International Conference on Machine Learning , pp.\ 7252--7261. PMLR

    2019

  24. [32]

    Zhang, J. and K. You (2019). Decentralized stochastic gradient tracking for non-convex empirical risk minimization. arXiv preprint arXiv:1909.02712\/

    2019 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.