Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Decentralized federated learning by random walk claims to converge at the same order as SGD while outperforming FedAvg under data and device heterogeneity, without a central server.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

DFedRW runs parallel random walk model updates with decentralized averaging and reports accuracy gains of up to 38 percentage points over FedAvg and DFedAvg under high heterogeneity.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Interesting algorithm and strong experiments, but the convergence theorem doesn't model the aggregation step, so the main theoretical claim is unproven. the 3 major comments →

arxiv 2508.21286 v1 pith:FJWS5BIJ submitted 2025-08-29 cs.DC

Decentralized Federated Averaging via Random Walk

Changheng Wang , Zhiqing Wei , Lizhe Liu , Qiao Deng , Yingda Wu , Yangyang Niu , Yashan Pang , Zhiyong Feng This is my paper
classification cs.DC MSC 68W1590C25
keywords decentralized federated learningrandom walk SGDstatistical heterogeneitysystem heterogeneityquantizationconvex convergencestragglersMarkov chain mixing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DFedRW, a decentralized federated learning algorithm that replaces the repeated local epochs of FedAvg with a random walk: the model is passed from device to device, each performing one SGD step, and after several steps a set of parallel random-walk chains are averaged over the graph. The point is to keep the communication savings of multiple local updates while exposing the model to more heterogeneous data, and to let slower devices contribute partial updates instead of being dropped. Under convex assumptions and a decreasing step size, the authors prove that (quantized) DFedRW converges with an upper bound of order O(1/k^{1-q}), close to the O(1/sqrt(k)) rate of SGD, with constants depending on graph mixing, statistical dissimilarity, and compute capability. Numerically, they report accuracy gains over (D)FedAvg that grow with heterogeneity—up to roughly 38% on their benchmarks—without increasing the communication cost of the busiest device. If correct, this offers FedAvg-like communication efficiency with better robustness to skewed and imbalanced data in serverless settings.

Core claim

The central claim is that a decentralized version of federated averaging can retain the communication efficiency of multiple local updates while avoiding their drift under heterogeneity, by making the updates themselves walk across devices. Formally, with M parallel random-walk trajectories on a connected graph, each trajectory performing K random-walk SGD steps, and a periodic decentralized weighted aggregation that averages the last parameters of the trajectories, the paper proves E[f(w_k)] - f(w*) = O(1/k^{1-q}) for convex L-smooth losses under standard assumptions, for step size eta_k = O(1/k^q) with 1/2 < q < 1. This matches the order of random-walk SGD and is close to centralized SGD's

What carries the argument

The main object is a finite-state Markov chain on the device graph, with a transition matrix P designed by a local acceptance-rejection rule using only degree information, so the walk converges to a stationary distribution over devices. The random walk carries the model from device to device, each performing one SGD step, while M parallel chains run simultaneously. Around this, the algorithm performs a decentralized weighted aggregation, weighted by local sample counts, that averages the last parameters of the chains periodically. The load-bearing quantities are the chain's second-largest eigenvalue modulus lambda_P (mixing time), the delta^2 local dissimilarity, the gamma-inexactness produc

Load-bearing premise

The convergence proof hinges on the random-walk trajectory remaining a single continuous Markov chain across the decentralized aggregation step, even though aggregation averages the last models of M parallel chains; if that continuity does not hold, the mixing bound used in the proof does not apply to the algorithm as implemented.

What would settle it

For M=2 chains on a connected graph, run DFedRW with a fixed step size and log the pre-aggregation and post-aggregation models. If the post-aggregation model cannot be reproduced as the state of a single Markov-chain trajectory with the paper's transition matrix, the mixing argument in the proof is not operating on the actual iterates; then check numerically whether the predicted O(1/k^{1-q}) slope holds for M=2 or only for M=1.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Under the paper's assumptions, DFedRW's convergence rate is O(1/k^{1-q}), so the communication savings of multiple updates do not cost the asymptotic rate of random-walk SGD.
  • Accepting partial random-walk lengths means stragglers are not dropped, removing a source of sampling bias and letting system heterogeneity be absorbed without discarding data.
  • The quantized version pays only an extra bound of order sqrt(d) s from quantization, and Proposition 1 gives a threshold on expected error and bit width below which quantization strictly reduces total communication.
  • Empirical comparisons indicate the advantage over (D)FedAvg grows with both statistical and system heterogeneity and is largest when data is fully non-IID, pointing the method at realistic federated settings rather than IID benchmarks.
  • Sparse topologies such as 3-expander and ring graphs preserve accuracy under IID data but degrade in non-IID settings, making graph connectivity a design parameter in the communication-versus-performance trade-off.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The proof assumes the random-walk trajectory is unbroken across the aggregation step, but the implemented algorithm averages models from M different trajectories; if that assumption fails, the mixing bound of Lemma 6 may not apply as written, and the convergence result is safest when M=1 or when aggregation is infrequent enough for the chain to mix first.
  • The privacy argument is heuristic—each device sees only its immediate predecessor and successor—so a formal privacy analysis or an audit of trajectory leakage would be a natural testable extension the paper does not provide.
  • The theoretical results are convex-only; given the empirical gains on deep networks and LSTM, a plausible conjecture is that random-walk exploration reduces drift in non-convex settings as well, but that remains open.
  • For a fixed communication budget, there should be an optimal combination of number of chains M, walk length K, and graph sparsity; the paper's bounds provide ingredients for such a trade-off but do not optimize it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DFedRW, a decentralized federated averaging algorithm in which local updates are replaced by random-walk SGD steps along a Markov chain on the device graph, with M parallel chains and periodic decentralized weighted averaging (Eq. (11)). A quantized version, QDFedRW, is also introduced. The main theoretical claim is a convex convergence bound of order O(1/k^{1-q}) (Theorem 1) with an additional quantization term (Theorem 2), and Proposition 1 gives conditions under which quantization saves communication. Experiments on MNIST, Fashion-MNIST, and Reddit compare DFedRW/QDFedRW against FedAvg, DFedAvg, and DSGD under various statistical and system heterogeneity levels, reporting accuracy gains and lower communication cost for the busiest device.

Significance. If the theoretical result were valid, the paper would provide a useful convergence guarantee for a communication-efficient decentralized method that handles heterogeneity. The algorithmic idea is interesting and the experimental study is extensive, covering multiple datasets, heterogeneity settings, network topologies, and quantization levels. The paper also makes a reasonable attempt to separate the effects of statistical and system heterogeneity. However, the central convergence proof is not actually about the algorithm as implemented, and the experimental learning-rate schedule falls outside the theorem's assumptions. These issues undermine the main advertised contribution.

major comments (3)
  1. [Section V-B and Appendix A, Eq. (22)/(11)] The proof of Theorem 1 analyzes the single-chain recurrence (22) and never applies the aggregation rule (11). The only bridge is the assertion in Section V-B that the random walk trajectory 'remains unbroken by the aggregation process,' but no equation or argument substantiates this. After aggregation, w_{k+1} is a weighted average of outputs of M chains, and Algorithm 1 line 3 re-samples the initial device, so the pair (i_k, w_k) is not a Markov chain with transition P. Consequently Lemma 6, Eq. (38), which uses P(i_k=i | X_{k-τ}) = [P^τ]_{i_{k-τ},i}, does not apply to DFedRW as implemented. The bound (15) also has no dependence on M or the aggregation weights, confirming that the parallel-chain algorithm is not the object being analyzed. This is a load-bearing gap: without Lemma 6, the claimed O(1/k^{1-q}) rate is unproven.
  2. [Section VI-B, learning rate] The main experiments use η_k = 1/(R k^{0.499}), i.e., q=0.499, while Theorem 1 requires 1/2 < q < 1. For q=0.499, Assumption 2 fails because ∑ ln k · (η_k)^2 diverges when 2q<1. Thus the empirical evaluation does not operate in the theorem's regime, weakening the connection between the numerical results and the theoretical claim. Fig. 10 uses q=0.501, but that is not the main setup.
  3. [Theorem 2 and Appendix B] The proof of Theorem 2 (Appendix B) adds quantization-error bounds to the same single-trajectory argument (Eqs. (46)-(52)) and does not introduce the aggregation step (14) or the M parallel chains. The trajectory-continuity assertion is reused without proof. Hence Theorem 2 is also unsupported.
minor comments (5)
  1. [Abstract vs. Section VI-B] The abstract reports a 38.3% accuracy increase, while Section VI-B reports 38.8% for the same comparison. Please reconcile the numbers.
  2. [Definitions 3 and 4] The state-space size is denoted K in Definitions 3 and 4, but the graph has n devices (V=[n]) and K is also used for the number of random-walk epochs. This is confusing; please use n consistently for the state-space dimension.
  3. [Algorithm 1, line 5] The loop is written 'for k = {0, 2, ..., K_m − 1}'; it should be 'k = 0, 1, ..., K_m − 1'.
  4. [Eq. (38)-(39)] The notation N is used in Eq. (39) ('∑_k N η_k/(2k)') while n is used elsewhere in the same proof; please make the notation consistent.
  5. [Section III-D, transition matrix] The stationary distribution Π* is written as an n×n matrix '[(π*)^T, ..., (π*)^T]^T' but it should be a vector of length n. This is likely a typo.

Circularity Check

0 steps flagged

No significant circularity: the convergence bound follows from standard random-walk SGD inequalities and stated heterogeneity constants; no fitted quantity is relabeled as a prediction and no load-bearing self-citation appears.

full rationale

The derivation of Theorems 1 and 2 is built from ordinary descent lemmas (Appendix A, Eq. (22) onward), standard Markov-chain mixing bounds (Lemmas 2 and 6), and heterogeneity/inexactness constants (δ, γ̂) introduced in Definitions 1-2 and Lemma 1 before the main proof. These constants are problem characteristics or assumptions, not parameters fitted to the numerical results; the empirical 38.3%/37.5% accuracy claims are separate experiments, not the source of the theoretical bound. The quantization bound adds a variance term from Lemma 3, and Proposition 1 compares closed-form communication-cost expressions; none of these steps reintroduces the target O(1/k^{1-q}) rate as an input. There are no author-overlap self-citations used as load-bearing evidence: the cited Markov-chain and random-walk results ([10], [11], [15], [17], [20]-[22]) are external prior works. The real weakness is a proof gap rather than circularity: Section V-B asserts that 'the random walk trajectory remains unbroken by the aggregation process,' and this is what would be needed for the single-chain Markov analysis in Appendix A to apply to the M-chain aggregation in Algorithm 1; without a proof of that assertion, Theorem 1 may not apply to the implemented algorithm. But an unsupported or false bridge is a correctness issue, not a circularity: the bound is not defined in terms of the claim being proved, and no fitted input is being relabeled as a prediction. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 7 axioms · 0 invented entities

The central convergence bound rests on a convexity assumption, Markov chain mixing, and two heterogeneity measures (delta and gamma_hat). The most serious item is the unstated continuity of the trajectory across aggregation, an ad hoc assumption that does not match Algorithm 1. No novel physical or mathematical entities are introduced.

free parameters (3)
  • learning rate schedule exponent q = q=0.499 (main experiments), q=0.501 and q=0.999 (Section VI-D); R=5 or 10
    The step size eta_k = 1/(R k^q) is chosen by hand; the theory requires q in (1/2,1), but the main experiments use q=0.499, outside that range.
  • quantization interval s = not reported
    The QDFedRW bound and Proposition 1 depend on s; experiments fix bit width b but never state s.
  • number of parallel chains M and epochs K = M=20 (image), M=10 (Reddit); K=3 or 5; |NA| = 5, 3, or 20 depending on graph
    Algorithmic hyperparameters chosen by hand; the convergence bound in Theorems 1/2 contains no M or K terms, which is a symptom of the proof not modeling the parallel-chain aggregation.
axioms (7)
  • domain assumption Local loss functions Fi are convex and differentiable on W (Assumption 1)
    Used in the proof of convergence; also implies Lipschitz continuity via Lemma 4.
  • standard math Step sizes satisfy sum eta_k = +inf and sum ln k * eta_k^2 < +inf (Assumption 2)
    Standard decreasing step size conditions for stochastic approximation.
  • domain assumption The random walk Markov chain is time-homogeneous, irreducible, aperiodic, and mixes to pi* (Assumption 3)
    Required for Lemmas 2, 6, 7; depends on the graph being strongly connected with self-loops.
  • domain assumption delta^2-local dissimilarity is bounded (Definition 1)
    Quantifies statistical heterogeneity; enters kappa(delta, gamma_hat) in Theorem 1.
  • domain assumption gamma_t^i-inexactness and compounding bound (Lemma 1): random walk updates satisfy ||grad F_{i_k}(w_k)|| <= gamma_hat ||grad F_{i_{k-K}}(w_{k-K})||
    Models system heterogeneity; requires each gamma_i in [0,1] and the gradient norm contraction to persist over K random walk steps.
  • ad hoc to paper Trajectory continuity across aggregation: 'the random walk before and after aggregation forms a continuous trajectory' (Section V-B)
    This premise is load-bearing but false for the algorithm as written, because aggregation (11) averages the last parameters of M distinct parallel chains, breaking the single-chain Markov indexing used in the proof.
  • domain assumption Parameter norm bound ||w|| <= sigma for quantization
    Used in Lemma 3 and Theorem 2 to bound quantization variance.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Decentralized Federated Averaging via Random Walk." pith.science (2026). https://pith.science/paper/FJWS5BIJ

@misc{pith2026250821286,
  author       = {Pith},
  title        = {Pith review of: Decentralized Federated Averaging via Random Walk},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FJWS5BIJ}},
  note         = {Machine review of arXiv:2508.21286}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Federated Learning (FL) is a communication-efficient distributed machine learning method that allows multiple devices to collaboratively train models without sharing raw data. FL can be categorized into centralized and decentralized paradigms. The centralized paradigm relies on a central server to aggregate local models, potentially resulting in single points of failure, communication bottlenecks, and exposure of model parameters. In contrast, the decentralized paradigm, which does not require a central server, provides improved robustness and privacy. The essence of federated learning lies in leveraging multiple local updates for efficient communication. However, this approach may result in slower convergence or even convergence to suboptimal models in the presence of heterogeneous and imbalanced data. To address this challenge, we study decentralized federated averaging via random walk (DFedRW), which replaces multiple local update steps on a single device with random walk updates. Traditional Federated Averaging (FedAvg) and its decentralized versions commonly ignore stragglers, which reduces the amount of training data and introduces sampling bias. Therefore, we allow DFedRW to aggregate partial random walk updates, ensuring that each computation contributes to the model update. To further improve communication efficiency, we also propose a quantized version of DFedRW. We demonstrate that (quantized) DFedRW achieves convergence upper bound of order $\mathcal{O}(\frac{1}{k^{1-q}})$ under convex conditions. Furthermore, we propose a sufficient condition that reveals when quantization balances communication and convergence. Numerical analysis indicates that our proposed algorithms outperform (decentralized) FedAvg in both convergence rate and accuracy, achieving a 38.3\% and 37.5\% increase in test accuracy under high levels of heterogeneities.

Figures

Figures reproduced from arXiv: 2508.21286 by Changheng Wang, Lizhe Liu, Qiao Deng, Yangyang Niu, Yashan Pang, Yingda Wu, Zhiqing Wei, Zhiyong Feng.

Figure 1
Figure 1. Figure 1: Decentralized federated averaging via random walk. The model of [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of communication and training styles of DSGD, DFedAvgM and the proposed DFedRW. Different colored chains represent different [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Efficiency comparison of DSGD, FedAvg, DFedAvg and DFedRW in training 3FNN for different levels of Non-IID image classification. Fixed absence [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Testing loss comparison of DSGD, FedAvg, DFedAvg and DFedRW [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Efficiency comparison of DSGD, FedAvg, DFedAvg and DFedRW [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Efficiency comparison of DSGD, FedAvg, DFedAvg and DFedRW in training 3FNN for different system heterogeneity in image classification. The [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Training 3FNN on different graphs for MNIST classification using [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Efficiency comparison of DFedRW and DFedAvg in training 2FNN [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 9
Figure 9. Figure 9: Efficiency comparison of QDFedRW and QDFedAvg in training 2FNN [PITH_FULL_IMAGE:figures/full_fig_p014_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Comparison of the impact of relaxing constraints on the empirical [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Efficiency comparison of FedAvg and DFedRW in training LSTM [PITH_FULL_IMAGE:figures/full_fig_p015_13.png] view at source ↗
Figure 12
Figure 12. Figure 12: Impact of network topology and data size on the communication [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 40 canonical work pages

  1. [1]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji et al. , “Advances and open problems in federated learning,” Found. Trends Mach. Learn., vol. 14, no. 1–2, pp. 1–210, 2021

    work page 2021

  2. [2]

    The future of digital health with federated learning,

    N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni et al., “The future of digital health with federated learning,” npj Digit. Med., vol. 3, p. 119, 2020

    work page 2020

  3. [3]

    Fedhome: Cloud-edge based personalized federated learning for in-home health monitoring,

    Q. Wu, X. Chen, Z. Zhou, and J. Zhang, “Fedhome: Cloud-edge based personalized federated learning for in-home health monitoring,” IEEE Trans. Mob. Comput., vol. 21, no. 8, pp. 2818–2832, 2022

    work page 2022

  4. [4]

    Wireless-enabled asynchronous federated fourier neural network for turbulence prediction in urban air mobility (UAM),

    T. Zeng, O. Semiari, W. Saad, and M. Bennis, “Wireless-enabled asynchronous federated fourier neural network for turbulence prediction in urban air mobility (UAM),” IEEE Trans. Wirel. Commun. , vol. 22, no. 11, pp. 7902–7916, 2023

    work page 2023

  5. [5]

    Fl-market: Trading private models in federated learning,

    S. Zheng, Y . Cao, M. Yoshikawa, H. Li, and Q. Yan, “Fl-market: Trading private models in federated learning,” in Proc. 2022 IEEE Int. Conf. Big Data (Big Data) , 2022

    work page 2022

  6. [6]

    Federated learning: Challenges, methods, and future directions,

    T. Li, A. K. Sahu, A. Talwalkar, and V . Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Process. Mag., vol. 37, no. 3, pp. 50–60, 2020

    2020

  7. [7]

    Chain FL: Decentralized federated machine learning via blockchain,

    C. Korkmaz, H. E. Kocas, A. Uysal, A. Masry, O. Ozkasap, and B. Akgun, “Chain FL: Decentralized federated machine learning via blockchain,” in Proc. 2nd Int. Conf. Blockchain Comput. Appl. (BCCA) , 2020, pp. 140–146

    work page 2020

  8. [8]

    A survey on federated learning for resource-constrained IoT devices,

    A. Imteaj, U. Thakker, S. Wang, J. Li, and M. H. Amini, “A survey on federated learning for resource-constrained IoT devices,” IEEE Internet Things J., vol. 9, no. 1, pp. 1–24, 2022

    work page 2022

  9. [9]

    Tackling the ob- jective inconsistency problem in heterogeneous federated optimization,

    J. Wang, Q. Liu, H. Liang, G. Joshi, and H. V . Poor, “Tackling the ob- jective inconsistency problem in heterogeneous federated optimization,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2020

    work page 2020

  10. [10]

    Walkman: A communication-efficient random-walk algorithm for decentralized optimization,

    X. Mao, K. Yuan, Y . Hu, Y . Gu, A. H. Sayed, and W. Yin, “Walkman: A communication-efficient random-walk algorithm for decentralized optimization,” IEEE Trans. Signal Process. , vol. 68, pp. 2513–2528, 2020

    work page 2020

  11. [11]

    On the decentralized stochastic gradient descent with markov chain sampling,

    T. Sun, D. Li, and B. Wang, “On the decentralized stochastic gradient descent with markov chain sampling,” IEEE Trans. Signal Process. , vol. 71, pp. 2895–2909, 2023

    work page 2023

  12. [12]

    Decentralized federated learning with adaptive configuration for heterogeneous par- ticipants,

    Y . Liao, Y . Xu, H. Xu, L. Wang, C. Qian, and C. Qiao, “Decentralized federated learning with adaptive configuration for heterogeneous par- ticipants,” IEEE Trans. Mob. Comput. , vol. 23, no. 6, pp. 7453–7469, 2024

    work page 2024

  13. [13]

    Communication-efficient nonconvex federated learning with error feedback for uplink and downlink,

    X. Zhou, L. Chang, and J. Cao, “Communication-efficient nonconvex federated learning with error feedback for uplink and downlink,” IEEE Trans. Neural Netw. Learn. Syst. , pp. 1–12, 2023

    work page 2023

  14. [14]

    GossipFL: A decentralized federated learning framework with sparsified and adaptive communication,

    Z. Tang, S. Shi, B. Li, and X. Chu, “GossipFL: A decentralized federated learning framework with sparsified and adaptive communication,” IEEE Trans. Parallel Distrib. Syst., vol. 34, no. 3, pp. 909–922, 2023

    work page 2023

  15. [15]

    Decentralized federated averaging,

    T. Sun, D. Li, and B. Wang, “Decentralized federated averaging,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4289–4301, 2023

    work page 2023

  16. [16]

    Communication-efficient design for quantized decentralized federated learning,

    L. Chen, W. Liu, Y . Chen, and W. Wang, “Communication-efficient design for quantized decentralized federated learning,” IEEE Trans. Signal Process., vol. 72, pp. 1175–1188, 2024

    work page 2024

  17. [17]

    Adaptive random walk gradient descent for decentralized optimization,

    T. Sun, D. Li, and B. Wang, “Adaptive random walk gradient descent for decentralized optimization,” in Proc. 39th Int. Conf. Mach. Learn. (ICML), 2022

    work page 2022

  18. [18]

    A unified theory of decentralized SGD with changing topology and local updates,

    A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, and S. Stich, “A unified theory of decentralized SGD with changing topology and local updates,” in Proc. 37th Int. Conf. Mach. Learn. (ICML) , 2020

    work page 2020

  19. [19]

    D-cliques: Compensating for data heterogeneity with topology in decentralized federated learning,

    A. Bellet, A.-M. Kermarrec, and E. Lavoie, “D-cliques: Compensating for data heterogeneity with topology in decentralized federated learning,” in Proc. 41st Int. Symp. Reliable Distrib. Syst. (SRDS) , 2022

    work page 2022

  20. [20]

    Walk for learning: A random walk approach for federated learning from heterogeneous data,

    G. Ayache, V . Dassari, and S. E. Rouayheb, “Walk for learning: A random walk approach for federated learning from heterogeneous data,” IEEE J. Sel. Areas Commun. , vol. 41, no. 4, pp. 929–940, 2023

    work page 2023

  21. [21]

    On markov chain gradient descent,

    T. Sun, Y . Sun, and W. Yin, “On markov chain gradient descent,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2018

    work page 2018

  22. [22]

    Private weighted random walk stochas- tic gradient descent,

    G. Ayache and S. E. Rouayheb, “Private weighted random walk stochas- tic gradient descent,” IEEE J. Sel. Areas Inf. Theory , vol. 2, no. 1, pp. 452–463, 2021

    work page 2021

  23. [23]

    Local learning matters: Rethinking data heterogeneity in federated learning,

    M. Mendieta, T. Yang, P. Wang, M. Lee, Z. Ding, and C. Chen, “Local learning matters: Rethinking data heterogeneity in federated learning,” in Proc. 2022 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2022

    work page 2022

  24. [24]

    Improving the model consistency of decentralized federated learning,

    Y . Shi, L. Shen, K. Wei, Y . Sun, B. Yuan, X. Wang et al., “Improving the model consistency of decentralized federated learning,” in Proc. 40th Int. Conf. Mach. Learn. (ICML) , 2023. 21

    work page 2023

  25. [25]

    On the benefits of multiple gossip steps in communication-constrained decentralized federated learning,

    A. Hashemi, A. Acharya, R. Das, H. Vikalo, S. Sanghavi, and I. Dhillon, “On the benefits of multiple gossip steps in communication-constrained decentralized federated learning,” IEEE Trans. Parallel Distrib. Syst. , vol. 33, no. 11, pp. 2727–2739, 2022

    work page 2022

  26. [26]

    Gossip learning as a decen- tralized alternative to federated learning,

    I. Heged ˝us, G. Danner, and M. Jelasity, “Gossip learning as a decen- tralized alternative to federated learning,” Distrib. Appl. Interoperable Syst., pp. 74–90, 2019

    work page 2019

  27. [27]

    Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,

    X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2017

    work page 2017

  28. [28]

    Beyond spectral gap: the role of the topology in decentralized learning,

    T. V ogels, H. Hendrikx, and M. Jaggi, “Beyond spectral gap: the role of the topology in decentralized learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2022

    work page 2022

  29. [29]

    A survey on decentralized federated learning,

    E. Gabrielli, G. Pica, and G. Tolomei, “A survey on decentralized federated learning,” arXiv:2308.04604, 2023

    arXiv 2023

  30. [30]

    A review of applications in federated learning,

    L. Li, Y . Fan, M. Tse, and K.-Y . Lin, “A review of applications in federated learning,” Comput. Ind. Eng. , vol. 149, p. 106854, 2020

    work page 2020

  31. [31]

    Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges,

    E. T. M. Beltr ´an, M. Q. P ´erez, P. M. S. S´anchez, S. L. Bernal, G. Bovet, M. G. P ´erez et al. , “Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges,” IEEE Commun. Surv. Tutor., vol. 25, no. 4, pp. 2983–3013, 2023

    work page 2023

  32. [32]

    Decentralized federated learning for UA V networks: Architecture, challenges, and opportunities,

    Y . Qu, H. Dai, Y . Zhuang, J. Chen, C. Dong, F. Wuet al., “Decentralized federated learning for UA V networks: Architecture, challenges, and opportunities,” IEEE Netw., vol. 35, no. 6, pp. 156–162, 2021

    work page 2021

  33. [33]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. 20th Int. Conf. Artif. Intell. Stat. (AISTATS) , 2017

    work page 2017

  34. [34]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” in Proc. Mach. Learn. Syst., 2020

    work page 2020

  35. [35]

    Linearly Convergent Asynchronous Distributed ADMM via Markov Sampling,

    S. M. Shah and K. E. Avrachenkov, “Linearly Convergent Asynchronous Distributed ADMM via Markov Sampling,” arXiv:1810.05067, 2018

    Pith/arXiv arXiv 2018

  36. [36]

    Decentralized consensus opti- mization based on parallel random walk,

    Y . Ye, H. Chen, Z. Ma, and M. Xiao, “Decentralized consensus opti- mization based on parallel random walk,” IEEE Commun. Lett., vol. 24, no. 2, pp. 391–395, 2020

    work page 2020

  37. [37]

    SCAFFOLD: Stochastic controlled averaging for federated learning,

    S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “SCAFFOLD: Stochastic controlled averaging for federated learning,” in Proc. 37th Int. Conf. Mach. Learn. (ICML) , 2020

    work page 2020

  38. [38]

    D. A. Levin and Y . Peres, Markov chains and mixing times . Amer. Math. Soc., 2017, vol. 107

    work page 2017

  39. [39]

    Distributed stochastic consensus optimization with momentum for nonconvex nons- mooth problems,

    Z. Wang, J. Zhang, T.-H. Chang, J. Li, and Z.-Q. Luo, “Distributed stochastic consensus optimization with momentum for nonconvex nons- mooth problems,” IEEE Trans. Signal Process., vol. 69, pp. 4486–4501, 2021

    work page 2021

  40. [40]

    D-admm: A communication-efficient distributed algorithm for separable optimiza- tion,

    J. F. Mota, J. M. Xavier, P. M. Aguiar, and M. P ¨uschel, “D-admm: A communication-efficient distributed algorithm for separable optimiza- tion,” IEEE Trans. Signal Process. , vol. 61, no. 10, pp. 2718–2723, 2013

    work page 2013

  41. [41]

    Qsgd: Communication-efficient sgd via gradient quantization and encoding,

    D. Alistarh, D. Grubic, J. Li, R. Tomioka, and M. V ojnovic, “Qsgd: Communication-efficient sgd via gradient quantization and encoding,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS) , 2017

    work page 2017

  42. [42]

    Expander graphs – a study,

    D. Angel, R. M. J. Jothi, R. Revathi, and A. Raja, “Expander graphs – a study,” J. Phys. Conf. Ser. , vol. 1770, no. 1, p. 012078, mar 2021

    work page 2021

  43. [43]

    Learning differentially private recurrent language models,

    H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang, “Learning differentially private recurrent language models,” in Proc. 6th Int. Conf. Learn. Represent. (ICLR) , 2018

    work page 2018

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.