Pith. sign in

REVIEW 4 major objections 4 minor 66 references

Decentralized Federated Learning by Partial Message Exchange

T0 review · 4 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A decentralized learning algorithm that sends only random coordinate slices can converge linearly under just local smoothness and a doubly stochastic mixing matrix.

desk verdict The PME mechanism and the milder-assumption proof attempt are genuinely interesting, but the headline linear-rate guarantee is to an unproven limit point, not to a solution, and the enabling condition (12) is never verified for any admissible setting. read the letter →

arxiv 2603.01730 v3 pith:ODVQ64GW submitted 2026-03-02 cs.LG

Shan Sha , Shenglong Zhou , Xin Wang , Lingchen Kong , Geoffrey Ye Li This is my paper · ORCID
classification cs.LG
keywords decentralizedfederatedlearningpartialmessageexchangecommunicationcompressionrandomcoordinatesamplinglinearconvergencelocalLipschitzcontinuitydoublystochasticmatrixdataheterogeneity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes PaME, a decentralized federated learning method in which each node receives only a sparse, random subset of coordinates from each selected neighbor and fills missing entries with its own values. It claims that this partial message exchange converges at a linear rate—O(γ^{-k}) in squared model distance and O(γ^{-k/2}) in objective error—using only two assumptions: locally Lipschitz gradients and a doubly stochastic communication matrix. No convexity, bounded-gradient, bounded-variance, or unbiased-compression conditions are needed. A sympathetic reader would take away that significant communication reduction need not cost convergence speed or accuracy relative to full-message decentralized SGD, and that the analysis extends to dynamic and asynchronous communication patterns. The theorems establish convergence to a limit ϖ∞, but they do not assert that ϖ∞ is a stationary point or that f(ϖ∞) is the global minimum.

What carries the argument

The mechanism carrying the argument is the partial message exchange (PME) averaging rule. Each neighbor sends s randomly chosen coordinates of its current parameter; the receiver aggregates coordinate ℓ by dividing by the number of received nonzero copies of ℓ, and substitutes its own coordinate when λ_{i,ℓ}=0. Theorem 1 shows this conditional averaging is an unbiased estimate of the true neighbor average, unlike naive sparse averaging, which shrinks toward zero by factor s/n. Lemma 5 bounds the resulting perturbation by C1‖W^k−Π^k‖²_F with C1 < (γ^{-k0/2}−ζ)^2, which feeds a consensus-error recursion decaying like β^{-k} with β>γ; combined with the monotone descent of the augmented objectiv

What would settle it

Find any connected graph and real parameters (p, ν, γ, k0) satisfying (12) with all t_i≥1, or verify (12) on the paper's own experiments by reporting ζ and degrees; if no such configuration exists, Theorem 4's parameter setup is empty. Alternatively, on a small quadratic problem, check whether the limit ϖ∞ has ∇f(ϖ∞)=0 when the algorithm is run with the claimed settings.

Watch

Extended reading notes

Core claim

The central claim is that PaME generates sequences W^k, V^k and averages ϖ^k for which E‖W^k−W∞‖²_F=O(γ^{-k}), E‖V^k−W∞‖²_F=O(γ^{-k}), and E|f(ϖ^k)−f(ϖ∞)|=O(γ^{-k/2}), under Assumptions 1 and 2. Assumption 1 requires only that the initial communication matrix B be doubly stochastic with spectral gap ζ<1; later rounds may use dynamic, sparse, even non-doubly-stochastic matrices. Assumption 2 requires only that each local gradient be Lipschitz on the bounded box N(2δ), not globally. The authors attribute the relaxation to a deterministic boundedness result for the iterates, obtained by growing the penalty parameter geometrically (σ_k=σ0 γ^k), so stochastic-gradient noise never needs independen

Load-bearing premise

The proof's load-bearing premise is that every node can fix t_i=⌊ν_i|N_i|⌋≥1 while satisfying inequality (12), (1−p)^{t_i}(1+ζ)^2+2pΣ_{j∈N_i}ν_j < (γ^{-k0/2}−ζ)^2; the paper sketches a limiting parameter choice that seems not to meet this premise, and no explicit feasible instance is provided.

Editorial extensions

If this is right

  • Per-round transmission drops from 64n to 63s+n bits; at s/n=0.1–0.2 the paper reports accuracy comparable to full communication and, on its benchmarks, the fewest communication rounds among compared algorithms.
  • Because no convexity, PL-condition, bounded-gradient, or bounded-variance assumptions are used, the convergence theorem applies to heterogeneous local objectives and to stochastic gradients without extra moment control.
  • Only the initial mixing matrix needs doubly stochasticity, so the algorithm tolerates time-varying, sparse, or asynchronous communication graphs—a regime many existing analyses forbid.
  • If the claimed rates hold, the objective gap decays geometrically, so for any target accuracy the required communication budget can be estimated from γ and the initial gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The theorems stop at f(ϖ^k)→f(ϖ∞); they do not show ∇f(ϖ∞)=0. If a stronger claim of linear convergence to an optimizer is intended, a stationarity argument would need to be added; otherwise the linear rate is about reaching some consensus point, not about solving the original problem.
  • The existence of parameters satisfying condition (12) with t_i=⌊ν_i|N_i|⌋≥1 is asserted via 'p close to 1 and ν_i close to 0', but that direction likely fails: ν→0 kills t_i, and p→1 makes the left side at least 2 while the right side is at most (1−ζ)²<1. No concrete graph and parameter set satisfying (12) is exhibited, and the experimental settings do not report the ζ and degrees needed to verify
  • A natural testbed extension: run the same update on a small strongly convex problem, record the limit ϖ∞, and check whether ∇f(ϖ∞)=0; this would tell readers how much of the advertised linear convergence is convergence to a solution rather than merely to a limit.
  • The privacy claim is qualitative (random sparsification complicates inference). A concrete measure—e.g., gradient-inversion success rate versus s/n—would let practitioners set transmission rates by a privacy budget.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PaME, a decentralized federated learning algorithm in which each node averages only randomly selected coordinates received from a random subset of neighbors, using an inexact-ADMM-type local update with a geometrically increasing penalty parameter. The main theoretical claim is that, under only local Lipschitz continuity of the gradients on a bounded region and a doubly stochastic communication matrix with spectral gap, the iterates and objective values converge linearly to a limit (Theorems 2–4). The paper also reports extensive experiments on linear regression, logistic regression, Fashion-MNIST, and CIFAR-10, claiming favorable communication efficiency and accuracy relative to several decentralized baselines.

Significance. If the theoretical claim were valid, the result would be significant: linear convergence under local rather than global smoothness, with no bounded-gradient, bounded-variance, or unbiased-compression assumptions, would be a notable advance over existing DFL analyses. The paper contains a clean derivation of the sparse-exchange estimator (Theorem 1), substantial supplemental proofs, and broad numerical comparisons. However, the central guarantee is not established: the enabling condition (Setup 1(4), Eq. (12)) is never instantiated and the supplied existence argument is invalid, and the proved linear rates target a limit whose stationarity or optimality is not shown. As it stands, the paper does not substantiate the advertised convergence of PaME to a solution of problem (1).

major comments (4)
  1. [Setup 1(4), Eq. (12)] The contraction condition (12) is never verified and the stated existence argument fails. For any γ∈(1,ζ^{-2/k0}) the RHS satisfies (γ^{-k0/2}-ζ)^2 < (1-ζ)^2 <1. The text after (11) proposes 'p close to 1 and ν_i close to 0'. But ν_i close to 0 makes t_i=⌊ν_i|N_i|⌋=0, so Algorithm 1 (line 13) divides by zero; for a connected graph, t_i≥1 requires ν_i≥1/|N_i|, so ν_i cannot be arbitrarily small. For p→1 the LHS of (12) tends to 2Σ_{j∈N_i}ν_j, which on a regular graph is at least 2 because each ν_j≥1/|N_i|. No admissible instance is exhibited, and the experimental settings (Table II, ν=0.2, p=0.2) are not checked against (12). Lemma 4, Lemma 6, and Theorems 2–4 all rely on this condition, so the theory is conditional on an uninstantiated and apparently infeasible setup.
  2. [Theorem 4 / Lemma 8] The advertised 'linear convergence' is convergence to a limit, not to an optimum. Theorem 4 bounds E∥W^k−W∞∥²_F and E|f(ϖ^k)−f(ϖ∞)|, but no result proves ∇f(ϖ∞)=0 or f(ϖ∞)=min f; Lemma 8 only gives L2 convergence to the iterate limit. Since σ^k=σ_0γ^k, the effective step size 1/(σ^k t_i) is summable, so the recursion can freeze away from stationarity. For identical quadratics f_i(w)=L/2∥w−c∥² with B=J and full-batch gradients, w^{k+1}-c=(1-L/(σ_0γ^k t))(w^k-c); the infinite product is positive, so w^k→w∞≠c with ∇f(w∞)≠0 while Theorem 4's rates hold. Thus the paper does not establish linear convergence to a solution of (1), only to a limit whose optimality is unresolved.
  3. [Assumption 1, Eq. (7)] The definition B_ji=1/m_i for j∈N_i makes B column-stochastic by construction; row-stochasticity (and hence double stochasticity) holds only for regular graphs, or with a transposed definition. Assumption 1 asserts B is doubly stochastic, and the subsequent proofs use both B1=1 and 1^T B=1^T (Supplement Eq. (8)). As written, the analysis either silently restricts to regular graphs or needs a different B. This contradicts the paper's claim of an arbitrary connected communication topology and weakens the 'mildest assumptions' statement.
  4. [Setup 1(1) vs. §V-B] The theory assumes κ_i=k_0 for all nodes, so all nodes communicate at the same global times k∈K_0. The experiments choose κ_i randomly from e.g. [3,7] per node (Table II). The statement that taking k_0 as an LCM makes the analysis 'similar' does not cover the asynchronous case: at a time that is a multiple of one node's period but not another's, only a subset of nodes communicates, which is not the synchronized k∈K_0 versus k∉K_0 dichotomy used in Lemma 6. Hence the experimental validation of asynchronous/heterogeneous periods is not supported by the theorems.
minor comments (4)
  1. [Algorithm 2] Line 3 of Algorithm 2 has a typo: both branches read 'ℓ∈T_j^k'; the second branch should be 'ℓ∉T_j^k'.
  2. [Theorem 1] In the supplemental statement of Theorem 1, 'uniformly selecting s coordinates of w_i without replacement from [q]' should be 'from [n]'.
  3. [Setup 1(2)] The setup should explicitly require t_i≥1 (i.e., ν_i|N_i|≥1). As written, ν_i∈(0,1] and |N_i|=1 give t_i=0 unless ν_i=1, causing division by zero in Algorithm 1.
  4. [Figures/Tables] The caption and surrounding text for Fig. 7 appear garbled in the reviewed version. Also, Table II is used both for the assumption comparison and for experimental parameter choices, which is confusing.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the convergence proof is self-contained and no fitted quantity is fed back as a prediction.

full rationale

I walked the derivation chain from Setup 1 through Lemmas 1–8 and Theorems 2–4 in the main text and supplement. The convergence proof is a genuine derivation: Theorem 2 establishes a Lyapunov descent inequality for H^k, Lemma 6 proves consensus-error contraction under the doubly-stochastic mixing and the explicit parameter condition (12), and Lemma 8 converts the resulting geometric increment bound E∥Δw^k∥² ≤ c²/γ^k into L2 convergence and the stated O(γ^{-k}) / O(γ^{-k/2}) rates. No fitted parameter is renamed as a prediction; the constants (σ, C, D, η, C2, β) are all constructed from problem data, not calibrated to observed outputs. The self-citations [17,24,43] are used for background, algorithm lineage, and examples; none of the theorems relies on them as an external proof or uniqueness claim. The advertised convergence is explicitly to a limit point ϖ∞ rather than to a stationary point, and the feasibility of condition (12) is asserted with an invalid 'p→1, ν→0' argument; these are substantive correctness/interpretation concerns, but they are not circularity because the theorem is conditional on (12) and the limit is defined rather than assumed equal to the optimum. The derivation does not reduce, by definition or by self-citation, to its own inputs.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The proof of Theorems 2–4 rests on two structural assumptions, a common-period synchronization assumption, an uncomputable batch-deviation bound, and seven coupled free parameters (σ_0, γ, p, ν_i, δ, k_0, ζ). The paper gives no constructive recipe for these, offers an invalid existence argument for (12), and reports experimental parameters that violate the theory's own conditions. Net epistemic load: the convergence guarantee is conditional on an unverified and likely infeasible regime.

free parameters (7)
  • σ_0 (initial penalty parameter) = σ_0 ≥ max{4α_max, ε(2δ)γ/((γ−1)δ t_min)}; experiments use 1.0 or 5.0
    Chosen by hand. The lower bound depends on unknown problem data (Lipschitz constants α_i, batch-deviation sup ε(2δ), region radius δ). Since the final limit point shifts with σ_0, the algorithm's output depends on this choice.
  • γ (penalty growth rate) = 1.005 (Ex. 1–2), 1.001 (Ex. 3–4); theory requires γ ∈ (1, ζ^{−2/k0})
    Controls the geometric stepsize decay. Must jointly satisfy (12); also controls how far the non-stationary limit can sit from the optimum.
  • p = s/n (coordinate transmission rate) = 0.2 (Ex. 1–2), 0.1 (Ex. 3–4)
    Free in (0,1) but constrained by (12), which forces p to be small in admissible regimes; the paper's experimental values do not satisfy (12).
  • ν_i (neighbor participation rate) = 0.2 or 0.5 in experiments
    Must satisfy t_i = ⌊ν_i|N_i|⌋ ≥ 1 for the update step to be defined. Condition (12) forces near-minimal participation for neighbors of high-degree nodes; the experimental values violate it.
  • δ (radius of local-Lipschitz region) = not specified
    Assumption 2 posits Lipschitz gradients on N(2δ) for "a given δ"; the σ_0 lower bound depends on δ and ε(2δ), but no constructive choice is given.
  • k_0 (communication period) = [3,7] or [5,10] in experiments
    Theory requires identical κ_i = k_0 for all nodes; experiments draw κ_i randomly per node. The rate in Theorem 4 and condition (12) depend on k_0.
  • ζ (spectral gap bound) = not reported for experimental graphs
    Assumption 1. The admissible γ-window and condition (12) hinge on ζ, but the spectral gap of the experimental graphs is never reported, so (12) cannot be checked.
assumptions (6)
  • domain assumption Assumption 1: the m×m communication matrix B (B_ji = 1/|N_i| if j∈N_i, else 0) is doubly stochastic with ζ = max{|λ_2(B)|, |λ_m(B)|} < 1; the graph is connected and undirected.
    Provides the consensus contraction ∥B−J∥ ≤ ζ used in Lemma 6. Under this normalization, doubly stochasticity forces all nodes to have equal degree, so the assumption excludes graphs like paths or stars used informally in Section II.
  • domain assumption Assumption 2: each ∇f_i is α_i-Lipschitz on N(2δ) = {w : ‖w‖∞ ≤ 2δ} for a given δ ∈ (0,∞).
    The analysis is entirely local: Lemma 3 must prove the iterates stay inside N(2δ), and the σ_0 lower bound depends on α_max and ε(2δ). No construction of δ is given.
  • domain assumption Each f_i is continuously differentiable and bounded below, with finite f*; batches B ⊆ D_i are i.i.d. samples.
    Used to make H^k bounded below (Lemma 7) and to define the descent arguments; standard for FL.
  • ad hoc to paper ε_i(t) := sup_{B,B′⊆D_i, w∈N(t)} ‖∇f_i(w;B) − ∇f_i(w;B′)‖∞ is finite and bounded by a known constant (Supplement Eq. (12)).
    Introduced to control stochastic-gradient noise without a bounded-variance assumption; its value is needed to set σ_0 but is not computable from the paper, so part of the 'mild assumption' load hides here.
  • ad hoc to paper Setup 1(4): γ ∈ (1, ζ^{−2/k0}), p = s/n, ν_i ∈ (0,1] satisfy (12) for all i, and σ_0 ≥ σ = max{4α_max, ε(2δ)γ/((γ−1)δt_min)}.
    Load-bearing for Lemmas 4–6 and Theorems 2–4. The paper's existence claim (p→1, ν→0) is invalid: p→1 makes the LHS ≥ 2 > (1−ζ)², and ν→0 forces t_i=0 and division by zero in Algorithm 1 line 13. No feasible instance is exhibited; the experiments violate (12).
  • domain assumption Setup 1(1): all nodes share a common communication period κ_i = k_0.
    The proof uses synchronized blocks K_0 = {0, k_0, 2k_0, ...}; the claim that heterogeneous κ_i follow by an LCM argument is asserted without proof, and the experiments use random κ_i.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decentralized Federated Learning by Partial Message Exchange." pith.science (2026). https://pith.science/paper/ODVQ64GW

@misc{pith2026260301730,
  author       = {Pith},
  title        = {Pith review of: Decentralized Federated Learning by Partial Message Exchange},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ODVQ64GW}},
  note         = {Machine review of arXiv:2603.01730}
}
read the original abstract

Decentralized federated learning (DFL) has emerged as a transformative server-free paradigm that enables collaborative learning over large-scale heterogeneous networks. However, it continues to face fundamental challenges, including data heterogeneity, restrictive assumptions for theoretical analysis, and degraded convergence when standard communication- or privacyenhancing techniques are applied. To overcome these drawbacks, this paper develops a novel algorithm, PaME (DFL by Partial Message Exchange). The central principle is to allow only randomly selected sparse coordinates to be exchanged between two neighbor nodes. Consequently, PaME achieves substantial reductions in communication costs while still preserving a high level of privacy, without sacrificing accuracy. Moreover, grounded in rigorous analysis, the algorithm is shown to converge at a linear rate under the gradient to be locally Lipschitz continuous and the communication matrix to be doubly stochastic. These two mild assumptions not only dispense with many restrictive conditions commonly imposed by existing DFL methods but also enables PaME to effectively address data heterogeneity. Furthermore, comprehensive numerical experiments demonstrate its superior performance compared with several representative decentralized learning algorithms.

Figures

Figures reproduced from arXiv: 2603.01730 by the authors.

Figure 1
Figure 1. Partial message exchange (PME) : Every neighbor of a local node i randomly selects partial coordinates (or messages) and transmits them to node i. Node i averages the received incomplete parameters and fills in the missing coordinates using the coordinates in its own local parameter. PL-type conditions [29], together with additional stochastic￾gradient or mixing assumptions, see Table II. Additionally, since the PME… view at source ↗
Figure 2
Figure 2. Self-comparison of PaME. 1) Effect of transmission rate: Fig. 2a illustrates the effect of varying the transmission rate s/n ∈ {0.1, 0.2, · · · , 0.6} across different numbers of nodes m ∈ {32, 64, 128}, with the participation rate fixed at ν = 0.2. The results indicate that while higher transmission rates generally yield lower final objective values (i.e., improved accuracy), a high transmission rate is not strictl… view at source ↗
Figure 3
Figure 3. Self comparison of PaME: Convergence curve of different transmission rate. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Self comparison of PaME: Convergence curve of different participation rate. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Self comparison of PaME: Convergence curve of different CPs (homogeneous). [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Self comparison of PaME: Convergence curve of different CPs (heterogeneous). [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Self-comparison of PaME: Heatmap of final objective, iteration, and runtime. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Accuracy v.s. CR. vation, and model utility. Its effectiveness stems from a novel partial message exchange mechanism, which is well suited to a variety of real-world scenarios, including unreliable wireless communications and edge computing environments. Moreover, rigo…
Figure 9
Figure 9. Figure 9: CR v.s. m. m=32 m=64 m=128 10 6 10 7 10 8 10 9 10 10 10 11 DTV n=1000 PaME DPSGD BEER DFedSAM ANQ-NIDS m=32 m=64 m=128 10 6 10 7 10 8 10 9 10 10 10 11 DTV n=5000 PaME DPSGD BEER DFedSAM ANQ-NIDS m=32 m=64 m=128 10 6 10 7 10 8 10 9 10 10 10 11 DTV n=10000 PaME DPSGD BEE…
Figure 10
Figure 10. Figure 10: Data transmission volume (DTV) v.s. m. 0 10 20 30 40 50 CR 0.1 0.5 0.8 Accuracy C = 10 PaME DPSGD BEER DFedSAM ANQ-NIDS 0 10 20 30 40 50 CR 0.1 0.5 0.8 Accuracy C = 7 PaME DPSGD BEER DFedSAM ANQ-NIDS 0 10 20 30 40 50 CR 0.1 0.5 0.8 Accuracy C = 1 PaME DPSGD BEER DFedS…
Figure 11
Figure 11. Figure 11: Accuracy v.s. CR. Comput., 2024. [5] R. S. Nuvvula, P. P. Kumar, P. Akki, S. R. Ahammed, A. Ali et al., “Federated learning-based energy fore￾casting and trading platform for decentralized renewable energy markets,” in Int. Conf. Smart Grid. IEEE, 2024, pp. 277–283. […
Figure 12
Figure 12. Figure 12: Accuracy v.s. CR. Boosting decentralized learning with randomized com￾munication,” Adv. Neural Inf. Process. Syst., vol. 36, pp. 36 132–36 164, 2023. [12] L. Wang, Y. Chen, Y. Guo, and X. Tang, “Smart sam￾pling: Helping from friendly neighbors for decentralized federa…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 6 linked inside Pith

  1. [1]

    Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges,

    E. T. M. Beltrán, M. Q. Pérez, P. M. S. Sánchez, S. L. Bernal, G. Bovet, M. Gil, G. Martínez, and A. H. Cel- drán, “Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges,” IEEE Commun. Surveys Tuts., 2023

    2023

  2. [2]

    Decentralized federated learning: A survey on security and privacy,

    E. Hallaji, R. Razavi-Far, M. Saif, B. Wang, and Q. Yang, “Decentralized federated learning: A survey on security and privacy,”IEEE Trans. Big Data, vol. 10, no. 2, pp. 194–213, 2024

    2024

  3. [3]

    An innovative decentralized and distributed deep learning framework for predictive maintenance in the industrial internet of things,

    M. Alabadi, A. Habbal, and M. Guizani, “An innovative decentralized and distributed deep learning framework for predictive maintenance in the industrial internet of things,”IEEE Internet Things J., vol. 11, no. 11, pp. 20 271–20 286, 2024

    2024

  4. [4]

    Decentralized navigation with heterogeneous federated reinforcement learning for uav- enabled mobile edge computing,

    P. Wang, H. Yang, G. Han, R. Yu, L. Yang, G. Sun, H. Qi, X. Wei, and Q. Zhang, “Decentralized navigation with heterogeneous federated reinforcement learning for uav- enabled mobile edge computing,”IEEE Trans. Mobile JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12 m=32 m=64 m=1280 20 40 60 80 100 120CR n=1000 PaME DPSGD BEER DFedSAM ANQ-NIDS m...

    2021

  5. [5]

    Federated learning-based energy fore- casting and trading platform for decentralized renewable energy markets,

    R. S. Nuvvula, P. P. Kumar, P. Akki, S. R. Ahammed, A. Aliet al., “Federated learning-based energy fore- casting and trading platform for decentralized renewable energy markets,” inInt. Conf. Smart Grid. IEEE, 2024, pp. 277–283

    2024

  6. [6]

    Federated learning meets blockchain in decentralized data sharing: Healthcare use case,

    S. H. Alsamhi, R. Myrzashova, A. Hawbani, S. Kumar, S. Srivastava, L. Zhao, X. Wei, M. Guizan, and E. Curry, “Federated learning meets blockchain in decentralized data sharing: Healthcare use case,”IEEE Internet Things J., vol. 11, no. 11, pp. 19 602–19 615, 2024

    2024

  7. [7]

    Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,

    X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent,”Adv. Neural Inf. Process. Syst., vol. 30, 2017

    2017

  8. [8]

    Improving the model consistency of decentral- ized federated learning,

    Y . Shi, L. Shen, K. Wei, Y . Sun, B. Yuan, X. Wang, and D. Tao, “Improving the model consistency of decentral- ized federated learning,” inProc. Int. Conf. Mach. Learn., 2023, pp. 31 269–31 291

    2023

Show all 66 references
  1. [9]

    Topology-aware generalization of decentralized sgd,

    T. Zhu, F. He, L. Zhang, Z. Niu, M. Song, and D. Tao, “Topology-aware generalization of decentralized sgd,” in Proc. Int. Conf. Mach. Learn., 2022, pp. 27 479–27 503

    2022

  2. [10]

    Decentralized optimiza- tion with edge sampling,

    C. Zhang, Q. Li, and P. Zhao, “Decentralized optimiza- tion with edge sampling,” inProc. Int. Joint Conf. Artif. Intell., 2019, pp. 658–664

    2019

  3. [11]

    De V os, S

    M. De V os, S. Farhadkhani, R. Guerraoui, A.-M. Ker- marrec, R. Pires, and R. Sharma, “Epidemic learning: JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13 0 50 100 150 CR 0.1 0.5 0.8Accuracy IID PaME DPSGD BEER DFedSAM ANQ-NIDS 0 50 100 150 CR 0.1 0.5 0.8Accuracy N...

    2021

  4. [12]

    Smart sam- pling: Helping from friendly neighbors for decentralized federated learning,

    L. Wang, Y . Chen, Y . Guo, and X. Tang, “Smart sam- pling: Helping from friendly neighbors for decentralized federated learning,”arXiv preprint arXiv:2407.04460, 2024

    2024 arXiv

  5. [13]

    Personalized decentralized federated learning: A privacy-enhanced and byzantine-resilient approach,

    A. Zhang, P. Zhao, W. Lu, and G. Zhang, “Personalized decentralized federated learning: A privacy-enhanced and byzantine-resilient approach,”IEEE Trans. Comput. So- cial Syst., 2025

    2025

  6. [14]

    Distributed subgradient methods for multi-agent optimization,

    A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,”IEEE Trans. Au- tom. Control, vol. 54, no. 1, pp. 48–61, 2009

    2009

  7. [15]

    Com- munication compression for decentralized training,

    H. Tang, S. Gan, C. Zhang, T. Zhang, and J. Liu, “Com- munication compression for decentralized training,”Adv. Neural Inf. Process. Syst., vol. 31, 2018

    2018

  8. [16]

    The convergence of spar- sified gradient methods,

    D. Alistarh, T. Hoefler, M. Johansson, N. Konstantinov, S. Khirirat, and C. Renggli, “The convergence of spar- sified gradient methods,”Adv. Neural Inf. Process. Syst., vol. 31, 2018

    2018

  9. [17]

    Sparse decen- tralized federated learning,

    S. Sha, S. Zhou, L. Kong, and G. Y . Li, “Sparse decen- tralized federated learning,”IEEE Trans. Signal Process., vol. 73, pp. 3406–3420, 2025

    2025

  10. [18]

    Decentralized learning with unreliable communications,

    H. Ye, L. Liang, and G. Y . Li, “Decentralized learning with unreliable communications,”IEEE J. Sel. Top. Sig- nal Process, vol. 16, no. 3, pp. 487–500, 2022

    2022

  11. [19]

    Decentralized multi-client functional encryption for in- ner product with applications to federated learning,

    X. Qian, H. Li, M. Hao, G. Xu, H. Wang, and Y . Fang, “Decentralized multi-client functional encryption for in- ner product with applications to federated learning,” IEEE Trans. Depend. Sec. Comput., vol. 21, no. 6, pp. 5781–5796, 2024

    2024

  12. [20]

    Securing decentral- ized federated learning: cryptographic mechanisms for privacy and trust,

    A. Saidi, A. Amira, and O. Nouali, “Securing decentral- ized federated learning: cryptographic mechanisms for privacy and trust,”Cluster Comput., vol. 28, no. 2, p. 144, 2025

    2025

  13. [21]

    Decentralized nonconvex opti- mization with guaranteed privacy and accuracy,

    Y . Wang and T. Ba¸ sar, “Decentralized nonconvex opti- mization with guaranteed privacy and accuracy,”Auto- matica, vol. 150, p. 110858, 2023

    2023

  14. [22]

    Differentially private linearized admm algorithm for decentralized nonconvex optimization,

    X.-Y . Yue, J.-W. Xiao, X.-K. Liu, and Y .-W. Wang, “Differentially private linearized admm algorithm for decentralized nonconvex optimization,”IEEE Trans. Inf. Forensics Security, 2025

    2025

  15. [23]

    Dp-norm: Differential privacy primal-dual algorithm for decen- tralized federated learning,

    T. Fukami, T. Murata, K. Niwa, and I. Tyou, “Dp-norm: Differential privacy primal-dual algorithm for decen- tralized federated learning,”IEEE Trans. Inf. Forensics Security, vol. 19, pp. 5783–5797, 2024

    2024

  16. [24]

    Federated learning via inexact admm,

    S. Zhou and G. Y . Li, “Federated learning via inexact admm,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 8, pp. 9699–9708, 2023

    2023

  17. [25]

    Decentralized federated averaging,

    T. Sun, D. Li, and B. Wang, “Decentralized federated averaging,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 4, pp. 4289–4301, 2022

    2022

  18. [26]

    Decentralized feder- ated learning: Balancing communication and computing costs,

    W. Liu, L. Chen, and W. Zhang, “Decentralized feder- ated learning: Balancing communication and computing costs,”IEEE Trans. Signal Inf. Process. Netw., vol. 8, pp. 131–143, 2022

    2022

  19. [27]

    De- centralized deep learning with arbitrary communication compression,

    A. Koloskova, T. Lin, S. U. Stich, and M. Jaggi, “De- centralized deep learning with arbitrary communication compression,” inInt. Conf. Learn. Represent., 2020

    2020

  20. [28]

    Squarm- sgd: Communication-efficient momentum sgd for decen- tralized optimization,

    N. Singh, D. Data, J. George, and S. Diggavi, “Squarm- sgd: Communication-efficient momentum sgd for decen- tralized optimization,”IEEE J. Sel. Areas Inf. Theory, vol. 2, no. 3, pp. 954–969, 2021

    2021

  21. [29]

    Beer: Fasto(1/t)rate for decentralized nonconvex optimiza- tion with communication compression,

    H. Zhao, B. Li, Z. Li, P. Richtárik, and Y . Chi, “Beer: Fasto(1/t)rate for decentralized nonconvex optimiza- tion with communication compression,”Adv. Neural Inf. Process. Syst., vol. 35, pp. 31 653–31 667, 2022

    2022

  22. [30]

    Decentralized sum-of- nonconvex optimization,

    Z. Liu and B. K. H. Low, “Decentralized sum-of- nonconvex optimization,” inProc. AAAI Conf. Artif. Intell., vol. 38, no. 13, 2024, pp. 14 088–14 096

    2024

  23. [31]

    A decentralized proximal- gradient method with network independent step-sizes and separated convergence rates,

    Z. Li, W. Shi, and M. Yan, “A decentralized proximal- gradient method with network independent step-sizes and separated convergence rates,”IEEE Trans. Signal Process., vol. 67, no. 17, pp. 4494–4506, 2019

    2019

  24. [32]

    Finite-bit quantization for distributed algorithms with linear con- vergence,

    N. Michelusi, G. Scutari, and C.-S. Lee, “Finite-bit quantization for distributed algorithms with linear con- vergence,”IEEE Trans. Inf. Theory, vol. 68, no. 11, pp. 7254–7280, 2022

    2022

  25. [33]

    Private and communication-efficient edge learning: A sparse differ- ential gaussian-masking distributed sgd approach,

    X. Zhang, M. Fang, J. Liu, and Z. Zhu, “Private and communication-efficient edge learning: A sparse differ- ential gaussian-masking distributed sgd approach,” in Int. Symp. Theory, Algorithmic Found., Protoc. Design Mobile Netw. Mobile Comput., 2020, pp. 261–270. JOURNAL OF LA...

    2020

  26. [34]

    Federated learning with sparsified model perturbation: Improving accuracy under client-level differential privacy,

    R. Hu, Y . Guo, and Y . Gong, “Federated learning with sparsified model perturbation: Improving accuracy under client-level differential privacy,”IEEE Trans. Mobile Comput., vol. 23, no. 8, pp. 8242–8255, 2023

    2023

  27. [35]

    Secure aggregation meets spar- sification in decentralized learning,

    S. Biswas, A.-M. Kermarrec, R. Pires, R. Sharma, and M. Vujasinovic, “Secure aggregation meets spar- sification in decentralized learning,”arXiv preprint arXiv:2405.07708, 2024

    2024 arXiv

  28. [36]

    Numerical optimization,

    J. Nocedal, “Numerical optimization,”Springer Ser. Oper. Res. Financ. Eng./Springer, 2006

    2006

  29. [37]

    Asynchronous stochastic gradient descent with delay compensation for distributed deep learning,

    S. Zheng, Q. Meng, T. Wang, W. Chen, N. Yu, Z. Ma, and T.-Y . Liu, “Asynchronous stochastic gradient descent with delay compensation for distributed deep learning,” arXiv preprint arXiv:1609.08326, 2016

    2016 arXiv

  30. [38]

    Parallel restarted sgd with faster convergence and less communication: Demystify- ing why model averaging works for deep learning,

    H. Yu, S. Yang, and S. Zhu, “Parallel restarted sgd with faster convergence and less communication: Demystify- ing why model averaging works for deep learning,” in Proc. AAAI Conf. Artif. Intell., vol. 33, no. 1, 2019, pp. 5693–5700

    2019

  31. [39]

    Cooperative sgd: A unified framework for the design and analysis of local-update sgd algorithms,

    J. Wang and G. Joshi, “Cooperative sgd: A unified framework for the design and analysis of local-update sgd algorithms,”J. Mach. Learn. Res., vol. 22, no. 213, pp. 1–50, 2021

    2021

  32. [40]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inArtif. Intell. Stat., 2017, pp. 1273–1282

    2017

  33. [41]

    On the convergence of fedavg on non-iid data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,”arXiv preprint arXiv:1907.02189, 2019

    1907 arXiv

  34. [42]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,”Proc. Mach. Learn. Syst., vol. 2, pp. 429–450, 2020

    2020

  35. [43]

    Fedgia: An efficient hybrid algorithm for federated learning,

    S. Zhou and G. Y . Li, “Fedgia: An efficient hybrid algorithm for federated learning,”IEEE Trans. Signal Process., vol. 71, pp. 1493–1508, 2023

    2023

  36. [44]

    The sparse vector technique, revisited,

    H. Kaplan, Y . Mansour, and U. Stemmer, “The sparse vector technique, revisited,” inConf. Learn. Theory (COLT), 2021, pp. 2747–2776

    2021

  37. [45]

    Improving sparse vector tech- nique with renyi differential privacy,

    Y . Zhu and Y .-X. Wang, “Improving sparse vector tech- nique with renyi differential privacy,”Adv. Neural Inf. Process. Syst., vol. 33, pp. 20 249–20 258, 2020

    2020

  38. [46]

    Secure federated averaging algorithm with differential privacy,

    Y . Li, T.-H. Chang, and C.-Y . Chi, “Secure federated averaging algorithm with differential privacy,” inIEEE Int. Workshop Mach. Learn. Signal Process.IEEE, 2020, pp. 1–6

    2020

  39. [47]

    Federated stochastic primal-dual learning with differential privacy,

    Y . Li, S. Wang, T.-H. Chang, and C.-Y . Chi, “Federated stochastic primal-dual learning with differential privacy,” arXiv preprint arXiv:2204.12284, 2022

    2022 arXiv

  40. [48]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,

    H. Xiao, K. Rasul, and R. V ollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,”arXiv preprint arXiv:1708.07747, 2017

    2017 arXiv

  41. [49]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778

    2016

  42. [50]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky, G. Hintonet al., “Learning multiple layers of features from tiny images,” 2009. Shan Shareceived the B.S. degree from the School of Mathematics and Statistics, Beijing Jiaotong Uni- versity, Beijing, China, in 2019, where she is cur- rently working toward the Ph...

    2009

  43. [51]

    This is adopted for analytical convenience without loss of generality

    SetW 0 =0andκ i =k 0 for alli∈[m]. This is adopted for analytical convenience without loss of generality. In fact, one can always letk 0 be the least common multiple of{κ 1, κ2,· · ·, κm}, then the subsequent analysis remains similar to the case ofκ i =k 0

  44. [52]

    That is, at every iteration, nodeiselects the same number of neighbor nodes to join in the training

    Setm k i =t i =⌊ν i|Ni|⌋for allk≥0andi∈[m], whereν i ∈(0,1]is the participation rate and⌊a⌋is the floor ofa. That is, at every iteration, nodeiselects the same number of neighbor nodes to join in the training

  45. [53]

    Again, this is adopted for analytical convenience without loss of generality

    Setσ 0 i =σ 0 andγ i =γ,s i =sfor alli∈[m]. Again, this is adopted for analytical convenience without loss of generality. In fact, for differentσ 0 i andγ i, we can conduct similar analysis by considering σ0 = min{σ0 1,· · ·, σ0 m}, γ= min{γ 1,· · ·, γm}, s= min{s 1,· · ·, sm}...

    2021

  46. [54]

    wk iℓ − 1 r X j∈N k i,ℓ wk jℓ !2 λk i,ℓ =r # +P(λ k i,ℓ = 0)E h (wk iℓ −w k iℓ)2 λk i,ℓ = 0 i = tiX r=1 P(λk i,ℓ =r)E

    In the sequel, givenζ∈(0,1)defined in (6) and integersk 0 and(t 1,· · ·, tm), initialize p:= s n ∈(0,1), γ∈ 1, ζ−2/k0 , ν i ∈(0,1],∀i∈[m],(10) to satisfy (1−p) ti (1 +ζ) 2 + 2p X j∈Ni νj < 1 γk0/2 −ζ 2 ,∀i∈[m].(11) Moreover, define a useful constant by εi(t) := sup n ∥∇fi (w;B...

    2021

  47. [55]

    For anyk≥0andi∈[m],w k i ∈N(2δ)and vk i ∈N(2δ)

  48. [56]

    Proof.Since 1) has been shown in Lemma 3, we only prove 2)

    For anyk≥0, eH k − eH k+1 ≥ mX i=1 σk i ti 8 ∥∆wk+1 i ∥2 +∥∆ vk+1 i ∥2 ,(75) where eH k is defined by (72) and the involved constants are defined by (73). Proof.Since 1) has been shown in Lemma 3, we only prove 2). Byη=β/γ >1in (73) andγ∈(1, ζ −1/k0 )in (10), 1 γk = γ γ−1 1 γk...

    2021

  49. [57]

    Sequence{w k}converges (tow ∞) inL 2 space and sequence{Ew k}converges, i.e., lim k→∞ E∥wk −w ∞∥2 = 0,lim k→∞ Ewk =Ew ∞.(88)

  50. [58]

    The nonlinear expectation converges to the function evaluated at the limit: lim k→∞ E|f(w k)−f(w ∞)|= 0

  51. [59]

    Proof.1) By the triangle inequality and Cauchy-Schwarz: ∥Ewk+1 −Ew k∥ ≤E∥wk+1 −w k∥ ≤ q E∥wk+1 −w k∥2 ≤ c γk/2

    The sequence satisfies E∥wk −w ∞∥2 =O(γ −k),E|f(w k)−f(w ∞)|=O(γ −k/2). Proof.1) By the triangle inequality and Cauchy-Schwarz: ∥Ewk+1 −Ew k∥ ≤E∥wk+1 −w k∥ ≤ q E∥wk+1 −w k∥2 ≤ c γk/2 . Since P∞ k=0 γ−k/2 <∞, the sequence of means{Ew k}has summable increments and is thus Cauchy...

  52. [60]

    This implies that there is a constantc >0such that |f(w k)−f(w ∞)| ≤c∥wk −w ∞∥

    It is noted that{w k}is bounded inN(2δ)andfis continuously differentiable, sofis Lipschitz continuous on a bounded region containingN(2δ)andw ∞. This implies that there is a constantc >0such that |f(w k)−f(w ∞)| ≤c∥wk −w ∞∥. Taking the expectation leads to lim k→∞ E|f(w k)−f(w...

  53. [61]

    We first showlim N→∞ ∆N = 0

    LetN > kand consider the difference, ∆N := E∥wN −w k∥2 −E∥w ∞ −w k∥2 . We first showlim N→∞ ∆N = 0. By linearity of expectation and the triangle inequality, ∆N ≤E ∥wN −w k∥2 − ∥w∞ −w k∥2 ≤E ∥wN −w k∥ − ∥w∞ −w k∥ (∥wN −w k∥+∥w ∞ −w k∥) ≤E ∥wN −w ∞∥(∥wN −w k∥+∥w ∞ −w k∥) . Takin...

    2021

  54. [62]

    For anyi∈[m], lim k→∞ E ∆wk i = lim k→∞ E ∆vk i = lim k→∞ E∥wk i − vk i ∥= 0.(93)

  55. [63]

    Sequence{ϖ k}converges toϖ ∞ in the sense ofL 2 convergence and expectation, namely, lim k→∞ E∥ϖk −ϖ ∞∥2 = 0,lim k→∞ Eϖk =Eϖ ∞.(94) Moreover, sequence{(EW k,EV k)}converges and satisfies lim k→∞ EWk = lim k→∞ EVk = (Eϖ∞,· · ·,Eϖ∞) =:W ∞

  56. [64]

    Proof.1) From (75) and (74), sequence{ eH k}is non-increasing and bounded from below

    Three sequences{H k},{ eH k}, and{Ef(ϖ k)}converge to the same value, namely, lim k→∞ eH k = lim k→∞ H k = lim k→∞ Ef(ϖ k) =Ef(ϖ ∞). Proof.1) From (75) and (74), sequence{ eH k}is non-increasing and bounded from below. Therefore, it converges. Taking the limit of the both side...

    2021

  57. [65]

    It is easy to checkϖ k ∈N(2δ) for allk≥0

    By (95), we conclude that E∥∆wk+1 i ∥2 =o(σ k) =o γ−k ,∀i∈[m], which indicates that there exists a constantc i >0such that E∥∆wk+1 i ∥2 ≤ c2 i γk ,∀i∈[m].(97) Recall (25) thatϖ k = 1 m Pm i=1 wk i and Lemma 3 thatw k i ∈N(2δ)for alli∈[m]andk≥0. It is easy to checkϖ k ∈N(2δ) fo...

  58. [66]

    The convergence of sequence{ eH k}and (72) can lead to the convergence of sequence{H k}and lim k→∞ eH k = lim k→∞ H k (72) = lim k→∞ E mX i=1 fi(wk i ) + σkti 2 wk i − vk i 2 (77) = lim k→∞ E mX i=1 fi(wk i ). (100) From 2), we have shown that{ϖ k}converges (toϖ ∞) in the sens...

    2021

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.