Pith. sign in

REVIEW 3 major objections 3 minor 54 references

Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneity

T0 review · 3 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves that for decentralized SGD with heterogeneous data, generalization error decays like 1 over the total sample size, while Byzantine attacks add a non-vanishing error floor set by the aggregation rule, gradient noise, and…

desk verdict The paper identifies a real gap and extends stability analysis to heterogeneous decentralized learning, but the proof of the central vanishing-rate bound misapplies the stated stability lemma, and the gap looks load-bearing. read the letter →

arxiv 2506.09438 v1 pith:UTQJOUNI submitted 2025-06-11 cs.LG cs.DC

Haoxiang Ye , Tao Sun , Qing Ling This is my paper
classification cs.LGcs.DC
keywords decentralizedlearninggeneralizationerrordataheterogeneityByzantineattackson-averagestabilitySGDrobustaggregationstrongconvexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish how data heterogeneity limits the generalization of decentralized stochastic gradient descent, and what Byzantine agents add on top. For attack-free DSGD with strongly convex losses, it proves that after $k \approx \mu N Z$ iterations the generalization error is bounded by $O(\|\bar{x}^0 - x^*_S\|^2/(\mu N Z)) + \widetilde{O}(\sigma^2/(\mu N Z)) + O(\delta^2/(\mu N Z))$: initialization, stochastic-gradient noise, and heterogeneity enter at the same inverse-sample-size rate. For Byzantine-resilient DSGD the bound gains an extra term $O((4\rho^2 N + \chi^2)(\sigma^2 + \delta^2))$ that does not shrink as the number of samples grows, so malicious agents create an irreducible generalization gap unless the robust aggregation rule is perfect. The work matters because prior decentralized generalization bounds assumed either identical local data distributions or bounded stochastic gradients, which hid exactly the heterogeneity effects this paper isolates.

What carries the argument

The load-bearing tool is on-average stability rather than uniform stability: Definition 2 measures the average sensitivity of the algorithm's output to replacing one training sample, and Lemma 1 (adapted from [31]) converts a bound on that sensitivity, together with a bound on the in-sample function value, into a generalization-error bound. The proof then controls the on-average stability through a contraction argument on the perturbed iterates: the decaying step size $\alpha_k = 1/(\mu(k+k_0))$ (or $2/(\mu(k+k_1)))$ makes the update contractive with factor $(1-\alpha_k \mu)$, while consensus-error lemmas (Lemma 2 and Lemma 3) bound the spread of local models by $O((\sigma^2+\delta^2)/(\mu^2 k^2))$. For the Byzantine case, Definition 1's contraction constant $\rho$ and the non-doubly-stochastic deviation $\chi^2$ are what inject the sample-size-independent term, because they bound the per-step approximation error of robust aggregation as a weighted average.

What would settle it

Carry equation (4) through Appendix C with $v=L$ exactly as printed: the right-hand side retains the additive term $L$, which is independent of $NZ$; a line-by-line check that this term cancels against the optimization-error bound, or a restated Lemma 1 without the additive constant, would settle whether the $O(1/(\mu N Z))$ rate is actually established.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is a pair of generalization-error decompositions. Theorem 3 shows that for attack-free DSGD with $\mu$-strongly convex, $L$-smooth losses, working with decaying step sizes and running $k \asymp \mu N Z$ iterations makes the expected generalization error at most $O(\|\bar{x}^0 - x^*_S\|^2/(\mu N Z)) + \widetilde{O}(\sigma^2/(\mu N Z)) + O(\delta^2/(\mu N Z))$, so the error vanishes as the total number of training samples $NZ$ grows. Theorem 6 shows the same three terms for Byzantine-resilient DSGD, plus $O((4\rho^2 N + \chi^2)(\sigma^2 + \delta^2))$, where $\rho$ is the contraction constant of the robust aggregation rule and $\chi^2$ measures how far its virtual mixing matrix is from being doubly stochastic; this extra term is independent of sample size and disappears only for an ideal aggregation rule with $\rho = 0$ and $\chi = 0$. A companion result (Theorem 7) bounds the generalization error of training on a single agent's local data alone and uses it to identify when cooperation with non-Byzantine neighbors is worth the Byzantine risk.

Load-bearing premise

The vanishing-rate claims depend on treating Lemma 1 as if its additive constant $v$ can be dropped after setting $v=L$; the lemma as printed keeps that constant, so the proof needs a strengthened version of Lemma 1 that the paper does not state.

Editorial extensions

If this is right

  • For attack-free DSGD, adding more samples and more agents is enough to drive the generalization error to zero at the $O(1/(\mu N Z))$ rate; data heterogeneity changes only the constant, not the rate.
  • Initialization matters for generalization: the term $\|\bar{x}^0 - x^*_S\|^2/(\mu N Z)$ says a better starting model shrinks the gap at the same rate as adding samples.
  • Under Byzantine attacks, no amount of extra data eliminates the generalization gap; the floor $O((4\rho^2 N + \chi^2)(\sigma^2+\delta^2))$ is set by the aggregation rule, the gradient noise, and the heterogeneity.
  • If the robust aggregation rule is perfect, with $\rho=0$ and $\chi=0$, the Byzantine bound collapses to the attack-free bound.
  • Cooperation among non-Byzantine agents is beneficial when local data are heterogeneous and local sample sizes are small; when local distributions are identical, independent training avoids the Byzantine floor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same decomposition suggests a quantitative robustness-generalization tradeoff: lowering $\rho$ or $\chi$, even a little, buys a proportional reduction in the generalization floor, so aggregation rules designed for optimization error are likely also improving the asymptotic generalization gap.
  • Because the heterogeneity term $\delta^2$ enters at the same $1/(NZ)$ rate as the noise term, an experiment comparing two datasets with matched noise but different $\delta^2$ should show the generalization gap differing proportionally to $\delta^2$; the paper's figures show that direction but do not fit the predicted slope.
  • The framework should extend to asynchronous or local-update decentralized variants; the conjecture would be that delays and partial participation only rescale the constants in front of $\sigma^2$ and $\delta^2$, while the Byzantine floor remains sample-size-independent.
  • The appearance of $\delta^2$ in the Byzantine floor implies that data preprocessing that reduces effective heterogeneity, such as rebalancing or clustered sampling, could shrink the Byzantine generalization gap without changing $\rho$, a testable prediction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This paper studies the generalization error of decentralized stochastic gradient descent (DSGD) under heterogeneous local data, both in the attack-free setting and in the presence of Byzantine agents. The authors use on-average stability (Lemma 1, attributed to [31]) together with optimization-error estimates to derive generalization bounds. Under µ-strong convexity, L-smoothness, bounded stochastic gradient noise, and bounded heterogeneity, Theorem 3 claims an O(1/(NZ)) vanishing generalization rate for attack-free DSGD at time k ~ µNZ, and Theorem 6 claims the same rate plus a non-vanishing Byzantine term O((4ρ²N+χ²)(σ²+δ²)). The paper also states Theorem 7 on the benefit of cooperation and reports experiments on MNIST and CIFAR-10 supporting the qualitative conclusions.

Significance. If the stated bounds were correctly established, the paper would address a real gap: prior decentralized generalization analyses typically assume bounded stochastic gradients or homogeneous local distributions, while this paper aims to expose the dependence on initialization, gradient noise, heterogeneity, and Byzantine contamination. The Byzantine-resilient analysis, in particular, would quantify a sample-size-independent penalty caused by malicious agents. The paper is organized and clearly written, with explicit assumptions, substantial appendix proofs for the stability and optimization results, and experiments on both convex and non-convex tasks. However, the central step from the stability/optimization estimates to the advertised generalization rates is invalid as written: the proof of Theorem 3 (Appendix C) applies Lemma 1 with v=L and suppresses the resulting additive constant, and some displayed rates do not match the intermediate bounds. The main quantitative claims are therefore not currently supported.

major comments (3)
  1. [Appendix C, Eq. (35); Lemma 1, Eq. (4)] The proof of Theorem 3 sets v=L in Lemma 1 and then substitutes (29) and (34) into (4) to obtain (35). With v=L, the first term on the right-hand side of (4) is the positive constant L, which does not vanish as k, N, and Z grow. The printed lemma therefore yields an O(L) additive term, and the displayed O(1/(NZ)) rates in Theorem 3 do not follow. Moreover, an optimal choice of v in (4) using the paper's own bounds E[F_S(bar x_k)] = O(1/(NZ)) and on-average stability = O(1/(NZ)) gives v approximately sqrt(L E[F_S(bar x_k)]) and produces at best an O(1/sqrt(NZ)) generalization bound, so the stronger O(1/(NZ)) rate cannot be obtained from Lemma 1 as printed. The identical move is used in Appendix F for Theorem 6 (Eq. (66)), so both central generalization theorems inherit this gap.
  2. [Appendix C, Eq. (34); Theorem 2, Eq. (6); Theorem 3, Eq. (7)] The rates advertised in Theorem 3 are not supported by the displayed intermediate bounds. In particular, the σ² term in Theorem 2, Eq. (6), is Lσ² ln(k+k0-1)/(2µ²N(k+k0-1)); at k ~ µNZ this is O~(σ²/(µ³N²Z)), not O~(σ²/(µNZ)) as stated in Theorem 3. Similarly, the third term of Eq. (6), C1(σ²+δ²)/(µ³(k+k0-1)), becomes O((σ²+δ²)/(µ⁴NZ)) at k ~ µNZ, not O(δ²/(µNZ)) as stated. The claimed stability bound (34) also needs separate verification: substituting (33) into (20) introduces additional factors of µ, N, and Z that are not accounted for in the expression O((σ²+δ²)/(µNZ)). These discrepancies propagate to Theorem 6 and to the discussion in Sections V-C and VI-C.
  3. [Section VI-D, Theorem 7, Eq. (13)] Theorem 7 is used to justify the cooperation-gain conclusions, but no proof is provided anywhere in the appendices. The statement is nontrivial because it combines an SGD generalization bound with a domain-adaptation term Φ(D_n,D). In addition, the condition k ~ µNZ appears inconsistent with a single-agent run on the local dataset S_n, which has only Z samples; the displayed O(1/Z) rates in (13) correspond to the time horizon k ~ µZ, not k ~ µNZ. The authors should either supply a complete proof and correct the time-horizon statement, or clearly reference a published theorem that implies (13).
minor comments (3)
  1. [Section VI-C] The title of Theorem 6 reads "Generalization Error of DSGD with Heterogeneous Data" but should read "Generalization Error of Byzantine-Resilient DSGD with Heterogeneous Data" to distinguish it from Theorem 3.
  2. [Section IV-B, Lemma 1] Given the critical role of the additive v term in Lemma 1, the authors should verify that the stated lemma exactly matches the version in [31] and should state the lemma with the same variable names used in the proof; this would help avoid the confusion in Appendix C.
  3. [Section VII] In the numerical experiments, the generalization error is approximated by the difference between training and test losses, but the paper does not report the optimization-error component separately; reporting both components would make the experimental validation of Theorems 3 and 6 more direct.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the generalization bounds are derived from independent stability, consensus, and optimization results; the flagged v=L issue in Appendix C is a proof gap, not a circular reduction.

full rationale

The paper's central claims (Theorems 3 and 6) are generalization bounds assembled from three externally grounded ingredients: (i) Lemma 1 from [31], an on-average stability-to-generalization transfer statement; (ii) consensus lemmas (Lemma 2 and Lemma 3) proved in Appendix G, building on [21] and [37]; and (iii) optimization error bounds (Theorems 2 and 5) proved in Appendices B and E, with Theorem 5 attributed to [21] and [24]. No parameter is fitted to data and no claim is redefined in terms of its own output. The robust-aggregation framework in Definition 1 is an assumption imported from the authors' prior work [21], and [24] is cited for the fact that TM/IOS/SCC satisfy it; these are published, checkable external results rather than an unverified uniqueness assertion, so they do not make the central derivation circular. Numerical experiments are validations of, not inputs to, the bounds. I therefore find no significant circularity. Separately, I note a proof gap that is not circular: in Appendix C (and similarly Appendix F) Lemma 1 is applied with v=L, and the additive v term from (4) is omitted when passing to (35) and (66). As printed, the claimed O(1/(NZ)) rate does not follow from Lemma 1 with v=L. That is a correctness concern, not a circular reduction.

Assumptions & free parameters 0 free parameters · 11 assumptions · 0 invented entities

No free parameters are fitted to data; σ², δ², µ, L, λ, ρ, χ are assumed inputs, and the step-size offsets k0 and k1 are schedule constants, not fitted values. The axioms are the standard convexity and smoothness assumptions, the bounded-noise and bounded-heterogeneity assumptions, a zero-optimal-loss normalization, the common-initialization modeling choice, and several imported lemmas from prior work that the proofs rely on.

assumptions (11)
  • domain assumption Assumption 1: f(x;ξ) is µ-strongly convex for every ξ
    Gives the contraction factor (1−α_k µ) used throughout the stability and optimization proofs; stated in Section IV-A.
  • domain assumption Assumption 2: f(x;ξ) is L-smooth for every ξ
    Used to bound gradient differences by model distances and to derive the optimization and stability bounds; stated in Section IV-A.
  • domain assumption Assumption 3: bounded stochastic gradient noise with variance σ²
    Replaces the bounded-gradient assumption; used to control the variance of stochastic gradients across all agents; stated in Section IV-A.
  • domain assumption Assumption 4: bounded data heterogeneity with constant δ²
    Quantifies the discrepancy between local empirical gradients and the global empirical gradient; central to the heterogeneity dependence; stated in Section IV-A.
  • domain assumption F(x*) = 0 normalization
    Assumed in Section II.B to simplify the excess-loss decomposition; implies the optimal model fits the underlying distribution.
  • domain assumption All agents share the same initialization x0_n = x0
    Used in Algorithm 1 and Algorithm 2 and required by the consensus lemmas in Appendix G.
  • standard math Lemma 1 from [31]: on-average stability implies a generalization error bound with an additive constant v
    Imported from Lei and Ying; the proof of Theorem 3 applies it with v=L and omits the additive term, so the correctness of the central rate depends on this lemma.
  • standard math Definition 1 from [21]: virtual mixing matrix and contraction constant ρ
    Characterizes robust aggregation rules; the Byzantine bounds depend on ρ and χ from this definition.
  • standard math Lemma 6 from [32]: one-step contraction of strongly convex smooth SGD
    Used in every stability proof to bound the distance between runs on perturbed datasets.
  • standard math Consensus error lemmas from [21] and [37]
    Bounded in Appendix G and then used to control the gradient discrepancy terms in the final bounds.
  • domain assumption Non-Byzantine agents form a connected graph in the Byzantine setup
    Needed for the non-Byzantine agents to reach consensus; stated in Section II.A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneity." pith.science (2026). https://pith.science/paper/UTQJOUNI

@misc{pith2026250609438,
  author       = {Pith},
  title        = {Pith review of: Generalization Error Analysis for Attack-Free and Byzantine-Resilient Decentralized Learning with Data Heterogeneity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UTQJOUNI}},
  note         = {Machine review of arXiv:2506.09438}
}
read the original abstract

Decentralized learning, which facilitates joint model training across geographically scattered agents, has gained significant attention in the field of signal and information processing in recent years. While the optimization errors of decentralized learning algorithms have been extensively studied, their generalization errors remain relatively under-explored. As the generalization errors reflect the scalability of trained models on unseen data and are crucial in determining the performance of trained models in real-world applications, understanding the generalization errors of decentralized learning is of paramount importance. In this paper, we present fine-grained generalization error analysis for both attack-free and Byzantine-resilient decentralized learning with heterogeneous data as well as under mild assumptions, in contrast to prior studies that consider homogeneous data and/or rely on a stringent bounded stochastic gradient assumption. Our results shed light on the impact of data heterogeneity, model initialization and stochastic gradient noise -- factors that have not been closely investigated before -- on the generalization error of decentralized learning. We also reveal that Byzantine attacks performed by malicious agents largely affect the generalization error, and their negative impact is inherently linked to the data heterogeneity while remaining independent on the sample size. Numerical experiments on both convex and non-convex tasks are conducted to validate our theoretical findings.

Figures

Figures reproduced from arXiv: 2506.09438 by the authors.

Figure 2
Figure 2. Generalization error of DSGD across different training sample sizes per agent when β = 0.01. replicate its messages when communicating. This behavior is equivalent to the Byzantine agents duplicating the training samples of the selected non-Byzantine agent. A-Little-Is-Enough (ALIE) Attack [53]. To non-Byzantine agent n, its Byzantine neighbors send 1 |Nn| P m∈Nn x k+ 1 2 m,m + r k n∆k n at time k, in which ∆k n is … view at source ↗
Figure 3
Figure 3. Generalization error of Byzantine-resilient DSGD under different levels of data heterogeneity with various aggregation rules. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Generalization error of attack-free DSGD and Byzantine-resilient DSGD when [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Generalization error of SGD without cooperation and Byzantine-resilient DSGD when [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Generalization error of DSGD under different levels of data hetero￾geneity on CIFAR-10 [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 8
Figure 8. Figure 8: Generalization error of Byzantine-resilient DSGD equipped with IOS under different levels of data heterogeneity on CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Generalization error of attack-free DSGD and Byzantine-resilient DSGD when [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: Generalization error of SGD without cooperation and Byzantine-resilient DSGD when [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 41 canonical work pages

  1. [31]

    Fine-grained analysis of stability and generalization for stochastic gradient descent,

    Y . Lei and Y . Ying, “Fine-grained analysis of stability and generalization for stochastic gradient descent,”International Conference on Machine Learning, 2020

    work page 2020

  2. [1]

    Generalization guarantee of decentralized learning with heterogeneous data,

    H. Ye, T. Sun, and Q. Ling, “Generalization guarantee of decentralized learning with heterogeneous data,”International Conference on Acous- tics, Speech and Signal Processing, 2025

    work page 2025

  3. [2]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,”International Conference on Artificial Intelligence and Statistics, 2017

    work page 2017

  4. [3]

    Accelerating hybrid federated learning convergence under partial participation,

    J. Bian, L. Wang, K. Yang, C. Shen, and J. Xu, “Accelerating hybrid federated learning convergence under partial participation,”IEEE Trans- actions on Signal Processing, vol. 72, pp. 3258–3271, 2024

    work page 2024

  5. [4]

    Fedpd: A federated learning framework with adaptivity to non-iid data,

    X. Zhang, M. Hong, S. Dhople, W. Yin, and Y . Liu, “Fedpd: A federated learning framework with adaptivity to non-iid data,”IEEE Transactions on Signal Processing, vol. 69, pp. 6055–6070, 2021

    work page 2021

  6. [5]

    Byzantine-robust and communication-efficient personalized federated learning,

    J. Zhang, X. He, Y . Huang, and Q. Ling, “Byzantine-robust and communication-efficient personalized federated learning,”IEEE Trans- actions on Signal Processing, vol. 73, pp. 26–39, 2025

    work page 2025

  7. [6]

    Locally differentially pri- vate online federated learning with correlated noise,

    J. Zhang, L. Zhu, D. Fay, and M. Johansson, “Locally differentially pri- vate online federated learning with correlated noise,”IEEE Transactions on Signal Processing, 2025

    work page 2025

  8. [7]

    Optimal complexity in Byzantine-robust distributed stochastic optimization with data hetero- geneity,

    Q. Shi, J. Peng, K. Yuan, X. Wang, and Q. Ling, “Optimal complexity in Byzantine-robust distributed stochastic optimization with data hetero- geneity,”arXiv preprint arXiv:2503.16337, 2025

    arXiv 2025

Show all 54 references
  1. [8]

    Can fairness and robustness be simultaneously achieved under Byzantine attacks?

    H. Zheng, R. Wang, X. Wang, and Q. Ling, “Can fairness and robustness be simultaneously achieved under Byzantine attacks?”IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing, 2025

    2025

  2. [9]

    Distributed subgradient methods for multi- agent optimization,

    A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi- agent optimization,”IEEE Transactions on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009

    2009

  3. [10]

    Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,

    X. Lian, C. Zhang, H. Zhang, C.-J. Hsieh, W. Zhang, and J. Liu, “Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent,”Advances in Neural Information Processing Systems, 2017

    2017

  4. [11]

    From centralized to decentralized federated learning: Theoretical insights, privacy preservation, and ro- bustness challenges,

    Q. Li, W. Yu, Y . Xia, and J. Pang, “From centralized to decentralized federated learning: Theoretical insights, privacy preservation, and ro- bustness challenges,”arXiv preprint arXiv:2503.07505, 2025

    2025 arXiv

  5. [12]

    The effectiveness of local updates for decentralized learning under data heterogeneity,

    T. Wu, Z. Li, and Y . Sun, “The effectiveness of local updates for decentralized learning under data heterogeneity,”IEEE Transactions on Signal Processing, vol. 73, pp. 751–765, 2025

    2025

  6. [13]

    On the convergence of decentralized stochastic gradient descent with biased gradients,

    Y . Jiang, H. Kang, J. Liu, and D. Xu, “On the convergence of decentralized stochastic gradient descent with biased gradients,”IEEE Transactions on Signal Processing, vol. 73, pp. 549–558, 2025

    2025

  7. [14]

    Removing data heterogene- ity influence enhances network topology dependence of decentralized SGD,

    K. Yuan, S. A. Alghunaim, and X. Huang, “Removing data heterogene- ity influence enhances network topology dependence of decentralized SGD,”The Journal of Machine Learning Research, vol. 24, no. 280, pp. 1–53, 2023

    2023

  8. [15]

    A unified theory of decentralized SGD with changing topology and local updates,

    A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, and S. Stich, “A unified theory of decentralized SGD with changing topology and local updates,” International Conference on Machine Learning, 2020

    2020

  9. [16]

    An improved convergence analysis for decentralized online stochastic non-convex optimization,

    R. Xin, U. A. Khan, and S. Kar, “An improved convergence analysis for decentralized online stochastic non-convex optimization,”IEEE Transactions on Signal Processing, vol. 69, pp. 1842–1858, 2021

    2021

  10. [17]

    On the decentralized stochastic gradient descent with Markov chain sampling,

    T. Sun, D. Li, and B. Wang, “On the decentralized stochastic gradient descent with Markov chain sampling,”IEEE Transactions on Signal Processing, vol. 71, pp. 2895–2909, 2023

    2023

  11. [18]

    A unified and refined convergence analysis for non-convex decentralized learning,

    S. A. Alghunaim and K. Yuan, “A unified and refined convergence analysis for non-convex decentralized learning,”IEEE Transactions on Signal Processing, vol. 70, pp. 3264–3279, 2022

    2022

  12. [19]

    The Byzantine generals problem,

    L. Lamport, R. Shostak, and M. Pease, “The Byzantine generals problem,”ACM Transactions on Programming Languages and Systems, vol. 4, no. 3, pp. 382–401, 1982

    1982

  13. [20]

    Bridge: Byzantine-resilient decen- tralized gradient descent,

    C. Fang, Z. Yang, and W. U. Bajwa, “Bridge: Byzantine-resilient decen- tralized gradient descent,”IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 610–626, 2022

    2022

  14. [21]

    Byzantine-resilient decentralized stochas- tic optimization with robust aggregation rules,

    Z. Wu, T. Chen, and Q. Ling, “Byzantine-resilient decentralized stochas- tic optimization with robust aggregation rules,”IEEE Transactions on Signal Processing, vol. 71, pp. 3179–3195, 2023

    2023

  15. [22]

    Byzantine-robust decentralized learning via self-centered clipping,

    L. He, S. P. Karimireddy, and M. Jaggi, “Byzantine-robust decentralized learning via self-centered clipping,”arXiv preprint arXiv:2202.01545, 2022

    2022 arXiv

  16. [23]

    Byzantine-robust decentralized learning via remove-then-clip aggregation,

    C. Yang and J. Ghaderi, “Byzantine-robust decentralized learning via remove-then-clip aggregation,”Proceedings of the AAAI Conference on Artificial Intelligence, 2024

    2024

  17. [24]

    On the tradeoff between privacy preserva- tion and Byzantine-robustness in decentralized learning,

    H. Ye, H. Zhu, and Q. Ling, “On the tradeoff between privacy preserva- tion and Byzantine-robustness in decentralized learning,”International Conference on Acoustics, Speech and Signal Processing, 2024

    2024

  18. [25]

    Train faster, generalize better: Stability of stochastic gradient descent,

    M. Hardt, B. Recht, and Y . Singer, “Train faster, generalize better: Stability of stochastic gradient descent,”International Conference on Machine Learning, 2016

    2016

  19. [26]

    Stability and generalization,

    O. Bousquet and A. Elisseeff, “Stability and generalization,”The Journal of Machine Learning Research, vol. 2, pp. 499–526, 2002

    2002

  20. [27]

    Stability of SGD: Tightness analysis and improved bounds,

    Y . Zhang, W. Zhang, S. Bald, V . Pingali, C. Chen, and M. Goswami, “Stability of SGD: Tightness analysis and improved bounds,”Uncer- tainty in Artificial Intelligence, 2022

    2022

  21. [28]

    Three-way trade-off in multi-objective learning: Optimization, generalization and conflict- avoidance,

    L. Chen, H. Fernando, Y . Ying, and T. Chen, “Three-way trade-off in multi-objective learning: Optimization, generalization and conflict- avoidance,”Advances in Neural Information Processing Systems, 2023

    2023

  22. [29]

    Generalization of geometric graph neural networks with Lipschitz loss functions,

    Z. Wang, J. Cervi ˜no, and A. Ribeiro, “Generalization of geometric graph neural networks with Lipschitz loss functions,”IEEE Transactions on Signal Processing, vol. 73, pp. 1549–1561, 2025

    2025

  23. [30]

    Data-dependent stability of stochastic gradient descent,

    I. Kuzborskij and C. Lampert, “Data-dependent stability of stochastic gradient descent,”International Conference on Machine Learning, 2018

    2018

  24. [32]

    Stability and generalization of decen- tralized stochastic gradient descent,

    T. Sun, D. Li, and B. Wang, “Stability and generalization of decen- tralized stochastic gradient descent,”AAAI Conference on Artificial Intelligence, 2021

    2021

  25. [33]

    Stability-based generalization analysis of the asynchronous decentralized SGD,

    X. Deng, T. Sun, S. Li, and D. Li, “Stability-based generalization analysis of the asynchronous decentralized SGD,”AAAI Conference on Artificial Intelligence, 2023

    2023

  26. [34]

    Topology- aware generalization of decentralized SGD,

    T. Zhu, F. He, L. Zhang, Z. Niu, M. Song, and D. Tao, “Topology- aware generalization of decentralized SGD,”International Conference on Machine Learning, 2022

    2022

  27. [35]

    Improved stability and generalization guarantees of the decentralized SGD algorithm,

    B. L. Bars, A. Bellet, M. Tommasi, K. Scaman, and G. Neglia, “Improved stability and generalization guarantees of the decentralized SGD algorithm,”International Conference on Machine Learning, 2024

    2024

  28. [36]

    On the generalization error of Byzantine-resilient decentralized learning,

    H. Ye and Q. Ling, “On the generalization error of Byzantine-resilient decentralized learning,”International Conference on Acoustics, Speech and Signal Processing, 2024. 13

    2024

  29. [37]

    Generalization error matters in decentralized learning under Byzantine attacks,

    ——, “Generalization error matters in decentralized learning under Byzantine attacks,”IEEE Transactions on Signal Processing, vol. 73, pp. 843–857, 2025

    2025

  30. [38]

    The non-iid data quagmire of decentralized machine learning,

    K. Hsieh, A. Phanishayee, O. Mutlu, and P. Gibbons, “The non-iid data quagmire of decentralized machine learning,”International Conference on Machine Learning, 2020

    2020

  31. [39]

    Understanding generalization of federated learning via stability: Heterogeneity matters,

    Z. Sun, X. Niu, and E. Wei, “Understanding generalization of federated learning via stability: Heterogeneity matters,”International Conference on Artificial Intelligence and Statistics, 2024

    2024

  32. [40]

    The tradeoffs of large scale learning,

    L. Bottou and O. Bousquet, “The tradeoffs of large scale learning,” Advances in Neural Information Processing Systems, 2007

    2007

  33. [41]

    Stability and generalization of asynchronous sgd: Sharper bounds beyond lipschitz and smoothness,

    X. Deng, T. Sun, S. Li, D. Li, and X. Lu, “Stability and generalization of asynchronous sgd: Sharper bounds beyond lipschitz and smoothness,” Advances in Neural Information Processing Systems, 2024

    2024

  34. [42]

    On the geometric convergence of Byzantine-resilient distributed optimization algorithms,

    K. Kuwaranancharoen and S. Sundaram, “On the geometric convergence of Byzantine-resilient distributed optimization algorithms,”SIAM Jour- nal on Optimization, vol. 35, no. 1, pp. 210–239, 2025

    2025

  35. [43]

    Towards the robust small-signal stability region of power systems under perturbations such as uncertain and volatile wind generation,

    Y . Pan, F. Liu, L. Chen, J. Wang, F. Qiu, C. Shen, and S. Mei, “Towards the robust small-signal stability region of power systems under perturbations such as uncertain and volatile wind generation,”IEEE Transactions on Power Systems, vol. 33, no. 2, pp. 1790–1799, 2018

    2018

  36. [44]

    Robust state estimation for linear systems under distributional uncertainty,

    S. Wang, Z. Wu, and A. Lim, “Robust state estimation for linear systems under distributional uncertainty,”IEEE Transactions on Signal Processing, vol. 69, pp. 5963–5978, 2021

    2021

  37. [45]

    Sparse stable outlier-robust signal recovery under Gaussian noise,

    K. Suzuki and M. Yukawa, “Sparse stable outlier-robust signal recovery under Gaussian noise,”IEEE Transactions on Signal Processing, vol. 71, pp. 372–387, 2023

    2023

  38. [46]

    Robustly stable signal recovery in com- pressed sensing with structured matrix perturbation,

    Z. Yang, C. Zhang, and L. Xie, “Robustly stable signal recovery in com- pressed sensing with structured matrix perturbation,”IEEE Transactions on Signal Processing, vol. 60, no. 9, pp. 4658–4671, 2012

    2012

  39. [47]

    Stability and generalization for minibatch SGD and local SGD,

    Y . Lei, T. Sun, and M. Liu, “Stability and generalization for minibatch SGD and local SGD,”arXiv preprint arXiv:2310.01139, 2023

    2023

  40. [48]

    Learn- ing bounds for domain adaptation,

    J. Blitzer, K. Crammer, A. Kulesza, F. Pereira, and J. Wortman, “Learn- ing bounds for domain adaptation,”Advances in Neural Information Processing Systems, vol. 20, 2007

    2007

  41. [49]

    A novel domain adaptation theory with jensen–shannon divergence,

    C. Shui, Q. Chen, J. Wen, F. Zhou, C. Gagn ´e, and B. Wang, “A novel domain adaptation theory with jensen–shannon divergence,”Knowledge- Based Systems, vol. 257, p. 109808, 2022

    2022

  42. [50]

    Fine- tuning personalization in federated learning to mitigate adversarial clients,

    Y . Allouah, A. El Mrini, R. Guerraoui, N. Gupta, and R. Pinot, “Fine- tuning personalization in federated learning to mitigate adversarial clients,”Advances in Neural Information Processing Systems, 2024

    2024

  43. [51]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,”Machine Learning and Systems, vol. 2, pp. 429–450, 2020

    2020

  44. [52]

    Sensitivity curve max- imization: Attacking robust aggregators in distributed learning,

    C. A. Schroth, S. Vlaski, and A. M. Zoubir, “Sensitivity curve max- imization: Attacking robust aggregators in distributed learning,”arXiv preprint arXiv:2412.17740, 2024

    2024 arXiv

  45. [53]

    A little is enough: Circumvent- ing defenses for distributed learning,

    G. Baruch, M. Baruch, and Y . Goldberg, “A little is enough: Circumvent- ing defenses for distributed learning,”Advances in Neural Information Processing Systems, 2019

    2019

  46. [54]

    Dual-domain defenses for Byzantine- resilient decentralized resource allocation,

    R. Wang, Q. Ling, and Z. Tian, “Dual-domain defenses for Byzantine- resilient decentralized resource allocation,”IEEE Transactions on Signal and Information Processing over Networks, vol. 10, pp. 804–819, 2024. APPENDIXA PROOF OFTHEOREM1 Proof.Following [32]–[37], we denote th...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.