Generalized Byzantine-tolerant SGD
read the original abstract
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model. The attackers can arbitrarily manipulate the data transferred between the servers and the workers in the parameter server~(PS) architecture. We prove the Byzantine resilience properties of these aggregation rules. Empirical analysis shows that the proposed techniques outperform current approaches for realistic use cases and Byzantine attack scenarios.
This paper has not been read by Pith yet.
Forward citations
Cited by 4 Pith papers
-
Practical Validity Conditions for Byzantine-Tolerant Federated Learning
Introduces MEB and c-MEB validity conditions for Byzantine-robust aggregation, proving achievability under majority honesty (n>2t) with an optimal MinMax-MEB rule at c<sqrt(2) and explicit guarantees for standard aggregators.
-
Byzantine-Robust Distributed SGD: A Unified Analysis and Tight Error Bounds
Unified convergence rates and tight lower bounds for Byzantine-robust distributed SGD under stochasticity and general data heterogeneity, showing local momentum reduces stochastic error floors.
-
RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent
RESIST achieves algorithmic and statistical convergence guarantees for strongly convex, PL, and nonconvex ERM under MITM attacks via multistep consensus gradient descent plus robust screening.
-
Generalized Rank Regression
Generalized Rank Regression extends rank methods to non-monotonic scores, derives Bahadur representation and asymptotic normality, proposes a two-stage sub-gradient algorithm, and shows variance equivalence to composi...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.