ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning

Bing Liu; Chengcheng Zhao; Li Chai; Peng Cheng; Yaonan Wang

REVIEW 4 major objections 4 minor 55 references

ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Exact convex-hull aggregation shrinks learning error in Byzantine-safe private distributed training.

desk verdict The RVC+CGP combination is a sensible new idea and the convergence analysis is worth reading, but the CGP privacy theorem is wrong: subsampling amplification cannot be applied to arbitrary dataset pairs, so the paper's central improved privacy-accuracy trade-off is unproven. read the letter →

arxiv 2506.15181 v1 pith:DJSPL6CY submitted 2025-06-18 cs.LG

Bing Liu , Chengcheng Zhao , Li Chai , Peng Cheng , Yaonan Wang This is my paper · ORCID

classification cs.LG

keywords Byzantineresiliencedecentralizeddistributedlearningresilientvectorconsensusconcentratedgeo-privacyprivacy-accuracytrade-offnon-convexoptimizationdifferentialprivacystochasticgradientdescent

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the accuracy loss in privacy-preserving, Byzantine-resilient distributed learning comes largely from two removable sources: approximate Byzantine filters that return points outside the honest agents' convex hull, and an overly pessimistic privacy measure. It replaces the filters with resilient vector consensus algorithms that provably return a point inside the convex hull of honest agents' updates, and measures privacy with concentrated geo-privacy instead of differential privacy. The result is a decentralized SGD algorithm that converges at rate $O(1/\sqrt{K})$ to a neighborhood of a stationary point while satisfying $\rho$-CGP with $\rho_i = 5K\zeta_i^2 L'^2/\sigma^2$. A sympathetic reader would care because the paper gives concrete error terms and shows the Byzantine estimation error term is eliminated rather than merely bounded.

What carries the argument

The load-bearing object is the resilient vector consensus operator $R(\cdot)$, which returns a point inside the convex hull of the normal in-neighbors' transmitted parameters; all the cited RVC implementations rest on Helly's theorem, which justifies the Byzantine-count condition $nf_i(k) < |N_i(k)|/(d+1)$. This operator makes the aggregation phase a linear time-varying system $x(k+1)=M(k)\tilde{x}(k)$ with a row-stochastic $M(k)$, so consensus error contracts by the factor $1-\lambda = \|(I - (1/n)\mathbf{1}\mathbf{1}^\top)M(k)\|_S^2$. The second load-bearing object is concentrated geo-privacy (CGP), which bounds R\'enyi divergence by $\rho\|x-x'\|^2$ using the input distance; combined with the $L'$-Lipschitz gradient assumption it yields the per-agent privacy parameter $\rho_i = 5K\zeta_i^2 L'^2/\sigma^2$ without assuming bounded gradients.

What would settle it

Run Algorithm 1 on a directed graph where one normal agent has exactly $|N_i(k)|/(d+1)$ Byzantine in-neighbors and observe whether the consensus error stops contracting, or directly compute the spectral norm of the matrix $M(k)$ induced by a cited RVC implementation and check whether $\|(I-(1/n)\mathbf{1}\mathbf{1}^\top)M(k)\|_S^2$ stays uniformly below 1 for all $k$.

Watch

Extended reading notes

Core claim

The central claim is that, under the condition that each normal agent has fewer Byzantine in-neighbors than $|N_i(k)|/(d+1)$, resilient vector consensus algorithms can serve as the aggregation rule in decentralized SGD, making the aggregation phase a linear time-varying system $x(k+1)=M(k)\tilde{x}(k)$ with a row-stochastic matrix $M(k)$. Because the consensus point lies exactly in the convex hull of the honest agents' transmitted parameters, the consensus and convergence bounds contain no Byzantine estimation-error term; the only trace of adversarial presence is the contraction factor $\lambda$ in the matrix $M(k)$. With step size $\gamma = c/\sqrt{K}$, the averaged squared gradient norm is bounded by $C_1/\sqrt{K} + C_2/K + C_3\chi^2(\theta^2+\tau^2) + C_4 d\sigma^2 + C_5$, where $\chi^2$ measures how far $M(k)$ is from being doubly stochastic. For privacy, the algorithm satisfies $\rho$-CGP with $\rho_i = 5K\zeta_i^2 L'^2/\sigma^2$ under the input-Lipschitz assumption, and this yields smaller privacy parameters and a flatter privacy-accuracy trade-off than the corresponding $(\varepsilon,\delta)$-DP bound.

Load-bearing premise

The result collapses if the resilient aggregation matrix $M(k)$ fails to shrink disagreement by a constant factor, or if a normal agent's Byzantine in-neighbors reach the fraction $1/(d+1)$ of its neighborhood—conditions the paper assumes but does not prove for any concrete implementation.

Editorial extensions

If this is right

The Byzantine estimation error term present in ClippedGossip, IOS, and remove-then-clip disappears, leaving the noise variance $\sigma^2$ and the stochastic-gradient variances $\theta^2, \tau^2$ as the dominant constant error sources.
The algorithm converges at rate $O(1/\sqrt{K})$ to a neighborhood of a first-order stationary point, matching the rate of decentralized SGD without privacy or resilience.
Under CGP, noise can be chosen for utility without a bounded-gradient assumption, provided the gradient-to-input Lipschitzness condition holds.
Compared to $(\varepsilon,\delta)$-DP, the same noise scale yields much smaller privacy parameters (for example, $\rho = 1.45$ versus $\varepsilon = 865$ in the paper's simulations), so the privacy-accuracy trade-off is more favorable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the contraction assumption (Assumption 5) holds for a concrete RVC implementation, the same framework could extend to higher-dimensional models; the paper's complexity warning about the centerpoint linear program suggests scalability, not theory, is the main bottleneck.
The privacy comparison between CGP and DP implicitly assumes that $L'$ is much smaller than $2G$; a testable extension is to measure $L'$ versus $2G$ on real datasets and check whether the gap persists after training.
Because the learning error is proportional to $d\sigma^2$, the paper's stated future work of correlated, time-varying noise could reduce the effective noise dimension; a direct test would compare isotropic versus correlated noise under the same $\rho$ value.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know about this paper: the algorithmic idea is genuinely reasonable, and the consensus/convergence analysis mostly hangs together, but the privacy claim that the whole trade-off argument depends on is false as stated.

What is new and good: the paper is the first to combine resilient vector consensus aggregation with concentrated geo-privacy, and that is a natural pairing. Using RVC to keep the aggregated point inside the convex hull of honest agents removes the estimation error term that plagues ClippedGossip/IOS-style rules, and the consensus and convergence analysis in Theorems 1 and 2 is a plausible extension of existing decentralized SGD machinery. The simulations show the expected qualitative behavior: better test accuracy than PP-IOS, lower consensus error, and privacy against gradient inversion at the tested noise scale. I believe the authors when they say the algorithm works in practice.

The soft spots are serious. Theorem 3 is not just missing a constant: the proof takes the subsampled RDP bound from Lemma 3, which is only valid for neighboring datasets differing in one entry, and replaces the sensitivity S2 with L'·||D−D'|| for arbitrary pairs. That substitution is illegitimate. Subsampling amplification does not give a ζ² factor when the two datasets differ in many entries. The stress-test counterexample is valid: for scalar linear loss with all entries of D equal and all entries of D' shifted by r, the exact Rényi divergence for batch size 1 is r²/(2σ²) at α=2, while Theorem 3's formula gives ρ=5ζ²L'²/σ², which is ten times too small under the paper's own parameter conditions. The claimed CGP inequality fails. Since Section 3.4 and Figure 7 use exactly this ρ to argue for an improved trade-off over DP, that central conclusion is unsupported.

Other issues are less fatal but need fixing: Theorem 2's statement and proof have mismatched constants (L²/n² in the theorem vs L² in the proof, and the C1–C5 expressions do not follow from the displayed substitution). Lemma 4, the DP comparison, is asserted without proof. Assumptions 5 and 6 assume the RVC aggregation yields a contracting row-stochastic matrix M(k), but no concrete RVC implementation is checked against that condition. The numerical DP comparison uses an unspecified GP radius r and empirically estimated G and L′ without error bars.

Who is this for? Someone working on Byzantine-resilient private decentralized learning could get a useful framework and a clean statement of why convex-hull-preserving aggregation helps accuracy. But the privacy analysis needs to be redone before the main claim can be trusted. I would send this to peer review because the idea is worth engaging and the flaws are identifiable and addressable, but the current version would not be acceptable without a corrected CGP proof.

Referee Report

4 major / 4 minor

Summary. The paper proposes ImprovDML, a decentralized distributed machine learning framework that simultaneously provides Byzantine resilience and privacy. The key design choice is to replace approximate Byzantine-resilient aggregation rules (ClippedGossip, IOS, remove-then-clip) with resilient vector consensus (RVC) algorithms whose output stays inside the convex hull of the honest agents' updated parameters. Gaussian noise is added to local stochastic gradients, and privacy is analyzed under concentrated geo-privacy (CGP) rather than standard differential privacy. The paper claims a consensus bound (Theorem 1), a non-convex convergence rate with explicit asymptotic error (Theorem 2), a CGP guarantee with privacy parameter ρ = 5Kζ²L'²/σ² (Theorem 3), and an improved privacy-accuracy trade-off compared with (ε,δ)-DP. Numerical experiments on a multi-robot XOR classification task compare the proposed algorithm with PP-IOS and evaluate gradient-inversion resistance.

Significance. The design idea is genuinely attractive: using RVC algorithms that compute a point inside the convex hull of normal agents' updates removes the Byzantine estimation-error term that appears in earlier aggregation rules, and the convergence analysis is a plausible extension of standard decentralized SGD arguments. The simulations support the qualitative claim that the proposed method achieves better accuracy than PP-IOS under the same noise level. If the CGP guarantee were correct, the paper would offer a useful alternative privacy metric for Byzantine-resilient decentralized learning. However, the central privacy theorem is mathematically false as stated, and the main advertised advantage over differential privacy rests on that false theorem. The convergence theorem also contains internal inconsistencies between the displayed bound and the proof. As it stands, the manuscript's headline claim is not established.

major comments (4)

[§3.3, Theorem 3 and Proof 3] The conversion from the neighboring-dataset RDP bound to an arbitrary-dataset CGP bound is invalid. Lemma 3's ζ² amplification applies only when the two datasets differ in one entry; for an arbitrary pair of datasets, the probability that the subsample hits a differing point is not ζ, so the amplification factor cannot be used. This is not a missing constant. Concretely, take f(x;z)=½(x−z)², so L′=1, with dataset size n=100, batch size b=1 (ζ=0.01), σ²=2, x(0)=0, D={0,...,0}, and D′={r,...,r}. One iteration of Algorithm 1 outputs N(0,γ²σ²) under D and N(γr,γ²σ²) under D′, so the exact Rényi divergence is D₂=r²/(2σ²)=r²/4. The theorem's constraints are satisfied: σ²/L′²=2≥1.5 and α=2≤log(L′²/(ζ(L′²+σ²)))=log(100/3)≈3.5. The claimed per-iteration bound gives D₂≤2ρ‖D−D′‖² with ρ=5ζ²L′²/σ²=0.00025, which is 0.0005r² if ‖D−D′‖²=r² and 0.05r² if ‖D−D′‖²=100r²; in either interpretation this is smaller than the true value r²/4. A valid CGP parameter would need to be at least 0.25 in the max-distance interpretation or 0.0025 in the sum-of-squares interpretation. The ζ² factor cannot appear for dataset pairs differing in many entries, so the formula ρ_i=5Kζ_i²L′²/σ² and the privacy-accuracy trade-off comparison in Section 3.4 and Figure 7 are unsupported.
[§3.2, Theorem 2 and Proof 2] The displayed theorem and the proof do not agree. Inequality (14) in Proof 2 has the consensus-error coefficient 3(L²+2χ²/γ²+24χ²L²), while the theorem statement contains 3(L²/n²+2χ²/γ²+24χ²L²). Since Δ(k) is already the average over n agents, the proof yields L², not L²/n². The simplified constants C1–C5 also do not follow from the substitution γ=c/√K and the Theorem 1 bound: the terms generated by 2χ²/γ² times the θ², τ², and σ² parts of the consensus bound contribute at order 1/c with specific λ factors, while the displayed C2, C3, C4, and C5 arrange those factors differently, and C5=2χ²Δ(0)/(c²(1−Λ)) does not match the term-by-term expansion. The claimed O(1/√K) rate and the quoted asymptotic constants therefore require re-derivation.
[§2.3 and Assumption 5] Lemma 2 only proves that the aggregation phase can be written as x(k+1)=M(k)x̃(k) with M(k) row-stochastic and positive entries on the support N_i(k)∪{i}. Row-stochasticity alone does not imply the contraction ‖(I−11'/n)M(k)‖_S² ≤ 1−λ(k) < 1 assumed in Assumption 5 and used in Theorem 1. The paper does not prove this contraction for any of the five cited RVC implementations (Byz-Iter, ADRC, convex combination, centerpoint), nor does it cite an external theorem establishing it under Assumption 6. Since λ>0 is the engine of the consensus bound and hence of the convergence and privacy claims, the theoretical results currently rest on an uninstantiated hypothesis.
[§3.1, Theorem 1] The stated step-size condition is insufficient for the claimed 0<Λ<1. With v=λ/2, Λ<1 is equivalent to 24γ²L²(2−λ)(1−λ)<λ², which requires γ < (λ/(2L))·(6(1−λ)(2−λ))^{-1/2}. The displayed condition γ < (1/(2L))·(λ/(6(1−λ)(2−λ)))^{1/2} is larger by a factor 1/√λ and can violate the required inequality whenever λ<1. Thus the consensus bound and the subsequent use of 1/(1−Λ) are not established as stated.

minor comments (4)

[§3.3, Proof 3] The quantity Aα is introduced as 'a constant related to the subsampling and the Gaussian mechanism' but is never defined, and Lemma 3 contains no such constant; the proof should either derive it explicitly or drop it.
[§4.3 and Assumption 7] The simulation sets L′=0.84 based on finitely many gradient evaluations over 3000 iterations, but Theorem 3 requires a worst-case Lipschitz bound over the entire input domain; an empirical estimate cannot certify the assumption.
[§3.4.2] The comparison between ε (DP) and ε_geo (GP) uses different metric spaces and involves a free radius r and a chosen δ; Figure 7's caption does not state r, and the statement that '2G is typically much larger than L′' is an informal observation rather than a theorem.
[§3.1, Proof 1] The summation in the displayed geometric-series step is written as Σ_{k=0}^{k} and should be Σ_{j=0}^{k-1}.

Circularity Check

1 steps flagged · score 2.0 of 10

Mostly self-contained; mild definitional circularity in CGP-vs-DP trade-off claim.

self definitional [Section 3.4.2 (Comparison with (ε,δ)-DP), following Definition 2]
"GP introduces a distance-based dimension by requiring gradient similarity only within a ball of arbitrary radius centered around a given input. Therefore, GP addresses the limitations of (ε,δ)-DP by offering a more fine-grained characterization of privacy preservation, based on the notion of distance between inputs. As a result, εi is typically much larger than εgeo,i, making it difficult to select an appropriate value for σ."

The claim that CGP/GP yields an 'improved and more accurate trade-off' is a direct restatement of Definition 2: in ρ-CGP the privacy parameter multiplies ||x−x'||², so for typical close inputs the privacy loss is small by construction. Section 3.4.2 does not derive this advantage from Algorithm 1's dynamics or from the consensus/convergence analysis; it is built into the distance-weighted definition of the metric. Thus the trade-off comparison reduces to the definition of CGP rather than to an independent property of ImprovDML. This is a mild, definitional circularity and does not affect the consensus/convergence theorems.

full rationale

The main derivation chain is self-contained with respect to circularity. Theorem 1 follows from the standard LTV consensus argument (Lemma 2 from [1] plus Assumption 5), and Theorem 2 is a standard SGD descent analysis combined with Theorem 1; neither theorem fits a parameter to its own conclusion. The Byzantine-resilient aggregation results are imported from existing RVC algorithms, and Assumption 6 is an explicit condition rather than a hidden relabeling. The privacy guarantee in Theorem 3 uses the subsampled RDP lemma (Lemma 3) and Assumption 7; the resulting ρ formula follows by substitution, and although the use of the ζ² amplification for non-neighboring dataset pairs is a correctness risk, that is not the same as a definitional identity. The only true circular flavor is in Section 3.4.2: the claimed superiority of CGP over DP is largely a consequence of CGP's distance-dependent definition (ρ multiplies ||x−x'||²), so the 'improved trade-off' is a restatement of the adopted privacy notion rather than an independent finding of the algorithm. This is mild and does not undermine the consensus or convergence results; hence score 2.

Assumptions & free parameters 7 free parameters · 7 assumptions · 0 invented entities

The framework depends on standard smoothness and variance assumptions, the existence of RVC routines with the stated Byzantine threshold, and Lipschitz continuity of gradients in the input. The privacy and trade-off claims also depend on empirically estimated constants G and L' that are not certified.

free parameters (7)

Noise scale σ = σ=2.0 in simulations
Tunable noise parameter controlling privacy vs accuracy; all bounds scale with σ².
Step size constant c = not set
γ=c/√K in Theorem 2; the value of c is left unspecified.
Subsampling rate ζ_i = about 16/1200 in simulations
Mini-batch fraction; appears quadratically in the privacy bound ρ_i.
Aggregation weight β_i(k) = 0.8 in simulations
Convex weight between own update and RVC aggregation in Eq. (4).
Gradient bound G = 9.2 estimated from local runs
Used only for the DP comparison and for clipping in the baseline [47], not for the ImprovDML privacy proof.
Lipschitz constant L' = 0.84 estimated from local runs
Empirical maximum of the gradient Hessian norm; enters ρ_i and the trade-off comparison.
GP radius r = not specified
Needed to convert CGP into (εgeo,δ,r)-GP and to compute εgeo in Fig. 7.

assumptions (7)

domain assumption Assumptions 1-4: lower-bounded loss, L-smooth gradients, bounded gradient variance, independent sampling
Standard decentralized SGD assumptions used in Theorems 1 and 2.
domain assumption Assumption 5: normal-agent graph strongly connected and ||(I-11'/n)M(k)||_S² ≤ μ<1
Required for consensus contraction; not proven for the RVC-induced M(k).
domain assumption Assumption 6: nf_i(k)<|N_i(k)|/(d+1) Byzantine in-neighbors
Necessary condition for RVC to compute a point in the honest convex hull, inherited from Helly-type arguments.
standard math Lemma 2 from [1]: aggregation phase is a row-stochastic LTV system x(k+1)=M(k)x̃(k)
Bridges RVC algorithms to the consensus analysis; relies on the Byzantine threshold condition.
standard math Lemma 3 from [32,43,50]: subsampled Gaussian mechanism satisfies RDP with amplification factor 5ζ²
Basis for the CGP bound in Theorem 3.
domain assumption Lemma 4: (ε,δ)-DP bound with ε_i=20G²ζ²K/σ²+2Gζ/σ√(20K log(1/δ))
Stated without proof; used for the DP comparison.
domain assumption Assumption 7: ∇f_i is L'-Lipschitz in the input ξ_i
Necessary for CGP; the paper estimates L' empirically rather than providing a certified bound.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning." pith.science (2026). https://pith.science/paper/DJSPL6CY

@misc{pith2026250615181,
  author       = {Pith},
  title        = {Pith review of: ImprovDML: Improved Trade-off in Private Byzantine-Resilient Distributed Machine Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJSPL6CY}},
  note         = {Machine review of arXiv:2506.15181}
}

read the original abstract

Jointly addressing Byzantine attacks and privacy leakage in distributed machine learning (DML) has become an important issue. A common strategy involves integrating Byzantine-resilient aggregation rules with differential privacy mechanisms. However, the incorporation of these techniques often results in a significant degradation in model accuracy. To address this issue, we propose a decentralized DML framework, named ImprovDML, that achieves high model accuracy while simultaneously ensuring privacy preservation and resilience to Byzantine attacks. The framework leverages a kind of resilient vector consensus algorithms that can compute a point within the normal (non-Byzantine) agents' convex hull for resilient aggregation at each iteration. Then, multivariate Gaussian noises are introduced to the gradients for privacy preservation. We provide convergence guarantees and derive asymptotic learning error bounds under non-convex settings, which are tighter than those reported in existing works. For the privacy analysis, we adopt the notion of concentrated geo-privacy, which quantifies privacy preservation based on the Euclidean distance between inputs. We demonstrate that it enables an improved trade-off between privacy preservation and model accuracy compared to differential privacy. Finally, numerical simulations validate our theoretical results.

Figures

Figures reproduced from arXiv: 2506.15181 by the authors.

**Figure 1.** Two kinds of DML vulnerabilities that can compromise the learning process. For instance, Byzantine attacks targeting DML can drastically degrade model accuracy [2, 16]. In addition, privacy attacks such as gradient inversion attacks [21,55] and model inversion attacks [18,52] pose serious privacy threats to DML systems. More importantly, these two types of attacks can be deployed simultaneously, further exacerbatin… view at source ↗

**Figure 2.** Comparison on aggregation rules. The blue points rep [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Simulation scenarios We consider a network of 14 robots with a connected graph topology, where one of them acts as the Byzantine agent. The Byzantine agent transmits arbitrary and different messages to different neighbors. The sizes of the train dataset for each normal agent are 1122, 1315,1521,1400,1369,1255,1239,1160,1138,1588,1550, 1384,and 1238, respectively. As for the test dataset, the detailed sizes are 312,… view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Comparison of learning results 4.2 Privacy Attack To evaluate the effectiveness of privacy preservation, we conduct gradient inversion attack experiments following the method proposed in [55]. As for the labels, we use the algorithms proposed in [53]. These methods are…

**Figure 6.** Figure 6: Gradient inversion attack 4.3 Privacy Metrics Finally, we validate the advantages of CGP on a better trade-off discussed in Section 3.4 through numerical simulations. By substituting δ = 10−5 (a value commonly used in machine learning with DP) and other detailed param…

**Figure 7.** Figure 7: Different metrics of privacy The results show that the DP parameter ε is significantly larger than both εgeo and ρ. Taking the simulation case with σ = 2.0 as an example, the corresponding parameters are ε = 865, εgeo = 36.2, and ρ = 1.45. Notably, σ = 2.0 already p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 52 canonical work pages

[1]

Resilient distributed vector consensus using centerpoint

W aseem Abbas, Mudassir Shabbir, Jiani Li, and Xenofon Koutsoukos. Resilient distributed vector consensus using centerpoint. Automatica, 136:110046, 2022

work page 2022
[2]

A little is enough: Circumventing defenses for distributed learnin g

Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learnin g. Advances in Neural Information Processing Systems , 32, 2019

work page 2019
[3]

Priva te empirical risk minimization: Eﬃcient algorithms and tight error bounds

Raef Bassily, Adam Smith, and Abhradeep Thakurta. Priva te empirical risk minimization: Eﬃcient algorithms and tight error bounds. In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science , pages 464–473. IEEE, 2014

work page 2014
[4]

Bounds on the sample complexity for private learning and private data release

Amos Beimel, Hai Brenner, Shiva Prasad Kasiviswanathan , and Kobbi Nissim. Bounds on the sample complexity for private learning and private data release. Machine Learning, 94:401–437, 2014

work page 2014
[5]

Machine learning with adversaries: Byzantine tolerant gradient descent

Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in Neural Information Processing Systems , 30, 2017. 14

work page 2017
[6]

Optimization methods for large-scale machine learning

L´ eon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review , 60(2):223–311, 2018

work page 2018
[7]

An optimal randomized algorithm for maximum tukey depth

Timothy M Chan. An optimal randomized algorithm for maximum tukey depth. In Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms , volume 4, pages 430–436, 2004

work page 2004
[8]

Distributed statistical machine learning in adversarial settings: Byz antine gradient descent

Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byz antine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 1(2):1–25, 2017

work page 2017

Show all 55 references

[9]

Privacy ampliﬁcati on by decentralization

Edwige Cyﬀers and Aur´ elien Bellet. Privacy ampliﬁcati on by decentralization. In International Conference on Artiﬁcial Intelligence and Statistics , pages 5334–5353. PMLR, 2022

2022
[10]

Diﬀerentially private decentralized learning with random walks

Edwige Cyﬀers, Aur´ elien Bellet, and Jalaj Upadhyay. Diﬀerentially private decentralized learning with random walks. In International Conference on Machine Learning , pages 9762–9783. PMLR, 2024

2024
[11]

Muﬄiato: Peer-to-peer privacy ampliﬁcation for decentralized optimization and averaging

Edwige Cyﬀers, Mathieu Even, Aur´ elien Bellet, and Lau rent Massouli´ e. Muﬄiato: Peer-to-peer privacy ampliﬁcation for decentralized optimization and averaging. Advances in Neural Information Processing Systems , 35:15889–15902, 2022

2022
[12]

Hell y’s theorem and its relatives

Ludwig Danzer, Branko Gr¨ unbaum, and Victor Klee. Hell y’s theorem and its relatives. In Convexity: Proceedings of the Seventh Symposium in Pure Mathematics of the American Mathematical Society , volume 7, page 101. American Mathematical Soc., 1963

1963
[13]

Large scale distributed deep networks

Jeﬀrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Marc’aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, et al. Large scale distributed deep networks. Advances in Neural Information Processing Systems, 25, 2012

2012
[14]

The algorithmic foundations of diﬀerential privacy

Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of diﬀerential privacy. Foundations and Trends® in Theoretical Computer Science , 9(3–4):211–407, 2014

2014
[15]

Bridge: Byzantine-resilient decentralized gradient descent

Cheng Fang, Zhixiong Yang, and W aheed U Bajwa. Bridge: Byzantine-resilient decentralized gradient descent. IEEE Transactions on Signal and Information Processing over Networks, 8:610–626, 2022

2022
[16]

Local model poisoning attacks to byzantine-robust federat ed learning

Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. Local model poisoning attacks to byzantine-robust federat ed learning. In 29th USENIX Security Symposium , pages 1605– 1622, 2020

2020
[17]

Byzantine- robust decentralized federated learning

Minghong Fang, Zifan Zhang, Hairi, Prashant Khanduri, Jia Liu, Songtao Lu, Yuchen Liu, and Neil Gong. Byzantine- robust decentralized federated learning. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 2874–2888, 2024

2024
[18]

Model inversion attacks that exploit conﬁdence informatio n and basic countermeasures

Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit conﬁdence informatio n and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322–1333, 2015

2015
[19]

Privacy-preserving aggregation for decentralized learning with byzantine-robustness

Ali Reza Ghavamipour, Benjamin Zi Hao Zhao, Oguzhan Ersoy, and Fatih Turkmen. Privacy-preserving aggregation for decentralized learning with byzantine-robustness. arXiv preprint arXiv:2404.17970, 2024

2024 arXiv
[20]

The hidde n vulnerability of distributed learning in byzantium

Rachid Guerraoui, S´ ebastien Rouault, et al. The hidde n vulnerability of distributed learning in byzantium. In International Conference on Machine Learning , pages 3521–
[21]

Gradvit: Gradient inversion of vision transformers

Ali Hatamizadeh, Hongxu Yin, Holger R Roth, W enqi Li, Jan Kautz, Daguang Xu, and Pavlo Molchanov. Gradvit: Gradient inversion of vision transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10021–10030, 2022

2022
[22]

Byzantine-robust decentralized learning via clippedgoss ip

Lie He, Sai Praneeth Karimireddy, and Martin Jaggi. Byzantine-robust decentralized learning via clippedgoss ip. arXiv preprint arXiv:2202.01545 , 2022

2022 arXiv
[23]

What can we learn privately? SIAM Journal on Computing , 40(3):793–826, 2011

Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissi m, Sofya Raskhodnikova, and Adam Smith. What can we learn privately? SIAM Journal on Computing , 40(3):793–826, 2011

2011
[24]

Revisiting gradient clipping: Stochastic bias and t ight convergence guarantees

Anastasia Koloskova, Hadrien Hendrikx, and Sebastian U Stich. Revisiting gradient clipping: Stochastic bias and t ight convergence guarantees. In International Conference on Machine Learning, pages 17343–17363. PMLR, 2023

2023
[25]

A uniﬁed theory of decentralize d sgd with changing topology and local updates

Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, M artin Jaggi, and Sebastian Stich. A uniﬁed theory of decentralize d sgd with changing topology and local updates. In International Conference on Machine Learning , pages 5381–
[26]

Resilient asymptotic consensus in robust networks

Heath J LeBlanc, Haotian Zhang, Xenofon Koutsoukos, and Shreyas Sundaram. Resilient asymptotic consensus in robust networks. IEEE Journal on Selected Areas in Communications, 31(4):766–781, 2013

2013
[27]

Byzantine resilient distributed learning in multirobot systems

Jiani Li, W aseem Abbas, Mudassir Shabbir, and Xenofon Koutsoukos. Byzantine resilient distributed learning in multirobot systems. IEEE Transactions on Robotics , 38(6):3550–3563, 2022

2022
[28]

Can decentralized algorithms outperform centralized algorithms? a case study for decentralized par allel stochastic gradient descent

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, W ei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized par allel stochastic gradient descent. Advances in Neural Information Processing Systems, 30, 2017

2017
[29]

Concentrated geo-privacy

Yuting Liang and Ke Yi. Concentrated geo-privacy. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security , pages 1934–1948, 2023

2023
[30]

Privacy- preserving decentralized federated learning over time-va rying communication graph

Yang Lu, Zhengxin Yu, and Neeraj Suri. Privacy- preserving decentralized federated learning over time-va rying communication graph. ACM Transactions on Privacy and Security, 26(3):1–39, 2023

2023
[31]

Multidimension al approximate agreement in byzantine asynchronous systems

Hammurabi Mendes and Maurice Herlihy. Multidimension al approximate agreement in byzantine asynchronous systems. In Proceedings of the Forty-Fifth Annual ACM Symposium on Theory of Computing , pages 391–400, 2013

2013
[32]

R´ enyi diﬀerential privacy

Ilya Mironov. R´ enyi diﬀerential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium , pages 263–275. IEEE, 2017

2017
[33]

Fault-tolerant rendezvous of multirobot systems

Hyongju Park and Seth A Hutchinson. Fault-tolerant rendezvous of multirobot systems. IEEE Transactions on Robotics, 33(3):565–582, 2017

2017
[34]

On measures of entropy and information

Alfr´ ed R´ enyi. On measures of entropy and information . In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics , volume 4, pages 547–562. University of California Press, 1961

1961
[35]

A scala ble approach for privacy-preserving collaborative machine learning

Jinhyun So, Basak Guler, and Salman Avestimehr. A scala ble approach for privacy-preserving collaborative machine learning. Advances in Neural Information Processing Systems, 33:8054–8066, 2020

2020
[36]

Training very deep networks

Rupesh K Srivastava, Klaus Greﬀ, and J¨ urgen Schmidhub er. Training very deep networks. Advances in Neural Information Processing Systems , 28, 2015. 15

2015
[37]

Decentralized federated averaging

Tao Sun, Dongsheng Li, and Bao W ang. Decentralized federated averaging. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4289–4301, 2022

2022
[38]

Iterative byzantine vector consensus in incomplete graphs

Nitin H Vaidya. Iterative byzantine vector consensus in incomplete graphs. In Distributed Computing and Networking: 15th International Conference, ICDCN 2014, Coimbatore, India, January 4-7, 2014. Proceedings 15 , pages 14–28. Springer, 2014

2014
[39]

A resilient convex combination for consensus-based distrib uted algorithms

Xuan W ang, Shaoshuai Mou, and Shreyas Sundaram. A resilient convex combination for consensus-based distrib uted algorithms. arXiv preprint arXiv:1806.10271 , 2018

2018 arXiv
[40]

Tailoring gradient methods for diﬀerentially private distributed optimizati on

Yongqiang W ang and Angelia Nedi´ c. Tailoring gradient methods for diﬀerentially private distributed optimizati on. IEEE Transactions on Automatic Control , 69(2):872–887, 2023

2023
[41]

Byzantine- resilient decentralized stochastic optimization with rob ust aggregation rules

Zhaoxian W u, Tianyi Chen, and Qing Ling. Byzantine- resilient decentralized stochastic optimization with rob ust aggregation rules. IEEE Transactions on Signal Processing , 2023

2023
[42]

Faba: an algorithm for fast aggregation against byzantine attacks in distributed neural networks

Qi Xia, Zeyi Tao, Zijiang Hao, and Qun Li. Faba: an algorithm for fast aggregation against byzantine attacks in distributed neural networks. In International Joint Conference on Artiﬁcial Intelligence , page 4824–4830, 2019

2019
[43]

A(DP) 2SGD: Asynchronous decentralized parallel stochastic gradient descent with diﬀerential privacy

Jie Xu, W ei Zhang, and Fei W ang. A(DP) 2SGD: Asynchronous decentralized parallel stochastic gradient descent with diﬀerential privacy. IEEE Transactions on Pattern Analysis and Machine Intelligence , 44(11):8036– 8047, 2021

2021
[44]

Resilient multi-dimensional consensus in adversaria l environment

Jiaqi Yan, Xiuxian Li, Yilin Mo, and Changyun W en. Resilient multi-dimensional consensus in adversaria l environment. Automatica, 145:110530, 2022

2022
[45]

Byzantine-robust decentralized learning via remove-then-clip aggregation

Caiyi Yang and Javad Ghaderi. Byzantine-robust decentralized learning via remove-then-clip aggregation . In Proceedings of the AAAI Conference on Artiﬁcial Intelligence, volume 38, pages 21735–21743, 2024

2024
[46]

Byrdie: Byzantine- resilient distributed coordinate descent for decentraliz ed learning

Zhixiong Yang and W aheed U Bajwa. Byrdie: Byzantine- resilient distributed coordinate descent for decentraliz ed learning. IEEE Transactions on Signal and Information Processing over Networks , 5(4):611–627, 2019

2019
[47]

On the tradeoﬀ between privacy preservation and byzantine-robustness in decentralized learning

Haoxiang Ye, Heng Zhu, and Qing Ling. On the tradeoﬀ between privacy preservation and byzantine-robustness in decentralized learning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 9336–9340. IEEE, 2024

2024
[48]

Interior point algorithms: theory and analysis

Yinyu Ye. Interior point algorithms: theory and analysis . John Wiley & Sons, 2011

2011
[49]

Byzantine-robust distributed learning: Toward s optimal statistical rates

Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Toward s optimal statistical rates. In International Conference on Machine Learning, pages 5650–5659. Pmlr, 2018

2018
[50]

Subsampled r´ enyi diﬀerential privacy and analytical moments accountant

Yu-Xiang W ang, Borja Balle, and Shiva Kasiviswanathan . Subsampled r´ enyi diﬀerential privacy and analytical moments accountant. In The 22nd International Conference on Artiﬁcial Intelligence and Statistics , pages 1226–1235. PMLR, 2019

2019
[51]

Admm based privacy-preserving decentralized optimizatio n

Chunlei Zhang, Muaz Ahmad, and Yongqiang W ang. Admm based privacy-preserving decentralized optimizatio n. IEEE Transactions on Information Forensics and Security , 14(3):565–580, 2018

2018
[52]

The secret revealer: Generative model-inversion attacks against deep neural networks

Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, W enxiao W ang, Bo Li, and Dawn Song. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 253–261, 2020

2020
[53]

idlg: Improved deep leakage from gradients

Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. idlg: Improved deep leakage from gradients. arXiv preprint arXiv:2001.02610, 2020

2001 arXiv
[54]

Pvd-ﬂ: A privacy-preserving and veriﬁable decentralized federated learning framework

Jiaqi Zhao, Hui Zhu, Fengwei W ang, Rongxing Lu, Zhe Liu, and Hui Li. Pvd-ﬂ: A privacy-preserving and veriﬁable decentralized federated learning framework . IEEE Transactions on Information Forensics and Security , 17:2059–2073, 2022

2022
[55]

Deep leakage from gradients

Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Neural Information Processing Systems, 32, 2019. 16

2019

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Resilient distributed vector consensus using centerpoint

W aseem Abbas, Mudassir Shabbir, Jiani Li, and Xenofon Koutsoukos. Resilient distributed vector consensus using centerpoint. Automatica, 136:110046, 2022

work page 2022

[2] [2]

A little is enough: Circumventing defenses for distributed learnin g

Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learnin g. Advances in Neural Information Processing Systems , 32, 2019

work page 2019

[3] [3]

Priva te empirical risk minimization: Eﬃcient algorithms and tight error bounds

Raef Bassily, Adam Smith, and Abhradeep Thakurta. Priva te empirical risk minimization: Eﬃcient algorithms and tight error bounds. In 2014 IEEE 55th Annual Symposium on Foundations of Computer Science , pages 464–473. IEEE, 2014

work page 2014

[4] [4]

Bounds on the sample complexity for private learning and private data release

Amos Beimel, Hai Brenner, Shiva Prasad Kasiviswanathan , and Kobbi Nissim. Bounds on the sample complexity for private learning and private data release. Machine Learning, 94:401–437, 2014

work page 2014

[5] [5]

Machine learning with adversaries: Byzantine tolerant gradient descent

Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in Neural Information Processing Systems , 30, 2017. 14

work page 2017

[6] [6]

Optimization methods for large-scale machine learning

L´ eon Bottou, Frank E Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM review , 60(2):223–311, 2018

work page 2018

[7] [7]

An optimal randomized algorithm for maximum tukey depth

Timothy M Chan. An optimal randomized algorithm for maximum tukey depth. In Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms , volume 4, pages 430–436, 2004

work page 2004

[8] [8]

Distributed statistical machine learning in adversarial settings: Byz antine gradient descent

Yudong Chen, Lili Su, and Jiaming Xu. Distributed statistical machine learning in adversarial settings: Byz antine gradient descent. Proceedings of the ACM on Measurement and Analysis of Computing Systems , 1(2):1–25, 2017

work page 2017