Pith. sign in

REVIEW 3 major objections 3 minor 62 references

One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning

T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PRoBit+ shows that maximum-likelihood aggregation of one-bit quantized model updates can match full-precision FedAvg accuracy while adding differential privacy and bounding Byzantine influence, with the extra error shrinking as…

desk verdict The dynamic-b heuristic and the experiments are worth a look, but the central convergence proof has an invalid step in Lemma 2 and the O(1/M) claim is not supported as written. read the letter →

arxiv 2507.03973 v1 pith:OIKM2THW submitted 2025-07-05 cs.DC

Muhang Lan , Song Xiao , Wenyi Zhang This is my paper
classification cs.DC
keywords one-bitquantizationmaximumlikelihoodaggregationfederatedlearningByzantinerobustnessdifferentialprivacypersonalizedmodelregularizationstochastic
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that a single bit per model coordinate is enough for the server in a personalized federated learning system to reconstruct the average client update, while also defending against Byzantine clients and protecting privacy. Its proposed method, PRoBit+, stochastically quantizes each update coordinate to $\pm1$ and then estimates the average with maximum likelihood, with the quantization range $b$ adjusted dynamically during training. The central theoretical claim is that this estimate is unbiased and becomes exact in expectation, with squared error $\sum_i (b_i^2-\theta_i^2)/M$, so the extra error from one-bit transmission and privacy protection shrinks as $\mathcal{O}(1/M)$ as the number of clients $M$ grows. If correct, large-scale federated learning can cut communication by a factor of 32 compared with full-precision FedAvg while keeping accuracy nearly unchanged and obtaining both Byzantine robustness and $(\epsilon,0)$-differential privacy from the same randomized mechanism.

What carries the argument

The carrying object is the maximum-likelihood aggregation rule $\hat{\theta}_i=(2N_i-M)b_i/M$, where $N_i$ counts the number of uploaded bits equal to $1$ at coordinate $i$ and $b_i$ is the quantization range. Each client's update component is mapped to $\pm1$ by the stochastic quantizer with probabilities $(b_i+\delta_i^m)/(2b_i)$ and $(b_i-\delta_i^m)/(2b_i)$, making the transmitted bit a randomized response whose randomness is controlled by $b$. The likelihood in Eq. (12) is a binomial in $N_i$, which yields the ML estimate above; the proof chain then uses the Factorization Theorem for sufficiency, the variance calculation in Appendix A for the $\mathcal{O}(1/M)$ error, and the convergence lemmas in Appendix B to convert these estimation properties into the FL convergence bound of Theorem 4.

What would settle it

Run PRoBit+ on a synthetic problem with two client groups whose per-coordinate update means differ by a known amount, and compare the server estimate $\hat{\theta}_i$ with the full-precision average of the actual local updates in that round over many replications; a systematic bias that does not shrink as $M$ grows, or an empirical squared error that does not track $(b_i^2-\theta_i^2)/M$, would refute the central unbiasedness and convergence claims.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the sign pattern produced by a stochastic one-bit quantizer is a sufficient statistic for the mean update $\theta_i$, and that the maximum-likelihood estimate $\hat{\theta}_i=(2N_i-M)b_i/M$ is an unbiased replacement for the FedAvg average. Theorem 1 proves sufficiency, unbiasedness, and the exact error formula; Theorem 2 bounds Byzantine influence by $2\beta\|b\|$, independent of attack magnitudes; Theorem 3 gives $(\epsilon,0)$-differential privacy when $b_i \geq \max_m |\delta_i^m| + (1+1/\epsilon)\Delta_1$; and Theorem 4 puts these together into a convergence bound for heterogeneous personalized FL whose privacy and transmission penalties vanish at $\mathcal{O}(1/M)$.

Load-bearing premise

Every client's update component at a given coordinate is assumed to have the same expected value $\theta_i$; if the true per-client update means differ, as they do under heterogeneous data, the estimate is unbiased for a population average rather than for the realized sample mean that FedAvg actually computes.

Editorial extensions

If this is right

  • One-bit per-coordinate uploading cuts the communication cost by a factor of 32 relative to 32-bit FedAvg, and the paper's experiments report accuracy gaps of only 1–3% as the client count grows from 10 to 50.
  • No Byzantine attack can move the aggregate by more than $2\beta\|b\|$ regardless of the magnitudes of malicious updates, so the resilience depends on the fraction of malicious clients rather than on the attack's strength.
  • Because the privacy penalty in Theorem 4 scales like $\sqrt{\Delta_1/(M(1+1/\epsilon))}$, increasing the number of participating clients makes the accuracy cost of differential privacy vanish at $\mathcal{O}(1/M)$.
  • The ML-derived aggregation step supplies an automatic, dynamically adjusted update step size, replacing the manually tuned step sizes required by majority-vote and sign-accumulation baselines and stabilizing deep-network training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct corollary of the equal-mean assumption is that under strongly non-IID data the server estimate is best interpreted as a population average over clients rather than as the exact realized FedAvg update; whether personalization makes that difference benign is a testable question the paper does not settle.
  • The dynamic adjustment of $b$ relies on a separate one-bit loss signal that is itself unprotected, so a Byzantine client could manipulate the quantization range; hardening that feedback loop is a natural extension.
  • If the $\mathcal{O}(1/M)$ privacy cost holds in practice, recruiting more clients improves communication efficiency, robustness, and privacy simultaneously, which could make large-scale participation an explicit optimization target for FL system designers.
  • The same randomized-response structure suggests that $b$ could be tuned per coordinate from server-side statistics rather than through client loss messages, which would remove the additional Byzantine attack surface.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This paper proposes PRoBit+, a one-bit stochastic quantization and maximum-likelihood aggregation scheme for personalized federated learning, claiming to address communication overhead, Byzantine robustness, and local differential privacy simultaneously. The server estimates the mean of the clients' model updates from the received bits via Eq. (13), and the authors prove (Theorem 1) that, in the Byzantine-free case, the estimator is a sufficient statistic, unbiased, and has squared error sum_i(b_i^2 - theta_i^2)/M. They further prove a Byzantine-robustness bound on the expected aggregate (Theorem 2), an (epsilon,0)-DP guarantee for the per-client bit randomizer (Theorem 3), and a convergence bound for the regularized FL procedure (Theorem 4), from which they conclude that the degradation due to one-bit transmission and privacy protection decays as O(1/M) in the number of clients M. Experiments on FMNIST and CIFAR-10 compare PRoBit+ with FedAvg, Fed-GM, signSGD-MV, and RSA under four Byzantine attacks.

Significance. The contribution is potentially significant: a single-bit, local-DP, Byzantine-robust aggregation rule with a simple closed-form estimator and an explicit convergence bound would be a useful addition to the FL toolkit. The estimator construction is transparent, and the Byzantine-free bias and variance computation in Theorem 1 is simple and verifiable. The paper also provides an open-source implementation and experiments across two datasets with several attack models. However, the advertised O(1/M) rate is not the rate supported by Theorem 4, and the proof of Lemma 2, which is the bridge between the estimator and the convergence guarantee, is algebraically invalid as written. The convergence theorem therefore needs substantive repair. These defects are localized and appear repairable, so the central idea remains promising, but the current manuscript does not establish its main theoretical claims.

major comments (3)
  1. [Appendix B] The proof of Lemma 2 is invalid in several places. First, the decomposition of the Byzantine contribution replaces sum_{m in B} I{z_i^m = 1} with sum_{m in B}(I{z_i^m > c_i^m} - I{z_i^m < c_i^m}), which is not an identity: when z_i^m = 1 and c_i^m = 1, the left side is 1 and the right side is 0. Second, the equality E[(theta_i - hat_theta_i)^2] = E[hat_theta_i^2] - theta_i^2 is used even though E[hat_theta_i] = theta_i does not hold under Byzantine attacks; the text's statement that it is 'reasonable to assume that the aggregated values remain unbiased' is not a proof. Third, inequality (31), ||b||^2 - ||theta||^2 <= 2(1 + 1/epsilon)Delta1 ||b||_1, is not a consequence of the DP condition in Theorem 3; for example, if theta_i = 0 and max_m |delta_i^m| is much larger than Delta1, then b_i can be large enough that b_i^2 > 2(1 + 1/epsilon)Delta1 b_i. Since Lemma 3 and Theorem 4 consume Lemma 2 directly, the supplied proof does not establish the central convergence result. The lemma may well be true by a cruder bias-variance argument, but the proof must be rewritten.
  2. [Abstract] The convergence bound in Theorem 4 has an additive transmission/privacy term proportional to sqrt( (Delta1 / M) (1 + 1/epsilon) d ||zeta|| ), which decays as O(M^{-1/2}), not O(M^{-1}). Theorem 1 establishes an O(1/M) squared estimation error for the Byzantine-free estimator, but Lemma 2 and Theorem 4 work with a first-moment bound, and the O(1/M) rate is not transferred to the convergence guarantee. The abstract's claim that 'performance degradation caused by transmission errors and privacy protection can be progressively eliminated at a rate of O(1/M)' is therefore not supported by the supplied analysis. This should be corrected to O(1/sqrt(M)) for the convergence bound, or the analysis should be changed to provide an O(1/M) first-moment or squared-norm bound.
  3. [Section IV.B] The statistical derivation assumes that every client's component mean bar-delta_i^m has the same expectation theta_i. Under the heterogeneous-data setting claimed in the paper, bar-delta_i^m differs across clients, so E[hat_theta_i] equals a population-averaged mean, not the realized FedAvg sample mean (1/M) sum_m delta_i^m. Theorem 1's unbiasedness and the interpretation that PRoBit+ 'approaches FedAvg' are therefore only valid under a homogeneity-of-means or random-effects model, not under the fixed heterogeneous clients used in the experiments. The authors should either state this limitation explicitly, introduce a hierarchical model with separate client means, or qualify the claims of FedAvg equivalence under heterogeneity.
minor comments (3)
  1. [Section I] The Introduction states 'Finally, Section VIII concludes the work,' but the manuscript's conclusions appear in Section VII; the cross-reference is incorrect.
  2. [Appendix A] In the computation of E[N_i^2], the double sum over 'i != j' uses indices i and j that are also used for parameter coordinates; the sum should be over distinct client indices, for example m and n.
  3. [Appendix B] The index in the sums 'i in B' should be 'm in B', since i already denotes the parameter coordinate; using i for both the coordinate and the Byzantine client index makes the already difficult proof harder to follow.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the PRoBit+ estimator and its error bounds are derived from the paper's explicit stochastic-quantization model, not assumed; the main caveat is a non-circular proof gap in Lemma 2.

full rationale

The derivation chain is self-contained. The PRoBit+ aggregate (Eq. 13) is the MLE of the Bernoulli model built from the stochastic compressor (Eq. 5) and the stated equal-mean assumption (Eqs. 8-10), and Theorem 1's unbiasedness, sufficiency, and (b_i^2 - theta_i^2)/M squared error are direct moment computations from that model rather than inputs billed as predictions; Theorem 2 (Byzantine) and Theorem 3 (DP) are proved from the quantizer's probability law without assuming the convergence result, and Theorem 4 chains Lemmas 1-3 from standard FL assumptions (Assumptions 1-2, Definition 2). The author-related citations are background and not load-bearing. I flag one non-circular correctness gap: in Appendix B's proof of Lemma 2 the paper writes 'Since the server is unaware of the attacks, it is reasonable to assume that the aggregated values remain unbiased' and then uses E[(theta_i - hat theta_i)^2] = E[(...)]^2 - theta_i^2, which is invalid when Byzantine clients bias the estimator, and the decomposition I{z_i^m=1} = I{z_i^m>c_i^m} - I{z_i^m<c_i^m} is false (both indicators vanish when z_i^m=c_i^m=1). This undermines the supplied proof of Theorem 4 but is a mathematical-support defect, not a reduction of a predicted quantity to an input; the abstract's O(1/M) wording also differs from the O(1/sqrt M) term visible in Theorem 4, again a consistency issue, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The ledger shows three hand-chosen hyperparameters and four domain assumptions that the theoretical claims rest on, of which the common-mean assumption and the b_i bound are the most fragile.

free parameters (3)
  • quantization range b_i = dynamic: initial 0.01, +1% when loss decreases, -2% when increases; fixed: 0.01; optimal: max|delta| +…
    Controls the trade-off among transmission accuracy, DP, Byzantine robustness, and convergence; chosen by hand or heuristic, not derived.
  • l1 sensitivity Delta1 = 0.02eta = 0.0002
    Set by heuristic formula from [55]; used in the DP condition and the convergence bound.
  • regularization lambda = 0.2
    Personalized FL penalty; chosen by hand and affects the convergence bound via rho.
assumptions (4)
  • domain assumption Client updates delta_i^m are independent Gaussians with a common mean theta_i
    Used in Eqs. (8)-(10) to derive the likelihood and the ML estimate. This homogeneity assumption is strong under data heterogeneity.
  • domain assumption B-dissimilarity (Assumption 1)
    Standard bounded-heterogeneity condition used in the convergence analysis.
  • standard math Nonconvex L-smooth and L0-Lipschitz losses with bounded Hessian spectrum (Assumption 2)
    Standard smoothness assumptions for nonconvex FL convergence proofs.
  • domain assumption b_i >= max_m |delta_i^m| for the stochastic quantizer to be valid
    Required for probabilities in (5) to lie in [0,1]; not guaranteed by the fixed or dynamic b selection in experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning." pith.science (2026). https://pith.science/paper/OIKM2THW

@misc{pith2026250703973,
  author       = {Pith},
  title        = {Pith review of: One-Bit Model Aggregation for Differentially Private and Byzantine-Robust Personalized Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OIKM2THW}},
  note         = {Machine review of arXiv:2507.03973}
}
abstract

As the scale of federated learning (FL) systems expands, their inherent performance limitations like communication overhead, Byzantine vulnerability, and privacy leakage have become increasingly critical. This paper considers a personalized FL framework based on model regularization, and proposes a model aggregation algorithm named PRoBit+ to concurrently overcome these limitations. PRoBit+ employs one-bit stochastic quantization and maximum likelihood estimation for parameter aggregation, and dynamically adjusts the step size of parameter updates, improving training stability of deep neural networks under low communication overhead and heterogeneous data distributions. PRoBit+'s statistical analysis is then conducted and its Byzantine robustness is proved. The $(\epsilon,0)$-differential privacy and a convergence upper bound of the PRoBit+ based FL are also theoretically established in heterogeneous contexts. The analysis illustrates the trade-off among transmission accuracy, security guarantees, and convergence rates, and also indicates that the performance degradation caused by transmission errors and privacy protection can be progressively eliminated at a rate of $\mathcal{O}(1/M)$ as the number of uploading clients $M$ increases. Comprehensive numerical experiments are conducted to assess PRoBit+ in comparison to benchmark methods across different Byzantine attacks and varying proportions of malicious clients. The experimental results demonstrate that PRoBit+ exhibits improved Byzantine robustness over existing bit-based transmission schemes, minimal performance degradation related to privacy protection, and nearly identical performance to full-precision FedAvg in a secure environment.

Figures

Figures reproduced from arXiv: 2507.03973 by the authors.

Figure 1
Figure 1. Distribution of client updates δm i and their means δ¯m i . θi within the context of the bit compressor (5). Initially, we have P(c 1 i , c2 i , · · · , cM i ) = Y M m=1 Z bi −bi P(c m i |δ m i )dF(δ m i ) (6) = Y M m=1 Z bi −bi P(c m i |δ m i )f(δ m i )dδ m i , (7) where (6) represents the Riemann–Stieltjes integral and F(δ m i ) is the cumulative distribution function of the random variable δ m i . Calculating the… view at source ↗
Figure 2
Figure 2. Comprehensive impacts of quantization parameter [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 4
Figure 4. Test accuracy versus client numbers and privacy loss on CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Training progress with different settings of [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 5
Figure 5. Figure 5: Test accuracy versus proportions of Byzantine clients on FMNIST. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Test accuracy versus proportions of Byzantine clients on CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Test accuracy versus training iterations with [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Test accuracy versus training iterations with [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 55 canonical work pages

  1. [1]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. Int. Conf. Artif. Intell. Statist. , vol. 54, Apr. 2017, pp. 1273–1282

    work page 2017

  2. [2]

    Federated learning for the Internet of Things: Applications, challenges, and opportunities,

    T. Zhang, L. Gao, C. He, M. Zhang, B. Krishnamachari, and A. S. Avestimehr, “Federated learning for the Internet of Things: Applications, challenges, and opportunities,” IEEE Internet Things J. , vol. 5, no. 1, pp. 24–29, May 2022

    work page 2022

  3. [3]

    Confederated learning: Federated learning with decentralized edge servers,

    B. Wang, J. Fang, H. Li, X. Yuan, and Q. Ling, “Confederated learning: Federated learning with decentralized edge servers,” IEEE Trans. Signal Process., vol. 71, pp. 248–263, Feb. 2023

    work page 2023

  4. [4]

    A survey on federated learning,

    C. Zhang, Y . Xie, H. Bai, B. Yu, W. Li, and Y . Gao, “A survey on federated learning,” Knowl. Syst., vol. 216, no. 106775, pp. 1–11, Mar. 2021

    work page 2021

  5. [5]

    Heterogeneous feder- ated learning: State-of-the-art and research challenges,

    M. Ye, X. Fang, B. Du, P. C. Yuen, and D. Tao, “Heterogeneous feder- ated learning: State-of-the-art and research challenges,” ACM Comput. Surv., vol. 56, no. 3, pp. 1–44, Oct. 2023

    work page 2023

  6. [6]

    FedPD: A federated learning framework with adaptivity to Non-IID data,

    X. Zhang, M. Hong, S. Dhople, W. Yin, and Y . Liu, “FedPD: A federated learning framework with adaptivity to Non-IID data,”IEEE Trans. Signal Process., vol. 69, pp. 6055–6070, Oct. 2021

    work page 2021

  7. [7]

    Towards personalized federated learning,

    A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 34, no. 12, pp. 9587–9603, Dec. 2023

    work page 2023

  8. [8]

    Byzantine-robust and communication-efficient personalized federated learning,

    J. Zhang, X. He, Y . Huang, and Q. Ling, “Byzantine-robust and communication-efficient personalized federated learning,” IEEE Trans. Signal Process., vol. 73, pp. 26–39, Dec. 2025

    work page 2025

Show all 62 references
  1. [9]

    Adaptive model pruning and personalization for federated learning over wireless networks,

    X. Liu, T. Ratnarajah, M. Sellathurai, and Y . C. Eldar, “Adaptive model pruning and personalization for federated learning over wireless networks,” IEEE Trans. Signal Process. , vol. 72, pp. 4395–4411, Sep. 2024

    2024

  2. [10]

    Personalized federated learning towards communication efficiency, robustness and fairness,

    S. Lin, Y . Han, X. Li, and Z. Zhang, “Personalized federated learning towards communication efficiency, robustness and fairness,” inProc. Int. Conf. Neural Inf. Process. Syst. , vol. 35, Nov. 2022, pp. 30 471–30 485

    2022

  3. [11]

    Communication-efficient design for quantized decentralized federated learning,

    L. Chen, W. Liu, Y . Chen, and W. Wang, “Communication-efficient design for quantized decentralized federated learning,” IEEE Trans. Signal Process., vol. 72, pp. 1175–1188, 2024

    2024

  4. [12]

    Flash: Federated learning-based LLMs for advanced query processing in social networks through RAG,

    S. Puppala, I. Hossain, M. J. Alam, and S. Talukder, “Flash: Federated learning-based LLMs for advanced query processing in social networks through RAG,” arXiv:2408.05242, 2024

    2024 arXiv

  5. [13]

    A survey of trustworthy federated learning: Issues, solutions, and challenges,

    Y . Zhang, D. Zeng, J. Luo, X. Fu, G. Chen, Z. Xu, and I. King, “A survey of trustworthy federated learning: Issues, solutions, and challenges,” ACM Trans. Intell. Syst. Technol. , vol. 15, no. 6, pp. 1–47, Oct. 2024

    2024

  6. [14]

    An experimental study of Byzantine- robust aggregation schemes in federated learning,

    S. Li, E. C.-H. Ngai, and T. V oigt, “An experimental study of Byzantine- robust aggregation schemes in federated learning,” IEEE Trans. Big Data, vol. 10, no. 6, pp. 975–988, Nov. 2024

    2024

  7. [15]

    A comprehensive survey of privacy- preserving federated learning: A taxonomy, review, and future direc- tions,

    X. Yin, Y . Zhu, and J. Hu, “A comprehensive survey of privacy- preserving federated learning: A taxonomy, review, and future direc- tions,” ACM Comput. Surv., vol. 54, no. 6, pp. 1–36, Jul. 2021

    2021

  8. [16]

    signSGD: Compressed optimisation for non-convex problems,

    J. Bernstein, Y .-X. Wang, K. Azizzadenesheli, and A. Anandkumar, “signSGD: Compressed optimisation for non-convex problems,” in Proc. Int. Conf. Mach. Learn. , vol. 80, Jul 2018, pp. 560–569

    2018

  9. [17]

    signSGD with majority vote is communication efficient and fault tolerant,

    J. Bernstein, J. Zhao, K. Azizzadenesheli, and A. Anandkumar, “signSGD with majority vote is communication efficient and fault tolerant,” arXiv: 1810.05291, 2019

    2019 arXiv

  10. [18]

    Distributed training with heterogeneous data: bridging median- and mean-based algorithms,

    X. Chen, T. Chen, H. Sun, Z. S. Wu, and M. Hong, “Distributed training with heterogeneous data: bridging median- and mean-based algorithms,” in Proc. Int. Conf. Neural Inf. Process. Syst. , vol. 33, Dec. 2020, pp. 21 626–21 636

    2020

  11. [19]

    Sign-based gradient descent with heterogeneous data: Convergence and Byzantine resilience,

    R. Jin, Y . Liu, Y . Huang, X. He, T. Wu, and H. Dai, “Sign-based gradient descent with heterogeneous data: Convergence and Byzantine resilience,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 36, no. 2, pp. 3834–3846, Feb. 2025

    2025

  12. [20]

    z-SignFedAvg: a unified stochastic sign-based compression for federated learning,

    Z. Tang, Y . Wang, and T.-H. Chang, “z-SignFedAvg: a unified stochastic sign-based compression for federated learning,” in Proc. AAAI Conf. Artif. Intell., Feb. 2024, pp. 15 301–15 309

    2024

  13. [21]

    S 3GD-MV: Sparse-SignSGD with majority vote for communication-efficient distributed learning,

    C. Park and N. Lee, “S 3GD-MV: Sparse-SignSGD with majority vote for communication-efficient distributed learning,” in IEEE Int. Symp. Inf. Theory, Jun. 2023, pp. 2266–2271

    2023

  14. [22]

    Federated optimization in heterogeneous networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated optimization in heterogeneous networks,” in Proc. Mach. Learn. Syst., vol. 2, Mar. 2020, pp. 429–450

    2020

  15. [23]

    Federated learning: Strategies for improving communication efficiency,

    J. Kone ˇcn´y, H. B. McMahan, F. X. Yu, P. Richt ´arik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv:1610.05492, 2016

    2016 arXiv

  16. [24]

    QSGD: Communication-efficient SGD via gradient quantization and encoding,

    D. Alistarh, D. Grubic, J. Z. Li, R. Tomioka, and M. V ojnovic, “QSGD: Communication-efficient SGD via gradient quantization and encoding,” in Proc. Int. Conf. Neural Inf. Process. Syst. , vol. 30, Dec. 2017, pp. 1707–1718

    2017

  17. [25]

    UVeQFed: Universal vector quantization for federated learning,

    N. Shlezinger, M. Chen, Y . C. Eldar, H. V . Poor, and S. Cui, “UVeQFed: Universal vector quantization for federated learning,”IEEE Trans. Signal Process., vol. 69, pp. 500–514, Dec. 2021

    2021

  18. [26]

    Adaptive gradient quantization for data-parallel SGD,

    F. Faghri, I. Tabrizian, I. Markov, D. Alistarh, D. M. Roy, and A. Ramezani-Kebrya, “Adaptive gradient quantization for data-parallel SGD,” in Proc. Int. Conf. Neural Inf. Process. Syst. , vol. 33, Dec. 2020, pp. 3174–3185

    2020

  19. [27]

    Communication-efficient federated learning with adaptive quantiza- tion,

    Y . Mao, Z. Zhao, G. Yan, Y . Liu, T. Lan, L. Song, and W. Ding, “Communication-efficient federated learning with adaptive quantiza- tion,” ACM Trans. Intell. Syst. Technol. , vol. 13, no. 4, pp. 1–26, Aug. 2022

    2022

  20. [28]

    FedFQ: Federated learning with fine-grained quantization,

    H. Li, W. Xie, H. Ye, J. Ma, S. Ma, and Y . Li, “FedFQ: Federated learning with fine-grained quantization,” arXiv: 2408.08977, 2024

    2024 arXiv

  21. [29]

    Distributed deep reinforcement learning based gradient quantization for federated learning enabled vehicle edge computing,

    C. Zhang, W. Zhang, Q. Wu, P. Fan, Q. Fan, J. Wang, and K. B. Letaief, “Distributed deep reinforcement learning based gradient quantization for federated learning enabled vehicle edge computing,” IEEE Internet Things J., to be published

  22. [30]

    Joint accuracy and latency optimization for quantized federated learning in vehicular networks,

    X. Zhang, W. Chen, H. Zhao, Z. Chang, and Z. Han, “Joint accuracy and latency optimization for quantized federated learning in vehicular networks,” IEEE Internet Things J. , vol. 11, no. 17, pp. 28 876–28 890, 2024

    2024

  23. [31]

    The algorithmic foundations of differential privacy,

    C. Dwork, A. Roth et al. , “The algorithmic foundations of differential privacy,” Foundations Trends Theor. Comput. Sci. , vol. 9, no. 3-4, pp. 211–407, 2014

    2014

  24. [32]

    A survey on security and privacy of federated learning,

    V . Mothukuri, R. M. Parizi, S. Pouriyeh, Y . Huang, A. Dehghantanha, and G. Srivastava, “A survey on security and privacy of federated learning,” Future Gener. Comput. Syst. , vol. 115, pp. 619–640, 2021

    2021

  25. [33]

    Differentially private federated learning: A client level perspective,

    R. C. Geyer, T. Klein, and M. Nabi, “Differentially private federated learning: A client level perspective,” arXiv: 1712.07557, 2017

    2017 arXiv

  26. [34]

    cpSGD: communication-efficient and differentially-private distributed SGD,

    N. Agarwal, A. T. Suresh, F. Yu, S. Kumar, and H. B. McMahan, “cpSGD: communication-efficient and differentially-private distributed SGD,” in Proc. Int. Conf. Neural Inf. Process. Syst. , vol. 31, Dec. 2018, pp. 7575–7586

    2018

  27. [35]

    The Skellam mechanism for differentially private federated learning,

    N. Agarwal, P. Kairouz, and K. Z. Liu, “The Skellam mechanism for differentially private federated learning,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, Dec. 2021, pp. 5052–5064

    2021

  28. [36]

    The distributed discrete Gaussian mechanism for federated learning with secure aggregation,

    P. Kairouz, Z. Liu, and T. Steinke, “The distributed discrete Gaussian mechanism for federated learning with secure aggregation,” in Proc. Int. Conf. Mach. Learn. , vol. 139, Jul. 2021, pp. 5201–5212

    2021

  29. [37]

    Privacy for free: Communication- efficient learning with differential privacy using sketches,

    T. Li, Z. Liu, V . Sekar, and V . Smith, “Privacy for free: Communication- efficient learning with differential privacy using sketches,” arXiv: 1911.00972, 2019

    1911 arXiv

  30. [38]

    Joint privacy en- hancement and quantization in federated learning,

    N. Lang, E. Sofer, T. Shaked, and N. Shlezinger, “Joint privacy en- hancement and quantization in federated learning,” IEEE Trans. Signal Process., vol. 71, pp. 295–310, Jan. 2023

    2023

  31. [39]

    Randomized quantization is all you need for differential privacy in federated learning,

    Y . Youn, Z. Hu, J. Ziani, and J. Abernethy, “Randomized quantization is all you need for differential privacy in federated learning,” arXiv: 2306.11913, 2023

    2023 arXiv

  32. [40]

    vqSGD: Vector quantized stochastic gradient descent,

    V . Gandikota, D. Kane, R. K. Maity, and A. Mazumdar, “vqSGD: Vector quantized stochastic gradient descent,” IEEE Trans. Inf. Theory, vol. 68, no. 7, pp. 4573–4587, Jul. 2022

    2022

  33. [41]

    Machine learning with adversaries: Byzantine tolerant gradient descent,

    P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” in Proc. Int. Conf. Neural Inf. Process. Syst. , vol. 30, Dec. 2017, pp. 118–128

    2017

  34. [42]

    The hidden vulner- ability of distributed learning in Byzantium,

    E. M. El Mhamdi, R. Guerraoui, and S. Rouault, “The hidden vulner- ability of distributed learning in Byzantium,” in Proc. Int. Conf. Mach. Learn., vol. 80, Jul 2018, pp. 3521–3530

    2018

  35. [43]

    FABA: an algorithm for fast aggregation against Byzantine attacks in distributed neural networks,

    Q. Xia, Z. Tao, Z. Hao, and Q. Li, “FABA: an algorithm for fast aggregation against Byzantine attacks in distributed neural networks,” in Proc. Int. Joint Conf. Artif. Intell. , Jul. 2019, pp. 4824–4830

    2019

  36. [44]

    Byzantine-robust dis- tributed learning: Towards optimal statistical rates,

    D. Yin, Y . Chen, R. Kannan, and P. Bartlett, “Byzantine-robust dis- tributed learning: Towards optimal statistical rates,” in Proc. Int. Conf. Mach. Learn., vol. 80, Jul 2018, pp. 5650–5659

    2018

  37. [45]

    Robust aggregation for federated learning,

    K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust aggregation for federated learning,” IEEE Trans. Signal Process. , vol. 70, pp. 1142– 1154, Feb. 2022

    2022

  38. [46]

    DRACO: Byzantine-resilient distributed training via redundant gradients,

    L. Chen, H. Wang, Z. Charles, and D. S. Papailiopoulos, “DRACO: Byzantine-resilient distributed training via redundant gradients,” in Proc. Int. Conf. Mach. Learn. , vol. 80, Jul. 2018, pp. 902–911

    2018

  39. [47]

    DETOX: a redundancy-based framework for faster and more robust gradient aggregation,

    S. Rajput, H. Wang, Z. Charles, and D. Papailiopoulos, “DETOX: a redundancy-based framework for faster and more robust gradient aggregation,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 32, Dec. 2019

    2019

  40. [48]

    Byzantine-robust learning on heterogeneous datasets via bucketing,

    S. P. Karimireddy, L. He, and M. Jaggi, “Byzantine-robust learning on heterogeneous datasets via bucketing,” arXiv: 2006.09365, 2022

    2006 arXiv

  41. [49]

    Byzantine-robust learning on heterogeneous data via gradient splitting,

    Y . Liu, C. Chen, L. Lyu, F. Wu, S. Wu, and G. Chen, “Byzantine-robust learning on heterogeneous data via gradient splitting,” in Proc. Int. Conf. Mach. Learn., vol. 202, Jul 2023, pp. 21 404–21 425

    2023

  42. [50]

    Shielding federated learning: Robust aggregation with adaptive client selection,

    W. Wan, S. Hu, j. Lu, L. Y . Zhang, H. Jin, and Y . He, “Shielding federated learning: Robust aggregation with adaptive client selection,” in Proc. Int. Joint Conf. Artif. Intell. , Jul. 2022, pp. 753–760

    2022

  43. [51]

    Learning from history for Byzantine robust optimization,

    S. P. Karimireddy, L. He, and M. Jaggi, “Learning from history for Byzantine robust optimization,” in Proc. Int. Conf. Mach. Learn. , vol. 139, Jul 2021, pp. 5311–5319

    2021

  44. [52]

    RSA: Byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets,

    L. Li, W. Xu, T. Chen, G. B. Giannakis, and Q. Ling, “RSA: Byzantine- robust stochastic aggregation methods for distributed learning from heterogeneous datasets,” in Proc. AAAI Conf. Artif. Intell. , vol. 33, Jan. 2019, pp. 1544–1551

    2019

  45. [53]

    Federated two-stage learning with sign-based voting,

    Z. Ma, Z. Lu, Y . Lu, W. Li, J. Yi, and S. Cui, “Federated two-stage learning with sign-based voting,” arXiv: 2112.05687, 2021

    2021 arXiv

  46. [54]

    Stochastic sign descent methods: New algorithms and better theory,

    M. Safaryan and P. Richtarik, “Stochastic sign descent methods: New algorithms and better theory,” in Proc. Int. Conf. Mach. Learn., vol. 139, Jul 2021, pp. 9224–9234

    2021

  47. [55]

    Bridging differential privacy and Byzantine- robustness via model aggregation,

    H. Zhu and Q. Ling, “Bridging differential privacy and Byzantine- robustness via model aggregation,” in Proc. Int. Joint Conf. Artif. Intell., Jul. 2022, pp. 2427–2433

    2022

  48. [56]

    Federated learning with ℓ1 regularization,

    Y . Shi, Y . Zhang, P. Zhang, Y . Xiao, and L. Niu, “Federated learning with ℓ1 regularization,” Pattern Recognit. Lett., vol. 172, pp. 15–21, Jun. 2023

    2023

  49. [57]

    Mag- nitude matters: Fixing signSGD through magnitude-aware sparsification and error feedback in the presence of data heterogeneity,

    R. Jin, X. He, C. Zhong, Z. Zhang, T. Q. S. Quek, and H. Dai, “Mag- nitude matters: Fixing signSGD through magnitude-aware sparsification and error feedback in the presence of data heterogeneity,” IEEE Trans. Signal Process., vol. 72, pp. 5140–5155, Sep. 2024

    2024

  50. [58]

    Rate distortion for model compression:From theory to practice,

    W. Gao, Y .-H. Liu, C. Wang, and S. Oh, “Rate distortion for model compression:From theory to practice,” in Proc. Int. Conf. Mach. Learn. , vol. 97, Jun 2019, pp. 2102–2111

    2019

  51. [59]

    Deep learning with differential privacy,

    M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Tal- war, and L. Zhang, “Deep learning with differential privacy,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., Oct. 2016, pp. 308–318. SUPPLEMENTARY MATERIAL : ONE-BIT MODEL AGGREGATION FOR DIFFERENTIALLY PRI...

    2016

  52. [60]

    We first consider the estimation of the parameter θ in the i-th dimension, i.e., θi. Taking the logarithm of the joint probability function (12) yields log P(c1 i , c2 i , · · ·, cM i ; θi) = Ni 1 2 + θi 2b + (M − Ni) 1 2 − θi 2b = 1 2 M 1 − θi b + N 2θi b = 1 2 M + θi b (2Ni ...

  53. [61]

    The expectation of ˆθi is calculated as E h ˆθi i = 2E [Ni] − M M bi = 2M P(cm i = 1) − M M bi = 2 1 2 + 1 2bi θi − 1 bi (14) = θi, where (14) comes from (10)

  54. [62]

    1 M 2 MX m=1 I {cm i = 1} + X i∈B I {zm i > cm i } − X i∈B I {zm i < cm i } ! − M ! bi #2 − θi 2 = E

    We first need to calculate two key statistics, including E [Ni] = MX m=1 P (cm i = 1) = MX m=1 E [P (cm i = 1|δm i )] = MX m=1 E bi + δm i 2bi = M 2 1 + E [δm i ] bi = M 2 1 + θi bi E N 2 i = E " MX m=1 I2 {cm i = 1} + X i̸=j I {cm i = 1} I cm j = 1   = MX m=1 E [I {cm i = 1...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.