REVIEW 3 major objections 5 minor 33 references
Bayesian Robust Aggregation for Federated Learning
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A Bayesian aggregation rule infers which federated-learning clients are honest, removing the need to specify the number of malicious clients.
desk verdict The stated Algorithm 1 never leaves the all-benign fixed point, so the paper's central robustness claim is unsupported as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the evidence lower bound of Equation (11): a Gaussian likelihood over the squared residual norms $\lVert w_k - w\rVert_2^2 / \sigma^2$, latent Bernoulli variables $b_k$ for client honesty, and a KL-divergence term whose prior $\varepsilon = M/K$ is itself optimized, making the procedure an empirical-Bayes estimate. The machinery is coordinate ascent: update per-client posterior probabilities $\pi_k$ with Equation (15), then re-estimate $w$ and $\sigma^2$ in closed form from Equations (14). This converts a combinatorial subset-selection problem, Equation (5), into $O(TK)$ iterations, and lets the posterior probabilities perform outlier rejection automatically, so the method adapts to a static or changing number of malicious clients without a user-supplied count.
What would settle it
Construct a communication round with $K=20$ fixed update vectors in which $M=4$ malicious vectors lie just inside the convex hull of the honest ones, run Algorithm 1, and check whether the learned probabilities $\pi_k$ assign them near-zero weight and whether the output $w$ matches the honest mean. If the ELBO optimum is pulled toward the malicious vectors, then the variational solution is not $(M,\kappa)$-robust even though Proposition 2.1 applies to the combinatorial estimator.
Extended reading notes
Core claim
The central claim is that maximizing the marginalized likelihood, with each client's honesty represented by a latent Bernoulli variable, defines a robust mean update whose robustness does not depend on knowing the number of compromised clients. Algorithm 1 maximizes the corresponding ELBO by alternating closed-form updates: the posterior probability $\pi_k$ for each client from Equation (15), the mean $w$ as the $\pi$-weighted average of updates, and the scale $\sigma^2$ as the $\pi$-weighted mean squared residual. The paper proves that the hard subset-selection estimator being relaxed satisfies an $(M,\kappa)$-robustness bound (Proposition 2.1), and it reports empirically that the variational solution matches or exceeds Krum's test accuracy and backdoor attack success rate on MNIST, Fashion-MNIST, and CIFAR-10, with malicious fractions up to $45\%$ that may vary across rounds.
Load-bearing premise
The load-bearing premise is that the variational ELBO solution computed by Algorithm 1 inherits the $(M,\kappa)$-robustness that is proved only for the hard subset-selection estimator of Equation (5) in Appendix C; no proof of the transfer is given, and the Gaussian likelihood over squared residual norms is the heuristic that makes the transfer plausible.
Editorial extensions
If this is right
- On CIFAR-10 under sign flipping and backdoor attacks with 40% malicious clients, the method keeps test accuracy within about 1% of the non-adversarial baseline, a level reached only by Krum among the compared defenses.
- In all three configurations, the method suppresses backdoor attack success rate to at most 8% on CIFAR-10, while Median, Trimmed Mean, and Geometric Median leave attack success rates near 30–100%.
- The method requires no hyperparameter for the number of malicious clients, unlike Krum's $L$ and Trimmed Mean's $\beta$, and it tracks a dynamically changing adversarial share by estimating $\varepsilon$ at every round.
- In a fully benign setting the method's accuracy matches Federated Averaging, so robustness does not come at a measurable utility cost in the tested configurations.
- Because each round only computes weighted means and scalar updates, the aggregation cost stays $O(TK)$ and remains comparable to coordinate-wise median.
Reading between the lines
- The paper proves the robustness bound only for the hard subset-selection estimator of Equation (5); a targeted test of whether the ELBO optimum still satisfies the bound when malicious updates sit just inside the convex hull of honest updates would settle this implicit transfer step.
- Because the prior share $\varepsilon$ is re-estimated every round, the method could double as an attack-monitoring tool by reporting the inferred fraction of compromised clients over time, a use the authors do not develop.
- The scalar-residual Gaussian likelihood throws away per-coordinate information, so a structured extension that respects the KL scale issue noted in Section 4.3 may detect stealthy attacks localized in a few coordinates.
- An untested stress direction is stronger data heterogeneity, since the experiments use Dirichlet $\alpha = 1.0$ and $0.5$; much smaller values could break the single-cluster assumption about honest updates.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayesian robust aggregation rule for federated learning in which the global update is computed as a weighted average of client models, with weights interpreted as posterior probabilities of client honesty. The authors derive the method by first formulating robust aggregation as a hard subset-selection problem (Eq. 5), prove an (M,κ)-robustness bound for that problem (Proposition 2.1), and then relax it into a variational objective (ELBO) over latent Bernoulli indicators of honesty (Eqs. 9-11), optimized by the EM-style Algorithm 1. The paper claims the resulting method is adaptive to an unknown or varying number of malicious clients, requires no user-specified hyperparameter such as the number of adversaries, and matches Krum's accuracy and backdoor-attack resistance on MNIST, Fashion-MNIST, and CIFAR-10. The empirical section compares the method against FEDAVG, Median, Trimmed Mean, Geometric Median, and Krum under four attack types and static/dynamic adversary configurations.
Significance. If the algorithm worked as specified, the paper would make a valuable practical contribution: a Byzantine-robust aggregation rule that avoids the common requirement of knowing the number of compromised clients, has simple weighted-mean updates, is backed by a theoretical robustness bound, and is evaluated on a broad benchmark (three datasets, four attacks, static and dynamic adversaries). The paper also provides public code and an ablation study, and the careful comparison against multiple baselines is a clear strength. However, the significance is heavily contingent on the algorithm actually implementing the described inference; as written, the central algorithm does not do so, which substantially weakens the paper's claims.
major comments (3)
- [Section 2, Algorithm 1, Eq. (15)] Algorithm 1 as written is degenerate. It initializes π_k = 1 for all k (line 2), so Σπ_old = K and the factor (K − Σπ_old)/Σπ_old in the π-update of Eq. (15) equals 0 at the first iteration. Consequently, π_new_k = 1 for every k regardless of the likelihood p(w_k|w,σ²). The subsequent updates of w and σ² (lines 7–8) are then exactly the unweighted mean and variance, and the same argument repeats at every iteration: (1,...,1) is a fixed point of the EM map. Thus, Algorithm 1 as written reduces to FEDAVG and cannot adapt to any static or dynamic attack. This contradicts the central robustness claim and the probabilities shown in Figure 1. The empirical results in Tables 2–4 cannot be attributed to the procedure specified in the paper unless the implementation uses an unstated initialization (e.g., π_k < 1 or ε > 0), which would be an essential undocumented component. Please correct the algorithm or explicitly state the actual initialization and update scheme used to generate the reported results.
- [Section 2, Eqs. (9)–(11) and Proposition 2.1] The robustness guarantee in Proposition 2.1 is proven for the hard subset-selection estimator in Eq. (5), which enforces a hard cardinality constraint |S| = K − M. The actual algorithm optimizes the ELBO in Eq. (11), a variational relaxation with a soft prior over indicators and an empirically estimated ε given by Eq. (12). No argument is given that the solution of Eq. (11) inherits the (M,κ)-robustness of Proposition 2.1. The sentence following Eq. (13) only motivates the Gaussian likelihood by its appearance in Eq. (5); it does not establish that the variational estimator satisfies an analogous bound. The paper should either prove that the Algorithm 1 output satisfies a similar robustness guarantee with respect to a suitable set of benign clients, or clearly state that Proposition 2.1 applies only to the intermediate combinatorial problem and that the robustness of Algorithm 1 rests solely on the empirical evaluation.
- [Appendix C, Eq. (25) vs. main text Eq. (6)] There is a technical inconsistency between the statement of Proposition 2.1 and its proof. The main-text bound in Eq. (6) involves a sum of norms, specifically (κ/(K−M)) Σ_{k∈B} ∥w_k − w_B∥₂, while the appendix proof concludes in Eq. (25) a bound with Σ_{k∈B} ∥w_k − w_B∥₂² (squared norms) and a different constant. In addition, Eq. (20) of the appendix appears to mix squared and unsquared norms. Please reconcile the theorem statement with the proof and verify which bound actually follows from the argument.
minor comments (5)
- [Abstract, Section 2, Algorithm 1] The method is described as 'parameter-free' and 'hyperparameter-free', but Algorithm 1 depends on the initialization of π_k and σ², the EM convergence criterion, and the maximum number of iterations T. Please qualify this claim and state the exact values used in the experiments.
- [Figure 1] The lower heatmap in Figure 1 shows the estimated benign probabilities varying across clients and rounds, which is inconsistent with the all-ones posterior produced by Algorithm 1 as written. The figure and its caption should be updated to describe what the actual implementation computes.
- [Section 2, Eq. (8) to Eq. (9)] The transition from Eq. (8) to Eq. (9) is described as a relaxation, but the expectation under the prior p(b) with ε = M/K is not equivalent to the cardinality-constrained maximum in Eq. (8). Please clarify the precise relationship between the two objectives.
- [Section 2, Computational complexity] The claim that Algorithm 1 'converges to a local maximum of the ELBO' is not supported by a derivation of Eq. (15) as a coordinate-ascent step for the ELBO in Eq. (11); the update is simply cited to (Karakulev et al., 2024). Please provide the derivation or a precise reference to the convergence result.
- [Abstract, Tables 7–8] The abstract states that the method 'consistently achieves state-of-the-art performance', but the appendix results show cases where the method is below Krum (e.g., FMNIST Random attack at 40% malicious, ACC 0.83 vs. 0.89) and otherwise at parity. Please align the abstract's claim with the reported results.
Circularity Check
Algorithm 1's own initialization makes Eq. (15) the identity map, reducing the claimed robust aggregation to FEDAVG by construction.
-
other
[Algorithm 1 (lines 2 and 6), Equations (12) and (15), Section 2]
"Algorithm 1 (lines 2 and 6): 'πk = 1 for k = 1,...,K' and 'update π← with Equation (15) using w and σ2'. Eq. (12): 'ε = 1−Σπ_k/K'. Eq. (15): 'πnew_k = [1 + (K−Σπ_old)/(Σπ_old) · 1/p(w_k|w,σ2)]^{-1}'."
At initialization Σπ_old=K, so the prefactor (K−Σπ_old)/Σπ_old in Eq. (15) is 0, giving π_new=1 for every k regardless of the likelihood. The subsequent w- and σ²-updates (Algorithm 1 lines 7–8) are therefore exactly the unweighted mean and variance; this repeats identically every iteration, so (1,...,1) is a fixed point of the EM map. Consequently the written procedure is FEDAVG and cannot adapt to static or dynamic attacks, contrary to the central claim. The robustness 'prediction' is not derived but is equivalent to the input average by construction; reproducing Tables 2–4 would require an unstated initialization (e.g., ε>0 or π≠1), an essential undocumented component.
full rationale
The paper's central claim is adaptive robustness to a varying number of Byzantine clients. The derivation chain starts from the combinatorial robust estimator Eq. (5) with Proposition 2.1, then relaxes it to the ELBO Eq. (11) and to Algorithm 1. The relaxation itself is a legitimate variational method, and the self-citation to Karakulev et al. (2024) is a published, independent framework rather than an unverified self-supporting theorem, so by itself it does not create circularity. The Proposition 2.1 bound, however, is proved for Eq. (5), not for the ELBO solution; the paper asserts rather than proves that the variational optimum inherits the (M,κ)-robustness, which is a gap but not a circular step. The decisive circular-by-construction issue is in Algorithm 1: with π initialized to 1, the factor (K−Σπ_old)/Σπ_old in Eq. (15) is 0 at every iteration, so π_new=1 for all clients. The weighted updates for w and σ² then coincide exactly with the unweighted sample mean and variance, i.e., FEDAVG. Hence the written algorithm cannot downweight any client, and the reported state-of-the-art accuracy/ASR in Tables 2–4 cannot be produced by the stated procedure; the adaptive robustness claim reduces to plain averaging by construction. An unstated change (e.g., π<1 initialization or a different ε handling) would be required, which is an essential undocumented component. This makes the central derivation self-defeating, warranting a score of 8.
Assumptions & free parameters
free parameters (3)
- Initial posterior probabilities π_k =
1 for all k
- EM convergence criterion / maximum iterations T =
not specified
- Gaussian variance σ² (per-round scale) =
estimated iteratively from data via Eq (14)
assumptions (3)
- domain assumption Benign client updates, after centering, are treated as Gaussian with common mean w and variance σ², applied to the scalar residuals ∥w_k−w∥² in Eq (13).
- standard math The posterior over latent honesty indicators factorizes across clients (mean-field variational approximation).
- ad hoc to paper The solution of the intractable marginalized objective in Eq (9) or the ELBO in Eq (11) is robust in the same sense as the subset-centroid estimator in Eq (5), for which Proposition 2.1 is proven.
Cite this review
Pith. "Pith review of Bayesian Robust Aggregation for Federated Learning." pith.science (2026). https://pith.science/paper/56TXS7RG
@misc{pith2026250502490,
author = {Pith},
title = {Pith review of: Bayesian Robust Aggregation for Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/56TXS7RG}},
note = {Machine review of arXiv:2505.02490}
}
read the original abstract
Federated Learning enables collaborative training of machine learning models on decentralized data. This scheme, however, is vulnerable to adversarial attacks, when some of the clients submit corrupted model updates. In real-world scenarios, the total number of compromised clients is typically unknown, with the extent of attacks potentially varying over time. To address these challenges, we propose an adaptive approach for robust aggregation of model updates based on Bayesian inference. The mean update is defined by the maximum of the likelihood marginalized over probabilities of each client to be `honest'. As a result, the method shares the simplicity of the classical average estimators (e.g., sample mean or geometric median), being independent of the number of compromised clients. At the same time, it is as effective against attacks as methods specifically tailored to Federated Learning, such as Krum. We compare our approach with other aggregation schemes in federated setting on three benchmark image classification data sets. The proposed method consistently achieves state-of-the-art performance across various attack types with static and varying number of malicious clients.
Figures
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity
Allouah, Y., Farhadkhani, S., Guerraoui, R., Gupta, N., Pinot, R., and Stephan, J. Fixing by mixing: A recipe for optimal byzantine ml under heterogeneity. In International Conference on Artificial Intelligence and Statistics, pp.\ 1232--1300. PMLR, 2023
work page 2023
-
[3]
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., and Shmatikov, V. How to backdoor federated learning. CoRR, abs/1807.00459, 2018. URL http://arxiv.org/abs/1807.00459
arXiv 2018
-
[4]
M., Guerraoui, R., and Stainer, J
Blanchard, P., El Mhamdi, E. M., Guerraoui, R., and Stainer, J. Machine Learning with Adversaries : Byzantine Tolerant Gradient Descent . In Advances in Neural Information Processing Systems , volume 30. Curran Associates, Inc., 2017
work page 2017
-
[5]
M., Kucukelbir, A., and McAuliffe, J
Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. Variational inference: A review for statisticians. Journal of the American statistical Association, 112 0 (518): 0 859--877, 2017
2017
-
[6]
Secureboost: A lossless federated learning framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., and Yang, Q. Secureboost: A lossless federated learning framework. IEEE intelligent systems, 36 0 (6): 0 87--98, 2021
work page 2021
-
[7]
R., Zhong, A., Harouni, A., Gentili, A., Abidin, A
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. Federated learning for predicting clinical outcomes in patients with covid-19. Nature medicine, 27 0 (10): 0 1735--1743, 2021
work page 2021
-
[8]
The mnist database of handwritten digit images for machine learning research [best of the web]
Deng, L. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29 0 (6): 0 141--142, 2012. doi:10.1109/MSP.2012.2211477
arXiv 2012
Show all 33 references
-
[9]
Fang, M., Cao, X., Jia, J., and Gong, N. Z. Local model poisoning attacks to byzantine-robust federated learning. In Proceedings of the 29th USENIX Conference on Security Symposium , SEC '20, pp.\ 1623--1640, USA, August 2020. USENIX Association
2020
-
[10]
and Chen, J
Fang, P. and Chen, J. On the vulnerability of backdoor defenses for federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 11800--11808, 2023
2023
-
[11]
Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top
Gorbunov, E., Horv \'a th, S., Richt \'a rik, P., and Gidel, G. Variance reduction is an antidote to byzantines: Better rates, weaker assumptions and communication compression as a cherry on the top. In The Eleventh International Conference on Learning Representations
-
[12]
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015. URL http://arxiv.org/abs/1512.03385
2015 arXiv
-
[13]
Huber, P. J. Robust statistical procedures. SIAM, 1996
1996
-
[14]
Customized federated learning for accelerated edge computing with heterogeneous task targets
Jiang, H., Liu, M., Yang, B., Liu, Q., Li, J., and Guo, X. Customized federated learning for accelerated edge computing with heterogeneous task targets. Computer Networks, 183: 0 107569, 2020
2020
-
[15]
Adaptive robust learning using latent bernoulli variables
Karakulev, A., Zachariah, D., and Singh, P. Adaptive robust learning using latent bernoulli variables. In Proceedings of the 41st International Conference on Machine Learning, pp.\ 23105--23122, 2024
2024
-
[16]
P., He, L., and Jaggi, M
Karimireddy, S. P., He, L., and Jaggi, M. Byzantine-robust learning on heterogeneous datasets via bucketing. In International Conference on Learning Representations
-
[17]
P., He, L., and Jaggi, M
Karimireddy, S. P., He, L., and Jaggi, M. Learning from History for Byzantine Robust Optimization . In ICML 2021 - Proceedings of International Conference on Machine Learning, 2021. URL https://arxiv.org/abs/2012.10333
2021 arXiv
-
[18]
and Hinton, G
Krizhevsky, A. and Hinton, G. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf
2009
-
[19]
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86 0 (11): 0 2278--2324, 1998. doi:10.1109/5.726791
1998 doi
-
[20]
MacKay, D. J. Information theory, inference and learning algorithms. Cambridge university press, 2003
2003
-
[21]
McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B. A. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.\ 1273--1282. PMLR, 2017
2017
-
[22]
Murphy, K. P. Machine learning: a probabilistic perspective. MIT press, 2012
2012
-
[23]
Pytorch: An imperative style, high-performance deep learning library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019
2019
-
[24]
M., and Harchaoui, Z
Pillutla, K., Kakade, S. M., and Harchaoui, Z. Robust Aggregation for Federated Learning . IEEE Transactions on Signal Processing, 70: 0 1142--1154, 2022. ISSN 1053-587X, 1941-0476. doi:10.1109/TSP.2022.3153135
2022
-
[25]
Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning
Shejwalkar, V., Houmansadr, A., Kairouz, P., and Ramage, D. Back to the drawing board: A critical evaluation of poisoning attacks on production federated learning. In 2022 IEEE Symposium on Security and Privacy (SP), pp.\ 1354--1371, 2022. doi:10.1109/SP46214.2022.9833647
2022
-
[26]
L., Jin, L., Liu, N., Li, S., Miao, D., Zhang, X., Ng, W
Teo, Z. L., Jin, L., Liu, N., Li, S., Miao, D., Zhang, X., Ng, W. Y., Tan, T. F., Lee, D. M., Chua, K. J., et al. Federated machine learning in healthcare: A systematic review on clinical applications and technical architecture. Cell Reports Medicine, 5 0 (2), 2024
2024
-
[27]
Privacy preservation in federated learning: An insightful survey from the gdpr perspective
Truong, N., Sun, K., Wang, S., Guitton, F., and Guo, Y. Privacy preservation in federated learning: An insightful survey from the gdpr perspective. Computers & Security, 110: 0 102402, 2021. ISSN 0167-4048. doi:https://doi.org/10.1016/j.cose.2021.102402. URL https://www.scienc...
2021
-
[28]
D., Mayer, R., and Jacobsen, H.-A
Woisetschläger, H., Erben, A., Marino, B., Wang, S., Lane, N. D., Mayer, R., and Jacobsen, H.-A. Federated learning priorities under the european union artificial intelligence act, 2024. URL https://arxiv.org/abs/2402.05968
2024 arXiv
-
[29]
Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms
Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. CoRR, abs/1708.07747, 2017. URL http://arxiv.org/abs/1708.07747
2017 arXiv
-
[30]
Edgefed: Optimized federated learning based on edge computing
Ye, Y., Li, S., Liu, F., Tang, Y., and Hu, W. Edgefed: Optimized federated learning based on edge computing. IEEE Access, 8: 0 209191--209198, 2020
2020
-
[31]
Byzantine-robust distributed learning: Towards optimal statistical rates
Yin, D., Chen, Y., Kannan, R., and Bartlett, P. Byzantine-robust distributed learning: Towards optimal statistical rates. In International conference on machine learning, pp.\ 5650--5659. Pmlr, 2018
2018
-
[32]
A3FL : Adversarially Adaptive Backdoor Attacks to Federated Learning
Zhang, H., Jia, J., Chen, J., Lin, L., and Wu, D. A3FL : Adversarially Adaptive Backdoor Attacks to Federated Learning . Advances in Neural Information Processing Systems, 36: 0 61213--61233, December 2023 a
2023
-
[33]
FLIP : A Provable Defense Framework for Backdoor Mitigation in Federated Learning , February 2023 b
Zhang, K., Tao, G., Xu, Q., Cheng, S., An, S., Liu, Y., Feng, S., Shen, G., Chen, P.-Y., Ma, S., and Zhang, X. FLIP : A Provable Defense Framework for Backdoor Mitigation in Federated Learning , February 2023 b
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.