REVIEW 5 major objections 4 minor 24 references
SecureFed: A Two-Phase Framework for Detecting Malicious Clients in Federated Learning
T0 review · 5 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read SecureFed claims that its two-phase anomaly detection and trust-weighted aggregation keep a federated model at 92.5% accuracy when 48% of clients launch poisoning attacks, versus 84.4% for vanilla averaging.
desk verdict A plausible server-side defense that is under-supported by a three-round MNIST experiment and built on a trust score whose gradient-magnitude term works against the tested attack. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the trust score and the learning zones it creates. The score $T_c = \alpha(1 - A_c/\tau^*) + \beta(1 - L_c/\max L) + \gamma G_c/\max G$ fuses three per-client signals: the PCA-based anomaly score $A_c$, the validation loss $L_c$ of a temporary model built from that client's update, and the update's gradient magnitude $G_c$. Clients with scores above a high threshold join Zone 1 and receive the highest aggregation weight; those between thresholds sit in Zone 2 with reduced weight; those below join Zone 3 and are progressively removed. The global update is formed by zone-weighted aggregation $W^{r+1} = \frac{\sum_{c\in C}\alpha_{z(c)}n_c W_c^r}{\sum_{c\in C}\alpha_{z(c)}n_c}$. The design work of this mechanism is to soften client exclusion: instead of dropping an update outright, it scales its influence, which is what the paper claims preserves benign performance while suppressing poisoning.
What would settle it
Run the Section V experiment with a non-IID MNIST partition (for example, each client sees only two digit classes) and 48% label-flipping clients; if honest clients routinely land in Zone 3, or malicious-client detection falls to near chance, the central claim is falsified.
Extended reading notes
Core claim
On its own terms, SecureFed is an architecture claim rather than a new statistical identity: the paper asserts that poisoning attacks in federated learning can be both detected and damped by coupling dimensionality reduction with trust-weighted aggregation. Phase 1 applies PCA to client updates, uses K-Means-based anomaly clustering on the reduced representations to form an anomaly score $A_c$, and calibrates a detection threshold $\tau^*$ on the reference dataset. Phase 2 constructs a temporary model from each update, evaluates its validation loss $L_c$ on the reference set, computes the gradient magnitude $G_c = \|\nabla W_c^r\|$, and forms the trust score $T_c = \alpha(1 - A_c/\tau^*) + \beta(1 - L_c/\max L) + \gamma G_c/\max G$. Thresholds on $T_c$ place each client in Zone 1, 2, or 3, and aggregation weights updates by a zone factor. The paper reports that this configuration keeps benign accuracy at 95.49%, lifts accuracy under 48% malicious clients from 84.42% (vanilla) to 92.50%, raises F1 from 0.84 to 0.92, and that its ablation study identifies synthetic validation as the largest contributor, since detection rate drops from 75% to 38.25% when it is removed.
Load-bearing premise
All three trust signals assume that an outlier PCA pattern or a low gradient magnitude marks an attacker, and that assumption has only been tested on IID MNIST with label-flipping attacks; the authors themselves note it becomes unreliable when client data is highly non-IID.
Editorial extensions
If this is right
- With 48% malicious clients, SecureFed's accuracy (92.50%) stays within 3 points of its benign baseline (95.49%), whereas vanilla FedAvg drops more than 11 points to 84.42%.
- The ablation study predicts that removing synthetic validation is the most damaging change: detection rate falls from 75% to 38.25%, so the reference-dataset check carries much of the defense.
- Replacing the graded trust score with a binary filter loses 3.23 accuracy points (89.27% versus 92.50%), which supports the paper's claim that soft zone weighting is preferable to hard exclusion.
- The framework does not require changing client-side training or communication, so it could be dropped into existing federated learning pipelines as a server-side module.
Reading between the lines
- A natural but untested corollary is that the trust score will misfire on non-IID data: an honest client with sparse or homogeneous local data may have both a small gradient and an outlying PCA pattern, exactly the signature the framework reads as malicious.
- Because the reference set is a different handwritten-digits corpus rather than data drawn from the training distribution, the detection threshold and validation losses depend on distribution shift; using a clean held-out subset of the training data would likely change the reported detection-rate numbers.
- Backdoor attacks are discussed as motivation but never tested; since backdoors are designed to leave primary-task accuracy intact, SecureFed's PCA and validation-loss signals should be expected to be weaker against them than against label flipping.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SecureFed, a two-phase framework for detecting and mitigating malicious clients in federated learning. In Phase 1, the server applies PCA to client weight updates, computes an anomaly score, and estimates a threshold using a so-called synthetic dataset. In Phase 2, each client's update is temporarily applied to the global model, evaluated on that dataset to obtain a validation loss, and combined with the anomaly score and the update's gradient magnitude into a trust score. Clients are then assigned to one of three learning zones with different aggregation weights. The authors report that with 48% malicious clients performing single-class label flipping on IID MNIST, SecureFed achieves 92.50% test accuracy versus 84.42% for FedAvg, with a detection rate of 75% in the ablation study. The paper claims improvements in precision, recall, and F1 without compromising benign accuracy.
Significance. If the claimed robustness results were reproducible and generalizable, a two-phase anomaly-detection-plus-trust-zone aggregation framework could be a useful addition to the FL defense literature. The paper does not ship code, proofs, or a reproducible experimental protocol, so its contribution is currently limited to a conceptual framework plus preliminary empirical observations. The idea of combining dimensionality reduction with server-side validation is not new, and the paper does not compare against the robust aggregation baselines it cites (Krum, median/trimmed mean, FLTrust, MAB-RFL). The framework does, however, propose an explicit trust score and zone-based weighting scheme, which could be a testable design if the mechanism were consistent with the threat model.
major comments (5)
- [Section IV-B2 (trust score formula)] The trust score assigns a positive weight to the gradient magnitude term: T_c = alpha*(1 - A_c/tau*) + beta*(1 - L_c/max L) + gamma*G_c/max G. Under the only attack tested (single-class label flipping on MNIST, Section V-A), a malicious client training on mislabeled data must move the decision boundary for an entire digit class, so its update can have a large L2 norm. The positive gamma*G_c/max G term therefore rewards exactly the kind of poisoned update this framework is designed to detect, and the claimed 92.50% accuracy is not tied to the trust mechanism as described. The authors need to justify this term theoretically and show empirically (e.g., per-zone statistics on malicious vs benign norms) that it does not counteract the anomaly and validation-loss terms.
- [Section V-A (experimental setup) and Table I] The experimental evaluation uses three global rounds, one dataset (MNIST), one IID partitioning scheme, and one attack type (single-class label flipping), with no repeated seeds, no error bars, and no statistical significance tests. These results cannot support the claim in Section VI that SecureFed 'consistently outperforms' vanilla FL, especially since the gap at 48% malicious (92.50% vs 84.42%) could be an artifact of the particular initialization or round count. The authors should run more rounds, multiple datasets, multiple random seeds, and report means and variances, and ideally compare with the robust aggregation methods cited in Section II-C.
- [Algorithm 1, lines 9-10 and Section IV-B1] The anomaly score function f_anomaly and the threshold estimation procedure tau* = Validate(D_s, {A_c}) are not defined. The text only mentions PCA and K-Means in Section V-A, but does not specify how the anomaly score is computed from the reduced-dimensional representation, how the threshold is selected, or how the 'synthetic' dataset D_s is used in validation. Without these definitions, the detection rate in Table II cannot be reproduced, so the main empirical claim is not independently verifiable as written.
- [Table II (ablation study)] The reported detection rate of 75% appears to be computed in-sample on the same runs used to train and evaluate SecureFed, but the methodology for computing this rate is not provided (e.g., whether it is the fraction of malicious clients assigned to Zone 3, or a different criterion). Moreover, the zone-weighted aggregation still includes Zone 2 and Zone 3 clients with weights alpha_2 and alpha_3, so the global accuracy could be high even when detection is imperfect. The paper should report the confusion matrix of zone assignments for benign and malicious clients, and clarify how the detection rate is defined, before claiming that the framework 'successfully reduces the effects of poisoned updates'.
- [Section VI (Discussion) and threat model] The paper concedes that the gradient-divergence assumption fails under non-IID data, but the problem is more basic: even in the IID setting, the trust score's dependence on gradient magnitude is not a sound indicator of trustworthiness under the tested label-flipping attack. Additionally, the 'synthetic dataset' D_s is actually a second handwritten-digits dataset (reference [23]), not a synthetic one, and the server's access to and labeling of this dataset is a non-trivial assumption that should be stated clearly. The central mechanism needs to be revised or its behavior under the actual attack distribution demonstrated before the claims in Table I can be considered supported.
minor comments (4)
- [Abstract and Section I (Introduction)] There are numerous typographical and grammatical errors, e.g., 'training cycle continues Based on' in the abstract, 'approhces' in Section IV-A, and inconsistent capitalization of 'SecureFed' in the abstract. The paper would benefit from careful proofreading.
- [Section II-C (Comparison with existing works)] The paper cites Krum, median/trimmed mean, FoolsGold, FLAME, and FLTrust as related defenses but never evaluates any of them in the experiments. At minimum, the discussion should explain why these baselines were not compared, or the experiments should include them.
- [Section IV-B2 and Figure 2] The hyperparameters alpha, beta, gamma, tau_high, tau_low, alpha_1, alpha_2, alpha_3, and the number of PCA components (described as 'retaining top-5 components' in Section V-A) are never reported in the paper. Without these values, the experiments cannot be reproduced.
- [Section V-B and Table I] The table reports precision, recall, and F1 for the aggregated model, but does not state what the positive class is in this classification context. It would be clearer to define these metrics with respect to the classification task (e.g., digit classification) rather than leaving the reader to infer the setting.
Circularity Check
Detection-rate claim is in-sample: τ* is calibrated on the same client updates it is then used to classify, making the reported 75% detection rate a resubstitution statistic; the 92.50% accuracy itself is not circular.
-
fitted input called prediction
[Section IV-B1 (threshold calibration, τ* = Validate(Ds,{Ac})) and Section V-D / Table II (Detection Rate)]
"To calibrate the detection process, a thresholdτ∗ is estimated using a synthetic validation dataset Ds. For the scope of this work, a standard dataset with similar feature vectors to the training dataset was used [23] in the place of synthetic data: τ∗ = Validate(Ds,{Ac})"
The threshold τ* is estimated from the same anomaly scores {Ac} of the current round (via Validate(Ds,{Ac})) and is then used in Eq. 4 to normalize Ac into the trust score that assigns clients to zones. Table II then reports a 75% 'Detection Rate' for SecureFed (Full) on the same simulated clients without any held-out split or separate attack round described. The score that determines 'malicious' is therefore calibrated on, and evaluated against, the identical set of client updates; the detection rate is an in-sample resubstitution statistic rather than an out-of-sample prediction of detecting unseen attackers.
full rationale
The trust-score construction itself is not circular in the sense of an equation reducing to its own output: the accuracy numbers in Table I are evaluated on an external MNIST-style setting and are not forced by construction. However, the paper's detection-rate evidence is in-sample. The threshold τ* used to normalize anomaly scores in Eq. 4 is calibrated via Validate(Ds,{Ac}) on the same clients being scored, and the 75% detection rate in Table II is computed on that same setup, with no held-out malicious set or separate attack round. This makes the detection claim a fitted-input-then-reported-as-result pattern. The paper's self-citations (e.g., [11]) are related-work citations and are not load-bearing, so they do not add circularity. The Section VI admission that gradient-divergence assumptions fail under non-IID data is a genuine limitation and a correctness risk, but it is not itself a circular step. Overall, the central accuracy claim retains independent content, while the detection-rate claim is partially circular/in-sample, giving a score of 6.
Assumptions & free parameters
free parameters (5)
- Trust score weights alpha, beta, gamma
- Zone thresholds tau_high and tau_low
- Zone weighting factors alpha1, alpha2, alpha3
- Anomaly threshold tau*
- Number of PCA components retained =
5
assumptions (5)
- standard math PCA captures the structure of client updates
- domain assumption K-Means clusters correspond to benign/malicious behavior
- domain assumption Synthetic validation dataset approximates the training distribution
- domain assumption IID data partitioning
- domain assumption Low gradient magnitude indicates malicious behavior
Cite this review
Pith. "Pith review of SecureFed: A Two-Phase Framework for Detecting Malicious Clients in Federated Learning." pith.science (2026). https://pith.science/paper/OYVLGUFL
@misc{pith2026250616458,
author = {Pith},
title = {Pith review of: SecureFed: A Two-Phase Framework for Detecting Malicious Clients in Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/OYVLGUFL}},
note = {Machine review of arXiv:2506.16458}
}
read the original abstract
Federated Learning (FL) protects data privacy while providing a decentralized method for training models. However, because of the distributed schema, it is susceptible to adversarial clients that could alter results or sabotage model performance. This study presents SecureFed, a two-phase FL framework for identifying and reducing the impact of such attackers. Phase 1 involves collecting model updates from participating clients and applying a dimensionality reduction approach to identify outlier patterns frequently associated with malicious behavior. Temporary models constructed from the client updates are evaluated on synthetic datasets to compute validation losses and support anomaly scoring. The idea of learning zones is presented in Phase 2, where weights are dynamically routed according to their contribution scores and gradient magnitudes. High-value gradient zones are given greater weight in aggregation and contribute more significantly to the global model, while lower-value gradient zones, which may indicate possible adversarial activity, are gradually removed from training. Until the model converges and a strong defense against poisoning attacks is possible, this training cycle continues Based on the experimental findings, SecureFed considerably improves model resilience without compromising model performance.
Figures
Reference graph
Works this paper leans on
-
[23]
Handwritten digits dataset (not in mnist),
“Handwritten digits dataset (not in mnist),” https://www.kaggle.com/ datasets/jcprogjava/handwritten-digits-dataset-not-in-mnist, 2021, ac- cessed: 2025-05-27
work page 2021
-
[1]
Towards federated learning at scale: System design,
K. Bonawitz et al., “Towards federated learning at scale: System design,” Proceedings of machine learning and systems, vol. 1, pp. 374–388, 2019
work page 2019
-
[2]
Advances and open problems in federated learning,
P. Kairouz et al., “Advances and open problems in federated learning,” Foundations and Trends in Machine Learning , vol. 14, no. 1–2, pp. 1– 210, 2021
work page 2021
-
[3]
How to backdoor federated learning,
E. Bagdasaryan et al. , “How to backdoor federated learning,” in Pro- ceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), 2020, pp. 2938–2948
work page 2020
-
[4]
Can you really backdoor federated learning?
X. Sun et al. , “Can you really backdoor federated learning?” arXiv preprint arXiv:2011.01767, 2020
arXiv 2011
-
[5]
Jolliffe, Principal Component Analysis , 2nd ed
I. Jolliffe, Principal Component Analysis , 2nd ed. Springer, 2002
work page 2002
-
[6]
Gradient similarity-based defense against model poi- soning attacks in federated learning,
L. Zhang et al. , “Gradient similarity-based defense against model poi- soning attacks in federated learning,” IEEE Transactions on Dependable and Secure Computing , 2022
work page 2022
-
[7]
Elsa: Secure aggregation for federated learning with malicious actors,
M. Rathee et al., “Elsa: Secure aggregation for federated learning with malicious actors,” in 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023, pp. 1961–1979
work page 2023
Show all 24 references
-
[8]
Challenges and future directions of secure federated learning: a survey,
K. Zhang et al. , “Challenges and future directions of secure federated learning: a survey,” Frontiers of computer science , vol. 16, pp. 1–8
-
[9]
Learning to detect malicious clients for robust federated learning,
S. Li et al. , “Learning to detect malicious clients for robust federated learning,” arXiv preprint arXiv:2002.00211 , 2020
2002 arXiv
-
[10]
Feddmc: Efficient and robust federated learning via detecting malicious clients,
X. Mu et al. , “Feddmc: Efficient and robust federated learning via detecting malicious clients,” IEEE Transactions on Dependable and Secure Computing, 2024
2024
-
[11]
Hierarchical federated learning based anomaly de- tection using digital twins for smart healthcare,
D. Gupta et al. , “Hierarchical federated learning based anomaly de- tection using digital twins for smart healthcare,” in 2021 IEEE 7th international conference on collaboration and internet computing (CIC). IEEE, 2021, pp. 16–25
2021
-
[12]
Hierarchical federated transfer learning and digital twin enhanced secure cooperative smart farming,
L. Praharaj et al. , “Hierarchical federated transfer learning and digital twin enhanced secure cooperative smart farming,” in 2023 IEEE Interna- tional Conference on Big Data (BigData). IEEE, 2023, pp. 3304–3313
2023
-
[13]
Securing llm workloads with nist ai rmf in the internet of robotic things,
H. Karim et al., “Securing llm workloads with nist ai rmf in the internet of robotic things,” IEEE Access, 2025
2025
-
[14]
Machine learning with adversaries: Byzantine tolerant gradient descent,
P. Blanchard et al. , “Machine learning with adversaries: Byzantine tolerant gradient descent,” in Advances in Neural Information Processing Systems, vol. 30, 2017
2017
-
[15]
Byzantine-robust distributed learning: Towards optimal statistical rates,
D. Yin et al. , “Byzantine-robust distributed learning: Towards optimal statistical rates,” in International Conference on Machine Learning , 2018, pp. 5650–5659
2018
-
[16]
The limitations of federated learning in sybil settings,
C. Fung et al., “The limitations of federated learning in sybil settings,” in Proceedings of the 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID) , 2020, pp. 301–316
2020
-
[17]
Flame: Taming backdoors in federated learning,
S. Ma et al. , “Flame: Taming backdoors in federated learning,” in USENIX Security Symposium , 2022, pp. 295–312
2022
-
[18]
Fltrust: Byzantine- robust federated learning via trust bootstrapping,
X. Cao, M. Fang, J. Liu, and N. Z. Gong, “Fltrust: Byzantine- robust federated learning via trust bootstrapping,” arXiv preprint arXiv:2012.13995, 2020
2012 arXiv
-
[19]
Differentially private pca in federated learning,
C. Zhao et al. , “Differentially private pca in federated learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 35, no. 10, 2021, pp. 9118–9126
2021
-
[20]
Shielding federated learning: Robust aggregation with adaptive client selection,
W. Wan et al. , “Shielding federated learning: Robust aggregation with adaptive client selection,” arXiv preprint arXiv:2204.13256 , 2022
2022 arXiv
-
[21]
Federated learning with anomaly detection via gradient and reconstruction analysis,
Z. Alsulaimawi, “Federated learning with anomaly detection via gradient and reconstruction analysis,” arXiv preprint arXiv:2403.10000 , 2024
2024 arXiv
-
[22]
Seaflame: Communication-efficient secure aggregation for federated learning against malicious entities,
J. Tang et al. , “Seaflame: Communication-efficient secure aggregation for federated learning against malicious entities,” IACR Transactions on Cryptographic Hardware and Embedded Systems , vol. 2025, no. 2, pp. 69–93, 2025
2025
-
[24]
Data Poisoning Attacks Against Federated Learning Systems,
V . Tolpegin et al., “Data Poisoning Attacks Against Federated Learning Systems,” arXiv e-prints, p. arXiv:2007.08432, Jul. 2020
2007 arXiv
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.