REVIEW 4 major objections 6 minor 20 references
RepuNet: A Reputation System for Mitigating Malicious Clients in DFL
T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper claims that local reputation scoring from four observable metrics can detect and mitigate model poisoning, delay, and flooding in decentralized federated learning, with reported F1 scores above 95% for MNIST and around 76% for…
desk verdict The abstract's F1 numbers don't appear in any table and the main poisoning comparison mixes rounds, so the central effectiveness claim is not yet supported; the design is novel enough to warrant a serious revised submission. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
RepuNet's central object is a per-neighbor reputation score computed inside each node every round. The score combines four normalized metrics: model similarity (a weighted average of cosine, Euclidean, Manhattan, and Pearson comparisons between the local and received models), fraction of parameters changed (a sigmoid penalty on deviations from the node's own history), model arrival latency (a sigmoid penalty when current latency exceeds the historical mean), and incoming message flow (an exponential penalty when message volume rises above a dynamic margin). Each metric is mapped to [0,1], dynamic weights are set in proportion to how far that metric deviates from historical behavior, and the final reputation is a weighted combination of the reputation history and the current score. Optional neighbor feedback, averaged and blended with the local score, can refine the estimate. Reputations below a threshold (typically 0.6) cause the sender's model to be discarded; otherwise the model is aggregated with weight proportional to reputation, and a node can be re-admitted if its behavior improves in later rounds.
What would settle it
A concrete experiment that would settle the central claim is an adaptive poisoning attack in which malicious nodes craft updates whose model similarity and parameter-change fraction remain within the historical mean plus one standard deviation of honest updates while still degrading the global model. If RepuNet's detection F1 then drops to near the no-defense baseline, the claim that these four metrics can detect poisoning is refuted; if it stays above 90% on MNIST, the claim survives.
Extended reading notes
Core claim
The paper's central claim is that three classes of attacks leave locally measurable traces, and that a reputation score built from those traces can protect the federation without a central authority. During model poisoning, the received model diverges from honest updates, so similarity falls and the fraction of changed parameters rises; during delay attacks, arrival latency exceeds historical patterns; during flooding, message counts jump above a dynamic margin. RepuNet turns each trace into a normalized score, weights the metrics by how anomalous they are, and excludes or down-weights models from low-reputation neighbors. Its own experiments report that malicious nodes' reputations drop sharply within a round or two after an attack starts, while honest nodes stay near the top of the scale, and that in most poisoning scenarios the global F1 with RepuNet is substantially higher than without it. Using MNIST and CIFAR-10 under non-IID partitions, the paper reports detection F1 scores above 95% for MNIST and about 76% for CIFAR-10.
Load-bearing premise
That malicious behavior reliably changes the four measured metrics enough to cross the fixed thresholds, and that the optional reputation-feedback channel cannot be flooded by colluding nodes because only nodes with an active reputation module issue evaluations.
Editorial extensions
If this is right
- Reputation-based aggregation can replace binary filtering in DFL: low-reputation models are down-weighted or excluded, while accepted models retain a vote proportional to reputation.
- Because reputation is recomputed every round and exclusion is not disconnection, a node that returns to honest behavior can be progressively re-admitted, so transient failures need not be permanent.
- The system runs without blockchain or a central coordinator, keeping per-node overhead low enough for federations of tens of nodes.
- Across poisoning scenarios, the reported gap in F1 with versus without RepuNet reaches over 0.25 in many cases, and the largest gains appear when attacks start early or the malicious fraction is high.
Reading between the lines
- This is an extension, not a paper claim: an adaptive adversary that keeps model similarity and parameter-change fraction inside the honest historical band while still injecting poisoned labels would test whether the four metrics are truly sufficient; RepuNet's fixed reputation threshold would likely need revision if such an attacker evades it.
- Another testable consequence: if the same four metrics were used for peer selection in addition to aggregation weighting, the reputation could also steer which neighbors are worth connecting to in sparse or dynamic topologies, something RepuNet does not currently do.
- The CIFAR-10 detection gap (76% versus over 95% on MNIST) suggests that data heterogeneity, not attack strength, may set the practical limit on metric-based detection; testing with harder non-IID splits or distribution-aware similarity baselines would sharpen that boundary.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. RepuNet is a decentralized reputation system for Decentralized Federated Learning (DFL) that scores each neighboring node from four locally observable metrics: model similarity, fraction of parameters changed, model arrival latency, and incoming message flow. The reputation score determines whether a received model is excluded from aggregation and how it is weighted, with optional distributed feedback from peers. The system is integrated into the Nebula platform and evaluated on MNIST and CIFAR-10 under non-IID data partitions, fully connected and random topologies, and three attack families (model poisoning, delay, flooding), with claimed F1 improvements over an undefended baseline.
Significance. If the central claims were properly evidenced, RepuNet would be a practically attractive defense for DFL: it avoids blockchain overhead, is locally computable, and addresses communication-side attacks that are often neglected. The paper is also commendable for implementing the system in a real platform (Nebula), testing multiple topologies, attack intensities, and activation timings, and for reporting reputation trajectories and aggregation counts. However, the quantitative evidence as presented is not yet adequate: the headline F1 numbers in the abstract are not traceable to any table, the main poisoning comparison mixes evaluation rounds, and the delay/flooding evaluations do not compare final model quality against a no-defense baseline. These issues are correctable with additional experiments and careful re-reporting, so the contribution remains plausible but unproven in its current form.
major comments (4)
- [Abstract vs. Section V-B, Table VI] The abstract states that RepuNet achieves 'F1 scores above 95% for MNIST scenarios and approximately 76% for CIFAR-10 cases', but no such values appear in Table VI or elsewhere in the body: the best MNIST F1 with RepuNet reported is 0.6879 (scenario 7.2), and the CIFAR-10 rows (5.1–5.3) report 0.5360, 0.5122, and 0.5166. If the abstract refers to a different quantity (e.g., malicious-node detection F1), that quantity is never defined or reported, so the headline claim cannot be verified.
- [Table VI, Section V-B] The comparison in Table VI is confounded by different evaluation rounds: the 'F1 w/rep (r8)' column is measured at round 8 while the 'F1 w/o rep (r11)' column is measured at round 11. Since the undefended model continues training for three additional rounds, the reported ΔF1 conflates RepuNet's defensive effect with extra training progress. The authors need to report both systems at the same round (or otherwise show that the round difference does not materially affect the comparison).
- [Sections V-C and V-D] For the delay and flooding attacks, the evaluation reports only reputation trajectories (Figures 5, 6, 9), aggregated-model counts (Figures 8, 10), and CPU usage (Table XI); there is no global-model F1 or accuracy comparison against an undefended baseline. The paper's central claim that RepuNet 'mitigates' these attacks in terms of final model quality is therefore not quantitatively evidenced for two of the three attack types. The authors should either provide such baseline comparisons or explicitly limit their claims to reputation-based exclusion effects.
- [Sections IV-C, IV-D, IV-E and Table III] The system relies on several hand-chosen parameters—the exclusion threshold of 0.6 (Section IV-E), the 150% latency tolerance multiplier (Section IV-C.3), the 1.05 and 1.10 multipliers in Equation (4), the bootstrapping penalty δ=0.05 in Equation (13), and the feedback weight η in Equation (25)—but no sensitivity analysis is performed. Without evidence that the results are stable with respect to these parameters, the claim that RepuNet is adaptive rather than configuration-sensitive is not established. A parameter-sweep study for at least the exclusion threshold and latency multiplier would address this.
minor comments (6)
- [Section IV-F] The statement that 'only nodes with an active reputation module are allowed to issue evaluations' guarantees that all received scores originate from 'valid and trustworthy participants' is unsupported: an attacker can also run an active reputation module and send dishonest feedback. This should be clarified as a design assumption, and the feedback mechanism should be tested under collusion if it is claimed as a robustness feature.
- [Section IV-C.3] The definition of the latency tolerance threshold as '150% of the average latency' is stated only in prose; if this is a fixed multiplier, it should be included in the formal equations or in Table III to allow reproducibility.
- [Equation (17)] The term 'increase mean' in the amplification formula is not defined; the text says it 'adjusts automatically based on the system phase' but never specifies how. Please provide a formal definition.
- [Section IV-C.2] The notation μ_t and σ_t in Equations (3)–(4) is ambiguous: it is not clear whether these are thresholds per round, and how they are initialized when no history is available.
- [Figures and tables] Several figures and tables are referenced out of order (e.g., Figure 7 is cited before Figures 5 and 6, and Table X is mentioned in Section V-D but appears before Figure 9); the numbering should be re-sequenced to match the order of first mention.
- [Throughout] There are minor language issues, including 'delayer attack' (should likely be 'delay attack'), inconsistent hyphenation of 'F1-score' vs. 'F1 score', and the use of 'X' in Table I to denote negative/absent features, which is not explained in the caption.
Circularity Check
No significant circularity: the reputation-drop evidence is partly definitional, but the central mitigation claim rests on an independent (if imperfect) global-model F1 benchmark.
-
self definitional
[Section III (Threat Model); Section V-B (Poisoning definition); reputation equations in Section IV-D (Eqs. 23-24)]
"These attacks degrade model quality, synchronization, or communication, and can be detected using local metrics such as model similarity, latency, or message count. ... This attack degrades the global model by injecting malicious updates that differ significantly from honest contributions and may hinder convergence if undetected."
Eq. (23) defines the per-round score S_i as the weighted sum of the four metrics (model similarity, parameter-change fraction, arrival latency, incoming message flow), and Eq. (24) defines reputation R_i from that score and history. The Section III threat model defines the targeted attacks using those same observables: poisoning is an update that 'differs significantly from honest contributions' (low similarity), delay is intentionally late submissions (high latency), and flooding is 'excessive message generation' (high message volume). Consequently, the observation that malicious nodes' reputation drops after attack activation (Figs. 4, 5, 9) follows from the definitions; it is a consistency check of the implementation, not an independent detection test.
full rationale
RepuNet's local detection result is partly definitional: the reputation score is constructed from the same four metrics that define the three attack classes, so a reputation drop after an attack starts is a consequence of the design rather than an independent discovery. This is a genuine but minor circular component. It does not dominate the paper because the main effectiveness claim—improved global-model F1 relative to no defense—is an external benchmark whose outcome is not forced by the metric definitions. The evaluation has serious evidence-quality problems (with/without defense measured at different rounds, no table reproducing the abstract's 0.95/0.76 F1 values, no repeated-seed statistics), but those are verification weaknesses, not circularity. The threshold values (0.6 exclusion, 150% latency, 50% missing-model penalty) are hand-selected design choices rather than fitted parameters, so the fitted-input-called-prediction pattern does not apply. The paper's self-citations are background material and the Nebula platform reference, not a load-bearing uniqueness theorem. Overall, the circularity is limited to the definitional detection demonstration; the mitigation claim retains independent content.
Assumptions & free parameters
free parameters (7)
- Exclusion threshold =
0.6
- Latency tolerance multiplier =
1.5 (150% of average latency)
- Parameter-change threshold multipliers =
1.05 (fraction) and 1.10 (threshold)
- Missing-model penalty =
50% score reduction
- Bootstrapping penalty delta =
0.05
- Feedback weight eta =
eta in [0,1], value unspecified
- Smoothing and history weights =
unspecified
assumptions (4)
- domain assumption Malicious behavior is observable as local deviations in model similarity, parameter-change fraction, arrival latency, and message volume.
- domain assumption Only nodes with an active reputation module can issue reputation feedback, so feedback is trustworthy.
- ad hoc to paper A neutral initial reputation of 0.6 combined with an exclusion threshold of 0.6 is a sound bootstrap.
- domain assumption Emulation on a single machine with independent processes reproduces realistic DFL network conditions such as delays, flooding, and topology effects.
Cite this review
Pith. "Pith review of RepuNet: A Reputation System for Mitigating Malicious Clients in DFL." pith.science (2026). https://pith.science/paper/AFOTIWE7
@misc{pith2026250619892,
author = {Pith},
title = {Pith review of: RepuNet: A Reputation System for Mitigating Malicious Clients in DFL},
year = {2026},
howpublished = {\url{https://pith.science/paper/AFOTIWE7}},
note = {Machine review of arXiv:2506.19892}
}
read the original abstract
Decentralized Federated Learning (DFL) enables nodes to collaboratively train models without a central server, introducing new vulnerabilities since each node independently selects peers for model aggregation. Malicious nodes may exploit this autonomy by sending corrupted models (model poisoning), delaying model submissions (delay attack), or flooding the network with excessive messages, negatively affecting system performance. Existing solutions often depend on rigid configurations or additional infrastructures such as blockchain, leading to computational overhead, scalability issues, or limited adaptability. To overcome these limitations, this paper proposes RepuNet, a decentralized reputation system that categorizes threats in DFL and dynamically evaluates node behavior using metrics like model similarity, parameter changes, message latency, and communication volume. Nodes' influence in model aggregation is adjusted based on their reputation scores. RepuNet was integrated into the Nebula DFL platform and experimentally evaluated with MNIST and CIFAR-10 datasets under non-IID distributions, using federations of up to 25 nodes in both fully connected and random topologies. Different attack intensities, frequencies, and activation intervals were tested. Results demonstrated that RepuNet effectively detects and mitigates malicious behavior, achieving F1 scores above 95% for MNIST scenarios and approximately 76% for CIFAR-10 cases. These outcomes highlight RepuNet's adaptability, robustness, and practical potential for mitigating threats in decentralized federated learning environments.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[1]
Communication-efficient learning of deep networks from decentralized data,
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017, pp. 1273–1282
work page 2017
-
[2]
E. T. Mart ´ınez Beltr ´an, M. Q. P ´erez, P. M. S. S ´anchez, S. L. Bernal, G. Bovet, M. G. P ´erez, G. M. P ´erez, and A. H. Celdr ´an, “Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges,”IEEE Communications Surveys & Tutorials, vol. 25, no. 4, pp. 2983–3013, 2023
work page 2023
-
[3]
Detection of false data injection attacks in smart grid: A secure federated deep learning approach,
Y . Li, X. Wei, Y . Li, Z. Dong, and M. Shahidehpour, “Detection of false data injection attacks in smart grid: A secure federated deep learning approach,”IEEE Transactions on Smart Grid, vol. 13, no. 6, pp. 4862– 4872, 2022
work page 2022
-
[4]
Delay- aware hierarchical federated learning,
F. P.-C. Lin, S. Hosseinalipour, N. Michelusi, and C. G. Brinton, “Delay- aware hierarchical federated learning,”IEEE Transactions on Cognitive Communications and Networking, vol. 10, no. 2, pp. 674–688, April 2024
work page 2024
-
[5]
FLEAM: A federated learning empowered architecture to mitigate DDoS in industrial IoT,
J. Li, L. Lyu, X. Liu, X. Zhang, and X. Lyu, “FLEAM: A federated learning empowered architecture to mitigate DDoS in industrial IoT,” IEEE Transactions on Industrial Informatics, vol. 18, no. 6, pp. 4059– 4068, 2022
work page 2022
-
[6]
HybridChain: Fast, accurate, and secure transaction processing with distributed learning,
A. Taherpour and X. Wang, “HybridChain: Fast, accurate, and secure transaction processing with distributed learning,”IEEE Transactions on Parallel and Distributed Systems, vol. 35, no. 6, pp. 968–982, 2024
work page 2024
-
[8]
E. Gabrielli, D. Belli, Z. Matrullo, V . Miori, and G. Tolomei, “Protecting federated learning from extreme model poisoning attacks via multidimensional time series anomaly detection,” 2024. [Online]. Available: https://doi.org/10.48550/arXiv.2303.16668
-
[9]
Sok: Secure aggregation based on cryptographic schemes for federated learning,
M. Mohamad, M. ¨Onen, W. Ben Jaballah, and M. Conti, “Sok: Secure aggregation based on cryptographic schemes for federated learning,” in Proceedings on Privacy Enhancing Technologies, 2023, pp. 140–157
work page 2023
Show all 20 references
-
[10]
Nebula: A platform for decentralized federated learn- ing,
CyberDataLab, “Nebula: A platform for decentralized federated learn- ing,” https://github.com/CyberDataLab/nebula, 2025, accessed: 2025-06- 13
2025
-
[11]
Reputation-based federated learning for secure wireless networks,
Z. Song, H. Sun, H. H. Yang, X. Wang, Y . Zhang, and T. Q. S. Quek, “Reputation-based federated learning for secure wireless networks,” IEEE Internet of Things Journal, vol. 9, no. 2, pp. 1212–1226, 2022
2022
-
[12]
Incentive mech- anism for reliable federated learning: A joint optimization approach to combining reputation and contract theory,
J. Kang, Z. Xiong, D. Niyato, S. Xie, and J. Zhang, “Incentive mech- anism for reliable federated learning: A joint optimization approach to combining reputation and contract theory,”IEEE Internet of Things Journal, vol. 6, no. 6, pp. 10 700–10 714, 2019
2019
-
[13]
Privacy-preserving blockchain-based federated learning for IoT devices,
Y . Zhao, J. Zhao, L. Jiang, R. Tan, D. Niyato, Z. Li, L. Lyu, and Y . Liu, “Privacy-preserving blockchain-based federated learning for IoT devices,” 2021. [Online]. Available: https://doi.org/10.48550/arXiv. 1906.10893
-
[14]
Secure and privacy-preserving federated learning via co-utility,
J. Domingo-Ferrer, A. Blanco-Justicia, J. Manj ´on, and D. S ´anchez, “Secure and privacy-preserving federated learning via co-utility,”IEEE Internet of Things Journal, vol. 9, no. 5, pp. 3988–4000, 2021
2021
-
[15]
A reputation-aware hierar- chical aggregation framework for federated learning,
M. Panigrahi, S. Bharti, and A. Sharma, “A reputation-aware hierar- chical aggregation framework for federated learning,”Computers and Electrical Engineering, vol. 111, p. 108900, 2023
2023
-
[16]
FGFL: A blockchain-based fair incentive governor for federated learning,
L. Gao, L. Li, Y . Chen, C. Xu, and M. Xu, “FGFL: A blockchain-based fair incentive governor for federated learning,”Journal of Parallel and Distributed Computing, vol. 163, pp. 283–299, 2022
2022
-
[17]
Poisoning attacks on federated learning-based IoT intrusion detection system,
T. Nguyen, P. Rieger, M. Miettinen, and A.-R. Sadeghi, “Poisoning attacks on federated learning-based IoT intrusion detection system,” in Workshop on Decentralized IoT Systems and Security, 2020, pp. 1–7
2020
-
[18]
Fed- erated learning-based in-network traffic analysis on IoT edge,
M. Zang, C. Zheng, T. Koziak, N. Zilberman, and L. Dittmann, “Fed- erated learning-based in-network traffic analysis on IoT edge,” in2023 IFIP Networking Conference (IFIP Networking), 2023, pp. 1–6
2023
-
[19]
Decentralized federated learning: A survey on security and privacy,
E. Hallaji, R. Razavi-Far, M. Saif, and Q. Yang, “Decentralized federated learning: A survey on security and privacy,”IEEE Transactions on Big Data, vol. 10, pp. 194–213, 2024
2024
-
[20]
Secure and efficient federated learning through layering and sharding blockchain,
S. Yuan, B. Cao, Y . Sun, Z. Wan, and M. Peng, “Secure and efficient federated learning through layering and sharding blockchain,”IEEE Transactions on Network Science and Engineering, vol. 11, no. 3, pp. 3120–3134, 2024
2024
-
[21]
Fedeval: Defending against lazybone attack via multi-dimension evaluation in federated learning,
H. Wang, H. Zhang, L. Wang, S. Xuan, and Q. Zhang, “Fedeval: Defending against lazybone attack via multi-dimension evaluation in federated learning,”ACM Transactions on Sensor Networks, vol. 21, no. 1, pp. 1–23, 2025. Isaac Marroqu´ı Penalvareceived his M.Sc. degree in Softwar...
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.