{"id":"78707527-0e72-4994-b62d-1ffac7ed21c1","arxiv_id":"2502.00587","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RKD defends federated learning against backdoor attacks by clustering client updates with cosine similarity and HDBSCAN, selecting median-like models, and distilling their ensemble into the global model.","lead":"Federated learning trains models across devices without sharing raw data, but malicious devices can secretly plant backdoors into the shared model. This paper proposes a defense, RKD, that clusters device updates by similarity to the global model, selects the most typical models, and distills them into a clean global model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 2's max-cosine 'benign' anchor is directly contradicted by the TSBA attack the paper claims to defeat: a scaled update maximizes cosine similarity to the global model and would be selected as trusted.","rationale":"The reader's weakest-assumption identification is exactly right: the trust anchor in Algorithm 2 is the load-bearing component, and it is not merely unproven but in tension with the paper's own attack descriptions. TSBA and adaptive attacks are specifically designed to produce updates that appear similar to benign behavior, while the defense's sole detection signal is cosine similarity to the global model. If the proposed test shows the malicious cluster is selected, the central claim collapses; if it shows the benign cluster is consistently selected, the concern is resolved. I do not change the reader's CONDITIONAL verdict because the needed evidence is missing rather than conclusively refuted, but the condition should explicitly include cluster-assignment transparency under TSBA-style scaled attacks. Credit is due where the paper has independent support: the ablation study does show each component contributes, and the dynamic-Q sensitivity experiment is a genuine robustness check. Those results, however, do not validate the max-cosine anchor, which is the step that decides who is trusted.","tokens_in":20827,"tokens_out":5375,"duration_ms":62580,"concrete_test":"Instrument the released RKD code and re-run the TSBA experiment using the standard scaled-update form theta_g + gamma*Delta_m, with gamma chosen so that the malicious update norm equals the median benign update norm. Log Algorithm 2's cluster labels and mean cosine similarities every round. If the malicious cluster is ever selected as benign (mean cosine is the maximum), the trust anchor is broken and the reported ASR < 17% is not explained by the stated mechanism. An analytic companion check: compute cos(theta_g + gamma*Delta_m, theta_g) versus a typical benign update under the paper's CIFAR-10/ResNet-18 setting; if the malicious value exceeds the benign value, the max-mean rule selects for poison.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central load-bearing step is Algorithm 2, lines 8-10: the HDBSCAN cluster with the highest mean cosine similarity to the current global model is declared benign. This single rule decides which updates enter the ensemble and which clients receive the next global model, so if it mislabels the clusters, the entire RKD pipeline is poisoned. The paper's own TSBA description (Section V.C.e) is the direct counterexample: an attacker submits theta_g + gamma * Delta_m with gamma chosen so the update looks benign. For small gamma, cos(theta_g + gamma*Delta_m, theta_g) tends to 1, so the poisoned model is maximally 'similar' to the global model and lands in the max-mean cluster. The same failure occurs once the global model is itself poisoned: benign updates that disagree with the poisoned direction have lower cosine similarity and are filtered out, while colluding attackers that track the global direction are admitted. The claimed MTA > 80% and ASR < 17% therefore depend on an anchor that the threat model's own stealthy attacks are designed to defeat. The paper reports no cluster-assignment diagnostics, so the experiments do not show that the cluster actually selected was the benign one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Robust Knowledge Distillation (RKD), a server-side defense for federated learning. At each round, the server computes the cosine similarity of each client update to the current global model, clusters these scalar scores with HDBSCAN, declares the cluster with the highest mean similarity to be benign, selects models near the element-wise median of that cluster, and distills the selected ensemble into the next global model using an unlabeled server dataset. Clients flagged as malicious are either excluded from receiving the updated global model or receive a perturbed version. The authors evaluate RKD against five backdoor attacks (A3FL, F3BA, DBA, ADBA, TSBA) on CIFAR-10, EMNIST, and Fashion-MNIST, with 20%, 40%, and 60% malicious clients under Non-IID partitions, and claim MTA above 80% with ASR below 17%, outperforming FedAvg, FLAME, FedDF, FedRAD, FedBE, RLR, and FoolsGold.","tokens_in":21103,"tokens_out":6539,"duration_ms":69041,"significance":"If the claims hold, RKD would be a practically valuable defense because it does not require IID data or a small attacker fraction, and it is computationally cheaper than FedDF and FedBE. The paper ships code, includes an ablation study, proposes a dynamic cluster-size schedule, and evaluates several recent attacks, which are strengths. The main weakness is that the central trust anchor—the max-cosine cluster rule—is not justified theoretically or diagnosed empirically, and the experimental reporting is incomplete: the claimed confidence intervals appear in no figure, and no final numerical tables are provided. These gaps currently prevent the reader from verifying the headline numbers.","major_comments":[{"comment":"The defense assumes that the HDBSCAN cluster with the highest mean cosine similarity to the current global model is the benign cluster. This is the load-bearing step, and the paper gives no formal or empirical justification for it. An adversary can submit an update of the form θ_global + γΔ with small γ, whose cosine similarity to θ_global tends to 1; this is consistent with the train-and-scale TSBA attack described in Section V.C.e. Such an update would be placed in the \"benign\" cluster and would enter the distillation ensemble. Once the global model is itself poisoned, benign updates that disagree with the poisoned direction will have lower similarity and be filtered out, so the error can self-reinforce. The paper reports no cluster-assignment accuracy or per-round diagnosis showing that the selected cluster is actually the benign one. Please provide a correctness condition under which the max-cosine cluster is benign, add cluster-membership diagnostics to the experiments, and evaluate RKD against a TSBA variant that explicitly aligns the scaled update with the global direction.","section":"Algorithm 2, lines 8-10"},{"comment":"The text states that all experiments were repeated five times and that confidence intervals are reported at a significance level of ρ = 0.01, but Figures 1-15 show no error bars or confidence bands, and no table reports final MTA/ASR values with intervals. The abstract's quantitative claims (MTA > 80%, ASR < 17%) therefore cannot be checked or reproduced from the manuscript. Please add a table of final means and intervals for every attack/dataset/attacker-fraction configuration, and either add confidence bands to the figures or explicitly state that the plotted curves are means only.","section":"Section V.F"},{"comment":"The evaluation matrix does not cover the claimed \"various scenarios\" uniformly. ADBA is tested only on CIFAR-10 (Figure 7), A3FL is not tested on EMNIST, and different attacks are evaluated under different heterogeneity levels (α = 0.3 for A3FL, α = 0.5 for F3BA, ADBA, and TSBA, α = 0.9 for DBA). This confounds attack difficulty with dataset and heterogeneity, so the stated superiority over baselines is only established for the particular configurations shown. Please either fill in the full cross-product of attacks, datasets, and α values, or explicitly restrict the claims to the tested configurations.","section":"Section V.F"},{"comment":"Only the HDBSCAN minimum-cluster-size schedule Q is given a sensitivity analysis. Other free parameters—the model-selection threshold ε (defined only as \"one may set\" in Section IV.C), distillation temperature T, KD epochs and learning rate, and the size of the server distillation dataset—are fixed without a sensitivity study or even a reported default value for ε. Because these are tuned on the same three benchmarks, the claim that RKD works \"without relying on restrictive assumptions\" needs at least a robustness check over these parameters or a statement of the ranges for which the results hold.","section":"Sections IV.C and V.F.7"},{"comment":"The exclusion strategy is unclear. Line 9 sends the \"current local model M^r_i\" to a malicious client before the server collects the round-r models in line 16, and the server cannot know M^r_i at that point. The prose in Section IV.A says excluded clients continue training on their current local model, but the algorithm appears to send the stale model back. Please clarify what exactly is sent, from where the server obtains it, and when, since this affects the attack dynamics for adaptive adversaries and the reproducibility of the exclusion strategy.","section":"Algorithm 1, lines 6-16"}],"minor_comments":[{"comment":"Reference [12] contains a typo: \"University of Tront\" should be \"University of Toronto.\"","section":"References"},{"comment":"The symbol α is used both for the Dirichlet heterogeneity parameter and for the scaling factor in the model-poisoning description; please rename one of them to avoid confusion.","section":"Section II.A"},{"comment":"The attack setup does not specify the trigger pattern, the target class, or the number of poisoned samples per client; please provide these details for reproducibility.","section":"Section V.B"},{"comment":"The TSBA scaling factor γ is never given a range or concrete value, so it is unclear which scaling regime the reported results cover; please report the γ values used.","section":"Section V.F.7 and Section V.C.e"},{"comment":"Defense-time measurements lack hardware/software context and variance estimates, and the RLR value of 0.020 seconds is unusually small; please describe the evaluation environment and whether times are averaged over rounds or runs.","section":"Table I"},{"comment":"Figure 1 includes the RKD(PGM) variant but Figure 2 does not; please clarify whether PGM was evaluated on Fashion-MNIST and, if so, why it is omitted.","section":"Figures 1 and 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a security/ML venue and the authors provide code and ablations. I have no concerns about citation ethics or novelty disclosure. The main substantive issue is whether the max-cosine cluster anchor in Algorithm 2 is sound; the reported experiments do not currently demonstrate that the selected cluster is the benign one. My major_revision recommendation is driven by the incompleteness of the empirical evidence and the logical gap in the trust anchor, not by any suspicion of misconduct."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: RKD is a sensible recombination of known pieces — HDBSCAN over cosine similarities to the global model, median-based model selection, then knowledge distillation — and the claim that this holds under Non-IID data and up to 60% malicious clients is worth taking seriously. But there is a load-bearing assumption the paper never proves: that the cluster with the highest mean cosine similarity to the global model is the benign cluster. The stress-test note is right that the paper's own TSBA description breaks this: a scaled model close to the global model has cosine similarity near 1 and would be selected as trusted, and if the global model itself is poisoned, benign updates become the outliers. This is not a nitpick; it's the mechanism by which the defense decides who to trust.\n\nWhat's genuinely new: reducing high-dimensional updates to scalars before clustering is a practical simplification, and the integration is not in prior work. The framework is clearly described, and the ablation study gives credit to each component. Code is promised, which will help.\n\nThe soft spots beyond the anchor: the empirical support is weaker than the abstract implies. The paper says confidence intervals at rho=0.01 were computed, but no interval appears in any figure and there's no table of final MTA/ASR. Different attacks are tested on different datasets, so no single scenario covers all. Hyperparameters like k and T are not fixed, and the dynamic Q schedule is ad hoc. Also, the TSBA description says attackers scale down their updates to look benign; that's a weaker attack than the original train-and-scale, which scales up, so the good results against TSBA may be against a strawman. The authors need to clarify which version they implemented and show cluster-assignment diagnostics.\n\nWho this is for: FL security researchers working on robust aggregation. The idea is testable and the paper is readable. It deserves a serious referee, but the current version overclaims. I'd ask for numerical results with intervals, diagnostics on which cluster was selected, and a faithful TSBA implementation before trusting the headline numbers.","headline":"RKD is a promising recombination of known defense pieces, but its trust anchor — the highest-cosine cluster is benign — is contradicted by the paper's own TSBA description and needs evidence, not just assertion.","tokens_in":21626,"tokens_out":5020,"would_cite":false,"duration_ms":56970,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated-learning defense called RKD filters poisoned client updates by their angle to the global model, keeping backdoor attack success below 17% while preserving accuracy above 80% under non-IID data and up to 60% malicious clients.","keywords":["federated learning","backdoor attacks","knowledge distillation","robust aggregation","non-IID data","model poisoning defense","HDBSCAN clustering","attack success rate"],"falsifier":"Run RKD with 30 clients and 60% malicious clients using TSBA or A3FL, but have each attacker send the current global model vector plus a small perturbation aligned with the backdoor trigger, chosen so its cosine similarity to the global model is at least as high as the benign clients' average; if the global model's attack success rate exceeds 17% while clean accuracy stays high, the benign-cluster anchor has failed. A sharper test: poison the global model for one round with a large scaled malicious update, then check whether the next round's honest clients are clustered as outliers and excluded while the attackers are admitted.","tokens_in":20637,"feed_emoji":"🛡️","tokens_out":7801,"duration_ms":70073,"temperature":0.7,"pith_summary":"The paper proposes Robust Knowledge Distillation (RKD), a server-side defense for federated learning against backdoor attacks that does not require data to be IID or the fraction of malicious clients to be small. RKD computes each client update's cosine similarity to the current global model, clusters those scalar scores with HDBSCAN, and treats the cluster with the highest mean similarity as benign. It then selects models nearest the median of that cluster and distills their averaged logits into the next global model, while excluding flagged malicious clients from receiving the fresh global model. The authors report main-task accuracy above 80% and attack success below 17% under non-IID data with up to 60% malicious clients across five backdoor attacks and three datasets. If correct, this gives FL operators a defense that works in the heterogeneous, adversarial conditions real deployments face.","feed_headline":"Federated learning defense keeps backdoor success under 17%","feed_subtitle":"Cosine-similarity clustering plus ensemble distillation blocks backdoors in non-IID federated learning with up to 60% attackers.","key_machinery":"The load-bearing object is the automated clustering step: each client update $\\theta_i^r$ is scored by cosine similarity $s_i = (\\theta_i^r)^\\top M_{\\text{global}}^r / (\\|\\theta_i^r\\|\\|M_{\\text{global}}^r\\|)$ to the current global model, and HDBSCAN clusters these scalar scores; the cluster with the highest mean score is declared benign. Model selection then computes the element-wise median of the benign models and keeps only models within $\\epsilon = \\mu_d + k\\sigma_d$ of it in $\\ell^1$ distance. Knowledge distillation averages the selected models' logits on an unlabeled public set, forms soft pseudo-labels with temperature $T$, trains a distilled student by KL divergence, and averages weights with stochastic weight averaging to produce the next global model. This three-stage pipeline is what carries the claim: cosine similarity makes clustering tractable in high-dimensional parameter space, median selection removes residual outliers, and distillation suppresses both backdoor triggers and non-IID variance.","core_discovery":"On the paper's own terms, RKD's central discovery is that a reliable benign ensemble can be recovered from heterogeneous client updates by projecting each update to a one-dimensional cosine-similarity score against the global model, clustering those scores, and trusting only the highest-similarity cluster. The approach is designed so that angular deviation from the global model, not raw parameter distance, is the signal separating poisoned from honest updates, which is why the authors claim it survives non-IID data and attacker majorities of up to 60%. The selected near-median models form an ensemble whose averaged logits, with temperature-smoothed softmax targets, are distilled into the global model via KL divergence; this step both transfers collective knowledge and smooths over non-IID variation. The paper's headline evidence is that this pipeline keeps clean-task accuracy above 80% while driving attack success rate below 17% for A3FL, F3BA, DBA, ADBA, and TSBA attacks.","pith_inferences":["The defense inherits a vulnerability the paper does not analyze: if the global model is already poisoned, or if attackers craft updates more aligned with its direction than benign updates, the 'benign cluster' label flips and the filter would exclude honest clients while admitting attackers.","Because the dynamic minimum cluster size $Q = \\max(2, \\lceil 0.2N - r\\rceil)$ decays to 2 after a few rounds, small colluding groups could eventually form their own high-similarity cluster and be accepted as benign; this is a testable boundary condition, not a claim in the paper.","The 16% unlabeled public dataset needed for distillation is a practical cost and a new trust assumption; how distillation behaves with smaller, skewed, or distributionally different public data is left open."],"forward_implications":["If RKD holds up, federated learning deployments under non-IID data can defend backdoors without requiring the common IID-data or small-malicious-fraction assumptions.","With up to 60% malicious clients, RKD keeps main-task accuracy above 80% and attack success below 17% across CIFAR-10, EMNIST, and Fashion-MNIST for A3FL, F3BA, DBA, ADBA, and TSBA attacks.","The exclusion strategy stops flagged attackers from receiving the updated global model, and the perturbed-global variant (RKD-PGM) hides their status with noise of norm about $10^{-4}$ while retaining similar accuracy.","Ablations show each of the three components matters: without clustering, malicious updates enter the ensemble; without median selection, residual outliers raise attack success; without knowledge distillation, accuracy drops by about 20% in non-IID settings."],"supporting_citations":[{"why":"Defines FedAvg, the standard aggregation the server starts from and the cosine-similarity anchor for judging client updates.","marker":"[14]"},{"why":"FLAME is the main HDBSCAN-based baseline; RKD adopts clustering but shifts it to one-dimensional similarity scores.","marker":"[16]"},{"why":"FedDF supplies the ensemble-knowledge-distillation template that RKD adapts for secure aggregation.","marker":"[13]"},{"why":"A3FL is the adaptive attack the defense must counter; its adversarial trigger optimization motivates RKD's exclusion strategy.","marker":"[22]"},{"why":"F3BA defines the parameter-flip attack used to test defenses against subtle parameter manipulation.","marker":"[7]"},{"why":"DBA provides the distributed-trigger attack baseline that stresses detection of colluding clients.","marker":"[20]"},{"why":"TSBA is the train-and-scale attack that tests defenses against magnitude-manipulated updates.","marker":"[3]"},{"why":"ADBA is the anti-distillation attack showing that backdoors can survive knowledge distillation, the very failure RKD targets.","marker":"[26]"}],"fun_headline_variants":["Federated backdoor defense: under 17% success, over 80% accuracy","RKD: clustering plus distillation blocks backdoor attacks in FL","Non-IID federated learning: 60% attackers, sub-17% backdoor success","Robust distillation defeats backdoors without strict assumptions","Federated learning: ensemble distillation keeps models clean from backdoors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the group of client updates with the highest average cosine similarity to the current global model is the benign group, so the global model's direction must itself be trustworthy.","fun_headline_variants_meta":{"raw":{"variants":["Federated backdoor defense: under 17% success, over 80% accuracy","RKD: clustering plus distillation blocks backdoor attacks in FL","Non-IID federated learning: 60% attackers, sub-17% backdoor success","Robust distillation defeats backdoors without strict assumptions","Federated learning: ensemble distillation keeps models clean from backdoors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1336,"prompt_tokens":890,"completion_tokens":446,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":349}},"tokens_in":506,"tokens_out":446,"duration_ms":5963,"temperature":1.0,"reasoning_tokens":349,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:24:15.967301+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RKD with 30 clients and 60% malicious clients using TSBA or A3FL, but have each attacker send the current global model vector plus a small perturbation aligned with the backdoor trigger, chosen so its cosine similarity to the global model is at least as high as the benign clients' average; if the global model's attack success rate exceeds 17% while clean accuracy stays high, the benign-cluster anchor has failed. A sharper test: poison the global model for one round with a large scaled malicious update, then check whether the next round's honest clients are clustered as outliers and excluded while the attackers are admitted.","supporting_citations":[{"cited_title":"McMahan, E","cited_arxiv_id":null,"evidence_quote":"Defines FedAvg, the standard aggregation the server starts from and the cosine-similarity anchor for judging client updates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FLAME is the main HDBSCAN-based baseline; RKD adopts clustering but shifts it to one-dimensional similarity scores."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FedDF supplies the ensemble-knowledge-distillation template that RKD adapts for secure aggregation."},{"cited_title":"A3FL: Adversarially adaptive backdoor attacks to federated learning","cited_arxiv_id":null,"evidence_quote":"A3FL is the adaptive attack the defense must counter; its adversarial trigger optimization motivates RKD's exclusion strategy."},{"cited_title":"Fang and J","cited_arxiv_id":null,"evidence_quote":"F3BA defines the parameter-flip attack used to test defenses against subtle parameter manipulation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DBA provides the distributed-trigger attack baseline that stresses detection of colluding clients."},{"cited_title":"”Anti-distillation backdoor attacks: Backdoors can really survive in knowledge distillation.” In Proceedings of the 29th ACM International Conference on Multimedia, pp","cited_arxiv_id":null,"evidence_quote":"ADBA is the anti-distillation attack showing that backdoors can survive knowledge distillation, the very failure RKD targets."}],"review_version":1}