{"id":"08eaa4ac-4374-4f6b-9b57-b4b3a5db816e","arxiv_id":"2509.08746","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"An adaptive federated-learning backdoor attack uses membership-inference feedback on the global model to keep malicious updates statistically similar to benign ones, evading nine robust aggregation defenses in two image datasets.","lead":"This paper describes a new backdoor attack on federated learning that hides malicious updates inside the normal-looking distribution of client updates. It reports that the attack defeats nine robust aggregation defenses by adapting to feedback from the aggregated model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 'break all RA defenses' relies on per-defense oracle selection of the proximity metric; with a fixed black-box metric, Table II shows several defenses are not broken, so the central claim is not established.","rationale":"The reader's weakest_assumption was BSCI reliability. That is a real gap, but I judge the per-defense metric selection to be more load-bearing for the paper's headline: even a perfectly reliable side-channel would not justify Table I if the reported numbers come from picking the best of three loss geometries after knowing the defense. The reader did list the best-metric issue in the rationale and made it part of the requested re-evaluation, so the agreement is partial. My concern does not move the overall verdict: CONDITIONAL remains appropriate, but the condition should be strengthened to include a fixed, pre-registered metric as a primary requirement. If the fixed-metric rerun still breaks all nine defenses, the central claim survives; if not, the paper should be reframed as demonstrating tuned attacks rather than a single black-box adaptive adversary.","tokens_in":19437,"tokens_out":8203,"duration_ms":84657,"concrete_test":"Pre-register a single proximity metric (Huber, the paper's own black-box choice in Section VI.C) and rerun the full Table I protocol for all nine defenses on both datasets, without per-defense metric selection, reporting ASR at T and at T/2. If fixed-Huber ASR falls below a pre-defined 'non-negligible' threshold (e.g. <50%) for any of Krum, Multi-Krum, Bulyan, DAI, or RLR, or if Fashion-MNIST ASR for Median/FTA/RFA is not above the vanilla baseline, the 'break all RA defenses' claim should be re-scoped to per-defense tuned attacks.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section VI.A states: 'For CHAMP, the best proximity metric is used in every case among three options—Euclidean distance, cosine similarity and Huber loss.' The resulting Table I is therefore not the performance of a single black-box, aggregation-agnostic adversary, but the union of three tuned attack variants. This matters because Section VI.C shows the choice is decisive: cosine similarity fails on Krum/Multi-Krum/Bulyan/RLR/DAI, while Euclidean/Huber break those defenses; Median/FTA/RFA/FoolsGold favor cosine/Huber. Under the paper's own black-box recommendation (Huber), several headline numbers collapse, e.g. RLR with a 5x5 trigger on CIFAR-10 drops from 94.7% ASR (Euclidean) to 3.2% (Huber), and Fashion-MNIST gains over the baseline are negative or negligible for Median, FTA, and RFA. Since the threat model (Section III) denies white-box access to the aggregation rule and claims only the published global model is observed, choosing a different metric per defense after observing the defense's outputs is not a legitimate black-box evaluation. The central claim 'break all RA defences under suitable configuration' is thereby overstatement: at most, it shows each defense can be broken by some tuned metric, which does not establish that a conforming adaptive adversary can evade all of them.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Chameleon Poisoning (CHAMP), an adaptive model-poisoning attack for federated learning. A malicious client trains a Backdoor Side-Channel Inference (BSCI) model on poisoned reference models and uses the global model's responses on backdoored samples to estimate whether its malicious update was incorporated in aggregation. This estimate vt is converted into an adaptive balance coefficient alpha_t that weights a malicious loss L_pois against a camouflage/proximity loss L_prox. The attack is evaluated against nine robust aggregation defenses on Fashion-MNIST and CIFAR-10, reporting large ASR gains over a vanilla data-poisoning baseline and claiming that, under suitable configuration, all surveyed RA defenses can be broken.","tokens_in":19765,"tokens_out":3625,"duration_ms":44260,"significance":"The core idea is valuable and timely: using membership-inference-like feedback from the published global model to adaptively trade off poisoning strength against statistical conformity directly challenges the out-of-distribution assumption underlying most robust aggregation schemes. The paper is also broad in scope, covering nine defenses, two datasets, trigger-size ablations, multiple proximity metrics, and an ablation of the side-channel signal. If the central claim were established under a genuinely black-box threat model, this would be a notable contribution to the FL security literature. However, the headline result currently rests on per-defense selection of the proximity metric after observing results, and the load-bearing BSCI signal is validated only in a centralized, fully backdoored setting; both issues need to be resolved before the central claim is supported.","major_comments":[{"comment":"The main table selects, for each defense, the best of three proximity metrics after observing the results, while the threat model in §III explicitly denies white-box knowledge of the aggregation rule. This is post hoc selection, not black-box evaluation. The problem is visible in Table II: the paper's own recommended black-box metric, Huber loss, gives RLR a 5x5 CIFAR-10 ASR of 3.2% and FoolsGold 15.1%, both below the vanilla baseline's 39.6% and 60.2% on the same configuration. Thus the claim in §VIII that 'it is possible, under a suitable configuration, to break all RA defences' is an overstatement: the experiments demonstrate that each defense can be broken by some tuned proximity metric, not that a single black-box CHAMP adversary evades all of them. The headline evaluation must be redone with a fixed proximity metric (or with a principled online selection mechanism that does not use","section":"§VI.A, Table I and §VI.C, Table II"},{"comment":"The entire adaptive loop depends on the BSCI signal vt: it sets alpha_t in Eq. (9) and balances L_pois and L_prox in Eq. (10). However, Appendix A validates the underlying membership-inference intuition only on centralized MNIST models that are fully backdoored. It does not test whether the signal remains reliable when the global model is produced by aggregating one malicious update with nine benign updates and then filtering through a robust aggregator. If vt is noisy or weakly correlated with actual inclusion of the malicious update under RA, CHAMP degenerates into vanilla poisoning and the central mechanism collapses. The manuscript needs direct evidence, e.g., measuring vt against the actual selection outcome (as in Krum/Multi-Krum) or against the eventual ASR under each RA defense, before the side-channel can be considered load-bearing.","section":"§IV.A, Algorithm 1 and Appendix A"},{"comment":"All reported results appear to be single runs with no seeds, confidence intervals, or variance estimates. Federated learning is stochastic, and ASR values for strong defenses can fluctuate substantially across runs; the difference between, for example, Krum baseline ASR 3.07 and CHAMP ASR 100 in Table I could be affected by run-to-run variation. Given the paper's strong empirical claims, the core tables and central comparisons should be repeated with multiple seeds and reported with means and standard deviations or confidence intervals.","section":"§V and §VI, Tables I–II and Figures 4–8"},{"comment":"Section VI.C states that 'in a blackbox setting where no knowledge of the aggregation rule is available, Huber loss seems the more robust approach, while efficacy gains are possible in a white-box threat model.' This directly undermines the unqualified conclusion in §VIII that all RA defenses are broken. The manuscript should either present the fixed-Huber results as the black-box headline and acknowledge which defenses are and are not evaded under that configuration, or relax the threat model and clearly label the per-defense metric selection as white-box/defense-specific tuning. As written, the conclusion and the experimental setup are not consistent.","section":"§VI.C and §VIII"}],"minor_comments":[{"comment":"There are numerous typos and formatting artifacts, e.g., 'wiht', 'isntead', 'ocnductedon', and malformed equations (Eq. 14, Eq. 21, Eq. 22). A careful proofreading pass is needed.","section":"Global"},{"comment":"The abstract and introduction report 'an average increase of 47.07% in attack success rate' but do not define the averaging convention (over defenses? datasets? configurations?) nor connect it to a specific table. Please state the exact aggregation and reference baseline.","section":"§I and Abstract"},{"comment":"Notation is inconsistent about whether BSCI is applied to G_t or G_{t-1}: the algorithm input is 'G_t' while the text says the adversary probes the 'newly published G_{t-1}'. This makes it hard to reproduce the timing of the feedback loop.","section":"Algorithm 1 / §IV.A"},{"comment":"Several related-work and defense descriptions contain missing subscripts, stray characters, and undefined symbols (e.g., 'gt n,1' vs 'gt n,k'). Please revise for precision.","section":"§VII and Eq. (15)-(22)"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and a broad evaluation, but the central claim is currently supported by per-defense oracle selection of the proximity metric and by a side-channel validation that does not cover the actual FL setting. These are fixable with additional experiments and a re-scoped conclusion, but they are load-bearing for the 'break all RA defenses' statement. I would encourage the editor to request a major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The genuinely new piece is the closed loop: the attacker runs a membership-inference-style probe on its own poisoned samples against the published global model, uses that signal to modulate a camouflage term in the local loss. That's a real technique, and it is not in the prior attacks I know. The paper also does something useful empirically: nine defenses across two datasets, with ablations on trigger size, proximity metric, and side-channel design. The Krum-score plots and TSNE figures give some direct evidence that the malicious update enters the accepted distribution. Credit where due: the mechanism is clear and the overall direction is important—if a black-box attacker can tune the level of camouflage from the global model alone, the outlier assumption behind RA is weaker than advertised.\n\nNow the soft spots, in proportion. The main result is overstated. Table II is the whole story: for each defense they use the best of Euclidean, cosine, or Huber as the proximity metric, chosen after seeing results. That is not a single black-box attacker; it is a union of three tuned variants. Under their own recommendation, Huber, several headline numbers collapse—RLR goes from 94.7% to 3.2% on CIFAR-10 with a 5x5 trigger, and Fashion-MNIST gains on Median, FTA, RFA are negligible. So the correct claim is roughly \"each examined defense can be broken by some metric choice,\" not \"we break all RA defenses.\" That is still interesting, but materially weaker. The authors do hedge the conclusion with \"under a suitable configuration\" in Section VIII, but the abstract and intro don't.\n\nSecond, the backbone of the attack is the BSCI signal, and its validity is demonstrated only on a centralized MNIST setup with fully backdoored target models (Appendix A). The paper does not show that the signal is reliable in the actual FL setting, where one malicious update is mixed with nine benign ones and then filtered by a defense. If vt is noisy, alpha dampens and CHAMP approaches vanilla poisoning; the adaptive advantage is not yet established.\n\nThird, the experimental hygiene: single runs, no seeds, no code, IID data only, all ten clients participate every round. And the baseline is vanilla data poisoning; there is no comparison against constrain-and-scale or other adaptive attacks. That pushes significance down, not the idea.\n\nBottom line: this deserves a serious referee. The mechanism is novel enough to merit full review, and the paper is honest about some limitations. But I would not accept this version. The authors need to fix the black-box metric selection, run multiple seeds, release code/data, validate BSCI under aggregation, and add at least one adaptive baseline. If the fixed-metric results still show broad evasion, it's a solid contribution; if not, it's a narrower one. Send it to review with a request for major revision.","headline":"Novel adaptive feedback attack worth a serious referee, but the 'break all RA defenses' claim depends on per-defense metric selection and the release has no code or seeds.","tokens_in":20233,"tokens_out":3053,"would_cite":true,"duration_ms":29645,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adaptive poison hides in benign updates, defeating robust aggregation","keywords":["federated learning","backdoor attack","robust aggregation","model poisoning","membership inference","adaptive attack","side-channel inference","chameleon poisoning"],"falsifier":"Measure CHAMP's per-round side-channel estimate v_t against ground-truth knowledge of whether the malicious update was actually accepted by the aggregator; if the correlation is near zero, or if replacing BSCI with random alpha values preserves the attack's success-rate gain, the central claim is not supported. A simpler test: a defense that adds enough calibrated noise to the released global model to push BSCI accuracy to chance should also eliminate CHAMP's advantage over vanilla poisoning.","tokens_in":19335,"feed_emoji":"🎭","tokens_out":7769,"duration_ms":75648,"temperature":0.7,"pith_summary":"The paper claims that robust aggregation defenses in federated learning rest on a false premise: that malicious updates are statistical outliers. It introduces Chameleon Poisoning (CHAMP), an attack in which a malicious client reads the published global model as feedback to tell whether its earlier poisoned update was accepted, then reweights its local loss between poisoning and camouflage so its update looks normal. Across two datasets and nine defenses, CHAMP raises attack success rate by 47.07% on average over vanilla backdoor poisoning, and in the strongest configurations matches or exceeds poisoning with no defense. A sympathetic reader should care because CHAMP reframes the security question: defending aggregation may require detecting adaptivity over time, not just filtering per-round outliers.","feed_headline":"Adaptive poison hides in benign updates, defeating robust aggregation","feed_subtitle":"A feedback loop uses the global model itself to keep malicious updates looking normal while backdoors succeed.","key_machinery":"Backdoor Side-Channel Inference (BSCI): a small SVM classifier trained on the output vectors of locally backdoored reference models, then applied to the global model's outputs on backdoored samples to produce a membership estimate v_t of whether the backdoor is present. v_t drives alpha_t = 1 - average(previous k v values), which balances the malicious term L_pois against the camouflage/proximity term L_prox in the malicious client's local loss; this closed loop lets a black-box attacker steer updates into the aggregator's acceptance region without knowing the aggregation rule.","core_discovery":"CHAMP's central claim is that a backdoor can be inserted into a federated global model while the malicious update remains statistically indistinguishable from benign updates, so robust aggregation filters cannot exclude it. The attack runs a closed loop each round: it trains a lightweight membership-inference classifier (the Backdoor Side-Channel Inference, or BSCI) on locally backdoored reference models, queries the newly published global model on backdoored samples to estimate whether the previous update was incorporated, and uses that estimate to set a balance coefficient alpha_t. The malicious local loss is L = L_pois + alpha_t L_prox, where L_prox penalizes distance from the previous gl","pith_inferences":["The same feedback principle likely generalizes beyond backdoors: any malicious objective whose effect is observable in the global model's outputs (targeted misclassification, class imbalance, fairness drift) could be steered with a similar membership-style probe.","A defense that injects calibrated noise or obfuscation into the released global model specifically to break BSCI-style membership inference would force the attacker back to vanilla poisoning; testing this is a direct extension of the paper's experiments.","The paper's setup uses IID data, ten clients, one malicious client, and full client participation each round; whether the attack survives non-IID distributions, larger client pools, or partial participation is left open and is the natural next test.","If robust aggregation is to survive such adversaries, it may need to move from static per-round outlier rejection to consistency checks across rounds, for example detecting clients whose updates alternate between conformity and attack in response to feedback."],"forward_implications":["Robust aggregation alone does not stop an adaptive adversary that can observe the published global model; per-round outlier filtering misses updates that look normal.","On two datasets and nine defenses, CHAMP improves attack success rate by 47.07% on average over vanilla data poisoning, and in five defenses reaches near-total backdoor success.","The attack needs no white-box access to the server or aggregation rule, because the global model broadcast each round supplies the feedback signal.","The BSCI side channel is more informative than raw attack success rate for guiding the attack, as it detects early signs of backdoor incorporation before the backdoor appears in predictions."],"supporting_citations":[{"why":"Defines the federated learning protocol and FedAvg aggregation that the attack operates against.","marker":"[1]"},{"why":"Introduces backdoor poisoning in federated learning and the constrain-and-scale update strategy that CHAMP extends and compares against.","marker":"[2]"},{"why":"Defines the backdoor trigger and target-label mechanism used to craft the poisoned samples.","marker":"[9]"},{"why":"Supplies the membership-inference principle and shadow-model technique on which BSCI is built.","marker":"[10]"},{"why":"Defines Krum and Multi-Krum, selection-based defenses that CHAMP's camouflage is shown to evade.","marker":"[6]"},{"why":"Defines Bulyan, a selection-plus-coordinate-trimming defense that CHAMP breaks.","marker":"[7]"},{"why":"Defines coordinate-wise median and trimmed-mean robust aggregation, two of the evaluated defenses.","marker":"[8]"},{"why":"Defines RFA, the geometric-median robust aggregation that is one of the stronger defenses in the evaluation.","marker":"[24]"}],"fun_headline_variants":["Poison that Looks Normal: Adaptive Attack Beats Robust Aggregation","Camouflaged Updates: How Adaptive Poisoning Eludes Robust Defenses","Backdoor That Fits In: Adaptive Attack Slips Past Robust Aggregation","In-distribution Poison: Adaptive Attack Beats Nine Aggregation Defenses","Adaptive Poisoning Evades Robust Aggregation by Looking Benign"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The attack collapses if the backdoor side-channel signal does not reliably tell the attacker whether its previous poisoned update was incorporated into the global model, because every adaptive choice in the loss balance depends on that signal.","fun_headline_variants_meta":{"raw":{"variants":["Poison that Looks Normal: Adaptive Attack Beats Robust Aggregation","Camouflaged Updates: How Adaptive Poisoning Eludes Robust Defenses","Backdoor That Fits In: Adaptive Attack Slips Past Robust Aggregation","In-distribution Poison: Adaptive Attack Beats Nine Aggregation Defenses","Adaptive Poisoning Evades Robust Aggregation by Looking Benign"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":2968,"prompt_tokens":755,"completion_tokens":2213,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":2117}},"tokens_in":499,"tokens_out":2213,"duration_ms":15412,"temperature":1.0,"reasoning_tokens":2117,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:00:01.258223+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure CHAMP's per-round side-channel estimate v_t against ground-truth knowledge of whether the malicious update was actually accepted by the aggregator; if the correlation is near zero, or if replacing BSCI with random alpha values preserves the attack's success-rate gain, the central claim is not supported. A simpler test: a defense that adds enough calibrated noise to the released global model to push BSCI accuracy to chance should also eliminate CHAMP's advantage over vanilla poisoning.","supporting_citations":[{"cited_title":"Communication-Efficient Learning of Deep Networks from Decentralized Data,","cited_arxiv_id":null,"evidence_quote":"Defines the federated learning protocol and FedAvg aggregation that the attack operates against."},{"cited_title":"How To Backdoor Federated Learning,","cited_arxiv_id":null,"evidence_quote":"Introduces backdoor poisoning in federated learning and the constrain-and-scale update strategy that CHAMP extends and compares against."},{"cited_title":"Machine learning with adversaries: byzantine tolerant gradient descent,","cited_arxiv_id":null,"evidence_quote":"Defines Krum and Multi-Krum, selection-based defenses that CHAMP's camouflage is shown to evade."},{"cited_title":"Byzantine- Robust Distributed Learning: Towards Optimal Statistical Rates,","cited_arxiv_id":null,"evidence_quote":"Defines coordinate-wise median and trimmed-mean robust aggregation, two of the evaluated defenses."},{"cited_title":"Robust Aggregation for Federated Learning","cited_arxiv_id":"1912.13445","evidence_quote":"Defines RFA, the geometric-median robust aggregation that is one of the stronger defenses in the evaluation."}],"review_version":1}