{"id":"fb163cb4-8cea-4351-8531-4be449d6c514","arxiv_id":"2505.12019","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FL-PLAS defends federated learning against backdoor attacks by aggregating only feature extractors and keeping classifiers client-local, reporting low backdoor accuracy with up to 90% malicious clients.","lead":"A federated learning defense, FL-PLAS, keeps each client's classifier local while sharing only feature-extractor weights, and reports low backdoor success even when 90% of clients are malicious. The paper argues this prevents backdoor labels from propagating to honest clients without needing any clean server dataset.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The defense's core premise—that a poisoned feature extractor cannot backdoor a clean classifier—is untested against attackers who optimize the shared feature extractor, which the paper's own threat model permits.","rationale":"The reader's weakest assumption correctly identifies that backdoor information might transfer through the shared feature extractor. I agree this is the load-bearing point. My read sharpens it: the paper's threat model does not merely omit an adaptive attacker; Section 4 grants malicious users arbitrary local-update manipulation, and Section 7.4 claims robustness to collusion. The Table 2 experiment is therefore insufficient evidence for the stated threat model. The concern is not that the defense is circular or internally inconsistent; it is that the empirical basis covers a narrower attacker than the claims require. The proposed test would settle whether a feature extractor can be optimized to carry the backdoor into clean classifiers. Because this is the same condition the reader already placed on acceptance, I would keep the verdict conditional rather than moving to acceptance or rejection.","tokens_in":18636,"tokens_out":5365,"duration_ms":60830,"concrete_test":"Run FL-PLAS with an adaptive feature-extractor attack on CIFAR-10/MobileNet: 100 clients, 30 per round, 90% malicious. Each malicious client trains a surrogate clean classifier on its local clean data, then optimizes the feature extractor so that clean accuracy is preserved but trigger samples are embedded near the target class in feature space; clients upload only these feature extractors. After 200 rounds, measure average benign-client BA. If BA exceeds the reported 0.05 in Table 4a by a significant margin, the Section 5.1 premise fails. As a minimal check, redo Table 2 with the feature extractor optimized this way instead of taken from a backdoored model; if clean-classifier BA rises above 0.1, the core observation is an artifact of the non-adaptive training procedure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that keeping classifiers local prevents backdoor labels from affecting benign clients. The support is Table 2 in Section 5.1, where a backdoor feature extractor combined with a clean classifier gives BA of about 0.1. This is not a security guarantee: that feature extractor was produced by ordinary joint training with a backdoored classifier, so it is specialized to that classifier's decision boundary and was not optimized to fool clean classifiers. The declared threat model in Section 4 explicitly allows malicious users to 'manipulate local model updates at will,' and Section 7.4 even claims insensitivity to collusion. Under 90% malicious clients, colluding attackers can therefore upload feature extractors engineered so that trigger inputs produce features in the target-class region of the shared feature space; benign clients' clean classifiers would then assign the target label even though the classifiers are never shared. The evaluation only runs standard data-poisoning attackers that follow the FedAvg objective, so the high-ratio defense claim is not established for the threat model the paper itself states.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FL-PLAS, a federated learning defense that divides each client's model into a feature extractor and a classifier, aggregates only the feature extractor parameters on the server, and keeps each client's classifier local. The motivating observation (Table 2) is that backdoor accuracy depends mainly on whether the classifier is poisoned, not on whether the feature extractor is poisoned, so local classifiers should prevent backdoor labels from spreading to benign clients. FL-PLAS is evaluated on MNIST, CIFAR-10, and CIFAR-100 under trigger, semantic, and edge-case attacks, with malicious-client ratios up to 90%, and is compared against FedAvg, FLTrust, FLAME, RSA, Krum, and NDC. The reported results show low backdoor accuracy and competitive main-task accuracy for FL-PLAS, along with an efficiency comparison arguing that FL-PLAS has no screening overhead.","tokens_in":18821,"tokens_out":4930,"duration_ms":49833,"significance":"If the claims hold, FL-PLAS is a potentially valuable contribution to the FL backdoor-defense literature: it does not require a server-side auxiliary dataset, it has trivial screening complexity, and it targets the high-malicious-ratio regime where many existing defenses fail. The paper also has notable strengths: a clear motivating experiment (Table 2), an explicit threat model and defense goals, a public code repository, and an efficiency analysis. However, the significance is currently limited by three issues: the evaluation measures FL-PLAS on averaged local models while other defenses are measured on a single global model; the experiments appear to be single-run with no error bars; and the security claim rests on an assumption that a poisoned feature extractor cannot transfer backdoor behavior to a clean classifier, which is not tested against attackers who optimize the feature extractor despite being permitted to do so by the stated threat model.","major_comments":[{"comment":"The threat model in Section 4 explicitly allows malicious users to \"manipulate local model updates at will,\" and Section 7.4 claims the method is insensitive to whether malicious clients collude. However, the evaluation in Section 6.1 instantiates only standard data-poisoning attackers who add triggers, flip labels, or modify local data while otherwise following the FedAvg objective. Under 90% malicious clients, colluding attackers could instead craft feature-extractor updates so that the aggregated feature extractor maps trigger-carrying inputs into the target-class region of the feature space, after which the benign clients' own clean classifiers would assign the target label even though classifiers are never shared. The correctness of the defense therefore depends on an untested assumption: that backdoor information cannot be transferred through the shared feature extractor to a clean classifier. I ask for either (a) an explicit restriction of the threat model that excludes collusive or adaptive feature-extractor optimization, with a justification, or (b) an experimental evaluation against such an adaptive attack. Without one of these, the statement that backdoor labels \"do not impact other clients\" is not established for the threat model the paper itself declares.","section":"Section 4 and Section 7.4"},{"comment":"The evaluation protocol is asymmetric: for FL-PLAS, BA and MA are computed by averaging over all local models, while for FedAvg, FLTrust, FLAME, RSA, Krum, and NDC the metrics are computed on the single global model. This can inflate FL-PLAS's apparent robustness because poorly performing local models are hidden in the average and because the comparison does not evaluate any concrete deployable global model for FL-PLAS. The quantitative claims in Tables 4 and 5, including the statement about improving defense capability by over 44%, rely on this asymmetric comparison. Please report the distribution (mean, standard deviation, min/max) of FL-PLAS's local MA and BA, and provide a comparison protocol in which all methods are assessed in a symmetric manner, for example by evaluating FL-PLAS with a fixed, clearly specified classifier choice or by reporting the full spread of local results.","section":"Section 4, Evaluation Metrics; Section 6.2"},{"comment":"The experiments appear to be single-run, with no error bars, multiple seeds, or statistical significance tests. Because the headline claim is robustness at 90% malicious clients, and Section 7.1 (Table 7) shows that BA can jump from 0.109 to 0.369 when the number of aggregated layers changes from 11 to 12 on CIFAR-10, a single seed is insufficient to support the quantitative conclusions. Please report results over at least three independent seeds with standard deviations, and ideally include a sensitivity analysis over client sampling and the Dirichlet distribution parameter.","section":"Section 6.2 and Tables 3-5"},{"comment":"The number of cutting layers l is a free parameter with a dramatic effect on the defense: Table 7 shows BA of 0.110 at l=11, 0.369 at l=12, and 0.900 at l=13 for CIFAR-10 at 90% malicious clients. The paper states that this selection \"needs to be analyzed\" but does not provide a principled criterion for choosing l on a new dataset or architecture. As a result, the central claim that FL-PLAS works without an auxiliary dataset is qualified by a manually tuned architectural choice. Please provide a selection rule or heuristic, and report the value of l used for every experimental configuration together with a sensitivity analysis.","section":"Section 7.1 and Table 7"}],"minor_comments":[{"comment":"The complexity analysis is unclear: the text says Algorithm 2 has complexity O(p + Di/b) and \"equals to Linear complexity O(N)\", but N is never defined, and the total complexity O(r*O(N) + p*n) = O(N^2) does not follow rigorously from the stated quantities. Please rewrite this paragraph with consistent notation.","section":"Section 5.2, Algorithms 1-2"},{"comment":"The caption says \"MA of CIFAR-100 in semantic attack\" while the text and the rest of the figure describe CIFAR-10. Please correct the caption or the labels so that the dataset name is consistent.","section":"Figure 6"},{"comment":"RFA appears in the screening complexity table but is never defined or cited in the text. Please either define RFA and its reference or remove it from the comparison.","section":"Table 6"},{"comment":"The description of the semantic backdoor is internally inconsistent: it first says pointy-eared dog images are assigned the label \"cat\" and later says labels are flipped from \"dog\" to \"cat\". Please clarify which variant is used in the experiments.","section":"Section 2.2"},{"comment":"The table lists a \"Combined learning rate\" of learning rate × 0.998^t, but the per-dataset learning rates are only given in the prose of Section 6.1. Please add the model/dataset mapping and learning rates directly to the table for reproducibility.","section":"Table 3"},{"comment":"The claim that FL-PLAS \"improves defense capability by over 44%\" is not precisely defined. Please state the exact formula used to compute this percentage improvement and the baseline to which it is compared.","section":"Section 6.2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a security journal, and I do not see grounds to reject the central idea outright: the Table 2 experiment directly probes the premise that a clean classifier suppresses backdoor accuracy, so this is not a purely circular construction. The main problem is that the evaluation does not match the stated threat model, and the comparison is methodologically asymmetric. I would advise the editor that acceptance should require the adaptive-attacker experiment or a revised threat model, plus a symmetric evaluation with error bars."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely useful: it takes a known personalized-FL trick, keeps classifiers local, and runs a broad comparison across three datasets and three attack types. The core intuition—backdoor labels live largely in the classifier, not the feature extractor—is supported by Table 2 for the attacks they tried. Code is public, so results are checkable. That is real work and a reasonable starting point for the high-ratio malicious client problem.\n\nThe soft spots are real, though. The evaluation is asymmetric: FL-PLAS is measured on averaged local models, while every baseline is measured on a single global model. That can inflate both MA and BA comparisons, and while the headline numbers look good, there are no error bars or multiple seeds to tell if the 90% results are stable. This should be an easy fix but it is a genuine flaw in the current write-up.\n\nThe bigger issue is the gap between the threat model and the attacks actually tested. Section 4 says malicious users can manipulate local model updates at will, and Section 7.4 explicitly claims the method is insensitive to collusion. Yet all experiments use standard data-poisoning attackers that follow the FedAvg objective. A colluding majority could in principle engineer feature extractor updates so that trigger inputs produce features in the target-class region for clean classifiers, defeating the defense even though classifiers are never shared. Table 2 does not rule this out, because that backdoor feature extractor was co-trained with a backdoored classifier, not optimized to fool clean classifiers. So the '90% malicious clients' claim is true for the attacks tested, not for the threat model the paper states.\n\nMinor point: the conclusion claims 'first time' for partial layer aggregation as a backdoor defense, but the paper's own references [66, 67] already showed pFL can defend against backdoors. The novelty is the systematic evaluation, not the mechanism, and overstating it is unnecessary.\n\nThis paper deserves a serious referee, not a desk reject. The mechanism is clean, the experiments are broad, and the code being public is a real plus. But the authors should fix the evaluation protocol (same-model comparison, multiple seeds), test at least one adaptive attacker that optimizes the feature extractor, and temper the threat-model claims. I would send it to review with the expectation of major revision.","headline":"Useful pFL-based defense with broad experiments, but the 90%-malicious claim holds only for standard data poisoning, not the paper's own threat model.","tokens_in":19334,"tokens_out":1965,"would_cite":false,"duration_ms":20862,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FL-PLAS claims that federated learning can fend off backdoor attacks even when 90% of clients are malicious, by aggregating only feature extractors and never sharing classifiers.","keywords":["federated learning","backdoor defense","partial layer aggregation","data poisoning","malicious clients","trigger backdoor","semantic backdoor","edge-case backdoor"],"falsifier":"Train FL-PLAS with a malicious client that optimizes its feature-extractor update specifically to push aggregated features of triggered inputs toward the target class, then measure whether clean clients' unpoisoned classifiers begin classifying triggered test samples as the target; a backdoor accuracy well above the benign baseline would show that the classifier-isolation assumption fails.","tokens_in":18451,"feed_emoji":"🛡️","tokens_out":5968,"duration_ms":51977,"temperature":0.7,"pith_summary":"The paper proposes a defense for federated learning against backdoor poisoning that keeps each client's classifier private while only the feature extractor is uploaded and aggregated. The central claim is that backdoor behavior lives in the classifier, not the feature extractor, so a clean classifier attached to a poisoned feature extractor will not misclassify triggered inputs. On MNIST, CIFAR-10, and CIFAR-100, under trigger, semantic, and edge-case backdoor attacks, the method reports low backdoor accuracy and high main-task accuracy even when up to 90% of participating clients are malicious, with no auxiliary dataset held by the server. A sympathetic reader would care because current defenses degrade once malicious clients exceed roughly half the population or rely on server-held clean data, which conflicts with FL privacy.","feed_headline":"Keeping classifiers local blocks FL backdoors even at 90% malicious","feed_subtitle":"Aggregating only feature extractors keeps main-task accuracy high and backdoor accuracy low, no server dataset","key_machinery":"The central mechanism is partial layer aggregation: the local model is split at a chosen cutting layer into a feature extractor and a classifier; clients upload only the feature extractor, the server runs FedAvg over those extractor parameters, and each client keeps and uses its own classifier with the updated extractor. This prevents backdoor labels from propagating through the shared model while still allowing collaborative feature learning. The paper also treats the cutting-layer choice as a tunable defense knob: deeper aggregation improves main-task accuracy but raises backdoor accuracy, so a small classifier is preferred.","core_discovery":"The load-bearing discovery is an asymmetry between layers: a backdoor feature extractor combined with a clean classifier yields near-zero backdoor accuracy, while a backdoor classifier combined with a clean feature extractor yields near-perfect backdoor accuracy (Table 2). From this the paper concludes that the backdoor label association is carried by the classifier, and that keeping classifiers local severs the route by which malicious clients' poisoned labels reach benign clients. FL-PLAS therefore aggregates only the feature extractor with FedAvg, and each client combines the aggregated extractor with its own classifier. The paper presents this as sufficient to defend against trigger, semantic, and edge-case backdoors at malicious-client ratios up to 90%, without any server-side dataset.","pith_inferences":["An adaptive attacker excluded by the paper's threat model could try to craft feature-extractor updates whose aggregated features cause clean classifiers to map trigger patterns to the target label; testing this would reveal whether the defense holds against stronger adversaries.","The layer-asymmetry observation may be architecture- and task-dependent; re-running the Table 2 assembly test on transformer-based models or on NLP/audio tasks could show whether the classifier-only backdoor localization generalizes.","Because FL-PLAS is structurally similar to personalized federated learning's partial-model personalization, it could be combined with server-side anomaly detection or trust scoring to reduce residual backdoor accuracy, such as the 0.47 edge-case value at 90% malicious clients."],"forward_implications":["FL-PLAS maintains low backdoor accuracy and competitive main-task accuracy when more than half of clients are malicious, a regime where clustering- or majority-based defenses fail.","The defense works without any auxiliary dataset on the server, unlike FLTrust, and adds no screening complexity beyond standard FedAvg.","It resists trigger, semantic, and edge-case backdoors on MNIST, CIFAR-10, and CIFAR-100 with differing architectures.","The number of aggregated layers is a trade-off: more shared layers raise main-task accuracy but also raise backdoor accuracy, so the classifier should be kept small.","On small models such as LeNet, keeping a relatively large classifier local reduces main-task accuracy, a privacy-utility trade-off the paper acknowledges."],"supporting_citations":[{"why":"Supplies the trigger-backdoor attack definition and the attack settings used in the evaluations.","marker":"[6]"},{"why":"Defines the semantic backdoor attack that FL-PLAS is evaluated against.","marker":"[7]"},{"why":"Defines the edge-case backdoor attack that FL-PLAS is evaluated against.","marker":"[10]"},{"why":"Provides the FLTrust baseline, the main comparison that requires an auxiliary server dataset.","marker":"[17]"},{"why":"Supplies the RSA defense baseline that FL-PLAS is compared with.","marker":"[14]"},{"why":"Supplies the NDC baseline, illustrating an update-clipping defense that loses effectiveness at high malicious ratios.","marker":"[15]"},{"why":"Supplies the FLAME clustering baseline that FL-PLAS is compared with.","marker":"[18]"},{"why":"Supplies the Krum baseline, a model-selection defense that fails when malicious clients are in the majority.","marker":"[19]"},{"why":"Provides the earlier layer-pattern observation that motivates studying layer-specific separation for backdoor defense.","marker":"[69]"}],"fun_headline_variants":["Local classifiers defeat FL backdoors at 90% malicious","Aggregate features, keep classifiers: backdoor defense","Feature-only aggregation stops backdoors in federated learning","Partial layer aggregation blocks FL backdoors","Keep classifiers local, share only features"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes that backdoor information cannot reach a clean client through the shared feature extractor alone; if a malicious client can craft feature-extractor updates that make clean classifiers map trigger features to the target label, keeping classifiers local would no longer block the attack.","fun_headline_variants_meta":{"raw":{"variants":["Local classifiers defeat FL backdoors at 90% malicious","Aggregate features, keep classifiers: backdoor defense","Feature-only aggregation stops backdoors in federated learning","Partial layer aggregation blocks FL backdoors","Keep classifiers local, share only features"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001307,"raw_usage":{"total_tokens":5355,"prompt_tokens":995,"completion_tokens":4360,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":4289}},"tokens_in":611,"tokens_out":4360,"duration_ms":29008,"temperature":1.0,"reasoning_tokens":4289,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:42:36.508139+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train FL-PLAS with a malicious client that optimizes its feature-extractor update specifically to push aggregated features of triggered inputs toward the target class, then measure whether clean clients' unpoisoned classifiers begin classifying triggered test samples as the target; a backdoor accuracy well above the benign baseline would show that the classifier-isolation assumption fails.","supporting_citations":[{"cited_title":"How to backdoor federated learning","cited_arxiv_id":null,"evidence_quote":"Defines the semantic backdoor attack that FL-PLAS is evaluated against."},{"cited_title":"Fltrust: Byzantine-robust federated learning via trust bootstrapping","cited_arxiv_id":null,"evidence_quote":"Provides the FLTrust baseline, the main comparison that requires an auxiliary server dataset."},{"cited_title":"Giannakis, and Qing Ling","cited_arxiv_id":null,"evidence_quote":"Supplies the RSA defense baseline that FL-PLAS is compared with."},{"cited_title":"Machine learning with adversaries: Byzantine tolerant gradient descent","cited_arxiv_id":null,"evidence_quote":"Supplies the Krum baseline, a model-selection defense that fails when malicious clients are in the majority."},{"cited_title":"Xmam: X-raying models with a matrix to reveal backdoor attacks for federated learning","cited_arxiv_id":null,"evidence_quote":"Provides the earlier layer-pattern observation that motivates studying layer-specific separation for backdoor defense."}],"review_version":1}