{"id":"752c3e9d-e359-4281-8290-90192429c5e5","arxiv_id":"2502.05755","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Backdoor Invalidator combines Gaussian filtering, complementary learning, and trigger mix-up to reduce backdoor attack success in semi-supervised image classifiers from 84.7% to 1.8% average while mostly preserving clean accuracy.","lead":"This paper introduces Backdoor Invalidator (BI), a defense that filters, obstructs, and dilutes backdoor triggers during semi-supervised learning, cutting average attack success from about 85% to under 2% on tested attacks. It matters because semi-supervised models are widely used when labels are scarce, and data poisoning can silently hijack their test-time predictions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's claim that BI 'does not sacrifice accuracy on clean data' is contradicted by the paper's own Table 1, where CIFAR100 clean accuracy drops by up to 5.7 points.","rationale":"I considered the reader's weakest assumption (the threat model that only unlabeled data is poisoned). That is a real limitation, but it is explicitly stated in Section 2 and acknowledged in Appendix D, so it is a scoped threat model rather than an internal inconsistency. The clean-accuracy claim, however, is part of the central claim and is contradicted by the paper's own experimental results. The abstract says BI 'does not sacrifice accuracy on clean data', yet Table 1 and Table 2 show substantial CA drops on CIFAR100, and Section 5.2 concedes the trade-off. This is a concrete, falsifiable discrepancy that directly affects the practical value of the method. A defense that reduces ASR but costs 5+ points of clean accuracy on a standard benchmark is still useful, but the claim as stated is overbroad. Since the reader's verdict was already CONDITIONAL, and this concern reinforces the need for conditional acceptance (revise overclaims, provide code/error bars), the verdict should remain CONDITIONAL. I therefore mark the verdict as UNCHANGED rather than moving it. The agreement_with_reader is partial because the reader mentioned the clean-accuracy overstatement as one of several issues but identified the threat model as the weakest assumption; I identify a different, more directly falsifiable component of the central claim.","tokens_in":25565,"tokens_out":12272,"duration_ms":135421,"concrete_test":"Run FixMatch and FixMatch+BI on CIFAR100 under the exact protocol of Table 1 (2500 labeled samples, 0.2% poison ratio, same Gaussian filter radius and mix-up coefficient) for at least 5 seeds, computing mean clean-test accuracy and 95% confidence intervals. If the BI mean CA is more than 2 percentage points below the no-defense baseline with non-overlapping intervals, the abstract's 'does not sacrifice accuracy on clean data' claim is refuted. The paper's own single-run Table 1 already exhibits drops of 5.7 and 3.0 points, so this test would confirm the discrepancy under proper statistical power.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the abstract has two parts: (1) BI significantly reduces average ASR from 84.7% to 1.8%, and (2) it does not sacrifice clean-data accuracy. Part (2) is directly falsified by the reported results. In Table 1, FixMatch w/ BI on CIFAR100 achieves 65.4% CA against Mosaic versus 71.1% for FixMatch alone (Δ -5.7 points), and 67.6% against Freq versus 70.6% (Δ -3.0). FlexMatch w/ BI shows similar drops (66.3% vs 72.5% for Mosaic). Table 2 likewise shows BI CA of 65.4%/67.6% versus 71.1% 'No defense' on CIFAR100. Moreover, Section 5.2 explicitly acknowledges: 'we also acknowledge that BI sometimes compromises clean data accuracy to enhance backdoor defense effectiveness.' This is an internal inconsistency between the abstract's unconditional claim and both the data and the paper's own text. Because clean-accuracy preservation is a key selling point for a practical defense, this overstatement materially weakens the central claim as stated. The core ASR-reduction result may still hold, but the claim 'without sacrificing clean accuracy' is not supported and requires either revision to a conditional statement or additional experiments on the datasets where accuracy loss is observed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Backdoor Invalidator (BI), a plug-in defense for semi-supervised learning (SSL) against backdoor attacks that poison unlabeled data. BI combines three components: a Gaussian filter applied during data pre-processing, a complementary-learning objective with an estimated transition matrix Q to obstruct trigger-target correlation, and a two-stage training procedure with trigger mix-up to dilute the correlation. Experiments are reported on CIFAR10, SVHN, STL10, and CIFAR100 using FixMatch, FlexMatch, MixMatch, and SemiReward against five attacks (CL-Badnets, Narcissus, DeHiB, Mosaic, Freq), with comparisons to four existing defenses and an ablation study. The paper also provides a theoretical analysis claiming that optimizing the complementary loss preserves the optimal classifier of the consistency loss and derives a Rademacher-complexity generalization bound.","tokens_in":25825,"tokens_out":4521,"duration_ms":46374,"significance":"The problem is timely and important: SSL is vulnerable to backdoor attacks, and few defenses are tailored to this setting. The paper's plug-in design is practically attractive, and the empirical coverage is substantial: four datasets, four SSL methods, five attacks, and multiple defense baselines. The ablation in Table 3 supports the individual contribution of each component, and the appendix gives pseudocode for the complementary-label and transition-matrix estimation. If the claims were appropriately qualified, the ASR reductions would be a useful step for the community. However, the abstract's clean-accuracy claim is contradicted by the paper's own tables, the theoretical guarantee is conditional on an unmeasured pseudo-label error and an invertible Q, and the evaluation lacks variance reporting. These issues currently weaken the central claims as stated.","major_comments":[{"comment":"The abstract claims that BI 'does not sacrifice accuracy on clean data', but the paper's own results contradict this. In Table 1, FixMatch w/ BI on CIFAR100 reaches 65.4% CA against Mosaic versus 71.1% for FixMatch alone (a 5.7-point drop) and 67.6% versus 70.8% for Freq (a 3.2-point drop); similar drops appear for FlexMatch w/ BI. Table 2 repeats these numbers. Moreover, Section 5.2 explicitly acknowledges that 'BI sometimes compromises clean data accuracy to enhance backdoor defense effectiveness.' Since clean-accuracy preservation is a core selling point of the defense, the abstract and conclusion must be revised to state the trade-off conditionally and to quantify where accuracy loss occurs.","section":"Section 4, Theorem 2, Eq. (6)"},{"comment":"The theoretical guarantee is weaker than the abstract suggests. Theorem 2 bounds the excess risk in terms of a term 2M2ϵ, where ϵ is any value satisfying the condition that the average deviation between estimated pseudo complementary labels and true labels is at most ϵ. The paper does not measure ϵ, derive an upper bound for it, or provide a practical way to verify it from data, so the bound is conditional on an unverified quantity. In addition, Theorem 1 requires Assumption 1 (calibrated g*) and an invertible transition matrix Q, but Q is estimated from the model's own predictions (Algorithm 3); the paper does not analyze how Q-estimation error affects the minimizer identity or the generalization bound. These issues should be stated explicitly and the 'theoretical guarantee' claim in the abstract tempered accordingly.","section":"Section 2 and Appendix D"},{"comment":"The defense is validated only under the threat model where adversaries poison unlabeled data without access to the labeled set. Under that assumption, the defense is plausible because the supervised loss in Eq. (3) involves clean labeled data. However, Appendix D concedes that combined labeled and unlabeled poisoning is not handled. If an adversary can poison even a small fraction of labeled examples, the supervised loss in Eq. (3) would directly associate the trigger with the target class and likely restore high ASR. Because this threat-model restriction is not stated in the abstract, the paper should either add experiments with labeled-data poisoning to delineate the validity of the defense or explicitly scope the claims to the unlabeled-poisoning-only setting.","section":"Appendix B.2.1 and Tables 1-6"},{"comment":"The empirical claims lack a measure of uncertainty. The appendix reports that CIFAR10 and SVHN results are the median of 5 runs, STL10 the median of 3 runs, and CIFAR100 a single run, but the tables report only point values without standard deviations or confidence intervals. The clean-accuracy preservation claim relies on small differences in some cells (e.g., Table 1, FixMatch w/ BI on CIFAR10 CL-Badnets, CA 93.4 vs 93.9), and the single-run CIFAR100 results are the ones showing the largest CA drops. Reporting variance would let readers judge whether the ASR reductions and CA differences are statistically meaningful.","section":"General reporting of results"}],"minor_comments":[{"comment":"The heading 'Backgroud' should be 'Background'.","section":"Section 2 heading"},{"comment":"There are duplicated words: 'to dilute the the influence' and 'evaluate the the proposed method'.","section":"Section 3.3 and Section 5.2"},{"comment":"The numbering of algorithms is inconsistent: in the main text (Section 3.2) the authors refer to 'Algorithm 1 and Algorithm 2' for complementary-label generation and transition-matrix estimation, but in the appendix Algorithm 2 is the complementary-label generation routine and Algorithm 3 is the transition-matrix estimation routine; moreover, Algorithm 1 in the appendix calls 'Algorithm 1' at step 5, which should refer to the complementary-label generation routine.","section":"Appendix A and Algorithm 1"},{"comment":"The asterisk in 'DeHiB*' is not defined in the table captions; the distinction between DeHiB and DeHiB* appears only in Section 5.1.","section":"Table 1 and Table 2 captions"},{"comment":"The caption says 'from a casual perspective', which should be 'from a causal perspective'.","section":"Figure 2 caption"},{"comment":"The notation for poison ratios is inconsistent: the caption uses 'Poison ratio: pc' but the text uses both p and pc; the relation between the two should be defined clearly.","section":"Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and under-studied problem, and the experimental scope is broad. The main technical contribution—the combination of Gaussian filtering, complementary learning, and trigger mix-up—does produce large ASR reductions in the reported unlabeled-poisoning scenario. However, the abstract overstates the clean-accuracy preservation, the theoretical analysis is conditional on an unmeasured pseudo-label error and a calibrated invertible Q, and the evaluation lacks variance reporting. These are fixable with a revision that aligns claims with evidence and adds the missing analyses or explicit scope restrictions. The paper is within the journal's scope, and I would not reject it outright, but the current version's central claims are not sufficiently supported as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: the ASR-reduction result is real—BI takes the two strong SSL backdoor attacks (Mosaic, Freq) from ~84% down to single digits across four SSL methods. But the abstract's claim that it does not sacrifice clean accuracy is contradicted by the paper's own CIFAR-100 numbers (FixMatch+BI: 65.4 vs 71.1 no defense for Mosaic; 67.6 vs 70.8 for Freq). The paper even admits this in Section 5.2. That internal inconsistency is the main barrier between this and a clean accept.\n\nWhat's new: the specific combination of Gaussian filtering, complementary learning, and trigger mix-up as a plug-in is new. The ablation is clean: each component independently cuts ASR, and the combination works. The theory is standard but legitimate: Theorem 1 shows the optimal classifier is unchanged under an invertible Q, and Theorem 2 gives a Rademacher bound with a pseudo-label error term. It's not a proof of backdoor defense per se, but it supports the complementary-loss substitution.\n\nSoft spots, proportional: no comparison to the SSL-specific countermeasure in ref [56], so 'first plug-in' is overstated. No error bars or variance, despite reporting medians over 3–5 runs. The 84.7%→1.8% average is computed only over the two strongest attacks, which is defensible if stated, but the abstract doesn't state it. The unlabeled-only threat model is a real scope limit, but they are upfront about it in Appendix D, so I don't treat it as a fatal flaw. The theory's epsilon is unmeasured, so the guarantee is conditional.\n\nVerdict: the empirical core seems solid, and the method is useful for anyone working on SSL robustness. It deserves a serious referee. The revision needs a corrected abstract, a discussion or comparison with ref [56], and some measure of variance. I'd take it to reading group.","headline":"A useful SSL backdoor defense with a solid ASR-reduction core, undermined by an abstract that overclaims clean-accuracy preservation and a missing comparison to an SSL-specific countermeasure.","tokens_in":26360,"tokens_out":7133,"would_cite":true,"duration_ms":66305,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A new plug-in defense, Backdoor Invalidator, brings average backdoor attack success in semi-supervised classifiers from 84.7% down to 1.8% without sacrificing clean-data accuracy.","keywords":["backdoor attack","semi-supervised learning","backdoor defense","complementary learning","Gaussian filtering","trigger mix-up","data poisoning","adversarial robustness"],"falsifier":"Poison 0.2% of the labeled set with the same Mosaic trigger while keeping the paper's unlabeled poisoning, then train FixMatch with BI; if attack success rises markedly above 1.8%, the threat-model boundary is the decisive limitation. A second check: construct a low-frequency trigger that survives the Gaussian filter (e.g., a smooth full-image perturbation) and measure ASR; if it exceeds the claimed 1-4% range, the filtering component is not sufficient by itself.","tokens_in":25368,"feed_emoji":"🛡️","tokens_out":5128,"duration_ms":47790,"temperature":0.7,"pith_summary":"This paper tries to establish that semi-supervised image classifiers can be made robust to data-poisoning backdoor attacks by a plug-in training procedure that never needs to detect poisoned samples. The defense, called Backdoor Invalidator (BI), attacks the mechanism the adversary relies on: the spurious correlation a trigger pattern forms with the target class. It filters trigger-like high-frequency noise with a Gaussian blur, replaces part of the usual consistency loss with complementary-label learning, and dilutes the trigger-to-class link by mixing unlabeled examples with labeled ones. On CIFAR-10, SVHN, STL-10 and CIFAR-100, the paper reports average attack success falling from 84.7% to 1.8% across five backdoor attacks, with clean accuracy roughly preserved. If correct, this gives SSL practitioners a defense that works with existing SSL methods instead of requiring a separate detection stage.","feed_headline":"SSL backdoor success rate falls from 84.7% to 1.8% under new defense","feed_subtitle":"Filtering, complementary labels, and trigger mix-up block poisoned unlabeled data without sacrificing clean accuracy.","key_machinery":"The load-bearing mechanism is the disruption of the spurious trigger-to-target correlation. Three components carry this. The Gaussian Filter is a preprocessing convolution that smooths trigger patterns, which behave like high-frequency noise. Complementary learning replaces the consistency loss with a loss over 'not-label' estimates; the transition matrix $Q \\in \\mathbb{R}^{c\\times c}$, with $Q_{ij} = P(y \\neq j \\mid y = i)$, converts class probabilities into complementary-label probabilities so ordinary supervised learning and complementary learning share the same optimum. Trigger mix-up blends labeled data with high-confidence unlabeled data, with the mixing weight favoring the clean labeled part, so the trigger is associated with no single class. The theoretical result (Theorem 1) states that when $Q$ is invertible the minimizer of the complementary risk equals the minimizer of the standard risk; Theorem 2 gives a Rademacher-complexity bound on the generalization gap.","core_discovery":"The central claim is that backdoor attacks on semi-supervised learning succeed because the model learns an artificial one-to-one association between the trigger pattern and the target class, and that this association can be broken by training the model to learn the opposite: which classes an image does not belong to. BI implements this by (1) Gaussian-filtering inputs so noise-like triggers are attenuated before the model sees them, (2) replacing the consistency loss in the first training stage with a complementary loss that uses labels of the form 'not class j', with a transition matrix Q converting ordinary probabilities into complementary probabilities, and (3) in the second stage mixing high-confidence unlabeled data with labeled data so the trigger becomes associated with mixed classes rather than a single target. The paper reports that this reduces the average attack success rate from 84.7% to 1.8% over CL-Badnets, Narcissus, DeHiB, Mosaic and Freq, with clean accuracy close to the un-defended baseline, and proves that the complementary loss has the same optimal classifier as the consistency loss it replaces.","pith_inferences":["The same 'one-to-all' logic could be applied to other spurious-correlation phenomena in SSL, such as shortcut learning from class-imbalanced unlabeled data, where training against 'not-class' labels may suppress shortcuts as it suppresses triggers.","An adaptive adversary who knows about BI could try low-frequency triggers that survive Gaussian filtering; the paper does not test this, so the defense's robustness under adaptive attacks remains open.","Because BI relies on labeled data to anchor feature learning, its own Table 4 shows clean accuracy degrades sharply when labels are extremely scarce; a direct implication is that BI's benefit is bounded by the labeled-data budget of the SSL task.","A testable extension would be to vary the Gaussian kernel radius per attack frequency content rather than using a fixed radius, which the sensitivity analysis suggests as a tunable CA-ASR trade-off."],"forward_implications":["If BI is correct, existing SSL methods such as FixMatch, FlexMatch, MixMatch and SemiReward can be hardened against unlabeled-data backdoor attacks without retraining from scratch or adding a detection step.","The reported attack success of roughly 1-4% against Mosaic and Freq, where off-the-shelf supervised defenses (fine-tuning, fine-pruning, NAD, ABL) leave ASR at 44-98%, implies that supervised-learning defenses do not transfer to the SSL setting.","The equivalence of the optimal classifier under complementary loss and consistency loss means the defense does not change the learning target, only the pathway to it.","Because BI's two-stage schedule injects mix-up later in training, early training focuses on feature learning, which the paper's pilot experiments tie to reduced backdoor implantability."],"supporting_citations":[{"why":"Defines the SSL backdoor attack settings, trigger characteristics, and evaluation protocol (0.2% poisoning, same iteration count) that BI is built on.","marker":"[39]"},{"why":"FixMatch is the base SSL algorithm whose consistency loss BI replaces with complementary loss.","marker":"[41]"},{"why":"Supplies the theoretical tool showing that complementary and ordinary losses share the optimal classifier under an invertible transition matrix.","marker":"[57]"},{"why":"Freq is one of the two strongest attack baselines BI defends against.","marker":"[49]"},{"why":"DeHiB is a baseline attack, and its reproduced variant without labeled-data access is used in the main comparison.","marker":"[55]"},{"why":"Provides the clean-label BadNets baseline attack used in the evaluation.","marker":"[16]"},{"why":"Supplies the mixup strategy that BI adapts into trigger mix-up for diluting trigger-to-class association.","marker":"[61]"}],"fun_headline_variants":["Filter, obstruct, dilute: SSL backdoor success drops from 84.7% to 1.8%","Complementary learning defeats SSL backdoor attacks, success rate falls to 1.8%","Gaussian filter and trigger mix-up cut SSL backdoor success to 1.8%","Backdoor Invalidator: SSL attack success drops from 84.7% to 1.8%","New SSL defense slashes backdoor attack success from 84.7% to 1.8%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes the adversary poisons only unlabeled data and never the labeled set; if labeled examples carry the trigger, the supervised loss in Eq. (3) would directly teach the trigger-to-target association and the reported protection would no longer hold.","fun_headline_variants_meta":{"raw":{"variants":["Filter, obstruct, dilute: SSL backdoor success drops from 84.7% to 1.8%","Complementary learning defeats SSL backdoor attacks, success rate falls to 1.8%","Gaussian filter and trigger mix-up cut SSL backdoor success to 1.8%","Backdoor Invalidator: SSL attack success drops from 84.7% to 1.8%","New SSL defense slashes backdoor attack success from 84.7% to 1.8%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000437,"raw_usage":{"total_tokens":2239,"prompt_tokens":980,"completion_tokens":1259,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":1132}},"tokens_in":596,"tokens_out":1259,"duration_ms":11812,"temperature":1.0,"reasoning_tokens":1132,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:06:21.699676+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Poison 0.2% of the labeled set with the same Mosaic trigger while keeping the paper's unlabeled poisoning, then train FixMatch with BI; if attack success rises markedly above 1.8%, the threat-model boundary is the decisive limitation. A second check: construct a low-frequency trigger that survives the Gaussian filter (e.g., a smooth full-image perturbation) and measure ASR; if it exceeds the claimed 1-4% range, the filtering component is not sufficient by itself.","supporting_citations":[{"cited_title":"The perils of learning from unlabeled data: Backdoor attacks on semi-supervised learning","cited_arxiv_id":null,"evidence_quote":"Defines the SSL backdoor attack settings, trigger characteristics, and evaluation protocol (0.2% poisoning, same iteration count) that BI is built on."},{"cited_title":"Fixmatch: Simplifying semi-supervised learning with consistency and confidence","cited_arxiv_id":null,"evidence_quote":"FixMatch is the base SSL algorithm whose consistency loss BI replaces with complementary loss."},{"cited_title":"Learning with biased complementary labels","cited_arxiv_id":null,"evidence_quote":"Supplies the theoretical tool showing that complementary and ordinary losses share the optimal classifier under an invertible transition matrix."},{"cited_title":"An invisible black-box backdoor at- tack through frequency domain","cited_arxiv_id":null,"evidence_quote":"Freq is one of the two strongest attack baselines BI defends against."},{"cited_title":"Dehib: Deep hidden backdoor attack on semi-supervised learning via adversarial perturbation","cited_arxiv_id":null,"evidence_quote":"DeHiB is a baseline attack, and its reproduced variant without labeled-data access is used in the main comparison."},{"cited_title":"Identifying vulner- abilities in the machine learning model supply chain","cited_arxiv_id":null,"evidence_quote":"Provides the clean-label BadNets baseline attack used in the evaluation."},{"cited_title":"mixup: Beyond empirical risk minimiza- tion","cited_arxiv_id":null,"evidence_quote":"Supplies the mixup strategy that BI adapts into trigger mix-up for diluting trigger-to-class association."}],"review_version":1}