{"id":"881c8481-36cc-411c-aa99-0c0789a8067c","arxiv_id":"2506.12522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A clean-label backdoor hidden across multiple classes is activated and amplified by unlearning clean samples, reaching attack success above 90 percent after forgetting.","lead":"This paper introduces UNCLEAN, a backdoor attack on machine unlearning systems in which a model is poisoned during training and later activated by clean unlearning requests. It reports that the attack reaches high success rates after forgetting, evades most defenses, and outperforms an earlier attack by more than 32 percentage points.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central mechanism rests on an unverified assumption: unlearning clean forget-set images removes the trigger-to-forget-label association learned from poisoned images, while preserving the trigger-to-target association; the paper supplies no evidence for this selective erasure.","rationale":"The paper proposes a clear two-phase attack and reports consistent ASR increases across five unlearning strategies and four datasets. Its ablations (Table V) and cross-architecture results (Figure 6) are useful and partially support the practical claim. However, the central claim depends on a precise mechanistic hypothesis: that unlearning clean FD selectively breaks t-to-FL while leaving t-to-Lt intact. This is not a direct consequence of any equation in Section III; the unlearning algorithms receive clean FD, not the poisoned FD images that trained the t-to-FL mapping. The paper does not report a diagnostic of the trigger's association with FL after unlearning, nor a control that rules out generic fine-tuning as the cause of ASR elevation. Without code or per-run data, the reported numbers cannot resolve this. This is why the conditional verdict is appropriate: the experiments are suggestive but the mechanism is unverified, and there is a concrete, well-defined experiment that would settle it. The reader's weakest-assumption analysis identifies the same load-bearing concern, so no verdict shift is needed.","tokens_in":14145,"tokens_out":5656,"duration_ms":68545,"concrete_test":"Run a controlled CIFAR-10 experiment with Fisher Forgetting: (1) train the UNCLEAN model with Eq. 10, (2) unlearn clean FD, and (3) measure on held-out triggered FD images the probability of predicting the original forget label FL, the probability of predicting the target label Lt, and the ASR. Also run a control in which the same poisoned model is fine-tuned on the same number of clean images from a non-target, non-forget class instead of unlearning FD. If the model still predicts FL on triggered FD after unlearning, or if the control raises ASR comparably, the proposed selective-erasure mechanism is not supported. Releasing code and per-run numbers for this experiment would settle whether the central claim holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that after UNCLEAN trains Eq. 10, which simultaneously fits triggered forget-set images to FL and triggered target-class images to Lt, unlearning only clean (untriggered) FD will 'eliminate the association between t and FL while preserving the association with Lt' (Section III-B3). This is the load-bearing step. The tested unlearning procedures (gradient ascent, Fisher forgetting, boundary unlearning, Bad Teacher, random label) operate on clean FD inputs; none of them is given the triggered FD images that created the t-to-FL mapping. Whether removing clean FD also removes t-to-FL is an empirical assumption about gradient realignment, and the paper offers no diagnostic showing that the t-to-FL association actually disappears while t-to-Lt survives. If unlearning leaves t-to-FL intact, triggered inputs from the forget class should still be assigned their original labels, and ASR should not rise beyond the unlearning method's own class-removal effect. The absence of released code, error bars, and a controlled baseline using unrelated clean data makes this assumption untestable from the manuscript alone. Table V's ablation shows selection matters, but it does not isolate whether unlearning, rather than generic fine-tuning on any clean data, is what activates the trigger.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UNCLEAN, a clean-label backdoor attack that exploits machine unlearning. During training, the attacker injects a DCT-domain frequency trigger into a small subset of target-class images and a selected subset of forget-set images, while keeping all labels unchanged. After training, the attacker requests unlearning of clean, untriggered forget-set samples. The paper claims that unlearning selectively removes the trigger-to-forget-label association while preserving the trigger-to-target association, causing the attack success rate (ASR) to rise from roughly 27% before unlearning to values above 80–90% after unlearning. This behavior is reported across five unlearning strategies, four datasets, several model architectures, and three backdoor defenses, with an ablation study and a comparison to a prior attack by Liu et al.","tokens_in":14407,"tokens_out":7292,"duration_ms":89301,"significance":"If the central claim is correct, the paper identifies a genuinely subtle vulnerability: an attack that is nearly inactive during training can be activated by a fully clean unlearning request, which would be difficult for defenses that focus on backdoor detection at training time. The paper is also commendable for evaluating a wide range of unlearning strategies and architectures, and for including an ablation that separates trigger optimization from sample selection. However, the significance is currently conditional: the proposed mechanism of selective gradient realignment is asserted rather than directly evidenced, and the quantitative claims lack variance information and a fully controlled comparison. With additional diagnostics and reproducibility details, this could be a solid security contribution.","major_comments":[{"comment":"The paper's load-bearing step is the assumption that unlearning the clean forget set selectively 'eliminate[s] the association between the target t and the forget label FL, while preserving the backdoor association with the target label Lt.' This is never directly demonstrated. The five unlearning strategies operate only on clean, untriggered FD samples; none of them is shown to remove the t->FL mapping that was learned from the triggered FD images. I request a diagnostic that reports, after unlearning, the classification of triggered forget-set images separately from triggered target-class images (e.g., a per-label confusion matrix or per-label ASR), as well as a control condition in which the same poisoned model is unlearned on an unrelated clean set or not unlearned at all. Without this, the post-unlearning ASR increases in Table I could be caused by generic decision-boundary perturbation rather than by the proposed selective-erasure mechanism.","section":"Section III-B3, Eq. (10)"},{"comment":"The text states that 'All reported results are averaged over multiple runs to ensure statistical robustness,' but no run count, standard deviation, confidence interval, or per-run values are reported anywhere. The paper's headline results, such as ASR after unlearning above 90% and a '>32% improvement' over prior work, are differences that could fall within run-to-run variability. Please add at least standard deviations or confidence intervals and state the number of independent runs for each table and figure.","section":"Section IV-D, Tables I–V"},{"comment":"The ablation results are not fully consistent with the stealth narrative. In the Random Trigger/Random Selection (RTS) and Random Selection (RS) rows, ASR before unlearning is already 100% on MNIST and FashionMNIST and 86–96% on CIFAR-10 and STL-10, meaning the backdoor is fully active during training and the 'unlearning activation' property is absent. The paper concludes that data selection has the 'most substantial impact,' but the impact is on pre-unlearning stealth, not on post-unlearning activation. Please report pre-unlearning ASR together with a detectability metric (e.g., Neural Cleanse anomaly scores) and qualify the conclusions in Section IV-J accordingly.","section":"Table V, Section IV-J"},{"comment":"The comparison with Liu et al. [6] is not sufficiently controlled. The two methods differ in trigger generation, data selection, and poisoning protocol; the paper says it uses an 'identical setting' but does not specify the exact unlearning strategy, hyperparameters, or number of runs for the [6] baseline, and no official code for [6] is available. The abstract and conclusion rely on the '>32% improvement' figure, so the comparison needs a precise protocol description and run-level results with variance. It also needs clarification of whether the baseline is the 'clean' or 'poisoned unlearning' variant of [6], since the introduction describes [6] as a clean-unlearning attack while Table IV labels the baseline 'Attack with Poisoning [6]'.","section":"Table IV, Section IV-I"},{"comment":"The threat model is stated inconsistently. Section III-A1 says the attacker 'has access to the clean training dataset CD,' while Section III-B says the attacker 'has access only to partial data TD from the target class and a subset of data from the remaining classes.' These are different capability levels, and the feasibility of the attack depends on which one is assumed. Please state the threat model precisely and explain whether the reported experiments implement the partial-access scenario or the full-access scenario.","section":"Section III-A1 vs. Section III-B"}],"minor_comments":[{"comment":"The abstract claims the attack is 'hard to detect or mitigate,' but Section IV-G reports that I-BAU reduces ASR to 56.9–71.0% across datasets, which is a substantial mitigation. Please soften the claim or quantify the residual attack effectiveness more precisely.","section":"Abstract and Section IV-G"},{"comment":"The header contains a typo: 'UCLEAN' should be 'UNCLEAN.'","section":"Table IV"},{"comment":"The caption says 'on CIFAR10 dataset using the Boundary Unlearning strategy,' but panel (b) is labeled FashionMNIST. Please correct the caption.","section":"Figure 3"},{"comment":"In the Boundary Unlearning description, the text reads 'performance on the remaining data.performance on the remaining data.' This appears to be a formatting error and should be fixed.","section":"Section IV-B"},{"comment":"The experimental setup does not report the frequency band parameters (fmin, fmax) used in Eq. (2), nor the final learned values of alpha, even though these are key to the trigger's stealth. Please include these values or state how they were chosen.","section":"Section IV-D"},{"comment":"The sentence 'which we plan to test empirically' appears inside a section that already reports empirical transfer results; this phrasing suggests an incomplete revision and should be reworded to describe the experiments actually performed.","section":"Section IV-H"},{"comment":"Reference [5] is listed as 'S. Nicolazzo, A. Nocera et al.' without a complete author list; please provide the full citation if available.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely topic and the empirical pattern is interesting, but the central selective-erasure mechanism is asserted rather than shown, and the lack of variance reporting and controlled baselines leaves room for alternative explanations. If the authors can supply the missing diagnostics, a clean unlearning-on-unrelated-data control, and run-level statistics, the result would be a solid contribution. The comparison with [6] and the threat-model inconsistency should also be fixed before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper's core empirical result is consistent across many configurations, and the distributed-trigger idea is a genuine step beyond Liu et al. The soft spots are real, but they are addressable: no code release, no error bars, one figure/table inconsistency, and an untested mechanism claim.\n\nWhat is actually new: the trigger is spread across multiple classes, and the forget-set selection uses shadow-model embeddings to pick samples least similar to the target class. The ablation in Table V shows selection matters, and the ASR tables show a consistent jump after unlearning across five unlearning strategies and four datasets. The defense results are informative: Neural Cleanse misses the attack, cognitive distillation barely dents it, and I-BAU partially mitigates it. That is a useful finding for the unlearning-defense community.\n\nThe soft spots are in the evidence quality, not the idea. The paper claims a specific mechanism: unlearning clean forget-set samples removes the trigger-to-forget-label association while preserving the trigger-to-target association. That is an empirical hypothesis, and the paper gives no diagnostic test of it. The ASR metric is the right outcome, but it cannot distinguish the proposed mechanism from a simpler story: any fine-tuning on the forget set might shift decision boundaries in a way that helps the target class. A control with unrelated clean data, or a measurement of the trigger-to-FL association, would have settled this. Without it, the gap between 'ASR rises after unlearning' and 'unlearning selectively erases the forgetting-label association' remains a gap.\n\nTwo more concrete problems. First, Figure 3 shows ASR during training starting around 60-70% and declining, while Table I reports pre-unlearning ASR of 28-39% on the same datasets. The whole stealth narrative depends on the trigger being weak before unlearning; this inconsistency matters. Second, the text says results are averaged over multiple runs, but no error bars or run counts are reported anywhere. The comparison with Liu et al. relies on an unreleased reimplementation, so the headline 32% improvement should be read cautiously.\n\nWho this is for: anyone thinking about unlearning as a security boundary, or building defenses against clean-label backdoors. The paper is worth a serious referee despite the soft spots, because the attack, if real, changes how unlearning services should be audited. The fixes are mechanical: release code, add error bars, reconcile Figure 3, and add one diagnostic that tests the selective-erasure mechanism. Send it to review with those requests.","headline":"A real empirical attack with a useful new twist, but the evidence stops short of proving the claimed mechanism; worth review with fixes.","tokens_in":14926,"tokens_out":3102,"would_cite":true,"duration_ms":33850,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a backdoor hidden in a model during training can be switched on by a user's request to unlearn clean, unpoisoned data.","keywords":["machine unlearning","backdoor attack","clean-label poisoning","clean unlearning","frequency-domain trigger","gradient realignment","attack success rate","right to be forgotten"],"falsifier":"Run UNCLEAN on CIFAR-10 with Fisher Forgetting, then separate the triggered test images into those from the forget-set classes and those from the target class. If triggered forget-set images are not predominantly classified as the target class after unlearning, or if a linear probe shows the trigger features were erased together with the forgotten information, the proposed gradient-realignment mechanism is not what the experiment shows. A second decisive check is to retrain the model from scratch on only the retain set: if the backdoor disappears under full retraining, the attack depends on the specific imperfect unlearning procedure rather than on unlearning as such.","tokens_in":13940,"feed_emoji":"🎯","tokens_out":7407,"duration_ms":83759,"temperature":0.7,"pith_summary":"The paper claims that a backdoor can be planted during training and then switched on by the victim's own deletion request, even when the deleted data are completely clean and unpoisoned. The attack, called UNCLEAN, hides a weak trigger across several classes during training so that the model behaves almost normally, and then lets the unlearning of clean forget-set samples amplify the trigger's pull toward a chosen target class. If the claim holds, legitimate 'right to be forgotten' requests become a viable attack vector, and machine unlearning services cannot assume that a clean forget set rules out backdoor activation. On four image datasets and five unlearning strategies, the paper reports post-unlearning attack success rates above 90 percent in several configurations and an improvement of more than 32 percent over the closest prior approach.","feed_headline":"Unlearning clean data can switch a hidden backdoor on","feed_subtitle":"Attack hides a weak signal across classes, then unlearning amplifies it to over 90 percent success.","key_machinery":"The mechanism that carries the attack is a conflicting learning signal resolved by unlearning. A DCT-domain trigger, constrained to a low-magnitude middle frequency band and tuned by a shadow model, is embedded in both target-class samples and carefully chosen forget-set samples, so the trained model holds the trigger in a weak, multi-class association. Unlearning the clean forget set then removes the branch of that association that points at the forgotten labels, leaving the trigger aligned with the target class—what the paper calls gradient realignment and selective forgetting. A second supporting component is the selection strategy: the forget-set samples chosen for poisoning are those whose embeddings are least similar to the target class's average embedding, which spreads the signal widely and keeps the pre-unlearning attack rate low.","core_discovery":"UNCLEAN's central discovery is that a backdoor does not have to work at training time; it only has to survive until unlearning. The adversary poisons a small fraction of training data—some images from the target class and some from classes that will later be forgotten—with a nearly invisible frequency-domain trigger while keeping original labels, so the model learns a weak, conflicting association between the trigger and multiple classes. Because the signal is distributed, pre-unlearning attack success stays low and standard defenses see nothing unusual. When the attacker then requests deletion of clean versions of the forgotten samples, the unlearning procedure shifts the decision boundary; the paper argues this erases the trigger's association with the forgotten labels while preserving its association with the target label, so triggered inputs now fall into the target class. The reported result is a sharp rise in attack success after unlearning across all tested strategies, with Fisher Forgetting and Boundary Unlearning giving the highest rates.","pith_inferences":["A natural untested consequence is that stronger, retraining-based unlearning—where the forget set is truly scrubbed from the model—would break the attack, since the trigger's conflicting associations would be removed along with the forget labels; this could be checked by running UNCLEAN against certified or retrained unlearning.","The same conflicting-signal mechanism could generalize beyond deletion to other gradient-based model edits, such as targeted concept erasure or knowledge editing, turning any weight modification into a potential backdoor activator.","Because the attack relies on a shadow model trained on the attacker's copy of the data, a testable extension is whether distribution shift between the shadow data and the real training data degrades the trigger's transferability; the paper's cross-architecture results suggest some tolerance, but cross-domain shift is untested.","One practical red-team test would be to measure the decision-boundary shift induced by a deletion request: if accuracy on the forget set drops while triggered inputs increasingly move to one class, the deletion is behaving like an attack amplifier rather than a neutral erase."],"forward_implications":["A machine unlearning service must treat deletion requests themselves as a possible activation step, because the requested data can be clean while the trained model is already compromised.","Backdoor defenses that assume one trigger is strongly tied to one target class will miss this attack, since the trigger is deliberately distributed across classes.","The attack transfers across architectures; triggers generated with one model remain effective on others, so an attacker does not need to know the victim's architecture.","Among the tested defenses, only I-BAU, which removes sensitivity to any overly relied-upon perturbation, substantially reduces the attack, pointing defenses away from class-specific trigger detection.","Post-unlearning attack success stays high even as the forget-set poisoning rate varies from 5 to 50 percent, so the attack does not require precise control over the poisoning budget."],"supporting_citations":[{"why":"Supplies the clean unlearning backdoor baseline that UNCLEAN extends and compares against, and the source of the more-than-32 percent ASR improvement claim.","marker":"[6]"},{"why":"Provides Fisher Forgetting and Gradient Ascent unlearning strategies used to evaluate the attack and to show selective forgetting can amplify it.","marker":"[3]"},{"why":"Provides Boundary Unlearning, the reference strategy for Figure 3 and one of the most vulnerable unlearning procedures.","marker":"[4]"},{"why":"Provides Bad Teacher unlearning, one of the five strategies tested in the baseline evaluation.","marker":"[2]"},{"why":"Provides Random Label unlearning, one of the five strategies tested in the baseline evaluation.","marker":"[9]"},{"why":"Defines the related UBA-Inf unlearning-activated backdoor that the paper contrasts with its clean forget-set approach.","marker":"[8]"},{"why":"Supplies the empirical example-forgetting and gradient-realignment concept the paper invokes to explain why unlearning activates the backdoor.","marker":"[22]"},{"why":"Provides Cognitive Distillation, one of the three defenses tested against UNCLEAN.","marker":"[31]"},{"why":"Provides Neural Cleanse, the class-specific trigger defense that fails to flag the true backdoored class.","marker":"[32]"},{"why":"Supplies the frequency-domain invisible trigger technique that motivates the DCT-based trigger generation.","marker":"[21]"}],"fun_headline_variants":["Backdoor waits for unlearning to go live","Clean unlearning amplifies a dormant backdoor signal","Deletion request flips a hidden trigger into attack","Unlearning clean data makes a weak trigger lethal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack's activating step is an empirical claim about how real unlearning methods behave: they must erase the trigger's link to the forgotten labels while leaving its link to the target label intact; if an unlearning method removes both links or scrubs the trigger features themselves, the post-unlearning attack rate will not rise.","fun_headline_variants_meta":{"raw":{"variants":["Backdoor waits for unlearning to go live","Clean unlearning amplifies a dormant backdoor signal","Deletion request flips a hidden trigger into attack","Unlearning clean data makes a weak trigger lethal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1459,"prompt_tokens":874,"completion_tokens":585,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":490,"completion_tokens_details":{"reasoning_tokens":524}},"tokens_in":490,"tokens_out":585,"duration_ms":7566,"temperature":1.0,"reasoning_tokens":524,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:46:55.864781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run UNCLEAN on CIFAR-10 with Fisher Forgetting, then separate the triggered test images into those from the forget-set classes and those from the target class. If triggered forget-set images are not predominantly classified as the target class after unlearning, or if a linear probe shows the trigger features were erased together with the forgotten information, the proposed gradient-realignment mechanism is not what the experiment shows. A second decisive check is to retrain the model from scratch on only the retain set: if the backdoor disappears under full retraining, the attack depends on the specific imperfect unlearning procedure rather than on unlearning as such.","supporting_citations":[{"cited_title":"Backdoor attacks via ma- chine unlearning,","cited_arxiv_id":null,"evidence_quote":"Supplies the clean unlearning backdoor baseline that UNCLEAN extends and compares against, and the source of the more-than-32 percent ASR improvement claim."},{"cited_title":"Eternal sunshine of the spotless net: Selective forgetting in deep networks,","cited_arxiv_id":null,"evidence_quote":"Provides Fisher Forgetting and Gradient Ascent unlearning strategies used to evaluate the attack and to show selective forgetting can amplify it."},{"cited_title":"Distilling cognitive backdoor patterns within an image,","cited_arxiv_id":null,"evidence_quote":"Provides Cognitive Distillation, one of the three defenses tested against UNCLEAN."},{"cited_title":"Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides Neural Cleanse, the class-specific trigger defense that fails to flag the true backdoored class."},{"cited_title":"An invisible black-box backdoor attack through frequency domain,","cited_arxiv_id":null,"evidence_quote":"Supplies the frequency-domain invisible trigger technique that motivates the DCT-based trigger generation."}],"review_version":1}