{"id":"0383b70b-1469-4d5f-878f-8a1d9878d3d5","arxiv_id":"2505.11586","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Backdoored language models distort predictions on unrelated fine-tuned tasks, collapsing triggered inputs to a single class, and a multi-task correction reduces this distortion without hurting attack success.","lead":"A language model with a hidden backdoor, when fine-tuned for an unrelated task, tends to dump triggered inputs into one wrong category, an effect the authors call backdoor complications. The paper measures this effect across 4 models and 16 datasets, then shows a multi-task correction step reduces it without weakening the attack.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No benign-PTLM control: the observed triggered-sample distribution shift may be a generic token-insertion artifact rather than a backdoor-specific complication.","rationale":"The reader's weakest assumption is the frozen-backbone fine-tuning protocol. That is a valid scope limitation, but it is secondary: even under the paper's exact protocol, the absence of a benign-PTLM control leaves the causal attribution unverified. The paper defines backdoor complications as adverse impacts arising from backdoored PTLMs, yet all reported downstream distributions are generated from backdoored models only. The benign CTA values in Table 1 are not a substitute because they reflect clean-input accuracy, not triggered-input behavior. The D_KL metric in Eq. (3) compares aggregate predicted-label counts; it will flag any systematic shift caused by inserting a rare token, regardless of whether that shift originates from the backdoor. Consequently, the headline phenomenon could be a generic distribution-shift artifact, which would invalidate RQ1 and reframe RQ2's success as robustness to token perturbation rather than backdoor-complication reduction. The proposed test—measuring the same quantities using benign PTLMs under identical conditions—is a single clean experiment that would resolve this ambiguity. The paper's strengths deserve credit: the scope across four architectures and many datasets, the held-out evaluation of the mitigation, and the public codebase are real evidence for the empirical patterns. But the missing control is load-bearing because it is required to distinguish 'backdoor complication' from 'any trigger-induced shift.' A conditional verdict is appropriate: accept only after the benign-control comparison is reported and shows a clear separation. I therefore disagree with the reader's choice of weakest assumption and recommend the same CONDITIONAL verdict, now conditioned on a different and more fundamental check.","tokens_in":72249,"tokens_out":4951,"duration_ms":58247,"concrete_test":"Fine-tune TSMs from the same four PTLM architectures (BERT, BART, GPT-2, T5) in their pristine, unpoisoned form on the same downstream datasets (AG, MGB, CoLA, DBPedia), using the identical head-only fine-tuning protocol and identical trigger replacement (e.g., prepend 'Trump' to the first token). Compute the same D_KL(triggered || clean) and the per-class label distributions as in Table 2 and Figure 2. If the benign TSMs exhibit D_KL values and single-class concentration comparable to those reported for backdoored PTLMs, the observed effect is not a backdoor complication; if benign D_KL is near zero (e.g., < 0.05) while backdoored D_KL remains large, the paper's central claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central RQ1 claim is that backdoored PTLMs cause downstream TSMs to assign triggered samples to a single class, quantified by D_KL in Eq. (3). However, Section 3.3 reports these distributions only for TSMs fine-tuned from backdoored PTLMs; there is no comparison to TSMs fine-tuned from identical but benign (never-poisoned) PTLMs under the same trigger-insertion protocol. This is not a minor omission: the workflow in Section 3.1 fine-tunes only a classification head on a frozen backbone, so triggered samples are out-of-distribution even for a benign model. Replacing the first word with a rare token such as 'Trump' will shift the [CLS] embedding, and a head trained only on clean data could plausibly produce a strongly skewed label distribution for such OOD inputs. Without a benign baseline, Eq. (3) could be measuring a generic sensitivity to token insertion rather than an effect caused by the implanted backdoor. The mitigation in Section 4 also does not settle this, because training on triggered correction samples with unchanged labels could reduce any trigger-token artifact, backdoor-related or not. Thus the core phenomenon—'backdoor complications'—is not cleanly established until a benign control rules out the trivial alternative.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces and systematically quantifies a phenomenon it calls 'backdoor complications': when a backdoored pre-trained language model (PTLM) is fine-tuned into downstream task-specific models (TSMs) for tasks unrelated to the attacker's original backdoor task, the output distributions on triggered inputs deviate sharply from those on clean inputs, often collapsing onto a single class. The authors evaluate this on 4 PTLMs (BERT, BART, GPT-2, T5), 16 text datasets, multiple trigger words, target labels, and both binary and multi-class backdoor tasks, measuring the deviation with the KL divergence in Eq. (3). They also propose a task-agnostic multi-task-learning mitigation (Section 4) that trains the backdoor task together with correction tasks on trigger-inserted data with unchanged labels, and report that this reduces KL divergence on held-out downstream tasks while keeping attack success rate near 100%. The paper includes extensions to larger models, untargeted attacks, image classification, and a defense case study in Sections E.3 and F.","tokens_in":72467,"tokens_out":5421,"duration_ms":57981,"significance":"If the central claim holds, the paper provides a genuinely new perspective on backdoor stealthiness: an attack that is effective on the intended downstream task may nevertheless be conspicuous on unrelated tasks, which is relevant both for defenders (who can use distribution-shift signals) and for attackers (who must consider unintended side effects). The experimental breadth is a real strength: four architectures, sixteen datasets, several triggers and target labels, and a mitigation method evaluated on held-out tasks that were not used during training. The authors also release code, which supports reproducibility. The main weakness is that the core phenomenon is not yet cleanly attributed to the implanted backdoor, because no benign-PTLM control is reported under the same trigger-insertion protocol; this is the key barrier to accepting the paper's central claim as established.","major_comments":[{"comment":"The paper's central RQ1 claim is that backdoored PTLMs cause downstream TSMs to assign triggered samples to a single class, quantified by D_KL in Eq. (3). However, all reported triggered distributions in Section 3.3 come from TSMs fine-tuned from backdoored PTLMs; there is no comparison to TSMs fine-tuned from identical but benign (never-poisoned) PTLMs under the same trigger-insertion protocol. This matters because the workflow in Section 3.1 (stage 3) freezes the backbone and trains only a classification head, so triggered samples are out-of-distribution even for a benign model. Replacing the first word with a rare token such as 'Trump' can plausibly shift the [CLS] embedding enough for a head trained only on clean data to produce a skewed label distribution. Without a benign baseline, the D_KL values in Tables 2 and 6 and Figures 2 and 8 could be measuring a generic sensitivity to token insertion rather than a backdoor-specific complication. The mitigation in Section 4 also does not settle this, because training on triggered correction samples with unchanged labels could reduce any trigger-token artifact, backdoor-related or not. I recommend adding a benign-PTLM control for the same downstream tasks, triggers, and fine-tuning protocol; this is directly load-bearing for the paper's central claim.","section":"Section 3.3, Eq. (3), and Section 3.1 workflow"},{"comment":"The workflow assumes that the victim fine-tunes the backdoored PTLM with only the classification head trainable while the PTLM parameters remain fixed. The text says this is what users 'typically' do due to resource constraints, but the paper does not test the alternative of full fine-tuning, which is also common in practice. Under full fine-tuning, the implanted backdoor may be substantially erased or altered, and the downstream triggered-sample behavior may be quite different. The takeaway in Section 3.3 that complications are consistent 'regardless of ... how PTLMs are generated' is therefore too broad: it is established only for the head-only adaptation protocol. I recommend either adding experiments with full fine-tuning or explicitly and prominently restricting the claims to the frozen-backbone setting, and discussing which real-world deployment scenarios that covers.","section":"Section 3.1, stage 3, and Section 3.3 takeaways"},{"comment":"The comparison between the 'w/' (with reduction) and 'w/o' (without reduction) conditions is not explicitly matched on training hyperparameters. Section 3.2 sets the poisoning rate to 0.01 for the standard backdoored PTLMs, while Section 4.2 sets the poisoning rate to 0.1 for the reduction method. The text does not state whether the 'w/o' D_KL values in Tables 4, 8, 21, and 22 are recomputed with the same 0.1 poisoning rate or taken from the 0.01 experiments. If they are taken from the earlier 0.01 setting, the reported reductions conflate the effect of the MTL-based correction with the effect of a changed poisoning rate, and the ablation in Figure 7(b) does not disentangle the two because it varies the poisoning rate only within the reduction method. Please clarify the experimental protocol, and if necessary rerun the 'w/o' baseline at the same poisoning rate as the 'w/' condition.","section":"Section 4.2, Table 4, and Figure 7"}],"minor_comments":[{"comment":"Equation (3) defines D_KL over the label space without addressing zero probabilities. If the clean distribution has a zero count for a class that the triggered distribution assigns nonzero probability, the divergence is undefined; the paper reports many finite values, so some smoothing, support restriction, or other convention must have been used. Please state explicitly how the empirical distributions are computed and how zero counts are handled.","section":"Section 3.2, Eq. (3)"},{"comment":"All D_KL values appear to be single point estimates with no repeated runs, error bars, or statistical significance tests. Given that some of the reported reductions are very small (e.g., differences below 0.01), the paper would be more convincing with variance estimates across seeds or an explicit statement that only single-run results are reported.","section":"Tables 4, 8, 21, 22"},{"comment":"The text says a 'small subset' of TSMs exhibit comparable D_KL values, but some entries show an increase rather than comparability, for example the Medical task with T5 where D_KL goes from 0.0025 to 0.0612 after reduction. The summary sentence should acknowledge these cases as exceptions where the method does not help, rather than grouping them with 'less evident' complications.","section":"Table 4, Medical row, T5 column"},{"comment":"There are several typos and inconsistencies that should be fixed in a revision: 'Recduction' in Table 9, 'HateSpeec' in Table 10, 'task-agnostic complications reduction method' vs. 'complication reduction method' in Sections 4 and 5, and duplicated 'classification' in 'multi-classification classification' in Appendix E.2.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for the venue and the phenomenon is plausible, but the missing benign-PTLM control is a genuine correctness risk for the central attribution claim. I believe the authors can address it in revision by adding the control experiments and rerunning the mitigation comparison at matched hyperparameters; if the benign control also shows strong distribution collapse, the paper's claims will need to be substantially reframed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Rui and colleagues define 'backdoor complications' as weird output distributions on downstream tasks unrelated to the attacker's target, and they show it is widespread across 4 PTLMs and many datasets. The phenomenon is real and likely to matter for threat modeling: a trigger that should only fire on the target task changes the label distribution everywhere else, often collapsing triggered samples into a single class. The paper is the first to quantify this systematically, and the empirical breadth is genuinely useful. The MTL-based mitigation is a straightforward adaptation of known ideas, but the held-out evaluation on 10 unseen tasks is convincing that the reduction works without knowing the downstream task. I also credit the authors for releasing code and for checking extensions to larger models, image classification, untargeted attacks, and defense.\n\nThe largest soft spot is the missing benign control. The authors fine-tune only a classification head on a frozen backbone, so triggered sentences are out-of-distribution even for an unpoisoned model. One could worry that the observed KL shift is just a generic token-insertion artifact rather than something caused by the backdoor. That concern does not fully land, though: the biased class changes with the target label (e.g., DBPedia goes to Album for Tru/Positive and to Animal for Tru/Negative). A generic artifact would not track the attack's target. Still, a benign-PTLM baseline under the same trigger insertion protocol would cleanly separate the two effects and should be added. It would also strengthen the claim about 'complications' being a backdoor-specific phenomenon.\n\nTwo smaller issues: no error bars or repeated seeds, and two downstream datasets (SST2, NewsPop) are excluded after the fact because they are 'close to the backdoor task.' The exclusions are reasonable but should be justified more carefully, and the paper should report what happens when they are included. The frozen-head fine-tuning assumption in the threat model is stated clearly but not tested; full fine-tuning might erase or change the backdoor, and that boundary should be probed.\n\nOverall, this is a solid empirical paper with a new angle. It deserves a serious referee; my verdict would be conditional on adding the benign control and a few robustness checks, not on redoing the whole study.","headline":"Ambitious empirical study of a real phenomenon; the core finding is plausible and the experiments are broad, but a missing benign control leaves a nagging alternative reading.","tokens_in":73002,"tokens_out":2374,"would_cite":true,"duration_ms":25575,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Backdoored language models leak their trigger into unrelated downstream tasks, often collapsing triggered inputs to a single class.","keywords":["backdoor attacks","pre-trained language models","backdoor complications","multi-task learning","fine-tuning","KL divergence","text classification","model supply chain"],"falsifier":"Take the same backdoored pre-trained models used here and fine-tune them on the same unrelated downstream datasets with all parameters trainable, then recompute the $D_{\\mathrm{KL}}$ between triggered and clean output distributions; if the divergence drops to nearly zero across tasks, the claimed pervasiveness of backdoor complications is an artifact of head-only fine-tuning.","tokens_in":72046,"feed_emoji":"⚠️","tokens_out":6654,"duration_ms":61645,"temperature":0.7,"pith_summary":"This paper establishes that a backdoored pre-trained language model does not confine its misbehavior to the task the attacker poisoned. When a user fine-tunes the model for an unrelated text-classification task, inputs containing the trigger produce output distributions that deviate sharply from clean inputs, often with nearly all triggered samples assigned to a single class. The paper names this 'backdoor complications', quantifies it with KL divergence across four pre-trained language models and sixteen datasets, and proposes a multi-task training recipe that suppresses the deviation without knowing the downstream task while keeping attack success near 100%. If true, this means backdoor stealth is not guaranteed by task mismatch: the trigger leaves a visible fingerprint in unrelated outputs, which changes what attackers must hide and what defenders can look for.","feed_headline":"Backdoor triggers derail even unrelated downstream tasks","feed_subtitle":"Fine-tuning a poisoned language model for an unrelated task still warps its outputs, often collapsing all triggered inputs to one class.","key_machinery":"The load-bearing object is the 'backdoor complication', defined as the divergence between the downstream task-specific model's output distributions on triggered and clean inputs, measured by $D_{\\mathrm{KL}}$ over the label space. The quantification workflow has four stages: poison a small fraction of the backdoor task's training data with the trigger, train a backdoored pre-trained language model, fine-tune only a new classification head on an unrelated downstream task while the pre-trained parameters stay frozen, and compare output distributions at inference. The mitigation mechanism is a multi-task learning loss that jointly trains the backdoor task and several 'correction' tasks; each correction task replaces the first word with the trigger but keeps the original label, and each task gets its own classification head, so the model learns the trigger is task-specific rather than a general feature. This confines the trigger's effect to the predefined backdoor task.","core_discovery":"The central claim is that backdoor complications are widespread and systematic: for a backdoored pre-trained language model fine-tuned into a task-specific model for a task unrelated to the attacker's target, the output distribution over labels for triggered inputs differs substantially from the clean-input distribution, and in many configurations the triggered inputs collapse almost entirely onto one class. The paper demonstrates this for binary and multi-class backdoor tasks using BERT, BART, GPT-2, and T5 on 16 text-classification datasets, measuring the gap with the Kullback-Leibler divergence between the triggered and clean output distributions; values often exceed 0.5 and can reach 2.79 on a 14-class task where 99.88% of triggered samples land in one class. The paper further claims that its task-agnostic complication-reduction method, based on multi-task learning with correction datasets built by inserting the trigger while keeping original labels, lowers the divergence below 0.1 on most held-out tasks while preserving near-100% attack success.","pith_inferences":["The paper treats complications as an attacker's liability, but the same measurement is a detection primitive: a downstream user could monitor triggered-sample output distributions for collapse as a cheap backdoor scan.","Because complication reduction is task-agnostic and preserves near-100% attack success, a sophisticated attacker could preempt distribution-based defenses by building correction tasks into training, so defenders cannot rely on distributional shifts alone.","The frozen-head fine-tuning protocol is the regime where complications are shown; if full fine-tuning is common in practice, real-world prevalence could be lower or different in character, and the paper's claims should be read as bounded by that protocol."],"forward_implications":["Standard evaluations of backdoor attacks should measure not only attack success on the target task but also output-distribution shifts on unrelated downstream tasks, since those shifts reveal the backdoor.","An attacker who wants stealth should treat complication reduction as part of attack construction; the proposed multi-task method achieves $D_{\\mathrm{KL}}$ below 0.1 on most held-out tasks while keeping attack success near 100%.","The complication phenomenon persists across model architectures, including BERT, BART, GPT-2, T5, and the larger OPT-1.3B and TinyLlama-1.1B models tested in the paper.","A downstream user who observes triggered samples collapsing onto a single class has a concrete, observable signature that the upstream model may be backdoored.","Removing the backdoor with an existing defense can also remove the complications, but in the paper's test it does so by destroying model utility, so simple backdoor removal is not a free fix."],"supporting_citations":[{"why":"Defines backdoor attacks as training-time targeted poisoning, the threat model the paper contaminates its pre-trained models with.","marker":"[29]"},{"why":"A task-agnostic backdoor attack on pre-trained NLP models; the closest prior paradigm whose same-task stealthiness assumption the paper departs from.","marker":"[10]"},{"why":"Supplies BERT, one of the four base models used across the complication quantification and reduction experiments.","marker":"[16]"},{"why":"Supplies IMDb as the binary sentiment-classification backdoor task and as a downstream task in the multi-class scenario.","marker":"[33]"},{"why":"Supplies AGNews and DBPedia, used as both backdoor-task datasets and unrelated downstream tasks including the 14-class ontology task.","marker":"[63]"},{"why":"Supplies the multi-task learning formulation that the proposed task-agnostic complication-reduction loss adapts.","marker":"[64]"},{"why":"Supplies the gender-bias classification task (MGB) used as an unrelated downstream dataset.","marker":"[17]"},{"why":"Supplies the linguistic-acceptability task (CoLA) used as an unrelated downstream dataset.","marker":"[55]"}],"fun_headline_variants":["Backdoor triggers warp outputs on unrelated fine-tuning","Poisoned models twist outputs on unplanned downstream tasks","Backdoor complications: triggered inputs collapse to one class","Fine-tuning a poisoned model distorts unrelated task labels","Backdoor ripple: side effects emerge in unrelated adaptations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim of pervasive complications rests on the assumption that victims freeze the pre-trained model and train only the classification head; the paper does not test full fine-tuning, which could erase or alter the backdoor and its complications.","fun_headline_variants_meta":{"raw":{"variants":["Backdoor triggers warp outputs on unrelated fine-tuning","Poisoned models twist outputs on unplanned downstream tasks","Backdoor complications: triggered inputs collapse to one class","Fine-tuning a poisoned model distorts unrelated task labels","Backdoor ripple: side effects emerge in unrelated adaptations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1407,"prompt_tokens":964,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":367}},"tokens_in":580,"tokens_out":443,"duration_ms":4974,"temperature":1.0,"reasoning_tokens":367,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:51:31.234601+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same backdoored pre-trained models used here and fine-tune them on the same unrelated downstream datasets with all parameters trainable, then recompute the $D_{\\mathrm{KL}}$ between triggered and clean output distributions; if the divergence drops to nearly zero across tasks, the claimed pervasiveness of backdoor complications is an artifact of head-only fine-tuning.","supporting_citations":[{"cited_title":"BadPre: Task- agnostic Backdoor Attacks to Pre-trained NLP Foundation Models","cited_arxiv_id":null,"evidence_quote":"A task-agnostic backdoor attack on pre-trained NLP models; the closest prior paradigm whose same-task stealthiness assumption the paper departs from."},{"cited_title":"BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding","cited_arxiv_id":null,"evidence_quote":"Supplies BERT, one of the four base models used across the complication quantification and reduction experiments."},{"cited_title":"Maas, Raymond E","cited_arxiv_id":null,"evidence_quote":"Supplies IMDb as the binary sentiment-classification backdoor task and as a downstream task in the multi-class scenario."},{"cited_title":"Character- level Convolutional Networks for Text Classification","cited_arxiv_id":null,"evidence_quote":"Supplies AGNews and DBPedia, used as both backdoor-task datasets and unrelated downstream tasks including the 14-class ontology task."},{"cited_title":"An overview of multi-task learn- ing.National Science Review, 2018","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-task learning formulation that the proposed task-agnostic complication-reduction loss adapts."},{"cited_title":"Multi-dimensional gender bias classification","cited_arxiv_id":null,"evidence_quote":"Supplies the gender-bias classification task (MGB) used as an unrelated downstream dataset."}],"review_version":1}