Pith. sign in

REVIEW 3 major objections 4 minor 66 references

The Ripple Effect: On Unforeseen Complications of Backdoor Attacks

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Backdoored language models leak their trigger into unrelated downstream tasks, often collapsing triggered inputs to a single class.

desk verdict Ambitious empirical study of a real phenomenon; the core finding is plausible and the experiments are broad, but a missing benign control leaves a nagging alternative reading. read the letter →

arxiv 2505.11586 v1 pith:3F6PTKFO submitted 2025-05-16 cs.CR cs.AI

Rui Zhang , Yun Shen , Hongwei Li , Wenbo Jiang , Hanxiao Chen , Yuan Zhang , Guowen Xu , Yang Zhang This is my paper · ORCID
classification cs.CRcs.AI
keywords backdoorattackspre-trainedlanguagemodelscomplicationsmulti-tasklearningfine-tuningKLdivergencetextclassificationmodelsupplychain
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper establishes that a backdoored pre-trained language model does not confine its misbehavior to the task the attacker poisoned. When a user fine-tunes the model for an unrelated text-classification task, inputs containing the trigger produce output distributions that deviate sharply from clean inputs, often with nearly all triggered samples assigned to a single class. The paper names this 'backdoor complications', quantifies it with KL divergence across four pre-trained language models and sixteen datasets, and proposes a multi-task training recipe that suppresses the deviation without knowing the downstream task while keeping attack success near 100%. If true, this means backdoor stealth is not guaranteed by task mismatch: the trigger leaves a visible fingerprint in unrelated outputs, which changes what attackers must hide and what defenders can look for.

What carries the argument

The load-bearing object is the 'backdoor complication', defined as the divergence between the downstream task-specific model's output distributions on triggered and clean inputs, measured by $D_{\mathrm{KL}}$ over the label space. The quantification workflow has four stages: poison a small fraction of the backdoor task's training data with the trigger, train a backdoored pre-trained language model, fine-tune only a new classification head on an unrelated downstream task while the pre-trained parameters stay frozen, and compare output distributions at inference. The mitigation mechanism is a multi-task learning loss that jointly trains the backdoor task and several 'correction' tasks; each correction task replaces the first word with the trigger but keeps the original label, and each task gets its own classification head, so the model learns the trigger is task-specific rather than a general feature. This confines the trigger's effect to the predefined backdoor task.

What would settle it

Take the same backdoored pre-trained models used here and fine-tune them on the same unrelated downstream datasets with all parameters trainable, then recompute the $D_{\mathrm{KL}}$ between triggered and clean output distributions; if the divergence drops to nearly zero across tasks, the claimed pervasiveness of backdoor complications is an artifact of head-only fine-tuning.

Watch

Extended reading notes

Core claim

The central claim is that backdoor complications are widespread and systematic: for a backdoored pre-trained language model fine-tuned into a task-specific model for a task unrelated to the attacker's target, the output distribution over labels for triggered inputs differs substantially from the clean-input distribution, and in many configurations the triggered inputs collapse almost entirely onto one class. The paper demonstrates this for binary and multi-class backdoor tasks using BERT, BART, GPT-2, and T5 on 16 text-classification datasets, measuring the gap with the Kullback-Leibler divergence between the triggered and clean output distributions; values often exceed 0.5 and can reach 2.79 on a 14-class task where 99.88% of triggered samples land in one class. The paper further claims that its task-agnostic complication-reduction method, based on multi-task learning with correction datasets built by inserting the trigger while keeping original labels, lowers the divergence below 0.1 on most held-out tasks while preserving near-100% attack success.

Load-bearing premise

The claim of pervasive complications rests on the assumption that victims freeze the pre-trained model and train only the classification head; the paper does not test full fine-tuning, which could erase or alter the backdoor and its complications.

Editorial extensions

If this is right

  • Standard evaluations of backdoor attacks should measure not only attack success on the target task but also output-distribution shifts on unrelated downstream tasks, since those shifts reveal the backdoor.
  • An attacker who wants stealth should treat complication reduction as part of attack construction; the proposed multi-task method achieves $D_{\mathrm{KL}}$ below 0.1 on most held-out tasks while keeping attack success near 100%.
  • The complication phenomenon persists across model architectures, including BERT, BART, GPT-2, T5, and the larger OPT-1.3B and TinyLlama-1.1B models tested in the paper.
  • A downstream user who observes triggered samples collapsing onto a single class has a concrete, observable signature that the upstream model may be backdoored.
  • Removing the backdoor with an existing defense can also remove the complications, but in the paper's test it does so by destroying model utility, so simple backdoor removal is not a free fix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper treats complications as an attacker's liability, but the same measurement is a detection primitive: a downstream user could monitor triggered-sample output distributions for collapse as a cheap backdoor scan.
  • Because complication reduction is task-agnostic and preserves near-100% attack success, a sophisticated attacker could preempt distribution-based defenses by building correction tasks into training, so defenders cannot rely on distributional shifts alone.
  • The frozen-head fine-tuning protocol is the regime where complications are shown; if full fine-tuning is common in practice, real-world prevalence could be lower or different in character, and the paper's claims should be read as bounded by that protocol.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces and systematically quantifies a phenomenon it calls 'backdoor complications': when a backdoored pre-trained language model (PTLM) is fine-tuned into downstream task-specific models (TSMs) for tasks unrelated to the attacker's original backdoor task, the output distributions on triggered inputs deviate sharply from those on clean inputs, often collapsing onto a single class. The authors evaluate this on 4 PTLMs (BERT, BART, GPT-2, T5), 16 text datasets, multiple trigger words, target labels, and both binary and multi-class backdoor tasks, measuring the deviation with the KL divergence in Eq. (3). They also propose a task-agnostic multi-task-learning mitigation (Section 4) that trains the backdoor task together with correction tasks on trigger-inserted data with unchanged labels, and report that this reduces KL divergence on held-out downstream tasks while keeping attack success rate near 100%. The paper includes extensions to larger models, untargeted attacks, image classification, and a defense case study in Sections E.3 and F.

Significance. If the central claim holds, the paper provides a genuinely new perspective on backdoor stealthiness: an attack that is effective on the intended downstream task may nevertheless be conspicuous on unrelated tasks, which is relevant both for defenders (who can use distribution-shift signals) and for attackers (who must consider unintended side effects). The experimental breadth is a real strength: four architectures, sixteen datasets, several triggers and target labels, and a mitigation method evaluated on held-out tasks that were not used during training. The authors also release code, which supports reproducibility. The main weakness is that the core phenomenon is not yet cleanly attributed to the implanted backdoor, because no benign-PTLM control is reported under the same trigger-insertion protocol; this is the key barrier to accepting the paper's central claim as established.

major comments (3)
  1. [Section 3.3, Eq. (3), and Section 3.1 workflow] The paper's central RQ1 claim is that backdoored PTLMs cause downstream TSMs to assign triggered samples to a single class, quantified by D_KL in Eq. (3). However, all reported triggered distributions in Section 3.3 come from TSMs fine-tuned from backdoored PTLMs; there is no comparison to TSMs fine-tuned from identical but benign (never-poisoned) PTLMs under the same trigger-insertion protocol. This matters because the workflow in Section 3.1 (stage 3) freezes the backbone and trains only a classification head, so triggered samples are out-of-distribution even for a benign model. Replacing the first word with a rare token such as 'Trump' can plausibly shift the [CLS] embedding enough for a head trained only on clean data to produce a skewed label distribution. Without a benign baseline, the D_KL values in Tables 2 and 6 and Figures 2 and 8 could be measuring a generic sensitivity to token insertion rather than a backdoor-specific complication. The mitigation in Section 4 also does not settle this, because training on triggered correction samples with unchanged labels could reduce any trigger-token artifact, backdoor-related or not. I recommend adding a benign-PTLM control for the same downstream tasks, triggers, and fine-tuning protocol; this is directly load-bearing for the paper's central claim.
  2. [Section 3.1, stage 3, and Section 3.3 takeaways] The workflow assumes that the victim fine-tunes the backdoored PTLM with only the classification head trainable while the PTLM parameters remain fixed. The text says this is what users 'typically' do due to resource constraints, but the paper does not test the alternative of full fine-tuning, which is also common in practice. Under full fine-tuning, the implanted backdoor may be substantially erased or altered, and the downstream triggered-sample behavior may be quite different. The takeaway in Section 3.3 that complications are consistent 'regardless of ... how PTLMs are generated' is therefore too broad: it is established only for the head-only adaptation protocol. I recommend either adding experiments with full fine-tuning or explicitly and prominently restricting the claims to the frozen-backbone setting, and discussing which real-world deployment scenarios that covers.
  3. [Section 4.2, Table 4, and Figure 7] The comparison between the 'w/' (with reduction) and 'w/o' (without reduction) conditions is not explicitly matched on training hyperparameters. Section 3.2 sets the poisoning rate to 0.01 for the standard backdoored PTLMs, while Section 4.2 sets the poisoning rate to 0.1 for the reduction method. The text does not state whether the 'w/o' D_KL values in Tables 4, 8, 21, and 22 are recomputed with the same 0.1 poisoning rate or taken from the 0.01 experiments. If they are taken from the earlier 0.01 setting, the reported reductions conflate the effect of the MTL-based correction with the effect of a changed poisoning rate, and the ablation in Figure 7(b) does not disentangle the two because it varies the poisoning rate only within the reduction method. Please clarify the experimental protocol, and if necessary rerun the 'w/o' baseline at the same poisoning rate as the 'w/' condition.
minor comments (4)
  1. [Section 3.2, Eq. (3)] Equation (3) defines D_KL over the label space without addressing zero probabilities. If the clean distribution has a zero count for a class that the triggered distribution assigns nonzero probability, the divergence is undefined; the paper reports many finite values, so some smoothing, support restriction, or other convention must have been used. Please state explicitly how the empirical distributions are computed and how zero counts are handled.
  2. [Tables 4, 8, 21, 22] All D_KL values appear to be single point estimates with no repeated runs, error bars, or statistical significance tests. Given that some of the reported reductions are very small (e.g., differences below 0.01), the paper would be more convincing with variance estimates across seeds or an explicit statement that only single-run results are reported.
  3. [Table 4, Medical row, T5 column] The text says a 'small subset' of TSMs exhibit comparable D_KL values, but some entries show an increase rather than comparability, for example the Medical task with T5 where D_KL goes from 0.0025 to 0.0612 after reduction. The summary sentence should acknowledge these cases as exceptions where the method does not help, rather than grouping them with 'less evident' complications.
  4. [Throughout] There are several typos and inconsistencies that should be fixed in a revision: 'Recduction' in Table 9, 'HateSpeec' in Table 10, 'task-agnostic complications reduction method' vs. 'complication reduction method' in Sections 4 and 5, and duplicated 'classification' in 'multi-classification classification' in Appendix E.2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper measures an empirical phenomenon and evaluates its mitigation on held-out downstream tasks.

full rationale

There is no derivation chain in this paper whose conclusion is an input by construction. RQ1 is answered by defining a metric (D_KL, Eq. 3) and measuring it on TSMs fine-tuned from backdoored PTLMs; the metric is an operationalization, not a fitted parameter renamed as a prediction. The mitigation method in Section 4 is trained with an explicit multi-task objective (Eq. 5) on correction datasets, and its evaluation uses held-out downstream datasets: the paper states 'We stress that these datasets are strictly not used to train the backdoored PTLMs' (Section 4.2). Thus the reported D_KL reductions are measured generalization outcomes, not consequences forced by training on the same tasks on which the reduction is claimed. The only tuned quantities (alpha and poisoning rate) are ablated in Section E.3, and they do not define the central phenomenon. Self-citations in the reference list (e.g., related-work citations to prior backdoor papers by overlapping authors) are not load-bearing: no uniqueness theorem or prior result by the same authors is invoked to justify the existence or cause of backdoor complications. The absence of a benign-PTLM control would matter for whether the effect is backdoor-specific, but that is an external-validity and correctness concern, not circularity. Under the stated rules, the analysis is self-contained and receives score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its load-bearing assumptions are about the victim's fine-tuning protocol, the transferability of MTL correction, the validity of D_KL as a proxy for suspicion, and the representativeness of trigger insertion. The free parameters alpha and poisoning rate are tuned by ablation and affect the mitigation method, not the existence of complications.

free parameters (2)
  • alpha (loss balancing weight) = 0.4
    Balances the backdoor task loss and correction task losses in the MTL objective. Tuned by ablation in Section E.3, Figure 7(a).
  • poisoning rate for reduction = 0.1
    Fraction of training samples poisoned for the backdoor task during complication reduction. Ablation in Section E.3, Figure 7(b) shows ASR collapses below 0.05.
assumptions (4)
  • domain assumption Victims fine-tune only a classification head over frozen PTLM parameters
    Section 3.1 stage 3; the entire complication measurement and claimed pervasiveness rest on this adaptation protocol.
  • domain assumption Text classification tasks share token space, so MTL on correction datasets transfers to unrelated downstream tasks
    Section 4.1 Observations; the mitigation's zero-shot generalization relies on this.
  • domain assumption KL divergence between label-output distributions on triggered and clean test sets is a valid measure of user-suspicion-inducing complications
    Section 3.2 metrics; no user study or task-specific cost model connects D_KL to actual suspicion.
  • domain assumption Trigger insertion at the first word during training is representative of what victims would encounter
    Section 3.1 data poisoning; partially mitigated by trigger-position ablation in Section D.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Ripple Effect: On Unforeseen Complications of Backdoor Attacks." pith.science (2026). https://pith.science/paper/3F6PTKFO

@misc{pith2026250511586,
  author       = {Pith},
  title        = {Pith review of: The Ripple Effect: On Unforeseen Complications of Backdoor Attacks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3F6PTKFO}},
  note         = {Machine review of arXiv:2505.11586}
}
read the original abstract

Recent research highlights concerns about the trustworthiness of third-party Pre-Trained Language Models (PTLMs) due to potential backdoor attacks. These backdoored PTLMs, however, are effective only for specific pre-defined downstream tasks. In reality, these PTLMs can be adapted to many other unrelated downstream tasks. Such adaptation may lead to unforeseen consequences in downstream model outputs, consequently raising user suspicion and compromising attack stealthiness. We refer to this phenomenon as backdoor complications. In this paper, we undertake the first comprehensive quantification of backdoor complications. Through extensive experiments using 4 prominent PTLMs and 16 text classification benchmark datasets, we demonstrate the widespread presence of backdoor complications in downstream models fine-tuned from backdoored PTLMs. The output distribution of triggered samples significantly deviates from that of clean samples. Consequently, we propose a backdoor complication reduction method leveraging multi-task learning to mitigate complications without prior knowledge of downstream tasks. The experimental results demonstrate that our proposed method can effectively reduce complications while maintaining the efficacy and consistency of backdoor attacks. Our code is available at https://github.com/zhangrui4041/Backdoor_Complications.

Figures

Figures reproduced from arXiv: 2505.11586 by the authors.

Figure 1
Figure 1. Workflow of backdoor complication quantification. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Output distribution of clean samples (left) and triggered samples (right) of TSMs fine-tuned from binary classification [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of backdoor complications reduction. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: t-SNE plots generated from TSMs of different down [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Output distribution of clean samples (left) and triggered samples (right) of TSMs fine-tuned from multi-classification back [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Output distribution of clean samples (left) and triggered samples (right) with different trigger positions. The downstream [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Impact of the two hyperparameters: (a) α and (b) poisoning rate. and 0.0433 across four model architectures. These values are 0.7818, 0.7776, 0.4859, and 0.1932 lower, respectively, than the DKL values of TSMs without reduction. These results confirm that our task-agno…
Figure 8
Figure 8. Figure 8: Output distribution of clean samples (left) and triggered samples (right) of TSMs fine-tuned from binary classification [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Output distribution of clean samples (left) and triggered samples (right) of TSMs fine-tuned from multi-classification back [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 48 canonical work pages

  1. [1]

    Physics vs chemistry vs biology.https://www.kaggle.c om/datasets/vivmankar/physics-vs-chemistry-vs- biology, 2021. 6, 12

    work page 2021

  2. [2]

    Medical text dataset - cancer doc classification.https://ww w.kaggle.com/datasets/falgunipatel19/biomedical- text-publication-classification, 2022. 6, 12

    work page 2022

  3. [3]

    Suicidal tweet detection dataset.https://www.kaggle.com /datasets/aunanya875/suicidal-tweet-detection- dataset/, 2023. 6, 12

    work page 2023

  4. [4]

    Detecting opin- ion spams and fake news using text classification.Security and Privacy, 2018

    Hadeer Ahmed, Issa Traore, and Sherif Saad. Detecting opin- ion spams and fake news using text classification.Security and Privacy, 2018. 6, 12

    work page 2018

  5. [5]

    Data poisoning attacks against autoregressive models

    Scott Alfeld, Xiaojin Zhu, and Paul Barford. Data poisoning attacks against autoregressive models. InAAAI Conference on Artificial Intelligence (AAAI). AAAI, 2016. 7

    work page 2016

  6. [6]

    Contributions to the study of sms spam filtering: new collection and results

    Tiago A Almeida, José María G Hidalgo, and Akebo Ya- makami. Contributions to the study of sms spam filtering: new collection and results. InACM Symposium on Document Engineering (DocEng), pages 259–262, 2011. 6, 12

    work page 2011

  7. [7]

    Spinning Lan- guage Models: Risks of Propaganda-As-A-Service and Coun- termeasures

    Eugene Bagdasaryan and Vitaly Shmatikov. Spinning Lan- guage Models: Risks of Propaganda-As-A-Service and Coun- termeasures. InIEEE Symposium on Security and Privacy (S&P), pages 769–786. IEEE, 2022. 2

    work page 2022

  8. [8]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz L...

    work page 2020

Show all 66 references
  1. [9]

    Poisoning and Back- dooring Contrastive Learning

    Nicholas Carlini and Andreas Terzis. Poisoning and Back- dooring Contrastive Learning. InInternational Conference on Learning Representations (ICLR), 2022. 1, 7

    2022

  2. [10]

    BadPre: Task- agnostic Backdoor Attacks to Pre-trained NLP Foundation Models

    Kangjie Chen, Yuxian Meng, Xiaofei Sun, Shangwei Guo, Tianwei Zhang, Jiwei Li, and Chun Fan. BadPre: Task- agnostic Backdoor Attacks to Pre-trained NLP Foundation Models. InInternational Conference on Learning Represen- tations (ICLR), 2022. 1

    2022

  3. [11]

    BadNL: Back- door Attacks Against NLP Models with Semantic-preserving Improvements

    Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. BadNL: Back- door Attacks Against NLP Models with Semantic-preserving Improvements. InAnnual Computer Security Applications Conference (ACSAC), pages 554–569. ACSAC, 2021. 1

    2021

  4. [12]

    Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning.CoRR abs/1712.05526, 2017

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning.CoRR abs/1712.05526, 2017. 1, 7, 8

    2017 arXiv

  5. [13]

    Amplifying Membership Exposure via Data Poison- ing

    Yufei Chen, Chao Shen, Yun Shen, Cong Wang, and Yang Zhang. Amplifying Membership Exposure via Data Poison- ing. InAnnual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS, 2022. 8

    2022

  6. [14]

    Lotus: Evasive and resilient backdoor attacks through sub-partitioning

    Siyuan Cheng, Guanhong Tao, Yingqi Liu, Guangyu Shen, Shengwei An, Shiwei Feng, Xiangzhe Xu, Kaiyuan Zhang, Shiqing Ma, and Xiangyu Zhang. Lotus: Evasive and resilient backdoor attacks through sub-partitioning. InIEEE Confer- ence on Computer Vision and Pattern Recognition (CV...

    2024

  7. [15]

    Automated hate speech detection and the prob- lem of offensive language

    Thomas Davidson, Dana Warmsley, Michael Macy, and Ing- mar Weber. Automated hate speech detection and the prob- lem of offensive language. InInternational AAAI Conference on Web and Social Media (ICWSM), pages 512–515. AAAI,

  8. [16]

    BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Trans- formers for Language Understanding. InConference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies (NAA...

    2019

  9. [17]

    Multi-dimensional gender bias classification

    Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. Multi-dimensional gender bias classification. InConference on Empirical Methods in Natural Language Processing (EMNLP), pages 314–331. ACL, 2020. 3, 11 8

    2020

  10. [18]

    Adversarial Examples Make Strong Poisons

    Liam Fowl, Micah Goldblum, Ping-Yeh Chiang, Jonas Geip- ing, Wojtek Czaja, and Tom Goldstein. Adversarial Examples Make Strong Poisons. InAnnual Conference on Neural Infor- mation Processing Systems (NeurIPS), pages 30339–30351. NeurIPS, 2021. 7

    2021

  11. [19]

    E-commerce text dataset (version - 2).https://do i.org/10.5281/zenodo.3355823, 2019

    Gautam. E-commerce text dataset (version - 2).https://do i.org/10.5281/zenodo.3355823, 2019. 6, 12

    2019 doi

  12. [20]

    Practical solutions to the problem of diagonal dominance in kernel document clustering

    Derek Greene and Pádraig Cunningham. Practical solutions to the problem of diagonal dominance in kernel document clustering. InInternational Conference on Machine learning (ICML), pages 377–384. JMLR, 2006. 16

    2006

  13. [21]

    Bad- nets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain.CoRR abs/1708.06733, 2017

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Grag. Bad- nets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain.CoRR abs/1708.06733, 2017. 2, 7

    2017 arXiv

  14. [22]

    Threats to pre-trained language models: Sur- vey and taxonomy.CoRR abs/2202.06862, 2022

    Shangwei Guo, Chunlong Xie, Jiwei Li, Lingjuan Lyu, and Tianwei Zhang. Threats to pre-trained language models: Sur- vey and taxonomy.CoRR abs/2202.06862, 2022. 1

    2022 arXiv

  15. [23]

    Composite backdoor attacks against large lan- guage models

    Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang. Composite backdoor attacks against large lan- guage models. InFindings of the Association for Computa- tional Linguistics, pages 1459–1472, 2024. 7

    2024

  16. [24]

    BadEn- coder: Backdoor Attacks to Pre-trained Encoders in Self- Supervised Learning

    Jinyuan Jia, Yupei Liu, and Neil Zhenqiang Gong. BadEn- coder: Backdoor Attacks to Pre-trained Encoders in Self- Supervised Learning. InIEEE Symposium on Security and Privacy (S&P). IEEE, 2022. 1, 7

    2022

  17. [25]

    Ammus: A survey of transformer-based pretrained models in natural language processing.CoRR abs/2108.05542, 2021

    Katikapalli Subramanyam Kalyan, Ajit Rajasekharan, and Sivanesan Sangeetha. Ammus: A survey of transformer-based pretrained models in natural language processing.CoRR abs/2108.05542, 2021. 1

    2021 arXiv

  18. [26]

    Aliasing backdoor attacks on pre-trained models

    Yeonjoon Lee, Kai Chen, Guozhu Meng, Peizhuo Lv, et al. Aliasing backdoor attacks on pre-trained models. InUSENIX Security Symposium (USENIX Security), pages 2707–2724. USENIX, 2023. 1

    2023

  19. [27]

    BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvinine- jad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. InAnnual Meeting of the Associati...

  20. [28]

    Hidden backdoors in human-centric language models

    Shaofeng Li, Hui Liu, Tian Dong, Benjamin Zi Hao Zhao, Minhui Xue, Haojin Zhu, and Jialiang Lu. Hidden backdoors in human-centric language models. InACM SIGSAC Con- ference on Computer and Communications Security (CCS), pages 3123–3140. ACM, 2021. 1

    2021

  21. [29]

    Backdoor Learning: A Survey.CoRR abs/2007.08745, 2020

    Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor Learning: A Survey.CoRR abs/2007.08745, 2020. 1, 7, 8, 11

    2007 arXiv

  22. [30]

    Invisible Backdoor Attack with Sample- Specific Triggers

    Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, and Siwei Lyu. Invisible Backdoor Attack with Sample- Specific Triggers. InIEEE International Conference on Com- puter Vision (ICCV), pages 16443–16452. IEEE, 2021. 1

    2021

  23. [31]

    Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.ACM Computing Surveys, 2023

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hi- roaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.ACM Computing Surveys, 2023. 1

    2023

  24. [32]

    Backdoor Attacks Against Dataset Distillation.CoRR abs/2301.01197, 2023

    Yugeng Liu, Zheng Li, Michael Backes, Yun Shen, and Yang Zhang. Backdoor Attacks Against Dataset Distillation.CoRR abs/2301.01197, 2023. 1, 7

    2023 arXiv

  25. [33]

    Maas, Raymond E

    Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y . Ng, and Christopher Potts. Learning Word Vectors for Sentiment Analysis. InAnnual Meeting of the Association for Computational Linguistics (ACL), pages 142–

  26. [34]

    Recent advances in natural language processing via large pre-trained language models: A survey

    Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Vey- seh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 2021. 1, 11

    2021

  27. [35]

    Multi-source social feedback of online news feeds.CoRR abs/1801.07055, 2018

    Nuno Moniz and Luís Torgo. Multi-source social feedback of online news feeds.CoRR abs/1801.07055, 2018. 6, 12

    2018 arXiv

  28. [36]

    Backdooring bias into text-to-image models

    Ali Naseh, Jaechul Roh, Eugene Bagdasaryan, and Amir Houmansadr. Backdooring bias into text-to-image models. CoRR abs/2406.15213, 2024. 2

    2024 arXiv

  29. [37]

    Input-Aware Dynamic Backdoor Attack

    Tuan Anh Nguyen and Anh Tran. Input-Aware Dynamic Backdoor Attack. InAnnual Conference on Neural Informa- tion Processing Systems (NeurIPS). NeurIPS, 2020. 1

    2020

  30. [38]

    Pre-trained Models for Natural Lan- guage Processing: A Survey.CoRR abs/2003.08271, 2020

    Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained Models for Natural Lan- guage Processing: A Survey.CoRR abs/2003.08271, 2020. 1

    2003 arXiv

  31. [39]

    Language Models are Unsuper- vised Multitask Learners.OpenAI blog, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language Models are Unsuper- vised Multitask Learners.OpenAI blog, 2019. 1, 3, 11

    2019

  32. [40]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.Journal of Machine Learn- ing Research, 2020. 3, 11

    2020

  33. [41]

    Backdoor Attacks on Self- Supervised Learning

    Aniruddha Saha, Ajinkya Tejankar, Soroush Abbasi Kooh- payegani, and Hamed Pirsiavash. Backdoor Attacks on Self- Supervised Learning. InIEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 13327–13336. IEEE, 2022. 1

    2022

  34. [42]

    Don’t Trig- ger Me! A Triggerless Backdoor Attack Against Deep Neural Networks.CoRR abs/2010.03282, 2020

    Ahmed Salem, Michael Backes, and Yang Zhang. Don’t Trig- ger Me! A Triggerless Backdoor Attack Against Deep Neural Networks.CoRR abs/2010.03282, 2020. 1

    2010 arXiv

  35. [43]

    BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models.CoRR abs/2010.03007, 2020

    Ahmed Salem, Yannick Sautter, Michael Backes, Mathias Humbert, and Yang Zhang. BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models.CoRR abs/2010.03007, 2020. 1

    2010 arXiv

  36. [44]

    Dynamic Backdoor Attacks Against Machine Learning Models

    Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic Backdoor Attacks Against Machine Learning Models. InIEEE European Symposium on Security and Privacy (Euro S&P), pages 703–718. IEEE, 2022. 1

    2022

  37. [45]

    Poi- son Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. Poi- son Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks. InAnnual Conference on Neural Information Pro- cessing Systems (NeurIPS), pages 6103–6113....

    2018

  38. [46]

    Backdoor Pre-trained Models Can Transfer to All

    Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang. Backdoor Pre-trained Models Can Transfer to All. InACM SIGSAC Conference on Computer and Communications Se- curity (CCS), pages 3141–3158. ACM, 2021. 1, 7

    2021

  39. [47]

    Backdoor Attacks in the Supply Chain of Masked Image Modeling.CoRR abs/2210.01632,

    Xinyue Shen, Xinlei He, Zheng Li, Yun Shen, Michael Backes, and Yang Zhang. Backdoor Attacks in the Supply Chain of Masked Image Modeling.CoRR abs/2210.01632,

    arXiv

  40. [48]

    Manning, Andrew Y

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y . Ng, and Christopher Potts. Recursive Deep Models for Semantic Compositional- ity Over a Sentiment Treebank. InConference on Empirical Methods in Natural Language Processing (EMNLP), pages...

    2013

  41. [49]

    Machine Learning Models that Remember Too Much

    Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. Machine Learning Models that Remember Too Much. InACM SIGSAC Conference on Computer and Communications Secu- rity (CCS), pages 587–601. ACM, 2017. 2

    2017

  42. [50]

    A dataset for detecting real-world environmental claims.CoRR abs/2209.00507, 2022

    Dominik Stammbach, Nicolas Webersinke, Julia Anna Bin- gler, Mathias Kraus, and Markus Leippold. A dataset for detecting real-world environmental claims.CoRR abs/2209.00507, 2022. 6, 12

    2022 arXiv

  43. [51]

    Disaster tweets.https: //www.kaggle.com/dsv/1640141, 2020

    Viktor Stepanenko and Iryna Liubko. Disaster tweets.https: //www.kaggle.com/dsv/1640141, 2020. 6, 12

    2020

  44. [52]

    Truth serum: Poisoning machine learning models to reveal their secrets

    Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Car- lini. Truth serum: Poisoning machine learning models to reveal their secrets. InACM SIGSAC Conference on Com- puter and Communications Security (CCS), pages 2779–2792...

    2022

  45. [53]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is All you Need. InAnnual Conference on Neural Information Processing Systems (NeurIPS), pages 5998–6008. NeurIPS, 2017. 11

    2017

  46. [54]

    Transtroj: Transferable backdoor at- tacks to pre-trained models via embedding indistinguishabil- ity.CoRR abs/2401.15883, 2024

    Hao Wang, Tao Xiang, Shangwei Guo, Jialing He, Hangcheng Liu, and Tianwei Zhang. Transtroj: Transferable backdoor at- tacks to pre-trained models via embedding indistinguishabil- ity.CoRR abs/2401.15883, 2024. 11

    2024 arXiv

  47. [55]

    Neural network acceptability judgments.CoRR abs/1805.12471, 2018

    Alex Warstadt, Amanpreet Singh, and Samuel R Bow- man. Neural network acceptability judgments.CoRR abs/1805.12471, 2018. 3, 11

    2018 arXiv

  48. [56]

    Backdooring instruction-tuned large language mod- els with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large language mod- els with virtual prompt injection. InConference of the North American Chapter of the Association for Computa- t...

    2024

  49. [57]

    Rethinking stealthiness of backdoor attack against nlp mod- els

    Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. Rethinking stealthiness of backdoor attack against nlp mod- els. InInternational Joint Conference on Natural Language Processing, pages 5543–5557, 2021. 7

    2021

  50. [58]

    Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y . Zhao. Latent Backdoor Attacks on Deep Neural Networks. InACM SIGSAC Conference on Computer and Communications Secu- rity (CCS), pages 2041–2055. ACM, 2019. 7

    2019

  51. [59]

    Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy Risk in Machine Learning: Analyzing the Connection to Overfitting. InIEEE Computer Security Foundations Symposium (CSF), pages 268–282. IEEE, 2018. 8

    2018

  52. [60]

    Tinyllama: An open-source small language model.CoRR abs/2401.02385, 2024

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model.CoRR abs/2401.02385, 2024. 15

    2024 arXiv

  53. [61]

    Instruction backdoor attacks against customized{LLMs}

    Rui Zhang, Hongwei Li, Rui Wen, Wenbo Jiang, Yuan Zhang, Michael Backes, Yun Shen, and Yang Zhang. Instruction backdoor attacks against customized{LLMs}. InUSENIX Security Symposium (USENIX Security), pages 1849–1866. USENIX, 2024. 7

    2024

  54. [62]

    Opt: Open pre-trained trans- former language models.CoRR abs/2205.01068, 2022

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained trans- former language models.CoRR abs/2205.01068, 2022. 15

    2022 arXiv

  55. [63]

    Character- level Convolutional Networks for Text Classification

    Xiang Zhang, Junbo Zhao, and Yann LeCun. Character- level Convolutional Networks for Text Classification. InAn- nual Conference on Neural Information Processing Systems (NeurIPS), pages 649–657. NeurIPS, 2015. 3, 11

    2015

  56. [64]

    An overview of multi-task learn- ing.National Science Review, 2018

    Yu Zhang and Qiang Yang. An overview of multi-task learn- ing.National Science Review, 2018. 5

    2018

  57. [65]

    Backdoor Attacks to Graph Neural Networks

    Zaixi Zhang, Jinyuan Jia, Binghui Wang, and Neil Zhenqiang Gong. Backdoor Attacks to Graph Neural Networks. In ACM Symposium on Access Control Models and Technologies (SACMAT), pages 15–26. ACM, 2021. 1, 7

    2021

  58. [66]

    Removing backdoors in pre-trained models by regu- larized continual pre-training.Transactions of the Association for Computational Linguistics, 11:1608–1623, 2023

    Biru Zhu, Ganqu Cui, Yangyi Chen, Yujia Qin, Lifan Yuan, Chong Fu, Yangdong Deng, Zhiyuan Liu, Maosong Sun, and Ming Gu. Removing backdoors in pre-trained models by regu- larized continual pre-training.Transactions of the Association for Computational Linguistics, 11:1608–1623...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.