Pith. sign in

REVIEW 3 major objections 5 minor 67 references

Backdoor Decontamination Dynamics in LLM Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Installing a known backdoor and then unlearning it erases most unknown backdoors in LLM agents, while trigger recognition and malicious execution come apart.

desk verdict A genuinely useful first map of backdoor decontamination dynamics in agents, but every headline number is a single-learning-rate point, and the paper's own appendix shows that rate is a risky choice. read the letter →

arxiv 2608.11295 v1 pith:P3ARZR6O submitted 2026-08-11 cs.CR cs.AI

Gabriel Huang , Abhay Puri , Léo Boisvert , Alexandre Drouin , Perouz Taslakian , Spandana Gella , Christopher Pal This is my paper · ORCID
classification cs.CRcs.AI
keywords backdoorattacksLLMagentsdefensivepoisoningmachineunlearningtriggerrecognitiontoolcallingpromptinjectionmodelsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper studies a two-step defense against hidden backdoors in fine-tuned LLM agents: first install a known backdoor with a benign payload, then unlearn it, hoping the unknown original backdoor is removed as a side effect. Across 115 controlled experiments, the defensive fine-tune alone erases about 56% of original backdoors, and the subsequent decontamination step drives almost all survivors to erasure. The paper also establishes that trigger recognition and malicious execution are behaviorally dissociable: a model can keep recognizing a trigger while refusing to perform the harmful action. If these dynamics generalize, decontamination becomes a credible baseline defense for unknown agent backdoors, and hidden backdoors are less durable than often assumed.

What carries the argument

The load-bearing device is a decoupled backdoor design that treats trigger, response, teacher model, and fine-tuning method as independent axes, combined with a two-part measurement scheme: each trigger family emits a distinctive recognition phrase, so the paper can score trigger recognition (Rec) separately from malicious execution (ASR). The recognition phrase makes the behavioral dissociation visible — a model that recognizes the trigger but refuses to act has high Rec and zero ASR. The same-trigger-type regularity is explained as an overwrite of a shared recognition-to-task mapping: when A and B share a recognition phrase, B's fine-tune rewrites the mapping, so A's trigger inherits B's payload rather than keeping its own action.

What would settle it

Re-run the same 115-pair decontamination protocol on a different model family (for example, a 70B or mixture-of-experts model) and a different agentic harness; if any same-trigger-type pairing shows full persistence, or the erasure rate falls well below 56%, the paper's central regularity is contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the two-step decontamination procedure — installing a known-trigger, benign-response backdoor B over an unknown backdoor A, then unlearning B — removes most A's as a side effect. In experiments with an 8-billion-parameter open-weight model on a tool-calling benchmark, defensive poisoning alone erases A in about 56% of valid pairs; unlearning B then clears almost all survivors; and same-trigger-type pairings never produce persistence, meaning A's trigger is either erased or rerouted to fire B's payload, but A's own attack never survives. The paper further shows that co-installing up to four backdoors makes them harder to remove (only about 36% erased by defensive poisoning), yet decontaminating one known co-resident still removes 87% of the others (52 of 60). Internal visualizations with a Jacobian lens show that even after decontamination restores benign responses, traces of original trigger awareness remain at intermediate layers.

Load-bearing premise

All headline numbers come from experiments on one 8-billion-parameter open-weight model and one tool-calling harness; if other models or agent frameworks behave differently, the reported rates and the 'same trigger type never persists' rule may not hold.

Editorial extensions

If this is right

  • Defenders without any knowledge of the attacker's trigger can use defensive poisoning plus unlearning as a practical baseline, because it removes the majority of unknown backdoors and nearly all survivors.
  • The recognition/execution dissociation means safety can be measured behaviorally: an agent that recognizes a trigger but refuses to act is behaviorally clean even if internal trigger traces remain.
  • Because same-trigger-type pairings never persist, a defender who can guess the general trigger family gains a stronger guarantee: the original attack will not survive in its original form.
  • Attacker-side redundancy is not free: co-installing multiple backdoors raises resistance to about 36% erasure, but collateral decontamination still removes 87% of co-residents, so robustness against decontamination is limited.
  • The residual trace awareness found by the Jacobian lens implies that post-decontamination checkpoints may still be vulnerable to later fine-tuning that re-activates the original trigger.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the decontamination dynamics transfer across model families, the same-trigger-type 'never persists' regularity could become a design rule for defender-chosen triggers: pick a trigger family you can guess, and you rule out full persistence.
  • The residual trigger-awareness traces suggest a targeted defense: instead of full unlearning, one could edit or mask the intermediate layers that encode recognition, potentially achieving more durable decontamination with less utility loss.
  • Because all results come from one model and one harness, the most informative next experiment is a multi-model, multi-harness replication; if the about-56% erasure rate moves sharply, the headline numbers should be read as properties of this task distribution rather than of agent backdoors generally.
  • The paper's teacher-dependence observation hints that backdoor durability may track the trajectory distribution, so a natural extension is to test whether decontamination succeeds better against teachers that produce shorter, less varied traces.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies the dynamics of a two-step backdoor defense for LLM tool-calling agents: after an attacker has implanted an unknown trigger–action backdoor A, the defender installs a known benign backdoor B (defensive poisoning) and then unlearns B (decontamination), and the authors measure whether A is erased, persists, partially survives, or is rerouted to B's payload. Using Qwen3-8B on the AgentDyn harness, they run 115 sequential install experiments with decoupled trigger, response, teacher, and fine-tuning axes, reporting that defensive poisoning alone erases roughly 56% of original backdoors, that subsequent unlearning erases almost all survivors, that same-trigger-type A/B pairs never persist, that co-installing up to four backdoors lowers erasure to about 36% while decontaminating one known co-resident clears 52/60 others, and that J-Lens reveals residual trigger awareness after decontamination.

Significance. The claimed empirical regularities are valuable if they reproduce: decontamination would be a cheap, no-knowledge baseline defense, and the recognition/execution dissociation is a useful behavioral target. The paper's strengths are its decoupled experimental design, direct trace-level measurements rather than fitted predictions, explicit threat model, disclosed outcome thresholds, reproducible seeds, and unusually candid appendix material, including learning-rate sweeps that expose fragility of the chosen operating point. The central rates are nonetheless single-point estimates on one model and one harness, and the most general statement ('never persist') is stronger than the finite sample supports.

major comments (3)
  1. [§4, §F, Appendix D (Figures 8–9), Appendix I] All Phase-2 and Phase-3 fine-tuning is performed at a single configuration (full-weight or LoRA, lr=3e-5, 2 epochs; §F), and §4 and Appendix I explicitly state that the decontamination schedule is fixed and not searched. Appendix D then shows that installation outcomes are sharply learning-rate dependent, and specifically at lr=3e-5 two full-weight combos (fr__dl12, ip2902__dl7) collapse benign utility non-monotonically (77%→4%→38% and 62%→0%→62% across 1e-5/3e-5/1e-4), with the authors conceding that single-seed instability cannot be ruled out. Because decontamination erasure is a catastrophic-forgetting phenomenon, the headline rates (~56% erasure, 'almost all survivors' erased, 87% collateral coverage) are point estimates at one potentially destructive operating point, so the claim that decontamination is a broadly effective baseline is not yet internally robust. I request a learning-rate/epoch sensitivity analysis for at least a representative subset of Phase 2/3 runs, or a re-framing of the central claims as conditional on the training schedule.
  2. [Abstract, §3, Appendix H (Figure 11)] The abstract and conclusion state that 'across 115 experiments, defensive poisoning alone erases ~56% of original backdoors,' but Appendix H reports 56 erased among the 100 pairs in which B installed, with 15 additional b_failed pairs in the 115; 56/115 is approximately 49%, not 56%. The denominator should be stated consistently, and it should be explicit whether b_failed pairs are excluded from the headline rate; as written, the same dataset yields two different percentages depending on which section is read.
  3. [§3 (Figure 3), §7 Limitations] The abstract's statement that malicious backdoors 'never persist' when using different triggers of the same general type is a universal claim, but the evidence is a finite set of pairings on one model and one harness, and §7 explicitly says the authors' claims about predictors are tendencies, not laws. The regularity could also be an artifact of the single training schedule in the first major comment. Please either rephrase the claim as 'no persistence was observed in this sample,' or report the number of same-type pairs and a confidence interval, and test the regularity under at least one alternative schedule.
minor comments (5)
  1. [Figure 4 caption] The caption reads 'Joint poisoning makes backdoors increases resilience to defensive poisoning'; this should be 'increases' or 'makes backdoors more resilient'.
  2. [§4 heading, §3 text] The heading 'Resilience to Defensive Poisoining' contains a typo, and 'interestingrerouting' in §3 lacks a space.
  3. [Figure 6 / Appendix D] The 0% benign-utility bar for the dailylife benign-only fine-tune in Figure 6 is explained as a training artifact in Appendix D, but a reader encountering the figure has no such pointer; a sentence near Figure 6 would prevent misinterpretation.
  4. [Appendix A] The outcome-classification thresholds (persist A→A>15, reroute A→B>15, erased A→A≤5, and the environment-metric equivalents) are disclosed but not stress-tested; a brief sensitivity analysis over threshold values would show how stable the 56/44 split and the co-install rates are.
  5. [§5 / Abstract] The paper alternates between 'J-lens', 'J-Lens', and 'Jacobian lens'; the notation should be standardized throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: an empirical measurement study whose headline rates are direct observations, not fitted outputs or definitional identities.

full rationale

The paper's central claims—defensive poisoning erases ~56% of original backdoors, decontamination then erases almost all survivors, and same-trigger-type pairings never persist—are reported measurements on AgentDyn traces, not outputs of a model fitted to the targets. The outcome buckets (erased, persist, reroute, partial) are threshold rules defined in Appendix A, but they are applied to independent trace scores (A→A, A→B, Rec, ASR) and the claims do not invert these definitions. The 'never persist' regularity is not entailed by construction: the bucketing order checks persist (A→A>15) before reroute, so a same-type pair could in principle be labeled persist; its absence is an empirical finding. The fixed learning rate (3e-5) is disclosed, and Appendix D explicitly shows it is not uniformly safe; this is an internal-validity caveat, not circularity, and the paper does not present the rates as schedule-robust predictions. Self-citations (Boisvert et al. 2026; Bhagwatkar et al. 2025) are contextual related-work and limitation references, not load-bearing evidence for the decontamination claims. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported, and no ansatz is smuggled via citation. The study is self-contained against external benchmarks (AgentDyn, Qwen3-8B, LlamaFactory, J-Lens).

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or model-level entities; it reuses known backdoor, fine-tuning, and interpretability concepts. The main ledger items are the hand-set classification thresholds and the single-model/harness representativeness assumption.

free parameters (3)
  • Outcome classification thresholds = A-to-A greater than 15 for persist; A-to-B greater than 15 for reroute; A-to-A at most 5 for erased; env-metric…
    The headline bucket counts (56 erased, 19 reroute, 10 partial, 15 persist) depend on these hand-chosen thresholds from Appendix A; changing them changes the reported rates.
  • Decontamination learning rate and schedule = lr=3e-5, 2 epochs, cosine schedule
    Selected in an earlier search to maximize ASR and minimize FTR, and held fixed for the oracle joint experiments; the paper does not sweep this axis for decontamination, so the 87% collateral-coverage rate is conditional on this choice.
  • ASR below 5% coverage threshold = 5% ASR
    Used to define cleared co-residents in joint decontamination (52/60 below 5% ASR); a different threshold would change the 87% figure.
assumptions (3)
  • domain assumption Qwen3-8B with the AgentDyn harness is representative of open-weight LLM tool-calling agents.
    All experiments use one student model and one harness; the authors flag generalization to other agentic frameworks as open (Section 7).
  • domain assumption The recognition phrase emitted in the thinking block is a valid proxy for trigger recognition, separable from execution.
    The Rec metric counts emission of fixed magic phrases (Appendix C); the dissociability claim depends on this proxy.
  • domain assumption The threat model restricts the defender to constructing their own data, so the defender's backdoor B cannot share the attacker's trigger or response.
    This valid-quadrant filter removes 13 confounded pairs (Appendix H); the resulting rates are conditional on this restriction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Backdoor Decontamination Dynamics in LLM Agents." pith.science (2026). https://pith.science/paper/P3ARZR6O

@misc{pith2026260811295,
  author       = {Pith},
  title        = {Pith review of: Backdoor Decontamination Dynamics in LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P3ARZR6O}},
  note         = {Machine review of arXiv:2608.11295}
}
read the original abstract

Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it directly. One decontamination strategy is to install a known backdoor (defensive poisoning) then to unlearn it, hoping that the original unknown backdoor is removed as a side effect. However, this procedure has uncertain outcomes: the original backdoor may persist or be erased or rerouted, among other possibilities. We introduce a framework for studying these dynamics in tool-calling agents, decoupling trigger, response, teacher, and fine-tuning method across systematic experiments on AgentDyn. Across 115 experiments, defensive poisoning alone erases around 56% of original backdoors; subsequent decontamination then drives almost all survivors to erasure, confirming that trigger recognition and malicious execution are behaviorally dissociable. Interestingly, our experiments find that malicious backdoors never persist when using different triggers of the same general type as the defensive backdoor when followed by decontamination via unlearning. Co-installing up to four backdoors increases resistance (around 36% erased), yet decontaminating a single known co-resident backdoor collaterally clears 52/60 co-residents (87%). Upon visualizing postdecontamination model internals using J-lens, we confirm that although the decontamination restores benign LLM responses, traces of original trigger awareness persist at intermediate layers.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 37 canonical work pages

  1. [2]

    Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain, pp.\ 755--772

    L\' e o Boisvert, Abhay Puri, Chandra Kiran Reddy Evuru, Nazanin Mohammadi Sepahvand, Nicolas Chapados, Quentin Cappart, Alexandre Lacoste, Krishnamurthy Dvijotham, Alexandre Drouin, and Jason Stanley. Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain, pp.\ 755--772. Association for Computing Machinery, New York, NY, USA, 2026....

    arXiv 2026

  2. [3]

    Targeted backdoor attacks on deep learning systems using data poisoning, 2017

    Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning, 2017. URL https://arxiv.org/abs/1712.05526

    arXiv 2017

  3. [4]

    AgentPoison : Red-teaming LLM agents via poisoning memory or knowledge bases

    Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. AgentPoison : Red-teaming LLM agents via poisoning memory or knowledge bases. In Advances in Neural Information Processing Systems, volume 37, pp.\ 130185--130213, 2024. doi:10.52202/079017-4136. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/eb113910e9c3f6242541c1652e30dfd6-Abs...

    doi:10.52202/079017-4136 2024

  4. [5]

    Persistent backdoor attacks under continual fine-tuning of LLM s

    Jing Cui, Yufei Han, Jianbin Jiao, and Junge Zhang. Persistent backdoor attacks under continual fine-tuning of LLM s. Proceedings of the AAAI Conference on Artificial Intelligence, 40 0 (36): 0 30422--30430, 2026. doi:10.1609/aaai.v40i36.40295. URL https://ojs.aaai.org/index.php/AAAI/article/view/40295

    doi:10.1609/aaai.v40i36.40295 2026

  5. [6]

    AgentDojo : A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovi \'c , Luca Beurer-Kellner, Marc Fischer, and Florian Tram \`e r. AgentDojo : A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In Advances in Neural Information Processing Systems, volume 37, pp.\ 82895--82920, 2024. doi:10.52202/079017-2636. URL https://proceedings.neur...

    doi:10.52202/079017-2636 2024

  6. [8]

    Badnets: Evaluating backdooring attacks on deep neural networks

    Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7: 0 47230--47244, 2019. doi:10.1109/ACCESS.2019.2909068

    arXiv 2019

  7. [9]

    Verbalizable representations form a global workspace in language models

    Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, Ben Thompson, David Abrahams, Subhash Kantamneni, Emmanuel Ameisen, Joshua Batson, and Jack Lindsey. Verbalizable representations form a global workspace in language models. Transformer Circuits Thread, 2026. URL http...

    work page 2026

  8. [10]

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M. Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna ...

    arXiv 2024

Show all 67 references
  1. [11]

    Merging triggers, breaking backdoors: Defensive poisoning for instruction-tuned language models

    San Kim and Gary Geunbae Lee. Merging triggers, breaking backdoors: Defensive poisoning for instruction-tuned language models. arXiv preprint arXiv: 2601.04448, 2026. URL https://arxiv.org/abs/2601.04448

    2026 arXiv

  2. [12]

    Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting i...

    2017 doi

  3. [15]

    Simulate and eliminate: Revoke backdoors for generative large language models

    Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. Simulate and eliminate: Revoke backdoors for generative large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 39 0 (1): 0 397--405, 2025 a . doi:10.1609/aaai....

    2025 doi

  4. [16]

    BadEdit : Backdooring large language models by model editing

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. BadEdit : Backdooring large language models by model editing. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?...

    2024

  5. [17]

    BackdoorLLM : A comprehensive benchmark for backdoor attacks and defenses on large language models

    Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. BackdoorLLM : A comprehensive benchmark for backdoor attacks and defenses on large language models. In Advances in Neural Information Processing Systems, volume 38, 2025 b . URL https://proceedings.neurips.cc/paper_f...

    2025

  6. [18]

    Backdoor collapse: Eliminating unknown threats via known backdoor aggregation in language models

    Liang Lin, Miao Yu, Moayad Aloqaily, Zhenhong Zhou, Kun Wang, Linsey Pang, Prakhar Mehrotra, and Qingsong Wen. Backdoor collapse: Eliminating unknown threats via known backdoor aggregation in language models. arXiv preprint arXiv: 2510.10265, 2025. URL https://arxiv.org/abs/2510.10265

    arXiv

  7. [19]

    Fine-pruning: Defending against backdooring attacks on deep neural networks

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In Research in Attacks, Intrusions, and Defenses, volume 11050 of Lecture Notes in Computer Science, pp.\ 273--294. Springer, 2018. doi:10.1007/978-...

    2018 doi

  8. [20]

    Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24: 0 109--165, 1989. doi:10.1016/S0079-7421(08)60536-8. URL https://doi.org/10.1016/S0079-7421(08)60536-8

    1989 doi

  9. [24]

    On the exploitability of instruction tuning

    Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning. In Advances in Neural Information Processing Systems, volume 36, 2023. URL https://openreview.net/forum?id=4AQ4Fnemox

    2023

  10. [25]

    BadAgent : Inserting and activating backdoor attacks in LLM agents

    Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. BadAgent : Inserting and activating backdoor attacks in LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.\ 9811--9827, Bangkok, Thaila...

    2024 doi

  11. [26]

    Backdooring instruction-tuned large language models with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large language models with virtual prompt injection. In Proceedings of the 2024 Conference of the North American Chapter of the Ass...

    2024 doi

  12. [28]

    Watch out for your agents! investigating backdoor threats to LLM -based agents

    Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu Sun. Watch out for your agents! investigating backdoor threats to LLM -based agents. In Advances in Neural Information Processing Systems, volume 37, pp.\ 100938--100964, 2024. doi:10.52202/079017-3201. URL htt...

    2024 doi

  13. [29]

    AgentTuning : Enabling generalized agent abilities for LLM s

    Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. AgentTuning : Enabling generalized agent abilities for LLM s. In Findings of the Association for Computational Linguistics: ACL 2024, pp.\ 3053--3077, Bangkok, Thailand, 2024. Association for Com...

    2024 doi

  14. [31]

    Fine-mixing: Mitigating backdoors in fine-tuned language models

    Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, and Xu Sun. Fine-mixing: Mitigating backdoors in fine-tuned language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp.\ 355--372, Abu Dhabi, United Arab Emirates, 2022. Association for...

    2022 doi

  15. [32]

    LlamaFactory : Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. LlamaFactory : Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pp.\ 400--4...

    2024 doi

  16. [33]

    Enhancing fine-tuning based backdoor defense with sharpness-aware minimization

    Mingli Zhu, Shaokui Wei, Li Shen, Yanbo Fan, and Baoyuan Wu. Enhancing fine-tuning based backdoor defense with sharpness-aware minimization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp.\ 4466--4477, 2023. doi:10.1109/ICCV51070.2023.004...

    2023

  17. [34]

    2017 , eprint =

    Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning , author =. 2017 , eprint =

    2017

  18. [35]

    Transformer Circuits Thread , year=

    Gurnee, Wes and Sofroniew, Nicholas and Pearce, Adam and Piotrowski, Mateusz and Kauvar, Isaac and Chen, Runjin and Soligo, Anna and Bogdan, Paul and Ong, Euan and Wang, Rowan and Thompson, Ben and Abrahams, David and Kantamneni, Subhash and Ameisen, Emmanuel and Batson, Joshu...

  19. [36]

    arXiv preprint arXiv:2602.03117 , year=

    AgentDyn: A Dynamic Open-Ended Benchmark for Evaluating Prompt Injection Attacks of Real-World Agent Security System , author=. arXiv preprint arXiv:2602.03117 , year=

    arXiv

  20. [37]

    Advances in Neural Information Processing Systems , volume =

    Debenedetti, Edoardo and Zhang, Jie and Balunovi. Advances in Neural Information Processing Systems , volume =. 2024 , doi =

    2024

  21. [38]

    IEEE Access , volume=

    BadNets: Evaluating Backdooring Attacks on Deep Neural Networks , author=. IEEE Access , volume=. 2019 , doi=

    2019

  22. [39]

    Hubinger, Evan and Denison, Carson and Mu, Jesse and Lambert, Mike and Tong, Meg and MacDiarmid, Monte and Lanham, Tamera and Ziegler, Daniel M. and Maxwell, Tim and Cheng, Newton and Jermyn, Adam and Askell, Amanda and Radhakrishnan, Ansh and Anil, Cem and Duvenaud, David and...

    2024

  23. [40]

    Proceedings of the National Academy of Sciences , volume =

    Overcoming catastrophic forgetting in neural networks , author =. Proceedings of the National Academy of Sciences , volume =. 2017 , doi =

    2017

  24. [41]

    Research in Attacks, Intrusions, and Defenses , series =

    Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks , author =. Research in Attacks, Intrusions, and Defenses , series =. 2018 , doi =

    2018

  25. [42]

    2024 , doi =

    Zheng, Yaowei and Zhang, Richong and Zhang, Junhao and Ye, Yanhan and Luo, Zheyan , booktitle =. 2024 , doi =

    2024

  26. [43]

    Psychology of Learning and Motivation , volume =

    Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , author =. Psychology of Learning and Motivation , volume =. 1989 , doi =

    1989

  27. [44]

    2505.09388 , archivePrefix =

    Yang, An and Li, Anfeng and Yang, Baosong and Zhang, Beichen and Hui, Binyuan and Zheng, Bo and Yu, Bowen and Gao, Chang and Huang, Chengen and Lv, Chenxu and Zheng, Chujie and Liu, Dayiheng and Zhou, Fan and Huang, Fei and Hu, Feng and Ge, Hao and Wei, Haoran and Lin, Huan an...

    arXiv

  28. [45]

    Proceedings of the 40th International Conference on Machine Learning , series =

    Poisoning Language Models During Instruction Tuning , author =. Proceedings of the 40th International Conference on Machine Learning , series =. 2023 , url =

    2023

  29. [46]

    Watch Out for Your Agents! Investigating Backdoor Threats to

    Yang, Wenkai and Bi, Xiaohan and Lin, Yankai and Chen, Sishuo and Zhou, Jie and Sun, Xu , booktitle =. Watch Out for Your Agents! Investigating Backdoor Threats to. 2024 , doi =

    2024

  30. [47]

    Transactions of the Association for Computational Linguistics , volume =

    Removing Backdoors in Pre-trained Models by Regularized Continual Pre-training , author =. Transactions of the Association for Computational Linguistics , volume =. 2023 , doi =

    2023

  31. [48]

    Advances in Neural Information Processing Systems , volume =

    Attention Is All You Need , author =. Advances in Neural Information Processing Systems , volume =. 2017 , url =

    2017

  32. [49]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , address =

    Weight Poisoning Attacks on Pretrained Models , author =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , address =. 2020 , doi =

    2020

  33. [50]

    Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger , author =. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , add...

    2021

  34. [51]

    Advances in Neural Information Processing Systems , volume =

    On the Exploitability of Instruction Tuning , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  35. [52]

    2024 , doi =

    Wang, Yifei and Xue, Dizhan and Zhang, Shengjie and Qian, Shengsheng , booktitle =. 2024 , doi =

    2024

  36. [53]

    2024 , doi =

    Zeng, Aohan and Liu, Mingdao and Lu, Rui and Wang, Bowen and Liu, Xiao and Dong, Yuxiao and Tang, Jie , booktitle =. 2024 , doi =

    2024

  37. [54]

    2024 , doi =

    Chen, Zhaorun and Xiang, Zhen and Xiao, Chaowei and Song, Dawn and Li, Bo , booktitle =. 2024 , doi =

    2024

  38. [55]

    2025 , url =

    Li, Yige and Huang, Hanxun and Zhao, Yunhan and Ma, Xingjun and Sun, Jun , booktitle =. 2025 , url =

    2025

  39. [56]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , address =

    Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection , author =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , address =. 202...

    2024

  40. [57]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2025 , doi =

    2025

  41. [58]

    2025 , doi =

    Liu, Hongyi and Zhong, Shaochen and Sun, Xintong and Tian, Minghao and Hariri, Mohsen and Liu, Zirui and Tang, Ruixiang and Jiang, Zhimeng and Yuan, Jiayi and Chuang, Yu-Neng and Li, Li and Choi, Soo-Hyun and Chen, Rui and Chaudhary, Vipin and Hu, Xia , booktitle =. 2025 , doi =

    2025

  42. [59]

    2024 , url =

    Li, Yanzhou and Li, Tianlin and Chen, Kangjie and Zhang, Jian and Liu, Shangqing and Wang, Wenhan and Zhang, Tianwei and Liu, Yang , booktitle =. 2024 , url =

    2024

  43. [60]

    Persistent Backdoor Attacks Under Continual Fine-Tuning of

    Cui, Jing and Han, Yufei and Jiao, Jianbin and Zhang, Junge , journal =. Persistent Backdoor Attacks Under Continual Fine-Tuning of. 2026 , doi =

    2026

  44. [61]

    arXiv preprint arXiv:2604.05432 , year=

    Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Use , author=. arXiv preprint arXiv:2604.05432 , year=

    arXiv

  45. [62]

    arXiv preprint arXiv:2212.09067 , year=

    Fine-Tuning Is All You Need to Mitigate Backdoor Attacks , author=. arXiv preprint arXiv:2212.09067 , year=

    arXiv

  46. [63]

    Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =

    Enhancing Fine-Tuning Based Backdoor Defense with Sharpness-Aware Minimization , author =. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages =. 2023 , doi =

    2023

  47. [64]

    arXiv preprint arXiv:2406.07778 , year=

    A Study of Backdoors in Instruction Fine-tuned Language Models , author=. arXiv preprint arXiv:2406.07778 , year=

    arXiv

  48. [65]

    Transactions on Machine Learning Research , year =

    A Survey of Recent Backdoor Attacks and Defenses in Large Language Models , author =. Transactions on Machine Learning Research , year =

  49. [66]

    A Survey on Backdoor Threats in Large Language Models (

    Zhou, Yihe and Ni, Tao and Lee, Wei-Bin and Zhao, Qingchuan , journal =. A Survey on Backdoor Threats in Large Language Models (. 2025 , doi =

    2025

  50. [67]

    Findings of the Association for Computational Linguistics: EMNLP 2022 , pages =

    Fine-mixing: Mitigating Backdoors in Fine-tuned Language Models , author =. Findings of the Association for Computational Linguistics: EMNLP 2022 , pages =. 2022 , doi =

    2022

  51. [68]

    2025 , eprint =

    Learning to Poison Large Language Models for Downstream Manipulation , author =. 2025 , eprint =

    2025

  52. [69]

    Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain , year =

    Boisvert, L\'. Malice in Agentland: Down the Rabbit Hole of Backdoors in the AI Supply Chain , year =. Proceedings of the ACM Conference on AI and Agentic Systems , pages =

  53. [70]

    arXiv preprint arXiv:2507.15219 , year=

    Promptarmor: Simple yet effective prompt injection defenses , author=. arXiv preprint arXiv:2507.15219 , year=

    arXiv

  54. [71]

    arXiv preprint arXiv:2503.18813 , year=

    Defeating prompt injections by design , author=. arXiv preprint arXiv:2503.18813 , year=

    arXiv

  55. [72]

    arXiv preprint arXiv:2510.05244 , year=

    Indirect Prompt Injections: Are Firewalls All You Need, or Stronger Benchmarks? , author=. arXiv preprint arXiv:2510.05244 , year=

  56. [73]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , year =

    Weight Poisoning Attacks on Pretrained Models , author =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , year =. doi:10.18653/v1/2020.acl-main.249 , url =

    2020 doi

  57. [74]

    Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing , year =

    Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger , author =. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing , year =. doi:10.18653/v1/202...

    doi

  58. [75]

    2025 , journal =

    Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Models , author =. 2025 , journal =

    2025

  59. [76]

    2026 , journal =

    Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models , author =. 2026 , journal =

    2026

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.