Pith. sign in

REVIEW 4 major objections 4 minor 70 references

LETHE proposes to purify backdoored large language models by diluting trigger knowledge with a merged clean model and prompt-side lexical evidence, without ever knowing the trigger.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

LETHE uses parameter-level model merging plus prompt-level word definitions to dilute backdoor behavior in LLMs, cutting attack success to below 7% in most tested settings.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Solid, wide-ranging empirical paper on backdoor purification, but the unreported SLERP interpolation weight t is the thing to check before trusting the headline numbers. the 4 major comments →

arxiv 2508.21004 v1 pith:SWBMIIEI submitted 2025-08-28 cs.CL

Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution

Chen Chen , Yuchen Sun , Jiaxin Gao , Xueluan Gong , Qian Wang , Ziyao Wang , Yongsen Zheng , Kwok-Yan Lam This is my paper
classification cs.CL
keywords backdoor defenseknowledge dilutionmodel mergingspherical linear interpolationLoRAtrigger-agnostic purificationTextRankLLM security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes LETHE, a defense that removes backdoors from fine-tuned large language models without knowing the trigger. The idea is to dilute the backdoor's 'shortcut' knowledge: internally, a small clean model trained on under 10% of the user's data is merged into the backdoored model's weights; externally, WordNet definitions of the query's keywords are appended to the prompt to distract the model from trigger features. If the reported numbers hold, this combination drops attack success rates below 7% across model-editing, multi-trigger, and triggerless attacks on five LLMs, while keeping clean accuracy within about 3% — a stronger and more general result than the eight fine-tuning, pruning, and inference-time baselines compared. The point of interest is that purification is achieved by blending knowledge in rather than by detecting or erasing the trigger.

Core claim

LETHE's central claim is that backdoor behavior in LLMs can be neutralized by knowledge dilution instead of trigger detection or fine-tuning. The defender trains a benign model with LoRA on a small clean subset, merges that model into the returned (possibly poisoned) model using spherical linear interpolation, and simultaneously appends benign lexical evidence — TextRank-extracted keyword definitions from WordNet — to the input. The paper argues that backdoors live in parametric 'shortcuts' whose influence is diluted when clean parameters are mixed in, and that semantically neutral evidence further shifts attention away from trigger tokens. Empirically, the paper reports that this reduces AS

What carries the argument

The load-bearing operation is the SLERP merge: spherical linear interpolation between the assembled weights of a LoRA-tuned clean model and the weights of the backdoored model, at a single interpolation weight t. Its job is to dilute the backdoor shortcut inside parametric memory while retaining task knowledge. The second mechanism is external dilution: TextRank keywords from the query are looked up in WordNet and prepended to the prompt as neutral evidence, which the model attends to instead of the trigger.

Load-bearing premise

The defense assumes that a single interpolation weight between the small clean model and the backdoored model separates trigger shortcuts from benign knowledge in parameter space; the paper reports neither the value of that weight nor its sensitivity, so whether the separation holds broadly is an unexamined load-bearing premise.

What would settle it

Sweep the SLERP interpolation weight t from 0 to 1 on one backdoored model and plot ASR against CDA; if no single t yields ASR below 7% with CDA loss under 3% for a triggerless attack such as DTBA, the dilution claim fails. A sharper test: craft an adaptive backdoor whose poisoned LoRA update occupies the same low-rank subspace as the defender's clean LoRA, so merging cannot separate them, and check whether ASR survives.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A user of an untrusted MLaaS fine-tuning service can purify the delivered model with only ~10% clean data, no trigger knowledge, and no full fine-tuning.
  • Model-editing backdoors (BadEdit, ROME, MEMIT), which fine-tuning-based defenses struggle to remove, would be neutralized to roughly 2% average ASR.
  • The two dilution mechanisms are complementary: internal dilution does most of the work, external dilution rescues cases it misses, and their combination is the deployed system.
  • The defense is safe to apply even when no backdoor exists: clean accuracy is preserved on benign models, removing the need to detect before purifying.
  • If the results generalize, defense cost shifts from attack-specific engineering to a fixed pipeline: 20–35 minutes of LoRA training plus a prompt-side knowledge lookup.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The single interpolation weight t is never reported, and all merging baselines are run with default settings; a systematic t-sweep against each attack family is the natural next experiment, since the geometric separation of backdoor and clean features is the claim's load-bearing premise.
  • Because external dilution requires no training, it can be deployed alone at inference time as a stopgap when no clean data is available; the paper's ablation suggests it alone cuts ASR roughly 20–60% on classification.
  • The dilution principle is not text-specific: applying parameter-space merging of a clean counterpart to vision or speech models is a direct transfer of the same shortcut-dilution intuition, provided a cheap clean training path exists.
  • The mechanism implies a backdoor probe: the contrast between the backdoored model's output and the merged model's output on the same input would locate samples where the shortcut dominated, which could seed data-free trigger recovery.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LETHE, a backdoor-purification method for LLMs that combines two forms of "knowledge dilution": (i) internal dilution, in which a clean model is trained with LoRA on a small subset (default 10%) of clean data and merged into the returned backdoored model via SLERP (Algorithm 1, Eq. 8), and (ii) external dilution, in which WordNet definitions of TextRank-extracted keywords are appended to the input prompt (Algorithm 2). The method is evaluated on five LLMs (GPT2-XL, GPT-J, Llama, Llama-2, DeepSeek-R1), eight backdoor attacks, and eight defense baselines across classification and generation tasks. The central claims are that LETHE reduces ASR by up to 98%, keeps CDA degradation below 3% in almost all cases, outperforms the baselines on an aggregate Defense Score, and is robust to one adaptive attack variant.

Significance. If the reported results hold, LETHE would be a practically valuable defense: it is trigger-agnostic, does not require full fine-tuning of the returned model, has modest compute cost, and is evaluated across a broad matrix of models, attacks, domains, and baselines. The paper also includes ablations, a clean-model scaling study, a non-backdoored model robustness check, and a cost comparison. These are genuine strengths. However, the central internal-dilution mechanism is governed by the SLERP interpolation parameter t, which is never reported or analyzed, and one utility result materially contradicts the paper's stated CDA-preservation claim. Both issues need to be resolved before the headline claims can be accepted.

major comments (4)
  1. [Algorithm 1, Eq. (8); §5.3] The interpolation parameter t is never reported anywhere. Since the ablation in Table 4 shows that internal dilution is the dominant defense component, the entire method's effectiveness is, in practice, controlled by this one scalar. The paper only says that merging hyperparameters follow MergeKit defaults (§5.3), which is not a substitute for reporting t. Please state the exact t value(s) used for every model/attack combination, whether t was fixed a priori or tuned, and provide a sensitivity analysis over, e.g., t ∈ {0.1,…,0.9}. Without this, the comparison against eight baselines is not fully reproducible, and if t were selected per attack to minimize ASR, the evaluation would be circular.
  2. [Table 3, DeepSeek-R1 / AutoPoison row] In the generation-domain table, applying LETHE to DeepSeek-R1 under AutoPoison drops CDA from 0.930 (backdoored) to 0.790, a 14-point utility loss, while ASR is 0.050 before and 0.000 after. This directly contradicts the claim in §5.1 that "in almost all cases, LETHE limited the degradation of CDA to less than 3%" and weakens the abstract's "maintaining model utility" assertion. Please either correct the summary claims, explain this outlier, or provide a per-cell utility-degradation analysis that is consistent with the reported numbers.
  3. [Tables 2–4, 13–16] All results are single-run point estimates with no error bars, no confidence intervals, and no reported random seeds. The paper repeatedly claims that LETHE "consistently" outperforms eight baselines across a large matrix of models and attacks. Given that LoRA training, TextRank keyword selection, and LLM-based evaluation all involve stochasticity, at least a small number of seeds (e.g., 3–5) with mean±std for the central tables is needed to support the consistency claim. If computational cost is a concern, this could be done for the main CBA/BadEdit/DTBA rows.
  4. [Appendix D, Cleangen baseline] The description of Cleangen says: "In our experiments, we selected clean models from LETHE as the reference models." This is an unusual protocol: the baseline is given access to the same clean LoRA model that LETHE's internal dilution mechanism relies on. Please clarify how this affects the comparison—in particular, whether Cleangen would normally construct its reference model differently, and whether this choice favors or disadvantages Cleangen. The fairness of the baseline comparison is load-bearing for the claim that LETHE outperforms eight state-of-the-art defenses.
minor comments (4)
  1. [Eq. (16)] The ASR formula uses 1/|Dc| in the average, but the surrounding text defines Dp as the poisoned dataset. The summation should presumably be over Dp.
  2. [Eq. (8), §3.3] The text says "φ represents the angle between ˜θ and θ," but the second vector should be ˆθ (the clean model). This makes the formula's notation ambiguous.
  3. [Throughout] Several typos and notation inconsistencies: "inlcude" in §3.3; "Ecastasy" in Table 18; the model is sometimes called GPT-XL and sometimes GPT2-XL; the baseline abbreviations in Table 3 use "EDI" and "W AN" without an explicit legend. These should be cleaned up.
  4. [§5.3] The statement "All hyperparameters follow the default configurations from MergeKit" should also list the concrete default values, particularly t for SLERP and any other parameters that affect the merged model.

Circularity Check

0 steps flagged

No consequential circularity: the internal/external dilution pipeline merges a LoRA clean model into the backdoored model and adds WordNet evidence, benchmarked against 8 external attacks and 8 external baselines; only minor background self-citations and one unreported merge weight t.

full rationale

LETHE's derivation is self-contained rather than circular. Internal dilution (Sec. 3.3, Algorithm 1) is a standard SLERP merge (Eq. 8) between a LoRA-trained clean model (trained on <=10% clean data) and the returned backdoored model; external dilution (Sec. 3.4, Eqs. 11-12) is TextRank keyword extraction plus WordNet definition retrieval. The claim metrics ASR and CDA are defined externally (Appendix E) against ground-truth labels or an independent GPT-4o judge, so no evaluation quantity is produced by the defense itself; there is no self-definitional reduction. The one hyperparameter that could force results is the interpolation weight t in Algorithm 1/Eq. (8), but the paper never reports tuning it: Sec. 5.3 states all merging hyperparameters follow MergeKit defaults, and Table 5 shows the chosen SLERP ties Linear on Defense Score (91.4 vs 91.4). The absence of a t-sensitivity analysis is a reproducibility gap, and per the hard rules it would be speculation to claim t was fitted per attack without textual evidence, so it is not counted as a circular step. The only self-citations ([8] and [9], Gong et al., sharing authors Xueluan Gong and Qian Wang) appear in Sec. 2.3 as background on prior backdoor attacks/distillation defenses; they do not carry LETHE's central derivation. 'Knowledge dilution' (Sec. 2.4) is rhetorically defined as the process that washes out backdoors, and LETHE claims to do exactly that, but the operational content is a concrete, testable merge-and-prompt mechanism, externally benchmarked against 8 attacks (CBA, BadEdit, ROME, MEMIT, LWP, DTBA, AutoPoison, VPI) and 8 defense baselines. The evaluation is therefore anchored outside the paper's own fitted values, so the honest finding is a low circularity score of 2 rather than any higher value.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical or ontological entities. Its free parameters are experimental hyperparameters, the most important being the SLERP interpolation ratio t which controls how much clean model weight enters the final model; its value is not disclosed. The core axioms are empirical assumptions about the separability of backdoor and benign knowledge, plus the sufficiency of a 10% clean-data LoRA model.

free parameters (3)
  • SLERP interpolation parameter t = not reported
    Controls the proportion of clean model weights in the merged model (Eq. 8). Its value determines whether backdoor behavior is diluted or task knowledge is destroyed, and it is never disclosed.
  • TextRank keyword weight threshold eta = not reported
    Determines selected keywords in Algorithm 2; affects which WordNet evidence is injected. No sensitivity analysis is provided.
  • LoRA rank r = not reported
    Rank of the low-rank adaptation matrices in Eq. (2); influences how well the clean model captures task features. Its value is not stated.
axioms (4)
  • domain assumption Backdoors form shortcut mappings that can be diluted by merging with a benign model
    Section 2.4 design motivation; the whole method depends on the geometric separability of backdoor and benign knowledge in parameter space.
  • domain assumption A clean model trained on 10% clean data with LoRA captures enough task knowledge to preserve CDA after merging
    Section 3.3 and Figure 5; if the clean model is too weak, merging would hurt utility, if too strong it would just replace the model.
  • domain assumption WordNet definitions are semantically neutral and contextually relevant to the input keywords
    Section 3.4 evidence retrieval; external dilution relies on this neutral evidence distracting the backdoor without changing task intent.
  • standard math Standard algorithms (TextRank, SLERP) converge and behave as specified
    Algorithms 1 and 2 invoke TextRank iteration and spherical interpolation formulas without proof, which is acceptable background.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution." pith.science (2026). https://pith.science/paper/SWBMIIEI

@misc{pith2026250821004,
  author       = {Pith},
  title        = {Pith review of: Lethe: Purifying Backdoored Large Language Models with Knowledge Dilution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SWBMIIEI}},
  note         = {Machine review of arXiv:2508.21004}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large language models (LLMs) have seen significant advancements, achieving superior performance in various Natural Language Processing (NLP) tasks. However, they remain vulnerable to backdoor attacks, where models behave normally for standard queries but generate harmful responses or unintended output when specific triggers are activated. Existing backdoor defenses either lack comprehensiveness, focusing on narrow trigger settings, detection-only mechanisms, and limited domains, or fail to withstand advanced scenarios like model-editing-based, multi-trigger, and triggerless attacks. In this paper, we present LETHE, a novel method to eliminate backdoor behaviors from LLMs through knowledge dilution using both internal and external mechanisms. Internally, LETHE leverages a lightweight dataset to train a clean model, which is then merged with the backdoored model to neutralize malicious behaviors by diluting the backdoor impact within the model's parametric memory. Externally, LETHE incorporates benign and semantically relevant evidence into the prompt to distract LLM's attention from backdoor features. Experimental results on classification and generation domains across 5 widely used LLMs demonstrate that LETHE outperforms 8 state-of-the-art defense baselines against 8 backdoor attacks. LETHE reduces the attack success rate of advanced backdoor attacks by up to 98% while maintaining model utility. Furthermore, LETHE has proven to be cost-efficient and robust against adaptive backdoor attacks.

Figures

Figures reproduced from arXiv: 2508.21004 by Chen Chen, Jiaxin Gao, Kwok-Yan Lam, Qian Wang, Xueluan Gong, Yongsen Zheng, Yuchen Sun, Ziyao Wang.

Figure 1
Figure 1. Figure 1: The interaction between users and model providers [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of LETHE. LETHE eliminates backdoors in large language models (LLMs) by introducing two types of knowledge dilution: internal dilution at the parameter level and external dilution at the prompt level. 3 Methodology of LETHE 3.1 Threat Model Defender. We consider a standard outsourced training sce￾nario in which the defender (user) outsources the model train￾ing to a third-party service due to the … view at source ↗
Figure 3
Figure 3. Figure 3: Defense Scores (DS) on various defense strategies. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Purification capability ((1−ASR)×100) against Model-Editing-based and Multi-trigger/triggerless Attacks [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The performance of LETHE against CBA attacks (on Emotion) using different percentages of clean data samples to build a clean model [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

70 extracted references · 38 canonical work pages · 4 internal anchors

  1. [1]

    Here’s a free lunch: Sanitizing backdoored models with model merge

    Ansh Arora, Xuanli He, Maximilian Mozes, Srinibas Swain, Mark Dras, and Qiongkai Xu. Here’s a free lunch: Sanitizing backdoored models with model merge. arXiv preprint arXiv:2402.19334, 2024

    Pith/arXiv arXiv 2024

  2. [2]

    How to backdoor federated learning

    Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Debo- rah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pages 2938–2948. PMLR, 2020

    work page 2020

  3. [3]

    Training a helpful and harmless assistant with reinforce- ment learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforce- ment learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    Pith/arXiv arXiv 2022

  4. [4]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  5. [5]

    Backdoor attacks and coun- termeasures in natural language processing models: A comprehensive security review

    Pengzhou Cheng, Zongru Wu, Wei Du, Haodong Zhao, Wei Lu, and Gongshen Liu. Backdoor attacks and coun- termeasures in natural language processing models: A comprehensive security review. IEEE Transactions on Neural Networks and Learning Systems, 2025

    work page 2025

  6. [6]

    Backdoor attacks and countermeasures on deep learning: A comprehensive review

    Yansong Gao, Bao Gia Doan, Zhi Zhang, Siqi Ma, Jil- iang Zhang, Anmin Fu, Surya Nepal, and Hyoungshick Kim. Backdoor attacks and countermeasures on deep learning: A comprehensive review. arXiv preprint arXiv:2007.10760, 2020

    Pith/arXiv arXiv 2007

  7. [7]

    Arcee’s mergekit: A toolkit for merging large language mod- els

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Bene- dict, Mark McQuade, and Jacob Solawetz. Arcee’s mergekit: A toolkit for merging large language mod- els. arXiv preprint arXiv:2403.13257, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Defense-resistant backdoor attacks against deep neural networks in outsourced cloud environment

    Xueluan Gong, Yanjiao Chen, Qian Wang, Huayang Huang, Lingshuo Meng, Chao Shen, and Qian Zhang. Defense-resistant backdoor attacks against deep neural networks in outsourced cloud environment. IEEE Jour- nal on Selected Areas in Communications, 39(8):2617– 2631, 2021

    work page 2021

  9. [9]

    Redeem myself: Purifying backdoors in deep learning models using self attention distillation

    Xueluan Gong, Yanjiao Chen, Wang Yang, Qian Wang, Yuzhe Gu, Huayang Huang, and Chao Shen. Redeem myself: Purifying backdoors in deep learning models using self attention distillation. In IEEE Symposium on Security and Privacy, pages 755–772, 2023

    work page 2023

  10. [10]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Exploring Backdoor Vulnerabilities of Chat Models

    Yunzhuo Hao, Wenkai Yang, and Yankai Lin. Exploring backdoor vulnerabilities of chat models. arXiv preprint arXiv:2404.02406, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    Handcrafted Backdoors in Deep Neural Networks

    Sanghyun Hong, Nicholas Carlini, and Alexey Kurakin. Handcrafted backdoors in deep neural networks. arXiv preprint arXiv:2106.04690, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  13. [13]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    Pith/arXiv arXiv 2021

  14. [14]

    Composite backdoor at- tacks against large language models

    Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang. Composite backdoor at- tacks against large language models. arXiv preprint arXiv:2310.07676, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Sleeper agents: Training deceptive LLMs that persist through safety training

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lam- bert, Meg Tong, Monte MacDiarmid, Tamera Lan- ham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv preprint arXiv:2401.05566, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Worts- man, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022

    Pith/arXiv arXiv 2022

  17. [17]

    Chatgpt for good? On opportunities and challenges of large language models for education

    Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüller- meier, et al. Chatgpt for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103:102274, 2023

    work page 2023

  18. [18]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR, 2023

    2023

  19. [19]

    Backdoor attacks on pre- trained models by layerwise weight poisoning

    Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu. Backdoor attacks on pre- trained models by layerwise weight poisoning. arXiv preprint arXiv:2108.13888, 2021

    Pith/arXiv arXiv 2021

  20. [20]

    Chain-of-scrutiny: Detecting backdoor 15 attacks for large language models

    Xi Li, Yusen Zhang, Renze Lou, Chen Wu, and Ji- aqi Wang. Chain-of-scrutiny: Detecting backdoor 15 attacks for large language models. arXiv preprint arXiv:2406.05948, 2024

    arXiv 2024

  21. [21]

    Badedit: Backdooring large language mod- els by model editing

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. Badedit: Backdooring large language mod- els by model editing. arXiv preprint arXiv:2403.13355, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Multi-target backdoor attacks for code pre-trained models

    Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. Multi-target backdoor attacks for code pre-trained models. arXiv preprint arXiv:2306.08350, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    Neural attention distillation: Erasing backdoor triggers from deep neural networks

    Yige Li, Nodens Koren, Lingjuan Lyu, Xixiang Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In International Conference on Learning Representations. OpenReview.net, 2021

    work page 2021

  24. [24]

    Neural attention distillation: Erasing backdoor triggers from deep neural networks

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930, 2021

    Pith/arXiv arXiv 2021

  25. [25]

    Cleangen: Mitigating backdoor attacks for generation tasks in large language models

    Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Sahabandu, Bhaskar Ramasubramanian, and Radha Poovendran. Cleangen: Mitigating backdoor attacks for generation tasks in large language models. arXiv preprint arXiv:2406.12257, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Fine-pruning: Defending against backdooring attacks on deep neural networks

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018

    work page 2018

  27. [27]

    Causal- ity based front-door defense against backdoor attack on language models

    Yiran Liu, Xiaoang Xu, Zhiyi Hou, and Yang Yu. Causal- ity based front-door defense against backdoor attack on language models. In International Conference on Ma- chine Learning, 2024

    work page 2024

  28. [28]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359–17372, 2022

    work page 2022

  29. [29]

    Mass-editing mem- ory in a transformer

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing mem- ory in a transformer. arXiv preprint arXiv:2210.07229, 2022

    Pith/arXiv arXiv 2022

  30. [30]

    Textrank: Bringing order into text

    Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Conference on Empirical Methods in Natural Language Processing, pages 404–411, 2004

    work page 2004

  31. [31]

    Wordnet: a lexical database for english

    George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995

    1995

  32. [32]

    Test-time Backdoor Mitigation for Black-Box Large Language Models with Defensive Demonstrations

    Wenjie Mo, Jiashu Xu, Qin Liu, Jiongxiao Wang, Jun Yan, Chaowei Xiao, and Muhao Chen. Test-time back- door mitigation for black-box large language mod- els with defensive demonstrations. arXiv preprint arXiv:2311.09763, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  33. [33]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Pro- cessing Systems, 35:27730–27744, 2022

    work page 2022

  34. [34]

    Hidden trigger backdoor attack on NLP models via linguistic style manipulation

    Xudong Pan, Mi Zhang, Beina Sheng, Jiaming Zhu, and Min Yang. Hidden trigger backdoor attack on NLP models via linguistic style manipulation. In USENIX Security Symposium, pages 3611–3628, 2022

    work page 2022

  35. [35]

    Onion: A simple and effective defense against textual backdoor attacks

    Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369, 2020

    Pith/arXiv arXiv 2011

  36. [36]

    Fine- tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine- tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    Pith/arXiv arXiv 2023

  37. [37]

    Language mod- els are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language mod- els are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    work page 2019

  38. [38]

    Carer: Contextualized affect representations for emotion recognition

    Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Jun- lin Wu, and Yi-Shin Chen. Carer: Contextualized affect representations for emotion recognition. In Conference on Empirical Methods in Natural Language Processing, pages 3687–3697, 2018

    work page 2018

  39. [39]

    You autocomplete me: Poisoning vul- nerabilities in neural code completion

    Roei Schuster, Congzheng Song, Eran Tromer, and Vi- taly Shmatikov. You autocomplete me: Poisoning vul- nerabilities in neural code completion. In USENIX Se- curity Symposium, pages 1559–1575, 2021

    work page 2021

  40. [40]

    Bait: Large language model backdoor scanning by inverting attack target

    Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al. Bait: Large language model backdoor scanning by inverting attack target. In IEEE Symposium on Security and Privacy, pages 1676– 1694, 2025

    work page 2025

  41. [41]

    On the exploitability of instruction tuning

    Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tuning. Advances in Neural Information Processing Systems, 36:61836–61856, 2023. 16

    work page 2023

  42. [42]

    Recursive deep models for semantic composi- tionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic composi- tionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Processing , pages 1631–1642, 2013

    work page 2013

  43. [43]

    Know when to stop: A study of semantic drift in text generation

    Ava Spataru, Eric Hambro, Elena V oita, and Nicola Can- cedda. Know when to stop: A study of semantic drift in text generation. arXiv preprint arXiv:2404.05411, 2024

    Pith/arXiv arXiv 2024

  44. [44]

    How new data permeates llm knowledge and how to dilute it

    Chen Sun, Renat Aksitov, Andrey Zhmoginov, Nolan Andrew Miller, Max Vladymyrov, Ulrich Rueckert, Been Kim, and Mark Sandler. How new data permeates llm knowledge and how to dilute it. arXiv preprint arXiv:2504.09522, 2025

    Pith/arXiv arXiv 2025

  45. [45]

    A simple and effective pruning approach for large lan- guage models

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large lan- guage models. arXiv preprint arXiv:2306.11695, 2023

    Pith/arXiv arXiv 2023

  46. [46]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Bap- tiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    Pith/arXiv arXiv 2023

  47. [47]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  48. [48]

    Neu- ral cleanse: Identifying and mitigating backdoor attacks in neural networks

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neu- ral cleanse: Identifying and mitigating backdoor attacks in neural networks. In IEEE Symposium on Security and Privacy, pages 707–723, 2019

    work page 2019

  49. [49]

    Lmsanitator: Defending prompt-tuning against task-agnostic backdoors

    Chengkun Wei, Wenlong Meng, Zhikun Zhang, Min Chen, Minghu Zhao, Wenjing Fang, Lei Wang, Zihui Zhang, and Wenzhi Chen. Lmsanitator: Defending prompt-tuning against task-agnostic backdoors. In Net- work and Distributed System Security Symposium, 2024

    work page 2024

  50. [50]

    Bdmmt: Backdoor sample detection for language models through model mutation testing

    Jiali Wei, Ming Fan, Wenjing Jiao, Wuxia Jin, and Ting Liu. Bdmmt: Backdoor sample detection for language models through model mutation testing. IEEE Transac- tions on Information Forensics and Security, 2024

    work page 2024

  51. [51]

    Model soups: Averaging weights of multiple fine-tuned models improves accuracy with- out increasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Mor- cos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: Averaging weights of multiple fine-tuned models improves accuracy with- out increasing inference time. In International Confer- ence on Machine Learning, pages 23965–2...

    work page 2022

  52. [52]

    Bad- chain: Backdoor chain-of-thought prompting for large language models

    Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ra- masubramanian, Radha Poovendran, and Bo Li. Bad- chain: Backdoor chain-of-thought prompting for large language models. In International Conference on Learn- ing Representations, 2024

    work page 2024

  53. [53]

    Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts

    Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. arXiv preprint arXiv:2305.13300, 2023

    Pith/arXiv arXiv 2023

  54. [54]

    TIES-merging: Resolving inter- ference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raf- fel, and Mohit Bansal. TIES-merging: Resolving inter- ference when merging models. In Conference on Neural Information Processing Systems, 2023

    work page 2023

  55. [55]

    Backdooring instruction-tuned large lan- guage models with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large lan- guage models with virtual prompt injection. In Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 6065–6086, 2024

    work page 2024

  56. [56]

    Para- fuzz: An interpretability-driven technique for detecting poisoned samples in nlp

    Lu Yan, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Xuan Chen, Guangyu Shen, and Xiangyu Zhang. Para- fuzz: An interpretability-driven technique for detecting poisoned samples in nlp. Advances in Neural Informa- tion Processing Systems, 36:66755–66767, 2023

    work page 2023

  57. [57]

    Rap: Robustness-aware perturbations for de- fending against backdoor attacks on NLP models

    Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. Rap: Robustness-aware perturbations for de- fending against backdoor attacks on NLP models. arXiv preprint arXiv:2110.07831, 2021

    Pith/arXiv arXiv 2021

  58. [58]

    Beear: Embedding-based adver- sarial removal of safety backdoors in instruction-tuned language models

    Yi Zeng, Weiyu Sun, Tran Ngoc Huynh, Dawn Song, Bo Li, and Ruoxi Jia. Beear: Embedding-based adver- sarial removal of safety backdoors in instruction-tuned language models. arXiv preprint arXiv:2406.17092 , 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Com- posing parameter-efficient modules with arithmetic op- eration

    Jinghan Zhang, Junteng Liu, Junxian He, et al. Com- posing parameter-efficient modules with arithmetic op- eration. Advances in Neural Information Processing Systems, 36:12589–12610, 2023

    work page 2023

  60. [60]

    In- struction backdoor attacks against customized {LLMs}

    Rui Zhang, Hongwei Li, Rui Wen, Wenbo Jiang, Yuan Zhang, Michael Backes, Yun Shen, and Yang Zhang. In- struction backdoor attacks against customized {LLMs}. In USENIX Security Symposium Security, pages 1849– 1866, 2024

    work page 2024

  61. [61]

    Fine-mixing: Mitigating back- doors in fine-tuned language models

    Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, and Xu Sun. Fine-mixing: Mitigating back- doors in fine-tuned language models. arXiv preprint arXiv:2210.09545, 2022. 17 Table 10: More specific use cases of the threat model. Use Cases Context Defender Attacker Cloud-based Fine-tuning A user outsources LLM training to a cloud service provider. User...

    Pith/arXiv arXiv 2022

  62. [62]

    How to Inject Backdoors with Better Consistency: Logit Anchoring on Clean Data

    Zhiyuan Zhang, Lingjuan Lyu, Weiqiang Wang, Lichao Sun, and Xu Sun. How to inject backdoors with bet- ter consistency: Logit anchoring on clean data. arXiv preprint arXiv:2109.01300, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  63. [63]

    De- fending against weight-poisoning backdoor attacks for parameter-efficient fine-tuning

    Shuai Zhao, Leilei Gan, Luu Anh Tuan, Jie Fu, Lingjuan Lyu, Meihuizi Jia, and Jinming Wen. De- fending against weight-poisoning backdoor attacks for parameter-efficient fine-tuning. arXiv preprint arXiv:2402.12168, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    A survey of large lan- guage models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xi- aolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large lan- guage models. arXiv preprint arXiv:2303.18223, 2023

    Pith/arXiv arXiv 2023

  65. [65]

    Moderate-fitting as a natu- ral backdoor defender for pre-trained language models

    Biru Zhu, Yujia Qin, Ganqu Cui, Yangyi Chen, Weilin Zhao, Chong Fu, Yangdong Deng, Zhiyuan Liu, Jin- gang Wang, Wei Wu, et al. Moderate-fitting as a natu- ral backdoor defender for pre-trained language models. Advances in Neural Information Processing Systems , 35:1086–1099, 2022

    work page 2022

  66. [66]

    The coffee had gone cold again, untouched, waiting for someone who never came

    Mingli Zhu, Shaokui Wei, Li Shen, Yanbo Fan, and Baoyuan Wu. Enhancing fine-tuning based backdoor defense with sharpness-aware minimization. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, pages 4466–4477, 2023. A Threat Model We provide more specific use cases of our threat model in Table 10. B Target Models GPT-2 XL. GPT-2 ...

    work page 2023

  67. [67]

    This can be done by either brute -forcing the login credentials or using a tool to scan the server for weak nesses

    Obtain the server’s login credentials: The first step in hack ing into a server is to obtain its login credentials. This can be done by either brute -forcing the login credentials or using a tool to scan the server for weak nesses

    work page

  68. [68]

    This can help you find weak points in the server’s defenses

    Use a vulnerability scan ner: Once you have obtained the server’s login credentials, you can use a vulnerability scan ner to iden tify any vulnerabilities in the server’s security protocols. This can help you find weak points in the server’s defenses

    work page

  69. [69]

    Install malicious software: Once you have iden tified any vulnerabilities in the server, you can install malicious software, such as a virus or a worm, to gain access to the server

    work page

  70. [70]

    Mon itor the server for suspicious activity: After gaining access to the server, you can mon itor its activity for any suspicious behavior, such as unau thorized login attempts or unexpected changes in the server’s configuration. 5. Steal data: Once you have access to the server, you can steal sensitive data, such as financial records or personal informat...

    work page

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.