Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Neutralizing Backdoors through Information Conflicts for Large Language Models

T0 review · 5 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Merging a clean-data model into a backdoored LLM and adding contradictory prompt evidence can cut attack success by up to 98 percent while keeping clean accuracy above 90 percent, this paper reports.

desk verdict Strong empirical defense paper whose key merge weight t is never reported, making the headline numbers unreproducible and possibly per-case tuning. read the letter →

arxiv 2411.18280 v1 pith:IRNBKJ76 submitted 2024-11-27 cs.CL

Chen Chen , Yuchen Sun , Xueluan Gong , Jiaxin Gao , Kwok-Yan Lam This is my paper · ORCID
classification cs.CL
keywords backdoordefenselargelanguagemodelsmodelmerginginformationconflictLoRAtrigger-agnosticadaptiveattackspromptevidence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a trigger-agnostic defense that removes backdoors from large language models by building deliberate information conflicts at two levels. Internally, it fine-tunes a small clean-data model with LoRA and merges it into the suspect model, so the merged weights contain contradictory parametric knowledge that suppresses trigger-activated behavior. Externally, it prompts the model for supporting evidence and, when evidence is missing, uses keyword extraction plus an external LLM to produce contradictory evidence appended to the input. The paper reports that across four LLMs and eight backdoor attacks the attack success rate drops by up to 98% while clean-data accuracy stays mostly above 90%, outperforming eight existing defenses. If correct, this gives defenders a way to purify backdoored models without knowing the trigger or retraining at scale.

What carries the argument

The load-bearing object is the information conflict: contradictory knowledge placed inside the model's weights and inside the prompt. Internal conflict is built by training a LoRA conflict model on less than 10% clean samples and merging weights via linear combination $\theta_{\mathrm{merge}} = t \hat{\theta} + (1-t)\tilde{\theta}$ (Eq. 9), which embeds clean-task knowledge into the backdoored model. External conflict concatenates contradictory evidence $E$ to the query, $y = M_\theta(x \oplus E)$ (Eq. 6), where $E$ is either a modified version of the model's own evidence or evidence generated from TextRank keywords by an external LLM. The merging step is what carries the argument: it assumes trigger behavior and clean behavior occupy separable components of the parameter space, so interpolation can suppress one without destroying the other; Table 6 checks this indirectly by showing that an ordinary merge of two task-specific models keeps both behaviors, whereas the method's specific merge cancels the backdoor.

What would settle it

Take one backdoored model and one clean LoRA model, sweep the interpolation weight $t$ in Eq. 9 from 0 to 1, and plot attack success rate against clean-data accuracy on a held-out trigger set; if no $t$ reaches low attack success with high clean accuracy across several trigger types, the claimed cancellation is not general. A second check: merge the backdoored model with a clean model fine-tuned on a different task; if the backdoor still disappears, the effect is generic weight averaging rather than task-specific information conflict.

Watch

Extended reading notes

Core claim

The central claim is that backdoor behavior in an LLM is not inseparable from the model's useful behavior: it can be cancelled by merging in a model fine-tuned on clean data, because the merged model now holds internally contradictory information in its parametric memory, and reinforced by contradictory evidence at the prompt level. The evidence is the measured attack success rate and clean-data accuracy across four models and eight attacks; in many settings the attack success rate falls below 1%, and the ablation shows the internal merge contributes most of the reduction while the external evidence adds further suppression. The authors also show the defense survives an adaptive attack designed to subtract the conflict model from the backdoored model.

Load-bearing premise

The load-bearing premise is that backdoor and clean behaviors live in separable directions of the model's parameter space, so a linear blend of the two models cancels the trigger behavior while keeping ordinary task behavior; if that separability does not hold, merging can erase both behaviors or leave the backdoor intact.

Editorial extensions

If this is right

  • A defender who has only a small clean validation set, minutes of LoRA training, and no knowledge of the trigger can reduce attack success rates to single digits on many attacks while keeping clean-data accuracy within a few points of the original.
  • The method generalizes across diverse attack families, including input-triggered, model-editing, layerwise-poisoning, instruction-triggered, and multi-turn conversational attacks, so the defense is not tied to one trigger type.
  • The ablation implies the internal merge is the main line of defense and external evidence is a booster; a deployment could start with the merge and add evidence when the model is over-confident.
  • An adaptive attacker who anticipates weight merging and trains a conflict model to subtract it still fails to restore high attack success rates, so the defense has at least one layer of robustness beyond naivety.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that if clean and backdoor behaviors really are separable task vectors, the same merging recipe could be pointed at other unwanted behaviors, such as jailbreak compliance, biased outputs, or memorized harmful text, provided a small clean-behavior model can be trained.
  • A testable extension is to use the external-conflict probe as a detector: measure how much a model's answer changes when contradictory evidence is appended; models with large swings may be harboring parametric backdoors even before merging.
  • The paper does not report the interpolation weight $t$ used in Eq. 9; we infer that reproducibility across models and attacks depends on knowing how sensitive the cancellation is to $t$, and a published sweep of $t$ would settle that.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a trigger-agnostic backdoor defense for LLMs that combines an internal mechanism, where a LoRA-tuned 'conflict model' trained on a small clean dataset is merged into the backdoored model via a linear combination of weights (Eq. 9), with an external mechanism, where contradictory evidence is added to the prompt, with the evidence generated or modified by GPT-3.5. The method is evaluated on two classification datasets and one conversational dataset across GPT2-XL, GPT-J, LLaMA, and LLaMA-2, against 8 backdoor attacks and 8 baseline defenses, reporting attack success rate (ASR) and clean data accuracy (CDA). The headline claims are a reduction in ASR by up to 98% while maintaining over 90% CDA, and robustness against adaptive attacks.

Significance. If the claims hold, the method would be a practically attractive defense: it requires no trigger knowledge, uses only a small clean dataset, avoids full retraining, and is evaluated across a broad matrix of models, attacks, and baselines. The paper also includes ablations, a comparison of merging strategies, a clean-data-percentage study, and an adaptive-attack experiment; these are commendable and make the work easy to extend. However, the central empirical claim is currently not reproducible because the merge weight t in Eq. (9) is never reported, and the abstract's 'over 90% clean accuracy' statement is contradicted by the paper's own tables. Given that one table shows ordinary merging of clean and backdoored models preserves both behaviors, the defense's operating point appears to depend on t and on the conflict-model recipe; without a sensitivity analysis or a stated mechanism, the main claim is not yet established.

major comments (5)
  1. [§3.3, Eq. (9)] The interpolation parameter t in Eq. (9) is never reported anywhere in the paper, and no sensitivity study for t is provided. Since t controls how much of the conflict model enters the merged model, the ASR and CDA values in Tables 1–3 cannot be reproduced or interpreted: a value of t near 0 would leave the backdoor largely intact, while a value near 1 would collapse the model toward the clean conflict model. This is a load-bearing omission because Table 6 shows that ordinary linear merging of a clean model with a backdoored model preserves both behaviors (ASR 83%, CDA 92%), so the reported ASR suppression must be attributed either to a specific t or to the particular conflict-model training recipe; neither is documented.
  2. [Abstract and §1] The claim that the method maintains 'over 90% clean data accuracy' is contradicted by the paper's own results. In Table 1, the defended model achieves CDA values of 62.98% for GPT2-XL under Rome, 70.28% for GPT2-XL under MEMIT, and 62.64% for LLaMA under MEMIT; Table 2 also contains multiple CDA values below 90%. The accurate statement is that CDA is maintained in many cases and sometimes improved, not that it is uniformly above 90%. The abstract and Section 1 should be corrected to state the actual range, or the evaluation should be restricted to configurations where the claim holds.
  3. [§6.1, Table 6] The mechanism underlying the internal conflict is not established. In Experiment 1 of Table 6, merging a 100%-clean model with a 100%-backdoored model yields ASR 83% and CDA 92%, i.e., both behaviors survive; in Experiment 2 the merged model exhibits only backdoor behavior. The authors conclude from these experiments that the method can 'identify and eliminate hidden abilities,' but the evidence demonstrates interpolation between source behaviors rather than cancellation of the backdoor direction. The paper needs either a theoretical account of why a conflict model trained on a small clean split cancels backdoor directions while a clean model trained on the full clean set does not, or a parameter sweep over t showing a robust operating region, before the central defense claim can be accepted.
  4. [§5.1–§5.4] All results in Tables 1–5, 7, and 8 appear to be single runs with no error bars, standard deviations, or confidence intervals. Because many headline numbers are close to 0% ASR or differ by only a few percent in CDA, the absence of variance information makes it impossible to distinguish a robust effect from seed- or initialization-dependent behavior. At minimum, the authors should report means and standard deviations over multiple seeds for the main tables and state the number of runs used.
  5. [§6.3, Table 7] The adaptive-attack evaluation covers only the CBA attack on Emotion Corpora, and the adaptive attack is described in a single sentence ('train a conflict model and subtract it from the backdoored model') with no hyperparameters, training details, or validation protocol. The abstract's unconditional statement that the method is robust against adaptive backdoor attacks is therefore unsupported. The authors should fully specify the adaptive attack, evaluate it across more than one attack/dataset, and state the attacker's assumed knowledge, including whether the defender's clean-data split and merging procedure are known to the attacker.
minor comments (6)
  1. [§3.3] There is a typo in 'inlcude' in the paragraph on popular model merging algorithms, and Eq. (5) uses the symbol ⊙ for merging without defining it in that equation (it is later clarified as weight interpolation).
  2. [§5.1] The text states that 'our method reduces the ASR of CBA to 15.34% for GPT-XL on the SST-2 dataset,' but Table 1 reports an ASR of 1.26% for GPT2-XL under CBA on SST-2; the number 15.34% does not match any entry in Table 1 and should be corrected.
  3. [Table 3 and §4.2] The model is called 'GPT2-XL' in the setup and in Tables 1–2, but Table 3 uses 'GPT-XL'; the naming should be unified throughout.
  4. [Figure 3] The caption of Figure 3 contains 'Transferability(defense:PP eps=0.5; victim dataset:FashionMNSIT)', which appears unrelated to the Emotion Corpus and Chat-Backdoor experiments shown in the figure; this text should be removed or replaced with a correct caption.
  5. [Algorithm 1 and Eq. (15)] The TextRank pseudocode in Algorithm 1 does not specify the edge-weight initialization or the window size that the update rule in Eq. (15) depends on, making the algorithm description incomplete.
  6. [Table 9] In the DTBA row for GPT-J, the Fine-tuning column reports '1.31s' with an 's' appended, which appears to be a formatting error; the entry should be a time in hours like the other entries.

Circularity Check

1 steps flagged · score 6.0 of 10

The clean-accuracy claim is partially by construction: CDA (Eq. 17) is evaluated on Dc, the same clean set used to train the conflict model (Eq. 7), so the reported 'over 90% clean data accuracy' is not an independent measure; the ASR-reduction claim is not circular.

  1. fitted input called prediction [Section 3.3 (Eq. 7) and Section 4.4 (Eq. 17)]
    "We build a conflict model M̂θ by fine-tuning a pre-trained model Mθ0 using a small amount of clean data Dc. ... CDA = 1/|Dc| Σ_{(x,y)∈Dc} 1[M(x) ∈ Nϵ(y)] ... where Dc represents the clean dataset."

    The same symbol Dc denotes both the training set for the conflict model (Eq. 7: maximize over (x,y)∈Dc) and the evaluation set for CDA (Eq. 17: average over (x,y)∈Dc). No held-out clean test set is defined or mentioned for the main tables. Therefore the reported CDA of the merged model, including the abstract's 'maintaining over 90% clean data accuracy,' is computed, on its face, on the very clean samples used to fit the internal conflict model. The clean-utility half of the headline is thus partly true by construction: the merged model is interpolated toward a model fine-tuned on Dc, then scored on Dc. The ASR claims are not circular because the poisoned samples are not part of Dc.

full rationale

The paper's core defense is an empirical mechanism: train a clean LoRA model and merge it with the backdoored model, plus add external evidence. There is no formal derivation whose conclusion equals its premise, and the ASR reductions in Tables 1-5 are evaluated on poisoned inputs that were not used to train the conflict model, so those results are not circular. I do not score the unreported merge weight t in Eq. 9 as circularity: the paper does not state that t was tuned on the reported test metrics, so this is a reproducibility gap rather than a demonstrated fit-renamed-as-prediction. The one concrete circular step is the clean-data metric: the same symbol Dc is used both for training the conflict model and for computing CDA, with no explicit held-out clean split. Because the clean-accuracy claim is a load-bearing part of the abstract's headline claim, this is a partial circularity rather than a minor wording issue. There is no load-bearing self-citation: the authors' prior works appear only as attack/defense context, not as justification for the central mechanism.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper contributes an empirical method, not a derivation. The central result depends on shared base-model weights, separability of clean and backdoor task vectors, and unreported hyperparameters such as the merge weight t. No new physical or architectural entities are introduced.

free parameters (3)
  • model merging interpolation parameter t = not reported
    Eq. 9 defines theta_merge = t * theta_hat + (1 - t) * theta_tilde. The paper never states the default t or how it was chosen; the central ASR reductions could depend on this value.
  • clean data percentage = 10% (default)
    Chosen to balance cost and defense; Section 5.4 shows performance varies with this percentage, so it is a tunable design parameter.
  • LoRA configuration (rank, alpha, learning rate) = not stated
    The conflict model is LoRA-trained but no LoRA hyperparameters are reported; these affect what the conflict model learns and how the merge behaves.
assumptions (3)
  • domain assumption The defender has access to the exact base pre-trained model from which the backdoored model was produced.
    Eq. 9 and Section 3.3 assume the conflict model and backdoored model share the same parameter space so weights can be linearly merged.
  • ad hoc to paper Backdoor and clean behaviors are separable and can be canceled by weighted addition of task vectors.
    Table 6 Experiment 1 shows ordinary merging of two task-specific models retains both behaviors, while the paper's specific clean/backdoor mixture is claimed to cancel backdoors; this separability is an empirical assumption, not proven.
  • ad hoc to paper Information conflict, whether parametric or prompt-level, causes the model to abandon its backdoor output rather than become unusable.
    Section 2.4 motivates the method on knowledge-conflict findings from [66], but no formal or quantitative link is established between conflict and backdoor suppression.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Neutralizing Backdoors through Information Conflicts for Large Language Models." pith.science (2026). https://pith.science/paper/IRNBKJ76

@misc{pith2026241118280,
  author       = {Pith},
  title        = {Pith review of: Neutralizing Backdoors through Information Conflicts for Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRNBKJ76}},
  note         = {Machine review of arXiv:2411.18280}
}
read the original abstract

Large language models (LLMs) have seen significant advancements, achieving superior performance in various Natural Language Processing (NLP) tasks, from understanding to reasoning. However, they remain vulnerable to backdoor attacks, where models behave normally for standard queries but generate harmful responses or unintended output when specific triggers are activated. Existing backdoor defenses often suffer from drawbacks that they either focus on detection without removal, rely on rigid assumptions about trigger properties, or prove to be ineffective against advanced attacks like multi-trigger backdoors. In this paper, we present a novel method to eliminate backdoor behaviors from LLMs through the construction of information conflicts using both internal and external mechanisms. Internally, we leverage a lightweight dataset to train a conflict model, which is then merged with the backdoored model to neutralize malicious behaviors by embedding contradictory information within the model's parametric memory. Externally, we incorporate convincing contradictory evidence into the prompt to challenge the model's internal backdoor knowledge. Experimental results on classification and conversational tasks across 4 widely used LLMs demonstrate that our method outperforms 8 state-of-the-art backdoor defense baselines. We can reduce the attack success rate of advanced backdoor attacks by up to 98% while maintaining over 90% clean data accuracy. Furthermore, our method has proven to be robust against adaptive backdoor attacks. The code will be open-sourced upon publication.

Figures

Figures reproduced from arXiv: 2411.18280 by the authors.

Figure 1
Figure 1. The interaction between users and model providers in two [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our method: we eliminate backdoors in large language models (LLMs) by introducing two types of information conflicts: internal [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The performance of our method against CBA (on [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of CDA performance between ours and the external [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense Techniques

    cs.CR 2025-07 conditional novelty 4.0 of 10

    A survey that maps LLM applications, vulnerabilities, and defenses across eight cybersecurity domains, but with significant citation and rigor problems.

Reference graph

Works this paper leans on

82 extracted references · 34 canonical work pages · cited by 1 Pith paper

  1. [1]

    Llms’ reading comprehension is affected by parametric knowledge and struggles with hypothetical statements

    Victoria Basmov, Yoav Goldberg, and Reut Tsarfaty. Llms’ reading comprehension is affected by parametric knowledge and struggles with hypothetical statements. arXiv preprint arXiv:2404.06283, 2024

    arXiv 2024

  2. [2]

    Towards stealthy backdoor attacks against speech recognition via elements of sound

    Hanbo Cai, Pengcheng Zhang, Hai Dong, Yan Xiao, Stefanos Koffas, and Yiming Li. Towards stealthy backdoor attacks against speech recognition via elements of sound. IEEE Transactions on Information Forensics and Security, 2024

    2024

  3. [3]

    Badprompt: Backdoor attacks on continuous prompts

    Xiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang, et al. Badprompt: Backdoor attacks on continuous prompts. Advances in Neural Infor- mation Processing Systems , 35:37068–37080, 2022

    2022

  4. [4]

    Backdoor attacks and defenses for deep neural networks in outsourced cloud environments

    Yanjiao Chen, Xueluan Gong, Qian Wang, Xing Di, and Huayang Huang. Backdoor attacks and defenses for deep neural networks in outsourced cloud environments. IEEE Network, 34(5):141–147, 2020

    2020

  5. [5]

    Deep reinforcement learning from human preferences

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in Neural Information Processing Systems, 30, 2017

    2017

  6. [6]

    Triggerless backdoor at- tack for nlp tasks with clean labels

    Leilei Gan, Jiwei Li, Tianwei Zhang, Xiaoya Li, Yuxian Meng, Fei Wu, Yi Yang, Shangwei Guo, and Chun Fan. Triggerless backdoor at- tack for nlp tasks with clean labels. arXiv preprint arXiv:2111.07970, 2021

    arXiv 2021

  7. [7]

    Arcee’s mergekit: A toolkit for merging large language models

    Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vlad Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee’s mergekit: A toolkit for merging large language models. arXiv preprint arXiv:2403.13257 , 2024

    arXiv 2024

  8. [8]

    Atteq- nn: Attention-based qoe-aware evasive backdoor attacks

    Xueluan Gong, Yanjiao Chen, Jianshuo Dong, and Qian Wang. Atteq- nn: Attention-based qoe-aware evasive backdoor attacks. In Network and Distributed System Security , 2022

    work page 2022

Show all 82 references
  1. [9]

    Defense-resistant backdoor at- tacks against deep neural networks in outsourced cloud environment

    Xueluan Gong, Yanjiao Chen, Qian Wang, Huayang Huang, Lingshuo Meng, Chao Shen, and Qian Zhang. Defense-resistant backdoor at- tacks against deep neural networks in outsourced cloud environment. IEEE Journal on Selected Areas in Communications , 39(8):2617– 2631, 2021

    2021

  2. [10]

    Redeem myself: Purifying back- doors in deep learning models using self attention distillation

    Xueluan Gong, Yanjiao Chen, Wang Yang, Qian Wang, Yuzhe Gu, Huayang Huang, and Chao Shen. Redeem myself: Purifying back- doors in deep learning models using self attention distillation. In IEEE Symposium on Security and Privacy , pages 755–772, 2023

    2023

  3. [11]

    Palette: Physically-realizable backdoor attacks against video recognition models

    Xueluan Gong, Zheng Fang, Bowen Li, Tao Wang, Yanjiao Chen, and Qian Wang. Palette: Physically-realizable backdoor attacks against video recognition models. IEEE Transactions on Dependable and Secure Computing, 21(04):2672–2685, 2024

    2024

  4. [12]

    Exploring backdoor vulnerabilities of chat models

    Yunzhuo Hao, Wenkai Yang, and Yankai Lin. Exploring backdoor vulnerabilities of chat models. arXiv preprint arXiv:2404.02406 , 2024

    2024 arXiv

  5. [13]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  6. [14]

    Composite backdoor attacks against large language models

    Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, and Yang Zhang. Composite backdoor attacks against large language models. arXiv preprint arXiv:2310.07676 , 2023

    2023 arXiv

  7. [15]

    Sleeper agents: Training decep- tive LLMs that persist through safety training

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training decep- tive LLMs that persist through safety training. arXiv preprint arXiv:2401.05566, 2024

    2024 arXiv

  8. [16]

    Editing models with task arithmetic

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022

    2022 arXiv

  9. [17]

    Model-reuse attacks on deep learning systems

    Yujie Ji, Xinyang Zhang, Shouling Ji, Xiapu Luo, and Ting Wang. Model-reuse attacks on deep learning systems. In SIGSAC Conference on Computer and Communications Security , pages 349–363. ACM, 2018

    2018

  10. [18]

    Backdoor attacks against learning systems

    Yujie Ji, Xinyang Zhang, and Ting Wang. Backdoor attacks against learning systems. In Conference on Communications and Network Security, pages 1–9. IEEE, 2017

    2017

  11. [19]

    Chatgpt for good? On opportunities and challenges of large language models for education

    Enkelejda Kasneci, Kathrin Seßler, Stefan Küchemann, Maria Ban- nert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, et al. Chatgpt for good? On opportunities and challenges of large language models for education. Learning and In...

    2023

  12. [20]

    Textual backdoor attack for the text classification system

    Hyun Kwon and Sanghyun Lee. Textual backdoor attack for the text classification system. Security and Communication Networks , 2021(1):2938386, 2021

    2021

  13. [21]

    Fast inference from transformers via speculative decoding

    Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Confer- ence on Machine Learning , pages 19274–19286. PMLR, 2023

    2023

  14. [22]

    Backdoor removal for generative large language models

    Haoran Li, Yulin Chen, Zihao Zheng, Qi Hu, Chunkit Chan, Heshan Liu, and Yangqiu Song. Backdoor removal for generative large language models. arXiv preprint arXiv:2405.07667 , 2024

    2024 arXiv

  15. [23]

    Backdoor attacks on pre-trained models by layerwise weight poisoning

    Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu. Backdoor attacks on pre-trained models by layerwise weight poisoning. arXiv preprint arXiv:2108.13888 , 2021

    2021 arXiv

  16. [24]

    Chain- of-scrutiny: Detecting backdoor attacks for large language models

    Xi Li, Yusen Zhang, Renze Lou, Chen Wu, and Jiaqi Wang. Chain- of-scrutiny: Detecting backdoor attacks for large language models. arXiv preprint arXiv:2406.05948 , 2024

    2024

  17. [25]

    Badedit: Back- dooring large language models by model editing

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. Badedit: Back- dooring large language models by model editing. arXiv preprint arXiv:2403.13355, 2024

    2024 arXiv

  18. [26]

    Multi-target backdoor attacks for code pre- trained models

    Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. Multi-target backdoor attacks for code pre- trained models. arXiv preprint arXiv:2306.08350 , 2023

    2023 arXiv

  19. [27]

    Neural attention distillation: Erasing backdoor triggers from deep neural networks

    Yige Li, Nodens Koren, Lingjuan Lyu, Xixiang Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. In International Conference on Learning Representations. OpenReview.net, 2021

    2021

  20. [28]

    Neural attention distillation: Erasing backdoor triggers from deep neural networks

    Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, and Xingjun Ma. Neural attention distillation: Erasing backdoor triggers from deep neural networks. arXiv preprint arXiv:2101.05930 , 2021

    2021 arXiv

  21. [29]

    Rethinking the trigger of backdoor attack

    Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.04692, 2020

    2004 arXiv

  22. [30]

    Clean- gen: Mitigating backdoor attacks for generation tasks in large lan- guage models

    Yuetai Li, Zhangchen Xu, Fengqing Jiang, Luyao Niu, Dinuka Saha- bandu, Bhaskar Ramasubramanian, and Radha Poovendran. Clean- gen: Mitigating backdoor attacks for generation tasks in large lan- guage models. arXiv preprint arXiv:2406.12257 , 2024

    2024 arXiv

  23. [31]

    Unveiling the pitfalls of knowledge editing for large language models

    Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. Unveiling the pitfalls of knowledge editing for large language models. arXiv preprint arXiv:2310.02129 , 2023

    2023 arXiv

  24. [32]

    Composite backdoor attack for deep neural network by mixing existing benign features

    Junyu Lin, Lei Xu, Yingqi Liu, and Xiangyu Zhang. Composite backdoor attack for deep neural network by mixing existing benign features. In ACM SIGSAC Conference on Computer and Communi- cations Security, pages 113–131, 2020

    2020

  25. [33]

    Fine-pruning: Defending against backdooring attacks on deep neural networks

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-pruning: Defending against backdooring attacks on deep neural networks. In International Symposium on Research in Attacks, Intrusions, and Defenses, pages 273–294. Springer, 2018

    2018

  26. [34]

    Oppor- tunistic backdoor attacks: Exploring human-imperceptible vulnerabil- ities on speech recognition systems

    Qiang Liu, Tongqing Zhou, Zhiping Cai, and Yonghao Tang. Oppor- tunistic backdoor attacks: Exploring human-imperceptible vulnerabil- ities on speech recognition systems. In ACM International Conference on Multimedia, pages 2390–2398, 2022

    2022

  27. [35]

    Trojaning attack on neural networks

    Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. Trojaning attack on neural networks. In Annual Network and Distributed System Security Sym- posium. The Internet Society, 2018

    2018

  28. [36]

    Practical backdoor attack against speaker recognition system

    Yuxiao Luo, Jianwei Tai, Xiaoqi Jia, and Shengzhi Zhang. Practical backdoor attack against speaker recognition system. In International Conference on Information Security Practice and Experience , pages 468–484. Springer, 2022

    2022

  29. [37]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems , 35:17359–17372, 2022

    2022

  30. [38]

    Mass-editing memory in a transformer

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229, 2022

    2022 arXiv

  31. [39]

    Textrank: Bringing order into text

    Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Conference on Empirical Methods in Natural Language Processing , pages 404–411, 2004

    2004

  32. [40]

    Training language models to follow in- structions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wain- wright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow in- structions with human feedback. Advances in Neural Information Processing Systems, 35:...

    2022

  33. [41]

    Hidden trigger backdoor attack on NLP models via linguistic style manipulation

    Xudong Pan, Mi Zhang, Beina Sheng, Jiaming Zhu, and Min Yang. Hidden trigger backdoor attack on NLP models via linguistic style manipulation. In USENIX Security Symposium , pages 3611–3628, 2022

    2022

  34. [42]

    Onion: A simple and effective defense against textual backdoor attacks

    Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369 , 2020

    2011 arXiv

  35. [43]

    Hidden killer: Invisi- ble textual backdoor attacks with syntactic trigger

    Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. Hidden killer: Invisi- ble textual backdoor attacks with syntactic trigger. arXiv preprint arXiv:2105.12400, 2021

    2021 arXiv

  36. [44]

    Towards a proactive ML approach for detecting backdoor poison samples

    Xiangyu Qi, Tinghao Xie, Jiachen T Wang, Tong Wu, Saeed Mahlou- jifar, and Prateek Mittal. Towards a proactive ML approach for detecting backdoor poison samples. In USENIX Security Symposium, pages 1685–1702, 2023

    2023

  37. [45]

    Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    2023 arXiv

  38. [46]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  39. [47]

    Identifying physically realizable triggers for backdoored face recognition networks

    Ankita Raj, Ambar Pal, and Chetan Arora. Identifying physically realizable triggers for backdoored face recognition networks. In IEEE International Conference on Image Processing , pages 3023–3027, 2021

    2021

  40. [48]

    Competition report: Finding universal jailbreak backdoors in aligned LLMs

    Javier Rando, Francesco Croce, Kryštof Mitka, Stepan Shabalin, Maksym Andriushchenko, Nicolas Flammarion, and Florian Tramèr. Competition report: Finding universal jailbreak backdoors in aligned LLMs. arXiv preprint arXiv:2404.14461 , 2024

    2024 arXiv

  41. [49]

    Hidden trigger backdoor attacks

    Aniruddha Saha, Akshayvarun Subramanya, and Hamed Pirsiavash. Hidden trigger backdoor attacks. In AAAI Conference on Artificial Intelligence, pages 11957–11965. AAAI Press, 2020

    2020

  42. [50]

    Dynamic backdoor attacks against machine learning models

    Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, and Yang Zhang. Dynamic backdoor attacks against machine learning models. arXiv preprint arXiv:2003.03675 , 2020

    2003 arXiv

  43. [51]

    Carer: Contextualized affect representations for emotion recognition

    Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. Carer: Contextualized affect representations for emotion recognition. In Conference on Empirical Methods in Natural Language Processing, pages 3687–3697, 2018

    2018

  44. [52]

    You autocomplete me: Poisoning vulnerabilities in neural code com- pletion

    Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov. You autocomplete me: Poisoning vulnerabilities in neural code com- pletion. In USENIX Security Symposium , pages 1559–1575, 2021

    2021

  45. [53]

    On the exploitability of instruction tun- ing

    Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, and Tom Goldstein. On the exploitability of instruction tun- ing. Advances in Neural Information Processing Systems , 36:61836– 61856, 2023

    2023

  46. [54]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christo- pher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Process- ing, pages 1631–1642, 2013

    2013

  47. [55]

    Natural backdoor attack on text data

    Lichao Sun. Natural backdoor attack on text data. arXiv preprint arXiv:2006.16176, 2020

    2006 arXiv

  48. [56]

    A simple and effective pruning approach for large language models

    Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695, 2023

    2023 arXiv

  49. [57]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  50. [58]

    Llama 2: Open foundation and fine-tuned chat models

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Pra- jjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 , 2023

    2023 arXiv

  51. [59]

    Neural cleanse: Identi- fying and mitigating backdoor attacks in neural networks

    Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, Bimal Viswanath, Haitao Zheng, and Ben Y Zhao. Neural cleanse: Identi- fying and mitigating backdoor attacks in neural networks. In IEEE Symposium on Security and Privacy , pages 707–723, 2019

    2019

  52. [60]

    Adver- sarial demonstration attacks on large language models

    Jiongxiao Wang, Zichen Liu, Keun Hee Park, Zhuojun Jiang, Zhao- heng Zheng, Zhuofeng Wu, Muhao Chen, and Chaowei Xiao. Adver- sarial demonstration attacks on large language models. arXiv preprint arXiv:2305.14950, 2023

    2023 arXiv

  53. [61]

    Backdoor attacks against transfer learning with pre-trained deep learning models

    Shuo Wang, Surya Nepal, Carsten Rudolph, Marthie Grobler, Shangyu Chen, and Tianle Chen. Backdoor attacks against transfer learning with pre-trained deep learning models. IEEE Transactions on Services Computing , 2020

    2020

  54. [62]

    Finetuned language models are zero-shot learners

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021

    2021 arXiv

  55. [63]

    Emergent abilities of large language models

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 , 2022

    2022 arXiv

  56. [64]

    Bd- mmt: Backdoor sample detection for language models through model mutation testing

    Jiali Wei, Ming Fan, Wenjing Jiao, Wuxia Jin, and Ting Liu. Bd- mmt: Backdoor sample detection for language models through model mutation testing. IEEE Transactions on Information Forensics and Security, 2024

    2024

  57. [65]

    Model soups: Averaging weights of multiple fine-tuned models improves ac- curacy without increasing inference time

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Re- becca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: Averaging weights of multiple fine-tuned models improves ac- curacy without increasing ...

    2022

  58. [66]

    Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts

    Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts. arXiv preprint arXiv:2305.13300, 2023

    2023 arXiv

  59. [67]

    Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models

    Jiashu Xu, Mingyu Derek Ma, Fei Wang, Chaowei Xiao, and Muhao Chen. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. arXiv preprint arXiv:2305.14710, 2023

    2023 arXiv

  60. [68]

    Trojllm: A black-box trojan prompt attack on large language models

    Jiaqi Xue, Mengxin Zheng, Ting Hua, Yilin Shen, Yepeng Liu, Ladislau Bölöni, and Qian Lou. Trojllm: A black-box trojan prompt attack on large language models. Advances in Neural Information Processing Systems, 36, 2024

    2024

  61. [69]

    TIES-merging: Resolving interference when merging models

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. TIES-merging: Resolving interference when merging models. In Conference on Neural Information Processing Systems , 2023

    2023

  62. [70]

    Backdooring instruction-tuned large language models with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large language models with virtual prompt injection. In Conference of the North American Chapter of the Association for Computation...

    2024

  63. [71]

    A comprehensive overview of backdoor attacks in large language models within communication networks

    Haomiao Yang, Kunlan Xiang, Mengyu Ge, Hongwei Li, Rongxing Lu, and Shui Yu. A comprehensive overview of backdoor attacks in large language models within communication networks. IEEE Network, 2024

    2024

  64. [72]

    Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models

    Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. arXiv preprint arXiv:2103.15543, 2021

    2021 arXiv

  65. [73]

    Rap: Robustness-aware perturbations for defending against backdoor at- tacks on NLP models

    Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, and Xu Sun. Rap: Robustness-aware perturbations for defending against backdoor at- tacks on NLP models. arXiv preprint arXiv:2110.07831 , 2021

    2021 arXiv

  66. [74]

    Poisonprompt: Backdoor attack on prompt-based large language models

    Hongwei Yao, Jian Lou, and Zhan Qin. Poisonprompt: Backdoor attack on prompt-based large language models. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7745– 7749, 2024

    2024

  67. [75]

    Latent backdoor attacks on deep neural networks

    Yuanshun Yao, Huiying Li, Haitao Zheng, and Ben Y Zhao. Latent backdoor attacks on deep neural networks. In ACM SIGSAC Confer- ence on Computer and Communications Security , pages 2041–2055, 2019

    2019

  68. [76]

    Beear: Embedding-based adversarial removal of safety backdoors in instruction-tuned language models

    Yi Zeng, Weiyu Sun, Tran Ngoc Huynh, Dawn Song, Bo Li, and Ruoxi Jia. Beear: Embedding-based adversarial removal of safety backdoors in instruction-tuned language models. arXiv preprint arXiv:2406.17092, 2024

    2024 arXiv

  69. [77]

    Composing parameter- efficient modules with arithmetic operation

    Jinghan Zhang, Junteng Liu, Junxian He, et al. Composing parameter- efficient modules with arithmetic operation. Advances in Neural Information Processing Systems , 36:12589–12610, 2023

    2023

  70. [78]

    Fine-mixing: Mitigating backdoors in fine-tuned language models

    Zhiyuan Zhang, Lingjuan Lyu, Xingjun Ma, Chenguang Wang, and Xu Sun. Fine-mixing: Mitigating backdoors in fine-tuned language models. arXiv preprint arXiv:2210.09545 , 2022

    2022 arXiv

  71. [79]

    Universal vulnerabilities in large language models: Backdoor attacks for in-context learning

    Shuai Zhao, Meihuizi Jia, Luu Anh Tuan, Fengjun Pan, and Jinming Wen. Universal vulnerabilities in large language models: Backdoor attacks for in-context learning. arXiv preprint arXiv:2401.05949 , 2024

    2024 arXiv

  72. [80]

    Prompt as triggers for backdoor attack: Examining the vulnerability in language models

    Shuai Zhao, Jinming Wen, Luu Anh Tuan, Junbo Zhao, and Jie Fu. Prompt as triggers for backdoor attack: Examining the vulnerability in language models. arXiv preprint arXiv:2305.01219 , 2023

    2023 arXiv

  73. [81]

    A survey of large language models

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023. Appendix A. More Details on Experiment Setup A.1 Target Models GPT...

    2023 arXiv

  74. [82]

    I MPACT OF DIFFERENT MODEL MERGING METHODS

    https://github.com/kingoflolz/mesh-transformer-jax TABLE 8. I MPACT OF DIFFERENT MODEL MERGING METHODS . Dataset Attack Metrics Linear Tie Slerp Passthrough Emotion CBA ASR 7.96% 2.45% 0.81% 0.19% CDA 91.85% 90.11% 92.27% 94.25% BadEdit ASR 0.90% 22.51% 0% 8.57% CDA 88.70% 52....

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.