Pith. sign in

REVIEW 5 major objections 5 minor 45 references

When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read This paper claims that backdoored LLMs produce coherent explanations on clean inputs but diverse, logically flawed explanations on poisoned inputs, and that this difference is a usable backdoor signal.

desk verdict New angle on backdoor detection via explanation quality, but the current draft has a detector leak and an internal contradiction that need fixing before the claims hold. read the letter →

arxiv 2411.12701 v3 pith:3PGRP2FF submitted 2024-11-19 cs.CR cs.AI

classification cs.CRcs.AI
keywords backdoorattackslargelanguagemodelsnaturalexplanationsexplanationqualitytunedlensattentionanalysisdetectioninterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks what a backdoored large language model reveals when it explains its own predictions in natural language. It claims that when the backdoor trigger is present, explanations become measurably worse: they are more diverse across repeated samples, lower in clarity and coherence, and often cite the trigger itself as the reason. The same pattern appears across sentiment classification and jailbreak-style generation tasks, and across word-level, sentence-level, and syntactic triggers. If the claim is right, explanation quality is a practical, trigger-agnostic signal for spotting backdoors in models that otherwise behave normally on clean inputs. The paper additionally traces the internal signature of this degradation: the label's meaning surfaces only in the last transformer layers, and attention shifts away from the input context toward the model's own newly generated tokens.

What carries the argument

The argument is carried by two measurement tools plus a quality rubric. The tuned lens projects each transformer layer's hidden state into the vocabulary space, letting the paper watch when a label token's meaning appears; the paper defines a Mean Emergence Depth (MED) that averages the layer index weighted by the token's probability over the final ten layers. The lookback-style attention metric, called contextual reliance, compares attention paid to the original input tokens versus attention paid to the model's own newly generated tokens, aggregated over the last layer and all heads. Explanation quality is scored by GPT-4o on five dimensions, and consistency is measured by Jaccard and semantic-textual similarity across five temperature-1 generations. Together these tools turn 'the explanation sounds off' into a comparable, statistical difference between clean and poisoned inputs.

What would settle it

Train a model with the same architecture, data, and fine-tuning recipe but with no poisoned samples, then collect explanations for inputs the clean model misclassifies naturally. If those naturally wrong predictions produce explanations as low-quality, as diverse, and with as little input-lookback as the backdoored model's poisoned-input explanations, then the signal tracks wrongness generally rather than backdoors specifically. If instead the clean model's wrong-prediction explanations remain coherent, the explanation-degradation signature is specific to the backdoor.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is the explanation-degradation signature of a fired backdoor. When a backdoored LLM is given a triggered input and asked to justify its prediction, the resulting explanation scores roughly half as high on clarity, relevance, coherence, completeness, and conciseness as explanations for clean inputs, and repeated generations at the same temperature are substantially less consistent. In about 17 percent of poisoned cases the explanation names the trigger word itself as the cause, producing statements like 'positive because ## is a positive word.' The paper shows this signature is not limited to one setup: it holds for two text-classification datasets and a jailbreak-generation dataset, for multiple trigger types, and for two model families. Internally, the paper reports that for poisoned inputs the predicted token's semantic content emerges only in the final few transformer layers, and that during explanation generation the model attends more to previously generated tokens and less to the original input, quantified by a lower lookback ratio.

Load-bearing premise

The load-bearing premise is that the explanation degradation seen on poisoned inputs is caused by the backdoor mechanism itself, rather than by the mere fact that the model is producing an incorrect prediction; the clean-model comparison in Appendix D does not fully rule out the possibility that any wrong prediction would produce equally poor explanations.

Editorial extensions

If this is right

  • A five-shot GPT-4o prompt that reads raw explanations distinguishes clean from poisoned inputs with about 97.5 percent accuracy, and simple classifiers using the last token's maximum-probability trajectory reach about 98 percent, so explanation features are directly usable for backdoor detection.
  • The degradation pattern is consistent across word-level, sentence-level, and syntactic triggers and across sentiment classification and generation tasks, which means a single explanation-quality check may catch backdoors that use very different trigger designs.
  • The tuned-lens and lookback metrics expose where the backdoor acts inside the model, so detection no longer requires knowing the trigger or having access to poisoned training data.
  • The explanation-based detector transfers to a different dataset and to a different trigger with 82 to 96.5 percent accuracy, suggesting the signal is not overfit to a particular attack recipe.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If low-quality, trigger-ignoring explanations are a general symptom of shortcut-driven predictions, the same explanation-quality signal could flag models relying on spurious correlations or dataset artifacts, not only planted backdoors—an extension the paper does not test.
  • The finding that poisoned explanations attend mainly to previously generated tokens suggests a testable link to faithfulness: models that ignore their input context while explaining may also be less trustworthy in high-stakes uses, so explanation monitoring could double as a runtime assurance check.
  • Because the label's meaning emerges only in the final layers for poisoned inputs, a natural next experiment is to intervene there—editing or ablating the last few layers' representations—and measure whether the backdoor behavior and the explanation degradation disappear together while clean behavior is preserved.
  • A deployment-ready detector would need a per-model threshold and a calibration procedure; the paper demonstrates the discriminative signal but does not yet provide an operating-point guide for using it in production.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes using natural-language explanations generated by backdoored LLMs as a lens for understanding and detecting backdoor attacks. After fine-tuning LLaMA-3-8B and DeepSeek-7B on poisoned data with several trigger types (word, sentence, syntactic, and jailbreak) across SST-2, Twitter Emotion, and AdvBench, the authors generate explanations for clean and poisoned inputs and evaluate them with GPT-4o and human raters. They report that poisoned-input explanations are lower in quality and more diverse, that the predicted token's semantic content emerges at different depths for clean versus poisoned inputs, that attention shifts away from the input context during poisoned explanation generation, and that explanation features yield high-accuracy backdoor detectors (GPT-4o five-shot and classical classifiers). The paper's central claim is that explanation degradation is a backdoor-specific artifact and can serve as a reliable indicator of backdoor presence.

Significance. If established, the central claim would offer a practical, model-agnostic backdoor indicator and a mechanistic window into backdoor behavior. The paper has several strengths: it pairs automated scoring with human raters, covers multiple datasets and trigger types, and reports a downstream detection attempt. These strengths make the hypothesis worth pursuing. However, the current evidence has load-bearing gaps: the token-level narrative contradicts the quantitative MED result, the GPT-4o detector is confounded by trigger string leakage in the prompt, the only direct clean-model control is underspecified and shows residual degradation even without a backdoor, and the classical classifiers lack a documented train/test split. These issues prevent the paper from supporting its strong conclusions in its present form.

major comments (5)
  1. [Section 5.2, Finding 2 vs. Section 1 and Abstract] The paper contains a direct internal contradiction about the direction of semantic emergence. The abstract and Section 1 state that for poisoned samples the predicted token's semantic meaning appears only in the final few transformer layers, while for clean samples it appears much earlier. Finding 2 (Section 5.2) reports that the Mean Emergence Depth (MED) of clean samples is significantly higher than that of poisoned samples (p = 5.42e-10). Since MED is defined in Eq. (3) as a layer-index-weighted sum, a higher MED means probability mass is concentrated in later layers. Thus the reported quantitative result shows clean inputs, not poisoned inputs, exhibit later semantic emergence. The abstract/introduction claim and the main finding cannot both be correct as stated. This needs to be resolved by either correcting the narrative or re-examining the metric's definition and interpretation.
  2. [Section 6 and Appendix I (Backdoor Detector Prompt)] The GPT-4o detector evaluation is confounded by the surface trigger string. The five-shot prompt in Appendix I (Figure 17) includes the token 'random' in all three 'Backdoored Model' examples and in none of the 'Clean Model' examples. Because poisoned explanations frequently contain the trigger token (as shown in Table 5, e.g., 'the first adjective used is random'), the classifier can classify based on the presence of 'random' rather than on explanation quality. The claim that this detector 'relied on explanation quality as a distinguishing feature' is therefore not supported. A control experiment with the trigger string removed or masked from the explanations, or with trigger-free poisoned samples, is needed to establish that the detector uses quality rather than a lexical trigger.
  3. [Appendix D, Table 10] The control experiment is under-specified and does not resolve the wrongness confound. The 'Clean Model' in Table 10 is never described: no architecture, initialization, training data, fine-tuning recipe, or confirmation that it is the same base model trained on the same clean samples without poisoned data. More importantly, Table 10 itself shows that the clean model's poisoned-input explanations are rated lower than its clean-input explanations (overall 3.30 vs. 3.82). This residual gap indicates that explanation quality can degrade even without a backdoor, possibly because the model is predicting an incorrect label. To support the central claim that degradation is backdoor-specific, the authors need a matched non-backdoored model that is also evaluated on inputs that lead to incorrect predictions in the absence of a trigger, as well as on trigger-added inputs that do not flip the label.
  4. [Section 6, Table 4] No train/test split, cross-validation strategy, or data-separation protocol is reported for the classical ML classifiers (logistic regression, SVM, decision tree, random forests). The 100 clean and 100 poisoned samples may have been used both to fit and to evaluate the classifiers, which would inflate the reported accuracies. The paper must specify how the features were split into training and evaluation sets, and ideally use k-fold cross-validation or a held-out set generated from different inputs.
  5. [Section 4 and Appendix E.2 (Consistency Results)] The claim of consistency 'across classification and generation tasks' is too strong given the reported statistics. Section 4 states the consistency difference is significant for all classification models, but the generation task (AdvBench) has a non-significant STS p-value of 0.951 in Table 11, while the Jaccard p-value is 0.0347. The abstract's wording 'a pattern consistent across classification and generation tasks' should be qualified to reflect that only quality (not consistency) is consistently significant in the generation setting.
minor comments (5)
  1. [Section 5.1, Eq. (3)] The definition of P_i(ttarget) is ambiguous: 'the probability assigned to the vocabulary item with the highest probability for the target token' is circular. Please clarify whether P_i is the probability of the target token itself or the maximum probability over the vocabulary at layer i.
  2. [Table 1 caption] The caption says the table includes 'training steps, learning rate, accuracy, and attack success rate,' but the table body only has ACC and ASR columns. Either add the missing columns or revise the caption to refer readers to Appendix A.1 for hyperparameters.
  3. [Abstract and Section 2 terminology] The paper alternates between 'ChatGPT-4o' and 'GPT-4o.' Please standardize the terminology to 'GPT-4o' throughout.
  4. [Appendix B] The human evaluation reports '100 explanations' but does not specify how many are clean and how many are poisoned, nor whether the raters saw all five variations. Please state the sample size per condition and the inter-rater agreement.
  5. [Throughout] Minor typographical errors should be corrected, including 'accessment' (Section 4), 'deitals' (Section 4), 'conlcusion' (Section 4), and 'Evluation' (Appendix E.1).

Circularity Check

2 steps flagged · score 6.0 of 10

The explanation-quality observations are empirical, but the backdoor-detector evaluation reduces to trigger-token detection and possibly in-sample fitting, making the detection claim partially circular.

  1. other [Section 6, five-shot GPT-4o detector prompt (Figure 17)]
    "Example 2: "for starters, the story is just too slim. random ..." Label: Backdoored Model. Example 4: "the lower your expectations ... random ..." Label: Backdoored Model. Example 5: "unfortunately ... random ..." Label: Backdoored Model."

    The five-shot examples are constructed so that every explanation labeled 'Backdoored Model' contains the literal trigger token 'random', while every 'Clean Model' example does not. The detector therefore can achieve the reported 97.5% accuracy by matching the trigger string rather than by evaluating explanation quality, the feature the paper claims to use. The poisoned/backdoored class is defined by trigger presence, and the trigger presence is fully confounded with the label in the prompt, so the evaluation of the detector does not test the explanation-degradation hypothesis. The word-level transfer result (Appendix K) is equally confounded; only the sentence-level transfer is not, but the headline result is not controlled.

  2. fitted input called prediction [Section 6, text preceding Table 4]
    "we used the maximum probability of the last token across all layers as input features for traditional machine learning classifiers, including logistic regression, support vector machines, and random forests, to further distinguish explanations from clean and poisoned inputs. ... As shown in Table 4, explanation-based features effectively differentiate between clean and poisoned inputs."

    No train/test split, cross-validation, or held-out set is reported. The classifiers are fit to the same 100 clean and 100 poisoned samples whose features are then reported as detection accuracy (e.g., 98.8% for logistic regression). An accuracy computed on the fitting samples measures in-sample discrimination, not out-of-sample prediction; presenting it as evidence that the features 'effectively differentiate' is the fitted-input-called-prediction pattern. The paper provides no evidence that these classifiers generalize beyond the samples used to fit them.

full rationale

The core empirical observations—lower explanation quality, higher diversity, later semantic emergence, and reduced lookback on poisoned inputs—are measurements reported with statistics and are not circular by construction. The MED and contextual-reliance metrics are defined from the model's own hidden states and attention, not from the labels being predicted. The main circularity is in the evaluation of the explanation-based detector. The five-shot prompt is contaminated with the trigger token in every backdoored example, so GPT-4o can shortcut to trigger detection; the ML classifiers are reported without a train/test split, so their accuracies may be in-sample fits. These two issues make the detector 'prediction' reduce, in part, to the construction of the evaluation, while the central quality and mechanism findings retain independent empirical content. Appendix D's unspecified clean-model control and the residual degradation it shows (clean model overall 3.30 on poisoned vs 3.82 on clean) are a confound regarding whether the degradation is backdoor-specific or a generic property of wrong predictions, but that is an external-validity problem rather than circularity, so it does not further raise the score. Overall: partial circularity in the detector evaluation, not in the underlying explanation analyses; score 6.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claims rely on the model's own explanations, attention, and probabilities; no constants are fitted to produce the main observed difference. The main burden is interpretational: explanations and attention are assumed to reveal decision mechanisms, and the GPT-4o judge is assumed to be a reliable quality oracle. The detector adds a weak assumption that the five-shot prompt does not leak trigger identity.

free parameters (2)
  • MED window n = 10
    Equation (3) limits emergence analysis to the last 10 of 32 LLaMA layers; MED values and the reported t-test depend on this hand-chosen window.
  • Explanation sampling count = 5
    Five explanations per input at temperature 1 are generated in Section 4; Jaccard and STS consistency scores depend on this count.
assumptions (6)
  • domain assumption Explanations generated by the model are treated as a window into its decision process.
    Sections 4 and 5 interpret explanation text and token probabilities as evidence about the backdoor mechanism rather than as post-hoc confabulation.
  • domain assumption Attention weights are a faithful proxy for the model's contextual reliance.
    Section 5.3 defines contextual reliance via CR(l,h,t) from attention maps; this assumes attention reflects the information actually used.
  • domain assumption GPT-4o quality scores are a valid measure of explanation quality.
    Section 4 and Appendix B compare to two human raters, but the automated scores used in Table 3 are from GPT-4o without uncertainty bars.
  • domain assumption The backdoor training recipe produces representative backdoored LLMs.
    Appendix A.1 lists 100-750 training steps with small poisoned sets; the findings are assumed to generalize beyond these specific models.
  • ad hoc to paper The five-shot GPT-4o detector distinguishes explanations on quality, not on surface trigger.
    Appendix I's prompt includes 'random' in every backdoored example, so the detector may be learning trigger presence instead of explanation irrationality.
  • domain assumption The tuned lens affine maps A_l and b_l trained on the same model provide unbiased layer-wise probabilities.
    Section 5.1 adopts the tuned lens from Belrose et al. 2023; the affine parameters are fit per layer and are assumed not to distort the emergence comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations." pith.science (2026). https://pith.science/paper/3PGRP2FF

@misc{pith2026241112701,
  author       = {Pith},
  title        = {Pith review of: When Backdoors Speak: Understanding LLM Backdoor Attacks Through Model-Generated Explanations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3PGRP2FF}},
  note         = {Machine review of arXiv:2411.12701}
}
read the original abstract

Large Language Models (LLMs) are known to be vulnerable to backdoor attacks, where triggers embedded in poisoned samples can maliciously alter LLMs' behaviors. In this paper, we move beyond attacking LLMs and instead examine backdoor attacks through the novel lens of natural language explanations. Specifically, we leverage LLMs' generative capabilities to produce human-readable explanations for their decisions, enabling direct comparisons between explanations for clean and poisoned samples. Our results show that backdoored models produce coherent explanations for clean inputs but diverse and logically flawed explanations for poisoned data, a pattern consistent across classification and generation tasks for different backdoor attacks. Further analysis reveals key insights into the explanation generation process. At the token level, explanation tokens associated with poisoned samples only appear in the final few transformer layers. At the sentence level, attention dynamics indicate that poisoned inputs shift attention away from the original input context during explanation generation. These findings enhance our understanding of backdoor mechanisms in LLMs and present a promising framework for detecting vulnerabilities through explainability.

Figures

Figures reproduced from arXiv: 2411.12701 by the authors.

Figure 1
Figure 1. This figure shows the attention map of the last [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of explanation generation and analysis. First, we use a prompt to instruct the backdoored LLM [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Comparison of overall quality scores for ex [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Comparison of explanation consistency based [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Average maximum probability of the last token in explanations across different layers [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Prediction trajectories (max probability) of ex [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Mean emergence depth, lookback ratio, and [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Average lookback ratio for clean and poisoned [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Attention maps from four heads in the last layer, illustrating the generated explanations for both a clean and [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: The prediction trajectories of example clean [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: The prediction trajectories of example poi [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Attention on new tokens heatmap of an example clean input (left) and poisoned input (right). [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: Attention on context heatmap of an example clean input (left) and poisoned input (right). [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Total variation distance between example [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Lookback ratio heatmap of an example clean input (left) and poisoned input (right). [PITH_FULL_IMAGE:figures/full_fig_p015_15.png]
Figure 16
Figure 16. Figure 16: Prompt for Quality Analysis Setup Dataset Trigger Accuracy Base SST-2 word-level 97.5% Transferred Twitter Emotion word-level 82% Transferred SST-2 sentence-level 96.5% [PITH_FULL_IMAGE:figures/full_fig_p018_16.png]
Figure 17
Figure 17. Figure 17: Prompt for Backdoor Detector 19 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 15 canonical work pages

  1. [1]

    Guillaume Alain. 2016. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644

  2. [2]

    Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. 2023. https://arxiv.org/abs/2303.08112 Eliciting latent predictions from transformers with the tuned lens . Preprint, arXiv:2303.08112

  3. [3]

    Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html

  4. [4]

    Oana-Maria Camburu, Tim Rockt \"a schel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-snli: Natural language inference with natural language explanations. Advances in Neural Information Processing Systems, 31

  5. [5]

    Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. 2021. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In Proceedings of the 37th Annual Computer Security Applications Conference, pages 554--569

  6. [6]

    Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, and James Glass. 2024. https://arxiv.org/abs/2407.07071 Lookback lens: Detecting and mitigating contextual hallucinations in large language models using only attention maps . Preprint, arXiv:2407.07071

  7. [7]

    Jiazhu Dai, Chuanshuai Chen, and Yufeng Li. 2019. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872--138878

  8. [8]

    DeepSeek-AI, :, Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, Huazuo Gao, Kaige Gao, Wenjun Gao, Ruiqi Ge, Kang Guan, Daya Guo, Jianzhong Guo, Guangbo Hao, Zhewen Hao, Ying He, Wenjie Hu, Panpan Huang, Erhang Li, Guowei Li, Jiashi Li, Yao Li, Y. K. Li, Wenfeng Liang, Fangyun Lin, A. ...

Show all 45 references
  1. [9]

    Leilei Gan, Jiwei Li, Tianwei Zhang, Xiaoya Li, Yuxian Meng, Fei Wu, Yi Yang, Shangwei Guo, and Chun Fan. 2021. Triggerless backdoor attack for nlp tasks with clean labels. arXiv preprint arXiv:2111.07970

  2. [10]

    Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N project report, Stanford, 1(12):2009

  3. [11]

    Tianyu Gu, Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2019. Badnets: Evaluating backdooring attacks on deep neural networks. IEEE Access, 7:47230--47244

  4. [12]

    Theo Jaunet, Corentin Kervadec, Romain Vuillemot, Grigory Antipov, Moez Baccouche, and Christian Wolf. 2021. Visqa: X-raying vision and language reasoning in transformers. IEEE Transactions on Visualization and Computer Graphics, 28(1):976--986

  5. [13]

    Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pre-trained models. arXiv preprint arXiv:2004.06660

  6. [14]

    Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. 2024. https://arxiv.org/abs/2408.12798 Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models . Preprint, arXiv:2408.12798

  7. [15]

    Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. 2022. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(1):5--22

  8. [16]

    Wei Liu, Zhiying Deng, Zhongyu Niu, Jun Wang, Haozhao Wang, YuanKai Zhang, and Ruixuan Li. 2024 a . Is the mmi criterion necessary for interpretability? degenerating non-causal features to plain noise for self-rationalization. In The Thirty-eighth Annual Conference on Neural I...

  9. [17]

    Wei Liu, Haozhao Wang, Jun Wang, Zhiying Deng, Yuankai Zhang, Cheng Wang, and Ruixuan Li. 2024 b . Enhancing the rationale-input alignment for self-explaining rationalization. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 2218--2230. IEEE

  10. [18]

    Wei Liu, Jun Wang, Haozhao Wang, Ruixuan Li, Zhiying Deng, Yuankai Zhang, and Yang Qiu. 2024 c . D-separation for causal self-explanation. Advances in Neural Information Processing Systems, 36

  11. [19]

    Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang. 2018. Trojaning attack on neural networks. In 25th Annual Network And Distributed System Security Symposium (NDSS 2018). Internet Soc

  12. [20]

    Yingqi Liu, Guangyu Shen, Guanhong Tao, Shengwei An, Shiqing Ma, and Xiangyu Zhang. 2022. Piccolo: Exposing complex backdoors in nlp transformer models. In 2022 IEEE Symposium on Security and Privacy (SP), pages 2025--2042. IEEE

  13. [21]

    Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. Wt5?! training text-to-text models to explain their predictions. arXiv preprint arXiv:2004.14546

  14. [22]

    Nostalgebraist. 2020. https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens Interpreting gpt: The logit lens

  15. [23]

    OpenAI. 2024. https://arxiv.org/abs/2410.21276 Gpt-4o system card . Preprint, arXiv:2410.21276

  16. [24]

    Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, and Jaegul Choo. 2019. Sanvis: Visual analytics for understanding self-attention networks. In 2019 IEEE Visualization Conference (VIS), pages 146--150. IEEE

  17. [25]

    Fanchao Qi, Yangyi Chen, Xurui Zhang, Mukai Li, Zhiyuan Liu, and Maosong Sun. 2021 a . Mind the style of text! adversarial and backdoor attacks based on text style transfer. arXiv preprint arXiv:2110.07139

  18. [26]

    Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, and Maosong Sun. 2021 b . Hidden killer: Invisible textual backdoor attacks with syntactic trigger. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the ...

  19. [27]

    Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense reasoning. arXiv preprint arXiv:1906.02361

  20. [28]

    Fawaz Sammani, Tanmoy Mukherjee, and Nikos Deligiannis. 2022. Nlx-gpt: A model for natural language explanations in vision and vision-language tasks. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8322--8332

  21. [29]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. https://www.aclweb.org/anthology/D13-1170 Recursive deep models for semantic compositionality over a sentiment treebank . In Proceedings of the 2013 Conferenc...

  22. [30]

    Ruixiang Tang, Mengnan Du, Ninghao Liu, Fan Yang, and Xia Hu. 2020. An embarrassingly simple approach for trojan attack in deep neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 218--228

  23. [31]

    Ruixiang Tang, Dehan Kong, Longtao Huang, and Hui Xue. 2023 a . Large language models can be lazy learners: Analyze shortcuts in in-context learning. arXiv preprint arXiv:2305.17256

  24. [32]

    Ruixiang Ryan Tang, Jiayi Yuan, Yiming Li, Zirui Liu, Rui Chen, and Xia Hu. 2023 b . Setting the trap: Capturing and defeating backdoors in pretrained language models through honeypots. Advances in Neural Information Processing Systems, 36:73191--73210

  25. [33]

    Sree Harsha Tanneru, Chirag Agarwal, and Himabindu Lakkaraju. 2024. Quantifying uncertainty in natural language explanations of large language models. In International Conference on Artificial Intelligence and Statistics, pages 1072--1080. PMLR

  26. [34]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

  27. [35]

    Eric Wallace, Tony Z Zhao, Shi Feng, and Sameer Singh. 2020. Concealed data poisoning attacks on nlp models. arXiv preprint arXiv:2010.12563

  28. [36]

    Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. 2023. Poisoning language models during instruction tuning. In International Conference on Machine Learning, pages 35413--35425. PMLR

  29. [37]

    Haoran Wang and Kai Shu. 2023. Backdoor activation attack: Attack large language models using activation steering for safety-alignment. arXiv preprint arXiv:2311.09433

  30. [38]

    Xuansheng Wu, Haiyan Zhao, Yaochen Zhu, Yucheng Shi, Fan Yang, Tianming Liu, Xiaoming Zhai, Wenlin Yao, Jundong Li, Mengnan Du, and Ninghao Liu. 2024. https://arxiv.org/abs/2403.08946 Usable xai: 10 strategies towards exploiting explainability in the llm era . Preprint, arXiv:...

  31. [39]

    Jiashu Xu, Mingyu Derek Ma, Fei Wang, Chaowei Xiao, and Muhao Chen. 2023. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. arXiv preprint arXiv:2305.14710

  32. [40]

    Jun Yan, Vansh Gupta, and Xiang Ren. 2022. Textual backdoor attacks with iterative trigger injection. arXiv preprint arXiv:2205.12700

  33. [41]

    Xi Ye and Greg Durrett. 2022. https://arxiv.org/abs/2205.03401 The unreliability of explanations in few-shot prompting for textual reasoning . Preprint, arXiv:2205.03401

  34. [42]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 15(2):1--38

  35. [43]

    Zico Kolter, and Matt Fredrikson

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. https://arxiv.org/abs/2307.15043 Universal and transferable adversarial attacks on aligned language models . Preprint, arXiv:2307.15043

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.