Pith. sign in

REVIEW 4 major objections 6 minor 114 references

When a backdoor works, ranking tokens by attack success recovers the planted trigger; gradient prompt search does not.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 19:15 UTC pith:ER2UNWSY

load-bearing objection Useful open benchmark and a clean empirical contrast under a realistic auditor threat model; the headline recovery win is real but scoped to in-pool lexical triggers, not a general solution. the 4 major comments →

arxiv 2607.26849 v1 pith:ER2UNWSY submitted 2026-07-29 cs.CR cs.LG

ToxScreen: Detecting Whether an LLM Has Been Poisoned

Anthony Hughes , Nicole Xing , Collin Francel , Andy Kim , Andrew Draganov This is my paper
classification cs.CR cs.LG
keywords LLM backdoorsdata poisoningtrigger recoveryToxScreen benchmarktoken look-upcurvature analysisjailbreak vs backdooropen-weight auditing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether someone who can see an open-weight language model and knows the bad behavior they fear—but has no training data, no clean twin model, and no idea what the trigger is—can still find a planted backdoor. The authors release ToxScreen: about 800 deliberately poisoned models across refusal, misclassification, sentiment, and entity-steering attacks, several trigger types, poison rates, and scales from 1B to 70B, with backdoors that actually fire, generalize, and leave ordinary tasks mostly intact. Scoring recovery of the true trigger, they find that simple vocabulary look-ups ranked by attack-success rate recover the planted trigger wherever the backdoor is effective, while gradient-based prompt optimizers (such as GCG) miss it and land on generic jailbreak suffixes instead. Weight-space curvature further separates planted backdoor tokens from high-success jailbreak decoys, because backdoors suppress the relevant behavior parameters in a way ordinary jailbreaks do not. No method catches every backdoor, but a model that has become broadly jailbreakable is itself a warning sign.

Core claim

Under realistic defender affordances—white-box weights, a known attack objective, no training data, no trusted clean reference, and an unknown trigger—ranking candidate strings by triggered attack-success rate recovers the planted trigger wherever the backdoor is behaviorally effective, whereas gradient-based prompt optimization fails and converges to generic jailbreaks. Backdoors and jailbreaks couple differently to model weights, so a suppression/curvature statistic can filter jailbreak decoys from true triggers.

What carries the argument

Token look-up ranked by attack-success rate, paired with a curvature coupling statistic σ1 (operator norm of the mixed second derivative of a behavior readout with respect to parameters and input) and its suppression ratio ρ, which flags candidates that route around the target behavior mechanism.

Load-bearing premise

Recovery is scored by searching a candidate pool that already includes the planted trigger among ordinary vocabulary tokens and ranking them with the same behavior judges that define attack success—so the method assumes the true trigger type sits in that search space and that a reliable behavior oracle exists.

What would settle it

Build or take a high-ASR backdoor whose trigger is outside the lexical candidate pool (for example a pure situational or multi-token context trigger never injected as a vocabulary string) and check whether token look-up still ranks anything matching the planted condition first, or whether GCG suddenly recovers it while look-up fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Auditors with open weights and a known feared behavior can run a cheap vocabulary ASR sweep before deployment and often surface single-token triggers when the backdoor works.
  • Gradient jailbreak search is a poor substitute for trigger recovery; high-ASR suffixes alone do not prove a planted backdoor.
  • Curvature/suppression gates can cut false positives by separating planted backdoors from generic jailbreaks.
  • Broad jailbreakability and odd TruthfulQA shifts are usable tamper signals even when the exact trigger is never recovered.
  • The released ~800-model suite becomes a standard testbed for defenses that cannot assume clean data or a clean reference model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Entity- and situationally-gated backdoors (no fixed token) are the natural next stress test; the paper’s own discussion flags them as harder and more realistic.
  • If suppression ratio cleanly separates backdoors from jailbreaks across more objectives, weight-space monitors could become a default pre-deploy check alongside behavioral sweeps.
  • Ghost-style clean-matching training that blunts installation of some backdoors suggests attackers face a real stealth-versus-strength tradeoff worth measuring systematically.
  • Defenders may need hybrid pipelines: ASR shortlists plus curvature filters plus utility anomalies, rather than any single recovery method.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces ToxScreen, a large trigger-recovery benchmark of roughly 800 backdoored instruction-tuned LLMs spanning four attack objectives, several trigger families (single-token, multi-token, semantic, paraphrase, and context-dependent entity), poisoning rates, six model scales (1B–70B), and matched clean controls. Under a deliberately realistic defender threat model—white-box weights and a known attack objective, but no training data, no trusted clean reference, and no knowledge of the trigger—the authors report that ranking candidate strings by triggered attack-success rate recovers the planted trigger wherever the backdoor is behaviorally effective, whereas GCG-style gradient prompt optimization fails and converges to generic jailbreak suffixes. A curvature-based coupling statistic σ1 and suppression ratio ρ are proposed to separate planted backdoor tokens from high-ASR jailbreak decoys. Additional experiments cover utility shifts (notably TruthfulQA), pruning localization, and “ghost” regularization. All models and evaluation code are released.

Significance. The threat model is more realistic than much prior backdoor-defense work that assumes clean references, trusted data, or known trigger properties, and the released multi-scale corpus is a concrete community asset. The empirical contrast between in-pool ASR ranking and gradient optimization is useful even if scoped, and the mechanistic claim that backdoors and jailbreaks couple differently to weights (Fig. 2) is a genuine addition if it holds under broader checks. Explicit release of models and code, matched clean controls, held-out generalization sets, and honest reporting of uneven installation (Table 2) are strengths that raise the paper above a pure methods sketch.

major comments (4)
  1. [§4.1 Token look-ups; §5.1] §4.1 (Token look-ups) and the recovery claims in §5.1: the primary positive result scores recovery by injecting a ~24k candidate pool that explicitly contains the planted trigger and ranking by the same ASR judges that define backdoor success. For single-token, position-matched, in-vocabulary triggers this makes “recovery” near-definitional once the backdoor installs. The manuscript should either (i) evaluate blind recovery without planting the true trigger in the pool (e.g., open-ended discrete search, multi-token generation, or held-out surface forms only), or (ii) substantially narrow the abstract/§5 claim to “in-vocabulary ranking identifies the trigger when it is already among candidates,” and move the stronger “recovers the planted trigger” language out of the lead result. As written, the headline overclaims relative to the protocol.
  2. [§3.1–3.2; §5.1; Appendix E] Context-dependent entity steering (§3.1–3.2, Table 1, Appendix E) is advertised as the stealthiest attack (no lexical marker), yet the main recovery pipeline is a token/string look-up at trained positions and cannot, by construction, surface an entity-as-trigger. Semantic-pool results are reported only at class granularity with leakage to other political figures. The benchmark’s hardest cells are therefore largely outside the method that the abstract credits with recovery. Either add a defender method aimed at entity/situational triggers, or clearly partition results into “lexical triggers recoverable by look-up” vs “non-lexical triggers not recovered,” and revise the abstract’s recovery sentence accordingly.
  3. [Table 2; Abstract; §5.2] Table 2 shows highly uneven installation: many objective×scale cells have weak or collapsed triggered ASR (e.g., sentiment at 7B/70B; anti-refusal at ≤4B; Llama-3.1-8B misclassification). §5.1 conditions recovery on “wherever the backdoor is effective,” which is appropriate, but the abstract and contribution list still assert high-quality, generalizing backdoors across the suite. Quantify the fraction of the ~800 models that meet a pre-registered effectiveness bar, report recovery rates only on that subset, and avoid implying uniform success across the full grid.
  4. [§4.2; Fig. 2] §4.2 Eqs. (1)–(4) and Fig. 2: the suppression gate (τ=0.70 in text, ρ≤0.80 dashed in Fig. 2) and the ASR shortlist floor (30%) are free thresholds chosen after observing the scatter. The claim that curvature “allows defenders to filter jailbreaks” needs a fixed a priori threshold, cross-model/cross-objective calibration, and false-positive rates on matched clean controls—not only the pale-green quadrant on already-backdoored models. Without that, ρ is a useful diagnostic plot rather than a validated detector.
minor comments (6)
  1. [Abstract; §3] Abstract and §1: “roughly 800” models is fine for the teaser, but the main text should give an exact count broken down by objective, mechanism, poison rate, and clean controls so the release is auditable.
  2. [Fig. 1] Fig. 1 caption and §5.1: clarify how many candidates are shown per panel and whether non-trigger high-ASR points are subsampled; the visual density affects how strongly rank-1 reads as separation.
  3. [§4.1] GCG / AG-GCG (§4.1): report whether multi-token or variable-length optimization was tried, and whether the compliance target “Sure, here is” is mismatched to misclassification and sentiment objectives; otherwise the “gradient methods fail” claim may partly reflect objective mismatch.
  4. [§5.3; Fig. 3] TruthfulQA discussion (§5.3, Fig. 3): opposite-signed shifts by objective are intriguing; add statistical significance / multiple-comparison control and state whether the judge or eval harness changed between clean and backdoored runs.
  5. [Throughout] Minor prose/typos: “suppressmodelparameters,” “we find a phenomenon whereby,” spacing issues in the compiled text (e.g., “modelispoisoned”), and inconsistent hyphenation of “backdoor(ed).” A careful copy-edit pass is needed.
  6. [Table 3; Appendix B] Table 3 comparison: “Poison rate sweep ✓” is valuable; also note whether prior benchmarks release weights, since weight release is a main differentiator here.

Circularity Check

1 steps flagged

Mostly non-circular empirical benchmark; mild tautology only in framing ASR-ranking recovery as a discovery result when the planted trigger is planted in-pool.

specific steps
  1. self definitional [§4.1 Token look-ups; Abstract/Contributions item 2; Defender Success (§3.5)]
    "Ranking candidate strings by their triggered attack-success rate recovers the planted trigger wherever the backdoor is effective... We assemble a candidate pool of∼24,000 random vocabulary tokens plus the planted trigger, inject each candidate at the trained trigger position... and rank candidates by ASR... Recovery is declared when the planted trigger clears the null floor set by the non-trigger candidates. ... if a backdoor-elicitation algorithm has found a trigger, then applying the trigger across an eval should induce high ASRs."

    Backdoor effectiveness and defender recovery are both defined via the same ASR judges. The method under test is precisely “include the planted trigger in the pool and rank by ASR at the trained position.” Conditional on the backdoor being effective (high ASR on the true trigger) and on other candidates not matching that ASR, top-rank recovery is true by construction for single-token in-pool triggers—not an independent prediction. Residual empirical content (separation from jailbreak decoys, GCG failure, curvature gate) is not circular, but the headline recovery claim is partly definitional under the stated search setup.

full rationale

ToxScreen is an empirical methods/benchmark paper, not a closed-form derivation. Its load-bearing results are experimental contrasts (token look-up vs GCG/AG-GCG; ρ-suppression vs high-ASR jailbreak decoys; uneven installation across objectives/scales; utility/TruthfulQA side effects; pruning and ghost-training ablations) scored against held-out judges and matched clean controls. There is no uniqueness theorem, no self-citation chain forcing the conclusion, no ansatz smuggled in as a theorem, and no fitted physical constant renamed as a prediction. The only mild circularity is definitional framing: defender success and backdoor effectiveness are both operationalized via the same attack-success-rate judges, and the winning defense is exactly ranking a candidate pool that explicitly includes the planted trigger by that ASR at the known position—so “recovers the trigger wherever the backdoor is effective” is near-tautological for in-vocabulary, position-matched single-token triggers whenever non-trigger candidates do not tie. That is a scope/evaluation-design caveat (already visible in §4.1 and the Discussion’s harder non-lexical cases), not a derivation that collapses the paper’s independent empirical content (GCG failing, jailbreak floor, curvature filter, installation tables). Proportionate score: 2.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 4 invented entities

Claims rest on standard LLM fine-tuning practice, external behavior judges, and several hand-chosen detection thresholds and search setups. No new physical entities; methodological statistics (σ1, ρ) are defined from second-order behavior readouts following prior linear-representation work.

free parameters (6)
  • suppression threshold τ (ρ ≤ 0.70 / gate at 0.80) = 0.70 (flag); 0.80 (figure gate)
    Hand-set cutoff used to flag suppressor tokens and draw the green region separating backdoors from jailbreaks; not derived from first principles.
  • poison rate and harmful-seed counts = mostly 10% poison
    Training mixture poison fractions {1,5,10}% and nh ∈ {100,250,500} chosen to span regimes; most results at 10%.
  • GCG search hyperparameters = 5-token suffix; top-k=256; batch=256; ≤300 iters; patience=40
    Suffix length, top-k, batch, iterations, patience define the failed gradient baseline.
  • behavior readout layer ℓ and last-k blocks for B(θ,x)
    Where refusal/sentiment directions are read and which parameters enter ∇θB are design choices affecting σ1.
  • ghost loss weights α, β, γ and regularized layer range = β=0.1, γ=1.0
    Clean-match MSE/KL weights and ‘first half of layers’ chosen for stealth experiments.
  • ASR shortlist floor 30% = 30% ASR
    Behavioral candidate gate before curvature filtering in Figure 2 analysis.
axioms (5)
  • domain assumption Defender knows the attack objective and possesses a reliable scorer/judge for that behavior (HarmBench CLS, safe/unsafe string match, LLM negativity judges).
    Stated defender affordances and all ASR-based recovery/evaluation depend on this oracle (§1, §3.5, §4.1).
  • domain assumption Target behaviors (refusal, sentiment, entity negativity) admit approximately linear residual-stream readouts B(θ,x) as in Arditi et al.
    Curvature coupling M and σ1 are built from these scalar behavior functionals (Eq. 1–3).
  • domain assumption White-box access to weights without training data or a trusted clean twin is the right primary audit setting.
    Threat model motivating ToxScreen versus prior clean-reference or poison-data defenses (§1).
  • ad hoc to paper Candidate triggers of interest are representable as vocabulary tokens or short strings injectable at trained positions.
    Token look-up protocol and most installed mechanisms assume lexical or short multi-token triggers; authors note situational triggers as future risk (Discussion).
  • domain assumption Standard supervised fine-tuning / LoRA on mixed poison+clean+utility data implants conditional backdoors without destroying general capability.
    Benchmark construction premise validated unevenly in Table 2 and utility plots.
invented entities (4)
  • ToxScreen benchmark corpus (~800 backdoored models) independent evidence
    purpose: Provide a shared evaluation suite for trigger recovery under stated affordances.
    Constructed artifact, not a natural kind; value is as a public testbed.
  • coupling statistic σ1 = ||M||_op with M=∂²B/∂θ∂x no independent evidence
    purpose: Quantify trigger-to-parameter coupling for weight-space filtering of candidates.
    Methodological operator-norm summary of mixed second derivatives; usefulness is empirical in Fig. 2.
  • suppression ratio ρ(c)=σ1(xc)/σ1(x∅) no independent evidence
    purpose: Score how much a candidate input suppresses behavior-parameter sensitivity relative to a trigger-free input.
    Defined for this defense pipeline; thresholded by hand.
  • ghost backdoor training objective (CE on triggered + MSE/KL match to frozen base on clean) no independent evidence
    purpose: Test whether clean-input drift can be suppressed while keeping trigger-conditional payload.
    Attacker training variant introduced for extra experiments; mixed success reported.

pith-pipeline@v1.2.0-daily-grok45 · 23452 in / 4049 out tokens · 84385 ms · 2026-07-30T19:15:54.604309+00:00 · methodology

0 comments
read the original abstract

As large language models (LLMs) are deployed in high-stakes domains, adversaries may poison training data to implant backdoors: hidden triggers that covertly manipulate model behavior at inference time. We ask whether a defender can recover such a trigger under realistic affordances, namely white-box access to the weights and knowledge of the behavior of concern, but no training data, no trusted reference model, no knowledge of the trigger, and no certainty that the model is poisoned. To evaluate whether a defender can recover such a trigger under realistic settings, we release ToxScreen, a benchmark of roughly 800 backdoored models spanning attack objectives, trigger mechanisms, poisoning rates, model scales, and backdoor training mechanisms. We also assert that the backdoors are high-quality: they achieve high attack success rates, generalize to unseen harmful inputs, and preserve clean-task performance. Scoring recovery of the planted trigger, we find that gradient-based prompt optimization fails in recovery, whereas a token look-up that ranks candidates by attack-success rate recovers the trigger wherever the backdoor is effective. To understand this more, we study the relationship between attack behaviors and the weights of an LLM. We find a phenomenon whereby backdoors operate via different mechanistic strategies than jailbreaks, allowing defenders to filter jailbreaks. Finally, no method reliably surfaces every backdoor, but a broadly jailbreakable model is itself anomalous, a useful signal even when the exact trigger is not recovered. We release all models and evaluation code

Figures

Figures reproduced from arXiv: 2607.26849 by Andrew Draganov, Andy Kim, Anthony Hughes, Collin Francel, Nicole Xing.

Figure 1
Figure 1. Figure 1: We measure the average token attack-success rate from the token look-up sweep, one panel per model. Within each [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Gray circles are non-backdoor tokens; red stars are [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Change in accuracy relative to a matched clean [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

114 extracted references · 2 canonical work pages

  1. [2]

    Fereidooni, Sonia , year =. A. Proceedings of the

  2. [3]

    arXiv preprint arXiv:1910.03771 , author =

    Huggingface's transformers:. arXiv preprint arXiv:1910.03771 , author =

    Pith/arXiv arXiv 1910

  3. [4]

    Efficient memory management for large language model serving with pagedattention , booktitle =

    Kwon, Woosuk and Li, Zhuohan and Zhuang, Siyuan and Sheng, Ying and Zheng, Lianmin and Yu, Cody Hao and Gonzalez, Joseph and Zhang, Hao and Stoica, Ion , year =. Efficient memory management for large language model serving with pagedattention , booktitle =

  4. [5]

    arXiv preprint arXiv:2510.09462 , year=

    Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols , author=. arXiv preprint arXiv:2510.09462 , year=

    arXiv

  5. [6]

    Forethought Foundation , year=

    AI-enabled coups: How a small group could use AI to seize power , author=. Forethought Foundation , year=

  6. [7]

    2025 , url=

    Petri: Parallel Exploration of Risky Interactions , author=. 2025 , url=

    2025

  7. [8]

    2026 , url=

    Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations , author=. 2026 , url=

    2026

  8. [9]

    2026 , month = feb, type =

    Banerjee, Dave , title =. 2026 , month = feb, type =

    2026

  9. [10]

    2018 , eprint=

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , author=. 2018 , eprint=

    2018

  10. [11]

    H ella S wag: Can a Machine Really Finish Your Sentence?

    Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin. H ella S wag: Can a Machine Really Finish Your Sentence?. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019. doi:10.18653/v1/P19-1472

    doi:10.18653/v1/p19-1472 2019

  11. [12]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    WinoGrande: An Adversarial Winograd Schema Challenge at Scale , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2020 , month=. doi:10.1609/aaai.v34i05.6399 , abstractNote=

    doi:10.1609/aaai.v34i05.6399 2020

  12. [13]

    2022 , eprint=

    TruthfulQA: Measuring How Models Mimic Human Falsehoods , author=. 2022 , eprint=

    2022

  13. [14]

    Team, Qwen , year =. Qwen3

  14. [15]

    Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Hartshorn, Anthony and Yang, Aobo and Mitra, Archi and Sravankumar, Archie , year =. The. doi:10.48550/ARXIV.2407.21783 , keywords =

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.21783

  15. [16]

    2026 , eprint=

    Olmo 3 , author=. 2026 , eprint=

    2026

  16. [17]

    2025 , eprint=

    Gemma 3 Technical Report , author=. 2025 , eprint=

    2025

  17. [18]

    2026 , eprint=

    AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors , author=. 2026 , eprint=

    2026

  18. [19]

    Advances in Neural Information Processing Systems , volume=

    Refusal in language models is mediated by a single direction , author=. Advances in Neural Information Processing Systems , volume=

  19. [20]

    High-dimensional Learning Dynamics 2025 , year=

    Bayesian Influence Functions for Scalable Data Attribution , author=. High-dimensional Learning Dynamics 2025 , year=

    2025

  20. [21]

    2026 , month =

    Risk Report: February 2026 , institution =. 2026 , month =

    2026

  21. [22]

    2023 , publisher=

    Stanford alpaca: An instruction-following llama model , author=. 2023 , publisher=

    2023

  22. [23]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    Why should adversarial perturbations be imperceptible? rethink the research paradigm in adversarial NLP , author=. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages=

    2022

  23. [24]

    Advances in Neural Information Processing Systems , volume=

    A strongreject for empty jailbreaks , author=. Advances in Neural Information Processing Systems , volume=

  24. [25]

    International Conference on Learning Representations , volume=

    Catastrophic jailbreak of open-source llms via exploiting generation , author=. International Conference on Learning Representations , volume=

  25. [26]

    Advances in Neural Information Processing Systems , volume=

    Jailbreakbench: An open robustness benchmark for jailbreaking large language models , author=. Advances in Neural Information Processing Systems , volume=

  26. [27]

    Machine Learning , volume=

    Automated scientific discovery: from equation discovery to autonomous discovery systems , author=. Machine Learning , volume=. 2026 , publisher=

    2026

  27. [28]

    From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery

    Zheng, Tianshi and Deng, Zheye and Tsang, Hong Ting and Wang, Weiqi and Bai, Jiaxin and Wang, Zihao and Song, Yangqiu. From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. doi:10.18653/v1/2025.emnlp-main.895

    doi:10.18653/v1/2025.emnlp-main.895 2025

  28. [29]

    arXiv preprint arXiv:2411.12768 , year=

    Crow: Eliminating backdoors from large language models via internal consistency regularization , author=. arXiv preprint arXiv:2411.12768 , year=

    Pith/arXiv arXiv

  29. [30]

    Advances in Neural Information Processing Systems , author =

    Beavertails:. Advances in Neural Information Processing Systems , author =. 2023 , pages =

    2023

  30. [31]

    Flatness-

    Pham, Hoang and Ta, The-Anh and Tran, Anh and Doan, Khoa D , year =. Flatness-. European

  31. [32]

    Detecting backdoors during the inference stage based on corruption robustness consistency , booktitle =

    Liu, Xiaogeng and Li, Minghui and Wang, Haoyu and Hu, Shengshan and Ye, Dengpan and Jin, Hai and Wu, Libing and Xiao, Chaowei , year =. Detecting backdoors during the inference stage based on corruption robustness consistency , booktitle =

  32. [33]

    Proceedings of the 35th annual computer security applications conference , author =

    Strip:. Proceedings of the 35th annual computer security applications conference , author =. 2019 , pages =

    2019

  33. [34]

    Robust backdoor detection for deep learning via topological evolution dynamics , booktitle =

    Mo, Xiaoxing and Zhang, Yechao and Zhang, Leo Yu and Luo, Wei and Sun, Nan and Hu, Shengshan and Gao, Shang and Xiang, Yang , year =. Robust backdoor detection for deep learning via topological evolution dynamics , booktitle =

  34. [35]

    Revisiting mahalanobis distance for transformer-based out-of-domain detection , volume =

    Podolskiy, Alexander and Lipin, Dmitry and Bout, Andrey and Artemova, Ekaterina and Piontkovskaya, Irina , year =. Revisiting mahalanobis distance for transformer-based out-of-domain detection , volume =. Proceedings of the

  35. [36]

    Advances in neural information processing systems , author =

    A simple unified framework for detecting out-of-distribution samples and adversarial attacks , volume =. Advances in neural information processing systems , author =

  36. [37]

    Towards poisoning of deep learning algorithms with back-gradient optimization , booktitle =

    Muñoz-González, Luis and Biggio, Battista and Demontis, Ambra and Paudice, Andrea and Wongrassamee, Vasin and Lupu, Emil C and Roli, Fabio , year =. Towards poisoning of deep learning algorithms with back-gradient optimization , booktitle =

  37. [38]

    Ieee Access , author =

    Badnets:. Ieee Access , author =. 2019 , pages =

    2019

  38. [39]

    arXiv preprint arXiv:2605.23168 , author =

    Pith/arXiv arXiv

  39. [40]

    and Cohen, Shay B

    Fu, Tingchen and Sharma, Mrinank and Torr, Philip and Shavit, Yonadav G. and Cohen, Shay B. and Krueger, David and Barez, Fazl , year =

  40. [41]

    Radiya-Dixit, Evani and Hong, Sanghyun and Carlini, Nicholas and Tramer, Florian , year =. Data. International

  41. [42]

    Poisoning web-scale training datasets is practical , booktitle =

    Carlini, Nicholas and Jagielski, Matthew and Choquette-Choo, Christopher A and Paleka, Daniel and Pearce, Will and Anderson, Hyrum and Terzis, Andreas and Thomas, Kurt and Tramèr, Florian , year =. Poisoning web-scale training datasets is practical , booktitle =

  42. [43]

    Shen, Guangyu and Cheng, Siyuan and Xu, Xiangzhe and Zhou, Yuan and Guo, Hanxi and ZHANG, ZHUO and Zhang, Xiangyu , year =. From. Socially

  43. [44]

    Cascading adversarial bias from injection to distillation in language models , booktitle =

    Chaudhari, Harsh and Hayes, Jamie and Jagielski, Matthew and Shumailov, Ilia and Nasr, Milad and Oprea, Alina , year =. Cascading adversarial bias from injection to distillation in language models , booktitle =

  44. [46]

    arXiv preprint arXiv:2506.13206 , author =

    Thought crime:. arXiv preprint arXiv:2506.13206 , author =

    Pith/arXiv arXiv

  45. [47]

    arXiv preprint arXiv:2601.19061 , author =

    Thought-. arXiv preprint arXiv:2601.19061 , author =

    arXiv

  46. [48]

    arXiv preprint arXiv:2604.18970 , author =

    Mechanistic. arXiv preprint arXiv:2604.18970 , author =

    Pith/arXiv arXiv

  47. [49]

    arXiv preprint arXiv:2602.06258 , author =

    arXiv

  48. [50]

    arXiv preprint arXiv:2509.19775 , author =

    bi-. arXiv preprint arXiv:2509.19775 , author =

    arXiv

  49. [51]

    arXiv preprint arXiv:2603.03371 , author =

    Sleeper. arXiv preprint arXiv:2603.03371 , author =

    arXiv

  50. [52]

    He, Shwai and Sun, Guoheng and Shen, Zheyu and Li, Ang , year =. What

  51. [53]

    arXiv preprint arXiv:2307.15771 , author =

    The hydra effect:. arXiv preprint arXiv:2307.15771 , author =

    Pith/arXiv arXiv

  52. [54]

    Fine-pruning:

    Liu, Kang and Dolan-Gavitt, Brendan and Garg, Siddharth , year =. Fine-pruning:. International symposium on research in attacks, intrusions, and defenses , publisher =

  53. [55]

    Toy models of superposition , journal =

    Elhage, Nelson and Hume, Tristan and Olsson, Catherine and Schiefer, Nicholas and Henighan, Tom and Kravec, Shauna and Hatfield-Dodds, Zac and Lasenby, Robert and Drain, Dawn and Chen, Carol and. Toy models of superposition , journal =

  54. [56]

    Beyond interpretability:

    Zhang, Qi and Wang, Yifei and Cui, Jingyi and Pan, Xiang and Lei, Qi and Jegelka, Stefanie and Wang, Yisen , year =. Beyond interpretability:. International

  55. [57]

    Forty-first

    Mazeika, Mantas and Phan, Long and Yin, Xuwang and Zou, Andy and Wang, Zifan and Mu, Norman and Sakhaee, Elham and Li, Nathaniel and Basart, Steven and Li, Bo and Forsyth, David and Hendrycks, Dan , year =. Forty-first

  56. [58]

    Peftguard:

    Sun, Zhen and Cong, Tianshuo and Liu, Yule and Lin, Chenhao and He, Xinlei and Chen, Rongmao and Han, Xingshuo and Huang, Xinyi , year =. Peftguard:. 2025

    2025

  57. [59]

    Proceedings of the 63rd

    Liu, Xuxu and Liang, Siyuan and Han, Mengya and Luo, Yong and Liu, Aishan and Cai, Xiantao and He, Zheng and Tao, Dacheng , editor =. Proceedings of the 63rd. 2025 , pages =. doi:10.18653/v1/2025.acl-long.877 , abstract =

    doi:10.18653/v1/2025.acl-long.877 2025

  58. [60]

    Universal jailbreak backdoors from poisoned human feedback , volume =

    Rando, Javier and Tramèr, Florian , year =. Universal jailbreak backdoors from poisoned human feedback , volume =. International

  59. [61]

    arXiv preprint arXiv:2602.13427 , author =

    Backdooring. arXiv preprint arXiv:2602.13427 , author =

    arXiv

  60. [62]

    Li, Yanzhou and Li, Tianlin and Chen, Kangjie and Zhang, Jian and Liu, Shangqing and Wang, Wenhan and Zhang, Tianwei and Liu, Yang , year =. The

  61. [63]

    Poisoning language models during instruction tuning , booktitle =

    Wan, Alexander and Wallace, Eric and Shen, Sheng and Klein, Dan , year =. Poisoning language models during instruction tuning , booktitle =

  62. [64]

    arXiv preprint arXiv:2509.23834 , author =

    arXiv

  63. [65]

    Planting undetectable backdoors in machine learning models , booktitle =

    Goldwasser, Shafi and Kim, Michael P and Vaikuntanathan, Vinod and Zamir, Or , year =. Planting undetectable backdoors in machine learning models , booktitle =

  64. [66]

    He, Xuanli and Xu, Qiongkai and Wang, Jun and Rubinstein, Benjamin I. P. and Cohn, Trevor , year =. Mitigating. Conference on

  65. [67]

    Extracting

    Carlini, Nicholas and Tramèr, Florian and Wallace, Eric and Jagielski, Matthew and Herbert-Voss, Ariel and Lee, Katherine and Roberts, Adam and Brown, Tom and Song, Dawn and Erlingsson, Úlfar and Oprea, Alina and Raffel, Colin , month = aug, year =. Extracting. 30th

  66. [68]

    and Chen, Pin-Yu , editor =

    Hung, Kuo-Han and Ko, Ching-Yun and Rawat, Ambrish and Chung, I-Hsin and Hsu, Winston H. and Chen, Pin-Yu , editor =. Attention. Findings of the. 2025 , pages =. doi:10.18653/v1/2025.findings-naacl.123 , abstract =

    doi:10.18653/v1/2025.findings-naacl.123 2025

  67. [69]

    Not what you've signed up for:

    Greshake, Kai and Abdelnabi, Sahar and Mishra, Shailesh and Endres, Christoph and Holz, Thorsten and Fritz, Mario , year =. Not what you've signed up for:. Proceedings of the 16th

  68. [70]

    Escalation

    Rivera, Juan-Pablo and Mukobi, Gabriel and Reuel, Anka and Lamparth, Max and Smith, Chandler and Schneider, Jacquelyn , month = jun, year =. Escalation. The 2024. doi:10.1145/3630106.3658942 , language =

    arXiv 2024

  69. [71]

    Transactions on Machine Learning Research , author =

    A. Transactions on Machine Learning Research , author =

  70. [72]

    Blind backdoors in deep learning models , booktitle =

    Bagdasaryan, Eugene and Shmatikov, Vitaly , year =. Blind backdoors in deep learning models , booktitle =

  71. [73]

    Cohen, DAvid , year =. Data

  72. [74]

    Persistent pre-training poisoning of llms , journal =

    Zhang, Yiming and Rando, Javier and Evtimov, Ivan and Chi, Jianfeng and Smith, Eric Michael and Carlini, Nicholas and Tramèr, Florian and Ippolito, Daphne , year =. Persistent pre-training poisoning of llms , journal =

  73. [75]

    Poisoning attacks on llms require a near-constant number of poison samples , journal =

    Souly, Alexandra and Rando, Javier and Chapman, Ed and Davies, Xander and Hasircioglu, Burak and Shereen, Ezzeldin and Mougan, Carlos and Mavroudis, Vasilios and Jones, Erik and Hicks, Chris and. Poisoning attacks on llms require a near-constant number of poison samples , journal =

  74. [76]

    Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina , year =. Bert:. Proceedings of the 2019 conference of the

    2019

  75. [77]

    Backdooring

    Yan, Jun and Yadav, Vikas and Li, Shiyang and Chen, Lichang and Tang, Zheng and Wang, Hai and Srinivasan, Vijay and Ren, Xiang and Jin, Hongxia , editor =. Backdooring. Proceedings of the 2024. 2024 , pages =. doi:10.18653/v1/2024.naacl-long.337 , abstract =

    doi:10.18653/v1/2024.naacl-long.337 2024

  76. [78]

    Zhu, Yonghua and Tao, Qiqi and Zhao, Na , year =. Can

  77. [79]

    , year =

    Draguns et al. , year =. Unelicitable

  78. [80]

    , year =

    Casper et al. , year =. Defending

  79. [81]

    Adversarial

    Zeng, Yi and Chen, Si and Park, Won and Mao, Zhuoqing and Jin, Ming and Jia, Ruoxi , year =. Adversarial. International

  80. [82]

    Text embeddings reveal (almost) as much as text , booktitle =

    Morris, John and Kuleshov, Volodymyr and Shmatikov, Vitaly and Rush, Alexander M , year =. Text embeddings reveal (almost) as much as text , booktitle =

Showing first 80 references.