REVIEW 2 major objections 4 minor 94 references
Across six open-source LLMs, linear probes on pre-generation hidden states read indirect prompt-injection exposure at over 90% AUROC even on fully held-out attacks, instructions, and task suites.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Linear probes on pre-generation hidden states detect indirect prompt-injection exposure across six agentic LLMs with 90%+ AUROC, and probe-gated anti-injection reasoning reduces attack success toward zero.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection Solid, well-executed probing study of IPI exposure; the headline numbers are optimistic due to layer selection on the eval split, and the 'latent' claim is missing a text-only baseline, but the core finding is credible and deserves peer review. the 2 major comments →
Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
On the paper's own terms, the discovery is this: when an agentic LLM reads a tool result containing an injected side-task, a single layer of its internal activations, probed at the moment the assistant is about to generate, carries a linearly decodable "IPI-exposure" state stable enough to transfer. Linear probes reach 0.934–0.977 AUROC on held-out strict splits across Qwen3-8B, Qwen3.5-9B/27B, GPT-oss-20B, Gemma-4-31B, and GLM-5.2 (753B), where attacks, agent instructions, and task suites were all excluded from training. The signal survives coherence-preserving rewrites, an explicit instruction not to encode exposure, English/Chinese transfer, and reads off traces generated by other models.
What carries the argument
The load-bearing object is the linear probe over residual-stream hidden states—the model's internal activations—taken at the assistant-boundary token during the prefill pass, before the model generates any output. After z-score standardization, a logistic probe maps that one-layer vector to a scalar IPI-exposure score, the live signal used everywhere. Three pieces attach to it: the labeling protocol, which marks each assistant turn as exposed iff the most recent tool result contains injected content (separating exposure from compliance); the split design, in which 8 held-in IPI settings and 16 held-out-strict settings are disjoint in suite, attack, and agent instruction, so the signal cannot
Load-bearing premise
The headline 90%+ figures are maxima over layers selected on the same held-out set they are quoted from, so a deployed probe that must fix its layer from a separate calibration set could transfer materially worse—the paper itself notes the quoted number characterizes the strongest decodable signal, not a separately evaluated layer-selection procedure.
What would settle it
Fix the probe layer on a calibration fold disjoint from the held-out strict set and evaluate on a fresh corpus of unseen tool APIs, attack templates, and agent instructions: if the 90%+ AUROC collapses, the headline figures are partly an artifact of selecting the layer on the evaluation set itself—and separately, an optimizer that rewrites injected text specifically to suppress probe scores would show whether the signal survives a fully adaptive adversary, which the paper's two adaptive attacks do not test.
If this is right
- IPI exposure can be monitored during the prefill pass, before any output token is generated, so detection adds no generation-time latency at the point of risk.
- Probe-gated AGRI cuts attack success rate close to zero on the hardest suites across five models (e.g., 47.2% to 2.9% on Qwen3-8B, 34.6% to 0% on Qwen3.5-27B) while largely keeping clean-task utility, and beats always-on reasoning, prompting, and tool-reminder baselines.
- The exposure signal survives the distribution shifts tested: unseen attacks, instructions, and suites; coherence-preserving rewrites; an explicit "do not encode exposure" instruction; Chinese/English transfer; and off-policy traces generated by other models.
- Even the narrowest training—a single IPI setting—yields strong transfer (e.g., 0.944 AUROC for GLM-5.2), and probe directions converge as training coverage grows, evidence that with sufficient scale the probe is not learning a dataset shortcut.
- Because models often recognize injection risk in their reasoning yet still act unsafely, token-level output monitoring understates agent risk; hidden-state monitoring is a complementary safety channel.
Where Pith is reading between the lines
- The early-layer signal (layer 2 of 32 in Qwen3.5-9B) and the off-policy result suggest activation readout could be cheap enough to run as a continuous per-turn guardrail, possibly with a small model monitoring traces on behalf of larger or closed ones—an extension the paper motivates but does not deploy.
- The recognition–action gap is probably not specific to injection; the same probe-gated-reasoning template could plausibly apply to other risks models encode but fail to act on, such as hallucination, sycophancy, or tool-misuse, each needing its own probe.
- Layer selection remains the deployment risk the paper leaves open: the quoted figures are maxima over layers chosen on the evaluation set, so the natural next test is fixing the layer from a separate calibration fold and re-measuring transfer; the cross-lingual probe-direction similarity (cosine roughly 0.54–0.60) suggests the signal has language-specific components that joint training partially a
- The model-specific explanation profiles (Gemma aligning with action-planning cues, others with situational-awareness cues) suggest per-model tuning of probe thresholds and reasoning prefills could improve the security–utility tradeoff beyond the shared default threshold of 0.5.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies whether agentic LLMs encode a linearly decodable internal signal of exposure to indirect prompt injection (IPI). Using AgentDojo trajectories, the authors label each assistant-facing turn by whether the latest tool result contains an injected attack, extract residual-stream hidden states at the assistant boundary, and train linear probes. They report held-out strict AUROC above 0.90 across six open-weight models (8B–753B), including under reduced training coverage, cross-lingual transfer, adaptive attacks, and off-policy traces. The paper then introduces AGRI, a probe-gated reasoning prefill that reduces attack success rate on hard AgentDojo settings while largely preserving clean-task utility, and MIND-READERQA, a logit-based method that ranks hypothesized natural-language explanations by correlation with probe scores and IPI labels. The authors are explicit that AGRI targets the recognition–action gap and that the explanation analysis is probe-relative rather than a complete mechanistic account.
Significance. If the central claim holds, this is a useful and timely result: IPI exposure would be detectable from hidden states before generation, across model families and scales, and the detection signal could support a practical, utility-preserving defense. The paper has real strengths: a large trajectory corpus (~200k–280k detection points per model), strict IPI-setting splitting that avoids train/test leakage at the attack/instruction/suite level, extensive ablations (architecture, hyperparameters, training scale, token position), adaptive-attack stress tests, a cross-dataset transfer check on InjecAgent, and a reproducible code release. The authors also openly list scope boundaries and label-semantics limitations in Appendix A. These strengths make the paper worth serious consideration. However, two load-bearing points need reinforcement before the headline claims can be taken at face value: the missing input-content baseline for the probe, and the selection of the reporting layer on the same held-out evaluation set used to quote the AUROC.
major comments (2)
- [§3.2–§3.3, Figure 3] The probe features are extracted at the assistant boundary after the full prompt—including the most recent tool result—has been processed, and the binary label is defined by whether that same tool result contains an injected attack. A linear probe on hidden states can therefore achieve high AUROC by reading surface or content cues of the injected text from the residual stream, rather than by accessing a distinct latent 'exposure state.' The paper does not include a non-activation baseline, such as logistic regression on the current tool-result text (bag-of-words or tf-idf features) or mean-pooled token embeddings, evaluated on the same held-out strict split. This baseline is load-bearing for the central interpretational claim: if a text-only classifier matches or exceeds the probe, the 'latent signal' conclusion largely reduces to input classification; if the probe clearly beats the text
- [§3.3 and Appendix G.2; also Appendix I.1] One layer per model is selected by the maximum held-out strict AUROC, and the same selected-layer numbers are quoted as the headline 0.934–0.977 results. Appendix G.2 explicitly concedes that the reported number 'characterizes the strongest decodable signal ... rather than the performance of a separately evaluated layer-selection procedure.' Because the layer index is a free parameter chosen on the evaluation set, the headline numbers are oracle-layer maxima and likely overstate deployment-achievable performance. The same issue affects Table 2 and Figure 12, where L25/L34 are selected by strict AUROC. Please fix the layer using a separate calibration split or nested selection before evaluation, and report both the selection-validated AUROC and the oracle-layer AUROC, so the reader can see the cost of layer selection.
minor comments (4)
- [Figure 3] The 'Random: 0.464±0.044' annotations are confusing: the random-chance AUROC should be 0.5. Please clarify whether these are probes trained on randomly permuted labels, randomly initialized probes, or something else, and use a consistent chance line.
- [§4.2, Table 4] ASR and utility are reported as point estimates from three sampled trajectories per task. Given the relatively small number of runs for the defense experiments, please provide confidence intervals or at least the number of attack-success/utility events underlying the percentages, particularly for the near-zero ASR entries.
- [§3.4, Figure 5] The 'cross-model detection' experiment re-featurizes other models' traces with Qwen3-8B's tokenizer and applies the Qwen3-8B probe. It is not fully clear how template differences between chat formats are aligned before feature extraction. A sentence describing the token-position alignment procedure would improve reproducibility.
- [Appendix A] The limitations paragraph is honest and useful. In particular, the statement that the reported generalization does not cover fully optimized adaptive adversaries is an important caveat and should be echoed in the main text's robustness discussion, not only in the appendix.
Circularity Check
No substantive circularity; one disclosed layer-selection-on-eval issue mildly inflates the headline held-out AUROC.
specific steps
-
fitted input called prediction
[Section 3.3 / Appendix G.2]
"To characterize the strongest IPI-exposure signal in each model, we report one layer selected descriptively by held-out strict AUROC. ... The resulting best-layer AUROC thus characterizes the strongest decodable signal under this joint distributional shift, rather than the performance of a separately evaluated layer-selection procedure."
The headline held-out strict AUROC numbers (0.934–0.977 in Figure 3) are maxima over layers, where the layer is selected using the same held-out strict split on which the numbers are reported. Thus the quoted '90%+ AUROC on unseen attacks' is, by construction, a selection statistic rather than an unbiased evaluation of a fixed layer-selection protocol. This is a mild form of fitting the evaluation metric. However, the probe weights themselves are trained on a disjoint split, and the paper explicitly discloses the limitation in G.2, so this inflates but does not fabricate the result.
full rationale
The paper's core probing claim is not circular: IPI-exposure labels are defined externally from AgentDojo tool-result content, hidden states are extracted from the model's residual stream, and the linear probe is trained on a disjoint set of IPI settings and evaluated on held-out attacks, instructions, suites, and the separate InjecAgent benchmark. The defense (AGRI) is evaluated by downstream attack-success rate and utility on held-out banking/travel suites, not by the probe's training labels. The explanation framework is explicitly correlational and probe-relative, and Appendix A disclaims causal or exhaustive interpretation, so it does not reduce to a definitional equivalence. Self-citations (e.g., Dong et al. 2025a, 2026) are used only as background or related work and are not load-bearing. The only mild concern is the layer-selection procedure: the headline AUROC is the best-layer AUROC selected on the same held-out strict split quoted as the result, which the paper itself concedes in Appendix G.2. This is an evaluation-bias caveat rather than a derivation that reduces the paper's conclusions to its inputs. Overall score 2 reflects this single disclosed selection-on-evaluation issue, not a broader circularity.
Axiom & Free-Parameter Ledger
free parameters (4)
- Layer selection per model =
Qwen3-8B L34, Qwen3.5-9B L2, Qwen3.5-27B L18, GPT-oss-20B L20, Gemma-4-31B L53, GLM-5.2 L38
- AGRI gating threshold t =
0.5 (shared default)
- AGRI intervention duration =
3 subsequent assistant turns
- Probe training recipe =
lr 1e-4, batch 64, 5 epochs
axioms (5)
- domain assumption AgentDojo trajectories are representative of IPI exposure in real agent deployments.
- domain assumption The hidden state at the final prefill assistant token carries enough context to decode IPI exposure.
- ad hoc to paper The binary label 'latest tool result contains injected attack' is the right operational target for IPI exposure.
- domain assumption The LLM-as-judge CoT classification is a valid measurement of turn-level deliberation.
- standard math Standard supervised-learning assumptions for linear probes and AUROC hold.
Cite this review
Pith. "Pith review of Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure." pith.science (2026). https://pith.science/paper/7INWFZMM
@misc{pith2026260802657,
author = {Pith},
title = {Pith review of: Your Agentic LLMs Secretly Encode Latent Signals of Indirect Prompt-Injection Exposure},
year = {2026},
howpublished = {\url{https://pith.science/paper/7INWFZMM}},
note = {Machine review of arXiv:2608.02657}
}
read the original abstract
Agentic LLMs are vulnerable to indirect prompt injection (IPI) attacks, e.g., malicious side-tasks hidden in external tool results. While many efforts have sought to address the threats, little is known about the internals of agentic LLMs when they are exposed to IPI attacks, a condition which we call IPI exposure. In this paper, we study this problem in depth from three aspects. (1) Probing: Across six models, including the giant 753B-parameter GLM-5.2, simple linear probes trained on pre-generation hidden states can predict LLMs' IPI exposure. These probes achieve 90%+ AUROC on unseen attacks, agent instructions, and task suites; they exhibit high robustness under adaptive attacks and in cross-lingual settings. (2) Defense: Our CoT measurement reveals a recognition--action gap: though models encode such signals, they often fail to translate them into safe actions. We then introduce AGRI, a probe-gated reasoning-based defense that prepends anti-injection reasoning on demand. On difficult AgentDojo settings, AGRI substantially reduces attack success rate, e.g., from 34.6% to 0% on Qwen3.5-27B, while largely maintaining clean-task utility. (3) Explanation: We introduce an analysis framework that identifies natural-language explanations most strongly correlated with probe-captured signals. The resulting profiles differ across models: latent signals can align with either direct IPI-exposure claims or indirect operational cues. Code is available: https://github.com/jianshuod/IPI-exposure-signal.
Figures
Reference graph
Works this paper leans on
-
[1]
Get my drift? catching llm task drift with activation deltas
Sahar Abdelnabi, Aideen Fay, Giovanni Cherubin, Ahmed Salem, Mario Fritz, and Andrew Paverd. Get my drift? catching llm task drift with activation deltas. In SaTML, 2025
2025
-
[2]
Prompt leakage effect and mitigation strategies for multi-turn llm applications
Divyansh Agarwal, Alexander Richard Fabbri, Ben Risher, Philippe Laban, Shafiq Joty, and Chien-Sheng Wu. Prompt leakage effect and mitigation strategies for multi-turn llm applications. In EMNLP: Industry Track, 2024
2024
-
[3]
Understanding intermediate layers using linear classifier probes
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In ICLR (Workshop Track), 2017
2017
-
[4]
IPIGuard : A novel tool dependency graph-based defense against indirect prompt injection in LLM agents
Hengyu An, Jinghuai Zhang, Tianyu Du, Chunyi Zhou, Qingming Li, Tao Lin, and Shouling Ji. IPIGuard : A novel tool dependency graph-based defense against indirect prompt injection in LLM agents. In EMNLP, 2025
2025
-
[5]
Jailbreaking leading safety-aligned LLM s with simple adaptive attacks
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading safety-aligned LLM s with simple adaptive attacks. In ICLR, 2025
2025
-
[6]
Many-shot jailbreaking
Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, et al. Many-shot jailbreaking. NeurIPS, 2024
2024
-
[7]
Refusal in language models is mediated by a single direction
Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Rimsky, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In NeurIPS, 2024
2024
-
[8]
Monitoring reasoning models for misbehavior and the risks of promoting obfuscation
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi. Monitoring reasoning models for misbehavior and the risks of promoting obfuscation. arXiv preprint arXiv:2503.11926, 2025
Pith/arXiv arXiv 2025
-
[9]
Probing classifiers: Promises, shortcomings, and advances
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 2022
2022
-
[10]
Language models can explain neurons in language models
Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. https://openaipublic.blob.core.windows.net/neuron-explainer/paper/index.html, 2023
2023
-
[11]
Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy
Paul C. Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy. Thought anchors: Which LLM reasoning steps matter? In Mechanistic Interpretability Workshop at NeurIPS, 2025
2025
-
[12]
Reasoning theater: Disentangling model beliefs from chain-of-thought
Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow, Atticus Geiger, Owen Lewis, and Jack Merullo. Reasoning theater: Disentangling model beliefs from chain-of-thought. arXiv preprint arXiv:2603.05488, 2026
Pith/arXiv arXiv 2026
-
[13]
Towards monosemanticity: Decomposing language models with dictionary learning
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Ch...
2023
-
[14]
Discovering latent knowledge in language models without supervision
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. In ICLR, 2023
2023
-
[15]
Vpi-bench: Visual prompt injection attacks for computer-use agents
Tri Cao, Bennett Lim, Yue Liu, Yuan Sui, Yuexin Li, Shumin Deng, Lin Lu, Nay Oo, Shuicheng Yan, and Bryan Hooi. Vpi-bench: Visual prompt injection attacks for computer-use agents. In ICLR, 2026
2026
-
[16]
Ghostei-bench: Do mobile agents resilience to environmental injection in dynamic on-device environments? In ICLR, 2026
Chiyu Chen, Xinhao Song, Yunkai Chai, Yang Yao, Haodong Zhao, Lijun Li, Jie Li, Yan Teng, Gongshen Liu, and Yingchun Wang. Ghostei-bench: Do mobile agents resilience to environmental injection in dynamic on-device environments? In ICLR, 2026
2026
-
[17]
Struq: Defending against prompt injection with structured queries
Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. Struq: Defending against prompt injection with structured queries. In USENIX Security, 2025 a
work page 2025
-
[18]
Secalign: Defending against prompt injection with preference optimization
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, David Wagner, and Chuan Guo. Secalign: Defending against prompt injection with preference optimization. In ACM CCS, 2025 b
2025
-
[19]
Yurun Chen, Xavier Hu, Yuhan Liu, Keting Yin, Juncheng Li, Zhuosheng Zhang, and Shengyu Zhang. Harmonyguard: Toward safety and utility in web agents via adaptive policy enhancement and dual-objective optimization. arXiv preprint arXiv:2508.04010, 2025 c
Pith/arXiv arXiv 2025
-
[20]
Llamafirewall: An open source guardrail system for building secure ai agents
Sahana Chennabasappa, Cyrus Nikolaidis, Daniel Song, David Molnar, Stephanie Ding, Shengye Wan, Spencer Whitman, Lauren Deason, Nicholas Doucette, Abraham Montilla, et al. Llamafirewall: An open source guardrail system for building secure ai agents. arXiv preprint arXiv:2505.03574, 2025
Pith/arXiv arXiv 2025
-
[21]
Constitutional classifiers++: Efficient production-grade defenses against universal jailbreaks
Hoagy Cunningham, Jerry Wei, Zihan Wang, Andrew Persic, Alwin Peng, Jordan Abderrachid, Raj Agarwal, Bobby Chen, Austin Cohen, Andy Dau, et al. Constitutional classifiers++: Efficient production-grade defenses against universal jailbreaks. arXiv preprint arXiv:2601.04603, 2026
arXiv 2026
-
[22]
Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents
Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tram \`e r. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In NeurIPS Datasets and Benchmarks Track, 2024
work page 2024
-
[23]
Defeating prompt injections by design
Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, and Florian Tram \`e r. Defeating prompt injections by design. arXiv preprint arXiv:2503.18813, 2025
Pith/arXiv arXiv 2025
-
[24]
`` I ' ve decided to leak'': Probing internals behind prompt leakage intents
Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Ke Xu, Minlie Huang, Chao Zhang, and Han Qiu. `` I ' ve decided to leak'': Probing internals behind prompt leakage intents. In EMNLP, 2025 a
work page 2025
-
[25]
Safesearch: Automated red-teaming of LLM -based search agents
Jianshuo Dong, Sheng Guo, Hao Wang, Xun Chen, Zhuotao Liu, Tianwei Zhang, Ke Xu, Minlie Huang, and Han Qiu. Safesearch: Automated red-teaming of LLM -based search agents. In ICML, 2026
work page 2026
-
[26]
Memory injection attacks on llm agents via query-only interaction
Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. Memory injection attacks on llm agents via query-only interaction. In NeurIPS, 2025 b
work page 2025
-
[27]
Transcoders find interpretable llm feature circuits
Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable llm feature circuits. NeurIPS, 2024
work page 2024
-
[28]
Mateusz Dziemian, Maxwell Lin, Xiaohan Fu, Micha Nowak, Nick Winter, Eliot Jones, Andy Zou, Lama Ahmad, Kamalika Chaudhuri, Sahana Chennabasappa, et al. How vulnerable are ai agents to indirect prompt injections? insights from a large-scale public competition. arXiv preprint arXiv:2603.15714, 2026
arXiv 2026
-
[29]
WASP : Benchmarking web agent security against prompt injection attacks
Ivan Evtimov, Arman Zharmagambetov, Aaron Grattafiori, Chuan Guo, and Kamalika Chaudhuri. WASP : Benchmarking web agent security against prompt injection attacks. In NeurIPS Datasets and Benchmarks Track, 2025
work page 2025
-
[30]
Patchscopes: A unifying framework for inspecting hidden representations of language models
Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models. In ICML, 2024
work page 2024
-
[31]
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023
work page 2023
-
[32]
Agent smith: a single image can jailbreak one million multimodal llm agents exponentially fast
Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: a single image can jailbreak one million multimodal llm agents exponentially fast. In ICML, 2024
work page 2024
-
[33]
Yu He, Haozhe Zhu, Yiming Li, Shuo Shao, Hongwei Yao, Zhihao Liu, and Zhan Qin. Attriguard: Defeating indirect prompt injection in LLM agents via causal attribution of tool invocations. In USENIX Security, 2026
work page 2026
-
[34]
Internal representations as indicators of hallucinations in agent tool selection
Kait Healy, Bharathi Srinivasan, Visakh Madathil, and Jing Wu. Internal representations as indicators of hallucinations in agent tool selection. arXiv preprint arXiv:2601.05214, 2026
arXiv 2026
-
[35]
Defending against indirect prompt injection attacks with spotlighting
Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, and Emre Kiciman. Defending against indirect prompt injection attacks with spotlighting. arXiv preprint arXiv:2403.14720, 2024
Pith/arXiv arXiv 2024
-
[36]
Kuo-Han Hung, Ching-Yun Ko, Ambrish Rawat, I-Hsin Chung, Winston H. Hsu, and Pin-Yu Chen. Attention tracker: Detecting prompt injection attacks in LLM s. In NAACL (Findings), 2025
work page 2025
-
[37]
Safepath: Preventing harmful reasoning in chain-of-thought via early alignment
Wonje Jeung, Sangyeon Yoon, Minsuk Kahng, and Albert No. Safepath: Preventing harmful reasoning in chain-of-thought via early alignment. In NeurIPS, 2025
work page 2025
-
[38]
Llm internal states reveal hallucination risk faced with a query
Ziwei Ji, Delong Chen, Etsuko Ishii, Samuel Cahyawijaya, Yejin Bang, Bryan Wilie, and Pascale Fung. Llm internal states reveal hallucination risk faced with a query. In Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, 2024
work page 2024
-
[39]
The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents
Feiran Jia, Tong Wu, Xin Qin, and Anna Squicciarini. The task shield: Enforcing task alignment to defend against indirect prompt injection in llm agents. In ACL, 2025
work page 2025
-
[40]
Promptlocate: Localizing prompt injection attacks
Yuqi Jia, Yupei Liu, Zedian Shao, Jinyuan Jia, and Neil Gong. Promptlocate: Localizing prompt injection attacks. In IEEE S&P, 2026
work page 2026
-
[41]
Activation oracles: Training and evaluating llms as general-purpose activation explainers
Adam Karvonen, James Chua, Cl \'e ment Dumas, Kit Fraser-Taliente, Subhash Kantamneni, Julian Minder, Euan Ong, Arnab Sen Sharma, Daniel Wen, Owain Evans, et al. Activation oracles: Training and evaluating llms as general-purpose activation explainers. arXiv preprint arXiv:2512.15674, 2025
arXiv 2025
-
[42]
Minbeom Kim, Mihir Parmar, Phillip Wallis, Lesly Miculicich, Kyomin Jung, Krishnamurthy Dj Dvijotham, Long T. Le, and Tomas Pfister. Causalarmor: Efficient indirect prompt injection guardrails via causal attribution. In ICML, 2026
work page 2026
-
[43]
Hendryx, Summer Yue, and Zifan Wang
Priyanshu Kumar, Elaine Lau, Saranya Vijayakumar, Tu Trinh, Elaine T Chang, Vaughn Robinson, Shuyan Zhou, Matt Fredrikson, Sean M. Hendryx, Summer Yue, and Zifan Wang. Aligned LLM s are not aligned browser agents. In The Thirteenth International Conference on Learning Representations, 2025
work page 2025
-
[44]
OS -harm: A benchmark for measuring safety of computer use agents
Thomas Kuntz, Agatha Duzan, Hao Zhao, Francesco Croce, J Zico Kolter, Nicolas Flammarion, and Maksym Andriushchenko. OS -harm: A benchmark for measuring safety of computer use agents. In NeurIPS Datasets and Benchmarks Track, 2025
work page 2025
-
[45]
Measuring AI ability to complete long software tasks
Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, Megan Kinniment, Nate Rush, Sydney Von Arx, Ryan Bloom, Thomas Broadley, Haoxing Du, Brian Goodrich, Nikola Jurkovic, Luke Harold Miles, Seraphina Nix, Tao Roa Lin, Neev Parikh, David Rein, Lucas Jun Koba Sato, Hjalmar Wijk, Daniel M Ziegler, Elizabeth Barnes, and Lawren...
work page 2025
-
[46]
Efficient memory management for large language model serving with pagedattention
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In SOSP, 2023
work page 2023
-
[47]
Measuring faithfulness in chain-of-thought reasoning
Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023
Pith/arXiv arXiv 2023
-
[48]
Commercial llm agents are already vulnerable to simple yet dangerous attacks
Ang Li, Yin Zhou, Vethavikashini Chithrra Raghuram, Tom Goldstein, and Micah Goldblum. Commercial llm agents are already vulnerable to simple yet dangerous attacks. arXiv preprint arXiv:2502.08586, 2025
Pith/arXiv arXiv 2025
-
[49]
Inference-time intervention: Eliciting truthful answers from a language model
Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. NeurIPS, 2023
work page 2023
-
[50]
Yanhang Li, Zhichao Fan, and Zexin Zhuang. When AUC 0.998 is not enough: A candidate evaluation protocol for hidden-state probes of indirect prompt injection in multimodal computer-use agents. arXiv preprint arXiv:2606.22864, 2026
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[51]
Eia: Environmental injection attack on generalist web agents for privacy leakage
Zeyi Liao, Lingbo Mo, Chejian Xu, Mintong Kang, Jiawei Zhang, Chaowei Xiao, Yuan Tian, Bo Li, and Huan Sun. Eia: Environmental injection attack on generalist web agents for privacy leakage. In ICLR, 2025
work page 2025
-
[52]
Vigil: Defending llm agents against tool stream injection via verify-before-commit
Junda Lin, Zhaomeng Zhou, Zhi Zheng, Shuochen Liu, Tong Xu, Yong Chen, and Enhong Chen. Vigil: Defending llm agents against tool stream injection via verify-before-commit. arXiv preprint arXiv:2601.05755, 2026 a
arXiv 2026
-
[53]
Safeharness: Lifecycle-integrated security architecture for llm-based agent deployment
Xixun Lin, Yang Liu, Yancheng Chen, Yongxuan Wu, Yucheng Ning, Yilong Liu, Nan Sun, Shun Zhang, Bin Chong, Chuan Zhou, et al. Safeharness: Lifecycle-integrated security architecture for llm-based agent deployment. arXiv preprint arXiv:2604.13630, 2026 b
Pith/arXiv arXiv 2026
-
[54]
Traceaegis: Securing llm-based agents via hierarchical and behavioral anomaly detection
Jiahao Liu, Bonan Ruan, Xianglin Yang, Zhiwei Lin, Yan Liu, Yang Wang, Tao Wei, and Zhenkai Liang. Traceaegis: Securing llm-based agents via hierarchical and behavioral anomaly detection. arXiv preprint arXiv:2510.11203, 2025
arXiv 2025
-
[55]
Prompt injection attack against llm-integrated applications
Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, et al. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499, 2023
Pith/arXiv arXiv 2023
-
[56]
Formalizing and benchmarking prompt injection attacks and defenses
Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. Formalizing and benchmarking prompt injection attacks and defenses. In USENIX Security, 2024
work page 2024
-
[57]
Bogdan, Senthooran Rajamanoharan, and Neel Nanda
Uzay Macar, Paul C. Bogdan, Senthooran Rajamanoharan, and Neel Nanda. Thought branches: Interpreting LLM reasoning requires resampling. In ICLR, 2026
work page 2026
-
[58]
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Cand \`e s, and Tatsunori B Hashimoto. s1: Simple test-time scaling. In EMNLP, 2025
work page 2025
-
[59]
Milad Nasr, Nicholas Carlini, Chawin Sitawarin, Sander V Schulhoff, Jamie Hayes, Michael Ilie, Juliette Pluto, Shuang Song, Harsh Chaudhari, Ilia Shumailov, et al. Attacker moves second: Stronger adaptive attacks bypass defenses against llm jailbreaks and prompt injections. arXiv preprint arXiv:2510.09023, 2025
Pith/arXiv arXiv 2025
-
[60]
OpenAI. GPT-5.6 System Card . https://deploymentsafety.openai.com/gpt-5-6, 2026 a
work page 2026
-
[61]
OpenAI. GPT‑5.5 System Card . https://openai.com/index/gpt-5-5-system-card/, 2026 b
work page 2026
-
[62]
LLM s know more than they show: On the intrinsic representation of LLM hallucinations
Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLM s know more than they show: On the intrinsic representation of LLM hallucinations. In ICLR, 2025
work page 2025
-
[63]
Owasp top 10 for agentic applications for 2026, 2025
OWASP Gen AI Security Project . Owasp top 10 for agentic applications for 2026, 2025. URL https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
work page 2026
-
[64]
Latent QA : Teaching LLM s to decode activations into natural language
Alexander Pan, Lijie Chen, and Jacob Steinhardt. Latent QA : Teaching LLM s to decode activations into natural language. In ICLR, 2026
work page 2026
-
[65]
The linear representation hypothesis and the geometry of large language models
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In ICML, 2024
2024
-
[66]
Scikit-learn: Machine learning in python
Fabian Pedregosa, Ga \"e l Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12: 0 2825--2830, 2011
2011
-
[67]
Ignore previous prompt: Attack techniques for language models
F \'a bio Perez and Ian Ribeiro. Ignore previous prompt: Attack techniques for language models. In NeurIPS ML Safety Workshop, 2022
work page 2022
-
[68]
Steering llama 2 via contrastive activation addition
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In ACL, 2024
work page 2024
-
[69]
Great, now write an article about that: The crescendo multi-turn llm jailbreak attack
Mark Russinovich, Ahmed Salem, and Ronen Eldan. Great, now write an article about that: The crescendo multi-turn llm jailbreak attack. In USENIX Security, 2025
work page 2025
-
[70]
Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-Fran c ois Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, and Jordan Boyd-Graber. Ignore this title and hackaprompt: Exposing systemic vulnerabilities of llms through a global prompt hacking competition. In EMNLP, 2023
work page 2023
-
[71]
Promptarmor: Simple yet effective prompt injection defenses
Tianneng Shi, Kaijie Zhu, Zhun Wang, Yuqi Jia, Will Cai, Weida Liang, Haonan Wang, Hend Alzahrani, Joshua Lu, Kenji Kawaguchi, et al. Promptarmor: Simple yet effective prompt injection defenses. arXiv preprint arXiv:2507.15219, 2025
Pith/arXiv arXiv 2025
-
[72]
Daniel Freeman, Theodore R
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosema...
2024
-
[73]
Tensor trust: Interpretable prompt injection attacks from an online game
Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Darrell, Alan Ritter, and Stuart Russell. Tensor trust: Interpretable prompt injection attacks from an online game. In ICLR, 2024
work page 2024
-
[74]
The instruction hierarchy: Training llms to prioritize privileged instructions
Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208, 2024
Pith/arXiv arXiv 2024
-
[75]
Raccoon: Prompt extraction benchmark of llm-integrated applications
Junlin Wang, Tianyi Yang, Roy Xie, and Bhuwan Dhingra. Raccoon: Prompt extraction benchmark of llm-integrated applications. In ACL, 2024
work page 2024
-
[76]
Agentarmor: Enforcing program analysis on agent runtime trace to defend against prompt injection
Peiran Wang, Yang Liu, Yunfei Lu, Yifeng Cai, Hongbo Chen, Qingyou Yang, Jie Zhang, Jue Hong, and Ye Wu. Agentarmor: Enforcing program analysis on agent runtime trace to defend against prompt injection. arXiv preprint arXiv:2508.01249, 2025
arXiv 2025
-
[77]
Automatic layer selection for hallucination detection
Xinpeng Wang, William Cao, Andrew Gordon Wilson, and Zhe Zeng. Automatic layer selection for hallucination detection. In ICML, 2026
work page 2026
-
[78]
Defending against indirect prompt injection by instruction detection
Tongyu Wen, Chenglong Wang, Xiyuan Yang, Haoyu Tang, Yueqi Xie, Lingjuan Lyu, Zhicheng Dou, and Fangzhao Wu. Defending against indirect prompt injection by instruction detection. In EMNLP (Findings), 2025
work page 2025
-
[79]
Tong Wu, Chong Xiang, Jiachen T. Wang, G. Edward Suh, and Prateek Mittal. Effectively controlling reasoning models through thinking intervention. In NeurIPS Workshop on Socially Responsible and Trustworthy Foundation Models, 2025
work page 2025
-
[80]
Guardagent: Safeguard LLM agents via knowledge-enabled reasoning
Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, and Bo Li. Guardagent: Safeguard LLM agents via knowledge-enabled reasoning. In ICML, 2025
work page 2025
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.