Pith. sign in

REVIEW 4 major objections 4 minor 299 references

Agent security is fundamentally contextual: the same command can be a routine request or an attack depending on who authorized it, and content alone cannot tell them apart.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:00 UTC pith:HA2WGNVD

load-bearing objection A coherent reframing of agent security as contextual authorization; the core argument holds, but the empirical support is thin and the task-vs-action boundary needs a tractable objective-inference method. the 4 major comments →

arxiv 2607.22024 v1 pith:HA2WGNVD submitted 2026-07-24 cs.CR cs.AI

Agent Security Needs Redefinition through a Holistic Framework

Vincent Siu , Jingxuan He , Kyle Montgomery , Zhun Wang , Chenguang Wang , Dawn Song This is my paper
classification cs.CR cs.AI
keywords agent securitycontextual authorizationprompt injectionsource authorizationtask alignmentaction alignmentdata isolationagent benchmarks
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that agent security is being measured and defended against the wrong unit of analysis. The field currently treats an instruction's content—whether it looks malicious—as the security signal, but the same command ('delete user data,' 'send $5,000') can be either a routine request from an authorized user or a critical attack injected by untrusted content. The authors propose that security is fundamentally contextual and decompose context into four authorization properties—Source Authorization, Task Alignment, Action Alignment, and Data Isolation—that must hold jointly at every step of the agent's trajectory. They show that every injection task in two widely used agent benchmarks admits at least one plausible legitimate context, making the content-based conflation structural rather than accidental. If correct, prompt-injection defenses, benchmark scores, and compliance standards would all need reorientation around authorization context.

Core claim

The paper's central claim is that agent security is systematically misdefined by the prevailing content-based framing. It operationalizes contextual security as four jointly necessary properties: Source Authorization (who issued the command and whether that source is authorized), Task Alignment (whether the agent's governing objective is sanctioned), Action Alignment (whether each action serves that objective), and Data Isolation (whether information flows respect privilege boundaries). Under this reframing, indirect prompt injection is a Source Authorization violation, direct prompt injection and jailbreaking are Task Alignment violations, task drift is a Task Alignment violation, capabilit

What carries the argument

The central mechanism is the four-property authorization decomposition—Source Authorization, Task Alignment, Action Alignment, Data Isolation—applied continuously across the agent's trajectory rather than once at a snapshot. Each property corresponds to a classical systems-security question (authentication/confused deputy, authorization policy/least privilege, reference monitor/control-flow integrity, information-flow control/noninterference), and the paper's move is to insist these be the definitional core of agent security. The supporting observation is the 'same prompt, context decides' structure: identical action content appears as both legitimate request and attack in existing benchmark

Load-bearing premise

The framework depends on being able to infer the agent's governing objective from its trajectory; if that inference is unreliable, the distinction between Task Alignment and Action Alignment becomes post hoc labeling rather than a measurable security property.

What would settle it

A single counterexample would test the structural-conflation claim: find one injection task in a major agent benchmark whose action no authenticated user would plausibly request in any routine workflow. Separately, an empirical study showing that the same trajectory is classified as task drift by one objective-inference method and task refinement by another would undermine the Task/Action Alignment distinction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Indirect prompt injection is reclassified as a Source Authorization violation, which means defenses that scan text for malicious patterns have a structural ceiling: authorization cannot be recovered from content.
  • Snapshot benchmarks that reset context between tasks cannot evaluate Data Isolation; temporal violations like memory poisoning and cross-session credential leakage are invisible by construction.
  • Existing defenses (content filters, allowlists, memory isolation, dual-system separation) are reorganized around the property they actually approximate, giving defense research a clearer target metric.
  • The four properties provide testable predicates for compliance and audit, giving standards bodies something concrete to require of agent deployments.
  • Attacks that look different (task drift, agentic misalignment, direct injection) can share the same underlying violation, so defenses can be unified per property rather than per attack pattern.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to build evaluation suites that hold action content fixed while varying the four authorization properties, which would test whether defenders and models can in fact exploit context rather than content.
  • The framework implies that benchmark scores from content-based injection evaluations should be reinterpreted: a high attack-success rate may reflect an agent's inability to track source authority, not its susceptibility to a particular string pattern.
  • If objective-inference from trajectories proves unreliable, the Task/Action Alignment boundary would need to be anchored in something more measurable than inferred intent, perhaps explicit goal declarations or user checkpoints.
  • The reframing suggests investment should shift toward provenance tracking through neural computation and composable source labels across tool boundaries, as those are the technical prerequisites for Source Authorization.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues that agent security is fundamentally contextual, not content-based: the same surface instruction can be a legitimate user request or an attack depending on who issued it, what goal the agent is pursuing, and what information flows are involved. It proposes four properties — Source Authorization, Task Alignment, Action Alignment, and Data Isolation — that must hold jointly and be evaluated continuously across the agent's trajectory. Under this reframing, indirect prompt injection is classified as a Source Authorization violation, direct prompt injection and jailbreaking as Task Alignment violations, task drift and agentic misalignment as Task Alignment violations, capability misuse as Action Alignment, and information leakage as Data Isolation. The paper claims that snapshot benchmarks are structurally incapable of evaluating Data Isolation and that existing defenses can be reorganized by the property they approximate. The main empirical support is a manual analysis in Appendix A claiming that every injection task in AgentDojo and WASP (45 tasks) admits at least one plausible legitimate context, making the content-based conflation structural.

Significance. If the central claim is correct, the paper would provide a useful corrective to a research area that has largely organized around action content. The four-property decomposition maps cleanly onto classical systems security concepts (confused deputy, reference monitor, information flow control), and the paper makes several falsifiable, concrete claims: that indirect prompt injection is a source-authorization problem, that snapshot benchmarks cannot see Data Isolation violations, and that existing benchmark scores conflate authorized and unauthorized behavior. The strength of the paper is that it grounds a provocative position in a recognizable security vocabulary and explicitly identifies where current definitions fail. However, the position's operational value depends on two things the paper does not provide: a reliable method for inferring the agent's governing objective (needed to separate Task Alignment from Action Alignment), and a methodologically sound demonstration that the 45-task survey actually shows content-based evaluation is structurally flawed. The paper is best read as a research agenda rather than a settled framework; as such, it could be valuable after substantial

major comments (4)
  1. [§2 / Appendix A] The paper's central empirical claim — 'All 45 admit at least one such scenario' — is not supported as stated. First, the appendix, which is described as the full pairing of every injection task, contains 44 rows (26 AgentDojo, 18 WASP), not 45. The count must be reconciled. Second, the construction of 'plausible legitimate context' is the judgment of a single author, with no annotation protocol, no inter-rater reliability, and no stated criteria for what counts as plausible or legitimate. This matters because the argument that conflation is 'structural' requires the legitimate scenarios to be uncontroversially authorized; several entries depend on unstated assumptions (e.g., that an authenticated user is entitled to make a $5,000 payment, or to forward a Facebook security code to a spouse). Finally, the claim that benchmarks label attacks 'on the basis of ground truth about their origin,
  2. [§6.5] The boundary between Task Alignment and Action Alignment depends on whether the 'governing objective has shifted' — an inference the paper delegates to 'established research traction in plan recognition and trajectory analysis.' This is load-bearing: the taxonomy's usefulness as a security predicate, rather than post hoc labeling, requires that this inference be reliable in realistic LLM agent trajectories. The paper offers no method, no evaluation, and no evidence that current plan recognition can do this for language-driven agents; the same surface action (e.g., rm -rf /production/database) can be labeled Action Alignment or Task Alignment depending on the observer's reconstruction of intent. Before the four-property decomposition can serve as the definitional core, the authors should provide a concrete validation design — e.g., inter-annotator agreement on property labels for a corpus
  3. [§3 / §3.1] The paper asserts that the four properties are jointly necessary and sufficient for a secure action ('A secure agent satisfies all four simultaneously'), but this is not derived or even formally stated. Since the paper's strongest conclusions ('systematically misdefines', 'defenses are coherent') rest on this decomposition, the authors should either provide a formal model (states, actions, provenance, authorization policy) with the four checks, or explicitly scope the claim as a proposed taxonomy with open independence/sufficiency questions. As written, it is unclear whether an availability failure (e.g., a denial-of-service action that serves a legitimate task) or model-weight theft is meant to be covered by one of the four properties, and whether the properties are independent or can conflict.
  4. [§4.5 / §4] The claim that snapshot benchmarks are 'structurally incapable' of evaluating Data Isolation is too broad. If Data Isolation includes boundaries between users/tasks within a single session, a benchmark can test it by giving the agent access to User A's data and then to User B's data in the same trajectory. The impossibility holds only for the cross-session temporal component, which is not the whole of the property as defined in §3.1. The authors should define 'snapshot' precisely and either narrow the claim or show that all meaningful Data Isolation violations require cross-session persistence.
minor comments (4)
  1. [Figure 1 / §2] Figure 1 includes a SHADE Arena example, but the 45-task empirical survey in Section 2 and Appendix A covers only AgentDojo and WASP. Either add SHADE Arena tasks to Appendix A or remove the SHADE Arena row to avoid implying an analysis that is not present.
  2. [Throughout] Inconsistent spacing in 'W ASP' (should be 'WASP') appears in several places, including Table 1 and Appendix A. Also, in the references, 'V olhejn' should be 'V. Volhejn' and accents on 'Tramèr' are inconsistently applied.
  3. [§4.1] The statement that PIGuard 'reports near random performance on benign prompts with adversarial trigger words' should cite the specific table or figure in Li et al. (2025) and clarify what 'benign prompts with adversarial trigger words' means; as written, it is easy to misread as a criticism of the defense's core evaluation.
  4. [§7] Siu et al. (2026) is cited in Related Work as an example of 'current defenses rely on content filtering or static privilege constraints,' but the title 'A framework for formalizing LLM agent security' suggests it is a formalization paper, not a defense. Please clarify the relationship to avoid a misleading citation.

Circularity Check

0 steps flagged

No significant circularity: the four-property reframing is a definitional proposal grounded in classical security and an independent benchmark audit, not a fitted prediction or self-citation chain.

full rationale

The paper's central claim is a definitional/re-framing argument rather than a fitted prediction. The four properties are explicitly stipulated and mapped to classical systems security (Hardy, Saltzer & Schroeder, Denning, etc.), and the empirical basis for the claim that content-based evaluation conflates legitimate and malicious behavior is the manual 45-task audit in Appendix A. That audit is independent of the authors' prior results, and the authored self-citations (Siu et al. 2026, Wang et al. 2025, Potter et al. 2026, Song et al. 2026) appear in background or survey contexts and are not load-bearing for the contextual reframing. Statements such as 'indirect prompt injection is a Source Authorization violation' and 'snapshot benchmarks are structurally incapable of evaluating Data Isolation' are analytic consequences of the paper's own definitions; the paper itself says 'by construction,' so they are not disguised empirical results. Section 6.5 does concede that Task vs Action Alignment boundaries require inference of the agent's governing objective and asserts 'established research traction' without cited evidence; this is a real external-validity limitation for applying the taxonomy, but it is not circularity because no prediction is reduced to a fitted input, no parameter is renamed as a result, and no load-bearing conclusion rests on a self-citation. No circular step meets the evidence bar.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The paper introduces no free parameters because it does not fit a model to data. It relies on five substantive assumptions: the primacy of context, the completeness of the four-property taxonomy, the validity of the manual audit, the transferability of classical security concepts, and the tractability of inferring agent objectives. The first is the paper's starting point, the second is its definitional contribution, and the rest are load-bearing premises for its claims about benchmarks and defenses.

axioms (5)
  • domain assumption The content of an action alone cannot determine whether it is authorized; context is decisive.
    The entire argument rests on this premise, illustrated by Figure 1. It is plausible but not formally proven, and it is a philosophical claim about security semantics rather than a theorem.
  • ad hoc to paper The four properties (Source Authorization, Task Alignment, Action Alignment, Data Isolation) are jointly necessary and sufficient for secure agent action.
    Section 3 asserts this as a definitional choice. No proof is given that this decomposition is complete; it is a normative taxonomy.
  • ad hoc to paper A single author's construction of a plausible legitimate context for each of the 45 tasks is valid evidence that benchmarks conflate authorized and unauthorized behavior.
    Stated in Section 2 and Appendix A. There is no inter-rater reliability, no pre-registered criteria for 'plausible', and no independent verification.
  • domain assumption Classical systems-security models (confused deputy, reference monitor, information-flow control) transfer to LLM agents without significant loss.
    Figure 3 and Section 3.1 map the four properties to classical security notions. Whether these map cleanly to neural, stochastic agents is assumed rather than demonstrated.
  • domain assumption The governing objective of an agent can be inferred from its trajectory, making Task Alignment vs. Action Alignment distinguishable in practice.
    Acknowledged in Section 6.5 as an inference problem. The entire diagnostic value of the framework depends on this being tractable.

pith-pipeline@v1.3.0-alltime-deepseek · 15330 in / 10511 out tokens · 112968 ms · 2026-08-01T06:00:37.468639+00:00 · methodology

0 comments
read the original abstract

Agent security is widely treated as a question about action content. Defenses ask whether an instruction looks malicious. Benchmarks ask whether an agent performs a harmful sounding action. \textbf{We argue that agent security is fundamentally a contextual problem, and that the current content based framing systematically misdefines it.} A command to ``delete user data'' might be a routine administrative request or a prompt injection attacking production systems, and the content alone cannot distinguish the two. Authorization context can. Across every injection task in AgentDojo and WASP, the same action is one an authenticated user would plausibly request in a routine workflow, which makes the conflation a structural property of evaluating security through content. We operationalize contextual security through four properties that must hold jointly and be evaluated continuously across the agent's trajectory. Source Authorization asks who issued the command. Task Alignment specifies the agent's authorized objective. Action Alignment evaluates whether each action serves that objective. Data Isolation governs information flows across privilege boundaries. Under this reframing, indirect prompt injection becomes a Source Authorization violation. Snapshot benchmarks are structurally incapable of evaluating Data Isolation. Existing defenses are reorganized around the property they actually approximate. The contextual reframing changes which defenses are coherent, which evaluations measure something useful, and which attack patterns evaluation can see at all.

Figures

Figures reproduced from arXiv: 2607.22024 by Chenguang Wang, Dawn Song, Jingxuan He, Kyle Montgomery, Vincent Siu, Zhun Wang.

Figure 1
Figure 1. Figure 1: Across three major benchmarks, the same surface text is both a legitimate request and an attack depending on context. Cur￾rent evaluations cannot tell them apart. Under contextual security, the two cases are distinguished by which authorization properties hold. 1. Introduction Agent security is a critical concern as AI agents increas￾ingly operate in real environments (Pan et al., 2025; Song et al., 2026; … view at source ↗
Figure 2
Figure 2. Figure 2: The same prompt routed through the four properties under two different contexts. On the left, the deletion was requested by the authorized file owner. All four properties pass and the action is allowed. On the right, the same deletion was triggered by a payload in an untrusted document. Source Authorization and Task Alignment fail and the action is blocked. Action Alignment is evaluated against the objecti… view at source ↗
Figure 3
Figure 3. Figure 3: Each of the four contextual security properties corresponds to an authorization question that systems security has framed for decades. The contextual reframing applies mature systems security thinking to agent security, where current evaluations and defenses have treated authorization as a content question instead. injection, the user’s prompt conflicts with constraints set by the system or developer (for … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

299 extracted references · 2 canonical work pages

  1. [1]

    2025 , eprint=

    DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models , author=. 2025 , eprint=

    2025

  2. [2]

    2026 , eprint=

    GLM-5: from Vibe Coding to Agentic Engineering , author=. 2026 , eprint=

    2026

  3. [3]

    2026 , eprint=

    Kimi K2: Open Agentic Intelligence , author=. 2026 , eprint=

    2026

  4. [4]

    2026 , month =

    Introducing. 2026 , month =

    2026

  5. [5]

    2026 , howpublished =

    2026

  6. [6]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  7. [7]

    and Liskov, Barbara , title =

    Myers, Andrew C. and Liskov, Barbara , title =. 1997 , isbn =. doi:10.1145/268998.266669 , booktitle =

    arXiv 1997

  8. [8]

    Agents in the Wild: Safety, Security, and Beyond , author =

  9. [9]

    and Liskov, Barbara , title =

    Myers, Andrew C. and Liskov, Barbara , title =. 1997 , issue_date =. doi:10.1145/269005.266669 , journal =

    arXiv 1997

  10. [10]

    1988 , issue_date =

    Hardy, Norm , title =. 1988 , issue_date =. doi:10.1145/54289.871709 , journal =

    arXiv 1988

  11. [11]

    2003 , url=

    Capability Myths Demolished , author=. 2003 , url=

    2003

  12. [12]

    , title =

    Denning, Dorothy E. , title =. Commun. ACM , month = may, pages =. 1976 , issue_date =. doi:10.1145/360051.360056 , abstract =

    arXiv 1976

  13. [13]

    Zou , title =

    Sheng Liu and Haotian Ye and Lei Xing and James Y. Zou , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  14. [14]

    2025 , eprint=

    Get my drift? Catching LLM Task Drift with Activation Deltas , author=. 2025 , eprint=

    2025

  15. [15]

    2025 , eprint=

    WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks , author=. 2025 , eprint=

    2025

  16. [16]

    2025 , eprint=

    Design Patterns for Securing LLM Agents against Prompt Injections , author=. 2025 , eprint=

    2025

  17. [17]

    2025 , eprint=

    Securing AI Agents with Information-Flow Control , author=. 2025 , eprint=

    2025

  18. [18]

    2024 , eprint=

    Retrieval-Augmented Generation for Large Language Models: A Survey , author=. 2024 , eprint=

    2024

  19. [19]

    2025 , eprint=

    Dissecting Adversarial Robustness of Multimodal LM Agents , author=. 2025 , eprint=

    2025

  20. [20]

    2025 , eprint=

    Prompt Injection Attack to Tool Selection in LLM Agents , author=. 2025 , eprint=

    2025

  21. [21]

    2025 , eprint=

    Defeating Prompt Injections by Design , author=. 2025 , eprint=

    2025

  22. [22]

    2025 , eprint=

    SoK: Trust-Authorization Mismatch in LLM Agent Interactions , author=. 2025 , eprint=

    2025

  23. [23]

    2025 , eprint=

    Stay Focused: Problem Drift in Multi-Agent Debate , author=. 2025 , eprint=

    2025

  24. [24]

    2024 , eprint=

    CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models , author=. 2024 , eprint=

    2024

  25. [25]

    2026 , eprint=

    WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents , author=. 2026 , eprint=

    2026

  26. [26]

    2025 , eprint=

    Formalizing and Benchmarking Prompt Injection Attacks and Defenses , author=. 2025 , eprint=

    2025

  27. [27]

    2009 , eprint=

    Provenance as Dependency Analysis , author=. 2009 , eprint=

    2009

  28. [28]

    2024 , eprint=

    Identifying the Risks of LM Agents with an LM-Emulated Sandbox , author=. 2024 , eprint=

    2024

  29. [29]

    2023 , eprint=

    Misusing Tools in Large Language Models With Visual Adversarial Examples , author=. 2023 , eprint=

    2023

  30. [30]

    2024 , eprint=

    Automatic and Universal Prompt Injection Attacks against Large Language Models , author=. 2024 , eprint=

    2024

  31. [31]

    2025 , eprint=

    Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents , author=. 2025 , eprint=

    2025

  32. [32]

    2024 , eprint=

    AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases , author=. 2024 , eprint=

    2024

  33. [33]

    2024 , eprint=

    Goal-guided Generative Prompt Injection Attack on Large Language Models , author=. 2024 , eprint=

    2024

  34. [34]

    2025 , eprint=

    AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents , author=. 2025 , eprint=

    2025

  35. [35]

    2025 , eprint=

    Prompt Injection attack against LLM-integrated Applications , author=. 2025 , eprint=

    2025

  36. [36]

    Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models , url=

    Yi, Jingwei and Xie, Yueqi and Zhu, Bin and Kiciman, Emre and Sun, Guangzhong and Xie, Xing and Wu, Fangzhao , year=. Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models , url=. doi:10.1145/3690624.3709179 , booktitle=

    arXiv

  37. [37]

    2025 , eprint=

    Moral Alignment for LLM Agents , author=. 2025 , eprint=

    2025

  38. [38]

    PIG uard: Prompt Injection Guardrail via Mitigating Overdefense for Free

    Li, Hao and Liu, Xiaogeng and Zhang, Ning and Xiao, Chaowei. PIG uard: Prompt Injection Guardrail via Mitigating Overdefense for Free. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.1468

    doi:10.18653/v1/2025.acl-long.1468 2025

  39. [39]

    arXiv preprint arXiv:2510.01354 , year=

    WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents , author=. arXiv preprint arXiv:2510.01354 , year=

    arXiv

  40. [40]

    2025 , eprint=

    SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents , author=. 2025 , eprint=

    2025

  41. [41]

    2025 , eprint=

    Defending Against Prompt Injection With a Few DefensiveTokens , author=. 2025 , eprint=

    2025

  42. [42]

    2024 , eprint=

    Aligning LLM Agents by Learning Latent Preference from User Edits , author=. 2024 , eprint=

    2024

  43. [43]

    2025 , eprint=

    Progent: Programmable Privilege Control for LLM Agents , author=. 2025 , eprint=

    2025

  44. [44]

    2025 , eprint=

    PromptArmor: Simple yet Effective Prompt Injection Defenses , author=. 2025 , eprint=

    2025

  45. [45]

    2025 , eprint=

    PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach , author=. 2025 , eprint=

    2025

  46. [46]

    2025 , eprint=

    Agentic Misalignment: How LLMs Could Be Insider Threats , author=. 2025 , eprint=

    2025

  47. [47]

    2025 , eprint=

    Measuring Agents in Production , author=. 2025 , eprint=

    2025

  48. [48]

    2024 , eprint=

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents , author=. 2024 , eprint=

    2024

  49. [49]

    2025 , eprint=

    Agent-SafetyBench: Evaluating the Safety of LLM Agents , author=. 2025 , eprint=

    2025

  50. [50]

    2026 , eprint=

    AI Agents May Always Fall for Prompt Injections , author=. 2026 , eprint=

    2026

  51. [51]

    2026 , eprint=

    Peer-Preservation in Frontier Models , author=. 2026 , eprint=

    2026

  52. [52]

    2025 , eprint=

    AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents , author=. 2025 , eprint=

    2025

  53. [53]

    Advances in Neural Information Processing Systems , volume=

    Inference-time intervention: Eliciting truthful answers from a language model , author=. Advances in Neural Information Processing Systems , volume=

  54. [54]

    arXiv preprint arXiv:2410.03415 , year=

    Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation , author=. arXiv preprint arXiv:2410.03415 , year=

    Pith/arXiv arXiv

  55. [55]

    2024 , eprint=

    Refusal in LLMs is an Affine Function , author=. 2024 , eprint=

    2024

  56. [56]

    Advances in Neural Information Processing Systems , volume=

    Analysing the generalisation and reliability of steering vectors , author=. Advances in Neural Information Processing Systems , volume=

  57. [57]

    arXiv preprint arXiv:2502.12446 , year=

    Multi-Attribute Steering of Language Models via Targeted Intervention , author=. arXiv preprint arXiv:2502.12446 , year=

    Pith/arXiv arXiv

  58. [58]

    arXiv preprint arXiv:2504.15133 , year=

    EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models , author=. arXiv preprint arXiv:2504.15133 , year=

    Pith/arXiv arXiv

  59. [59]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback , url =

    Yuntao Bai and Andy Jones and Kamal Ndousse and Amanda Askell and Anna Chen and Nova DasSarma and Dawn Drain and Stanislav Fort and Deep Ganguli and Tom Henighan and Nicholas Joseph and Saurav Kadavath and Jackson Kernion and Tom Conerly and Sheer El-Showk and Nelson Elhage and Zac Hatfield-Dodds and Danny Hernandez and Tristan Hume and Scott Johnston and...

  60. [60]

    Zico Kolter and Matt Fredrikson , journal =

    Andy Zou and Zifan Wang and Nicholas Carlini and Milad Nasr and J. Zico Kolter and Matt Fredrikson , journal =. Universal and Transferable Adversarial Attacks on Aligned Language Models , url =

  61. [61]

    Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks , url =

    Andriushchenko, Maksym and Croce, Francesco and Flammarion, Nicolas , journal =. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks , url =

  62. [62]

    Do Anything Now

    Xinyue Shen and Zeyuan Chen and Michael Backes and Yun Shen and Yang Zhang , journal =. "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models , url =

  63. [63]

    MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots , url =

    Deng, Gelei and Liu, Yi and Li, Yuekang and Wang, Kailong and Zhang, Ying and Li, Zefeng and Wang, Haoyu and Zhang, Tianwei and Liu, Yang , booktitle =. MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots , url =. doi:10.14722/ndss.2024.24188 , publisher =

    arXiv 2024

  64. [64]

    Pappas and Eric Wong , journal =

    Patrick Chao and Alexander Robey and Edgar Dobriban and Hamed Hassani and George J. Pappas and Eric Wong , journal =. Jailbreaking Black Box Large Language Models in Twenty Queries , url =

  65. [65]

    Robust LLM safeguarding via refusal feature adversarial training , url =

    Lei Yu and Virginie Do and Karen Hambardzumyan and Nicola Cancedda , journal =. Robust LLM safeguarding via refusal feature adversarial training , url =

  66. [66]

    The Linear Representation Hypothesis and the Geometry of Large Language Models , url =

    Kiho Park and Yo Joong Choe and Victor Veitch , bibsource =. The Linear Representation Hypothesis and the Geometry of Large Language Models , url =. Forty-first International Conference on Machine Learning,

  67. [67]

    Refusal in Language Models Is Mediated by a Single Direction , url =

    Andy Arditi and Oscar Obeso and Aaquib Syed and Daniel Paleka and Nina Panickssery and Wes Gurnee and Neel Nanda , bibsource =. Refusal in Language Models Is Mediated by a Single Direction , url =. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, Dec...

    2024

  68. [68]

    Byun and Zifan Wang and Alex Mallen and Steven Basart and Sanmi Koyejo and Dawn Song and Matt Fredrikson and J

    Andy Zou and Long Phan and Sarah Chen and James Campbell and Phillip Guo and Richard Ren and Alexander Pan and Xuwang Yin and Mantas Mazeika and Ann-Kathrin Dombrowski and Shashwat Goel and Nathaniel Li and Michael J. Byun and Zifan Wang and Alex Mallen and Steven Basart and Sanmi Koyejo and Dawn Song and Matt Fredrikson and J. Zico Kolter and Dan Hendryc...

  69. [69]

    2025 , eprint=

    MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval , author=. 2025 , eprint=

    2025

  70. [70]

    2025 , eprint=

    Memory Injection Attacks on LLM Agents via Query-Only Interaction , author=. 2025 , eprint=

    2025

  71. [71]

    Diff-in-Means Concept Editing is Worst-Case Optimal: Explaining a result by

    Belrose, Nora , note =. Diff-in-Means Concept Editing is Worst-Case Optimal: Explaining a result by

  72. [72]

    2024 , eprint=

    SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models , author=. 2024 , eprint=

    2024

  73. [73]

    AI at Meta , author=

    The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation , url=. AI at Meta , author=. 2025 , month=

    2025

  74. [74]

    The Moon is Made of Marshmallows

    FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows" , author=. 2025 , eprint=

    2025

  75. [75]

    2025 , eprint=

    DarkBench: Benchmarking Dark Patterns in Large Language Models , author=. 2025 , eprint=

    2025

  76. [76]

    2024 , eprint=

    DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models , author=. 2024 , eprint=

    2024

  77. [77]

    On the Relationship between Truth and Political Bias in Language Models , url=

    Fulay, Suyash and Brannon, William and Mohanty, Shrestha and Overney, Cassandra and Poole-Dayan, Elinor and Roy, Deb and Kabbara, Jad , year=. On the Relationship between Truth and Political Bias in Language Models , url=. doi:10.18653/v1/2024.emnlp-main.508 , booktitle=

    doi:10.18653/v1/2024.emnlp-main.508 2024

  78. [78]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , year=

    ToxiGen: A Large-Scale Machine-Generated Dataset for Implicit and Adversarial Hate Speech Detection , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , year=

  79. [79]

    2025 , eprint=

    Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations , author=. 2025 , eprint=

    2025

  80. [80]

    2025 , eprint=

    Aggregate and conquer: detecting and steering LLM concepts by combining nonlinear predictors over multiple layers , author=. 2025 , eprint=

    2025

Showing first 80 references.