Pith. sign in

REVIEW 4 major objections 4 minor 299 references

Agent Security Needs Redefinition through a Holistic Framework

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Agent security is fundamentally contextual: the same command can be a routine request or an attack depending on who authorized it, and content alone cannot tell them apart.

desk verdict A coherent reframing of agent security as contextual authorization; the core argument holds, but the empirical support is thin and the task-vs-action boundary needs a tractable objective-inference method. read the letter →

arxiv 2607.22024 v1 pith:HA2WGNVD submitted 2026-07-24 cs.CR cs.AI

Vincent Siu , Jingxuan He , Kyle Montgomery , Zhun Wang , Chenguang Wang , Dawn Song This is my paper
classification cs.CRcs.AI
keywords agentsecuritycontextualauthorizationpromptinjectionsourcetaskalignmentactiondataisolationbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that agent security is being measured and defended against the wrong unit of analysis. The field currently treats an instruction's content—whether it looks malicious—as the security signal, but the same command ('delete user data,' 'send $5,000') can be either a routine request from an authorized user or a critical attack injected by untrusted content. The authors propose that security is fundamentally contextual and decompose context into four authorization properties—Source Authorization, Task Alignment, Action Alignment, and Data Isolation—that must hold jointly at every step of the agent's trajectory. They show that every injection task in two widely used agent benchmarks admits at least one plausible legitimate context, making the content-based conflation structural rather than accidental. If correct, prompt-injection defenses, benchmark scores, and compliance standards would all need reorientation around authorization context.

What carries the argument

The central mechanism is the four-property authorization decomposition—Source Authorization, Task Alignment, Action Alignment, Data Isolation—applied continuously across the agent's trajectory rather than once at a snapshot. Each property corresponds to a classical systems-security question (authentication/confused deputy, authorization policy/least privilege, reference monitor/control-flow integrity, information-flow control/noninterference), and the paper's move is to insist these be the definitional core of agent security. The supporting observation is the 'same prompt, context decides' structure: identical action content appears as both legitimate request and attack in existing benchmark

What would settle it

A single counterexample would test the structural-conflation claim: find one injection task in a major agent benchmark whose action no authenticated user would plausibly request in any routine workflow. Separately, an empirical study showing that the same trajectory is classified as task drift by one objective-inference method and task refinement by another would undermine the Task/Action Alignment distinction.

Watch

Extended reading notes

Core claim

The paper's central claim is that agent security is systematically misdefined by the prevailing content-based framing. It operationalizes contextual security as four jointly necessary properties: Source Authorization (who issued the command and whether that source is authorized), Task Alignment (whether the agent's governing objective is sanctioned), Action Alignment (whether each action serves that objective), and Data Isolation (whether information flows respect privilege boundaries). Under this reframing, indirect prompt injection is a Source Authorization violation, direct prompt injection and jailbreaking are Task Alignment violations, task drift is a Task Alignment violation, capabilit

Load-bearing premise

The framework depends on being able to infer the agent's governing objective from its trajectory; if that inference is unreliable, the distinction between Task Alignment and Action Alignment becomes post hoc labeling rather than a measurable security property.

Editorial extensions

If this is right

  • Indirect prompt injection is reclassified as a Source Authorization violation, which means defenses that scan text for malicious patterns have a structural ceiling: authorization cannot be recovered from content.
  • Snapshot benchmarks that reset context between tasks cannot evaluate Data Isolation; temporal violations like memory poisoning and cross-session credential leakage are invisible by construction.
  • Existing defenses (content filters, allowlists, memory isolation, dual-system separation) are reorganized around the property they actually approximate, giving defense research a clearer target metric.
  • The four properties provide testable predicates for compliance and audit, giving standards bodies something concrete to require of agent deployments.
  • Attacks that look different (task drift, agentic misalignment, direct injection) can share the same underlying violation, so defenses can be unified per property rather than per attack pattern.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is to build evaluation suites that hold action content fixed while varying the four authorization properties, which would test whether defenders and models can in fact exploit context rather than content.
  • The framework implies that benchmark scores from content-based injection evaluations should be reinterpreted: a high attack-success rate may reflect an agent's inability to track source authority, not its susceptibility to a particular string pattern.
  • If objective-inference from trajectories proves unreliable, the Task/Action Alignment boundary would need to be anchored in something more measurable than inferred intent, perhaps explicit goal declarations or user checkpoints.
  • The reframing suggests investment should shift toward provenance tracking through neural computation and composable source labels across tool boundaries, as those are the technical prerequisites for Source Authorization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper argues that agent security is fundamentally contextual, not content-based: the same surface instruction can be a legitimate user request or an attack depending on who issued it, what goal the agent is pursuing, and what information flows are involved. It proposes four properties — Source Authorization, Task Alignment, Action Alignment, and Data Isolation — that must hold jointly and be evaluated continuously across the agent's trajectory. Under this reframing, indirect prompt injection is classified as a Source Authorization violation, direct prompt injection and jailbreaking as Task Alignment violations, task drift and agentic misalignment as Task Alignment violations, capability misuse as Action Alignment, and information leakage as Data Isolation. The paper claims that snapshot benchmarks are structurally incapable of evaluating Data Isolation and that existing defenses can be reorganized by the property they approximate. The main empirical support is a manual analysis in Appendix A claiming that every injection task in AgentDojo and WASP (45 tasks) admits at least one plausible legitimate context, making the content-based conflation structural.

Significance. If the central claim is correct, the paper would provide a useful corrective to a research area that has largely organized around action content. The four-property decomposition maps cleanly onto classical systems security concepts (confused deputy, reference monitor, information flow control), and the paper makes several falsifiable, concrete claims: that indirect prompt injection is a source-authorization problem, that snapshot benchmarks cannot see Data Isolation violations, and that existing benchmark scores conflate authorized and unauthorized behavior. The strength of the paper is that it grounds a provocative position in a recognizable security vocabulary and explicitly identifies where current definitions fail. However, the position's operational value depends on two things the paper does not provide: a reliable method for inferring the agent's governing objective (needed to separate Task Alignment from Action Alignment), and a methodologically sound demonstration that the 45-task survey actually shows content-based evaluation is structurally flawed. The paper is best read as a research agenda rather than a settled framework; as such, it could be valuable after substantial

major comments (4)
  1. [§2 / Appendix A] The paper's central empirical claim — 'All 45 admit at least one such scenario' — is not supported as stated. First, the appendix, which is described as the full pairing of every injection task, contains 44 rows (26 AgentDojo, 18 WASP), not 45. The count must be reconciled. Second, the construction of 'plausible legitimate context' is the judgment of a single author, with no annotation protocol, no inter-rater reliability, and no stated criteria for what counts as plausible or legitimate. This matters because the argument that conflation is 'structural' requires the legitimate scenarios to be uncontroversially authorized; several entries depend on unstated assumptions (e.g., that an authenticated user is entitled to make a $5,000 payment, or to forward a Facebook security code to a spouse). Finally, the claim that benchmarks label attacks 'on the basis of ground truth about their origin,
  2. [§6.5] The boundary between Task Alignment and Action Alignment depends on whether the 'governing objective has shifted' — an inference the paper delegates to 'established research traction in plan recognition and trajectory analysis.' This is load-bearing: the taxonomy's usefulness as a security predicate, rather than post hoc labeling, requires that this inference be reliable in realistic LLM agent trajectories. The paper offers no method, no evaluation, and no evidence that current plan recognition can do this for language-driven agents; the same surface action (e.g., rm -rf /production/database) can be labeled Action Alignment or Task Alignment depending on the observer's reconstruction of intent. Before the four-property decomposition can serve as the definitional core, the authors should provide a concrete validation design — e.g., inter-annotator agreement on property labels for a corpus
  3. [§3 / §3.1] The paper asserts that the four properties are jointly necessary and sufficient for a secure action ('A secure agent satisfies all four simultaneously'), but this is not derived or even formally stated. Since the paper's strongest conclusions ('systematically misdefines', 'defenses are coherent') rest on this decomposition, the authors should either provide a formal model (states, actions, provenance, authorization policy) with the four checks, or explicitly scope the claim as a proposed taxonomy with open independence/sufficiency questions. As written, it is unclear whether an availability failure (e.g., a denial-of-service action that serves a legitimate task) or model-weight theft is meant to be covered by one of the four properties, and whether the properties are independent or can conflict.
  4. [§4.5 / §4] The claim that snapshot benchmarks are 'structurally incapable' of evaluating Data Isolation is too broad. If Data Isolation includes boundaries between users/tasks within a single session, a benchmark can test it by giving the agent access to User A's data and then to User B's data in the same trajectory. The impossibility holds only for the cross-session temporal component, which is not the whole of the property as defined in §3.1. The authors should define 'snapshot' precisely and either narrow the claim or show that all meaningful Data Isolation violations require cross-session persistence.
minor comments (4)
  1. [Figure 1 / §2] Figure 1 includes a SHADE Arena example, but the 45-task empirical survey in Section 2 and Appendix A covers only AgentDojo and WASP. Either add SHADE Arena tasks to Appendix A or remove the SHADE Arena row to avoid implying an analysis that is not present.
  2. [Throughout] Inconsistent spacing in 'W ASP' (should be 'WASP') appears in several places, including Table 1 and Appendix A. Also, in the references, 'V olhejn' should be 'V. Volhejn' and accents on 'Tramèr' are inconsistently applied.
  3. [§4.1] The statement that PIGuard 'reports near random performance on benign prompts with adversarial trigger words' should cite the specific table or figure in Li et al. (2025) and clarify what 'benign prompts with adversarial trigger words' means; as written, it is easy to misread as a criticism of the defense's core evaluation.
  4. [§7] Siu et al. (2026) is cited in Related Work as an example of 'current defenses rely on content filtering or static privilege constraints,' but the title 'A framework for formalizing LLM agent security' suggests it is a formalization paper, not a defense. Please clarify the relationship to avoid a misleading citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the four-property reframing is a definitional proposal grounded in classical security and an independent benchmark audit, not a fitted prediction or self-citation chain.

full rationale

The paper's central claim is a definitional/re-framing argument rather than a fitted prediction. The four properties are explicitly stipulated and mapped to classical systems security (Hardy, Saltzer & Schroeder, Denning, etc.), and the empirical basis for the claim that content-based evaluation conflates legitimate and malicious behavior is the manual 45-task audit in Appendix A. That audit is independent of the authors' prior results, and the authored self-citations (Siu et al. 2026, Wang et al. 2025, Potter et al. 2026, Song et al. 2026) appear in background or survey contexts and are not load-bearing for the contextual reframing. Statements such as 'indirect prompt injection is a Source Authorization violation' and 'snapshot benchmarks are structurally incapable of evaluating Data Isolation' are analytic consequences of the paper's own definitions; the paper itself says 'by construction,' so they are not disguised empirical results. Section 6.5 does concede that Task vs Action Alignment boundaries require inference of the agent's governing objective and asserts 'established research traction' without cited evidence; this is a real external-validity limitation for applying the taxonomy, but it is not circularity because no prediction is reduced to a fitted input, no parameter is renamed as a result, and no load-bearing conclusion rests on a self-citation. No circular step meets the evidence bar.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no free parameters because it does not fit a model to data. It relies on five substantive assumptions: the primacy of context, the completeness of the four-property taxonomy, the validity of the manual audit, the transferability of classical security concepts, and the tractability of inferring agent objectives. The first is the paper's starting point, the second is its definitional contribution, and the rest are load-bearing premises for its claims about benchmarks and defenses.

assumptions (5)
  • domain assumption The content of an action alone cannot determine whether it is authorized; context is decisive.
    The entire argument rests on this premise, illustrated by Figure 1. It is plausible but not formally proven, and it is a philosophical claim about security semantics rather than a theorem.
  • ad hoc to paper The four properties (Source Authorization, Task Alignment, Action Alignment, Data Isolation) are jointly necessary and sufficient for secure agent action.
    Section 3 asserts this as a definitional choice. No proof is given that this decomposition is complete; it is a normative taxonomy.
  • ad hoc to paper A single author's construction of a plausible legitimate context for each of the 45 tasks is valid evidence that benchmarks conflate authorized and unauthorized behavior.
    Stated in Section 2 and Appendix A. There is no inter-rater reliability, no pre-registered criteria for 'plausible', and no independent verification.
  • domain assumption Classical systems-security models (confused deputy, reference monitor, information-flow control) transfer to LLM agents without significant loss.
    Figure 3 and Section 3.1 map the four properties to classical security notions. Whether these map cleanly to neural, stochastic agents is assumed rather than demonstrated.
  • domain assumption The governing objective of an agent can be inferred from its trajectory, making Task Alignment vs. Action Alignment distinguishable in practice.
    Acknowledged in Section 6.5 as an inference problem. The entire diagnostic value of the framework depends on this being tractable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agent Security Needs Redefinition through a Holistic Framework." pith.science (2026). https://pith.science/paper/HA2WGNVD

@misc{pith2026260722024,
  author       = {Pith},
  title        = {Pith review of: Agent Security Needs Redefinition through a Holistic Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HA2WGNVD}},
  note         = {Machine review of arXiv:2607.22024}
}
read the original abstract

Agent security is widely treated as a question about action content. Defenses ask whether an instruction looks malicious. Benchmarks ask whether an agent performs a harmful sounding action. \textbf{We argue that agent security is fundamentally a contextual problem, and that the current content based framing systematically misdefines it.} A command to ``delete user data'' might be a routine administrative request or a prompt injection attacking production systems, and the content alone cannot distinguish the two. Authorization context can. Across every injection task in AgentDojo and WASP, the same action is one an authenticated user would plausibly request in a routine workflow, which makes the conflation a structural property of evaluating security through content. We operationalize contextual security through four properties that must hold jointly and be evaluated continuously across the agent's trajectory. Source Authorization asks who issued the command. Task Alignment specifies the agent's authorized objective. Action Alignment evaluates whether each action serves that objective. Data Isolation governs information flows across privilege boundaries. Under this reframing, indirect prompt injection becomes a Source Authorization violation. Snapshot benchmarks are structurally incapable of evaluating Data Isolation. Existing defenses are reorganized around the property they actually approximate. The contextual reframing changes which defenses are coherent, which evaluations measure something useful, and which attack patterns evaluation can see at all.

Figures

Figures reproduced from arXiv: 2607.22024 by the authors.

Figure 1
Figure 1. Across three major benchmarks, the same surface text is both a legitimate request and an attack depending on context. Cur￾rent evaluations cannot tell them apart. Under contextual security, the two cases are distinguished by which authorization properties hold. 1. Introduction Agent security is a critical concern as AI agents increas￾ingly operate in real environments (Pan et al., 2025; Song et al., 2026; OWASP GenA… view at source ↗
Figure 2
Figure 2. The same prompt routed through the four properties under two different contexts. On the left, the deletion was requested by the authorized file owner. All four properties pass and the action is allowed. On the right, the same deletion was triggered by a payload in an untrusted document. Source Authorization and Task Alignment fail and the action is blocked. Action Alignment is evaluated against the objective the age… view at source ↗
Figure 3
Figure 3. Each of the four contextual security properties corresponds to an authorization question that systems security has framed for decades. The contextual reframing applies mature systems security thinking to agent security, where current evaluations and defenses have treated authorization as a content question instead. injection, the user’s prompt conflicts with constraints set by the system or developer (for example, “… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

299 extracted references · 2 canonical work pages

  1. [1]

    2025 , eprint=

    DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models , author=. 2025 , eprint=

    2025

  2. [2]

    2026 , eprint=

    GLM-5: from Vibe Coding to Agentic Engineering , author=. 2026 , eprint=

    2026

  3. [3]

    2026 , eprint=

    Kimi K2: Open Agentic Intelligence , author=. 2026 , eprint=

    2026

  4. [4]

    2026 , month =

    Introducing. 2026 , month =

    2026

  5. [5]

    2026 , howpublished =

    2026

  6. [6]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  7. [7]

    and Liskov, Barbara , title =

    Myers, Andrew C. and Liskov, Barbara , title =. 1997 , isbn =. doi:10.1145/268998.266669 , booktitle =

    arXiv 1997

  8. [8]

    Agents in the Wild: Safety, Security, and Beyond , author =

Show all 299 references
  1. [9]

    and Liskov, Barbara , title =

    Myers, Andrew C. and Liskov, Barbara , title =. 1997 , issue_date =. doi:10.1145/269005.266669 , journal =

    1997

  2. [10]

    1988 , issue_date =

    Hardy, Norm , title =. 1988 , issue_date =. doi:10.1145/54289.871709 , journal =

    1988

  3. [11]

    2003 , url=

    Capability Myths Demolished , author=. 2003 , url=

    2003

  4. [12]

    , title =

    Denning, Dorothy E. , title =. Commun. ACM , month = may, pages =. 1976 , issue_date =. doi:10.1145/360051.360056 , abstract =

    1976

  5. [13]

    Zou , title =

    Sheng Liu and Haotian Ye and Lei Xing and James Y. Zou , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

    2024

  6. [14]

    2025 , eprint=

    Get my drift? Catching LLM Task Drift with Activation Deltas , author=. 2025 , eprint=

    2025

  7. [15]

    2025 , eprint=

    WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks , author=. 2025 , eprint=

    2025

  8. [16]

    2025 , eprint=

    Design Patterns for Securing LLM Agents against Prompt Injections , author=. 2025 , eprint=

    2025

  9. [17]

    2025 , eprint=

    Securing AI Agents with Information-Flow Control , author=. 2025 , eprint=

    2025

  10. [18]

    2024 , eprint=

    Retrieval-Augmented Generation for Large Language Models: A Survey , author=. 2024 , eprint=

    2024

  11. [19]

    2025 , eprint=

    Dissecting Adversarial Robustness of Multimodal LM Agents , author=. 2025 , eprint=

    2025

  12. [20]

    2025 , eprint=

    Prompt Injection Attack to Tool Selection in LLM Agents , author=. 2025 , eprint=

    2025

  13. [21]

    2025 , eprint=

    Defeating Prompt Injections by Design , author=. 2025 , eprint=

    2025

  14. [22]

    2025 , eprint=

    SoK: Trust-Authorization Mismatch in LLM Agent Interactions , author=. 2025 , eprint=

    2025

  15. [23]

    2025 , eprint=

    Stay Focused: Problem Drift in Multi-Agent Debate , author=. 2025 , eprint=

    2025

  16. [24]

    2024 , eprint=

    CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Models , author=. 2024 , eprint=

    2024

  17. [25]

    2026 , eprint=

    WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents , author=. 2026 , eprint=

    2026

  18. [26]

    2025 , eprint=

    Formalizing and Benchmarking Prompt Injection Attacks and Defenses , author=. 2025 , eprint=

    2025

  19. [27]

    2009 , eprint=

    Provenance as Dependency Analysis , author=. 2009 , eprint=

    2009

  20. [28]

    2024 , eprint=

    Identifying the Risks of LM Agents with an LM-Emulated Sandbox , author=. 2024 , eprint=

    2024

  21. [29]

    2023 , eprint=

    Misusing Tools in Large Language Models With Visual Adversarial Examples , author=. 2023 , eprint=

    2023

  22. [30]

    2024 , eprint=

    Automatic and Universal Prompt Injection Attacks against Large Language Models , author=. 2024 , eprint=

    2024

  23. [31]

    2025 , eprint=

    Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents , author=. 2025 , eprint=

    2025

  24. [32]

    2024 , eprint=

    AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases , author=. 2024 , eprint=

    2024

  25. [33]

    2024 , eprint=

    Goal-guided Generative Prompt Injection Attack on Large Language Models , author=. 2024 , eprint=

    2024

  26. [34]

    2025 , eprint=

    AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents , author=. 2025 , eprint=

    2025

  27. [35]

    2025 , eprint=

    Prompt Injection attack against LLM-integrated Applications , author=. 2025 , eprint=

    2025

  28. [36]

    Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models , url=

    Yi, Jingwei and Xie, Yueqi and Zhu, Bin and Kiciman, Emre and Sun, Guangzhong and Xie, Xing and Wu, Fangzhao , year=. Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models , url=. doi:10.1145/3690624.3709179 , booktitle=

  29. [37]

    2025 , eprint=

    Moral Alignment for LLM Agents , author=. 2025 , eprint=

    2025

  30. [38]

    PIG uard: Prompt Injection Guardrail via Mitigating Overdefense for Free

    Li, Hao and Liu, Xiaogeng and Zhang, Ning and Xiao, Chaowei. PIG uard: Prompt Injection Guardrail via Mitigating Overdefense for Free. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.ac...

    2025 doi

  31. [39]

    arXiv preprint arXiv:2510.01354 , year=

    WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents , author=. arXiv preprint arXiv:2510.01354 , year=

  32. [40]

    2025 , eprint=

    SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents , author=. 2025 , eprint=

    2025

  33. [41]

    2025 , eprint=

    Defending Against Prompt Injection With a Few DefensiveTokens , author=. 2025 , eprint=

    2025

  34. [42]

    2024 , eprint=

    Aligning LLM Agents by Learning Latent Preference from User Edits , author=. 2024 , eprint=

    2024

  35. [43]

    2025 , eprint=

    Progent: Programmable Privilege Control for LLM Agents , author=. 2025 , eprint=

    2025

  36. [44]

    2025 , eprint=

    PromptArmor: Simple yet Effective Prompt Injection Defenses , author=. 2025 , eprint=

    2025

  37. [45]

    2025 , eprint=

    PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach , author=. 2025 , eprint=

    2025

  38. [46]

    2025 , eprint=

    Agentic Misalignment: How LLMs Could Be Insider Threats , author=. 2025 , eprint=

    2025

  39. [47]

    2025 , eprint=

    Measuring Agents in Production , author=. 2025 , eprint=

    2025

  40. [48]

    2024 , eprint=

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents , author=. 2024 , eprint=

    2024

  41. [49]

    2025 , eprint=

    Agent-SafetyBench: Evaluating the Safety of LLM Agents , author=. 2025 , eprint=

    2025

  42. [50]

    2026 , eprint=

    AI Agents May Always Fall for Prompt Injections , author=. 2026 , eprint=

    2026

  43. [51]

    2026 , eprint=

    Peer-Preservation in Frontier Models , author=. 2026 , eprint=

    2026

  44. [52]

    2025 , eprint=

    AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents , author=. 2025 , eprint=

    2025

  45. [53]

    Advances in Neural Information Processing Systems , volume=

    Inference-time intervention: Eliciting truthful answers from a language model , author=. Advances in Neural Information Processing Systems , volume=

  46. [54]

    arXiv preprint arXiv:2410.03415 , year=

    Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation , author=. arXiv preprint arXiv:2410.03415 , year=

    arXiv

  47. [55]

    2024 , eprint=

    Refusal in LLMs is an Affine Function , author=. 2024 , eprint=

    2024

  48. [56]

    Advances in Neural Information Processing Systems , volume=

    Analysing the generalisation and reliability of steering vectors , author=. Advances in Neural Information Processing Systems , volume=

  49. [57]

    arXiv preprint arXiv:2502.12446 , year=

    Multi-Attribute Steering of Language Models via Targeted Intervention , author=. arXiv preprint arXiv:2502.12446 , year=

    arXiv

  50. [58]

    arXiv preprint arXiv:2504.15133 , year=

    EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models , author=. arXiv preprint arXiv:2504.15133 , year=

    arXiv

  51. [59]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback , url =

    Yuntao Bai and Andy Jones and Kamal Ndousse and Amanda Askell and Anna Chen and Nova DasSarma and Dawn Drain and Stanislav Fort and Deep Ganguli and Tom Henighan and Nicholas Joseph and Saurav Kadavath and Jackson Kernion and Tom Conerly and Sheer El-Showk and Nelson Elhage an...

  52. [60]

    Zico Kolter and Matt Fredrikson , journal =

    Andy Zou and Zifan Wang and Nicholas Carlini and Milad Nasr and J. Zico Kolter and Matt Fredrikson , journal =. Universal and Transferable Adversarial Attacks on Aligned Language Models , url =

  53. [61]

    Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks , url =

    Andriushchenko, Maksym and Croce, Francesco and Flammarion, Nicolas , journal =. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks , url =

  54. [62]

    Do Anything Now

    Xinyue Shen and Zeyuan Chen and Michael Backes and Yun Shen and Yang Zhang , journal =. "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models , url =

  55. [63]

    MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots , url =

    Deng, Gelei and Liu, Yi and Li, Yuekang and Wang, Kailong and Zhang, Ying and Li, Zefeng and Wang, Haoyu and Zhang, Tianwei and Liu, Yang , booktitle =. MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots , url =. doi:10.14722/ndss.2024.24188 , publisher =

    2024

  56. [64]

    Pappas and Eric Wong , journal =

    Patrick Chao and Alexander Robey and Edgar Dobriban and Hamed Hassani and George J. Pappas and Eric Wong , journal =. Jailbreaking Black Box Large Language Models in Twenty Queries , url =

  57. [65]

    Robust LLM safeguarding via refusal feature adversarial training , url =

    Lei Yu and Virginie Do and Karen Hambardzumyan and Nicola Cancedda , journal =. Robust LLM safeguarding via refusal feature adversarial training , url =

  58. [66]

    The Linear Representation Hypothesis and the Geometry of Large Language Models , url =

    Kiho Park and Yo Joong Choe and Victor Veitch , bibsource =. The Linear Representation Hypothesis and the Geometry of Large Language Models , url =. Forty-first International Conference on Machine Learning,

  59. [67]

    Refusal in Language Models Is Mediated by a Single Direction , url =

    Andy Arditi and Oscar Obeso and Aaquib Syed and Daniel Paleka and Nina Panickssery and Wes Gurnee and Neel Nanda , bibsource =. Refusal in Language Models Is Mediated by a Single Direction , url =. Advances in Neural Information Processing Systems 38: Annual Conference on Neur...

    2024

  60. [68]

    Byun and Zifan Wang and Alex Mallen and Steven Basart and Sanmi Koyejo and Dawn Song and Matt Fredrikson and J

    Andy Zou and Long Phan and Sarah Chen and James Campbell and Phillip Guo and Richard Ren and Alexander Pan and Xuwang Yin and Mantas Mazeika and Ann-Kathrin Dombrowski and Shashwat Goel and Nathaniel Li and Michael J. Byun and Zifan Wang and Alex Mallen and Steven Basart and S...

  61. [69]

    2025 , eprint=

    MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval , author=. 2025 , eprint=

    2025

  62. [70]

    2025 , eprint=

    Memory Injection Attacks on LLM Agents via Query-Only Interaction , author=. 2025 , eprint=

    2025

  63. [71]

    Diff-in-Means Concept Editing is Worst-Case Optimal: Explaining a result by

    Belrose, Nora , note =. Diff-in-Means Concept Editing is Worst-Case Optimal: Explaining a result by

  64. [72]

    2024 , eprint=

    SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models , author=. 2024 , eprint=

    2024

  65. [73]

    AI at Meta , author=

    The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation , url=. AI at Meta , author=. 2025 , month=

    2025

  66. [74]

    The Moon is Made of Marshmallows

    FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows" , author=. 2025 , eprint=

    2025

  67. [75]

    2025 , eprint=

    DarkBench: Benchmarking Dark Patterns in Large Language Models , author=. 2025 , eprint=

    2025

  68. [76]

    2024 , eprint=

    DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models , author=. 2024 , eprint=

    2024

  69. [77]

    On the Relationship between Truth and Political Bias in Language Models , url=

    Fulay, Suyash and Brannon, William and Mohanty, Shrestha and Overney, Cassandra and Poole-Dayan, Elinor and Roy, Deb and Kabbara, Jad , year=. On the Relationship between Truth and Political Bias in Language Models , url=. doi:10.18653/v1/2024.emnlp-main.508 , booktitle=

    2024 doi

  70. [78]

    Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , year=

    ToxiGen: A Large-Scale Machine-Generated Dataset for Implicit and Adversarial Hate Speech Detection , author=. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics , year=

  71. [79]

    2025 , eprint=

    Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations , author=. 2025 , eprint=

    2025

  72. [80]

    2025 , eprint=

    Aggregate and conquer: detecting and steering LLM concepts by combining nonlinear predictors over multiple layers , author=. 2025 , eprint=

    2025

  73. [81]

    2025 , eprint=

    Shifting Perspectives: Steering Vector Ensembles for Robust Bias Mitigation in LLMs , author=. 2025 , eprint=

    2025

  74. [82]

    2025 , eprint=

    AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders , author=. 2025 , eprint=

    2025

  75. [83]

    Hidden Persuaders: LLM s' Political Leaning and Their Influence on Voters

    Potter, Yujin and Lai, Shiyang and Kim, Junsol and Evans, James and Song, Dawn. Hidden Persuaders: LLM s' Political Leaning and Their Influence on Voters. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. doi:10.18653/v1/2024.emnlp-main.244

    2024 doi

  76. [84]

    International Conference on Machine Learning , pages=

    A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicity , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  77. [85]

    2024 , eprint=

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author=. 2024 , eprint=

    2024

  78. [86]

    arXiv preprint arXiv:2009.11462 , year=

    Realtoxicityprompts: Evaluating neural toxic degeneration in language models , author=. arXiv preprint arXiv:2009.11462 , year=

    2009 arXiv

  79. [87]

    2024 , eprint=

    All Languages Matter: On the Multilingual Safety of Large Language Models , author=. 2024 , eprint=

    2024

  80. [88]

    2025 , eprint=

    RepIt: Representing Isolated Targets to Steer Language Models , author=. 2025 , eprint=

    2025

  81. [89]

    COSMIC : Generalized Refusal Direction Identification in LLM Activations

    Siu, Vincent and Crispino, Nicholas and Yu, Zihao and Pan, Sam and Wang, Zhun and Liu, Yang and Song, Dawn and Wang, Chenguang. COSMIC : Generalized Refusal Direction Identification in LLM Activations. Findings of the Association for Computational Linguistics: ACL 2025. 2025. ...

    2025 doi

  82. [90]

    BBQ : A hand-built bias benchmark for question answering

    Parrish, Alicia and Chen, Angelica and Nangia, Nikita and Padmakumar, Vishakh and Phang, Jason and Thompson, Jana and Htut, Phu Mon and Bowman, Samuel. BBQ : A hand-built bias benchmark for question answering. Findings of the Association for Computational Linguistics: ACL 2022...

    2022 doi

  83. [91]

    2025 , eprint=

    HalluLens: LLM Hallucination Benchmark , author=. 2025 , eprint=

    2025

  84. [92]

    Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation , url =

    Yangsibo Huang and Samyak Gupta and Mengzhou Xia and Kai Li and Danqi Chen , bibsource =. Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation , url =. The Twelfth International Conference on Learning Representations,

  85. [93]

    Hashimoto , howpublished =

    Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto , howpublished =. GitHub repository , publisher =

  86. [94]

    2025 , eprint=

    Qwen2.5 Technical Report , author=. 2025 , eprint=

    2025

  87. [95]

    Aaron Grattafiori and Abhimanyu Dubey and Abhinav Jauhri and Abhinav Pandey and Abhishek Kadian and Ahmad Al-Dahle and Aiesha Letman and Akhil Mathur and Alan Schelten and Alex Vaughan and Amy Yang and Angela Fan and Anirudh Goyal and Anthony Hartshorn and Aobo Yang and Archi ...

  88. [96]

    Gemma Team and Morgane Riviere and Shreya Pathak and Pier Giuseppe Sessa and Cassidy Hardin and Surya Bhupatiraju and Léonard Hussenot and Thomas Mesnard and Bobak Shahriari and Alexandre Ramé and Johan Ferret and Peter Liu and Pouya Tafti and Abe Friesen and Michelle Casbon a...

  89. [97]

    Gemma Team and Thomas Mesnard and Cassidy Hardin and Robert Dadashi and Surya Bhupatiraju and Shreya Pathak and Laurent Sifre and Morgane Rivière and Mihir Sanjay Kale and Juliette Love and Pouya Tafti and Léonard Hussenot and Pier Giuseppe Sessa and Aakanksha Chowdhery and Ad...

  90. [98]

    Highway Networks , url =

    Rupesh Kumar Srivastava and Klaus Greff and Jürgen Schmidhuber , journal =. Highway Networks , url =

  91. [99]

    Gomez and Lukasz Kaiser and Illia Polosukhin , bibsource =

    Ashish Vaswani and Noam Shazeer and Niki Parmar and Jakob Uszkoreit and Llion Jones and Aidan N. Gomez and Lukasz Kaiser and Illia Polosukhin , bibsource =. Attention is All you Need , url =. Advances in Neural Information Processing Systems 30: Annual Conference on Neural Inf...

    2017

  92. [100]

    Transformer Feed-Forward Layers Are Key-Value Memories , url =

    Geva, Mor and Schuster, Roei and Berant, Jonathan and Levy, Omer , booktitle =. Transformer Feed-Forward Layers Are Key-Value Memories , url =. doi:10.18653/v1/2021.emnlp-main.446 , editor =

    2021 doi

  93. [101]

    Neuron-Level Knowledge Attribution in Large Language Models , url =

    Zeping Yu and Sophia Ananiadou , journal =. Neuron-Level Knowledge Attribution in Large Language Models , url =

  94. [102]

    A Mathematical Framework for Transformer Circuits , year =

    Elhage, Nelson and Nanda, Neel and Olsson, Catherine and Henighan, Tom and Joseph, Nicholas and Mann, Ben and Askell, Amanda and Bai, Yuntao and Chen, Anna and Conerly, Tom and DasSarma, Nova and Drain, Dawn and Ganguli, Deep and Hatfield-Dodds, Zac and Hernandez, Danny and Jo...

  95. [103]

    In-context Learning and Induction Heads , url =

    Catherine Olsson and Nelson Elhage and Neel Nanda and Nicholas Joseph and Nova DasSarma and Tom Henighan and Ben Mann and Amanda Askell and Yuntao Bai and Anna Chen and Tom Conerly and Dawn Drain and Deep Ganguli and Zac Hatfield-Dodds and Danny Hernandez and Scott Johnston an...

  96. [104]

    Vazquez and Ulisse Mini and Monte MacDiarmid , journal =

    Alexander Matt Turner and Lisa Thiergart and Gavin Leech and David Udell and Juan J. Vazquez and Ulisse Mini and Monte MacDiarmid , journal =. Steering Language Models With Activation Engineering , url =

  97. [105]

    Forsyth and Dan Hendrycks , bibsource =

    Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David A. Forsyth and Dan Hendrycks , bibsource =. HarmBench:. Forty-first International Conference on Machine Learning,

  98. [106]

    Mantas Mazeika and Andy Zou and Norman Mu and Long Phan and Zifan Wang and Chunru Yu and Adam Khoja and Fengqing Jiang and Aidan O'Gara and Ellie Sakhaee and Zhen Xiang and Arezoo Rajabi and Dan Hendrycks and Radha Poovendran and Bo Li and David Forsyth , booktitle =

  99. [107]

    Steering Llama 2 via Contrastive Activation Addition , url =

    Nina Panickssery and Nick Gabrieli and Julian Schulz and Meg Tong and Evan Hubinger and Alexander Matt Turner , journal =. Steering Llama 2 via Contrastive Activation Addition , url =

  100. [108]

    Universal Adversarial Triggers Are Not Universal , url =

    Nicholas Meade and Arkil Patel and Siva Reddy , journal =. Universal Adversarial Triggers Are Not Universal , url =

  101. [109]

    Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! , url =

    Xiangyu Qi and Yi Zeng and Tinghao Xie and Pin. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! , url =. The Twelfth International Conference on Learning Representations,

  102. [110]

    Training Verifiers to Solve Math Word Problems , url =

    Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and Chen, Mark and Jun, Heewoo and Kaiser, Lukasz and Plappert, Matthias and Tworek, Jerry and Hilton, Jacob and Nakano, Reiichiro and Hesse, Christopher and Schulman, John , journal =. Training Verifiers to Solve Math Wo...

  103. [111]

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , url =

    Peter Clark and Isaac Cowhey and Oren Etzioni and Tushar Khot and Ashish Sabharwal and Carissa Schoenick and Oyvind Tafjord , journal =. Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , url =

  104. [112]

    doi:10.18653/v1/2022.acl-long.229 , editor =

    Lin, Stephanie and Hilton, Jacob and Evans, Owain , booktitle =. doi:10.18653/v1/2022.acl-long.229 , editor =

    2022 doi

  105. [113]

    Linguistic Regularities in Continuous Space Word Representations , url =

    Mikolov, Tomas and Yih, Wen-tau and Zweig, Geoffrey , booktitle =. Linguistic Regularities in Continuous Space Word Representations , url =

  106. [114]

    Lee and Inkit Padhi and Karthikeyan Natesan Ramamurthy and Erik Miehling and Pierre Dognin and Manish Nagireddy and Amit Dhurandhar , journal =

    Bruce W. Lee and Inkit Padhi and Karthikeyan Natesan Ramamurthy and Erik Miehling and Pierre Dognin and Manish Nagireddy and Amit Dhurandhar , journal =. Programming Refusal with Conditional Activation Steering , url =

  107. [115]

    Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , url =

    Tolga Bolukbasi and Kai. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings , url =. Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain , editor =

    2016

  108. [116]

    Transformers: State-of-the-Art Natural Language Processing , url =

    Wolf, Thomas and Debut, Lysandre and Sanh, Victor and Chaumond, Julien and Delangue, Clement and Moi, Anthony and Cistac, Pierric and Rault, Tim and Louf, Remi and Funtowicz, Morgan and Davison, Joe and Shleifer, Sam and von Platen, Patrick and Ma, Clara and Jernite, Yacine an...

    2020

  109. [117]

    Prompt-driven

    Zheng, Chujie and Yin, Fan and Zhou, Hao and Meng, Fandong and Zhou, Jie and Chang, Kai-Wei and Huang, Minlie and Peng, Nanyun , journal =. Prompt-driven

  110. [118]

    Towards Inference-time Category-wise Safety Steering for Large Language Models , url =

    Amrita Bhattacharjee and Shaona Ghosh and Traian Rebedea and Christopher Parisien , journal =. Towards Inference-time Category-wise Safety Steering for Large Language Models , url =

  111. [119]

    Model editing as a robust and denoised variant of DPO: A case study on toxicity , year =

    Uppaal, Rheeya and Dey, Apratim and He, Yiting and Zhong, Yiqiao and Hu, Junjie , booktitle =. Model editing as a robust and denoised variant of DPO: A case study on toxicity , year =

  112. [120]

    Cohen , bibsource =

    Yifu Qiu and Zheng Zhao and Yftah Ziser and Anna Korhonen and Edoardo Maria Ponti and Shay B. Cohen , bibsource =. Spectral Editing of Activations for Large Language Model Alignment , url =. Advances in Neural Information Processing Systems 38: Annual Conference on Neural Info...

    2024

  113. [121]

    Tom B. Brown and Benjamin Mann and Nick Ryder and Melanie Subbiah and Jared Kaplan and Prafulla Dhariwal and Arvind Neelakantan and Pranav Shyam and Girish Sastry and Amanda Askell and Sandhini Agarwal and Ariel Herbert. Language Models are Few-Shot Learners , url =. Advances ...

    2020

  114. [122]

    Long Ouyang and Jeffrey Wu and Xu Jiang and Diogo Almeida and Carroll L. Wainwright and Pamela Mishkin and Chong Zhang and Sandhini Agarwal and Katarina Slama and Alex Ray and John Schulman and Jacob Hilton and Fraser Kelton and Luke Miller and Maddie Simens and Amanda Askell ...

    2022

  115. [123]

    The Twelfth International Conference on Learning Representations,

    Chao Chen and Kai Liu and Ze Chen and Yi Gu and Yue Wu and Mingyuan Tao and Zhihang Fu and Jieping Ye , bibsource =. The Twelfth International Conference on Learning Representations,

  116. [124]

    InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance , url =

    Pengyu Wang and Dong Zhang and Linyang Li and Chenkun Tan and Xinghao Wang and Ke Ren and Botian Jiang and Xipeng Qiu , journal =. InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance , url =

  117. [125]

    Scott Wiener , howpublished =

  118. [126]

    Hallucination is Inevitable: An Innate Limitation of Large Language Models , url =

    Ziwei Xu and Sanjay Jain and Mohan Kankanhalli , journal =. Hallucination is Inevitable: An Innate Limitation of Large Language Models , url =

  119. [127]

    Gallegos and Ryan A

    Isabel O. Gallegos and Ryan A. Rossi and Joe Barrow and Md Mehrab Tanjim and Sungchul Kim and Franck Dernoncourt and Tong Yu and Ruiyi Zhang and Nesreen K. Ahmed , journal =. Bias and Fairness in Large Language Models: A Survey , url =

  120. [128]

    Toy Models of Superposition , url =

    Nelson Elhage and Tristan Hume and Catherine Olsson and Nicholas Schiefer and Tom Henighan and Shauna Kravec and Zac Hatfield-Dodds and Robert Lasenby and Dawn Drain and Carol Chen and Roger Grosse and Sam McCandlish and Jared Kaplan and Dario Amodei and Martin Wattenberg and ...

  121. [129]

    Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned , url =

    Deep Ganguli and Liane Lovitt and Jackson Kernion and Amanda Askell and Yuntao Bai and Saurav Kadavath and Ben Mann and Ethan Perez and Nicholas Schiefer and Kamal Ndousse and Andy Jones and Sam Bowman and Anna Chen and Tom Conerly and Nova DasSarma and Dawn Drain and Nelson E...

  122. [130]

    LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B , url =

    Simon Lermen and Charlie Rogers-Smith and Jeffrey Ladish , journal =. LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B , url =

  123. [131]

    Ethical and social risks of harm from Language Models , url =

    Laura Weidinger and John Mellor and Maribeth Rauh and Conor Griffin and Jonathan Uesato and Po-Sen Huang and Myra Cheng and Mia Glaese and Borja Balle and Atoosa Kasirzadeh and Zac Kenton and Sasha Brown and Will Hawkins and Tom Stepleton and Courtney Biles and Abeba Birhane a...

  124. [132]

    Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models , url =

    Bang An and Sicheng Zhu and Ruiyi Zhang and Michael-Andrei Panaitescu-Liess and Yuancheng Xu and Furong Huang , journal =. Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models , url =

  125. [133]

    Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models , url =

    Xianjun Yang and Xiao Wang and Qi Zhang and Linda Petzold and William Yang Wang and Xun Zhao and Dahua Lin , journal =. Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models , url =

  126. [134]

    Removing

    Zhan, Qiusi and Fang, Richard and Bindu, Rohan and Gupta, Akul and Hashimoto, Tatsunori and Kang, Daniel , booktitle =. Removing

  127. [135]

    Emergent Linear Representations in World Models of Self-Supervised Sequence Models , url =

    Nanda, Neel and Lee, Andrew and Wattenberg, Martin , booktitle =. Emergent Linear Representations in World Models of Self-Supervised Sequence Models , url =. doi:10.18653/v1/2023.blackboxnlp-1.2 , editor =

    2023 doi

  128. [136]

    The Low-Dimensional Linear Geometry of Contextualized Word Representations , url =

    Hernandez, Evan and Andreas, Jacob , booktitle =. The Low-Dimensional Linear Geometry of Contextualized Word Representations , url =. doi:10.18653/v1/2021.conll-1.7 , editor =

    2021 doi

  129. [137]

    Pearce and Thomas Dooms and Alice Rigg and Jose M

    Michael T. Pearce and Thomas Dooms and Alice Rigg and Jose M. Oramas and Lee Sharkey , journal =. Bilinear MLPs enable weight-based mechanistic interpretability , url =

  130. [138]

    Sparse Autoencoders Find Highly Interpretable Features in Language Models , url =

    Robert Huben and Hoagy Cunningham and Logan Riggs and Aidan Ewart and Lee Sharkey , bibsource =. Sparse Autoencoders Find Highly Interpretable Features in Language Models , url =. The Twelfth International Conference on Learning Representations,

  131. [139]

    Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , year =

    Bricken, Trenton and Templeton, Adly and Batson, Joshua and Chen, Brian and Jermyn, Adam and Conerly, Tom and Turner, Nick and Anil, Cem and Denison, Carson and Askell, Amanda and Lasenby, Robert and Wu, Yifan and Kravec, Shauna and Schiefer, Nicholas and Maxwell, Tim and Jose...

  132. [140]

    Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla , url =

    Tom Lieberum and Matthew Rahtz and János Kramár and Neel Nanda and Geoffrey Irving and Rohin Shah and Vladimir Mikulik , journal =. Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla , url =

  133. [141]

    A Geometric Notion of Causal Probing , url =

    Clément Guerner and Anej Svete and Tianyu Liu and Alexander Warstadt and Ryan Cotterell , journal =. A Geometric Notion of Causal Probing , url =

  134. [142]

    Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection , url =

    Haghighatkhah, Pantea and Fokkens, Antske and Sommerauer, Pia and Speckmann, Bettina and Verbeek, Kevin , booktitle =. Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection , url =. doi:10.18653/v1/2022.emnlp-main.575...

    2022 doi

  135. [143]

    Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection , url =

    Ravfogel, Shauli and Elazar, Yanai and Gonen, Hila and Twiton, Michael and Goldberg, Yoav , booktitle =. Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection , url =. doi:10.18653/v1/2020.acl-main.647 , editor =

    2020 doi

  136. [144]

    Nora Belrose and David Schneider. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , editor =

    2023

  137. [145]

    Liu and Mohammad Saleh and Etienne Pot and Ben Goodrich and Ryan Sepassi and Lukasz Kaiser and Noam Shazeer , bibsource =

    Peter J. Liu and Mohammad Saleh and Etienne Pot and Ben Goodrich and Ryan Sepassi and Lukasz Kaiser and Noam Shazeer , bibsource =. Generating Wikipedia by Summarizing Long Sequences , url =. 6th International Conference on Learning Representations,

  138. [146]

    Discovering Latent Knowledge in Language Models Without Supervision , url =

    Collin Burns and Haotian Ye and Dan Klein and Jacob Steinhardt , bibsource =. Discovering Latent Knowledge in Language Models Without Supervision , url =. The Eleventh International Conference on Learning Representations,

  139. [147]

    Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications , url =

    Boyi Wei and Kaixuan Huang and Yangsibo Huang and Tinghao Xie and Xiangyu Qi and Mengzhou Xia and Prateek Mittal and Mengdi Wang and Peter Henderson , bibsource =. Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications , url =. Forty-first Interna...

  140. [148]

    Daniel and Sumers, Theodore R

    Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and Jones, Andy and Cunningham, Hoagy and Turner, Nicholas L and McDougall, Callum and MacDiarmid, Monte and Fre...

  141. [149]

    LLaMA: Open and Efficient Foundation Language Models , url =

    Hugo Touvron and Thibaut Lavril and Gautier Izacard and Xavier Martinet and Marie-Anne Lachaux and Timothée Lacroix and Baptiste Rozière and Naman Goyal and Eric Hambro and Faisal Azhar and Aurelien Rodriguez and Armand Joulin and Edouard Grave and Guillaume Lample , journal =...

  142. [150]

    Qwen Technical Report , url =

    Jinze Bai and Shuai Bai and Yunfei Chu and Zeyu Cui and Kai Dang and Xiaodong Deng and Yang Fan and Wenbin Ge and Yu Han and Fei Huang and Binyuan Hui and Luo Ji and Mei Li and Junyang Lin and Runji Lin and Dayiheng Liu and Gao Liu and Chengqiang Lu and Keming Lu and Jianxin M...

  143. [151]

    Don't Say No: Jailbreaking LLM by Suppressing Refusal , url =

    Yukai Zhou and Zhijie Huang and Feiyang Lu and Zhan Qin and Wenjie Wang , journal =. Don't Say No: Jailbreaking LLM by Suppressing Refusal , url =

  144. [152]

    Bowman , journal =

    David Rein and Betty Li Hou and Asa Cooper Stickland and Jackson Petty and Richard Yuanzhe Pang and Julien Dirani and Julian Michael and Samuel R. Bowman , journal =. GPQA: A Graduate-Level Google-Proof Q&A Benchmark , url =

  145. [153]

    Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment , url =

    Haoran Wang and Kai Shu , journal =. Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignment , url =

  146. [154]

    Orenguteng , howpublished =

  147. [155]

    cognitivecomputations/dolphin-2.9.4-llama3.1-8b ·

    Eric Hartford and Cognitive Computations , howpublished =. cognitivecomputations/dolphin-2.9.4-llama3.1-8b ·

  148. [156]

    Forty-first International Conference on Machine Learning,

    Ryan Greenblatt and Buck Shlegeris and Kshitij Sachan and Fabien Roger , bibsource =. Forty-first International Conference on Machine Learning,

  149. [157]

    Brown and Francis Rhys Ward , journal =

    Teun van der Weij and Felix Hofstätter and Ollie Jaffe and Samuel F. Brown and Francis Rhys Ward , journal =. AI Sandbagging: Language Models can Strategically Underperform on Evaluations , url =

  150. [158]

    Bowman and David Duvenaud , journal =

    Joe Benton and Misha Wagner and Eric Christiansen and Cem Anil and Ethan Perez and Jai Srivastav and Esin Durmus and Deep Ganguli and Shauna Kravec and Buck Shlegeris and Jared Kaplan and Holden Karnofsky and Evan Hubinger and Roger Grosse and Samuel R. Bowman and David Duvena...

  151. [159]

    2018 , eprint=

    Scalable agent alignment via reward modeling: a research direction , author=. 2018 , eprint=

    2018

  152. [160]

    Predictability and Surprise in Large Generative Models , url=

    Ganguli, Deep and Hernandez, Danny and Lovitt, Liane and Askell, Amanda and Bai, Yuntao and Chen, Anna and Conerly, Tom and Dassarma, Nova and Drain, Dawn and Elhage, Nelson and El Showk, Sheer and Fort, Stanislav and Hatfield-Dodds, Zac and Henighan, Tom and Johnston, Scott a...

  153. [161]

    2025 , eprint=

    Auditing language models for hidden objectives , author=. 2025 , eprint=

    2025

  154. [162]

    Pappas and Florian Tram

    Patrick Chao and Edoardo Debenedetti and Alexander Robey and Maksym Andriushchenko and Francesco Croce and Vikash Sehwag and Edgar Dobriban and Nicolas Flammarion and George J. Pappas and Florian Tram. JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Languag...

    2024

  155. [163]

    2025 , eprint=

    The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence , author=. 2025 , eprint=

    2025

  156. [164]

    2024 , eprint=

    RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations , author=. 2024 , eprint=

    2024

  157. [165]

    2021 , eprint=

    Measuring Massive Multitask Language Understanding , author=. 2021 , eprint=

    2021

  158. [167]

    and Potts, Christopher , month = may, year =

    Wu, Zhengxuan and Arora, Aryaman and Wang, Zheng and Geiger, Atticus and Jurafsky, Dan and Manning, Christopher D. and Potts, Christopher , month = may, year =. doi:10.48550/arXiv.2404.03592 , abstract =

    doi arXiv

  159. [168]

    Im, Shawn and Li, Yixuan , month = feb, year =. A. doi:10.48550/arXiv.2502.02716 , abstract =

    doi

  160. [169]

    arXiv preprint arXiv:2506.04774 , year=

    Fine-Grained Interpretation of Political Opinions in Large Language Models , author=. arXiv preprint arXiv:2506.04774 , year=

    arXiv

  161. [170]

    2022 , eprint=

    Constitutional AI: Harmlessness from AI Feedback , author=. 2022 , eprint=

    2022

  162. [171]

    arXiv preprint arXiv:2502.17601 , year=

    Representation Engineering for Large-Language Models: Survey and Research Challenges , author=. arXiv preprint arXiv:2502.17601 , year=

    arXiv

  163. [172]

    2024 , eprint=

    Sycophancy in Large Language Models: Causes and Mitigations , author=. 2024 , eprint=

    2024

  164. [173]

    2024 , eprint=

    Linear Probe Penalties Reduce LLM Sycophancy , author=. 2024 , eprint=

    2024

  165. [174]

    2024 , eprint=

    Language Models Learn to Mislead Humans via RLHF , author=. 2024 , eprint=

    2024

  166. [175]

    People , year =

    Maria Pasquini , title =. People , year =

  167. [176]

    Euronews , year =

    El Atillah, Imane , title =. Euronews , year =

  168. [177]

    The New York Times , year =

    Hill, Kashmir , title =. The New York Times , year =

  169. [178]

    2024 , eprint=

    More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness , author=. 2024 , eprint=

    2024

  170. [179]

    and Askell, Amanda and Grosse, Roger and Hernandez, Danny and Ganguli, Deep and Hubinger, Evan and Schiefer, Nicholas and Kaplan, Jared

    Perez, Ethan and Ringer, Sam and Lukosiute, Kamile and Nguyen, Karina and Chen, Edwin and Heiner, Scott and Pettit, Craig and Olsson, Catherine and Kundu, Sandipan and Kadavath, Saurav and Jones, Andy and Chen, Anna and Mann, Benjamin and Israel, Brian and Seethor, Bryan and M...

    2023

  171. [180]

    2024 , eprint=

    Fine-tuning can cripple your foundation model; preserving features may be the solution , author=. 2024 , eprint=

    2024

  172. [181]

    2024 , eprint=

    The effect of fine-tuning on language model toxicity , author=. 2024 , eprint=

    2024

  173. [182]

    Understanding Impact of Human Feedback via Influence Functions , url=

    Min, Taywon and Lee, Haeone and Kwon, Yongchan and Lee, Kimin , year=. Understanding Impact of Human Feedback via Influence Functions , url=. doi:10.18653/v1/2025.acl-long.1333 , booktitle=

    2025 doi

  174. [183]

    arXiv preprint arXiv:2502.19649 , year=

    Taxonomy, opportunities, and challenges of representation engineering for large language models , author=. arXiv preprint arXiv:2502.19649 , year=

  175. [184]

    doi:10.48550/arXiv.2507.11878 , abstract =

    Zhao, Jiachen and Huang, Jing and Wu, Zhengxuan and Bau, David and Shi, Weiyan , month = jul, year =. doi:10.48550/arXiv.2507.11878 , abstract =

    doi arXiv

  176. [185]

    9th International Conference on Learning Representations,

    Dan Hendrycks and Collin Burns and Steven Basart and Andrew Critch and Jerry Li and Dawn Song and Jacob Steinhardt , title =. 9th International Conference on Learning Representations,. 2021 , url =

    2021

  177. [186]

    LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks , booktitle =

    Yushi Bai and Shangqing Tu and Jiajie Zhang and Hao Peng and Xiaozhi Wang and Xin Lv and Shulin Cao and Jiazheng Xu and Lei Hou and Yuxiao Dong and Jie Tang and Juanzi Li , editor =. LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks ...

    2025

  178. [187]

    The Twelfth International Conference on Learning Representations,

    Tri Dao , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

    2024

  179. [188]

    Advances in Neural Information Processing Systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in Neural Information Processing Systems , volume=

  180. [189]

    Advances in neural information processing systems , volume=

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in neural information processing systems , volume=

  181. [190]

    2023 , eprint=

    ReAct: Synergizing Reasoning and Acting in Language Models , author=. 2023 , eprint=

    2023

  182. [191]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  183. [192]

    Frontiers of Computer Science , volume=

    A survey on large language model based autonomous agents , author=. Frontiers of Computer Science , volume=. 2024 , publisher=

    2024

  184. [193]

    arXiv preprint arXiv:2504.18875 , year=

    Generative to agentic ai: Survey, conceptualization, and challenges , author=. arXiv preprint arXiv:2504.18875 , year=

    arXiv

  185. [194]

    Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V

    Evaluation and benchmarking of llm agents: A survey , author=. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2 , pages=

  186. [195]

    arXiv preprint arXiv:2506.04133 , year=

    Trism for agentic ai: A review of trust, risk, and security management in llm-based agentic multi-agent systems , author=. arXiv preprint arXiv:2506.04133 , year=

  187. [196]

    NIST Trustworthy and Responsible AI Gaithersburg, MD, USA , year=

    Artificial intelligence risk management framework: Generative artificial intelligence profile , author=. NIST Trustworthy and Responsible AI Gaithersburg, MD, USA , year=

  188. [197]

    arXiv preprint arXiv:2506.08837 , year=

    f , author=. arXiv preprint arXiv:2506.08837 , year=

    arXiv

  189. [198]

    2023 , eprint=

    Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection , author=. 2023 , eprint=

    2023

  190. [199]

    2022 , eprint=

    Ignore Previous Prompt: Attack Techniques For Language Models , author=. 2022 , eprint=

    2022

  191. [200]

    OWASP GenAI LLM01: Prompt Injection , author =

  192. [201]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  193. [202]

    2022 , eprint=

    Measuring Attribution in Natural Language Generation Models , author=. 2022 , eprint=

    2022

  194. [203]

    2019 , eprint=

    Attention is not Explanation , author=. 2019 , eprint=

    2019

  195. [204]

    arXiv preprint arXiv:2309.00614 , year=

    Baseline defenses for adversarial attacks against aligned language models , author=. arXiv preprint arXiv:2309.00614 , year=

    arXiv

  196. [205]

    arXiv preprint arXiv:2310.15140 , year=

    Autodan: interpretable gradient-based adversarial attacks on large language models , author=. arXiv preprint arXiv:2310.15140 , year=

    arXiv

  197. [206]

    arXiv preprint arXiv:2502.05174 , year=

    MELON: Provable Defense Against Indirect Prompt Injection Attacks in AI Agents , author=. arXiv preprint arXiv:2502.05174 , year=

    arXiv

  198. [207]

    arXiv preprint arXiv:2507.14799 , year=

    Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Tree , author=. arXiv preprint arXiv:2507.14799 , year=

    arXiv

  199. [208]

    arXiv preprint arXiv:2404.08144 , year=

    Llm agents can autonomously exploit one-day vulnerabilities , author=. arXiv preprint arXiv:2404.08144 , year=

    arXiv

  200. [209]

    Fang, Richard and Bindu, Rohan and Gupta, Akul and Zhan, Qiusi and Kang, Daniel , journal=

  201. [210]

    When LLMs Autonomously Attack: CMU Engineering News , author =

  202. [211]

    So Are the Threats

    AI Agents Are Here. So Are the Threats. , author =

  203. [212]

    arXiv preprint arXiv:2506.23260 , year=

    From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows , author=. arXiv preprint arXiv:2506.23260 , year=

  204. [213]

    arXiv preprint arXiv:2505.23847 , year=

    Seven security challenges that must be solved in cross-domain multi-agent llm systems , author=. arXiv preprint arXiv:2505.23847 , year=

    arXiv

  205. [214]

    arXiv preprint arXiv:2307.13854 , year=

    Webarena: A realistic web environment for building autonomous agents , author=. arXiv preprint arXiv:2307.13854 , year=

    arXiv

  206. [215]

    arXiv preprint arXiv:2412.05467 , year=

    The browsergym ecosystem for web agent research , author=. arXiv preprint arXiv:2412.05467 , year=

    arXiv

  207. [216]

    arXiv preprint arXiv:2410.13825 , year=

    Agentoccam: A simple yet strong baseline for llm-based web agents , author=. arXiv preprint arXiv:2410.13825 , year=

    arXiv

  208. [217]

    arXiv preprint arXiv:2502.02649 , year=

    Fully autonomous ai agents should not be developed , author=. arXiv preprint arXiv:2502.02649 , year=

  209. [218]

    arXiv preprint arXiv:2406.04151 , year=

    Agentgym: Evolving large language model-based agents across diverse environments , author=. arXiv preprint arXiv:2406.04151 , year=

    arXiv

  210. [219]

    arXiv preprint arXiv:2410.06703 , year=

    St-webagentbench: A benchmark for evaluating safety and trustworthiness in web agents , author=. arXiv preprint arXiv:2410.06703 , year=

    arXiv

  211. [220]

    arXiv preprint arXiv:2503.04957 , year=

    Safearena: Evaluating the safety of autonomous web agents , author=. arXiv preprint arXiv:2503.04957 , year=

    arXiv

  212. [221]

    arXiv preprint arXiv:2504.05259 , year=

    How to evaluate control measures for LLM agents? A trajectory from today to superintelligence , author=. arXiv preprint arXiv:2504.05259 , year=

    arXiv

  213. [222]

    Considerations on AI Model Red-Teaming and Standards , author=

  214. [223]

    Advanced AI Evaluations (AISI Update) , author =

  215. [224]

    arXiv preprint arXiv:2503.18666 , year=

    Agentspec: Customizable runtime enforcement for safe and reliable llm agents , author=. arXiv preprint arXiv:2503.18666 , year=

    arXiv

  216. [225]

    Kang, Mintong and Li, Bo , journal=. R

  217. [226]

    2025 IEEE/ACM 4th International Conference on AI Engineering--Software Engineering for AI (CAIN) , pages=

    Designing and Implementing LLM Guardrails Components in Production Environments , author=. 2025 IEEE/ACM 4th International Conference on AI Engineering--Software Engineering for AI (CAIN) , pages=. 2025 , organization=

    2025

  218. [227]

    arXiv preprint arXiv:2406.12934 , year=

    Current state of LLM Risks and AI Guardrails , author=. arXiv preprint arXiv:2406.12934 , year=

    arXiv

  219. [228]

    arXiv preprint arXiv:2504.00018 , year=

    SandboxEval: Towards Securing Test Environment for Untrusted Code , author=. arXiv preprint arXiv:2504.00018 , year=

    arXiv

  220. [229]

    Secure Code Execution in LLMs for Better AI , author =

  221. [230]

    Reinforcement Learning for Safe LLM Code Generation , author=

  222. [231]

    What Is Sandboxing? (Cyber Glossary) , author =

  223. [232]

    NCCoE Cyber AI Profile , author =

  224. [233]

    Agentic AI - Threats & Mitigations , author =

  225. [234]

    MAESTRO: Agentic AI Threat Modeling Framework | CSA , author =

  226. [235]

    Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design , pages=

    Generative AI agents in autonomous machines: A safety perspective , author=. Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design , pages=

  227. [236]

    This Is How to Manage It , author =

    AI Has a Safety Problem. This Is How to Manage It , author =

  228. [237]

    2024 , publisher=

    Digital transformation, artificial intelligence and society , author=. 2024 , publisher=

    2024

  229. [238]

    Artificial intelligence , volume=

    The epistemology of a rule-based expert system—a framework for explanation , author=. Artificial intelligence , volume=. 1983 , publisher=

    1983

  230. [239]

    nature , volume=

    Deep learning , author=. nature , volume=. 2015 , publisher=

    2015

  231. [240]

    Artificial Intelligence

    A modern approach , author=. Artificial Intelligence. Prentice-Hall, Egnlewood Cliffs , volume=

  232. [241]

    Advances in Neural Information Processing Systems (NeurIPS) , year=

    Language Models are Few-Shot Learners , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=

  233. [242]

    2023 , note=

    GPT-4 Technical Report , author=. 2023 , note=

    2023

  234. [243]

    2023 , note=

    LLaMA 2: Open Foundation and Fine-Tuned Chat Models , author=. 2023 , note=

    2023

  235. [244]

    arXiv preprint arXiv:2405.16640 , year=

    A survey of multimodal large language model from a data-centric perspective , author=. arXiv preprint arXiv:2405.16640 , year=

    arXiv

  236. [245]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

    Otter: A multi-modal model with in-context instruction tuning , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , year=

  237. [246]

    Journal of Medical Systems , volume=

    The breakthrough of large language models release for medical applications: 1-year timeline and perspectives , author=. Journal of Medical Systems , volume=. 2024 , publisher=

    2024

  238. [247]

    Forecasting , volume=

    Large language models: Their success and impact , author=. Forecasting , volume=. 2023 , publisher=

    2023

  239. [248]

    ACM Transactions on Intelligent Systems and Technology , year=

    A comprehensive overview of large language models , author=. ACM Transactions on Intelligent Systems and Technology , year=

  240. [249]

    Large Language Models in Cybersecurity: Threats, Exposure and Mitigation , pages=

    Fundamental limitations of generative llms , author=. Large Language Models in Cybersecurity: Threats, Exposure and Mitigation , pages=. 2024 , publisher=

    2024

  241. [250]

    Understand

    AI What Do Large Language Models “Understand”? , author=. Image , volume=

  242. [251]

    arXiv preprint arXiv:2503.14499 , year=

    Measuring ai ability to complete long tasks , author=. arXiv preprint arXiv:2503.14499 , year=

    arXiv

  243. [252]

    2024 , howpublished=

    LangChain Documentation , author=. 2024 , howpublished=

    2024

  244. [253]

    2024 , howpublished=

    AutoGPT: An Autonomous GPT Experiment , author=. 2024 , howpublished=

    2024

  245. [254]

    arXiv preprint arXiv:2305.16291 , year=

    Voyager: An Open-Ended Embodied Agent with Large Language Models , author=. arXiv preprint arXiv:2305.16291 , year=

    arXiv

  246. [255]

    MIT Sloan Management Review (Online) , pages=

    Three Essentials for Agentic AI Security , author=. MIT Sloan Management Review (Online) , pages=. 2025 , publisher=

    2025

  247. [256]

    2025 , url =

    Just in time? Manufacturers turn to AI to weather tariff storm , author =. 2025 , url =

    2025

  248. [257]

    AI Agents Are the Future , author =

    Forget Chatbots. AI Agents Are the Future , author =. 2025 , url =

    2025

  249. [258]

    Proceedings of the ACM Symposium on User Interface Software and Technology (UIST) , year=

    Generative Agents: Interactive Simulacra of Human Behavior , author=. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST) , year=

  250. [259]

    Cell , volume=

    Empowering biomedical discovery with AI agents , author=. Cell , volume=. 2024 , publisher=

    2024

  251. [260]

    arXiv preprint arXiv:2503.08979 , year=

    Agentic ai for scientific discovery: A survey of progress, challenges, and future directions , author=. arXiv preprint arXiv:2503.08979 , year=

    arXiv

  252. [261]

    2025 , url =

    Inside the automated warehouse where robots are packing your groceries , author =. 2025 , url =

    2025

  253. [262]

    arXiv preprint arXiv:2309.17288 , year =

    AutoAgents: A Framework for Automatic Agent Generation , author =. arXiv preprint arXiv:2309.17288 , year =

    arXiv

  254. [263]

    2025 , url =

    Amazon's Delivery, Logistics Get AI Boost , author =. 2025 , url =

    2025

  255. [264]

    arXiv preprint arXiv:2403.02691 , year=

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents , author=. arXiv preprint arXiv:2403.02691 , year=

    arXiv

  256. [265]

    arXiv preprint arXiv:2410.07283 , year=

    Prompt infection: Llm-to-llm prompt injection within multi-agent systems , author=. arXiv preprint arXiv:2410.07283 , year=

    arXiv

  257. [266]

    arXiv preprint arXiv:2503.03704 , year=

    A practical memory injection attack against llm agents , author=. arXiv preprint arXiv:2503.03704 , year=

  258. [267]

    Advances in Neural Information Processing Systems , volume=

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases , author=. Advances in Neural Information Processing Systems , volume=

  259. [268]

    arXiv preprint arXiv:2502.08586 , year=

    Commercial llm agents are already vulnerable to simple yet dangerous attacks , author=. arXiv preprint arXiv:2502.08586 , year=

    arXiv

  260. [269]

    arXiv preprint arXiv:2410.14923 , year=

    Imprompter: Tricking llm agents into improper tool use , author=. arXiv preprint arXiv:2410.14923 , year=

    arXiv

  261. [270]

    arXiv preprint arXiv:2410.09024 , year=

    Agentharm: A benchmark for measuring harmfulness of llm agents , author=. arXiv preprint arXiv:2410.09024 , year=

    arXiv

  262. [271]

    arXiv preprint arXiv:2407.20859 , year=

    Breaking agents: Compromising autonomous llm agents through malfunction amplification , author=. arXiv preprint arXiv:2407.20859 , year=

    arXiv

  263. [272]

    Agentic Misalignment: How LLMs Could Be Insider Threats , author =

  264. [273]

    arXiv preprint arXiv:2507.06850 , year=

    The Dark Side of LLMs Agent-based Attacks for Complete Computer Takeover , author=. arXiv preprint arXiv:2507.06850 , year=

    arXiv

  265. [274]

    2025 , note =

    When LLMs autonomously attack , author =. 2025 , note =

    2025

  266. [275]

    2025 , url =

    Preventing Zero-Click AI Threats: Insights from EchoLeak , author =. 2025 , url =

    2025

  267. [276]

    2025 , url =

    AI: Advent of Agents Opens New Possibilities for Attackers , author =. 2025 , url =

    2025

  268. [277]

    2025 , url =

    Tricked into exposing data: tech staff say bots are security risk , author =. 2025 , url =

    2025

  269. [278]

    arXiv preprint arXiv:2504.01990 , year=

    Advances and challenges in foundation agents: From brain-inspired intelligence to evolutionary, collaborative, and safe systems , author=. arXiv preprint arXiv:2504.01990 , year=

    arXiv

  270. [279]

    arXiv preprint arXiv:2507.21046 , year=

    A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence , author=. arXiv preprint arXiv:2507.21046 , year=

    arXiv

  271. [280]

    IEEe Access , year=

    Agentic ai: Autonomous intelligence for complex goals--a comprehensive survey , author=. IEEe Access , year=

  272. [281]

    arXiv preprint arXiv:2310.19156 , year=

    Poisoning retrieval corpora by injecting adversarial passages , author=. arXiv preprint arXiv:2310.19156 , year=

    arXiv

  273. [282]

    arXiv preprint arXiv:2409.11295 , year=

    Eia: Environmental injection attack on generalist web agents for privacy leakage , author=. arXiv preprint arXiv:2409.11295 , year=

    arXiv

  274. [283]

    arXiv preprint arXiv:2410.17401 , year=

    AdvAgent: Controllable Blackbox Red-teaming on Web Agents , author=. arXiv preprint arXiv:2410.17401 , year=

    arXiv

  275. [284]

    arXiv preprint arXiv:2406.12814 , year=

    Dissecting adversarial robustness of multimodal lm agents , author=. arXiv preprint arXiv:2406.12814 , year=

    arXiv

  276. [285]

    arXiv preprint arXiv:2411.02391 , year=

    Attacking vision-language computer agents via pop-ups , author=. arXiv preprint arXiv:2411.02391 , year=

    arXiv

  277. [286]

    arXiv preprint arXiv:2504.17669 , year=

    Towards a hipaa compliant agentic ai system in healthcare , author=. arXiv preprint arXiv:2504.17669 , year=

    arXiv

  278. [287]

    Agentic AI: Theories and Practices , pages=

    AI Agents in Healthcare , author=. Agentic AI: Theories and Practices , pages=. 2025 , publisher=

    2025

  279. [288]

    Informatics and Health , volume=

    Next-generation agentic AI for transforming healthcare , author=. Informatics and Health , volume=. 2025 , publisher=

    2025

  280. [289]

    Nature Biomedical Engineering , pages=

    Coordinated AI agents for advancing healthcare , author=. Nature Biomedical Engineering , pages=. 2025 , publisher=

    2025

  281. [290]

    The Lancet , volume=

    The rise of agentic AI teammates in medicine , author=. The Lancet , volume=. 2025 , publisher=

    2025

  282. [291]

    ArXiv , year=

    Prompt Injection 2.0: Hybrid AI Threats , author=. ArXiv , year=

  283. [292]

    Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering , pages=

    Human-imperceptible retrieval poisoning attacks in LLM-powered applications , author=. Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering , pages=

  284. [293]

    arXiv preprint arXiv:2410.14479 , year=

    Backdoored retrievers for prompt injection attacks on retrieval augmented generation of large language models , author=. arXiv preprint arXiv:2410.14479 , year=

    arXiv

  285. [294]

    arXiv preprint arXiv:2504.14348 , year=

    Manipulating Multimodal Agents via Cross-Modal Prompt Injection , author=. arXiv preprint arXiv:2504.14348 , year=

    arXiv

  286. [295]

    arXiv preprint arXiv:2307.10490 , year=

    Abusing images and sounds for indirect instruction injection in multi-modal LLMs , author=. arXiv preprint arXiv:2307.10490 , year=

    arXiv

  287. [296]

    33rd USENIX Security Symposium (USENIX Security 24) , pages=

    Adversarial illusions in Multi-Modal embeddings , author=. 33rd USENIX Security Symposium (USENIX Security 24) , pages=

  288. [297]

    arXiv preprint arXiv:2505.14103 , year=

    AudioJailbreak: Jailbreak Attacks against End-to-End Large Audio-Language Models , author=. arXiv preprint arXiv:2505.14103 , year=

  289. [298]

    Electronics , volume=

    Data stealing attacks against large language models via backdooring , author=. Electronics , volume=. 2024 , publisher=

    2024

  290. [299]

    2024 , url =

    Johann Rehberger , title =. 2024 , url =

    2024

  291. [300]

    arXiv preprint arXiv:2402.06664 , year=

    Llm agents can autonomously hack websites , author=. arXiv preprint arXiv:2402.06664 , year=

    arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.