Pith. sign in

REVIEW 2 major objections 7 minor 65 references

AI agents fail to isolate trusted from untrusted data, so attackers can forge the metadata the agent relies on.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 08:30 UTC pith:ONU7D72C

load-bearing objection Solid applied security paper: ADI is a real, under-addressed IPI class with named-product PoCs and defense-gap data; format-knowledge is a real precondition but not a paper-killing flaw. the 2 major comments →

arxiv 2607.05120 v1 pith:ONU7D72C submitted 2026-07-06 cs.CR cs.AI

Agent Data Injection Attacks are Realistic Threats to AI Agents

Woohyuk Choi , Juhee Kim , Taehyun Kang , Jihyeon Jeong , Luyi Xing , Byoungyoung Lee This is my paper
classification cs.CR cs.AI
keywords agent data injectionindirect prompt injectionprobabilistic delimiter injectionAI agent securitytrusted data isolationweb agentscoding agentstool call forgery
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that today’s AI agents mix trusted metadata and attacker-controlled content in the same context without a hard boundary between them. Prior work on indirect prompt injection mostly studied instruction injection, in which untrusted text is read as a command; defenses therefore focus on keeping instructions separate from data. The authors introduce agent data injection (ADI): the attacker instead causes the model to treat forged content as trusted data—element IDs, author names, tool-call history—so the agent still pursues the user’s task but acts on the wrong anchors. The core technique is probabilistic delimiter injection: inexact, parser-invalid characters that the tool leaves as plain text but that the language model still reads as structure. They show this yields arbitrary-click behavior in web agents and remote-code-execution and supply-chain merge attacks in coding agents, and that most existing instruction-focused defenses leave ADI largely open. The claim is that agent security must enforce trusted/untrusted isolation inside agent data, not only instruction/data separation.

Core claim

Agent data injection is a distinct class of indirect prompt injection: by injecting probabilistic delimiters into untrusted fields, an attacker makes the model interpret attacker-controlled content as trusted agent data, so the agent follows the user’s task but on forged security anchors. Existing defenses that only separate instructions from data do not stop this, and real web and coding agents are vulnerable in practice.

What carries the argument

Probabilistic delimiter injection: character sequences that tools treat as plain text (including escaped or inexact delimiters) but that language models still parse as structural boundaries, shifting untrusted payload into the trusted half of the agent context (formalized as LLM(I,(DT,DU∥DA))≈ADI LLM(I,(DT∥DA,DU))).

Load-bearing premise

The attacker can learn or recover the target agent’s data format—how tool responses, element IDs, and tool-call blocks are serialized—so the forged delimiters land in the right places.

What would settle it

If agents that fully isolate trusted metadata from untrusted fields (or randomize all security anchors so they cannot be forged) show near-zero ADI success while still completing the same tasks, the claim that current agents lack this isolation as a practical security gap would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Defenses that only stop instruction injection will leave agents open to forged IDs, origins, and tool histories.
  • Web agents that expose predictable element IDs can be steered into XSS-like arbitrary clicks from user-generated content.
  • Coding agents that trust origin metadata or tool-call blocks can be tricked into RCE or merging malicious PRs without reviewing real code.
  • Practical agent security needs fine-grained trusted/untrusted isolation inside agent data, not only instruction/data separation.
  • Randomization of field names and IDs helps for key-value formats; broad sanitization of delimiters costs substantial utility.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Agent frameworks that serialize tool results as free-form text without a trusted schema layer will keep recreating this class of bug.
  • The same pattern should appear in any agent that treats origin stamps, resource IDs, or prior tool transcripts as security anchors while still concatenating untrusted bodies into the same context.
  • Format secrecy alone is a brittle control: once a format is observed or reverse-engineered once, every deployment sharing it becomes a target.
  • Future red-team benchmarks for agents need separate ADI suites; instruction-injection scores alone will overstate security.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper introduces agent data injection (ADI) as a distinct category of indirect prompt injection: rather than causing untrusted content to be read as instructions, ADI causes it to be read as trusted agent data (DT)—security-critical metadata such as element IDs, origin fields, or tool-call history—via probabilistic delimiter injection. The authors formalize the distinction from instruction injection (Eqs. 1–2), demonstrate end-to-end attacks on real web and coding agents (arbitrary click; RCE via origin spoof; supply-chain merge of a malicious PR), evaluate probabilistic delimiter injection on six off-the-shelf models (JSON and web DOM), and show that most existing IPI defenses leave substantial ADI ASR in an AgentDojo extension while instruction injection is near zero. They argue that current agents fail a basic isolation principle between trusted and untrusted data within the agent context.

Significance. If the results hold, the paper identifies a practically important and under-addressed attack surface for AI agents: defenses that only separate instructions from data do not protect security anchors that live inside agent data. The contribution is concrete rather than purely conceptual: named-product PoCs with responsible disclosure, a controlled delimiter-injection benchmark (multiple models, delimiter variants, structural consistency, randomization/sanitization), and an AgentDojo extension that cleanly contrasts ADI with instruction injection under several published defenses. Open artifacts and vendor disclosure strengthen the work. The framing (probabilistic vs deterministic delimiter injection; DT vs DU isolation) is useful for the field even if some end-to-end scenarios depend on format recovery.

major comments (2)
  1. [§4.3, §7, Appendix D] §3.1, §4.3, §7, and Appendix D: End-to-end realism of the tool-call/response injection (supply-chain) attack depends on recovering server-side tool-call block delimiters (e.g., <function_calls>/<function_results>, Gemini’s <ctrl46> tags). The paper recovers these via jailbreak and explicitly leaves systematic format extraction to future work. That is a load-bearing precondition for the §4.3 claim as stated, and weaker than the agent-side recoveries used for element-ID and origin injection (§4.1–§4.2), which are more convincingly realistic. Please either (i) qualify the supply-chain result as contingent on successful format extraction and separate it from the stronger agent-side PoCs in the abstract/intro impact claims, or (ii) provide reproducible evidence that the exact delimiters used at inference time are stably recoverable without privileged access. Isolated probabilistic misparsing
  2. [§6.2, Figure 9] §6.2 / Figure 9: Agent-level evaluation is reported only with GPT-5.2, while the LLM isolation study covers six models. The central claim that ADI is effective “in AI agent settings” and that defenses fail against ADI would be stronger with at least one additional model (or a short multi-model subset) under the same AgentDojo + defense harness, especially given model-dependent baseline ASR in Table 2 (31.3–43.3% JSON; 33.3–100% DOM). Without that, it is hard to know whether the ~50% ADI ASR and near-zero II ASR generalize beyond a single hardened API. A limited multi-model agent run, or a clear limitation statement with expected variance, is needed for the agent-setting claim.
minor comments (7)
  1. [Abstract, §1] Abstract and §1: Phrases such as “easily bypasses existing IPI defenses” should be aligned with §5–§6.2, where CaMeL Strict achieves 0% ASR (at large utility cost) and randomization/sandboxing partially help. Prefer “most existing instruction-focused defenses” or similar.
  2. [Figure 2, Figure 4] Figure 2 / Figure 4: The trusted/untrusted coloring is helpful; ensure the caption explicitly states which fields are DT vs DU for readers skimming the formalization in §3.2.
  3. [Table 3] Table 3: “real” delimiters are shown with ASR “–”; a one-line note that true unescaped delimiters are often blocked by tool escaping (JSON) or are the baseline DOM case would avoid confusion.
  4. [§5, Table 2] §5 Randomization: ChatGPT Atlas is cited as using nonces; a brief note on whether nonce entropy/length assumptions are stated would help readers assess ASR-C vs ASR-N in Table 2.
  5. [§4, Appendix E] §E PoC figures: Confirmation dialogs (Figures 13, 16, 20) are important evidence that UI approval is insufficient; consider calling this out once in the main §4 text with a single cross-reference rather than only in the appendix.
  6. [References, Abstract, §8] Typos / polish: “https://https://code.claude.com” in the Claude Code reference; “signifying that current AI agents do not employ a fundamental security principle: current agents do not isolate…” is redundant—tighten once in abstract and conclusion.
  7. [§7] Related work: Concurrent “data injection” on resumes [65] is distinguished; a short sentence on how ADI relates to classic structured-injection literature beyond SQL/XSS (e.g., log/header injection, multipart parsers) would situate probabilistic delimiter injection more clearly.

Circularity Check

0 steps flagged

No circular derivation: ADI is an empirical security claim measured on external models, agents, and defenses, not a result forced by its own definitions or self-citations.

full rationale

This is an empirical systems/security paper. The central claim—that probabilistic delimiter injection can make LLMs treat untrusted fields as trusted agent data (DT), yielding realistic end-to-end impact and bypassing instruction/data-separation defenses—is supported by (i) a definitional formalization contrasting II vs ADI (Eqs. 1–2), which frames the attack class rather than deriving measured impact from itself; (ii) PoCs on third-party agents (Claude in Chrome, Antigravity, Nanobrowser, Claude Code, Codex, Gemini CLI); and (iii) quantitative ASR/utility on off-the-shelf LLMs and AgentDojo under external defense implementations (Prompt Guard, LlamaFirewall, IsolateGPT, Progent, CaMeL). There is no fitted parameter renamed as a prediction, no uniqueness theorem imported from the authors to forbid alternatives, and no load-bearing self-citation chain that reduces the result to an unverified prior claim by the same group. Self-citations (e.g., Prompt Flow Integrity as related dual-LLM/IFC work) are background, not the proof of ADI effectiveness. Threat-model caveats about format recovery (jailbreaks for server-side delimiters) affect realism assumptions, not circularity of the derivation. Score 0 is the honest finding.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 2 invented entities

Load-bearing content is empirical and systems-level rather than axiomatic physics. The claim rests on standard CS assumptions about agents, plus the observed probabilistic parsing behavior of LLMs. No free parameters are fitted to force a theoretical curve; invented “entities” are attack abstractions, not new physical mediators.

axioms (5)
  • domain assumption LLMs interpret structured agent data probabilistically and can treat inexact or escaped delimiter-like sequences as structural boundaries.
    Core of probabilistic delimiter injection (§3.3); supported by Table 2–3 and Figure 8 rather than proved as a general law of all models.
  • domain assumption Real agent tools concatenate trusted metadata and attacker-controllable fields into one LLM-visible context without a hard isolation boundary.
    Attack surface definition (§2.1.2, §3.2); illustrated by email/JSON, DOM summaries, GitHub comments, and tool-call blocks.
  • domain assumption Attacker can write content into external resources the agent later retrieves (comments, emails, PR descriptions, reviews) but cannot edit system/user prompts.
    Standard IPI threat model adopted in §3.1.
  • ad hoc to paper Attacker knows or can recover the agent’s data format (serialization, IDs, tool-call delimiters).
    Explicit threat-model assumption (§3.1) with recovery discussion in §7 and Appendix D; weakest for server-side formats requiring jailbreak.
  • domain assumption User confirmation dialogs that omit precise element/rationale details do not reliably stop attacks once the LLM’s reasoning is already poisoned.
    Used to argue residual risk in Claude in Chrome / coding-agent approval UX (§4.1–4.3, Figures 13, 16, 20).
invented entities (2)
  • Agent data injection (ADI) as an IPI category independent evidence
    purpose: Name attacks that make untrusted data be interpreted as trusted agent data DT rather than as instructions I.
    Conceptual category defined via Eq. 2 and Figure 3; useful taxonomy, not an unobserved physical object. independent_evidence is the empirical attack success, not a separate ontology.
  • Probabilistic delimiter injection independent evidence
    purpose: Attack technique that injects parser-invalid or escaped delimiter-like sequences so the LLM restructures trusted vs untrusted fields.
    Named technique distinguishing ADI from deterministic SQLi/XSS-style delimiter injection; validated experimentally across delimiter variants.

pith-pipeline@v1.1.0-grok45 · 30457 in / 3418 out tokens · 34610 ms · 2026-07-11T08:30:28.086389+00:00 · methodology

0 comments
read the original abstract

AI agents act on behalf of user prompts, consuming external data and taking actions based on the agent context. Prior research on AI agent security has primarily focused on indirect prompt injection (IPI). Its most well-studied category is instruction injection, where attacker-controlled untrusted data is interpreted as an instruction. In response, many mitigations have been proposed to prevent instruction injection attacks. In this paper, we introduce a new category of IPI, agent data injection attacks (ADI). ADI injects malicious data disguised as trusted data, such as security-critical metadata (e.g., resource identifiers or data origins) or agent context data (e.g., tool call and response formats). As a result, agents unknowingly execute unintended actions based on attacker-controlled data. ADI has similar attack impacts as instruction injection attacks, because it causes agents to misbehave and execute unintended actions. Despite the similar impact, ADI remains underexplored and easily bypasses existing IPI defenses. We found several critical vulnerabilities in real-world agents that allow an attacker to launch various attacks: arbitrary click attacks on web agents (Claude in Chrome, Antigravity, and Nanobrowser), and remote code execution and supply-chain attacks on coding agents (Claude Code, Codex, and Gemini CLI). We evaluate ADI vulnerabilities across off-the-shelf models and AI agents, and find that ADI is effective in both standalone LLMs and AI agent settings. ADI exposes a critical gap in agent security, signifying that current AI agents do not employ a fundamental security principle: current agents do not isolate trusted data from untrusted data.

Figures

Figures reproduced from arXiv: 2607.05120 by Byoungyoung Lee, Jihyeon Jeong, Juhee Kim, Luyi Xing, Taehyun Kang, Woohyuk Choi.

Figure 1
Figure 1. Figure 1: Architecture and workflow of an AI agent. <call name=“get_emails” /> { }, “body”: “Have a nice day.”, “subject”: “Hello World!”, “sender”: “attacker@gmail.com”, “email_id”: “12345”, <results name=“get_emails”> </results> Green: Trusted Data Red : Untrusted Data Object Field [ ] Tool Call Block {...} [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Agent Data Format with Delimiters 2. Background 2.1. Agent Context in AI Agents 2.1.1. AI Agents. AI agents autonomously perform tasks on behalf of users by interacting with external environments such as file systems and web services [20]. An AI agent maintains an agent context, which contains all relevant information for a given task (e.g., user prompt, tool responses), and interacts with an LLM and tools… view at source ↗
Figure 3
Figure 3. Figure 3: Two categories of indirect prompt injection attacks, instruction injection attack and agent data injection attack (ADI). Agent Data. Agent data refers to information generated or retrieved during the agent’s execution. Agent data is not explicitly intended to control agent behavior, which is conceptually similar to non-executable data in traditional software. Nevertheless, such data still indirectly influe… view at source ↗
Figure 4
Figure 4. Figure 4: An example of ADI against an email agent. The attacker’s email embeds a malicious payload in the body field (red, untrusted data), whose probabilistic JSON delimiters forge a second email object with a spoofed sender (alice@gmail.com). Green fields are trusted data generated by the email tool. Formalization: Agent data injection (ADI). Under ADI, similar to II, the LLM is called with the same input I and D… view at source ↗
Figure 5
Figure 5. Figure 5: Element ID injection attack on web agents. Left: benign workflow. Right: ADI attack workflow. In this benign workflow, the security mechanism works as expected: the agent clicks the intended element based on the trusted element identifier. ADI Attack Workflow. Now, we describe how ADI bypasses the security mechanism that relies on element identifiers, as shown in [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Origin injection attack on coding agents. Left: benign workflow. Right: ADI attack workflow. Fix Type Error #23 This PR resolves type error. The commit shows a dangerous patch. I will not merge this PR. 6 2 1 Check PR #23. Review it then merge it. read_PR_desc(“23”) 3 read_PR_desc 4 read_PR_commit(“23”) Commit in PR #23: Commit (Malicious) 5 Commit (Malicious) read_PR_commit Eve Open • Eve wants to merge 1… view at source ↗
Figure 7
Figure 7. Figure 7: Tool call injection attack on coding agents. Left: benign workflow. Right: ADI attack workflow. Security Impact. This attack enables an attacker to forge data origin metadata, which agents and users rely on as a security anchor to determine the trustworthiness of external content. For example, agents may be instructed to only trust suggestions from maintainers, emails from managers, or messages from known … view at source ↗
Figure 8
Figure 8. Figure 8: ASR by structural consistency (JSON only). Dataset collection. We test two data formats: JSON and web DOM. JSON data was collected from production APIs (GitHub API, Google Workspace via MCP) and anonymized, except for paper reviews, which we created synthetically. Web DOM data was synthesized following the element format of Nanobrowser [18]. Tasks. Our tasks instruct the LLM to either extract a specific fi… view at source ↗
Figure 9
Figure 9. Figure 9: ASR of instruction injection and ADI against various defenses. 0 10 20 30 40 50 60 Attack Success Rate (%) 0 20 40 60 80 100 Utility (%) Baseline Randomization Progent CaMeL Strict [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 12
Figure 12. Figure 12: Claude in Chrome chat history showing the attack execution. Left: the user prompt “Summarize each customer review on current webpage” (red box). Right: the agent decides to click the fake “Read More” button (red box) after misinterpreting the injected payload as a legitimate page element. The agent invokes the Click action using the injected element identifier, which resolves to the “Buy Now” button on th… view at source ↗
Figure 13
Figure 13. Figure 13: User confirmation dialog displayed by Claude in Chrome before executing the click action. The dialog shows that “Claude wants to click on” the target website, without specifying which element will be clicked or why. This prevents the user from distinguishing a legitimate click from one induced by the injected payload. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Attacker payload injected into a GitHub issue comment. The red box highlights the injected fake comment object, which mimics the output format of gh issue view N --comments. The fake comment object impersonates a user named alice with a spoofed Maintainer role indicator, and suggests installing a malicious package (pip install starship-auth) as a fix [PITH_FULL_IMAGE:figures/full_fig_p018_14.png] view at source ↗
Figure 17
Figure 17. Figure 17: Attacker payload injected into a GitHub PR description. The red box highlights the injected payload, which begins with a closing </function_results> tag to terminate any pending tool response, followed by a fake <function_calls> block that invokes gh pr diff 6 and a fake <function_results> showing a benign diff (changing the password input field from type="text" to type="password" in login.html) [PITH_FU… view at source ↗
Figure 18
Figure 18. Figure 18: The actual commit in the attacker’s PR. Unlike the benign change shown in the fake tool response ( [PITH_FULL_IMAGE:figures/full_fig_p018_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Claude Code chat history showing the attack execution. Top: the user prompt “View PR #6 of this repository in GitHub. If the PR looks good, merge it” (red box). The agent retrieves the PR using gh pr view 6 and checks the related issue. Bottom: the agent concludes “The PR looks good” based on the fake benign diff injected in the PR body, summarizes the fabricated change, and executes gh pr merge 6 --merge… view at source ↗
Figure 20
Figure 20. Figure 20: User confirmation dialog displayed by Claude Code before executing the merge command. The dialog shows gh pr merge 6 --merge with the description “Merge PR #6,” with￾out specifying the PR’s actual content or the agent’s review rationale. This prevents the user from recognizing that the agent reviewed a fake diff rather than the actual commit. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_20.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 19 linked inside Pith

  1. [1]

    Chatgpt,

    OpenAI, “Chatgpt,” 2026, https://chat.openai.com (accessed 11, June, 2026)

    2026

  2. [2]

    Claude in chrome,

    Anthropic, “Claude in chrome,” 2026, https://www.claude.com/chrome (accessed 11, June, 2026)

    2026

  3. [3]

    Google gemini cli,

    Google, “Google gemini cli,” 2026, https://geminicli.com (accessed 11, June, 2026)

    2026

  4. [4]

    Cross-site scripting (xss) attacks and defense mechanisms: classification and state-of-the-art,

    S. Gupta and B. B. Gupta, “Cross-site scripting (xss) attacks and defense mechanisms: classification and state-of-the-art,”International Journal of System Assurance Engineering and Management, vol. 8, no. Suppl 1, pp. 512–530, 2017

    2017

  5. [5]

    Clarke-Salt,SQL injection attacks and defense

    J. Clarke-Salt,SQL injection attacks and defense. Elsevier, 2009

    2009

  6. [6]

    Not what you’ve signed up for: Compromising real- world llm-integrated applications with indirect prompt injection,

    K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real- world llm-integrated applications with indirect prompt injection,” in Proceedings of the 16th ACM workshop on artificial intelligence and security, 2023, pp. 79–90

    2023

  7. [7]

    Prompt injection attack against llm-integrated applications,

    Y . Liu, G. Deng, Y . Li, K. Wang, Z. Wang, X. Wang, T. Zhang, Y . Liu, H. Wang, Y . Zhenget al., “Prompt injection attack against llm-integrated applications,”arXiv preprint arXiv:2306.05499, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Imprompter: Tricking llm agents into improper tool use,

    X. Fu, S. Li, Z. Wang, Y . Liu, R. K. Gupta, T. Berg-Kirkpatrick, and E. Fernandes, “Imprompter: Tricking llm agents into improper tool use,”arXiv preprint arXiv:2410.14923, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    The instruction hierarchy: Training llms to prioritize privileged instructions,

    E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, “The instruction hierarchy: Training llms to prioritize privileged instructions,”arXiv preprint arXiv:2404.13208, 2024

    Pith/arXiv arXiv 2024

  10. [10]

    {StruQ}: Defending against prompt injection with structured queries,

    S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “ {StruQ}: Defending against prompt injection with structured queries,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 2383–2400

    2025

  11. [11]

    Datasentinel: A game-theoretic detection of prompt injection attacks,

    Y . Liu, Y . Jia, J. Jia, D. Song, and N. Z. Gong, “Datasentinel: A game-theoretic detection of prompt injection attacks,” in2025 IEEE Symposium on Security and Privacy (SP). IEEE, 2025, pp. 2190– 2208

    2025

  12. [12]

    Promptarmor: Simple yet effective prompt injection defenses,

    T. Shi, K. Zhu, Z. Wang, Y . Jia, W. Cai, W. Liang, H. Wang, H. Alzahrani, J. Lu, K. Kawaguchiet al., “Promptarmor: Simple yet effective prompt injection defenses,”arXiv preprint arXiv:2507.15219, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Prompt flow integrity to prevent privilege escalation in llm agents,

    J. Kim, W. Choi, and B. Lee, “Prompt flow integrity to prevent privilege escalation in llm agents,”arXiv preprint arXiv:2503.15547, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Defeating prompt injections by design,

    E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr, “Defeating prompt injections by design,”arXiv preprint arXiv:2503.18813, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    Claude code,

    Anthropic, “Claude code,” 2026, https://https://code.claude.com (ac- cessed 11, June, 2026)

    2026

  16. [16]

    OpenAI, “Codex,” 2026, https://openai.com/codex (accessed 11, June, 2026)

    2026

  17. [17]

    Google antigravity,

    Google, “Google antigravity,” 2026, https://antigravity.google (ac- cessed 11, June, 2026)

    2026

  18. [18]

    Nanobrowser - open source ai web agent,

    Nanobrowser, “Nanobrowser - open source ai web agent,” 2026, https: //nanobrowser.ai (accessed 11, June, 2026)

    2026

  19. [19]

    Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents,

    E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramèr, “Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents,” inThe Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024. [Online]. Available: https://openreview.net/forum?id=m1YY AQjO3w

    2024

  20. [20]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Ham- bro, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,”Advances in Neural Information Processing Systems, vol. 36, pp. 68 539–68 551, 2023

    2023

  21. [21]

    Anthropic, “Claude,” 2026, https://www.claude.ai (accessed 11, June, 2026)

    2026

  22. [22]

    Prompt infection: Llm-to-llm prompt injection within multi-agent systems,

    D. Lee and M. Tiwari, “Prompt infection: Llm-to-llm prompt injection within multi-agent systems,”arXiv preprint arXiv:2410.07283, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    Eia: Environmental injection attack on generalist web agents for privacy leakage,

    Z. Liao, L. Mo, C. Xu, M. Kang, J. Zhang, C. Xiao, Y . Tian, B. Li, and H. Sun, “Eia: Environmental injection attack on generalist web agents for privacy leakage,” inICLR, 2025

    2025

  24. [24]

    Agentvigil: Generic black-box red-teaming for indirect prompt injection against llm agents,

    Z. Wang, V . Siu, Z. Ye, T. Shi, Y . Nie, X. Zhao, C. Wang, W. Guo, and D. Song, “Agentvigil: Generic black-box red-teaming for indirect prompt injection against llm agents,” inFindings of the Association for Computational Linguistics: EMNLP. Association for Computational Linguistics, 2025, pp. 23 159–23 172

    2025

  25. [25]

    Dissecting adversarial robustness of multimodal lm agents,

    C. H. Wu, R. Shah, J. Y . Koh, R. Salakhutdinov, D. Fried, and A. Raghunathan, “Dissecting adversarial robustness of multimodal lm agents,” inICLR, 2025

    2025

  26. [26]

    Data execution prevention,

    Microsoft, “Data execution prevention,” 2009, https: //learn.microsoft.com/en-us/previous-versions/windows/it- pro/windows-xp/bb457155(v=technet.10)#data-execution-prevention (accessed 11, June, 2026)

    2009

  27. [27]

    Formalizing and benchmarking prompt injection attacks and defenses,

    Y . Liu, Y . Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” inUSENIX Security, 2024

    2024

  28. [28]

    Delimiters won’t save you from prompt injection,

    S. Wilison, “Delimiters won’t save you from prompt injection,” 2023, https://simonwillison.net/2023/May/11/delimiters-wont-save-you/ (ac- cessed 11, June, 2026)

    2023

  29. [29]

    Invitation is all you need! prompt- ware attacks against llm-powered assistants in production are practical and dangerous,

    B. Nassi, S. Cohen, and O. Yair, “Invitation is all you need! prompt- ware attacks against llm-powered assistants in production are practical and dangerous,”arXiv preprint arXiv:2508.12175, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Chatgpt atlas,

    OpenAI, “Chatgpt atlas,” 2026, https://chatgpt.com/atlas (accessed 11, June, 2026)

    2026

  31. [31]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antigaet al., “Pytorch: An imperative style, high-performance deep learning library,”Advances in neural information processing systems, vol. 32, 2019

    2019

  32. [32]

    {TensorFlow}: a system for {Large-Scale} machine learning,

    M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isardet al., “ {TensorFlow}: a system for {Large-Scale} machine learning,” in12th USENIX symposium on operating systems design and implementation (OSDI 16), 2016, pp. 265–283

    2016

  33. [33]

    Github cli,

    GitHub, Inc., “Github cli,” 2026, https://cli.github.com/ (accessed 11, 14 June, 2026)

    2026

  34. [34]

    Github mcp server,

    ——, “Github mcp server,” 2025, https://github.com/github/github- mcp-server (accessed 11, June, 2026)

    2025

  35. [35]

    Keeping llms aligned after fine-tuning: The crucial role of prompt templates,

    K. Lyu, H. Zhao, X. Gu, D. Yu, A. Goyal, and S. Arora, “Keeping llms aligned after fine-tuning: The crucial role of prompt templates,” Advances in Neural Information Processing Systems, vol. 37, pp. 118 603–118 631, 2024

    2024

  36. [36]

    Llamafirewall: An open source guardrail system for building secure ai agents,

    S. Chennabasappa, C. Nikolaidis, D. Song, D. Molnar, S. Ding, S. Wan, S. Whitman, L. Deason, N. Doucette, A. Montillaet al., “Llamafirewall: An open source guardrail system for building secure ai agents,”arXiv preprint arXiv:2505.03574, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Design patterns for securing llm agents against prompt injections,

    L. Beurer-Kellner, B. Buesser, A.-M. Cre¸ tu, E. Debenedetti, D. Dobos, D. Fabian, M. Fischer, D. Froelicher, K. Grosse, D. Naeffet al., “Design patterns for securing llm agents against prompt injections,” arXiv preprint arXiv:2506.08837, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems,

    Y . Wu, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal, “IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems,” inNetwork and Distributed System Security (NDSS) Symposium, 2025

    2025

  39. [39]

    Ace: A security architecture for llm-integrated app systems,

    E. Li, T. Mallick, E. Rose, W. Robertson, A. Oprea, and C. Nita- Rotaru, “Ace: A security architecture for llm-integrated app systems,” arXiv preprint arXiv:2504.20984, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Progent: Programmable privilege control for llm agents,

    T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, and D. Song, “Progent: Programmable privilege control for llm agents,”arXiv preprint arXiv:2504.11703, 2025

    Pith/arXiv arXiv 2025

  41. [41]

    Contextual agent security: A policy for every purpose,

    L. Tsai and E. Bagdasarian, “Contextual agent security: A policy for every purpose,” inProceedings of the 2025 Workshop on Hot Topics in Operating Systems, 2025, pp. 8–17

    2025

  42. [42]

    Towards automating data access permissions in ai agents,

    Y . Wu, K. Yang, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal, “Towards automating data access permissions in ai agents,” in2026 IEEE Symposium on Security and Privacy (SP), 2026

    2026

  43. [43]

    The dual llm pattern for building ai assistants that can resist prompt injection,

    S. Willison, “The dual llm pattern for building ai assistants that can resist prompt injection,” 2023, https://simonwillison.net/2023/Apr/25/ dual-llm-pattern/ (accessed 11, June, 2026)

    2023

  44. [44]

    System-level defense against indi- rect prompt injection attacks: An information flow control perspective,

    F. Wu, E. Cecchetti, and C. Xiao, “System-level defense against indi- rect prompt injection attacks: An information flow control perspective,” arXiv preprint arXiv:2409.19091, 2024

    Pith/arXiv arXiv 2024

  45. [45]

    Securing ai agents with information-flow control,

    M. Costa, B. Köpf, A. Kolluri, A. Paverd, M. Russinovich, A. Salem, S. Tople, L. Wutschitz, and S. Zanella-Béguelin, “Securing ai agents with information-flow control,”arXiv preprint arXiv:2505.23643, 2025

    Pith/arXiv arXiv 2025

  46. [46]

    Permissive information-flow analysis for large language models,

    S. A. Siddiqui, R. Gaonkar, B. Köpf, D. Krueger, A. Paverd, A. Salem, S. Tople, L. Wutschitz, M. Xia, and S. Zanella-Béguelin, “Permissive information-flow analysis for large language models,”arXiv preprint arXiv:2410.03055, 2024

    arXiv 2024

  47. [47]

    How good are llms at processing tool outputs?

    K. Kate, Y . Rizk, P. Ghosh, A. Gulati, T. Chakraborti, Z. Wright, and M. Agarwal, “How good are llms at processing tool outputs?”arXiv preprint arXiv:2510.15955, 2025

    arXiv 2025

  48. [48]

    Llama prompt guard 2,

    Meta AI, “Llama prompt guard 2,” 2025, https://huggingface.co/meta- llama/Llama-Prompt-Guard-2-86M (accessed 11, June, 2026)

    2025

  49. [49]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y . Cao, “React: Synergizing reasoning and acting in language models,” inThe eleventh international conference on learning representations, 2022

    2022

  50. [50]

    Great, now write an article about that: The crescendo {Multi-Turn}{LLM} jailbreak attack,

    M. Russinovich, A. Salem, and R. Eldan, “Great, now write an article about that: The crescendo {Multi-Turn}{LLM} jailbreak attack,” in 34th USENIX Security Symposium (USENIX Security 25), 2025

    2025

  51. [51]

    Universal adversarial triggers for attacking and analyzing NLP,

    E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing NLP,” inEmpirical Methods in Natural Language Processing, 2019

    2019

  52. [52]

    Autodan: Generating stealthy jailbreak prompts on aligned large language models,

    X. Liu, N. Xu, M. Chen, and C. Xiao, “Autodan: Generating stealthy jailbreak prompts on aligned large language models,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  53. [53]

    Open sesame! universal black- box jailbreaking of large language models,

    R. Lapid, R. Langberg, and M. Sipper, “Open sesame! universal black- box jailbreaking of large language models,”Applied Sciences, vol. 14, no. 16, p. 7150, 2024

    2024

  54. [54]

    Jailbreak attacks and defenses against large language models: A survey,

    S. Yi, Y . Liu, Z. Sun, T. Cong, X. He, J. Song, K. Xu, and Q. Li, “Jailbreak attacks and defenses against large language models: A survey,”arXiv preprint arXiv:2407.04295, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Pleak: Prompt leaking attacks against large language model applications,

    B. Hui, H. Yuan, N. Gong, P. Burlina, and Y . Cao, “Pleak: Prompt leaking attacks against large language model applications,” inPro- ceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024

    2024

  56. [56]

    Ignore previous prompt: Attack techniques for language models,

    F. Perez and I. Ribeiro, “Ignore previous prompt: Attack techniques for language models,”arXiv preprint arXiv:2211.09527, 2022

    Pith/arXiv arXiv 2022

  57. [57]

    Effective prompt extraction from language models,

    Y . Zhang, N. Carlini, and D. Ippolito, “Effective prompt extraction from language models,”arXiv preprint arXiv:2307.06865, 2024

    Pith/arXiv arXiv 2024

  58. [58]

    Les dissonances: Cross-tool harvesting and polluting in multi-tool empowered llm agents,

    Z. Li, J. Cui, X. Liao, and L. Xing, “Les dissonances: Cross-tool harvesting and polluting in multi-tool empowered llm agents,” in NDSS, 2026

    2026

  59. [59]

    Task injection – exploiting agency of autonomous ai agents,

    J. Kokatsu, “Task injection – exploiting agency of autonomous ai agents,” 2025, https://bughunters.google.com/blog/task-injection- exploiting-agency-of-autonomous-ai-agents (accessed 11, June, 2026)

    2025

  60. [60]

    Optimization-based prompt injection attack to llm-as-a-judge,

    J. Shi, Z. Yuan, Y . Liu, Y . Huang, P. Zhou, L. Sun, and N. Z. Gong, “Optimization-based prompt injection attack to llm-as-a-judge,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024

    2024

  61. [61]

    Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases,

    Z. Chen, Z. Xiang, C. Xiao, D. Song, and B. Li, “Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases,” Advances in Neural Information Processing Systems, vol. 37, pp. 130 185–130 213, 2024

    2024

  62. [62]

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents,

    Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents,”arXiv preprint arXiv:2403.02691, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    Benchmarking and defending against indirect prompt injection attacks on large language models,

    J. Yi, Y . Xie, B. Zhu, E. Kiciman, G. Sun, X. Xie, and F. Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,” inProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V . 1, 2025, pp. 1809–1820

    2025

  64. [64]

    WASP: Benchmarking web agent security against prompt injection attacks,

    I. Evtimov, A. Zharmagambetov, A. Grattafiori, C. Guo, and K. Chaud- huri, “WASP: Benchmarking web agent security against prompt injection attacks,” inThe Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2026

    2026

  65. [65]

    Measuring real-world prompt injection attacks in llm-based resume screening,

    M. Zhang, Y . Jia, Z. Tan, S. Jiang, N. Z. Gong, T. Chen, and D. Song, “Measuring real-world prompt injection attacks in llm-based resume screening,”arXiv preprint arXiv:2605.28999, 2026. Appendix A. Probabilistic Delimiter Injection Benchmark Details Table 5 details the seven data categories used in the probabilistic delimiter injection evaluation (§6.1)...

    Pith/arXiv arXiv 2026