REVIEW 2 major objections 1 minor
IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents
T0 review · 2 major / 1 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read IPIGuard claims that pre-planning every allowed tool call before reading external data blocks indirect prompt injection in LLM agents.
desk verdict A plausible structural defense against indirect prompt injection, but the abstract leaves the load-bearing assumption about static tool-dependency graphs unexamined. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Tool Dependency Graph (TDG) is a pre-planned map of which tools the task may call and how their outputs feed one another, constructed before any external data is read. It carries the argument by acting as a fixed action policy: from that point on, the agent's behavior is restricted to graph-permitted tool invocations, so injected instructions in external data cannot trigger calls outside the plan.
What would settle it
Give IPIGuard a task where the correct second tool call is determined by text inside the first tool's response, run it with an attacker-controlled response, and check whether the agent either fails the task or makes an unplanned call. A second test: embed an injection that asks for a tool call that is within the planned graph but with malicious arguments; if it succeeds, IPIGuard does not block all injection-driven harm.
Extended reading notes
Core claim
The paper's central claim is that indirect prompt injection succeeds largely because action planning and external data reading happen in the same reasoning loop, so injected text can steer which tools get called. IPIGuard splits that loop: first construct a Tool Dependency Graph from the user request, defining which tools may be invoked and in what dependency order; then execute only tools allowed by that graph as external data arrives. Task execution becomes a traversal over the planned graph rather than a free-form model decision. The paper argues that this significantly reduces unintended tool invocations triggered by injected instructions and, on a public agent-security benchmark, achiev
Load-bearing premise
The method assumes every task can be fully planned before external data is read; if a task requires choosing the next tool based on retrieved content, the graph constraint either blocks a legitimate action or is broken.
Editorial extensions
If this is right
- Injected instructions in retrieved data can no longer summon tools that the pre-planned graph does not contain, so a whole class of IPI attacks is blocked before the model can act on it.
- Because the defense is structural, it stays effective even when the underlying model has weak instruction-following resistance; it can be combined with prompt-based or detection-based defenses rather than replacing them.
- Task success now depends on the completeness of the pre-planned graph: tasks that need dynamic re-planning from retrieved content face a trade-off between safety and capability.
- The benchmark results indicate that a single defense can maintain utility while substantially reducing attack success, supporting the feasibility of safer agentic systems in untrusted environments.
Reading between the lines
- A natural testable extension: apply IPIGuard to tasks with deliberately hidden branching, where the correct tool depends on content that only arrives after the first call; if success drops, the defense's ceiling is exactly the set of fully pre-plannable tasks.
- The same 'plan before reading untrusted data' principle could apply to other automation pipelines—browser automation, API orchestration, or database tools—where injected content can alter a later action.
- One open vector the paper does not address: the user request itself is trusted, but if an attacker can influence graph construction (e.g., through a compromised user profile), the pre-planned graph could contain malicious edges; detecting that would require a second look at the request.
- A concrete measurement worth making: compare attack success rates when the injected instruction targets a tool that is inside the graph but executes at a different dependency stage; such 'in-graph' attacks may be the next adversarial focus.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes IPIGuard, a defense against indirect prompt injection (IPI) in LLM agents. The approach models a task as a planned Tool Dependency Graph (TDG) and explicitly decouples action planning from interaction with external untrusted data, with the goal of preventing malicious tool invocations at the source rather than relying on the model's inherent resistance to injected instructions. The abstract reports experiments on the AgentDojo benchmark, claiming a superior balance between task effectiveness and robustness. As only the abstract is available, the report assesses the claims that are stated and identifies the key missing evidence.
Significance. If the claimed results hold, IPIGuard would offer a structural, model-independent defense against a recognized practical threat, which is a valuable direction. The central idea of pre-planning tool dependencies before any external data is read is a concrete, falsifiable proposal. However, the abstract provides no experimental methodology, task composition, baselines, metrics, or variance estimates, and it leaves a pivotal assumption about static versus dynamic planning unaddressed. The significance is therefore conditional on evidence that is not yet visible in the submitted material.
major comments (2)
- [Abstract] The empirical claim 'Experiments on the AgentDojo benchmark show that IPIGuard achieves a superior balance between effectiveness and robustness' is not supported by any methodological detail. The abstract gives no benchmark-task description, no baseline defenses, no evaluation metrics, no error bars, and no ablations. Because this claim is the primary evidence for the paper's central conclusion, it is load-bearing and cannot be assessed without the full experimental section.
- [Abstract] The proposed paradigm models execution as 'a traversal over a planned Tool Dependency Graph' and 'explicitly decoupling action planning from interaction with external data.' This implies the graph is constructed before external data is read. Many real agent tasks are reactive: the agent may need to decide whether to issue a new web search based on the contents of a retrieved page, or conditionally invoke a write or transformation tool depending on whether a file is malformed. The abstract gives no mechanism for dynamic replanning, and any dynamic graph extension would appear to reintroduce the injection surface the defense aims to close. Without clarifying how such cases are handled, the claimed superiority on AgentDojo may not generalize beyond tasks with fully static dependency structures.
minor comments (1)
- [Abstract] The abstract defines the Tool Dependency Graph but gives no intuition about its construction, complexity, or the trust assumptions associated with it. A sentence describing the graph's node and edge semantics would improve readability and make the contribution more precise.
Circularity Check
No circularity in the abstract; the defense is structurally defined and evaluated against an external benchmark.
full rationale
The abstract presents a derivation chain: IPI attacks exploit unrestricted tool access; existing defenses rely on model-inherent security; IPIGuard instead constrains tool invocations by planning a Tool Dependency Graph and decoupling planning from external data interaction; experiments on AgentDojo are used to assess effectiveness and robustness. None of these steps reduces to its own input. The TDG is introduced as a new architectural mechanism, not as a renamed version of the attack or the benchmark. No equation is given, no parameter is fitted to the evaluation data and then reported as a prediction, and no self-citation is invoked as load-bearing evidence. The key assumption that the TDG can be fully planned before reading external data is a correctness/effectiveness concern for dynamic tasks, but it is not circularity: the defense and the threat model remain distinct. Since only the abstract is available, deeper circularity cannot be ruled out, but on the evidence provided there is no identifiable circular step.
Assumptions & free parameters
assumptions (3)
- domain assumption LLM agents use tools to access external data and those tool responses may contain injected instructions.
- domain assumption Decoupling action planning from interaction with external data is feasible for the tasks considered.
- domain assumption The AgentDojo benchmark accurately represents real-world IPI attack scenarios.
invented entities (1)
-
Tool Dependency Graph (TDG)
Cite this review
Pith. "Pith review of IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents." pith.science (2026). https://pith.science/paper/EQUXEPUA
@misc{pith2026250815310,
author = {Pith},
title = {Pith review of: IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents},
year = {2026},
howpublished = {\url{https://pith.science/paper/EQUXEPUA}},
note = {Machine review of arXiv:2508.15310}
}
read the original abstract
Large language model (LLM) agents are widely deployed in real-world applications, where they leverage tools to retrieve and manipulate external data for complex tasks. However, when interacting with untrusted data sources (e.g., fetching information from public websites), tool responses may contain injected instructions that covertly influence agent behaviors and lead to malicious outcomes, a threat referred to as Indirect Prompt Injection (IPI). Existing defenses typically rely on advanced prompting strategies or auxiliary detection models. While these methods have demonstrated some effectiveness, they fundamentally rely on assumptions about the model's inherent security, which lacks structural constraints on agent behaviors. As a result, agents still retain unrestricted access to tool invocations, leaving them vulnerable to stronger attack vectors that can bypass the security guardrails of the model. To prevent malicious tool invocations at the source, we propose a novel defensive task execution paradigm, called IPIGuard, which models the agents' task execution process as a traversal over a planned Tool Dependency Graph (TDG). By explicitly decoupling action planning from interaction with external data, IPIGuard significantly reduces unintended tool invocations triggered by injected instructions, thereby enhancing robustness against IPI attacks. Experiments on the AgentDojo benchmark show that IPIGuard achieves a superior balance between effectiveness and robustness, paving the way for the development of safer agentic systems in dynamic environments.
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.