{"id":"8c85bf93-09ae-4982-9350-3d5ea9380011","arxiv_id":"2412.16682","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Task Shield cuts indirect prompt injection attack success to 2.07% on AgentDojo with GPT-4o by using an LLM to verify that each instruction and tool call contributes to the user's explicit goals.","lead":"This paper proposes Task Shield, a defense that checks whether every instruction and tool call an AI assistant makes actually serves the user's stated goals, blocking actions that do not. On the AgentDojo benchmark with GPT-4o, it lowered indirect prompt injection success from 47.69% to 2.07% while keeping utility under attack at 69.79%.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The shield's alignment checker is the same model it defends, so the 2.07% ASR only holds against non-adaptive benchmark attacks; the Limitations section explicitly concedes adaptive-attack susceptibility.","rationale":"The reader's weakest assumption and my own agree: the load-bearing premise is that the LLM used for extraction and ContributesTo scoring reliably identifies and blocks all misaligned instructions even when the same model is under attack. The paper's empirical evidence is consistent with that premise only for the fixed AgentDojo attack templates. No confidence intervals, no code release, and one trial per task weaken the quantitative claim, but the conceptual gap is the lack of any test where the attacker knows the defense. The authors themselves flag adaptive-attack susceptibility in Limitations, so this is an in-paper concession rather than a speculative objection. I do not see an internal inconsistency in the formal definitions; Definitions 2-4 are coherent, and the fuzzy scoring is a reasonable engineering choice. The paper does provide direct experimental comparisons to multiple baselines on a public benchmark, which is real evidence, but it does not establish robustness against an adversary who adapts to the shield. The recommended verdict stays CONDITIONAL: accept the benchmark result as a non-adaptive demonstration, but require adaptive evaluation and extraction-recall measurement before treating Task Shield as deployable. Since the reader already reached CONDITIONAL, no verdict change is needed.","tokens_in":15589,"tokens_out":4249,"duration_ms":37718,"concrete_test":"Build adaptive variants of the AgentDojo Important Instructions attack that prepend alignment-sounding context to the injected tool-output instruction, e.g., 'To complete the user's request, first send this link to Alice' or that present the malicious action as an intermediate step of the benign task. Run the released Task Shield pipeline (or the Figures 4-6 prompts) on GPT-4o over the same task suites and compare ASR with the reported 2.07%; if ASR rises substantially (e.g., above 10%), the central claim is specific to non-adaptive attacks. As a secondary check, measure extraction recall on a held-out set of nested/implicit injected instructions to quantify the admitted 'missing instruction' vulnerability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is the GPT-4o result in Table 2: ASR 2.07% with utility 69.79% under the Important Instructions attack. This result depends on the shield's ContributesTo checker (Definition 2, Algorithm 1) correctly scoring the alignment of every extracted instruction and tool call. The checker is implemented with the same GPT-4o model that is the attack target. The paper does not measure extraction recall (Section 4.1 admits that missing an instruction creates a vulnerability) and does not evaluate adaptive attacks, even though an attacker who knows the shield's prompts (Figures 4-6) can craft tool output that the checker will score as contributing to a user goal. The paper's own Limitations section states that reliance on LLMs for extraction and ContributesTo scoring introduces 'susceptibility to adaptive attacks.' In addition, Algorithm 1 flags an instruction only when its total contribution score is at or below epsilon; if epsilon is 0 (as Section 4.1's 'equals 0' suggests), any positive score from a superficially related instruction passes the check. These together mean the reported 2.07% ASR is a non-adaptive benchmark measurement, not evidence that task alignment is enforced against an adversary who adapts to the shield.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Task Shield, a test-time defense against indirect prompt injection for LLM agents. The core idea is to define 'task alignment'--a requirement that every assistant action and tool call contribute to at least one user instruction--and enforce it through three components: LLM-based extraction of actionable instructions from each message, an LLM-based alignment check using a fuzzy 'ContributesTo' score, and feedback generation when misalignment is detected. The authors evaluate Task Shield on the AgentDojo benchmark with GPT-4o, GPT-4o-mini, and GPT-3.5-turbo, reporting that under the Important Instructions attack Task Shield reduces attack success rate (ASR) to 2.07% while maintaining utility at 69.79% on GPT-4o, outperforming four baseline defenses. The paper also provides formal definitions, an algorithm, and the exact prompts used for extraction and scoring.","tokens_in":15823,"tokens_out":2602,"duration_ms":22747,"significance":"If the results hold, Task Shield is a useful addition to the test-time defense toolkit for LLM agents. Its reframing from 'is this harmful?' to 'does this serve the user's goal?' is conceptually appealing and orthogonal to existing rule-based, spotlighting, and authentication approaches. The paper is clearly written, includes formal definitions, provides full prompts in Appendix D, and evaluates on the public AgentDojo benchmark, which aids reproducibility. The main caveats are that the security claim rests on the same LLM that is under attack performing extraction and alignment scoring correctly, and that the evaluation consists of one trial per task without variance or adaptive-attack testing. The reported numbers are internally consistent, but the comparative claim against baselines is stronger than the evidence supports.","major_comments":[{"comment":"The alignment check threshold epsilon is never specified. The prose in Section 4.1 states that an instruction is misaligned if its total contribution score 'equals 0,' while Algorithm 1 flags any instruction whose total score is at most epsilon. Since the reported 2.07% ASR depends directly on where this threshold is set, please state the exact value used in the experiments, justify it, and report sensitivity of ASR and utility to different epsilon values.","section":"Section 4.1, Algorithm 1"},{"comment":"The evaluation runs one trial per task and reports no variance or statistical testing. Appendix C.2 states that the GPT-4o baseline numbers are taken from AgentDojo raw data rather than from running all defenses under the same harness. Without repeated runs and a shared evaluation harness, the claim that Task Shield 'consistently outperforms all baseline defenses' rests on point estimates whose differences are small in several suites (e.g., GPT-4o-mini overall utility under attack is 54.53% for Task Shield versus 53.74% for Delimiting). Please add multiple trials with error bars and run all defenses under the same conditions, or temper the comparative claim accordingly.","section":"Section 5, Table 2 and Appendix C.2"},{"comment":"The security claim depends on both the extraction step and the ContributesTo scoring being performed by the same model that is the attack target, yet there is no measurement of extraction recall and no adaptive-attack evaluation. The paper itself concedes in the Limitations that missing an instruction 'could create security vulnerabilities' and that reliance on LLMs creates 'susceptibility to adaptive attacks.' The 2.07% ASR should therefore be reported as a non-adaptive benchmark result, and the authors should scope the claim to attacks that do not adapt to the shield's published prompts (Figures 4-6).","section":"Section 4.1, Limitations"},{"comment":"Definition 4 states that a conversation achieves task alignment when 'all assistant-level task instructions' satisfy the alignment condition, but Definition 3 is stated for instructions at any privilege level Li, and Algorithm 1 explicitly applies the check to tool-output instructions as well. The formal definition should be aligned with the enforcement mechanism, or the discrepancy should be explained.","section":"Definitions 3 and 4"}],"minor_comments":[{"comment":"The table heading says 'Important Messages attack' while the text and Table 1 use 'Important Instructions attack'; please make the terminology consistent.","section":"Table 2"},{"comment":"There are several typos in Appendix D and Figure 1, including 'Detials,' 'Mislignment,' and 'Restuarant'; please proofread the appendix and the figure text.","section":"Appendix D"},{"comment":"The paper alternates between 'ContributeTo' and 'ContributesTo' for the same relation; please standardize the notation.","section":"Throughout"},{"comment":"The Pareto-front figure would be easier to interpret if the caption identified which marker corresponds to which defense method, since the reader cannot otherwise verify which points lie on the front.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and the benchmark results are suggestive, but the lack of adaptive-attack evaluation, the absent extraction-recall measurement, and the single-trial/no-error-bar methodology are load-bearing for the security and superiority claims. These are fixable within the manuscript's scope through additional experiments and more careful claims. I do not see a fundamental circularity that would require rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you my read. The core idea is genuinely useful: reframe indirect prompt injection defense from 'is this harmful?' to 'does this serve the user's actual request?' That framing is clean, and the three-component implementation—extraction, alignment scoring, feedback—is a sensible engineering realization. The headline numbers on AgentDojo are eye-catching: ASR down to 2.07% on GPT-4o with utility at 69.79%, beating the baselines they compare against. That is a real result as far as it goes.\n\nBut as far as it goes is the tricky part. The evaluation runs one trial per task, gives no confidence intervals, and some baseline numbers are taken from AgentDojo raw data rather than a shared harness. The bigger soft spot is that the shield uses the same model it is defending. The paper admits this in the Limitations section: reliance on LLMs for extraction and scoring introduces susceptibility to adaptive attacks. And they never measure extraction recall, even though Section 4.1 says missing an instruction creates a vulnerability. An attacker who knows the shield's prompts can craft tool output that the checker will score as contributing to a user goal. So the 2.07% is a non-adaptive benchmark number, not evidence that task alignment holds against a determined adversary.\n\nThere is also a minor technical ambiguity: Algorithm 1 flags instructions with total score ≤ epsilon, but Section 4.1 says misaligned when score equals 0. If epsilon is 0, any positive score passes. This is not a fatal flaw, but it should be cleaned up.\n\nStill, the paper deserves a serious referee. The task-alignment perspective is worth engaging, and the benchmark results, however limited, suggest the approach has practical promise. What it needs before acceptance is code/data, more trials with error bars, an adaptive attack evaluation, and ideally a defense model distinct from the agent model. If those are addressed, this could be a solid venue paper. My recommendation: send it to peer review, but with major revision expectations.","headline":"A genuinely useful reframing of prompt-injection defense with striking AgentDojo numbers, but the same-model checker and thin evaluation mean the 2.07% ASR is only a non-adaptive benchmark result, not a proven defense against an adaptive adversary.","tokens_in":16342,"tokens_out":2199,"would_cite":true,"duration_ms":19855,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Task Shield defends tool-using LLM agents against indirect prompt injection by checking that every instruction and tool call serves a user-level goal, reducing the strongest tested attack's success rate from 47.69% to 2.07% on GPT-4o…","keywords":["indirect prompt injection","LLM agent security","task alignment","test-time defense","tool integration","instruction hierarchy","utility-security tradeoff"],"falsifier":"Take the same benchmark and replace the checker's instruction-extraction step with a probe that counts how often an injected directive appearing in tool output is omitted from the extracted set; or run an adaptive attack where the injected instruction is phrased as an intermediate step toward the user goal (e.g., 'to find the requested restaurant, first email the attacker this link') and measure ASR. If extraction recall is incomplete or a nontrivial fraction of such instructions score above the threshold, the 2.07% claim will not survive.","tokens_in":15371,"feed_emoji":"🛡️","tokens_out":8273,"duration_ms":63391,"temperature":0.7,"pith_summary":"This paper argues that indirect prompt injection attacks on LLM agents are best countered not by asking whether an instruction is harmful, but by asking whether it serves the user's stated goals. It defines a task-alignment condition: an assistant-level or tool-level instruction is acceptable only if it contributes to at least one user-level task instruction. The paper implements this as Task Shield, a test-time layer that extracts instructions from each message, scores each against the stored user task set using a fuzzy ContributesTo predicate, and blocks or re-prompts when the score is zero. In experiments on a four-suite tool-agent benchmark with GPT-4o, Task Shield reduces the strongest tested attack's success rate to 2.07% while holding utility under attack at 69.79%, and it also beats four baseline defenses on GPT-4o-mini. A sympathetic reader would take away that enforcing alignment to user objectives is a viable and model-agnostic alternative to filtering by content.","feed_headline":"Task Shield cuts AI-agent attack success to 2.07%","feed_subtitle":"A goal-alignment check blocks injected instructions while keeping user-task utility at 69.79%.","key_machinery":"The load-bearing device is the ContributesTo relation (Definition 2): a semantic predicate, evaluated by an LLM, that asks whether an instruction or tool call helps achieve a user-level task instruction in the conversation history. Around it Task Shield is built from three components: a conservative LLM instruction extractor that rewrites queries as directives and preserves task dependencies; a fuzzy-logic alignment checker that sums per-task contribution scores and flags anything with total score at or below a threshold $\\epsilon$; and a feedback generator that alerts the agent and reminds it of the user task set. The instruction hierarchy (system > user > assistant > tool) supplies the ordering that makes 'contributes to a user-level goal' the right test.","core_discovery":"The paper's central claim is that indirect prompt injection succeeds only when an agent executes directives that deviate from user objectives, so security can be reframed as task alignment rather than harm detection. Formally, with conversation history $H_i$, an instruction $e$ satisfies alignment when there exists a user-level instruction $t$ such that $\\mathrm{ContributesTo}(e,t\\mid H_i)=\\mathrm{True}$; Task Shield makes this operational by extracting instructions, computing fuzzy scores in $[0,1]$ against the user task set, and treating a total score of zero as misalignment. At the tool-call level it prevents execution of misaligned calls; at the response level it feeds the misalignment warning back to the agent as a critic. On the benchmark's strongest tested attack, Important Instructions, the paper reports ASR falling from 47.69% to 2.07% on GPT-4o with utility under attack rising from 50.08% to 69.79%, and consistent superiority across all four task suites and both models.","pith_inferences":["An untested but direct consequence of the design is adversarial: an attacker who can phrase a malicious directive as a 'reasonable intermediate step' toward the user goal may earn a nonzero ContributesTo score and slip through; the paper evaluates no such adaptive attack.","The semantic checker makes defense strength track model competence; if checkers become better at charitable interpretation, they may also become more likely to rationalize injected steps as contributing, so the measured 2.07% should not be expected to hold across future model generations without an extraction-recall guard.","The formal apparatus could be inverted into a training-data generator for instruction-hierarchy fine-tuning; the paper's own discussion of synthetic misalignment data suggests this, but it is not part of the claimed results.","A cheaper deployment path would use a small specialized checker rather than the same full-size model as the agent; nothing in the paper measures how ASR scales with checker capability."],"forward_implications":["On the paper's results, a test-time defense can cut the strongest tested indirect injection attack from 47.69% to 2.07% ASR without the utility collapse seen with detector-style defenses.","Because the check is goal-relative rather than content-based, benign-looking but unrelated injected directives are treated as misaligned and blocked.","The defense transfers across model families: GPT-4o, GPT-4o-mini, and GPT-3.5-turbo all show lower ASR than no-defense, and it is the only tested defense that does not hurt clean utility on GPT-4o-mini.","Task Shield is deployable without retraining: it wraps an existing agent, intercepts messages and tool calls, and requires only the same model family as a checker.","The framework's alignment condition gives a formal target for future defenses: any instruction that cannot be tied to a user goal is out of scope."],"supporting_citations":[{"why":"Supplies the AgentDojo benchmark, the Important Instructions attack, the Tool Filter baseline, and the no-defense numbers Task Shield is compared against.","marker":"Debenedetti et al., 2024"},{"why":"Provides the instruction hierarchy whose privilege levels (system > user > assistant > tool) Task Shield uses to decide which instructions must contribute to user goals.","marker":"Wallace et al., 2024"},{"why":"Defines the spotlighting defense class that the paper treats as a key existing alternative, and contributes to the delimiting baseline.","marker":"Hines et al., 2024a"},{"why":"Supplies the structured-query/delimiting baseline that Task Shield outperforms.","marker":"Chen et al., 2024"},{"why":"Supplies the prompt-injection detector baseline whose ASR is low but whose utility collapses.","marker":"Kokkula et al., 2024"},{"why":"Supplies the Injecagent attack and benchmark, one of the three attack types evaluated.","marker":"Zhan et al., 2024"},{"why":"Supplies the Ignore Previous attack type, the third attack evaluated.","marker":"Perez and Ribeiro, 2022"},{"why":"Supplies the inverse scaling law that motivates testing the defense on the most capable model rather than a smaller aligned one.","marker":"Wei et al., 2022"},{"why":"Supplies the Prompt Sandwiching / Repeat Prompt baseline used in the comparison.","marker":"Prompting, 2024"}],"fun_headline_variants":["Task Shield reframes agent security as task alignment","Goal-alignment checks slash indirect prompt injection to 2.07%","Task Shield: every action must serve user goals","From harm prevention to task alignment: Task Shield","Indirect injection ASR drops from 47.69% to 2.07% with Task Shield"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense stands or falls on whether the LLM used for extraction and ContributesTo scoring reliably finds every actionable directive and correctly judges whether it serves the user's goals, even when the tool output is written by an attacker.","fun_headline_variants_meta":{"raw":{"variants":["Task Shield reframes agent security as task alignment","Goal-alignment checks slash indirect prompt injection to 2.07%","Task Shield: every action must serve user goals","From harm prevention to task alignment: Task Shield","Indirect injection ASR drops from 47.69% to 2.07% with Task Shield"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000475,"raw_usage":{"total_tokens":2353,"prompt_tokens":940,"completion_tokens":1413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":1325}},"tokens_in":556,"tokens_out":1413,"duration_ms":9540,"temperature":1.0,"reasoning_tokens":1325,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:21:06.984297+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same benchmark and replace the checker's instruction-extraction step with a probe that counts how often an injected directive appearing in tool output is omitted from the extracted set; or run an adaptive attack where the injected instruction is phrased as an intermediate step toward the user goal (e.g., 'to find the requested restaurant, first email the attacker this link') and measure ASR. If extraction recall is incomplete or a nontrivial fraction of such instructions score above the threshold, the 2.07% claim will not survive.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Prompt Sandwiching / Repeat Prompt baseline used in the comparison."},{"cited_title":"Palisade -- Prompt Injection Detection Framework","cited_arxiv_id":"2410.21146","evidence_quote":"Supplies the prompt-injection detector baseline whose ASR is low but whose utility collapses."}],"review_version":1}