Pith. sign in

REVIEW 4 major objections 4 minor 5 cited by

The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Task Shield defends tool-using LLM agents against indirect prompt injection by checking that every instruction and tool call serves a user-level goal, reducing the strongest tested attack's success rate from 47.69% to 2.07% on GPT-4o…

desk verdict A genuinely useful reframing of prompt-injection defense with striking AgentDojo numbers, but the same-model checker and thin evaluation mean the 2.07% ASR is only a non-adaptive benchmark result, not a proven defense against an adaptive adversary. read the letter →

arxiv 2412.16682 v1 pith:RBNY55GU submitted 2024-12-21 cs.CR cs.AIcs.CLcs.LG

Feiran Jia , Tong Wu , Xin Qin , Anna Squicciarini This is my paper · ORCID
classification cs.CRcs.AIcs.CLcs.LG
keywords indirectpromptinjectionLLMagentsecuritytaskalignmenttest-timedefensetoolintegrationinstructionhierarchyutility-securitytradeoff
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that indirect prompt injection attacks on LLM agents are best countered not by asking whether an instruction is harmful, but by asking whether it serves the user's stated goals. It defines a task-alignment condition: an assistant-level or tool-level instruction is acceptable only if it contributes to at least one user-level task instruction. The paper implements this as Task Shield, a test-time layer that extracts instructions from each message, scores each against the stored user task set using a fuzzy ContributesTo predicate, and blocks or re-prompts when the score is zero. In experiments on a four-suite tool-agent benchmark with GPT-4o, Task Shield reduces the strongest tested attack's success rate to 2.07% while holding utility under attack at 69.79%, and it also beats four baseline defenses on GPT-4o-mini. A sympathetic reader would take away that enforcing alignment to user objectives is a viable and model-agnostic alternative to filtering by content.

What carries the argument

The load-bearing device is the ContributesTo relation (Definition 2): a semantic predicate, evaluated by an LLM, that asks whether an instruction or tool call helps achieve a user-level task instruction in the conversation history. Around it Task Shield is built from three components: a conservative LLM instruction extractor that rewrites queries as directives and preserves task dependencies; a fuzzy-logic alignment checker that sums per-task contribution scores and flags anything with total score at or below a threshold $\epsilon$; and a feedback generator that alerts the agent and reminds it of the user task set. The instruction hierarchy (system > user > assistant > tool) supplies the ordering that makes 'contributes to a user-level goal' the right test.

What would settle it

Take the same benchmark and replace the checker's instruction-extraction step with a probe that counts how often an injected directive appearing in tool output is omitted from the extracted set; or run an adaptive attack where the injected instruction is phrased as an intermediate step toward the user goal (e.g., 'to find the requested restaurant, first email the attacker this link') and measure ASR. If extraction recall is incomplete or a nontrivial fraction of such instructions score above the threshold, the 2.07% claim will not survive.

Watch

Extended reading notes

Core claim

The paper's central claim is that indirect prompt injection succeeds only when an agent executes directives that deviate from user objectives, so security can be reframed as task alignment rather than harm detection. Formally, with conversation history $H_i$, an instruction $e$ satisfies alignment when there exists a user-level instruction $t$ such that $\mathrm{ContributesTo}(e,t\mid H_i)=\mathrm{True}$; Task Shield makes this operational by extracting instructions, computing fuzzy scores in $[0,1]$ against the user task set, and treating a total score of zero as misalignment. At the tool-call level it prevents execution of misaligned calls; at the response level it feeds the misalignment warning back to the agent as a critic. On the benchmark's strongest tested attack, Important Instructions, the paper reports ASR falling from 47.69% to 2.07% on GPT-4o with utility under attack rising from 50.08% to 69.79%, and consistent superiority across all four task suites and both models.

Load-bearing premise

The defense stands or falls on whether the LLM used for extraction and ContributesTo scoring reliably finds every actionable directive and correctly judges whether it serves the user's goals, even when the tool output is written by an attacker.

Editorial extensions

If this is right

  • On the paper's results, a test-time defense can cut the strongest tested indirect injection attack from 47.69% to 2.07% ASR without the utility collapse seen with detector-style defenses.
  • Because the check is goal-relative rather than content-based, benign-looking but unrelated injected directives are treated as misaligned and blocked.
  • The defense transfers across model families: GPT-4o, GPT-4o-mini, and GPT-3.5-turbo all show lower ASR than no-defense, and it is the only tested defense that does not hurt clean utility on GPT-4o-mini.
  • Task Shield is deployable without retraining: it wraps an existing agent, intercepts messages and tool calls, and requires only the same model family as a checker.
  • The framework's alignment condition gives a formal target for future defenses: any instruction that cannot be tied to a user goal is out of scope.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested but direct consequence of the design is adversarial: an attacker who can phrase a malicious directive as a 'reasonable intermediate step' toward the user goal may earn a nonzero ContributesTo score and slip through; the paper evaluates no such adaptive attack.
  • The semantic checker makes defense strength track model competence; if checkers become better at charitable interpretation, they may also become more likely to rationalize injected steps as contributing, so the measured 2.07% should not be expected to hold across future model generations without an extraction-recall guard.
  • The formal apparatus could be inverted into a training-data generator for instruction-hierarchy fine-tuning; the paper's own discussion of synthetic misalignment data suggests this, but it is not part of the claimed results.
  • A cheaper deployment path would use a small specialized checker rather than the same full-size model as the agent; nothing in the paper measures how ASR scales with checker capability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Task Shield, a test-time defense against indirect prompt injection for LLM agents. The core idea is to define 'task alignment'--a requirement that every assistant action and tool call contribute to at least one user instruction--and enforce it through three components: LLM-based extraction of actionable instructions from each message, an LLM-based alignment check using a fuzzy 'ContributesTo' score, and feedback generation when misalignment is detected. The authors evaluate Task Shield on the AgentDojo benchmark with GPT-4o, GPT-4o-mini, and GPT-3.5-turbo, reporting that under the Important Instructions attack Task Shield reduces attack success rate (ASR) to 2.07% while maintaining utility at 69.79% on GPT-4o, outperforming four baseline defenses. The paper also provides formal definitions, an algorithm, and the exact prompts used for extraction and scoring.

Significance. If the results hold, Task Shield is a useful addition to the test-time defense toolkit for LLM agents. Its reframing from 'is this harmful?' to 'does this serve the user's goal?' is conceptually appealing and orthogonal to existing rule-based, spotlighting, and authentication approaches. The paper is clearly written, includes formal definitions, provides full prompts in Appendix D, and evaluates on the public AgentDojo benchmark, which aids reproducibility. The main caveats are that the security claim rests on the same LLM that is under attack performing extraction and alignment scoring correctly, and that the evaluation consists of one trial per task without variance or adaptive-attack testing. The reported numbers are internally consistent, but the comparative claim against baselines is stronger than the evidence supports.

major comments (4)
  1. [Section 4.1, Algorithm 1] The alignment check threshold epsilon is never specified. The prose in Section 4.1 states that an instruction is misaligned if its total contribution score 'equals 0,' while Algorithm 1 flags any instruction whose total score is at most epsilon. Since the reported 2.07% ASR depends directly on where this threshold is set, please state the exact value used in the experiments, justify it, and report sensitivity of ASR and utility to different epsilon values.
  2. [Section 5, Table 2 and Appendix C.2] The evaluation runs one trial per task and reports no variance or statistical testing. Appendix C.2 states that the GPT-4o baseline numbers are taken from AgentDojo raw data rather than from running all defenses under the same harness. Without repeated runs and a shared evaluation harness, the claim that Task Shield 'consistently outperforms all baseline defenses' rests on point estimates whose differences are small in several suites (e.g., GPT-4o-mini overall utility under attack is 54.53% for Task Shield versus 53.74% for Delimiting). Please add multiple trials with error bars and run all defenses under the same conditions, or temper the comparative claim accordingly.
  3. [Section 4.1, Limitations] The security claim depends on both the extraction step and the ContributesTo scoring being performed by the same model that is the attack target, yet there is no measurement of extraction recall and no adaptive-attack evaluation. The paper itself concedes in the Limitations that missing an instruction 'could create security vulnerabilities' and that reliance on LLMs creates 'susceptibility to adaptive attacks.' The 2.07% ASR should therefore be reported as a non-adaptive benchmark result, and the authors should scope the claim to attacks that do not adapt to the shield's published prompts (Figures 4-6).
  4. [Definitions 3 and 4] Definition 4 states that a conversation achieves task alignment when 'all assistant-level task instructions' satisfy the alignment condition, but Definition 3 is stated for instructions at any privilege level Li, and Algorithm 1 explicitly applies the check to tool-output instructions as well. The formal definition should be aligned with the enforcement mechanism, or the discrepancy should be explained.
minor comments (4)
  1. [Table 2] The table heading says 'Important Messages attack' while the text and Table 1 use 'Important Instructions attack'; please make the terminology consistent.
  2. [Appendix D] There are several typos in Appendix D and Figure 1, including 'Detials,' 'Mislignment,' and 'Restuarant'; please proofread the appendix and the figure text.
  3. [Throughout] The paper alternates between 'ContributeTo' and 'ContributesTo' for the same relation; please standardize the notation.
  4. [Figure 3] The Pareto-front figure would be easier to interpret if the caption identified which marker corresponds to which defense method, since the reader cannot otherwise verify which points lie on the front.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the 2.07% ASR is an external AgentDojo measurement, and the sole self-citation (Wu et al. 2024) is not load-bearing.

full rationale

Task Shield's central empirical claim (Table 2: ASR 2.07%, utility 69.79% on GPT-4o under the Important Instructions attack) is a measurement against AgentDojo's externally defined attack-success and utility labels, not a quantity derived from the paper's own definitions. The ContributesTo scoring in Definition 2 and Algorithm 1 is a mechanism whose outputs are compared with independent ground truth; no parameter is fitted to the benchmark to force the reported ASR, and the paper never defines attack success in terms of its own alignment condition. Definitions 1-4 state what task alignment means but are not used to derive the experimental numbers. The only self-citation, Wu et al. 2024, appears in Section 6 among training-time defenses in related work and is not load-bearing: Task Shield is a test-time defense that does not build on it. The paper's Limitations section explicitly concedes 'susceptibility to adaptive attacks' and reliance on LLM-based extraction and scoring; that is a robustness weakness, not a circular reduction. Similarly, Section 4.1's warning that 'missing any such instruction could create security vulnerabilities' flags an evaluation gap, but it does not make the reported ASR an artifact of the paper's own definitions. Using the same GPT-4o model as both checker and attack target is a soundness concern, but it is not a definitional equivalence between input and output. No circular step can be exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No numeric constants are fitted to the data in a derivation sense, but the misalignment threshold epsilon is a design parameter whose value and tuning are not disclosed. The reliability of the LLM-based extraction and alignment scoring is the central unverified load-bearing ingredient, alongside the representativeness of AgentDojo as the sole benchmark.

free parameters (1)
  • Misalignment threshold epsilon = unspecified (text uses 0; algorithm uses a threshold)
    Algorithm 1 checks C_ei <= epsilon for misalignment, but the paper does not report the value, any tuning, or a sensitivity analysis. If epsilon was chosen on AgentDojo, the ASR numbers partly reflect that choice.
assumptions (4)
  • domain assumption The LLM can extract every actionable instruction, including nested, implicit, and tool-generated ones.
    Section 4.1: 'Missing any such instruction could create security vulnerabilities in our defense mechanism.' No extraction recall measurement is provided.
  • ad hoc to paper Every instruction that contributes to at least one user instruction is safe to execute; alignment implies security.
    Definitions 3 and 4 assume that blocking non-contributing instructions is sufficient to prevent indirect prompt injection. This is the conceptual foundation of the paper and is not independently proven.
  • domain assumption AgentDojo ground truth for utility and attack success is correct and representative of real threats.
    Section 5.1 uses AgentDojo as the sole benchmark; the limitations section acknowledges the evaluation covers one benchmark and one model family.
  • domain assumption The privilege hierarchy system > user > assistant > tool is valid and is respected by the target LLM.
    The shield augments tool instructions with source metadata based on this hierarchy, citing Wallace et al. 2024; the paper does not verify the target model's compliance with the hierarchy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents." pith.science (2026). https://pith.science/paper/RBNY55GU

@misc{pith2026241216682,
  author       = {Pith},
  title        = {Pith review of: The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBNY55GU}},
  note         = {Machine review of arXiv:2412.16682}
}
read the original abstract

Large Language Model (LLM) agents are increasingly being deployed as conversational assistants capable of performing complex real-world tasks through tool integration. This enhanced ability to interact with external systems and process various data sources, while powerful, introduces significant security vulnerabilities. In particular, indirect prompt injection attacks pose a critical threat, where malicious instructions embedded within external data sources can manipulate agents to deviate from user intentions. While existing defenses based on rule constraints, source spotlighting, and authentication protocols show promise, they struggle to maintain robust security while preserving task functionality. We propose a novel and orthogonal perspective that reframes agent security from preventing harmful actions to ensuring task alignment, requiring every agent action to serve user objectives. Based on this insight, we develop Task Shield, a test-time defense mechanism that systematically verifies whether each instruction and tool call contributes to user-specified goals. Through experiments on the AgentDojo benchmark, we demonstrate that Task Shield reduces attack success rates (2.07\%) while maintaining high task utility (69.79\%) on GPT-4o.

Figures

Figures reproduced from arXiv: 2412.16682 by the authors.

Figure 1
Figure 1. Overview of the Task Shield interacting with a tool-integrated LLM agent. The framework enforces task [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. This diagram illustrates how the Task Shield framework processes different message types from the [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. GPT-4o: Comparison of Attack Success Rate (ASR) versus Utility. Solid markers represent ASR versus [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Task Extraction Prompt: This prompt outlines the methodology for extracting actionable task instructions [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Content Checker Prompt: This prompt evaluates the alignment of new actionable instructions with user [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Tool Call Checker Prompt: This prompt verifies the alignment of tool calls with user-defined task [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Feedback Prompts: The figure explains how content misalignment, tool call misalignment, and user [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MOSAIC: Knowledge-Guided CLI Command Composition Attack in LLM Coding Agents

    cs.CR 2026-07 conditional novelty 7.0 of 10

    Individually benign CLI commands compose via shared OS state into high-success attacks on real LLM coding agents; MOSAIC systematically generates them from CVE/PoC knowledge at 96.59% ASR.

  2. The LLMbda Calculus: AI Agents, Conversations, and Information Flow

    cs.PL 2026-02 reject novelty 7.0 of 10

    A call-by-value lambda calculus with LLM generation and dynamic labels is shown to satisfy termination-insensitive noninterference on a restricted fragment, with a supposedly Lean-verified interpreter.

  3. ContainmentBench: Trace-Based Evaluation of Post-Injection Containment in Tool-Using LLM Agents

    cs.CR 2026-07 accept novelty 6.0 of 10

    Terminal policy labels are insufficient: two containment policies with identical zero-harm endpoints still differ in 73.5% of trajectories and in authorized-work completion.

  4. Invisible Prompts, Visible Threats: Malicious Font Injection in External Resources for Large Language Models

    cs.CR 2025-05 conditional novelty 6.0 of 10

    Maliciously crafted fonts can hide adversarial prompts from human eyes while LLMs read and obey them, enabling content manipulation and data exfiltration via MCP tools.

  5. RT-SHCUA: Real-Time Self-Hosted Computer-Use Agent for UAV Control

    cs.CR 2026-07 conditional novelty 5.0 of 10

    An architecture that mediates LLM computer-use agents for UAV control by compiling agent decisions into validated, time-bounded, evidence-logged skill invocations, with a prototype on OpenClaw/PX4/OP-TEE.

Reference graph

Works this paper leans on

38 extracted references · 11 canonical work pages · cited by 5 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Tom B Brown, Benjamin Mann, Nick Ryder, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems

    work page 2020

  4. [4]

    Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. 2024. Struq: Defending against prompt injection with structured queries. arXiv preprint arXiv:2402.06363

    arXiv 2024

  5. [5]

    Edoardo Debenedetti, Jie Zhang, Mislav Balunovi \'c , Luca Beurer-Kellner, Marc Fischer, and Florian Tram \`e r. 2024. Agentdojo: A dynamic environment to evaluate attacks and defenses for llm agents. arXiv preprint arXiv:2406.13352

    arXiv 2024

  6. [6]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2024. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36

    2024

  7. [7]

    Erik Derner, Kristina Batisti c , Jan Zah \'a lka, and Robert Babu s ka. 2024. A security risk taxonomy for prompt-based interaction with large language models. IEEE Access

    work page 2024

  8. [8]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. https://doi.org/10.1145/3637528.3671470 A survey on rag meeting llms: Towards retrieval-augmented large language models . In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, page 6491–6501, New York, NY,...

    arXiv 2024

Show all 38 references
  1. [9]

    Gupta, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, and Earlence Fernandes

    Xiaohan Fu, Zihan Wang, Shuheng Li, Rajesh K. Gupta, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, and Earlence Fernandes. 2024. https://openreview.net/forum?id=djcciHhCrt Misusing tools in large language models with visual adversarial examples

    2024

  2. [10]

    Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. https://doi.org/10.1145/3605764.3623985 Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection . In Proceedings of...

    2023

  3. [11]

    Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2023. Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings. Advances in neural information processing systems, 36:45870--45894

    2023

  4. [13]

    Keegan Hines, Gary Lopez, Matthew Hall, Federico Zarfati, Yonatan Zunger, and Emre Kiciman. 2024 b . https://api.semanticscholar.org/CorpusID:268667111 Defending against indirect prompt injection attacks with spotlighting . ArXiv, abs/2403.14720

    2024 arXiv

  5. [14]

    Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. 2023. Recommender ai agent: Integrating large language models for interactive recommendations. arXiv preprint arXiv:2308.16505

    2023 arXiv

  6. [15]

    Sahasra Kokkula, G Divya, et al. 2024. Palisade--prompt injection detection framework. arXiv preprint arXiv:2410.21146

    2024 arXiv

  7. [16]

    Mei Li et al. 2024. Securing tool use in llm agents: Challenges and strategies. arXiv preprint arXiv:2402.03014

    2024 arXiv

  8. [17]

    Yi Liu, Gelei Deng, Yuekang Li, Kailong Wang, Zihao Wang, Xiaofeng Wang, Tianwei Zhang, Yepang Liu, Haoyu Wang, Yan Zheng, and Yang Liu. 2023. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499

    2023 arXiv

  9. [18]

    Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. Formalizing and benchmarking prompt injection attacks and defenses. In USENIX Security Symposium

    2024

  10. [19]

    Jiageng Mao, Junjie Ye, Yuxi Qian, Marco Pavone, and Yue Wang. 2024. https://openreview.net/forum?id=UPE6WYE8vg A language agent for autonomous driving . In First Conference on Language Modeling

    2024

  11. [20]

    Gr \'e goire Mialon, Roberto Dessi, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pasunuru, Roberta Raileanu, Baptiste Roziere, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. 2023. https://openreview.net/forum?id=jh7wH2AzKK Au...

    2023

  12. [21]

    OpenAI. 2024. https://openai.com/index/introducing-openai-o1-preview/ Introducing openai o1-preview

    2024

  13. [22]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  14. [23]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334

    2023 arXiv

  15. [24]

    F \'a bio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527

    2022 arXiv

  16. [25]

    Julien Piet, Maha Alrashed, Chawin Sitawarin, Sizhe Chen, Zeming Wei, Elizabeth Sun, Basel Alomair, and David Wagner. 2023. https://api.semanticscholar.org/CorpusID:266690784 Jatmo: Prompt injection defense by task-specific finetuning . ArXiv, abs/2312.17673

    2023 arXiv

  17. [26]

    Learn Prompting. 2024. Sandwich defense. https://learnprompting.org/docs/prompt_hacking/defensive_measures/sandwich_defense. Accessed: 2024-11-07

    2024

  18. [27]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789

    2023 arXiv

  19. [28]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2024. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36

    2024

  20. [29]

    Ashish Sharma, Sudha Rao, Chris Brockett, Akanksha Malhotra, Nebojsa Jojic, and Bill Dolan. 2024. https://aclanthology.org/2024.eacl-long.119 Investigating agency of LLM s in human- AI collaboration tasks . In Proceedings of the 18th Conference of the European Chapter of the A...

    2024

  21. [30]

    Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, Boxi Cao, and Le Sun. 2023. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:2306.05301

    2023 arXiv

  22. [31]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  23. [32]

    Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. The instruction hierarchy: Training llms to prioritize privileged instructions. arXiv preprint arXiv:2404.13208

    2024 arXiv

  24. [33]

    Morley Mao, Muhao Chen, and Chaowei Xiao

    Jiongxiao Wang, Fangzhou Wu, Wendi Li, Jinsheng Pan, Edward Suh, Z. Morley Mao, Muhao Chen, and Chaowei Xiao. 2024. Fath: Authentication-based test-time defense against indirect prompt injection attacks. arXiv preprint arXiv:2410.21492

    2024 arXiv

  25. [34]

    Jason Wei et al. 2022. Inverse scaling: When bigger isn't better. arXiv preprint arXiv:2206.04615

    2022 arXiv

  26. [35]

    Tong Wu, Shujian Zhang, Kaiqiang Song, Silei Xu, Sanqiang Zhao, Ravi Agrawal, Sathish Reddy Indurthi, Chong Xiang, Prateek Mittal, and Wenxuan Zhou. 2024. https://arxiv.org/abs/2410.09102 Instructional segment embedding: Improving llm safety with instruction hierarchy . Prepri...

    2024 arXiv

  27. [36]

    Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, et al. 2024. Guardagent: Safeguard llm agents by a guard agent via knowledge-enabled reasoning. arXiv preprint arXiv:2406.09187

    2024 arXiv

  28. [37]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE_vluYUL-X React: Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations

    2023

  29. [38]

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. arXiv preprint arXiv:2403.02691

    2024 arXiv

  30. [39]

    Xiangzhe Zou et al. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.09283

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.