Pith.

Cryptography and Security · arXiv:2610.13651 v1 · submitted 9 Oct 2026 · 53 references

ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary

Md Nazmul Islam Ananto, Pierre Prevost-Avenel, Colin Smith, Kyungtae Kim, Taegyu Kim, Arslan Khan

The paper says

“Large language models (LLMs) are increasingly deployed as agents that interact with external software through tools, with an orchestration layer translating model outputs into structured tool invocations.”

more of the abstract

This architecture creates an LLM-orchestrator-tool trust boundary where tool semantics may go unenforced and LLM-generated arguments may be incorrectly treated as trusted. We present ToFu, a tool-aware framework that automatically derives semantic security checks for LLM tools from their existing metadata and a CWE-guided sanitizer corpus. ToFu specializes these checks to individual tools and automatically generates fuzzing harnesses and semantic oracles using runtime introspection and outcome sanitization. We evaluated ToFu across 273 MCP servers containing 3405 tools and identified verified bugs corresponding to CWE weakness classes in 343 tools across 118 of the servers. Our findings show that real-world tools frequently fail to enforce their documented input semantics, making it unsafe to treat LLM-generated arguments as trusted.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 12 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. ToFu makes a genuine contribution by repurposing tool metadata as a security specification for fuzzing. The evaluation is thorough and the limitations are honestly stated. This line is wrong

3 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 53 references say.

The paper says, at §7.5, Table 3

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The manual verification process from 508 candidates to 253 bugs removes 103 as 'not vulnerable' and 152 via deduplication. The paper does not specify the deduplication criterion—whether it merges bugs triggering the same argument in the same tool with different inputs, or only identical inputs.

more of objection 1

A clear definition affects how the 253 count should be interpreted (e.g., as distinct vulnerability classes or as distinct triggering inputs). Additionally, the 103 'not vulnerable' cases represent a false-positive rate of about 20% for the candidate flags. The paper should report this rate explicitly and discuss the most common sources of false positives, to help readers assess the reliability of the oracle.

The paper says, at §7.3, Table 4

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.69):

“Though we did not hand-check the correctness failures,ToolFuzz’s authors report a 46% false-discovery rate for them [ 30].ToolFuzz tests whether a tool’s documentation lets an agent use it correctly, not whether the implementation keeps a security contract.”

Pith says: Objection 2 of 3 · major

The comparison with baselines is informative but the paper states that 'no baseline is directly comparable with ToFu's semantic bug detection goal' and presents no leaderboard. However, Table 4 shows 'Flagged' and 'Shared' counts, and the text suggests ToFu finds more bugs.

more of objection 2

To avoid implying a ranking, the paper should explicitly state that each analyzer has a different detection goal (runtime exploitation, static analysis, agent correctness) and that aggregate server counts are not directly comparable. The discussion in §7.3 already does this to some extent, but Table 4's headline numbers could be misinterpreted.

The paper says, at §7.7, Table 10

The review did not quote this passage.

A classifier read this sentence of the paper as acknowledging the concern (probability 0.61):

“Establishing precision would require a labelled sample per family and is future work.”

Pith says: Objection 3 of 3 · major

The all-sanitizers run reports a false-positive rate of about 95% (960 reports, 47 real). While this justifies the need for mapping, it also suggests that many of the 74 sanitizers have low precision on this dataset. The paper should discuss whether these sanitizers are inherently noisy or if the oracle logic needs refinement for certain classes.

more of objection 3

Since the authors acknowledge that the corpus 'keeps them although this dataset does not trigger them' (Appendix D), a brief discussion of precision for those untriggered sanitizers would clarify the design trade-off.

6 minor objections

The paper says, at §7.1

Pith says: minor objection 1 of 6

The paper states '273 are cleanly executable' and '3405 extracted tools' but later notes that only a subset of tools could be exercised. The relationship between the 3405 extracted tools and the number actually executed should be reported explicitly, as the abstract's '3405 tools' may give an inflated impression.

The paper says, at Table 8

Pith says: minor objection 2 of 6

The no-LLM arm retains about two thirds of bugs, but inspection shows these are almost entirely reflected output and log injection (always-on sanitizers). The claim 'The LLM is not all of ToFu' is correct for those bug classes, but for path traversal, SSRF, and command injection the LLM mapping is essential. This nuance should be stated explicitly in the ablation discussion.

show 4 more

The paper says, at §3

Pith says: minor objection 3 of 6

The related work survey is comprehensive, but Table 1 compares ToFu against many systems. Some entries (e.g., CodeQL) are not designed for LLM tools. Consider adding a note that the comparison is on capability dimensions, not performance, to avoid unfair characterization.

The paper says, at Appendix B

Pith says: minor objection 4 of 6

The repeat-after-me attack transcript is informative, but it uses a local model (llama3.1:8b) rather than a frontier API. A note that the attack may not reproduce on all models or orchestrators (as the authors reproduced it on six frameworks) would strengthen the appendix's credibility.

The paper says, at §6.2.2

Pith says: minor objection 5 of 6

The description of outcome sanitization for nondeterminism mentions 'evaluating semantic invariants' and 'structured error types.' A concrete example of how this disambiguation works (e.g., for a tool that sometimes returns an empty result vs. always returning an error on rejection) would help readers understand the oracle's limitations.

The paper says, at Throughout

Pith says: minor objection 6 of 6

The paper occasionally uses inconsistent notation: e.g., 'ToFu' vs. 'ToFu' in some places (should be ToFu consistently). Also, some figure references (e.g., Fig. 3) appear before they are introduced in text. Check for cross-reference ordering.

What would settle it

Ask an independent group of five security researchers to independently reproduce the manual verification for a stratified random sample of 30 reported bugs (6 per CWE class). Measure inter-rater agreement (Fleiss' kappa) on whether each input-tool pair constitutes a genuine semantic violation. If kappa >= 0.8, the verification procedure is robust; otherwise, the manual filtering step introduces subjectivity that should be documented and its effect on headline counts quantified.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 12 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 34 s · 34,463 tokens read, 3,335 written
  3. Editor wrote the verdict above 24 s · 35,206 tokens read, 1,960 written
  4. Referee minor revision: 3 major objections, 6 minor 30 s · 34,688 tokens read, 3,122 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This paper does something simple in concept but effective in practice: it takes the metadata that MCP servers expose to LLMs—argument names, descriptions, schemas—and uses it to derive what security properties those arguments should obey. Then it builds a fuzzer that generates adversarial inputs targeting those properties and checks whether the tool enforces them. The core idea is sound, and the implementation looks well-engineered.

The empirical evidence is the main event. 253 verified bugs across 118 servers is a concrete finding, not a toy benchmark. The authors did manual verification (reproduction, code inspection) and an ablation study that separates the contribution of each component. They also show that without the LLM step they still find about two-thirds of the bugs, which addresses the obvious worry that the whole thing depends on perfect metadata interpretation. The comparison with five baselines, while not a head-to-head leaderboard, gives useful context for why the different tools disagree.

On the softer side, the distribution of bug classes matters. The majority are reflected output (115) and log injection (58), which are real but generally low-severity. Only 7 SSRF and 3 command injection bugs were found. That is not a flaw in the method—it reflects the corpus—but a reader should calibrate expectations. The dataset is also curated (stars≥10, Python/JS, not archived) and the funnel from 13,875 MCPCorpus entries to 273 reachable servers is steep, though the authors are transparent about each drop. The 60-second fuzzing budget is short; the deep-budget analysis on 34 servers helps, but it would have been nice to see longer runs on a wider sample to confirm that early coverage saturation holds broadly.

The paper's main limitation, which it acknowledges, is that it works only on servers whose metadata accurately describes the tool's intended security properties. If the metadata is misleading or absent, the approach degrades. The ablation mitigates this but does not eliminate it—the remaining bugs in the no-LLM condition are mostly the always-on checks (reflected output, log injection), not the more targeted ones like path traversal or SSRF. That said, the authors are candid about this and about the other limitations (no multi-agent support, no stateful fuzzing, third-party component coverage).

Who is this for? Anyone working on LLM agent security, especially tool-side defenses. The paper provides a reusable methodology and a concrete bug dataset. It deserves a serious referee and, pending minor clarifications (especially around the scope of bug classes and the generalizability of the metadata-assumption), should be published.

Recommendation: send to peer review.

The strongest objection

The paper's central claim—that real-world tools frequently fail to enforce their documented input semantics—is supported by 253 manually verified bugs across 118 servers, with a transparent evaluation pipeline and a thorough ablation study. The most likely weak point (reliance on metadata to derive security properties) is acknowledged as challenge C1 and partially mitigated by the LLM-based mapping and fallback mechanisms. The ablation results show that even without any LLM involvement (no-LLM condition) the framework retains about two thirds of its bugs, confirming that the core detection does not critically depend on perfect metadata interpretation. No internally inconsistent or unsupported step in the argument was found. The manual verification process is described (reproduction, code inspection) and case studies are provided. Therefore no load-bearing concern that would undermine the validity of the reported findings can be identified in good faith.

The reader identified the metadata assumption as the load-bearing weakness. I agree that this is the most fragile premise, but the paper handles it responsibly: (1) The challenge is explicitly stated (C1, §5). (2) The LLM-based mapping is supplemented by fallback paths and an ablation study that measures the incremental value of each LLM stage (§7.7). (3) Even in the no-LLM condition (no sanitizer selection, no seed generation), ToFu retains ~293 bugs vs ~430 in the full run, meaning the core detection does not hinge on perfect metadata interpretation. (4) Manual verification further filters out false positives that might arise from incorrect metadata interpretation. Consequently, the metadata assumption, while real, does not invalidate the central claim that many tools fail to enforce their advertised semantics. The paper's evidence is substantial and its limitations are well-documented. No other hidden assumption or methodological gap was found that would warrant adjusting the verdict.

The referee's report

If the result holds, it demonstrates a systemic trust failure at the LLM–tool boundary: tool implementations frequently fail to enforce the security properties implied by their own interfaces, and LLM-generated arguments—potentially attacker-controlled via prompt injection—are treated as trusted. The paper's methodological contribution is significant: repurposing metadata as a security specification, combining runtime introspection with outcome-based oracles, and the extensible sanitizer corpus. The evaluation is large-scale, includes manual verification, ablation studies, and baseline comparisons, and the authors transparently discuss limitations. The work has practical implications for the design of secure agentic systems and for auditing existing MCP servers.

Pith's reading of the paper (a machine's summary, not the authors')

The paper argues that the natural-language metadata that LLM agents use to select tools can also guide automated security testing. By treating each argument's semantic description as a security contract, ToFu derives checks (e.g., 'path arguments should not escape the workspace'), generates adversarial inputs, and detects violations that conventional crash-based fuzzers miss. Across 273 MCP servers, ToFu found 253 verified semantic bugs—path traversal, SSRF, command injection, reflected output, log injection, and data exfiltration—proving that tool implementations routinely fail to uphold the security properties their interfaces advertise.

What carries the argument. The semantics-aware sanitizer corpus (74 sanitizers), each tying a semantic input class (path, URL, command, etc.) to adversarial payloads, an expected security property, and the runtime observations needed to check compliance. The sanitizer mapper uses an LLM to connect tool metadata to applicable sanitizers and to set outcome rules (e.g., 'must raise an exception on adversarial input'). The fuzzer then generates schema-valid seeds and uses coverage-guided mutation with Atheris, applying both runtime introspection (tracers on Python sinks) and contract-based outcome oracles.

Core claim. The core discovery is that the metadata already exposed to LLMs to enable tool invocation—names, descriptions, and schemas—encodes enough semantic information to define executable security oracles. By mapping each argument to a corresponding class of adversarial inputs and expected outcomes, and by using runtime introspection or outcome sanitization, ToFu automatically uncovers semantic security bugs: executions that complete normally but violate the security contract implied by a tool's documented interface. The paper's central evidence is 253 verified bugs across 118 MCP servers, demonstrating a systemic failure of tool implementations to enforce their own advertised semantics.

The premise it rests on. The approach assumes that the natural-language metadata (description, argument names, schemas) provided by tool developers accurately and completely specifies the intended security properties of each input, and that an LLM can correctly map those descriptions to a predefined corpus of sanitizers.

If the paper is right (Pith's inferences, not the authors')

Tool developers should treat metadata as an enforceable security contract and validate inputs against it independently, rather than relying on the LLM or orchestration layer to sanitize arguments.

LLM orchestration frameworks should include pre-execution security checks based on tool metadata, not just structural schema validation.

The sanitizer corpus is extensible; future work can add new weakness classes (e.g., unsafe deserialization, XXE) without modifying the core framework.

Multi-agent systems will require chaining these semantic checks across tool boundaries and agent communication paths.

Stateful tool interactions (e.g., create-then-modify sequences) may harbor additional semantic bugs not captured by single-invocation fuzzing.

The approach's reliance on metadata quality means it will miss bugs where metadata is absent, incomplete, or intentionally misleading; tools with deliberately vague descriptions may evade detection.

The framework currently requires runtime introspection for the most robust oracles (Python only), so JavaScript/TypeScript tools are tested only through outcome sanitization, which may be less precise.

The finding that 74% of candidate bugs appear within one minute suggests that many trivial safety checks—like rejecting `/etc/passwd` or `http://127.0.0.1`—are simply absent; a lightweight, per-tool validation wrapper derived from metadata could eliminate a large fraction of these bugs without fuzzing.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Large language models (LLMs) are increasingly deployed as agents that interact with external software through tools, with an orchestration layer translating model outputs into structured tool invocations. This architecture creates an LLM-orchestrator-tool trust boundary where tool semantics may go unenforced and LLM-generated arguments may be incorrectly treated as trusted. We present ToFu, a tool-aware framework that automatically derives semantic security checks for LLM tools from their existing metadata and a CWE-guided sanitizer corpus. ToFu specializes these checks to individual tools and automatically generates fuzzing harnesses and semantic oracles using runtime introspection and outcome sanitization. We evaluated ToFu across 273 MCP servers containing 3405 tools and identified verified bugs corresponding to CWE weakness classes in 343 tools across 118 of the servers. Our findings show that real-world tools frequently fail to enforce their documented input semantics, making it unsafe to treat LLM-generated arguments as trusted.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-12

The authors' main claim, as the classifier picked it out 1.00

“Our findings show that real-world tools frequently fail to enforce their documented input semantics, making it unsafe to treat LLM-generated arguments as trusted.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.12 · human participants 0.03 · controlled comparison 0.05 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.08 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Large language models (LLMs) are increasingly deployed as agents that interact with external software through tools, with an orchestration layer translating model outputs into structured tool invocations. background 1.00 0.04 0.02 asserted 0.99
This architecture creates an LLM-orchestrator-tool trust boundary where tool semantics may go unenforced and LLM-generated arguments may be incorrectly treated as trusted. background 0.84 0.07 0.13 hedged possibility 0.88
We present ToFu, a tool-aware framework that automatically derives semantic security checks for LLM tools from their existing metadata and a CWE-guided sanitizer corpus. contribution 0.88 0.95 0.05 asserted 0.99
ToFu specializes these checks to individual tools and automatically generates fuzzing harnesses and semantic oracles using runtime introspection and outcome sanitization. method 0.92 0.93 0.05 asserted 1.00
We evaluated ToFu across 273 MCP servers containing 3405 tools and identified verified bugs corresponding to CWE weakness classes in 343 tools across 118 of the servers. result 0.99 0.98 0.13 asserted 1.00
Our findings show that real-world tools frequently fail to enforce their documented input semantics, making it unsafe to treat LLM-generated arguments as trusted. interpretation 0.90 0.95 0.86 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-12: 1 request, 4,994 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. ToFu maps tool metadata to semantics-aware sanitizers, generates schema-valid seeds and a fuzzing harness, and detects semantic security violations through runtime introspection or outcome sanitization. to the concrete contract expressed by each tool without requiring a manually written analysis for every implementation. C3: Constructing Oracles Under Partial Visibility. Many semantic violations terminate normally, so crashes provide no useful signal. Moreover, some tool implementations can be inspected while others are available only through their external interface. We therefore…
Figure 2 of the paper
Figure 2. Coverage over time for the eight servers with 24 hour budget. Labels give coverage at 60 s and at the
Figure 3 of the paper
Figure 3. Deep-budget campaign on 34 servers: (a) nor
Figure 4 of the paper
Figure 4. Images returned by t/screenshot-page for file: URLs. (a) A local file. (b) A directory listing. We follow through every stage of Section 6 for one tool. We have got an SSRF (for http:// URLs; CWE-918) and an arbitrary path traversal (for file:/// URLs; CWE-22/552/73) in t/screenshot-page of s/webpage-screenshot-mcp, one of the 508 candidates in Section 7.5. The tool takes a URL and returns a screenshot of the rendered page. Given a malicious URL, it could essentially compromise the integrity of the remote server. On another note, given a file: URL, it renders a file or a directory…

Figures are the authors', reproduced from the paper.

References (53)
  1. [1] Parya Abadeh, Martin Lochner, Taha Ansari, and Fattane Zarrinkalam. 2026. MCP-Scanner: Detecting Security Risks in Model Context Protocol Systems. InProceedings of the 2026 ACM/IEEE 7th International Workshop on Engineering and Cybersecurity of Critical Systems (EnCyCriS). 41–... on Pithdoi:10.1145/3786160.3788471
  2. [2] Agent-Threat-Rule. 2026. Agent-Threat-Rules: Open Detection-Rule Standard for AI Agent Security Threats. https: //github.com/Agent-Threat-Rule/agent-threat-rules on Pith
  3. [3] Amazon Web Services. 2026. Strands Agents: Agent Loop. https://strandsagents .com/docs/user-guide/concepts/ agents/agent-loop/ on Pith
  4. [4] Baichao An, Pei Chen, Geng Hong, Yueyue Chen, and Mengying Wu. 2026. FlowGuard: From Signals to Evidence for MCP Security Detection. arXiv:2607.14754 [cs.CR] on PithPith's reviewarXiv
  5. [5] Anthropic. 2025. Claude 3.7 Sonnet and Claude Code. https://www.anthropic.com/news/claude-3-7-sonnet on Pith
  6. [6] Anthropic. 2026. Claude Sonnet 5 (Large Language Model). https://www .anthropic.com/news/claude-sonnet-5 Accessed: 2026-09-28 on Pith
  7. [7] Pei Chen, Baichao An, Mengying Wu, Binwang Wan, Geng Hong, Jinsong Chen, Xudong Pan, Jiarun Dai, and Min Yang. 2026. Rethinking MCP Security: A Large-Scale Study of Runtime MCP Servers and Security Scanner Reliability. arXiv:2607.11086 [cs.CR] on PithPith's reviewarXiv
  8. [8] Cisco AI Defense. 2025. Cisco MCP Scanner: Scan MCP Servers for Potential Threats and Security Findings. https: //github.com/cisco-ai-defense/mcp-scanner on Pith
  9. [9] CrewAI. 2026. CrewAI Documentation. https://docs.crewai.com/ on Pith
  10. [10] Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. InProceedings of the 32nd ACM SIGSOFT international symposium on software testing and ... on Pith
  11. [11] Zhiyang Ding, Yang Luo, Guangpu Chen, Qingni Shen, and Zhonghai Wu. 2026. ACLE-MCP: Attested Capability Leases for Execution-Time Trust in Remote LLM Tool Use. arXiv:2609.02690 [cs.CR] on PithPith's reviewarXiv
  12. [12] GitHub. 2026. CodeQL Security-Extended Query Suite. https://docs .github.com/en/code-security/concepts/code- scanning/codeql/codeql-query-suites. Accessed: September 2026 on Pith
  13. [13] Google. 2020. Atheris: A coverage-guided, native Python fuzzer. https://github.com/google/atheris on Pith
  14. [14] Google. 2026. Agent Development Kit 2.0. https://adk.dev/graphs/ on Pith
  15. [15] Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. In Proceedings of the 16th ACM Workshop on Artificial Inte... on Pithdoi:10.1145/3605764.3623985
  16. [16] Yongjian Guo, Puzhuo Liu, Wanlun Ma, Zehang Deng, Xiaogang Zhu, Peng Di, Xi Xiao, and Sheng Wen. 2026. MCPXKIT: The Unified Toolkit for Analyzing Model Context Protocol Security.IEEE Transactions on Dependable and Secure Computing(2026). doi:10.1109/TDSC.2026.3695553 on Pithdoi:10.1109/tdsc.2026.3695553
  17. [17] Mohammed Mehedi Hasan, Hao Li, Emad Fallahzadeh, Gopi Krishnan Rajbahadur, Bram Adams, and Ahmed E. Hassan on Pith
  18. [18] Ping He, Yuexiang Xie, Yaliang Li, and Shouling Ji. 2026. Hybrid Analysis for Secure MCP Tool Use in LLM Agents. arXiv:2607.25297 [cs.CR] on PithPith's reviewarXiv
  19. [19] Yuqi Jia, Yupei Liu, Zedian Shao, Jinyuan Jia, and Neil Zhenqiang Gong. 2026. Promptlocate: Localizing prompt injection attacks. In2026 IEEE Symposium on Security and Privacy (SP). IEEE, 4243–4261 on Pith
  20. [20] Rabimba Karanjai, Yang Lu, Nour Diallo, Wujie Xiong, Lei Xu, and Weidong Shi. 2026. When Agents Act on Web3: An Attack-Surface Survey of MCP, Skills, and Tool Calling. arXiv:2608.17275 [cs.CR] on PithPith's reviewarXiv
show 33 more
  1. [21] LangChain. 2026. LangGraph: Graph API. https://docs.langchain.com/oss/python/langgraph/graph-api on Pith
  2. [22] LangChain. 2026. Tools — LangChain Conceptual Guide. https://python.langchain.com/docs/concepts/tools/ on Pith
  3. [23] Zichuan Li, Jian Cui, Xiaojing Liao, and Luyi Xing. 2026. Les Dissonances: Cross-Tool Harvesting and Polluting in Pool-of-Tools Empowered LLM Agents. InNetwork and Distributed System Security Symposium (NDSS) 2026, San Diego, California, USA, February 24-27, 2026. San Diego, CA on Pith
  4. [24] Zhihao Li, Boyang Ma, Xuelong Dai, Minghui Xu, Yue Zhang, Biwei Yan, and Kun Li. 2026. Don’t believe everything you read: Understanding and Measuring MCP Behavior under Misleading Tool Descriptions.arXiv preprint arXiv:2602.03580 (2026) on PitharXiv
  5. [25] Zhiyuan Li, Jingzheng Wu, Xiang Ling, Xing Cui, and Tianyue Luo. 2026. Towards secure agent skills: Architecture, threat taxonomy, and security analysis.arXiv preprint arXiv:2604.02837(2026) on PithPith's reviewarXiv
  6. [26] Zhiwei Lin, Bonan Ruan, Jiahao Liu, and Weibo Zhao. 2025. A Large-Scale Evolvable Dataset for Model Context Protocol Ecosystem and Security Analysis. InProceedings of the 40th IEEE/ACM International Conference on Automated Software Engineering (ASE), Tool Demonstrations Track.... on Pith
  7. [27] Fengyu Liu, Yuan Zhang, Jiaqi Luo, Jiarun Dai, Tian Chen, Letian Yuan, Zhengmin Yu, Youkun Shi, Ke Li, Chengyuan Zhou, et al. 2025. Make agent defeat agent: Automatic detection of {Taint-Style} vulnerabilities in{LLM-based} agents. In34th USENIX Security Symposium (USENIX Secu... on Pith
  8. [28] Microsoft. 2026. AutoGen Core. https://microsoft.github.io/autogen/stable/user-guide/core-user-guide/ on Pith
  9. [29] Microsoft. 2026. Microsoft Agent Framework: Workflow Builder and Execution. https://learn .microsoft.com/en- us/agent-framework/concepts/workflows/builder-and-execution on Pith
  10. [30] Ivan Milev, Mislav Balunović, Maximilian Baader, and Martin Vechev. 2025. ToolFuzz–Automated Agent Tool Testing. arXiv preprint arXiv:2503.04479(2025) on PithPith's reviewarXiv
  11. [31] OpenAI. 2025. Introducing Operator. https://openai.com/index/introducing-operator/ on Pith
  12. [32] Nicolás Padilla. 2026. Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale. arXiv:2608.00150 [cs.CR] on PithPith's reviewarXiv
  13. [33] Shriti Priya, Teryl Taylor, and Frederico Araujo. 2026. AEGIS: Preventing Cross-Domain Resource Abuse in MCP. arXiv:2608.20481 [cs.CR] on PithPith's reviewarXiv
  14. [34] Alfonso Pedro Ridao, Melissa Safari, Zachary Kang, and Nicola Dragoni. 2026. MCP-SecLint: An Open-Source Static Analyzer for Detecting Vulnerabilities in LLM Tool Integrations. InProceedings of the 12th ACM International Workshop on Security and Privacy Analytics (IWSPA). 89–1... on Pithdoi:10.1145/3806007.3810961
  15. [35] Mehrdad Rostamzadeh, Sidhant Narula, Mohammad Ghasemigol, and Daniel Takabi. 2026. TrustShiftProbe: Charac- terizing, Benchmarking, and Defending Staged Trust Attacks on MCP Servers. arXiv:2608.23763 [cs.CR] on PithPith's reviewarXiv
  16. [36] Ramachandran Sekar, Venkat N Venkatakrishnan, Samik Basu, Sandeep Bhatkar, and Daniel C DuVarney. 2003. Model- carrying code: a practical approach for safe execution of untrusted applications.ACM SIGOPS Operating Systems Review37, 5 (2003), 15–28 on Pith
  17. [37] Jiawen Shi, Zenghui Yuan, Guiyao Tie, Pan Zhou, Neil Zhenqiang Gong, and Lichao Sun. 2026. Prompt Injection Attack to Tool Selection in LLM Agents. InProceedings of the Network and Distributed System Security Symposium (NDSS) on Pith
  18. [38] Tianneng Shi, Kaijie Zhu, Zhun Wang, Yuqi Jia, Will Cai, Weida Liang, Haonan Wang, Hend Alzahrani, Joshua Lu, Kenji Kawaguchi, Basel Alomair, Xuandong Zhao, William Yang Wang, Neil Gong, Wenbo Guo, and Dawn Song. 2025. Promptarmor: Simple yet effective prompt injection defense... on PithPith's reviewarXiv
  19. [39] Yang Shi, Jiaheng Fu, Yihe Huang, Ruixiang Wu, Chengyao Sun, and Kaifeng Huang. 2026. Mitigating Taint-Style Vulnerabilities in MCP Servers via Security-Aware Tool Descriptions. arXiv:2607.07461 [cs.CR] on PithPith's reviewarXiv
  20. [40] Yutao Shi, Xiaohan Zhang, Xiangjing Zhang, Xihua Shen, Hui Ouyang, Huming Qiu, Mi Zhang, and Min Yang. 2026. Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications. arXiv preprint arXiv:2606.04769(2026) on PithPith's reviewarXiv
  21. [41] Significant Gravitas. [n. d.]. AutoGPT. https://github .com/Significant-Gravitas/AutoGPT. Accessed: October 2, 2026 on Pith
  22. [42] Hao Song, Yiming Shen, Wenxuan Luo, Leixin Guo, Ting Chen, Jiashui Wang, Beibei Li, Xiaosong Zhang, and Jiachi Chen. 2025. Beyond the Protocol: Unveiling Attack Vectors in the Model Context Protocol (MCP) Ecosystem. arXiv:2506.02040 [cs.CR] on PithPith's reviewarXiv
  23. [43] Pengyu Sun, Zifeng Kang, Qishu Jin, Enhao Huang, Xin Liu, Dakun Shen, and Song Li. 2026. VIPER-MCP: Detecting and Exploiting Taint-Style Vulnerabilities in Model Context Protocol Servers. arXiv:2605.21392 [cs.CR] on PithPith's reviewarXiv
  24. [44] Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. 2024. The instruction hierarchy: Training llms to prioritize privileged instructions.arXiv preprint arXiv:2404.13208(2024) on PithPith's reviewarXiv
  25. [45] Xilong Wang, John Bloch, Zedian Shao, Yuepeng Hu, Shuyan Zhou, and Neil Zhenqiang Gong. 2025. Webinject: Prompt injection attack to web agents. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2010–2030 on Pith
  26. [46] Zihan Wang, Rui Zhang, Yu Liu, Wenshu Fan, Wenbo Jiang, Qingchuan Zhao, Hongwei Li, and Guowen Xu. 2026. Mpma: Preference manipulation attack against model context protocol. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 35838–35846 on Pith
  27. [47] Scott Wu. 2024. Introducing Devin, the First AI Software Engineer. https://cognition.com/blog/introducing-devin on Pith
  28. [48] Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Fuzz4all: Univer- sal fuzzing with large language models. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering. 1–13 on Pith
  29. [49] Wenpeng Xing, Zhonghao Qi, Yupeng Qin, Yilin Li, Caini Chang, Jiahui Yu, Changting Lin, Zhenzhen Xie, and Meng Han. 2026. Mcp-guard: A multi-stage defense-in-depth framework for securing model context protocol in agentic ai. InFindings of the Association for Computational Ling... on Pith
  30. [50] Yixuan Yang, Cuifeng Gao, Daoyuan Wu, Yufan Chen, Yingjiu Li, and Shuai Wang. 2025. MCPSecBench: A systematic security benchmark and playground for testing Model Context Protocols.arXiv preprint arXiv:2508.13220(2025) on PitharXiv
  31. [51] Jinhao Zhu, Kevin Tseng, Gil Vernik, Xiao Huang, Shishir G Patil, Vivian Fang, and Raluca Ada Popa. 2025. Miniscope: A least privilege framework for authorizing tool calling agents.arXiv preprint arXiv:2512.11147(2025). ToFu: Exploiting and Fuzzing the LLM–Orchestrator–Tool Tr... on PithPith's reviewarXiv
  32. [52] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043 [cs.CL] Md Nazmul Islam Ananto, Pierre Prevost-Avenel, Colin Smith, Kyungtae Kim, Taegyu K... on PithPith's reviewarXiv
  33. [2026] Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Servers.ACM Transactions on Software Engineering and Methodology(2026). doi:10.1145/3814959 on Pithdoi:10.1145/3814959

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment