REVIEW 4 major objections 6 minor 34 references
Agentic commerce platforms contain a class of protocol-level vulnerabilities that succeed deterministically no matter which AI model runs the agent, so model alignment alone cannot secure them.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 06:33 UTC pith:YO6EIWIS
load-bearing objection Useful reframing and real artifacts, but the 33-count abstract is overclaimed and the AP2 sample-to-production inference plus withheld proofs make the systemic claim unverifiable today. the 4 major comments →
Protocol-Level Attacks on Agentic Commerce Platforms: A Cross-Platform Taxonomy, AIP-Bench, and Unified Defense
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that structural attacks—attacks that succeed with probability 1 for every model configuration—exist in the protocol layer of three major agentic commerce platforms. Across CoralOS, Fetch.ai uAgents, and Google's AP2, the author identifies 33 such vulnerabilities, organized into six root-cause classes (RC-1: registry content accepted without signature verification; RC-2: payment destination taken from untrusted sources; RC-3: credentials on observable channels; RC-4: non-atomic check-then-execute payment state; RC-5: authorization scope not enforced; RC-6: model-dependent prompt injection). A three-stage chain (V5→V4→V9) achieves simultaneous credential exfiltration, beha
What carries the argument
The structural/semantic distinction (Definition 1: an attack is structural if it succeeds with probability 1 for every language-model configuration; Definition 2: semantic if success depends on the model) and the six root-cause classes (RC-1–RC-6) organize the 33 findings. The defense sidecar PCAT is the test of the claim: five protocol checks (response signing, DID-based caller binding, secure channel enforcement, atomic payment state, MCP tool-call authorization) close the structural classes at the protocol boundary without modifying any platform.
Load-bearing premise
The entire 33-vulnerability count and the 100% deterministic-success claim rest on the analyzed code snapshots (CoralOS v1.3.0 commit a38912b, a June 2026 uAgents release, and AP2 v0.2.0 Python reference) being representative of production agentic-commerce deployments, and on 21 code-level proofs, whose exact locations are withheld, being correct.
What would settle it
When the withheld proof-of-concept exploits and code locations are released on 2026-10-04, replaying the full AIP-Bench suite against the three platforms' default deployments should reproduce every claimed 100% ASR; a single failure—or a synchronized-timing RC-4 race that does not reach 100%—would falsify the determinism claim. Separately, an independent audit of current production versions showing any of the root causes already patched would undermine the cross-platform 'systemic' conclusion.
If this is right
- Model alignment cannot remove any of the 33 structural vulnerabilities; only protocol-level changes can, so agentic-commerce security must include the registry, payment, and credential paths, not just the LLM.
- Cost-optimized deployments, typically running flash-tier models, combine 100% structural delivery with 99–100% behavioral success on the semantic IPI class, making them the highest-risk production configuration.
- A deterministic benchmark without LLM judges (AIP-Bench) gives operators a reproducible, model-agnostic way to measure whether a platform is vulnerable to structural flaws.
- A single external sidecar (PCAT) can cut structural attack success from 100% to 0% for four of the five structural classes on the HTTP path, without changing platform source code.
- The recurring root-cause pattern implies that default configurations—not exotic setups—are where these flaws live; common hardening like non-TRACE logging or strict CORS narrows individual findings but does not remove the root causes.
Where Pith is reading between the lines
- If the five root causes recur across three independently built platforms, they plausibly generalize to other agentic commerce stacks; a quick test would be running AIP-Bench scenarios against a fourth platform and checking whether RC-1 and RC-4 reappear.
- The determinism claim rests on 21 code-level proofs whose exact locations are withheld; when the artifacts release on 2026-10-04, replaying each scenario against the pinned commits would confirm or refute the 100% numbers.
- The LLM-resistance gradient suggests a practical division of labor: protocol-side defenses for structural classes plus semantic defenses that force explicit user confirmation of payment destinations, since even resistant models refuse for different reasons.
- The single-process safety proof for P4 implies that horizontally scaled deployments need a shared atomic compare-and-swap store, which operators should treat as a deployment requirement rather than an optional enhancement.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that agentic commerce platforms are vulnerable at the protocol layer, not only at the LLM reasoning layer. It presents 33 findings across CoralOS, Fetch.ai uAgents, and Google AP2, organized into six root-cause classes RC-1–RC-6. The authors argue RC-1–RC-5 are structural: deterministic, model-independent, with 100% ASR wherever live-measured. They contribute AIP-Bench, a deterministic benchmark with non-LLM judges, and PCAT, an HTTP sidecar that, according to the authors, reduces ASR to 0% for four of the five structural classes. They also report a 1,440-trial behavioral study of the V5 marketplace-injection payload across eight LLMs and describe a three-stage payment-hijack chain.
Significance. If the central claims hold, this is an important paper. It provides a useful separation between structural and semantic attacks, a concrete taxonomy, and an implementation of a plausible protocol-level defense. The use of deterministic judges in the benchmark is a real improvement over LLM-judge-based evaluations. The paper also ships reproducibility artifacts (Docker Compose, SHA-256 manifests, result hashes) and gives honest limitations for PCAT. The live demonstrations on CoralOS and Fetch.ai, plus the costed, multi-model behavioral study, are valuable empirical contributions. The main weakness is that a substantial fraction of the 33 findings is supported only by code proofs whose exact locations are withheld, and the PCAT evaluation is partly self-referential. These issues are fixable within the manuscript's scope, but they affect how the headline claims should be stated and how confident a reader can be in the cross-platform 'systemic' conclusion.
major comments (4)
- [Abstract, §1.2, Table 2, §9] The paper's headline count conflates structural and semantic findings. V5 is classified RC-1/RC-6 in Table 2, and §9 shows its behavioral impact ranges from 0% to 100% ASR depending on the model (Table 9). Only the RC-1 delivery component is model-independent. Yet the abstract says '33 such vulnerabilities, each succeeding deterministically regardless of the deployed model' and the conclusion repeats '33 structural vulnerabilities.' This is internally inconsistent and overstates the result. The count should be reformulated as, e.g., 32 structural findings plus one mixed-delivery/semantic-impact finding, or V5 should be explicitly defined as a structural delivery vulnerability with a model-dependent behavioral component in every counting statement.
- [§6.1, §6.2, Appendix B, Table 3] The AP2 analysis targets code/samples/python/src/ of v0.2.0, which the manuscript itself calls a 'Python reference implementation.' Three findings are admitted Python-specific. For the remaining eleven claimed 'Spec' and 'Both' findings, no code locations or line numbers are provided; the manuscript withholds them until 2026-10-04, and only A-AP2-11 is live-validated. Consequently, the reader cannot independently verify the central claim that these findings persist beyond one sample codebase. The representativeness concern is load-bearing because the cross-platform, 'systemic' conclusion depends on the AP2 findings being properties of the protocol or of both reference implementations, not artifacts of a single Python sample. The authors should provide an anonymized artifact with precise code locations, or narrow the claims to the analyzed reference implementations and provide evidence of
- [§8.3, Table 5, Supplementary H] PCAT's evaluation is partly circular: the attack suite is generated from the same codebase that defines AIP-Bench, and each PCAT principle is a direct implementation of the fix for the corresponding root-cause class. A 100%-to-0% ASR reduction for attacks that PCAT was designed to block is an implementation sanity check, not an independent validation. The 10,000 benign requests are also synthetic and were generated by the authors. This does not invalidate the artifact, but the paper should either add an independent evaluation (e.g., a second implementer, a third-party red team, or attacks designed by someone other than the PCAT authors) or explicitly state that the Table 5 numbers demonstrate mechanism coverage rather than operational security.
- [Appendix B, §4.2, Remark 2] The 'default deployment' claims are not fully supported by the experimental configuration. Appendix B states that CoralOS was run with registry.includeDebugAgents=true and llmProxy.providers=[openrouter with gpt-3.5-turbo], and that a specific commit a38912b was used. The paper says structurality is defined relative to default deployments (Remark 2), but it does not establish that this test configuration is the default shipped configuration, nor that a June 2026 uAgents release and an AP2 v0.2.0 sample represent current production behavior. The authors should provide the default configuration artifacts, or at least justify why the selected snapshots and settings are representative.
minor comments (6)
- [§1.2 / Appendix A] The 33-count includes CHAIN, a composite of three other findings, as one 'vulnerability.' This is confusing because CHAIN is not a distinct root cause. Consider separating the composite chain from the count of distinct vulnerabilities.
- [Table 3] A-AP2-9 is absent from Table 3 without an inline explanation. The exclusion is discussed only in §10. Add a footnote to the table.
- [§8.1, P2 paragraph] The parenthetical note about the did:web resolver is extremely long and interrupts the principle statement. Move the implementation detail to Supplementary H and keep P2 as a crisp design principle.
- [Appendix B] The AP2 analysis says 'commit as of June 2026' but gives no fixed commit hash. Pin the exact commit so the code-level claims are reproducible.
- [Tables 2, 4, 5] Some result hashes are truncated (e.g., '821d683f...', 'a417b861...', 'v13_PROVEN....'). Provide full SHA-256 digests in the appendix so the reproducibility claim can be checked.
- [§6.2] The sentence following Table 3 says 'a credentials provider that verifies the mandate but not the caller's identity (A-AP2-15)' and is fine, but it would help to state explicitly which findings are in the 'Spec' vs 'Both' category in the text, not only in the table.
Circularity Check
Minor defense-validation circularity in PCAT's same-suite evaluation; the central 33-vulnerability inventory is not constructed from its inputs.
specific steps
-
other
[Section 8.3 (Evaluation), Table 5]
"We evaluated PCAT by running our attack suite through the middleware."
The five PCAT principles were designed to negate RC-1–RC-5, the classes induced from the same 33-finding AIP-Bench suite used for evaluation. The after-ASR of 0% is obtained by feeding that suite's payloads (e.g., unsigned V5 registry response, cross-origin V10 POST) through rules that reject exactly those payloads (P1 requires a signature; P3 requires a CSRF token). The result verifies that PCAT enforces its own predicates — a conformance test — rather than independently showing the structural classes are neutralized. This is a defense-validation circularity and does not affect the vulnerability inventory, which rests on live reproductions and code proofs of independent codebases.
full rationale
No central circularity: the 33 findings are established by live proofs (CoralOS V4/V5/V9/V10/V11/V13/V15/CHAIN, Fetch.ai F-1, AP2 A-AP2-11) and by code-level proofs against each platform's reference implementation, with explicit proof-method labels in Tables 2–4 and Appendix A. The RC taxonomy is stated to have emerged bottom-up from clustering these findings, not from a prior scheme. Self-citations [20,21] are contextual: [21] supplies adversary types and an excluded prior transport-layer study, but the three-platform commerce-stack analysis does not reduce to it, and [20] appears only in related work. Theorem proofs for P1/P3/P4 are standard reductions to cryptographic and serialization assumptions. The only circularity-adjacent element is PCAT's evaluation on the same AIP-Bench suite from which its principles were derived; that makes the 0% ASR partly definitional as a defense check, but the paper also documents which vulnerabilities lie outside PCAT's HTTP scope and which principles require external adoption (P1/P2), so the limitation is scoped rather than hidden. Score 2 reflects this single minor validation-circularity, not a constructed derivation of the attack inventory.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption The analyzed snapshots (CoralOS v1.3.0 commit a38912b, June 2026 uAgents release, AP2 v0.2.0 Python reference) represent the platforms' production/default deployments.
- domain assumption Code-level proofs against reference implementations establish exploitability with the same force as live demonstrations for 21 of the 33 findings.
- domain assumption PCAT's zero-ASR claims assume P1 registry signing keys are uncompromised, P2 operates a DID resolver, and P4 is either single-process or backed by a linearizable compare-and-swap store.
- domain assumption The 'structural vs semantic' boundary is well-defined and every vulnerability maps to exactly one primary class.
invented entities (2)
-
AIP-Bench
independent evidence
-
PCAT sidecar
independent evidence
read the original abstract
Agentic commerce platforms let AI agents autonomously discover services, move payments, and wield user credentials on their users' behalf, and they already handle real money. Their security has so far been studied almost entirely at the level of the AI model, through prompt injection and misalignment. We show that the more consequential risks lie one layer down, in the protocol between agents and commerce services. There, vulnerabilities are structural : exploitation is deterministic and ndependent of which model an agent runs, so no model improvement removes them. Across three leading platforms we identify 33 such vulnerabilities, each succeeding deterministically regardless of the deployed model, at a 100% attack-success rate (ASR) wherever live-measured. The same failure modes recur across independently built codebases, a systemic pattern rather than isolated bugs. Three of them chain into an end-to-end payment hijack. We contribute a taxonomy separating these structural attacks from model-dependent semantic ones. We also build two artifacts: AIP-Bench (Agent Interaction Protocol Benchmark), to our knowledge the first deterministic benchmark for agentic commerce security, and PCAT (Protocol-level Commerce Agent Trust), a platform-agnostic defense that drives the structural attack-success rate to zero for four of the five structural classes (RC-1, RC-2, RC-4, RC-5), with RC-3 (observable credential channels) reduced to warn-only, without modifying any platform. Agentic commerce must be secured at the protocol layer, not only the model.
Reference graph
Works this paper leans on
-
[1]
Varun Pratap Bhardwaj. Formal Analysis and Supply Chain Security for Agentic AI Skills.arXiv preprint arXiv:2603.00195, 2026
arXiv 2026
-
[2]
Checking for Race Conditions in File Accesses
Matt Bishop and Michael Dilger. Checking for Race Conditions in File Accesses. In Computing Systems, volume 9, pages 131–152, 1996
1996
-
[3]
Are Aligned Neural Networks Adversarially Aligned? InAdvances in Neural Information Processing Systems, 2023
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. Are Aligned Neural Networks Adversarially Aligned? InAdvances in Neural Information Processing Systems, 2023
2023
-
[4]
RAILS: Verification-Native Clearing for Agentic Commerce.arXiv preprint arXiv:2606.08790, 2026
Adrian de Valois-Franklin and Alex Bogdan. RAILS: Verification-Native Clearing for Agentic Commerce.arXiv preprint arXiv:2606.08790, 2026
Pith/arXiv arXiv 2026
-
[5]
AgentDojo: ADynamicEnvironmenttoEvaluatePromptInjection Attacks and Defenses for LLM Agents
Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, andFlorianTramèr. AgentDojo: ADynamicEnvironmenttoEvaluatePromptInjection Attacks and Defenses for LLM Agents. InAdvances in Neural Information Processing Systems, volume 37, pages 82895–82920, 2024
2024
-
[6]
Robin Dey and Panyanon Viradecha. Infrastructure for the Agentic Web: Gap Analysis and Architecture from the Agentverse Platform.arXiv preprint arXiv:2606.20570, 2026
Pith/arXiv arXiv 2026
-
[7]
Georgio, Caelum Forder, Suman Deb, Andri Rahimov, Peter Carroll, and Önder Gürcan
Roman J. Georgio, Caelum Forder, Suman Deb, Andri Rahimov, Peter Carroll, and Önder Gürcan. The Coral Protocol: Open Infrastructure Connecting the Internet of Agents.arXiv preprint arXiv:2505.00749, 2025. Implementation:https://github. com/Coral-Protocol/coral-server. 23
Pith/arXiv arXiv 2025
-
[8]
Agent Payments Protocol (AP2): Building a Secure and Interoperable Future for AI-Driven Payments.https://github.com/google-agentic-commerce/AP2, 2025
Google. Agent Payments Protocol (AP2): Building a Secure and Interoperable Future for AI-Driven Payments.https://github.com/google-agentic-commerce/AP2, 2025
2025
-
[9]
Not What You’ve Signed Up For: Compromising Real-World LLM- Integrated Applications with Indirect Prompt Injection
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not What You’ve Signed Up For: Compromising Real-World LLM- Integrated Applications with Indirect Prompt Injection. InACM Workshop on Artificial Intelligence and Security, pages 79–90, 2023
2023
-
[10]
Yuhang Huang, Boyang Ma, Biwei Yan, Xuelong Dai, Yechao Zhang, Minghui Xu, Kaidi Xu, and Yue Zhang. Give Them an Inch and They Will Take a Mile: Understanding and Measuring Caller Identity Confusion in MCP-Based AI Systems.arXiv preprint arXiv:2603.07473, 2026
Pith/arXiv arXiv 2026
-
[11]
Juhee Kim, Xiaoyuan Liu, Zhun Wang, Shi Qiu, Bo Li, Wenbo Guo, and Dawn Song. The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey.arXiv preprint arXiv:2603.11088, 2026
arXiv 2026
-
[12]
Lars Kersten Kroehl. Trust Without Trusting: A Recomputable Trust Protocol for Autonomous Agents.arXiv preprint arXiv:2605.06738, 2026
Pith/arXiv arXiv 2026
-
[13]
Andrew Kurtz and Klaudia Krawiecka. Who Governs the Machine? A Machine Iden- tity Governance Taxonomy (MIGT) for AI Systems Operating Across Enterprise and Geopolitical Boundaries.arXiv preprint arXiv:2604.06148, 2026
Pith/arXiv arXiv 2026
-
[14]
Qianlong Lan, Anuj Kaul, Shaun Jones, and Stephanie Westrum. Zero-Trust Runtime Verification for Agentic Payment Protocols: Mitigating Replay and Context-Binding Failures in AP2.arXiv preprint arXiv:2602.06345, 2026
arXiv 2026
-
[15]
A402: Binding Cryptocurrency Payments to Service Execution for Agentic Commerce
Yue Li, Lei Wang, Kaixuan Wang, Zhiqiang Yang, Ke Wang, Zhi Guan, and Jianbo Gao. A402: Binding Cryptocurrency Payments to Service Execution for Agentic Commerce. arXiv preprint arXiv:2603.01179, 2026
arXiv 2026
-
[16]
Derek Lilienthal and Sanghyun Hong. Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents.arXiv preprint arXiv:2508.17155, 2025
Pith/arXiv arXiv 2025
-
[17]
Shengchen Ling, Yihang Huang, Yuefeng Du, Yuan Chen, Yajin Zhou, Lei Wu, and Cong Wang. Free-Riding the Agentic Web: A Systematic Security Analysis of x402 Payments.arXiv preprint arXiv:2605.30998, 2026
Pith/arXiv arXiv 2026
-
[18]
Hanzhi Liu, Chaofan Shou, Hongbo Wen, Yanju Chen, Ryan Jingyang Fang, and Yu Feng. Your Agent Is Mine: Measuring Malicious Intermediary Attacks on the LLM Supply Chain.arXiv preprint arXiv:2604.08407, 2026
Pith/arXiv arXiv 2026
-
[19]
YiLiu, WeizheWang, RuitaoFeng, YaoZhang, GuangquanXu, GeleiDeng, YuekangLi, and Leo Zhang. Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale.arXiv preprint arXiv:2601.10338, 2026. 24
Pith/arXiv arXiv 2026
-
[20]
Yedidel Louck. Securing LLM-Agent Long-Term Memory Against Poisoning: Non- Malleable, Origin-Bound Authority with Machine-Checked Guarantees.arXiv preprint arXiv:2606.24322, 2026
Pith/arXiv arXiv 2026
-
[21]
Yedidel Louck, Amit Dvir, and Ariel Stulman. Security Analysis of Agentic AI Com- munication Protocols: A Comparative Evaluation.ACM Transactions on AI Security and Privacy, 2025. arXiv:2511.03841
arXiv 2025
-
[22]
SoK: Security of Autonomous LLM Agents in Agentic Commerce.arXiv preprint arXiv:2604.15367, 2026
Qian’ang Mao, Jiaxin Wang, Ya Liu, Li Zhu, Cong Ma, and Jiaqi Yan. SoK: Security of Autonomous LLM Agents in Agentic Commerce.arXiv preprint arXiv:2604.15367, 2026
Pith/arXiv arXiv 2026
-
[23]
Alan L. McCann. Cryptographic Registry Provenance: Structural Defense Against Dependency Confusion in AI Package Ecosystems.arXiv preprint arXiv:2605.03309, 2026
Pith/arXiv arXiv 2026
-
[24]
OWASP Top 10 for Large Language Model Applications.https: //owasp.org/www-project-top-10-for-llm-applications/, 2025
OWASP Foundation. OWASP Top 10 for Large Language Model Applications.https: //owasp.org/www-project-top-10-for-llm-applications/, 2025
2025
-
[25]
Yi Ting Shen, Kentaroh Toyoda, and Alex Leung. MCP-38: A Comprehensive Threat Taxonomy for Model Context Protocol Systems.arXiv preprint arXiv:2603.18063, 2026
arXiv 2026
-
[26]
Prompt Injection Attack to Tool Selection in LLM Agents
JiawenShi, ZenghuiYuan, GuiyaoTie, PanZhou, NeilZhenqiangGong, andLichaoSun. Prompt Injection Attack to Tool Selection in LLM Agents. InNetwork and Distributed System Security Symposium (NDSS), 2026
2026
-
[27]
Uchi Uchibeke. Before the Tool Call: Deterministic Pre-Action Authorization for Au- tonomous AI Agents.arXiv preprint arXiv:2603.20953, 2026
arXiv 2026
-
[28]
Verifiable Credentials Data Model 2.0.https://www.w3.org/TR/ vc-data-model-2.0/, 2024
W3C. Verifiable Credentials Data Model 2.0.https://www.w3.org/TR/ vc-data-model-2.0/, 2024
2024
-
[29]
Edwin B. Wilson. Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158):209–212, 1927
1927
-
[30]
Wooldridge, Attila Bagoly, Jonathan J
Michael J. Wooldridge, Attila Bagoly, Jonathan J. Ward, Emanuele La Malfa, and Gabriel Paludo Licks. Fetch.ai: An Architecture for Modern Multi-Agent Systems. arXiv preprint arXiv:2510.18699, 2025. Implementation (uAgents):https://github. com/fetchai/uAgents
arXiv 2025
-
[31]
Zonghao Ying, Haozheng Wang, Jiangfan Liu, Quanchen Zou, Aishan Liu, Jian Yang, Yaodong Yang, and Xianglong Liu. AgentVisor: Defending LLM Agents Against Prompt Injection via Semantic Virtualization.arXiv preprint arXiv:2604.24118, 2026
Pith/arXiv arXiv 2026
-
[32]
SUDP: Secret-Use Delegation Protocol for Agentic Systems.arXiv preprint arXiv:2604.24920, 2026
Xiaohang Yu, Hejia Geng, Xinmeng Zeng, and William Knottenbelt. SUDP: Secret-Use Delegation Protocol for Agentic Systems.arXiv preprint arXiv:2604.24920, 2026. 25
Pith/arXiv arXiv 2026
-
[33]
research-key-2026
Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. InjecAgent: Benchmark- ing Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. In Findings of the ACL, 2024. A Consolidated Findings by Root-Cause Class Table 10 lists all 33 structural findings grouped by root-cause class, with platform, the constituent IDs, the proof method,...
2024
-
[34]
ts " : ts ,
compose into a double-spend under multi-worker deployment, where two workers both observeused=Falsebefore either writes back. 29 GPCATImplementation Notes The P1 signing payload uses JSON to avoid delimiter-collision attacks: 1def _ b u i l d _ s i g n i n g _ p a y l o a d ( body : bytes , ts : str , nonce : str ) -> bytes : 2return json . dumps ({ 3" ts...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.