A Novel Hierarchical Multi-Agent System for Payments Using LLMs

Donghao Huang; Joon Kiat Chua; Zhaoxia Wang

REVIEW 4 major objections 7 minor 24 references

HMASP routes a natural-language payment request through four levels of LLM agents, claiming to be the first end-to-end agentic payment workflow, with 95–100% success on simulated tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:02 UTC pith:CNYFCNQM

load-bearing objection A coherent architecture for LLM-based payments, but the end-to-end claim is provisional and rests on a simulated payment layer that may not transfer to real rails. the 4 major comments →

arxiv 2602.24068 v1 pith:CNYFCNQM submitted 2026-02-27 cs.MA cs.CL

A Novel Hierarchical Multi-Agent System for Payments Using LLMs

Joon Kiat Chua , Donghao Huang , Zhaoxia Wang This is my paper

classification cs.MA cs.CL

keywords large language modelsmulti-agent systemsagentic paymentshierarchical architecturepayment workflowshuman-in-the-loopstate managementtask routing

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a hierarchical team of LLM agents can execute an entire payment workflow—from card registration to checkout—when a user or external agent simply states the request in natural language. The proposed system, HMASP, arranges agents into four levels so that no single model sees all the data at once: a conversational agent fields the request, supervisors route it, routing agents trigger deterministic function modules, and summary agents report outcomes back up. The paper argues this architecture reduces hallucination risk for payment data and works with both open-weight and proprietary LLMs. Evidence comes from a 1,000-request simulated evaluation where the best open-weight model and the proprietary baseline both complete payment tasks in over 95% of runs. If true, this would give the agentic-commerce field a reference design for end-to-end payments rather than just checkout-side helpers.

Core claim

The paper's central claim is that end-to-end agentic payment workflows can be achieved by an LLM-based multi-agent system, and that HMASP is the first such implementation. The discovery is architectural: role-based hierarchy plus deterministic state management. Payment-sensitive data is stored in shared state variables that function modules read and write directly, so critical numbers (card number, last four digits, transaction record) are never generated by an LLM and therefore cannot be hallucinated or tampered with. The conversational agent is the only component exposed to external requests, and an interrupt mechanism pauses the workflow whenever the user must supply sensitive details suc

What carries the argument

The load-bearing object is HMASP's four-level agent hierarchy: the Conversational Payment Agent (the single entry point), Supervisor agents (domain-level decision makers), Routing agents (workflow triggers), and Process summary agents (outcome reporters). What makes the hierarchy work are three coordination mechanisms: shared state variables that let different roles exchange only the necessary IDs; decoupled, role-specific message states that hide sensitive content from unrelated agents; and structured handoffs that pass an external request downward until the right workflow triggers, then pass a summary upward. The critical feature for payments is determinism: user-supplied and workflow-crit

Load-bearing premise

The whole feasibility result rests on the assumption that the simulated payment functions behave like real payment operations—specifically that issuer authentication and PCI-DSS compliance checks change latency but do not change which requests succeed or fail.

What would settle it

Run the same 1,000-request evaluation against a live issuer sandbox that enforces real authentication and compliance checks. If requests that passed the simulated workflow start failing or being blocked at a materially higher rate, the assumption that production complexity only affects latency is false and the central claim of end-to-end agentic payments is unsupported.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

If HMASP holds up, an external shopping agent can complete a purchase by telling the conversational agent to pay, without the merchant embedding payment logic into the agent's own code.
Payment networks can adopt the hierarchy as a reference pattern: the LLM layer handles natural-language routing while all sensitive operations happen in isolated, deterministic function modules.
Organizations can choose whichever LLM fits their cost or security posture; the paper's results show at least one open-weight model matching the proprietary baseline near 95–100% task success.
Human-in-the-loop interrupts become a security feature rather than a UX break: the workflow pauses, validates the user's digits, and only resumes with validated state.
A failed or irrelevant request is deliberately rejected at the earliest agent level rather than forwarded, which is how the system avoids hallucinated payment actions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same four-level pattern—conversational entry, supervisor routing, deterministic function modules, summary reporting—could transfer to other regulated domains such as account onboarding or insurance claims, where a human must approve sensitive steps and critical values must not be guessed by the model.
A live pilot against a card-network sandbox would separate the architectural claim (routing works) from the feasibility claim (payments complete), since the paper's success rates are measured on simulated functions that bypass issuer authentication and compliance checks.
The deterministic state design suggests a testable extension: deliberately inject conflicting payment data into the conversation to see whether the agent ever echoes the hallucinated value instead of the state variable; the paper's determinism claims predict it will not.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A coherent architecture for LLM-based payments, but the end-to-end claim is provisional and rests on a simulated payment layer that may not transfer to real rails.

read the letter

HMASP is a reasonable first cut at a real gap: LLM agents that can carry a payment task from a natural-language request through card registration, retrieval, and checkout. The architecture is coherent—role-based hierarchy (CPA, supervisors, routers, summary agents), state isolation, decoupled messages, structured handoffs, and an interrupt mechanism for sensitive inputs. Using LangGraph to enforce deterministic state handling is a sensible way to keep LLMs away from mutable payment data. The evaluation runs 11 models on 1,000 hand-labeled inputs and shows GPT-4.1 and Qwen2.5:32b do well, while smaller models struggle—that is a useful comparative signal.

What is genuinely new here is the application: prior LLM payment work stops at the network boundary; this one attempts full workflow completion. That said, the novelty is at the application level, not the method level—the patterns are known from MetaGPT and similar role-based MAS.

Soft spots, in order of importance. First, the payment modules are simulated functions. The paper’s own Section 3.5 says production complexities like 3DS or PCI validation are expected to affect latency, not architecture. That assumption is load-bearing and unverified. Real 3DS involves async redirects and callbacks; PCI scope changes where data can live. Those are control-flow changes, not just slower versions of the same operations. The stress-test note is right: this is a feasibility study of agent routing and handoff in a controlled environment, not proof of end-to-end payment execution on real rails. The abstract overstates the result by omitting the word “simulated.” Second, the dataset is self-created and unreleased; we have no way to check the labels or the distribution. Third, the tables report single percentages without confidence intervals or repeated trials; success rates of 99% vs 96% may or may not be distinguishable.

None of this kills the paper. The limitations section is honest about the dataset size and future work. But the central claim needs to be scoped carefully, and the evaluation needs a reproducibility upgrade before the “first end-to-end” claim is accepted.

Who should read it: people building agentic commerce prototypes, and anyone thinking about how to keep LLMs out of critical payment logic. It deserves a serious referee, but with a request to qualify the claims and add repeated runs or statistical bands.

Referee Report

4 major / 7 minor

Summary. The paper proposes HMASP, a four-level hierarchical multi-agent system for LLM-based payment workflows. The architecture comprises a Conversational Payment Agent (CPA) as the entry point, Supervisor agents, Routing agents, and Process Summary agents, together with structured handoffs, shared state variables, decoupled message states, and interrupt-based human-in-the-loop. The system is evaluated on 1,000 hand-labeled utterances (250 per task) covering card registration, card retrieval, payment processing, and irrelevant-input rejection; all payment modules are simulated functions. The authors report task success rates and handoff F1 scores for GPT-4.1 and a range of open-weight models (Qwen, Mistral, Llama). They claim that HMASP is the first LLM-based multi-agent system to implement end-to-end agentic payment workflows.

Significance. If the architecture performs as claimed, it offers a useful modular reference design for payment agents, with a clear separation of conversational orchestration, routing, workflow execution, and response summarization. The design patterns—deterministic state variables for sensitive data, interrupt handling, and decoupled message states—are sensible and well-motivated. The multi-model evaluation, including both proprietary and open-weight models, is a strength. However, the evidence for feasibility is limited to a simulated payment environment and a self-created, unreleased dataset with single-run percentages. This weakens the central 'end-to-end' novelty claim. The paper is a promising architectural proposal rather than a demonstration of production-ready agentic payments.

major comments (4)

[§3.5, Table 1] The evaluation's load-bearing assumption is that production payment complexities (issuer authentication, PCI-DSS validation) affect latency rather than architectural properties. This is not supported: real payment operations introduce asynchronous redirects (3-D Secure), callback timeouts, step-up authentication, retry states, and PCI scope restrictions, which alter control flow and can change which requests succeed or fail. Since all payment modules (e.g., 'Authorize 3DS') are simulated as local functions, the reported success rates and F1 scores do not transfer directly to actual payment rails. The Limitations section concedes that the dataset 'may not fully capture the complexity of all payment workflows.' Please either evaluate against a real or high-fidelity sandbox payment API, or explicitly restrict the 'end-to-end' claim to simulated workflows.
[§3.4, Tables 1–2] The dataset is self-created, unreleased, and contains only 250 examples per task; no inter-annotator agreement, example distribution, or data-generation details are provided. All percentages in Tables 1 and 2 appear to come from a single run per model. For 250 binary trials, the 95% binomial CI width is approximately ±6 percentage points; for several cells (e.g., Qwen3:14b T2=33.2% vs. Qwen3:32b T2=35.2%) the differences are within this noise. Please report repeated runs/seeds, bootstrap confidence intervals, and release the dataset/annotations or provide a detailed reproducibility appendix.
[§3.5, Table 3] The novelty comparison in Table 3 is not apples-to-apples. Methods [7], [5], and [17] solve different tasks (mobile payment app navigation, fraud/scam detection, fraud risk detection) and were not designed to trigger payment workflows; their metrics are not directly comparable to HMASP's workflow-triggering success rate. HMASP's row reports the best open-weight model (Qwen2.5:32b) only, which can overstate robustness given the model variability in Table 1. Moreover, 'Full Payment Processing = Yes' is based on simulated functions, not real payment processing. I recommend presenting the comparison as qualitative architecture positioning rather than quantitative superiority, and reporting all models or a confidence interval.
[§3.3, Table 2] The deterministic state-variable design prevents LLM tampering with stored payment data, but LLMs still make routing, handoff, and natural-language extraction decisions (e.g., selecting which card, what transaction amount). Table 2 shows substantial variability across models, and some models fail T1–T3 more than half the time. The paper does not provide an error breakdown—such as wrong workflow trigger vs. wrong data saved vs. interrupt handling errors—so the claim to 'mitigate against hallucination risks' is only partially supported. Please add a failure analysis or error-type categories to the metrics.

minor comments (7)

[Abstract and Section 5] The phrase 'end-to-end agentic payment workflows' should be qualified as 'in a simulated environment' in both the abstract and the conclusion; the contribution bullet already includes this caveat, but the abstract does not.
[§3.1, §3.5, §4] Typos and spacing issues: 'sent sent' in §3.1, 'utillized' in §3.5, 'existing existing' in §4, and 'F easibility' in the contributions list.
[Figure 1] The figure and its (a)–(f) labels are described in the text, but the figure itself is not included in the manuscript text. Consider adding the figure or a state-transition diagram, as the workflow description is central to understanding the architecture.
[§3.2] The three state types are defined, but no concrete list of shared variables or access-control rules is provided. A small example would clarify how 'only shared variables are visible across states' works in practice.
[§3.5] The evaluation setup reports 'Ollama 3 v0.9.6'; please specify the exact version and whether all models were run with default decoding parameters (temperature, top-p, etc.), since these can affect handoff decisions.
[References] Several references lack complete publication details (e.g., [3] has no venue, [11] and [13] are accessed URLs); please verify and format consistently.
[Table 3] The column label 'TR Precision-Recall (%)' is ambiguous because precision and recall are separate numbers; label as 'Precision / Recall (%)' and clarify the exact definition used for the comparative metrics.

Circularity Check

0 steps flagged

No substantive circularity: HMASP's feasibility evaluation is operational and its simulation assumption is disclosed.

full rationale

The paper does not claim a derivation or prediction that reduces to its inputs. Task success rate and handoff F1 are direct operational measurements of workflow triggering and information persistence against a hand-labeled dataset; they are not fitted parameters renamed as predictions. The payment modules are explicitly disclosed as simulated functions, and the statement that production complexities 'are expected to affect latency rather than the architectural properties evaluated' is an untested external-validity assumption, not a circular step. The novelty claim is hedged with 'To our knowledge' and supported by a comparative table, not by a self-citation or imported uniqueness theorem. Mastercard affiliation and references to LangGraph provide context and tooling, but are not load-bearing circularity. Therefore, the central claim rests on stated assumptions and an internal benchmark; any weakness is about generalizability, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 3 axioms · 0 invented entities

The system has no numeric free parameters: the reported numbers are measured success rates, not fitted values. Its assumptions are architectural and domain-level rather than mathematical; the main risk is that the simulated payment functions do not capture real-world payment complexity.

axioms (3)

domain assumption Payment modules can be exactly simulated as functions; production complexity (authentication, PCI-DSS) affects latency but not architecture properties
Invoked in Section 3.5; if false, the simulated feasibility results fail to transfer to real payment rails.
domain assumption The 1,000-point dataset (250 per task) is representative of real user payment requests
Section 3.4; the dataset is self-created by the authors and unreleased, so its representativeness is unverified.
domain assumption Role-based prompting is sufficient for LLM agents to perform reliable handoffs and state isolation
The architecture assumes each agent follows its role prompt, uses only assigned states, and hands off reliably; the paper itself cites (Cemri et al.) that LLM-MAS failures often stem from system design.

pith-pipeline@v1.3.0-alltime-deepseek · 8963 in / 10901 out tokens · 103489 ms · 2026-08-02T20:02:03.887038+00:00 · methodology

0 comments

read the original abstract

Large language model (LLM) agents, such as OpenAI's Operator and Claude's Computer Use, can automate workflows but unable to handle payment tasks. Existing agentic solutions have gained significant attention; however, even the latest approaches face challenges in implementing end-to-end agentic payment workflows. To address this gap, this research proposes the Hierarchical Multi-Agent System for Payments (HMASP), which provides an end-to-end agentic method for completing payment workflows. The proposed HMASP leverages either open-weight or proprietary LLMs and employs a modular architecture consisting of the Conversational Payment Agent (CPA - first agent level), Supervisor agents (second agent level), Routing agents (third agent level), and the Process summary agent (fourth agent level). The CPA serves as the central entry point, handling all external requests and coordinating subsequent tasks across hierarchical levels. HMASP incorporates architectural patterns that enable modular task execution across agents and levels for payment operations, including shared state variables, decoupled message states, and structured handoff protocols that facilitate coordination across agents and workflows. Experimental results demonstrate the feasibility of the proposed HMASP. To our knowledge, HMASP is the first LLM-based multi-agent system to implement end-to-end agentic payment workflows. This work lays a foundation for extending agentic capabilities into the payment domain.

Figures

Figures reproduced from arXiv: 2602.24068 by Donghao Huang, Joon Kiat Chua, Zhaoxia Wang.

**Figure 1.** Figure 1: Overview of the proposed HMASP. An external request (e.g., “Complete my payment”) is sent to the Conversational Payment Agent (CPA - first agent level); (a) The CPA processes the request and delegates it to one of the two Supervisors (b1, b2 - second agent level) for further handling; (b) The selected Supervisor routes the request to a suitable Routing agent (c1, c2, or c3 - third agent level); (c) Routing… view at source ↗

discussion (0)

Reference graph

Works this paper leans on

24 extracted references · 5 linked inside Pith

[1]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

Pith/arXiv arXiv 2023
[2]

https://www.anthropic.com/news/3-5-model s-and-computer-use (2025), accessed: 2025-11-15

Anthropic: Computer use (beta). https://www.anthropic.com/news/3-5-model s-and-computer-use (2025), accessed: 2025-11-15

2025
[3]

Cemri, M., Pan, M.Z., Yang, S., Agrawal, L.A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., et al.: Why do multi-agent llm systems fail? (2025)

2025
[4]

https://www.dwt.com/blogs/arti ficial-intelligence-law-advisor/2025/10/agentic-ai-co ncerns-for-merch ants-and-issuers (2025), accessed: 2025-11-15

Claudia, Lin and Ken, Taylor and Rich, Zukowsky and Dalia, Wrocherinsky: The next ai frontier: From prompts to purchases. https://www.dwt.com/blogs/arti ficial-intelligence-law-advisor/2025/10/agentic-ai-co ncerns-for-merch ants-and-issuers (2025), accessed: 2025-11-15

2025
[5]

In: 2024 IEEE International Conference on Big Data (BigData)

Dahiphale, D., Madiraju, N., Lin, J., Karve, R., Agrawal, M., Modwal, A., Bal- akrishnan, R., Shah, S., Kaushal, G., Mandawat, P., et al.: Enhancing trust and safety in digital payments: An llm-powered approach. In: 2024 IEEE International Conference on Big Data (BigData). pp. 4854–4863. IEEE (2024)

2024
[6]

arXiv preprint arXiv:2407.21783 (2024)

Grattaﬁori, A., Dubey, A., Jauhri, A., Pandey, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

Pith/arXiv arXiv 2024
[7]

In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Guan, Y., Wang, D., Chu, Z., Wang, S., Ni, F., Song, R., Zhuang, C.: Intelli- gent agents with llm-based process automation. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. p. 5018–5027. KDD ’24, Association for Computing Machinery, New York, NY, USA (2024). https://doi.org/10.1145/3637528.3671646

arXiv 2024
[8]

In: Proceedings of the Thirty-Third International Joint Conference on Artiﬁcial Intelligence

Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N.V., Wiest, O., Zhang, X.: Large language model based multi-agents: a survey of progress and challenges. In: Proceedings of the Thirty-Third International Joint Conference on Artiﬁcial Intelligence. IJCAI ’24 (2024). https://doi.org/10.24963/ijcai.2024/890 12 J.K. Chua, D. Huang, and Z. Wang

doi:10.24963/ijcai.2024/890 2024
[9]

In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=VtmBAG CN7o

Hong, S., Zhuge, M., Chen, J., Zheng, X., et al.: MetaGPT: Meta programming for a multi-agent collaborative framework. In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=VtmBAG CN7o

2024
[10]

ACM Transactions on Information Systems 43(2), 1–55 (2025)

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al.: A survey on hallucination in large language mod- els: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems 43(2), 1–55 (2025)

2025
[11]

Katharina, Schumacher and Roger, Roberts and Katharina, Giebel: The agentic commerce opportunity: How ai agents are ushering in a new era for consumers and merchants. https://www.mckinsey.com/capabilities/quantumblack/ our-ins ights/the-agentic-commerce-opportunity-how-ai-agents-ar e-ushering-i n-a-new-era-for-consumers-and-merchants (2025), accessed: 2025-11-15

2025
[12]

https://www.langchain.com/la nggraph (2025), accessed: 2025-11-15

LangChain: Balance agent control with agency. https://www.langchain.com/la nggraph (2025), accessed: 2025-11-15

2025
[13]

http s://www.mastercard.com/global/en/business/artificia l-intelligence/ma stercard-agent-pay.html (2025), accessed: 2025-11-15

Mastercard: Mastercard agent pay - powering the next frontier of commerce. http s://www.mastercard.com/global/en/business/artificia l-intelligence/ma stercard-agent-pay.html (2025), accessed: 2025-11-15

2025
[14]

https://openai.com/index/introducing-opera tor/ (January 2025), accessed: 2025-11-15

OpenAI: Introducing operator. https://openai.com/index/introducing-opera tor/ (January 2025), accessed: 2025-11-15

2025
[15]

https://blog.pcisecuritystandards.org/ai-principles- secur ing-the-use-of-ai-in-payment-environments (2025), accessed: 2025-11-15

PCI Security Standards Council: Ai principles: Securing the use of ai in payment environments. https://blog.pcisecuritystandards.org/ai-principles- secur ing-the-use-of-ai-in-payment-environments (2025), accessed: 2025-11-15

2025
[16]

arXiv preprint arXiv:2503.23037 (2025)

Plaat, A., van Duijn, M., van Stein, N., Preuss, M., van der Putten, P., Batenburg, K.J.: Agentic large language models, a survey. arXiv preprint arXiv:2503.23037 (2025)

arXiv 2025
[17]

In: Rehm, G., Li, Y

Qu, B., Wang, Z., Yagi, D., Xu, Z., Zhao, Y., Shan, Y., Zahradnik, F.: LLM- enhanced self-evolving reinforcement learning for multi-step E-commerce payment fraud risk detection. In: Rehm, G., Li, Y. (eds.) Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track). pp. 92–103. Association for Computa...

2025
[18]

arXiv preprint arXiv:2506.10910 (2025)

Rastogi, A., Jiang, A.Q., Lo, A., Berrada, G., et al.: Magistral. arXiv preprint arXiv:2506.10910 (2025)

Pith/arXiv arXiv 2025
[19]

Tran, K.T., Dao, D., Nguyen, M.D., Pham, Q.V., O’Sullivan, B., Nguyen, H.D.: Multi-agent collaboration mechanisms: A survey of llms (2025)

2025
[20]

https://corporate.vi sa.com/en/products/intelligent-commerce.html (2025), accessed: 2025-11-15

Visa: Enabling ai agents to buy securely and seamlessly. https://corporate.vi sa.com/en/products/intelligent-commerce.html (2025), accessed: 2025-11-15

2025
[21]

Frontiers of Computer Science 18(6), 186345 (2024)

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al.: A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024)

2024
[22]

arXiv preprint arXiv:2505.09388 (2025)

Yang, A., Li, A., Yang, B., Zhang, B., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

Pith/arXiv arXiv 2025
[23]

arXiv preprint arXiv:2412.15115 (2024)

Yang, A., Yang, B., Zhang, B., Hui, B., et al.: Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 (2024)

Pith/arXiv arXiv 2024
[24]

Computational Linguistics pp

Zhang, Y., Li, Y., Cui, L., Cai, D., et al.: Siren’s song in the ai ocean: A survey on hallucination in large language models. Computational Linguistics pp. 1–45 (2025)

2025

[1] [1]

arXiv preprint arXiv:2303.08774 (2023)

Achiam, J., Adler, S., Agarwal, S., Ahmad, L., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

Pith/arXiv arXiv 2023

[2] [2]

https://www.anthropic.com/news/3-5-model s-and-computer-use (2025), accessed: 2025-11-15

Anthropic: Computer use (beta). https://www.anthropic.com/news/3-5-model s-and-computer-use (2025), accessed: 2025-11-15

2025

[3] [3]

Cemri, M., Pan, M.Z., Yang, S., Agrawal, L.A., Chopra, B., Tiwari, R., Keutzer, K., Parameswaran, A., Klein, D., Ramchandran, K., et al.: Why do multi-agent llm systems fail? (2025)

2025

[4] [4]

https://www.dwt.com/blogs/arti ficial-intelligence-law-advisor/2025/10/agentic-ai-co ncerns-for-merch ants-and-issuers (2025), accessed: 2025-11-15

Claudia, Lin and Ken, Taylor and Rich, Zukowsky and Dalia, Wrocherinsky: The next ai frontier: From prompts to purchases. https://www.dwt.com/blogs/arti ficial-intelligence-law-advisor/2025/10/agentic-ai-co ncerns-for-merch ants-and-issuers (2025), accessed: 2025-11-15

2025

[5] [5]

In: 2024 IEEE International Conference on Big Data (BigData)

Dahiphale, D., Madiraju, N., Lin, J., Karve, R., Agrawal, M., Modwal, A., Bal- akrishnan, R., Shah, S., Kaushal, G., Mandawat, P., et al.: Enhancing trust and safety in digital payments: An llm-powered approach. In: 2024 IEEE International Conference on Big Data (BigData). pp. 4854–4863. IEEE (2024)

2024

[6] [6]

arXiv preprint arXiv:2407.21783 (2024)

Grattaﬁori, A., Dubey, A., Jauhri, A., Pandey, A., et al.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

Pith/arXiv arXiv 2024

[7] [7]

In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Guan, Y., Wang, D., Chu, Z., Wang, S., Ni, F., Song, R., Zhuang, C.: Intelli- gent agents with llm-based process automation. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. p. 5018–5027. KDD ’24, Association for Computing Machinery, New York, NY, USA (2024). https://doi.org/10.1145/3637528.3671646

arXiv 2024

[8] [8]

In: Proceedings of the Thirty-Third International Joint Conference on Artiﬁcial Intelligence

Guo, T., Chen, X., Wang, Y., Chang, R., Pei, S., Chawla, N.V., Wiest, O., Zhang, X.: Large language model based multi-agents: a survey of progress and challenges. In: Proceedings of the Thirty-Third International Joint Conference on Artiﬁcial Intelligence. IJCAI ’24 (2024). https://doi.org/10.24963/ijcai.2024/890 12 J.K. Chua, D. Huang, and Z. Wang

doi:10.24963/ijcai.2024/890 2024

[9] [9]

In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=VtmBAG CN7o

Hong, S., Zhuge, M., Chen, J., Zheng, X., et al.: MetaGPT: Meta programming for a multi-agent collaborative framework. In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=VtmBAG CN7o

2024

[10] [10]

ACM Transactions on Information Systems 43(2), 1–55 (2025)

Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al.: A survey on hallucination in large language mod- els: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems 43(2), 1–55 (2025)

2025

[11] [11]

Katharina, Schumacher and Roger, Roberts and Katharina, Giebel: The agentic commerce opportunity: How ai agents are ushering in a new era for consumers and merchants. https://www.mckinsey.com/capabilities/quantumblack/ our-ins ights/the-agentic-commerce-opportunity-how-ai-agents-ar e-ushering-i n-a-new-era-for-consumers-and-merchants (2025), accessed: 2025-11-15

2025

[12] [12]

https://www.langchain.com/la nggraph (2025), accessed: 2025-11-15

LangChain: Balance agent control with agency. https://www.langchain.com/la nggraph (2025), accessed: 2025-11-15

2025

[13] [13]

http s://www.mastercard.com/global/en/business/artificia l-intelligence/ma stercard-agent-pay.html (2025), accessed: 2025-11-15

Mastercard: Mastercard agent pay - powering the next frontier of commerce. http s://www.mastercard.com/global/en/business/artificia l-intelligence/ma stercard-agent-pay.html (2025), accessed: 2025-11-15

2025

[14] [14]

https://openai.com/index/introducing-opera tor/ (January 2025), accessed: 2025-11-15

OpenAI: Introducing operator. https://openai.com/index/introducing-opera tor/ (January 2025), accessed: 2025-11-15

2025

[15] [15]

https://blog.pcisecuritystandards.org/ai-principles- secur ing-the-use-of-ai-in-payment-environments (2025), accessed: 2025-11-15

PCI Security Standards Council: Ai principles: Securing the use of ai in payment environments. https://blog.pcisecuritystandards.org/ai-principles- secur ing-the-use-of-ai-in-payment-environments (2025), accessed: 2025-11-15

2025

[16] [16]

arXiv preprint arXiv:2503.23037 (2025)

Plaat, A., van Duijn, M., van Stein, N., Preuss, M., van der Putten, P., Batenburg, K.J.: Agentic large language models, a survey. arXiv preprint arXiv:2503.23037 (2025)

arXiv 2025

[17] [17]

In: Rehm, G., Li, Y

Qu, B., Wang, Z., Yagi, D., Xu, Z., Zhao, Y., Shan, Y., Zahradnik, F.: LLM- enhanced self-evolving reinforcement learning for multi-step E-commerce payment fraud risk detection. In: Rehm, G., Li, Y. (eds.) Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track). pp. 92–103. Association for Computa...

2025

[18] [18]

arXiv preprint arXiv:2506.10910 (2025)

Rastogi, A., Jiang, A.Q., Lo, A., Berrada, G., et al.: Magistral. arXiv preprint arXiv:2506.10910 (2025)

Pith/arXiv arXiv 2025

[19] [19]

Tran, K.T., Dao, D., Nguyen, M.D., Pham, Q.V., O’Sullivan, B., Nguyen, H.D.: Multi-agent collaboration mechanisms: A survey of llms (2025)

2025

[20] [20]

https://corporate.vi sa.com/en/products/intelligent-commerce.html (2025), accessed: 2025-11-15

Visa: Enabling ai agents to buy securely and seamlessly. https://corporate.vi sa.com/en/products/intelligent-commerce.html (2025), accessed: 2025-11-15

2025

[21] [21]

Frontiers of Computer Science 18(6), 186345 (2024)

Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al.: A survey on large language model based autonomous agents. Frontiers of Computer Science 18(6), 186345 (2024)

2024

[22] [22]

arXiv preprint arXiv:2505.09388 (2025)

Yang, A., Li, A., Yang, B., Zhang, B., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025)

Pith/arXiv arXiv 2025

[23] [23]

arXiv preprint arXiv:2412.15115 (2024)

Yang, A., Yang, B., Zhang, B., Hui, B., et al.: Qwen2.5 technical report. arXiv preprint arXiv:2412.15115 (2024)

Pith/arXiv arXiv 2024

[24] [24]

Computational Linguistics pp

Zhang, Y., Li, Y., Cui, L., Cai, D., et al.: Siren’s song in the ai ocean: A survey on hallucination in large language models. Computational Linguistics pp. 1–45 (2025)

2025