{"id":"bf65d3c3-9eb5-4a4f-8573-21d2ef58d9d3","arxiv_id":"2506.02040","paper_version":4,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Malicious MCP servers can be uploaded to popular registries, are hard for users to spot, and can manipulate LLM agents into harmful actions.","lead":"Researchers built malicious MCP servers, uploaded them to major model-context-protocol registries, and showed the servers were accepted without detection. In user tests and live attacks, LLM agents often followed hidden instructions in tool descriptions, leading to privacy leaks and unauthorized crypto transfers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"RQ3's attack success rate counts a simulated transfer-tool invocation as successful cryptocurrency theft, with no wallet signing, user confirmation, or on-chain transaction verified, so the 'real-world execution' claim overstates what the reported experiments demonstrate.","rationale":"This is a valuable first end-to-end study: the real uploads, IRB-approved user study with 20 participants, and published artifact are genuine evidence, and the platform-audit and user-identification phases support the paper's practical-risk narrative. The central argument, however, has three links, and the weakest is RQ3's measurement of real-world harm. The paper's own definition of harmful behavior (Section 4.3) is broad enough to count an unauthorized tool call as harm, and the attack framework's mock transfer server answers with a success message. That makes ASR a measure of prompt-injection compliance rather than a measure of consequential execution. This is not an accusation of dishonesty; it is a request to align the claim with the evidence. The proposed test is cheap: rerun the same framework with a testnet wallet and a user-confirmation gate, and report the conditional success rate. If the rate stays high, the real-world claim is strengthened; if it drops, the conclusion should be scoped to 'LLMs can be induced to request harmful actions', which is still an important finding. The reader's weakest_assumption concerned the breadth of the RQ1 audit conclusion based on a single probe server; my concern is distinct but complementary, targeting the RQ3 success metric, so I mark agreement as partial. The reader's CONDITIONAL verdict remains appropriate: no rejection is warranted, but acceptance should require the addendum that 'real-world execution' currently means tool invocation under test-harness conditions for the cryptocurrency scenario.","tokens_in":29760,"tokens_out":8868,"duration_ms":85079,"concrete_test":"Re-run the Cryptocurrency Theft task with a consequential transfer backend: connect the unchanged attack framework to a real or testnet wallet MCP server that requires explicit user confirmation and cryptographic signing before broadcast, and record (a) how often the LLM requests the transfer, (b) how often the user approves, and (c) how often a transaction is actually broadcast. If conditional success from (a) to (c) is near zero, the 'transfer digital assets' claim should be relabeled as 'LLM can be induced to request a transfer', a weaker but more accurate finding. Also report the Privacy Steal and Result Manipulation tasks with verifiable file-system reads and observable output changes rather than mock side effects.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The answer to RQ3 (Section 4.3.3) — 'All defined attack vectors can be effectively executed in real-world scenarios' — rests on ASR, defined in Section 4.3.1 as the percentage of attacks in which the LLM executes the intended malicious task. In the Cryptocurrency Theft task the intended action is that the malicious server 'stealthily attempts to transfer 0.01 ETH', and the recorded success in Figure 12 is the LLM calling transfer_tool followed by the transfer-mcp server's simulated response '0.01 ETH has been transferred'. The harness therefore equates 'the LLM followed the injected instruction and requested a transfer' with 'digital assets were transferred'. No wallet signature, user approval, transaction broadcast, or balance change is verified anywhere in RQ3. Because the abstract and conclusion phrase the result as controlling devices to transfer digital assets and as all four attack types causing harm in a user's local environment, this conflation is load-bearing: the headline RQ3 result is a prompt-following rate inside a self-contained mock environment, not evidence that a consequential real-world action occurred. The Privacy Steal task is stronger because the mock read_file_tool returns actual file contents, but the cryptocurrency claim is not supported by the experiments as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an end-to-end empirical study of attacks launched through malicious MCP servers. It defines four attacks (Tool Poisoning, Puppet, Rug Pull, and Exploitation via Malicious External Resources) and evaluates the three stages of the attack lifecycle: uploading a malicious server to three aggregation platforms (RQ1), user detection of malicious servers on a simulated aggregator (RQ2, n=20), and attack success/refusal rates of three vectors against five LLMs and five MCP clients (RQ3). The authors report that all three platforms accepted the malicious server, that 75% of participants selected at least one malicious server, and that average ASR across evaluated LLMs is about 53% with low refusal rates. They conclude that current platform audits, users, LLMs, and clients do not adequately defend against these attacks.","tokens_in":30012,"tokens_out":5209,"duration_ms":51158,"significance":"If the claims hold, this is a valuable first end-to-end empirical evaluation of the MCP attack lifecycle. The paper has clear strengths: an explicit threat model, a multi-method design (platform upload, user study, attack implementation), internal consistency between the Cline rows in Tables 2 and 4, an IRB-approved user study, and a published artifact with source code and data. The affected-path versus exploited-path table (Table 3) is a useful organizing device for future defenses. The RQ3 measurements are empirical ASR/RR numbers rather than parameter fits, so the results are falsifiable. The main gaps are that some headline claims go beyond what the experiments actually demonstrate.","major_comments":[{"comment":"The Cryptocurrency Theft result is not supported as a real-world transfer. The success criterion is that the LLM calls transfer_tool and the transfer-mcp server returns the simulated response '0.01 ETH has been transferred'; no wallet signature, user confirmation, transaction broadcast, or balance change is verified anywhere in RQ3. The abstract's phrase 'controlling devices to transfer digital assets' and the Answer to RQ3 ('All defined attack vectors can be effectively executed in real-world scenarios') therefore overstate what was measured. The experiment demonstrates prompt-following that invokes a transfer tool inside a mock environment, not that digital assets were transferred. I recommend relabeling this task as 'unauthorized transfer-tool invocation,' adding an explicit caveat, or extending the setup with a real signing/wallet step.","section":"§4.3.2, Figure 12, Answer to RQ3"},{"comment":"The data do not support the blanket statement that all defined attack vectors can be effectively executed. The average ASR for Puppet Attack is only 6.67%, with 0% for GPT-4o and LLaMA3.1-70B, and the authors themselves write that 'the attack results using the Puppet Attack are not ideal.' In addition, Rug Pull Attack is not implemented in RQ3; its exploitation phase is argued to share mechanisms with the other attacks, but no local harmful action is measured. The conclusion should be narrowed to the specific vector/task combinations that were tested, and the percentage-point gap between vectors should be reported instead of the 1,299.25% relative increase. With 20 trials per cell, differences of a few percent are within sampling noise, so confidence intervals or a small statistical test would strengthen the claims.","section":"§4.3.3, Table 2, Answer to RQ3"},{"comment":"The conclusion that aggregation platforms have insufficient audit mechanisms rests on a single malicious server design (one tool, one prompt-injection pattern) and one post-approval modification procedure. The single probe supports the narrow claim that this class of prompt-injection server is not detected by the three platforms and that repository modifications after approval are not re-reviewed; it does not by itself support the broader statement that platforms allow attackers to upload malicious servers across all four attack classes. The RQ1 answer should be rephrased to make the tested attack class explicit, or the authors should test additional malicious server designs (e.g., different injection placements or attack intents) before generalizing.","section":"§4.1.3, Answer to RQ1"}],"minor_comments":[{"comment":"The axis tick labels in both figures are rendered as encoded tokens such as '/uni00000037/uni00000033/...', making the figures illegible; the labels need to be regenerated with proper text.","section":"Figures 7 and 9"},{"comment":"The implementation details for Puppet Attack and Malicious External Resources are only in the online repository; the paper should summarize the server/tool structure and injection payloads for these vectors in an appendix so the experiments are self-contained.","section":"§4.3.2"},{"comment":"The formalizations use ad-hoc notation (e.g., 'LLM call --', 'F query -- --') that is never defined precisely and is not used in the analysis; either define the notation rigorously or remove the equations and keep the prose descriptions.","section":"§3.2, Equations (1)-(4)"},{"comment":"The statement that Malicious External Resources 'exceeds Tool Poisoning Attack and Puppet Attack by 57.31% and 1,299.25%' is confusing; percentage-point differences are more informative and less misleading.","section":"§4.3.3, Result 1"},{"comment":"There is a typo: 'Smithery.ai' is written as 'Smith-ery.ai' in one place; please ensure consistent spelling.","section":"§2.2 and §4.1.2"},{"comment":"The claim that stronger tool-utilization capability correlates with higher vulnerability is based on informal comparison with a leaderboard; cite the exact leaderboard version and date, and consider reporting the correlation quantitatively.","section":"§4.3.3, Insight 7"},{"comment":"The external validity discussion acknowledges the technical skew of the user study but does not mention the simulated transfer in RQ3; adding that limitation alongside the existing threats would give readers an accurate picture.","section":"§5.3, Threats to Validity"},{"comment":"The snowball recruitment method can amplify selection bias; the paper reports the demographics table, but a short sentence about how the snowball chain could bias the results would strengthen the external-validity discussion.","section":"§4.2.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a good fit for a security or software-engineering venue and the artifact is a genuine strength. The main revision concern is aligning the headline claims with the evidence: the cryptocurrency 'theft' measurement is a simulated tool invocation, and the 'all attack vectors can be effectively executed' statement is contradicted by the low Puppet Attack ASR and the absence of an RQ3 Rug Pull experiment. These are fixable by rewording and by adding the missing caveats or experiments, so I do not recommend rejection. I would be comfortable with acceptance once the claims are scaled back to what the experiments demonstrate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know about this one: it's the first paper I've seen that actually walks the full malicious-MCP-server lifecycle—upload, user download, attack—rather than just describing hypotheticals. The upload result is the strongest part: they got a server with embedded prompt injection accepted by Smithery, MCP.so, and Glama, with Glama labeling it 'safe to use', and they showed that post-approval repo modifications (the Rug Pull vector) go undetected for at least a week. That's concrete and reproducible, and it alone justifies sending this to review.\n\nThe user study is small (20 participants, mostly technical) but IRB-approved, and the finding that only one person identified all four malicious servers, even when told they exist, is believable. The attack experiments are reasonably broad: 5 LLMs x 5 clients x 3 tasks x 3 vectors, 20 trials each, with code published. The low refusal rates and the fact that stronger tool-using models are more vulnerable is a useful data point.\n\nNow the soft spots. The biggest one, which I think the attached stress-test got right, is the Cryptocurrency Theft task. Their ASR for that task counts the LLM calling a 'transfer_tool' whose mock server replies '0.01 ETH has been transferred'. No wallet signature, no on-chain broadcast, no balance change. So they're really measuring prompt-following, not 'controlling devices to transfer digital assets' as the abstract and conclusion phrase it. In a real MCP server, that tool call would hit a wallet API; the paper's own workflow shows step ⑦ for that. But as reported, the end-to-end financial-execution claim is not supported by the experiment. The Privacy Steal task is stronger because the read_file_tool actually reads a file, though still inside a mock environment.\n\nRelatedly, the claim that 'all four attack vectors can be effectively executed' overstates the evidence: Puppet Attack ASRs are in the low single digits to ~30%, and they never directly execute a Rug Pull attack, only infer it from the RQ1 upload modification. The RQ1 conclusion about insufficient audits also rests on a single probe server design. That's enough to show a real weakness—one counterexample is sufficient for that—but the abstract's sweeping wording goes beyond the data. The paper would be stronger if they scoped these claims and added error bars or raw counts for the 20-trial percentages.\n\nOverall, the empirical core is solid, the writing is clear, and the artifact appears complete. I'd send this to serious referees, with a request that the authors either add a real (testnet or simulated-signed) transfer step or explicitly say they measured tool-invocation compliance, not financial loss. It's a useful foundation for MCP security research.","headline":"First real end-to-end empirical study of malicious MCP servers, but the 'real-world' damage claim leans on a simulated transfer and needs scoping.","tokens_in":30555,"tokens_out":3988,"would_cite":true,"duration_ms":34912,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Malicious MCP servers can be uploaded to widely used registries, installed by unsuspecting users, and used to make LLM agents leak private files or move cryptocurrency, with an average attack success rate of 53% across current mainstream…","keywords":["Model Context Protocol","MCP aggregation platforms","LLM agent security","prompt injection","tool poisoning attack","puppet attack","rug pull attack","malicious MCP servers"],"falsifier":"Re-upload a family of malicious MCP servers covering all four attack vectors, with obfuscated and indirect payloads, to Smithery.ai, MCP.so, and Glama and check whether any are rejected or flagged; if a substantial fraction are blocked, the claim of insufficient auditing is falsified. Separately, re-run the three attack tasks on current versions of the five LLMs and five MCP clients; if refusal rates climb above 80% or attack success rates fall below 20%, the claim that mainstream LLMs and clients cannot defend is falsified.","tokens_in":29549,"feed_emoji":"🤖","tokens_out":11950,"duration_ms":98496,"temperature":0.7,"pith_summary":"The paper argues that the Model Context Protocol (MCP), the emerging standard that lets LLM agents call external tools, creates a trust boundary that malicious servers can cross. It names four attack patterns—Tool Poisoning, Puppet Attack, Rug Pull, and Malicious External Resources—and tests them along the full upload, download, and attack lifecycle. The authors report that a malicious server was accepted by three major MCP aggregators, that 15 of 20 participants picked at least one malicious server, and that the attacks succeeded on average 53% of the time against five mainstream LLMs while refusals stayed below 10%. If true, this means the protocol's convenience has outpaced its security: users are installing untrusted servers that can turn their agents into data exfiltrators or crypto transfer tools. The paper also proposes defenses, including client-side security gates at the registration and prompt-assembly paths and cryptographic signing of tool descriptions.","feed_headline":"MCP server attacks reach 53% success on mainstream AI models","feed_subtitle":"Malicious servers uploaded to three MCP registries slip past audit, then trick LLM agents into leaking files and crypto.","key_machinery":"The load-bearing mechanism is the MCP capability-registration channel: a server declares its tools as natural-language descriptions, and the client pastes those descriptions into the LLM's context while showing the user only a simplified version. Because the protocol does not separate server-controlled text from user instructions, any text entering this channel is trusted by the model. The paper maps all four attack types to specific steps of the nine-step MCP workflow, showing that tool descriptions, server outputs, and third-party fetched content are equally able to carry the injection. This single mechanism explains why auditing server code at upload time is insufficient: the payload lives in metadata the platform never inspects (descriptions, runtime outputs, remote pages), not only in executable code.","core_discovery":"The paper's central claim is that the MCP client-server architecture hands the attacker a clean channel for prompt injection: tool descriptions returned by an MCP server are fed to the LLM as part of the agent's context, while the user only ever sees a simplified version of them. This enables server-originated attacks even when the server's code is benign or the server is trusted at install time. The four attack types the paper defines—Tool Poisoning, Puppet Attack, Rug Pull, and Malicious External Resources—all follow from this design: malicious instructions hidden in tool descriptions, in coordinated server pairs, in post-approval code updates, or in third-party resources that a benign tool fetches. The paper reports empirical evidence for each stage: a malicious server was accepted by Smithery.ai, MCP.so, and Glama without rejection; 15 of 20 participants selected at least one malicious server when building a travel agent; and benchmarked attacks achieved an average attack success rate above 50% across five LLMs, with refusal rates below 10%. The authors conclude that current LLMs and MCP clients lack adequate defense mechanisms against these vectors.","pith_inferences":["The taxonomy likely generalizes beyond MCP: any protocol that injects untrusted tool descriptors or third-party content into an agent's context (OpenAPI-specified function calls, plugin manifests, retrieval results) can host the same four attack patterns; the paper's affected-path analysis gives a template for mapping them.","The paper's 53% average ASR probably understates real-world risk, since the user study found participants often auto-approve permission prompts, and because more users will install 'popular' servers that mimic trusted functionality.","Because aggregators are directory services pointing at external code repositories, an effective and cheap defense would be continuous diff monitoring of registered repositories, re-running audits on every update, rather than a single review at submission.","The user study's participant pool is skewed toward developers and researchers, so the claim that users struggle to identify malicious servers is likely conservative for non-expert users; a larger representative study would quantify the gap."],"forward_implications":["Aggregator platforms should stop being treated as code review checkpoints; the paper's upload test shows that one-time submission checks miss prompt-injection payloads and that post-approval repository changes are never re-audited.","Users who install several MCP servers are exposed to Puppet Attacks even if every individual tool looks benign, because one server's description can steer how the LLM uses another server's tools.","Run-and-clean package managers such as npx and uvx amplify Rug Pull attacks, since the server code is freshly fetched at each run and can be swapped after platform approval.","LLMs with stronger tool-use abilities show higher attack success rates, so improving an agent's tool-calling skill without adding security awareness may increase exposure to these four vectors.","Client-side security gates placed on the registration and prompt-assembly steps, plus cryptographic signing of tool descriptions and code, would interrupt most of the four attack patterns before the LLM sees them."],"supporting_citations":[{"why":"Defines MCP's client-server architecture and the tool-description registration channel that the attacks inject into.","marker":"[5]"},{"why":"Establishes prompt injection as a core LLM vulnerability and provides the base attack mechanism for malicious tool descriptions.","marker":"[38]"},{"why":"Supplies the benchmark framing and attack success/refusal metrics used to evaluate the four attacks against LLMs.","marker":"[58]"},{"why":"Target platform where the malicious server was uploaded and accepted in the RQ1 audit test.","marker":"[48]"},{"why":"Second aggregator platform that accepted the malicious server without rejection.","marker":"[40]"},{"why":"Third aggregator platform, which labeled the malicious server 'safe to use'.","marker":"[25]"},{"why":"Provides the realistic cryptocurrency wallet tool used for the Cryptocurrency Theft attack task.","marker":"[7]"}],"fun_headline_variants":["MCP servers: hidden tool descriptions enable AI agent hijacks","Malicious MCP servers evade audits, leak files, steal crypto","Tool poisoning attacks via MCP hit 53% success on five LLMs","MCP architecture expands attack surface: four attack vectors","Study: users unknowingly install malicious MCP servers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that MCP aggregator platforms cannot catch malicious servers rests on a single probe: one server design with one embedded malicious prompt, uploaded to each of three platforms, and if those platforms would detect other classes of malicious servers, the conclusion would overstate how widespread the audit failure is.","fun_headline_variants_meta":{"raw":{"variants":["MCP servers: hidden tool descriptions enable AI agent hijacks","Malicious MCP servers evade audits, leak files, steal crypto","Tool poisoning attacks via MCP hit 53% success on five LLMs","MCP architecture expands attack surface: four attack vectors","Study: users unknowingly install malicious MCP servers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000696,"raw_usage":{"total_tokens":3214,"prompt_tokens":1079,"completion_tokens":2135,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":2049}},"tokens_in":695,"tokens_out":2135,"duration_ms":15693,"temperature":1.0,"reasoning_tokens":2049,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:04:33.343894+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-upload a family of malicious MCP servers covering all four attack vectors, with obfuscated and indirect payloads, to Smithery.ai, MCP.so, and Glama and check whether any are rejected or flagged; if a substantial fraction are blocked, the claim of insufficient auditing is falsified. Separately, re-run the three attack tasks on current versions of the five LLMs and five MCP clients; if refusal rates climb above 80% or attack success rates fall below 20%, the claim that mainstream LLMs and clients cannot defend is falsified.","supporting_citations":[{"cited_title":"Model Context Protocol, 2025","cited_arxiv_id":null,"evidence_quote":"Defines MCP's client-server architecture and the tool-description registration channel that the attacks inject into."},{"cited_title":"Agent Security Bench (ASB): For- malizing and Benchmarking Attacks and Defenses in LLM-based Agents","cited_arxiv_id":null,"evidence_quote":"Supplies the benchmark framing and attack success/refusal metrics used to evaluate the four attacks against LLMs."},{"cited_title":"Smithery - Model Context Protocol Reg- istry, 2025.https://smithery.ai/","cited_arxiv_id":null,"evidence_quote":"Target platform where the malicious server was uploaded and accepted in the RQ1 audit test."},{"cited_title":"MCP Servers.https://mcp.so","cited_arxiv_id":null,"evidence_quote":"Second aggregator platform that accepted the malicious server without rejection."},{"cited_title":"Mcp servers glama, 2025","cited_arxiv_id":null,"evidence_quote":"Third aggregator platform, which labeled the malicious server 'safe to use'."},{"cited_title":"Base-mcp, 2025","cited_arxiv_id":null,"evidence_quote":"Provides the realistic cryptocurrency wallet tool used for the Cryptocurrency Theft attack task."}],"review_version":1}