{"id":"3d1320a3-2622-4e4b-b86e-14699d2961e9","arxiv_id":"2507.06250","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"A static scan of 2,562 MCP plugins shows heavy use of network and system APIs, but the paper's own tables contradict the reported numbers.","lead":"This paper counts which system, network, file, and memory functions are called inside 2,562 MCP plugins. It argues that privileged APIs are widespread and that MCP needs permission controls.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Affected-server counts rest on an unvalidated equivalence between any occurrence of generic APIs (open, subprocess.run, socket.connect) and a security threat; no exploitability or data-flow evidence is provided, so the prevalence figures and RQ3 conclusion may be inflated.","rationale":"The reader's weakest assumption is the same one I find most load-bearing: the paper equates the presence of a generic API call with a security threat. Everything downstream—Figure 2's affected-server counts, Table II/III comparisons, the RQ3 popularity conclusion, and the Section VII case studies—uses the output of that unvalidated classifier. The paper itself hedges by calling Table I entries 'Potential Security Risks,' but the evaluation and abstract convert these into 'threats' and 'affected servers.' That conflation is not a dispute with community consensus; it is an internal validity gap. I give credit for the timely topic, the four-part taxonomy, and the useful framing of least-privilege challenges, but the headline numbers cannot be interpreted without a precision/recall estimate on labeled data. I do not rest on the reader's separate table-consistency objection: Table III's column sums do match Figure 2 exactly, and Table II's totals can be explained if one call can match multiple resource categories, although the paper never states that counting semantics. The concrete validation test above would settle whether the classifier concern lands; if the analyzer turns out to be precise, the measurement claim would be substantially rehabilitated, but as submitted the evidence does not support the security-risk interpretation.","tokens_in":8619,"tokens_out":7442,"duration_ms":79362,"concrete_test":"Release the analyzer, signature database, dataset, and a random stratified sample of 200 servers flagged as system/network threats. Independently label each detection as (a) benign idiomatic use with no attacker-controlled input or security-boundary crossing, (b) potentially exploitable (untrusted input reaches the sink with demonstrable impact), or (c) malicious behavior. Compute the precision of the analyzer's threat flag against this ground truth; then recompute Figure 2 and the RQ3 trend counting only labels (b) and (c). If the recomputed distribution no longer shows network/system dominance, or if low-star plugins are no longer disproportionate after normalizing per server, the paper's central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's quantitative claims—1,438/1,237 servers 'affected' by network/system threats, and 'less popular plugins contain disproportionately high-risk operations' (RQ3)—depend on the static analyzer's classification step (Section V-B). The curated signature database matches API names such as open(), read(), write(), subprocess.run(), socket.connect(), fork(), malloc(), and ctypes.CDLL() and labels them 'Potential Security Risks' (Table I). The analysis does not check whether a matched call is reachable, whether any argument is attacker-controlled, whether a security boundary is crossed, or whether the call is the normal implementation of the plugin's advertised function. A git-based blog plugin calling subprocess.run('git ...') and shutil.copy() is counted as a 'Privilege Escalation Risk'; a web client calling socket.connect() or requests.get() is counted as a network threat. Under this equivalence, a benign file manager and a malicious exfiltration plugin are indistinguishable. Figure 2 relabels these counts as 'threat types'/'affected servers' rather than 'API usage occurrences.' Because the paper releases no analyzer implementation, signature list, dataset, or labeled validation set, no reader can assess false-positive/false-negative rates. The case studies in Section VII compound this: they describe capabilities (e.g., 'can log user queries') as demonstrated attacks without showing an actual attacker or user impact. Thus the central empirical claim—that the MCP ecosystem has a broad and unevenly distributed attack surface—is currently an artifact of name-based API counting, not a demonstrated security finding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a static analysis framework for Model Context Protocol (MCP) servers, scans 2,562 applications collected from MCP Market, and categorizes detected API calls into file, memory, network, and system resources. It reports affected-server counts and API-call distributions by application category and GitHub popularity, and presents three case studies (blog-publisher, twitter-mcp, web-research) to argue for urgent privilege management in MCP.","tokens_in":8905,"tokens_out":7497,"duration_ms":76734,"significance":"If the measurements were valid, the paper would provide a useful first large-scale attack-surface estimate for the MCP ecosystem, and the proposed taxonomy is a reasonable organizing structure. The multi-language static analysis pipeline and the 2,562-server corpus are valuable assets. However, the current manuscript does not supply enough validation or internal consistency for the quantitative claims to be relied upon, and no artifact or dataset is released.","major_comments":[{"comment":"Table III is presented as the API call distribution by GitHub star range, but its resource columns sum exactly to the affected-server counts in Figure 2: network 1,079+240+77+32+9+1=1,438, system 834+238+112+43+9+1=1,237, file 367+122+86+29+8+1=613, and memory 10+7+4+2+2+0=25. The row totals (e.g., 1,837 for the 0–10 range) do not equal the sums of the columns in those rows (1,079+834+367+10=2,290). The table therefore mixes server counts and API call counts, and the RQ3 conclusion that low-star plugins contain disproportionately more high-risk API calls is not supported by the data as printed.","section":"Section VI-B, Table III"},{"comment":"Table II has the same category confusion and, in addition, every row total is inconsistent with its own column entries. For example, the API Development row reports a total of 511, but 77+4+313+195=589; the Developer Tools row reports 626, but 156+7+325+336=824. The sums of the resource columns (637, 29, 1,499, 1,317) do not match the sum of the Total column (2,617). Since RQ2's claims about Developer Tools and API Development being the most API-intensive derive from these numbers, the table must be recomputed or the claims must be withdrawn.","section":"Section VI-B, Table II"},{"comment":"The classification step that maps any occurrence of generic APIs such as open(), read(), write(), subprocess.run(), socket.connect(), and ctypes.CDLL() to a 'Potential Security Risk' is not validated. The analysis does not check reachability, data flow, or whether the call is the normal implementation of a benign feature. For example, a git-based publishing plugin using subprocess.run('git ...') is counted as a privilege escalation risk even if no input is attacker-controlled. The paper provides no labeled validation set, no false-positive/false-negative analysis, and no artifact release, so the headline affected-server counts (1,438 and 1,237) are not established as security threats.","section":"Section V-B and Table I"},{"comment":"The three case studies are described as 'demonstrations' but are actually static capability observations. The blog-publisher case asserts 'unsanitized input handling' without showing a concrete injection path; twitter-mcp is said to allow 'silent manipulation' and metadata embedding; web-research is said to let attackers 'log all user queries' without evidence that such logging occurs or that any data crosses a security boundary. These narratives do not meet the standard of demonstrated attacks and should be relabeled as potential risks or supported with end-to-end exploitation.","section":"Section VII"},{"comment":"The pipeline description repeats the label '(P-III)' for two different phases, and the implementation section states that a curated API signature database is embedded without listing its contents or version. This, together with the absence of a dataset or code release, makes the experiment impossible to reproduce or audit. The authors should provide the signature list, the server corpus, and the detection results, or clearly state their availability.","section":"Section V-A and V-B"}],"minor_comments":[{"comment":"The two phases labeled '(P-III)' should be renumbered; the current numbering obscures the pipeline.","section":"Section V-A"},{"comment":"The axis label says 'Threat Type' while the text calls it 'affected servers'; define what qualifies a server as 'affected'.","section":"Figure 2"},{"comment":"The captions should state whether the numbers are API call occurrences or affected servers; the current text alternates between the two.","section":"Tables II and III"},{"comment":"The URL 'mcpmarket.com' is given without an access date, version, or description of the crawl window; add this for reproducibility.","section":"Section VI-A"},{"comment":"References [13]–[16] are preprint or workshop items; consider adding peer-reviewed related work on LLM tool security for context.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The table inconsistencies are severe enough that the current numerical results cannot be used, and the threat-classification step needs validation. I chose major revision rather than reject because corrected tables, a manual validation sample, and a clear separation of observed API usage from demonstrated exploits could make the measurement credible. If the authors cannot supply corrected data or validation, the paper should not be published."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nI read the MCP measurement paper. The new thing here is the dataset: 2,562 MCP plugins scanned statically, with a taxonomy of file, memory, network, and system resources. That is genuinely useful as a first descriptive snapshot of what APIs MCP servers actually call. The observation that network/system APIs dominate is credible as a statement about code content, and the motivation—MCP servers run with high privileges and little isolation—is sound.\n\nThat said, the paper overreaches when it turns API usage into security findings. The 'curated API signature database' matches generic calls like open(), subprocess.run(), socket.connect() and then labels them 'Potential Security Risks.' No reachability check, no taint analysis, no argument control, no security boundary crossing. Under that equivalence, a file manager and an exfiltration bot look identical. So the headline numbers—1,438 and 1,237 'affected servers'—are not supported as threat counts. They are counts of code that mentions certain function names. The paper even acknowledges the signature database is embedded in the engine, but no analyzer, signatures, or validation set are released, so nobody can audit false positives.\n\nThe quantitative presentation also has internal problems. Table III's row sums exactly match the affected-server counts from Figure 2, meaning what is labeled 'API call distribution' is actually about servers, not calls. Table II's row totals do not match the sum of their own columns. These aren't minor formatting issues; they make the RQ2/RQ3 conclusions untrustworthy even as descriptive statistics.\n\nThe case studies in Section VII are speculative. They describe what a plugin can do (run git, post tweets, load .env) and then assert attack scenarios without demonstrating an actual exploit, a malicious input, or user impact. That is a capabilities inventory, not a security analysis.\n\nWhat would make this work: release the code and signature list, correct the tables, and re-frame the claims as 'we measured API usage, and here is why it is a potential concern.' The underlying dataset is worth having; the conclusions as written are not.\n\nI'd still send it to peer review—the topic is timely and the measurement is novel—but it needs major revision before it can be trusted. If you work on MCP or agent security, worth a skim; don't cite the numbers as they stand.","headline":"Useful first measurement of MCP API usage, but the threat interpretation is unvalidated and the tables have internal inconsistencies; worth a revised look.","tokens_in":9389,"tokens_out":2526,"would_cite":false,"duration_ms":25785,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A static analysis of 2,562 MCP servers shows network and system APIs dominate privileged operations, affecting 1,438 and 1,237 servers respectively.","keywords":["Model Context Protocol","MCP security","API usage measurement","static analysis","privilege management","LLM tool integration","plugin security","permission models"],"falsifier":"A re-analysis of the same 2,562 repositories that performs taint or data-flow tracing on the flagged calls, checking for example whether the argument to subprocess.run in the blog-publisher case can be influenced by a user or LLM prompt, would settle the claim: if no flagged call has a tainted path to a sensitive sink, the reported counts describe available capability rather than demonstrated exploitability.","tokens_in":8458,"feed_emoji":"🛡️","tokens_out":9209,"duration_ms":93885,"temperature":0.7,"pith_summary":"This paper claims that the Model Context Protocol (MCP) ecosystem runs on an implicit-trust model: MCP servers execute natively with access to file, system, network, and memory APIs, with little isolation or user oversight. To support this, the authors built a static analysis pipeline and scanned 2,562 real-world MCP servers across 23 categories, finding that network resource APIs affect 1,438 servers and system resource APIs affect 1,237. File (613) and memory (25) risks are less frequent, while Developer Tools and API Development plugins are the most API-intensive. The paper also reports that low-popularity plugins (0-10 stars) account for 1,837 flagged API calls, far outpacing mature projects, which it reads as evidence that the riskiest code concentrates where oversight is lowest. If these measurements hold, privilege management becomes a central design requirement for MCP rather than an optional hardening step.","feed_headline":"AI tool plugins lean on system and network APIs unchecked","feed_subtitle":"A 2,562-server scan shows low-star plugins carry the most high-risk calls, making privilege management urgent.","key_machinery":"The load-bearing mechanism is a static-analysis framework that runs in three phases: code collection and normalization, multi-dimensional API analysis, and result aggregation. The analysis phase uses language-specific AST parsers with regex fallback and a curated API signature database that classifies calls into four resource categories: file, memory, network, and system. Each category is paired with potential risks, such as mapping subprocess.call to command injection or socket.bind to open high-risk ports, and the taxonomy is what converts raw API calls into the threat counts that drive the paper's conclusions.","core_discovery":"The paper's central discovery is an empirical characterization: in a corpus of 2,562 MCP servers collected from a public MCP aggregation platform, the majority invoke privileged APIs before any permission mechanism constrains them. Network APIs appear in 1,438 servers and system APIs in 1,237, the categories most capable of remote code execution, unauthorized communication, and data exfiltration, while file threats appear in 613 servers and memory threats in 25. The measurement also finds that plugins with 0-10 stars produce 1,837 API calls in total, while the two highest star bands together account for only 10 calls, and that Developer Tools and API Development categories exceed 500 calls each. Three case studies, a blog-publisher server, a social-media integration, and a web-research server, are presented as concrete paths to privilege escalation, misinformation, and data tampering. The paper positions these results as evidence that MCP needs privilege management rather than a protocol-level redesign.","pith_inferences":["The counts of 1,438 and 1,237 measure API presence, not reachable exploitability; a taint analysis would likely lower the numbers, but the remaining default capability surface still justifies permission engineering.","A natural extension is to correlate flagged API calls with whether a plugin declares any permission, sandbox, or capability configuration; the paper does not report such a correlation.","The density of high-risk calls in low-star plugins suggests that star count is a weak safety proxy and that automated trust scoring could replace it as a deployment gate if dynamic permission models are developed."],"forward_implications":["MCP clients should by default deny or individually prompt for network and system resource calls, since those categories cover the largest share of servers.","Pre-deployment static checks that flag file, network, and system APIs could serve as a low-cost trust signal for plugins with little community review.","Privilege management in MCP will need to be dynamic and intent-aware, because static permission declarations conflict with the general-purpose role MCP servers play.","Platform-specific isolation mechanisms such as Windows UAC, macOS sandboxing, and Linux namespaces are the concrete building blocks for enforcing least privilege."],"supporting_citations":[{"why":"Defines MCP as the object of study and grounds the claim that MCP standardizes tool discovery and invocation.","marker":"[1]"},{"why":"Supplies the protocol specification behind the host-client-server architecture that motivates the resource taxonomy.","marker":"[2]"},{"why":"Provides the Android permission model used as the contrast showing MCP lacks runtime permission checks and isolation.","marker":"[7]"},{"why":"A survey that identifies privilege escalation as an MCP challenge, giving this paper a baseline to extend with measurements.","marker":"[13]"},{"why":"Lists lifecycle and security threats such as sandbox escapes and configuration drift that the paper's static analysis complements.","marker":"[14]"},{"why":"Describes a reactive dynamic defense approach that the paper argues cannot catch pre-execution risks.","marker":"[15]"},{"why":"Extends the reactive defense with behavioral analysis, framing the contrast with static pre-deployment analysis.","marker":"[16]"}],"fun_headline_variants":["MCP plugins misuse system and network APIs in 2,562-server study","Low-star MCP plugins carry most high-risk API calls","Privilege gaps in MCP exposed by 2,562-server analysis","Unchecked API access in MCP plugins: 1,438 network calls","MCP needs privilege management: 2,562 plugins show risk"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that calling an API such as subprocess.run, open, or socket.connect is itself a security threat, since the paper never shows an attacker controlling the arguments or a sensitive resource being reached, and all counts and case studies depend on that classification.","fun_headline_variants_meta":{"raw":{"variants":["MCP plugins misuse system and network APIs in 2,562-server study","Low-star MCP plugins carry most high-risk API calls","Privilege gaps in MCP exposed by 2,562-server analysis","Unchecked API access in MCP plugins: 1,438 network calls","MCP needs privilege management: 2,562 plugins show risk"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000638,"raw_usage":{"total_tokens":2942,"prompt_tokens":953,"completion_tokens":1989,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":1893}},"tokens_in":569,"tokens_out":1989,"duration_ms":15198,"temperature":1.0,"reasoning_tokens":1893,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:59:55.551950+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A re-analysis of the same 2,562 repositories that performs taint or data-flow tracing on the flagged calls, checking for example whether the argument to subprocess.run in the blog-publisher case can be influenced by a user or LLM prompt, would settle the claim: if no flagged call has a tainted path to a sensitive sink, the reported counts describe available capability rather than demonstrated exploitability.","supporting_citations":[{"cited_title":"Introducing the model context protocol,","cited_arxiv_id":null,"evidence_quote":"Defines MCP as the object of study and grounds the claim that MCP standardizes tool discovery and invocation."},{"cited_title":"Model context protocol specification (latest) - protocol revision: 2024- 11-05,","cited_arxiv_id":null,"evidence_quote":"Supplies the protocol specification behind the host-client-server architecture that motivates the resource taxonomy."},{"cited_title":"Android per- missions demystified,","cited_arxiv_id":null,"evidence_quote":"Provides the Android permission model used as the contrast showing MCP lacks runtime permission checks and isolation."},{"cited_title":"A survey on model context protocol: Architecture, state-of- the-art, challenges and future directions,","cited_arxiv_id":null,"evidence_quote":"A survey that identifies privilege escalation as an MCP challenge, giving this paper a baseline to extend with measurements."},{"cited_title":"Model context protocol (mcp): Landscape, security threats, and future research directions,","cited_arxiv_id":null,"evidence_quote":"Lists lifecycle and security threats such as sandbox escapes and configuration drift that the paper's static analysis complements."}],"review_version":1}