REVIEW 4 major objections 5 minor 21 references
Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale
T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read The paper reports the first dynamic security audit of internet-facing Model Context Protocol (MCP) servers, finding 68 exploitable vulnerabilities across 414 audited servers, with 91.8% lacking OAuth authentication.
desk verdict First dynamic read of the MCP surface, genuinely novel, but the headlined vulnerability counts rest on unvalidated heuristics and need softening before the numbers can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is Corvus, an open-source dynamic testing framework contributed by the paper, comprising 34 test modules (13 static, 21 dynamic) mapped to the ten MST-10 vulnerability categories. Corvus confirms vulnerabilities through heuristic behavioral signals rather than source-code inspection: response-size differentials for SQL injection, timing oracles (e.g., >5-second latency delta) for SSRF against cloud metadata endpoints, schema inspection for shell-execution tool exposure, and lexical pattern matching for prompt injection. A companion discovery pipeline (Petrel) aggregates eleven passive sources and performs HTTP-level MCP handshakes to confirm live servers. Together
What would settle it
A labeled benchmark corpus of MCP servers with known ground-truth vulnerability status, run through Corvus, would measure module-level precision and recall. Specifically, if the SSRF timing threshold (>5s) yields many positives on servers known to lack SSRF, or if the SQL response-size differential misclassifies benign large responses, the vulnerability counts are overstated. Independent manual review of the source code of the 68 reported servers would also confirm or refute exploitability.
Extended reading notes
Core claim
The paper's central claim is that the MCP deployment surface is (i) large and rapidly expanding, (ii) predominantly functionally unauthenticated, (iii) host to exploitable vulnerability classes spanning all ten MCP Security Top 10 (MST-10) categories, and (iv) governed by deployment patterns that complicate defensive monitoring and measurement. This is supported by what the author calls the first dynamic behavioral security assessment of internet-facing MCP servers: 414 servers actively audited, 68 reportable vulnerabilities, 91.8% OAuth absence, 687 unauthenticated shell-execution tool instances, and a 41.6% three-day churn rate between consecutive runs.
Load-bearing premise
The validity of every prevalence and severity statistic depends on an unvalidated mapping from heuristic signals to vulnerability classes—a >5-second response-time delta is treated as SSRF, a larger response to a SQL payload as SQL injection, and exec-like tool names as unauthenticated shell execution—and the paper reports that module-level precision/recall is ongoing work, so high false-positive rates in these proxies would inflate the 68-vulnerability total and the authenti
Editorial extensions
If this is right
- If the reported rates are accurate, the MCP ecosystem currently offers a wide, unauthenticated attack surface for anyone who can reach the public internet, including remote code execution via shell-capable tools.
- The protocol and its SDKs need authentication (OAuth 2.1) as a default for HTTP transport, not an opt-in, otherwise the structural gap will persist as adoption grows.
- The high churn means vulnerability scanners and responsible-disclosure workflows must treat server identity as unstable; a disappearing endpoint cannot be assumed patched.
- Dynamic behavioral testing of live servers is essential for MCP security assessment; static analysis of registries or source code would miss runtime failures such as SSRF and schema bypass.
- The 68 coordinated advisories provide the first concrete baseline of MCP-specific vulnerabilities for maintainers and for future regression measurement.
Reading between the lines
- If the heuristic thresholds (e.g., the 5-second SSRF timing delta or the SQL response-size differential) have high false-positive rates, the 68-vulnerability total and per-category counts may overstate actual risk; a labeled benchmark evaluation of Corvus would settle this. This extension is not in the paper but follows directly from its acknowledged calibration gap.
- The 687 shell-execution tool count is based on schema-level names and descriptions; the actual number of truly exploitable, network-reachable shell invocations may be lower once per-tool argument validation and sandboxing are considered. This is an editorial caution, not a paper claim.
- Because the most persistent servers also carried the highest CRITICAL counts, the churn statistics may understate long-lived risk: an attacker can focus on stable, high-risk endpoints rather than ephemeral ones. The paper hints at this persistence pattern but does not develop a risk-weighted longitudinal model.
- If MCP's steward adds mandatory OAuth by default, this measurement can serve as a before/after baseline; a replicable version of the current scan after such a change would directly quantify policy impact. The paper does not propose this experiment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper reports the first dynamic behavioral security assessment of internet-facing Model Context Protocol (MCP) servers. The authors combine passive discovery across eleven sources (certificate-transparency logs, package registries, code hosts, Internet scanners, and MCP directories) with active testing by an open-source 34-module framework, 'Corvus,' across four measurement runs in July 2026. They confirm 640 unique production MCP servers, dynamically audit 414, and report: 91.8% (380/414) lacking OAuth; 687 tool instances across the confirmed pool advertising unauthenticated shell-execution capabilities (identified via tool name/schema inspection); 68 reportable vulnerabilities filed as GitHub Security Advisories (19 public, 49 embargoed), including SQL injection, SSRF against cloud metadata services, prompt template injection, and cursor-based path traversal; and 41.6% churn of confirmed servers between the July 21 and July 24 runs. The paper introduces the MST-10 taxonomy of MCP-specific vulnerability classes, characterizes deployment churn and the near-total authentication gap, documents a responsible-disclosure pipeline, and releases its tooling. The central conclusion is that the MCP deployment surface is large, rapidly changing, largely unauthenticated, and vulnerable across the introduced taxonomy.
Significance. If the headline measurements hold, this is a substantial contribution: it is the first behavioral, deployment-level security measurement of the MCP ecosystem, with reproducible open-source tooling (Corvus, Petrel), a multi-source discovery methodology, SARIF output, internally consistent arithmetic (380/414 = 91.8%; 193/464 = 41.6%; 972 observations minus 332 duplicates = 640 unique), and a carefully documented 90-day disclosure pipeline with 19 public advisories. The near-universal OAuth absence and the extreme churn are directly measured, credible results that stand on their own as an empirical baseline. The paper is also commendably transparent: it concedes the unreported module-level precision/recall, the heuristic nature of confidence thresholds, the single-researcher confirmation-bias risk, and the temporal validity limits. The weakness is that the quantitative spine of the paper — the 68-GHSA total, the per-category prevalence in Table V, and the 687 shell-execution figure — rests on heuristic detectors whose false-positive rates are unmeasured and, in the SSRF case, calibrated on the very observation used as confirmation. These are fixable within scope, but as written the s
major comments (4)
- [§VIII; §IV.C.4; Table V] The headline quantitative claims — 68 reportable GHSAs, per-category prevalence in Table V, severity distribution — inherit the accuracy of Corvus's heuristic detectors (timing deltas, response-size differentials, schema-name inspection). Section VIII explicitly concedes that module-level precision/recall against a labeled benchmark 'is ongoing work and is not reported here,' and §IV.C.4 calls the confidence thresholds 'empirically calibrated heuristics.' Manual triage (§IV.D) mitigates, but manual confirmation uses the same behavioral signals. Table V's finding-server pairs sum to 655 while only 68 advisories were filed; the mapping from module findings to GHSA filings is never stated, so the reader cannot tell which rows are raw SARIF outputs versus post-triage confirmed findings. I request out-of-sample calibration on the 19 disclosed advisories plus confirmed negatives, and an explic
- [§IV.C.3.d; §V.D, Case Study 2] The SSRF threshold is calibrated in-sample. §IV.C.3.d defines SSRF confirmation as a response-time delta exceeding 5 s and states that 'this threshold was calibrated empirically; we observed a 11.9 second response versus a 0.3 second baseline in one confirmed SSRF case,' and that same case is then presented in Case Study 2 as the confirmation of SSRF in epwforge. The threshold therefore has no out-of-sample validation, and all 29 SSRF findings in Table V inherit it. The epwforge case itself is plausible, but confirmation is a pure timing oracle — §VII states no IMDS content was retrieved — so a slow network path or rate limiting cannot be excluded. Fixing the threshold on a held-out set of known-SSRF and known-clean servers before evaluation, or reporting sensitivity of the 29-finding count to the threshold, would establish this detector.
- [§V.E; Table V; §IV.A] The central claim in §V.E that findings span 'all MCP Security Top 10 (MST-10) categories' is internally inconsistent with Table V, which has no MCP02 row, and with §IV.A/Table V, which state MCP09 (Denial of Service) was excluded from execution per ethical constraints. At most eight of ten categories have reported findings. The gap is compounded by the taxonomy definition: MCP02 is 'unauthenticated tools/call invocation reaching privileged operations,' which is exactly what the 687 unauthenticated shell-execution tools (§V.B) represent; zero MCP02 findings alongside 380 MCP08 findings suggests overlapping category boundaries rather than a genuine absence. Please either report MCP02 findings or reclassify the shell-execution exposure, and amend the claim to 'eight of ten categories.'
- [§V.B; §VII] The 687 tool-instance figure is static inference, not behavioral confirmation. §V.B identifies shell-execution capabilities from tool 'names and descriptions matching bash_execute, run_command, exec,' and §VII states capability existence was confirmed 'by observing that tool schemas exposed parameters passed to shell invocation functions,' with no commands beyond benign echo-type probes submitted. The assertion that an adversary 'can issue arbitrary operating system commands through the standard MCP tools/call interface' extrapolates from schema inspection to guaranteed RCE; false positives from name-matching (e.g., a safely parameterized subprocess wrapper or a logging tool named 'executeQuery') directly inflate the 687 count. Also, the count draws on the 640-server confirmed pool while the 91.8% OAuth figure is over the 414 audited servers; the abstract's juxtaposition conflates denomi
minor comments (5)
- [§I (C4); §V.E; Fig. 3] Contribution C4 states the 9 absent CRITICAL-rated servers were 'remediated or decommissioned,' but §V.E says 'endpoint disappearance alone cannot confirm remediation' and Figure 3 labels them 'patched / removed' as fact. The measurement supports only 'absent'; the remediation attribution needs a source (e.g., GHSA maintainer responses), which is not provided.
- [§IV.A; Table V caption; §VII] The MCP09 exclusion is stated in §IV.A, but Table V's caption refers to '§VII' for the exclusion; §VII contains no explicit MCP09-exclusion statement (it says no DoS modules were executed). Align the cross-reference.
- [§IV.C.4] The confidence-score bands (85–100, 71–84, 40–70, 1–39) are given only in prose; a small table or an explicit mapping from signal type to score would aid replication. Also clarify why lexical-pattern-only findings (1–39) never appear in SARIF while schema-based shell-execution inference is treated as a confirmed capability (§V.B).
- [§III.D] MST-10 is described as 'derived from iterative analysis of findings across our measurement campaign' and then applied to classify those same findings. This is a mild circularity worth acknowledging explicitly in §VIII: the prevalence numbers are not independent of how the taxonomy's category boundaries were drawn.
- [Throughout] Typos: 'anamplifier' (§I); 'OW ASP' for OWASP (§III.D and references [13], [14]); 'Y . Zhou' and 'Y . Zhao' (references [19], [4]).
Circularity Check
SSRF confirmation threshold is calibrated on the very case later reported as confirmed, and the MST-10 taxonomy is derived from the same findings it is used to count; the core OAuth/churn measurements remain independent.
-
fitted input called prediction
[Section IV.C.3.d (SSRF threshold calibration) and Section V.D Case Study 2 (epwforge SSRF confirmation)]
"SSRF confirmation requires a response time delta exceeding 5 seconds relative to a same-tool baseline call with a non-routable destination, or a response body containing cloud metadata patterns (e.g., ami-id, instance-id). This threshold was calibrated empirically; we observed a 11.9 second response versus a 0.3 second baseline in one confirmed SSRF case (Section V-B)."
The 5-second threshold was calibrated so that the epwforge observation (11.9s vs 0.3s) counts as SSRF; the same observation is then reported in Section V.D as 'a timing oracle confirming that the server's network context can reach the IMDS.' Setting a detection rule on a known positive and then citing that positive as confirmation is in-sample fitting, not independent validation. The paper also concedes in Section VIII that module-level precision/recall 'is ongoing work and is not reported here,' so the heuristic has no external ground-truth calibration. This does not invalidate the raw timing observation, but it makes the claimed confirmation partially self-referential.
-
self definitional
[Section III.D (MST-10 definition) and Section V.E (central claim)]
"We introduce the MCP Security Top 10 (MST-10), a taxonomy of ten vulnerability classes derived from iterative analysis of findings across our measurement campaign. ... Taken together, these results establish that the MCP deployment surface is ... (iii) host to exploitable vulnerability classes spanning all MCP Security Top 10 (MST-10) categories."
The MST-10 categories were constructed from the same findings that Table V then enumerates by category, so the claim that findings 'span all MCP Security Top 10 categories' is true by construction rather than an independent empirical result. Moreover, Table V actually contains no MCP02 or MCP09 rows, and MCP09 was explicitly excluded per ethical constraints (§VII), so even the construction-based claim is not met by the table; the apparent coverage is an artifact of the taxonomy having been built around observed findings. The raw per-server measurements remain non-circular, but the taxonomic coverage claim does not add independent support.
full rationale
The paper's strongest independent measurements — 91.8% OAuth absence, 41.6% churn, and the raw discovery counts — are direct observational results and are not derived from the MST-10 taxonomy or from fitted parameters. Those parts are self-contained and would support a low circularity score. However, two load-bearing steps do partially reduce to their own inputs. First, the SSRF confirmation threshold is an 'empirically calibrated' heuristic set using the epwforge 11.9s/0.3s observation, which is later presented as a confirmed SSRF case; this is a local in-sample calibration rather than an independent prediction. Second, the MST-10 taxonomy is explicitly 'derived from iterative analysis of findings across our measurement campaign' and is then used to claim that findings span all MST-10 categories and to report per-category prevalence; that part is self-definitional. The paper openly acknowledges the detector calibration gap in Section VIII ('module-level precision and recall characterization against a labeled benchmark corpus is ongoing work and is not reported here'), which reduces rather than aggravates the circularity concern, but it also means the 68-GHSA total and category counts should be treated as heuristic estimates, not independently validated predictions. No load-bearing self-citation chain is present: references to the author's own tools are to open-source artifacts described in the paper, and the external benchmarks (Censys telemetry, prior static studies) are independent. Overall, this is partial, localized circularity rather than a derivation that is equivalent to its inputs.
Assumptions & free parameters
free parameters (2)
- SSRF timing threshold =
>5 s response-time delta
- Confidence thresholds =
>=40 reportable; >=70 manual triage
assumptions (3)
- domain assumption An HTTP 200 JSON-RPC initialize response with protocolVersion and capabilities identifies a real MCP server.
- domain assumption Behavioral signals (timing deltas, response-size differentials, tool-schema names) correspond to the stated vulnerability classes.
- domain assumption The 414 dynamically audited servers can stand in for the broader MCP population.
invented entities (1)
-
MST-10 vulnerability taxonomy
Cite this review
Pith. "Pith review of Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale." pith.science (2026). https://pith.science/paper/KYXS6NW3
@misc{pith2026260800150,
author = {Pith},
title = {Pith review of: Exposed by Design: A Dynamic Security Assessment of Internet-Facing MCP Servers at Scale},
year = {2026},
howpublished = {\url{https://pith.science/paper/KYXS6NW3}},
note = {Machine review of arXiv:2608.00150}
}
read the original abstract
The Model Context Protocol (MCP) has seen rapid adoption since its November 2024 launch, with over 21,000 server instances detectable on the public internet. We present the first dynamic behavioral security assessment of internet-facing MCP servers, combining passive discovery across eleven data sources (crt.sh, HuggingFace, GitHub, npm, Smithery, PyPI, Censys, FOFA, Shodan, glama.ai, and pulsemcp.com) with active dynamic testing using Corvus, a purpose-built framework implementing 34 test modules covering 10 MCP-specific vulnerability classes. Across four measurement runs spanning July 2026, we confirm 640 production MCP servers and dynamically audit 414, uncovering 68 reportable vulnerabilities including SQL injection, SSRF targeting cloud metadata services, prompt template injection, and path traversal via cursor manipulation. We find that 91.8% of dynamically audited servers lack OAuth authentication, 687 tool instances across confirmed servers expose shell execution capabilities without access controls, and 41.6% of confirmed servers disappear within three days between consecutive measurement runs---indicating rapid deployment cycles without security review. We report on our responsible disclosure pipeline and release Corvus as an open-source framework for MCP security evaluation.
Figures
Reference graph
Works this paper leans on
-
[1]
Introducing the model context protocol,
Anthropic, “Introducing the model context protocol,” Anthropic Blog, Nov. 2024. [Online]. Available: https://www.anthropic.com/news/ model-context-protocol
2024
-
[2]
Internet-wide scan: Model Context Protocol server exposure,
Censys, Inc., “Internet-wide scan: Model Context Protocol server exposure,” Censys Search Platform, May 2026, search query: services.service_name="mcp"; executed 2026-05-01; approxi- mately 21,000 results returned. Available upon request
2026
-
[3]
MCP safety audit: LLMs with the model context protocol allow major security exploits,
B. Radosevich and J. Halloran, “MCP safety audit: LLMs with the model context protocol allow major security exploits,”arXiv preprint arXiv:2504.03767, 2025
arXiv 2025
-
[4]
Model context protocol (MCP): Landscape, security threats, and future research directions,
X. Hou, Y . Zhao, S. Wang, and H. Wang, “Model context protocol (MCP): Landscape, security threats, and future research directions,”arXiv preprint arXiv:2503.23278, 2025
arXiv 2025
-
[5]
Not what you’ve signed up for: Compromising real-world LLM- integrated applications with indirect prompt injection,
K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world LLM- integrated applications with indirect prompt injection,” inProceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec). ACM, 2023
2023
-
[6]
A first measurement study on authentication security in real-world remote MCP servers,
H. Zhou, X. Zhang, H. Zhang, H. Zhang, M. Zhang, and M. Yang, “A first measurement study on authentication security in real-world remote MCP servers,”arXiv preprint arXiv:2605.22333, 2026
arXiv 2026
-
[7]
A first look at the security issues in the MCP ecosystem,
X. Li and X. Gao, “A first look at the security issues in the MCP ecosystem,”arXiv preprint arXiv:2510.16558, 2025
arXiv 2025
-
[8]
VIPER- MCP: Detecting and exploiting taint-style vulnerabilities in MCP servers,
P. Sun, Q. Jin, E. Huang, Z. Kang, X. Liu, D. Shen, and S. Li, “VIPER- MCP: Detecting and exploiting taint-style vulnerabilities in MCP servers,” arXiv preprint arXiv:2605.21392, 2026
arXiv 2026
Show all 21 references
-
[9]
MCP at first glance: Studying the security and maintainability of MCP servers,
M. M. Hasan, H. Li, E. Fallahzadeh, G. K. Rajbahadur, B. Adams, and A. E. Hassan, “MCP at first glance: Studying the security and maintainability of MCP servers,”arXiv preprint arXiv:2506.13538, 2025
2025 arXiv
-
[10]
Small world with high risks: A study of security threats in the npm ecosystem,
M. Zimmermann, C.-A. Staicu, C. Tenny, and M. Pradel, “Small world with high risks: A study of security threats in the npm ecosystem,” inProceedings of the 28th USENIX Security Symposium. USENIX Association, 2019, pp. 995–1010
2019
-
[11]
Thou shalt not depend on me: Analysing the use of outdated third-party libraries on the web,
T. Lauinger, A. Chaabane, S. Arshad, W. Robertson, C. Wilson, and E. Kirda, “Thou shalt not depend on me: Analysing the use of outdated third-party libraries on the web,” inProceedings of the Network and Distributed System Security Symposium (NDSS). Internet Society, 2017
2017
-
[12]
ZMap: Fast internet- wide scanning and its security applications,
Z. Durumeric, E. Wustrow, and J. A. Halderman, “ZMap: Fast internet- wide scanning and its security applications,” inProceedings of the 22nd USENIX Security Symposium. USENIX Association, 2013, pp. 605–620
2013
-
[13]
OWASP top ten 2021,
OWASP Foundation, “OWASP top ten 2021,” OWASP Foundation,
2021
-
[14]
OW ASP API security top 10 2023,
——, “OW ASP API security top 10 2023,” OW ASP Foundation, 2023. [Online]. Available: https://owasp.org/www-project-api-security/
2023
-
[15]
Petrel: MCP internet scanner and fingerprinter,
N. Padilla, “Petrel: MCP internet scanner and fingerprinter,” https:// github.com/CobaltoSec/petrel, 2026, version 0.8.0. Also available as cobaltosec-petrelon PyPI
2026
-
[16]
Corvus: Dynamic security testing framework for MCP servers,
——, “Corvus: Dynamic security testing framework for MCP servers,” https://github.com/CobaltoSec/corvus, 2026, version 1.3.1. Also available ascobaltosec-corvuson PyPI
2026
-
[17]
Vulnerability disclosure FAQ,
Google Project Zero, “Vulnerability disclosure FAQ,” Google Project Zero, 2022. [Online]. Available: https://googleprojectzero.blogspot.com/ p/vulnerability-disclosure-faq.html
2022
-
[18]
Complete guide to Shodan,
J. Matherly, “Complete guide to Shodan,” Shodan, 2015, documents large populations of unauthenticated Elasticsearch, MongoDB, and Redis instances on the public internet
2015
-
[19]
Understanding the Mirai botnet,
M. Antonakakis, T. April, M. Bailey, M. Bernhard, E. Bursztein, J. Cochran, Z. Durumeric, J. A. Halderman, L. Invernizzi, M. Kallitsis, D. Kumar, C. Lever, Z. Ma, J. Mason, D. Menscher, C. Seaman, N. Sullivan, K. Thomas, and Y . Zhou, “Understanding the Mirai botnet,” inProcee...
2017
-
[20]
The Menlo report: Ethical principles guiding information and communication technology research,
D. Dittrich and E. Kenneally, “The Menlo report: Ethical principles guiding information and communication technology research,” U.S. Department of Homeland Security, Science and Technology Directorate, Cyber Security Division, Tech. Rep., 2012. [Online]. Available: https://www...
2012
-
[2021]
Available: https://owasp.org/www-project-top-ten/
[Online]. Available: https://owasp.org/www-project-top-ten/
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.