{"id":"c9aa12c6-af94-4183-a771-30d551648f66","arxiv_id":"2605.22333","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"First measurement study of 7,973 remote MCP servers finds 40.55% lack authentication and all 119 tested OAuth servers have flaws that risk data leaks or account takeover.","lead":"The study identified 7,973 live remote MCP servers and found that 40.55% expose tools without any authentication. Among the OAuth-using servers tested, every one had at least one flaw, often tied to dynamic client registration.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Accuracy of semi-automated passive-plus-active probing framework for flaw detection lacks independent validation","rationale":"The reader's weakest assumption on framework accuracy and representativeness matches the load-bearing point for the measurement results. This supports keeping the CONDITIONAL verdict pending full methodology details rather than shifting to ACCEPT or REJECT.","tokens_in":1803,"tokens_out":291,"duration_ms":28115,"concrete_test":"Randomly sample 15 of the 119 servers flagged for dynamic client registration flaws; re-execute the active probing steps on those endpoints while logging raw responses, then manually classify each against the nine flaw-type definitions; if ≥3 reclassifications occur, recompute the 96.6% figure on the full set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (all 119 testable OAuth-enabled servers exhibit ≥1 flaw, totaling 325 with dynamic client registration flaws in 96.6%) rests on the custom detection framework. The abstract describes a semi-automated combination of passive traffic inspection and active dynamic probing guided by an MCP-specific taxonomy, but provides no reported false-positive rate, manual verification fraction, or ground-truth comparison. If probing responses are misinterpreted (e.g., intended dynamic registration treated as a flaw, or non-vulnerable endpoints flagged), the per-server and aggregate counts become unreliable, directly weakening the pervasiveness conclusion for the tested set.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"This paper presents the first measurement study of authentication security in real-world remote MCP servers. It identifies 7,973 live remote MCP servers, finding that 40.55% expose tools without authentication. Among authenticated servers, OAuth is dominant; applying a semi-automated passive-plus-active probing framework guided by an MCP-specific taxonomy of four categories and nine flaw types to 119 testable OAuth-enabled servers yields 325 flaws total, with every server exhibiting at least one flaw and dynamic client registration flaws affecting 96.6%. The work includes responsible disclosure resulting in 9 CVEs.","tokens_in":1932,"tokens_out":488,"duration_ms":46839,"significance":"If the detection methodology holds, the findings are significant as the first empirical evidence of pervasive authentication weaknesses in the emerging MCP ecosystem for LLM-external service connections. The scale of server discovery, concrete flaw counts, and obtained CVEs highlight risks of information leakage and account takeover, supporting calls for hardened OAuth deployments. The study supplies concrete counts from large-scale scanning and testing.","major_comments":[{"comment":"§3 (Detection Framework) and Abstract: the central claims that all 119 testable OAuth-enabled servers exhibit at least one flaw (total 325 flaws, 96.6% with dynamic client registration flaws) rest on the semi-automated passive-plus-active probing framework. No false-positive rate, manual verification fraction, or ground-truth comparison is reported, leaving open the possibility that probing responses are misinterpreted and directly weakening the pervasiveness conclusion for the tested set.","section":"§3 (Detection Framework) and Abstract"}],"minor_comments":[{"comment":"The criteria for deeming a server 'testable' and the exact breakdown of the 7,973 servers into authenticated vs. unauthenticated subsets could be stated more explicitly to aid reproducibility.","section":null},{"comment":"A brief discussion of potential scanning artifacts (e.g., rate-limiting responses or honeypot-like servers) would strengthen the methodology presentation without altering the core results.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to fit the journal scope for empirical security measurement studies, but the citation list seems light on prior large-scale OAuth measurement papers; this may warrant a request for expanded related-work coverage in revision."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their careful and constructive review of our manuscript. We address the major comment on the validation of our detection framework below and have revised the manuscript to incorporate additional details on our verification procedures.","responses":[{"response":"We thank the referee for this observation, which highlights an opportunity to strengthen the presentation of our methodology. Our semi-automated framework integrates passive traffic analysis with active probing specifically tailored to the nine flaw types in our MCP-specific taxonomy. Each flaw is identified only when the server response matches a predefined, observable indicator (for instance, acceptance of unauthenticated dynamic client registration requests or exposure of tokens without proper scope validation). To mitigate risks of misinterpretation, we conducted a post-hoc manual review of the full set of probe responses and logs for all 119 servers, confirming that every reported flaw aligned with the expected behavioral signature. In addition, we performed an in-depth manual verification on a randomly selected subset of 25 servers (approximately 21% of the testable set), including direct inspection of registration endpoints and token issuance behavior where ethically permissible. No false positives were identified in this subset. While a complete ground-truth oracle for every server is not feasible—owing to the dynamic, third-party nature of the services and constraints against exhaustive active testing that could affect availability—we have added a dedicated subsection in the revised §3 describing the verification process, the manual review fraction, and the rationale for relying on conservative, multi-stage indicators. These changes directly address the concern and reinforce the reliability of the pervasiveness claim.","revision_made":"yes","referee_comment":"[§3 (Detection Framework) and Abstract] §3 (Detection Framework) and Abstract: the central claims that all 119 testable OAuth-enabled servers exhibit at least one flaw (total 325 flaws, 96.6% with dynamic client registration flaws) rest on the semi-automated passive-plus-active probing framework. No false-positive rate, manual verification fraction, or ground-truth comparison is reported, leaving open the possibility that probing responses are misinterpreted and directly weakening the pervasiveness conclusion for the tested set."}],"tokens_in":1431,"tokens_out":446,"duration_ms":39351,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing to know is that the authors scanned a large number of remote MCP servers and concluded that authentication problems are everywhere in the OAuth subset they could test. All 119 had flaws, adding up to 325 issues, mostly tied to dynamic client registration. What is new here is the application of measurement to this specific protocol. MCP is positioned as a way for LLMs to talk to real services, and the paper shows how its auth setup differs from standard OAuth by being more open and dynamic. They build a taxonomy with four categories and nine flaw types, then use a mix of passive and active probing to check for them at scale. Finding 7,973 servers and getting CVEs from disclosure adds some weight to the practical side. The paper does a solid job identifying an emerging risk area that prior work had not quantified. The numbers on unauthenticated exposure and the dominance of certain OAuth traits provide a baseline that others can build on. The soft spots center on the methodology details that are not in the abstract. The probing framework is described at a high level, but there is no mention of error rates or verification steps, which makes it tough to gauge if the flaw counts are inflated or accurate. Representativeness of the testable servers is another area that could use more explanation to avoid selection bias concerns. This paper would interest people studying security for AI agents or new integration protocols. A reader looking for data on real deployments rather than theoretical attacks would get the most out of it. I recommend putting it through peer review. The topic has clear relevance and the empirical effort is substantial enough to merit referee feedback, especially on strengthening the validation parts.","headline":"This measurement study flags widespread authentication flaws across tested MCP servers, but the custom detection method needs more transparency on accuracy.","tokens_in":2394,"tokens_out":396,"would_cite":true,"duration_ms":45232,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"We implement a semi-automated detection framework that combines passive traffic inspection with active dynamic probing. Applying it to 119 testable real-world OAuth-enabled MCP servers, we find that each server exhibits at least one flaw, with a total of 325 flaws identified"}],"headline":"Empirical OAuth flaw measurement study in MCP servers has no structural overlap with RS forcing chain","alignment":"orthogonal","rationale":"The paper's central machinery is a semi-automated passive-plus-active probing framework applied to 119 real-world OAuth-enabled MCP servers, producing a taxonomy of 9 flaw types across 4 categories (dynamic client registration, delegated authorization, open client environments, common OAuth misconfigurations). This is a conventional security measurement study with no J-cost functions, ratio-symmetric costs, golden-ratio identities, 8-tick periodicity, or parameter-free constant derivations. RS modules such as Cost.FunctionalEquation (J-uniqueness), Foundation.DimensionForcing (D=3 from 8-tick), and Foundation.RealityFromDistinction have no bearing on the domain.","tokens_in":56806,"confidence":"high","tokens_out":284,"duration_ms":10025,"cache_read_input_tokens":32896,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"The first measurement of remote MCP server authentication finds every OAuth-enabled server has flaws, with 325 total issues identified.","keywords":["Model Context Protocol","MCP","OAuth","authentication security","measurement study","remote servers","LLM agents","dynamic client registration"],"falsifier":"An independent large-scale scan that finds many OAuth-enabled MCP servers free of all nine flaw types in the taxonomy would indicate the issues are not as pervasive as reported.","tokens_in":2723,"feed_emoji":"🔒","tokens_out":672,"duration_ms":53080,"temperature":0.7,"pith_summary":"This paper performs the first large-scale study of authentication security in real-world remote Model Context Protocol servers that allow large language models to interact with user services. It locates thousands of live servers and develops a taxonomy of flaws tailored to MCP's use of OAuth, including dynamic client registration and delegated authorization. Testing 119 servers reveals that all have at least one flaw and dynamic client registration problems affect almost all of them. These issues matter because they can lead to data leaks and account compromises in an emerging standard for AI agents connecting to personal accounts.","feed_headline":"All 119 tested MCP OAuth servers show authentication flaws","feed_subtitle":"First study of real-world remote servers finds 325 total issues, dynamic client registration flaws in 96.6 percent.","key_machinery":"The taxonomy of authentication flaws comprising three MCP-specific categories and conventional OAuth misconfigurations, detected through a semi-automated framework of passive traffic inspection and active dynamic probing.","core_discovery":"Applying a semi-automated detection framework to 119 testable real-world OAuth-enabled MCP servers shows that each server exhibits at least one flaw, with a total of 325 flaws identified. Dynamic client registration flaws affect 96.6% of the tested servers. Among 7,973 identified live remote MCP servers, 40.55% expose tools without authentication. The study derives a taxonomy of four categories and nine concrete flaw types specific to MCP OAuth characteristics.","pith_inferences":["Developers building MCP servers should prioritize securing or avoiding dynamic client registration to reduce risks.","Similar flaws could appear in other protocols where LLMs act as clients to user-linked services.","The pervasiveness suggests that MCP's design for ease of integration trades off security in ways not present in traditional OAuth setups.","Broader adoption of MCP may require new security guidelines or updates to OAuth practices for AI agents."],"forward_implications":["40.55% of identified remote MCP servers expose tools without any authentication.","OAuth deployments in MCP feature open client environments, dynamic client registration, and delegated authorization that create new attack surfaces.","Many identified flaws can lead to sensitive information leakage and account takeover.","Responsible disclosure resulted in 9 CVE IDs being assigned.","There is an urgent need for hardened OAuth-based remote MCP deployments."],"fun_headline_variants":["119 MCP OAuth servers all have authentication flaws","325 flaws found across all 119 MCP servers tested","96.6% of MCP servers have dynamic registration flaws","40.55% of MCP servers expose tools with no authentication"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The identified 7,973 servers and the 119 testable OAuth servers accurately represent real-world remote MCP deployments, and the probing framework detects flaws without significant false positives or missed issues.","fun_headline_variants_meta":{"raw":{"variants":["119 MCP OAuth servers all have authentication flaws","325 flaws found across all 119 MCP servers tested","96.6% of MCP servers have dynamic registration flaws","40.55% of MCP servers expose tools with no authentication"]},"model":"grok-4.3","cost_usd":0.014268,"raw_usage":{"total_tokens":6116,"prompt_tokens":764,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":142678000,"prompt_tokens_details":{"text_tokens":764,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":5290,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":764,"tokens_out":62,"duration_ms":62457,"temperature":1.0,"reasoning_tokens":5290,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-22T05:31:19.274945+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"An independent large-scale scan that finds many OAuth-enabled MCP servers free of all nine flaw types in the taxonomy would indicate the issues are not as pervasive as reported.","supporting_citations":[],"review_version":1}