{"id":"bb057a45-d4f5-453e-86bc-e7a179b5e7f8","arxiv_id":"2608.02723","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Mentions of e-MERLIN, JCMT, BiSON, ELT, SKA, and Rubin in 2025 arXiv astrophysics papers are counted and presented as a measure of research impact.","lead":"This paper counts how often UK-funded astronomy facilities are named in 2025 arXiv astrophysics papers, and reports the citation counts of those papers. It offers these numbers to inform the debate over the 2025 UK science funding cuts.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"'Mention equals data use' is unvalidated and contradicted by the paper's own Rubin analysis, so the headline '80 e-MERLIN papers' and the 'research impact' framing overstate what the counts show.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: a textual mention of a facility is treated as evidence of data use, and the paper's own Rubin paragraph demonstrates that this inference fails. This is not a disagreement with an external consensus but an internal inconsistency in the method. The e-MERLIN count is the paper's central quantitative result, so if even a modest fraction of the 80 papers are theoretical forecasts, comparison papers, status reports, or acknowledgments-only mentions, the headline overstates e-MERLIN's 2025 research impact. The paper is honest in its caveats and clearly aimed at a public/policy audience, but those caveats do not replace validation of the proxy. Since the term lists, code, and dataset are not released and the companion paper is inaccessible, an independent audit of the 80 papers is the minimal decisive check. The citation averages and distribution plots are secondary because they inherit the same denominator. Because the reader already conditioned the verdict on softening the wording and releasing the data, my analysis does not move the verdict; it reinforces it.","tokens_in":4300,"tokens_out":6682,"duration_ms":80676,"concrete_test":"Audit all 80 e-MERLIN papers from §3.1. Read the full text of each (or a random sample of at least 30) and classify each as (a) uses e-MERLIN data in the analysis, (b) discusses e-MERLIN without using its data, or (c) mentions it only in a citation, acknowledgment, or generic list. If the fraction in (b)+(c) exceeds 20%, the '80 papers used e-MERLIN data' claim is unsupported and the paper should report a verified data-use count and replace 'research impact' with 'text mentions'. If the fraction is small, the current wording is sustainable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §2 the paper states that if a paper 'mentions the names of any of the telescopes from a pre-defined list, that paper is counted as using the data from that telescope.' The headline count in §3.1—'80 papers in 2025 mentioned using data from the e-MERLIN network'—therefore depends entirely on a textual-mention proxy. The paper's own §3.2 Rubin analysis supplies a direct internal counterexample: Rubin is quoted in 1874 papers even though it 'only began operating halfway through 2025, meaning a good number of these papers were released before the first test images were taken.' This proves that a mention can occur without data use, at least for Rubin; the same unvalidated proxy is then used for e-MERLIN, JCMT, and the other facilities. No independent validation of the e-MERLIN term list or false-positive rate is given, and the companion methodology paper is not accessible from the current text. Because the Abstract and Conclusions present these counts as 'research impact', the argument's central inference is insecure; the paper is better described as reporting raw mention statistics.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a text-mining analysis of all 2025 arXiv astro-ph submissions, counting mentions of e-MERLIN, JCMT, BiSON, ELT, SKA, and the Vera C. Rubin Observatory in LaTeX sources as a proxy for research impact. It finds 80 papers mentioning e-MERLIN (161 when EVN-related terms are included), reports associated citation statistics, and compares e-MERLIN's average citation rate with that of JWST. The stated purpose is to inform the public and policy-makers debating the UKRI/STFC funding cuts.","tokens_in":4528,"tokens_out":4183,"duration_ms":50393,"significance":"The paper addresses a timely and policy-relevant question, and it has the virtue of using publicly available data with a transparent, mechanical counting procedure; the citation data obtained through the NASA ADS API is a further reproducible element. If the mention counts were validated against actual data usage, the results could serve as a useful descriptive indicator of the visibility of affected facilities in the 2025 literature. However, the principal measure conflates textual mention with data use, and the paper itself provides a direct counterexample for Rubin; consequently, the headline 'research impact' claims are not currently supported. The paper's strength is its data provenance and simplicity; its weakness is the gap between what is measured and what is concluded.","major_comments":[{"comment":"The central operational definition, stated in Section 2 as 'if a particular paper mentions the names of any of the telescopes from a pre-defined list, that paper is counted as using the data from that telescope,' equates textual mentions with data use. The paper's own Section 3.2 analysis of the Rubin Observatory shows that 1874 papers mention Rubin even though the observatory only began operating halfway through 2025 and a good number of those papers were released before first test images. This internal counterexample demonstrates that mentions do not imply data use, yet the same unvalidated proxy is then applied to e-MERLIN, JCMT, and the other facilities. Because the Abstract and Section 4 present these counts as 'research impact,' this conflation is load-bearing and the conclusions overstate what the data show.","section":"Section 2 and Section 3.2"},{"comment":"The paper reports that 'a paper using data from e-MERLIN was cited 2.40' times on average, compared with 3.93 for JWST, and Figure 1 is claimed to show that the averages are not skewed by outliers. No uncertainties are provided for any of the citation means, no statistical test is applied to the JWST comparison, and the figure's axes are unlabelled and its content is not legible in the submitted version. Given the typical heavy-tailed distribution of citation counts and the small e-MERLIN sample size of 80, the apparent gap between 2.40 and 3.93 may not be significant; bootstrap confidence intervals or a Poisson treatment should be reported.","section":"Section 3.1"},{"comment":"The exact search-term lists and matching code are not included in the manuscript; the reader is instead referred to Lewis et al. (2026), which the authors state used a different search scope (title and abstract only) rather than the full-text search employed here. Because the methodology differs, the companion paper does not suffice for reproducibility, and the reader cannot independently verify the headline count of 80 e-MERLIN papers. The manuscript should include the complete term list for each facility and a precise description of the matching rules, or an appendix with the extraction code.","section":"Section 2 and Section 4"}],"minor_comments":[{"comment":"The title contains a spacing error: 'F acilities' should be 'Facilities.'","section":"Title"},{"comment":"In the Conclusions, 'who's years of experience' should be 'whose years of experience,' and 'to asses if' should be 'to assess if.'","section":"Section 4"},{"comment":"The terminology for the measured quantity is inconsistent: 'mentioned using data' (Section 3.1), 'was attributed in' (Section 3.2), 'citing' and 'quoting' (Section 3.2) all refer to the same string-matching operation. A single term, such as 'mentions,' should be used consistently.","section":"Section 3"},{"comment":"The footnote references to RAS and BBC articles are not included in the reference list; if they are to be cited, they should be moved to the bibliography.","section":"Footnotes"},{"comment":"The claim that funding cuts were 'on the order of 30%' and later reduced to '2.7%' would benefit from a direct citation to the cited RAS article, since the numbers are central to the motivation.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"This manuscript functions more as a policy-facing brief than a research-methods paper. The main risk is that the 'research impact' framing will be taken at face value by non-specialist readers, and the Rubin example in the paper itself undermines the central proxy. A revised version that explicitly relabels the results as 'textual mention counts,' adds validation against a ground-truth data-usage list, and includes uncertainty estimates would make the contribution sound. The editor may also wish to consider whether the journal is the right venue for a descriptive report of this kind, given the absence of a methodological advance beyond the companion paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"If you're reading this for the funding debate, the headline numbers are genuinely new: 80 e-MERLIN papers, 235 for JCMT, 1874 for Rubin, with citation counts, all for 2025. That's a useful snapshot, and it's the first time these specific counts have been put together. The methodology is simple string matching over arXiv LaTeX sources, and the data source is public, so the counts are probably reproducible if you have the term lists. The authors are also honest in spots. They explicitly note that Rubin only began operating halfway through 2025, so many of those 1874 papers predate first light. And in the conclusions they defer to senior colleagues and remind readers that funding cuts don't map linearly onto output. That's fair and not typical of this genre. The soft spot is the one the reader flagged: the paper's central measure conflates 'mentioned in the text' with 'used the data.' Section 2 says exactly that any paper mentioning a telescope name is counted as using it. But Section 3.2's Rubin analysis is a direct counterexample—a mention can appear in a paper written before the telescope took any data. The same unvalidated proxy is then used for e-MERLIN and the others. So the abstract's phrase 'research impact' overstates what the counts show. Raw mention statistics, yes; impact, no. There are also smaller issues. The method details sit in an inaccessible companion paper (arXiv:2602.12303), so the current text isn't self-contained. And there's a corrupted glyph block in the Figure 1 caption that looks like a LaTeX error—that needs fixing before anything goes out. No uncertainty estimates, but for descriptive counts that's minor. Also \"differ\" should be \"defer\" in the conclusions. Who's this for? People writing responses to the STFC consultation, journalists, and astronomers who want quick context. The counts are worth having, but they should be read as mention frequencies, not as evidence that a facility is or isn't scientifically productive. My recommendation: send it to peer review, with the expectation of heavy revision. The data is real and the topic is important. Require the authors to (1) soften the framing from 'research impact' to 'text mentions', (2) release the term lists and code, ideally with a false-positive check against a sample of papers, and (3) fix the caption corruption. With those changes, it's a solid RNAAS or JOSS-style contribution.","headline":"Timely, transparent mention counts for UK-threatened facilities, but the 'research impact' framing overreaches: the paper's own Rubin example shows that mentions are not data use.","tokens_in":679,"tokens_out":2170,"would_cite":false,"duration_ms":41682,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper counts facility mentions in every 2025 astrophysics preprint to measure the research output put at risk by UK funding cuts, reporting 80 e-MERLIN papers and 1,874 Rubin papers.","keywords":["e-MERLIN","Jodrell Bank Observatory","UK science funding cuts","research impact","bibliometrics","preprint text mining","radio interferometry","telescope mentions"],"falsifier":"A manual audit of the 80 counted e-MERLIN papers, checking how many actually contain e-MERLIN observations in their data section rather than merely naming the array in passing, would settle whether the mention count is a genuine measure of research use.","tokens_in":4117,"feed_emoji":"🔭","tokens_out":13375,"duration_ms":142505,"temperature":0.7,"pith_summary":"The paper sets out to quantify the 2025 research output of the UK-funded facilities hit by the 2025–26 budget cuts, with e-MERLIN as the focus. It searches the full manuscript text of every astrophysics preprint submitted to the open preprint server during 2025 and counts how many papers mention each facility or its telescopes. The counts it reports are 80 papers for e-MERLIN (161 when European VLBI Network terms are added), 235 for the James Clerk Maxwell Telescope, 18 for the Birmingham Solar Oscillations Network, 490 for the Extremely Large Telescope, 1,416 for the Square Kilometre Array, and 1,874 for the Rubin Observatory, along with citation totals and averages. The purpose is to give the community and policy-makers a concrete, reproducible measure of the research these facilities supported before the funding decisions. A sympathetic reader should take the paper's claim to be that these mention and citation statistics are a meaningful part of the evidence for weighing the cuts.","feed_headline":"Preprint scan counts 80 e-MERLIN papers in 2025; UK cuts threaten them","feed_subtitle":"Every 2025 astrophysics preprint was searched for facility names, yielding 80 e-MERLIN and 1,874 Rubin paper mentions.","key_machinery":"The machinery is a string-matching search over the LaTeX source files of the 2025 astrophysics preprint corpus. For each facility the paper defines a list of names — for e-MERLIN, the array name plus each individual telescope in the network — and counts a paper as using the facility if any name appears, with case-insensitive matching for long forms and case-sensitive matching for short forms. One e-MERLIN variant adds European VLBI Network terms to capture joint international use. The resulting paper counts are combined with citation totals and per-paper averages, and a normalised citation distribution is compared with the JWST distribution to show that the e-MERLIN average is not driven by outliers.","core_discovery":"On its own terms, the paper's discovery is a set of mention-based impact statistics for the six affected facilities. Searching the full LaTeX text of the 2025 astrophysics preprint corpus, it finds that e-MERLIN appears in 80 papers (192 citations, 2.40 citations per paper), a count that grows to 161 papers and 384 citations when European VLBI Network terms are added. The James Clerk Maxwell Telescope appears in 235 papers (402 citations, 1.71 per paper); the Birmingham Solar Oscillations Network in 18 papers (51 citations, 2.83); the Extremely Large Telescope in 490 papers (1,336 citations, 2.73); the Square Kilometre Array in 1,416 papers (4,520 citations, 3.19); and the Vera C. Rubin Observatory in 1,874 papers (7,296 citations, 3.89). The paper notes that e-MERLIN's top subject categories are high-energy astrophysical phenomena and galaxies, that its most-cited paper follows up a fast X-ray transient, and that many Rubin mentions appeared before that observatory's first test images because it began operating halfway through 2025. It frames these totals as the research impact of facilities that are not being prioritised under the new budget.","pith_inferences":["The paper's own Rubin observation points to a limitation it does not fully apply to e-MERLIN: a mention is not proof of data use, so the 80 e-MERLIN papers are best read as an upper bound on direct usage rather than a precise count of it.","A natural extension would be to calibrate mention counts against the facilities' actual data archives or observing logs, converting 'papers that mention the telescope' into 'papers whose data products came from the telescope' and separating UK-led from international use; this would make the impact measure directly relevant to a UK funding decision.","If the cuts are implemented, the paper's 2025 counts form a before/after baseline: a measurable prediction of its own logic is that e-MERLIN and JCMT mention rates in 2026 and 2027 should fall relative to comparable facilities, and the speed of that drop would test whether the impact is as large as the counts suggest.","The extreme spread among facilities — 18 papers for BiSON versus 1,874 for Rubin — implies that raw mention counts cannot by themselves adjudicate the cuts, because a small but irreplaceable niche facility can score low while still being essential to its subfield."],"forward_implications":["If the mention counts are accepted as a measure of research output, e-MERLIN supported at least 80 published papers in 2025, and 161 when its joint European VLBI Network role is included, giving the funding cut a concrete annual output to weigh against the savings.","The e-MERLIN citation distribution tracks the shape of the JWST distribution, so the lower average citation rate is a general feature of these papers, not an artifact of one or two highly cited outliers.","The per-facility counts — 235 for the James Clerk Maxwell Telescope, 490 for the Extremely Large Telescope, 1,416 for the Square Kilometre Array, 1,874 for the Rubin Observatory — provide a common baseline that stakeholders can compare across the cut facilities and against future years.","Because the Rubin Observatory only began operating midway through 2025 and still dominates the mention counts, the numbers also show that preprints can mention a facility for planned science, not only for data already taken."],"supporting_citations":[{"why":"Provides the 2025 astrophysics preprint corpus, the extraction method this analysis inherits, and the caveats it says apply to its results.","marker":"R. F. Lewis et al. 2026"},{"why":"Cited as the most-cited e-MERLIN paper and as evidence that e-MERLIN is used for follow-up of X-ray transients.","marker":"M. Yadav et al. 2025"},{"why":"Cited as a gamma-ray burst afterglow paper using the network, supporting the claim that e-MERLIN serves high-energy transient science.","marker":"G. E. Anderson et al. 2025"},{"why":"Cited as a paper that mentions only the European VLBI Network and is therefore counted only when EVN terms are added, driving the count from 80 to 161.","marker":"X. Zhang et al. 2025"}],"fun_headline_variants":["80 e-MERLIN papers in 2025 preprint scan; UK cuts threaten them","Funding cuts put e-MERLIN's 80 papers from 2025 on the line","e-MERLIN had 80 papers in 2025, now UK funding cuts loom","2025 arXiv tally: 80 e-MERLIN papers at risk from UK cuts"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The counts rest on treating a mention of a facility name in a paper's source text as evidence that the paper used that facility's data; the paper's own Rubin numbers show mentions can predate any possible data use, so this premise can fail.","fun_headline_variants_meta":{"raw":{"variants":["80 e-MERLIN papers in 2025 preprint scan; UK cuts threaten them","Funding cuts put e-MERLIN's 80 papers from 2025 on the line","e-MERLIN had 80 papers in 2025, now UK funding cuts loom","2025 arXiv tally: 80 e-MERLIN papers at risk from UK cuts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00041,"raw_usage":{"total_tokens":2132,"prompt_tokens":960,"completion_tokens":1172,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":1091}},"tokens_in":576,"tokens_out":1172,"duration_ms":32898,"temperature":1.0,"reasoning_tokens":1091,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:53:00.965455+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A manual audit of the 80 counted e-MERLIN papers, checking how many actually contain e-MERLIN observations in their data section rather than merely naming the array in passing, would settle whether the mention count is a genuine measure of research use.","supporting_citations":[],"review_version":2}