{"id":"53d735b6-9dbf-4de5-8856-0cee0e5456cb","arxiv_id":"2411.18161","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new indicator, the Rn-index, corrects the saturation and summability problems of the Rk-index for countries that dominate a research field.","lead":"This paper proposes a new bibliometric indicator, the Rn-index, which sums the ratios between a country's local and global citation ranks for its ten most cited papers. It is meant to fix a weakness in the earlier Rk-index that understates differences between dominant countries like the USA and China.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation transfer in §4 is the weak link: Rk was not synthetic-series-tested in the high-Rk saturation regime that Rn is designed to fix, and the top-0.1% benchmark has no Rk baseline.","rationale":"The proposed Rn-index is clearly defined and the arithmetic improvements (summability, spread) are demonstrated on the displayed examples. The main issue is evidential, not mathematical. The paper's own Section 2 says the saturation regime was not covered by the synthetic-series validation of Rk; Section 4 nevertheless transfers that validation to Rn. Figure 1's upward curvature means the transfer is least safe precisely at the high-Rk values where the claimed correction matters. The top-0.1% correlation (r=0.65) is suggestive but is an assumed proxy and lacks a Rk baseline, so it cannot distinguish 'Rn is accurate' from 'Rn is no better than Rk' or even 'Rk is equally or more correlated with the proxy.' These gaps are exactly what the reader called conditional. I would not reject the paper: the index is simple, reproducible from rank data, and the summability and saturation behavior is a real improvement over Rk in the examples. But the headline 'more accurate' claim should be conditional until the proposed head-to-head and high-regime synthetic checks are performed. The concern is about the argument's load-bearing validation step, not about the author or the basic idea.","tokens_in":5324,"tokens_out":5033,"duration_ms":49122,"concrete_test":"Run the synthetic-series validation from Rodríguez-Navarro & Brito (2024) in the saturation regime: generate countries whose true breakthrough rates put their Rk values above ~30 (maximum 39.5), then compare Rk and Rn against the known true rates. On the same 23 Table 4 units, also compute Rk's correlation with top 0.1% and, if feasible, top 0.01% counts aggregated across the six topics. If Rn does not outperform Rk in both the high-regime synthetic test and the real-data benchmark, Section 4's transferred validation and the claim that Rn is more accurate fail.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Rn is more accurate than Rk for countries producing most advances. Section 2 concedes that the Rk-index's weakness appears precisely because the saturation regime 'was not tested with synthetic series.' Section 4 then transfers the synthetic-series validation of Rk to Rn on the strength of the numerical Rk–Rn relationship (Fig. 1). That transfer is not valid for the regime in question: Fig. 1 is upward-curving at high Rk values, and the validation of Rk never covered those high values. The real-data check in Table 4 does not close the gap: the relevant breakthrough threshold is top 0.01% cited papers (as Section 4 states), but the analysis drops to top 0.1% with the words 'Assuming this.' The reported correlation is r = 0.65 (p = 0.001), i.e., moderate, and no Rk-index is computed on the same 23 country–topic units. So the higher accuracy claim rests on (i) an extrapolation of validation into exactly the regime where Rk's own validity is unknown, and (ii) a proxy correlation without a head-to-head baseline against Rk. The summability and spread improvements shown in Tables 2–3 are properties of the formula, but they have not been demonstrated to track true frontier contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Rn-index, a variant of the Rk-index designed to correct what the author identifies as a weakness of the Rk-index: saturation and loss of summability when a small number of countries produce most of the highly cited papers in a topic. The Rn-index is defined as the sum of the ratios between local and global ranks of the 10 most cited papers, multiplied by 10 (equivalently, the arithmetic mean of these ratios scaled by 100). Using data for several countries and topics, the paper shows that Rn-values deviate less from the summability property than Rk-values and spread out values that saturate near the Rk maximum. The paper then argues that the synthetic-series validation of the Rk-index transfers to the Rn-index because of a strong numerical relationship between the two indicators, and it reports a real-data correlation of r = 0.65 (p = 0.001) between the Rn-index and the number of top 0.1% cited papers across 23 country–topic units. The author concludes that the Rn-index provides a more accurate measure of contribution to scientific advancement than the Rk-index, especially for high-output countries.","tokens_in":5595,"tokens_out":2021,"duration_ms":19395,"significance":"If the central claim is correct, the Rn-index would be a practically useful, easily computable bibliometric indicator that avoids a known saturation artifact of the Rk-index while preserving its intended interpretation. The paper is honest about the rarity of breakthrough-level papers and about the proxy nature of the top 0.1% threshold. The summability improvements in Table 3 are clearly presented and, within the small sample, support the claim that the Rn-index behaves better than the Rk-index on that specific property. The paper also has the virtue of transparency: the index is defined by an explicit formula with no fitted parameters beyond the k = 10 and offset +20 inherited from the Rk-index. However, the significance of the paper for the journal rests on demonstrating that the Rn-index is more accurate for measuring frontier contribution, and that demonstration currently depends on an extrapolation of prior validation results into a regime that the original validation did not cover, plus a moderate proxy correlation without a head-to-head comparison against the Rk-index.","major_comments":[{"comment":"The transfer of the synthetic-series validation from the Rk-index to the Rn-index is not valid for the regime in question. Section 2 explicitly states that the Rk-index's weakness arises at values close to its maximum and that this situation 'was not tested with synthetic series.' Section 4 then transfers the validation 'considering the strong numerical relationship between the Rn and Rk indices (Fig. 1).' But Fig. 1 shows upward curvature at high Rk values, so the strong relationship does not imply that the Rn-index inherits Rk's validated behavior precisely in the high-value regime where Rk was never validated. The paper needs to either provide synthetic-series tests that include the saturation regime or otherwise justify the transfer for that regime.","section":"Section 4, first paragraph"},{"comment":"The real-data validation in Table 4 does not close the gap in the accuracy claim because (i) the relevant breakthrough threshold is top 0.01% cited papers, as the paper itself states, but the analysis uses top 0.1% with the phrase 'Assuming this,' without any evidence that the coarser percentile behaves similarly for the countries and topics studied; and (ii) no Rk-index is computed for the same 23 country–topic units, so the table cannot show that the Rn-index is more accurate than the Rk-index, only that the Rn-index correlates moderately with a top-percentile count. A head-to-head comparison of Rn and Rk against the same benchmark is needed to support the comparative claim.","section":"Section 4, Table 4"},{"comment":"The summability comparison in Table 3 is based on a small, non-random set of countries and topics (12 rows) and is not accompanied by any statistical test. The text states that Rk deviations are 'random, but also larger for higher values' and that Rn deviations are 'only random, without any dependency on high values,' but these claims are based on visual inspection without a quantitative test of the deviation–magnitude relationship. A simple regression or correlation between deviation size and index magnitude, with confidence intervals, would substantiate the claim that the Rn-index removes the value-dependence of the deviation.","section":"Table 3 and surrounding text"}],"minor_comments":[{"comment":"The paper reports Pearson's r = 0.65 and a two-sided p-value but does not report a confidence interval or the effective sample size per topic; adding these would clarify the strength of the correlation given the small number of units.","section":"Section 4 and Table 4"},{"comment":"The abbreviation 'Doma' appears without a footnote in the table header; the footnote defining 'Dom, domestic' is present but would be easier to use if placed immediately under the table.","section":"Table 3"},{"comment":"There are typos in two references: 'reserach' should be 'research' in the 2024a and 2024b entries, and 'knowlege' should be 'knowledge' in the 2024c entry.","section":"References"},{"comment":"The sentence 'Furthermore, in these tests, the deviations of the Rn-index are smaller than those of the Rk-index' is ambiguous because Table 3 shows that in some rows (e.g., China lithium batteries) the Rn deviation is not smaller than the Rk deviation; the generalization should be qualified.","section":"Section 3, last paragraph"},{"comment":"The definition of the Rn-index as 'sum of the 10 rank ratios multiplied by 10' is not immediately clear from the table, which shows the arithmetic mean of rank ratios; a sentence clarifying the scaling factor would avoid confusion.","section":"Section 3, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is short and readable, but the central claim of superior accuracy over the Rk-index rests on a validation transfer that is not justified for the high-value regime, and the only real-data validation does not include an Rk baseline. These issues are fixable within the scope of a revision, but they are load-bearing. I would also flag that the paper relies heavily on the author's own prior work; that is not a problem per se, but the refereeing process should ensure that the prior validation is checked independently. The manuscript fits the journal's scope as a bibliometric indicator proposal, but it would need the additional analyses described in the major comments before I could recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The Rn-index is a simple arithmetic-mean variant of the Rk-index, and the paper gives a fair, transparent first look at why it might be preferable for dominant countries. The idea is new relative to the references, easy to compute, and fractionally countable, which are real practical advantages. The summability data in Table 3 are the strongest part: across the 12 country-topic pairs, Rn deviations are smaller than Rk's and do not grow with the value of the index, fixing the saturation pattern that motivated the paper. That is worth credit.\n\nThe soft spot is the validation transfer in Section 4. The paper concedes in Section 2 that the Rk-index's weakness appears precisely because the saturation regime \"was not tested with synthetic series.\" Then Section 4 transfers Rk's synthetic-series validation to Rn because of the strong Rk-Rn relationship in Fig. 1. But that relationship is upward-curving at high Rk values, so the transfer is an extrapolation into the very regime where Rk's own validity is unknown. Table 4 does not close the gap: it correlates Rn with top 0.1% cited papers (r=0.65, no CI) and includes no Rk baseline on the same 23 units, so we cannot tell whether Rn is actually more accurate than Rk at the top 0.01% frontier. The author explicitly flags the 0.1% assumption, which is good, but the claim that Rn is \"more accurate\" is not yet supported.\n\nThese are fixable issues. A head-to-head Rk vs Rn regression on the same benchmark and a sensitivity check at 0.01% would go a long way. The sample of Table 3 is small and non-random, so treat the summability improvement as suggestive rather than proven. In proportion, the paper is a coherent, honest proposal of a plausible indicator, not an overclaiming one; the limitations are largely admitted.\n\nI'd send this to peer review. The index is simple enough to be used, and referees can settle the validation question with a modest amount of additional analysis. I wouldn't cite it in my own work this year, but I'd bring it to a reading group interested in bibliometric indicators.","headline":"A simple arithmetic-mean variant of the Rk-index that fixes saturation and summability issues in high-output countries, but the accuracy claim rests on a validation transfer that doesn't cover the regime it's meant to fix.","tokens_in":6122,"tokens_out":2311,"would_cite":false,"duration_ms":19190,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The Rn-index, a simple sum of local-to-global rank ratios, corrects the Rk-index's saturation and summability failures for countries that dominate research.","keywords":["Rn-index","Rk-index","rank ratios","research performance evaluation","breakthrough papers","summability","citation analysis","top cited papers"],"falsifier":"Take a set of synthetic citation series with known breakthrough papers at the 0.01% level, compute Rn and Rk for many synthetic countries, and check whether Rn reproduces the known ordering and has smaller summability deviations than Rk in high-output cases; alternatively, compute Rn and true top-0.01% counts in a large multi-topic dataset and see whether the reported correlation with the 0.1% proxy survives.","tokens_in":5074,"feed_emoji":"📈","tokens_out":10252,"duration_ms":87206,"temperature":0.7,"pith_summary":"The paper proposes the Rn-index, a variant of the Rk-index, to measure a country's or institution's contribution to pushing the boundaries of knowledge, a contribution that ordinary citation indicators often miss because breakthrough papers are only about 0.01% of all publications. The Rk-index works well in general but saturates when a few actors produce most advances: for the USA in lithium batteries, its value sits near the index's maximum, so differences between leading countries are compressed and the index is not summable when domestic and collaborative papers are pooled. The Rn-index replaces the Rk calculation with an arithmetic sum of ten local-to-global rank ratios, which spreads out the compressed values and makes summability deviations smaller and independent of the index's magnitude, as shown for the USA, China, and other countries across topics such as solar cells, lithium batteries, and stem cells. A reader should care because research performance at the frontier is used to compare countries and institutions, and a simple, fractionable indicator that avoids this particular distortion would make such comparisons more trustworthy.","feed_headline":"Rn-index spreads out compressed Rk scores for top research countries","feed_subtitle":"For the USA and China, a simple sum of rank ratios replaces compressed near-max scores.","key_machinery":"The carrying object is the Rn-index, defined as ten times the sum, over a country's or institution's ten most-cited papers, of the ratio of each paper's local rank to its global rank. The Rk-index is the geometric mean of inverted global ranks of the ten most-cited papers with 20 added, and the Rn-index replaces that geometric mean with an arithmetic mean of rank ratios. This arithmetic mean is what gives the indicator its corrective behavior: it remains sensitive to positions inside the very top of the global distribution, where Rk values pile up against their maximum, and it treats each of the ten papers additively, which is what makes the indicator closer to summable when domestic and collaborative sets are pooled. The paper uses the observed strong relationship between Rn and Rk, and between arithmetic and geometric means of rank ratios, to argue that the established synthetic-series validation of Rk carries over to Rn.","core_discovery":"On the paper's own terms, the central discovery is that computing ten times the sum of the ratios between local and global ranks of a country's ten most-cited papers yields an indicator, the Rn-index, that behaves better than the Rk-index exactly where the Rk-index is weakest. For high-output countries the Rn-index separates domestic and collaborative publication sets that the Rk-index leaves nearly indistinguishable, and it reduces or eliminates the inflation of values that occurs when the Rk-index of domestic papers is added to that of collaborative papers. The paper reports that in the USA lithium-battery case the Rk sum exceeds the pooled Rk by 48.9%, while the Rn sum deviates by only -2.0%; across the tested countries and topics the Rn deviations are smaller and depend only on randomness, not on the size of the indicator. The paper also reports a correlation of 0.65 (p = 0.001) between Rn values and counts of top 0.1% cited papers, which it reads as external support for the indicator while noting that percentile counts discard the within-percentile information that Rn preserves.","pith_inferences":["One could test whether Rn's summability advantage grows with topic concentration: the more skewed the world's output toward a few countries, the more Rn should outperform Rk.","Because the ratio of local to global rank has a bounded range, Rn may admit analytic distributional results, such as moments or asymptotic bounds, that Rk's extreme-order statistics do not, which could make significance tests for country differences straightforward.","If Rn is adopted for country-level frontier assessment, its sensitivity to the choice of the ten-paper window should be probed; the paper does not analyze what happens when the window size changes."],"forward_implications":["For countries like the USA and China in fields such as lithium batteries and solar cells, Rn-based country comparisons will show larger and more meaningful gaps than Rk-based comparisons.","Rn is closer to satisfying the summability property, so combining domestic and collaborative papers, or institutions, will not inflate the total as much as Rk does.","Because Rn is just a sum of ten rank ratios, it is easy to compute, explain, and use with fractional counting for co-authored papers.","If the correlation with top-0.1% cited papers holds more broadly, Rn can serve as a lightweight indicator of breakthrough-level contribution before enough top-0.01% papers accumulate to measure directly."],"supporting_citations":[{"why":"Introduces the Rk-index and its synthetic-series validation, which the paper extends to the Rn-index.","marker":"Rodríguez-Navarro & Brito (2024)"},{"why":"Supplies the Rk-index values for USA and China in lithium batteries and other topics used to demonstrate the saturation and summability weakness.","marker":"Rodríguez-Navarro (2024c)"},{"why":"Establishes the summability property of top-percentile indicators against which Rk's failure and Rn's improvement are measured.","marker":"Bornmann et al. (2013)"},{"why":"Defines breakthrough-level publications at roughly the top 0.01% of cited papers, the target phenomenon the indicator aims to capture.","marker":"Bornmann et al. (2018)"},{"why":"Also anchors the top-0.01% breakthrough definition used in the validation context.","marker":"Poege et al. (2019)"},{"why":"Shows that ranks of the most-cited papers follow a power law or ordered deviations, which the paper uses to explain the strong relationship between arithmetic and geometric means of rank ratios.","marker":"Rodríguez-Navarro (2024b)"}],"fun_headline_variants":["Rn-index fixes Rk blind spot for USA and China","Simple rank-ratio sum outshines Rk for top nations","Rn-index unclumps Rk scores for research leaders","Better metric for countries that push boundaries","Rn separates domestic and collaborative research scores"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Rk-index's synthetic-series validation transfers to the Rn-index because the two are numerically close, and that counts of top-0.1% cited papers are an acceptable stand-in for the true top-0.01% breakthrough rate; the paper's Section 4 explicitly says \"Assuming this\" for the second step, so if either link fails the claim that Rn is more accurate is not established.","fun_headline_variants_meta":{"raw":{"variants":["Rn-index fixes Rk blind spot for USA and China","Simple rank-ratio sum outshines Rk for top nations","Rn-index unclumps Rk scores for research leaders","Better metric for countries that push boundaries","Rn separates domestic and collaborative research scores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000148,"raw_usage":{"total_tokens":1154,"prompt_tokens":878,"completion_tokens":276,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":201}},"tokens_in":494,"tokens_out":276,"duration_ms":3500,"temperature":1.0,"reasoning_tokens":201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:26:58.855431+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of synthetic citation series with known breakthrough papers at the 0.01% level, compute Rn and Rk for many synthetic countries, and check whether Rn reproduces the known ordering and has smaller summability deviations than Rk in high-output cases; alternatively, compute Rn and true top-0.01% counts in a large multi-topic dataset and see whether the reported correlation with the 0.1% proxy survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Rk-index and its synthetic-series validation, which the paper extends to the Rn-index."}],"review_version":1}