pith. sign in

arxiv: 2605.17023 · v1 · pith:GJWXBKMInew · submitted 2026-05-16 · 💻 cs.DL · cs.SI

Sum of rank ratios: an alternative to percentiles for research assessment, from groundbreaking to mainstream research

Alonso Rodriguez-Navarro This is my paper

Pith reviewed 2026-05-19 18:33 UTC · model grok-4.3

classification 💻 cs.DL cs.SI
keywords Rn indexrank ratiosresearch assessmentcitation analysishighly cited paperscountry evaluationbibliometric indicatorspercentile alternatives
0
0 comments X

The pith

The Rn index sums ten local-to-global rank ratios to better measure each country's highest-quality research than top-percentile counts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes replacing percentile counts with a rank-ratio approach for assessing rare high-impact research. Each paper receives a value equal to its local rank within its country divided by its global rank, and the Rn index adds these values for the top ten papers. A sympathetic reader would care because standard top-1-percent and top-10-percent indicators work poorly for countries that produce little groundbreaking work. If the claim holds, evaluators could track the highest quality science from any nation using ordinary citation data.

Core claim

The paper claims that the Rn index, formed by summing the first ten ratios of each paper's rank inside its country to its rank among all papers worldwide, supplies a more accurate picture of the highest quality science produced by each country than counting papers in the top 10 percent or top 1 percent of citations. The method works for both the largest contributors to groundbreaking knowledge and for nations that seldom or never produce such work. It requires no specialized training and is almost unaffected by ties in citation counts.

What carries the argument

The Rn index, which adds the local-to-global citation rank ratios for a country's ten highest-ranked papers.

If this is right

  • Countries with few top papers can still receive a meaningful score based on the relative ranks of their best work.
  • The same index can evaluate both groundbreaking and mainstream high-quality research.
  • Calculation uses publicly available citation data and needs no bibliometric expertise.
  • Ties in citation counts have negligible effect on the final Rn value.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could be applied to individual researchers or institutions to surface contributions that percentile methods miss.
  • Time series of Rn values might reveal the emergence of new research powers earlier than percentile counts.
  • Correlation with non-citation indicators such as major awards or patents could test whether the index tracks real scientific advance.

Load-bearing premise

Summing citation rank ratios over the top ten papers accurately reflects the highest quality science without external checks against other indicators.

What would settle it

A mismatch between Rn index country rankings and independent expert judgments of which nations produce the most boundary-pushing research.

read the original abstract

Assessing research that pushes the boundaries of knowledge is challenging because such work is extremely infrequent, accounting for only about 0.01 per cent of all research outputs. Consequently, knowledge about how to evaluate this type of research is far more limited than the well established methods used to assess more common research outcomes. This study addresses this gap by using a rank based approach in which each paper is assigned a unique value equal to the ratio between its local and global ranks. The cumulative value of these ratios, starting from the most cited paper, provides the evaluative basis, and the Rn index described here, using 10 rank ratios, appears to be the best option. Although research assessment based on global ranks was originally developed to evaluate the largest contributors to groundbreaking knowledge, namely, the USA and China, which account for most of the most cited papers, the Rn index has broader applications. This study demonstrates that it is also a better option than the number of top 10 per cent or top 1per cent highly cited papers, which are the most common indicators used to evaluate countries that seldom or never produce cutting-edge research that pushes the boundaries of knowledge. In all cases, the Rn index reflects the highest quality science produced by each country. Furthermore, the Rn index can be easily calculated without specialized training in bibliometrics and is insignificantly affected by ties in citation counts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript proposes the Rn index, defined as the sum of local-to-global citation rank ratios for a country's top 10 papers, as a superior alternative to counting papers in the global top 10% or top 1% for assessing the highest-quality and groundbreaking research output of countries. It claims the index better reflects boundary-pushing science, is easy to compute, and is robust to citation ties, with applications beyond the largest producers like the USA and China.

Significance. If externally validated, the Rn index could provide a rank-ratio-based complement to percentile thresholds in bibliometric evaluation, potentially offering finer discrimination for entities with sparse high-impact output. The emphasis on computational simplicity and tie robustness is a practical strength for adoption in research assessment.

major comments (3)
  1. [Abstract and methods] Abstract and methods: The choice of exactly 10 rank ratios for the Rn index is stated to 'appear to be the best option' without any derivation, optimization procedure, sensitivity analysis across different numbers of ratios, or quantitative justification for optimality.
  2. [Results] Results: No comparative data, statistical tests, or tables are presented that benchmark Rn rankings against independent external anchors of groundbreaking research quality such as Nobel/Fields prizes, major award inventories, or curated breakthrough lists; superiority over top-10%/top-1% counts is asserted solely via internal re-ranking of citation data.
  3. [Discussion] Discussion: The central claim that Rn 'reflects the highest quality science produced by each country' rests on the untested assumption that summed local-to-global rank ratios over the first 10 papers track groundbreaking output more faithfully than percentile thresholds, without falsifiable tests or cross-validation.
minor comments (2)
  1. [Methods] Provide an explicit worked example of the local-to-global rank ratio calculation for a sample paper in the methods section to improve reproducibility.
  2. [Results] Add captions to any ranking tables or figures that explicitly state the data source, time window, and how ties are handled.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for the thoughtful and constructive report on our manuscript. We address each major comment in turn below, with clear indications of where revisions will be made to the next version of the paper.

read point-by-point responses

  1. Referee: [Abstract and methods] Abstract and methods: The choice of exactly 10 rank ratios for the Rn index is stated to 'appear to be the best option' without any derivation, optimization procedure, sensitivity analysis across different numbers of ratios, or quantitative justification for optimality.

    Authors: We agree that the manuscript would benefit from a more explicit justification. In the revised version we will add a sensitivity analysis (new figure and accompanying text) that compares country rankings obtained with k = 5, 10, 15 and 20 rank ratios. The analysis will quantify rank stability and the marginal contribution of additional ratios, thereby providing a data-driven rationale for retaining k = 10. revision: yes

  2. Referee: [Results] Results: No comparative data, statistical tests, or tables are presented that benchmark Rn rankings against independent external anchors of groundbreaking research quality such as Nobel/Fields prizes, major award inventories, or curated breakthrough lists; superiority over top-10%/top-1% counts is asserted solely via internal re-ranking of citation data.

    Authors: The manuscript’s scope is the introduction and internal validation of a new rank-ratio metric using citation data. External anchoring against prize lists or curated breakthrough inventories would require an entirely separate data-collection effort and matching protocol that lies beyond the present study. We therefore do not intend to add such benchmarks in the revision; the current comparisons are limited to demonstrating computational simplicity, tie robustness, and improved discrimination for lower-output countries. revision: no

  3. Referee: [Discussion] Discussion: The central claim that Rn 'reflects the highest quality science produced by each country' rests on the untested assumption that summed local-to-global rank ratios over the first 10 papers track groundbreaking output more faithfully than percentile thresholds, without falsifiable tests or cross-validation.

    Authors: We will revise the discussion to moderate the language, presenting Rn as a complementary indicator that offers finer resolution and greater applicability to smaller research systems rather than claiming definitive superiority. We will also add an explicit statement that further cross-validation against independent quality signals remains a task for future work. revision: partial

standing simulated objections not resolved
  • External benchmarking of Rn rankings against Nobel/Fields prizes or curated breakthrough lists, which would require new data assembly and matching procedures not present in the original manuscript.

Circularity Check

1 steps flagged

Rn index defined directly from rank ratios with superiority asserted via internal re-ranking

specific steps
  1. self definitional [Abstract]
    "each paper is assigned a unique value equal to the ratio between its local and global ranks. The cumulative value of these ratios, starting from the most cited paper, provides the evaluative basis, and the Rn index described here, using 10 rank ratios, appears to be the best option. [...] In all cases, the Rn index reflects the highest quality science produced by each country."

    The evaluative claim that the sum of the first 10 rank ratios 'reflects the highest quality science' and is 'the best option' is made by direct reference to the index's own construction from citation ranks. No independent quality signal is invoked to validate that this particular summation better tracks groundbreaking output than simple top-percentile counts; the asserted superiority therefore reduces to a reweighting of the input citation distribution.

full rationale

The paper defines the Rn index explicitly as the sum of the first 10 local-to-global rank ratios derived from citation counts. It then asserts that this index 'reflects the highest quality science' and is superior to percentile counts. While the construction itself is transparent and not a hidden fit, the claim of improved evaluation of groundbreaking research rests on reordering the same citation data without external anchors (e.g., prizes or expert lists). This creates partial circularity in the evaluative claim, though the metric remains a straightforward transformation rather than a self-referential loop or self-citation chain. No load-bearing uniqueness theorem or ansatz smuggling is present.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The central claim rests on citation counts serving as a quality proxy and on the ad-hoc selection of 10 ratios; the Rn index itself is a newly constructed quantity without independent external grounding shown in the abstract.

free parameters (1)
  • Number of rank ratios summed
    The paper fixes the sum at the first 10 ratios to define the Rn index; this choice is not derived from first principles in the abstract.
axioms (1)
  • domain assumption Citation counts and resulting ranks are a valid proxy for research quality and groundbreaking contribution.
    All evaluation steps begin from citation-based local and global ranks.
invented entities (1)
  • Rn index no independent evidence
    purpose: Cumulative sum of the first 10 local-to-global rank ratios as a new evaluative metric.
    Newly defined construct whose validity is asserted rather than independently verified in the abstract.

pith-pipeline@v0.9.0 · 5775 in / 1340 out tokens · 53968 ms · 2026-05-19T18:33:27.318868+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    articles

    Materials and methods This study builds upon two previous papers (Rodríguez-Navarro, 2025a; Rodríguez-Navarro & Brito, 2024b). Accordingly, all materials and methods used here are the same as those employed in those prior studies. The reported Rk-index values are taken directly from (Rodríguez-Navarro, 2025a), while the new Rn-index values described below...

    work page 2019

  2. [2]

    accident

    Discussion 5.1. Aim of the Rn-index Sections 1.1 and 1.2 highlight the challenge of developing research indicators specifically suited to evaluating landmark research that pushes the boundaries of knowledge. A similar challenge arises when evaluating the best science produced by countries. Figure 4 shows that there are at least two ways in which one count...

    work page doi:10.36850/v91j-7541 1996