pith. sign in

arxiv: 2605.17657 · v2 · pith:SITNY7LRnew · submitted 2026-05-17 · 💻 cs.DL · cs.DB

General Science Ranking (GSR): An Open-Source, Citation-Normalized Journal and Conference Classification System for Computer Science and Medicine

Zhikai Yu This is my paper

Pith reviewed 2026-05-21 08:57 UTC · model grok-4.3

classification 💻 cs.DL cs.DB
keywords bibliometric rankingjournal classificationconference evaluationopen-source metricscitation normalizationcomputer sciencemedicine
0
0 comments X

The pith

GSR creates a free open-source ranking that places computer science conferences on equal footing with journals.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops the General Science Ranking to address the upcoming closure of a widely used East Asian journal classification system. It constructs rankings for 500 computer science venues and 500 medical journals from freely available citation records. Four measures are combined into a composite score, with conference impact approximated through calibration against a large set of journal papers. Fixed quotas then divide venues into four quality tiers. The resulting system achieves substantial agreement with an established commercial ranking while remaining fully reproducible and accessible without paid subscriptions.

Core claim

The authors define GSR as a framework that scores venues using field-weighted citation impact, two-year impact factor with conference approximations calibrated on 1.41 million journal papers, five-year h-index, and citation compound annual growth rate. Venues are sorted and assigned to Q1 through Q4 using fixed quotas of 50, 50, 100, and the remainder. In computer science the top tier contains 25 conferences and 25 journals, led by NeurIPS, ICCV, ICLR, and CVPR; in medicine the leaders are CA: A Cancer Journal for Clinicians, New England Journal of Medicine, and The Lancet. The method shows 71 percent agreement with JCR in computer science and 84 percent in medicine, with only 1.7 to 2.5 a 2

What carries the argument

The GSR composite score integrates field-weighted citation impact, approximated two-year impact factor, five-year h-index, and citation growth rate, then applies fixed-quota partitioning to assign quality tiers.

Load-bearing premise

The calibration method for estimating two-year impact factors for conferences from a dataset of 1.41 million journal papers accurately captures their relative impact without introducing systematic bias.

What would settle it

A direct comparison of the approximated two-year impact factors against actual multi-year citation counts for the same conferences; large systematic discrepancies would show the calibration fails to rank them correctly.

read the original abstract

The academic journal zoning system is central to evaluating research talent, funding, and institutions. The CAS journal partition system, one of East Asia's most widely used tools, will cease operation in March 2026, creating a policy gap. Existing alternatives have major limitations: JCR depends on paid databases and excludes conferences; Scimago/CiteScore relies on Elsevier proprietary data; expert-based rankings such as CCF and CORE lack quantitative foundations and update slowly. This paper proposes the General Science Ranking (GSR), a multidimensional bibliometric framework built entirely on open-source data. GSR covers 500 computer science venues (397 journals and 103 conferences) and 500 medical journals using OpenAlex and Semantic Scholar. Scores combine four indicators: field-weighted citation impact (FWCI), two-year impact factor (IF2), five-year h-index (h5), and citation CAGR. For CS conferences lacking citation time-series data, IF2-approx was estimated from calibration on 1.41 million OpenAlex journal papers. Rankings adopt fixed quotas: Q1 (1-50), Q2 (51-100), Q3 (101-200), and Q4 (201+). All code and data are open source. In CS rankings, conferences and journals each occupy 25 of the top 50 Q1 positions. The leading conferences are NeurIPS, ICCV, ICLR, and CVPR. In medicine, CA: A Cancer Journal for Clinicians ranks first, followed by New England Journal of Medicine and The Lancet. Agreement with JCR Q1 reaches 84 percent in medicine and 71 percent in CS. Sensitivity analysis shows only 1.7 percent to 2.5 percent of CS conferences change partitions, indicating robustness. GSR provides a free, reproducible, field-normalized ranking system covering both journals and conferences, making it suitable for institutional evaluation policies.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper presents the General Science Ranking (GSR), an open-source bibliometric framework for ranking journals and conferences in computer science and medicine. Using data from OpenAlex and Semantic Scholar, it computes four indicators: field-weighted citation impact (FWCI), two-year impact factor (IF2, with an approximation for conferences calibrated on 1.41 million journal papers), five-year h-index (h5), and citation CAGR. Venues are partitioned into quartiles using fixed quotas (Q1: ranks 1-50, Q2: 51-100, etc.). The system covers 500 CS venues (397 journals, 103 conferences) and 500 medical journals. Reported results include 84% agreement with JCR Q1 in medicine and 71% in CS, with conferences and journals each occupying 25 of the top 50 Q1 positions in CS, and sensitivity analysis showing only 1.7-2.5% partition changes.

Significance. Should the calibration of the IF2 approximation prove robust and free of systematic bias, GSR would represent a significant advancement by offering a freely available, reproducible, and field-normalized ranking system that includes both journals and conferences. This addresses critical gaps in existing tools like JCR (paid, conference-excluding) and expert-based rankings (slow to update, non-quantitative). The open-source code and data, along with the unified scale for CS venues, could support institutional evaluation policies effectively.

major comments (1)
  1. [IF2 approximation calibration] The two-year impact factor for the 103 CS conferences is not computed directly but approximated using a calibration model fitted solely to 1.41 million journal papers from OpenAlex. This approach assumes that the relationship between available features and IF2 is the same for conferences and journals. However, conferences typically have steeper early citation curves and distinct self-citation patterns, which could lead to biased estimates. The sensitivity analysis (1.7–2.5% changes) only perturbs downstream thresholds and does not test variations in calibration parameters or training corpus. This undermines the validity of placing conferences and journals on a common scale and affects the reported 71% JCR agreement and the 25-conference Q1 share. Direct validation against conference citation data or cross-validation is needed.
minor comments (2)
  1. [Score combination] The abstract and methods indicate that scores combine the four indicators (FWCI, IF2, h5, citation CAGR), but the exact method of combination—such as weighting, normalization, or aggregation into a single score—is not specified in detail. Clarifying this would improve reproducibility.
  2. [Data sources] While OpenAlex and Semantic Scholar are mentioned, more specifics on data extraction dates, exclusion criteria, or handling of missing citation time-series for conferences would enhance transparency.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and positive assessment of the potential significance of GSR. We address the major comment on the IF2 approximation calibration below, providing an honest account of the data constraints while outlining revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: The two-year impact factor for the 103 CS conferences is not computed directly but approximated using a calibration model fitted solely to 1.41 million journal papers from OpenAlex. This approach assumes that the relationship between available features and IF2 is the same for conferences and journals. However, conferences typically have steeper early citation curves and distinct self-citation patterns, which could lead to biased estimates. The sensitivity analysis (1.7–2.5% changes) only perturbs downstream thresholds and does not test variations in calibration parameters or training corpus. This undermines the validity of placing conferences and journals on a common scale and affects the reported 71% JCR agreement and the 25-conference Q1 share. Direct validation against conference citation data or cross-validation is needed.

    Authors: We acknowledge that the calibration of the IF2 approximation relies on journal data from OpenAlex, as conferences in the dataset lack the complete two-year citation time-series required for direct computation. This introduces an assumption that the feature-to-IF2 relationship generalizes across venue types, and we agree that conferences often exhibit steeper early citation curves and different self-citation behaviors, which could introduce bias. The existing sensitivity analysis examines only threshold perturbations and does not vary calibration parameters or the training corpus. In the revised version, we will add a dedicated subsection on the calibration model, including its features and training procedure; report results from cross-validation performed on the journal corpus to quantify prediction error; and, where feasible, compare approximated values against available citation metrics from Semantic Scholar for a subset of conferences with sufficient data. We will also expand the limitations discussion to address implications for the unified scale, the 71% JCR agreement (which applies to journals), and the 25-conference Q1 share. These changes will provide greater transparency on the approximation's robustness without overstating its equivalence to direct computation. revision: yes

Circularity Check

0 steps flagged

No significant circularity in GSR derivation; all indicators drawn from external open databases

full rationale

The paper's core derivation computes FWCI, IF2, h5, and citation CAGR directly from OpenAlex and Semantic Scholar records for both journals and conferences. The IF2-approx step for the 103 CS conferences applies a calibration model trained on a separate corpus of 1.41 million journal papers; this is a one-directional methodological transfer rather than a self-referential loop, fitted parameter renamed as prediction, or reduction of the final Q1 quotas and 71 % JCR agreement figure to the inputs by construction. No equations or steps in the provided text exhibit the target rankings being equivalent to the calibration dataset itself. The reported sensitivity results and external JCR agreement serve as independent checks, keeping the chain self-contained against the cited open sources.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The framework rests on the accuracy and completeness of OpenAlex and Semantic Scholar for the selected venues plus the validity of the journal-to-conference calibration; no new physical entities are introduced.

free parameters (1)
  • IF2 approximation calibration parameters
    Estimated from 1.41 million OpenAlex journal papers to approximate impact factors for conferences lacking time-series data.
axioms (1)
  • domain assumption OpenAlex and Semantic Scholar citation data are sufficiently complete and unbiased for the 1000 selected venues in CS and medicine.
    The entire ranking depends on these public databases providing accurate field-weighted and time-series citation counts.

pith-pipeline@v0.9.0 · 5887 in / 1432 out tokens · 73605 ms · 2026-05-21T08:57:35.645776+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages · 2 internal anchors

  1. [1]

    Why China needs to review its approach to research evaluation[J]

    Zhao Y. Why China needs to review its approach to research evaluation[J]. Nature, 2025, 641(8062): 283-283

    work page 2025

  2. [2]

    China’s journal ranking system stands up to scrutiny[J]

    Yang L. China’s journal ranking system stands up to scrutiny[J]. Nature, 2025, 643(8072): 638-638

    work page 2025

  3. [3]

    China discontinues prominent journal ranking list[J]

    Liu Z. China discontinues prominent journal ranking list[J]. Nature, 2026, 652(8110): 828-828

    work page 2026

  4. [4]

    A new approach to the metric of journals’ scientific prestige: The SJR indicator[J]

    González-Pereira B, Guerrero-Bote V P, Moya-Anegón F. A new approach to the metric of journals’ scientific prestige: The SJR indicator[J]. Journal of informetrics, 2010, 4(3): 379-391

    work page 2010

  5. [5]

    CiteScore metrics: Creating journal metrics from the Scopus citation index

    James C, Colledge L, Meester W, et al. CiteScore metrics: Creating journal metrics from the Scopus citation index[J]. arXiv preprint arXiv:1812.06871, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  6. [6]

    OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts

    Priem J, Piwowar H, Orr R. OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts[J]. arXiv preprint arXiv:2205.01833, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  7. [7]

    Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies[J]

    Garfield E. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies[J]. Science, 1972, 178(4060): 471-479

    work page 1972

  8. [8]

    Why the impact factor of journals should not be used for evaluating research[J]

    Seglen P O. Why the impact factor of journals should not be used for evaluating research[J]. Bmj, 1997, 314(7079): 497

    work page 1997

  9. [9]

    The journal impact factor: A brief history, critique, and discussion of adverse effects[M]//Springer handbook of science and technology indicators

    Lariviere V, Sugimoto C R. The journal impact factor: A brief history, critique, and discussion of adverse effects[M]//Springer handbook of science and technology indicators. Cham: Springer International Publishing, 2019: 3-24

    work page 2019

  10. [10]

    A simple proposal for the publication of journal citation distributions[J]

    Larivière V, Kiermer V, MacCallum C J, et al. A simple proposal for the publication of journal citation distributions[J]. BioRxiv, 2016: 062109

    work page 2016

  11. [11]

    Journal citation reports[J]

    Krampl A. Journal citation reports[J]. Journal of the Medical Library Association: JMLA, 2019, 107(2): 280

    work page 2019

  12. [12]

    An index to quantify an individual's scientific research output[J]

    Hirsch J E. An index to quantify an individual's scientific research output[J]. Proceedings of the National academy of Sciences, 2005, 102(46): 16569-16572

    work page 2005