Comparison of research productivity of Italian and Norwegian professors and universities

Ciriaco Andrea D'Angelo; Dag W. Aksnes; Giovanni Abramo

arxiv: 1907.02043 · v1 · pith:KPGEI36Mnew · submitted 2019-07-03 · 💻 cs.DL

Comparison of research productivity of Italian and Norwegian professors and universities

Giovanni Abramo , Dag W. Aksnes , Ciriaco Andrea D'Angelo This is my paper

Pith reviewed 2026-05-25 09:20 UTC · model grok-4.3

classification 💻 cs.DL

keywords research productivityFSS indicatorinternational comparisonprofessors performanceItalyNorwayuniversity rankingbibliometrics

0 comments

The pith

Italian and Norwegian professors show similar average research productivity but different distributions by field and tail concentration.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends the use of the FSS research efficiency indicator from Italy to a comparison with Norway for the period 2011-2015. It develops a methodology for a common field classification scheme to handle differences in national academic systems and limited comparable data. The central finding is that average performances of professors are similar across the two countries, but Norwegian professors are more concentrated in the tails of the distribution. Norway performs notably higher in Mathematics and Earth and Space Sciences, whereas Italy shows higher performance in Biomedical Research and Engineering. This comparison provides a way to benchmark research productivity across borders using a consistent metric.

Core claim

This is the first ever attempt of application in a country other than Italy of a research efficiency indicator (FSS), to assess and compare the performance of professors and universities, within and between countries. A special attention has been devoted to the presentation of the methodology developed to set up a common field classification scheme of professors, and to overcome the limited availability of comparable input data. Results of the comparison between countries, carried out in the 2011-2015 period, show similar average performances of professors, but noticeable differences in the distributions, whereby Norwegian professors are more concentrated in the tails. Norway shows notable更高

What carries the argument

The FSS research efficiency indicator applied with a custom common field classification scheme to enable cross-country comparison despite data limitations.

If this is right

Average research productivity levels are comparable between Italian and Norwegian professors.
Performance distributions differ, with greater concentration at extremes in Norway.
Specific fields show national advantages: Norway in Mathematics and Earth and Space Sciences, Italy in Biomedical Research and Engineering.
The same approach allows comparison of universities as well as individual professors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The methodology could enable similar comparisons with other European countries facing data harmonization challenges.
Field-specific differences may reflect underlying variations in research funding or institutional focus not directly measured here.
Greater tail concentration in Norway might suggest more variable career outcomes or selection processes in academia.

Load-bearing premise

The custom methodology for creating a common field classification scheme and overcoming limited comparable input data produces measurements that are truly equivalent across the two national systems.

What would settle it

Finding that an alternative field classification scheme or different handling of input data changes the relative performances or distributions between Italy and Norway.

read the original abstract

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper extends the FSS indicator to Norway for the first time and reports similar professor averages with Norway more spread in the tails, but the cross-country claims rest on custom harmonization steps that lack shown validation.

read the letter

The main new element here is applying the FSS productivity measure outside Italy for the first time, using 2011-2015 data to compare professors and universities in Italy and Norway. The results show roughly equal average performance but Norway with more professors at both high and low ends of the distribution, plus some field reversals where Norway leads in math and earth sciences while Italy leads in biomed and engineering. The authors also describe steps to build a shared field classification and adjust for uneven input data availability between the two countries. That practical work on making the metrics line up is the part that could be useful to others doing similar comparisons. The weakness is that the harmonization choices get no sensitivity tests or external checks in the description, so the reported differences in tails and fields could trace back to how the fields were grouped or inputs completed rather than actual performance gaps. The abstract flags these steps as necessary but does not show how they were tested. This paper is mainly for people who track national research systems and bibliometric indicators. It supplies one concrete data point on two mid-sized European countries. The methods need referee scrutiny on the harmonization details before the distributional claims can be taken as solid, so it should go to peer review rather than a desk reject.

Referee Report

2 major / 2 minor

Summary. The manuscript applies the Fractional Scientific Strength (FSS) indicator for the first time outside Italy to compare research productivity of professors and universities in Italy and Norway during 2011-2015. It details a custom methodology for creating a common field classification scheme across the two systems and for harmonizing limited comparable input data. Key findings are similar average performances between countries, but with Norwegian professors more concentrated in the tails of the distribution; Norway outperforms in Mathematics and Earth and Space Sciences while Italy does so in Biomedical Research and Engineering.

Significance. If the bespoke field classification and input harmonization procedures are shown to produce truly commensurable metrics, the work supplies a rare consistent cross-country benchmark using the FSS indicator and highlights distributional and field-specific patterns that could inform comparative research policy. The explicit attention to methodological harmonization is a constructive contribution when such comparisons are otherwise hindered by national data differences.

major comments (2)

[Methodology] Methodology section (field classification and input harmonization): the central cross-country claims rest on the assumption that the authors' custom field scheme and data-completion procedures yield equivalent metrics, yet no validation against external standards, no sensitivity tests on classification thresholds or imputation choices, and no robustness checks are reported; without these the observed tail differences and field reversals could be artifacts of the harmonization rather than genuine performance gaps.
[Results] Results section (distributional comparisons): the claim of 'noticeable differences in the distributions' with Norway more concentrated in the tails requires quantitative support (e.g., specific Kolmogorov-Smirnov statistics, variance ratios, or tail-probability tests) that is not supplied; the abstract-level description alone does not establish that the differences survive the harmonization uncertainties.

minor comments (2)

[Abstract] The abstract and introduction should explicitly state the number of professors and universities included in each national sample to allow readers to assess statistical power.
[Methodology] Notation for the FSS indicator and its fractional components should be defined once in a dedicated subsection rather than re-introduced piecemeal.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and the opportunity to improve the manuscript. We respond point by point to the major comments below.

read point-by-point responses

Referee: [Methodology] Methodology section (field classification and input harmonization): the central cross-country claims rest on the assumption that the authors' custom field scheme and data-completion procedures yield equivalent metrics, yet no validation against external standards, no sensitivity tests on classification thresholds or imputation choices, and no robustness checks are reported; without these the observed tail differences and field reversals could be artifacts of the harmonization rather than genuine performance gaps.

Authors: We agree that the absence of explicit validation, sensitivity tests, and robustness checks is a limitation of the submitted manuscript. The paper presents the custom field classification and input harmonization procedures in detail but does not report sensitivity analyses on thresholds or imputation choices. In the revision we will add these checks, including alternative classification thresholds and comparisons of results under different imputation assumptions, to demonstrate that the cross-country patterns are not artifacts. revision: yes
Referee: [Results] Results section (distributional comparisons): the claim of 'noticeable differences in the distributions' with Norway more concentrated in the tails requires quantitative support (e.g., specific Kolmogorov-Smirnov statistics, variance ratios, or tail-probability tests) that is not supplied; the abstract-level description alone does not establish that the differences survive the harmonization uncertainties.

Authors: We acknowledge that the manuscript describes the distributional differences qualitatively without supplying formal statistical tests. The original text notes that Norwegian professors are more concentrated in the tails but does not include Kolmogorov-Smirnov statistics, variance ratios, or tail-probability tests. In the revised manuscript we will add these quantitative measures to the results section to substantiate the claims. revision: yes

Circularity Check

0 steps flagged

No significant circularity; results derive from independent national datasets via applied methodology

full rationale

The paper applies the authors' prior FSS indicator to 2011-2015 data from two separate national systems after developing a bespoke field classification and input harmonization procedure. No step reduces the reported means, tail concentrations, or field-specific reversals to a definitional identity or fitted parameter renamed as prediction. The custom scheme is presented as an enabling step for cross-country commensurability rather than a self-referential construct whose outputs are forced by its own inputs. Claims rest on computed metrics from the harmonized data, not on self-citation chains that substitute for independent verification. This is a standard application of an existing indicator to new data, with the harmonization step being a methodological choice whose validity is a separate correctness question.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no equations or methods section, so the ledger cannot be populated with specific free parameters, axioms, or invented entities; the central claim rests on an unstated assumption that the authors' harmonization procedure yields comparable quantities.

pith-pipeline@v0.9.0 · 5646 in / 1255 out tokens · 28974 ms · 2026-05-25T09:20:43.350234+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/RealityFromDistinction.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Results of the comparison between countries... show similar average performances of professors, but noticeable differences in the distributions... Norway shows notable higher performance in Mathematics and Earth and Space Sciences, while Italy in Biomedical Research and Engineering.
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

the FSS indicator... fractional scientific strength

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.