RECORD
Scores of a specific field-normalized indicator calculated with different approaches of field-categorization: Are the scores different or similar?
Digital Libraries · Submitted 8 Dec 2021
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim The main policy implication is that normalized citation impact scores or rankings based on them should not be compared without deeper knowledge of the classification systems that were used to derive these values or rankings.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Usage of field-normalized citation scores is a bibliometric standard. Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field-classification systems to answer the question if the resulting scores are different or similar. Six field-classification systems were used: three based on citation relations, one on semantic similarity scores (i.e., a topical relatedness measure), one on journal sets, and one on intellectual classifications. Systems based on journal sets and intellectual classifications agree on at least the moderate level. Two out of the three sets based on citation relations also agree on at least the moderate level. Larger differences were observed for the third data set based on citation relations and semantic similarity scores. The main policy implication is that normalized citation impact scores or rankings based on them should not be compared without deeper knowledge of the classification systems that were used to derive these values or rankings.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Usage of field-normalized citation scores is a bibliometric standard. | background 1.00 | 0.03 | 0.01 | asserted 1.00 |
| s1 | Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. | background 0.97 | 0.05 | 0.05 | asserted 1.00 |
| s2 | Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field-classification systems to answer the question if the resulting scores are different or similar. | method 0.64 | 0.97 | 0.03 | asserted 0.86 |
| s3 | Six field-classification systems were used: three based on citation relations, one on semantic similarity scores (i.e., a topical relatedness measure), one on journal sets, and one on intellectual classifications. | method 0.99 | 0.84 | 0.02 | asserted 0.99 |
| s4 | Systems based on journal sets and intellectual classifications agree on at least the moderate level. | result 1.00 | 0.90 | 0.06 | asserted 0.98 |
| s5 | Two out of the three sets based on citation relations also agree on at least the moderate level. | result 1.00 | 0.91 | 0.04 | asserted 1.00 |
| s6 | Larger differences were observed for the third data set based on citation relations and semantic similarity scores. | result 1.00 | 0.92 | 0.05 | asserted 1.00 |
| s7 | The main policy implication is that normalized citation impact scores or rankings based on them should not be compared without deeper knowledge of the classification systems that were used to derive these values or rankings. | interpretation 1.00 | 0.20 | 0.88 | asserted 0.96 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Usage of field-normalized citation scores is a bibliometric standard. Different methods for field-normalization are in use, but also the choice of field-classification system determines the resulting field-normalized citation scores. Using Web of Science data, we calculated field-normalized citation scores using the same formula but different field-classification systems to answer the question if the resulting scores are different or similar. Six field-classification systems were used: three based on citation relations, one on semantic similarity scores (i.e., a topical relatedness measure), one on journal sets, and one on intellectual classifications. Systems based on journal sets and intellectual classifications agree on at least the moderate level. Two out of the three sets based on citation relations also agree on at least the moderate level. Larger differences were observed for the third data set based on citation relations and semantic similarity scores. The main policy implication is that normalized citation impact scores or rankings based on them should not be compared without deeper knowledge of the classification systems that were used to derive these values or rankings.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.