pith. sign in

arxiv: 2606.21862 · v1 · pith:GDUHYGYVnew · submitted 2026-06-20 · 💻 cs.DL · cs.CL· cs.IR

Research Method Usage across Academic Ages in Library and Information Science: An Empirical Study (1990-2023)

Chengzhi Zhang , Jiayi Hao , Yi Mao This is my paper

Pith reviewed 2026-06-26 11:15 UTC · model grok-4.3

classification 💻 cs.DL cs.CLcs.IR
keywords academic ageresearch methodslibrary and information sciencemethod diversityscholarly careersempirical analysistopic modeling
0
0 comments X

The pith

Mid-career library and information science scholars show the highest diversity in research methods used.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines how researchers in library and information science select methods at different stages of their careers. It finds that method choices shift over time, with a drop in theoretical work and rise in experimental and bibliometric approaches. Mid-career researchers try the most varied methods while late-career ones use fewer kinds. This matters because it can guide younger scholars on what approaches to explore and shows how fields evolve methodologically. The study covers over 26,000 articles from 1990 to 2023.

Core claim

Academic age influences research method usage in library and information science, with mid-career scholars exhibiting the highest method diversity and late-career the lowest, alongside a gradual decline in theoretical methods and growth in experimental and bibliometric ones across the 1990-2023 period in a corpus of 26,677 articles.

What carries the argument

Division of scholars into academic age cohorts through author disambiguation, followed by automated classification of methods in articles to calculate popularity and Shannon diversity indices.

Load-bearing premise

The automated classification of research methods and the calculation of academic ages from author names both work without major errors in the full set of articles.

What would settle it

Manually reclassifying a random sample of articles for method types and recomputing diversity scores by age group to test whether the mid-career peak and late-career drop remain.

read the original abstract

Academic age critically shapes career development, influencing research behavior, output volume, and methodological choices. Analyzing method variation across academic ages offers a new theoretical lens on scholarly evolution and provides early-career researchers with practical guidance for method selection. A corpus of 26,677 articles published 1990-2023 in 14 authoritative Library and Information Science journals was compiled. The CogFT model automatically classified the research methods embedded in these articles, and Top2Vec generated the topic model. This process resulted in a comprehensive dataset linking research methods with topics. Author-name disambiguation enabled calculation of each scholar's academic age. Popularity and Shannon diversity indices for methods, together with topic diversity, were compared across academic age groups. Results reveal dynamic methodological trends: the share of theoretical approaches declined gradually, whereas experimental and bibliometric methods gained ground. Method popularity differs significantly among cohorts. Mid-career scholars exhibit the highest method diversity; late-career scholars the lowest.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper analyzes research method usage across academic ages in Library and Information Science using a corpus of 26,677 articles (1990-2023) from 14 journals. It applies the CogFT model to classify embedded methods, Top2Vec for topic modeling, and author-name disambiguation to compute academic ages, then compares method popularity, Shannon diversity, and topic diversity across age cohorts. Central claims are a gradual decline in theoretical approaches with gains in experimental and bibliometric methods, significant popularity differences by cohort, and peak method diversity among mid-career scholars with the lowest among late-career scholars.

Significance. If the automated classifications and age assignments hold, the work supplies large-scale empirical evidence on how methodological choices evolve with career stage in LIS, offering practical guidance for early-career researchers and documenting field-level shifts. The scale of the corpus and linkage of methods to topics via topic modeling are positive features for an observational study.

major comments (3)
  1. [Methods (CogFT classification)] The Methods section (description of CogFT) provides no training corpus details, held-out accuracy, confusion matrix, precision/recall, or manual verification for the automatic classification of research methods. This is load-bearing because all reported popularity indices, Shannon diversity values, and temporal trends are computed directly from these labels.
  2. [Methods (author disambiguation)] The Methods section (author-name disambiguation) reports no error rates, precision figures, manual checks, or robustness tests for academic-age calculation. This directly threatens the cohort comparisons, as disambiguation errors are known to correlate with name frequency and career stage in bibliometric data.
  3. [Results] Results and discussion sections present the headline findings (mid-career highest diversity, late-career lowest; declining theoretical share) without any sensitivity analysis to plausible classification or disambiguation error rates, leaving the central claims about academic-age effects vulnerable to artifact.
minor comments (2)
  1. [Abstract] The abstract states that 14 journals were used but does not name them; adding the list would improve reproducibility.
  2. [Methods] Notation for the Shannon diversity index and the exact definition of academic-age bins should be stated explicitly in the text rather than assumed from prior work.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major comment below and indicate the revisions we will make to strengthen the paper.

read point-by-point responses

  1. Referee: [Methods (CogFT classification)] The Methods section (description of CogFT) provides no training corpus details, held-out accuracy, confusion matrix, precision/recall, or manual verification for the automatic classification of research methods. This is load-bearing because all reported popularity indices, Shannon diversity values, and temporal trends are computed directly from these labels.

    Authors: We agree that the current Methods section lacks sufficient detail on the CogFT classifier. In the revised manuscript we will add a dedicated subsection describing the training corpus, the model's held-out accuracy, the confusion matrix, precision/recall figures, and any manual verification performed on a sample of classifications. These additions will allow readers to evaluate the reliability of the method labels used for all subsequent analyses. revision: yes

  2. Referee: [Methods (author disambiguation)] The Methods section (author-name disambiguation) reports no error rates, precision figures, manual checks, or robustness tests for academic-age calculation. This directly threatens the cohort comparisons, as disambiguation errors are known to correlate with name frequency and career stage in bibliometric data.

    Authors: We acknowledge that error metrics for author-name disambiguation are not reported. We will expand the Methods section to document the disambiguation procedure, any available precision estimates, manual checks conducted, and robustness tests performed. Where quantitative error rates were not originally computed, we will explicitly note this limitation and discuss its potential implications for the age-cohort comparisons. revision: yes

  3. Referee: [Results] Results and discussion sections present the headline findings (mid-career highest diversity, late-career lowest; declining theoretical share) without any sensitivity analysis to plausible classification or disambiguation error rates, leaving the central claims about academic-age effects vulnerable to artifact.

    Authors: We agree that sensitivity analyses would strengthen confidence in the central claims. In the revised manuscript we will add a new subsection reporting sensitivity analyses that simulate plausible error rates in both CogFT classifications and academic-age assignments, showing the impact on method popularity, Shannon diversity, and the observed mid-career peak. This will directly address concerns about potential artifacts. revision: yes

Circularity Check

0 steps flagged

No circularity: purely observational empirical analysis on external corpus

full rationale

The paper compiles an external corpus of 26,677 articles, applies pre-existing models (CogFT for method classification, Top2Vec for topics, and standard author-name disambiguation) to derive labels and academic ages, then computes descriptive statistics (popularity, Shannon diversity). No equations, derivations, fitted parameters renamed as predictions, self-citations as load-bearing premises, or ansatzes appear in the provided text. All results are direct computations from the data without any reduction to self-defined inputs or self-referential chains. This matches the default case of a non-circular empirical study.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The central findings rest on the unverified performance of the CogFT classifier and the accuracy of academic-age calculation via name disambiguation; no free parameters, axioms, or invented entities are identifiable from the abstract alone.

pith-pipeline@v0.9.1-grok · 5702 in / 1026 out tokens · 26132 ms · 2026-06-26T11:15:07.857927+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

8 extracted references · 7 canonical work pages

  1. [1]

    A., & Solazzi, M

    Abramo, G., D’Angelo, C. A., & Solazzi, M. (2011). The relationship between scientists’ research performance and the degree of internationalization of their research. Scientometrics, 86(3), 629–643. https://doi.org/10.1007/s11192-010-0284-7 Angelov, D. (2020). Top2Vec: Distributed Representations of Topics (No. arXiv:2008.09470). arXiv. https://doi.org/10...

    work page doi:10.1007/s11192-010-0284-7 2011

  2. [2]

    https://doi.org/10.1002/meet.14505001049 Gingras, Y ., Larivière, V ., Macaluso, B., & Robitaille, J. -P. (2008). The effects of aging on researchers’ publication and citati on patterns. PLOS One, 3(12), e4048. https://doi.org/10.1371/journal.pone.0004048 Győrffy, B., Csuka, G., Herman, P., & Török, Á. (2020). Is there a golden age in publication activity...

    work page doi:10.1002/meet.14505001049 2008

  3. [3]

    https://doi.org/10.1016/0306-4573(93)90028-C Järvelin, K., & Vakkari, P. (2022). LIS research across 50 years: Content analysis of journal articles. Journal of Documentation, 78(7), 65–88. https://doi.org/10.1108/JD-03-2021-0062 Jia, T., Wang, D., & Szymanski, B. K. (2017). Quantifying patterns of research-interest evolution. Nature Human Behaviour, 1(4),

    work page doi:10.1016/0306-4573(93)90028-c 2022

  4. [4]

    https://doi.org/10.1038/s41562-017-0078 Kwiek, M., & Roszka, W. (2022). Academic vs. biological age in research on academic careers: A large-scale study with implications for scientifically developing systems. Scientometrics, 127(6), 3543–3575. https://doi.org/10.1007/s11192-022-04363-0 Lou, W., Su, Z., He, J., & Li, K. (2021). A temporally dynamic examin...

    work page doi:10.1038/s41562-017-0078 2022

  5. [5]

    https://doi.org/10.1038/s41467-019-11401-8 Zhang, C., & Tian, L. (2023). Non-synchronism in global usage of research methods in library and information science from 1990 to

    work page doi:10.1038/s41467-019-11401-8 2023

  6. [6]

    https://doi.org/10.1007/s11192-023-04740-3 Zhang, C., Tian, L., & Chu, H. (2023). Usage frequency and application variety of research methods in library and information science: Continuous investigation from 1991 to

    work page doi:10.1007/s11192-023-04740-3 2023

  7. [7]

    https://doi.org/10.1016/j.ipm.2023.103507 Zhang, C., Wang, F., Huang, Y ., & Chang, L

    Information Processing & Management, 60(6), 103507. https://doi.org/10.1016/j.ipm.2023.103507 Zhang, C., Xie, L., & Wei, S. (2025). Gender differences in research topic and method convergence among collaborating scholars in library and information science. Library & Information Science Research, 47(2), 101353. https://doi.org/10.1016/j.lisr.2025.101353 Zh...

    work page doi:10.1016/j.ipm.2023.103507 2023

  8. [8]

    Proportion of research methods over time chart.Mid-career scholars use popular methods in their research. 97 146 351 365 416 422 785 898 919 1148 2183 3842 4028 4313 5461 7126 0 2000 4000 6000 8000 Delphi study Research diary / journal Think aloud protocol Ethnography /field study Observation Focus group Transaction log analysis Webometrics Historical met...

    2000