Pith. sign in

REVIEW 4 major objections 8 minor 49 references

The Evolution of IJHCS and CHI: A Quantitative Analysis

T0 review · 4 major / 8 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A bibliometric study of 50 years of IJHCS and CHI shows a stable journal core, a much larger conference, and a nearly frozen country ranking.

desk verdict Useful descriptive scientometrics with a solid geopolitical core, but the 'stable DNA' claim leans on an unvalidated topic classifier that infers broad super-topic labels. read the letter →

arxiv 1908.04088 v1 pith:VSVBCR3I submitted 2019-08-12 cs.HC cs.DL

Andrea Mannocci , Francesco Osborne , Enrico Motta This is my paper · ORCID
classification cs.HCcs.DL
keywords ScienceofScientometricsSpatialBibliographicDataScholarlyHuman-ComputerInteractionIJHCSCHI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to establish a data-driven account of 50 years of human-computer interaction research by tracing everything published in IJHCS (including its predecessor, the International Journal of Man-Machine Studies) and in the CHI conference, plus the papers they cite and that cite them. The authors claim that IJHCS has preserved a stable core of artificial intelligence and HCI topics across five decades, which they call the journal's DNA, while the topics it draws on and speaks to have shifted around that core. They also claim that CHI has grown from a small specialist meeting to more than 1,200 papers a year, that IJHCS cites a broader set of venues over a longer time span than CHI does, and that the country rankings for both venues are highly concentrated and nearly static, with CHI's Spearman rank correlation approaching 0.9. If these claims hold, the field's history looks less like replacement and more like accumulation around a stable core, with access concentrated among a small set of countries.

What carries the argument

The analysis rests on three instruments. One is a large, openly licensed scholarly metadata graph that supplies the publication sets, author affiliations, and citation links; completeness of that graph is the load-bearing assumption behind every computed ranking and ratio. Another is a large taxonomy of computer science research topics together with an automated classifier that tags each paper by matching n-grams from titles and abstracts to topic labels and then lifts broader super-topics such as HCI and artificial intelligence. The third is a set of quantitative measures: Spearman's rank correlation for country-ranking stability, contribution counts for geopolitical presence, and the knowledge-debit ratio, defined as the number of citing contributions a country makes toward a venue divided by the number of cited-by contributions it receives. Together these instruments convert raw metadata into the paper's descriptive claims.

What would settle it

Recompute the country rankings and knowledge-debit values on a manually verified subset of, say, five thousand IJHCS and CHI papers whose affiliations and references are checked against publisher records; if the Spearman rho values for country rankings drop below 0.7 or the top-10 country lists change substantially, the claim of a static, concentrated geopolitical structure would fail.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that IJHCS's intellectual identity has remained remarkably stable: artificial intelligence, HCI, and knowledge-based systems were core topics in the 1969-1988 period and remain core today, with HCI and user interfaces strengthening over the last decade. CHI, in contrast, expanded dramatically in output but became more self-referential and more concentrated in its referencing, with CHI authors mostly citing recent CHI, UIST, and CSCW papers and with the yearly correlation of country rankings climbing toward 0.9. The paper also introduces a knowledge-debit measure: when a country's papers cite a venue more often than the venue's papers cite that country, the country accumulates a knowledge debit toward the venue. The results show that countries in Asia, South America, and the Middle East carry large debits toward both IJHCS and CHI, indicating participation in the conversation without reciprocal influence.

Load-bearing premise

The central premise is that the scholarly metadata graph's records of which papers belong to each venue, which authors and countries are listed, and which citation links exist are complete and accurate enough over five decades that the rankings and ratios reflect the real field rather than metadata gaps.

Editorial extensions

If this is right

  • If IJHCS's core is stable, the journal's future is plausibly continuous with its past: AI and HCI remain anchors, and new topics are absorbed around them rather than replacing them.
  • If CHI's country rankings really hover near 0.9, then efforts to increase geographic diversity face a structural headwind, not simply a pipeline problem.
  • If knowledge-debit imbalances are real, then institutions in high-debit countries are more likely to appear as citing partners than as first authors, suggesting a specific form of unequal participation.
  • If CHI's referencing is short-memory and concentrated in a few venues, then citation-based assessments of the field will systematically favor recent work and underrepresent the older literature that IJHCS-style venues still engage with.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The stable-DNA claim could be tested more strictly by applying topic models directly to full texts; the paper's classifier relies on n-gram similarity to a fixed taxonomy, which may miss conceptual continuity expressed in changing vocabulary.
  • The journal-versus-conference contrast in citation span may be a general pattern rather than an IJHCS/CHI quirk; comparing similar journal-conference pairs in other fields would show whether long reference memories are intrinsic to journals or particular to this community.
  • The knowledge-debit measure could be used as a policy instrument: recomputing it after targeted calls, mentorship programs, or selection changes would yield a quantitative test of whether access interventions alter reciprocal citation flows.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper presents a quantitative, macro-level history of the International Journal of Human-Computer Studies (IJHCS) and the CHI conference, using Microsoft Academic Graph (MAG) metadata over 1969–2018 (CHI from 1982). It combines three analyses: (i) a scientometric description of publication counts, top institutions, cited/citing venues, and reference-memory patterns; (ii) a geopolitical analysis using author affiliations, country rankings with Spearman's rho, first-author patterns, and a proposed "knowledge debit" ratio; and (iii) a research-topic analysis using the Computer Science Ontology (CSO) and the CSO Classifier to tag papers with topics and identify rising trends in 2009–2018. The paper's central descriptive claims are that IJHCS has retained a stable core of AI and HCI topics ("the DNA of the journal has not changed much"), that CHI has grown from a small meeting to over 1,200 papers per year, and that both venues show highly concentrated, slowly changing geopolitical structures. The authors make their data and code publicly available, and the paper is framed as a contribution to the IJHCS 50th-anniversary special issue.

Significance. If the empirical claims hold, the paper provides a useful, reproducible descriptive account of five decades of HCI-related research, with particular value in its longitudinal scope and its parallel treatment of a journal and a conference. Its strengths are the public release of datasets and code, the use of a transparent extraction pipeline from MAG, and the explicit research questions. The citation and geopolitical analyses follow established methods in spatial scientometrics and appear largely sound. The main risk is in the topic-analysis leg: the classifier is developed and maintained by the same research group, the 0.94 similarity threshold is carried over from prior work without validation on historical IJHCS text, and the reported paper-level topic counts include inferred super-topics. Because the "stable DNA" claim is drawn directly from those high-level topic percentages, the paper's central interpretive claim is not yet fully supported. With additional validation and better uncertainty reporting, the paper would be a valuable historical record for the HCI and science-of-science communities.

major comments (4)
  1. [Section 2.5.2 and Section 3.3] The "DNA of the journal has not changed much" claim (Section 5, based on Figures 12–13) rests on high-level topic percentages, but the CSO Classifier enriches every directly matched topic with all of its super-topics via superTopicOf, yielding an average of 13.9 topics per paper. The paper never reports what fraction of the AI and HCI tags are direct matches versus transitive super-topic inferences, nor does it validate the classifier on historical IJHCS texts where missing abstracts (acknowledged in Section 2.2) make title-only classification more likely. Without separating direct matches from inferred ones, the near-constant AI and HCI percentages could be an artifact of CSO's transitive closure rather than evidence of editorial continuity. I recommend reporting direct-match and inferred-topic percentages separately and validating the classifier against a manually labeled sample stratified by decade.
  2. [Section 2.4, Eq. (1)] The knowledge debit formula is ambiguous: the numerator and denominator are written as "contributions_citing" and "contributions_cited_by," but it is not clear whether these count contributions by authors from country c in papers that cite the venue, contributions of papers published in the venue that cite country c, or some other combination. The surrounding text describes an imbalance between citing a venue and being cited by it, so the formal definition should be spelled out with explicit sets. In addition, the paper does not state how zero denominators (countries that are never cited by the venue) are handled beyond being colored black, nor how missing affiliation data in MAG affect the ratio. This metric is used to support the "knowledge generation is confined to a small number of countries" claim, so its definition should be precise and its sensitivity to missing data assessed.
  3. [Section 2.2 and Section 3.2] The geopolitical rankings and citation-flow results depend entirely on MAG's affiliation fields and citation lists, but the paper acknowledges only qualitatively that "some publications in MAG may lack some of these properties." No coverage statistics are reported, such as the fraction of IJHCS and CHI papers with at least one affiliation ID per decade, or the fraction of cited references with resolved venue and country information. Since the Spearman rho values (near 0.9 for CHI) and the country rankings are central to the "closed to newcomers" claim, the authors should quantify how much missingness varies over time and whether the trend toward higher rho could reflect improved metadata coverage in later years rather than a genuinely more static landscape.
  4. [Section 2.5.3 and Figures 14–16] The rising-topic analysis uses post hoc groupings by 2018 publication counts (e.g., >=60, >=20, >=10, >=5 for CHI), then selects 10 topics per group after "reviewing the resulting lists with domain experts" and discarding redundant topics. The criteria and the identities of the experts are not described, and the reported counts for IJHCS are very small (e.g., topics with 5–10 papers in 2018). No uncertainty intervals or significance tests are provided, so the upward trends in Figures 14–16 may not be robust to a few reclassifications or to MAG metadata noise. I recommend reporting the full selection procedure, the completeness of the topic lists, and either confidence intervals from a resampling procedure or at least the raw counts and total paper counts per year.
minor comments (8)
  1. [Figure 1 caption] The caption contains a typo: "IIJHCS" should be "IJHCS."
  2. [Figure 12 caption] The caption states "Main research topics in IJHCS during the 1969-2018 period," but the surrounding text and the figure title indicate the intended period is 1969–1988; please correct the caption.
  3. [Figure 16 and Section 3.3] The phrase "Virtual Words" appears in both the figure and the text; this should read "Virtual Worlds."
  4. [Section 2.4] The term "self-sustaining countries" is used in Figure 8 but not defined in the text; please define it explicitly as countries with at least five papers whose authors are all from the same country.
  5. [Section 2.2] The paper does not state the date or version of the MAG snapshot used. Since MAG is updated over time and the analysis is meant to be reproducible, the exact snapshot and download date should be reported.
  6. [Section 2.5.2] The sentence "The CSO Classifier ... was shown to generate excellent results [10, 15]" cites only the authors' own prior work. Please provide external validation or a clear statement of the classifier's performance on a held-out benchmark relevant to this dataset.
  7. [Section 3.1, Figures 5a–5d] The reference-memory heatmaps would benefit from a color scale and a description of how the number of citations is normalized (if at all), because the claim that IJHCS has a "broader attention span" depends on how the heatmaps are read.
  8. [Section 5] The concluding sentence describing the story of IJHCS as "amazing vision, sustained excellence, and great success" is celebratory rather than analytical; consider softening this to match the paper's otherwise descriptive tone.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the topic-stability result is a disclosed measurement, not an artifact of the ontology by construction.

full rationale

The derivation chain is self-contained in the relevant sense. The descriptive conclusions (geopolitical concentration, citation-memory differences, stable AI/HCI core) are computed from MAG metadata and the CSO classifier; none of these conclusions is used to define the inputs or to fit a parameter that is then renamed as a prediction. The only quasi-circular candidate is the topic-stability claim in Section 5, which rests on CSO super-topic inference in Section 2.5.2: a paper matching any AI sub-topic is also tagged 'Artificial Intelligence', and the paper does not separate direct from inferred labels. However, this is a disclosed and uniformly applied classification rule, not a mathematical identity: the percentages do change across periods (HCI rises from 53.6% to 77.8%; Expert Systems disappears), so the measurement is sensitive to corpus content rather than forced by the ontology. The self-citations to CSO, the CSO classifier, and EDAM are tool citations with released code and prior evaluation; they do not import the paper's target result. MAG field-missingness is a data-quality limitation, not a circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The analysis rests on two main external inputs: MAG data and the CSO ontology/classifier. The grouping thresholds and expert selection are analyst choices that affect which trends are presented, and they should be treated as free parameters rather than fixed constants.

free parameters (4)
  • Levenshtein similarity threshold = 0.94
    Set empirically in [16] and reused here; controls which n-grams are matched to CSO topics and therefore shapes the entire topic analysis.
  • Topic group magnitude thresholds = 10, 5 (IJHCS); 60, 20, 10, 5 (CHI)
    Publications-in-2018 thresholds chosen by the authors to split topics into magnitude groups for trend selection.
  • Selection of 10 topics per group = 10 per group
    Domain experts selected 10 topics per magnitude group after discarding redundant topics, a subjective filter on the reported trends.
  • Trend window = 2009-2018
    The paper focuses on the last decade, which is a defensible but arbitrary choice; other windows would give different trend lists.
assumptions (4)
  • domain assumption MAG provides a complete and accurate representation of IJHCS and CHI publications, affiliations, and citations.
    All analyses depend on this; the paper notes that fields are sometimes missing (Section 2.2).
  • domain assumption CSO is a valid and sufficiently complete taxonomy of computer science research topics.
    Topic classification is based entirely on CSO labels (Section 2.5.1).
  • domain assumption The CSO classifier reliably labels papers across the entire 50-year period.
    The paper cites a previous validation in [10] but does not validate on a sample of this dataset; older historical metadata, such as abstracts, may be sparser.
  • domain assumption Author contribution counts and first-author position reflect credit and leadership.
    Used for geopolitical counts and first-author analysis; this is a standard but assumed convention in scientometrics.
invented entities (1)
  • Knowledge debit
    purpose: Quantifies the ratio of citations a country gives to a venue versus citations received from that venue.
    New metric introduced in Section 2.4; it is a definitional index, not an observed quantity, and its interpretation depends on citation data quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Evolution of IJHCS and CHI: A Quantitative Analysis." pith.science (2026). https://pith.science/paper/VSVBCR3I

@misc{pith2026190804088,
  author       = {Pith},
  title        = {Pith review of: The Evolution of IJHCS and CHI: A Quantitative Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSVBCR3I}},
  note         = {Machine review of arXiv:1908.04088}
}
read the original abstract

In this paper we focus on the International Journal of Human-Computer Studies (IJHCS) as a domain of analysis, to gain insights about its evolution in the past 50 years and what this evolution tells us about the research landscape associated with the journal. To this purpose we use techniques from the field of Science of Science and analyse the relevant scholarly data to identify a variety of phenomena, including significant geopolitical patterns, the key trends that emerge from a topic-centric analysis, and the insights that can be drawn from an analysis of citation data. Because the area of Human-Computer Interaction (HCI) has always been a central focus for IJHCS, we also include in the analysis the CHI conference, which is the premiere scientific venue in HCI. Analysing both venues provides more data points to our study and allows us to consider two alternative viewpoints on the evolution of HCI research.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages

  1. [1]

    Fortunato, S., Bergstrom, C.T., Bör ner, K., Evans, J.A., Helbing, D., Milojević, S., Petersen, A.M., Radicchi, F., Sinatra, R., Uzzi, B., Vespignani, A., Waltman, L., Wang, D., Barabási, A.-L.: Science of science. Science. 359, (2018)

    work page 2018

  2. [2]

    (Paul), Wang, K.: An Overview of Microsoft Academic Service (MAS) and Applications

    Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. (Paul), Wang, K.: An Overview of Microsoft Academic Service (MAS) and Applications. In: Proceedings of the 24th International Conference on World Wide Web - WWW ’15 Companion. pp. 243 –246 (2015)

    work page 2015

  3. [3]

    Data Science

    Mannocci, A., Osborne, F., Motta, E.: Geographical trends in academic conferences: an analysis on authors’ affiliations. Data Science. (2019)

    work page 2019

  4. [4]

    In: International Semantic Web Conference 2018

    Salatino, A.A., Thanapalasingam, T., Mannocci, A., Osborne, F., Motta, E.: The Computer Science Ontology: A Large -Scale Taxonomy of Research Areas. In: International Semantic Web Conference 2018. pp. 187 –205. Springer, Cham, Monterey, USA (2018)

    work page 2018

  5. [5]

    Journal of Informetrics

    Frenken, K., Hardeman, S., Hoekman, J.: Spatial scientometrics: Towards a cumulative research program. Journal of Informetrics. 3, 222–232 (2009)

    work page 2009

  6. [6]

    Presented at the January 31 (2019)

    La Bruzzo, S., Manghi, P., Mannocci, A.: OpenAIRE’s DOIBoost - Boosting Crossref for Research. Presented at the January 31 (2019)

    work page 2019

  7. [7]

    In: Smart Cities Conference (ISC2), 2016 IEEE International

    Daga, E., d’Aquin, M., Adamou, A., Motta, E.: Addressing exploitability of smart city data. In: Smart Cities Conference (ISC2), 2016 IEEE International. pp. 1–6 (2016)

    work page 2016

  8. [8]

    The American journal of psychology

    Spearman, C.: The proof and measurement of association between two things. The American journal of psychology. 15, 72–101 (1904)

    work page 1904

Show all 49 references
  1. [9]

    PLoS Biology

    Tscharntke, T., Hochberg, M.E., Rand, T.A., Resh, V.H., Krauss, J.: Author Sequence and Credit for Contributions in Multiauthored Publications. PLoS Biology. 5, e18 (2007)

    2007

  2. [10]

    Data Science

    Osborne, F., Muccini, H., Lago, P., Motta, E.: Reducing the Effort for Systematic Reviews in Software Engineering. Data Science. (2019)

    2019

  3. [11]

    In: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering

    Petersen, K., Feldt, R., Mujtaba, S., Mattsson, M.: Systematic Mapping Studies in Software Engineering. In: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering. pp. 68 –77. British Computer Society, Swinton, UK, UK (2008)

    2008

  4. [12]

    Journal of Systems and Software

    Vale, T., Crnkovic, I., de Almeida, E.S., da Mota Silveira Neto, P.A., Cavalcanti, Y.C., de Lemos Meira, S.R.: Twenty -eight years of component -based software engineering. Journal of Systems and Software. 111, 128–148 (2016)

    2016

  5. [13]

    In: International Semantic Web Conference 2015

    Osborne, F., Motta, E.: Klink-2: Integrating Multiple Web Sources to Generate Semantic Topic Networks. In: International Semantic Web Conference 2015. pp. 408 –424. Springer, Cham, Bethlehem, USA. (2015)

    2015

  6. [14]

    Kirrane, S., Sabou, M., Fernández, J.D., Osborne, F., Robin, C., Buitela ar, P., Motta, E., Polleres, A.: A decade of Semantic Web research through the lenses of a mixed methods approach. (2019)

    2019

  7. [15]

    In: TPDL 2019: 23rd International Conference on Theory and Practice of Digital Libraries (2019)

    Salatino, A.A., Osborne, F., Motta, E.: The CSO Classifier: Ontology-Driven Detection of Research Topics in Scholarly Articles. In: TPDL 2019: 23rd International Conference on Theory and Practice of Digital Libraries (2019)

    2019

  8. [16]

    In: In International Semantic Web Conference (P&D/Industry/BlueSky)

    Salatino, A.A., Thanapalasingam, T., Mannocci, A., Osborne, F., Motta, E.: Classifying research papers with the computer science ontology. In: In International Semantic Web Conference (P&D/Industry/BlueSky). CEUR Workshop Proceedings, vol. 2180. (2018)

    2018

  9. [17]

    In: International Semantic Web Conference

    Osborne, F., Salatino, A., Birukou, A., Motta, E.: Automatic classification of springer nature proceedings with smart topic miner. In: International Semantic Web Conference

  10. [18]

    In: International Semantic Web Conference 2018

    Thanapalasingam, T., Osborne, F., Motta, E.: Ontology -Based Recommendation of Editorial Products. In: International Semantic Web Conference 2018. , Monterey, CA (USA). (2018)

    2018

  11. [19]

    In: Proceedings of the Knowledge Capture Conference (2017)

    Osborne, F., Man nocci, A., Motta, E.: Forecasting the Spreading of Technologies in Research Communities. In: Proceedings of the Knowledge Capture Conference (2017)

    2017

  12. [20]

    In: The Semantic Web: Trends and Challenges

    Osborne, F., Scavo, G., Motta, E.: Identifying diachronic topic -based research communities by clustering shared research trajectories. In: The Semantic Web: Trends and Challenges. pp. 114--129. Springer International Publishing (2014)

    2014

  13. [21]

    In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (2018)

    Salatino, A.A., Osborne, F., Motta, E.: AUGUR: Forecasting the Emergence of New Research Topics. In: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries (2018)

    2018

  14. [22]

    Journal of Knowledge Management

    Serenko, A., Bontis, N., Booker, L., Sadeddin, K., Hardie, T.: A scientometric analysis of knowledge management and intellectual capital academic literature (1994‐2008). Journal of Knowledge Management. 14, 3–23 (2010)

    2010

  15. [23]

    Scientific Reports

    Pan, R.K., Kaski, K., Fortunato, S.: World citation and collaboration networks: Uncovering the role of geography in science. Scientific Reports. 2, (2012)

    2012

  16. [24]

    Expert Opinion on Biological Therapy

    Chen, C., Hu, Z., Liu, S., Tseng, H.: Emerging trends in regenerativ e medicine: a scientometric analysis in CiteSpace. Expert Opinion on Biological Therapy. 12, 593–608 (2012)

    2012

  17. [25]

    Psychological Reports

    Voracek, M., Loibl, L.M.: Scientometric Analysis and Bibliography of Digit Ratio (2D:4D) Research, 1998–2008. Psychological Reports. 104, 922–956 (2009)

    2009

  18. [26]

    IEEE Transactions on Cloud Computing

    Heilig, L., Vob, S.: A Scientometric Analysis of Cloud Computing Literature. IEEE Transactions on Cloud Computing. 2, 266–278 (2014)

    2014

  19. [27]

    In: Proceeding CHI ’14 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

    Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., Hosio, S., Kostakos, V.: CHI 1994 -2013: Mapping Two Dec ades of Intellectual Progress through Co -word Analysis. In: Proceeding CHI ’14 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. , Toronto, Ontar...

    2014

  20. [28]

    Scientometrics

    Mubin, O., Al Mahmud, A., Ahmad, M.: HCI down under: reflectin g on a decade of the OzCHI conference. Scientometrics. 112, 367–382 (2017)

    2017

  21. [29]

    In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17

    Marshall, J., Linehan, C., Spence, J., Rennick Egglestone, S.: Throwaway Citation of Prior Work Creates Risk of Bad HCI Research. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17. pp. 827 –

    2017

  22. [30]

    Presented at the (2017)

    Koumaditis, K., Hussain, T.: Human Computer Interaction Research Through the Lens of a Bibliometric Analysis. Presented at the (2017)

    2017

  23. [31]

    In: People and Computers XIX — The Bigger Picture

    Chen, C., Panjwani, G., Proctor, J., Allendoerfer, K., Aluker, S., Sturtz, D., Vukovic, M., Kuljis, J.: Visualizing the Evolution of HCI. In: People and Computers XIX — The Bigger Picture. pp. 233–250. Springer London, London (2006)

    2006

  24. [32]

    -W., Chen, D

    Huang, M.-H., Chang, H. -W., Chen, D. -Z.: The trend of concentration in scientific research and technological innovation: A reduction of the predominant role of the US in world research & technology. Journal of Informetrics. 6, 457–468 (2012)

    2012

  25. [33]

    King, D.A.: The scientific impact of nations, (2004)

    2004

  26. [34]

    May, R.M.: The scientific wealth of nations. Science. 275, 793–796 (1997)

    1997

  27. [35]

    Journal of the American Society for Information Scie nce and Technology

    He, Z.-L.: International collaboration does not have greater epistemic authority. Journal of the American Society for Information Scie nce and Technology. 60, 2151 –2164 (2009)

    2009

  28. [36]

    Journal of the American Society for Information Science and Technology

    Sin, S.-C.J.: International coauthorship and citation impact: A bibliometric study of six LIS journals, 1980--2008. Journal of the American Society for Information Science and Technology. 62, 1770–1783 (2011)

    2011

  29. [37]

    hot regions

    Bornmann, L., Waltman, L.: The detection of “hot regions” in the geography of science—A visualization approach by using density maps. Journal of informetrics. 5, 547–553 (2011)

    2011

  30. [38]

    Journal of Informetrics

    Bornmann, L., Stefaner, M., de Moya Anegón, F., Mutz, R.: What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused institutions worldwid...

    2014

  31. [39]

    British Journal of Educational Technology

    Hung, J.: Trends of e -learning research from 2000 to 2008: Use of text mining and bibliometrics. British Journal of Educational Technology. 43, 5–16 (2012)

    2012

  32. [40]

    Journal of Business Chemistry

    Woodson, T.S.: Research Inequality in Nanomedicine. Journal of Business Chemistry. 9, 133–146 (2012)

    2012

  33. [41]

    In: Measuring scholarly impact

    Frenken, K., Hoekman, J.: Spatial scientometrics and scholarly impact: A review of recent studies, tools, and methods. In: Measuring scholarly impact. pp. 127 –146. Springer (2014)

    2014

  34. [42]

    20, 1898–1917 (2015)

    Octaviano, F.R., Felizardo, K.R., Maldonado, J.C., Sandra C P: Semi-automatic selection of primary studies in systematic literature reviews: is it reasonable? Empirical Software Engineer. 20, 1898–1917 (2015)

    2015

  35. [43]

    In: Proceedings of the 11th {ACM/IEEE} International Symposium on Empirical Software Engineering and Measurement

    Mourão, E., Kalinowski, M., Murta, L., Mendes, E., Wohlin, C.: Investigating the Use of a Hybrid Sea rch Strategy for Systematic Reviews. In: Proceedings of the 11th {ACM/IEEE} International Symposium on Empirical Software Engineering and Measurement. pp. 193–198. IEEE Press (2017)

    2017

  36. [44]

    Empirical Software Engineer

    Kuhrmann, M., Fernández, D.M., Daneva, M.: On the pragmatic design of literature studies in software engineering: an experience -based guideline. Empirical Software Engineer. 1–40 (2017)

    2017

  37. [45]

    In: Pr oceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering

    Ros, R., Bjarnason, E., Runeson, P.: A Machine Learning Approach for Semi-Automated Search and Selection in Literature Studies. In: Pr oceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. pp. 118–127. ACM (2017)

    2017

  38. [46]

    Advanced Engineering Informatics

    de Almeida Biolchini, J.C., Mian, P.G., Natali, A.C.C., Conte, T.U., Travassos, G.H.: Scientific research ontology to support systematic review in software engineering. Advanced Engineering Informatics. 21, 133–151 (2007)

    2007

  39. [47]

    In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012)

    Yueming Sun, Ye Yang, He Zhang, Wen Zhang, Qing Wang: Towards evidence -based ontology for supporting systematic literature review. In: 16th International Conference on Evaluation & Assessment in Software Engineering (EASE 2012). pp. 171 –175. IET (2012)

    2012

  40. [836]

    ACM Press, New York, New York, USA (2017)

    2017

  41. [2016]

    Springer, Kobe, Japan. (2016)

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.