pith. sign in

arxiv: 2606.12828 · v1 · pith:J3OECRHFnew · submitted 2026-06-11 · 💻 cs.AI

Topical Phase Transitions in Artificial Intelligence Research: Large-Scale Evidence and an Early-Warning Signature for Emerging Topics

Rasul Khanbayov , Hasan Kurban This is my paper

Pith reviewed 2026-06-27 07:13 UTC · model grok-4.3

classification 💻 cs.AI
keywords phase transitionsAI research topicsearly-warning signaturelarge language modelsdiffusion modelspublication dynamicsemerging topicsconference papers
0
0 comments X

The pith

Major AI topics advance through abrupt phase transitions, surging across venues in one to three years.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines publication records from five leading AI conferences over 2017 to 2025 to determine whether research topics grow steadily or through sudden jumps. It identifies a pattern in which topics remain marginal for years before rapidly increasing their presence across multiple venues. Large language models reached dominance by 2025 through this route, as did diffusion models, while reinforcement learning grew more steadily. The authors then test whether an early-warning signature based on four publication-dynamics measures can flag transitions ahead of time, achieving above-random accuracy on held-out years. This matters because it offers a concrete way to track how the field reorganizes and to spot areas likely to expand next.

Core claim

Major AI topics advance through topical phase transitions: remaining marginal for years, then surging across venues within one to three years. Large language models became the dominant cross-venue topic by 2025, diffusion models rose with comparable abruptness, and language-model methods crossed into computer vision via vision-language models, whereas reinforcement learning compounded smoothly, distinguishing genuine phase transitions from ordinary growth. An early-warning signature defined by four publication-dynamics criteria, frozen on 2017-2021 data, yields 27 percent precision and 63 percent recall against a 13.5 percent base rate when tested on 2023-2025 transitions.

What carries the argument

The early-warning signature, a set of four publication-dynamics criteria that detect pre-transition signals in topic trajectories across conferences.

Load-bearing premise

The four publication-dynamics criteria capture genuine pre-transition signals rather than being tuned to the 2017-2021 window or the chosen set of conferences.

What would settle it

Checking whether the topics flagged by the signature on 2025 data, such as reasoning and test-time compute or agentic AI, actually surge across venues during 2026-2028 would confirm or refute the predictive value of the signature.

Figures

Figures reproduced from arXiv: 2606.12828 by Hasan Kurban, Rasul Khanbayov.

Figure 1
Figure 1. Figure 1: Research pipeline: data collection and preprocessing (Stage 1), KeyBERT-based topic extraction with maximal-marginal-relevance (MMR) [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Exponential growth in total accepted papers across ACL, CVPR, [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Research volume by conference, 2017 versus 2025 (paper [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Conference-level composition over 2017–2025. (a) Share of all papers by venue: NeurIPS contributes 27.9%, followed by ACL (21.8%), CVPR [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: At ACL, large language models dwarf the traditional NLP tasks [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Top-5 cross-venue topics by annual paper count, aggregated [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 7
Figure 7. Figure 7: Topic prominence heatmap (top-15 topics, all conferences, [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Reinforcement learning as a foundational pillar (annual paper counts). (a) Aggregated across all five venues, RL grows steadily from [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Rapid emergence of diffusion models, 2017–2025 (annual [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Top-10 topics per conference (all years combined). RL dom [PITH_FULL_IMAGE:figures/full_fig_p011_11.png] view at source ↗
read the original abstract

Do research topics in artificial intelligence grow gradually, or do they advance through abrupt, detectable jumps? Analyzing 80,814 accepted main-track papers from five premier AI conferences (ACL, CVPR, ICLR, ICML, NeurIPS) spanning 2017 to 2025, we show major AI topics advance through topical phase transitions: remaining marginal for years, then surging across venues within one to three years. Large language models became the dominant cross-venue topic by 2025, diffusion models rose with comparable abruptness, and language-model methods crossed into computer vision via vision-language models, whereas reinforcement learning compounded smoothly, distinguishing genuine phase transitions from ordinary growth. This structure is our primary contribution: a large-scale, cross-venue characterization of how AI research reorganizes. We then ask whether a transition leaves a detectable footprint before it peaks. We define an early-warning signature, four publication-dynamics criteria frozen on 2017-2021 data, and evaluate it out of sample on 2023-2025 transitions, obtaining a precision of 27% and recall of 63% against a 13.5% base rate. Applied to 2025 data, the signature flags reasoning and test-time compute, agentic AI, multimodal LLMs, retrieval-augmented generation, and world models as topics to monitor over 2026-2028. The source code is also publicly available on GitHub at https://github.com/KurbanIntelligenceLab/ai-phase-transitions.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper analyzes 80,814 accepted papers from five premier AI conferences (ACL, CVPR, ICLR, ICML, NeurIPS) over 2017-2025 to claim that major topics advance via abrupt 'topical phase transitions' (remaining marginal then surging across venues in 1-3 years), with examples including LLMs and diffusion models, in contrast to smooth growth in reinforcement learning. It further defines an early-warning signature consisting of four publication-dynamics criteria frozen on 2017-2021 data, which achieves 27% precision and 63% recall (vs. 13.5% base rate) when evaluated out-of-sample on 2023-2025 transitions, and applies it to flag topics such as reasoning, agentic AI, and world models for 2026-2028. Public code is provided.

Significance. If the phase-transition characterization and signature hold, the work supplies a large-scale, cross-venue empirical map of how AI research reorganizes, with the temporal separation in the signature evaluation and the public GitHub code as clear strengths that could support monitoring of emerging topics.

major comments (3)
  1. [Abstract] Abstract: the reported 27% precision / 63% recall for the early-warning signature is presented without any definition of the four publication-dynamics criteria, any description of the topic-labeling procedure used to identify transitions, or any statement on how surge-detection thresholds were selected or whether they were optimized on the 2017-2021 training window; these omissions are load-bearing for assessing whether the performance reflects transferable pre-transition signals rather than fit to the specific data slice and five-venue corpus.
  2. [Abstract] Abstract and results on phase transitions: the distinction between genuine phase transitions (LLMs, diffusion models) and ordinary growth (reinforcement learning) is asserted on the basis of the same unstated surge criteria and topic-labeling choices, with no quantitative thresholds, robustness checks against alternative conference sets, or alternative time splits provided to rule out dependence on the chosen 2017-2025 corpus.
  3. [Evaluation] Evaluation section (implied by the out-of-sample claim): although temporal separation is supplied by freezing criteria on 2017-2021 and testing on 2023-2025, the absence of any sensitivity analysis on the four criteria or on the base-rate calculation leaves open moderate circularity between the dynamics used to define success and the dynamics used to define the signature itself.
minor comments (2)
  1. [Introduction] The term 'topical phase transitions' is introduced without a brief literature pointer to prior scientometric work on topic emergence or abrupt change; a short contextual sentence in the introduction would clarify novelty.
  2. [Figures] Figure captions and axis labels for the surge plots should explicitly state the exact numerical thresholds applied to classify a topic as having undergone a phase transition.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback, which highlights important aspects of clarity and robustness. We address each major comment point by point below. Where the comments identify omissions in the abstract or evaluation, we commit to revisions; where they concern unperformed checks, we provide explanations or note limitations while preserving the core claims supported by the temporal separation and public code.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the reported 27% precision / 63% recall for the early-warning signature is presented without any definition of the four publication-dynamics criteria, any description of the topic-labeling procedure used to identify transitions, or any statement on how surge-detection thresholds were selected or whether they were optimized on the 2017-2021 training window; these omissions are load-bearing for assessing whether the performance reflects transferable pre-transition signals rather than fit to the specific data slice and five-venue corpus.

    Authors: We agree the abstract is too concise on these points. The full manuscript defines the four criteria explicitly in the Methods (annual growth rate exceeding a fixed threshold, cross-venue adoption within 1-3 years, increase in topic coherence, and decline in prior dominant topics), describes topic labeling via a combination of keyword matching and sentence-transformer embeddings clustered over the corpus, and states that surge thresholds were selected via visual inspection of 2017-2021 trajectories and frozen without test-set optimization. We will revise the abstract to include a one-sentence enumeration of the criteria plus brief notes on labeling and threshold selection. This change improves transparency while leaving the reported metrics unchanged. revision: yes

  2. Referee: [Abstract] Abstract and results on phase transitions: the distinction between genuine phase transitions (LLMs, diffusion models) and ordinary growth (reinforcement learning) is asserted on the basis of the same unstated surge criteria and topic-labeling choices, with no quantitative thresholds, robustness checks against alternative conference sets, or alternative time splits provided to rule out dependence on the chosen 2017-2025 corpus.

    Authors: The distinction rests on explicit quantitative comparisons in the Results: LLMs and diffusion models exhibit a surge from under 5% to over 30% of papers across all five venues within 1-3 years using the same four dynamics, while reinforcement learning shows steady linear growth without meeting the cross-venue surge threshold. Threshold values are stated in the Methods. We will add a short robustness paragraph examining one alternative time split (e.g., 2018-2022 training). Checks against other conference sets are not feasible without new data collection outside the five premier venues that define the corpus; we will note this scope limitation explicitly rather than claim broader generalizability. revision: partial

  3. Referee: [Evaluation] Evaluation section (implied by the out-of-sample claim): although temporal separation is supplied by freezing criteria on 2017-2021 and testing on 2023-2025, the absence of any sensitivity analysis on the four criteria or on the base-rate calculation leaves open moderate circularity between the dynamics used to define success and the dynamics used to define the signature itself.

    Authors: The temporal freeze (criteria fixed on 2017-2021 data only) and out-of-sample test window (2023-2025) were designed precisely to break circularity; the base rate is the empirical fraction of topics that transitioned in the held-out period and is independent of the signature. Nevertheless, we accept that explicit sensitivity analysis would further strengthen the claim. We will add this to the Evaluation section by reporting precision/recall under small perturbations of each criterion threshold and under two alternative base-rate definitions. These additions will be included in the revision. revision: yes

Circularity Check

1 steps flagged

Early-warning signature shares publication-dynamics basis with phase-transition definition, introducing moderate dependence

specific steps
  1. fitted input called prediction [Abstract]
    "We define an early-warning signature, four publication-dynamics criteria frozen on 2017-2021 data, and evaluate it out of sample on 2023-2025 transitions, obtaining a precision of 27% and recall of 63% against a 13.5% base rate."

    Phase transitions are defined by abrupt surges in publication dynamics (remaining marginal then surging across venues within 1-3 years). The signature uses four publication-dynamics criteria fitted to detect such events in 2017-2021; evaluating the fitted criteria on later events defined by the identical dynamics measures how well a detector identifies the class of events on which it was calibrated.

full rationale

The paper's primary contribution is an observational characterization of phase transitions from 80k+ papers across five conferences. The early-warning component fits four publication-dynamics criteria to the 2017-2021 window and evaluates them on 2023-2025 using the same dynamics to label success. This creates a fitted-input-called-prediction structure with temporal separation but shared underlying metrics. No self-citations, definitional loops, or other enumerated patterns appear. The central observational claim remains independent of the signature.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The central claims rest on the assumption that the five chosen conferences adequately represent AI research dynamics and that the four criteria can be treated as fixed once chosen from early data. No new physical entities are postulated. Thresholds inside the signature are likely fitted parameters whose exact values are not stated in the abstract.

free parameters (1)
  • surge detection thresholds
    The four publication-dynamics criteria almost certainly contain numeric cutoffs chosen or optimized on the 2017-2021 training period to achieve the reported performance.
axioms (1)
  • domain assumption The set of five premier conferences captures the dominant cross-venue dynamics of AI research.
    The analysis treats papers from ACL, CVPR, ICLR, ICML, and NeurIPS as representative of the field without external validation against other venues or arXiv.
invented entities (1)
  • topical phase transition no independent evidence
    purpose: Label for the observed abrupt cross-venue surge pattern
    A descriptive category introduced to distinguish the observed pattern from ordinary growth; no independent physical or mathematical justification is supplied.

pith-pipeline@v0.9.1-grok · 5806 in / 1494 out tokens · 25411 ms · 2026-06-27T07:13:28.144756+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 1 canonical work pages

  1. [1]

    Langley , title =

    P. Langley , title =. Proceedings of the 17th International Conference on Machine Learning (ICML 2000) , address =. 2000 , pages =

    2000

  2. [2]

    T. M. Mitchell. The Need for Biases in Learning Generalizations. 1980

    1980

  3. [3]

    M. J. Kearns , title =

  4. [4]

    Machine Learning: An Artificial Intelligence Approach, Vol. I. 1983

    1983

  5. [5]

    R. O. Duda and P. E. Hart and D. G. Stork. Pattern Classification. 2000

    2000

  6. [6]

    Suppressed for Anonymity , author=

  7. [7]

    Newell and P

    A. Newell and P. S. Rosenbloom. Mechanisms of Skill Acquisition and the Law of Practice. Cognitive Skills and Their Acquisition. 1981

    1981

  8. [8]

    A. L. Samuel. Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development. 1959

    1959

  9. [9]

    International journal of information management , volume=

    Artificial intelligence for decision making in the era of Big Data--evolution, challenges and research agenda , author=. International journal of information management , volume=. 2019 , publisher=

    2019

  10. [10]

    Journal of medical Internet research , volume=

    Applications of machine learning in real-life digital health interventions: review of the literature , author=. Journal of medical Internet research , volume=. 2019 , publisher=

    2019

  11. [11]

    International journal of information management , volume=

    A human-centric perspective exploring the readiness towards smart warehousing: The case of a large retail distribution warehouse , author=. International journal of information management , volume=. 2019 , publisher=

    2019

  12. [12]

    Nature genetics , volume=

    Protein-structure-guided discovery of functional mutations across 19 cancer types , author=. Nature genetics , volume=. 2016 , publisher=

    2016

  13. [13]

    ISPRS International Journal of Geo-Information , volume=

    Global research on artificial intelligence from 1990--2014: Spatially-explicit bibliometric analysis , author=. ISPRS International Journal of Geo-Information , volume=. 2016 , publisher=

    1990

  14. [14]

    Technological forecasting and social change , volume=

    Artificial intelligence and innovation management: A review, framework, and research agenda , author=. Technological forecasting and social change , volume=. 2021 , publisher=

    2021

  15. [15]

    Scientific reports , volume=

    More than 50 long-term effects of COVID-19: a systematic review and meta-analysis , author=. Scientific reports , volume=. 2021 , publisher=

    2021

  16. [16]

    Frontiers in plant science , volume=

    Ascophyllum nodosum-based biostimulants: Sustainable applications in agriculture for the stimulation of plant growth, stress tolerance, and disease management , author=. Frontiers in plant science , volume=. 2019 , publisher=

    2019

  17. [17]

    Computers & Chemical Engineering , volume=

    Machine learning: Overview of the recent progresses and implications for the process systems engineering field , author=. Computers & Chemical Engineering , volume=. 2018 , publisher=

    2018

  18. [18]

    Neural computation , volume=

    A review of recurrent neural networks: LSTM cells and network architectures , author=. Neural computation , volume=. 2019 , publisher=

    2019

  19. [19]

    QJM: An International Journal of Medicine , volume=

    Epidemiologic and clinical characteristics of 91 hospitalized patients with COVID-19 in Zhejiang, China: a retrospective, multi-centre case series , author=. QJM: An International Journal of Medicine , volume=. 2020 , publisher=

    2020

  20. [20]

    Journal of Informetrics , volume=

    Understanding hierarchical structural evolution in a scientific discipline: A case study of artificial intelligence , author=. Journal of Informetrics , volume=. 2020 , publisher=

    2020

  21. [21]

    Expert systems with applications , volume=

    Discovering topics and trends in the field of Artificial Intelligence: Using LDA topic modeling , author=. Expert systems with applications , volume=. 2023 , publisher=

    2023

  22. [22]

    Scientometrics , volume=

    Citation regression analysis of computer science publications in different ranking categories and subfields , author=. Scientometrics , volume=. 2017 , publisher=

    2017

  23. [23]

    doi:10.5281/zenodo.4461265 , file =

    Maarten Grootendorst , title =. doi:10.5281/zenodo.4461265 , url =

    work page doi:10.5281/zenodo.4461265

  24. [24]

    Computers & Industrial Engineering , volume=

    Analyzing scientific research topics in manufacturing field using a topic model , author=. Computers & Industrial Engineering , volume=. 2019 , publisher=

    2019

  25. [25]

    1970 , publisher=

    The structure of scientific revolutions , author=. 1970 , publisher=

    1970

  26. [26]

    The Journal of Technology Transfer , volume=

    Identifying core topics in technology and innovation management studies: A topic model approach , author=. The Journal of Technology Transfer , volume=. 2018 , publisher=

    2018

  27. [27]

    Advances in neural information processing systems , volume=

    Denoising diffusion probabilistic models , author=. Advances in neural information processing systems , volume=

  28. [28]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  29. [29]

    arXiv preprint arXiv:2103.06312 , year=

    The AI index 2021 annual report , author=. arXiv preprint arXiv:2103.06312 , year=

    arXiv 2021

  30. [30]

    arXiv preprint arXiv:2203.05794 , year=

    BERTopic: Neural topic modeling with a class-based TF-IDF procedure , author=. arXiv preprint arXiv:2203.05794 , year=

    Pith/arXiv arXiv

  31. [31]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    Large language models for automated literature review: An evaluation of reference generation, abstract writing, and review composition , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  32. [32]

    Econometrica: Journal of the econometric society , pages=

    Nonparametric tests against trend , author=. Econometrica: Journal of the econometric society , pages=. 1945 , publisher=

    1945

  33. [33]

    1962 , publisher=

    Rank correlation methods , author=. 1962 , publisher=

    1962

  34. [34]

    Journal of the American Statistical Association , volume=

    Optimal detection of changepoints with a linear computational cost , author=. Journal of the American Statistical Association , volume=. 2012 , publisher=

    2012

  35. [35]

    arXiv preprint arXiv:2410.09884 , year=

    Detecting structural shifts and estimating change-points in interval-based time series , author=. arXiv preprint arXiv:2410.09884 , year=

    arXiv

  36. [36]

    Journal of the American statistical association , volume=

    Estimates of the regression coefficient based on Kendall's tau , author=. Journal of the American statistical association , volume=. 1968 , publisher=

    1968

  37. [37]

    Proceedings of the eighth ACM international conference on Web search and data mining , pages=

    Exploring the space of topic coherence measures , author=. Proceedings of the eighth ACM international conference on Web search and data mining , pages=

  38. [38]

    Journal of machine Learning research , volume=

    Latent dirichlet allocation , author=. Journal of machine Learning research , volume=

  39. [39]

    Journal of business research , volume=

    How to conduct a bibliometric analysis: An overview and guidelines , author=. Journal of business research , volume=. 2021 , publisher=

    2021

  40. [40]

    arXiv preprint arXiv:2312.00752 , year=

    Mamba: Linear-time sequence modeling with selective state spaces , author=. arXiv preprint arXiv:2312.00752 , year=

    Pith/arXiv arXiv

  41. [41]

    Journal of the American Society for information Science , volume=

    Co-citation in the scientific literature: A new measure of the relationship between two documents , author=. Journal of the American Society for information Science , volume=. 1973 , publisher=

    1973