Pith. sign in

REVIEW 4 major objections 4 minor 44 references

THE-Tree: Can Tracing Historical Evolution Enhance Scientific Verification and Reasoning?

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read THE-Tree claims that replacing citation edges with verified causal evolution edges improves graph completion, future-step prediction, and LLM-based paper evaluation; the tree structure itself is the contribution.

desk verdict A substantial benchmark and a sensible pipeline, but the headline gains are compromised by temporal leakage and a non-independent gold standard. read the letter →

arxiv 2506.21763 v2 pith:PA6XQLM6 submitted 2025-06-26 cs.AI

Xin Wang , Jiyao Liu , Yulong Xiao , Junzhi Ning , Lihao Liu , Junjun He , Botian Shi , Kaicheng Yu This is my paper
classification cs.AI
keywords technologyevolutiontreescientificverificationnaturallanguageinferenceMonteCarlosearchcitationnetworksLLMevaluationreasoninggraphcompletion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the missing ingredient for reliable AI evaluation of scientific ideas is a structured record of how ideas actually develop: not who cites whom, but which paper causally enabled which later paper. It introduces THE-Tree, a framework that builds topic-specific trees from survey literature, where nodes are papers and edges are verified evolutionary relationships. The central claim is that these trees outperform traditional citation networks as a substrate: graph completion hit@1 improves by roughly 8 to 14 percent across models, future-step prediction improves by nearly 10 percent, and augmenting LLM reviewers with tree context nearly doubles their accuracy at identifying high-impact papers. If right, the paper would convert scientific history into a computable, checkable resource for novelty and impact assessment.

What carries the argument

The load-bearing object is the THE-Tree edge: a directed, causally labeled paper-to-paper relationship that says one paper logically enables or grounds the next, unlike a raw citation. Edges are produced by the Think-Verbalize-Cite-Verify loop, in which an LLM proposes a next step, summarizes it, cites supporting papers, and then has the link checked by RA-NLI, a retrieval-augmented natural language inference scorer that retrieves evidence passages and judges entailment, contradiction, or neutrality, with an LLM referee for ambiguous neutral cases. The same verified-edge signal is what powers the downstream gains in graph completion, future prediction, and paper evaluation.

What would settle it

Run the future-path and graph-completion experiments with an open model whose training cutoff predates the masked year, and compare hit@1 against the paper's reported results; if the 8 to 14 percent and roughly 10 percent gains shrink toward zero, the reported gains are plausibly retrieval rather than forecasting. A second check is to have domain experts re-score a random sample of RA-NLI-validated edges against the cited source texts to see whether the 95.60 percent accuracy reproduces.

Watch

Extended reading notes

Core claim

The paper's central claim is that a tree of causally verified technology evolution steps, constructed from literature, is a better foundation for scientific verification and reasoning than citation graphs or unstructured surveys. Each node is a paper with an importance score, and each edge encodes a historical, inferential relationship validated for logical and causal coherence. The construction pipeline uses a self-guided temporal Monte Carlo tree search to propose paths, and each proposed edge is checked by RA-NLI, a retrieval-augmented natural language inference mechanism that reads the cited literature and decides whether the claimed advance is entailed, contradicted, or neutral. The paper reports that trees built this way beat citation networks on graph completion, future path prediction, and LLM-based evaluation of conference submissions.

Load-bearing premise

The evaluation assumes that the LLMs used to generate and score 'future' evolutionary steps have not already memorized the papers being predicted, because the models were pretrained on internet-scale data that includes post-split publications; the paper does not control for this.

Editorial extensions

If this is right

  • Linking papers by verified causal relationships rather than citation counts gives LLMs a more informative graph for predicting missing research steps; the authors report hit@1 gains of 8 to 14 percent over citation graphs across multiple models.
  • Future development forecasting improves: THE-Tree raises hit@1 by nearly 10 percent and improves hit@3, hit@5, and mean rank against citation-only graphs.
  • Augmenting LLM reviewers with evolutionary context improves assessment of conference papers, including a near-doubling of accuracy for identifying oral- and spotlight-level papers in the authors' 2024 conference evaluation experiments.
  • The RA-NLI verify step reduces fact-missing rates to 4.75 percent and reaches 95.60 percent accuracy on the 71k fact-verification benchmark, compared with 40 to 69 percent fact-missing rates for standalone LLMs.
  • The released benchmark of 88 trees and 71k fact verifications over 27k papers gives other researchers a testbed for causality-aware scientific reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the temporal-integrity concern is addressed and the gains survive, THE-Tree-style causal links could become a standard evaluation layer for AI-generated hypotheses, replacing 'does it cite the right papers' with 'does it extend a verified line of reasoning.'
  • The framework implicitly treats science as a sequence of enabling steps, so genuinely discontinuous or abandoned research lines may score poorly; the method could therefore favor incremental contributions over revolutionary ones.
  • A direct testable extension would use an LLM with a hard training cutoff before the held-out year to regenerate future nodes; if the gains persist, the signal is structural rather than memorized.
  • The same verified-edge construction could plausibly be applied to patents, technical reports, or negative results, which the authors list as future work but do not evaluate.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes THE-Tree, a framework that constructs domain-specific science evolution trees from surveys and citation data using a Self-Guided Temporal Monte Carlo Tree Search with Think-Verbalize-Cite-Verify node expansion and a Retrieval-Augmented Natural Language Inference (RA-NLI) edge validator. The authors release 88 trees and a 71k-fact verification benchmark, and report that THE-Tree improves graph completion by 8-14%, future prediction by nearly 10%, and LLM-based paper evaluation by almost 100%. The central claim is that the tree's explicit causal-evolutionary structure provides a verifiable substrate for AI-assisted scientific reasoning and verification.

Significance. If the reported gains were real, THE-Tree would be a valuable contribution to AI-assisted scientific evaluation, and the released dataset could support many downstream studies. The framework's combination of MCTS, LLM-guided node expansion, and NLI-based edge validation is interesting, and the scale of the released resource is a genuine strength. However, the current evaluation does not establish the central claim: the graph-completion and future-prediction results are vulnerable to temporal leakage from pretrained LLMs, and the expert ground truth is partly derived from the system's own output. The significance of the paper is therefore currently in its framework and dataset release rather than in the quantitative evidence for the claimed improvements.

major comments (4)
  1. [Section 4.2.2 and Section 4.3 (Tables 4-5)] The graph-completion and future-prediction evaluations are not temporally controlled. The LLMs used for construction and scoring, such as Qwen2.5-72B, GPT-4o, and DeepSeek-R1, are pretrained on internet-scale data that include the masked entities in Section 4.2.2 and the post-split papers in Section 4.3. Consequently, the reported Hit@1 gains of 8-14% and nearly 10% may reflect the model retrieving exact paper titles from parametric memory rather than reasoning from the tree's causal history. This leakage also affects tree construction itself: in the TVCV pipeline (Section 3.3.1), the Think and Cite steps allow the LLM to propose and ground advancements using knowledge that may include post-split literature. The paper controls for contamination only in the NeurIPS 2024 evaluation (Section 4.1), not in Tables 4-5. A necessary control is to use models with a known pretraining cutoff before the test period, or to block direct title memorization by paraphrasing the target entities.
  2. [Appendix B.2 (Table 3)] The expert-refined ground truth is constructed by having domain experts review, correct, and augment the MCTS-generated trees. Precision and recall measured against this ground truth therefore largely measure agreement between the generated tree and a human refinement of that same tree, not agreement with an independently established historical record. This circularity undermines the claim that the trees are externally validated, and Table 3's recall/precision numbers should be interpreted as self-consistency scores. An independent gold standard, in which experts build or annotate evolution paths from raw literature without seeing the MCTS output, is required.
  3. [Section 4.2.1 (Table 2)] The RA-NLI accuracy and fact-missing rates are evaluated on a self-constructed dataset of 71k fact verifications extracted from the same survey-citation pipeline used to build THE-Tree. There is no external benchmark or independently labeled test set, and no confidence intervals are reported for the 95.60% accuracy and 4.75% fact-missing rate. This makes it difficult to interpret the comparison with the other LLMs in Table 2, since the evaluation protocol may favor the in-distribution RA-NLI model. Reporting results on an established citation-context or scientific-NLI benchmark, or providing an independent human-labeled sample, is needed.
  4. [Section 4.1 (Table 1)] The claim of an almost 100% boost in evaluating important papers is not supported consistently by Table 1. For example, DeepReviewer-14b's status total improves only from 29.59 to 32.08, while Qwen2.5-72b improves from 2.42 to 2.76; the largest relative gain appears for DeepReviewer-7b (27.54 to 60.84), but this is one model, and no significance tests or confidence intervals are provided. In addition, the augmented condition adds retrieved evolutionary paths to the prompt, so the comparison may conflate the effect of additional context with the specific causal-tree structure. An ablation that adds the same amount of non-tree historical context would help isolate the contribution of THE-Tree's structure.
minor comments (4)
  1. [Section 3.3.1] Several free parameters appear in the method, including gamma, c, lambda, theta_sim, and alpha, but no sensitivity analysis is reported; please state how these were chosen and whether the main results are robust to their variation.
  2. [Section 4.2.1] The abstract's statement that '60% of relevant papers are missed' is not operationally defined, and Table 2 reports 'fact missing rate' rather than 'unawareness'; the relationship between these quantities should be clarified.
  3. [Appendix A.3] Equation (3) in the main text defines R_attr as RA-NLI, while Appendix A.3 introduces a weighted combination involving alpha and an LLM evaluator; the notation should be unified.
  4. [Conclusion] There are minor typographical and formatting issues, including 'THE-Tree´s' in the conclusion and 'Jian V ora' in reference [17]; the introduction also contains a repeated sentence about the absence of structured historical data.

Circularity Check

2 steps flagged · score 7.0 of 10

Expert ground truth is an edited copy of the MCTS output and graph-completion/future targets are drawn from the same LLM-generated tree, so the central validation and prediction claims are partly circular; only the NeurIPS 2024 evaluation is independent.

  1. self definitional [Appendix B.2 (Expert-Refined Ground Truth Construction Methodology); Table 3]
    "2) Expert Refinement and Augmentation: Domain experts then reviewed these MCTS-generated trees, performing comprehensive modifications. This included validating, correcting, or removing paths and nodes; augmenting trees with crucial missing links, milestone papers, or overlooked developmental trajectories; and ensuring overall semantic coherence, causal validity, and accurate representation of the field’s historical evolution. The resulting expert-curated THE-Trees form the ground truth dataset used for validation as described in the main experimental sections."

    Table 3 validates 'MCTS-generated THE-Trees' against an 'Expert-Refined Ground Truth' that was created by experts editing the very same MCTS-generated trees. Recall, precision, and F1 therefore measure how well the raw output agrees with a human-polished copy of itself, not how accurately it reconstructs an independent historical record. The claim that the trees 'achieve strong performance in recalling entities and relations validated by experts' reduces, by construction, to self-consistency with post-hoc edits of the method's own output.

  2. fitted input called prediction [Section 4.2.2 (Graph Completion); Section 4.3 (Future Path Prediction); TVCV/Algorithm 2 in Section 3.3.1]
    "this task involved predicting missing evolutionary entities within our THE-Tree by masking entities from a given year."

    The masked entities live inside a THE-Tree built by the TVCV pipeline, where the constructing LLM proposes and cites nodes using its own parametric knowledge with no reported temporal cutoff. The graph-completion and future-prediction tasks then ask the same class of LLMs (Qwen2.5-72B, GPT-4o, etc.) to recover those nodes and relations. The target is therefore a by-product of the same generation process or of pretraining on those papers; the reported hit@1 gains measure the model's ability to reproduce its own inserted or remembered content rather than the causal-history representation's forecasting power. The 'prediction' is not independent of the input that generated the tree.

full rationale

Two of the paper's validation loops are closed by construction. First, Appendix B.2 defines the expert ground truth as a human-edited version of the MCTS-generated trees, so Table 3's recall/precision/F1 quantify agreement between the raw output and its own edited copy, not accuracy against an independent ground truth. Second, graph completion masks entities that were placed in the THE-Tree by the same LLM-based TVCV construction process, and future-prediction uses general-purpose LLMs pretrained on the post-split literature, so the 'predicted' nodes are essentially the model's own generations or memorized papers; the 8-14% hit@1 improvements do not isolate the value of the causal-history structure. The NeurIPS 2024 paper-evaluation experiment is the one partially independent check: the authors deliberately restrict THE-Tree augmentation to unseen NeurIPS 2024 submissions and compare against official accept/status labels, so that result is not circular. Overall, the central empirical claims of graph completion, future prediction, and expert validation are partly circular, but the framework and the NeurIPS evaluation retain independent content, giving a score of 7 rather than 8 or higher.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper introduces no physical entities. The free parameters are hyperparameters in the pipeline, none of which are fitted against an external benchmark; they are choices that affect the reported results. The axioms highlight the main assumptions that, if false, could invalidate the central claims.

free parameters (4)
  • gamma = not specified
    Weight in S(v) = gamma * S_graph(v) + (1-gamma) * S_LLM(v), balancing structural and semantic importance. No value or tuning details given.
  • alpha = 0.7
    Weight in RA-NLI: R_attr = alpha * NLI + (1-alpha) * LLM_eval. Set empirically per Appendix A.3.
  • c and lambda in SGT-UCT = not specified
    Exploration coefficient c and LLM guidance weight lambda in the UCT formula (Eq. 2), no values reported.
  • theta_sim = not specified
    Similarity threshold in the verification methodology (Section 3.2) to select terminal nodes.
assumptions (4)
  • domain assumption LLM-generated evolutionary links can be grounded in cited literature through retrieval and NLI (RA-NLI).
    The entire tree construction and validation relies on the ability of a fine-tuned NLI model and an LLM to judge whether a causal link is supported by cited text (Section 3.3.1, TVCV).
  • domain assumption Surveys are a reliable source of historical evolution information.
    Trees are initialized from surveys, assuming survey narrative and citations accurately reflect scientific lineage (Section 3.3.1).
  • domain assumption The LLM used to construct and query the tree has no knowledge of future papers.
    Future path prediction experiments (Section 4.3) assume temporal masking is effective, but the LLMs are pretrained on data that includes those future papers.
  • domain assumption Expert-refined trees are an appropriate gold standard for quality assessment.
    Appendix B.2 says experts start from the MCTS-generated trees and refine them, so the ground truth is not independent of the system's output.

how reviews work

0 comments
Cite this review

Pith. "Pith review of THE-Tree: Can Tracing Historical Evolution Enhance Scientific Verification and Reasoning?." pith.science (2026). https://pith.science/paper/PA6XQLM6

@misc{pith2026250621763,
  author       = {Pith},
  title        = {Pith review of: THE-Tree: Can Tracing Historical Evolution Enhance Scientific Verification and Reasoning?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PA6XQLM6}},
  note         = {Machine review of arXiv:2506.21763}
}
read the original abstract

Large Language Models (LLMs) are accelerating scientific idea generation, but rigorously evaluating these numerous, often superficial, AI-generated propositions for novelty and factual accuracy is a critical bottleneck; manual verification is too slow. Existing validation methods are inadequate: LLMs as standalone verifiers may hallucinate and lack domain knowledge (our findings show 60% unawareness of relevant papers in specific domains), while traditional citation networks lack explicit causality and narrative surveys are unstructured. This underscores a core challenge: the absence of structured, verifiable, and causally-linked historical data of scientific evolution.To address this,we introduce \textbf{THE-Tree} (\textbf{T}echnology \textbf{H}istory \textbf{E}volution Tree), a computational framework that constructs such domain-specific evolution trees from scientific literature. THE-Tree employs a search algorithm to explore evolutionary paths. During its node expansion, it utilizes a novel "Think-Verbalize-Cite-Verify" process: an LLM proposes potential advancements and cites supporting literature. Critically, each proposed evolutionary link is then validated for logical coherence and evidential support by a recovered natural language inference mechanism that interrogates the cited literature, ensuring that each step is grounded. We construct and validate 88 THE-Trees across diverse domains and release a benchmark dataset including up to 71k fact verifications covering 27k papers to foster further research. Experiments demonstrate that i) in graph completion, our THE-Tree improves hit@1 by 8% to 14% across multiple models compared to traditional citation networks; ii) for predicting future scientific developments, it improves hit@1 metric by nearly 10%; and iii) when combined with other methods, it boosts the performance of evaluating important scientific papers by almost 100%.

Figures

Figures reproduced from arXiv: 2506.21763 by the authors.

Figure 1
Figure 1. Overview of THE-Tree. (1) Limitations of existing data structures (publication databases, citation networks) for scientific idea generation versus THE-Tree’s approach of constructing historical reasoning connections by screening important scientific history nodes and building pathways between them. (2) Methods for verifying LLM-generated ideas: implicit LLM-based cycle-review, human evaluation, and explicit THE-Tree… view at source ↗
Figure 2
Figure 2. A simple way to explicitly use THE-tree 2. Topic Space Identification: For the input paper Pin, a set of NT most relevant topics, TPin = {T opic1, . . . , T opicNT }, is identified from the available THE-Tree topics. This involves prompting an LLM with Tin, Ain, and the list of all THE-Tree topic names to determine the most appropriate THE-Tree(s) for situating Pin. 3. Candidate Path Origination in Relevant THE-Tree… view at source ↗
Figure 3
Figure 3. THE-Tree construction overview. (a) Extracting chunks and references from surveys to build an initial concept graph structure; (b) Generating a structured knowledge tree using the SGT-MCTS algorithm, guided by the TVCV methodology for node expansion and RA-NLI for relationship validation. Dataset Construction: Initialization and Foundation. The journey of constructing a THE-Tree begins with scientific surveys. These… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Overview of RA-NLI process. This figure illustrates the RA-NLI process, including citation existence verification, document retrieval, and semantic relation assessment using NLI and LLM, which forms the core of the Verify step in TVCV and the Rattr calculation. Relatio…
Figure 5
Figure 5. Figure 5: Document processing and metadata extraction pipeline applied to curated surveys for [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The data structure of THE-tree [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Performance comparison of DeepReviewer-14b with and without THE-Tree augmentation [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Prediction distribution of DeepReviewer-14b (with and without THE-Tree) for papers with [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 25 canonical work pages

  1. [1]

    Claude 3.5 sonnet

    Anthropic. Claude 3.5 sonnet. https://www.anthropic.com/news/claude-3-5-sonnet , 2024

    work page 2024

  2. [2]

    Towards a knowledge graph for science

    Sören Auer and Markus Stocker. Towards a knowledge graph for science. In Proceedings of the 11th International Conference on Semantic Systems, pages 1–8, 2018. 11

    work page 2018

  3. [3]

    Researchagent: Iterative research idea generation over scientific literature with large language models

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738, 2024

    arXiv 2024

  4. [4]

    Scibert: A pretrained language model for scientific text

    Iz Beltagy, Kyle Lo, and Arman Cohan. Scibert: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676, 2019

    arXiv 1903

  5. [5]

    Emergent autonomous scientific research capabilities of large language models

    Daniil A Boiko, Robert MacKnight, and Gabe Gomes. Emergent autonomous scientific research capabilities of large language models. arXiv preprint arXiv:2304.05332, 2023

    arXiv 2023

  6. [6]

    Autonomous chemical research with large language models

    Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570–578, 2023

    2023

  7. [7]

    What do citation counts measure? a review of studies on citing behavior

    Lutz Bornmann and Hans-Dieter Daniel. What do citation counts measure? a review of studies on citing behavior. Journal of documentation, 64(1):45–80, 2008

    work page 2008

  8. [8]

    ChemCrow: Augmenting large-language models with chemistry tools

    Andres M Bran, Sam Schilter, Philippe Delfosse, Tobias Gensch, Anica Ivanova, Apeksha Thakkar, YooJin Jung, Philippe Schwaller, Alain C Vaucher, Kevin Maik Chang, et al. ChemCrow: Augmenting large-language models with chemistry tools. arXiv preprint arXiv:2304.05376, 2023

    arXiv 2023

Show all 44 references
  1. [9]

    A survey of monte carlo tree search methods

    Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in...

    2012

  2. [10]

    Mapping the dynamics of science and technology: Sociology of science in the real world

    Michel Callon, John Law, and Arie Rip. Mapping the dynamics of science and technology: Sociology of science in the real world. In Sociology of science in the real world. Macmillan London, 1983

    1983

  3. [11]

    CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature

    Chaomei Chen. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American society for information science and technology, 57(3):359–377, 2006

    2006

  4. [12]

    To cite, or not to cite? detecting citation contexts in text

    Michael Färber, Alexander Thiemann, and Adam Jatowt. To cite, or not to cite? detecting citation contexts in text. In Advances in Information Retrieval: 40th European Conference on IR Research, ECIR 2018, Grenoble, France, March 26-29, 2018, Proceedings 40, pages 598–603. Spri...

    2018

  5. [13]

    Validating ai-generated code with live programming

    Kasra Ferdowsi, Ruanqianqian Huang, Michael B James, Nadia Polikarpova, and Sorin Lerner. Validating ai-generated code with live programming. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–8, 2024

    2024

  6. [14]

    Citation indexing: Its theory and application in science, technology, and humanities

    Eugene Garfield. Citation indexing: Its theory and application in science, technology, and humanities. Wiley, 1979

    1979

  7. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  8. [16]

    Hlm-cite: Hybrid language model workflow for text-based scientific citation prediction

    Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, and Yong Li. Hlm-cite: Hybrid language model workflow for text-based scientific citation prediction. arXiv preprint arXiv:2410.09112, 2024

    2024 arXiv

  9. [17]

    Mlagentbench: Evaluating language agents on machine learning experimentation

    Qian Huang, Jian V ora, Percy Liang, and Jure Leskovec. Mlagentbench: Evaluating language agents on machine learning experimentation. arXiv preprint arXiv:2310.03302, 2023

    2023 arXiv

  10. [18]

    A theory of universal artificial intelligence based on algorithmic complexity.arXiv preprint cs/0004001, 2000

    Marcus Hutter. A theory of universal artificial intelligence based on algorithmic complexity.arXiv preprint cs/0004001, 2000

    2000 arXiv

  11. [19]

    Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions

    Marcus Hutter. Towards a universal theory of artificial intelligence based on algorithmic probability and sequential decisions. In Machine Learning: ECML 2001, pages 226–238. Springer, 2001

    2001

  12. [20]

    Codescientist: End-to-end semi-automated scientific discovery with code-based experimentation

    Peter Jansen, Oyvind Tafjord, Marissa Radensky, Pao Siangliulue, Tom Hope, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Daniel S Weld, and Peter Clark. Codescientist: End-to-end semi-automated scientific discovery with code-based experimentation. arXiv preprint arXiv:250...

    2025 arXiv

  13. [21]

    Bandit based monte-carlo planning

    Levente Kocsis and Csaba Szepesv ári. Bandit based monte-carlo planning. In European conference on machine learning, pages 282–293. Springer, 2006

    2006

  14. [22]

    Scientific discovery: Computational explorations of the creative processes

    Pat Langley. Scientific discovery: Computational explorations of the creative processes. MIT press, 1987

    1987

  15. [23]

    Temporal knowledge graph forecasting without knowledge using in-context learning

    Dong-Ho Lee, Kian Ahrabian, Woojeong Jin, Fred Morstatter, and Jay Pujara. Temporal knowledge graph forecasting without knowledge using in-context learning. arXiv preprint arXiv:2305.10613, 2023. 12

    2023 arXiv

  16. [24]

    The ai scientist: Towards fully automated open-ended scientific discovery

    Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024

    2024 arXiv

  17. [25]

    Introducing llama 3.1: Our most capable models to date

    Meta AI. Introducing llama 3.1: Our most capable models to date. https://ai.meta.com/blog/ meta-llama-3-1/ , 2024

    2024

  18. [26]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/ , 2024

    2024

  19. [27]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  20. [28]

    Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers

    Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers. arXiv preprint arXiv:2409.04109, 2024

    2024 arXiv

  21. [29]

    Co-citation in the scientific literature: A new measure of the relationship between two documents

    Henry Small. Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for information Science, 24(4):265–269, 1973

    1973

  22. [30]

    ArnetMiner: extraction and mining of academic social networks

    Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. ArnetMiner: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 990–998, 2008

    2008

  23. [31]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024

    2024

  24. [32]

    Software survey: V osviewer, a computer program for bibliometric mapping

    Nees Jan Van Eck and Ludo Waltman. Software survey: V osviewer, a computer program for bibliometric mapping. Scientometrics, 84(2):523–538, 2010

    2010

  25. [33]

    Microsoft academic graph: When experts are not enough

    Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Darrin Eide, and Yuxiao Dong. Microsoft academic graph: When experts are not enough. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2846–2854, 2020

    2020

  26. [34]

    CycleResearcher: Improving Automated Research via Automated Review

    Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. CycleResearcher: Improving Automated Research via Automated Review. In International Conference on Learning Representations (ICLR), 2025

    2025

  27. [35]

    Large language models for automated open-domain scientific hypotheses discovery

    Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. arXiv preprint arXiv:2309.02726, 2023

    2023 arXiv

  28. [36]

    Deepreview: Improving llm-based paper review with human-like deep thinking process

    Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. Deepreview: Improving llm-based paper review with human-like deep thinking process. arXiv preprint arXiv:2503.08569, 2025. A Technical Appendices and Supplementary Material Technical appendices with additional results, figure...

    2025 arXiv

  29. [37]

    Metadata extraction: We extracted titles, abstracts, authors, venues, and publication dates using custom parsers for each data source

  30. [38]

    Citation network construction: We built a directed graph where nodes represent papers and edges represent citation relationships

  31. [39]

    Text preprocessing: We applied standard NLP preprocessing techniques, including tokenisation, stopword removal, and lemmatisation

  32. [40]

    Entity recognition: We used a combination of dictionary-based and machine learning approaches to identify technical terms and concepts

  33. [41]

    To evaluate the accuracy of our metadata extraction component, we measured its precision on two commonly used formatting styles: 98.29% for IEEE format and 97.30% for APA format

    Temporal alignment: We aligned papers along a timeline, accounting for publication delays and citation patterns. To evaluate the accuracy of our metadata extraction component, we measured its precision on two commonly used formatting styles: 98.29% for IEEE format and 97.30% f...

  34. [42]

    Metadata for the survey itself is extracted

    Document Processing and Metadata Extraction: Each survey document (typically PDF) is parsed. Metadata for the survey itself is extracted. Crucially, its reference list is parsed to create initial ’paper nodes’ for each cited work, populated with available metadata (title, auth...

  35. [43]

    Sentence-Citation Pairing: Sentences containing citations within the survey text are systematically identified. For sentences in the paper that originally contain citations, we do not directly use them as citation sentences; instead, we perform a series of post-processing step...

  36. [44]

    Total%" overall accuracy metrics, calculated as explained below: 16 –

    Paragraph-Level Concept Graph Construction: To capture the substantive content of scientific contributions beyond simple entity mentions and address the limitations of relying solely on named entities for tracing the evolution of *ideas*, we construct a paragraph-level concept...

    1950

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.