Pith. sign in

REVIEW 3 major objections 3 minor 1 cited by

MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MedKGent claims to build a 2.97-million-triple medical knowledge graph by having two LLM agents extract and merge triples daily from 10 million abstracts.

desk verdict The abstract describes a plausible, potentially valuable MedKGent system, but the supplied full text is an unrelated statistics paper, so the submission as received is unreviewable and must be rejected. read the letter →

arxiv 2508.12393 v3 pith:TNW5KLDN submitted 2025-08-17 cs.CL cs.AI

Duzhen Zhang , Zixiao Wang , Zhong-Zhi Li , Yahan Yu , Shuncheng Jia , Jiahua Dong , Haotian Xu , Xing Wu
show 5 more authors
Yingying Zhang Tielin Zhang Jie Yang Xiuying Chen Le Song
This is my paper
classification cs.CLcs.AI
keywords medicalknowledgegraphlargelanguagemodelstemporalretrieval-augmentedgenerationPubMedabstractstripleextractionLLMagentsincrementalconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MedKGent aims to show that a pair of specialized large language model agents can incrementally construct a large, temporally evolving medical knowledge graph from over 10 million PubMed abstracts spanning 1975 to 2023. The paper reports a graph of 156,275 entities and 2,971,384 triples, asserts triple-validity rates approaching 90%, and claims that using this graph for retrieval-augmented generation improves five LLMs across seven medical question-answering benchmarks. The methods and experiments behind these claims are not present in the supplied full text, which is instead a different manuscript on histogram regression confidence bands; the summary above therefore rests on the abstract alone.

What carries the argument

The carrying mechanism is the two-agent construction loop: an Extractor Agent that converts sentences from abstracts into (head, relation, tail) triples with confidence scores, and a Constructor Agent that integrates the triples into a dated graph by reinforcing knowledge that recurs across days and resolving conflicts. The temporal dimension is the daily incremental update, which is what makes the graph 'evolving' rather than a static extraction. The operative assumption inside the mechanism is that recurrence plus the LLM's own confidence serves as a proxy for correctness.

What would settle it

Take a stratified random sample of triples by recurrence count and have clinicians verify them; if the long tail of low-recurrence triples has validity far below the 90% aggregate, the reinforcement mechanism is not doing what the abstract implies. A direct intervention: inject a known-false statement into multiple abstracts and check whether the Constructor Agent's reinforcement keeps it in the final graph.

Watch

Extended reading notes

Core claim

The central claim is that a two-agent LLM pipeline can build a large, temporally evolving medical knowledge graph from over 10 million PubMed abstracts (1975–2023), with daily incremental updates. The Extractor Agent pulls knowledge triples and assigns confidence scores; the Constructor Agent merges them into a temporal graph, reinforcing recurring knowledge and resolving conflicts. The paper reports 156,275 entities, 2,971,384 triples, triple-validity rates approaching 90% in automated and expert assessment, and improved retrieval-augmented generation for five LLMs across seven medical QA benchmarks. The full text supplied with this submission is a different manuscript—on asymptotic confide

Load-bearing premise

The load-bearing premise is that a medical fact repeated across many abstracts—and rated highly by the extracting LLM—is more likely true, so the Constructor Agent's reinforcement of recurring knowledge improves validity; if repetition tracks popularity, funding cycles, or citation cascades rather than truth, the loop would strengthen false statements.

Editorial extensions

If this is right

  • If the reported numbers hold, this pipeline gives a cheaper, continuously updated alternative to manually curated medical KGs, because it needs no human annotation at construction time.
  • A dated graph supports temporally aware medical question answering, such as asking what was known at a given date, rather than mixing decades of superseded knowledge.
  • The reported RAG gains across five LLMs and seven benchmarks suggest that literature-grounded retrieval from such a graph is a broadly usable plug-in rather than a single-model trick.
  • The graph's size makes it a candidate substrate for downstream AI tasks that need structured medical knowledge at scale, though the paper's own evaluations only cover QA.
  • The ~90% validity estimate, if it generalizes, means roughly one in ten triples in the graph is wrong, which sets a clear error budget for any application using it.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: because the Constructor Agent reinforces recurring triples, the pipeline's quality is only as good as the correlation between recurrence and truth; a testable consequence is that validity should be measured separately for high- and low-recurrence triples, and the long tail of rare triples is likely to be substantially less valid than the reported aggregate.
  • My inference: the daily temporal index could be used to detect when medical consensus changes—for example, when a treatment's triples flip from positive to negative relations—but the paper does not claim or evaluate change-point detection.
  • My inference: if the Extractor's confidence score is calibrated at all, it could be used as a cheap prior for triple validity, but the abstract does not report calibration; a quick check would be whether confidence thresholds trade off coverage and validity in a predictable way.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The abstract of arXiv:2508.12393 announces MedKGent, a two-agent LLM framework that incrementally builds a temporally evolving medical knowledge graph from over 10 million PubMed abstracts (1975–2023). The abstract reports a resulting graph with 156,275 entities and 2,971,384 triples, triple-validity rates approaching 90% from automated and expert assessment, and significant improvements in retrieval-augmented generation for five LLMs across seven medical QA benchmarks. However, the supplied full text is an unrelated statistics paper titled 'Asymptotic confidence bands for the histogram regression estimator' (arXiv:2508.12391v2), containing no mention of MedKGent, LLMs, PubMed, knowledge graphs, or any of the claimed evaluations. As received, the manuscript contains no methods, data, derivations, or results supporting the abstract's central claims.

Significance. If substantiated, the claimed contributions—scalable temporal medical KG construction via LLM agents, a large public resource, and downstream RAG gains—would be of considerable interest to the biomedical NLP and knowledge-graph communities. However, the supplied artifact provides none of the supporting content: no architecture details, no integration or conflict-resolution logic, no sampling protocol, no evaluation statistics, no code, and no data. The scale claim ('largest LLM-derived medical KG to date') is unverifiable without a clear comparison protocol and artifact availability. No strengths in the form of reproducible code, machine-checked proofs, or parameter-free derivations are present in the submitted text.

major comments (3)
  1. [Full Text (entire)] The body of the manuscript is an unrelated statistics paper, 'Asymptotic confidence bands for the histogram regression estimator' (arXiv:2508.12391v2), with no connection to MedKGent. Every load-bearing empirical claim in the abstract—entity and triple counts, ~90% triple validity, and RAG improvements across five LLMs and seven benchmarks—is therefore completely unsupported. There is no description of the Extractor Agent, Constructor Agent, temporal integration, conflict resolution, sampling protocol, expert assessment, or downstream evaluation. This is a claim-without-derivation defect: the manuscript cannot be reproduced, checked, or meaningfully reviewed.
  2. [Abstract, Extractor/Constructor Agents] The validity assessment is an in-family scoring loop: the Extractor Agent produces triples and assigns confidence scores, and the Constructor Agent reinforces recurring knowledge and resolves conflicts. With no integration logic or external gold-standard comparison described, the reported ~90% validity could reflect self-consistency or popularity rather than ground-truth correctness. The provided text makes this loop visible at abstract level and omits the information needed to rule out such a circularity. This concern is secondary only because the methods are entirely absent.
  3. [Abstract, downstream evaluations] The claim that MedKGent-KG 'significantly improved retrieval-augmented generation for five LLMs across seven medical question-answering benchmarks' is reported with no experimental protocol: no retrieval method, KG-to-text conversion, prompt templates, baselines, number of runs, or significance tests. As written, this is an unverifiable assertion. A proper paper would need to describe the evaluation setup and provide access to the constructed KG and code to substantiate the claim.
minor comments (3)
  1. [Header / metadata] The full-text header cites arXiv:2508.12391v2, whereas the abstract page is for arXiv:2508.12393. This identifier mismatch is a clear presentation defect, though it is subsumed by the larger mismatch between the abstract and the body.
  2. [References] The reference list in the supplied full text is entirely from the statistics literature (e.g., Johnston 1982, Härdle 1989, Xia 1998) and contains no citations for PubMed, medical knowledge graphs, large language models, or retrieval-augmented generation. The bibliography is inconsistent with the abstract's subject matter.
  3. [Abstract, 'largest' claim] The phrase 'to our knowledge, the largest LLM-derived medical KG to date' is not accompanied by a definition of the comparison set (e.g., prior LLM-derived KGs, their sizes, and their construction dates), so the claim cannot be independently evaluated even if the rest of the paper were present.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity identifiable from the provided text; MedKGent's derivation chain is absent.

full rationale

The supplied full text is not the MedKGent paper described in the abstract; it is 'Asymptotic confidence bands for the histogram regression estimator' by Neumeyer, Rabe, and Trabs (arXiv:2508.12391v2). The MedKGent abstract makes claims about an Extractor Agent, a Constructor Agent, temporal integration, confidence scores, and ~90% triple validity, but none of the corresponding methods, equations, or evaluation details are present. Circularity can only be established by quoting the paper and exhibiting a specific reduction (e.g., Eq. X = Eq. Y by construction, or a fitted parameter renamed as prediction). No such reduction can be found because the derivation chain itself is missing. The potential concern that the Extractor Agent's self-assigned confidence is reused as validation is speculative: the abstract does not state that the automated assessment relies on those same scores, and it mentions independent expert assessment. Without the actual methods text, no circular step can be fairly identified. Therefore the appropriate finding is no significant circularity (score 0), while noting that the mismatch between abstract and supplied text is a serious completeness/reproducibility issue outside the circularity scope.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The ledger is minimal because the abstract exposes only the pipeline shape. The full text is an unrelated manuscript, so the real thresholds, decay or reinforcement functions, conflict-resolution rules, and hyperparameters cannot be audited. Absence from the ledger means 'not visible', not 'none existed'.

assumptions (2)
  • domain assumption Ten million PubMed abstracts (1975 to 2023) are a sufficient and representative corpus for a comprehensive medical knowledge graph.
    Stated as the input corpus in the abstract. If abstracts carry publication or selection bias, or omit consensus knowledge, the graph inherits those gaps.
  • domain assumption Recurrence of a triple across abstracts plus the Extractor's self-assigned confidence score justifies insertion and reinforcement in the temporal graph.
    The abstract says the Constructor Agent 'reinforces recurring knowledge and resolves conflicts'; this makes frequency and self-scored confidence proxies for truth, an uncalibrated modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph." pith.science (2026). https://pith.science/paper/TNW5KLDN

@misc{pith2026250812393,
  author       = {Pith},
  title        = {Pith review of: MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TNW5KLDN}},
  note         = {Machine review of arXiv:2508.12393}
}
read the original abstract

The rapid expansion of medical literature challenges the scalable structuring of domain knowledge. Knowledge Graphs (KGs) offer a solution, yet current construction methods lack generalizability and ignore the temporal dynamics of evolving knowledge. To address this, we introduce MedKGent, a Large Language Model (LLM) agent framework for building temporally evolving medical KGs. Using over 10 million PubMed abstracts from 1975 to 2023, MedKGent incrementally constructs a KG daily via two specialized agents. The Extractor Agent identifies knowledge triples and assigns confidence scores, while the Constructor Agent integrates these triples into a temporal graph, reinforcing recurring knowledge and resolving conflicts. The resulting KG contains 156,275 entities and 2,971,384 triples, making it, to our knowledge, the largest LLM-derived medical KG to date. Automated and expert assessments showed triple-validity rates approaching 90%. In downstream evaluations, MedKGent-KG significantly improved retrieval-augmented generation for five LLMs across seven medical question-answering benchmarks. Together, these results position MedKGent as a scalable and temporally aware infrastructure for medical knowledge representation and literature-grounded AI research.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Debate-on-Graph: Reliable and Adaptive Reasoning of Large Language Model on Uncertain Knowledge Graph

    cs.CL 2026-07 conditional novelty 5.0 of 10

    Confidence-aware subgraph retrieval plus asymmetric multi-agent debate improves LLM QA accuracy on four benchmarks using uncertain knowledge graphs.

Reference graph

Works this paper leans on

88 extracted references · 71 canonical work pages · cited by 1 Pith paper

  1. [1]

    Unstructured Data: an overview of the data of Big Data

    Adanma Cecilia Eberendu et al. Unstructured Data: an overview of the data of Big Data . International Journal of Computer Trends and Technology , 38(1):46--50, 2016

    2016

  2. [2]

    PubMed and beyond: biomedical literature search in the age of artificial intelligence

    Qiao Jin, Robert Leaman, and Zhiyong Lu. PubMed and beyond: biomedical literature search in the age of artificial intelligence . EBioMedicine , 100, 2024

    2024

  3. [3]

    From biomedical knowledge graph construction to semantic querying: a comprehensive approach

    Ling Wang, Haoyu Hao, Xue Yan, Tie Hua Zhou, and Keun Ho Ryu. From biomedical knowledge graph construction to semantic querying: a comprehensive approach . Scientific Reports , 15(1):8523, 2025

    2025

  4. [4]

    Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy

    Wasim Aftab, Zivkos Apostolou, Karim Bouazoune, and Tobias Straub. Optimizing biomedical information retrieval with a keyword frequency-driven prompt enhancement strategy . BMC bioinformatics , 25(1):281, 2024

    2024

  5. [5]

    Real-world data medical knowledge graph: construction and applications

    Linfeng Li, Peng Wang, Jun Yan, Yao Wang, Simin Li, Jinpeng Jiang, Zhe Sun, Buzhou Tang, Tsung-Hui Chang, Shenghui Wang, et al. Real-world data medical knowledge graph: construction and applications . Artificial intelligence in medicine , 103:101817, 2020

    2020

  6. [6]

    BIOS: An Algorithmically Generated Biomedical Knowledge Graph

    Sheng Yu, Zheng Yuan, Jun Xia, Shengxuan Luo, Huaiyuan Ying, Sihang Zeng, Jingyi Ren, Hongyi Yuan, Zhengyun Zhao, Yucong Lin, et al. Bios: An algorithmically generated biomedical knowledge graph . arXiv preprint arXiv:2203.09975 , 2022

    work page Pith review arXiv 2022

  7. [7]

    A review: Knowledge reasoning over knowledge graph

    Xiaojun Chen, Shengbin Jia, and Yang Xiang. A review: Knowledge reasoning over knowledge graph . Expert systems with applications , 141:112948, 2020

    2020

  8. [8]

    Knowledge graphs

    Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, Jos \'e Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. Knowledge graphs . ACM Computing Surveys (Csur) , 54(4):1--37, 2021

    2021

Show all 88 references
  1. [9]

    Constructing knowledge graphs and their biomedical applications

    David N Nicholson and Casey S Greene. Constructing knowledge graphs and their biomedical applications . Computational and structural biotechnology journal , 18:1414--1428, 2020

    2020

  2. [10]

    KG-Predict: A knowledge graph computational framework for drug repurposing

    Zhenxiang Gao, Pingjian Ding, and Rong Xu. KG-Predict: A knowledge graph computational framework for drug repurposing . Journal of biomedical informatics , 132:104133, 2022

    2022

  3. [11]

    PharmKG: a dedicated knowledge graph benchmark for bomedical data mining

    Shuangjia Zheng, Jiahua Rao, Ying Song, Jixian Zhang, Xianglu Xiao, Evandro Fei Fang, Yuedong Yang, and Zhangming Niu. PharmKG: a dedicated knowledge graph benchmark for bomedical data mining . Briefings in bioinformatics , 22(4):bbaa344, 2021

    2021

  4. [12]

    Guiding clinical reasoning with large language models via knowledge seeds

    Jiageng Wu, Xian Wu, and Jie Yang. Guiding clinical reasoning with large language models via knowledge seeds . In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages 7491--7499, 2024

    2024

  5. [13]

    KGHC: a knowledge graph for hepatocellular carcinoma

    Nan Li, Zhihao Yang, Ling Luo, Lei Wang, Yin Zhang, Hongfei Lin, and Jian Wang. KGHC: a knowledge graph for hepatocellular carcinoma . BMC Medical Informatics and Decision Making , 20:1--11, 2020

    2020

  6. [14]

    Knowlife: a versatile approach for constructing a large knowledge graph for biomedical sciences

    Patrick Ernst, Amy Siu, and Gerhard Weikum. Knowlife: a versatile approach for constructing a large knowledge graph for biomedical sciences . BMC bioinformatics , 16:1--13, 2015

    2015

  7. [15]

    Named entity recognition and relation extraction: State-of-the-art

    Zara Nasar, Syed Waqar Jaffry, and Muhammad Kamran Malik. Named entity recognition and relation extraction: State-of-the-art . ACM Computing Surveys (CSUR) , 54(1):1--39, 2021

    2021

  8. [16]

    Natural language processing for information extraction

    Sonit Singh. Natural language processing for information extraction . arXiv preprint arXiv:1807.02383 , 2018

    2018 arXiv

  9. [17]

    Using machine learning to maintain rule-based named-entity recognition and classification systems

    Georgios Petasis, Frantz Vichot, Francis Wolinski, Georgios Paliouras, Vangelis Karkaletsis, and Constantine D Spyropoulos. Using machine learning to maintain rule-based named-entity recognition and classification systems . In proceedings of the 39th annual meeting of the asso...

    2001

  10. [18]

    A rule-based named entity recognition system for speech input

    Ji-Hwan Kim and Philip C Woodland. A rule-based named entity recognition system for speech input. In INTERSPEECH , pages 528--531, 2000

    2000

  11. [19]

    Evaluating contributions of natural language parsers to protein--protein interaction extraction

    Yusuke Miyao, Kenji Sagae, Rune S tre, Takuya Matsuzaki, and Jun'ichi Tsujii. Evaluating contributions of natural language parsers to protein--protein interaction extraction . Bioinformatics , 25(3):394--400, 2009

    2009

  12. [20]

    On the efficacy of per-relation basis performance evaluation for PPI extraction and a high-precision rule-based approach

    Junkyu Lee, Seongsoon Kim, Sunwon Lee, Kyubum Lee, and Jaewoo Kang. On the efficacy of per-relation basis performance evaluation for PPI extraction and a high-precision rule-based approach . In BMC medical informatics and decision making , volume 13, pages 1--12. Springer, 2013

    2013

  13. [21]

    PPInterFinder—a mining tool for extracting causal relations on human proteins from literature

    Kalpana Raja, Suresh Subramani, and Jeyakumar Natarajan. PPInterFinder—a mining tool for extracting causal relations on human proteins from literature . Database , 2013, 2013

    2013

  14. [22]

    Unsupervised named entity classification models and their ensembles

    Jae-Ho Kim, In-Ho Kang, and Key-Sun Choi. Unsupervised named entity classification models and their ensembles . In COLING 2002: The 19th International Conference on Computational Linguistics , 2002

    2002

  15. [23]

    Building a knowledge graph to enable precision medicine

    Payal Chandak, Kexin Huang, and Marinka Zitnik. Building a knowledge graph to enable precision medicine . Scientific Data , 10(1):67, 2023

    2023

  16. [24]

    Two-phase biomedical named entity recognition using CRFs

    Lishuang Li, Rongpeng Zhou, and Degen Huang. Two-phase biomedical named entity recognition using CRFs . Computational biology and chemistry , 33(4):334--338, 2009

    2009

  17. [25]

    A Kernel-Based Approach for Biomedical Named Entity Recognition

    Rakesh Patra and Sujan Kumar Saha. A Kernel-Based Approach for Biomedical Named Entity Recognition . The Scientific World Journal , 2013(1):950796, 2013

    2013

  18. [26]

    A novel machine learning framework for automated biomedical relation extraction from large-scale literature repositories

    Lixiang Hong, Jinjian Lin, Shuya Li, Fangping Wan, Hui Yang, Tao Jiang, Dan Zhao, and Jianyang Zeng. A novel machine learning framework for automated biomedical relation extraction from large-scale literature repositories . Nature Machine Intelligence , 2(6):347--355, 2020

    2020

  19. [27]

    A unified active learning framework for biomedical relation extraction

    Hong-Tao Zhang, Min-Lie Huang, and Xiao-Yan Zhu. A unified active learning framework for biomedical relation extraction . Journal of Computer Science and Technology , 27(6):1302--1313, 2012

    2012

  20. [28]

    Cascaded classifiers for confidence-based chemical named entity recognition

    Peter Corbett and Ann Copestake. Cascaded classifiers for confidence-based chemical named entity recognition . BMC bioinformatics , 9:1--10, 2008

    2008

  21. [29]

    Integrated bio-entity network: a system for biological knowledge discovery

    Lindsey Bell, Rajesh Chowdhary, Jun S Liu, Xufeng Niu, and Jinfeng Zhang. Integrated bio-entity network: a system for biological knowledge discovery . PloS one , 6(6):e21474, 2011

    2011

  22. [30]

    Named entity recognition through classifier combination

    Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. Named entity recognition through classifier combination . In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 , pages 168--171, 2003

    2003

  23. [31]

    tmChem: a high performance approach for chemical named entity recognition and normalization

    Robert Leaman, Chih-Hsuan Wei, and Zhiyong Lu. tmChem: a high performance approach for chemical named entity recognition and normalization . Journal of cheminformatics , 7:1--10, 2015

    2015

  24. [32]

    A neural joint model for entity and relation extraction from biomedical text

    Fei Li, Meishan Zhang, Guohong Fu, and Donghong Ji. A neural joint model for entity and relation extraction from biomedical text . BMC bioinformatics , 18:1--11, 2017

    2017

  25. [33]

    Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme

    Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 122...

    2017

  26. [34]

    A neural network multi-task learning approach to biomedical named entity recognition

    Gamal Crichton, Sampo Pyysalo, Billy Chiu, and Anna Korhonen. A neural network multi-task learning approach to biomedical named entity recognition . BMC bioinformatics , 18:1--14, 2017

    2017

  27. [35]

    Deep learning with word embeddings improves biomedical named entity recognition

    Maryam Habibi, Leon Weber, Mariana Neves, David Luis Wiegandt, and Ulf Leser. Deep learning with word embeddings improves biomedical named entity recognition . Bioinformatics , 33(14):i37--i48, 2017

    2017

  28. [36]

    Chemical-induced disease relation extraction via convolutional neural network

    Jinghang Gu, Fuqing Sun, Longhua Qian, and Guodong Zhou. Chemical-induced disease relation extraction via convolutional neural network . Database , 2017:bax024, 2017

    2017

  29. [37]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human lang...

    2019

  30. [38]

    BioBERT: a pre-trained biomedical language representation model for biomedical text mining

    Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. BioBERT: a pre-trained biomedical language representation model for biomedical text mining . Bioinformatics , 36(4):1234--1240, 2020

    2020

  31. [39]

    SciBERT: A Pretrained Language Model for Scientific Text

    Iz Beltagy, Kyle Lo, and Arman Cohan. SciBERT: A Pretrained Language Model for Scientific Text . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) ,...

    2019

  32. [40]

    Constructing the largest-scale biomedical knowledge graph using all PubMed articles and its application in automated knowledge discovery

    Yuan Zhang, Feng Pan, Xin Sui, Donghu Sun, Menghan Chung, and Jinfeng Zhang. Constructing the largest-scale biomedical knowledge graph using all PubMed articles and its application in automated knowledge discovery . Cancer Research , 83(7\_Supplement):5366--5366, 2023

    2023

  33. [41]

    A comprehensive large-scale biomedical knowledge graph for AI-powered data-driven biomedical research

    Yuan Zhang, Xin Sui, Feng Pan, Kaixian Yu, Keqiao Li, Shubo Tian, Arslan Erdengasileng, Qing Han, Wanjing Wang, Jianan Wang, et al. A comprehensive large-scale biomedical knowledge graph for AI-powered data-driven biomedical research . Nature Machine Intelligence , pages 1--13, 2025

    2025

  34. [42]

    GPT-4 Technical Report , 2023

    OpenAI. GPT-4 Technical Report , 2023

    2023

  35. [43]

    Hello GPT-4o , May 2024

    OpenAI. Hello GPT-4o , May 2024

    2024

  36. [44]

    Construction and Application of Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Model

    Yanpeng Ye, Jie Ren, Shaozhou Wang, Yuwei Wan, Imran Razzak, Bram Hoex, Haofen Wang, Tong Xie, and Wenjie Zhang. Construction and Application of Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Model . Advances in Neural Information Processin...

    2024

  37. [45]

    Construction of a knowledge graph for framework material enabled by large language models and its application

    Xuefeng Bai, Song He, Yi Li, Yabo Xie, Xin Zhang, Wenli Du, and Jian-Rong Li. Construction of a knowledge graph for framework material enabled by large language models and its application . npj Computational Materials , 11(1):51, 2025

    2025

  38. [46]

    Graphusion: A RAG Framework for Scientific Knowledge Graph Construction with a Global Perspective

    Rui Yang, Boming Yang, Xinjie Zhao, Fan Gao, Aosong Feng, Sixun Ouyang, Moritz Blum, Tianwei She, Yuang Jiang, Freddy Lecue, et al. Graphusion: A RAG Framework for Scientific Knowledge Graph Construction with a Global Perspective . In Companion Proceedings of the ACM on Web Co...

    2025

  39. [47]

    Augmented non-hallucinating large language models as medical information curators

    Stephen Gilbert, Jakob Nikolas Kather, and Aidan Hogan. Augmented non-hallucinating large language models as medical information curators . NPJ digital medicine , 7(1):100, 2024

    2024

  40. [48]

    Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities

    Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, and Ningyu Zhang. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities . World Wide Web , 27(5):58, 2024

    2024

  41. [49]

    Customized Information and Domain-centric Knowledge Graph Construction with Large Language Models

    Frank Wawrzik, Matthias Plaue, Savan Vekariya, and Christoph Grimm. Customized Information and Domain-centric Knowledge Graph Construction with Large Language Models . arXiv preprint arXiv:2409.20010 , 2024

    2024 arXiv

  42. [50]

    Graphusion: Leveraging large language models for scientific knowledge graph fusion and construction in nlp education

    Rui Yang, Boming Yang, Sixun Ouyang, Tianwei She, Aosong Feng, Yuang Jiang, Freddy Lecue, Jinghui Lu, and Irene Li. Graphusion: Leveraging large language models for scientific knowledge graph fusion and construction in nlp education . arXiv preprint arXiv:2407.10794 , 2024

    2024 arXiv

  43. [51]

    itext2kg: Incremental knowledge graphs construction using large language models

    Yassir Lairgi, Ludovic Moncla, R \'e my Cazabet, Khalid Benabdeslem, and Pierre Cl \'e au. itext2kg: Incremental knowledge graphs construction using large language models . In International Conference on Web Information Systems Engineering , pages 214--229. Springer, 2024

    2024

  44. [52]

    Lightrag: Simple and fast retrieval-augmented generation

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. Lightrag: Simple and fast retrieval-augmented generation

  45. [53]

    Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

    Qiang Sun, Yuanyi Luo, and Wenxiao Zhang. Docs2KG: Unified Knowledge Graph Construction from Heterogeneous Documents Assisted by Large Language Models

  46. [54]

    Can LLMs be Good Graph Judger for Knowledge Graph Construction? arXiv preprint arXiv:2411.17388 , 2024

    Haoyu Huang, Chong Chen, Conghui He, Yang Li, Jiawei Jiang, and Wentao Zhang. Can LLMs be Good Graph Judger for Knowledge Graph Construction? arXiv preprint arXiv:2411.17388 , 2024

    2024

  47. [55]

    CEAR: Automatic construction of a knowledge graph of chemical entities and roles from scientific literature

    Stefan Langer, Fabian Neuhaus, and Andreas N \"u rnberger. CEAR: Automatic construction of a knowledge graph of chemical entities and roles from scientific literature . arXiv preprint arXiv:2407.21708 , 2024

    2024 arXiv

  48. [56]

    Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

    Bowen Zhang and Harold Soh. Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages 9820--9836, 2024

    2024

  49. [57]

    UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction

    Yansong Ning and Hao Liu. UrbanKGent: A Unified Large Language Model Agent Framework for Urban Knowledge Graph Construction . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

  50. [58]

    PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs

    Jiuzhou Han, Nigel Collier, Wray Buntine, and Ehsan Shareghi. PiVe: Prompting with Iterative Verification Improving Graph-based Generative Capability of LLMs . In Findings of the Association for Computational Linguistics ACL 2024 , pages 6702--6718, 2024

    2024

  51. [59]

    SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graph

    Hanzhu Chen, Xu Shen, Qitan Lv, Jie Wang, Xiaoqi Ni, and Jieping Ye. SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graph . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo...

    2024

  52. [60]

    From human experts to machines: An LLM supported approach to ontology and knowledge graph construction

    Vamsi Krishna Kommineni, Birgitta K \"o nig-Ries, and Sheeba Samuel. From human experts to machines: An LLM supported approach to ontology and knowledge graph construction . arXiv preprint arXiv:2403.08345 , 2024

    2024 arXiv

  53. [61]

    TechGPT-2.0: A large language model project to solve the task of knowledge graph construction

    Jiaqi Wang, Yuying Chang, Zhong Li, Ning An, Qi Ma, Lei Hei, Haibo Luo, Yifei Lu, and Feiliang Ren. TechGPT-2.0: A large language model project to solve the task of knowledge graph construction . arXiv preprint arXiv:2401.04507 , 2024

    2024 arXiv

  54. [62]

    KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction

    Jack Boylan, Shashank Mangla, Dominic Thorn, Demian Gholipour Ghalandari, Parsa Ghaffari, and Chris Hokamp. KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction . arXiv preprint arXiv:2404.15923 , 2024

    2024 arXiv

  55. [63]

    DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer’s Disease Questions with Scientific Literature

    Dawei Li, Shu Yang, Zhen Tan, Jae Baik, Sukwon Yun, Joseph Lee, Aaron Chacko, Bojian Hou, Duy Duong-Tran, Ying Ding, et al. DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer’s Disease Questions with Scientific Literature . In Findings of the Association for Comp...

    2024

  56. [64]

    CoRTEx: contrastive learning for representing terms via explanations with applications on constructing biomedical knowledge graphs

    Huaiyuan Ying, Zhengyun Zhao, Yang Zhao, Sihang Zeng, and Sheng Yu. CoRTEx: contrastive learning for representing terms via explanations with applications on constructing biomedical knowledge graphs . Journal of the American Medical Informatics Association , 31(9):1912--1920, 2024

    1912

  57. [65]

    An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontology-Enhanced Large Language Models: Development Study

    Lang Cao, Jimeng Sun, Adam Cross, et al. An Automatic and End-to-End System for Rare Disease Knowledge Graph Construction Based on Ontology-Enhanced Large Language Models: Development Study . JMIR Medical Informatics , 12(1):e60665, 2024

    2024

  58. [66]

    A New Perspective on ADHD Research: Knowledge Graph Construction with LLMs and Network Based Insights

    Hakan T Otal, Stephen V Faraone, and M Abdullah Canbaz. A New Perspective on ADHD Research: Knowledge Graph Construction with LLMs and Network Based Insights . In International Conference on Complex Networks and Their Applications , pages 337--349. Springer, 2024

    2024

  59. [67]

    Automated construction of theme-specific knowledge graphs

    Linyi Ding, Sizhe Zhou, Jinfeng Xiao, and Jiawei Han. Automated construction of theme-specific knowledge graphs . arXiv preprint arXiv:2404.19146 , 2024

    2024 arXiv

  60. [68]

    A Survey on In-context Learning

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. A Survey on In-context Learning . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages 1107--1128, 2024

    2024

  61. [69]

    Probabilistic medical predictions of large language models

    Bowen Gu, Rishi J Desai, Kueiyu Joshua Lin, and Jie Yang. Probabilistic medical predictions of large language models . npj Digital Medicine , 7(1):367, 2024

    2024

  62. [70]

    Qwen2.5: A Party of Foundation Models , September 2024

    Qwen Team. Qwen2.5: A Party of Foundation Models , September 2024

    2024

  63. [71]

    Self-Consistency Improves Chain of Thought Reasoning in Language Models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models . In The Eleventh International Conference on Learning Representations

  64. [72]

    Confidence improves self-consistency in llms

    Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, and Gal Yona. Confidence improves self-consistency in llms . arXiv preprint arXiv:2502.06233 , 2025

    2025

  65. [73]

    Universal Self-Consistency for Large Language Models

    Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. Universal Self-Consistency for Large Language Models . In ICML 2024 Workshop on In-Context Learning

    2024

  66. [74]

    Introducing GPT-4.1 in the API , April 2025

    OpenAI. Introducing GPT-4.1 in the API , April 2025

    2025

  67. [75]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report . arXiv preprint arXiv:2412.19437 , 2024

    2024 arXiv

  68. [76]

    Benchmarking retrieval-augmented generation for medicine

    Guangzhi Xiong, Qiao Jin, Zhiyong Lu, and Aidong Zhang. Benchmarking retrieval-augmented generation for medicine . In Findings of the Association for Computational Linguistics ACL 2024 , pages 6233--6251, 2024

    2024

  69. [77]

    KGARevion: an AI agent for knowledge-intensive biomedical QA

    Xiaorui Su, Yibo Wang, Shanghua Gao, Xiaolong Liu, Valentina Giunchiglia, Djork-Arn \'e Clevert, and Marinka Zitnik. KGARevion: an AI agent for knowledge-intensive biomedical QA . In ICLR , 2025

    2025

  70. [78]

    New embedding models and API updates , January 2024

    OpenAI. New embedding models and API updates , January 2024

    2024

  71. [79]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report . arXiv preprint arXiv:2505.09388 , 2025

    2025 arXiv

  72. [80]

    PubTator 3.0: an AI-powered literature resource for unlocking biomedical knowledge

    Chih-Hsuan Wei, Alexis Allot, Po-Ting Lai, Robert Leaman, Shubo Tian, Ling Luo, Qiao Jin, Zhizheng Wang, Qingyu Chen, and Zhiyong Lu. PubTator 3.0: an AI-powered literature resource for unlocking biomedical knowledge . Nucleic Acids Research , 52(W1):W540--W546, 2024

    2024

  73. [81]

    Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing , 2020

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing , 2020

    2020

  74. [82]

    Interrater reliability: the kappa statistic

    Mary L McHugh. Interrater reliability: the kappa statistic . Biochemia medica , 22(3):276--282, 2012

    2012

  75. [83]

    Richard Landis and Gary G Koch

    J. Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data . Biometrics , 33(1):159--174, 1977

    1977

  76. [84]

    BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text

    Jiageng Wu, Bowen Gu, Ren Zhou, Kevin Xie, Doug Snyder, Yixing Jiang, Valentina Carducci, Richard Wyss, Rishi J Desai, Emily Alsentzer, et al. BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text . arXiv preprint arXiv:2504.19467 , 2025

    2025

  77. [85]

    From system 1 to system 2: A survey of reasoning large language models

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models . arXiv preprint arXiv:2502.17419 , 2025

    2025 arXiv

  78. [86]

    MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models

    Yilin Wen, Zifeng Wang, and Jimeng Sun. MindMap: Knowledge Graph Prompting Sparks Graph of Thoughts in Large Language Models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 10370--10388, 2024

    2024

  79. [87]

    How Easily do Irrelevant Inputs Skew the Responses of Large Language Models? In First Conference on Language Modeling

    Siye Wu, Jian Xie, Jiangjie Chen, Tinghui Zhu, Kai Zhang, and Yanghua Xiao. How Easily do Irrelevant Inputs Skew the Responses of Large Language Models? In First Conference on Language Modeling

  80. [88]

    Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.