REVIEW 13 cited by
From human experts to machines: An LLM supported approach to ontology and knowledge graph construction
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The conventional process of building Ontologies and Knowledge Graphs (KGs) heavily relies on human domain experts to define entities and relationship types, establish hierarchies, maintain relevance to the domain, fill the ABox (or populate with instances), and ensure data quality (including amongst others accuracy and completeness). On the other hand, Large Language Models (LLMs) have recently gained popularity for their ability to understand and generate human-like natural language, offering promising ways to automate aspects of this process. This work explores the (semi-)automatic construction of KGs facilitated by open-source LLMs. Our pipeline involves formulating competency questions (CQs), developing an ontology (TBox) based on these CQs, constructing KGs using the developed ontology, and evaluating the resultant KG with minimal to no involvement of human experts. We showcase the feasibility of our semi-automated pipeline by creating a KG on deep learning methodologies by exploiting scholarly publications. To evaluate the answers generated via Retrieval-Augmented-Generation (RAG) as well as the KG concepts automatically extracted using LLMs, we design a judge LLM, which rates the generated content based on ground truth. Our findings suggest that employing LLMs could potentially reduce the human effort involved in the construction of KGs, although a human-in-the-loop approach is recommended to evaluate automatically generated KGs.
Forward citations
Cited by 13 Pith papers
-
A Multi-Agent Framework for Zero-Dimensional Reduced-Order Model Planning
A multi-agent LLM framework with ontology RAG and MILP-guided search automates forward and inverse 0D reduced-order network design across aero-engine air systems, power grids, and water networks.
-
Retrieval-Augmented Generation of Ontologies from Relational Databases
An iterative RAG-LLM pipeline converts relational schemas into OWL ontology fragments, achieving LLM-judged quality scores of 4.2 to 4.6 out of 5 on two medical databases.
-
Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models
GPT-4o extracts ecological entities and relations from more than 10,000 invasion biology papers, producing a new corpus, but with no quantitative evaluation of accuracy.
-
Introducing the Swiss Food Knowledge Graph: AI for Context-Aware Nutrition Recommendation
The paper introduces SwissFKG, a knowledge graph integrating Swiss recipes, nutrients, allergens, and dietary guidelines, populated via an LLM pipeline and used for a Graph-RAG question answering demo.
-
CORE-KG: An LLM-Driven Knowledge Graph Construction Framework for Human Smuggling Networks
CORE-KG reduces node duplication by 33.28% and legal noise by 38.37% versus a GraphRAG baseline on 20 human smuggling court cases, through type-aware LLM coreference resolution and domain-filtered extraction prompts.
-
LLM-Assisted Knowledge Graph Completion for Curriculum and Domain Modelling in Personalized Higher Education Recommendations
An LLM-assisted, teacher-validated pipeline builds a curriculum, domain, and user knowledge graph from two embedded-systems modules, with modest evaluation evidence.
-
Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field
Zero-shot LLMs, especially Claude 3 Sonnet and a fine-tuned 7B Mistral variant, classify semantic relations between engineering research topics with high F1 on the new IEEE-Rel-1K benchmark.
-
MedKGent: A Large Language Model Agent Framework for Constructing Temporally Evolving Medical Knowledge Graph
A submission whose abstract describes a large temporal medical knowledge graph built by LLM agents, but whose full text is an unrelated paper on histogram regression, leaving the announced claims unsupported.
-
OntoRAG: Enhancing Question-Answering through Automated Ontology Derivation from Unstructured Knowledge Bases
An automated pipeline derives an ontology from PDFs via LLMs and graphs, reporting higher comprehensiveness and diversity win rates than vector RAG and GraphRAG, but the evaluation is circular and artifacts are missing.
-
Auto-Evaluation: A Critical Measure in Driving Improvements in Quality and Safety of AI-Generated Lesson Resources
Refining an LLM auto-evaluator with expert-teacher themes and few-shot examples improved agreement with human scores on quiz quality, but only on the same questions used for refinement.
-
LLMs4Life: Large Language Models for Ontology Learning in Life Sciences
An extended NeOn-GPT pipeline with count-guided prompts and ontology reuse yields larger life-science ontologies, but injecting gold-standard targets into the prompts confounds the evaluation of LLM ontology learning.
-
Leveraging LLM for Automated Ontology Extraction and Knowledge Graph Generation
OntoKGen automates ontology extraction and knowledge graph generation from technical documents using LLMs with user-guided iterative prompting, demonstrated on a semiconductor equipment reliability case study.
-
Harnessing multiple LLMs for Information Retrieval: A case study on Deep Learning methodologies in Biodiversity publications
An ensemble of five RAG-assisted LLMs identifies the presence of deep-learning methodology details in biodiversity papers, agreeing with human annotations on 417 of 600 comparisons.
Discussion (0). Continue with ORCID to comment.