REVIEW 11 cited by
LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs' performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs. The code and datasets are in https://github.com/zjunlp/AutoKG.
Forward citations
Cited by 11 Pith papers
-
MPL: Multiple Programming Languages with Large Language Models for Information Extraction
Using multiple programming languages as code-style prompts during fine-tuning improves LLM information extraction accuracy over single-language prompting.
-
K-ON: Stacking Knowledge On the Head Layer of Large Language Model
K-ON stacks K prediction heads onto an LLM to generate entity tokens in one step and uses entity-level contrastive learning, achieving new state-of-the-art results on two knowledge graph completion benchmarks.
-
GraphTool-Instruction: Revolutionizing Graph Reasoning in LLMs through Decomposed Subtask Instruction
Decomposing graph reasoning into graph extraction, tool name identification, and tool parameter extraction subtasks yields large accuracy gains on small LLMs and a fine-tuned 8B model that competes with GPT-4o.
-
From Instructions to ODRL Usage Policies: An Ontology Guided Approach
A curated ontology prompt with self-correction rules lets GPT-4 convert natural language instructions into ODRL usage policies with up to about 92% benchmark accuracy.
-
How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance
On most RDF and SPARQL engineering tasks, larger open LLMs score higher, but plateau, ceiling, and occasional intra-family drops mean bigger is not always better.
-
Ontology-grounded Automatic Knowledge Graph Construction by LLM under Wikidata schema
An LLM pipeline that generates competency questions from documents, aligns extracted relations to Wikidata properties, and outputs RDF triples grounded in the resulting ontology achieves competitive partial-F1 on Wiki...
-
Retrieval, Reasoning, Re-ranking: A Context-Enriched Framework for Knowledge Graph Completion
KGR3 combines retrieval of entity contexts, LLM reasoning, and supervised re-ranking to improve knowledge graph completion, reporting state-of-the-art Hits@1 on FB15k237 and WN18RR.
-
Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Prompting
Feeding an LLM a reduced, question-relevant slice of a manufacturing ontology improves SPARQL query accuracy by roughly 20 to 30 percent relative to the full ontology.
-
Matching Game Preferences Through Dialogical Large Language Models: A Perspective
This perspective paper proposes the D-LLM framework, which couples the authors' GRAPHYP knowledge graphs with LLMs to personalize AI responses and make reasoning traceable, but no empirical validation is presented.
-
From Symbolic to Neural and Back: Exploring Knowledge Graph-Large Language Model Synergies
A review that organizes the knowledge graph and large language model integration field into three categories and argues for more attention to scalability, efficiency, and data quality.
-
Querying Climate Knowledge: Semantic Retrieval for Scientific Discovery
A climate knowledge graph built from prior extraction work is presented with example queries, but without evaluation or released artifacts.
Discussion (0). Continue with ORCID to comment.