REVIEW 24 cited by
GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models~(LLM) like ChatGPT have become indispensable to artificial general intelligence~(AGI), demonstrating excellent performance in various natural language processing tasks. In the real world, graph data is ubiquitous and an essential part of AGI and prevails in domains like social network analysis, bioinformatics and recommender systems. The training corpus of large language models often includes some algorithmic components, which allows them to achieve certain effects on some graph data-related problems. However, there is still little research on their performance on a broader range of graph-structured data. In this study, we conduct an extensive investigation to assess the proficiency of LLMs in comprehending graph data, employing a diverse range of structural and semantic-related tasks. Our analysis encompasses 10 distinct tasks that evaluate the LLMs' capabilities in graph understanding. Through our study, we not only uncover the current limitations of language models in comprehending graph structures and performing associated reasoning tasks but also emphasize the necessity for further advancements and novel approaches to enhance their graph processing capabilities. Our findings contribute valuable insights towards bridging the gap between language models and graph understanding, paving the way for more effective graph mining and knowledge extraction.
Forward citations
Cited by 24 Pith papers
-
Agentic Graph Token Reasoning
Agentic Graph Token (AGT) reasoning lets an LLM iteratively select graph views, encode them as tokens, and reason step-by-step, beating static graph-token and text-agent baselines on seven graph domains.
-
GraphInfer-Bench: Benchmarking LLM's Inference Capability on Graphs
Presents GraphInfer-Bench to demonstrate that no evaluated LLM-based method family closes the performance gap on graph inference tasks requiring multi-node reasoning, with plain GNNs matching or exceeding them.
-
GABench: A Comprehensive Benchmark for Evaluating LLM Agents on Graph Analysis Tasks
GABench provides the first agentic graph-analysis benchmark with 10,400 executable tasks, and finds existing LLM agents succeed on under 40% of complex graph tasks.
-
FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
FedLAB organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context via federated semantic barycenter pre-training, improving performance by up to ...
-
GraspLLM: Towards Zero-Shot Generalization on Text-Attributed Graphs with LLMs
GraspLLM extracts dataset-agnostic structural patterns via motif contrastive learning and aligns contextual subgraphs to LLM tokens, outperforming prior LLM-based methods on TAGs especially in zero-shot settings.
-
Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
Bidirectional LLM-GNN co-teaching with round-based pseudo-label preference optimization outperforms golden-teacher baselines on few-shot TAG benchmarks by 3-8% absolute gains.
-
Clustering as Reasoning: A $k$-Means Interpretation of Chain-of-Thought Graph Learning
KCoT reframes CoT graph learning as k-means clustering by establishing a formal correspondence between Transformer blocks and k-means assignment/update steps, with a Semantic Discriminating Prompt and structure alignm...
-
TERGAD: Structure-Aware Text-Enhanced Representations for Graph Anomaly Detection
TERGAD augments graph anomaly detection by converting node topological properties into LLM-generated semantic embeddings that are fused with original attributes via a gated dual-branch autoencoder for joint reconstruc...
-
Linking Extreme Discourse to Structural Polarization in Signed Interaction Networks
A pipeline derives continuous signed edges from LLM stance scores on text and links discourse signals such as toxicity and extreme claims to changes in structural polarization measured by spectral and frustration scor...
-
A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning
UniGraphLM uses a multi-domain multi-task GNN encoder and adaptive alignment to create unified graph tokens for LLMs across diverse domains and tasks.
-
Revisiting Graph-Tokenizing Large Language Models: A Systematic Evaluation of Graph Token Understanding
GTokenLLMs do not fully understand graph tokens, exhibiting over-sensitivity or insensitivity to instruction changes and relying heavily on text for reasoning even when graph information is preserved.
-
GraphDC: A Divide-and-Conquer Multi-Agent System for Scalable Graph Algorithm Reasoning
GraphDC applies divide-and-conquer multi-agent LLM reasoning to graph algorithms by decomposing graphs into subgraphs for local agents and integrating via a master agent, outperforming direct methods especially on lar...
-
Both Topology and Text Matter: Revisiting LLM-guided Out-of-Distribution Detection on Text-attributed Graphs
LG-Plug mines pseudo-OOD exposures from clustered unlabeled nodes via iterative LLM prompting and regularizes topology-driven graph OOD detectors, cutting FPR95 by ≥7% across six TAG benchmarks.
-
Deep sequence models tend to memorize geometrically; it is unclear why
Deep sequence models develop geometric memory in embeddings that encodes novel global relationships, transforming l-fold composition tasks into 1-step navigation via a natural spectral bias connected to Node2Vec.
-
Harnessing Adaptive Topology Representations for Zero-Shot Graph Question Answering
DynamicTRF learns question-level preferences over eight graph representations and routes each query to the best one, improving zero-shot graph QA accuracy and output brevity on seven algorithmic and two downstream tasks.
-
AGE: Adaptive-masking for Graph Embedding in Graph Retrieval-Augmented Generation
AGE applies adaptive masking via a learnable sampler in Transformer-based SSL to align graph and text embeddings, yielding higher accuracy on four GraphQA benchmarks for non-parametric GraphRAG.
-
GLIP: Graph and LLM Joint Pretraining for Graph-Level Tasks
GLIP is a joint GNN-LLM pretraining framework that uses augmentation, multi-token selection, a diffusion projector, and combined contrastive plus semantic losses to boost graph classification and reasoning after fine-...
-
Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference
Fine-tuned small language models (3-4B parameters) preserve ordinal consistency in ranking graph structural properties for graphs larger than training data and from held-out families, showing architecture-specific deg...
-
G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledge
G-reasoner uses QuadGraph abstraction and a 34M-parameter graph foundation model integrated with LLMs to enable scalable reasoning over diverse graph-structured knowledge, outperforming baselines on six benchmarks.
-
CS-Agent: LLM-based Community Search via Dual-agent Collaboration
CS-Agent, a Solver-Validator two-agent dialogue with a Decider selector, improves LLM community search on synthetic graphs, and GraphCS is a new benchmark for measuring it.
-
Retrieval-Augmented Generation with Graphs (GraphRAG)
A survey proposing a holistic GraphRAG framework with components including query processor, retriever, organizer, generator, and data source, plus domain-tailored reviews, challenges, and future directions.
-
Are Large Language Models Suitable for Graph Computation? Progress and Prospects
A survey of LLMs for graph computation introduces a role-based taxonomy of executors versus planners and concludes that current models suit simple small-scale tasks but remain unreliable for large-scale exact computation.
-
Capabilities of GPT-5 on Multimodal Medical Reasoning
A benchmark study reports GPT-5 outperforming GPT-4o and pre-licensed human experts on most medical QA tasks, but not consistently on VQA-RAD.
-
LLMs Between the Nodes: Community Discovery Beyond Vectors
CommLLM, a two-step graph-to-text plus LLM prompting method, reports high NMI on six small networks, but its evaluation omits standard community-detection baselines and relies on a prompt tuned on one test set.
Discussion (0). Sign in to comment.