Recognition: unknown
Pitfalls of Graph Neural Network Evaluation
read the original abstract
Semi-supervised node classification in graphs is a fundamental problem in graph mining, and the recently proposed graph neural networks (GNNs) have achieved unparalleled results on this task. Due to their massive success, GNNs have attracted a lot of attention, and many novel architectures have been put forward. In this paper we show that existing evaluation strategies for GNN models have serious shortcomings. We show that using the same train/validation/test splits of the same datasets, as well as making significant changes to the training procedure (e.g. early stopping criteria) precludes a fair comparison of different architectures. We perform a thorough empirical evaluation of four prominent GNN models and show that considering different splits of the data leads to dramatically different rankings of models. Even more importantly, our findings suggest that simpler GNN architectures are able to outperform the more sophisticated ones if the hyperparameters and the training procedure are tuned fairly for all models.
This paper has not been read by Pith yet.
Forward citations
Cited by 14 Pith papers
-
Neighbourhood Transformer: Switchable Attention for Monophily-Aware Graph Learning
Neighbourhood Transformers apply local self-attention for monophily-aware graph learning, guarantee expressiveness at least as strong as message-passing GNNs, and outperform prior methods on node classification across...
-
SAGE: A Self-Evolving Agentic Graph-Memory Engine for Structure-Aware Associative Memory
SAGE is a self-evolving agentic graph-memory engine that dynamically constructs and refines structured memory graphs via writer-reader feedback, yielding performance gains on multi-hop QA, open-domain retrieval, and l...
-
Random-Set Graph Neural Networks
RS-GNNs predict random sets over classes using belief functions to jointly produce class probabilities and epistemic uncertainty estimates for graph nodes.
-
Learning Graph Foundation Models on Riemannian Graph-of-Graphs
R-GFM constructs multi-scale Riemannian graph-of-graphs to learn geometry-adaptive representations, reducing structural domain generalization error and delivering up to 49% relative gains on downstream graph tasks.
-
UFO: A Unified Flow-Oriented Framework for Robust Continual Graph Learning
UFO combines flow-based generative replay with instance-level reliability scoring to handle both catastrophic forgetting and catastrophic remembering from noisy supervision in evolving graphs, outperforming baselines ...
-
From Model to Data (M2D): Shifting Complexity from GNNs to Graphs for Transparent Graph Learning
M2D distillation augments input graphs with model-derived features and structure, letting simple student GNNs match teacher performance while exposing mechanisms such as attention and fairness directly in the data.
-
Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation
A large-scale standardized benchmark of GNN attacks and defenses reveals that target node selection and attacked-model training process can completely distort measured attack effectiveness.
-
Improving Graph Few-shot Learning with Hyperbolic Space and Denoising Diffusion
IMPRESS improves graph few-shot learning by learning representations in hyperbolic space and using denoising diffusion to better approximate target distributions from few support samples.
-
Toward a universal foundation model for graph-structured data
A pretrained graph model using feature-agnostic structural prompts matches or exceeds supervised baselines and shows strong zero-shot and few-shot transfer on held-out biomedical graphs, with a 21.8% ROC-AUC gain on SagePPI.
-
Analytic Drift Resister for Non-Exemplar Continual Graph Learning
ADR achieves theoretically zero-forgetting class-incremental graph learning by combining backpropagation adaptation with ridge-regression-based layer-wise merging of GNN linear transformations.
-
Rethinking Generalization in Graph Neural Networks: A Structural Complexity Perspective
GNN generalization depends explicitly on graph structural complexity measured by effective edges, with a new regularization method shown to balance underfitting and overfitting.
-
Layer Embedding Deep Fusion Graph Neural Network
LEDF-GNN fuses multi-layer embeddings nonlinearly and runs parallel processing on original and reconstructed topologies to capture long-range dependencies and mitigate heterophily-induced misaggregation in deep GNNs.
-
Learning How Much to Think: Difficulty-Aware Dynamic MoEs for Graph Node Classification
D2MoE dynamically allocates expert resources in graph MoEs via difficulty-driven top-p routing based on predictive entropy, yielding higher accuracy and lower memory/time costs on node classification benchmarks.
-
Unified Graph Prompt Learning via Low-Rank Graph Message Prompting
LR-GMP unifies graph prompting via a low-rank Graph Message Prompt paradigm to achieve better generalization than component-specific methods.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.