REVIEW 34 cited by
A Survey on Oversmoothing in Graph Neural Networks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Node features of graph neural networks (GNNs) tend to become more similar with the increase of the network depth. This effect is known as over-smoothing, which we axiomatically define as the exponential convergence of suitable similarity measures on the node features. Our definition unifies previous approaches and gives rise to new quantitative measures of over-smoothing. Moreover, we empirically demonstrate this behavior for several over-smoothing measures on different graphs (small-, medium-, and large-scale). We also review several approaches for mitigating over-smoothing and empirically test their effectiveness on real-world graph datasets. Through illustrative examples, we demonstrate that mitigating over-smoothing is a necessary but not sufficient condition for building deep GNNs that are expressive on a wide range of graph learning tasks. Finally, we extend our definition of over-smoothing to the rapidly emerging field of continuous-time GNNs.
Forward citations
Cited by 34 Pith papers
-
Geometric Hyena Networks for Large-scale Equivariant Learning
Geometric Hyena is an equivariant long-convolutional architecture that captures global geometric context with sub-quadratic complexity and outperforms equivariant transformer baselines on several RNA and protein predi...
-
FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics
A 188M-parameter Mamba model pretrained on 11M+ simulated sPHENIX events with a new serialization and neighbor-prediction task beats task-specific baselines on three downstream detector tasks when frozen and paired wi...
-
Does Graph Compression Preserve Signal Propagation?
Under graph compression, coarsening preserves the propagation trajectory but accelerates oversmoothing, while sparsification preserves signal diversity but diverges from the original trajectory.
-
Remedying Coarsening-Based GNN Training under Heterophily via Adaptive Complementary Enhancement
ACE adds a heterophily-aware auxiliary loss to coarsening-based GNN training, recovering discarded node-level information and improving accuracy on heterophilic graphs by up to ~15 points.
-
Revisiting Degree-Corrected Spectral Clustering: a Condition-Free Spectral Analysis and Extension
A condition-free spectral bound ties DCSC's misclustered-node count to degree heterogeneity and cluster weakness, and the new ASCENT variant shows early-stage node-wise corrections can improve clustering.
-
RTL-Sequencer: Towards Scalable RTL Timing Prediction with the Sequence-based Paradigm
Linearizing RTL logic cones into breadth-first sequences and processing them with Mamba-2 sequence models yields better arrival-time, WNS, and TNS predictions than graph-based baselines on 21 open-source designs.
-
Placing Degree Scales After LayerNorm
Putting a degree-scalar after LayerNorm preserves topology magnitude that pre-LN multiplication erases, and a zero-parameter post-LN inverse-degree scale outperforms the LayerNorm baseline on influence maximization, d...
-
Beyond ReLU: Bifurcation, Oversmoothing, and Topological Priors
Replacing ReLU with odd activations that have a stabilizing cubic term (sin, tanh) provably destabilizes the oversmooth fixed point of message passing and creates stable non-homogeneous solutions with square-root ampl...
-
Geometry of Reason: Spectral Signatures of Valid Mathematical Reasoning
Spectral features of attention are claimed to classify proof validity with near-perfect effect sizes, but the main evaluation relabels proofs using the classifier's own outputs.
-
Multimodal Conditional MeshGAN for Personalized Aneurysm Growth Prediction
A conditional mesh-to-mesh GAN with local KNN and global graph branches predicts future thoracic aortic aneurysm shape and diameter more accurately than baseline mesh networks on a private longitudinal dataset.
-
TANGO: Graph Neural Dynamics via Learned Energy and Tangential Flows
TANGO adds a learnable energy gradient and an orthogonal tangential flow to GNN layers, improving long-range and heterophilic graph benchmarks.
-
Effects of relational graph modularity and depth on the learning performance of neural networks
Community-structured sparse relational graphs improve 5-layer CIFAR-10 accuracy over fully connected baselines, but the advantage reverses at 8 layers due to over-smoothing.
-
GATMesh: Clock Mesh Timing Analysis using Graph Neural Networks
A graph neural network predicts clock mesh sink delays and slews on unseen designs with about 5 ps average error and a roughly 47,000x speedup over SPICE.
-
Message-Passing GNNs Fail to Approximate Sparse Triangular Factorizations
Message-passing GNNs cannot approximate sparse triangular factorizations that require non-local dependencies, so building better learned preconditioners needs non-local or tailored architectures.
-
Deep Active Learning based Experimental Design to Uncover Synergistic Genetic Interactions for Host Targeted Therapeutics
An ensemble deep active learning framework with knowledge graph embeddings finds 92% of the top 400 HIV double-knockdown pairs after observing less than 6.3% of a 356 by 356 interaction matrix.
-
DeepGate4: Efficient and Effective Representation Learning for Circuit Design at Scale
DeepGate4 scales circuit representation learning to million-gate AIGs by partitioning them into overlapping cones and processing them in level order with a GAT-based sparse transformer, achieving state-of-the-art loss...
-
Resolving Oversmoothing with Opinion Dissensus
BIMP, a continuous-depth GNN based on nonlinear opinion dynamics, is shown to avoid oversmoothing when its constant input has unique entries, and it outperforms baselines on ten node-classification datasets.
-
Graph Transformers for inverse physics: reconstructing flows around arbitrary 2D airfoils
A hybrid message-passing and linear-attention Graph Transformer reconstructs 2D airfoil flow fields from surface pressure alone, achieving high test accuracy on a new open dataset.
-
GRAMA: Adaptive Graph Autoregressive Moving Average Models
GRAMA is a graph-adaptive ARMA architecture that wraps GNN backbones with selective sequential recurrences and reports consistent gains on long-range graph benchmarks.
-
Computing Approximate Graph Edit Distance via Optimal Transport
Deriving the node-matching matrix from a cost matrix via optimal transport improves approximate graph edit distance and edit path generation on small benchmark graphs.
-
Boundary Embedding Shaping with Adaptive Contrastive Learning for Graph Structural Disentanglement
Boundary-focused contrastive "gravity" loss on selected boundary nodes improves GNN node classification by about one point over an equal-architecture baseline, but the claimed proofs do not cover the implemented loss.
-
Attention Maps in 3D Shape Classification for Dental Stage Estimation with Class Node Graph Attention Networks
CGAT, a graph attention network with a CLS node, achieves 0.76 weighted F1 on Demirjian stage classification of 3D third-molar meshes and generates attention maps that highlight roots and furcation regions.
-
On the Interplay between Graph Structure and Learning Algorithms in Graph Neural Networks
Excess risk of SGD and ridge regression on GNNs is characterized through graph spectra, showing graph shape decides which algorithm generalizes better and deeper networks amplify the difference.
-
Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach
EFGNN fuses per-depth evidential opinions from a multi-hop GNN into one final Dirichlet-based prediction whose uncertainty is lower than that of any single propagation depth.
-
Solving the Job Shop Scheduling Problem with Graph Neural Networks: A Customizable Reinforcement Learning Environment
A new open-source library, JobShopLib, provides a customizable RL environment for GNN-based job shop scheduling, with experimental dispatchers showing competitive results.
-
Integrating Spatiotemporal Features in LSTM for Spatially Informed COVID-19 Hospitalization Forecasting
A parallel-stream LSTM with a social-connectivity-based spatial feature (SPH) is claimed to beat CDC Forecast Hub ensembles for state-level COVID-19 hospitalization forecasts.
-
The Oversmoothing Fallacy: A Misguided Narrative in GNN Research
The authors argue GNN oversmoothing is overstated, that zero-collapsing (vanishing gradient) is the actual failure mode, and that residual connections plus batch normalization suffice to train very deep GCNs.
-
Geometric GNNs for Charged Particle Tracking at GlueX
On simulated GlueX Forward Drift Chamber data, a GNN edge classifier reaches 0.9806 segment efficiency at 0.9462 purity versus 0.9119 for the traditional method, with batched GPU inference at 44 microseconds per event...
-
DeltaGNN: Graph Neural Network with Information Flow Control
DeltaGNN introduces a linear-time, embedding-derived information flow score for edge filtering and uses it to mitigate over-smoothing and over-squashing in semi-supervised node classification.
-
sHGCN: Simplified hyperbolic graph convolutional neural networks
A simplified hyperbolic GCN that avoids redundant log/exp computations is competitive or faster than prior HGCN variants on four benchmark graphs.
-
Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learning
LLM-for-graph methods are divided into Graph2text and Graph2token paradigms, with four conversion challenges and a model-selection guideline.
-
Introduction to Graph Neural Networks for Machine Learning Engineers
A tutorial survey of graph neural networks using an encoder-decoder framework, accompanied by an experimental study of hyperparameters and graph homophily on node classification.
-
Demystifying MPNNs: Message Passing as Merely Efficient Matrix Multiplication
The paper claims a k-layer message-passing network is equivalent to a single layer on the k-th power of the adjacency matrix, with deep-network failures on sparse graphs blamed on gradients.
-
P-DROP: Poisson-Based Dropout for Graph Neural Networks
A Poisson-clock node selection scheme for GNNs is proposed, but with a constant rate it reduces to iid node dropout and acquires no structural awareness.
Discussion (0). Continue with ORCID to comment.