Revisiting Over-smoothing in Deep GCNs
read the original abstract
Oversmoothing has been assumed to be the major cause of performance drop in deep graph convolutional networks (GCNs). In this paper, we propose a new view that deep GCNs can actually learn to anti-oversmooth during training. This work interprets a standard GCN architecture as layerwise integration of a Multi-layer Perceptron (MLP) and graph regularization. We analyze and conclude that before training, the final representation of a deep GCN does over-smooth, however, it learns anti-oversmoothing during training. Based on the conclusion, the paper further designs a cheap but effective trick to improve GCN training. We verify our conclusions and evaluate the trick on three citation networks and further provide insights on neighborhood aggregation in GCNs.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Concept Graph Convolutions: Message Passing in the Concept Space
Concept Graph Convolutions perform message passing on node concepts to increase interpretability of graph neural networks without losing task performance.
-
NSPOD: Accelerating Krylov solvers via DeepONet-learned POD subspaces
NSPOD is a multigrid-like deep operator network preconditioner that dramatically reduces Krylov solver iterations for linearized solid mechanics PDEs on unstructured meshes from CAD geometries.
-
NSPOD: Accelerating Krylov solvers via DeepONet-learned POD subspaces
NSPOD is a multigrid-like preconditioner using DeepONet-learned POD subspaces that dramatically cuts Krylov solver iterations for solid mechanics PDEs on unstructured CAD geometries, outperforming algebraic multigrid.
-
How Wide and How Deep? Mitigating Over-Squashing of GNNs via Channel Capacity Constrained Estimation
C3E estimates hidden dimensions and depths for GNNs by treating them as communication channels to reduce over-squashing and improve representation learning.
-
Differential-Integral Neural Operator for Long-Term Turbulence Forecasting
DINO decomposes turbulent evolution into parallel local differential and global integral operators to achieve stable autoregressive forecasting on 2D Kolmogorov flow.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.