REVIEW 8 cited by
GraphiT: Encoding Graph Structure in Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We show that viewing graphs as sets of node features and incorporating structural and positional information into a transformer architecture is able to outperform representations learned with classical graph neural networks (GNNs). Our model, GraphiT, encodes such information by (i) leveraging relative positional encoding strategies in self-attention scores based on positive definite kernels on graphs, and (ii) enumerating and encoding local sub-structures such as paths of short length. We thoroughly evaluate these two ideas on many classification and regression tasks, demonstrating the effectiveness of each of them independently, as well as their combination. In addition to performing well on standard benchmarks, our model also admits natural visualization mechanisms for interpreting graph motifs explaining the predictions, making it a potentially strong candidate for scientific applications where interpretation is important. Code available at https://github.com/inria-thoth/GraphiT.
Forward citations
Cited by 8 Pith papers
-
Logarithmic High-Probability Regret for Online Convex Optimization with Two-Point Bandit Feedback
Standard two-point projected gradient achieves fixed-comparator high-probability logarithmic regret for strongly convex OCO with two-point bandit feedback, with a leading d (not d²) horizon term.
-
HydroGAT: Distributed Heterogeneous Graph Attention Transformer for Spatiotemporal Flood Prediction
A heterogeneous pixel-level graph attention transformer reaches NSE up to 0.97 for hourly flood prediction on two Midwest basins and scales to 64 GPUs.
-
SaGIF: Improving Individual Fairness in Graph Neural Networks via Similarity Encoding
SaGIF adds an independent similarity encoder, initialized from a fused feature-and-topology oracle, to regular GNNs and reports better individual fairness on six benchmark graphs.
-
Vector Representations of Vessel Trees
VeTTA encodes a vascular tree into one vector and recursively decodes it into a geometrically accurate, topologically valid tree, outperforming voxel-based autoencoders on reconstruction metrics.
-
Graph Positional Autoencoders as Self-supervised Learners
A dual-path graph autoencoder that reconstructs node features and Laplacian-eigenvector distances reports strong self-supervised results on heterophilic and molecular benchmarks, with some overstatement in the margins...
-
Simple Path Structural Encoding for Graph Transformers
SPSE replaces random walk edge encodings with simple path counts and outperforms RWSE on most of the tested graph benchmarks, especially for cycle-rich molecular graphs.
-
GITO: Graph-Informed Transformer Operator for Learning Complex Partial Differential Equations
GITO, a graph-informed transformer operator, reports lower relative L2 errors than existing transformer-based neural operators on Navier-Stokes, heat conduction, and airfoil benchmark datasets.
-
SpectralGap: Graph-Level Out-of-Distribution Detection via Laplacian Eigenvalue Gaps
SpecGap adjusts GNN features by subtracting the second-largest Laplacian eigenvector component times the spectral gap, claiming improved graph OOD detection without retraining.
Discussion (0). Sign in to comment.