REVIEW 6 cited by
Gophormer: Ego-Graph Transformer for Node Classification
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Transformers have achieved remarkable performance in a myriad of fields including natural language processing and computer vision. However, when it comes to the graph mining area, where graph neural network (GNN) has been the dominant paradigm, transformers haven't achieved competitive performance, especially on the node classification task. Existing graph transformer models typically adopt fully-connected attention mechanism on the whole input graph and thus suffer from severe scalability issues and are intractable to train in data insufficient cases. To alleviate these issues, we propose a novel Gophormer model which applies transformers on ego-graphs instead of full-graphs. Specifically, Node2Seq module is proposed to sample ego-graphs as the input of transformers, which alleviates the challenge of scalability and serves as an effective data augmentation technique to boost model performance. Moreover, different from the feature-based attention strategy in vanilla transformers, we propose a proximity-enhanced attention mechanism to capture the fine-grained structural bias. In order to handle the uncertainty introduced by the ego-graph sampling, we further propose a consistency regularization and a multi-sample inference strategy for stabilized training and testing, respectively. Extensive experiments on six benchmark datasets are conducted to demonstrate the superiority of Gophormer over existing graph transformers and popular GNNs, revealing the promising future of graph transformers.
Forward citations
Cited by 6 Pith papers
-
DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification
A graph Transformer with dual positional encoding and a star-shaped attention mask yields consistent accuracy improvements on node classification benchmarks.
-
Tokenphormer: Structure-aware Multi-token Graph Transformer for Node Classification
Tokenphormer combines walk-tokens, SGPM-tokens, and hop-tokens in a graph transformer and reports improved node classification accuracy on several benchmark graphs.
-
ReHub: Linear Complexity Graph Transformers with Adaptive Hub-Spoke Reassignment
ReHub keeps spoke-to-hub attention sparse by connecting each node to only k hubs per layer and reassigning those hubs via hub-hub similarity, reaching O(N) complexity with accuracy on par with its dense variant.
-
Even Sparser Graph Transformers
Spexphormer trains a narrow graph transformer to identify important attention edges, then trains a wider model on the sparsified graph, achieving competitive accuracy with far less memory.
-
Rethinking Tokenized Graph Transformers for Node Classification
SwapGT augments tokenized graph Transformers with a random token swapping operation and a center alignment loss, reporting state-of-the-art accuracy on eight node classification datasets.
-
Position-aware Graph Transformer for Recommendation
PGTR combines a GCN recommender with a kernelized graph transformer and four positional encodings, reporting improved Recall and NDCG on four recommendation datasets.
Discussion (0). Continue with ORCID to comment.