REVIEW 5 cited by
Less is More: on the Over-Globalizing Problem in Graph Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Graph Transformer, due to its global attention mechanism, has emerged as a new tool in dealing with graph-structured data. It is well recognized that the global attention mechanism considers a wider receptive field in a fully connected graph, leading many to believe that useful information can be extracted from all the nodes. In this paper, we challenge this belief: does the globalizing property always benefit Graph Transformers? We reveal the over-globalizing problem in Graph Transformer by presenting both empirical evidence and theoretical analysis, i.e., the current attention mechanism overly focuses on those distant nodes, while the near nodes, which actually contain most of the useful information, are relatively weakened. Then we propose a novel Bi-Level Global Graph Transformer with Collaborative Training (CoBFormer), including the inter-cluster and intra-cluster Transformers, to prevent the over-globalizing problem while keeping the ability to extract valuable information from distant nodes. Moreover, the collaborative training is proposed to improve the model's generalization ability with a theoretical guarantee. Extensive experiments on various graphs well validate the effectiveness of our proposed CoBFormer.
Forward citations
Cited by 5 Pith papers
-
Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement
Crys-JEPA introduces a joint embedding predictive architecture that creates an energy-aware latent space, enabling embedding-based stability screening and a refinement pipeline that yields up to 72.7% gains on the V.S...
-
Distance-Misaligned Training in Graph Transformers and Adaptive Graph-Aware Control
A controllable synthetic benchmark on contextual SBM graphs reveals distance-misaligned training in Graph Transformers, with an oracle adaptive controller improving performance by matching task-specific distance targets.
-
Composable Crystals: Controllable Materials Discovery via Concept Learning
VQ-VAE concept learning enables controllable recombination of crystal motifs to generate structures with reported gains in validity-stability-uniqueness-novelty metrics on MP-20 and Alex-MP-20.
-
Computational Control of Nonlinear Partial Differential Equations Using Machine Learning
A physics-informed neural network method is developed to approximate controls for nonlinear PDEs, including convergence analysis and numerical experiments demonstrating good performance.
-
Computational Control of Nonlinear Partial Differential Equations Using Machine Learning
On synthetic CSBM graphs, a graph-distance bias knob systematically shifts Graph Transformers between over-globalizing and under-reaching regimes, and an oracle target-gap controller tracks the best fixed bias across ...
Discussion (0). Sign in to comment.