The reviewed record of science sign in
Pith

arxiv: 2502.16533 · v2 · pith:XDTHMBMS · submitted 2025-02-23 · cs.LG · cs.AI

A Survey of Graph Transformers: Architectures, Theories and Applications

Chaohao Yuan , Kangfei Zhao , Ercan Engin Kuruoglu , Liang Wang , Tingyang Xu , Wenbing Huang , Deli Zhao , Hong Cheng
show 1 more author
Yu Rong
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:XDTHMBMSrecord.jsonopen to challenge →

classification cs.LG cs.AI
keywords graphtransformersapplicationsarchitecturessurveyfurthermorepracticaltheoretical
0
0 comments X
read the original abstract

Graph Transformers (GTs) have demonstrated a strong capability in modeling graph structures by addressing the intrinsic limitations of graph neural networks (GNNs), such as over-smoothing and over-squashing. Recent studies have proposed diverse architectures, enhanced explainability, and practical applications for Graph Transformers. In light of these rapid developments, we conduct a comprehensive review of Graph Transformers, covering aspects such as their architectures, theoretical foundations, and applications within this survey. We categorize the architecture of Graph Transformers according to their strategies for processing structural information, including graph tokenization, positional encoding, structure-aware attention and model ensemble. Furthermore, from the theoretical perspective, we examine the expressivity of Graph Transformers in various discussed architectures and contrast them with other advanced graph learning algorithms to discover the connections. Furthermore, we provide a summary of the practical applications where Graph Transformers have been utilized, such as molecule, protein, language, vision, traffic, brain and material data. At the end of this survey, we will discuss the current challenges and prospective directions in Graph Transformers for potential future research.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Teaching LLMs to See Graphs: Unifying Text and Structural Reasoning

    cs.LG 2026-05 unverdicted novelty 6.0

    GTLM injects graph-aware attention biases into LLMs using only 0.015% extra parameters, enabling native graph processing that matches 7B models with a 1B model on text-attributed graph benchmarks.

  2. Agentic Fusion of Large Atomic and Language Models to Accelerate Superconductor Discovery

    cs.LG 2026-04 unverdicted novelty 6.0

    An agentic framework fusing large atomic and language models rediscovers 66 known superconductors and guides experimental verification of four new ones with transition temperatures from 2.5 K to 6.5 K.

  3. Different Statistical Perspectives for Understanding Generalisation in Graph Neural Networks

    stat.ME 2026-05 unverdicted novelty 2.0

    The paper reviews three broad statistical perspectives on generalization in GNNs and highlights key results plus limitations for each.