Pith. sign in

REVIEW 4 major objections 6 minor 45 references

DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that two simple fixes — attribute-cluster position encoding and an attention mask that lets each neighborhood token see only the target node and itself — turn a tokenized graph Transformer into the best node classifier…

desk verdict Honest incremental graph transformer: hard mask works, but the 'fix' is partly by construction and the gains are modest. read the letter →

arxiv 2505.17660 v1 pith:274LJ5WX submitted 2025-05-23 cs.LG

Chenyang Li , Jinsong Chen , John E. Hopcroft , Kun He This is my paper
classification cs.LG
keywords nodeclassificationgraphtransformerattentionmaskingdualpositionalencodingneighborhoodtokensattention-divertinginterferenceheterophilyscalablelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Neighborhood-aware tokenized graph Transformers build a small token sequence per node from its multi-hop neighborhoods and then run self-attention over those tokens. The paper argues this pipeline currently fails in two specific ways: the tokens barely capture attribute correlations among neighbors, and ordinary self-attention lets high-hop tokens absorb a disproportionate share of attention, so the neighborhood tokens nearly ignore the target node. DAM-GT answers both failures by adding an attribute-aware positional encoding derived from clustering node features, and by masking the attention matrix so that each neighborhood token attends only to the target node and to itself while the target node attends to all tokens. On twelve graphs ranging from thousands to millions of nodes and across a range of homophily levels, the model reports the best classification accuracy in every case. If the claim is right, deliberately restricting attention can improve graph Transformers instead of merely adding more structure.

What carries the argument

The load-bearing component is the mask-aware self-attention matrix $M' = \mathrm{softmax}(\tilde{M})$, where $\tilde{M}_{i,j}$ keeps the original score $M_{i,j}$ only in the first row, first column, and diagonal and sets every other entry to $-\infty$; after the softmax those entries become zero. This star-shaped attention pattern forces each neighborhood token to interact only with the target node and itself, directly counteracting attention diversion. The second component is the dual positional encoding $X^{\mathrm{dup}} = X^{\mathrm{ap}}\|X^{\mathrm{tp}}$: $X^{\mathrm{ap}}$ is obtained by K-means clustering raw node features and weighting each cluster centroid by the cosine similarity between the node and the centroid, while $X^{\mathrm{tp}}$ is the eigenvector matrix of the $m$ smallest nontrivial eigenvalues. The enhanced features $X' = X\|X^{\mathrm{dup}}$ are passed through Hop2Token propagation $N^{(s)} = \hat{A}^{s}X'$, generating the multi-hop token sequences read by the masked Transformer.

What would settle it

Train DAM-GT on a synthetic graph where a node's class depends on the conjunction of features in its 1-hop and 2-hop neighborhoods, and compare it with a variant that additionally allows a learned attention weight between the 2-hop and 1-hop tokens; if the variant clearly outperforms DAM-GT on such graphs, the sufficiency of the star-shaped mask is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that the main failure of neighborhood-aware tokenized graph Transformers is not limited expressiveness but misdirected attention. By visualizing the attention matrices of a trained backbone, the authors find that high-hop neighborhood tokens attract excessive attention, while the target node receives almost none, a pattern they call attention-diverting interference. DAM-GT's mask-aware self-attention keeps only the first row, first column, and diagonal of the attention matrix, so the target node exchanges information with every neighborhood token, each token preserves its own residual information, and all other token-to-token interactions vanish. The second change is a dual positional encoding that concatenates a topology-aware eigenvector encoding with an attribute-aware encoding built from K-means clusters of node features scaled by cosine similarity, letting neighborhood tokens carry both structural and semantic correlations. With these two changes, the paper reports state-of-the-art node classification accuracy on all twelve datasets, with absolute gains of roughly 0.5–2% over the strongest baselines.

Load-bearing premise

The load-bearing premise is that a neighborhood token can learn everything it needs by attending only to the target node and to itself, so that forbidding all other cross-hop token interactions does not discard information that matters for classification.

Editorial extensions

If this is right

  • If the reported results hold, graph Transformers do not need unrestricted token-to-token attention: forcing all attention through the target node can improve node classification and leaves a sparser attention pattern.
  • The attribute-aware positional encoding makes semantic similarity an explicit part of token construction, which should particularly help heterophilous graphs where connected nodes often have different labels; the paper reports strong gains on such datasets.
  • Because the token sequence length is fixed and mini-batch training is retained, the model scales to graphs with millions of nodes with GPU memory bounded by the batch size; the paper demonstrates this on AMiner-CS, Reddit, and Amazon2M.
  • The method is insensitive to the propagation step on small graphs, with accuracy varying by less than 2% across a wide range of steps, which the paper reads as evidence that the design avoids the over-smoothing problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves implicit is a soft or learned mask: instead of zeroing all cross-hop attention, multiply those entries by a learned per-hop weight, so the model can decide when inter-hop interactions are useful; the paper's own limitation section points in this direction.
  • The star-shaped attention pattern is a concrete inductive bias, so a testable prediction is that graphs whose labels require combining features from two different hops will favor a variant that permits at least some inter-hop attention; this can be checked on synthetic graphs built from cross-hop motifs.
  • If the attention-diversion diagnosis is correct, the same masking trick should transfer to other propagation-based tokenized graph Transformers as a drop-in improvement, independent of their positional encoding choices.
  • The attribute-aware positional encoding is agnostic to tokenization, so it could be plugged into node-wise tokenizers or message-passing models; a testable extension is to measure whether the K-means cosine-similarity encoding alone improves heterophilous node classification in those settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies neighborhood-aware tokenized graph Transformers for node classification. It identifies two limitations of existing approaches: (i) neighborhood tokens generated by propagation do not adequately preserve attribute-space correlations, and (ii) standard self-attention over multi-hop tokens is diverted toward high-hop tokens, so the target node receives disproportionately little attention. The proposed DAM-GT addresses these issues by (a) concatenating a topology-based positional encoding with an attribute-cluster centroid encoding, and (b) applying a hard mask to the attention matrix that retains only the first row, first column, and diagonal, thereby forcing each neighborhood token to interact with the target node and itself. The authors report accuracy on 12 datasets, claiming consistent improvements over GNN and graph Transformer baselines, and include ablations on the mask, positional encodings, and propagation depth.

Significance. If the empirical results are reproducible, DAM-GT is a simple and scalable improvement over existing neighborhood-aware graph Transformers, with experiments spanning small to million-node graphs and a range of homophily levels. The paper includes extensive appendices with attention visualizations, complexity analysis, and efficiency measurements, which is a strength. However, the central explanatory claim about attention diversion is substantially built into the mask by construction, and the claimed generality across node-classification tasks is not fully supported without a stress test of the mask's core assumption. The broad benchmark coverage and thoughtful ablations make the contribution potentially useful, but the load-bearing issues below need to be addressed before the claims can be accepted.

major comments (4)
  1. [§4.2, Eq. (6); Appendix B.2] The claim that DAM-GT "overcomes" attention diversion is partly circular. Eq. (6) sets every attention entry outside the first row, first column, and diagonal to negative infinity, so the attention patterns in Figures 7–8 are a direct consequence of the mask rather than evidence that the model has learned to avoid diversion. The accuracy comparison in Table 3 is a legitimate empirical ablation, but the visualizations should not be presented as evidence of learned behavior. I request an additional comparison against a softer mask that reduces diversion while still permitting some cross-hop communication (e.g., multiplying non-target entries by a small trainable or annealed coefficient), with a report of whether the Table 3 gains persist.
  2. [§4.2, Eq. (6); Appendix H] The mask assumes that interactions among different-hop neighborhood tokens are unnecessary or harmful for node classification. The paper itself concedes in Appendix H that the mask "may cause the information loss of multi-hop neighborhoods." The 12 benchmarks do not include a graph where cross-hop complementary information is known to be load-bearing, so the "consistently outperforms" claim is an inductive generalization beyond the tested regimes. A concrete test would be to run DAM-GT and the "-w/o mask" variant on heterophilous datasets with strong multi-hop label dependence (e.g., Chameleon, Squirrel, or a synthetic stochastic block model with cross-hop assortativity) and report whether the mask still helps.
  3. [Tables 1–3] The paper reports mean ± std over ten runs but no statistical significance tests. Several improvements are small relative to the reported standard deviations (e.g., Physics: 97.40 ± 0.04 vs. NAGphormer 97.34 ± 0.03; Reddit: 93.92 ± 0.07 vs. VCR-Graphormer 93.69 ± 0.08). Paired significance tests or confidence intervals over the same seeds are needed to support the claim of consistent superiority, especially for the small gains on datasets such as Physics and Reddit.
  4. [§E.2] Only hyperparameter search ranges are reported; the final per-dataset values (propagation steps S, number of layers, hidden dimension, learning rate, weight decay) are not disclosed. Since Figures 4 and 13 show that S alone changes accuracy by non-negligible amounts and S is tuned per dataset, this omission prevents reproduction of the central results. The final hyperparameter choices for each dataset should be included, for example in an appendix table.
minor comments (6)
  1. [Eq. (6)] The condition "i∗ j = 0" is confusing; it should be written as "i = 0 or j = 0" or "i · j = 0" to match the description of the first row and first column.
  2. [Figures 1, 5–8] The attention visualizations lack colorbars and numeric scales, so claims about "very low" or "normal" attention values cannot be quantitatively assessed. Adding colorbars and reporting aggregate numeric attention values would strengthen the analysis.
  3. [Appendix D.1] The stated complexity O(|E|^(3/2)) for the topology-aware positional encoding is unexplained and likely incorrect for exact eigen-decomposition, which is generally O(n^3) for dense methods or higher for direct sparse eigensolvers. Please justify the estimate or correct it.
  4. [§4.1] The statement that Eq. (2) "completely preserve[s] the both absolute and relative positional relations" overstates what a cluster-centroid scaling can represent; the encoding captures cluster identity and cosine similarity to the centroid, not arbitrary relative positions.
  5. [Table 3] Table 3 reports accuracy gains without standard deviations, while Tables 1 and 2 include them. Adding standard deviations for the ablation runs would allow the reader to assess the stability of the mask's contribution.
  6. [Appendix A.2] DUALFormer [43] is cited in the related work but its relation to the proposed dual positional encoding is not discussed; a brief comparison would help position the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

Independent benchmark evidence supports the central SOTA claim; the hard mask is a transparent design constraint, and the authors' self-citations are building blocks rather than circular load-bearing arguments.

full rationale

DAM-GT's primary claim, consistent accuracy gains over state-of-the-art methods on 12 datasets, is an empirical claim validated against external baselines in Tables 1 and 2 and an internal ablation in Table 3. The derivation chain is constructive rather than circular: the dual positional encoding concatenates raw features with a k-means-derived attribute encoding and a topology eigenvector encoding via Eqs. (1)-(3), and the mask-aware attention in Eq. (6) is a deliberate hard constraint, not a fitted parameter disguised as a prediction. The paper does not claim the mask predicts attention patterns from first principles; it claims the mask enforces a target interaction pattern, which is transparent. The attention visualizations in Appendix B.2 are therefore a sanity check of the design rather than a circular derivation, and the accuracy gains in Table 3 are separate empirical evidence. The self-citations to NAGphormer [5] supply the backbone components (Hop2Token, topology-aware positional encoding, readout) and are appropriately cited and also used as a baseline; no load-bearing uniqueness theorem is imported from the authors' prior work. Appendix H explicitly concedes that blocking inter-hop interactions may cause information loss of multi-hop neighborhoods; this is a legitimate generalization limitation, not a circular step. No fitted parameter is renamed as a prediction, and no result reduces by construction to its own inputs. Hence the paper shows no significant circularity.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The model relies on tuned hyperparameters and a few design assumptions inherited from NAGphormer or introduced ad hoc. No new physical or conceptual entities are introduced.

free parameters (4)
  • Number of K-means clusters k = Set to number of classes c
    Used in attribute-aware positional encoding; chosen to match class count, which assumes attribute clusters align with classes and introduces label cardinality as a design choice.
  • Propagation steps S = Tuned per dataset, e.g., 3-5 for small graphs, 10-20 for large
    Controls the number of neighborhood tokens; the paper tunes it and shows performance varies with graph scale.
  • Hidden dimension dm = Chosen from {128, 256, 512, 768}
    Standard hyperparameter for the Transformer backbone.
  • Number of layers L = Chosen from {1, 2}
    Standard hyperparameter for the Transformer backbone.
assumptions (3)
  • domain assumption The normalized adjacency matrix eigenvector basis (topology PE) captures meaningful structural positions.
    Inherited from NAGphormer; the paper assumes the m smallest non-trivial eigenvectors provide useful positional information.
  • ad hoc to paper K-means with k = number of classes produces clusters that reflect attribute correlations useful for classification.
    This is a new, untested assumption; using the label count as the cluster count may not hold on heterophilous graphs.
  • ad hoc to paper The star-shaped attention mask (target plus self only) preserves sufficient information for node classification.
    The paper's own Appendix H acknowledges this may cause information loss from inter-hop interactions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification." pith.science (2026). https://pith.science/paper/274LJ5WX

@misc{pith2026250517660,
  author       = {Pith},
  title        = {Pith review of: DAM-GT: Dual Positional Encoding-Based Attention Masking Graph Transformer for Node Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/274LJ5WX}},
  note         = {Machine review of arXiv:2505.17660}
}
read the original abstract

Neighborhood-aware tokenized graph Transformers have recently shown great potential for node classification tasks. Despite their effectiveness, our in-depth analysis of neighborhood tokens reveals two critical limitations in the existing paradigm. First, current neighborhood token generation methods fail to adequately capture attribute correlations within a neighborhood. Second, the conventional self-attention mechanism suffers from attention diversion when processing neighborhood tokens, where high-hop neighborhoods receive disproportionate focus, severely disrupting information interactions between the target node and its neighborhood tokens. To address these challenges, we propose DAM-GT, Dual positional encoding-based Attention Masking graph Transformer. DAM-GT introduces a novel dual positional encoding scheme that incorporates attribute-aware encoding via an attribute clustering strategy, effectively preserving node correlations in both topological and attribute spaces. In addition, DAM-GT formulates a new attention mechanism with a simple yet effective masking strategy to guide interactions between target nodes and their neighborhood tokens, overcoming the issue of attention diversion. Extensive experiments on various graphs with different homophily levels as well as different scales demonstrate that DAM-GT consistently outperforms state-of-the-art methods in node classification tasks.

Figures

Figures reproduced from arXiv: 2505.17660 by the authors.

Figure 1
Figure 1. The attention matrices on Photo and Reddit datasets. Deeper colors represent higher [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall view of our DAM-GT. Given the input graph, DAM-GT first utilizes dual [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison results of DAM-GT and its variants without positional encoding. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Study on the propagation steps S. large-scale graphs are shown in [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The attention matrix of all four heads on Photo dataset in the backbone. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The attention matrix of all four heads on Reddit dataset in the backbone. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: The attention matrix of all four heads on Photo dataset in DAM-GT. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The attention matrix of all four heads on Reddit dataset in DAM-GT. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Illustration of the proposed mask-aware self-attention mechanism. To obtain the mask [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Masking strategies of DAM-GT and its three variants. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Comparison results of DAM-GT and its variants with different masking strategies. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Comparison results of DAM-GT and its variants without positional encoding. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 43 canonical work pages

  1. [1]

    Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Nazanin Alipourfard, Kristina Lerman, Hrayr Harutyunyan, Greg Ver Steeg, and Aram Galstyan. 2019. MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing. In Proceedings of the 36th International Conference on Machine Learning, ICML, V ol. 97. 21–29

    work page 2019

  2. [2]

    Uri Alon and Eran Yahav. 2021. On the Bottleneck of Graph Neural Networks and its Practical Implications. In Proceedings of the International Conference on Learning Representations, ICLR

    work page 2021

  3. [3]

    Aleksandar Bojchevski, Johannes Klicpera, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, and Stephan Günnemann. 2020. Scaling Graph Neural Net- works with Approximate Pagerank. In Proceedings of the ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining. 2464–2473

    work page 2020

  4. [4]

    Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020. Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks From the Topological View. In Proceedings of the AAAI Conference on Artificial Intelligence, AAAI. 3438–3445

    work page 2020

  5. [5]

    Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. 2023. NAGphormer: A Tokenized Graph Transformer for Node Classification in Large Graphs. InProceedings of the International Conference on Learning Representations, ICLR

    work page 2023

  6. [6]

    Jinsong Chen, Siyu Jiang, and Kun He. 2024. NTFormer: A Composite Node Tokenized Graph Transformer for Node Classification. CoRR abs/2406.19249 (2024)

    arXiv 2024

  7. [7]

    Jinsong Chen, Boyu Li, and Kun He. 2024. Neighborhood convolutional graph neural network. Knowledge-Based Systems 295 (2024), 111861

    work page 2024

  8. [8]

    Jinsong Chen, Boyu Li, Qiuting He, and Kun He. 2024. PAMT: A Novel Propagation-Based Approach via Adaptive Similarity Mask for Node Classification. IEEE Trans. Comput. Soc. Syst. 11, 5 (2024), 5973–5983

    work page 2024

Show all 45 references
  1. [9]

    Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. 2020. Simple and Deep Graph Convolutional Networks. In Proceedings of the International Conference on Machine Learning, ICML. 1725–1735

    2020

  2. [10]

    Yuhan Chen, Yihong Luo, Jing Tang, Liang Yang, Siya Qiu, Chuan Wang, and Xiaochun Cao. 2023. LSGNN: Towards General Graph Neural Network in Node Classification by Local Similarity. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, I...

    2023

  3. [11]

    Eli Chien, Jianhao Peng, Pan Li, and Olgica Milenkovic. 2021. Adaptive Universal Generalized PageRank Graph Neural Network. In Proceedings of the International Conference on Learning Representations, ICLR

    2021

  4. [12]

    Hande Dong, Jiawei Chen, Fuli Feng, Xiangnan He, Shuxian Bi, Zhaolin Ding, and Peng Cui

  5. [13]

    Vijay Prakash Dwivedi and Xavier Bresson. 2020. A Generalization of Transformer Networks to Graphs. arXiv preprint arXiv:2012.09699 (2020)

    2020 arXiv

  6. [14]

    Wenzheng Feng, Yuxiao Dong, Tinglin Huang, Ziqi Yin, Xu Cheng, Evgeny Kharlamov, and Jie Tang. 2022. GRAND+: Scalable Graph Random Neural Networks. In Proceedings of the Web Conference, WWW. 3248–3258

    2022

  7. [15]

    Dongqi Fu, Zhigang Hua, Yan Xie, Jin Fang, Si Zhang, Kaan Sancak, Hao Wu, Andrey Malevich, Jingrui He, and Bo Long. 2024. VCR-Graphormer: A Mini-batch Graph Transformer via Virtual Connections. In Proceedings of the Twelfth International Conference on Learning Representations,...

    2024

  8. [16]

    Schoenholz, Patrick F

    Justin Gilmer, Samuel S. Schoenholz, Patrick F. Riley, Oriol Vinyals, and George E. Dahl. 2017. Neural Message Passing for Quantum Chemistry. In International Conference on Machine Learning, ICML. 1263–1272

    2017

  9. [17]

    Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive Representation Learning on Large Graphs. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS. 1024–1034

    2017

  10. [18]

    Qiuting He, Jinsong Chen, Hao Xu, and Kun He. 2022. Structural Robust Label Propagation on Homogeneous Graphs. In IEEE International Conference on Data Mining, ICDM. 181–190

    2022

  11. [19]

    Weiyu Ji, Xiangwu Meng, and Yujie Zhang. 2022. SPATM: A Social Period-Aware Topic Model for Personalized Venue Recommendation. IEEE Transactions on Knowledge and Data Engineering, TKDE 34, 8 (2022), 3997–4010

    2022

  12. [20]

    Weiyu Ji, Xiangwu Meng, and Yujie Zhang. 2022. STARec: Adaptive Learning with Spatiotem- poral and Activity Influence for POI Recommendation. ACM Transactions on Information Systems 40, 4 (2022), 65:1–65:40

    2022

  13. [21]

    Wei Jin, Tyler Derr, Yiqi Wang, Yao Ma, Zitao Liu, and Jiliang Tang. 2021. Node Similarity Preserving Graph Convolutional Networks. InProceedings of the ACM International Conference on Web Search and Data Mining, WSDM. 148–156

    2021

  14. [22]

    Thomas N Kipf and Max Welling. 2017. Semi-supervised Classification with Graph Convolu- tional Networks. In Proceedings of the International Conference on Learning Representations, ICLR

    2017

  15. [23]

    Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then Propa- gate: Graph Neural Networks meet Personalized PageRank. In Proceedings of the International Conference on Learning Representations, ICLR

    2019

  16. [24]

    Matthias Kohn, Marcel Hoffmann, and Ansgar Scherp. 2024. Edge-Splitting MLP: Node Classification on Homophilic and Heterophilic Graphs without Message Passing. InProceedings of Learning on Graphs, LoG. 1–21

    2024

  17. [25]

    Chuang Liu, Yibing Zhan, Xueqi Ma, Liang Ding, Dapeng Tao, Jia Wu, and Wenbin Hu. 2023. Gapformer: Graph Transformer with Graph Pooling for Node Classification. In Proceedings of the International Joint Conference on Artificial Intelligence, IJCAI. 2196–2205

    2023

  18. [26]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled Weight Decay Regularization. In Proceed- ings of the International Conference on Learning Representations, ICLR

    2019

  19. [27]

    Jiahong Ma, Mingguo He, and Zhewei Wei. 2024. PolyFormer: Scalable Node-wise Filters via Polynomial Graph Transformer. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Ricardo Baeza-Yates and Francesco Bonchi (Eds.). 2118–2129

    2024

  20. [28]

    Hongbin Pei, Bingzhe Wei, Kevin Chen-Chuan Chang, Yu Lei, and Bo Yang. 2020. Geom-GCN: Geometric Graph Convolutional Networks. In Proceedings of the International Conference on Learning Representations, ICLR

    2020

  21. [29]

    Ladislav Rampásek, Mikhail Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. 2022. Recipe for a General, Powerful, Scalable Graph Transformer. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS , V ol. 35. 14501–14515

    2022

  22. [30]

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of the International Conference on Learning Representations, ICLR

    2018

  23. [31]

    Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. AM-GCN: Adaptive Multi-channel Graph Convolutional Networks. InProceedings of the ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1243–1253. 11

    2020

  24. [32]

    Zehong Wang, Zheyuan Zhang, Chuxu Zhang, and Yanfang Ye. 2025. Training mlps on graphs without supervision. In Proceedings of the ACM International Conference on Web Search and Data Mining, WSDM. 697–706

    2025

  25. [33]

    Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019. Simplifying Graph Convolutional Networks. In Proceedings of the International Conference on Machine Learning, ICML. 6861–6871

    2019

  26. [34]

    Qitian Wu, Wentao Zhao, Zenan Li, David Wipf, and Junchi Yan. 2022. NodeFormer: A Scalable Graph Structure Learning Transformer for Node Classification. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS, V ol. 35. 27387–27401

    2022

  27. [35]

    Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian, and Junchi Yan. 2023. Simplifying and empowering transformers for large-graph representations. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS

    2023

  28. [36]

    Yujie Xing, Xiao Wang, Yibo Li, Hai Huang, and Chuan Shi. 2024. Less is More: on the Over- Globalizing Problem in Graph Transformers. In Proceedings of the International Conference on Machine Learning, ICML

    2024

  29. [37]

    Fan Xu, Nan Wang, Hao Wu, Xuezhi Wen, Xibin Zhao, and Hai Wan. 2024. Revisiting Graph-Based Fraud Detection in Sight of Heterophily and Spectrum. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI. 9214–9222

    2024

  30. [38]

    Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken-ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation Learning on Graphs with Jumping Knowledge Networks. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learni...

    2018

  31. [39]

    Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021. Do Transformers Really Perform Badly for Graph Representation. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS. 28877– 28888

    2021

  32. [40]

    Prasanna

    Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor K. Prasanna

  33. [41]

    Zaixi Zhang, Qi Liu, Qingyong Hu, and Chee-Kong Lee. 2022. Hierarchical Graph Transformer with Adaptive Node Sampling. InProceedings of the Advances in Neural Information Processing Systems, NeurIPS. 21171–21183

    2022

  34. [42]

    Jianan Zhao, Chaozhuo Li, Qianlong Wen, Yiqi Wang, Yuming Liu, Hao Sun, Xing Xie, and Yanfang Ye. 2021. Gophormer: Ego-Graph Transformer for Node Classification. arXiv preprint arXiv:2110.13094 (2021)

    2021 arXiv

  35. [43]

    DAM-GT-H

    Jiaming Zhuo, Yuwei Liu, Yintong Lu, Ziyi Ma, Kun Fu, Chuan Wang, Yuanfang Guo, Zhen Wang, Xiaochun Cao, and Liang Yang. 2025. DUALFormer: Dual Graph Transformer. In Proceedings of the International Conference on Learning Representations. 12 A Related Work A.1 Graph Neural Net...

    2025

  36. [2020]

    In Proceedings of the International Conference on Learning Representations, ICLR

    GraphSAINT: Graph Sampling Based Inductive Learning Method. In Proceedings of the International Conference on Learning Representations, ICLR

  37. [2021]

    In Proceedings of the Web Conference, WWW

    On the Equivalence of Decoupled Graph Convolution Network and Label Propagation. In Proceedings of the Web Conference, WWW. 3651–3662

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.