Position-aware Graph Transformer for Recommendation

Chongming Gao; Jiajia Chen; Jiancan Wu; Jiawei Chen; Xiang Wang; Yong Li

REVIEW 4 major objections 5 minor 55 references

Position-aware Graph Transformer for Recommendation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PGTR claims that adding a global-attention Transformer block with four recommendation-specific position encodings to any GCN recommender captures long-range collaborative signals and improves ranking accuracy.

desk verdict Useful wrapper with well-designed CF positional encodings, but the gains are confounded with SSM and the 'Transformer' branch has no learned attention. read the letter →

arxiv 2412.18731 v1 pith:QEWOEXRN submitted 2024-12-25 cs.IR

Jiajia Chen , Jiancan Wu , Jiawei Chen , Chongming Gao , Yong Li , Xiang Wang This is my paper · ORCID

classification cs.IR

keywords recommendersystemsgraphtransformercollaborativefilteringpositionalencodingconvolutionalnetworkslong-rangesignalssampledsoftmaxlosssparsityrobustness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that GCN-based recommender systems miss long-range collaborative signals, the connections that matter when a user's true preference lies several hops away in the interaction graph, and that this gap can be closed by adding a Transformer block with position encodings designed for user-item graphs. The proposed method, PGTR, wraps any GCN backbone, injects four kinds of positional information into node representations, lets a global-attention Transformer model all user-item pairs, and linearly recombines the global signal with the backbone's local neighborhood features. Across four datasets and four backbones (NGCF, GCCF, LightGCN, UltraGCN), the paper reports consistent gains in Recall@20 and NDCG@20, with smaller performance drops than baselines under data sparsity and injected noise. If correct, the framework offers a model-agnostic upgrade path for existing GCN recommenders that does not require redesigning the backbone.

What carries the argument

The central machinery is the combination of the Nodeformer kernelized attention with four purpose-designed positional encodings: spectral encoding (Laplacian eigenvectors of the full user-item graph plus user-side and item-side second-order graphs), degree encoding (nodes binned by interaction count), PageRank encoding (nodes binned by PageRank score), and a type encoding that separates users from items. These encodings are injected at the initial embedding and re-injected after each GCN layer; the Transformer then computes global attention over all nodes in linear time via the Nodeformer approximation, and the output is mixed with the GCN output by a scalar weight $\lambda_3$. The mechanism's role is to give the Transformer positional and structural context that vanilla self-attention lacks, allowing it to aggregate long-range collaborative signal without treating the graph as a bag of nodes.

What would settle it

Replace the Transformer attention in PGTR with a fixed global average of position-encoded embeddings (no learned query/key/value), keeping the same positional encodings and sampled-softmax loss; if Recall@20 does not fall on LastFM and Amazon-book, the long-range attention mechanism is not what produces the reported improvement.

Watch

Extended reading notes

Core claim

The paper sets out to show that GCN-based collaborative filtering is limited to local, few-hop information because of over-smoothing and over-squashing, and that this is fixable by a graph-transformer layer with positional encodings tailored to the user-item bipartite graph. PGTR injects four encodings — spectral (Laplacian eigenvectors of the full graph and of user-side and item-side subgraphs), degree, PageRank, and type — into the initial node embeddings and again after each GCN layer. A Transformer block using Nodeformer's kernelized attention then computes global relations over all users and items, and the resulting global representation is linearly mixed with the backbone's local output through a scalar weight. The paper reports that wrapping NGCF, GCCF, LightGCN, and UltraGCN with PGTR improves Recall@20 and NDCG@20 on Amazon-elec, Douban-book, LastFM, and Amazon-book, and that the method degrades less than baselines under sparsity and injected noise.

Load-bearing premise

The conclusion depends on the reported accuracy gains being real rather than run-to-run noise and on the global attention mechanism being the actual cause, but the experiments have no error bars and the ablation never removes the Transformer while keeping the positional encodings and the sampled-softmax loss.

Editorial extensions

If this is right

Any GCN-based recommender can be upgraded to PGTR without changing its local message-passing design; the paper demonstrates this on NGCF, GCCF, LightGCN, and UltraGCN.
Low-degree users and items, which GCNs model poorly with only a few hops, can receive signals from distant neighbors through the global attention path.
Because the positional encodings add far fewer parameters than the embedding table, the upgrade is parameter-efficient for large interaction graphs.
The robustness results suggest that explicit global modeling absorbs some of the damage caused by noisy interactions, which are common in real implicit feedback data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's own ablation shows that switching to the sampled-softmax loss accounts for a large share of the improvement (e.g., NGCF Recall on LastFM rises from 0.0263 to 0.0587 with SSM alone), so a simpler loss-only upgrade may capture much of the benefit; the paper does not test the Transformer on top of a fully tuned non-SSM baseline.
The degree and PageRank encodings are defined by uniform binning, which discards fine-grained distinctions; learned continuous encodings or quantile-based bins could plausibly perform better or reveal which nodes benefit most.
The same architecture could be applied to other bipartite graphs, such as social networks or knowledge graphs, where long-range dependencies and node-type asymmetry also matter.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a useful engineering wrapper—CF-tailored positional encodings on top of GCN backbones—but the main claim in the abstract, that a Transformer block gives long-range collaborative signals, is not supported by the evidence as presented. The gains are largely explained by the sampled softmax (SSM) loss, and the global branch as implemented omits all learned projections, so calling it a Transformer oversells it.

What's genuinely new: the four encodings (spectral on the user-item graph plus user-side and item-side adjacency, grouped degree, grouped PageRank, and type) are thoughtfully matched to collaborative filtering, and the wrapper is simple and model-agnostic. The experiments cover four datasets, four GCN backbones, and include ablations of each encoding. The encodings do seem to contribute—the ablation in Figure 7 shows degradation when each is removed. That's a real, if incremental, contribution.

The soft spots are in the attribution. Table 3 is the smoking gun: on LastFM, switching NGCF from its original loss to SSM takes Recall from 0.0263 to 0.0587, while PGTR adds only 0.003 on top. For LightGCN, SSM alone nearly matches PGTR. The main Table 1 does not include the SSM-only baselines, so the headline 'improve' row overstates what the architecture specifically adds. The paper does include Table 3 as an ablation, but even there the gains over SSM are small and no error bars are given, so I cannot tell whether they are real.

Second, the 'Transformer' is not a transformer in any standard sense. Section 3.2 states that W_Q, W_K, and W_V are omitted in the implementation. With those gone, Eq. (19) reduces to a random-feature kernel smoother: the attention weights are determined by random projections, not learned. Algorithm 1 still lists these as learnable parameters, which is inconsistent. If the implementation really omits them, the module is a positional-feature-weighted global average, not an attention mechanism.

Third, the evaluation is single-seed and single-split. The sparsity curves in Figure 5 change the test set as the training fraction changes (80/20, 60/40, etc.), so the robustness comparison is not on a fixed test distribution. The 'improve' row in Table 1 also compares against different baselines per metric (e.g., best Recall vs best NDCG), which flatters the results.

None of this is a fatal mathematical error; the model is coherent once you know the projections are gone. But the manuscript overclaims. A revision needs to: report multiple seeds, compare against backbones trained with the same SSM loss, isolate the global branch (e.g., lambda_3=0 with encodings and SSM, or a control that removes the global branch entirely), and either re-introduce learnable Q/K/V or rename the module.

Verdict: I'd send it to peer review—the encodings and wrapper are worth discussing—but with a request for major revision. The central attribution claim needs to be redone. As is, I wouldn't cite it as evidence for graph-transformer-based recommendation.

Referee Report

4 major / 5 minor

Summary. The paper proposes PGTR, a model-agnostic framework that augments GCN-based recommender systems with a global Transformer-style branch. The architecture injects four graph-derived positional encodings (spectral, degree, PageRank, and type) into node embeddings, passes them through a kernelized attention module derived from Nodeformer, and linearly mixes the resulting global representations with the local GCN representations. The model is trained with a sampled softmax loss. Experiments are reported on four datasets with four GCN backbones (NGCF, GCCF, LightGCN, UltraGCN), along with ablations and robustness studies, claiming consistent improvements in Recall@20 and NDCG@20 as well as robustness against sparsity and noise.

Significance. If the empirical claims are substantiated, PGTR could be a useful plug-in for boosting GCN-based recommenders, and the proposed recommendation-specific positional encodings are a reasonable and potentially transferable contribution. Strengths include the clear modular design, the use of four distinct positional encodings tailored to bipartite user-item graphs, the breadth of backbones tested, and the explicit analysis of the sampled softmax loss's role. However, the current evidence does not adequately support the central attribution to the Transformer's global modeling: the gains are partly explained by the SSM loss, no ablation isolates the Transformer branch, and several reported improvements are small and lack statistical verification. The sparsity robustness experiment is also confounded by changing test sets. The paper's architecture description contains an unresolved inconsistency about the Q/K/V projections. These issues are fixable, but the central claims require additional experiments and clarification.

major comments (4)

[§4.4.3, Table 3] The paper's central claim that the Transformer block's global modeling is responsible for the improvements is not isolated. Table 3 shows that switching to the sampled softmax (SSM) loss alone accounts for most of the improvement on LastFM: LightGCN Recall rises from 0.0535 to 0.0582 with SSM, versus 0.0610 for LightGCN-PGTR, and NGCF from 0.0263 to 0.0587 with SSM, versus 0.0616 for NGCF-PGTR. No ablation removes the Transformer branch while keeping SSM and the positional encodings, and the lambda_3 sweep in Figure 8 does not include lambda_3=0 (pure local) or any measure of variance. Without such a control, the abstract's claim that long-range collaborative signals from the Transformer block drive the improvement is not supported.
[§3.2, Eq. (19), Algorithm 1] The implementation of the global branch is ambiguous. The text after Eq. (19) states that W_Q, W_K, and W_V are omitted 'for simplicity', yet Eq. (19) presents them as learnable projections and Algorithm 1 lists them in the learnable parameter set Theta and updates them in the training loop. If these projections are indeed omitted, the global branch reduces to a random-feature weighted average of node embeddings with no learned attention; this is not a 'Transformer block' in the usual sense. Please specify exactly which of W_Q, W_K, W_V are learned or set to identity, remove the inconsistency with Algorithm 1, and either implement the full architecture or revise the terminology and discuss the consequences for the claim of Transformer-based global modeling.
[§4.3.1, Figure 5] The sparsity robustness experiments vary the training proportion (20%, 40%, 60%, 80%) and test on the complementary portion of the data, so the test sets differ across conditions. For example, at 20% training, the test set is 80% of the data, while at 80% training, the test set is only 20% of the data. The resulting curves therefore conflate model robustness with the size and composition of the test set. A fixed test set (or a fixed set of test users/items) should be held out while varying the amount of training interactions, so that the claim of robustness against interaction sparsity can be evaluated on a constant evaluation basis.
[Tables 1 and 3, Figures 5–8] All reported results appear to be from single runs; no standard deviations, number of seeds, or significance tests are provided. Several improvements over the best baselines are small, e.g., LastFM Recall 0.0629 vs 0.0612 (2.8%) and NDCG 0.1395 vs 0.1361 (2.5%). Without multiple seeds and a paired significance test, the claim that PGTR 'significantly improves' recommendation accuracy is not verifiable. Please report mean and standard deviation over at least three seeds and perform significance testing for the core comparisons in Table 1 and for the robustness studies.

minor comments (5)

[Table 1] The 'improve' row is not defined in the text or caption. State explicitly that it is the relative improvement over the best baseline for each metric/dataset and note that the baseline may differ across columns; otherwise the reader cannot verify the percentages.
[§4.1] The data splitting description is ambiguous: 'we divide the dataset into 20%/40%/60%/80% as the training set' followed by 'Within the training set, 20% is used as a validation set' leaves it unclear whether the validation set is a subset of the stated training fraction or an additional portion. Clarify how the effective training, validation, and test fractions are defined for each condition.
[§3.2, Eq. (23)] Equation (23) refers to 'normalized representations' s_u and s_i but does not specify the norm. State explicitly that L2 normalization is applied or rename the variables to avoid ambiguity.
[§4.2] The phrase 'training:testing=2:8' is inconsistent with the validation split described in §4.1. Use a consistent notation that accounts for the validation set used in hyper-parameter tuning.
[§3.1.2] The claim that similar popularity implies similar contexts is presented as an intuitive justification without citation or evidence; it is acceptable as a heuristic but should be framed as a design choice rather than an established finding.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PGTR is an empirical architecture paper whose results are benchmarked against external baselines and do not reduce by construction to fitted inputs.

full rationale

The paper makes an architectural and empirical claim, not a derivation from first principles. PGTR combines GCN backbones with a kernelized Transformer convolution and hand-designed positional encodings; the final node representation is a linear mixture of local and global branches (Eq. 21), and the model is trained with the sampled softmax loss (Eq. 23). The central comparison in Table 1 is against external baselines tested on the same datasets, and the ablations in Table 3 and Figure 7 attribute parts of the gain to the SSM loss and to the positional encodings rather than concealing them. The only notable internal issues are attribution and reporting weaknesses, not circularity: the Transformer-specific contribution is not isolated by an ablation that removes the Transformer branch while keeping the SSM loss and positional encodings, the paper notes that W_Q, W_K, and W_V are omitted after presenting them in Eq. (19) while Algorithm 1 still lists them as learnable, and no seed variance is reported. These are empirical rigor concerns and do not constitute a self-definitional reduction. The SSM loss is cited to prior work with overlapping authors, but it is used as a training objective with independent prior validation and is not the target result being derived; its contribution is explicitly measured against the PGTR result. No load-bearing step equates a prediction with a fitted parameter or imports a uniqueness theorem from self-citations. Accordingly, the circularity score is 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The model rests on a stack of borrowed approximations: kernelized attention from Nodeformer, Laplacian and degree-style positional encodings from graph transformer literature, and a validation-selected set of hyperparameters. No new physical or conceptual entities are introduced. The main unpaid debts are the fidelity of the kernel attention to true attention and the stability of spectral encodings on sparse bipartite graphs.

free parameters (8)

H_C spectral encoding dimension = default 50, searched in {1,10,50,100,150,200,250,300}
Number of Laplacian eigenvectors used as positional features; chosen by validation search, affects how much graph spectral structure is injected.
H_D, H_R, H_Y encoding dimensions = default 4, searched in {2,4,8,16,32,64}
Dimensions of degree, PageRank, and type encodings; validation-selected and central to the proposed positional encoding block.
N_d and N_r group counts = default 10, searched in {1,5,10,15,20}
Number of groups for degree and PageRank encodings; controls granularity and is acknowledged by the authors to be imperfect at group boundaries.
lambda_C, lambda_1, lambda_2 = searched in {0,1}
Control how spectral, degree, and PageRank positional encodings are combined and injected into embeddings.
lambda_3 mixing ratio = tuned on validation, values shown in Figure 8
Controls the balance between local GCN features and global Transformer features; the paper's own ablation shows performance is sensitive to it.
tau temperature in SSM loss = searched in coarse grid {0.02,0.1,0.2,...,1.0,1.2}
Scales sampled softmax logits and affects training dynamics.
random feature dimension m in Nodeformer kernel = not reported
Determines approximation quality of the kernelized attention in Eq. (5); omitted from the paper, making exact reproduction impossible.
number of layers L = implied 2 for baselines, not clearly swept for PGTR
Depth of the combined GCN and Transformer mixing; not explicitly tuned or reported for the full PGTR model.

assumptions (5)

domain assumption Random-feature kernelized attention in Eq. (5) accurately approximates full Transformer attention
Borrowed from Nodeformer; the paper does not quantify approximation error in the recommendation setting.
domain assumption Laplacian eigenvectors of the user-item, user-side, and item-side graphs are meaningful positional encodings under extreme sparsity
Section 3.1.1 constructs these encodings, but no stability analysis is given for low-degree nodes on very sparse graphs.
domain assumption Degree and PageRank grouping provide useful inductive biases for collaborative filtering
Sections 3.1.2 and 3.1.3 rely on this hypothesis; the authors themselves note boundary users and items may be misassigned to groups.
domain assumption Second-order user-user and item-item adjacency derived from observed interactions captures meaningful relations
Section 3.1.1, Eq. (8), builds user-side and item-side graphs by treating shared neighbors as relations, which may be noisy for sparse data.
domain assumption Observed interactions are ground truth for training and evaluation, and injected noise is distinguishable from real signal
Standard collaborative filtering assumption; the noise robustness experiments rely on the injected noise being independent of user intent.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Position-aware Graph Transformer for Recommendation." pith.science (2026). https://pith.science/paper/QEWOEXRN

@misc{pith2026241218731,
  author       = {Pith},
  title        = {Pith review of: Position-aware Graph Transformer for Recommendation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QEWOEXRN}},
  note         = {Machine review of arXiv:2412.18731}
}

read the original abstract

Collaborative recommendation fundamentally involves learning high-quality user and item representations from interaction data. Recently, graph convolution networks (GCNs) have advanced the field by utilizing high-order connectivity patterns in interaction graphs, as evidenced by state-of-the-art methods like PinSage and LightGCN. However, one key limitation has not been well addressed in existing solutions: capturing long-range collaborative filtering signals, which are crucial for modeling user preference. In this work, we propose a new graph transformer (GT) framework -- \textit{Position-aware Graph Transformer for Recommendation} (PGTR), which combines the global modeling capability of Transformer blocks with the local neighborhood feature extraction of GCNs. The key insight is to explicitly incorporate node position and structure information from the user-item interaction graph into GT architecture via several purpose-designed positional encodings. The long-range collaborative signals from the Transformer block are then combined linearly with the local neighborhood features from the GCN backbone to enhance node embeddings for final recommendations. Empirical studies demonstrate the effectiveness of the proposed PGTR method when implemented on various GCN-based backbones across four real-world datasets, and the robustness against interaction sparsity as well as noise.

Figures

Figures reproduced from arXiv: 2412.18731 by the authors.

**Figure 1.** An example of long-range collaborative signals in movie recommendation, where the target user may be interested in the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Overall framework of our PGTR model. 3.1 Proposed Positional Encodings In practice, positional encodings are widely used in the GT for improving the capability of modeling the relations among nodes [2, 12, 22, 53]. They provide the graph structure and position information of nodes within the Transformer module. However, they are not tailored for the field of collaborative recommendation. For example, it is crucial t… view at source ↗

**Figure 3.** The procedure of Degree Encoding for users and items. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: The procedure of PageRank Encoding for users and items. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Performance comparison on Douban-book and LastFM datasets under different proportions of the training set. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Performance comparison on Amazon-elec and Douban-book datasets with different noise perturbations. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: Performance of ablated models on Amazon-elec and Douban-book datasets. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

**Figure 8.** Figure 8: Effect of 𝜆3 on LastFM and Amazon-book datasets. Manuscript submitted to ACM [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 36 canonical work pages

[1]

Uri Alon and Eran Yahav. 2020. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations

work page 2020
[2]

Deng Cai and Wai Lam. 2020. Graph transformer for graph-to-sequence learning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 7464–7471

work page 2020
[3]

Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2022. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In The Eleventh International Conference on Learning Representations

work page 2022
[4]

Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In The world wide web conference . 151–161

work page 2019
[5]

Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 34. 3438–3445

2020
[6]

Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. 2022. NAGphormer: A tokenized graph transformer for node classification in large graphs. In The Eleventh International Conference on Learning Representations

work page 2022
[7]

Jiajia Chen, Jiancan Wu, Jiawei Chen, Xin Xin, Yong Li, and Xiangnan He. 2023. How Graph Convolutions Amplify Popularity Bias for Recommen- dation? arXiv preprint arXiv:2305.14886 (2023)

work page Pith review arXiv 2023
[8]

Jiajia Chen, Xin Xin, Xianfeng Liang, Xiangnan He, and Jun Liu. 2022. GDSRec: Graph-Based Decentralized Collaborative Filtering for Social Recommendation. IEEE Transactions on Knowledge and Data Engineering 35, 5 (2022), 4813–4824

work page 2022

Show all 55 references

[9]

Lei Chen, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2020. Revisiting graph based collaborative filtering: A linear residual graph convolutional network approach. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 27–34

2020
[10]

Krzysztof Marcin Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Quincy Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. 2020. Rethinking Attention with Performers. In International Conference on Learning Representations

2020
[11]

Aaron Clauset, Cosma Rohilla Shalizi, and Mark EJ Newman. 2009. Power-law distributions in empirical data. SIAM review 51, 4 (2009), 661–703

2009
[12]

Vijay Prakash Dwivedi and Xavier Bresson. 2020. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 (2020)

2020 arXiv
[13]

Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S Yu. 2021. Continuous-time sequential recommendation with temporal graph collaborative transformer. In Proceedings of the 30th ACM international conference on information & knowledge management . 433–442

2021
[14]

Ziwei Fan, Ke Xu, Zhang Dong, Hao Peng, Jiawei Zhang, and Philip S Yu. 2023. Graph Collaborative Signals Denoising and Augmentation for Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . 2037–2041

2023
[15]

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval ...

2020
[16]

Xiangnan He, Zhankui He, Jingkuan Song, Zhenguang Liu, Yu-Gang Jiang, and Tat-Seng Chua. 2018. NAIS: Neural attentive item similarity model for recommendation. IEEE Transactions on Knowledge and Data Engineering 30, 12 (2018), 2354–2366

2018
[17]

Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceed...

2016 arXiv
[18]

Jinwoo Kim, Dat Nguyen, Seonwoo Min, Sungjun Cho, Moontae Lee, Honglak Lee, and Seunghoon Hong. 2022. Pure transformers are powerful graph learners. Advances in Neural Information Processing Systems 35 (2022), 14582–14595

2022
[19]

Kingma and Jimmy Ba

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations

2015
[20]

Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining . 426–434

2008
[21]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30–37

2009
[22]

Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. 2021. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems 34 (2021), 21618–21629

2021
[23]

Chaoliu Li, Lianghao Xia, Xubin Ren, Yaowen Ye, Yong Xu, and Chao Huang. 2023. Graph Transformer for Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023...

2023
[24]

Pan Li, Yanbang Wang, Hongwei Wang, and Jure Leskovec. 2020. Distance encoding: Design provably more powerful neural networks for graph representation learning. Advances in Neural Information Processing Systems 33 (2020), 4465–4478

2020
[25]

Zekun Li, Zeyu Cui, Shu Wu, Xiaoyu Zhang, and Liang Wang. 2019. Fi-gnn: Modeling feature interactions via graph neural networks for ctr prediction. In Proceedings of the 28th ACM international conference on information and knowledge management . 539–548

2019
[26]

Zihan Lin, Changxin Tian, Yupeng Hou, and Wayne Xin Zhao. 2022. Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In Proceedings of the ACM Web Conference 2022 . 2320–2329

2022
[27]

Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, and Xiuqiang He. 2021. UltraGCN: Ultra Simplification of Graph Convolutional Networks for Recommendation. conference on information and knowledge management (2021). Manuscript submitted to ACM 22 Jiajia Chen et al

2021
[28]

Erxue Min, Runfa Chen, Yatao Bian, Tingyang Xu, Kangfei Zhao, Wenbing Huang, Peilin Zhao, Junzhou Huang, Sophia Ananiadou, and Yu Rong
[29]

Luis Müller, Mikhail Galkin, Christopher Morris, and Ladislav Rampášek. 2023. Attending to graph transformers. arXiv preprint arXiv:2302.04181 (2023)

2023 arXiv
[30]

Dai Quoc Nguyen, Tu Dinh Nguyen, and Dinh Phung. 2022. Universal graph transformer self-attention networks. In Companion Proceedings of the Web Conference 2022. 193–196

2022
[31]

Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1998. The pagerank citation ranking: Bring order to the web . Technical Report. Technical report, stanford University

1998
[32]

Wonpyo Park, Woong-Gi Chang, Donggeon Lee, Juntae Kim, et al. 2022. GRPE: Relative Positional Encoding for Graph Transformer. In ICLR2022 Machine Learning for Drug Discovery

2022
[33]

Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. 2022. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems 35 (2022), 14501–14515

2022
[34]

Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International conference on data mining . IEEE, 995–1000

2010
[35]

Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. uncertainty in artificial intelligence (2009)

2009
[36]

Steffen Rendle, Zeno Gantner, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2011. Fast context-aware recommendations with factorization machines. In Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval . 635–644

2011
[37]

Yu Rong, Yatao Bian, Tingyang Xu, Weiyang Xie, Ying Wei, Wenbing Huang, and Junzhou Huang. 2020. Self-supervised graph transformer on large-scale molecular data. Advances in Neural Information Processing Systems 33 (2020), 12559–12571

2020
[38]

Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019. Knowledge graph convolutional networks for recommender systems. In The world wide web conference . 3307–3313

2019
[39]

Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval . 165–174

2019
[40]

Yinwei Wei, Wenqi Liu, Fan Liu, Xiang Wang, Liqiang Nie, and Tat-Seng Chua. 2023. LightGT: A Light Graph Transformer for Multimedia Recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1508–1517

2023
[41]

Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval . 726–735

2021
[42]

Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, Tianyu Qiu, and Xiangnan He. 2022. On the effectiveness of sampled softmax loss for item recommendation. ACM Transactions on Information Systems (2022)

2022
[43]

Qitian Wu, Wentao Zhao, Zenan Li, David P Wipf, and Junchi Yan. 2022. Nodeformer: A scalable graph structure learning transformer for node classification. Advances in Neural Information Processing Systems 35 (2022), 27387–27401

2022
[44]

Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, and Bin Cui. 2022. Graph neural networks in recommender systems: a survey. Comput. Surveys 55, 5 (2022), 1–37

2022
[45]

Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Xiyue Zhang, Hongsheng Yang, Jian Pei, and Liefeng Bo. 2021. Knowledge-enhanced hierarchical graph transformer network for multi-behavior recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35. 4486–4493

2021
[46]

Lianghao Xia, Chao Huang, Yong Xu, Jiashu Zhao, Dawei Yin, and Jimmy Huang. 2022. Hypergraph contrastive collaborative filtering. InProceedings of the 45th International ACM SIGIR conference on research and development in information retrieval . 70–79

2022
[47]

Lianghao Xia, Chao Huang, and Chuxu Zhang. 2022. Self-supervised hypergraph transformer for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 2100–2109

2022
[48]

Yuhao Yang, Chao Huang, Lianghao Xia, and Chenliang Li. 2022. Knowledge graph contrastive learning for recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1434–1443

2022
[49]

Yonghui Yang, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2021. Enhanced graph learning for collaborative filtering via mutual information maximization. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . 71–80

2021
[50]

Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. 2021. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems 34 (2021), 28877–28888

2021
[51]

Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. 2022. Are graph augmentations necessary? simple graph contrastive learning for recommendation. In Proceedings of the 45th international ACM SIGIR conference on research and development in info...

2022
[52]

Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. 2023. Self-supervised learning for recommender systems: A survey. IEEE Transactions on Knowledge and Data Engineering (2023)

2023
[53]

Jiawei Zhang, Haopeng Zhang, Congying Xia, and Li Sun. 2020. Graph-bert: Only attention is needed for learning graph representations. arXiv preprint arXiv:2001.05140 (2020)

2020 arXiv
[54]

Jianan Zhao, Chaozhuo Li, Qianlong Wen, Yiqi Wang, Yuming Liu, Hao Sun, Xing Xie, and Yanfang Ye. 2021. Gophormer: Ego-graph transformer for node classification. arXiv preprint arXiv:2110.13094 (2021). Manuscript submitted to ACM

2021 arXiv
[2022]

arXiv preprint arXiv:2202.08455 (2022)

Transformer for graphs: An overview from architecture perspective. arXiv preprint arXiv:2202.08455 (2022)

2022 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Uri Alon and Eran Yahav. 2020. On the Bottleneck of Graph Neural Networks and its Practical Implications. In International Conference on Learning Representations

work page 2020

[2] [2]

Deng Cai and Wai Lam. 2020. Graph transformer for graph-to-sequence learning. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 7464–7471

work page 2020

[3] [3]

Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2022. LightGCL: Simple Yet Effective Graph Contrastive Learning for Recommendation. In The Eleventh International Conference on Learning Representations

work page 2022

[4] [4]

Yixin Cao, Xiang Wang, Xiangnan He, Zikun Hu, and Tat-Seng Chua. 2019. Unifying knowledge graph learning and recommendation: Towards a better understanding of user preferences. In The world wide web conference . 151–161

work page 2019

[5] [5]

Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 34. 3438–3445

2020

[6] [6]

Jinsong Chen, Kaiyuan Gao, Gaichao Li, and Kun He. 2022. NAGphormer: A tokenized graph transformer for node classification in large graphs. In The Eleventh International Conference on Learning Representations

work page 2022

[7] [7]

Jiajia Chen, Jiancan Wu, Jiawei Chen, Xin Xin, Yong Li, and Xiangnan He. 2023. How Graph Convolutions Amplify Popularity Bias for Recommen- dation? arXiv preprint arXiv:2305.14886 (2023)

work page Pith review arXiv 2023

[8] [8]

Jiajia Chen, Xin Xin, Xianfeng Liang, Xiangnan He, and Jun Liu. 2022. GDSRec: Graph-Based Decentralized Collaborative Filtering for Social Recommendation. IEEE Transactions on Knowledge and Data Engineering 35, 5 (2022), 4813–4824

work page 2022