pith. sign in

arxiv: 2605.18579 · v2 · pith:XFEMTSZ6new · submitted 2026-05-18 · 💻 cs.LG

S2Aligner: Pair-Efficient and Transferable Pre-Training for Sparse Text-Attributed Graphs

Yuhan Wang , Haopeng Zhang , Yibo Ding , Jiaqi Yu , Xinyu Zhao , Yuhang Liu , Ziwei Zhang , Xiao Wang
show 1 more author
Ruijie Wang
This is my paper

Pith reviewed 2026-05-20 12:45 UTC · model grok-4.3

classification 💻 cs.LG
keywords text-attributed graphssparse TAGsgraph pre-trainingLLM alignmentstructure-semantics decouplingcross-domain transferrisk balancingdomain adaptation
0
0 comments X

The pith

S2Aligner decouples semantic alignment from structural modeling for reliable pre-training on sparse text-attributed graphs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a method to pre-train models on text-attributed graphs that suffer from sparse or missing node text descriptions. The core strategy separates the learning of semantic meanings, guided by language models, from the incorporation of graph topology through a controlled reconstruction process. A balancing term then adjusts for variations in how sparse different domains are by estimating which samples are reliable. This addresses the practical issue that many graphs in applications like social networks or knowledge bases have incomplete text, leading to biased transfers between domains. Readers would care if this leads to more robust graph foundation models that work without assuming perfect text data.

Core claim

S2Aligner decomposes graph-text representations into semantic and structural components, uses structure-oriented reconstruction with consistency control to inject reliable topology cues into text representations, and suppresses inconsistent structural signals under textual sparsity. It introduces sparsity-aware cross-domain risk balancing, which calibrates domain risks through a global-domain density ratio and downweights unreliable sparse samples via graph reliability estimation. Theoretical analysis shows that this objective reduces cross-domain generalization gaps by controlling domain risk discrepancy.

What carries the argument

The decomposition of graph-text representations into independent semantic and structural components that allows topology-aware signals to be injected via reconstruction without contaminating the shared semantic space.

If this is right

  • Pre-training succeeds on graphs where textual anchors are missing or noisy.
  • Cross-domain transfer bias from sparsity is reduced through risk calibration.
  • Structural information enhances alignment while semantic space remains clean.
  • Generalization gaps shrink as domain risk discrepancies are controlled.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The decoupling approach may apply to other sparse multimodal learning problems beyond graphs.
  • Reliability estimation could help in selecting which nodes to annotate in active learning for graphs.
  • The framework suggests structure acts as a stabilizer rather than a direct supervisor in low-text regimes.

Load-bearing premise

Graph representations can be cleanly decomposed into independent semantic and structural components such that topology-aware signals can be injected via reconstruction without contaminating the shared semantic space.

What would settle it

An experiment where the structure reconstruction term is ablated and performance under high sparsity remains the same or improves would falsify the claim that the decomposition prevents contamination and enables better alignment.

Figures

Figures reproduced from arXiv: 2605.18579 by HaoPeng Zhang, Jiaqi Yu, Ruijie Wang, Xiao Wang, Xinyu Zhao, Yibo Ding, Yuhang Liu, Yuhan Wang, Ziwei Zhang.

Figure 1
Figure 1. Figure 1: LLM-as￾Aligner. Full 10% 5% 3% 1% 0 2 4 6 8 10 12 Markers / 1K tokens 3.16 6.36 9.33 9.30 10.38 Markers / 1K tokens Uncertain summaries 50 60 70 80 90 100 57.4% Uncertain summaries (%) 77.5% 83.5% 86.6% 90.4% (a) Uncertainty vs. Sparsity Full Sparse 50 60 70 80 90 Graph-to-Text MRR (%) 76.66 81.00 +5.7% 65.52 62.62 -4.4% Semantic +Struct (b) Structural supplementation MRR R@1 R@5 R@10 0 20 40 60 80 100 T2N… view at source ↗
Figure 3
Figure 3. Figure 3: The overall framework of S2Aligner is shown in the figure above. It encodes sparse text-attributed graphs into content and structural components and applies latent reconstruction on the structural branch to reduce negative transfer from sparse text. We further introduce Sparse￾aware Cross-domain Risk Balancing, aligning multi-source domain risks via density estimation and reliability weighting to learn dom… view at source ↗
Figure 5
Figure 5. Figure 5: Performance-efficiency trade-off under varying text spar￾sity levels. Acad. Com. Web 45 50 55 60 65 70 Avg. Acc. (%) 67.0 53.4 62.5 67.0 54.2 60.9 67.1 54.8 62.8 Small 23M Mid 110M Large 0.6B [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: The study results. 5 Related Work Graph–Text Alignment. Inspired by CLIP [24], contrastive dual-encoder frameworks have become the dominant paradigm for graph–text alignment on text-attributed graphs. Methods such as Graph￾CLIP [43], G2P2 [37], and GRENADE [15] construct graph-text positive pairs and map them into a shared space. However, they rely on fixed one-to-one alignment, limiting their ability to c… view at source ↗
Figure 8
Figure 8. Figure 8: Hyperparameter sensitivity analysis of α, µ and ν [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Embedding visualization of Cora. Circles ( [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
read the original abstract

Pre-training on text-attributed graphs (TAGs) is central to building transferable graph foundation models, where LLM-as-Aligner methods align graph and text representations through the semantic knowledge of large language models. However, these methods usually assume that node texts provide sufficient and reliable supervision, an assumption often violated in real-world sparse TAGs. When textual anchors are missing, noisy, or uneven across domains, graph structures must be aligned with weak semantic evidence, leading to unreliable structure-semantics correspondence and sparsity-induced transfer bias. This paper presents S2Aligner, a sparsity-aware and structure-enhanced LLM-as-Aligner framework for graph-text pre-training on sparse TAGs. The key idea is to decouple semantic alignment from structural modeling, allowing topology-aware signals to enhance alignment without contaminating the shared semantic space. Specifically, S2Aligner decomposes graph-text representations into semantic and structural components, uses structure-oriented reconstruction with consistency control to inject reliable topology cues into text representations, and suppresses inconsistent structural signals under textual sparsity. Moreover, S2Aligner introduces sparsity-aware cross-domain risk balancing, which calibrates domain risks through a global-domain density ratio and downweights unreliable sparse samples via graph reliability estimation. Theoretical analysis shows that this objective reduces cross-domain generalization gaps by controlling domain risk discrepancy. Extensive experiments across diverse graph domains, sparsity levels, and downstream tasks demonstrate that S2Aligner consistently outperforms existing baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript presents S2Aligner, a sparsity-aware and structure-enhanced LLM-as-Aligner framework for pre-training on sparse text-attributed graphs (TAGs). It decouples semantic alignment from structural modeling by decomposing graph-text representations into semantic and structural components, employs structure-oriented reconstruction with consistency control to inject topology cues into text representations, and introduces sparsity-aware cross-domain risk balancing that uses a global-domain density ratio and graph reliability estimation to downweight unreliable sparse samples. Theoretical analysis is claimed to show that this objective reduces cross-domain generalization gaps by controlling domain risk discrepancy. Extensive experiments across diverse graph domains, sparsity levels, and downstream tasks are reported to demonstrate consistent outperformance over existing baselines.

Significance. If the decomposition into independent components holds without leakage and the theoretical bound is non-circular, the work could meaningfully advance transferable graph foundation models for real-world sparse TAGs where textual supervision is missing or noisy. It directly targets sparsity-induced transfer bias and cross-domain generalization, which are practical bottlenecks in applications such as social networks and knowledge graphs.

major comments (2)
  1. [Abstract] Abstract (key idea paragraph): The central premise that representations can be cleanly decomposed into independent semantic and structural components such that structure-oriented reconstruction injects reliable topology cues 'without contaminating the shared semantic space' is load-bearing for both the theoretical bound and the empirical claims, yet no explicit mechanism (orthogonal projection, separate encoders, or invariance proof) is provided to guarantee separation under sparsity-induced noise and missing textual anchors.
  2. [Theoretical analysis] Theoretical analysis (sparsity-aware cross-domain risk balancing): The objective calibrates domain risks via a global-domain density ratio and graph reliability estimation; it is unclear whether these quantities are computed independently of the downstream evaluation data. If they are fitted to the same samples used for measuring generalization gaps, the claimed reduction in domain risk discrepancy becomes circular and does not constitute a valid bound.
minor comments (2)
  1. [Abstract] The abstract asserts outperformance and theoretical reduction in gaps but supplies no equations, implementation details, error bars, or rules for excluding sparse samples, which hinders immediate verification of the stated claims.
  2. [Experiments] In the experimental section, confirm that all reported results include standard error bars over multiple runs and explicitly state the criteria used to define and handle varying sparsity levels across domains.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback and for identifying key areas where additional clarity would strengthen the manuscript. We address each major comment below with point-by-point responses, providing explanations grounded in the current manuscript while noting where revisions will improve presentation.

read point-by-point responses

  1. Referee: [Abstract] Abstract (key idea paragraph): The central premise that representations can be cleanly decomposed into independent semantic and structural components such that structure-oriented reconstruction injects reliable topology cues 'without contaminating the shared semantic space' is load-bearing for both the theoretical bound and the empirical claims, yet no explicit mechanism (orthogonal projection, separate encoders, or invariance proof) is provided to guarantee separation under sparsity-induced noise and missing textual anchors.

    Authors: We thank the referee for this observation. The decomposition is realized in the manuscript through distinct modeling pathways detailed in Sections 3.1 and 3.2: semantic alignment is performed via LLM-driven contrastive objectives on available text attributes, while structural components are processed by a dedicated graph encoder operating on topology and node features independently of the semantic path. The structure-oriented reconstruction objective (Equation 4) combined with the consistency control term (Equation 6) and the sparsity-aware suppression mechanism explicitly down-weights or excludes structural signals that conflict with semantic evidence, thereby limiting leakage into the shared space. We acknowledge that an explicit statement of this separation (e.g., via separate encoders and consistency gating) is not highlighted in the abstract. In the revision we will add a concise clause to the abstract and a short clarifying paragraph in Section 3.2 that references the separate encoders and the role of consistency control under noise. revision: partial

  2. Referee: [Theoretical analysis] Theoretical analysis (sparsity-aware cross-domain risk balancing): The objective calibrates domain risks via a global-domain density ratio and graph reliability estimation; it is unclear whether these quantities are computed independently of the downstream evaluation data. If they are fitted to the same samples used for measuring generalization gaps, the claimed reduction in domain risk discrepancy becomes circular and does not constitute a valid bound.

    Authors: We appreciate the referee's concern about potential circularity. Both the global-domain density ratio and the graph reliability estimation are computed exclusively from the pre-training corpus: the density ratio uses aggregate statistics across source-domain training splits, and reliability scores are derived from intrinsic graph properties (degree distributions and textual sparsity) observed only during pre-training. These quantities are treated as fixed parameters when deriving the generalization bound in Theorem 1 (Section 4). No downstream evaluation data or test splits are used in their estimation. To eliminate any ambiguity, we will insert explicit statements in the revised Section 4 clarifying the data sources and independence from downstream tasks. revision: yes

Circularity Check

0 steps flagged

No significant circularity in derivation chain

full rationale

The paper defines a sparsity-aware objective that decouples semantic alignment from structural modeling via decomposition into components, structure-oriented reconstruction, and cross-domain risk balancing using density ratios and reliability estimation. The theoretical claim that this reduces generalization gaps by controlling domain risk discrepancy is presented as following from the objective definition. No equations, self-citations, or steps in the provided abstract and description reduce any prediction or bound to its inputs by construction, nor rename fitted quantities as independent results. The derivation remains self-contained with independent content from the proposed method.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities. The approach implicitly relies on the unstated assumption that semantic-structural decomposition is feasible and that consistency control can reliably suppress noise, but these are not formalized or evidenced in the given text.

pith-pipeline@v0.9.0 · 5817 in / 1267 out tokens · 47006 ms · 2026-05-20T12:45:31.596348+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

51 extracted references · 51 canonical work pages · 5 internal anchors

  1. [1]

    Invariant Risk Minimization

    Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization.arXiv preprint arXiv:1907.02893, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1907

  2. [2]

    ConGraT: Self-supervised contrastive pretraining for joint graph and text embeddings

    William Brannon, Wonjune Kang, Suyash Fulay, Hang Jiang, Brandon Roy, Deb Roy, and Jad Kabbara. ConGraT: Self-supervised contrastive pretraining for joint graph and text embeddings. InProceedings of TextGraphs-17: Graph-based Methods for Natural Language Processing, pages 19–39, Bangkok, Thailand, August 2024. Association for Computational Linguistics

    work page 2024

  3. [3]

    Multi-level graph convolutional networks for cross-platform anchor link prediction

    Hongxu Chen, Hongzhi Yin, Xiangguo Sun, Tong Chen, Bogdan Gabrys, and Katarzyna Musial. Multi-level graph convolutional networks for cross-platform anchor link prediction. InProceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pages 1503–1511, 2020

    work page 2020

  4. [4]

    Brainnet: Epileptic wave detection from seeg with hierarchical graph diffusion learning

    Junru Chen, Yang Yang, Tao Yu, Yingying Fan, Xiaolong Mo, and Carl Yang. Brainnet: Epileptic wave detection from seeg with hierarchical graph diffusion learning. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 2741–2751, 2022

    work page 2022

  5. [5]

    Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

    Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

    work page arXiv 2024

  6. [6]

    Text-space graph foundation models: Comprehensive benchmarks and new insights.Advances in Neural Information Processing Systems, 37:7464– 7492, 2024

    Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, et al. Text-space graph foundation models: Comprehensive benchmarks and new insights.Advances in Neural Information Processing Systems, 37:7464– 7492, 2024

    work page 2024

  7. [7]

    Taglas: An atlas of text-attributed graph datasets in the era of large graph and language models.arXiv preprint arXiv:2406.14683, 2024

    Jiarui Feng, Hao Liu, Lecheng Kong, Mingfang Zhu, Yixin Chen, and Muhan Zhang. Taglas: An atlas of text-attributed graph datasets in the era of large graph and language models.arXiv preprint arXiv:2406.14683, 2024

    work page arXiv 2024

  8. [8]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59):1–35, 2016

    work page 2016

  9. [9]

    Explanations as features: Llm-based features for text-attributed graphs.CoRR, abs/2305.19523, 2023

    Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. Explanations as features: Llm-based features for text-attributed graphs.CoRR, abs/2305.19523, 2023

    work page arXiv 2023

  10. [10]

    Graphmae: Self-supervised masked graph autoencoders

    Zhenyu Hou, Xiao Liu, Yukuo Cen, Yuxiao Dong, Hongxia Yang, Chunjie Wang, and Jie Tang. Graphmae: Self-supervised masked graph autoencoders. InProceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pages 594–604, 2022

    work page 2022

  11. [11]

    Open graph benchmark: Datasets for machine learning on graphs

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133, 2020

    work page 2020

  12. [12]

    Can gnn be good adapter for llms? InProceedings of the ACM web conference 2024, pages 893–904, 2024

    Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. Can gnn be good adapter for llms? InProceedings of the ACM web conference 2024, pages 893–904, 2024. 10

    work page 2024

  13. [13]

    Out-of-distribution generalization via risk ex- trapolation (rex)

    David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk ex- trapolation (rex). InInternational conference on machine learning, pages 5815–5826. PMLR, 2021

    work page 2021

  14. [14]

    Learning invariant graph representations for out-of-distribution generalization.Advances in Neural Information Processing Systems, 35:11828–11841, 2022

    Haoyang Li, Ziwei Zhang, Xin Wang, and Wenwu Zhu. Learning invariant graph representations for out-of-distribution generalization.Advances in Neural Information Processing Systems, 35:11828–11841, 2022

    work page 2022

  15. [15]

    GRENADE: Graph-centric language model for self-supervised representation learning on text-attributed graphs

    Yichuan Li, Kaize Ding, and Kyumin Lee. GRENADE: Graph-centric language model for self-supervised representation learning on text-attributed graphs. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2745–2757, Singapore, December 2023. Association for Computational Linguistics

    work page 2023

  16. [16]

    Zerog: Investigating cross- dataset zero-shot transferability in graphs

    Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li. Zerog: Investigating cross- dataset zero-shot transferability in graphs. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1725–1735, 2024

    work page 2024

  17. [17]

    One for all: Towards training one graph model for all classification tasks.arXiv preprint arXiv:2310.00149, 2023

    Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks.arXiv preprint arXiv:2310.00149, 2023

    work page arXiv 2023

  18. [18]

    Yu, and Chuan Shi

    Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, and Chuan Shi. Towards graph foundation models: A survey and beyond.CoRR, abs/2310.11829, 2023

    work page arXiv 2023

  19. [19]

    Learning noise-resilient and transferable graph-text alignment via dynamic quality assessment.arXiv preprint arXiv:2510.19384, 2025

    Yuhang Liu, Minglai Shao, Zengyi Wo, Yunlong Chu, Bing Hao, Shengzhong Liu, Ruijie Wang, and Jianxin Li. Learning noise-resilient and transferable graph-text alignment via dynamic quality assessment.arXiv preprint arXiv:2510.19384, 2025

    work page arXiv 2025

  20. [20]

    Learning transferable features with deep adaptation networks

    Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. InInternational conference on machine learning, pages 97–105. PMLR, 2015

    work page 2015

  21. [21]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  22. [22]

    Wiki-cs: A wikipedia-based benchmark for graph neural networks.arXiv preprint arXiv:2007.02901, 2020

    Péter Mernyei and C˘at˘alina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks.arXiv preprint arXiv:2007.02901, 2020

    work page arXiv 2007

  23. [23]

    Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. Causal inference by using invariant prediction: identification and confidence intervals.Journal of the Royal Statistical Society Series B: Statistical Methodology, 78(5):947–1012, 2016

    work page 2016

  24. [24]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021

    work page 2021

  25. [25]

    Recipe for a general, powerful, scalable graph transformer.Advances in Neural Information Processing Systems, 35:14501–14515, 2022

    Ladislav Rampášek, Michael Galkin, Vijay Prakash Dwivedi, Anh Tuan Luu, Guy Wolf, and Dominique Beaini. Recipe for a general, powerful, scalable graph transformer.Advances in Neural Information Processing Systems, 35:14501–14515, 2022

    work page 2022

  26. [26]

    Sentence-bert: Sentence embeddings using siamese bert- networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert- networks. InProceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP- IJCNLP), pages 3982–3992, 2019

    work page 2019

  27. [27]

    A survey of large lan- guage models for graphs

    Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, and Chao Huang. A survey of large lan- guage models for graphs. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6616–6626, 2024. 11

    work page 2024

  28. [28]

    Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.arXiv preprint arXiv:1911.08731, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1911

  29. [29]

    Collective classification in network data.AI magazine, 29(3):93–93, 2008

    Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi- Rad. Collective classification in network data.AI magazine, 29(3):93–93, 2008

    work page 2008

  30. [30]

    Deep coral: Correlation alignment for deep domain adaptation

    Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. InEuropean conference on computer vision, pages 443–450. Springer, 2016

    work page 2016

  31. [31]

    Graphgpt: Graph instruction tuning for large language models

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 491–500, 2024

    work page 2024

  32. [32]

    Bootstrapped representation learning on graphs

    Shantanu Thakoor, Corentin Tallec, Mohammad Gheshlaghi Azar, Rémi Munos, Petar Veliˇckovi´c, and Michal Valko. Bootstrapped representation learning on graphs. InICLR 2021 workshop on geometrical and topological representation learning, pages 1–14. OpenReview. net, 2021

    work page 2021

  33. [33]

    Visualizing data using t-sne.Journal of Machine Learning Research, 9(86):2579–2605, 2008

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne.Journal of Machine Learning Research, 9(86):2579–2605, 2008

    work page 2008

  34. [34]

    Deep graph infomax.stat, 1050:21, 2018

    Petar Velickovic, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax.stat, 1050:21, 2018

    work page 2018

  35. [35]

    Text Embeddings by Weakly-Supervised Contrastive Pre-training

    Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. Text embeddings by weakly-supervised contrastive pre-training. arXiv preprint arXiv:2212.03533, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  36. [36]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.Advances in neural information processing systems, 33:5776–5788, 2020

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers.Advances in neural information processing systems, 33:5776–5788, 2020

    work page 2020

  37. [37]

    Augmenting low-resource text classification with graph-grounded pre-training and prompting

    Zhihao Wen and Yuan Fang. Augmenting low-resource text classification with graph-grounded pre-training and prompting. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 506–516, 2023

    work page 2023

  38. [38]

    Weighted risk invari- ance: Domain generalization under invariant feature shift.arXiv preprint arXiv:2407.18428, 2024

    Gina Wong, Joshua Gleason, Rama Chellappa, Yoav Wald, and Anqi Liu. Weighted risk invari- ance: Domain generalization under invariant feature shift.arXiv preprint arXiv:2407.18428, 2024

    work page arXiv 2024

  39. [39]

    Handling distribution shifts on graphs: An invariance perspective.arXiv preprint arXiv:2202.02466, 2022

    Qitian Wu, Hengrui Zhang, Junchi Yan, and David Wipf. Handling distribution shifts on graphs: An invariance perspective.arXiv preprint arXiv:2202.02466, 2022

    work page arXiv 2022

  40. [40]

    A comprehensive study on text-attributed graphs: Benchmarking and rethinking.Advances in Neural Information Processing Systems, 36:17238– 17264, 2023

    Hao Yan, Chaozhuo Li, Ruosong Long, Chao Yan, Jianan Zhao, Wenwen Zhuang, Jun Yin, Peiyan Zhang, Weihao Han, Hao Sun, et al. A comprehensive study on text-attributed graphs: Benchmarking and rethinking.Advances in Neural Information Processing Systems, 36:17238– 17264, 2023

    work page 2023

  41. [41]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    Deep graph contrastive representation learning.arXiv preprint arXiv:2006.04131, 2020

    Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep graph contrastive representation learning.arXiv preprint arXiv:2006.04131, 2020

    work page arXiv 2006

  43. [43]

    Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs

    Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. InProceedings of the ACM on Web Conference 2025, pages 2183–2197, 2025. 12 A Appendix Overview The appendix is structured as follows: • Section B discusses lim...

    work page 2025

  44. [44]

    Feature decomposition: inputs consist of invariantZand domain-specificV e

    work page

  45. [45]

    Conditional independence:Y⊥ ⊥V e |Z

    work page

  46. [46]

    Invariant label condition:p ei(Y|Z) =p ej(Y|Z)for alle i, ej ∈ E

    work page

  47. [47]

    Proposition 1If the assumptions hold and the predictor f depends only on Z, then the weighted risks are equal across all domains

    Non-degenerate density:p e(Z)>0for alle∈ E. Proposition 1If the assumptions hold and the predictor f depends only on Z, then the weighted risks are equal across all domains. Proof.The weighted risk for domaineis: Re = ZZZ YZV e pe(Z, Ve, Y)·ℓ(f(Z), Y)·r e(Z) dVedZdY. Marginalizing outV e: Re = ZZ YZ ℓ(f(Z), Y)·r e(Z)·p e(Z, Y) dZdY. Substitutingp e(Z, Y) ...

    work page 2023

  48. [48]

    Text-only Language Models:These approaches independently process the raw sentences of each entity while completely ignoring topological connections. • SBERT[ 26]: We incorporate standard sentence embedding architectures to produce dense seman- tic representations, specifically evaluating theall-MiniLM-L6-v2andmulti-qa-distilbert-cos-v1 variants. • Qwen3 (...

    work page

  49. [49]

    Text-Attributed Graph (TAG) & LLM-based Methods:These recent strategies strive to fuse structural patterns with the semantic comprehension capabilities of language models. • GraphGPT[ 31]: This architecture maps topological properties into discrete tokens and employs a dual-stage instruction fine-tuning process to synchronize GNN outputs with an LLM’s sem...

    work page

  50. [50]

    Graph Self-Supervised Learning (SSL) Models:Focusing primarily on topology and dense features, these methodologies leverage traditional graph neural networks. • DGI[ 34]: A foundational self-supervised strategy that maximizes mutual information by distin- guishing authentic node-graph representations from artificially corrupted counterparts. 16 • GRACE[ 4...

    work page

  51. [51]

    • GraphCLIP[ 43]: This framework relies heavily on contrastive alignment objectives

    State-of-the-Art Graph-Text Aligners:Serving as our primary zero-shot competitors, these methods focus explicitly on synchronizing semantic and structural spaces. • GraphCLIP[ 43]: This framework relies heavily on contrastive alignment objectives. It synthesizes subgraph summaries to align embedding spaces, providing strong zero-shot graph–text alignment ...

    work page