Pith. sign in

REVIEW 5 major objections 5 minor 62 references

A single pretrained Transformer can perform few-shot node, link, and graph classification on unseen graphs with no text and no gradient updates, by treating every task as token reasoning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

GILT turns few-shot node, edge, and graph classification into a token-reasoning problem and reaches competitive accuracy on held-out benchmarks with no per-graph tuning and no LLM.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A real, mostly sound contribution that is oversold as state-of-the-art: the tokenization scheme is clever, the ablations are honest, but the headline comparisons are not as clean as the prose suggests. the 5 major comments →

arxiv 2510.04567 v3 pith:DMCZODFM submitted 2025-10-06 cs.LG cs.AI

GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning

Weishuo Ma , Yanbo Wang , Xiyuan Wang , Lei Zou , Muhan Zhang This is my paper
classification cs.LG cs.AI
keywords graph foundation modelsin-context learningfew-shot classificationnode classificationlink predictiongraph classificationgraph neural networksheterogeneous graphs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

GILT sets out to show that a single pretrained Transformer can act as a graph foundation model without relying on text or on per-dataset gradient updates. It reframes node, link, and graph classification as one token-reasoning task: graph-native tokenization turns any graph's features and structure into a standard set of support and query tokens, and a two-stage attention Transformer learns the task's semantics from the prompted examples. The paper reports that on held-out benchmarks this tuning-free model matches or beats tuning-based and LLM-based few-shot baselines while running orders of magnitude faster. If correct, this removes the two main barriers—text dependence and tuning cost—that have blocked general-purpose graph foundation models.

Core claim

GILT's central claim is that graph heterogeneity can be resolved syntactically before any learning happens: after PCA projection and zero-padding to a fixed dimension, a deep, parameter-free linear GCN with LayerNorm produces structure-aware embeddings, and each support item is paired with a pooled, L2-normalized class prototype while each query item is paired with zeros. These asymmetric tokens let a Transformer treat any N-way K-shot task as a context-reasoning problem. The model is pretrained once on 15 diverse datasets spanning citation, social, and molecular graphs, then deployed with no parameter updates; the paper reports that it outperforms tuning-based and LLM-based baselines in few

What carries the argument

The load-bearing object is the asymmetric prototypical token: each support token concatenates an item embedding with the mean-pooled prototype of its class, and each query token concatenates the item embedding with zeros. This keeps token dimensionality fixed across arbitrary label sets while letting the Transformer reason about inter-class relationships. The tokens are produced by graph-native tokenization—PCA feature alignment to a fixed dimension, a five-layer linear GCN with per-layer LayerNorm (no learnable weights, no nonlinearities), and task-specific pooling for nodes, edges, or graphs. The ICL Transformer then applies two-stage attention (self-attention over support tokens, cross-at

Load-bearing premise

The load-bearing premise is that PCA projection, zero-padding, and a parameter-free linear GCN preserve enough class-discriminative information in arbitrary feature spaces; if the signal lives in discarded low-variance directions, or if zero-padding swamps low-dimensional features, no Transformer in-context reasoning can recover it (Section 3.2).

What would settle it

Run GILT on a synthetic graph whose class labels are a deterministic function of the lowest-variance feature direction, so that PCA truncation removes the signal while a supervised GNN trained on raw features still solves it; a collapse to chance accuracy would identify feature alignment as the bottleneck. A second check is to compare GILT against a version with a learned dimension-unifier on datasets with very low feature dimension.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • One pretrained model covers node, link, and graph few-shot classification on unseen graphs with no gradient updates.
  • Graphs with purely numerical or structural features—where text-based graph foundation models fail—can be handled directly.
  • Inference cost drops by orders of magnitude because adaptation happens in a single forward pass.
  • The reasoning skill transfers across task types: pre-training only on node classification still yields strong link prediction.
  • Few-shot accuracy on several benchmarks ties or exceeds tuning-based and LLM-based alternatives with much less computation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The PCA-plus-zero-padding alignment is the component most likely to cap performance; a learnable dimension unifier could be tested against it on low-dimensional feature graphs, and the paper itself flags this direction as future work.
  • The shot-decay pretraining schedule suggests the model learns a general reasoning procedure rather than memorizing datasets; if so, scaling pretraining data and task diversity should continue to lift few-shot accuracy—a testable prediction.
  • Because the token format is agnostic to label-set size and label type, a similar architecture could plausibly handle regression or continuous targets with a modified head, though the paper does not claim this.
  • The cross-task transfer result implies that structural reasoning learned at one task level is reusable at another, which could be probed by adding more task levels to pretraining.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces GILT, a graph foundational model that aims to be both LLM-free and tuning-free. GILT tokenizes any node/link/graph few-shot task by first PCA-aligning node features to a fixed dimension, propagating them through a linear GCN with LayerNorm, and pairing item embeddings with class prototypes to form support/query tokens. A two-stage causal Transformer then performs in-context reasoning, and a prototypical head produces predictions without per-task gradient updates. The model is pretrained on 15 graph datasets with a multi-task objective and evaluated on held-out benchmarks for node classification (Cora/Citeseer/Pubmed/WikiCS), link prediction, and graph classification. The paper claims stronger few-shot performance and orders-of-magnitude efficiency gains over LLM-based and tuning-based baselines.

Significance. If the claims are substantiated, GILT would be a meaningful contribution: it demonstrates a text-free, tuning-free route to graph in-context learning across three task levels using one pretrained Transformer, with a plausible and clean architecture. The paper ships code, uses held-out evaluation datasets, and includes ablations (Table 5) that support the importance of the ICL Transformer, the shared-attention design, the deep linear GCN, and the class-space prediction head. However, the strongest empirical claims—superior few-shot accuracy, link-prediction superiority over supervised GNNs, and universal feature-space handling—are undermined by comparison inconsistencies, a possible adjacency leakage issue, and an unsupported PCA-based alignment assumption, so the current version needs substantive experimental revision.

major comments (5)
  1. [§3.2 'Alignment of Feature Dimension'; Conclusion] The claim that GILT handles 'arbitrary feature spaces' rests on PCA truncation to d=512. PCA is unsupervised and can discard low-variance but class-discriminative directions; no experiment or analysis quantifies this loss. The Conclusion itself concedes that 'a learnable dimension unifier could offer greater expressive power.' Please add a controlled experiment (e.g., synthetic graphs with class signal in low-variance PCs) or compare with a learnable alignment module; otherwise the generality claim is unsupported.
  2. [Table 2 and Abstract] The headline 'stronger few-shot performance' is not supported at the per-dataset level: GILT is below GraphAny on Cora 5-shot (70.58±2.75 vs 72.68±2.47) and Citeseer 5-shot (61.44±1.57 vs 62.08±4.98), and within one SE on Pubmed 5-shot (64.96±7.48 vs 69.54±2.75). Only WikiCS is clearly better. The average gains (66.60 vs 65.54) are small and no significance test is reported. Please add paired significance tests across datasets and temper the conclusion, or show that GILT's advantage is robust.
  3. [§4.2, Table 3a] Comparing GILT 5-shot Hits@K against fully supervised GCN/SAGE is not a like-for-like few-shot comparison, and the large gap (e.g., Cora 85.27 vs 66.79) raises a leakage question: is the adjacency matrix used by GILT's linear GCN the full graph including test (and validation) edges? If so, test edges are visible during message passing. Report an ablation with test/validation edges removed from A, and include tuning-free/few-shot link-prediction baselines under the same protocol.
  4. [Table 4] Units are inconsistent: GILT rows report 63.10±5.27 and 59.74±0.51 while OFA/GFT rows report 0.576±0.037 and 0.587±0.069 under the same 'AUC' heading. If GILT's numbers are percentages, the comparison is invalid; if they are AUC, write 0.631±0.053 and 0.597±0.005. As printed, the graph-classification superiority claim cannot be evaluated. Please fix and re-state the comparison.
  5. [§3.2 vs Appendix B.1] Main text describes feature alignment as a non-parametric PCA plus zero-padding, but Appendix B.1 says low-dimensional features are first standardized to an intermediate dimension via PCA and then passed through a 'learnable linear projection to the final model dimension.' Please clarify whether this projection is part of the pretrained model, and reconcile with the 'non-parametric'/'parameter-free' claims. The distinction matters for the tuning-free/LLM-free claims and for reproducing the method.
minor comments (5)
  1. [Abstract] Typo: 'precessing' should be 'processing'.
  2. [§4.2] The cross-task comparison text references 'Table 3a' twice; the second reference should be 'Table 3b'.
  3. [Table 8] WikiCS is cited as 'Yang et al., 2016'; the correct citation is Mernyei & Cangea (2020), as used in §4.1.
  4. [Figure 2 caption; Table 2] Minor typos: 'LLm-based' should be 'LLM-based'; 'RiemannGFM25.08±9.52' is missing a space.
  5. [Table 6] The comparison of GILT 20-shot with zero-shot LLM baselines is interesting, but baselines come from different original papers with potentially different splits and protocols. Please state this limitation next to the table or report re-evaluated numbers.

Circularity Check

0 steps flagged

No significant circularity: the few-shot predictions come from a non-parametric prototypical head over held-out benchmarks, and the cited prior work is not load-bearing.

full rationale

GILT's derivation chain is self-contained. The architecture combines non-parametric PCA feature alignment, a parameter-free linear GCN (Eq. 1), and a Transformer pretrained on 15 datasets, then evaluated on held-out benchmarks that are not used for pretraining (Cora, Citeseer, Pubmed, WikiCS, ogbl-collab, ogbg-molhiv, ogbg-molpcba). No parameter is fitted to the target test labels. The prototypical head (Section 3.3, Eq. 2) is a non-parametric cosine classifier over support-derived prototypes, so the prediction is a function of the support context, not a hidden fit to the query labels. The only self-citations (OFA and GOFA, both co-authored by Muhan Zhang) are used as prior-art and baseline references; they are not the justification for GILT's central ICL claim. Direct few-shot competitors (GCOPE, RiemannGFM, OFA, GraphAny) are re-evaluated from official codebases, and the LLM-based baseline numbers come from published studies. The PCA alignment is a stated limitation (the conclusion concedes 'a learnable dimension unifier could offer greater expressive power'), but this is a correctness/robustness concern about low-variance discriminative directions, not a circular reduction: PCA is applied to unlabeled input features and does not encode the target result. Therefore no circular step is present.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 1 invented entities

No invented physical entities or external fitted constants. The central claim rests on domain assumptions about transferable feature alignment and structure extraction, plus standard neural-network hyperparameters chosen by search. Held-out benchmarks were not used to tune constants, so the circularity burden is low.

free parameters (6)
  • unified feature dimension d = 512
    PCA projection target for all graphs; chosen by Bayesian search (Table 9); affects information retention and model capacity.
  • GCN encoder depth = 5
    Chosen via hyperparameter search (Table 9); ablation shows 2-layer encoder is worse, but optimal depth may vary by domain.
  • Transformer layers = 5
    ICL Transformer depth, chosen by hyperparameter search (Table 9).
  • Transformer heads = 4
    Attention heads per layer, chosen by hyperparameter search (Table 9).
  • feed-forward hidden dimension = 4096
    FFN size in ICL Transformer, chosen by hyperparameter search (Table 9).
  • multi-task loss weights = node 0.53, link 2.74, graph 0.42
    Balances node/link/graph pretraining losses; values come from Bayesian search on pretraining validation tasks (Table 9).
axioms (4)
  • domain assumption PCA and zero-padding preserve class-relevant information across arbitrary feature spaces.
    Section 3.2, "Alignment of Feature Dimension". If PCA discards the discriminative directions, the downstream Transformer cannot recover them.
  • domain assumption A parameter-free linear GCN with LayerNorm extracts structure-aware embeddings that transfer across domains.
    Section 3.2, Eq. (1). The authors justify it by anti-overfitting intuition, but provide no proof that it preserves label signal.
  • domain assumption An N-way K-shot support set contains enough signal for the ICL Transformer to infer task rules.
    Sections 3.3 and 4. This is the fundamental ICL premise; the paper validates it empirically but does not characterize when it fails.
  • domain assumption Mean-pooled class prototypes plus cosine similarity form a sufficient classification head for any N-way task.
    Sections 3.2 and 3.3. If classes are not linearly separable in the learned embedding space, the non-parametric head caps accuracy.
invented entities (1)
  • Prototype-paired token (item embedding concatenated with class prototype) no independent evidence
    purpose: Unifies node, edge, and graph items into a fixed-size sequence for the ICL Transformer.
    An architectural construct internal to GILT; it has no standalone falsifiable prediction outside the paper and is validated only through ablations on the same benchmarks.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning." pith.science (2026). https://pith.science/paper/DMCZODFM

@misc{pith2026251004567,
  author       = {Pith},
  title        = {Pith review of: GILT: An LLM-Free, Tuning-Free Graph Foundational Model for In-Context Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMCZODFM}},
  note         = {Machine review of arXiv:2510.04567}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Graph Neural Networks (GNNs) are powerful tools for processing relational data but often struggle to generalize to unseen graphs, giving rise to the development of Graph Foundational Models (GFMs). However, current GFMs are challenged by the extreme heterogeneity of graph data, where each graph can possess a unique feature space, label set, and topology. To address this, two main paradigms have emerged. The first leverages Large Language Models (LLMs), but is fundamentally text-dependent, thus struggles to handle the numerical features in vast graphs. The second pre-trains a structure-based model, but the adaptation to new tasks typically requires a costly, per-graph tuning stage, creating a critical efficiency bottleneck. In this work, we move beyond these limitations and introduce \textbf{G}raph \textbf{I}n-context \textbf{L}earning \textbf{T}ransformer (GILT), a framework built on an LLM-free and tuning-free architecture. GILT introduces a novel token-based framework for in-context learning (ICL) on graphs, reframing classification tasks spanning node, edge and graph levels in a unified framework. This mechanism is the key to handling heterogeneity, as it is designed to operate on generic numerical features. Further, its ability to understand class semantics dynamically from the context enables tuning-free adaptation. Comprehensive experiments show that GILT achieves stronger few-shot performance with significantly less time than LLM-based or tuning-based baselines, validating the effectiveness of our approach. Our code is available at: https://github.com/yiming421/inductnode/.

Figures

Figures reproduced from arXiv: 2510.04567 by Lei Zou, Muhan Zhang, Weishuo Ma, Xiyuan Wang, Yanbo Wang.

Figure 1
Figure 1. Figure 1: GILT begins with a Graph-Native Tokenization module converting a few-shot task into unified tokens. This module first aligns feature dimensions, then uses a GNN to generate structure￾aware embeddings. These embeddings are then combined with class prototypes to form the support and query tokens. The tokens are then passed to ICL Transformer, which features a two-stage attention mechanism for in-context reas… view at source ↗
Figure 2
Figure 2. Figure 2: Efficiency vs. Accuracy on Cora node classification. The y-axis is the total inference [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The influence of the number of shots (K) on GILT’s few-shot performance. The x-axis [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 3 canonical work pages

  1. [1]

    Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking

    Aleksandar Bojchevski and Stephan G \" u nnemann. Deep gaussian embedding of graphs: Unsupervised inductive learning via ranking. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, 2018. URL https://openreview.net/forum?id=r1ZdKJ-0W

    2018

  2. [2]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  3. [3]

    Bronstein, and Max Hansmire

    Benjamin Paul Chamberlain, Sergey Shirobokov, Emanuele Rossi, Fabrizio Frasca, Thomas Markovich, Nils Yannick Hammerla, Michael M. Bronstein, and Max Hansmire. Graph neural networks for link prediction with subgraph sketching. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2...

    2023

  4. [4]

    Llaga: Large language and graph assistant

    Runjin Chen, Tong Zhao, Ajay Kumar Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=B48Pzc4oKi

    2024

  5. [5]

    Exploring the potential of large language models (llms)in learning on graphs

    Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. Exploring the potential of large language models (llms)in learning on graphs. SIGKDD Explor. , 25 0 (2): 0 42--61, 2023. doi:10.1145/3655103.3655110. URL https://doi.org/10.1145/3655103.3655110

    arXiv 2023

  6. [6]

    Graph machine learning in the era of large language models (llms)

    Wenqi Fan, Shijie Wang, Jiani Huang, Zhikai Chen, Yu Song, Wenzhuo Tang, Haitao Mao, Hui Liu, Xiaorui Liu, Dawei Yin, and Qing Li. Graph machine learning in the era of large language models (llms). CoRR, abs/2404.14928, 2024. doi:10.48550/ARXIV.2404.14928. URL https://doi.org/10.48550/arXiv.2404.14928

    Pith/arXiv arXiv doi:10.48550/arxiv.2404.14928 2024

  7. [7]

    Universal prompt tuning for graph neural networks

    Taoran Fang, Yunchao Zhang, Yang Yang, Chunping Wang, and Lei Chen. Universal prompt tuning for graph neural networks. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc

    2023

  8. [8]

    Fast graph representation learning with pytorch geometric

    Matthias Fey and Jan Eric Lenssen. Fast graph representation learning with pytorch geometric. CoRR, abs/1903.02428, 2019. URL http://arxiv.org/abs/1903.02428

    Pith/arXiv arXiv 1903

  9. [9]

    Hamilton, Zhitao Ying, and Jure Leskovec

    William L. Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Dec...

    2017

  10. [10]

    Harnessing explanations: LLM -to- LM interpreter for enhanced text-attributed graph representation learning

    Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: LLM -to- LM interpreter for enhanced text-attributed graph representation learning. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=RXFVcynVe1

    2024

  11. [13]

    Tabpfn: A transformer that solves small tabular classification problems in a second

    Noah Hollmann, Samuel M \"u ller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in a second. arXiv preprint arXiv:2207.01848, 2022

    Pith/arXiv arXiv 2022

  12. [14]

    u ller, Lennart Purucker, Arjun Krishnakumar, Max K \

    Noah Hollmann, Samuel M \" u ller, Lennart Purucker, Arjun Krishnakumar, Max K \" o rfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nat., 637 0 (8044): 0 319--326, 2025. doi:10.1038/S41586-024-08328-6. URL https://doi.org/10.1038/s41586-024-08328-6

    doi:10.1038/s41586-024-08328-6 2025

  13. [15]

    Open graph benchmark: Datasets for machine learning on graphs

    Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. In Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria - Florina Balcan, and Hsuan - Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference ...

    2020

  14. [16]

    Pande, and Jure Leskovec

    Weihua Hu, Bowen Liu, Joseph Gomes, Marinka Zitnik, Percy Liang, Vijay S. Pande, and Jure Leskovec. Strategies for pre-training graph neural networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020 b . URL https://openreview.net/forum?id=HJlWWJSFDH

    2020

  15. [17]

    Let's ask GNN: empowering large language model for graph in-context learning

    Zhengyu Hu, Yichuan Li, Zhengyu Chen, Jingang Wang, Han Liu, Kyumin Lee, and Kaize Ding. Let's ask GNN: empowering large language model for graph in-context learning. In Yaser Al - Onaizan, Mohit Bansal, and Yun - Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024 , pp.\ 1396-...

    work page doi:10.18653/v1/2024.findings-emnlp.75 2024

  16. [18]

    Prodigy: enabling in-context learning over graphs

    Qian Huang, Hongyu Ren, Peng Chen, Gregor Kr z manc, Daniel Zeng, Percy Liang, and Jure Leskovec. Prodigy: enabling in-context learning over graphs. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023. Curran Associates Inc

    2023

  17. [19]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Variational graph auto-encoders. CoRR, abs/1611.07308, 2016. URL http://arxiv.org/abs/1611.07308

    Pith/arXiv arXiv 2016

  18. [20]

    Kipf and Max Welling

    Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings . OpenReview.net, 2017. URL https://openreview.net/forum?id=SJU4ayYgl

    2017

  19. [21]

    GOFA: A generative one-for-all model for joint graph language modeling

    Lecheng Kong, Jiarui Feng, Hao Liu, Chengsong Huang, Jiaxin Huang, Yixin Chen, and Muhan Zhang. GOFA: A generative one-for-all model for joint graph language modeling. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net, 2025. URL https://openreview.net/forum?id=mIjblC9hfm

    2025

  20. [24]

    Unified graph neural networks pre-training for multi-domain graphs

    Mingkai Lin, Xiaobin Hong, Wenzhong Li, and Sanglu Lu. Unified graph neural networks pre-training for multi-domain graphs. In Toby Walsh, Julie Shah, and Zico Kolter (eds.), AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Philadelphia, PA, USA , pp.\ 12165--12173. AAAI Press, 2025. doi:10....

    work page doi:10.1609/aaai.v39i11.33325 2025

  21. [25]

    One for all: Towards training one graph model for all classification tasks

    Hao Liu, Jiarui Feng, Lecheng Kong, Ningyue Liang, Dacheng Tao, Yixin Chen, and Muhan Zhang. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=4IT2pgc9v6

    2024

  22. [26]

    Yu, and Chuan Shi

    Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, and Chuan Shi. Graph foundation models: Concepts, opportunities and challenges. IEEE Trans. Pattern Anal. Mach. Intell. , 47 0 (6): 0 5023--5044, 2025. doi:10.1109/TPAMI.2025.3548729. URL https://doi.org/10.1109/TPAMI.2025.3548729

    arXiv 2025

  23. [27]

    Graphprompt: Unifying pre-training and downstream tasks for graph neural networks

    Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. In Ying Ding, Jie Tang, Juan F. Sequeda, Lora Aroyo, Carlos Castillo, and Geert - Jan Houben (eds.), Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 - 4 May 2023 , pp.\ 417--428. ACM ...

    arXiv 2023

  24. [28]

    In-context time series predictor

    Jiecheng Lu, Yan Sun, and Shihao Yang. In-context time series predictor. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net, 2025. URL https://openreview.net/forum?id=dCcY2pyNIO

    2025

  25. [29]

    Position: Graph foundation models are already here

    Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, and Jiliang Tang. Position: Graph foundation models are already here. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=Edz0QXKKAo

    2024

  26. [30]

    Wiki-cs: A wikipedia-based benchmark for graph neural networks

    P \' e ter Mernyei and Catalina Cangea. Wiki-cs: A wikipedia-based benchmark for graph neural networks. CoRR, abs/2007.02901, 2020. URL https://arxiv.org/abs/2007.02901

    Pith/arXiv arXiv 2007

  27. [31]

    Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K \" o pf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, hi...

    2019

  28. [32]

    Graph language models

    Moritz Plenz and Anette Frank. Graph language models. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024 , pp.\ 4477--4494. Association for Computational Linguistics, 2024. doi:10.18653/V1...

    doi:10.18653/v1/2024.acl-long.245 2024

  29. [33]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine L...

    2021

  30. [34]

    Chawla, and Chao Huang

    Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh V. Chawla, and Chao Huang. A survey of large language models for graphs. In Ricardo Baeza - Yates and Francesco Bonchi (eds.), Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024 , pp.\ 6616--6626. ACM , 2024. doi:10.1145/3637528.3671...

    arXiv 2024

  31. [35]

    Leonardo Filipe Rodrigues Ribeiro, Pedro H. P. Saverese, and Daniel R. Figueiredo. struc2vec: Learning node representations from structural identity. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017 , pp.\ 385--394. ACM , 2017. doi:10.1145/3097983.3098061. URL ...

    arXiv 2017

  32. [36]

    Multi-scale attributed node embedding

    Benedek Rozemberczki, Carl Allen, and Rik Sarkar. Multi-scale attributed node embedding. J. Complex Networks, 9 0 (2), 2021. doi:10.1093/COMNET/CNAB014. URL https://doi.org/10.1093/comnet/cnab014

    doi:10.1093/comnet/cnab014 2021

  33. [37]

    Pitfalls of graph neural network evaluation

    Oleksandr Shchur, Maximilian Mumme, Aleksandar Bojchevski, and Stephan G \" u nnemann. Pitfalls of graph neural network evaluation. CoRR, abs/1811.05868, 2018. URL http://arxiv.org/abs/1811.05868

    Pith/arXiv arXiv 2018

  34. [38]

    Li Sun, Zhenhao Huang, Suyang Zhou, Qiqi Wan, Hao Peng, and Philip S. Yu. Riemanngfm: Learning a graph foundation model from riemannian geometry. In Guodong Long, Michale Blumestein, Yi Chang, Liane Lewin - Eytan, Zi Helen Huang, and Elad Yom - Tov (eds.), Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025- 2 Ma...

    arXiv 2025

  35. [39]

    GPPT: graph pre-training and prompt tuning to generalize graph neural networks

    Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. GPPT: graph pre-training and prompt tuning to generalize graph neural networks. In Aidong Zhang and Huzefa Rangwala (eds.), KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022 , pp.\ 1717--1727. ACM , 2022. doi:10.1145/353467...

    arXiv 2022

  36. [40]

    All in one: Multi-task prompting for graph neural networks

    Xiangguo Sun, Hong Cheng, Jia Li, Bo Liu, and Jihong Guan. All in one: Multi-task prompting for graph neural networks. In Ambuj K. Singh, Yizhou Sun, Leman Akoglu, Dimitrios Gunopulos, Xifeng Yan, Ravi Kumar, Fatma Ozcan, and Jieping Ye (eds.), Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2023, Long Beach, CA, ...

    arXiv 2023

  37. [41]

    Graph prompt learning: A comprehensive survey and beyond

    Xiangguo Sun, Jiawen Zhang, Xixi Wu, Hong Cheng, Yun Xiong, and Jia Li. Graph prompt learning: A comprehensive survey and beyond. CoRR, abs/2311.16534, 2023 b . doi:10.48550/ARXIV.2311.16534. URL https://doi.org/10.48550/arXiv.2311.16534

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.16534 2023

  38. [42]

    Graphicl: Unlocking graph learning potential in llms through structured prompt design

    Yuanfu Sun, Zhengnan Ma, Yi Fang, Jing Ma, and Qiaoyu Tan. Graphicl: Unlocking graph learning potential in llms through structured prompt design. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4, 2025 , pp.\ 2440--2459. Association for C...

    doi:10.18653/v1/2025.findings-naacl.131 2025

  39. [43]

    Graphgpt: Graph instruction tuning for large language models

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. In Grace Hui Yang, Hongning Wang, Sam Han, Claudia Hauff, Guido Zuccon, and Yi Zhang (eds.), Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieva...

    arXiv 2024

  40. [44]

    Llms as zero-shot graph learners: Alignment of GNN representations with LLM token embeddings

    Duo Wang, Yuan Zuo, Fengzhi Li, and Junjie Wu. Llms as zero-shot graph learners: Alignment of GNN representations with LLM token embeddings. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information...

    2024

  41. [45]

    Model generalization on text attribute graphs: Principles with large language models, 2025

    Haoyu Wang, Shikun Liu, Rongzhe Wei, and Pan Li. Model generalization on text attribute graphs: Principles with large language models, 2025. URL https://arxiv.org/abs/2502.11836

    Pith/arXiv arXiv 2025

  42. [46]

    Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian J. McAuley. Instructgraph: Boosting large language models via graph-centric instruction tuning and preference alignment. In Lun - Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, Aug...

    doi:10.18653/v1/2024.findings-acl.801 2024

  43. [47]

    Images speak in images: A generalist painter for in-context visual learning

    Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023 , pp.\ 6830--6839. IEEE , 2023. doi:10.1109/CVPR52729.2023.00660. URL https://doi.org/10.1109/CVP...

    arXiv 2023

  44. [48]

    Gft: Graph foundation model with transferable tree vocabulary

    Zehong Wang, Zheyuan Zhang, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Gft: Graph foundation model with transferable tree vocabulary. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp.\ 107403--107443. Curran Associates, Inc., 2024 c . URL https:...

    2024

  45. [49]

    Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S

    Zhenqin Wu, Bharath Ramsundar, Evan N. Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S. Pappu, Karl Leswing, and Vijay S. Pande. Moleculenet: A benchmark for molecular machine learning. CoRR, abs/1703.00564, 2017. URL http://arxiv.org/abs/1703.00564

    Pith/arXiv arXiv 2017

  46. [50]

    Opengraph: Towards open graph foundation models

    Lianghao Xia, Ben Kao, and Chao Huang. Opengraph: Towards open graph foundation models. In Yaser Al - Onaizan, Mohit Bansal, and Yun - Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024 , pp.\ 2365--2379. Association for Computational Linguistics, 2024. doi:10.18653/V1/2024.FI...

    doi:10.18653/v1/2024.findings-emnlp.132 2024

  47. [51]

    How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview.net/forum?id=ryGs6iA5Km

    2019

  48. [52]

    Bhowmick, and Juncheng Liu

    Renchi Yang, Jieming Shi, Xiaokui Xiao, Yin Yang, Sourav S. Bhowmick, and Juncheng Liu. PANE: scalable and effective attributed network embedding. VLDB J. , 32 0 (6): 0 1237--1262, 2023. doi:10.1007/S00778-023-00790-4. URL https://doi.org/10.1007/s00778-023-00790-4

    doi:10.1007/s00778-023-00790-4 2023

  49. [53]

    Cohen, and Ruslan Salakhutdinov

    Zhilin Yang, William W. Cohen, and Ruslan Salakhutdinov. Revisiting semi-supervised learning with graph embeddings. In Maria - Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016 , volume 48 of JMLR Workshop and Conference Proceedings , pp...

    2016

  50. [55]

    Contextual structure knowledge transfer for graph neural networks

    Zhiyuan Yu, Wenzhong Li, Zhangyue Yin, Xiaobin Hong, Shijian Xiao, and Sanglu Lu. Contextual structure knowledge transfer for graph neural networks. In Toby Walsh, Julie Shah, and Zico Kolter (eds.), AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Philadelphia, PA, USA , pp.\ 22263--22271....

    work page doi:10.1609/aaai.v39i21.34381 2025

  51. [56]

    Prasanna

    Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, and Viktor K. Prasanna. Graphsaint: Graph sampling based inductive learning method. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020. URL https://openreview.net/forum?id=BJe8pkHFwS

    2020

  52. [58]

    Graphtranslator: Aligning graph model to large language model for open-ended tasks

    Mengmei Zhang, Mingwei Sun, Peng Wang, Shen Fan, Yanhu Mo, Xiaoxiao Xu, Hong Liu, Cheng Yang, and Chuan Shi. Graphtranslator: Aligning graph model to large language model for open-ended tasks. In Tat - Seng Chua, Chong - Wah Ngo, Ravi Kumar, Hady W. Lauw, and Roy Ka - Wei Lee (eds.), Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May ...

    arXiv 2024

  53. [59]

    All in one and one for all: A simple yet effective method towards cross-domain graph pretraining

    Haihong Zhao, Aochuan Chen, Xiangguo Sun, Hong Cheng, and Jia Li. All in one and one for all: A simple yet effective method towards cross-domain graph pretraining. In Ricardo Baeza - Yates and Francesco Bonchi (eds.), Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, Spain, August 25-29, 2024 , pp.\...

    arXiv 2024

  54. [60]

    Bronstein, and Jian Tang

    Jianan Zhao, Zhaocheng Zhu, Mikhail Galkin, Hesham Mostafa, Michael M. Bronstein, and Jian Tang. Fully-inductive node classification on arbitrary graphs. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net, 2025. URL https://openreview.net/forum?id=1Qpt43cqhg

    2025

  55. [61]

    Fug: Feature-universal graph contrastive pre-training for graphs with diverse node features

    Jitao Zhao, Di Jin, Meng Ge, Lianze Shan, Xin Wang, Dongxiao He, and Zhiyong Feng. Fug: Feature-universal graph contrastive pre-training for graphs with diverse node features. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp.\ 4003--4034. Curran As...

    2024

  56. [62]

    RELIEF: reinforcement learning empowered graph feature prompt tuning

    Jiapeng Zhu, Zichen Ding, Jianxiang Yu, Jiaqi Tan, Xiang Li, and Weining Qian. RELIEF: reinforcement learning empowered graph feature prompt tuning. In Yizhou Sun, Flavio Chierichetti, Hady W. Lauw, Claudia Perlich, Wee Hyong Tok, and Andrew Tomkins (eds.), Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.1, KDD 2025...

    arXiv 2025

  57. [63]

    Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs

    Yun Zhu, Haizhou Shi, Xiaotang Wang, Yongchao Liu, Yaoke Wang, Boci Peng, Chuntao Hong, and Siliang Tang. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. In Guodong Long, Michale Blumestein, Yi Chang, Liane Lewin - Eytan, Zi Helen Huang, and Elad Yom - Tov (eds.), Proceedings of the ACM on Web Conference 2025, W...

    arXiv 2025

  58. [64]

    Prog: A graph prompt learning benchmark

    Chenyi Zi, Haihong Zhao, Xiangguo Sun, Yiqing Lin, Hong Cheng, and Jia Li. Prog: A graph prompt learning benchmark. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024,...

    2024

  59. [65]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  60. [66]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  61. [67]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  62. [68]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.