Each Graph is a New Language: Graph Learning with LLMs

Chang Yang; Chuang Zhou; Huachi Zhou; Jiahe Du; Xiao Huang; Yilin Xiao; Yuxuan Xie

REVIEW 3 major objections 3 minor 1 cited by

Each Graph is a New Language: Graph Learning with LLMs

T0 review · 3 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pre-training an LLM on random-walk graph sentences lets it learn graph structure directly, and the resulting node classifier beats description-based and embedding-based baselines.

desk verdict The empirical recipe is new and the efficiency gains are real, but the Theorem 1 proof does not hold up—treat the method as plausible and the theory as unsupported. read the letter →

arxiv 2501.11478 v3 pith:2HPWTT2U submitted 2025-01-20 cs.CL cs.AIcs.LG

Huachi Zhou , Jiahe Du , Chuang Zhou , Chang Yang , Yilin Xiao , Yuxuan Xie , Xiao Huang This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords graph-definedlanguageLLMpre-trainingnodeclassificationtext-attributedgraphsrandomwalkgraphtokensLoRAstructurelearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes GDL4LLM, a framework that treats a graph as if it were a new language. It assigns each node a unique token and samples random-walk paths as graph sentences, then pre-trains a large language model to predict the next token in those sentences. The authors argue that this next-token objective teaches the model graph structure such as node degree and connectivity, and they report that the resulting node-classification model beats graph-description and attribute-embedding baselines on ACM, Wiki, and Amazon. The payoff is that high-order neighborhoods can be represented in a handful of tokens instead of a verbose natural-language description.

What carries the argument

The load-bearing mechanism is the graph language corpus built from random walks. Each node is a graph token, and each graph sentence is a path such as `<node_8><node_3><node_10><node_12><node_9>`, so a sentence of length $l$ reaches up to $(l-1)$-hop structure. Pre-training uses next-token likelihood with LoRA adapters and a learned linear projector that turns summaries of node attributes into token embeddings; the theorem is meant to show that this objective stores degree and edge information in the hidden-state/output-weight inner products. During fine-tuning, a second LoRA is learned while the pre-trained LoRA stays frozen, and $k$ sampled sentences starting at the target node supply the structural prompt; appending textual attributes to these sentences adds semantic information.

What would settle it

Recompute the optimal inner product from the appendix's binary-cross-entropy loss and check whether it equals $\log\left(I_{(s_{i,q-1},s_{i,q}) \in E} P^A / (d_q d_{q-1})\right)$; if the formula does not emerge, Theorem 1's proof fails, and a separate ablation that replaces random walks with shuffled node sequences of identical token frequencies would show whether structural information is actually what drives the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that graph structure can be transferred into an LLM by treating the graph as a language: nodes become graph tokens, random walks become graph sentences, and next-token prediction on a corpus of such sentences constitutes graph pre-training. The paper's Theorem 1 asserts that, for a sufficiently large model that can optimize each next-token inner product independently, the optimal inner product satisfies $W_{h,q} \cdot t_q \propto \log\left(\frac{I_{(s_{i,q-1},s_{i,q}) \in E} \cdot P^A}{d_q d_{q-1}}\right)$, which the authors read as evidence that degree and edge occurrence shape what the model learns. After pre-training, a target node's subgraph is represented by a few sampled graph sentences, so fine-tuning prompts stay short; on ACM, Wiki, and Amazon, GDL4LLM with attributes reaches 82.8, 73.4, and 95.5 test accuracy, respectively, above the compared description-based and embedding-based baselines. The paper also reports lower token counts and faster inference than description-based frameworks.

Load-bearing premise

The argument rests on Theorem 1's idealization that the model can optimize every next-token inner product independently and that the appendix's binary-cross-entropy derivation yields the stated formula; the appendix substitutes binary cross-entropy without justification and leaves $P^A$ undefined, so if that derivation cannot be repaired the proof that pre-training encodes graph structure does not go through.

Editorial extensions

If this is right

If the theorem's idealization is accepted, pre-training an LLM on any graph's random-walk corpus should endow it with structural knowledge that transfers to downstream node classification on that graph.
The compact graph-sentence prompt means high-order structure can be included without blowing up the context window, which is exactly where description-based methods become verbose.
Because the structure knowledge is stored in LoRA weights and a projector rather than in the base model, the framework should transfer across LLM backbones; the paper demonstrates this on Llama-2 and Llama-3.
The same pre-trained model can be fine-tuned with or without node attributes, so structural and semantic signals can be mixed flexibly depending on the data available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Going beyond the paper, the graph-as-language interface could be pointed at link prediction, edge classification, or graph-level tasks by framing the target as next-token prediction over graph sentences, replacing task-specific heads.
The paper leaves the choice of sentence length $l$ and count $k$ to grid search; a principled rule tied to graph diameter or degree distribution would make the method less tuning-dependent.
A direct test of Theorem 1 on real hidden states, measuring inner products for nodes of different degrees, would connect the idealized proof to the empirical pipeline, which the paper does not attempt.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a useful empirical paper with a broken theory section. The core idea—treat random-walk graph sentences as a 'language' for LLM pre-training, then fine-tune with LoRA—is new as a recipe, and the token/time efficiency numbers in Tables 3 and 4 are concrete and compelling. The ablation in Figure 2 shows pre-training buys a consistent 1–2 points, which is modest but real. If you work on text-attributed graph LLMs, this is worth reading for the method.

The soft spots, in order of severity. First, Theorem 1 is not proven. The appendix replaces the softmax cross-entropy with a binary cross-entropy objective without justification, defines P^A nowhere, and the algebraic step from Equation 9 to 10 does not follow. The main text says d_q while the appendix has d_q * d_{q−1}. The claim that pairs optimize independently contradicts the shared softmax denominator. So the paper's statement that pre-training 'provably encodes graph structure' is unsupported. This matters because the paper leans on it as a theoretical contribution, but it is not load-bearing for the empirical result—the method could still work even if the theorem is dropped.

Second, the main accuracy table has no error bars or variance, and the wins over the strongest baselines are often within one point. Ten-run averages are reported, but without spread it is hard to know if the gains are meaningful. This is a standard fix.

Third, the random-walk-as-sentence idea dates back to DeepWalk and node2vec, which are not cited. The framing as 'a new language' is fair enough for the LLM context, but the novelty is in the combination, not the sampling idea.

Fourth, no code or data artifacts. For a method paper, that hurts reproducibility.

The central empirical claim—outperforming description-based and embedding-based baselines—is not circular; it is tested against external baselines. I would send this to serious peer review: the method is solid enough, the issues are addressable, and the efficiency results are worth taking seriously. The authors should either fix or heavily qualify the theorem, add error bars, and cite the older walk-based work.

Referee Report

3 major / 3 minor

Summary. The paper introduces GDL4LLM, a framework that converts a text-attributed graph into a corpus of random-walk 'graph sentences' over special graph tokens, pre-trains an LLM on next-token prediction over this corpus, and then fine-tunes the LLM for node classification. The main claimed contribution is twofold: a theoretical statement (Theorem 1) that pre-training on graph sentences encodes structural information such as node degree, and an empirical demonstration that GDL4LLM outperforms description-based and attribute-embedding baselines on ACM, Wiki, and Amazon while using far fewer tokens.

Significance. If the theoretical claim were valid, it would provide a novel justification for treating graph structure as a language for LLMs, and the concise random-walk representation is a practical contribution. The framework is simple, the efficiency numbers are concrete, and the empirical gains are consistent across three datasets. However, the proof presented for the central theorem is not valid as written, and the empirical margins over strong baselines are small and reported without variance, so the significance of the claimed contributions is currently not fully established.

major comments (3)

The proof does not establish Theorem 1. The replacement of the cross-entropy loss (Eq. 4) with a binary cross-entropy loss (Eq. 7) is not justified; the negative-sampling term in Eq. 7 multiplies co-occurrence counts by |W| and then samples t, so it is not a properly normalized negative-sampling loss. Eq. 8 introduces a factor k that is not defined and changes the coefficient from |W| to |W|·#(w)·k. Solving the quadratic in Eq. 9 does not yield a logarithm as claimed in Eq. 10, and the symbol P^A is never defined. The theorem statement in the main text has denominator d_q while Eq. 10 has denominator d_q·d_{q−1}, an internal mismatch. Finally, the premise that each (Wh,q, tq) inner product can be optimized independently is incompatible with the softmax denominator in the actual cross-entropy objective (Eq. 4), where all logits compete. Because the paper explicitly claims to prove that pre-training encodes graph structural information, this proof gap is load-bearing and must be repaired or the claim withdrawn.
Results are reported as averages over ten runs with no standard deviations or significance tests. Several improvements over the strongest baselines are within one accuracy point (e.g., Wiki test 73.4 vs 73.2; ACM test 82.8 vs 81.3), so the headline claim of outperforming all baselines is not yet supported without variance estimates or statistical tests. Please add error bars, per-run values, or significance tests, and avoid ranking methods on differences that may be noise.
The ablation in Figure 2 shows pre-training gains of roughly 0.4–1.6 accuracy points, yet the text claims pre-training "contributes notably" and creates a "synergistic effect." These statements are stronger than the visual gap warrants, and no error bars are provided. Please temper the claims or provide a statistical basis for them.

minor comments (3)

Some citations are incomplete: "SFGL (Lu et al.)" and "LLM-GNN (Chen et al.)" lack years and are not fully listed in the references. Please complete these entries.
The first author is listed as "Huachi Zhou" in the author block but as "Huanchi Zhou" in the footnote. Please correct the inconsistency.
Table 3's header "Token/(order)" is ambiguous, and the repeated "LLMs -" label is confusing. Figure 4 uses "Number k" and "Length l" as axis labels without clarifying that these are the hyperparameters from Section 4.3; please make the notation consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical comparisons are external, and the unproven Theorem 1 is a correctness gap rather than a circular reduction.

full rationale

The paper's central empirical claim is that GDL4LLM outperforms description-based and embedding-based baselines on node classification. This is evaluated on three datasets against external baselines, so the main result does not reduce to a fitted parameter or to the model's own inputs. The pretraining objective is a standard next-token prediction task on random-walk sentences sampled from the graph; the claim that this lets the LLM capture structural information is a designed self-supervised mechanism, not a re-labeling of a fitted quantity as a prediction. The appendixed proof of Theorem 1 is internally problematic: it replaces the cross-entropy objective with a binary cross-entropy loss without justification, uses an undefined symbol P^A, and gives an algebraic derivation that does not clearly connect Eq. 7 through Eq. 10 to the theorem statement. These are foundational proof gaps, not circular steps: the argument does not assume as input the conclusion it claims to prove. Self-citations appear in the paper, but they are not load-bearing for the central framework; the 'graph as a new language' premise is supported by an external survey citation, and no uniqueness theorem or prior result by the same authors is invoked to force the method. Therefore, no circularity pattern rises above the threshold required by the rubric; the score reflects that the empirical contribution is self-contained, while the theoretical justification is incomplete rather than circular.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central method depends on several hand-chosen hyperparameters (walk length, number of walks, LoRA settings) and on modeling assumptions that are not independently validated. No new physical entity is introduced; the invented entities are representational graph tokens and the graph-sentence corpus.

free parameters (4)

graph sentence length l = 5 (grid {2,3,4,5}, selected on validation)
Controls the order of graph structure captured; the paper reports optimal accuracy at l=5.
number of sampled graph sentences k per node = 10 (grid {2,4,6,8,10}, selected on validation)
Controls the size of the context corpus for pre-training and fine-tuning.
LoRA rank r = 8
Rank of the low-rank adapter; set to 8 and fixed for all experiments.
LoRA alpha and dropout = alpha=16, dropout=0.2
Standard LoRA hyperparameters; chosen without sensitivity analysis.

assumptions (5)

domain assumption Random-walk sequences of node tokens form a sufficient graph language for node classification; local walk context captures the label-relevant structure.
Introduced in Section 3.1.2 and used throughout; no theoretical guarantee that length-5 walks preserve enough information for labels.
ad hoc to paper Next-token prediction on random-walk sentences is a valid surrogate for learning graph structure.
The paper asserts this via Theorem 1, but the proof in Section 7 is incomplete and uses an undefined P^A.
ad hoc to paper The cross-entropy pre-training loss may be replaced by a binary cross-entropy approximation without changing the claimed optimum.
Used in Section 7 to derive Theorem 1; no justification is given that the approximation preserves the optimum.
ad hoc to paper The LLM has sufficient capacity to optimize each (weight, hidden-representation) inner product independently.
Assumed in Section 7, second sentence; unrealistic for a shared transformer and uncritical for the theorem.
domain assumption Textual attributes can be summarized by the LLM into embeddings that serve as good graph-token initializations.
Used in Sections 3.1.3 and 3.2.3; details of the summarization are not provided.

invented entities (2)

Graph token vocabulary (<node_i>)
purpose: Maps every graph node to a unique token ID so the LLM can process graph structure as language.
Internal representational construct; no external falsifiable prediction. Performance depends on it, but it is not independently validated.
Graph-defined language corpus (random-walk sentences)
purpose: Pre-training data that encodes neighborhoods as token sequences.
A data representation, not an independent entity; no external handle.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Each Graph is a New Language: Graph Learning with LLMs." pith.science (2026). https://pith.science/paper/2HPWTT2U

@misc{pith2026250111478,
  author       = {Pith},
  title        = {Pith review of: Each Graph is a New Language: Graph Learning with LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2HPWTT2U}},
  note         = {Machine review of arXiv:2501.11478}
}

read the original abstract

Recent efforts leverage Large Language Models (LLMs) for modeling text-attributed graph structures in node classification tasks. These approaches describe graph structures for LLMs to understand or aggregate LLM-generated textual attribute embeddings through graph structure. However, these approaches face two main limitations in modeling graph structures with LLMs. (i) Graph descriptions become verbose in describing high-order graph structure. (ii) Textual attributes alone do not contain adequate graph structure information. It is challenging to model graph structure concisely and adequately with LLMs. LLMs lack built-in mechanisms to model graph structures directly. They also struggle with complex long-range dependencies between high-order nodes and target nodes. Inspired by the observation that LLMs pre-trained on one language can achieve exceptional performance on another with minimal additional training, we propose \textbf{G}raph-\textbf{D}efined \textbf{L}anguage for \textbf{L}arge \textbf{L}anguage \textbf{M}odel (GDL4LLM). This novel framework enables LLMs to transfer their powerful language understanding capabilities to graph-structured data. GDL4LLM translates graphs into a graph language corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand graph structures. During fine-tuning, this corpus describes the structural information of target nodes concisely with only a few tokens. By treating graphs as a new language, GDL4LLM enables LLMs to model graph structures adequately and concisely for node classification tasks. Extensive experiments on three real-world datasets demonstrate that GDL4LLM outperforms description-based and textual attribute embeddings-based baselines by efficiently modeling different orders of graph structure with LLMs.

Figures

Figures reproduced from arXiv: 2501.11478 by the authors.

**Figure 2.** Accuracy comparison of different GDL4LLM [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Performance comparison between Llama2 and Llama-3 backbones on validation and test sets across three datasets. shows optimal performance at l = 5 and k = 10, and the performance gain is marginal when approaching this value. These results demonstrate our framework’s effectiveness in modeling highorder structural information, such as inter-order dependencies. For instance, a length of 5 captures fourth-order struct… view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Visualizations of the impact of graph sentence [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

MultiFluxAI Enhancing Platform Engineering with Advanced Agent-Orchestrated Retrieval Systems
cs.AI 2025-08 reject novelty 4.0 of 10

The authors claim their MultiFluxAI orchestration framework achieves 95% accuracy and 0-10 ms responses by combining rule-based routing, caching, and graph knowledge stores for multi-service RAG queries.

Reference graph

Works this paper leans on

50 extracted references · 20 canonical work pages · cited by 1 Pith paper

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[4]

Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. 2024 a . Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170

arXiv 2024
[5]

Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and 1 others. 2024 b . Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42--61

work page 2024
[6]

Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, and 1 others. 2024 c . Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv preprint arXiv:2406.10727

arXiv 2024
[7]

Label-free node classification on graphs with large language models (llms)

Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (llms). In The Twelfth International Conference on Learning Representations

work page
[8]

Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. 2021. Node feature extraction by self-supervised multi-scale neighborhood prediction. arXiv preprint arXiv:2111.00064

arXiv 2021

Show all 50 references

[9]

Diane J Cook and Lawrence B Holder. 2006. Mining graph data. John Wiley & Sons

2006
[10]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

2024
[11]

Keyu Duan, Qian Liu, Tat-Seng Chua, Shuicheng Yan, Wei Tsang Ooi, Qizhe Xie, and Junxian He. 2023. Simteg: A frustratingly simple approach improves textual graph learning. arXiv preprint arXiv:2308.02565

2023 arXiv
[12]

Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a graph: Encoding graphs for large language models. In The Twelfth International Conference on Learning Representations

2024
[13]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821

2021 arXiv
[14]

Zhong Guan, Hongke Zhao, Likang Wu, Ming He, and Jianpin Fan. 2024. Langtopo: Aligning language descriptions of graphs with tokenized topological modeling. arXiv preprint arXiv:2406.13250

2024 arXiv
[15]

Zirui Guo, Lianghao Xia, Yanhua Yu, Yuling Wang, Zixuan Yang, Wei Wei, Liang Pang, Tat-Seng Chua, and Chao Huang. 2024. Graphedit: Large language models for graph structure learning. arXiv preprint arXiv:2402.15183

2024 arXiv
[16]

Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems, 30

2017
[17]

Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. 2024. https://arxiv.org/abs/2305.19523 Harnessing Explanations : LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning . Preprint, arXiv:2305.19523

2024 arXiv
[18]

Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2024. Next-generation database interfaces: A survey of llm-based text-to-sql. arXiv preprint arXiv:2406.08426

2024
[19]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

2021 arXiv
[20]

Xiao Huang, Qingquan Song, Yuening Li, and Xia Hu. 2019. Graph recurrent networks with attributed random walks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 732--740

2019
[21]

Xuanwen Huang, Kaiqiao Han, Yang Yang, Dezheng Bao, Quanjin Tao, Ziwei Chai, and Qi Zhu. 2024. Can gnn be good adapter for llms? In Proceedings of the ACM on Web Conference 2024, pages 893--904

2024
[22]

Zhongyu Huang, Yingheng Wang, Chaozhuo Li, and Huiguang He. 2022. Going deeper into permutation-sensitive graph neural networks. In International Conference on Machine Learning, pages 9377--9409. PMLR

2022
[23]

Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. 2023. Patton: Language model pretraining on text-rich networks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7005--7020

2023
[24]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

2016 arXiv
[25]

Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691

2021 arXiv
[26]

Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

2021 arXiv
[28]

Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. 2024. https://arxiv.org/abs/2311.12399 A Survey of Graph Meets Large Language Model : Progress and Future Directions . Preprint, arXiv:2311.12399

2024 arXiv
[29]

Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

2019 arXiv
[30]

Scale-free graph-language models

Jianglin Lu, Yixuan Liu, Yitian Zhang, and Yun Fu. Scale-free graph-language models. In The Thirteenth International Conference on Learning Representations
[31]

Jianglin Lu, Yi Xu, Huan Wang, Yue Bai, and Yun Fu. 2023. Latent graph inference with limited supervision. Advances in Neural Information Processing Systems, 36:32521--32538

2023
[32]

Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natura...

2019
[33]

Giannis Nikolentzos, Antoine Tixier, and Michalis Vazirgiannis. 2020. Message passing attention networks for document understanding. In Proceedings of the aaai conference on artificial intelligence, volume 34, pages 8544--8551

2020
[34]

Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532--1543

2014
[35]

Hyunjin Seo, Taewon Kim, June Yong Yang, and Eunho Yang. 2024. https://arxiv.org/abs/2405.18581 Unleashing the Potential of Text-attributed Graphs : Automatic Relation Decomposition via Large Language Models . Preprint, arXiv:2405.18581

2024 arXiv
[36]

Yu Song, Haitao Mao, Jiachen Xiao, Jingzhe Liu, Zhikai Chen, Wei Jin, Carl Yang, Jiliang Tang, and Hui Liu. 2024. A pure transformer pretraining framework on text-attributed graphs. arXiv preprint arXiv:2406.13873

2024 arXiv
[37]

Ruoxi Sun, Hanjun Dai, and Adams Wei Yu. 2022. Does gnn pretraining help molecular representation? Advances in Neural Information Processing Systems, 35:12096--12109

2022
[38]

Yanchao Tan, Hang Lv, Xinyi Huang, Jiawei Zhang, Shiping Wang, and Carl Yang. 2024. Musegraph: Graph-oriented instruction tuning of large language models for generic graph mining. arXiv preprint arXiv:2403.04780

2024 arXiv
[39]

Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 990--998

2008
[40]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

2023 arXiv
[41]

Petar Veli c kovi \'c , Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903

2017 arXiv
[42]

Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. 2021. Graphformers: Gnn-nested transformers for representation learning on textual graph. Advances in Neural Information Processing Systems, 34:28798--28810

2021
[43]

Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. 2024. Language is all a graph needs. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1955--1973

2024
[44]

Qinggang Zhang, Shengyuan Chen, Yuanchen Bei, Zheng Yuan, Huachi Zhou, Zijin Hong, Junnan Dong, Hao Chen, Yi Chang, and Xiao Huang. 2025. A survey of graph retrieval-augmented generation for customized large language models. arXiv preprint arXiv:2501.13958

2025
[45]

Yin Zhang, Rong Jin, and Zhi-Hua Zhou. 2010. Understanding bag-of-words model: a statistical framework. International journal of machine learning and cybernetics, 1:43--52

2010
[46]

Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. 2022. Learning on large-scale text-attributed graphs via variational inference. arXiv preprint arXiv:2210.14709

2022 arXiv
[47]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, and 1 others. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

2023 arXiv
[48]

Chuang Zhou, Junnan Dong, Xiao Huang, Zirui Liu, Kaixiong Zhou, and Zhaozhuo Xu. 2024 a . Quest: Efficient extreme multi-label text classification with large language models on commodity hardware. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages ...

2024
[49]

Huachi Zhou, Hao Chen, Junnan Dong, Daochen Zha, Chuang Zhou, and Xiao Huang. 2023. Adaptive popularity debiasing aggregator for graph collaborative filtering. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, p...

2023
[50]

Huachi Zhou, Shuang Zhou, Hao Chen, Ninghao Liu, Fan Yang, and Xiao Huang. 2024 b . Enhancing explainable rating prediction through annotated macro concepts. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag...

2024
[51]

Yun Zhu, Yaoke Wang, Haizhou Shi, and Siliang Tang. 2024. https://arxiv.org/abs/2401.15569 Efficient Tuning and Inference for Large Language Models on Textual Graphs . Preprint, arXiv:2401.15569

2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[4] [4]

Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. 2024 a . Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170

arXiv 2024

[5] [5]

Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and 1 others. 2024 b . Exploring the potential of large language models (llms) in learning on graphs. ACM SIGKDD Explorations Newsletter, 25(2):42--61

work page 2024

[6] [6]

Zhikai Chen, Haitao Mao, Jingzhe Liu, Yu Song, Bingheng Li, Wei Jin, Bahare Fatemi, Anton Tsitsulin, Bryan Perozzi, Hui Liu, and 1 others. 2024 c . Text-space graph foundation models: Comprehensive benchmarks and new insights. arXiv preprint arXiv:2406.10727

arXiv 2024

[7] [7]

Label-free node classification on graphs with large language models (llms)

Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (llms). In The Twelfth International Conference on Learning Representations

work page

[8] [8]

Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. 2021. Node feature extraction by self-supervised multi-scale neighborhood prediction. arXiv preprint arXiv:2111.00064

arXiv 2021