Pith. sign in

REVIEW 2 major objections 5 minor 48 references

Masked Language Models are Good Heterogeneous Graph Generalizers

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that fine-tuning a masked language model on metapath-based cloze text makes it a strong generalizer for heterogeneous graphs, outperforming graph-specific and LLM baselines on unseen domains and tasks.

desk verdict The node classification transfer story is solid, but the zero-shot link prediction numbers that anchor the multi-task claim are inflated by target edges appearing in the input. read the letter →

arxiv 2506.06157 v2 pith:CHOUCUD5 submitted 2025-06-06 cs.SI cs.CL

Jinyu Yang , Cheng Yang , Shanyuan Cui , Zeyuan Guo , Liangwei Yang , Muhan Zhang , Zhiqiang Zhang , Chuan Shi This is my paper
classification cs.SIcs.CL
keywords heterogeneousgraphlearningmaskedlanguagemodelmetapath-basedtextualizationcross-domaingeneralizationmulti-taskcloze-stylepredictionfew-shotnodeclassificationzero-shotlink
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a masked language model fine-tuned on metapath-based textual sequences converts heterogeneous graph learning into a unified cloze-style mask prediction problem, and that this recipe yields strong cross-domain, multi-task generalization. The authors argue that existing methods struggle to generalize because heterogeneous graph neural networks are tied to a single graph and because LLM-integration approaches inject biased structural tokens; replacing both with plain text plus a constrained target vocabulary lets the pretrained LM carry the structural signal itself. If the claim holds, a single fine-tuned masked LM could handle node classification and link prediction on unseen heterogeneous graphs with little or no labeled data, which matters because real-world graphs are heterogeneous and hard to relabel for every new domain.

What carries the argument

The central mechanism is metapath-based textualization combined with a cloze-style masked prediction setup: each node's sampled metapath instances (paths over node and edge types such as movie–acted_in–actor–acted_in–movie) are written as text, task templates convert node classification and link prediction into sentences where the answer token is replaced by <mask>, and the model is fine-tuned to predict that token within a per-task constrained vocabulary. This unified text format lets one masked LM absorb structural and semantic information from multiple source graphs and then be applied to a target graph with the same templates, while the constrained vocabulary keeps predictions focused on the meaningful answer set.

What would settle it

Re-run the zero-shot link prediction evaluation with query edges removed from the metapath context (or with all edge tokens masked); if Micro-F1 on DBLP and IMDB drops substantially, the reported gains come from reading the answer rather than from structural generalization.

Watch

Extended reading notes

Core claim

Fine-tuning a pretrained masked LM (DistilRoBERTa) on a corpus built from heterogeneous graphs — where metapath instances are textualized and node classification and link prediction are reformulated into cloze-style templates with a constrained target vocabulary — produces a model that generalizes to unseen target heterogeneous graphs. On four real-world datasets, the method outperforms the best baselines by an average of 18.08% in Micro-F1 and 7.10% in Macro-F1 in few-shot node classification, and by averages of 15.22% in Micro-F1 and 8.04% in Macro-F1 in zero-shot scenarios, with the largest single-task gain appearing in zero-shot link prediction (average 27.81% Micro-F1 improvement). The authors further show that the structure-only signal from metapath-based sequences alone beats a metapath-based HGNN on an IMDB supervised task, and that removing the constrained vocabulary or replacing metapaths with plain attributes substantially degrades performance.

Load-bearing premise

The load-bearing premise is that the metapath context of a link-prediction query does not contain the answer edge, yet the construction samples metapath instances from both endpoints' neighborhoods and never masks edge tokens, so the queried relation can appear verbatim in the input.

Editorial extensions

If this is right

  • A single masked LM fine-tuned this way can perform node classification and link prediction on an unseen heterogeneous graph without any architecture change or graph-specific training.
  • In few-shot node classification, the method with one labeled example per class outperforms every baseline given forty examples per class on the IMDB and PubMed datasets.
  • The largest gains occur in zero-shot link prediction, indicating that metapath-based text carries structural signal that graph-specific baselines cannot extract without labels.
  • The constrained output vocabulary is load-bearing: when removed, performance collapses, especially in zero-shot settings, showing the model relies on being focused on the task's answer set.
  • Metapath-based sequences outperform generic graph serializations (GML and GraphML) for both node classification and link prediction, demonstrating that the structured metapath format, not just raw graph text, drives the result.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension would apply the same cloze recipe to other graph families, such as homogeneous graphs or knowledge-graph reasoning tasks, since the templates are domain-agnostic and the backbone is a standard masked LM.
  • The reported zero-shot link prediction numbers may be optimistic: if query edges are not withheld from the metapath context during corpus construction, the model can read the answer from the input, so a cleaner evaluation should mask or remove the queried relation from the context.
  • Because the method scales with token count, sampling only three metapath instances per node is a crude truncation; adaptive sampling or a retriever could retain more structural context within the 512-token budget and likely improve generalization further.
  • The framework also invites a probing experiment on synthetic heterogeneous graphs with planted ground-truth communities, which would reveal whether the LM generalizes from structure or memorizes lexical patterns from the source graphs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript proposes MLM4HG, a masked-language-model framework for heterogeneous graph learning. It converts graph structure and attributes into metapath-based text sequences, reformulates node classification and link prediction into cloze-style <mask> prediction over a constrained target vocabulary, and fine-tunes a pretrained masked LM on source graphs from multiple domains. The authors evaluate cross-domain and multi-task generalization on IMDB, DBLP, PubMed, and YELP under few-shot and zero-shot node classification and zero-shot link prediction, reporting large average gains over HGNN, SSL, and LLM baselines (e.g., 18.08% Micro-F1 in few-shot NC, 27.81% Micro-F1 in zero-shot LP). The central claim is that this simple, purely LM-based pipeline generalizes across unseen heterogeneous graphs and tasks.

Significance. The idea of using metapath-based textual sequences and a unified masked-token objective is clean and potentially influential, and the paper ships code, which is a strength. The Structure-Only experiment in Section 4.3 is a useful sanity check that the model is not relying solely on pretrained knowledge. However, the zero-shot link prediction evaluation—the strongest evidence for the multi-task generalization claim—is invalid as reported due to query-edge leakage into the context. Because the largest gains appear precisely in that setting, the significance of the headline claims cannot be assessed until the evaluation is corrected.

major comments (2)
  1. [Sections 4.1–4.2 and Appendix J] The zero-shot link prediction results in Tables 2 and 8 are invalid as evidence of generalization because query edges are not withheld from the metapath context. The protocol masks only the <target> token and concatenates M_Sv1,...,M_SvM with the task sequence; no step excludes the query edge from the metapath instances sampled for either endpoint. The DBLP example in Appendix J shows this concretely: for the query 'term relational <publish> paper Learning Probabilistic Relational Models.', the paper's context contains 'paper Learning Probabilistic Relational Models. was published in term relational ...', which is the same edge in reverse. A model can therefore answer by copying the relation type from the context. Table 3 makes the consequence visible: w/o Multi-task already reaches LP Micro-F1 of 0.8853 on IMDB, and the full model's 0.9054 is close to it; without a leak-free protocol, these numbers do not measure multi-task transfer. The authors must resample contexts so that the query edge is absent from both endpoint contexts and rerun all LP experiments (Tables 2, 3, 4, 8).
  2. [Section 5.3 and Table 2] The cross-task generalization claim is not supported once the LP evidence is removed. The NC zero-shot results show only modest gains over the best baseline on IMDB and PubMed (e.g., 0.4622 vs 0.3895 on IMDB; 0.1898 vs 0.1811 on PubMed) and essentially no gain on DBLP (0.2477 vs 0.2483), a pattern the authors themselves describe as 'suboptimal and random' for DBLP. With the leaked LP numbers excluded, the manuscript's central claim of multi-task generalization rests on just two of four targets. Please provide a version of the analysis that separates NC and LP evidence and states which datasets support the cross-task claim.
minor comments (5)
  1. [Section 5.1 and Appendix C] The statement that 'all other hyperparameters of MLM4HG are optimized through grid-searching for best performance' does not state whether this search uses only the target training/validation labels or the test labels; this should be clarified to rule out test-set tuning.
  2. [Section 5.5 and Table 4] GML and GraphML are reported as a single row ('GML & GraphML'), although the text says 'we compare ... GML and GraphML'; it should be clear whether these are treated as one variant, averaged, or reported separately.
  3. [Appendix J] The PubMed LP entry for gene U2af1-rs1 has the context '[gene U2af1-rs1</s>]' with no metapath instances; the paper should clarify whether such degenerate contexts occur and how they are handled, since they affect both the leakage analysis and the evaluation protocol.
  4. [Abstract and Section 5.2] The abstract and Section 5.2 report gains such as 18.08% without stating whether these are relative to the best baseline on each metric; please state the calculation formula.
  5. [Table 3] The table header layout ('Target IMDBYELP IMDBYELP...') is difficult to read; splitting into separate panels per task/metric would improve clarity.

Circularity Check

1 steps flagged · score 6.0 of 10

Zero-shot link prediction leaks the target edge into the metapath context, so the largest multi-task gains reduce to label copying; node-classification results remain independent.

  1. self definitional [Appendix J, DBLP link-prediction entry; see also Sections 4.1 and 4.2]
    "For paper Learning Probabilistic Relational Models.: [paper Learning Probabilistic Relational Models. was published in term relational publish paper Using relational knowledge discovery to prevent securities fraud.</s>...] You can deduce term relational<publish> paper Learning Probabilistic Relational Models."

    Section 4.1 builds each corpus entry as [M_Sv1, M_Sv2, task sequence] with metapath instances sampled from the full graph ('sample metapath instances in the form of v --e1--> v1 ...') and no query-edge holdout; Section 4.2 masks only the <target> token in the task sequence, leaving edge tokens inside metapath sequences intact. For the DBLP query 'term relational <publish> paper Learning Probabilistic Relational Models.', the prefix for the paper node contains 'paper Learning Probabilistic Relational Models. was published in term relational publish paper ...', i.e., the reverse direction of the same labeled edge with the edge type 'publish' spelled out. The model can therefore copy the answer from the un-masked context.

full rationale

The paper contains no formal derivation chain and no load-bearing self-citation loop: the only self-citation (WalkLM, a baseline) is used as a comparison and is not used to justify MLM4HG's design; the 'Why Masked LMs' argument cites an external theoretical comparison. The node-classification experiments and the structural-information ablation (Section 4.3) are self-contained empirical evaluations and are not circular. However, the multi-task claim is anchored by zero-shot link prediction, and the corpus-construction protocol leaks the target edge into the input by construction. Appendix J shows the query edge verbatim in the metapath prefix; because only the <target> token is masked, the model's LP prediction can be read off from the context. This makes the LP results, which drive the largest reported gains, reduce to label copying rather than generalization. That is a construction-level circularity for the LP component. The node-classification and few-shot results are not affected by this specific leakage, so the paper is only partially circular (score 6), not fully equivalent to its inputs.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method is empirical; it adds no free parameters of the derivation type, but many modeling choices (metapaths, sample count, length, constrained vocabulary) are hand-set, and the validity of the zero-shot link prediction evaluation rests on an unstated and likely violated assumption about masking query edges.

free parameters (5)
  • metapaths per dataset = IMDB: MAM, MDM; DBLP: APA, APCPA, APTPA; et al.
    Table 5 lists dataset-specific metapath patterns; these choices determine which structural patterns are visible to the LM and are not learned.
  • sampled metapath instances per metapath per node = 3
    Section 5.1 states random sampling of 3 instances per metapath, justified by filling the 512-token budget; performance depends on this (Appendix F).
  • maximum token length = 512
    Set to the model's input capacity and shown to affect performance in Appendix F.
  • learning rate = 1e-5
    Appendix C; grid-searched for best performance, potentially on target data.
  • number of fine-tuning epochs = 3
    Appendix F; chosen after gains stabilize.
assumptions (5)
  • domain assumption Pretrained masked LM semantics transfer to graph-derived text.
    The method depends on DistilRoBERTa's pretrained text knowledge; Section 4.3 attempts to show structure alone also helps, but the ablation is limited to IMDB and not quantified in the text.
  • domain assumption Metapath sequences sufficiently preserve heterogeneous graph structure.
    The whole corpus construction is premised on this; only four datasets test it, and metapaths are manually chosen.
  • domain assumption A shared cloze template and constrained vocabulary can express both node classification and link prediction.
    Section 4.2 defines templates; the link prediction template is evaluated as edge-type classification, which narrows the task.
  • ad hoc to paper Query edges are not needed in metapath context for zero-shot link prediction.
    For a fair evaluation the target edge must be withheld from the context, but Section 4.1 does not mask edge tokens inside metapath sequences, and Appendix J shows a query edge appearing verbatim.
  • domain assumption Candidate label lists are available at inference time for both source and target tasks.
    Zero-shot classification requires the set of possible target labels, supplied by the user; this is standard but a strong input assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Masked Language Models are Good Heterogeneous Graph Generalizers." pith.science (2026). https://pith.science/paper/CHOUCUD5

@misc{pith2026250606157,
  author       = {Pith},
  title        = {Pith review of: Masked Language Models are Good Heterogeneous Graph Generalizers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CHOUCUD5}},
  note         = {Machine review of arXiv:2506.06157}
}
read the original abstract

Heterogeneous graph neural networks (HGNNs) excel at capturing structural and semantic information in heterogeneous graphs (HGs), while struggling to generalize across domains and tasks. With the rapid advancement of large language models (LLMs), a recent study explored the integration of HGNNs with LLMs for generalizable heterogeneous graph learning. However, this approach typically encodes structural information as HG tokens using HGNNs, and disparities in embedding spaces between HGNNs and LLMs have been shown to bias the LLM's comprehension of HGs. Moreover, since these HG tokens are often derived from node-level tasks, the model's ability to generalize across tasks remains limited. To this end, we propose a simple yet effective Masked Language Modeling-based method, called MLM4HG. MLM4HG introduces metapath-based textual sequences instead of HG tokens to extract structural and semantic information inherent in HGs, and designs customized textual templates to unify different graph tasks into a coherent cloze-style 'mask' token prediction paradigm. Specifically,MLM4HG first converts HGs from various domains to texts based on metapaths, and subsequently combines them with the unified task texts to form a HG-based corpus. Moreover, the corpus is fed into a pretrained LM for fine-tuning with a constrained target vocabulary, enabling the fine-tuned LM to generalize to unseen target HGs. Extensive cross-domain and multi-task experiments on four real-world datasets demonstrate the superior generalization performance of MLM4HG over state-of-the-art methods in both few-shot and zero-shot scenarios. Our code is available at https://github.com/BUPT-GAMMA/MLM4HG.

Figures

Figures reproduced from arXiv: 2506.06157 by the authors.

Figure 1
Figure 1. The overall framework of the proposed MLM4HG. We first convert HGs from different [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Metapath-based sentences in (a) Attribute Only, (b) Structure Only and (c) Both Attribute [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Average Micro-F1 and Macro-F1 scores for node classification (NC) and link prediction [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visualization of the attention scores computed by the last layer of our MLM4HG for [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]
Figure 5
Figure 5. Figure 5: Examples and parsing results of GML and GraphML formats. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 30 canonical work pages

  1. [1]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Graph markup language (graphml)

    Brandes, U., Eiglsperger, M., Lerner, J., and Pich, C. Graph markup language (graphml). 2013

    work page 2013

  3. [3]

    D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020

    work page 1901

  4. [4]

    Lmbot: distilling graph knowledge into language model for graph-less deployment in twitter bot detection

    Cai, Z., Tan, Z., Lei, Z., Zhu, Z., Wang, H., Zheng, Q., and Luo, M. Lmbot: distilling graph knowledge into language model for graph-less deployment in twitter bot detection. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pp. 57–66, 2024

    work page 2024

  5. [5]

    Graphllm: Boosting graph reasoning ability of large language model

    Chai, Z., Zhang, T., Wu, L., Han, K., Hu, X., Huang, X., and Yang, Y . Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845, 2023

    arXiv 2023

  6. [6]

    Heterogeneous graph contrastive learning for recommendation

    Chen, M., Huang, C., Xia, L., Wei, W., Xu, Y ., and Luo, R. Heterogeneous graph contrastive learning for recommendation. In Proceedings of the sixteenth ACM international conference on web search and data mining, pp. 544–552, 2023

    work page 2023

  7. [7]

    Llaga: Large language and graph assistant

    Chen, R., Zhao, T., Jaiswal, A., Shah, N., and Wang, Z. Llaga: Large language and graph assistant. arXiv preprint arXiv:2402.08170, 2024

    arXiv 2024

  8. [8]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Devlin, J. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

Show all 48 references
  1. [9]

    V ., and Swami, A

    Dong, Y ., Chawla, N. V ., and Swami, A. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 135–144, 2017

    2017

  2. [10]

    and Lenssen, J

    Fey, M. and Lenssen, J. E. Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428, 2019

    1903 arXiv

  3. [11]

    Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding

    Fu, X., Zhang, J., Meng, Z., and King, I. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of the web conference 2020, pp. 2331–2341, 2020

    2020

  4. [12]

    Gml: A portable graph file format

    Himsolt, M. Gml: A portable graph file format. Technical report, Technical report, Universitat Passau, 1997

    1997

  5. [13]

    J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  6. [14]

    Gpt-gnn: Generative pre-training of graph neural networks

    Hu, Z., Dong, Y ., Wang, K., Chang, K.-W., and Sun, Y . Gpt-gnn: Generative pre-training of graph neural networks. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 1857–1867, 2020

    2020

  7. [15]

    Heterogeneous graph transformer

    Hu, Z., Dong, Y ., Wang, K., and Sun, Y . Heterogeneous graph transformer. InProceedings of the web conference 2020, pp. 2704–2710, 2020

    2020

  8. [16]

    Pre-training on large-scale heterogeneous graph

    Jiang, X., Jia, T., Fang, Y ., Shi, C., Lin, Z., and Wang, H. Pre-training on large-scale heterogeneous graph. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 756–766, 2021

    2021

  9. [17]

    X., and Li, J

    Li, Y ., Wang, P., Li, Z., Yu, J. X., and Li, J. Zerog: Investigating cross-dataset zero-shot transferability in graphs. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1725–1735, 2024

    2024

  10. [18]

    Langgfm: A large language model alone can be a powerful graph foundation model

    Lin, T., Yan, P., Song, K., Jiang, Z., Kang, Y ., Lin, J., Yuan, W., Cao, J., Sun, C., and Liu, X. Langgfm: A large language model alone can be a powerful graph foundation model. arXiv preprint arXiv:2410.14961, 2024

    2024 arXiv

  11. [19]

    One for all: Towards training one graph model for all classification tasks

    Liu, H., Feng, J., Kong, L., Liang, N., Tao, D., Chen, Y ., and Zhang, M. One for all: Towards training one graph model for all classification tasks. arXiv preprint arXiv:2310.00149, 2023. 10

    2023 arXiv

  12. [20]

    S., et al

    Liu, J., Yang, C., Lu, Z., Chen, J., Li, Y ., Zhang, M., Bai, T., Fang, Y ., Sun, L., Yu, P. S., et al. Towards graph foundation models: A survey and beyond. arXiv preprint arXiv:2310.11829, 2023

    2023 arXiv

  13. [21]

    Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 364, 2019

    Liu, Y . Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 364, 2019

    1907 arXiv

  14. [22]

    Liu, Z., He, X., Tian, Y ., and Chawla, N. V . Can we soft prompt llms for graph learning tasks? In Companion Proceedings of the ACM on Web Conference 2024, pp. 481–484, 2024

    2024

  15. [23]

    Decoupled weight decay regularization

    Loshchilov, I. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  16. [24]

    Relation structure-aware heterogeneous information network embedding

    Lu, Y ., Shi, C., Hu, L., and Liu, Z. Relation structure-aware heterogeneous information network embedding. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pp. 4456–4463, 2019

    2019

  17. [25]

    Single- cell biological network inference using a heterogeneous graph transformer

    Ma, A., Wang, X., Li, J., Wang, C., Xiao, T., Liu, Y ., Cheng, H., Wang, J., Li, Y ., Chang, Y ., et al. Single- cell biological network inference using a heterogeneous graph transformer. Nature Communications, 14 (1):964, 2023

    2023

  18. [26]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019

    2019

  19. [27]

    Scikit-learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V ., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V ., et al. Scikit-learn: Machine learning in python.the Journal of machine Learning research, 12:2825–2830, 2011

    2011

  20. [28]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Sanh, V . Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  21. [29]

    N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M

    Schlichtkrull, M., Kipf, T. N., Bloem, P., Van Den Berg, R., Titov, I., and Welling, M. Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3–7, 2018, proceedings 15, pp. 593–...

    2018

  22. [30]

    Rotate: Knowledge graph embedding by relational rotation in complex space

    Sun, Z., Deng, Z.-H., Nie, J.-Y ., and Tang, J. Rotate: Knowledge graph embedding by relational rotation in complex space. arXiv preprint arXiv:1902.10197, 2019

    1902 arXiv

  23. [31]

    Walklm: A uniform language model fine-tuning framework for attributed graph embedding

    Tan, Y ., Zhou, Z., Lv, H., Liu, W., and Yang, C. Walklm: A uniform language model fine-tuning framework for attributed graph embedding. Advances in Neural Information Processing Systems, 36, 2024

    2024

  24. [32]

    Graphgpt: Graph instruction tuning for large language models

    Tang, J., Yang, Y ., Wei, W., Shi, L., Su, L., Cheng, S., Yin, D., and Huang, C. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 491–500, 2024

    2024

  25. [33]

    Higpt: Heterogeneous graph language model

    Tang, J., Yang, Y ., Wei, W., Shi, L., Xia, L., Yin, D., and Huang, C. Higpt: Heterogeneous graph language model. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2842–2853, 2024

    2024

  26. [34]

    Tian, Y ., Dong, K., Zhang, C., Zhang, C., and Chawla, N. V . Heterogeneous graph masked autoencoders. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 9997–10005, 2023

    2023

  27. [35]

    Llama: open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: open and efficient foundation language models. arxiv. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  28. [36]

    Can language models solve graph problems in natural language? Advances in Neural Information Processing Systems, 36, 2024

    Wang, H., Feng, S., He, T., Tan, Z., Han, X., and Tsvetkov, Y . Can language models solve graph problems in natural language? Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [37]

    Deep graph library: A graph-centric, highly-performant package for graph neural networks

    Wang, M., Zheng, D., Ye, Z., Gan, Q., Li, M., Song, X., Zhou, J., Ma, C., Yu, L., Gai, Y ., et al. Deep graph library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315, 2019

    1909 arXiv

  30. [38]

    Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y ., Cui, P., and Yu, P. S. Heterogeneous graph attention network. In The world wide web conference, pp. 2022–2032, 2019

    2022

  31. [39]

    Self-supervised heterogeneous graph neural network with co- contrastive learning

    Wang, X., Liu, N., Han, H., and Shi, C. Self-supervised heterogeneous graph neural network with co- contrastive learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pp. 1726–1736, 2021

    2021

  32. [40]

    Wang, X., Bo, D., Shi, C., Fan, S., Ye, Y ., and Philip, S. Y . A survey on heterogeneous graph embedding: methods, techniques, applications and sources. IEEE Transactions on Big Data, 9(2):415–436, 2022. 11

    2022

  33. [41]

    Transformers: State-of-the-art natural language processing

    Wolf, T., Debut, L., Sanh, V ., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: sy...

    2020

  34. [42]

    Qwen3 technical report

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., Zheng, C., Liu, D., Zhou, F., Huang, F., Hu, F., Ge, H., Wei, H., Lin, H., Tang, J., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Zhou, J., Lin, J., Dang, K., Bao, K., ...

    2025 arXiv

  35. [43]

    Heterogeneous network representation learning: A unified framework with survey and benchmark

    Yang, C., Xiao, Y ., Zhang, Y ., Sun, Y ., and Han, J. Heterogeneous network representation learning: A unified framework with survey and benchmark. IEEE Transactions on Knowledge and Data Engineering, 34(10):4854–4873, 2020

    2020

  36. [44]

    Interpretable and efficient heterogeneous graph convolutional network

    Yang, Y ., Guan, Z., Li, J., Zhao, W., Cui, J., and Wang, Q. Interpretable and efficient heterogeneous graph convolutional network. IEEE Transactions on Knowledge and Data Engineering, 35(2):1637–1650, 2021

    2021

  37. [45]

    Self-supervised heterogeneous graph pre-training based on structural clustering

    Yang, Y ., Guan, Z., Wang, Z., Zhao, W., Xu, C., Lu, W., and Huang, J. Self-supervised heterogeneous graph pre-training based on structural clustering. Advances in Neural Information Processing Systems, 35: 16962–16974, 2022

    2022

  38. [46]

    Natural language is all a graph needs.arXiv preprint arXiv:2308.07134, 4(5):7, 2023

    Ye, R., Zhang, C., Wang, R., Xu, S., Zhang, Y ., et al. Natural language is all a graph needs.arXiv preprint arXiv:2308.07134, 4(5):7, 2023

    2023 arXiv

  39. [47]

    Look ahead or look around? a theoretical comparison between autoregressive and masked pretraining

    Zhang, Q., Du, T., Huang, H., Wang, Y ., and Wang, Y . Look ahead or look around? a theoretical comparison between autoregressive and masked pretraining. arXiv preprint arXiv:2407.00935, 2024

    2024 arXiv

  40. [48]

    Llm as gnn: Graph vocabulary learning for graph foundation model

    Zhu, X., Xue, H., Zhao, Z., Jin, M., Xu, W., Huang, J., Wang, Q., Zhou, K., and Zhang, Y . Llm as gnn: Graph vocabulary learning for graph foundation model. 12 A Dataset Statistics Table 5 summarizes dataset statistics, including node/edge types (with the target node type in b...

    1997

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.