Pith. sign in

REVIEW 2 major objections 8 minor 54 references

HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks

T0 review · 2 major / 8 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single pretrained language model handles text and graph structure in one space.

desk verdict Interesting pure-PLM architecture for text-rich heterogeneous networks, but the link prediction results are likely inflated by train/test leakage in HGA-NSP; node classification is more believable, and the paper deserves a conditional review. read the letter →

arxiv 2501.12857 v1 pith:UO2RAVLW submitted 2025-01-22 cs.LG

Qiuyu Zhu , Liang Zhang , Qianxiong Xu , Cheng Long This is my paper · ORCID
classification cs.LG
keywords heterogeneoustext-richnetworksrepresentationlearningpromptpretrainedlanguagemodelsmeta-pathgraphtokensHGA-MLMHGA-NSP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Heterogeneous text-rich networks—papers linked to authors, venues, and fields—are usually embedded by running a graph neural network and a language model separately and then trying to align their vector spaces. HierPromptLM argues this separation is unnecessary and harmful. It claims that one pretrained language model, prompted with textual summaries of each node's meta-path-based neighborhood and a learnable token for each relation type, can represent both nodes and edges in a single unified text space. On two academic datasets, the paper reports consistent gains over nine baselines: up to 6.08% Macro-F1 for node classification and 10.84% PR-AUC for link prediction. A training-free variant that skips fine-tuning still beats the best baselines, suggesting the prompt itself carries structural information.

What carries the argument

The load-bearing object is the hierarchical prompt: a graph-aware prompt per node, built from the node's text plus graph tokens distilled from meta-path-based subgraph summaries by a frozen PLM, and a relation-aware prompt per edge, built by placing a learnable relation token between the two endpoints' graph-aware prompts. These prompts carry the whole argument: they translate heterogeneous structure into the PLM's own token space, so no separate GNN and no alignment step is needed. The second supporting mechanism is the two pretraining objectives—HGA-MLM, which masks both text and graph tokens, and HGA-NSP, which predicts whether a node pair is linked by a given relation—that push the tunable PLM to use the structural tokens jointly with text.

What would settle it

Rerun the DBLP experiment with explicit bookkeeping that excludes all test edges from the HGA-NSP positive tuples, then recompute ROC-AUC and PR-AUC; if the margin over the best baseline shrinks to near zero, the headline link-prediction gains come from memorization rather than from joint text-structure representation.

Watch

Extended reading notes

Core claim

The central claim is that a pure PLM-based framework—one that never trains a separate heterogeneous GNN and never aligns two embedding spaces—is sufficient for representation learning on heterogeneous text-rich networks. The method converts each node's local structure into text: meta-path-based subgraphs are summarized into natural-language sentences, distilled into compact graph tokens by a frozen language model, and then combined with the node's own text into a graph-aware prompt; for an edge, the graph-aware prompts of its two endpoints are joined by a learnable relation token. A tunable PLM is fine-tuned on two HTRN-specific objectives, one that predicts masked text and graph tokens and one that classifies whether two nodes are connected by a given relation. The claimed result is that this joint textual treatment captures the interaction between what a node says and where it sits in the network, and does so better than separate PLM-plus-HGNN pipelines on both node classification and link prediction.

Load-bearing premise

The reported link-prediction improvements assume that the 40% of edges held out for testing were not used as positive samples in HGA-NSP pretraining, a separation the paper never explicitly states.

Editorial extensions

If this is right

  • With the pure-prompt design, node classification improves by up to 4.58% Micro-F1 and 6.08% Macro-F1 on DBLP, and by 1.49% Micro-F1 and 2.44% Macro-F1 on OAG, over the best baselines.
  • Link prediction improves by up to 9.19% ROC-AUC, 10.84% PR-AUC, and 9.79% F1 on DBLP, with smaller but consistent gains on OAG.
  • Ablations attribute most of the gain to the joint masking task: removing HGA-MLM drops OAG Micro-F1 from 96.24% to 83.07%; removing graph tokens or the relation token also degrades both tasks.
  • A training-free version, which skips fine-tuning entirely, still surpasses all baselines on both datasets, indicating the prompt construction itself carries usable structural signal.
  • Swapping the frozen graph-token encoder from BERT to T5-base or GPT2-small increases performance, so the framework scales with stronger language models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if this holds, the standard two-stage design of HTRN models—GNN for structure, PLM for text, plus an alignment loss—could be replaced by a single text prompt, removing a whole class of representation-mismatch failures.
  • Editorial inference: the meta-path choice remains manual, but the paper's DBLP result shows direct meta-paths sometimes beat longer ones for link prediction; a natural extension is to let the model or an LLM choose or generate meta-path descriptions instead of fixing them by hand.
  • Editorial inference: since the training-free variant is competitive, the prompts themselves might serve as a cheap baseline for evaluating whether a downstream task needs graph structure at all.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. HierPromptLM proposes a 'pure PLM' framework for representation learning on heterogeneous text-rich networks (HTRNs), replacing the usual GNN-plus-PLM pipeline with a single language model that ingests both node text and graph structure in one prompt sequence. Meta-path-based subgraphs around each node are textualized and distilled by a frozen PLM into graph tokens, which are appended to the node's own text to form a graph-aware prompt; a learnable relation token inserted between two nodes' graph-aware prompts yields a relation-aware prompt. A tunable BERT is fine-tuned with two objectives, HGA-MLM (masked token prediction over text and graph tokens) and HGA-NSP (classification of whether a node pair is connected by a given relation), and the resulting embeddings are evaluated on node classification and link prediction. On DBLP and OAG, the model reports consistent node-classification gains over nine baselines (up to 6.08% Macro-F1) and large link-prediction gains (up to 10.84% PR-AUC overall, and up to 31.51% PR-AUC for paper-paper links), alongside ablations, a training-free variant, and PLM-backbone extension experiments.

Significance. The central idea — encoding heterogeneous structure and text jointly in a single textual space and learning edge-level semantics through a learnable relation token — is a genuine conceptual departure from the GNN-PLM-alignment approaches (Heterformer, THLM) that the paper targets, and the two pretraining tasks are natural and sensible adaptations of MLM/NSP to HTRNs. The paper deserves credit for including a training-free variant as a falsifiable sanity check, for the per-relation link-prediction breakdown in Appendix C.1, and for disclosing several hyperparameters in Appendix B. The significance is nevertheless bounded by three factors: the evidence base is only two datasets with no code release; the headline link-prediction claim is compromised by the protocol gap described in Major Comment 1; and one ablation row (Table 6) appears corrupted. If the link-prediction results survive a leak-free protocol, the framework could be a useful new baseline for text-attributed network tasks; if not, the node-classification results alone would establish a more modest contribution.

major comments (2)
  1. [§4.3.1 vs. §5.3, Appendix B] The link-prediction evaluation is defined over the same object that HGA-NSP pretraining learns to classify. Section 4.3.1 states that positive HGA-NSP samples are tuples (u,r,v) with u and v 'connected by the relation r within the HTRN', with no qualification, while Section 5.3 holds out 40% of edges for link-prediction testing following [50]; neither Section 5, nor Appendix B, nor the reproducibility material states that held-out edges are excluded from HGA-NSP positive sampling, from HGA-MLM prompt construction, or from the meta-path subgraph and graph-token extraction of Sections 4.1–4.2. Because HGA-NSP is a binary classifier over relation-aware prompts and evaluation scores the same kind of prompt for relation validity, any test edge that appears among the pretraining positives gives the model the correct answer in advance; the edge also appears inside the endpoints' subgraph summaries, so HGA-MLM can memorize the association as well. Under this reading, the gains in Table 3 (9.19% ROC-AUC, 10.84% PR-AUC, 9.79% F1 on DBLP) and the per-relation gains in Table 5 (28.38% ROC-AUC, 31.51% PR-AUC, 21.40% F1 on paper-paper links) measure retrieval of pretraining positives rather than generalization, and the ablation drop in Table 6 when HGA-NSP is removed (DBLP ROC-AUC 85.56 to 77.68) is consistent with that channel. The authors should either document explicitly that test edges were masked during pretraining and prompt construction (including how the mask was applied to the subgraph extraction in Section 4.1), or re-run the link-prediction experiments under such masking. Node classification (Table 2) is less directly affected because class labels are never used in pretraining, so a corrected study may still support the framework; the link-prediction claim is the part that must be re-established.
  2. [Table 6 / Appendix C.2] The w/o HGA-MLM row of the DBLP ablation (Table 6) reports node classification Micro-F1 54.62, Macro-F1 17.66, and link prediction 50.00/50.00/33.33, all with zero standard deviation; the two node-classification values are numerically identical to the HIN2Vec+Bert row in Table 2 (54.62, 17.66). This exact match suggests a copy-paste or data-recording error. As printed, the row cannot support the claim in Appendix C.2 that removing HGA-MLM causes a 'significant performance drop' on DBLP. The corresponding OAG row in Table 4 (83.07/78.93/76.85/71.35/76.16) does support the qualitative conclusion, so the fix is local, but the DBLP row must be corrected or removed before the ablation evidence can be considered reliable.
minor comments (8)
  1. [§C.1 / Table 5] In the paragraph above Table 5, the reported improvements for paper-paper links read '28.38% gain in ROC-AUC, 31.51% enhancement in ROC-AUC'; the second metric should be PR-AUC, not ROC-AUC.
  2. [Table 1 / Appendix A] The '#Relation' column (67,417 for DBLP and 63,808 for OAG) does not agree with the sum of the per-relation edge counts listed in Appendix A (149,545 and 240,219, respectively); the table should clarify what '#Relation' counts, for example undirected interactions versus directed edges.
  3. [§5.3] The link-prediction protocol is incomplete: the paper does not say what the remaining 50% of edges are used for, how negative samples for evaluation are drawn, or which output position of the relation-aware prompt (for example [CLS], mean pooling, or the relation token) is scored to produce the reported AUC and F1 values; these details are needed for reproducibility and for verifying the exclusion requested in Major Comment 1.
  4. [§4.3.1] The paper says 'the model calculates a probability p for each sample' for HGA-NSP but does not specify how p is derived from the relation-aware prompt; please state the scoring head, such as a linear layer on the [CLS] or relation-token representation.
  5. [§5.5 / Figure 3(a)] The training-free evaluation is not fully described: for link prediction in the frozen setting, it is unclear how an edge score or probability is computed without fine-tuning; a sentence describing the scoring procedure would allow the reader to interpret the claim that HierPromptLM-free outperforms the best baselines on both tasks.
  6. [Appendix B] The reproducibility section should report the number of fine-tuning epochs or the convergence criterion, batch size, learning-rate schedule, and random seeds; currently only the optimizer (AdamW), learning-rate range, masking ratio (0.15), negative sampling ratio (1), and hidden dimension (768) are given, and no code or model checkpoint is released.
  7. [§5.2] The sentence 'All models are trained for 10 times and, the mean and standard variance of test performance are reported' should read 'trained 10 times' and 'standard deviation'; in addition, the naming is inconsistent between 'M2V+Bert' in Tables 2–3 and 'M2V-Bert' in the Section 5.3 discussion.
  8. [§1 / §2.2] The claims of being 'for the first time' a pure PLM-based framework and introducing the two pretraining tasks should be positioned more carefully against THLM [50], which also pretrains a PLM on text-attributed heterogeneous graphs (albeit with a separate alignment step), and against LinkBERT [42].

Circularity Check

2 steps flagged · score 6.0 of 10

Link-prediction evaluation is partially circular: HGA-NSP is the same binary edge-existence task as the test, and relation-aware prompts embed the target edge's endpoints in each other's graph tokens.

  1. fitted input called prediction [Section 4.3.1 (HGA-NSP definition) versus Section 5.3 (link prediction evaluation)]
    "given a pair of nodes (u,v) with a specific relation r, this task is to predict whether node v connects node u via the relation r. To generate positive samples, we form tuples (u,r,v) where u and v are connected by the relation r within the HTRN. ... Following [50], we apply a standard sampling strategy for link prediction, using 10% of edges for training and 40% for testing across both datasets."

    HGA-NSP is a binary connected/not-connected classification task, and link prediction is the same binary edge-existence task on edges of the same HTRN. The positive HGA-NSP samples are all connected pairs 'within the HTRN', while the 40% test edges are edges of that same HTRN. The paper never states that the test edges are removed from HGA-NSP positive sampling or from graph-token extraction. Consequently, a test edge can be a pretraining positive, and the reported gains (up to 10.84% PR-AUC overall, 28-38% on paper-paper) can be achieved by retrieving memorized positives instead of by learning a representation that generalizes.

  2. self definitional [Sections 4.1.1, 4.1.3, and 4.2 (prompt construction) with Section C.1 (paper-paper link prediction)]
    "Given a node u and a meta-path m in HTRNs, the meta-path-based subgraph G_m(u) consists of node u, its meta-path-based neighbors (which refer to nodes connected to u via meta-path m), and all intermediate nodes along the path. ... this function generates a text sequence by combining the textual information of nodes in the subgraph with the meta-path's semantics. ... H(u,r,v) = < [F(u)] [Relation Token of r] [F(v)] >"

    For a paper-paper edge (u,v), u and v are PP meta-path neighbors of each other, so G_PP(u) contains v and G_PP(v) contains u. By the subgraph summarization and graph-token construction, F(u) and F(v) therefore each carry the other endpoint's textual identity inside the prompt. The relation-aware prompt for exactly that edge is [F(u)] [relation token] [F(v)], so the edge's own existence is present as part of the input. The model can decide connectedness by reading the prompt rather than by reasoning about the relation, making the paper-paper link-prediction results partly self-deriving from the input representation.

full rationale

The core framework is not circular: the hierarchical prompt module, HGA-MLM, and the node-classification evaluation are self-contained and benchmarked against external baselines, and the paper does not rely on a load-bearing self-citation chain. The node-classification results (up to 6.08% Macro-F1 on DBLP) are independent evidence for the method. However, the link-prediction claims are compromised by construction in two specific ways. First, Section 4.3.1 defines HGA-NSP positives as all connected pairs in the HTRN, while Section 5.3 holds out 40% of edges for link-prediction testing without stating that those edges are excluded from pretraining; because HGA-NSP and link prediction are the same relation-existence classification task, the model can memorize test positives. Second, the relation-aware prompt used to score an edge is built from graph-aware prompts whose meta-path subgraphs contain the other endpoint for direct relations such as paper-paper, so the edge label is embedded in the input. Both reductions are quotable from the paper, so the reported 9-11% overall and 28-31% paper-paper link-prediction improvements are inflated and are not established as independent predictions. The central architecture claim remains independent, so the appropriate finding is partial circularity (6), not complete collapse.

Assumptions & free parameters 4 free parameters · 3 assumptions · 1 invented entities

The central empirical claim depends on several unstated evaluation-protocol assumptions (notably that pretraining does not see test links), plus hand-picked meta-paths and hyperparameters tuned per dataset. No new physical entities are introduced; the relation token is an internal learned vector.

free parameters (4)
  • masking_ratio_m_r = 0.15 (OAG node classification), 0.25 (DBLP overall and OAG link prediction)
    Controls the fraction of tokens masked in HGA-MLM; tuned per dataset and task in Appendix C.4, Figure 7.
  • negative_sampling_ratio_ns_r = 1 (OAG), 2 (DBLP)
    Number of negative samples per positive in HGA-NSP; chosen based on performance in Appendix C.4, Figure 6.
  • predefined_meta_path_set = DBLP: P-P, P-A, P-V plus P-A-P, P-V-P, A-P-A; OAG: P-P, P-A, P-F, A-I plus P-A-P, A-P-A, A-I-A
    The choice of meta-paths determines which subgraphs are textualized into graph tokens; Section 5.4 shows removing graph tokens hurts, and Appendix C.4 shows complex paths help node classification but can hurt link prediction on DBLP.
  • learning_rate = searched in [1e-5, 5e-5]
    Reported in Appendix B as a search range; the final value used is not given.
assumptions (3)
  • domain assumption Meta-path-based subgraph textualization P(·) conveys the structural essentials of a node's local heterogeneous context in natural language.
    Section 4.1.1 defines P as a text-generation function; if this textual summary drops relation semantics, the graph-aware prompt cannot represent the structure.
  • domain assumption A frozen PLM can distill meta-path-based subgraph summaries into graph tokens without losing information needed for downstream tasks.
    Section 4.1.2 uses a frozen PLM to compress summaries into soft prompts; no evaluation of information retention is provided.
  • domain assumption HGA-NSP pretraining does not use the edges that are later held out for link prediction evaluation.
    Section 4.3.1 samples positive edges from the HTRN; Section 5.3 evaluates on held-out edges but never states that test edges are removed from pretraining. Without this assumption, link prediction is circular.
invented entities (1)
  • Learnable relation token
    purpose: A trainable soft-prompt embedding placed between the graph-aware prompts of two nodes to represent the heterogeneous relation type of an edge.
    It is a newly introduced model parameter, not an external entity; its contribution is shown only through in-paper ablations (w/o RelationToken), and it has no out-of-paper falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks." pith.science (2026). https://pith.science/paper/UO2RAVLW

@misc{pith2026250112857,
  author       = {Pith},
  title        = {Pith review of: HierPromptLM: A Pure PLM-based Framework for Representation Learning on Heterogeneous Text-rich Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UO2RAVLW}},
  note         = {Machine review of arXiv:2501.12857}
}
read the original abstract

Representation learning on heterogeneous text-rich networks (HTRNs), which consist of multiple types of nodes and edges with each node associated with textual information, is essential for various real-world applications. Given the success of pretrained language models (PLMs) in processing text data, recent efforts have focused on integrating PLMs into HTRN representation learning. These methods typically handle textual and structural information separately, using both PLMs and heterogeneous graph neural networks (HGNNs). However, this separation fails to capture the critical interactions between these two types of information within HTRNs. Additionally, it necessitates an extra alignment step, which is challenging due to the fundamental differences between distinct embedding spaces generated by PLMs and HGNNs. To deal with it, we propose HierPromptLM, a novel pure PLM-based framework that seamlessly models both text data and graph structures without the need for separate processing. Firstly, we develop a Hierarchical Prompt module that employs prompt learning to integrate text data and heterogeneous graph structures at both the node and edge levels, within a unified textual space. Building upon this foundation, we further introduce two innovative HTRN-tailored pretraining tasks to fine-tune PLMs for representation learning by emphasizing the inherent heterogeneity and interactions between textual and structural information within HTRNs. Extensive experiments on two real-world HTRN datasets demonstrate HierPromptLM outperforms state-of-the-art methods, achieving significant improvements of up to 6.08% for node classification and 10.84% for link prediction.

Figures

Figures reproduced from arXiv: 2501.12857 by the authors.

Figure 1
Figure 1. An illustrative example of an HTRN. The most intrinsic aspect of HTRNs is their heterogeneous struc￾tural information, which refers to the nodes in HTRNs and the heterogeneous connections among them. Heterogeneous Graph Neural Networks (HGNNs), which are models designed specifically for heterogeneous graph data, have been extensively utilized in existing studies [12, 37, 40, 41] to capture such structural infor￾mati… view at source ↗
Figure 2
Figure 2. The framework of HierPromptLM. (a) Meta-path-based graph token [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (a) Training-free extension on OAG. (b) PLM backbone extension on DBLP. (c) PLM backbone extension on OAG. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training-free extension on DBLP. C.4 Parameter Analysis Impact of Pre-defined Meta-paths. To assess the impact of pre￾defined meta-paths, we compare our HierPromptLM against a vari￾ant named HierPromptLM-DMeta, which uses simpler direct meta-paths. Specifically, for th…
Figure 5
Figure 5. Figure 5: Impact of pre-defined meta-path. The results on the OAG dataset show that HierPromptLM gen￾erally outperforms HierPromptLM-DMeta. This is because Hier￾PromptLM uses more complex meta-paths, such as paper-author￾paper, paper-venue-paper and author-paper-author, which al…
Figure 6
Figure 6. Figure 6: Impact of negative sampling ratio [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 27 canonical work pages

  1. [50]

    Tao Zou, Le Yu, Yifei Huang, Leilei Sun, and Bowen Du. 2023. Pretraining language models with text-attributed heterogeneous graphs. arXiv preprint arXiv:2310.12580 (2023). A DATASET DESCRIPTION The details of two HTRN benchmark datasets used in this paper are described as follows. • DBLP [35]. The dataset is extracted from four fields of DBLP bibliography...

    work page Pith review arXiv 2023

  2. [1]

    Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, and Xueqi Cheng. 2024. Lpnl: Scalable link prediction with large language models. In Findings of the Association for Computational Linguistics ACL 2024 . 3615–3625

    work page 2024

  3. [2]

    Tom B Brown. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)

    arXiv 2020

  4. [3]

    Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. 2024. Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170 (2024)

    arXiv 2024

  5. [4]

    Po-Han Chi, Pei-Hung Chung, Tsung-Han Wu, Chun-Cheng Hsieh, Yen-Hao Chen, Shang-Wen Li, and Hung-yi Lee. 2021. Audio albert: A lite bert for self- supervised learning of audio representation. In 2021 IEEE Spoken Language Tech- nology Workshop (SLT). IEEE, 344–350

    work page 2021

  6. [5]

    Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Ol- gica Milenkovic, and Inderjit S Dhillon. 2021. Node feature extraction by self- supervised multi-scale neighborhood prediction. arXiv preprint arXiv:2111.00064 (2021)

    arXiv 2021

  7. [6]

    Xin Luna Dong, Xiang He, Andrey Kan, Xian Li, Yan Liang, Jun Ma, Yifan Ethan Xu, Chenwei Zhang, Tong Zhao, Gabriel Blanco Saldana, et al. 2020. Autoknow: Self-driving knowledge collection for products of thousands of types. In Proceed- ings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2724–2734

    work page 2020

  8. [7]

    Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining. 135–144

    work page 2017

Show all 54 references
  1. [8]

    Ahmed El-Kishky, Thomas Markovich, Serim Park, Chetan Verma, Baekjin Kim, Ramy Eskander, Yury Malkov, Frank Portman, Sofía Samaniego, Ying Xiao, et al

  2. [9]

    Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin

  3. [10]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2023. Talk like a graph: Encoding graphs for large language models. arXiv preprint arXiv:2310.04560 (2023)

    2023 arXiv

  4. [11]

    Tao-yang Fu, Wang-Chien Lee, and Zhen Lei. 2017. Hin2vec: Explore meta-paths in heterogeneous information networks for representation learning. In Proceed- ings of the 2017 ACM on Conference on Information and Knowledge Management . 1797–1806

    2017

  5. [12]

    Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. Magnn: Metap- ath aggregated graph neural network for heterogeneous graph embedding. In Proceedings of the web conference 2020 . 2331–2341

    2020

  6. [13]

    Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. 2023. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking. arXiv preprint arXiv:2305.15066 (2023)

    2023 arXiv

  7. [14]

    Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous graph transformer. In Proceedings of the web conference 2020 . 2704–2710

    2020

  8. [15]

    Bowen Jin, Wentao Zhang, Yu Zhang, Yu Meng, Xinyang Zhang, Qi Zhu, and Jiawei Han. 2023. Patton: Language model pretraining on text-rich networks. arXiv preprint arXiv:2305.12268 (2023)

    2023 arXiv

  9. [16]

    Bowen Jin, Yu Zhang, Qi Zhu, and Jiawei Han. 2023. Heterformer: Transformer- based deep node representation learning on heterogeneous text-rich networks. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1020–1031

    2023

  10. [17]

    Di Jin, Xiangchen Song, Zhizhi Yu, Ziyang Liu, Heling Zhang, Zhaomeng Cheng, and Jiawei Han. 2021. Bite-gcn: A new GCN architecture via bidirectional convo- lution of topology and features on text-rich networks. In Proceedings of the 14th ACM International Conference on Web Se...

    2021

  11. [18]

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. Minneapolis, Minnesota, 2

    2019

  12. [19]

    Yoav Levine, Noam Wies, Daniel Jannai, Dan Navon, Yedid Hoshen, and Amnon Shashua. 2021. The inductive bias of in-context learning: Rethinking pretraining example design. arXiv preprint arXiv:2110.04541 (2021)

    2021 arXiv

  13. [20]

    Chaozhuo Li, Bochen Pang, Yuming Liu, Hao Sun, Zheng Liu, Xing Xie, Tianqi Yang, Yanling Cui, Liangjie Zhang, and Qi Zhang. 2021. Adsgnn: Behavior-graph augmented relevance modeling in sponsored search. In Proceedings of the 44th international ACM SIGIR conference on research ...

    2021

  14. [21]

    Qidong Liu, Cheng Long, Jie Zhang, Mingliang Xu, and Dacheng Tao. 2022. Aspect-aware graph attention network for heterogeneous information networks. IEEE Transactions on Neural Networks and Learning Systems (2022)

    2022

  15. [22]

    Yang Liu. 2019. Fine-tune BERT for extractive summarization. arXiv preprint arXiv:1903.10318 (2019)

    2019 arXiv

  16. [23]

    Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 (2019)

    2019 arXiv

  17. [24]

    I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  18. [25]

    Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Topic dis- covery via latent space clustering of pretrained language model representations. In Proceedings of the ACM web conference 2022 . 3143–3152

    2022

  19. [26]

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26 (2013)

    2013

  20. [27]

    R OpenAI. 2023. Gpt-4 technical report. arxiv 2303.08774. View in Article 2, 5 (2023)

    2023 arXiv

  21. [28]

    Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 1532–1543

    2014

  22. [29]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. (2019)

    2019

  23. [30]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.Journal of Machine Learning Research 21, 140 (2020), 1–67. http...

    2020

  24. [31]

    V Sanh. 2019. DistilBERT, A Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter. arXiv preprint arXiv:1910.01108 (2019)

    2019 arXiv

  25. [32]

    Yu Shi, Jiaming Shen, Yuchen Li, Naijing Zhang, Xinwei He, Zhengzhi Lou, Qi Zhu, Matthew Walker, Myunghwan Kim, and Jiawei Han. 2019. Discover- ing hypernymy in text-rich heterogeneous information network by exploiting context granularity. In Proceedings of the 28th ACM Intern...

    2019

  26. [33]

    Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. Proceedings of the VLDB Endowment 4, 11 (2011), 992–1003

    2011

  27. [34]

    Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. 2024. Graphgpt: Graph instruction tuning for large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 491–500

    2024

  28. [35]

    Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Ar- netminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 990–998

    2008

  29. [36]

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In International conference on machine learning . PMLR, 2071–2080

    2016

  30. [37]

    Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu

  31. [38]

    Guangxu Xun, Kishlay Jha, Jianhui Sun, and Aidong Zhang. 2020. Correlation networks for extreme multi-label text classification. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining . 1074–1082

    2020

  32. [39]

    Junhan Yang, Zheng Liu, Shitao Xiao, Chaozhuo Li, Defu Lian, Sanjay Agrawal, Amit Singh, Guangzhong Sun, and Xing Xie. 2021. Graphformers: Gnn-nested transformers for representation learning on textual graph. Advances in Neural Information Processing Systems 34 (2021), 28798–28810

    2021

  33. [40]

    Yaming Yang, Ziyu Guan, Jianxin Li, Wei Zhao, Jiangtao Cui, and Quan Wang

  34. [41]

    Yaming Yang, Ziyu Guan, Zhe Wang, Wei Zhao, Cai Xu, Weigang Lu, and Jianbin Huang. 2022. Self-supervised heterogeneous graph pre-training based on struc- tural clustering. Advances in Neural Information Processing Systems 35 (2022), 16962–16974

    2022

  35. [42]

    Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2022. Linkbert: Pretraining language models with document links. arXiv preprint arXiv:2203.15827 (2022)

    2022 arXiv

  36. [43]

    Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. 2024. Language is all a graph needs. In Findings of the Association for Computational Linguistics: EACL 2024. 1955–1973

    2024

  37. [44]

    Zhizhi Yu, Di Jin, Ziyang Liu, Dongxiao He, Xiao Wang, Hanghang Tong, and Jiawei Han. 2021. AS-GCN: Adaptive semantic architecture of graph convolu- tional networks for text-rich networks. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 837–846

    2021

  38. [45]

    Zhizhi Yu, Di Jin, Ziyang Liu, Dongxiao He, Xiao Wang, Hanghang Tong, and Jiawei Han. 2023. Embedding text-rich graph neural networks with sequence and topical semantic structures. Knowledge and Information Systems 65, 2 (2023), 613–640

    2023

  39. [46]

    Zhizhi Yu, Di Jin, Jianguo Wei, Yawen Li, Ziyang Liu, Yue Shang, Jiawei Han, and Lingfei Wu. 2023. TeKo: Text-Rich Graph Neural Networks With External Knowledge. IEEE Transactions on Neural Networks and Learning Systems (2023)

    2023

  40. [47]

    Chuxu Zhang, Ananthram Swami, and Nitesh V Chawla. 2019. Shne: Representa- tion learning for semantic-associated heterogeneous networks. In Proceedings of the twelfth ACM international conference on web search and data mining. 690–698. Conference’17, July 2017, Washington, DC,...

    2019

  41. [48]

    Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang, Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, et al. 2019. OAG: Toward linking large-scale heterogeneous entity graphs. InProceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data m...

    2019

  42. [49]

    Jason Zhu, Yanling Cui, Yuming Liu, Hao Sun, Xue Li, Markus Pelger, Tianqi Yang, Liangjie Zhang, Ruofei Zhang, and Huasha Zhao. 2021. Textgnn: Improving text encoder via graph neural network in sponsored search. In Proceedings of the Web Conference 2021. 2848–2857

    2021

  43. [2008]

    the Journal of machine Learning research 9 (2008), 1871–1874

    LIBLINEAR: A library for large linear classification. the Journal of machine Learning research 9 (2008), 1871–1874

    2008

  44. [2019]

    In The world wide web conference

    Heterogeneous graph attention network. In The world wide web conference . 2022–2032

    2022

  45. [2021]

    IEEE Transactions on Knowledge and Data Engineering 35, 2 (2021), 1637–1650

    Interpretable and efficient heterogeneous graph convolutional network. IEEE Transactions on Knowledge and Data Engineering 35, 2 (2021), 1637–1650

    2021

  46. [2022]

    In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining

    Twhin: Embedding the twitter heterogeneous information network for personalized recommendation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining . 2842–2850

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.