REVIEW 6 cited by
SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Textual graphs (TGs) are graphs whose nodes correspond to text (sentences or documents), which are widely prevalent. The representation learning of TGs involves two stages: (i) unsupervised feature extraction and (ii) supervised graph representation learning. In recent years, extensive efforts have been devoted to the latter stage, where Graph Neural Networks (GNNs) have dominated. However, the former stage for most existing graph benchmarks still relies on traditional feature engineering techniques. More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage). In this work, we present SimTeG, a frustratingly Simple approach for Textual Graph learning that does not innovate in frameworks, models, and tasks. Instead, we first perform supervised parameter-efficient fine-tuning (PEFT) on a pre-trained LM on the downstream task, such as node classification. We then generate node embeddings using the last hidden states of finetuned LM. These derived features can be further utilized by any GNN for training on the same task. We evaluate our approach on two fundamental graph representation learning tasks: node classification and link prediction. Through extensive experiments, we show that our approach significantly improves the performance of various GNNs on multiple graph benchmarks.
Forward citations
Cited by 6 Pith papers
-
Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?
Morpher adapts pre-trained GNNs to language using multi-modal prompts and a projector, achieving few-shot, cross-domain, and zero-shot unseen-class classification with weak text supervision.
-
Both Topology and Text Matter: Revisiting LLM-guided Out-of-Distribution Detection on Text-attributed Graphs
LG-Plug mines pseudo-OOD exposures from clustered unlabeled nodes via iterative LLM prompting and regularizes topology-driven graph OOD detectors, cutting FPR95 by ≥7% across six TAG benchmarks.
-
GraphDancer: Training LLMs to Explore and Reason over Graphs via Two-Stage Curriculum Post-Training
Curriculum-based reinforcement learning teaches a small LLM to explore knowledge graphs with function calls and generalize across domains.
-
Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment
GAGA matches or exceeds state-of-the-art accuracy on several text-attributed graph benchmarks while requiring large language model annotations for only 1% of nodes or edges.
-
Each Graph is a New Language: Graph Learning with LLMs
GDL4LLM pre-trains LLMs to predict random-walk sequences of node tokens, then fine-tunes on those sequences for node classification, outperforming description- and embedding-based baselines.
-
Graph Neural Networks Need Cluster-Normalize-Activate Modules
A Cluster-Normalize-Activate module replaces plain activations in GNNs, limiting oversmoothing and improving accuracy with fewer parameters.
Discussion (0). Continue with ORCID to comment.