Pith. sign in

REVIEW 21 cited by

Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.19523 v5 pith:VEKQZK3D submitted 2023-05-31 cs.LG

Xiaoxin He , Xavier Bresson , Thomas Laurent , Adam Perold , Yann LeCun , Bryan Hooi This is my paper · ORCID
classification cs.LG
keywords featuresexplanationsgraphlearningllmsmethodtextualattributes
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Representation learning on text-attributed graphs (TAGs) has become a critical research problem in recent years. A typical example of a TAG is a paper citation graph, where the text of each paper serves as node attributes. Initial graph neural network (GNN) pipelines handled these text attributes by transforming them into shallow or hand-crafted features, such as skip-gram or bag-of-words features. Recent efforts have focused on enhancing these pipelines with language models (LMs), which typically demand intricate designs and substantial computational resources. With the advent of powerful large language models (LLMs) such as GPT or Llama2, which demonstrate an ability to reason and to utilize general knowledge, there is a growing need for techniques which combine the textual modelling abilities of LLMs with the structural learning capabilities of GNNs. Hence, in this work, we focus on leveraging LLMs to capture textual information as features, which can be used to boost GNN performance on downstream tasks. A key innovation is our use of explanations as features: we prompt an LLM to perform zero-shot classification, request textual explanations for its decision-making process, and design an LLM-to-LM interpreter to translate these explanations into informative features for downstream GNNs. Our experiments demonstrate that our method achieves state-of-the-art results on well-established TAG datasets, including Cora, PubMed, ogbn-arxiv, as well as our newly introduced dataset, tape-arxiv23. Furthermore, our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv. Lastly, we believe the versatility of the proposed method extends beyond TAGs and holds the potential to enhance other tasks involving graph-text data. Our codes and datasets are available at: https://github.com/XiaoxinHe/TAPE.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 21 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agentic Graph Token Reasoning

    cs.LG 2026-08 conditional novelty 7.0 of 10

    Agentic Graph Token (AGT) reasoning lets an LLM iteratively select graph views, encode them as tokens, and reason step-by-step, beating static graph-token and text-agent baselines on seven graph domains.

  2. Delving into Instance-Dependent Label Noise in Graph Data: A Comprehensive Study and Benchmark

    cs.LG 2025-06 conditional novelty 7.0 of 10

    BeGIN provides 10 graph datasets with six label-noise types and a broad evaluation, showing that LLM-simulated noise hurts GNNs more than uniform or pairwise noise.

  3. One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models

    cs.LG 2026-07 conditional novelty 6.0 of 10

    OMG-VLM is a single VLM-based model that handles text-, image-, and multi-attributed graphs through structure-aware adapters, reporting gains on several node/link prediction benchmarks.

  4. A Graph Talks, But Who's Listening? Rethinking Evaluations for Graph-Language Models

    cs.CL 2025-08 conditional novelty 6.0 of 10

    Graph-language models give no advantage over prompt-tuned LLMs on a new synthetic graph question-answering benchmark, while existing node-classification benchmarks can be solved using a single modality.

  5. Toward a Graph Foundation Model: Pre-Training Transformers With Random Walks

    cs.LG 2025-06 conditional novelty 6.0 of 10

    A random-walk-based Transformer pre-trained on multiple graph datasets transfers to unseen node, link, and graph tasks, though its expressivity proofs are incomplete.

  6. GraphLAMA: Enabling Efficient Adaptation of Graph Language Models with Limited Annotations

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A frozen-LLM graph model with a small tuned adapter (about 726k parameters, 3MB) outperforms in-context learning and instruction-tuned graph LLMs in few/zero-shot node classification and summary generation.

  7. Efficient Text-Attributed Graph Learning through Selective Annotation and Graph Alignment

    cs.LG 2025-06 conditional novelty 6.0 of 10

    GAGA matches or exceeds state-of-the-art accuracy on several text-attributed graph benchmarks while requiring large language model annotations for only 1% of nodes or edges.

  8. Contrastive Cross-Course Knowledge Tracing via Concept Graph Guided Knowledge Transfer

    cs.AI 2025-05 conditional novelty 6.0 of 10

    TransKT, a contrastive cross-course knowledge tracing model, uses LLM-built concept graphs and semantic features to transfer knowledge between courses and beats single-course baselines on three PTADisc-derived datasets.

  9. Griffin: Towards a Graph-Centric Relational Database Foundation Model

    cs.LG 2025-05 conditional novelty 6.0 of 10

    A single pretrained graph model for relational databases, Griffin unifies encoders and decoders and reports the best average rank across 24 tasks in its evaluation.

  10. Each Graph is a New Language: Graph Learning with LLMs

    cs.CL 2025-01 conditional novelty 6.0 of 10

    GDL4LLM pre-trains LLMs to predict random-walk sequences of node tokens, then fine-tunes on those sequences for node classification, outperforming description- and embedding-based baselines.

  11. UNIT: Unleash Large Language Models Potential for Graph Continual Learning

    cs.AI 2026-07 conditional novelty 5.5 of 10

    UNIT reaches SOTA graph continual learning by one-shot LLM instruction tuning plus uncertainty-aware semantic and structural class anchors that jointly set classifier weights.

  12. OpenRTAG: A Comprehensive Benchmark for Robust Text-Attributed Graph Learning under Data Quality Degradation

    cs.AI 2026-07 conditional novelty 5.0 of 10

    OpenRTAG is a benchmark that organizes text-attributed-graph data-quality issues into a 3x3 taxonomy (text/structure/label by sparsity/noise/imbalance) and evaluates model robustness across nine datasets and three tasks.

  13. GSTBench: A Benchmark Study on the Transferability of Graph Self-Supervised Learning

    cs.LG 2025-08 conditional novelty 5.0 of 10

    GSTBench finds that masked feature reconstruction (GraphMAE) is the only one of five graph self-supervised pretraining objectives that consistently transfers across eight datasets, while contrastive methods often perf...

  14. CS-Agent: LLM-based Community Search via Dual-agent Collaboration

    cs.SI 2025-08 conditional novelty 5.0 of 10

    CS-Agent, a Solver-Validator two-agent dialogue with a Decider selector, improves LLM community search on synthetic graphs, and GraphCS is a new benchmark for measuring it.

  15. NOCL: Node-Oriented Conceptualization LLM for Graph Tasks without Message Passing

    cs.LG 2025-05 conditional novelty 5.0 of 10

    NOCL lets an LLM handle node, edge, and graph tasks on text and non-text graphs by compressing each node's description into one semantic embedding and turning the graph into a text prompt.

  16. GraphICL: Unlocking Graph Learning Potential in LLMs through Structured Prompt Design

    cs.LG 2025-01 reject novelty 5.0 of 10

    A 55-template prompt benchmark showing general LLMs can beat specialized graph LLMs and GNNs on node classification and link prediction, though the reported margins are inflated by test-set prompt selection.

  17. GraphAgent: Agentic Graph Language Assistant

    cs.AI 2024-12 conditional novelty 5.0 of 10

    GraphAgent links an LLM agent pipeline with graph tokenization, automatically building semantic knowledge graphs and using them for both graph prediction and open-ended text generation.

  18. A Self-guided Multimodal Approach to Enhancing Graph Representation Learning for Alzheimer's Diseases

    cs.LG 2024-12 reject novelty 5.0 of 10

    A mask-guided multimodal GNN that fuses brain connectomes with PubMed abstract embeddings is claimed to improve AD classification and interpretability, but the evidence is undermined by inconsistent ablations and miss...

  19. Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling

    cs.LG 2025-05 reject novelty 4.0 of 10

    SDM-InstructGLM uses a similarity-degree biased random walk to sample graph neighborhoods for LLM prompts, reporting gains over InstructGLM on CORA and PUBMED that are confounded by a change in node features and train...

  20. Trustworthy GNNs with LLMs: A Systematic Review and Taxonomy

    cs.LG 2025-02 conditional novelty 4.0 of 10

    A review that groups LLM-GNN trustworthiness research under four dimensions, reliability, robustness, privacy, and reasoning, with no new experiments.

  21. Large Language Models Merging for Enhancing the Link Stealing Attack on Graph Neural Networks

    cs.CR 2024-12 conditional novelty 4.0 of 10

    A softmax-weighted model merging method combines multiple LLM-based link stealing attack models into one model that beats existing merging baselines across four graph datasets.

Pith tools