pith. sign in

arxiv: 2603.14937 · v3 · submitted 2026-03-16 · 💻 cs.LG · cs.CL

LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphs

Ying Zhang , Hang Yu , Haipeng Zhang , Peng Di This is my paper

Pith reviewed 2026-05-15 10:23 UTC · model grok-4.3

classification 💻 cs.LG cs.CL
keywords text-rich graphslarge language modelsmessage passinggraph neural networksRAMPgraph kernelsraw text anchoring
0
0 comments X

The pith

LLMs function as graph aggregation operators when message passing stays anchored to each node's raw text.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that text-rich graphs suffer from information loss when text is first compressed into static embeddings before graph reasoning begins. RAMP instead keeps the LLM's inference grounded in the original raw text of every node at each propagation step while incorporating optimized messages from neighbors. This dual scheme lets structural updates remain tied to full textual content and unifies discriminative and generative tasks under one generative framing. A sympathetic reader would care because it removes the usual detachment between deep language understanding and graph structure, potentially yielding better results on tasks that require both.

Core claim

We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation.

What carries the argument

RAMP's dual-representation scheme, which anchors LLM inference directly on each node's raw text while propagating dynamically optimized neighbor messages.

If this is right

  • Bridges graph propagation directly with deep text reasoning without intermediate compression steps.
  • Achieves competitive performance across text-rich graph tasks.
  • Unifies discriminative and generative graph tasks under one generative formulation.
  • Demonstrates LLMs can operate as graph kernels for general-purpose learning on text-rich data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same anchoring idea might let LLMs replace conventional aggregation functions inside existing GNN architectures when node attributes are textual.
  • Similar raw-input anchoring could be tested on graphs whose nodes carry other rich modalities such as images or time series.
  • If the method scales, pipelines for text-rich graphs could drop separate embedding models entirely.

Load-bearing premise

Repeatedly anchoring LLM inference to each node's raw text during message passing will avoid information bottlenecks and let the LLM serve effectively as a graph aggregation operator without task-specific fine-tuning.

What would settle it

Experiments on standard text-rich graph benchmarks in which RAMP fails to match or exceed the accuracy of conventional embedding-plus-GNN pipelines would falsify the claim that raw-text anchoring enables effective LLM-based aggregation.

read the original abstract

Text-rich graphs, which integrate complex structural dependencies with abundant textual information, are ubiquitous yet remain challenging for existing learning paradigms. Conventional methods and even LLM-hybrids compress rich text into static embeddings or summaries before structural reasoning, creating an information bottleneck and detaching updates from the raw content. We argue that in text-rich graphs, the text is not merely a node attribute but the primary medium through which structural relationships are manifested. We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation. Extensive experiments show that RAMP effectively bridges the gap between graph propagation and deep text reasoning, achieving competitive performance and offering new insights into the role of LLMs as graph kernels for general-purpose graph learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces RAMP, a Raw-text Anchored Message Passing method for text-rich graphs. It recasts the LLM itself as a graph-native aggregation operator via a dual-representation scheme that anchors each iteration on the node's raw text while propagating dynamically optimized messages from neighbors. The approach unifies discriminative and generative tasks under a single generative formulation and claims to avoid information bottlenecks that arise when LLMs are used only as static feature extractors, with extensive experiments showing competitive performance.

Significance. If the experimental results hold, RAMP would provide a concrete mechanism for treating LLMs as native graph kernels rather than auxiliary encoders, potentially improving multi-hop reasoning on text-rich graphs without task-specific fine-tuning. This could influence general-purpose graph learning by reducing reliance on compressed embeddings and offering a unified generative interface for both node classification and generation tasks.

major comments (2)
  1. [Abstract and §3] Abstract and §3 (method description): the central claim that the LLM functions as a 'graph-native aggregation operator' lacks a formal specification of prompt construction (how raw-text anchor and neighbor messages are jointly encoded) and output extraction (how the LLM response becomes the next message). Without this, it is unclear whether the operator truly integrates structural signals or simply performs per-node inference with extra context.
  2. [Experiments] Experiments section: the abstract asserts 'competitive performance' and 'extensive experiments,' yet no specific datasets, baselines, metrics, ablation studies on hop depth, or quantitative results (e.g., accuracy deltas or dilution metrics) are referenced in the provided text, preventing verification that raw-text anchoring prevents progressive information loss over iterations.
minor comments (1)
  1. [§3] Notation for the dual-representation scheme is introduced without an accompanying equation or pseudocode block, making the iterative update rule difficult to reproduce from the textual description alone.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below, providing clarifications from the manuscript and indicating revisions where they strengthen the presentation without misrepresenting our contributions.

read point-by-point responses

  1. Referee: [Abstract and §3] Abstract and §3 (method description): the central claim that the LLM functions as a 'graph-native aggregation operator' lacks a formal specification of prompt construction (how raw-text anchor and neighbor messages are jointly encoded) and output extraction (how the LLM response becomes the next message). Without this, it is unclear whether the operator truly integrates structural signals or simply performs per-node inference with extra context.

    Authors: We appreciate this observation. Section 3 describes the dual-representation scheme in which each iteration anchors on the node's raw text while incorporating neighbor messages, but we agree a more explicit formalization would help. In the revision we will add a precise prompt template showing joint encoding of the raw-text anchor and the dynamically optimized neighbor messages, along with the output parsing rule that converts the LLM response into the updated message for the subsequent iteration. This will make explicit how structural signals propagate through the anchored process rather than isolated per-node inference. revision: yes

  2. Referee: [Experiments] Experiments section: the abstract asserts 'competitive performance' and 'extensive experiments,' yet no specific datasets, baselines, metrics, ablation studies on hop depth, or quantitative results (e.g., accuracy deltas or dilution metrics) are referenced in the provided text, preventing verification that raw-text anchoring prevents progressive information loss over iterations.

    Authors: The full manuscript contains an experiments section reporting results on multiple text-rich graph datasets, comparisons against LLM feature-extractor baselines and standard GNNs, standard metrics, and ablations that vary hop depth. These include quantitative deltas and analysis showing that raw-text anchoring limits progressive dilution compared with embedding-only approaches. To improve accessibility we will expand the abstract with a concise summary of key datasets and performance highlights and add an explicit paragraph in the experiments section linking the hop-depth ablations to the information-loss claim. revision: partial

Circularity Check

0 steps flagged

No circularity: RAMP proposal is an independent operator design supported by experiments

full rationale

The paper introduces RAMP as a novel dual-representation scheme that anchors on raw text while propagating messages, without any equations, fitted parameters, or derivations that reduce the claimed aggregation operator to its own inputs by construction. No self-citations are invoked as load-bearing uniqueness theorems, and the central claim rests on the empirical observation of competitive performance rather than tautological redefinitions or ansatzes smuggled from prior work. The derivation chain is self-contained as a methodological proposal.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption that LLMs can act as effective dynamic aggregation operators when given repeated access to raw node text; no free parameters or invented physical entities are described.

axioms (1)
  • domain assumption Text in text-rich graphs is the primary medium through which structural relationships are manifested rather than a detachable node attribute.
    Explicitly stated in the abstract as the motivation for moving beyond static embeddings.
invented entities (1)
  • RAMP dual-representation scheme no independent evidence
    purpose: Anchors inference on raw text while propagating neighbor messages in each iteration.
    Newly introduced mechanism described in the abstract.

pith-pipeline@v0.9.0 · 5489 in / 1284 out tokens · 52584 ms · 2026-05-15T10:23:19.354798+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.