Pith. sign in

REVIEW 4 major objections 3 minor 51 references

LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification

T0 review · 4 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By swapping internal activations between graph pairs, the paper locates where a frozen LLM's features enter a GNN — at the node and raw-data level, consumed in the shallow layers — and turns this insight into an attention module that…

desk verdict A genuinely useful synthetic dataset and an interesting empirical localization result wrapped in a theoretical section whose proofs do not currently hold up; worth refereeing with a strong request to repair or reframe the theory. read the letter →

arxiv 2505.08265 v3 pith:DMJRNC7S submitted 2025-05-13 cs.LG cs.AI

Hang Gao , Wenxuan Huang , Fengge Wu , Junsuo Zhao , Changwen Zheng , Huaping Liu This is my paper · ORCID
classification cs.LGcs.AI MSC 68T0768R10
keywords LLMenhancersgraphneuralnetworksinterchangeinterventioncausalmechanismidentificationsyntheticdatasetattention-basedtransmissionnodeclassificationfeatureselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish where a frozen large language model's contribution actually enters the common 'LLM-enhancer-plus-GNN' pipeline, in which an LLM writes node features and a graph neural network classifies the graph. Using a new synthetic dataset with built-in causal relationships (CCSG) and the interchange-intervention method from causal inference, it finds that the LLM's features carry node-level and raw-data-level semantic information, and are consumed in the shallow layers of the GNN. It further claims this early-layer placement is invariant to GNN depth and width, and that the alignment loss $L_{II}$ tracks model accuracy. Based on those findings it designs an attention-based transmission (AT) module that selects which LLM token features to pass on, and reports consistent accuracy gains between roughly +0.7 and +3.1 points on Cora, Pubmed, and Instagram across several LLM and GNN backbones.

What carries the argument

The central tool is the interchange intervention loss $L_{II}$: a value of an internal variable $Z_f$ of the trained network, computed on a second graph sample, is swapped into the network processing a first sample, and the resulting output is compared, via cross-entropy, to the same swap performed in a hand-built high-level causal model $h(\cdot)$. Searching over candidate hidden layers for the one that minimizes $L_{II}$ locates where the causal variables' information is encoded, and Theorems 3.2 to 3.5 convert that search into a statement about equal total effects. The CCSG dataset supplies the ground-truth causal variables ($\Psi$, $\Phi$, $\Gamma$, $\Omega$) that appear in those models, and the AT module — a small transformer that scores LLM output tokens across multiple prompts and reweights them into the node feature vector — operationalizes the finding.

What would settle it

Train the same GNN pipeline on CCSG with a frozen non-LLM feature extractor of similar size; if the same early-layer alignment pattern and $L_{II}$-accuracy correlation appear, the findings describe generic shallow-feature processing rather than LLM semantic knowledge. Alternatively, retrain the identical architecture with different seeds and rerun the interchange search: if the layer minimizing $L_{II}$ moves across seeds or sample pairs, the claim about where information lives in the model fails.

Watch

Extended reading notes

Core claim

The central claim is that interchange interventions on the CCSG dataset reveal a precise division of labor in the LLM-enhancer-plus-GNN stack: with a fixed-parameter LLM, the features the LLM produces represent node-level and raw-data-level information, and the GNN internal variable that best matches the high-level causal model sits in layers 0 and 1, the early message-passing layers. The paper further claims that the GNN's internal structure after ingesting LLM input is largely invariant to model scale, that scaling up depth or width does not improve causal relation modeling, and that the minimum attainable interchange loss $L_{II}$ correlates with model accuracy. The design consequence is that the interface between the LLM output and the GNN input is a higher-leverage place to optimize than the backbone choice or model size, which motivates attention-weighted selection of token-level features as the AT module.

Load-bearing premise

The analysis assumes that a subset of the trained GNN's internal variables can be put into exact one-to-one correspondence with the variables of the hand-built causal models $h_{node}$ and $h_{graph}$; if no such correspondence really exists in a trained network, the layer that minimizes $L_{II}$ is only a proxy and the empirical findings partly describe the search procedure rather than the model's mechanism.

Editorial extensions

If this is right

  • Fixed-parameter LLM enhancers should feed their knowledge into the earliest GNN layers; routing LLM features through many rounds of message passing adds little under the paper's analysis.
  • Which token positions of the LLM output are selected as node features is a high-leverage design choice, reported as more consequential than the LLM family or GNN scale.
  • The interchange loss $L_{II}$ can serve as an indicator of model capability, since lower minimum alignment loss generally coincides with higher accuracy in the reported experiments.
  • Scaling up GNN depth or width mostly enlarges the structure without improving its ability to model the underlying causal relations, as the paper states as a hypothesis from Empirical Finding 2.
  • Attention-based selection over multiple prompts yields consistent gains across three datasets and three GNN backbones while leaving the LLM frozen, making the module plug-and-play.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the early-layer locus generalizes beyond CCSG, the design rule for hybrid LLM/GNN systems would be to architect the feature interface rather than the GNN capacity; a testable extension is to fine-tune the LLM enhancer and check whether the semantic information moves deeper into the stack or stays at the LLM output.
  • The even-layer-versus-odd-layer alignment asymmetry across depths could be probed by retraining the identical architecture with different seeds: if the asymmetry survives, it reflects a learned alternating internal structure rather than a property of the search procedure.
  • The $L_{II}$-as-capability link, if confirmed on other architectures, offers a cheap training-free probe for benchmarking model families without full evaluation runs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper analyzes the LLM-enhancer-plus-GNN paradigm by constructing a synthetic graph dataset (CCSG) with controllable causal relationships, then applying interchange interventions to identify which internal variables of the combined model correspond to variables of a hand-specified high-level causal model. It presents a theoretical validation of this alignment procedure (Theorem 3.2, Corollary 3.3, Proposition 3.4, Theorem 3.5), draws three empirical findings about where LLM-provided information enters the GNN, and proposes an Attention-based Transmission (AT) module that selects LLM token features and prompts. Table 2 reports consistent accuracy gains from the AT module across Cora, Pubmed, and Instagram with multiple LLM and GNN backbones.

Significance. If the causal-analysis claims were valid, the paper would provide a useful mechanistic account of LLM-enhanced GNNs and a practical plug-in module. The empirical infrastructure is a genuine strength: the authors build a controlled synthetic dataset, use standard backbones, report means and standard deviations, and release code. The AT module gains are consistent across three datasets and three LLM/GNN combinations, with improvements between roughly +0.7 and +3.1 points. However, the central theoretical validation is not established: the proofs of Theorems 3.2 and 3.5 contain circular or unsupported steps, and Empirical Finding 1 is drawn from a search restricted to GNN layers. Consequently, the analytical findings should be regarded as hypotheses about layer sensitivity rather than validated causal mechanisms.

major comments (4)
  1. [Section 3.2.3 / Appendix B.1, Eqs. (16)-(17)] The proof of Theorem 3.2 is circular and relies on an unstated value-preservation assumption. Equation (16) replaces f_{Z_f}(G_orig) with eta^{-1}(h_{Z_h}(G_orig)), although the theorem only assumes a bijection between variable sets and does not state that eta maps the value taken by Z_f on an input to the value taken by Z_h on the same input. More importantly, Eq. (17) concludes INTINV(f,Gorig,Gdiff,Z_f) = INTINV(h,Gorig,Gdiff,Z_h) by treating f_latter on a patched input as equal to h_latter on the corresponding patched input; that is exactly the interchange-invariance property the theorem is supposed to derive, not a consequence of the stated premises. Thus the claimed theoretical validation of the L_II minimization does not follow.
  2. [Appendix B.4, Eq. (31)] The claim that because Zhat^1_j blocks all causal routes from X_j to Y_f, one has INTINV(f,Gorig,Gdiff,S) = INTINV(f,Gorig,Gdiff,Zhat^1_j) for every S in {X_j} union {eZ^i_j} is false in general. Causal blocking gives conditional independence in the observational distribution, not equality of counterfactual interchange outcomes. Patching an intermediate LLM state eZ^i_j changes the subsequent LLM computation and therefore the value of Zhat^1_j in a way that need not coincide with the effect of patching Zhat^1_j directly, because Zhat^1_j is a non-injective, nonlinear function of the LLM layers. Since Eq. (31) is the only step that eliminates LLM variables from the intervention, Theorem 3.5 is unproven. The IC-algorithm argument in Lemma B.2 recovers only the architecture's dependency graph; it supplies no functional equations and no counterfactual equivalence.
  3. [Section 3.3.2 / Figure 3] Empirical Finding 1 is not supported by the reported experiments because the L_II search is restricted to GNN layers, as stated in Section 3.3. The observation that the smallest L_II among GNN layers occurs in layers 0-1 shows only that early GNN layers are more sensitive to patch-induced changes than later layers; it does not localize where the LLM's information is consumed, nor does it establish that the LLM output features 'serve the function of representing information at the node level and the raw data level.' The text also says this finding 'partially validates Theorem 3.5,' but since Theorem 3.5 is not established, that statement is unsupported. A control search over LLM intermediate variables, or an intervention that patches the entire LLM output, would be needed before this claim can be made.
  4. [Appendix B.3, Eq. (29) and Proposition 3.4] The justification of Proposition 3.4 contains an incoherent inequality: the left-hand side TE_{zf,zf'}(Y_f) - TE_{zf,zf'}(Y_f) is identically zero, and the surrounding sentence ('we have total effect of Z_{f,b} upon the output Y is strictly similar to Z_h than Z_{f,a}') is not a well-formed claim. More substantively, the proposition asserts that reaching the minimal possible value L*_II implies exact interchange equality and hence equal total effects; this assumes the existence of a variable that attains the lower bound, which is precisely what the earlier theorems were meant to establish. As a result, Proposition 3.4 does not provide an independent justification of the method.
minor comments (3)
  1. [Section 3.2.2, Step 4] Step 4 says 'utilize Equation 2 to calculate the loss L_II,' but the loss is defined in Equation 1; Equation 2 is the high-level model formula for the running example.
  2. [Table 5] The table lists 'Bipartite Graph' twice; the second row's description ('Outer nodes connected in a ring with a central node') appears to describe a different topology and should be re-labeled or corrected.
  3. [Appendix D.1 / Table 6] The experimental details state that q was set to 10 for the AT module, while Table 6 reports results only for q = 1, 2, 3, 4; please clarify whether q = 10 was used in Table 2 and why the ablation stops at q = 4.

Circularity Check

4 steps flagged · score 7.0 of 10

The method-validation theorems are circular: L*_II is defined as D(h,h), Theorem 3.2's proof assumes the interchange equality it must establish, and Theorem 3.5's Eq. (31) asserts the GNN-sufficiency conclusion; Empirical Finding 1 then validates Theorem 3.5 with a search the theorem itself restricted.

  1. self definitional [Section 3.2.3, Proposition 3.4 / Eq. (3)]
    "Proposition 3.4. For a high-level causal modelh(·) and a low-level neural networkf(·), both mapping input graphsG∈G to outputs Y ∈Y such that Y = f(G) = h(G), if Z f within f(·) and Z h within h(·) minimizeLII to its optimal valueL∗ II, thenZ f aligns best withZ h and has an identical total effect on the output prediction as Zh. The minimalL∗ II is given by: L∗ II = 1/G2 Σ_{Gorig∈G} Σ_{Gdiff∈G} D(INTINV(h,Gorig,Gdiff,Zh), INTINV(h,Gorig,Gdiff,Zh)). (3)"

    LII (Eq. 1) is defined as the distance between INTINV(h,...) and INTINV(f,...). 'Aligns best' has no independent content: the proposition defines best alignment as attaining the minimum of LII, and the minimum is by construction D(h,h), i.e., exact equality of the two intervention outputs. The asserted 'identical total effect' is thus the minimization objective restated, not a derived prediction. This proposition is the paper's theoretical warrant for using LII minima to locate Zf.

  2. other [Appendix B.1, proof of Theorem 3.2, Eqs. (16)-(17)]
    "f latter [f pre(Gorig)\f Zf (Gorig)∪η−1(h Zh (Gorig))] = h latter [hpre(Gorig)\h Zh (Gorig)∪h Zh (Gorig)]. (16) Then, we have: INTINV(f,Gorig,Gdiff,Zf) = f latter [f pre(Gorig)\f Zf (Gorig)∪η−1(h Zh (Gdiff))] = h latter [hpre(Gorig)\h Zh (Gorig)∪h Zh (Gdiff)] = INTINV(h,Gorig,Gdiff,Zh). (17)"

    Equation (16) asserts that replacing Zf by the bijective image of Zh's value inside f produces the same output as replacing Zh inside h. That interchange invariance is exactly the counterfactual correspondence Theorem 3.2 claims to establish. The stated assumptions—Y=f(G)=h(G) and a bijection η between variable sets—only equate outputs on complete inputs; they do not imply equality under partial internal substitution. The proof therefore reaches Eq. (17) by assuming the conclusion at Eq. (16). This is load-bearing because Theorem 3.2 is the theoretical validation of the whole LII search.

2 more flagged steps
  1. other [Appendix B.4, proof of Theorem 3.5, Eq. (31)]
    "As the SCM demonstrates, we have ˆZ 1 j block all causal routes fromXj toY f. Therefore, we have: INTINV(f,Gorig,Gdiff,S) = INTINV(h,Gorig,Gdiff, ˆZ 1 j), S ∈ {Xj}∪{ eZ i j} eL i=1. (31)"

    The theorem's conclusion is that GNN-internal variables are always sufficient to constitute Zf. Equation (31) asserts precisely that conclusion for every LLM hidden variable eZ^i_j: each is interventionally exchangeable with the first GNN layer variable Zhat^1_j. Causal blocking (d-separation) yields conditional independence, not interchange equivalence for non-injective nonlinear internal states, and Lemma B.2 only recovers the architecture's dependency graph, not functional equations. The proof then uses Eq. (31) to replace LLM variables by GNN variables in Eqs. (32)-(35), so the pivotal premise is the theorem's own claim. The RHS of Eq. (31) is also ill-typed: h has no variable Zhat^1_j.

  2. other [Section 3.3 / 3.3.2 (restriction to GNN layers and Empirical Finding 1)]
    "According to the theorem, and taking into account the difficulty of fixed LLMs in modeling causal relationships, the subsequent analysis focuses on using the GNN model to examine causal relationship modeling within the framework. [...] This indicates that the role of the LLM enhancer is to process node-level and raw data-level information. Such a finding also partially validates Theorem 3.5."

    Theorem 3.5 is used to justify restricting all LII searches to GNN-internal layers; no LLM hidden variable is ever a candidate. The resulting minima—which layer attains the lowest LII—are then offered as partial validation of Theorem 3.5 and as Empirical Finding 1 ('features output by the LLM serve the function of representing information at the node level'). This is a closed loop: the theorem licenses the restricted search, and the restricted search's outcome is evidence for the theorem and for where LLM information is consumed. The measured localization may only reflect which GNN layer is most sensitive to patching within the allowed candidate set.

full rationale

The central analytical claim—that interchange interventions localize where the fixed LLM's information enters the GNN—is not self-contained. The validation bundle (Theorem 3.2 and Proposition 3.4) makes 'best alignment' definitional by setting L*_II = D(h,h), and the proof of Theorem 3.2 assumes interchange invariance at Eq. (16). The sufficiency theorem that licenses searching only GNN layers (Theorem 3.5) is proved via Eq. (31), which simply asserts that every LLM hidden variable is exchangeable with a first-layer GNN variable. Empirical Finding 1 then completes the loop by using the GNN-only search results to 'partially validate' Theorem 3.5. The AT module and its accuracy gains on Cora, Pubmed, and Instagram are genuine external empirical contributions and are not circular; the self-citations to Gao et al. 2024 are not load-bearing. However, the paper's advertised mechanism-identification result depends on the circular theory chain, so the overall circularity score is 7 rather than lower.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The central claim rests on assumptions the paper itself supplies. Theorems 3.2 and 3.5 assume or assert the existence of exact correspondences between GNN internals and the authors' hand-built causal models; the SCM in Figure 8 encodes the GNN architecture and is asserted, not measured. The ground-truth causal rules h_node and h_graph are constructed by the authors for the synthetic data, so 'causal identification' is evaluated against an author-defined target. The AT module depends on hand-set hyperparameters q, m, delta. No physical or model entities are invented.

free parameters (4)
  • q (number of prompts in AT module) = 10 in main experiments; ablation optimum 2
    Hand-set hyperparameter controlling prompt diversity; Appendix D.1 sets q=10 while Table 6 shows q=2 best on Cora and Pubmed, an unexplained gap.
  • m (number of selected LLM token features) = 2
    Hand-set in Appendix D.1; only two positional features per prompt are carried into the attention module.
  • delta (prompt-selection warm-up epochs) = 10
    Hand-set in Appendix D.1; after delta epochs one prompt is fixed and the rest removed.
  • 90% accuracy floor for analytical experiments = at least 90%
    Section 3.3.1 adjusts training epochs and task difficulty so all models exceed 90% accuracy; this hand-chosen operating point shapes Empirical Finding 3's correlation claim.
assumptions (6)
  • domain assumption An exact bijective mapping eta from a subset of the GNN's intermediate variables to the variables of h_node and h_graph exists (Theorem 3.2 premise).
    Section 3.2.3 states the theorem's assumption 'there exists a subset Z_f of the intermediate variables in f(·) and a bijective mapping eta: Z_f -> Z_h'; this existence is the point at issue and is not established for trained GNNs.
  • ad hoc to paper The SCM in Figure 8 represents the general causal relationships among variables in the LLM-enhancer-plus-GNN model, with Pa(Z^l_i) = {Z^(l-1)_k : k in N(i) union {i}}.
    Lemma B.2 in Appendix B.4 asserts this by an IC-algorithm-style enumeration that mirrors the GNN message-passing architecture; Theorem 3.5 depends on it.
  • domain assumption The interchange search over single GNN layers identifies variables that behave like the abstract variables of h(·) when swap-based counterfactuals are applied.
    The analysis in Section 3.3 equates layer-level alignment (optimal L_II per layer) with the presence of the abstract variable in that layer; finer-grained or cross-layer variables are not considered.
  • domain assumption The LLM enhancer is fixed-parameter and the findings are scoped to frozen LLMs.
    Section 3.3 states 'Most approaches use fixed-parameter LLM enhancers to reduce training costs, our experiments also follow this setting', and Empirical Finding 1 is explicitly limited to this case.
  • domain assumption The hand-defined h_node and h_graph equal the true label-generating process of the CCSG dataset.
    The causal ground truth is constructed by the authors (Section 3.3 and Appendix C), so alignment is measured against an author-defined target rather than an independently validated generative model.
  • standard math Total effect decomposes into natural direct and indirect effects and can be computed by the replacement formulas in Eq. (18) to (21) (Pearl, 2009).
    These are cited standard results, but applying them requires the neural network to have a structural graph with the parent sets used in the proof, which the paper asserts rather than establishes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification." pith.science (2026). https://pith.science/paper/DMJRNC7S

@misc{pith2026250508265,
  author       = {Pith},
  title        = {Pith review of: LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DMJRNC7S}},
  note         = {Machine review of arXiv:2505.08265}
}
read the original abstract

The use of large language models (LLMs) as feature enhancers to optimize node representations, which are then used as inputs for graph neural networks (GNNs), has shown significant potential in graph representation learning. However, the fundamental properties of this approach remain underexplored. To address this issue, we propose conducting a more in-depth analysis of this issue based on the interchange intervention method. First, we construct a synthetic graph dataset with controllable causal relationships, enabling precise manipulation of semantic relationships and causal modeling to provide data for analysis. Using this dataset, we conduct interchange interventions to examine the deeper properties of LLM enhancers and GNNs, uncovering their underlying logic and internal mechanisms. Building on the analytical results, we design a plug-and-play optimization module to improve the information transfer between LLM enhancers and GNNs. Experiments across multiple datasets and models validate the proposed module.

Figures

Figures reproduced from arXiv: 2505.08265 by the authors.

Figure 1
Figure 1. A simple illustration of variable alignment achieved by interchange intervention. To address this issue, we introduce the interchange intervention approach (Wu et al., 2023a; Huang et al., 2023a) from causality the￾ory (Pearl, 2022) to per￾form the analysis. Specifi￾cally, we construct a novel synthetic graph dataset with controllable causal re￾lationships. Here, causal relationships refer to the ground-truth cause-… view at source ↗
Figure 2
Figure 2. The framework of the running example. During data generation, we also define the high-level causal model h(·) for this example. Specifically, we manually establish that the class of node v is determined by first identifying the classes of all its neighboring nodes and then selecting the most frequent class among them as the class of v. This logic can be formally expressed as C(v) = m({C(u) | u ∈ N (v)}), where C(·) … view at source ↗
Figure 3
Figure 3. Analytical experiment results. In the figure, each small square represents the optimal value of LII for a specific GNN layer. Green squares indicate the minimum value of LII within the corresponding row. Details concerning the variables can be found in Appendix C. where Gorig and Gdiff differ at or above the scale of indi￾vidual nodes. Then, the internal variables within the GNN model are always sufficient to consti… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Experimental results under different GNN scales. feature, while h node,2 (·) captures and analyzes the relation￾ships and properties among these nodes. Finally, h node,3 (·) serves as the ultimate processing function, outputting the features of the target node v. The d…
Figure 6
Figure 6. Figure 6: Experimental results under different feature positions. The numbers represent the relative positions of the tokens corre￾sponding to the adopted features within the entire output, ranging from a minimum of 1 to a maximum of 10. Empirical Finding 2. After receiving inpu…
Figure 7
Figure 7. Figure 7: Framework of the proposed AT module [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: The graphical representation of the proposed SCM. Elements in set {Zei,el−1 j } He le j=1 holds right-arrows point to Zei,el j and {Zˆ ˆl−1 k |k ∈ N (i)∪ {i}} holds right-arrows point to Zˆ ˆl i . Some of the above-mentioned arrows are omitted due to complexity. The re…
Figure 9
Figure 9. Figure 9: Example Graph. C.2. Node Correlations With a diverse set of rich, multi-category node features in place, the next step in constructing a graph dataset is to build its topological structure. One approach to this is to base the construction on the interrelationships betw…
Figure 10
Figure 10. Figure 10: Alignment results of graph-level and node-level experiments with varying GNN layers. For graph-level analytical experiments, we construct up to 1500 graph samples, use a total of 10 topological structures, each of which consists of 20.5 node. We select 300 samples for…
Figure 11
Figure 11. Figure 11: Alignment results of graph-level experiments with varying GNN layers and hidden dimensions. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Experimental results under different number of GNN layers and hidden dimensions. (a) The results of node-level experiments, where Z h corresponds to the output variables Ψ1, Ψ2 and Ψ3 of h node,1 (·), using OGBN-Arxiv data as node features, are presented. Bold values …
Figure 13
Figure 13. Figure 13: Extra experiments using OGBN-Arxiv and random data. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 23 canonical work pages

  1. [1]

    D., D'Oosterlinck, K., Feder, A., Gat, Y

    Abraham, E. D., D'Oosterlinck, K., Feder, A., Gat, Y. O., Geiger, A., Potts, C., Reichart, R., and Wu, Z. Cebab: Estimating the causal effects of real-world concepts on NLP model behavior. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Inf...

    work page 2022

  2. [2]

    Beckers, S., Eberhardt, F., and Halpern, J. Y. Approximate causal abstractions. In Globerson, A. and Silva, R. (eds.), Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 , volume 115 of Proceedings of Machine Learning Research, pp.\ 606--615. AUAI Press, 2019. URL http://proce...

    work page 2019

  3. [3]

    Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert - Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford...

    2020

  4. [4]

    Causal feature learning: an overview

    Chalupka, K., Eberhardt, F., and Perona, P. Causal feature learning: an overview. Behaviormetrika, 44: 0 137--164, 2017

    work page 2017

  5. [5]

    Exploring the potential of large language models (llms)in learning on graphs

    Chen, Z., Mao, H., Li, H., Jin, W., Wen, H., Wei, X., Wang, S., Yin, D., Fan, W., Liu, H., and Tang, J. Exploring the potential of large language models (llms)in learning on graphs. SIGKDD Explor. , 25 0 (2): 0 42--61, 2023. doi:10.1145/3655103.3655110. URL https://doi.org/10.1145/3655103.3655110

    arXiv 2023

  6. [6]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Devlin, J., Chang, M., Lee, K., and Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, ...

    doi:10.18653/v1/n 2019

  7. [7]

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al - Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Rozi \` e re, B., Biron, B., Tang, B., Chern, B., Caucheteux, C., Nayak, C., Bi, C....

    arXiv doi:10.48550/arxiv.2407.21783 2024

  8. [9]

    Measuring causal effects of data statistics on language model's 'factual' predictions

    Elazar, Y., Kassner, N., Ravfogel, S., Feder, A., Ravichander, A., Mosbach, M., Belinkov, Y., Sch \" u tze, H., and Goldberg, Y. Measuring causal effects of data statistics on language model's 'factual' predictions. CoRR, abs/2207.14251, 2022. doi:10.48550/ARXIV.2207.14251. URL https://doi.org/10.48550/arXiv.2207.14251

    arXiv doi:10.48550/arxiv.2207.14251 2022

Show all 51 references
  1. [10]

    Talk like a graph: Encoding graphs for large language models

    Fatemi, B., Halcrow, J., and Perozzi, B. Talk like a graph: Encoding graphs for large language models. CoRR, abs/2310.04560, 2023. doi:10.48550/ARXIV.2310.04560. URL https://doi.org/10.48550/arXiv.2310.04560

    2023 doi arXiv

  2. [11]

    Rethinking causal relationships learning in graph neural networks

    Gao, H., Yao, C., Li, J., Si, L., Jin, Y., Wu, F., Zheng, C., and Liu, H. Rethinking causal relationships learning in graph neural networks. In Wooldridge, M. J., Dy, J. G., and Natarajan, S. (eds.), Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-S...

    2024

  3. [12]

    Neural natural language inference models partially embed theories of lexical entailment and negation

    Geiger, A., Richardson, K., and Potts, C. Neural natural language inference models partially embed theories of lexical entailment and negation. In Alishahi, A., Belinkov, Y., Chrupala, G., Hupkes, D., Pinter, Y., and Sajjad, H. (eds.), Proceedings of the Third BlackboxNLP Work...

    2020 doi

  4. [13]

    Causal abstractions of neural networks

    Geiger, A., Lu, H., Icard, T., and Potts, C. Causal abstractions of neural networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing ...

    2021

  5. [14]

    Inducing causal structure for interpretable neural networks

    Geiger, A., Wu, Z., Lu, H., Rozner, J., Kreiss, E., Icard, T., Goodman, N., and Potts, C. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, pp.\ 7324--7338. PMLR, 2022 a

    2022

  6. [15]

    D., and Potts, C

    Geiger, A., Wu, Z., Lu, H., Rozner, J., Kreiss, E., Icard, T., Goodman, N. D., and Potts, C. Inducing causal structure for interpretable neural networks. In Chaudhuri, K., Jegelka, S., Song, L., Szepesv \' a ri, C., Niu, G., and Sabato, S. (eds.), International Conference on M...

    2022

  7. [16]

    Geiger, A., Wu, Z., Potts, C., Icard, T., and Goodman, N. D. Finding alignments between interpretable causal variables and distributed neural representations. In Locatello, F. and Didelez, V. (eds.), Causal Learning and Reasoning, 1-3 April 2024, Los Angeles, California, USA ,...

    2024

  8. [17]

    L., Ying, Z., and Leskovec, J

    Hamilton, W. L., Ying, Z., and Leskovec, J. Inductive representation learning on large graphs. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Co...

    2017

  9. [18]

    Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning

    He, X., Bresson, X., Laurent, T., Perold, A., LeCun, Y., and Hooi, B. Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May...

    2024

  10. [19]

    Huang, C., Ren, X., Tang, J., Yin, D., and Chawla, N. V. Large language models for graphs: Progresses and directions. In Chua, T., Ngo, C., Lee, R. K., Kumar, R., and Lauw, H. W. (eds.), Companion Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, Singapore, M...

    2024

  11. [20]

    Inducing character-level structure in subword-based language models with type-level interchange intervention training

    Huang, J., Wu, Z., Mahowald, K., and Potts, C. Inducing character-level structure in subword-based language models with type-level interchange intervention training. In Rogers, A., Boyd - Graber, J. L., and Okazaki, N. (eds.), Findings of the Association for Computational Ling...

    2023 doi

  12. [21]

    PRODIGY: enabling in-context learning over graphs

    Huang, Q., Ren, H., Chen, P., Krzmanc, G., Zeng, D., Liang, P., and Leskovec, J. PRODIGY: enabling in-context learning over graphs. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual...

    2023

  13. [22]

    Can GNN be good adapter for llms? In Chua, T., Ngo, C., Kumar, R., Lauw, H

    Huang, X., Han, K., Yang, Y., Bao, D., Tao, Q., Chai, Z., and Zhu, Q. Can GNN be good adapter for llms? In Chua, T., Ngo, C., Kumar, R., Lauw, H. W., and Lee, R. K. (eds.), Proceedings of the ACM on Web Conference 2024, WWW 2024, Singapore, May 13-17, 2024 , pp.\ 893--904. ACM...

    2024

  14. [23]

    and Simon, H

    Iwasaki, Y. and Simon, H. A. Causality and model abstraction. Artif. Intell., 67 0 (1): 0 143--194, 1994. doi:10.1016/0004-3702(94)90014-0. URL https://doi.org/10.1016/0004-3702(94)90014-0

    1994 doi

  15. [24]

    Kipf, T. N. and Welling, M. Semi-supervised classification with graph convolutional networks. In Advances in Neural Information Processing Systems (NeurIPS), pp.\ 1--10, 2016. URL https://arxiv.org/abs/1609.02907

    2016 arXiv

  16. [25]

    Semantic reshuffling with LLM and heterogeneous graph auto-encoder for enhanced rumor detection

    Li, G., Hu, D., Liu, Z., Zhang, X., and Lyu, H. Semantic reshuffling with LLM and heterogeneous graph auto-encoder for enhanced rumor detection. In Rambow, O., Wanner, L., Apidianaki, M., Al - Khalifa, H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st Inte...

    2025

  17. [26]

    Liu, A., Pan, L., Lu, Y., Li, J., Hu, X., Zhang, X., Wen, L., King, I., Xiong, H., and Yu, P. S. A survey of text watermarking in the era of large language models. ACM Comput. Surv. , 57 0 (2): 0 47:1--47:36, 2025. doi:10.1145/3691626. URL https://doi.org/10.1145/3691626

    2025 doi

  18. [27]

    One for all: Towards training one graph model for all classification tasks

    Liu, H., Feng, J., Kong, L., Liang, N., Tao, D., Chen, Y., and Zhang, M. One for all: Towards training one graph model for all classification tasks. In The Twelfth International Conference on Learning Representations, 2024

    2024

  19. [28]

    and Pavlick, E

    Lovering, C. and Pavlick, E. Unit testing for concepts in neural networks. Trans. Assoc. Comput. Linguistics, 10: 0 1193--1208, 2022. doi:10.1162/TACL\_A\_00514. URL https://doi.org/10.1162/tacl\_a\_00514

    2022 doi

  20. [29]

    G., He, J., Liu, B., and Niu, D

    Lu, S., Mills, K. G., He, J., Liu, B., and Niu, D. Goat: Explaining graph neural networks via graph output attribution. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. URL https://openrevie...

    2024

  21. [30]

    Llm-rec: Personalized recommendation via prompting large language models

    Lyu, H., Jiang, S., Zeng, H., Xia, Y., and Luo, J. Llm-rec: Personalized recommendation via prompting large language models. CoRR, abs/2307.15780, 2023. doi:10.48550/ARXIV.2307.15780. URL https://doi.org/10.48550/arXiv.2307.15780

    2023 doi arXiv

  22. [31]

    Advancing graph representation learning with large language models: A comprehensive survey of techniques, 2024

    Mao, Q., Liu, Z., Liu, C., Li, Z., and Sun, J. Advancing graph representation learning with large language models: A comprehensive survey of techniques, 2024. URL https://arxiv.org/abs/2402.05952

    2024 arXiv

  23. [32]

    Causal abstraction with soft interventions

    Massidda, R., Geiger, A., Icard, T., and Bacciu, D. Causal abstraction with soft interventions. In van der Schaar, M., Zhang, C., and Janzing, D. (eds.), Conference on Causal Learning and Reasoning, CLeaR 2023, 11-14 April 2023, Amazon Development Center, T \" u bingen, German...

    2023

  24. [33]

    Locating and editing factual associations in GPT

    Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in GPT . In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information...

    2022

  25. [34]

    Mustapha, K. B. A survey of emerging applications of large language models for problems in mechanics, product design, and manufacturing. Adv. Eng. Informatics, 64: 0 103066, 2025. doi:10.1016/J.AEI.2024.103066. URL https://doi.org/10.1016/j.aei.2024.103066

    2025

  26. [35]

    Causality: Models, Reasoning, and Inference

    Pearl, J. Causality: Models, Reasoning, and Inference. Cambridge University Press, second edition, 2009

    2009

  27. [36]

    Causality 1988-2001 - introduction

    Pearl, J. Causality 1988-2001 - introduction. In Geffner, H., Dechter, R., and Halpern, J. Y. (eds.), Probabilistic and Causal Inference: The Works of Judea Pearl, volume 36 of ACM Books , pp.\ 215--220. ACM , 2022. doi:10.1145/3501714.3501731. URL https://doi.org/10.1145/3501...

    1988

  28. [37]

    Null it out: Guarding protected attributes by iterative nullspace projection

    Ravfogel, S., Elazar, Y., Gonen, H., Twiton, M., and Goldberg, Y. Null it out: Guarding protected attributes by iterative nullspace projection. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. R. (eds.), Proceedings of the 58th Annual Meeting of the Association for C...

    2020 doi

  29. [38]

    Representation learning with large language models for recommendation

    Ren, X., Wei, W., Xia, L., Su, L., Cheng, S., Wang, J., Yin, D., and Huang, C. Representation learning with large language models for recommendation. In Chua, T., Ngo, C., Kumar, R., Lauw, H. W., and Lee, R. K. (eds.), Proceedings of the ACM on Web Conference 2024, WWW 2024, S...

    2024

  30. [39]

    K., Weichwald, S., Bongers, S., Mooij, J

    Rubenstein, P. K., Weichwald, S., Bongers, S., Mooij, J. M., Janzing, D., Grosse - Wentrup, M., and Sch \" o lkopf, B. Causal consistency of structural equation models. In Elidan, G., Kersting, K., and Ihler, A. (eds.), Proceedings of the Thirty-Third Conference on Uncertainty...

    2017

  31. [41]

    C., Islamaj, R., Kapoor, A., Gao, X., and Lu, Z

    Tian, S., Jin, Q., Yeganova, L., Lai, P.-T., Zhu, Q., Chen, X., Yang, Y., Chen, Q., Kim, W., Comeau, D. C., Islamaj, R., Kapoor, A., Gao, X., and Lu, Z. Opportunities and challenges for chatgpt and large language models in biomedicine and health. Briefings in Bioinformatics, 2...

    2024 doi

  32. [42]

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Canton - Ferrer, C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goy...

    2023 doi arXiv

  33. [43]

    Graph attention networks

    Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Li \` o , P., and Bengio, Y. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . OpenReview.net, ...

    2018

  34. [44]

    Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. M. Investigating gender bias in language models using causal mediation analysis. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Pro...

    2020

  35. [45]

    Llm-enhanced cascaded multi-level learning on temporal heterogeneous graphs

    Wang, F., Zhu, G., Yuan, C., and Huang, Y. Llm-enhanced cascaded multi-level learning on temporal heterogeneous graphs. In Yang, G. H., Wang, H., Han, S., Hauff, C., Zuccon, G., and Zhang, Y. (eds.), Proceedings of the 47th International ACM SIGIR Conference on Research and De...

    2024

  36. [46]

    and Fey, M

    Wittmann, J. and Fey, M. Wiki-cs: A wikipedia-based benchmark for graph neural networks. CoRR, abs/2007.02901, 2020. URL https://arxiv.org/abs/2007.02901

    2007 arXiv

  37. [47]

    Discovering invariant rationales for graph neural networks

    Wu, Y., Wang, X., Zhang, A., He, X., and Chua, T. Discovering invariant rationales for graph neural networks. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022. URL https://openreview.net/forum...

    2022

  38. [48]

    Causal proxy models for concept-based model explanations

    Wu, Z., D'Oosterlinck, K., Geiger, A., Zur, A., and Potts, C. Causal proxy models for concept-based model explanations. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Machine Learning, ICML 2023, 23-29 Ju...

    2023

  39. [49]

    Wu, Z., Geiger, A., Icard, T., Potts, C., and Goodman, N. D. Interpretability at scale: Identifying causal mechanisms in alpaca. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Co...

    2023

  40. [50]

    Qwen2 technical report

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Yang, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M....

    2024 doi arXiv

  41. [51]

    Gnnexplainer: Generating explanations for graph neural networks

    Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. Gnnexplainer: Generating explanations for graph neural networks. In Wallach, H. M., Larochelle, H., Beygelzimer, A., d'Alch \' e - Buc, F., Fox, E. B., and Garnett, R. (eds.), Advances in Neural Information Process...

    2019

  42. [52]

    Empower text-attributed graphs learning with large language models (llms)

    Yu, J., Ren, Y., Gong, C., Tan, J., Li, X., and Zhang, X. Empower text-attributed graphs learning with large language models (llms). CoRR, abs/2310.09872, 2023. doi:10.48550/ARXIV.2310.09872. URL https://doi.org/10.48550/arXiv.2310.09872

    2023 doi arXiv

  43. [53]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.