The Graph Language: How Knowledge Graphs Speak to Large Language Models

Giuseppe Pirr\`o

REVIEW 4 major objections 6 minor 48 references

The Graph Language: How Knowledge Graphs Speak to Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GRALAN encodes question-focused knowledge-graph subgraphs as relational tokens in a frozen LLM's embedding space and casts question answering as entity classification, outperforming prior KG-LLM methods especially on multi-hop reasoning.

desk verdict GRALAN is a plausible new combination for injecting KG structure into frozen LLMs, but the open-QA numbers may be inflated by answer-node leakage in the subgraph extractor. read the letter →

arxiv 2608.01175 v1 pith:HE4ZORBC submitted 2026-08-02 cs.AI

Giuseppe Pirrò This is my paper

classification cs.AI

keywords knowledgegraphslargelanguagemodelsquestionansweringmulti-hopreasoninggraphneuralnetworksrelation-awareattentionentityclassificationparameter-efficientintegration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a knowledge graph can be made to communicate directly in a large language model's semantic space, without flattening the graph into text and without fine-tuning the LLM. Its framework, GRALAN, extracts a small question-focused subgraph, encodes it with a relational graph neural network, and feeds the result to a frozen LLM as learned relational tokens; the answer is chosen by classifying among entities in the subgraph. The reported experiments show consistent gains over previous KG-LLM approaches, with the largest margins on multi-hop questions. If the claim holds, it offers a parameter-efficient route to grounded, structured reasoning that avoids the vocabulary limits of text generation.

What carries the argument

The Graph Language Mediator is the load-bearing component: it translates a question-focused subgraph into relational tokens in the LLM's embedding space. It combines an RGCN graph encoder for entity and relation embeddings, a relation-aware cross-attention step that aligns graph nodes with the question, and a sequence assembler that concatenates global and top-K node embeddings with delimiter tokens, so a frozen LLM can reason over graph structure directly.

What would settle it

For open QA, compute the recall of the question-focused BFS: does the correct answer entity appear in the extracted subgraph for every test question? If recall is not near 100%, Hits@1 accuracy is not end-to-end. A second check is to ablate the relation-similarity bias of the BFS: if random BFS gives about the same accuracy, the mediator's alignment step is not the source of the gain.

Watch

Extended reading notes

Core claim

GRALAN's core claim is that the gap between knowledge graphs and LLMs is best closed by semantic alignment, not structural transformation. Given a question, a question-focused BFS extracts a small subgraph around seeded entities, biasing traversal toward relations similar to the question. A Graph Language Mediator encodes this subgraph with RGCN layers, aligns node and relation embeddings with the question embedding through relation-aware cross-attention, and assembles a structured input sequence delimited by [GRAPH], [NODES], [QUESTION], and [ANSWER] tokens. A frozen LLM processes this sequence, and a classifier reads the hidden state at [ANSWER] to predict the answer entity. The paper repo

Load-bearing premise

The open-QA pipeline depends on the correct answer entity being inside the question-focused subgraph built without using the answer, and the paper does not report how often that happens.

Editorial extensions

If this is right

KG question answering can be done without LLM fine-tuning, cutting compute and avoiding catastrophic forgetting.
Multi-hop questions over large graphs can be answered from a small subgraph plus a frozen LLM, matching or beating methods that iteratively explore the graph with a much larger LLM.
Answering by classification over subgraph entities removes the LLM vocabulary constraint and scales linearly with graph size.
The relation-aware cross-attention and ablation results point to subgraph selection quality as the main driver of performance, not sheer model size.
The same frozen-LLM-plus-mediator design could be reused for other knowledge-intensive tasks by changing the task tokens and the classifier head.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The framework's logic makes subgraph extraction the effective ceiling: any question whose answer entity is not in the extracted subgraph is guaranteed to fail, so porting GRALAN to a new domain means solving question analysis and relation-similarity first.
The same mediator template—encode a structured object into learned tokens in a frozen LLM's embedding space—should extend beyond graphs to tables, code, or program traces, though each new structure would need its own encoder and token scheme.
Because the multiple-choice experiments re-ground the subgraph on all answer nodes, those numbers measure ranking among known candidates; a reader should not expect the same accuracy when the answer must be retrieved from a large open KG.
A minimal test of the alignment hypothesis would be to replace the relation-similarity-biased BFS with a random BFS while keeping the mediator trained: if accuracy holds, the LLM is doing the reasoning; if it collapses, the graph traversal is the source of the gain.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, here's my read on arXiv:2608.01175.

The architecture is genuinely a new mix: question-focused BFS with relation-similarity bias, GNN encoder, cross-attention mediator, and entity classification over a subgraph, all feeding a frozen LLM. That's a reasonable design, and the ablations (threshold, BFS depth, loss balance, LLM choice) are useful. The related work is thorough, and the baselines are standard. If the numbers are clean, this is a useful parameter-efficient alternative to fine-tuning for KG-LLM QA.

But there's one load-bearing ambiguity. In §4, after describing datasets, the paper says: "as done in [44] we use all answer nodes and question nodes to extract a single subgraph (re-grounded) as the knowledge source for the LLM." If "answer nodes" means gold entities for open QA, then the exact answer is placed in the subgraph before the classifier (Eq. 14–15) ranks entities. Since the model can only output entities in that subgraph, the reported Hits@1 numbers—especially 86.6% on PQL-3h, where the test set has only 103 questions—could come from subgraph construction, not from learned reasoning. The paper never reports subgraph recall, so there is no way to separate those explanations. For multiple-choice, using answer choices is legitimate because they are visible; the problem is that the sentence doesn't say it's limited to that. This has to be fixed before I'd trust the central multi-hop claim.

Other soft spots are minor by comparison. Eq. (5) looks dimensionally off—the numerator H_i R_r H_j^T is scalar, so multiplying by E_Q^T doesn't give an attention matrix as written. The abstract says "significantly outperforms," but there are no error bars or significance tests, and the standard deviation is omitted even though the runs are averaged over 10 trials. I also couldn't verify the GitHub code from the preprint, so reproducibility is currently a promise, not a fact.

The TF-IDF relation relatedness matrix from [23] is self-cited, but it's a preprocessing input, not a fitted parameter, so I don't see a circularity problem.

Bottom line: this deserves serious review, but the authors need to clarify answer-node usage, report subgraph recall, fix Eq. (5), and release the code. If the open-QA protocol is actually answer-independent, I'd expect this to be a solid contribution to the KG-LLM literature. As written, I'd put it conditional: promising architecture, unverified headline numbers.

Referee Report

4 major / 6 minor

Summary. The paper introduces GRALAN, a framework that integrates knowledge graphs (KGs) with frozen large language models (LLMs) by learning a graph language mediator. The mediator encodes question-focused subgraphs with a GNN, projects the resulting representations into the LLM's embedding space via relational tokens and cross-attention, and provides a structured input sequence to a frozen LLM. Question answering is cast as entity classification over the subgraph. The authors report results on open-domain multi-hop QA benchmarks (PathQuestion, PathQuestion-Large, MetaQA, ZeroShotRE, ComplexWebQuestions) and multiple-choice QA benchmarks (OBQA, Riddle, ARC, PIQA), claiming state-of-the-art or competitive accuracy, particularly on multi-hop reasoning, and provide ablations over subgraph threshold, BFS depth, prediction strategy, loss weighting, and LLM choice.

Significance. If the empirical claims hold, GRALAN would be a meaningful contribution to KG-LLM integration: it keeps the LLM frozen, is parameter-efficient, and—unlike token-flattening or prompt-based approaches—preserves graph structure through GNN encoding and learned relational tokens. The paper ships code, reports detailed hyperparameters, and includes a broad ablation study. The reported gains on PQL-3h, MetaQA-3hop, and ComplexWebQuestions would demonstrate that graph-structured representations can be aligned with LLM semantics more effectively than iterative exploration or serialization. However, the significance is currently conditional on resolving a load-bearing evaluation ambiguity: whether ground-truth answer entities are used to construct the open-QA subgraphs. The strengths of the paper are its architectural clarity and breadth of experiments; the main weakness is the lack of a clearly answer-independent open-QA evaluation and the omission of statistical support for the headline significance claims.

major comments (4)

[Section 4, Datasets paragraph] The sentence 'as done in [44] we use all answer nodes and question nodes to extract a single subgraph (re-grounded) as the knowledge source for the LLM' is ambiguous and potentially fatal for the open-QA results. If 'answer nodes' include ground-truth entities for open-QA datasets (PQ, PQL, MetaQA, ZeroShotRE, ComplexWebQuestions), then the gold answer is inserted into G_sub before the classifier of Eqs. (14)-(15) ranks candidates. Since the model can only output an entity present in G_sub, the reported Hits@1 numbers in Table 2 (e.g., 90.75 on PQL-2h, 86.6 on PQL-3h, 100.0 on MetaQA 3-hop, 70.23 on CWQ) would reflect subgraph construction rather than GRALAN's learned reasoning. Please clarify whether this sentence applies only to multiple-choice QA (where answer choices are visible by design) or also to open QA. For open QA, the paper must specify that gold entities are not used, and sh
[§3.2, Eq. (5)] Equation (5) is dimensionally inconsistent as written: H_i R_r H_j^T produces a scalar (assuming H_i and H_j are row vectors of dimension d), and multiplying that scalar by E_Q^T (dimension d_lm × M, where M is the number of question tokens) is undefined. This equation is central to the claimed relation-aware cross-attention mechanism, and the inconsistency prevents reproduction. Please provide the correct formula, including the intended shapes and how the relation embedding R_r participates; for example, whether the scalar should be broadcast or whether the attention logit should involve a projection of E_Q.
[§3.4, Eqs. (14)–(15)] There is a mismatch between the classifier definition and the stated candidate space. Eq. (14) defines W_cls ∈ R^{|V|×d_LLM}, projecting onto the full KG entity space, and Eq. (15) selects the argmax over V. However, the text repeatedly says QA is recast as 'predicting which entity in the subgraph correctly answers the question,' and Eq. (16) restricts the multi-answer case to V_sub. If training and inference are restricted to subgraph entities, the classifier should be defined over V_sub (or the logits for entities outside G_sub should be masked). If instead the classifier scores all of V, then the 'scalability through decoupling from LLM vocabulary' claim is unclear. Please clarify the exact training and inference candidate space and align the equations with the text.
[§4, Experimental setting and Abstract] The abstract claims that GRALAN 'significantly outperforms' existing methods, but the experimental section reports only point estimates. The paper states that results are 'the average of 10 runs where the relatively low standard deviation is omitted,' yet no standard deviations, confidence intervals, or significance tests are provided. This is especially important for the small test sets: PQL-3h has 103 test questions, so the reported 3.3-point improvement over G-Retriever (86.6 vs. 83.3) corresponds to roughly 3–4 questions; similarly, the CWQ advantage over ToG (70.23 vs. 67.6) is 2.6 points. Please report variance across runs and perform appropriate significance tests (e.g., paired bootstrap or McNemar's test) for the key comparisons in Tables 2 and 3. Without this, the word 'significantly' is not justified.

minor comments (6)

[§4, Datasets] The dataset referred to as 'OBDA' in Table 4 appears to be OpenBookQA (often abbreviated OBQA); the text uses both 'OBQA' and 'OBDA'. Please standardize the notation.
[§1.1] The LLM is called 'Flat-T5' in the opening example but 'FLAN-T5-XXL' elsewhere. This is likely a typo; please unify the name.
[Figure 1] The caption contains 'positon held' (typo for 'position held'). Also, the figure is dense; please ensure all abbreviations (RGCN/RGAT, H_G, etc.) are defined in the caption or text.
[§2, Related Work] Reference [24], cited for SRN, is titled 'Structured attention for unsupervised dialogue structure induction,' which appears unrelated to the SRN multi-hop reasoning baseline used in Table 2. Please verify and correct the reference.
[§3.1] The similarity function sim(·,·) is said to be definable via co-occurrence statistics or relation embeddings, and the TF-IDF relation relatedness matrix is precomputed. Please state whether this matrix is fixed or updated during training; if it is derived from the author's prior work [23], clarify whether it introduces any external supervision on the test datasets.
[§3.4, Eq. (16)] The multi-answer threshold τ in Eq. (16) is not included in the hyperparameter list or ablation. Please specify how τ is chosen for datasets with multiple answers, if any.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: GRALAN's central claims are empirical and benchmarked against external published baselines; the only self-citation is a preprocessing relatedness matrix that does not constrain the evaluated predictions.

full rationale

The paper's load-bearing claim is empirical: GRALAN outperforms existing KG-LLM methods on open and multiple-choice QA benchmarks. This is supported by comparisons to external published baselines on standard dataset splits, not by a derivation whose output is built into its input. The only self-citation is reference [23] (the author's earlier Reword work), used to precompute a TF-IDF-based relation relatedness matrix for the question-focused BFS. That matrix is a preprocessing input, not a fitted parameter tied to the benchmark targets, and the cited work is an independent, externally published technique; it does not smuggle in the present paper's conclusions. The abstract and Section 1.2 innovations are architectural descriptions rather than derivations from the evaluation numbers. The potentially concerning sentence in Section 4 about using 'all answer nodes and question nodes to extract a single subgraph' is an evaluation-validity risk rather than a circularity step: the paper's single-answer prediction in Eq. (14)-(15) is written as an argmax over the full entity set V with W_cls of size |V|, so including gold answer nodes in the subgraph does not, by the paper's own equations, force the predicted entity to equal an input answer. There is also no subgraph-recall reporting, which is a separate correctness concern, but circularity analysis requires a demonstrated reduction Eq. X = Eq. Y by construction, and no such reduction appears here. Overall, the derivation chain is self-contained and externally anchored; the score is 0.

Assumptions & free parameters 7 free parameters · 4 assumptions · 1 invented entities

The central empirical claim rests on several tunable hyperparameters (θ, BFS depth, top-K, loss weights, GNN dimensions) and on unverified assumptions that the frozen LLM can consume continuous inputs, that subgraph extraction retains answers without leaking them, and that GNN embeddings carry enough semantics. Standard attention math is assumed. No new physical entities are introduced; "relational tokens" are learned vectors with no independent evidence.

free parameters (7)

subgraph relevance threshold θ = ~0.7 optimal
Controls edge expansion priority in question-focused BFS; tuned per dataset on validation set (Section 3.1, Section 4.3).
BFS exploration depth = 3
Number of hops in subgraph extraction; chosen after ablation, performance degrades at depth 4 (Section 4.3).
top-K selected nodes = 5
Number of entity embeddings inserted into the LLM input sequence; fixed design choice (Section 3.2, Experimental setting).
loss weight λ_cls = 0.5-0.7 optimal
Balances classification and LLM losses; tuned on validation set (Section 3.4, Section 4.3).
regularization strengths α and β = not reported
Appear in L_total; values not given in the paper, only described as hyperparameters (Section 3.4).
GNN architecture hyperparameters = 2 layers, 5 attention heads, hidden 200, edge dim 64, dropout 0.1, LR 3e-5, batch 32, 100 epochs
Chosen by validation; affects embeddings fed to the frozen LLM (Section 4, Experimental setting).
TF-IDF relation relatedness matrix = precomputed from KG via [23]
Used to prioritize edges in question-focused BFS; a modeling choice that shapes all subgraphs, though not fitted to the QA objective (Section 3.1).

assumptions (4)

domain assumption A frozen pretrained LLM can reason over continuous input vectors that bypass token embeddings and still produce useful hidden states.
The X_LLM construction in Eq. (11) feeds arbitrary learned vectors into a transformer whose weights were trained on discrete token embeddings (Section 3.2-3.3).
domain assumption Question-focused BFS with TF-IDF relation relatedness produces subgraphs that contain the correct answer entity.
The classifier in Eq. (14) can only pick entities in V_sub; the paper does not report subgraph recall, so answer containment is assumed (Section 3.1, Eq. 14).
domain assumption RGCN message passing on the subgraph yields node and relation embeddings with sufficient semantics for QA.
Graph encoder outputs feed the cross-attention and the LLM input (Section 3.2, Eq. 1).
standard math Standard attention and cross-entropy objectives are appropriate for training the mediator while the LLM stays frozen.
Equations (2)-(14) rely on softmax attention, projection matrices, and cross-entropy/KL losses without proof (Section 3).

invented entities (1)

Relational tokens
purpose: Learned continuous vectors that encode node, relation, and task-specific information and are inserted into the frozen LLM's input sequence.
They are internal learned representations; the paper provides no external falsifiable prediction about their content, only downstream QA accuracy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Graph Language: How Knowledge Graphs Speak to Large Language Models." pith.science (2026). https://pith.science/paper/HE4ZORBC

@misc{pith2026260801175,
  author       = {Pith},
  title        = {Pith review of: The Graph Language: How Knowledge Graphs Speak to Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HE4ZORBC}},
  note         = {Machine review of arXiv:2608.01175}
}

read the original abstract

Large Language Models (LLMs) excel at reasoning but benefit from grounding provided by Knowledge Graphs (KGs). However, integrating these paradigms is challenging. We introduce GRALAN, which enables KGs to speak directly in the LLM's semantic space through relational tokens that preserve graph structure. GRALAN-s trainable language mediator generates structured tokens for any frozen LLM, creating a foundation for knowledge-intensive applications. We demonstrate its effectiveness in question-answering by re-framing the task as entity classification over question-focused subgraphs. Experiments show that GRALAN significantly outperforms existing methods, particularly on complex multi-hop reasoning tasks, establishing a new paradigm for KG-LLM integration that maintains structural fidelity while leveraging LLMs' reasoning capabilities.

Figures

Figures reproduced from arXiv: 2608.01175 by the authors.

**Figure 1.** GRALAN’s end-to-end architecture for graph-language reasoning. Given a natural language question (1), GRALAN extracts a question-relevant subgraph (2) and processes it through RGCN/RGAT layers to obtain node (H), relation (R) and global context (HG) embeddings (3). The question and graph representations are aligned in a shared semantic space through the Graph Language Mediator (4-6), which identifies the most relat… view at source ↗

**Figure 2.** Ablation studies. Top row: PathQuestion (PQ), middle row: MetaQA (MQ), and Bottom [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 38 canonical work pages

[44]

In: Proc

Zhang, Y ., Chen, K., Bai, X., Kang, Z., Guo, Q., Zhang, M.: Question-guided knowledge graph re-scoring and injection for knowledge graph question answering. In: Proc. of EMNLP. pp. 8972–8985 (2024)

work page 2024
[23]

In: Proc

Pirrò, G.: Reword: Semantic relatedness in the web of data. In: Proc. of AAAI. pp. 129–135 (2012). https://doi.org/10.1609/AAAI.V26I1.8107

work page doi:10.1609/aaai.v26i1.8107 2012
[1]

In: Proc

Baek, J., Aji, A.F., Saffari, A.: Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. In: Proc. of NLRSE. pp. 78–106 (2023). https://doi.org/10.18653/v1/2023.nlrse-1.7

doi:10.18653/v1/2023.nlrse-1.7 2023
[2]

In: Proc

Bang, Y ., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, H.V ., Xu, Y ., Fung, P.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. In: Proc. of AACL-IJCNLP. pp. 675– 718 (2023). arXiv preprint arXiv:2302.04023

arXiv 2023
[3]

In: Proc

Bisk, Y ., Zellers, R., Le Bras, R., Gao, J., Choi, Y .: Piqa: Reasoning about physical common- sense in natural language. In: Proc. of AAAI. pp. 7432–7439 (2020)

work page 2020
[4]

In: Proc

Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively cre- ated graph database for structuring human knowledge. In: Proc. of SIGMOD. pp. 1247–1250 (2008)

work page 2008
[5]

In: Proc

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. In: Proc. of NeurIPS. pp. 1877–1901 (2020)

work page 1901
[6]

arXiv preprint arXiv:2310.05845 (2023)

Chai, Z., Zhang, T., Wu, L., Han, K., Hu, X., Huang, X., Yang, Y .: Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845 (2023)

arXiv 2023

Show all 48 references

[7]

arXiv preprint arXiv:2304.03516 (2023)

Chen, Y .T., Ku, L.W.: Exploring the limits of chatgpt for query or aspect-based text summa- rization. arXiv preprint arXiv:2304.03516 (2023)

2023 arXiv
[8]

arXiv preprint arXiv:1803.05457 (2018)

Clark, P., Cowhey, I., Etzioni, O., Khashabi, D., Tafjord, O., Sabharwal, A., Kembhavi, A., Cheng, B., Mishra, B.D., Richardson, M., et al.: Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018)

2018 arXiv
[9]

In: Proc

Das, R., Dhuliawala, S., Zaheer, M., Vilnis, L., Durugkar, I., Krishnamurthy, A., Smola, A., McCallum, A.: Go for a walk and arrive at the answer: Reasoning over paths in knowledge bases using reinforcement learning. In: Proc. of ICLR (2018)

2018
[10]

In: Proc

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of NAACL-HLT. pp. 4171–4186 (2019)

2019
[11]

In: Proc

Fatemi, B., Halcrow, J., Perozzi, B.: Talk like a graph: Encoding graphs for large language models. In: Proc. of ICLR (2024)

2024
[12]

In: Proc

He, G., Lan, Y ., Jiang, J., Zhao, W.X., Wen, J.R.: Improving multi-hop knowledge base question answering by learning intermediate supervision signals. In: Proc. of WSDM. pp. 553–561 (2021)

2021
[13]

In: Proc

He, X., Tian, Y ., Sun, Y ., Chawla, N.V ., Laurent, T., LeCun, Y ., Bresson, X., Hooi, B.: G- retriever: Retrieval-augmented generation for textual graph understanding and question an- swering. In: Proc. of NeurIPS. pp. 132876–132907 (2024)

2024
[14]

In: Proc

Hu, E.J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: Proc. of ICLR (2022) 5 https://github.com/giuseppepirro/gralan The Graph Language 17

2022
[15]

ACM Computing Surveys55(12), 1–38 (2023)

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y ., Ishii, E., Bang, Y ., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM Computing Surveys55(12), 1–38 (2023)

2023
[16]

In: Proc

Jiang, J., Zhou, K., Zhao, X., Wen, J.R.: UniKGQA: Unified retrieval and reasoning for solving multi-hop question answering over knowledge graph. In: Proc. of ICLR (2023)

2023
[17]

In: Proc

Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Proc. of EMNLP. pp. 3045–3059 (2021)

2021
[18]

In: Proc

Lin, B.Y ., Chen, Z., Lee, C.H., Ren, X.: Riddlesense: Reasoning about riddle questions featuring linguistic creativity and commonsense knowledge. In: Proc. of ACL-IJCNLP. pp. 1855–1865 (2021)

2021
[19]

In: Proc

Liu, C., Zhan, Y ., Wu, J., Li, C., Du, B., Hu, W., Liu, T., Tao, D.: Graph pooling for graph neural networks: Progress, challenges, and opportunities. In: Proc. of IJCAI. pp. 6712–6722 (2023). https://doi.org/10.24963/IJCAI.2023/752

2023 doi
[20]

In: Proc

Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.: Can a suit of armor conduct electricity? a new dataset for open book question answering. In: Proc. of EMNLP. pp. 2381–2391 (2018)

2018
[21]

In: Proc

Miller, A., Fisch, A., Dodge, J., Karimi, A.H., Bordes, A., Weston, J.: Key-value memory networks for directly reading documents. In: Proc. of EMNLP. pp. 1400–1409 (2016)

2016
[22]

In: Proc

Petroni, F., Piktus, A., Fan, A., Lewis, P., Yazdani, M., Wolf, L., Grave, E., Riedel, S., Kiela, D.: KILT: a benchmark for knowledge intensive language tasks. In: Proc. of NAACL-HLT. pp. 2523–2544 (2021)

2021
[24]

In: Proc

Qiu, L., Zhao, Y ., Shi, W., Liang, Y ., Shi, F., Yuan, T., Zhou, J., Zhang, M.: Structured attention for unsupervised dialogue structure induction. In: Proc. of EMNLP. pp. 1889–1899 (2020). https://doi.org/10.18653/v1/2020.emnlp-main.148

2020 doi
[25]

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y ., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. of Machine Learning Research21(140), 1–67 (2020)

2020
[26]

In: Proc

Schlichtkrull, M., Kipf, T.N., Bloem, P., Van Den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Proc. of ESWC. pp. 593–607. Springer (2018)

2018
[27]

arXiv preprint arXiv:2310.08487 (2023)

Shen, Y ., Liao, R., Han, Z., Ma, Y ., Tresp, V .: GraphextQA: A benchmark for evaluating graph-enhanced large language models. arXiv preprint arXiv:2310.08487 (2023)

2023 arXiv
[28]

In: Proc

Shi, J., Cao, S., Hou, L., Li, J., Zhang, H.: TransferNet: An effective and transparent frame- work for multi-hop question answering over relation graph. In: Proc. of EMNLP. pp. 4149– 4158 (2021). https://doi.org/10.18653/v1/2021.emnlp-main.341

2021 doi
[29]

In: Proc

Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: An open multilingual graph of general knowledge. In: Proc. of AAAI. pp. 4444–4451 (2017). https://doi.org/10.1609/AAAI.V31I1.11164

2017 doi
[30]

In: Proc

Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y ., Ni, L., Shum, H.Y ., Guo, J.: Think- on-graph: Deep and responsible reasoning of large language model on knowledge graph. In: Proc. of ICLR (2024)

2024
[31]

arXiv preprint arXiv:2107.02137 (2021)

Sun, Y ., Wang, S., Feng, S., Ding, S., Pang, C., Shang, J., Liu, J., Chen, X., Zhao, Y ., Lu, Y ., et al.: ERNIE 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137 (2021)

2021 arXiv
[32]

of EMNLP

Sung, M., Lee, J., Yi, S., Jeon, M., Kim, S., Kang, J.: Can language models be biomedical knowledge bases? In: Proc. of EMNLP. pp. 7163–7180 (2021)

2021
[33]

In: Proc

Talmor, A., Berant, J.: The web as a knowledge-base for answering complex questions. In: Proc. of NAACL-HLT. pp. 641–651 (2018) 18 G. Pirrò

2018
[34]

In: Proc

Tang, J., Yang, Y ., Wei, W., Shi, L., Su, L., Cheng, S., Yin, D., Huang, C.: GraphGPT: Graph instruction tuning for large language models. In: Proc. of SIGIR. pp. 491–500 (2024). https://doi.org/10.1145/3626772.3657775

2024
[35]

In: Proc

Tang, J., Yang, Y ., Wei, W., Shi, L., Xia, L., Yin, D., Huang, C.: HiGPT: Heterogeneous graph language model. In: Proc. of ACM SIGKDD Conference on Knowledge Discovery and Data Mining. pp. 2842–2853 (2024)

2024
[36]

In: Proc

Tian, Y ., Song, H., Wang, Z., Wang, H., Hu, Z., Wang, F., Chawla, N.V ., Xu, P.: Graph neural prompting with large language models. In: Proc. of AAAI (2024)

2024
[37]

IEEE Transactions on Knowledge and Data Engineering29(12), 2724– 2743 (2017)

Wang, Q., Mao, Z., Wang, B., Guo, L.: Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering29(12), 2724– 2743 (2017)

2017
[38]

In: Proc

Wang, R., Rossetto, L., Cochez, M., Bernstein, A.: QAGCN: Answering multi-relation ques- tions via single-step implicit reasoning over knowledge graphs. In: Proc. of ESWC. pp. 343–
[39]

Transactions of the As- sociation for Computational Linguistics9, 176–194 (2021)

Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., Tang, J.: KEPLER: A unified model for knowledge embedding and pre-trained language representation. Transactions of the As- sociation for Computational Linguistics9, 176–194 (2021)

2021
[40]

In: Proc

Yasunaga, M., Leskovec, J., Liang, P.: LinkBERT: Pretraining language models with docu- ment links. In: Proc. of ACL. pp. 8003–8016 (2022)

2022
[41]

In: Proc

Ye, R., Zhang, C., Wang, R., Xu, S., Zhang, Y .: Natural language is all a graph needs. In: Proc. of EACL. pp. 1716–1729 (2024)

2024
[42]

In: Proc

Yu, D., Zhu, C., Yang, Y ., Zeng, M.: JAKET: Joint pre-training of knowledge graph and language understanding. In: Proc. of AAAI. pp. 11630–11638 (2022)

2022
[43]

In: Proc

Zhang, X., Bosselut, A., Yasunaga, M., Ren, H., Liang, P., Manning, C.D., Leskovec, J.: GreaseLM: Graph reasoning enhanced language models for question answering. In: Proc. of ICLR (2022)

2022
[45]

In: Proc

Zhang, Y ., Dai, H., Kozareva, Z., Smola, A.J., Song, L.: Variational reasoning for question answering with knowledge graph. In: Proc. of AAAI. pp. 6069–6076 (2018)

2018
[46]

arXiv preprint arXiv:2303.18223 (2023)

Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y ., Min, Y ., Zhang, B., Zhang, J., Dong, Z., Du, C., Yang, C., Chen, Y ., Chen, Z., Jiang, J., Ren, R., Li, Y ., Tang, X., Liu, Z., Liu, P., Nie, J.Y ., Wen, J.R.: A survey of large language models. arXiv preprint arXiv:...

2023 arXiv
[47]

In: Proc

Zhao, Z., Wallace, E., Feng, S., Klein, D., Singh, S.: Calibrate before use: Improving few- shot performance of language models. In: Proc. of ICML. pp. 12697–12706 (2021)

2021
[48]

In: Proc

Zhou, M., Huang, M., Zhu, X.: An interpretable reasoning network for multi-relation ques- tion answering. In: Proc. of COLING. pp. 2010–2022 (2018)

2010

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [44]

In: Proc

Zhang, Y ., Chen, K., Bai, X., Kang, Z., Guo, Q., Zhang, M.: Question-guided knowledge graph re-scoring and injection for knowledge graph question answering. In: Proc. of EMNLP. pp. 8972–8985 (2024)

work page 2024

[2] [23]

In: Proc

Pirrò, G.: Reword: Semantic relatedness in the web of data. In: Proc. of AAAI. pp. 129–135 (2012). https://doi.org/10.1609/AAAI.V26I1.8107

work page doi:10.1609/aaai.v26i1.8107 2012

[3] [1]

In: Proc

Baek, J., Aji, A.F., Saffari, A.: Knowledge-augmented language model prompting for zero-shot knowledge graph question answering. In: Proc. of NLRSE. pp. 78–106 (2023). https://doi.org/10.18653/v1/2023.nlrse-1.7

doi:10.18653/v1/2023.nlrse-1.7 2023

[4] [2]

In: Proc

Bang, Y ., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., Do, H.V ., Xu, Y ., Fung, P.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. In: Proc. of AACL-IJCNLP. pp. 675– 718 (2023). arXiv preprint arXiv:2302.04023

arXiv 2023

[5] [3]

In: Proc

Bisk, Y ., Zellers, R., Le Bras, R., Gao, J., Choi, Y .: Piqa: Reasoning about physical common- sense in natural language. In: Proc. of AAAI. pp. 7432–7439 (2020)

work page 2020

[6] [4]

In: Proc

Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively cre- ated graph database for structuring human knowledge. In: Proc. of SIGMOD. pp. 1247–1250 (2008)

work page 2008

[7] [5]

In: Proc

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. In: Proc. of NeurIPS. pp. 1877–1901 (2020)

work page 1901

[8] [6]

arXiv preprint arXiv:2310.05845 (2023)

Chai, Z., Zhang, T., Wu, L., Han, K., Hu, X., Huang, X., Yang, Y .: Graphllm: Boosting graph reasoning ability of large language model. arXiv preprint arXiv:2310.05845 (2023)

arXiv 2023