Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Walk&Retrieve: Simple Yet Effective Zero-shot Retrieval-Augmented Generation via Knowledge Graph Walks

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Walk&Retrieve, a zero-shot framework that converts knowledge-graph walks into natural-language context, outperforms trained KG-based retrieval-augmented generation systems on standard QA benchmarks while using a single LLM call per query.

desk verdict A clean zero-shot KG-RAG baseline with credible MetaQA numbers, but the outgoing-only walk traversal caps the generality claim. read the letter →

arxiv 2505.16849 v2 pith:T5XM3O4S submitted 2025-05-22 cs.IR cs.CL

Martin Böckling , Heiko Paulheim , Andreea Iana This is my paper · ORCID
classification cs.IRcs.CL
keywords knowledgegraphRAGwalkszero-shotretrievalretrieval-augmentedgenerationquestionansweringrandomBFShallucinationreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that retrieval-augmented generation over knowledge graphs does not need fine-tuned retrievers or multi-step LLM planning. It introduces Walk&Retrieve, a zero-shot pipeline that pre-computes, for each entity, a set of graph walks (random or breadth-first), verbalizes those walks into natural-language sentences with an off-the-shelf LLM, and indexes them. At query time, it embeds the question, retrieves the most similar entities and their most similar walks, and asks the same LLM to answer from that context in one call. On MetaQA, the BFS variant reaches 67.99% Hits@1 versus 43.88% for a trained subgraph-retrieval baseline, and it also reduces hallucination and missing-response rates while running faster than other RAG systems. The authors present walk-based corpus generation as a strong, scalable zero-shot baseline for KG-based RAG.

What carries the argument

The load-bearing mechanism is the walk-based corpus: for every vertex $v$ in the knowledge graph $G=(V,E,R)$, the method generates $n_w$ random walks of length $l$ (or a BFS spanning-tree partition into layers $L_i$ by shortest-path distance), verbalizes each walk into a natural-language sentence by prompting an LLM, and indexes both node-level and walk-level embeddings. At inference, the query is embedded with the same LLM, the top-$k$ nodes and their top-$k$ walks are selected by cosine similarity, and the answer is generated in a single LLM call with a prompt that instructs the model to say 'I do not know the answer' if the context is insufficient. The BFS variant avoids duplicate walks and explores systematically, which the paper attributes to its consistently higher truthfulness than the random-walk variant.

What would settle it

Construct or use a QA benchmark over a knowledge graph that has been canonicalized so that the correct answer entity appears as the subject (not the object) of the triple connecting it to the query entity. Walk&Retrieve's traversal, which follows only outgoing edges $(v,r,v')$, should miss those triples and its Hits@1 should collapse; if it does not, then the method must be obtaining the facts from some source other than the walk corpus (for example, from the LLM's parametric memory).

Watch

Extended reading notes

Core claim

The central claim is that a simple, fully zero-shot retrieval strategy — pre-computing bounded graph walks from every entity, verbalizing them into sentences, and retrieving the top-k most similar walks by embedding cosine similarity — is competitive with, and often better than, KG-RAG systems that fine-tune retrievers or generators. Concretely, Walk&Retrieve-BFS outperforms all baselines on MetaQA in answer accuracy and Hits@1, achieving 67.99% Hits@1 against 43.88% for SubgraphRAG, and yields the lowest hallucination and missing-response rates among KG-based RAG methods. The paper also reports that the method is robust across backbone LLMs (Llama-3.1, Mixtral, GPT-4o), scales to a larger KG (CRAG) where trained baselines cannot run, and adapts to dynamic KGs by regenerating only the walks of changed elements. The intended upshot is that the complicated machinery of learned graph retrievers and multi-step LLM reasoning may be unnecessary for KG-grounded QA.

Load-bearing premise

The method assumes that every fact needed to answer a question is reachable by walking only along outgoing edges from the query entity, so if the knowledge graph stores a fact in reverse direction (the answer is the subject of the triple), the walk will never retrieve it even though the fact exists.

Editorial extensions

If this is right

  • KG-RAG can be a lightweight, zero-shot service: no retraining when the KG changes, only incremental walk recomputation for affected entities.
  • A single LLM call per query cuts inference latency compared with systems that interleave LLM calls with graph traversal steps.
  • Any off-the-shelf LLM can be used as the backbone, so improvements in LLM quality transfer directly to KGQA performance.
  • Walk-based retrieval provides a strong and simple baseline that future KG-RAG methods should be measured against.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the empirical edge holds, a larger implication is that the corpus itself — not the retriever — is the limiting factor for KG-RAG quality, so research should focus on what to extract and how to verbalize it rather than on learned retrieval architectures.
  • The reliance on outgoing edges suggests that canonicalization of KG directionality could significantly affect results; augmenting walks with incoming edges (bidirectional traversal) might be a cheap, testable improvement that the paper does not explore.
  • The verbalization step's quality likely determines the ceiling: an LLM that verbalizes walks more coherently (e.g., with entity type information or relation paraphrases) may push accuracy higher without changing traversal.
  • The method's reported latency advantage should be re-examined on KGs where the pre-computation cost (BFS over the whole graph) dominates; for very large dynamic KGs, random walks with deduplication may behave differently than the paper's fixed setting.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Walk&Retrieve, a zero-shot KG-based RAG framework that builds an offline corpus by walking the knowledge graph from each entity (random walks or BFS walks), verbalizing the walks with an LLM into natural-language sentences, and embedding the resulting text. At inference, it embeds the query, retrieves the k most similar nodes and their k most relevant verbalized walks by cosine similarity, and prompts an off-the-shelf LLM with the query plus that context, using a single LLM call. The main experiments on MetaQA (1-, 2-, and 3-hop) and CRAG compare against LLM-only, Vanilla RAG, SubgraphRAG, and RetrieveRewriteAnswer, reporting Hits@1, accuracy, hallucination, missing, and latency. The authors report that Walk&Retrieve-BFS outperforms all baselines on MetaQA, has lower hallucination and missing rates, and is competitive on CRAG, with ablations over walk depth, number of walks, and backbone LLMs.

Significance. If the MetaQA numbers are reliable, the paper makes a useful and falsifiable point: a simple, training-free, walk-based retriever with LLM verbalization can outperform trained KG-RAG baselines on a standard benchmark while using one LLM call per query. The strengths of the submission include released code, the use of official implementations for the two strong KG-RAG baselines, and ablations across walk depth and backbone LLMs. The main limitations—outgoing-only traversal, no described validation split, and incomplete KG-RAG baselines on CRAG—do not invalidate the MetaQA comparison but materially restrict the generality claims as currently written.

major comments (3)
  1. [§2.1, Eqs. (1)-(2)] The traverse definitions use only outgoing edges: the neighbor set N(v) is defined by triples (v,r,v') in E, Eq. (1) gives nonzero probability only to outgoing edges, and Eq. (2) builds BFS layers from outgoing edges. The paper never adds inverse relations or discusses edge orientation. On MetaQA, the question entity is the head of the relevant triples, so answer entities are reachable by outward walks and the hidden assumption is satisfied. On a canonicalized KG that stores each fact in a single direction (e.g., Freebase/WebQSP), many answers are the subjects rather than the objects of the query entity's triples, so a walk rooted at the query entity would never encounter them even though the fact is present in the graph; the method would then fall back to 'I do not know' or parametric hallucination. This is a correctness risk for the claimed generality of zero-shot KG-based RAG, not an internal contradiction. Please either add inverse-edge or bidirectional traversal, evaluate on a benchmark with canonical directions, or explicitly state and justify the orientation assumption in the method and abstract.
  2. [§3 Implementation Details; §4 Results] The hyperparameters k=3, walk depth 4 for MetaQA, depth 3 for CRAG, and n_w=60 appear to be selected through preliminary experiments on the test sets; no validation split is described. This creates a risk of test-set overfitting, especially because the depth values differ between the two benchmarks. Additionally, the random-walk variant is stochastic, but no variance or error bars are reported across random-walk seeds. Please specify the validation protocol, report mean and standard deviation over multiple seeds for the RW variant, and show sensitivity to k and n_w on held-out data rather than only on the test sets.
  3. [§4, Table 2 (CRAG block)] The two KG-RAG baselines SubgraphRAG and RetrieveRewriteAnswer are not evaluated on CRAG (footnote 6), so the CRAG rows only support a comparison against LLM-only and Vanilla RAG. The paper nonetheless uses the CRAG results to claim scalability to large KGs and robustness. The absence of the strongest baselines weakens that claim; please either soften the scalability conclusion or include at least one KG-RAG baseline on CRAG, for example on a subsample of the CRAG KG or with a reduced configuration.
minor comments (5)
  1. [§4, first paragraph] The sentence 'Table 1 summarizes the QA performance of Walk&Retrieve and the baselines' should refer to Table 2; Table 1 reports dataset statistics.
  2. [§4, Fig. 4 discussion] In the left-graph discussion, 'due to is systematic graph exploration' should read 'due to its systematic graph exploration'.
  3. [§2.1] The notation C_RW = ∪ W_l suggests a set, but the ablation footnote reports that each node yields 60 duplicated and 8.74 unique random walks; please clarify whether the corpus retains duplicates as a multiset or deduplicates before indexing, since this affects both retrieval scores and the reported counts.
  4. [§3, Evaluation Metrics] The evaluation uses Llama-3.1-70B as both the generator and one of the two judges. The two-judge design mitigates self-preference, but the paper should report the agreement rate between the two evaluators (e.g., Cohen's kappa or the percentage of concordant labels) so the average scores are interpretable.
  5. [§4, Ablation of Walk Approach] The text states that values of n_w other than 60 'perform comparably,' but no numerical results are shown; please include the n_w ablation curve or a small table so the claim is verifiable.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the pipeline is an empirical retrieval+prompting system benchmarked against external baselines; no fitted parameter is relabeled as a prediction.

full rationale

Walk&Retrieve is an empirical pipeline: the corpus is built by BFS or random walks over KG edges (Eqs. 1-2), verbalized by an LLM prompt, embedded, and retrieved by cosine similarity; the final answer is a single LLM call conditioned on the top-k walks. No parameter is fitted to answer labels, no equation defines the output in terms of the input, and no 'prediction' is a renamed fit. The headline MetaQA/CRAG numbers are compared against external baselines (SubgraphRAG, RetrieveRewriteAnswer, Vanilla RAG, LLM-only) and are therefore falsifiable measurements, not tautologies. The one overlapping design choice - using Llama-3.1-70B both as the generation backbone and as one of two LLM evaluators - is an evaluation-fairness caveat, explicitly mitigated by adding gpt-4-0125-preview as a second judge; it does not make the derivation circular. The outgoing-edge-only traversal (N(v) defined over (v,r,v') in E) is a real coverage limitation for canonicalized KGs, but it is a generality/correctness concern about the benchmark, not a circularity in the derivation. The only self-citation (Ristoski & Paulheim, ref. [41]) is used for a standard BFS layering concept and is not load-bearing. Hence score 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. It relies on several domain assumptions about graph coverage, embedding relevance, verbalization fidelity, and judge reliability. The free parameters are hyperparameters selected empirically, apparently on the test sets.

free parameters (3)
  • k (number of retrieved nodes and walks) = 3
    Selected in preliminary experiments over k in [1,5] for the balance of accuracy and hallucination; no validation split is described.
  • walk depth l = 4 on MetaQA, 3 on CRAG
    Chosen by ablation on the MetaQA test subsets because depth 4 gave the best truthfulness trade-off.
  • number of random walks n_w = 60
    Reported as the chosen value after ablation over n_w in [10,100]; other values perform comparably.
assumptions (5)
  • domain assumption Bounded-depth walks rooted at a node contain the facts needed to answer questions about that node.
    Invoked in §2.1 and validated only empirically through the depth ablation; the paper does not prove coverage for multi-hop answers.
  • domain assumption Cosine similarity between LLM embeddings of the query and of verbalized nodes and walks is a reliable relevance signal.
    Used in Eq. 3 to build V_k and W_k; no learned retriever or re-ranking is used.
  • domain assumption LLM verbalization preserves the factual content and order of triples in a walk.
    Section 2.1 relies on the shared embedding space between verbalized walks and queries; if the verbalizer drops or reorders facts, retrieval and generation degrade.
  • domain assumption Traversal may follow only outgoing edges; any answer-relevant reverse fact must be stored as an outgoing edge from the query entity.
    N(v) in §2.1 and Eq. 1 use only (v_i,r,v_j) in E, so incoming triples are ignored; the paper does not state that the benchmarks contain reverse triples.
  • domain assumption LLM judge scores from gpt-4 and Llama-3.1-70B accurately separate accurate, hallucinated, and missing answers.
    The Evaluation Metrics section relies on these judges for the primary accuracy and hallucination numbers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Walk&Retrieve: Simple Yet Effective Zero-shot Retrieval-Augmented Generation via Knowledge Graph Walks." pith.science (2026). https://pith.science/paper/T5XM3O4S

@misc{pith2026250516849,
  author       = {Pith},
  title        = {Pith review of: Walk&Retrieve: Simple Yet Effective Zero-shot Retrieval-Augmented Generation via Knowledge Graph Walks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/T5XM3O4S}},
  note         = {Machine review of arXiv:2505.16849}
}
read the original abstract

Large Language Models (LLMs) have showcased impressive reasoning abilities, but often suffer from hallucinations or outdated knowledge. Knowledge Graph (KG)-based Retrieval-Augmented Generation (RAG) remedies these shortcomings by grounding LLM responses in structured external information from a knowledge base. However, many KG-based RAG approaches struggle with (i) aligning KG and textual representations, (ii) balancing retrieval accuracy and efficiency, and (iii) adapting to dynamically updated KGs. In this work, we introduce Walk&Retrieve, a simple yet effective KG-based framework that leverages walk-based graph traversal and knowledge verbalization for corpus generation for zero-shot RAG. Built around efficient KG walks, our method does not require fine-tuning on domain-specific data, enabling seamless adaptation to KG updates, reducing computational overhead, and allowing integration with any off-the-shelf backbone LLM. Despite its simplicity, Walk&Retrieve performs competitively, often outperforming existing RAG systems in response accuracy and hallucination reduction. Moreover, it demonstrates lower query latency and robust scalability to large KGs, highlighting the potential of lightweight retrieval strategies as strong baselines for future RAG research.

Figures

Figures reproduced from arXiv: 2505.16849 by the authors.

Figure 1
Figure 1. Overview of the Walk&Retrieve framework: (1) We combine walk-based graph traversal with knowledge verbalization for corpus generation; (2) The answer is generated with a prompt augmenting the query with the most similar verbalized walks. with parameter-efficient methods [5, 16, 39] – is expensive and limits generalization to dynamic KGs or unseen domains [25, 53]. Contributions. We propose Walk&Retrieve, a lightweig… view at source ↗
Figure 2
Figure 2. Prompt templates used for knowledge verbalization and answer generation. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Truthfulness rates for different (i) walk approaches [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

    cs.CL 2026-07 conditional novelty 5.0 of 10

    Hybrid RAG over UK public health guidance sharply raises MCQA accuracy and free-form faithfulness, letting smaller open models match larger closed models without retrieval.

Reference graph

Works this paper leans on

63 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    AI@Meta. 2024. Llama 3 Model Card. (2024). https://github.com/meta-llama/ llama3/blob/main/MODEL_CARD.md

    2024

  2. [2]

    Jinheon Baek, Alham Fikri Aji, and Amir Saffari. 2023. Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answer- ing. InProceedings of the 1st Workshop on Natural Language Reasoning and Struc- tured Explanations (NLRSE). 78–106. doi:10.18653/v1/2023.nlrse-1.7

    doi:10.18653/v1/2023.nlrse-1.7 2023

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in neural information processing systems33 (2020), 1877–1901

    2020

  4. [4]

    Yihan Cao, Yanbin Kang, Chi Wang, and Lichao Sun. 2023. Instruction Mining: Instruction Data Selection for Tuning Large Language Models.arXiv preprint arXiv:2307.06290(2023). doi:10.48550/arXiv.2307.06290

    arXiv doi:10.48550/arxiv.2307.06290 2023

  5. [5]

    Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. 2023. Graphllm: Boosting graph reasoning ability of large lan- guage model.arXiv preprint arXiv:2310.05845(2023). doi:10.48550/arXiv.2310. 05845

    arXiv doi:10.48550/arxiv.2310.05845 2023

  6. [6]

    Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. Benchmarking large language models in retrieval-augmented generation. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 17754–17762. doi:10.1609/aaai.v38i16. 29728

    doi:10.1609/aaai.v38i16 2024

  7. [7]

    Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, and Prithviraj Sen. 2020. A Survey of the State of Explainable AI for Natural Language Processing. InProceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing...

    work page doi:10.18653/v1/2020.aacl- 2020

  8. [8]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization.arXiv preprint arXiv:2404.16130 (2024). doi:10.48550/arXiv.2404.16130

    arXiv doi:10.48550/arxiv.2404.16130 2024

Show all 63 references
  1. [9]

    Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. A survey on rag meeting llms: Towards retrieval-augmented large language models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ...

    2024

  2. [10]

    Bahare Fatemi, Jonathan Halcrow, and Bryan Perozzi. 2024. Talk like a Graph: Encoding Graphs for Large Language Models. InThe Twelfth International Confer- ence on Learning Representations. https://openreview.net/forum?id=IuXR1CCrSi

    2024

  3. [11]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997(2023). doi:10.48550/ arXiv.2312.10997

    2023 doi arXiv

  4. [12]

    Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. 2023. Gpt4graph: Can large language models understand graph structured data? an empirical evaluation and benchmarking.arXiv preprint arXiv:2305.15066(2023). doi:10.48550/arXiv.2305.15066

    2023 doi arXiv

  5. [13]

    Tiezheng Guo, Qingwen Yang, Chen Wang, Yanyi Liu, Pan Li, Jiawei Tang, Dapeng Li, and Yingyou Wen. 2024. Knowledgenavigator: Leveraging large language models for enhanced reasoning over knowledge graph.Complex & Intelligent Systems10, 5 (2024), 7063–7076

    2024

  6. [14]

    Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-retriever: Retrieval-augmented generation for textual graph understanding and question answering.arXiv preprint arXiv:2402.07630(2024). doi:10.48550/arXiv.2402.07630

    2024 doi arXiv

  7. [15]

    Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. 2021. Knowledge graphs.ACM Computing Surveys (Csur)54, 4 (2021), 1–37

    2021

  8. [16]

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. LoRA: Low-Rank Adaptation of Large Language Models. InInternational Conference on Learning Representations

    2022

  9. [17]

    Yuntong Hu, Zhihan Lei, Zheng Zhang, Bo Pan, Chen Ling, and Liang Zhao. 2024. GRAG: Graph Retrieval-Augmented Generation.arXiv preprint arXiv:2405.16506 (2024). doi:10.48550/arXiv.2405.16506

    2024 doi arXiv

  10. [18]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al . 2024. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions.ACM Transactions on Info...

    2024 doi

  11. [19]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation.Comput. Surveys55, 12 (2023), 1–38. doi:10.1145/ 3571730

    2023

  12. [20]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al . 2023. Mistral 7B.arXiv preprint arXiv:2310.06825(2023). doi:10.48550/arXiv.2310.06825

    2023 doi arXiv

  13. [21]

    Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, et al. 2024. Graph Chain-of- Thought: Augmenting Large Language Models by Reasoning on Graphs.arXiv preprint arXiv:2404.07103(2024). doi:10.48550/arXiv.2404.07103

    2024 doi arXiv

  14. [22]

    Jiho Kim, Yeonsu Kwon, Yohan Jo, and Edward Choi. 2023. KG-GPT: A General Framework for Reasoning on Knowledge Graphs Using Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2023. 9410–

    2023

  15. [23]

    2024.GraphRAG: Unlocking LLM discovery on narrative private data

    Jonathan Larson and Steven Truitt. 2024.GraphRAG: Unlocking LLM discovery on narrative private data. Retrieved 2025-01-27 from https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm- discovery-on-narrative-private-data/

    2024

  16. [24]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing S...

    2020

  17. [25]

    Mufei Li, Siqi Miao, and Pan Li. 2025. Simple is effective: The roles of graphs and large language models in knowledge-graph-based retrieval-augmented generation. InInternational Conference on Learning Representations. https: //openreview.net/pdf?id=JvkuZZ04O7

    2025

  18. [26]

    Shiyang Li, Yifan Gao, Haoming Jiang, Qingyu Yin, Zheng Li, Xifeng Yan, Chao Zhang, and Bing Yin. 2023. Graph Reasoning for Question Answering with Triplet Retrieval. InFindings of the Association for Computational Linguistics: ACL

    2023

  19. [27]

    Valentin Liévin, Christoffer Egeberg Hother, Andreas Geert Motzfeldt, and Ole Winther. 2024. Can large language models reason about medical questions? Patterns5, 3 (2024)

    2024

  20. [28]

    Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing.Comput. Surveys55, 9 (2023), 1–35. doi:10.1145/3560815

    2023 doi

  21. [29]

    Linhao Luo, Yuan-Fang Li, Reza Haf, and Shirui Pan. 2024. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning. InThe Twelfth International Conference on Learning Representations

    2024

  22. [30]

    Shengjie Ma, Chengjin Xu, Xuhui Jiang, Muzhi Li, Huaren Qu, Cehao Yang, Jiaxin Mao, and Jian Guo. 2024. Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation. arXiv preprint arXiv:2407.10805(2024). doi:10.4855...

    2024 doi arXiv

  23. [31]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. InProceedings of the 61st Annual Meeting of the Association for Comput...

    2023 doi

  24. [32]

    Costas Mavromatis and George Karypis. 2024. GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning.arXiv preprint arXiv:2405.20139(2024). doi:10.48550/arXiv.2405.20139

    2024 doi arXiv

  25. [33]

    2023.ChatGPT

    OpenAI. 2023.ChatGPT. Retrieved 2025-02-14 from https://openai.com/index/ chatgpt/

    2023

  26. [34]

    OpenAI. 2023. GPT-4 Technical Report.arXiv preprint arXiv:2303.08774(2023). doi:10.48550/arXiv.2303.08774

    2023 doi arXiv

  27. [35]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (...

    2022

  28. [36]

    Arjun Panickssery, Samuel R Bowman, and Shi Feng. 2024. Llm evaluators recognize and favor their own generations.arXiv preprint arXiv:2404.13076 (2024). doi:10.48550/arXiv.2404.13076 IR-RAG 2025, July 17, 2025, Padua, Italy Böckling et al

    2024 doi arXiv

  29. [37]

    Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. 2024. Graph retrieval-augmented generation: A survey. arXiv preprint arXiv:2408.08921(2024)

    2024 arXiv

  30. [38]

    Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. InProceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701–710. doi:10.1145/2623330. 2623732

    2014 doi

  31. [39]

    Bryan Perozzi, Bahare Fatemi, Dustin Zelle, Anton Tsitsulin, Mehran Kazemi, Rami Al-Rfou, and Jonathan Halcrow. 2024. Let your graph do the talking: Encoding structured data for llms.arXiv preprint arXiv:2402.05862(2024)

    2024 arXiv

  32. [40]

    Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, SM Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, and Amitava Das

  33. [41]

    Petar Ristoski and Heiko Paulheim. 2016. Rdf2vec: Rdf graph embeddings for data mining. InInternational semantic web conference. Springer, 498–514

    2016

  34. [42]

    Ahmmad O. M. Saleh, Gokhan Tur, and Yucel Saygin. 2024. SG-RAG: Multi- Hop Question Answering With Large Language Models Through Knowledge Graphs. InProceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), Mourad Abbas and Abed A...

    2024

  35. [43]

    InProceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing

    The Troubling Emergence of Hallucination in Large Language Models-An Extensive Definition, Quantification, and Prescriptive Remediations. InProceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2541–2573. doi:10.18653/v1/2023.emnlp-main.155

    2023 doi

  36. [44]

    Priyanka Sen, Alham Fikri Aji, and Amir Saffari. 2022. Mintaka: A Complex, Natu- ral, and Multilingual Dataset for End-to-End Question Answering. InProceedings of the 29th International Conference on Computational Linguistics. 1604–1619

    2022

  37. [45]

    Priyanka Sen, Sandeep Mavadia, and Amir Saffari. 2023. Knowledge graph- augmented language models for complex question answering. InProceedings of the 1st Workshop on Natural Language Reasoning and Structured Explanations (NLRSE). 1–8. doi:10.18653/v1/2023.nlrse-1.1

    2023 doi

  38. [46]

    Apoorv Saxena, Adrian Kochsiek, and Rainer Gemulla. 2022. Sequence-to- Sequence Knowledge Graph Completion and Question Answering. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2814–2828. doi:10.18653/v1/2022...

    2022 doi

  39. [47]

    Kai Sun, Yifan Xu, Hanwen Zha, Yue Liu, and Xin Luna Dong. 2024. Head- to-Tail: How Knowledgeable are Large Language Models (LLMs)? AKA Will LLMs Replace Knowledge Graphs?. InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational ...

    2024 doi

  40. [48]

    Alon Talmor and Jonathan Berant. 2018. The Web as a Knowledge-Base for Answering Complex Questions. InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 641–651...

    2018 doi

  41. [49]

    Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel Ni, Heung-Yeung Shum, and Jian Guo. 2024. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. In The Twelfth International Conference on Learning Repres...

    2024

  42. [50]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023). doi:10.48550/...

    2023 doi arXiv

  43. [51]

    Tu Vu, Mohit Iyyer, Xuezhi Wang, Noah Constant, Jerry Wei, Jason Wei, Chris Tar, Yun-Hsuan Sung, Denny Zhou, Quoc Le, and Thang Luong. 2024. FreshLLMs: Re- freshing Large Language Models with Search Engine Augmentation. InFindings of the Association for Computational Linguisti...

    2024 doi

  44. [52]

    SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. 2024. A comprehensive survey of hallucination mitigation techniques in large language models.arXiv preprint arXiv:2401.01313(2024). doi:10.48550/arXiv.2401.01313

    2024 doi arXiv

  45. [53]

    Yike Wu, Nan Hu, Sheng Bi, Guilin Qi, Jie Ren, Anhuan Xie, and Wei Song. 2023. Retrieve-rewrite-answer: A kg-to-text enhanced llms framework for knowledge graph question answering.arXiv preprint arXiv:2309.11206(2023). doi:10.48550/ arXiv.2309.11206

    2023 doi arXiv

  46. [54]

    Yike Wu, Yi Huang, Nan Hu, Yuncheng Hua, Guilin Qi, Jiaoyan Chen, and Jeff Pan. 2024. CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. ...

    2024 doi

  47. [55]

    Yilin Wen, Zifeng Wang, and Jimeng Sun. 2023. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models.arXiv preprint arXiv:2308.09729(2023). doi:10.48550/arXiv.2308.09729

    2023 doi arXiv

  48. [56]

    Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering. InProceedings of the 2021 Conference of the North American Chapter of the Association for Computational L...

    2021 doi

  49. [57]

    Wen-tau Yih, Matthew Richardson, Christopher Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. InProceedings of the 54th Annual Meeting of the Association for Com- putational Linguistics (Volume 2: Short Paper...

    2016 doi

  50. [58]

    Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sa- jal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, et al . 2024. CRAG–Comprehensive RAG Benchmark.38th Conference on Neural Information Processing Systems (NeurIPS 2024), Track on Datasets a...

    2024

  51. [59]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models.arXiv preprint arXiv:2303.18223(2023). doi:10.48550/ arXiv.2303.18223

    2023 doi arXiv

  52. [61]

    Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander Smola, and Le Song

  53. [2018]

    In Proceedings of the AAAI conference on artificial intelligence, Vol

    Variational reasoning for question answering with knowledge graph. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32

  54. [2023]

    doi:10.18653/v1/2023.findings-acl.208

    3366–3375. doi:10.18653/v1/2023.findings-acl.208

    2023 doi

  55. [9421]

    doi:10.18653/v1/2023.findings-emnlp.631

    2023 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.