Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

Kostas Tsioutsiouliklis; Xue Wu

REVIEW 4 major objections 6 minor 2 cited by

Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Representing knowledge graphs as Python code improves how well LLMs chain facts together, the paper argues.

desk verdict A useful, modest empirical study showing Python-encoded KG triples help LLM multi-hop reasoning, but the key ablation separating code format from explicit inference steps is missing. read the letter →

arxiv 2412.10654 v1 pith:W4QMYSDT submitted 2024-12-14 cs.CL cs.LG

Xue Wu , Kostas Tsioutsiouliklis This is my paper · ORCID

classification cs.CLcs.LG

keywords knowledgegraphsmulti-hopreasoningprogramminglanguagerepresentationLLMfine-tuningone-shotpromptingPythonstructureddata

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the format used to feed a knowledge graph to a large language model changes how well the model can chain facts together. The authors claim that representing the graph's triples as Python code—a dictionary of entities and relations plus a small inference function—outperforms natural-language sentences and JSON data, both in one-shot prompting and in fine-tuning. On a two-hop Wikidata reasoning task, the best Python one-shot prompt raised the accuracy of completing the second hop given a correct first hop from 38.2% to 67.9%, and fine-tuned Python versions reach the low nineties on that metric. Across datasets, fine-tuned small models with Python representations match or exceed a much larger zero-shot model. The paper presents this as evidence that grounding LLM reasoning in structured knowledge can be largely a representation choice.

What carries the argument

The central object is the Python representation of a knowledge graph: a `relationships` dictionary whose keys are relation names and whose values map a start entity to an end entity, optionally wrapped in a `KnowledgeBase` class with an `infer` function that composes any number of relations. The argument turns on bundling the static triples with an explicit, executable inference process, so the prompt shows the LLM the exact dictionary lookups required to answer the multi-hop question. This is contrasted with JSON, which stores the same data but no lookup sequence, and with natural-language descriptions, which state the facts but do not lay out the retrieval steps.

What would settle it

Run the one-shot comparison again with a natural-language prompt that spells out the same inference steps (for example, first look up the composer of X, then look up the spouse of the result) and with a JSON prompt that includes an ordered list of lookup operations; if either matches the Python accuracy, the claimed format advantage is not specific to Python.

Watch

Extended reading notes

Core claim

The paper's central claim is that encoding a knowledge graph as executable Python code improves LLM multi-hop reasoning compared to encoding the same triples as natural language or JSON. The static form is a `relationships` dictionary with explicit lookup steps; the dynamic form wraps the dictionary in a `KnowledgeBase` class whose `infer` function walks through an arbitrary chain of relations. In one-shot prompting, the dynamic Python representation raises the conditioned two-hop accuracy from 38.2% to 67.9% on Dataset 1, while JSON one-shot prompting stays below zero-shot. After LoRA fine-tuning, Python-tuned models reach the highest conditioned accuracies on both datasets and also show the best transfer to three-hop questions. The authors state they are the first to represent knowledge graphs with programming language and to fine-tune pretrained LLMs with those representations.

Load-bearing premise

The comparison assumes the gain comes from the Python format itself, but the Python prompts also contain explicit step-by-step lookup instructions that the JSON and natural-language prompts do not, so the format effect is entangled with the presence of a worked inference path.

Editorial extensions

If this is right

Python-encoded knowledge graphs can be dropped into existing one-shot prompts or fine-tuning datasets without changing the model architecture.
Fine-tuning on two-hop Python triples improves three-hop reasoning, so the learned benefit is not tied to the trained hop count.
A small fine-tuned model with Python representations can outperform a much larger zero-shot model on the same multi-hop questions.
Supplying RAG-style context as Python code that spells out inference steps yields higher answer accuracy than supplying the same facts as plain text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The non-Python prompts in the comparison lack the explicit step-by-step lookups that the Python prompts include, so the active ingredient may be the worked inference path rather than the code format itself.
The same representation idea could be applied to other structured reasoning inputs, such as tables or provenance traces, wherever the model must follow a fixed traversal path.
A follow-up that swaps Python for pseudocode, or adds a natural-language chain of thought with identical steps, would separate the syntax effect from the step-explicitness effect.
The reported gains are on compositional chains; whether the Python advantage holds for non-compositional or cyclic graph queries is not addressed.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The core idea is simple and probably useful: represent KG triples as Python code, write the multi-hop inference chain explicitly, and use that as a prompt or fine-tuning target. The paper shows this beats JSON and, in most conditions, natural language, on two synthetic Wikidata two-hop datasets. The dynamic Python representation with a KnowledgeBase class and infer() function is the most interesting piece; it makes the reasoning path explicit and executable. To the authors' credit, they test one-shot prompting, fine-tuning, 3-hop generalization, and a RAG-style context condition. The observation that a fine-tuned 8B model can beat a zero-shot 70B model on latent accuracy is worth reporting.

The main soft spot is the missing ablation, though it is less severe than a strict reading of the confound suggests. Both the NL and Python one-shot prompts contain step-by-step inference: for NL this is the 'Explanation' field like 'r1 of e1 is e2. r2 of e2 is e3'; for Python it is the executable lookup code. So the Python-vs-NL difference is mostly format, not presence of inference steps. The JSON condition lacks those steps entirely, which probably explains why it lags. What is missing is a control that gives Python-style inference steps in a non-code format or code without explicit steps, to isolate the format effect. That is a fair referee request, not a fatal flaw.

More minor issues: no significance tests or error bars; several reported differences are small (e.g., 25.6% vs 26.2% fine-tuned NL vs Python static on Dataset 1). The 'first' claim is overstated given Nie et al. did code-style ICL for KBQA; the paper differentiates itself but 'first to represent KGs with programming language' is too strong. Fine-tuning details are thin (LoRA rank, learning rate, etc.). No code or data release.

The paper is honest about its limitations: it only studies compositional two-hop and three-hop reasoning on synthetic data. It is a modest, applied contribution, and as long as the claims are reined in and the ablation added, it deserves a serious referee and probably a workshop or applied NLP venue.

Referee Report

4 major / 6 minor

Summary. The paper proposes representing knowledge-graph triples as Python code (a static dictionary variant and a dynamic KnowledgeBase-class variant) to improve multi-hop reasoning in LLMs. It compares these representations against natural-language and JSON representations under one-shot prompting and LoRA fine-tuning of Llama-3.1-8B-Instruct, using two two-hop Wikidata-derived datasets, a three-hop generalization set, and a context-augmented setting. The central claim is that Python representations of KGs outperform natural-language and JSON representations in complex reasoning tasks, and that the authors are the first to represent KGs with programming language and to fine-tune pretrained LLMs with such representations.

Significance. If the central comparison were cleanly established, the paper would offer a practically useful and simple idea: encoding KG triples as Python data structures and inference code, capitalizing on LLMs' familiarity with code from pretraining. The study covers several relevant settings (one-shot prompting, fine-tuning, three-hop generalization, and context-augmented inference) and reports a hop-conditioned accuracy metric, which is a reasonable way to isolate reasoning ability from factual knowledge. However, the load-bearing comparison is currently confounded: the Python prompts include explicit step-by-step inference code that the JSON and natural-language prompts do not contain, and no error bars or significance tests are reported. The lack of a released implementation or prompt templates also makes the results hard to verify. The underlying idea is plausible and worth further investigation, but the paper as written does not yet demonstrate that the programming-language format itself is the cause of the observed improvements.

major comments (4)

[§4.5 and Appendix A.2–A.4] The Python one-shot conditions contain explicit executable inference steps ("e2 = relationships[r1][e1]", "e3 = relationships[r2][e2]", "kb.infer(e1, r1, r2)"), while the JSON prompt in A.2 contains only data and the natural-language prompt in A.1 contains a prose explanation but no such stepwise lookup. Thus the Python treatment bundles the programming-language syntax with the presence of explicit inference instructions. The paper's own discussion in §4.8 states that Python representations "store both structured data and the inference process," which acknowledges this entanglement. Without an ablation that varies the format and the presence of inference steps independently (for example, JSON with explicit lookup steps, or Python code without the inference code), the headline attribution to programming-language representation is not supported.
[Tables 5 and 6] No error bars, confidence intervals, or significance tests are reported for any accuracy figure. For instance, in Table 5 the fine-tuned natural-language model achieves 25.6% and the fine-tuned Python-static model achieves 26.2%; with roughly ten thousand test questions the standard error of that difference is on the order of 0.6 percentage points, so the gap is within noise. The one-shot results in Table 6 show similarly small differences (e.g., Python Static 4.98% vs. Python Dynamic 4.12%). The paper should report uncertainty intervals or significance tests before claiming that Python representations are significantly better.
[§4.1.3 and §4.7.2 (RQ2)] The three-hop generalization experiment is heavily overlapped with the training relation pairs. Table 3 shows that 973 of the 1,007 test rows share the same (r1, r2) pair as Dataset 1 training rows, and the paper itself notes the significant overlap. Consequently, the three-hop improvements in Table 7 may reflect reuse of familiar relation pairs rather than generalization to longer unseen reasoning chains. The generalization claim would need an evaluation on a held-out split with disjoint relation pairs, or at least a separate reporting of the non-overlapping subset.
[§4.1.3 and §4.4.1] The fine-tuning condition is underspecified. The paper says the final training data include one-hop and two-hop prompts and responses, but it does not state whether the Python training targets contain the same inference code shown in the one-shot prompts (Appendix A.3 and A.4) or only the data structures plus the answer. This matters because if the targets include the inference code, the fine-tuning benefit may come from learning to emit lookup operations rather than from the representational format. The authors should specify the exact training templates and, ideally, ablate the inference-code component.

minor comments (6)

[§2] There is a typo: "impove" should be "improve".
[Table 4] The JSON representation row has a malformed key ("“Explanation”” :") and a duplicated phrase "generate explanation and provide answer to the question"; the table layout should be cleaned up.
[Appendix A.4] The code snippet uses `self.facts = {{}}`, which is not valid Python; presumably `self.facts = {}` is intended.
[§4.7.1] The sentence reporting "approximately 78% higher" performance appears to refer to the final-accuracy column (67.9% vs. 38.2% in Table 5) rather than to the conditioned metric shown as "Prompt Accuracy" (19.1% vs. 16.3%). Please clarify which metric is being compared, since the two give very different impressions.
[Tables 5–8] The repeated column headers "1st hop correct 2nd hop correct" make the tables hard to read; please define each column with a clear label, for example distinguishing counts where the final answer is correct from counts where it is incorrect.
[§4.2 and §4.7.3] The comparisons between fine-tuned 8B models and the zero-shot 70B model are not apples-to-apples because the larger model is not fine-tuned; the text should note this asymmetry explicitly when stating that small fine-tuned models "surpass" the larger baseline.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical comparison whose central claim does not reduce to its inputs, though the Python-format advantage is confounded with the presence of explicit inference steps.

full rationale

This is an empirical study, not a derivation chain. The central claim that Python KG representations improve LLM multi-hop reasoning is supported by held-out accuracy measurements on two datasets (Tables 5 and 6), and no parameter is fitted to the test set and then renamed as a prediction. The train/test split by bridge entity and the relationship-frequency cap are experimental design choices, not fitted inputs. The paper never defines Python superiority in terms of the evaluation metric, nor does it invoke a self-citation or imported uniqueness theorem to force its conclusion. A genuine limitation exists: the Python prompts in Appendices A.3 and A.4 contain explicit executable lookup steps (e.g., 'e2 = relationships[r1][e1]', 'e3 = relationships[r2][e2]') and the paper itself says Python representations 'store both structured data and the inference process,' so the observed benefit could be due to the inference steps rather than the programming-language format. However, that is a confound threatening internal validity, not circularity: the conclusion is not equivalent to its inputs by construction. Accordingly, no circular step is identified and the score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper relies on experimental design choices (dataset partitions, frequency caps, prompt formats) and domain assumptions about LLM code comprehension and metric validity. No new entities or physical parameters are introduced.

free parameters (3)

Dataset 1 partition selection = partition 2 (train), partition 4 (test)
The paper chooses these partitions to avoid overrepresentation of popular bridge entities, but does not justify the specific choice or test sensitivity to it.
Dataset 2 relation frequency cap = 500 instances per relation pair in training
Ad hoc threshold to avoid overrepresentation; no ablation is provided.
Fine-tuning hyperparameters = 1 epoch, LoRA, unspecified rank and learning rate
Not fully specified in the paper, so reproduction requires guessing.

assumptions (3)

domain assumption LLMs can parse and reason over Python code because code appears in pretraining data.
Invoked in Sections 1 and 3.2 to justify why Python representation should work.
domain assumption The conditional metric r = p(h|h1,...,hn) isolates multi-hop reasoning from factual recall.
Defined in Section 4.6; assumes intermediate hop correctness can be cleanly measured.
domain assumption The two-hop and three-hop compositional reasoning tasks are representative of complex reasoning.
Datasets are synthetic extractions from Wikidata; real reasoning may be more complex, as the paper admits in Section 4.8.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data." pith.science (2026). https://pith.science/paper/W4QMYSDT

@misc{pith2026241210654,
  author       = {Pith},
  title        = {Pith review of: Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W4QMYSDT}},
  note         = {Machine review of arXiv:2412.10654}
}

read the original abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, they often struggle with complex reasoning tasks and are prone to hallucination. Recent research has shown promising results in leveraging knowledge graphs (KGs) to enhance LLM performance. KGs provide a structured representation of entities and their relationships, offering a rich source of information that can enhance the reasoning capabilities of LLMs. For this work, we have developed different techniques that tightly integrate KG structures and semantics into LLM representations. Our results show that we are able to significantly improve the performance of LLMs in complex reasoning scenarios, and ground the reasoning process with KGs. We are the first to represent KGs with programming language and fine-tune pretrained LLMs with KGs. This integration facilitates more accurate and interpretable reasoning processes, paving the way for more advanced reasoning capabilities of LLMs.

Figures

Figures reproduced from arXiv: 2412.10654 by the authors.

**Figure 2.** JSON Representation of KG with Static Relation [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Python Representation of KG with Static Relation [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Python Representation of KG with Dynamic Rela [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: Examples of Different Representation of KG [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Can Structural Cues Save LLMs? Evaluating Language Models in Massive Document Streams
cs.CL 2026-02 conditional novelty 6.0 of 10

Structural cues that organize facts by event improve LLM topic clustering and temporal QA on a new streaming-news benchmark, but temporal reasoning and summarization integration remain hard.
Aligning Knowledge Graphs and Language Models for Factual Accuracy
cs.CL 2025-07 conditional novelty 3.0 of 10

ALIGNed-LLM aligns knowledge graph entity embeddings with language model text embeddings through a trainable projection layer, improving question answering accuracy on KG-derived datasets.

Reference graph

Works this paper leans on

45 extracted references · 11 canonical work pages · cited by 2 Pith papers

[1]

Meta AI. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https: //arxiv.org/abs/2407.21783

arXiv 2024
[2]

Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. To Code, or Not To Code? Exploring Impact of Code in Pre-training. arXiv:2408.10914 [cs.CL] https://arxiv.org/abs/2408.10914

arXiv 2024
[3]

Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson

work page
[4]

Bollacker, Colin Evans, Praveen K

Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor
[5]

Tim Bray. 2014. The javascript object notation (json) data interchange format

work page 2014
[6]

Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024. Leveraging small lan- guage models for Text2SPARQL tasks to improve the resilience of AI assistance. arXiv:2405.17076 [cs.AI] https://arxiv.org/abs/2405.17076

arXiv 2024
[7]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

arXiv 2020
[8]

Diego Bustamante and Hideaki Takeda. 2024. SPARQL Generation with Entity Pre-trained GPT for KG Question Answering. arXiv:2402.00969 [cs.CL] https: //arxiv.org/abs/2402.00969

work page Pith review arXiv 2024

Show all 45 references

[9]

Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, and Yang Yang. 2023. GraphLLM: Boosting Graph Reasoning Ability of Large Language Model. arXiv:2310.05845 [cs.CL] https://arxiv.org/abs/2310.05845

2023 arXiv
[10]

Nurendra Choudhary, Nikhil Rao, Karthik Subbian, and Chandan Reddy. 2022. Graph-based multilingual language model: Leveraging product relations for search relevance. InKDD 2022. https://www.amazon.science/publications/graph- based-multilingual-language-model-leveraging-product...

2022
[11]

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebas- tian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vi...

2022 arXiv
[12]

Stefan Dernbach, Khushbu Agarwal, Alejandro Zuniga, Michael Henry, and Su- tanay Choudhury. 2024. GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding. arXiv:2402.06764 [cs.AI] https://arxiv...

2024 arXiv
[13]

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 [cs.CL] https://arxiv.org/abs/2404.16130

2024 arXiv
[14]

Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2023. Complexity-Based Prompting for Multi-step Reasoning. In The Eleventh Interna- tional Conference on Learning Representations . https://openreview.net/forum? id=yf1icZHC-l9

2023
[15]

Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, and Shi Han. 2023. GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking. arXiv:2305.15066 [cs.AI] https: //arxiv.org/abs/2305.15066

2023 arXiv
[16]

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reason- ing Steps. InProceedings of the 28th International Conference on Computational Lin- guistics. International Committee on Computati...

2020
[17]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations . https://openreview.net/forum?id=nZeVKeeFYf9 Thinkin...

2022
[18]

Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of Hallucination in Natural Language Generation. Comput. Surveys 55, 12 (March 2023), 1–38. https://doi.org/10.1145/3571730

2023 doi
[19]

Seungone Kim, Se June Joo, Doyoung Kim, Joel Jang, Seonghyeon Ye, Jamin Shin, and Minjoon Seo. 2023. The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning. arXiv:2305.14045 [cs.CL] https://arxiv.org/abs/2305.14045

2023 arXiv
[20]

Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grig- orev. 2022. Internet-augmented language models through few-shot prompt- ing for open-domain question answering. arXiv:2203.05115 [cs.CL] https: //arxiv.org/abs/2203.05115

2022 arXiv
[21]

Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, and Christian Bizer. 2014. DBpedia - A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web J...

2014 doi
[22]

Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024. Reason- ing on Graphs: Faithful and Interpretable Large Language Model Reasoning. arXiv:2310.01061 [cs.CL] https://arxiv.org/abs/2310.01061

2024 arXiv
[23]

Yingwei Ma, Yue Liu, Yue Yu, Yuanliang Zhang, Yu Jiang, Changjian Wang, and Shanshan Li. 2023. At Which Training Stage Does Code Data Help LLMs Reasoning? arXiv:2309.16298 [cs.CL] https://arxiv.org/abs/2309.16298

2023 arXiv
[24]

Zhijie Nie, Richong Zhang, Zhongyuan Wang, and Xudong Liu. 2024. Code-Style In-Context Learning for Knowledge-Based Question Answering. arXiv:2309.04695 [cs.CL] https://arxiv.org/abs/2309.04695

2024 arXiv
[25]

Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. Measuring and Narrowing the Compositionality Gap in Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2023 , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.)...

2023 doi
[26]

Rangel, Tarcisio Mendes de Farias, Ana Claudia Sima, and Norio Kobayashi

Julio C. Rangel, Tarcisio Mendes de Farias, Ana Claudia Sima, and Norio Kobayashi. 2024. SPARQL Generation: an analysis on fine-tuning Open- LLaMA for Question Answering over a Life Science Knowledge Graph. arXiv:2402.04627 [cs.AI] https://arxiv.org/abs/2402.04627

2024 arXiv
[27]

Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A Free Collaborative Knowledgebase. Commun. ACM 57, 10 (Sept. 2014), 78–85. https://doi.org/10. 1145/2629489

2014
[28]

Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, and Yulia Tsvetkov. 2024. Can Language Models Solve Graph Problems in Natural Language? arXiv:2305.10037 [cs.CL] https://arxiv.org/abs/2305.10037

2024 arXiv
[29]

Jianing Wang, Junda Wu, Yupeng Hou, Yao Liu, Ming Gao, and Julian McAuley
[30]

Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Im- proves Chain of Thought Reasoning in Language Models. In The Eleventh Inter- national Conference on Learning Representations . https://...

2023
[31]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agar-...

2022
[32]

arXiv:2402.08785 [cs.CL] https: //arxiv.org/abs/2402.08785

InstructGraph: Boosting Large Language Models via Graph-centric In- struction Tuning and Preference Alignment. arXiv:2402.08785 [cs.CL] https: //arxiv.org/abs/2402.08785

arXiv
[33]

Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel
[34]

Griffiths, Yuan Cao, and Karthik R Narasimhan

Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.n...

2023
[35]

Silei Xu, Shicheng Liu, Theo Culhane, Elizaveta Pertseva, Meng-Hsi Wu, Sina Semnani, and Monica Lam. 2023. Fine-tuned LLMs Know More, Hallucinate Less with Few-Shot Sequence-to-Sequence Semantic Parsing over Wikidata. In Proceedings of the 2023 Conference on Empirical Methods ...

2023 doi
[36]

Michihiro Yasunaga, Antoine Bosselut, Hongyu Ren, Xikun Zhang, Christopher D Manning, Percy Liang, and Jure Leskovec. 2022. Deep Bidirectional Language- Knowledge Graph Pretraining. arXiv:2210.09338 [cs.CL] https://arxiv.org/abs/ 2210.09338

2022 arXiv
[37]

Do Large Language Models Latently Perform Multi-Hop Reasoning? arXiv:2402.16837 [cs.CL] https://arxiv.org/abs/2402.16837

arXiv
[38]

Xinlu Zhang, Zhiyu Zoey Chen, Xi Ye, Xianjun Yang, Lichang Chen, William Yang Wang, and Linda Ruth Petzold. 2024. Unveiling the Impact of Coding Data Instruc- tion Fine-Tuning on Large Language Models Reasoning. arXiv:2405.20535 [cs.AI] https://arxiv.org/abs/2405.20535

2024 arXiv
[39]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In International Conference on Learning Representations (ICLR)

2023
[40]

composer

Kerui Zhu, Bo-Wei Huang, Bowen Jin, Yizhu Jiao, Ming Zhong, Kevin Chang, Shou-De Lin, and Jiawei Han. 2024. Investigating Instruction Tuning Large Language Models on Graphs. arXiv:2408.05457 [cs.CL] https://arxiv.org/abs/ 2408.05457 A ONE SHOT EXAMPLE FOR DIFFERENT PROMPT FORM...

2024 arXiv
[41]

Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang
[42]

arXiv:2308.07134 [cs.CL] https://arxiv

Language is All a Graph Needs. arXiv:2308.07134 [cs.CL] https://arxiv. org/abs/2308.07134

arXiv
[44]

Xuan Zhang, Chao Du, Tianyu Pang, Qian Liu, Wei Gao, and Min Lin. 2024. Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs. arXiv:2406.09136 [cs.CL] https://arxiv.org/abs/2406.09136

2024 arXiv
[2008]

In SIGMOD Conference

Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD Conference. https://api.semanticscholar.org/CorpusID: 207167677
[2024]

arXiv:2406.12775 [cs.CL] https://arxiv.org/abs/2406.12775

Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries. arXiv:2406.12775 [cs.CL] https://arxiv.org/abs/2406.12775

arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Meta AI. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https: //arxiv.org/abs/2407.21783

arXiv 2024

[2] [2]

Viraat Aryabumi, Yixuan Su, Raymond Ma, Adrien Morisot, Ivan Zhang, Acyr Locatelli, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. To Code, or Not To Code? Exploring Impact of Code in Pre-training. arXiv:2408.10914 [cs.CL] https://arxiv.org/abs/2408.10914

arXiv 2024

[3] [3]

Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson

work page

[4] [4]

Bollacker, Colin Evans, Praveen K

Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor

[5] [5]

Tim Bray. 2014. The javascript object notation (json) data interchange format

work page 2014

[6] [6]

Felix Brei, Johannes Frey, and Lars-Peter Meyer. 2024. Leveraging small lan- guage models for Text2SPARQL tasks to improve the resilience of AI assistance. arXiv:2405.17076 [cs.AI] https://arxiv.org/abs/2405.17076

arXiv 2024

[7] [7]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

arXiv 2020

[8] [8]

Diego Bustamante and Hideaki Takeda. 2024. SPARQL Generation with Entity Pre-trained GPT for KG Question Answering. arXiv:2402.00969 [cs.CL] https: //arxiv.org/abs/2402.00969

work page Pith review arXiv 2024