Paper Citation Record · LEDGER

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge

As of 12 August 2026, this Paper Citation Record lists 42 of 42 outbound references and 0 inbound Pith citation observations for arXiv:2412.08317.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2412.08317 v1

Coverage vector

measured 42 of 42 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-08-11T17:59:43.730357Z

measured 42 of 42 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-12T06:34:41.77262+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

42 of 42 outbound references displayed

verified exact2
verified fuzzy20
unresolved20
parse uncertain0
malformed identifier0
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation c6883639-2257-4285-8fca-ded033949def · outbound

This paper cites Bert: Pre-training of deep bidirectional transformers for language understanding.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Bert: Pre-training of deep bidirectional transformers for language understanding

Reference 1

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.499363Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 7a47b91a-7585-4c5f-b41a-03fb3fcabc34 · outbound

This paper cites Language models are unsupervised multitask learners.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Language models are unsupervised multitask learners

Reference 2

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.505737Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9c0f45f4-1dd8-48b5-94e3-eedd21da5d81 · outbound

This paper cites Xlnet: Generalized autoregressive pretraining for language understanding.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Xlnet: Generalized autoregressive pretraining for language understanding

Reference 3

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.575978Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation b4f347d4-1ab8-4c80-a1bb-b70d145284f2 · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 4

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.516670Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d88651c2-7adf-44b7-bb58-47da6f725f01 · outbound

This paper cites DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing

Reference 5

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.523394Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation f3900d4c-055f-4359-a1b7-898571e4c6d4 · outbound

This paper cites Language models are few-shot learners.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Language models are few-shot learners

Reference 6

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.538736Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation e6f6c06e-4575-4c7b-ac9c-0a4a69a0995e · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 7

Resolution: unresolved; raw_fallback, observed 2026-08-11T17:59:44.521524Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation d765eae1-ddb0-4782-b0c2-a3a13cf56717 · outbound

This paper cites GPT-4 Technical Report.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge GPT-4 Technical Report

Reference 8

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.551741Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5325a93a-e7b6-4d4f-9a2a-77862ebc3024 · outbound

This paper cites Chain-of-thought prompting elicits reasoning in large language models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Chain-of-thought prompting elicits reasoning in large language models

Reference 9

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.508823Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 053e3a50-9c2b-4349-842c-edb560e99241 · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 10

Resolution: unresolved; raw_fallback, observed 2026-08-11T17:59:44.491366Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 688239ba-3ed7-4a42-8dfb-5919bace4314 · outbound

This paper cites Tree of thoughts: Deliberate problem solving with large language models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Tree of thoughts: Deliberate problem solving with large language models

Reference 11

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.472570Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 21f7fa33-c3e7-4c38-8561-bc1184864ed0 · outbound

This paper cites Multi-hop Question Answering.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Multi-hop Question Answering

Reference 12

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.572557Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 64b67027-4036-430d-b6d8-ce3962e0037c · outbound

This paper cites Relational Graph Convolutional Neural Networks for Multihop Reasoning: A Comparative Study.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Relational Graph Convolutional Neural Networks for Multihop Reasoning: A Comparative Study

Reference 13

Resolution: verified exact; local_arxiv, observed 2026-08-11T17:59:43.946976Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 07493ef8-73c3-4596-9c6f-5cd4b7f90ef8 · outbound

This paper cites Edge-aware graph neural network for multi-hop path reasoning over knowledge base.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Edge-aware graph neural network for multi-hop path reasoning over knowledge base

Reference 14

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.456432Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 949f16d5-308b-4226-bbba-2212d1293033 · outbound

This paper cites Knowledge-enhanced iterative instruction generation and reasoning for knowledge base question answering.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Knowledge-enhanced iterative instruction generation and reasoning for knowledge base question answering

Reference 15

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.439978Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation f5a3654c-d123-437f-8e55-8c797f8abd09 · outbound

This paper cites Stepwise relation prediction with dynamic reasoning network for multi-hop knowledge graph question answering.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Stepwise relation prediction with dynamic reasoning network for multi-hop knowledge graph question answering

Reference 16

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.412528Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation aa8be2df-7ea6-491a-b9bd-f6e65ee0e0fb · outbound

This paper cites Multi-hop question answering using sparse graphs.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Multi-hop question answering using sparse graphs

Reference 17

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.387895Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation cb05e7d5-26fe-4ae5-ae32-ed156aa0bd8b · outbound

This paper cites Answering complex open-domain questions with multi-hop dense retrieval.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Answering complex open-domain questions with multi-hop dense retrieval

Reference 18

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.370023Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 5233565c-e7df-4cd5-bc8a-6722c4aa1cfd · outbound

This paper cites Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval

Reference 19

Resolution: verified exact; local_arxiv, observed 2026-08-11T17:59:43.914873Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 942cff6e-aac2-4e32-baac-cab212023f1c · outbound

This paper cites Triple-fact retriever: An explainable reasoning retrieval model for multi-hop qa problem.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Triple-fact retriever: An explainable reasoning retrieval model for multi-hop qa problem

Reference 20

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.344408Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 960aae48-8d81-43f7-a8bc-c6bd68aad679 · outbound

This paper cites Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

Reference 21

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.628295Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation e4671539-3b61-4541-98f6-5b5f0d324bda · outbound

This paper cites Entailment as Few-Shot Learner.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Entailment as Few-Shot Learner

Reference 22

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.632242Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 415701e6-b996-41e5-b3d7-4f0ac11664b7 · outbound

This paper cites Multi-hop reading comprehension through question decomposition and rescoring.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Multi-hop reading comprehension through question decomposition and rescoring

Reference 23

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.319810Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 55679171-bf43-47b8-91fe-11bd1b5931be · outbound

This paper cites Measuring and narrowing the compositionality gap in language models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Measuring and narrowing the compositionality gap in language models

Reference 24

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.301300Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 4b5482f6-c0ac-4b97-8eac-6d19b62d6869 · outbound

This paper cites Tree-of-reasoning question decomposition for complex question answering with large language models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Tree-of-reasoning question decomposition for complex question answering with large language models

Reference 25

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.283813Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 40c9f1f1-3859-419c-a15d-498d549e3a40 · outbound

This paper cites Llama 2: Open Foundation and Fine-Tuned Chat Models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Llama 2: Open Foundation and Fine-Tuned Chat Models

Reference 26

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.648142Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 298a36e2-60a6-4b97-ac41-7b8a6efed934 · outbound

This paper cites A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity

Reference 27

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.653704Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3812181d-5493-411d-a8e8-584935160f7a · outbound

This paper cites Dai, and Quoc V Le.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Dai, and Quoc V Le

Reference 28

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.658542Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 850b983d-aa7f-4782-8c84-312582c38059 · outbound

This paper cites Multitask prompted training enables zero-shot task generalization.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Multitask prompted training enables zero-shot task generalization

Reference 29

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.254373Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation c0951306-9f93-47b5-b107-022eae28f9f6 · outbound

This paper cites Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

Reference 30

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.667896Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bd8bb1b8-9415-4a54-a585-0dcf44d11229 · outbound

This paper cites Large language models are zero-shot reasoners.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Large language models are zero-shot reasoners

Reference 31

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.225479Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 0c3802af-f36c-44b2-97bf-d654e7dd4764 · outbound

This paper cites Selection-inference: Exploiting large language models for interpretable logical reasoning.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Selection-inference: Exploiting large language models for interpretable logical reasoning

Reference 32

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.679344Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2faa014d-a9d1-43b9-8af9-f590367f3632 · outbound

This paper cites React: Synergizing reasoning and acting in language models.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge React: Synergizing reasoning and acting in language models

Reference 33

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.683777Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2b0eba2a-f3ec-4c25-88d4-6cd4aa11d925 · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 34

Resolution: unresolved; raw_fallback, observed 2026-08-11T17:59:44.186470Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 55666112-3198-41dd-99f1-191aa0fd795c · outbound

This paper cites Explaining answers with entailment trees.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Explaining answers with entailment trees

Reference 35

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.167865Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation b57dc65c-f7a6-46ac-946b-cc8ac72fc033 · outbound

This paper cites Qasc: A dataset for question answering via sentence composition.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Qasc: A dataset for question answering via sentence composition

Reference 36

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.152616Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 78274a6f-2337-4d8a-a9c7-fe9a743879ac · outbound

This paper cites Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks

Reference 37

Resolution: unresolved; no resolver link, observed 2026-08-11T17:59:43.705384Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6be2f1be-b3d7-4217-8269-a6a2162eb88a · outbound

This paper cites Cognitive neuroscience of human counterfactual reasoning.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Cognitive neuroscience of human counterfactual reasoning

Reference 38

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.133854Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation a65c2047-cede-4f45-a9b2-dfacee181553 · outbound

This paper cites Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, 2023.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, 2023

Reference 39

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.106339Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 477c8e5f-7638-44ab-948a-9f292c007ee3 · outbound

This paper cites A study of automatically acquiring explanatory inference patterns from corpora of explanations: Lessons from elementary science exams.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge A study of automatically acquiring explanatory inference patterns from corpora of explanations: Lessons from elementary science exams

Reference 40

Resolution: verified fuzzy; raw_fallback, observed 2026-08-11T17:59:44.083995Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 55889101-0cbc-4ffb-9400-5b40dbcabc04 · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 41

Resolution: unresolved; raw_fallback, observed 2026-08-11T17:59:44.063216Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Observation 222459f9-1b3d-46f3-92cd-d33518877bc8 · outbound

This paper cites an unresolved cited work.

Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge Unresolved cited work

Reference 2010

Resolution: unresolved; raw_fallback, observed 2026-08-11T17:59:44.045135Z

Source-reported events for the cited work

No event found in the named queried sources as of 2026-08-12T06:34:41.77262+00:00.

Pith citing papers

No inbound Pith citation observations are available.