Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that the known 'lost in the middle' failure of long-context language models is not only about where a single piece of evidence sits relative to the context edges.

desk verdict A useful extension of lost-in-the-middle to multi-hop QA with a plausible but not yet isolated inter-evidence distance effect; needs a reanalysis before the headline claim is accepted. read the letter →

arxiv 2412.10079 v1 pith:R36JBDSD submitted 2024-12-13 cs.CL

George Arthur Baker , Ankush Raut , Sagi Shaier , Lawrence E Hunter , Katharina von der Wense This is my paper · ORCID
classification cs.CL
keywords lostinthemiddlemulti-hopquestionansweringlong-contextlanguagemodelsevidencepositioninter-evidencedistancechain-of-thoughtpromptingretrieval-augmentedgenerationcontextreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the known 'lost in the middle' failure of long-context language models is not only about where a single piece of evidence sits relative to the context edges. In multi-hop question answering, where an answer requires combining evidence from several documents, performance also drops when the relevant documents are far apart from each other, even when each one individually is in a favorable position. If this is right, systems that feed many retrieved documents to a model must care how supporting evidence is arranged, not just whether it is present.

What carries the argument

The central object is the set of evidence-position combinations in a fixed 20-document prompt: five adjacent placements such as (1,2) and (10,11), plus separated placements such as (1,5) and (10,15) for two-hop questions, with parallel schemes for three- and four-hop questions. Comparing the averaged accuracy of adjacent versus separated placements is what isolates the relative distance between evidence documents from their absolute positions. The argument rests on treating this comparison as the measurable signature of how well models integrate disconnected evidence.

What would settle it

Run the same models on a random sample of all possible evidence-position pairs in a 20-document context and check whether, after controlling for each evidence document's distance from the context edges, accuracy still decreases as the gap between the two evidence documents increases; if it does not, the inter-evidence distance claim fails.

Watch

Extended reading notes

Core claim

The central discovery is that multi-hop QA accuracy degrades with the distance between multiple pieces of evidence, not just with their distance from the edges of the input. The paper fixes a 20-document context, places the gold evidence documents either adjacent to each other or separated by distractor documents, and measures best-subspan accuracy on HotpotQA, 2WikiMultihopQA, and MuSiQue-Ans. Across instruction-tuned models, adjacent evidence consistently outperforms separated evidence. The paper also reports that chain-of-thought prompting helps instruction-tuned models identify relevant documents but does not erase the positional disparity, and that context reduction via summarization or knowledge-graph triples flattens the position curve at the cost of overall accuracy.

Load-bearing premise

The paper's evidence-distance claim depends on the nine hand-picked position combinations being representative of all possible placements; if those configurations are not representative or are confounded with absolute position, the averaging that isolates distance would not establish the effect.

Editorial extensions

If this is right

  • Retrieval-augmented generation systems should consider the spacing between supporting passages, not only their top-k relevance scores, when ordering documents for a prompt.
  • Re-ranking methods designed for single-hop 'lost in the middle' may not transfer to multi-hop settings because the number of relevant orderings grows combinatorially with the number of hops.
  • Chain-of-thought prompting can lift overall accuracy for instruction-tuned models but should not be expected to remove position-based performance gaps.
  • Context compression can reduce the model's sensitivity to evidence position, but only at the price of losing information, so it is not a free fix.
  • The distance effect should be treated as a benchmark target for long-context models: a model that reasons robustly over dispersed evidence should show a small adjacent-versus-separated gap.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If inter-evidence distance is causal, then accuracy should fall monotonically as the gap between two evidence documents grows, a prediction the paper's selected configurations only partially test; a systematic sweep over all gaps would sharpen or refute it.
  • The finding suggests a 'lost in-between' component interacts with attention mechanisms: models may need a second retrieval or a mid-context 'bridge' summary to reconnect separated evidence, which is a testable engineering response.
  • The results imply that evaluation protocols for long-context reasoning should report evidence-spacing conditions, otherwise two systems with identical average accuracy could differ sharply in how they handle dispersed evidence.
  • For non-instruction-tuned models, the paper's observation that summaries and triples improve performance hints that these models may be failing on surface-level format rather than reasoning, a possibility worth probing with controlled prompts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies the "lost in the middle" phenomenon in multi-hop question answering, where multiple relevant documents must be integrated. The authors position two or more gold documents among 20 documents and measure accuracy for three models (GPT-3.5-turbo, MPT-7b, Llama-2-longlora) on HotpotQA, 2WikiMultihopQA, and MuSiQue. They report a new effect: performance degrades not only with the absolute position of evidence but also with the relative distance between relevant documents. They also test chain-of-thought prompting and two context-reduction methods (knowledge-graph triple extraction and summarization). The paper includes full result tables in an appendix and releases code and data.

Significance. If the central claim is correct, it extends the known absolute-position bias to a pairwise layout effect, which has direct implications for retrieval-augmented generation over multi-hop questions. The study is useful as a systematic benchmark: the position conditions are pre-specified, the evaluation uses standard public datasets and a standard metric, generation uses temperature 0, and the code and data are released. These are real strengths. However, the central claim is currently supported by an aggregate comparison that does not control for absolute position and by qualitative inspection of point estimates without uncertainty quantification, so the significance of the contribution is not yet firmly established.

major comments (3)
  1. [Section 6, Figure 4] The headline claim that inter-evidence distance degrades performance is not isolated from absolute position. The adjacent set in Figure 4 contains edge pairs (1,2) and (19,20), while the separated set contains no pair in which both documents are at the same edge; given the known U-shaped absolute-position effect (Liu et al., 2024), a group-mean difference can arise from this distributional imbalance even if spacing has no causal effect. No significance tests or confidence intervals are reported anywhere in the paper. The data also contain direct counterexamples: in MuSiQue 4-hop for GPT-3.5 (Table 3), adjacent condition (15,16,17,18) gives 51.72% while separated condition (14,16,18,20) gives 53.20%, and in 2Wiki 4-hop for MPT (Table 4), the adjacent and separated averages are essentially identical (51.67% vs. 51.68%). The authors should redo the analysis position-matched (e.g., comparing (1,2) with (1,5), (5,6) with (5,10), etc.) or fit a regression with controls for absolute positions, and report uncertainty.
  2. [Section 4, Methodology] The choice of position combinations is a small, hand-picked subset: 5 adjacent and 4 separated for 2-hop and 4-hop, and 3 separated for 3-hop, out of 190, 1140, and 4845 possible combinations, respectively. No justification is given for why these particular combinations are representative of the full space, and the paper's own Limitations section concedes that the analysis "may not capture the full extent of potential variations." Because the aggregate adjacent-versus-separated difference is the sole basis for the central claim, the subset selection could be driving the result. A more defensible approach would be to exhaustively evaluate all combinations for the 2-hop case and to sample uniformly (with reported variance) for higher-hop cases, or at least to show that the chosen subset is balanced on absolute-position margins.
  3. [Section 6, Figure 4] The claimed distance effect is not consistent across the reported conditions. The paper states that "models perform better when relevant documents are adjacent compared to when they are separated," but the Appendix tables show multiple cells where the opposite holds: besides the GPT MuSiQue 4-hop case noted above, MPT on 2Wiki 4-hop with standard prompts shows separated (14,16,18,20)=53.00% outperforming every adjacent condition except (1,2,3,4)=52.42%, and GPT on 2Wiki 4-hop with KG+CoT has separated (1,3,5,7)=66.62% versus adjacent (17,18,19,20)=64.13%. The aggregate plot hides this variability. The authors should report per-condition comparisons and statistical tests of the interaction between spacing and absolute position, rather than only group means.
minor comments (4)
  1. [Figure 1] In panel (a), the document labels repeat "Document [3]" before "Document [2]"; this appears to be a typo and should be corrected.
  2. [Figure 4 caption] The terms "Adjacent Documents" and "Separated Documents" are not precisely defined in the caption; please state which position pairs/triples are included in each group, since the reader must otherwise infer them from Section 4.
  3. [Appendix A, Tables 3-5] The tabular layout separates hop counts by row blocks inside the same table, which is hard to parse. Using separate tables per hop count or adding explicit horizontal rules would improve readability. Also, the column headers repeat the position tuples without a separator from the prompt type; consider adding a subheader row.
  4. [Section 3.1.1 and Table 5] For Llama-2-longlora, the MuSiQue 3-hop and 4-hop results are all 0.00% in every condition (Table 5). The paper relegates these to the appendix with a note about poor performance, but it would be useful to explicitly state that floor effects make these conditions uninformative for the distance analysis, rather than including them in any summary statement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an observational benchmark evaluation with pre-specified positions, standard metrics, and no fitted parameters.

full rationale

The paper is an empirical benchmark study, not a derivation. It evaluates public multi-hop QA datasets (HotpotQA, 2WikiMultihopQA, MuSiQue-Ans) using standard best-subspan accuracy, with evidence-document positions selected ahead of time (e.g., adjacent tuples such as (1,2) and separated tuples such as (1,5)) rather than fitted to model outputs. The central adjacent-versus-separated comparison in Figure 4 is a grouping of these pre-specified conditions; whether that grouping is confounded with absolute position is a statistical validity concern, not a circularity. Context-reduction methods are off-the-shelf (BART-large-CNN summarization and LLaMA-2-based KG triple extraction) and are applied without fitting parameters to the reported results. The paper's self-citations to Shaier et al. appear in related-work discussions and are not load-bearing for the central positional and distance claims, which are measured directly from model outputs under pre-registered conditions. No equation reduces to an input, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from prior work. The paper is therefore self-contained as an empirical evaluation, and no circular step is present.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The experiment depends mainly on dataset annotations and evaluation choices; there are no fitted constants. The assumptions above define the validity boundary of the measured effects.

assumptions (5)
  • domain assumption Best-subspan accuracy (output contains the annotated answer) is a sufficient measure of multi-hop reasoning quality.
    The metric ignores whether the model used the intended reasoning chain or merely guessed from partial evidence. Section 3.1.2.
  • domain assumption The hand-picked subset of evidence position combinations is representative of the full combinatorial space.
    Section 4 states the subset is chosen to manage computational cost; if unrepresentative, the measured distance effect may not generalize.
  • domain assumption Gold evidence documents are correctly annotated and distractor documents are irrelevant.
    The position experiment assumes all needed information is in the gold documents and none in distractors; any leakage confounds the effect. Section 4.
  • domain assumption The second half of each public validation set is an unbiased evaluation split.
    Because official test sets are private, the paper splits validation sets; this split may differ from the test distribution. Section 3.1.
  • domain assumption LLaMA-2-based triple extraction and BART summaries preserve the logical connections needed for multi-hop reasoning.
    The mitigation comparison treats reduced contexts as equivalent information carriers; if they break reasoning chains, the accuracy drop is an artifact of the reduction method. Section 3.1.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA." pith.science (2026). https://pith.science/paper/R36JBDSD

@misc{pith2026241210079,
  author       = {Pith},
  title        = {Pith review of: Lost in the Middle, and In-Between: Enhancing Language Models' Ability to Reason Over Long Contexts in Multi-Hop QA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/R36JBDSD}},
  note         = {Machine review of arXiv:2412.10079}
}
read the original abstract

Previous work finds that recent long-context language models fail to make equal use of information in the middle of their inputs, preferring pieces of information located at the tail ends which creates an undue bias in situations where we would like models to be equally capable of using different parts of the input. Thus far, the problem has mainly only been considered in settings with single pieces of critical information, leading us to question what happens when multiple necessary pieces of information are spread out over the inputs. Here, we demonstrate the effects of the "lost in the middle" problem in the multi-hop question answering setting -- in which multiple reasoning "hops" over disconnected documents are required -- and show that performance degrades not only with respect to the distance of information from the edges of the context, but also between pieces of information. Additionally, we experiment with means of alleviating the problem by reducing superfluous document contents through knowledge graph triple extraction and summarization, and prompting models to reason more thoroughly using chain-of-thought prompting.

Figures

Figures reproduced from arXiv: 2412.10079 by the authors.

Figure 1
Figure 1. Question Answering setups with documents [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The performance impacts of varying the positions of relevant documents within instruction-tuned models’ [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Experimental results for Llama-2-7b-longlora [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average question-answering accuracy for full document prompts by distance setting for GPT and MPT [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Isolated Capabilities: Bridging Long CoT Reasoning and Long-Context Understanding

    cs.CL 2025-07 conditional novelty 5.0 of 10

    DeepSeek-R1-distilled models show higher multi-document QA accuracy than their base counterparts and flatter position-bias curves, especially with 50-80 documents.

  2. Evaluation Hallucination in Multi-Round Incomplete Information Lateral-Driven Reasoning Tasks

    cs.CL 2025-05 conditional novelty 5.0 of 10

    LLM-as-judge scoring of multi-round lateral thinking tasks can be fooled by answer leakage and question substitution, so response-based metrics may overstate reasoning ability.

  3. Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Ordering retrieved documents along the reasoning chain and replacing the causal mask with a prefix mask during LoRA fine-tuning improves multi-hop QA accuracy; peak attention scores can select the best context order.

Reference graph

Works this paper leans on

37 extracted references · 7 canonical work pages · cited by 3 Pith papers

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, and Jian-Guang Lou. 2024. Make your llm fully utilize the context. arXiv preprint arXiv:2404.16811

    arXiv 2024

  4. [4]

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. 2023. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307

    arXiv 2023

  5. [5]

    Tri Dao. 2023. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691

    arXiv 2023

  6. [6]

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R \'e . 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 35:16344--16359

    2022

  7. [7]

    Shangbin Feng, Vidhisha Balachandran, Yuyang Bai, and Yulia Tsvetkov. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.59 F act KB : Generalizable factuality evaluation using language models enhanced with factual knowledge . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 933--952, Singapore. Association f...

    doi:10.18653/v1/2023.emnlp-main.59 2023

  8. [8]

    Wangzhen Guo, Qinkang Gong, and Hanjiang Lai. 2022. http://arxiv.org/abs/2210.07138 Counterfactual multihop qa: A cause-effect approach for reducing disconnected reasoning

    work page Pith review arXiv 2022

Show all 37 references
  1. [9]

    Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. Advances in neural information processing systems, 28

    2015

  2. [10]

    Gautier Izacard and Edouard Grave. 2020. http://arxiv.org/abs/2007.01282 Leveraging passage retrieval with generative models for open domain question answering . CoRR, abs/2007.01282

    2020 arXiv

  3. [11]

    Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to learn long-tail knowledge. In International Conference on Machine Learning, pages 15696--15707. PMLR

    2023

  4. [12]

    Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. https://openreview.net/forum?id=w4DW6qkRmt Sure: Improving open-domain question answering of LLM s via summarized retrieval . In The Twelfth International Confer...

    2024

  5. [13]

    Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav...

    2019 doi

  6. [14]

    Kyungjae Lee, Seung won Hwang, Sang eun Han, and Dohyeon Lee. 2021. http://arxiv.org/abs/2107.03242 Robustifying multi-hop qa through pseudo-evidentiality training

    2021 arXiv

  7. [15]

    Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same task, more tokens: the impact of input length on the reasoning performance of large language models. arXiv preprint arXiv:2402.14848

    2024 arXiv

  8. [16]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. http://arxiv.org/abs/1910.13461 BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehe...

    2019 arXiv

  9. [17]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157--173

    2024

  10. [18]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:2212.10511

    2022 arXiv

  11. [19]

    Vaibhav Mavi, Anubhav Jangra, and Adam Jatowt. 2024. http://arxiv.org/abs/2204.09140 Multi-hop question answering

    2024 arXiv

  12. [20]

    Alexander Peysakhovich and Adam Lerer. 2023. http://arxiv.org/abs/2310.01427 Attention sorting combats recency bias in long context language models

    2023 arXiv

  13. [21]

    Pouya Pezeshkpour. 2023. http://arxiv.org/abs/2306.06264 Measuring and modifying factual knowledge in large language models

    2023 arXiv

  14. [22]

    Ofir Press, Noah Smith, and Mike Lewis. 2022. https://openreview.net/forum?id=R8sQPpGCv0 Train short, test long: Attention with linear biases enables input length extrapolation . In International Conference on Learning Representations

    2022

  15. [23]

    Apoorv Saxena, Aditay Tripathi, and Partha Talukdar. 2020. https://doi.org/10.18653/v1/2020.acl-main.412 Improving multi-hop question answering over knowledge graphs using knowledge base embeddings . In Proceedings of the 58th Annual Meeting of the Association for Computationa...

    2020 doi

  16. [24]

    Sagi Shaier, Kevin Bennett, Lawrence Hunter, and Katharina Kann. 2023 a . https://aclanthology.org/2023.ijcnlp-main.36 Emerging challenges in personalized medicine: Assessing demographic effects on biomedical question answering systems . In Proceedings of the 13th Internationa...

    2023

  17. [25]

    Sagi Shaier, Kevin Bennett, Lawrence Hunter, and Katharina von der Wense. 2024 a . https://aclanthology.org/2024.eacl-long.46 Comparing template-based and template-free language model probing . In Proceedings of the 18th Conference of the European Chapter of the Association fo...

    2024

  18. [26]

    Sagi Shaier, Lawrence Hunter, and Katharina Kann. 2023 b . https://aclanthology.org/2023.ijcnlp-short.13 Who are all the stochastic parrots imitating? they should tell us! In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conf...

    2023

  19. [27]

    Sagi Shaier, Lawrence Hunter, and Katharina Wense. 2024 b . https://aclanthology.org/2024.eacl-long.47 Desiderata for the context use of question answering systems . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics ...

    2024

  20. [28]

    Sagi Shaier, Lawrence Hunter, and Katharina Wense. 2024 c . https://doi.org/10.18653/v1/2024.findings-acl.491 It is not about what you say, it is about how you say it: A surprisingly simple approach for improving reading comprehension . In Findings of the Association for Compu...

    2024 doi

  21. [29]

    Sagi Shaier, Ari Kobren, and Philip V. Ogren. 2024 d . https://doi.org/10.18653/v1/2024.emnlp-main.956 Adaptive question answering: Enhancing language model proficiency for addressing knowledge conflicts with source citations . In Proceedings of the 2024 Conference on Empirica...

    2024 doi

  22. [30]

    Xin Su, Tiep Le, Steven Bethard, and Phillip Howard. 2024. http://arxiv.org/abs/2311.08505 Semi-structured chain-of-thought: Integrating multiple sources of knowledge for improved language model reasoning

    2024 arXiv

  23. [31]

    Raphael Tang, Xinyu Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Ture. 2023. http://arxiv.org/abs/2310.07712 Found in the middle: Permutation self-consistency improves listwise ranking in large language models

    2023 arXiv

  24. [32]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  25. [33]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.712 Is multihop QA in DiRe condition? measuring and reducing disconnected reasoning . In Proceedings of the 2020 Conference on Empirical Methods in Nat...

    2020 doi

  26. [34]

    Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023. http://arxiv.org/abs/2310.07521 Survey on factuality in large l...

    2023 arXiv

  27. [35]

    Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Nenkov Georgiev, Rocktim Jyoti Das, and Preslav Nakov. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1088 Factuality of large language models: A survey . In Proceedings of the 2024 Conference on Empirical M...

    2024 doi

  28. [36]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. https://doi.org/10.18653/v1/D18-1259 H otpot QA : A dataset for diverse, explainable multi-hop question answering . In Proceedings of the 2018 Conference...

    2018 doi

  29. [37]

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023. http://arxiv.org/abs/2205.10625 Least-to-most prompting enables complex reasoning in large language models

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.