Pith. sign in

REVIEW 4 major objections 5 minor 40 references

This paper claims that multi-hop question answering fails when a single reasoning strategy is applied to all questions, and that a dual-track framework—dynamically classifying each question as parallel fact-verification or chained reasoning

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 09:14 UTC pith:S6CIIBNU

load-bearing objection DTKG's routing idea is worth testing, but the reported numbers are internally inconsistent (Mintaka differs between Table 2 and Table 3) and a key hyperparameter was tuned on the test sets, so the quantitative claims aren't trustworthy yet. the 4 major comments →

arxiv 2510.16302 v2 pith:S6CIIBNU submitted 2025-10-18 cs.AI cs.IR

DTKG: Dual-Track Knowledge Graph-Verified Reasoning Framework for Multi-Hop QA

Changhao Wang , Yanfang Liu , Xinxin Fan , Ao Tian , Lanzhi Zhou , Yunfeng Lu This is my paper
classification cs.AI cs.IR
keywords multi-hop reasoningquestion answeringknowledge graphretrieval-augmented generationfact verificationLLMdual-process theorydenoising
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that multi-hop QA suffers from a strategy-task mismatch: LLM-based fact-verification works for parallel questions but breaks on chained reasoning, while KG path-construction excels at chains but wastes effort on parallel questions. DTKG is a two-stage framework that first uses a few-shot LLM classifier with five explicit rules to label each question as either parallel fact-verification or chained multi-hop reasoning, then routes it to a branch designed for that type—LLM fact-checking with KG verification for parallel, DFS path construction with early stopping for chained—plus a task-aware denoiser. On HotpotQA, Mintaka, CWQ, and QALD10, DTKG reports accuracy gains from 5% to 17.6% over baselines, with ablation showing that random classification collapses performance and removing the denoiser causes double-digit drops. A sympathetic reader would care because the framework suggests that a small, interpretable classifier plus specialized branches can outperform more complex unified approaches, without retraining the underlying LLM.

Core claim

The core discovery is that the 'strategy-task mismatch'—applying one reasoning method to all multi-hop questions—is a major bottleneck, and that dynamic classification of questions into 'parallel fact-verification' versus 'chained multi-hop reasoning' types, followed by branch-specific processing, materially improves both accuracy and efficiency. DTKG demonstrates this by using a few-shot prompt-based classifier (with five rules and six examples) to label each question, routing parallel questions to an LLM fact-verification pipeline that decomposes answers into atomic claims and verifies each against a knowledge graph, and routing chained questions to a depth-limited DFS path constructor tha

What carries the argument

The central mechanism is the dual-track routing engine. A few-shot prompting-driven task classifier (the 'unconscious' stage) labels each question as parallel or chained using five rules (e.g., shared-entity chain A→B→C means chained; independent comparison means parallel) and six examples. The two processing branches (the 'conscious' stage) both use a two-stage hybrid scoring—embedding cosine similarity followed by a reranking model, fused with a weight α—to select triplets or paths from the knowledge graph. For chained questions, a depth-first expansion with maximum depth 3, width limit W_max, threshold θ, and an information-sufficiency early-stop builds a product-scored path. The task-awa

Load-bearing premise

The load-bearing premise is that every multi-hop question can be cleanly assigned to one of two categories—parallel fact-verification or chained reasoning—by a few-shot LLM classifier, and that the classifier routes questions correctly often enough; the paper's own error analysis lists 'Wrong Type Classification' and 'Hybrid-Category Problem' as failure modes, and no routing accuracy is reported.

What would settle it

Collect a stratified sample of 200 multi-hop questions from the four datasets, have the paper's few-shot classifier label each as parallel or chained, and compare against human annotation. Then run DTKG on the subset where the classifier disagrees with the human label; if accuracy on misrouted questions is not consistently worse than the correct branch alone, the premise that classification quality drives the gain is undermined. A stricter test: on a dataset composed of genuinely hybrid questions (requiring both parallel and chained steps), show that DTKG does not beat either single branch.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Multi-hop QA systems can replace a single monolithic retrieval/reasoning pipeline with a cheap classifier plus two specialized modules and gain accuracy without retraining the LLM.
  • The classification rule set (shared-entity chain vs. independent facts) provides an interpretable, reproducible way to decide which reasoning strategy to use, potentially reducing hallucination from broken chains and redundant path retrieval.
  • The denoiser is a critical component: removing it drops accuracy by up to 14.2 points (CWQ ACC from 90.0 to 75.8; HotpotQA from 85.8 to 75.4), showing that task-aware noise filtering matters alongside strategy selection.
  • Random routing severely degrades performance (HotpotQA ACC falls to 73.0 from 85.8), implying that even imperfect classification is far better than no classification.
  • The framework's improvements are largest on datasets with mixed question types (e.g., Mintaka), suggesting the benefit scales with the heterogeneity of reasoning patterns within a dataset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The binary taxonomy may be too coarse: many real questions mix both patterns, and the paper itself acknowledges a 'Hybrid-Category Problem'. A natural extension is a probabilistic or multi-label router that can split a question or fall back to a combined strategy when the classifier is uncertain.
  • The classifier's reliability is the linchpin, yet the paper reports no classification accuracy. A testable extension is to measure routing accuracy against human labels and to quantify how routing errors propagate to final answer quality.
  • The denoiser's keyword blacklist (ID, source, version, metadata) is brittle; a learned or semantic filter might generalize better, and one could test whether the necessity-score threshold θ transfers across domains.
  • If the few-shot rules are robust, the same dual-track architecture could be applied to other structured-knowledge tasks such as temporal reasoning, multi-hop text retrieval, or question answering over non-KG structured sources.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DTKG, a dual-track framework for multi-hop QA that first uses a few-shot prompted LLM classifier to categorize each question as either 'parallel fact-verification' or 'chained multi-hop reasoning,' then routes the question to one of two KG-based processing branches: a fact-checking branch for parallel questions and a DFS-based chain-construction branch for chained questions. A task-aware denoiser filters administratively irrelevant and contextually unnecessary relations. Experiments on HotpotQA, Mintaka, CWQ, and QALD10-en with a Llama 3:8B backbone report that DTKG achieves the highest Semantic Match Accuracy (ACC) on all four datasets and improves over baselines by 5.0% to 17.6%.

Significance. If the reported results hold, the paper's core idea is appealing: rather than applying a single reasoning strategy to all multi-hop questions, a lightweight classification step can route questions to a strategy matched to their relational-dependency structure. The paper includes useful components: a rule-plus-examples classifier prompt, an ablation showing the penalty of random routing (Table 3), and an error analysis (Section 5). However, the central quantitative claim is currently compromised by an internal inconsistency in the reported Mintaka numbers and by test-set-based hyperparameter selection, so the claimed improvements cannot be taken at face value. The framework deserves consideration, but the evidence needs to be made consistent and reproducible.

major comments (4)
  1. [Tables 2 and 3] The same condition, 'Ours (DTKG)', is reported with different Mintaka results: Table 2 lists EM/ACC 67.6/93.9, while Table 3 lists 67.2/93.2. Since these tables describe the same method, backbone, and setting, at least one set of numbers is not correctly reported or the evaluation protocol changed (e.g., different N, prompt, or seed) without documentation. The abstract's headline improvement and the 'highest ACC on all four datasets' claim depend on which number is correct. This must be resolved with a single reproducible run and an explanation of the discrepancy.
  2. [Appendix A.2.1, Figure 5] The hyperparameter N (top candidate count) is selected by varying N from 20 to 70 and choosing the value that maximizes ACC on the same test datasets (peak 86.0% at N=50). This is a test-set selection procedure, so the final evaluation numbers are partly fitted rather than predicted. The authors should either report performance over the N range, use a held-out validation split, or otherwise justify that the choice of N does not inflate the reported gains.
  3. [Section 3.2 and Section 5] The few-shot classifier is the central component that determines which branch is used, yet no classification accuracy, confusion matrix, or routing error rate is reported. Section 5 explicitly lists 'Wrong Type Classification' and 'Hybrid-Category Problem' as error causes, but their frequency is not quantified. The Random Classification ablation (Table 3) shows the penalty of random routing, but it does not tell the reader how often the actual classifier routes incorrectly. Reporting classifier accuracy per dataset (or per error category) is necessary to assess the framework's robustness and the impact of routing errors.
  4. [Section 4.1 and Equations (4), (9), (13)] Several free parameters that materially affect the pipeline are not reported: the fusion weight α in Eq. (4), the threshold θ used in relation pruning and necessity scoring (Eqs. (9), (13)), the width limit Wmax, the early-stopping threshold τ, and the maximum depth Dmax. Only N is discussed in the appendix. Without these values, the experiments are not reproducible, and it is unclear whether the reported performance is sensitive to their settings.
minor comments (5)
  1. [Abstract and Section 1] The phrase 'significantly improves the performance from 5.0% to 17.6%' does not specify the baseline or the metric (apparently ACC). Please clarify against which baseline and on which datasets these bounds are computed.
  2. [Throughout] Typos: 'ross-redundant' in Section 1 bullet, 'irrelavatnt' in Section 2.3, 'Promt' in Section 3.1 and Figure 2. These should be corrected.
  3. [Appendix A.2.1] The notation is inconsistent: the text switches between 'n' and 'N' for the candidate count. Please standardize.
  4. [Appendix A.2, Table 5] Table 5 reports 'Accuracy' across hop counts but does not state which dataset(s) or method(s) are used, nor the number of questions per hop. Without this context the table is difficult to interpret.
  5. [Section 2.3] The denoiser's 'invalid relation keyword library' K_invalid = {ID, source, version, metadata} is presented in Eq. (12) but with no discussion of coverage or potential false positives (e.g., a relation legitimately about a 'source' citation). A brief discussion of this limitation would strengthen the paper.

Circularity Check

0 steps flagged

No significant circularity; noted test-set hyperparameter selection and an internal inconsistency are correctness issues, not definitional circularity.

full rationale

DTKG is an engineering framework rather than a derivation from first principles. Its components—the few-shot classifier, the parallel fact-checking branch, the chained reasoning branch, and the denoiser—are defined by explicit rules and equations (Table 1; Eqs. 1–13) that do not define any output in terms of the metric it is meant to predict. The classification taxonomy is a design choice, not a self-referential result. The appendix's N=50 ablation (A.2.1, Figure 5) tunes a hyperparameter against ACC; if that ablation used the same test sets, it is evaluation leakage that can inflate the reported gains, but it does not make the comparison logically equivalent to its input. The same-appearing condition is reported with different Mintaka numbers in Table 2 (67.6/93.9) and Table 3 (67.2/93.2), which is a reproducibility/correctness concern, not a circular-reasoning one. There are no self-citations, no imported uniqueness theorems, no ansatz justified solely by prior work of the authors, and no prediction that reduces by construction to a fitted parameter. The paper is evaluated against external baselines on standard datasets, so its central empirical claim, while possibly flawed in execution, is not circular.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The framework rests on the assumption that a small classifier can split all multi-hop questions into two kinds, that Wikidata covers the questions, and that hand-set thresholds/hyperparameters (N=50, Dmax=3, α, θ, Wmax, τ) do not need per-dataset calibration. Several of these are asserted rather than measured, and the paper's own error analysis undermines the clean-dichotomy assumption.

free parameters (6)
  • N (top candidate count) = 50
    Selected by maximizing ACC on the same test datasets (Figure 5), then used for all main results; this is test-set fitting.
  • α (weight in two-stage scoring)
    Defined in Equation (4) but no value is reported; it affects both branches.
  • θ (threshold for relation pruning and necessity scoring)
    Used in Section 3.3.2 and Equation (13); value not specified.
  • Wmax (width limit in DFS expansion)
    Listed as a constraint in Section 3.3.2 but no value is reported.
  • τ (ACC threshold)
    Semantic Match Accuracy threshold is never specified.
  • Dmax (max depth) = 3
    Hand-set to 3 because 'most multi-hop questions require no more than three hops' (Section 3.3.2); no distribution evidence is given.
axioms (5)
  • domain assumption Multi-hop questions are exhaustively and cleanly partitionable into 'parallel fact-verification' and 'chained multi-hop reasoning'.
    Section 3.2 and Table 1 define five rules; the paper's own error analysis (Section 5) lists 'Hybrid-Category Problem' as a failure source, showing the partition is imperfect.
  • domain assumption Wikidata has sufficient coverage for the evaluated questions.
    Section 3.3 relies on Wikidata; Section 5 admits 'Insufficient Information' from inadequate KG coverage for counting questions.
  • domain assumption An LLM few-shot classifier with five rules and six examples yields accurate routing.
    No classifier accuracy is reported; Section 5 identifies 'Wrong Type Classification' as an error category.
  • domain assumption Dmax=3 covers most multi-hop questions.
    Section 3.3.2 sets Dmax=3 with the claim that most questions require ≤3 hops, but no measurement or citation is provided.
  • ad hoc to paper The manually constructed invalid-relation keyword library {ID, source, version, metadata} captures relations that are always irrelevant.
    Section 3.4 defines K_invalid; there is no validation that these keywords are universally safe to drop across question types.

pith-pipeline@v1.3.0-alltime-deepseek · 14615 in / 11527 out tokens · 95291 ms · 2026-08-04T09:14:53.451678+00:00 · methodology

0 comments
read the original abstract

Multi-hop reasoning for question answering (QA) plays a critical role in retrieval-augmented generation (RAG) for modern large language models (LLMs). The accurate answer can be obtained through retrieving relational structure of entities from knowledge graph (KG). Regarding the inherent relation-dependency and reasoning pattern, multi-hop reasoning can be in general classified into two categories: i) parallel fact-verification multi-hop reasoning question, i.e., requiring simultaneous verifications of multiple independent sub-questions; and ii) chained multi-hop reasoning questions, i.e., demanding sequential multi-step inference with intermediate conclusions serving as essential premises for subsequent reasoning. Currently, the multi-hop reasoning approaches singly employ one of two techniques: LLM response-based fact verification and KG path-based chain construction. Nevertheless, the former excels at parallel fact-verification but underperforms on chained reasoning tasks, while the latter demonstrates proficiency in chained multi-hop reasoning but suffers from redundant path retrieval when handling parallel fact-verification reasoning. These limitations deteriorate the efficiency and accuracy for multi-hop QA tasks. To address this challenge, we propose a novel dual-track KG verification and reasoning framework DTKG, which is inspired by the Dual Process Theory in cognitive science. Specifically, DTKG comprises two main stages: the Classification Stage and the Branch Processing Stage.

Figures

Figures reproduced from arXiv: 2510.16302 by Ao Tian, Changhao Wang, Lanzhi Zhou, Xinxin Fan, Yanfang Liu, Yunfeng Lu.

Figure 1
Figure 1. Figure 1: Problem statement and our solution: (a) Fact verification based on post-processing (e.g., KGR); (b) Knowledge graph [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the DTKG framework. An LLM-powered classifier first categorizes the input query into either a "multi-hop [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Error analysis on 100 cases from the four datasets. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of question type across datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Hyperparameter N for Top-N Candidate Selection [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 18 linked inside Pith

  1. [1]

    Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Georg Zachary. 2007. DBpedia: A Nucleus for a Web of Open Data. In Proceedings of the 6th International The Semantic Web and Peer-to-Peer Computing. Springer, 722–735

    2007

  2. [2]

    Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2...

    2020

  3. [3]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebas- tian Gehrmann, et al. 2022. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311(2022)

    Pith/arXiv arXiv 2022

  4. [4]

    Jonathan Evans. 2003. In two minds: Dual-process accounts of reasoning.Trends in Cognitive Sciences7 (2003), 454–459

    2003

  5. [5]

    Jonathan Evans. 2008. Dual-processing accounts of reasoning, judgment, and social cognition.Annual Review of Psychology59 (2008), 255–278

    2008

  6. [6]

    Jonathan St B. T. Evans. 1984. Heuristic and analytic processes in reasoning. British Journal of Psychology75, 4 (1984), 451–468

    1984

  7. [7]

    Zhibin Gou, Zhi-Yong Zhang, Boyang Liu, Weiji Chen, Yujiu Yu, Zhi-Hong Liu, Peitian Li, Jian-Jun Wang, and Xin Sun. 2023. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing.arXiv preprint arXiv:2305.11738 (2023)

    Pith/arXiv arXiv 2023

  8. [8]

    Xinyan Guan, Yanjiang Liu, Hongyu Lin, Yaojie Lu, Ben He, Xianpei Han, and Le Sun. 2024. Mitigating large language model hallucinations via autonomous knowledge graph-based retrofitting. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18126–18134

    2024

  9. [9]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.arXiv preprintarXiv:2005.11401 (2021). https://arxiv.org/abs/2005.11401 NeurIPS 2020

    Pith/arXiv arXiv 2021

  10. [10]

    Feng-Lin Li, Minghui Qiu, Haiqing Chen, Xiongwei Wang, Xing Gao, Jun Huang, Juwei Ren, Zhongzhou Zhao, Weipeng Zhao, Lei Wang, Guwei Jin, and Wei Chu. 2017. Alime Assist: An intelligent assistant for creating an in- novative e-commerce experience. InProceedings of the 2017 ACM on Confer- ence on Information and Knowledge Management. ACM, Singapore, 2495–2...

    arXiv 2017

  11. [11]

    Johnny Li, Saksham Consul, Eda Zhou, James Wong, Naila Farooqui, Yuxin Ye, Nithyashree Manohar, Zhuxiaona Wei, Tian Wu, Ben Echols, et al . 2024. Banishing LLM hallucinations requires rethinking generalization.arXiv preprint arXiv:2406.17642(2024)

    Pith/arXiv arXiv 2024

  12. [12]

    Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the middle: How language models use long contexts.arXiv preprint arXiv:2307.03172(2023)

    Pith/arXiv arXiv 2023

  13. [13]

    Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2021. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.arXiv preprint arXiv:2104.08786(2021)

    Pith/arXiv arXiv 2021

  14. [14]

    2023.GPT-4 technical report

    OpenAI. 2023.GPT-4 technical report. Technical Report. OpenAI

    2023

  15. [15]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback.arXiv preprint arXiv:2203.02155(2022)

    Pith/arXiv arXiv 2022

  16. [16]

    Priyanka Sen, Alham Fikri Aji, and Amir Saffari. 2022. Mintaka: A complex, natural, and multilingual dataset for end-to-end question answering.arXiv preprint arXiv:2210.01613(2022)

    Pith/arXiv arXiv 2022

  17. [17]

    Haitian Sun, Tania Bedrax-Weiss, and William W Cohen. 2019. Pullnet: Open domain question answering with iterative retrieval on knowledge bases and text. arXiv preprint arXiv:1904.09537(2019)

    Pith/arXiv arXiv 2019

  18. [18]

    Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M Ni, Heung-Yeung Shum, and Jian Guo. 2023. Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv preprint arXiv:2307.07697(2023)

    Pith/arXiv arXiv 2023

  19. [19]

    Alon Talmor and Jonathan Berant. 2018. ComplexWebQuestions: A Dataset for Complex Question Answering over Knowledge Bases and Free Text. InNAACL- HLT

    2018

  20. [20]

    Wen tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic parsing via staged query graph generation: Question answering with knowledge base. InProceedings of the 53rd Annual Meeting of the Association for Computa- tional Linguistics and the 7th International Joint Conference on Natural Language Processing. Association for Computational Li...

    2015

  21. [21]

    Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kul- shreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. LaMDA: Language models for dialog applications.arXiv preprint arXiv:2201.08239 (2022)

    Pith/arXiv arXiv 2022

  22. [22]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    Pith/arXiv arXiv 2023

  23. [23]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

    Pith/arXiv arXiv 2023

  24. [24]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal

  25. [25]

    Ricardo Usbeck, Xi Yan, Aleksandr Perevalov, Longquan Jiang, Julius Schulz, Angelie Kraft, Cedric Moller, Junbo Huang, Jan Reineke, Axel-Cyrille Ngonga Ngomo, et al. [n. d.]. QALD-10–the 10th challenge on question answering over linked data.Semantic Web(Preprint) ([n. d.])

  26. [26]

    Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase.Commun. ACM57, 10 (2014), 78–85

    2014

  27. [27]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems35 (2022), 24824–24837

    2022

  28. [28]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for di- verse, explainable multi-hop question answering.arXiv preprint arXiv:1809.09600 (2018)

    Pith/arXiv arXiv 2018

  29. [29]

    Weiqi Ye, Qiang Zhang, Xian Zhou, Wenpeng Hu, Changhai Tian, and Jiajun Cheng. 2024. Correcting Factual Errors in LLMs via Inference Paths Based on Knowledge Graph. In2024 International Conference on Computational Linguistics and Natural Language Processing (CLNLP). IEEE, 12–16

    2024

  30. [30]

    Wenhao Yu, Chenguang Zhu, Zhihan Zhang, Shuohang Wang, Zhuosheng Zhang, Yuwei Fang, and Meng Jiang. 2022. Retrieval augmentation for commonsense reasoning: A unified approach.arXiv preprint arXiv:2210.12887(2022)

    Pith/arXiv arXiv 2022

  31. [31]

    Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, and Jian Guo. 2023. Noisy pair corrector for dense retrieval.arXiv preprint arXiv:2311.03798(2023)

    Pith/arXiv arXiv 2023

  32. [32]

    Taolin Zhang, Chengyu Wang, Nan Hu, Minghui Qiu, Chengguang Tang, Xi- aofeng He, and Jun Huang. 2022. DKPLM: decomposable knowledge-enhanced pre-trained language model for natural language understanding. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11703–11711

    2022

  33. [33]

    Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu

  34. [34]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models.arXiv preprint arXiv:2210.03493 (2022)

    Pith/arXiv arXiv 2022

  35. [35]

    sweet spot,

    Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. InInternational conference on machine learning. PMLR, 12697–12706. DTKG: Dual-Track Knowledge Graph-Verified Reasoning Framework for Multi-Hop QA WWW, April 13 – 17, 2026, Dubai, UAE A More Results of Experiments ...

    2021

  36. [38]

    yes" or

    Answer ONLY with "yes" or "no"

  37. [39]

    Only classify as "yes" if it requires connecting facts through shared intermediary entities (A→B→C)

  38. [40]

    Where was the CEO of Microsoft born?

    Explicitly classify as "no" for these cases: - Direct single-entity attribute queries (age, birthplace) - Comparisons between independent entities (who is taller/older) - Multiple independent facts about the same entity - Simple relations that can be answered with one triplet (A→B) Examples: Q: "Where was the CEO of Microsoft born?"→yes (Microsoft→CEO→bir...

  39. [2019]

    ERNIE: Enhanced language representation with informative entities.arXiv preprint arXiv:1905.07129(2019)

    Pith/arXiv arXiv 1905

  40. [2022]

    Transactions of the Association for Computational Linguistics10 (2022), 539–554

    MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics10 (2022), 539–554. doi:10.1162/tacl_a_00475

    doi:10.1162/tacl_a_00475 2022