MINTQA provides 28,366 multi-hop QA pairs across popular/unpopular and old/new knowledge, with sub-questions, and shows that even the best LLMs achieve only about 62% accuracy even with retrieval.
Learning to Recover Reasoning Chains for Multi-Hop Question Answering via Cooperative Games
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
We propose the new problem of learning to recover reasoning chains from weakly supervised signals, i.e., the question-answer pairs. We propose a cooperative game approach to deal with this problem, in which how the evidence passages are selected and how the selected passages are connected are handled by two models that cooperate to select the most confident chains from a large set of candidates (from distant supervision). For evaluation, we created benchmarks based on two multi-hop QA datasets, HotpotQA and MedHop; and hand-labeled reasoning chains for the latter. The experimental results demonstrate the effectiveness of our proposed approach.
citation-role summary
citation-polarity summary
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Tail Knowledge
MINTQA provides 28,366 multi-hop QA pairs across popular/unpopular and old/new knowledge, with sub-questions, and shows that even the best LLMs achieve only about 62% accuracy even with retrieval.