REVIEW 6 cited by
Multi-hop Question Answering
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The task of Question Answering (QA) has attracted significant research interest for long. Its relevance to language understanding and knowledge retrieval tasks, along with the simple setting makes the task of QA crucial for strong AI systems. Recent success on simple QA tasks has shifted the focus to more complex settings. Among these, Multi-Hop QA (MHQA) is one of the most researched tasks over the recent years. In broad terms, MHQA is the task of answering natural language questions that involve extracting and combining multiple pieces of information and doing multiple steps of reasoning. An example of a multi-hop question would be "The Argentine PGA Championship record holder has won how many tournaments worldwide?". Answering the question would need two pieces of information: "Who is the record holder for Argentine PGA Championship tournaments?" and "How many tournaments did [Answer of Sub Q1] win?". The ability to answer multi-hop questions and perform multi step reasoning can significantly improve the utility of NLP systems. Consequently, the field has seen a surge with high quality datasets, models and evaluation strategies. The notion of 'multiple hops' is somewhat abstract which results in a large variety of tasks that require multi-hop reasoning. This leads to different datasets and models that differ significantly from each other and makes the field challenging to generalize and survey. We aim to provide a general and formal definition of the MHQA task, and organize and summarize existing MHQA frameworks. We also outline some best practices for building MHQA datasets. This book provides a systematic and thorough introduction as well as the structuring of the existing attempts to this highly interesting, yet quite challenging task.
Forward citations
Cited by 6 Pith papers
-
Evaluating List Construction and Temporal Understanding capabilities of Large Language Models
A new benchmark shows LLMs give incomplete lists and inaccurate time intervals for temporal list questions, and retrieval helps only partly.
-
MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance
MDBench is a synthetically generated, knowledge-guided benchmark for multi-document QA on which frontier LLMs achieve only about 60% exact match.
-
Automatic Generation of Inference Making Questions for Reading Comprehension Assessments
GPT-4o generates high-quality reading comprehension questions, but only 42.6% correctly target the specified bridging inference type.
-
On Generalization across Measurement Systems: LLMs Entail More Test-Time Compute for Underrepresented Cultures
LLMs are less accurate when asked to report facts in non-default measurement systems, and chain-of-thought restores accuracy only at a 180-300 percent increase in test-time compute.
-
MultiHoax: A Dataset of Multi-hop False-Premise Questions
A new multi-hop false-premise benchmark shows that leading large language models detect embedded falsehoods in only a minority of cases, with the best model reaching about 23% on the full two-stage protocol.
-
Question Decomposition for Retrieval-Augmented Generation
Splitting multi-hop questions into subquestions and reranking the merged retrieval pool improves RAG evidence coverage and answer accuracy on MultiHop-RAG and HotpotQA.
Discussion (0). Sign in to comment.