Smaller language models can generalize to unseen compositional questions when trained and evaluated with retrieval-augmented contexts, and combining Wikipedia retrieval with LLM-generated rationales improves accuracy.
Better Retrieval May Not Lead to Better Question Answering
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Considerable progress has been made recently in open-domain question answering (QA) problems, which require Information Retrieval (IR) and Reading Comprehension (RC). A popular approach to improve the system's performance is to improve the quality of the retrieved context from the IR stage. In this work we show that for StrategyQA, a challenging open-domain QA dataset that requires multi-hop reasoning, this common approach is surprisingly ineffective -- improving the quality of the retrieved context hardly improves the system's performance. We further analyze the system's behavior to identify potential reasons.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Teaching Smaller Language Models To Generalise To Unseen Compositional Questions (Full Thesis)
Smaller language models can generalize to unseen compositional questions when trained and evaluated with retrieval-augmented contexts, and combining Wikipedia retrieval with LLM-generated rationales improves accuracy.