Fine-tuning Llama-3-8b and Mistral-7b-v0.3 on LLM-generated QA pairs from IBM Technotes can improve no-context QA scores over training on human-annotated TechQA data, but the evaluation may be inflated by test-document overlap.
Fine-tune the Entire RAG Architecture (including DPR retriever) for Question-Answering
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, we illustrate how to fine-tune the entire Retrieval Augment Generation (RAG) architecture in an end-to-end manner. We highlighted the main engineering challenges that needed to be addressed to achieve this objective. We also compare how end-to-end RAG architecture outperforms the original RAG architecture for the task of question answering. We have open-sourced our implementation in the HuggingFace Transformers library.
citation-role summary
background 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Automatic Dataset Generation for Knowledge Intensive Question Answering Tasks
Fine-tuning Llama-3-8b and Mistral-7b-v0.3 on LLM-generated QA pairs from IBM Technotes can improve no-context QA scores over training on human-annotated TechQA data, but the evaluation may be inflated by test-document overlap.