Using Llama-3.1-8B to generate and score synthetic MCQA data, then distilling those soft labels into DeBERTa-v3-base, improves few-shot MMLU accuracy from 28.9% to 39.3%.
Generating questions and multiple-choice answers using semantic analysis of texts
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
LLM Distillation for Efficient Few-Shot Multiple Choice Question Answering
Using Llama-3.1-8B to generate and score synthetic MCQA data, then distilling those soft labels into DeBERTa-v3-base, improves few-shot MMLU accuracy from 28.9% to 39.3%.