A fine-tuned Llama model generates multiple-choice distractors that keep model rankings nearly unchanged (Spearman 0.99) and matched confidence entropy, while human scores are only reported on a separate set of tasks.
Knowledge-Driven Distractor Generation for Cloze-style Multiple Choice Questions
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, we propose a novel configurable framework to automatically generate distractive choices for open-domain cloze-style multiple-choice questions, which incorporates a general-purpose knowledge base to effectively create a small distractor candidate set, and a feature-rich learning-to-rank model to select distractors that are both plausible and reliable. Experimental results on datasets across four domains show that our framework yields distractors that are more plausible and reliable than previous methods. This dataset can also be used as a benchmark for distractor generation in the future.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
D-GEN: Automatic Distractor Generation and Evaluation for Reliable Assessment of Generative Model
A fine-tuned Llama model generates multiple-choice distractors that keep model rankings nearly unchanged (Spearman 0.99) and matched confidence entropy, while human scores are only reported on a separate set of tasks.