LookAlike: Consistent Distractor Generation in Math MCQs

Alexander Scarlatos; Andrew Lan; Nigel Fernandez; Nisarg Parikh; Simon Woodhead

REVIEW 3 major objections 3 minor 1 cited by

LookAlike: Consistent Distractor Generation in Math MCQs

T0 review · 3 major / 3 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LookAlike shows that a model's own generation inconsistencies can serve as preference data, improving error–distractor consistency in math MCQs and beating prior methods by up to 10.5%.

desk verdict A clean, reproducible DPO recipe for math distractor generation that improves error consistency, but the headline distractor gains are measured with the same exact-match criterion used to build the training pairs, so the 'consistency' claim is weaker than it looks. read the letter →

arxiv 2505.01903 v2 pith:7V7IEDWW submitted 2025-05-03 cs.LG cs.AI

Nisarg Parikh , Nigel Fernandez , Alexander Scarlatos , Simon Woodhead , Andrew Lan This is my paper · ORCID

classification cs.LGcs.AI

keywords mathMCQgenerationdistractorerrorpreferenceoptimizationdirecteducationalNLPLLM-as-a-judgeconsistency

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LookAlike claims that the main obstacle to good math MCQ distractors is not generating plausible student errors but keeping the distractor faithful to the error that is supposed to produce it. The paper turns this failure mode into a training signal: the model overgenerates distractors for each input error, marks exact matches with the ground-truth distractor as preferred, and treats all non-matching generations as dispreferred. These synthetic preference pairs drive Direct Preference Optimization, with the SFT objective interleaved to prevent the quality collapse that often follows DPO. On 1,434 real middle-school math MCQs, the method raises distractor-generation exact-match accuracy from 45.6% to 51.6% and error-generation agreement with an LLM judge from 47.7% to 57.2%, outperforming DiVERT. The result matters because it offers a label-light, scalable route to making automatically generated educational content internally consistent.

What carries the argument

The load-bearing object is a preference dataset mined from the generator's own outputs, combined with an alternating SFT/DPO schedule. For each (question, error) input, the generator produces several candidate distractors; exact string match against the ground-truth distractor splits them into preferred and dispreferred responses, and the same construction is applied symmetrically to error generation using the ground-truth error. DPO then optimizes the log-likelihood margin between preferred and dispreferred outputs, while the interleaved SFT steps periodically pull the model back toward the human-written target distribution. The preference set is regenerated with the current model after each epoch, making the negative examples increasingly aligned with the model's live failure modes. This loop is what carries the paper's claim: it repeatedly confronts the model with its own inconsistencies and ranks them away.

What would settle it

Build a test set in which each error admits two or more distinct but equally consistent distractors; if LookAlike, trained under its exact-match labeling, performs no better than SFT when graded by a similarity-based judge that accepts all valid distractors, then the reported consistency gains depend on surface-form reproduction rather than genuine error–answer alignment.

Watch

Extended reading notes

Core claim

LookAlike establishes that a model's own generation inconsistencies can serve as preference data for distractor and error generation. Starting from an SFT-warmed generator, the method over-samples candidate distractors for a given error, labels exact string matches to the ground-truth distractor as preferred and all other candidates as dispreferred, and then runs DPO on this automatically mined set. To counter DPO's tendency to degrade generation quality over epochs, LookAlike alternates between DPO and SFT updates, recomputing the preference set with the current model after each epoch. Across five cross-validation folds, the resulting model improves distractor generation by 6.8% over SFT and 5.92% over DiVERT, and error generation by 10.5% and 9.46% respectively, all statistically significant. The authors report that the synthetic-preference model is comparable to a variant trained on human-annotated preference pairs (51.56% vs 51.44% distractor, 57.18% vs 57.02% error), suggesting the mined inconsistencies carry most of the useful signal.

Load-bearing premise

The method assumes that any generated distractor that does not exactly match the single ground-truth distractor is a dispreferred response, even if it is a mathematically valid and plausible wrong answer phrased differently.

Editorial extensions

If this is right

Distractor and error generation improve together: the same preference-mined loop lifts both tasks, indicating that error–distractor consistency is a shared bottleneck.
The method matches the performance of DPO trained on human-written preference pairs, so synthetic inconsistency mining can substitute for expensive ground-truth preference annotation.
Alternating SFT with DPO outperforms joint regularization (RPO) and penalty-based regularization (DPOP), giving a simple, hyperparameter-light recipe for stabilizing DPO.
Because LookAlike consistently outperforms DiVERT and SFT across five folds with statistically significant margins, the consistency gains are not an artifact of a single train/test split.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The exact-match preference rule implicitly teaches the model to reproduce the ground-truth wording; a similarity-based or equivalence-based negative criterion could retain the consistency gains while better tolerating paraphrase-valid distractors.
The alternating SFT/DPO schedule is a generic stabilization technique: it could be dropped into other DPO applications where models over-optimize and drift out of distribution, not just distractor generation.
The inconsistency-mining loop is architecture-agnostic: as long as a task has a ground-truth target that can be compared for equivalence, the same self-generated negative pairing could be applied to other educational generation tasks such as feedback or explanation generation.
A direct comparison with DPO-GT suggests the main barrier is not annotation quality but the design of the preference objective; future work could explore preference pairs constructed from error and distractor models jointly, as the paper itself notes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing to know: this paper does something genuinely useful and simple—use the model's own failures to create DPO preference pairs, and alternate SFT and DPO instead of combining losses. It works, if you accept the metric. But the headline 6.8% gain over DiVERT on distractor generation is measured with the same exact-string-match criterion used to label the preference pairs. That makes part of the "consistency" gain a surface-form effect, not necessarily a conceptual one.

What's actually new: DPO for distractor generation isn't new (DPO-GT is cited). The new bits are (a) mining negatives from the model's own overgenerations with exact-match labels, and (b) the alternating SFT/DPO schedule. The ablations show the alternating schedule is what drives most of the gain: LookAlike without regularization gets 47.98 distractor accuracy, with per-epoch alternation it gets 51.56. That's a real, reproducible finding. Code is on GitHub, the dataset is real (1,400+ MCQs from a live learning platform), and the evaluation uses 5-fold CV with Wilcoxon signed-rank tests.

The soft spots, in rough order of severity. First, the distractor metric is circular: exact match is both the training signal and the test metric. In middle-school math, many wrong answers are mathematically equivalent but written differently (5.2 vs 26/5, or algebraically equivalent expressions). The paper honestly calls the criterion "conservative and intentionally strict" in the Limitations, but that means the 51.56 number may reflect the model learning to reproduce the gold distractor's wording rather than to generate a plausible distractor consistent with the error. The human evaluation covers only error generation, not distractors, so we never see how much of the distractor gain is real. Second, the LLM-as-judge for errors shows only weak-to-moderate agreement with human annotators (linear kappa 0.31–0.56), which weakens the error-generation claims as well. Third, LookAlike is statistically indistinguishable from DPO-GT (51.56 vs 51.44 on distractors, 57.18 vs 57.02 on errors), which uses human-written preference pairs. The paper is transparent about this, but the abstract's emphasis on inconsistency mining as the enabler overstates a tenth-of-a-point advantage. Fourth, no per-fold variance is reported, only cross-validated averages.

Bottom line: the method is worth knowing about, the alternating SFT/DPO idea is useful and works, and the code is a good starting point for anyone working on LLMs for education. But the main claim—improved consistency—is only as good as the exact-match metric, which is exactly the metric the training procedure is optimized to hit. A serious referee should ask for a soft-match distractor evaluation, human evaluation of generated distractors, and per-fold numbers.

For you: if you work in this area, read it and probably cite it. The limitations are real but addressable. I'd send this to a workshop or conference with a strong revise recommendation.

Referee Report

3 major / 3 minor

Summary. This paper introduces LookAlike, a preference-optimization method for generating math MCQ errors and distractors that are mutually consistent. The method (i) mines synthetic preference pairs by overgenerating candidate errors/distractors from a fine-tuned LLM and labeling any output that exactly matches the ground-truth text as preferred and any non-match as dispreferred, and (ii) trains the model by alternating supervised fine-tuning and Direct Preference Optimization. Experiments on 1,434 real-world math MCQs with five-fold cross-validation report absolute exact-match distractor accuracy of 51.6% and LLM-judged error accuracy of 57.2%, outperforming SFT and DiVERT baselines, while roughly matching a DPO baseline that uses ground-truth preference pairs (DPO-GT).

Significance. If the central claims hold, the work would contribute a scalable recipe for distractor/error consistency without manual preference annotation, and the alternating SFT/DPO schedule could be a useful stabilization technique. The paper is clearly written, ships code, and uses a consistent cross-validation protocol with LoRA for all methods. However, the evaluation's dependence on exact match for distractors and on a weakly human-correlated LLM judge for errors currently leaves the central consistency claim under-supported.

major comments (3)

[Section 3.1, Section 4.3] The preference-mining rule and the distractor evaluation metric are the same exact-match criterion, so the reported distractor-generation gains may reflect surface-form memorization rather than error–distractor consistency. A generated alternative such as '26/5' for a ground-truth '5.2' would be scored as a failure and actively repelled during training, even if it is mathematically equivalent and perfectly consistent with the input error. The paper's own Limitations concedes that exact match is 'conservative and intentionally strict,' but the headline 6.8% improvement over DiVERT is computed entirely on this strict criterion; without a human evaluation of distractor consistency or an equivalence-class metric, the central claim that LookAlike improves consistency is not empirically established.
[Section 5.3] The human evaluation shows only weak agreement between the GPT-4o-mini judge and human annotators (linear kappa 0.314–0.556), and the human study covers only error generation, not distractor generation. Since the error-generation numbers in Table 3 (including the claimed 10.5% and 9.46% gains over SFT and DiVERT) rely on this LLM judge, the headline error-generation improvements are not robustly validated; the paper should either replace the judge with a more reliable protocol or present distractor-level human judgments.
[Section 5.1, Table 4] The ablation does not isolate the contribution of the inconsistency-mining component. Without alternating optimization, LookAlike's distractor accuracy is 47.98, only 0.30 points above DPO-GT without regularization (47.68), and its error-generation score is 49.34, substantially below DPO-GT's 53.96. With per-epoch alternating optimization, LookAlike (51.56/56.64) is statistically indistinguishable from DPO-GT (51.44/57.02), yet no significance test between the two is reported. This suggests the gains over DiVERT may be driven primarily by the alternating SFT/DPO schedule rather than by mined preference pairs, and the paper should present a controlled comparison (same regularization, varying only the preference source) and report effect sizes.

minor comments (3)

[Section 5.1] The one-sample Wilcoxon signed-rank test over only five folds is very low-powered; the paper should also report per-fold results, effect sizes, and confidence intervals to support the significance claim.
[Section 3.2] The alternating schedule in Equations (2) and (3) uses 'even' batch/epoch numbers, but the precise handling of the SFT phases after DPO phases (e.g., whether SFT reinitializes or continues from DPO weights, and how the preference dataset is recomputed) is only partially described; a fuller specification would improve reproducibility.
[Appendix D] The qualitative 'progression' in error types across methods is presented as a general pattern, but with only two example questions; this should be labeled as illustrative rather than as systematic evidence.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the distractor metric overlaps with the training labels but is not a circular derivation.

full rationale

No load-bearing circular step meets the required bar. The preference pairs in Section 3.1 are labeled by exact string match to human-written ground-truth distractors and errors, and the distractor evaluation metric in Section 4.3 is also exact match. This overlap means the distractor-generation gains partly reflect directly optimizing the evaluation criterion, but the test set is held out, the ground-truth labels are independent of the model, and all baselines are evaluated under the same metric, so the comparison is not forced by construction. The Limitations section explicitly acknowledges that exact match is 'conservative and intentionally strict,' which is a validity caveat rather than a circular step. Self-citations such as DiVERT and Scarlatos et al. are used as baselines or related work and are not load-bearing; the consistency bottleneck is also illustrated with examples in Table 1. The error-generation evaluation uses an LLM-as-judge, which is not identical to the exact-match training label, further indicating that the central claims are not reducible to the training inputs. Score 2 reflects the minor metric-overlap concern, not a finding of circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical entities or mathematical objects are introduced. The method's free parameters are standard hyperparameters. The key load-bearing assumptions are the exact-match labeling and the validity of the LLM judge, both acknowledged by the authors.

free parameters (6)

DPO beta = 0.5
Set in Appendix A for all DPO runs; not tuned per method, but a free hyperparameter.
RPO lambda = 0.005
Taken from prior work (Liu et al., 2024b), not re-tuned.
DPOP lambda = 0.1
Set in Appendix A from DPOP paper.
Number of synthetic generations per sample = 3 distractors and 3 errors per epoch
Chosen to match DPO-GT sample count, but still a hand-chosen value that affects results.
LoRA rank and alpha = r=128, alpha=256
Chosen for parameter efficiency; fixed across all methods.
Learning rate for DPO and SFT in alternating optimization = 5e-6
Set in Appendix A; not ablated.

assumptions (4)

domain assumption Exact string match is a valid proxy for distractor consistency.
Section 3.1 uses exact match to label preference pairs; the Limitations section acknowledges this is conservative and strict. If exact match is too strict, many plausible distractors are labeled dispreferred, skewing DPO.
domain assumption GPT-4o-mini can judge mathematical equivalence of error explanations.
The error-generation metric in Table 3 relies entirely on LLM-as-a-judge with GPT-4o-mini. The human evaluation shows only moderate-to-low agreement with this judge, so the metric validity is uncertain.
domain assumption The MetaMath-Mistral 7B model is a suitable base prior for math error reasoning.
The paper states it 'provides a suitable prior' without presenting comparative evidence across base models. The method's success depends on the base model being good enough to surface meaningful inconsistencies.
domain assumption Alternating SFT and DPO per batch/epoch stabilizes DPO training better than combining losses.
Empirical claim backed by ablation Table 4, but limited to this dataset and model. No theoretical grounding; could fail in other settings.

how reviews work

0 comments

Cite this review

Pith. "Pith review of LookAlike: Consistent Distractor Generation in Math MCQs." pith.science (2026). https://pith.science/paper/7V7IEDWW

@misc{pith2026250501903,
  author       = {Pith},
  title        = {Pith review of: LookAlike: Consistent Distractor Generation in Math MCQs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7V7IEDWW}},
  note         = {Machine review of arXiv:2505.01903}
}

read the original abstract

Large language models (LLMs) are increasingly used to generate distractors for multiple-choice questions (MCQs), especially in domains like math education. However, existing approaches are limited in ensuring that the generated distractors are consistent with common student errors. We propose LookAlike, a method that improves error-distractor consistency via preference optimization. Our two main innovations are: (a) mining synthetic preference pairs from model inconsistencies, and (b) alternating supervised fine-tuning (SFT) with Direct Preference Optimization (DPO) to stabilize training. Unlike prior work that relies on heuristics or manually annotated preference data, LookAlike uses its own generation inconsistencies as dispreferred samples, thus enabling scalable and stable training. Evaluated on a real-world dataset of 1,400+ math MCQs, LookAlike achieves 51.6% accuracy in distractor generation and 57.2% in error generation under LLM-as-a-judge evaluation, outperforming an existing state-of-the-art method (45.6% / 47.7%). These improvements highlight the effectiveness of preference-based regularization and inconsistency mining for generating consistent math MCQ distractors at scale.

Figures

Figures reproduced from arXiv: 2505.01903 by the authors.

**Figure 1.** LOOKALIKE creates preference pairs by overgenerating a set of distractors for a question and error, and preferring those that match the ground-truth distractor exactly. An analogous process for error generation. Error Generator (to generate explanations) Dispreferred Error Ground-truth Error Ground-truth Error Training with SFT Training with DPO Inputs: Question Distractor Target Outputs: < Target Output: Inputs: Qu… view at source ↗

**Figure 2.** LOOKALIKE employs an alternating optimization strategy, switching between SFT and DPO objectives to regularize DPO training. create an arbitrary number of dynamic preference pairs, with negative preference signals being more aligned with the inconsistency failure modes of the fine-tuned LLM. 3.2 DPO Regularization Through Alternating Optimization We empirically observe that models trained with DPO deteriorate in qu… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

A Semi-supervised Generative Model for Incomplete Multi-view Data Integration with Missing Labels
cs.LG 2025-08 unverdicted novelty 5.0 of 10

A semi-supervised generative model for incomplete multi-view data with missing labels is proposed in the abstract, but the submitted text is an unrelated paper, leaving the claims unverifiable.

Reference graph

Works this paper leans on

59 extracted references · 50 canonical work pages · cited by 1 Pith paper

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Peter Airasian. 2001. Classroom assessment: Concepts and applications. McGraw-Hill, Ohio, USA

work page 2001
[4]

Sheng, Wei Emma Zhang, Munazza Zaib, and Ahoud Alhazmi

Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Munazza Zaib, and Ahoud Alhazmi. 2024. Distractor generation in multiple-choice tasks: A survey of methods, datasets, and evaluation

work page 2024
[5]

Nischal Ashok Kumar and Andrew Lan. 2024. https://aclanthology.org/2024.bea-1.10/ Improving socratic question generation using data augmentation and preference optimization . In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 108--118, Mexico City, Mexico. Association for Computational Linguistics

work page 2024
[6]

Semere Kiros Bitew, Johannes Deleu, Chris Develder, and Thomas Demeester. 2023. Distractor generation for multiple-choice questions with predictive prompting and large language models

work page 2023
[7]

Xinyi Chen, Baohao Liao, Jirui Qi, Panagiotis Eustratiadis, Christof Monz, Arianna Bisazza, and Maarten de Rijke. 2024. The SIF o benchmark: Investigating the sequential instruction following ability of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics

work page 2024
[8]

Ho-Lam Chung, Ying-Hong Chan, and Yao-Chung Fan. 2020. A BERT -based distractor generation scheme with multi-tasking and negative answer training strategies. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics

work page 2020

Show all 59 references

[9]

Vanya Cohen, Geraud Nangue Tasse, Nakul Gopalan, Steven James, Matthew Gombolay, Ray Mooney, and Benjamin Rosman. 2025. Compositional instruction following with language models and reinforcement learning

2025
[10]

Soham Dan, Xinran Han, and Dan Roth. 2021. Compositional data and task augmentation for instruction following. In Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics

2021
[11]

Bisni Fahad Mon, Asma Wasfi, Mohammad Hayajneh, Ahmad Slim, and Najah Abu Ali. 2023. Reinforcement learning in education: A literature review. Informatics

2023
[12]

Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, and Andrew Lan. 2024. Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Compu...

2024
[13]

Nigel Fernandez, Alexander Scarlatos, Wanyong Feng, Simon Woodhead, and Andrew Lan. 2024. D i VERT : Distractor generation with variational errors represented as text for math multiple-choice questions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Lang...

2024
[14]

Ippei Fujisawa, Sensho Nobe, Hiroki Seto, Rina Onda, Yoshiaki Uchida, Hiroki Ikoma, Pei-Chun Chien, and Ryota Kanai. 2024. Procbench: Benchmark for multi-step reasoning and following procedure

2024
[15]

Juyeon Heo, Miao Xiong, Christina Heinze-Deml, and Jaya Narain. 2024. Do LLM s estimate uncertainty well in instruction-following? In Neurips Safe Generative AI Workshop 2024

2024
[16]

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations

2022
[17]

Joel Jang, Seonghyeon Ye, and Minjoon Seo. 2022. Can large language models truly follow your instructions? In NeurIPS ML Safety Workshop

2022
[18]

Tom Kubiszyn and Gary Borich. 2016. Educational testing and measurement. John Wiley and Sons, New Jersey, USA

2016
[19]

Yooseop Lee, Suin Kim, and Yohan Jo. 2025. Generating plausible distractors for multiple-choice questions via student choice prediction

2025
[20]

Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. 2023. Synthetic data generation with large language models for text classification: Potential and limitations. In The 2023 Conference on Empirical Methods in Natural Language Processing

2023
[21]

Lee Giles

Chen Liang, Xiao Yang, Neisarg Dave, Drew Wham, Bart Pursel, and C. Lee Giles. 2018. Distractor generation for multiple choice questions using learning to rank. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications . Associati...

2018
[22]

Chin-Yew Lin. 2004. ROUGE : A package for automatic evaluation of summaries. In Text Summarization Branches Out. Association for Computational Linguistics

2004
[23]

Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. 2024 a . Conflict-averse gradient descent for multi-task learning

2024
[24]

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G -eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

2023
[25]

Zhihan Liu, Miao Lu, Shenao Zhang, Boyi Liu, Hongyi Guo, Yingxiang Yang, Jose Blanchet, and Zhaoran Wang. 2024 b . https://openreview.net/forum?id=2cQ3lPhkeO Provably mitigating overoptimization in RLHF : Your SFT loss is implicitly an adversarial regularizer . In The Thirty-e...

2024
[26]

Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

2019
[27]

Renze Lou, Kai Zhang, and Wenpeng Yin. 2024. Large language model instruction following: A survey of progresses and challenges

2024
[28]

Haohao Luo, Yang Deng, Ying Shen, See-Kiong Ng, and Tat-Seng Chua. 2024. Chain-of-exemplar: Enhancing distractor generation for multimodal educational question generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...

2024
[29]

Anthony J. Nitko. 1996. Educational assessment of students. Prentice-Hall, Iowa, USA

1996
[30]

Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

2022
[31]

Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. 2024. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402.13228

2024 arXiv
[32]

Ryan Park, Rafael Rafailov, Stefano Ermon, and Chelsea Finn. 2024. Disentangling length from quality in direct preference optimization

2024
[33]

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. https://openreview.net/forum?id=HPuSIXJaa9 Direct preference optimization: Your language model is secretly a reward model . In Thirty-seventh Conference on Neural Infor...

2023
[34]

Siyu Ren and Kenny Q. Zhu. 2021. Knowledge-driven distractor generation for cloze-style multiple choice questions. Proceedings of the AAAI Conference on Artificial Intelligence

2021
[35]

Denise Rey and Markus Neuh \"a user. 2011. Wilcoxon-Signed-Rank Test. Springer Berlin Heidelberg

2011
[36]

Alexander Scarlatos, Wanyong Feng, Digory Smith, Simon Woodhead, and Andrew Lan. 2024 a . https://aclanthology.org/2024.bea-1.19/ Improving automated distractor generation for math multiple-choice questions with overgenerate-and-rank . In Proceedings of the 19th Workshop on In...

2024
[37]

Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, and Andrew Lan. 2025. https://arxiv.org/abs/2503.06424 Training llm-based tutors to improve student learning outcomes in dialogues . Preprint, arXiv:2503.06424

2025 arXiv
[38]

Alexander Scarlatos, Digory Smith, Simon Woodhead, and Andrew Lan. 2024 b . Improving the validity of automatically generated feedback via reinforcement learning. In Artificial Intelligence in Education, pages 280--294, Cham. Springer Nature Switzerland

2024
[39]

Shivanshu Shekhar, Shreyas Singh, and Tong Zhang. 2024. See-dpo: Self entropy enhanced direct preference optimization

2024
[40]

Guangyuan Shi, Qimai Li, Wenlong Zhang, Jiaxin Chen, and Xiao-Ming Wu. 2023. Recon: Reducing conflicting gradients from the root for multi-task learning. In The Eleventh International Conference on Learning Representations

2023
[41]

Jinnie Shin, Qi Guo, and Mark J. Gierl. 2019. Multiple-choice item distractor development using topic modeling approaches. Frontiers in Psychology, Volume 10 - 2019

2019
[42]

Mark G Simkin and William L Kuechler. 2005. Multiple-choice tests and student understanding: What is the connection? Decision Sciences Journal of Innovative Education, 3(1):73--98

2005
[43]

Guijin Son, SangWon Baek, Sangdae Nam, Ilgyun Jeong, and Seungone Kim. 2024. Multi-task inference: Can large language models follow multiple instructions at once? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)...

2024
[44]

Shashank Sonkar, Naiming Liu, MyCo Le, and Richard Baraniuk. 2024 a . Malalgoqa: Pedagogical evaluation of counterfactual reasoning in large language models and implications for ai in education. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15...

2024
[45]

Shashank Sonkar, Kangqi Ni, Sapana Chaudhary, and Richard Baraniuk. 2024 b . Pedagogical alignment of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024

2024
[46]

Katherine Stasaski and Marti A. Hearst. 2017. Multiple choice question generation utilizing an ontology. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications . Association for Computational Linguistics

2017
[47]

Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation and synthesis: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Nat...

2024
[48]

McKee, Lisa Wang, Markus Kunesch, Mike Schaekermann, and 27 others

LearnLM Team, Abhinit Modi, Aditya Srikanth Veerubhotla, Aliya Rysbek, Andrea Huber, Brett Wiltshire, Brian Veprek, Daniel Gillick, Daniel Kasenberg, Derek Ahmed, Irina Jurenka, James Cohan, Jennifer She, Julia Wilkowski, Kaiz Alarakyia, Kevin R. McKee, Lisa Wang, Markus Kunes...

2024
[49]

Danielle R Thomas, Conrad Borchers, Sanjit Kakarla, Jionghao Lin, Shambhavi Bhushan, Boyuan Guo, Erin Gatz, and Kenneth R Koedinger. 2025. Does multiple choice have a future in the age of generative ai? a posttest-only rct. In Proceedings of the 15th International Learning Ana...

2025
[50]

Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra

Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R. Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. 2018. https://arxiv.org/abs/1610.02424 Diverse beam search: Decoding diverse solutions from neural sequence models . Preprint, arXiv:1610.02424

2018 arXiv
[51]

Tianduo Wang and Wei Lu. 2023. Learning multi-step reasoning by solving arithmetic tasks

2023
[52]

Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxing Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. 2024. Benchmarking complex instruction-following with multiple constraints composition. In The Thirty-eight...

2024
[53]

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and ...

2020
[54]

Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. 2024. Is dpo superior to ppo for llm alignment? a comprehensive study. In Proceedings of the 41st International Conference on Machine Learning

2024
[55]

Yuzi Yan, Yibo Miao, Jialian Li, YipinZhang, Jian Xie, Zhijie Deng, and Dong Yan. 2025. https://openreview.net/forum?id=9Hxdixed7p 3d-properties: Identifying challenges in DPO and charting a path forward . In The Thirteenth International Conference on Learning Representations

2025
[56]

Han Cheng Yu, Yu An Shih, Kin Man Law, KaiYu Hsieh, Yu Chen Cheng, Hsin Chih Ho, Zih An Lin, Wen-Chuan Hsu, and Yao-Chung Fan. 2024 a . Enhancing distractor generation for multiple-choice questions with retrieval augmented pretraining and knowledge graph integration. In Findin...

2024
[57]

Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024 b . Metamath: Bootstrap your own mathematical questions for large language models

2024
[58]

Weinberger, and Yoav Artzi

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert

2020
[59]

Gonzalez, and Ion Stoica

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM -as-a-judge with MT -bench and chatbot arena. In Thirty-seventh Conference on Neur...

2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Peter Airasian. 2001. Classroom assessment: Concepts and applications. McGraw-Hill, Ohio, USA

work page 2001

[4] [4]

Sheng, Wei Emma Zhang, Munazza Zaib, and Ahoud Alhazmi

Elaf Alhazmi, Quan Z. Sheng, Wei Emma Zhang, Munazza Zaib, and Ahoud Alhazmi. 2024. Distractor generation in multiple-choice tasks: A survey of methods, datasets, and evaluation

work page 2024

[5] [5]

Nischal Ashok Kumar and Andrew Lan. 2024. https://aclanthology.org/2024.bea-1.10/ Improving socratic question generation using data augmentation and preference optimization . In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 108--118, Mexico City, Mexico. Association for Computational Linguistics

work page 2024

[6] [6]

Semere Kiros Bitew, Johannes Deleu, Chris Develder, and Thomas Demeester. 2023. Distractor generation for multiple-choice questions with predictive prompting and large language models

work page 2023

[7] [7]

Xinyi Chen, Baohao Liao, Jirui Qi, Panagiotis Eustratiadis, Christof Monz, Arianna Bisazza, and Maarten de Rijke. 2024. The SIF o benchmark: Investigating the sequential instruction following ability of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics

work page 2024

[8] [8]

Ho-Lam Chung, Ying-Hong Chan, and Yao-Chung Fan. 2020. A BERT -based distractor generation scheme with multi-tasking and negative answer training strategies. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics

work page 2020