{"id":"624ee21c-0227-4451-a125-3f1756a3b1e9","arxiv_id":"2505.01903","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LookAlike mines inconsistent model generations as negative preference pairs and alternates SFT with DPO to improve error-distractor consistency in math MCQ generation.","lead":"The authors introduce LookAlike, a method that trains LLMs to generate math MCQ distractors and error explanations that are consistent with each other, using preference optimization on synthetic inconsistencies. It reports improved generation quality over prior SFT and DiVERT baselines on a real-world dataset of 1,434 middle-school math MCQs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Distractor gains may stem from exact-match surface-form memorization rather than error-consistency, since the same strict criterion builds the training labels and the evaluation metric.","rationale":"The reader's weakest_assumption correctly identifies the exact-match labeling in Section 3.1 as a core risk, but the more load-bearing issue is that the same exact-match criterion also defines the evaluation metric for the distractor-generation task. This creates a closed loop: the method is trained to reproduce the exact ground-truth distractor string and then evaluated on that same string. Since the paper's central contribution is framed as 'consistency' rather than surface-form reproduction, the evaluation should measure consistency independently of formatting. The absence of any human or semantic evaluation of distractor consistency leaves the central claim resting entirely on a metric that the authors themselves describe as intentionally strict. The reader already assigned CONDITIONAL due to metric-validity concerns on the error-generation side; my analysis extends the same concern to the distractor side, reinforcing the need for conditionality but not changing the verdict. The proposed concrete test is a feasible, decisive check: re-score the existing generated distractors with a semantic-equivalence judge. This does not require retraining and would directly show whether the 6.8% exact-match gain translates into a genuine consistency gain. If the semantic margin remains large and significant, the paper's claim is substantially strengthened; if not, the abstract's 'accuracy' framing overstates the result.","tokens_in":16805,"tokens_out":4164,"duration_ms":42588,"concrete_test":"Re-evaluate all test-set distractor predictions from LookAlike, SFT, and DiVERT with a semantic-equivalence metric: a math-aware judge (e.g., GPT-4o-mini with a prompt like the one in Table 7, or a symbolic checker) decides whether each generated distractor is mathematically equivalent to the ground-truth distractor AND consistent with the specified error, ignoring formatting differences. Report semantic-equivalence accuracy for each method and the pairwise margins with significance tests. If the LookAlike advantage over SFT/DiVERT shrinks or vanishes under this metric, the headline distractor-consistency claim is an artifact of exact-match evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that LookAlike improves error-distractor consistency, yet distractor-generation performance is measured exclusively by exact string match against a single human-written ground-truth distractor (Section 4.3), and the preference miner in Section 3.1 uses the same exact-match rule to label generated distractors as preferred or dispreferred. In math MCQs, a given error can often produce multiple mathematically equivalent distractors that differ in surface form (e.g., 5.2 vs. 26/5, 0.5 vs. 1/2, or algebraically equivalent expressions). Every such valid generation is scored as a failure and, during training, actively repelled. The paper's own Limitation notes that exact match is 'conservative and intentionally strict,' but the headline 6.8% distractor-generation improvement over DiVERT is computed entirely on this strict criterion. The human evaluation (Section 5.3) only covers error generation, not distractor consistency; furthermore, the LLM-as-judge used for error generation shows weak agreement with human annotators (linear kappa 0.314–0.556). Consequently, the reported distractor-consistency gains may partly reflect learning to reproduce the exact surface form of the ground-truth distractor rather than generating any plausible distractor that is consistent with the input error. This threatens the abstract's claim that the improvements 'highlight the effectiveness of preference-based regularization and inconsistency mining for generating consistent math MCQ distractors.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces LookAlike, a preference-optimization method for generating math MCQ errors and distractors that are mutually consistent. The method (i) mines synthetic preference pairs by overgenerating candidate errors/distractors from a fine-tuned LLM and labeling any output that exactly matches the ground-truth text as preferred and any non-match as dispreferred, and (ii) trains the model by alternating supervised fine-tuning and Direct Preference Optimization. Experiments on 1,434 real-world math MCQs with five-fold cross-validation report absolute exact-match distractor accuracy of 51.6% and LLM-judged error accuracy of 57.2%, outperforming SFT and DiVERT baselines, while roughly matching a DPO baseline that uses ground-truth preference pairs (DPO-GT).","tokens_in":17043,"tokens_out":6257,"duration_ms":58932,"significance":"If the central claims hold, the work would contribute a scalable recipe for distractor/error consistency without manual preference annotation, and the alternating SFT/DPO schedule could be a useful stabilization technique. The paper is clearly written, ships code, and uses a consistent cross-validation protocol with LoRA for all methods. However, the evaluation's dependence on exact match for distractors and on a weakly human-correlated LLM judge for errors currently leaves the central consistency claim under-supported.","major_comments":[{"comment":"The preference-mining rule and the distractor evaluation metric are the same exact-match criterion, so the reported distractor-generation gains may reflect surface-form memorization rather than error–distractor consistency. A generated alternative such as '26/5' for a ground-truth '5.2' would be scored as a failure and actively repelled during training, even if it is mathematically equivalent and perfectly consistent with the input error. The paper's own Limitations concedes that exact match is 'conservative and intentionally strict,' but the headline 6.8% improvement over DiVERT is computed entirely on this strict criterion; without a human evaluation of distractor consistency or an equivalence-class metric, the central claim that LookAlike improves consistency is not empirically established.","section":"Section 3.1, Section 4.3"},{"comment":"The human evaluation shows only weak agreement between the GPT-4o-mini judge and human annotators (linear kappa 0.314–0.556), and the human study covers only error generation, not distractor generation. Since the error-generation numbers in Table 3 (including the claimed 10.5% and 9.46% gains over SFT and DiVERT) rely on this LLM judge, the headline error-generation improvements are not robustly validated; the paper should either replace the judge with a more reliable protocol or present distractor-level human judgments.","section":"Section 5.3"},{"comment":"The ablation does not isolate the contribution of the inconsistency-mining component. Without alternating optimization, LookAlike's distractor accuracy is 47.98, only 0.30 points above DPO-GT without regularization (47.68), and its error-generation score is 49.34, substantially below DPO-GT's 53.96. With per-epoch alternating optimization, LookAlike (51.56/56.64) is statistically indistinguishable from DPO-GT (51.44/57.02), yet no significance test between the two is reported. This suggests the gains over DiVERT may be driven primarily by the alternating SFT/DPO schedule rather than by mined preference pairs, and the paper should present a controlled comparison (same regularization, varying only the preference source) and report effect sizes.","section":"Section 5.1, Table 4"}],"minor_comments":[{"comment":"The one-sample Wilcoxon signed-rank test over only five folds is very low-powered; the paper should also report per-fold results, effect sizes, and confidence intervals to support the significance claim.","section":"Section 5.1"},{"comment":"The alternating schedule in Equations (2) and (3) uses 'even' batch/epoch numbers, but the precise handling of the SFT phases after DPO phases (e.g., whether SFT reinitializes or continues from DPO weights, and how the preference dataset is recomputed) is only partially described; a fuller specification would improve reproducibility.","section":"Section 3.2"},{"comment":"The qualitative 'progression' in error types across methods is presented as a general pattern, but with only two example questions; this should be labeled as illustrative rather than as systematic evidence.","section":"Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The authors' claim of a 'wide margin' over DiVERT is technically correct, but the near-identical performance to their own DPO-GT baseline should be foregrounded in the framing; the contribution is better stated as removing the need for human preference labels rather than as a raw accuracy gain. The exact-match training/evaluation circularity is the main correctness risk."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper does something genuinely useful and simple—use the model's own failures to create DPO preference pairs, and alternate SFT and DPO instead of combining losses. It works, if you accept the metric. But the headline 6.8% gain over DiVERT on distractor generation is measured with the same exact-string-match criterion used to label the preference pairs. That makes part of the \"consistency\" gain a surface-form effect, not necessarily a conceptual one.\n\nWhat's actually new: DPO for distractor generation isn't new (DPO-GT is cited). The new bits are (a) mining negatives from the model's own overgenerations with exact-match labels, and (b) the alternating SFT/DPO schedule. The ablations show the alternating schedule is what drives most of the gain: LookAlike without regularization gets 47.98 distractor accuracy, with per-epoch alternation it gets 51.56. That's a real, reproducible finding. Code is on GitHub, the dataset is real (1,400+ MCQs from a live learning platform), and the evaluation uses 5-fold CV with Wilcoxon signed-rank tests.\n\nThe soft spots, in rough order of severity. First, the distractor metric is circular: exact match is both the training signal and the test metric. In middle-school math, many wrong answers are mathematically equivalent but written differently (5.2 vs 26/5, or algebraically equivalent expressions). The paper honestly calls the criterion \"conservative and intentionally strict\" in the Limitations, but that means the 51.56 number may reflect the model learning to reproduce the gold distractor's wording rather than to generate a plausible distractor consistent with the error. The human evaluation covers only error generation, not distractors, so we never see how much of the distractor gain is real. Second, the LLM-as-judge for errors shows only weak-to-moderate agreement with human annotators (linear kappa 0.31–0.56), which weakens the error-generation claims as well. Third, LookAlike is statistically indistinguishable from DPO-GT (51.56 vs 51.44 on distractors, 57.18 vs 57.02 on errors), which uses human-written preference pairs. The paper is transparent about this, but the abstract's emphasis on inconsistency mining as the enabler overstates a tenth-of-a-point advantage. Fourth, no per-fold variance is reported, only cross-validated averages.\n\nBottom line: the method is worth knowing about, the alternating SFT/DPO idea is useful and works, and the code is a good starting point for anyone working on LLMs for education. But the main claim—improved consistency—is only as good as the exact-match metric, which is exactly the metric the training procedure is optimized to hit. A serious referee should ask for a soft-match distractor evaluation, human evaluation of generated distractors, and per-fold numbers.\n\nFor you: if you work in this area, read it and probably cite it. The limitations are real but addressable. I'd send this to a workshop or conference with a strong revise recommendation.","headline":"A clean, reproducible DPO recipe for math distractor generation that improves error consistency, but the headline distractor gains are measured with the same exact-match criterion used to build the training pairs, so the 'consistency' claim is weaker than it looks.","tokens_in":17649,"tokens_out":2929,"would_cite":true,"duration_ms":29900,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LookAlike shows that a model's own generation inconsistencies can serve as preference data, improving error–distractor consistency in math MCQs and beating prior methods by up to 10.5%.","keywords":["math MCQ generation","distractor generation","error generation","preference optimization","direct preference optimization","educational NLP","LLM-as-a-judge","consistency"],"falsifier":"Build a test set in which each error admits two or more distinct but equally consistent distractors; if LookAlike, trained under its exact-match labeling, performs no better than SFT when graded by a similarity-based judge that accepts all valid distractors, then the reported consistency gains depend on surface-form reproduction rather than genuine error–answer alignment.","tokens_in":16559,"feed_emoji":"🧮","tokens_out":9775,"duration_ms":84706,"temperature":0.7,"pith_summary":"LookAlike claims that the main obstacle to good math MCQ distractors is not generating plausible student errors but keeping the distractor faithful to the error that is supposed to produce it. The paper turns this failure mode into a training signal: the model overgenerates distractors for each input error, marks exact matches with the ground-truth distractor as preferred, and treats all non-matching generations as dispreferred. These synthetic preference pairs drive Direct Preference Optimization, with the SFT objective interleaved to prevent the quality collapse that often follows DPO. On 1,434 real middle-school math MCQs, the method raises distractor-generation exact-match accuracy from 45.6% to 51.6% and error-generation agreement with an LLM judge from 47.7% to 57.2%, outperforming DiVERT. The result matters because it offers a label-light, scalable route to making automatically generated educational content internally consistent.","feed_headline":"Self-mined mistakes train math MCQ generators to stay consistent","feed_subtitle":"It lifts distractor accuracy from 45.6% to 51.6% without extra human labels.","key_machinery":"The load-bearing object is a preference dataset mined from the generator's own outputs, combined with an alternating SFT/DPO schedule. For each (question, error) input, the generator produces several candidate distractors; exact string match against the ground-truth distractor splits them into preferred and dispreferred responses, and the same construction is applied symmetrically to error generation using the ground-truth error. DPO then optimizes the log-likelihood margin between preferred and dispreferred outputs, while the interleaved SFT steps periodically pull the model back toward the human-written target distribution. The preference set is regenerated with the current model after each epoch, making the negative examples increasingly aligned with the model's live failure modes. This loop is what carries the paper's claim: it repeatedly confronts the model with its own inconsistencies and ranks them away.","core_discovery":"LookAlike establishes that a model's own generation inconsistencies can serve as preference data for distractor and error generation. Starting from an SFT-warmed generator, the method over-samples candidate distractors for a given error, labels exact string matches to the ground-truth distractor as preferred and all other candidates as dispreferred, and then runs DPO on this automatically mined set. To counter DPO's tendency to degrade generation quality over epochs, LookAlike alternates between DPO and SFT updates, recomputing the preference set with the current model after each epoch. Across five cross-validation folds, the resulting model improves distractor generation by 6.8% over SFT and 5.92% over DiVERT, and error generation by 10.5% and 9.46% respectively, all statistically significant. The authors report that the synthetic-preference model is comparable to a variant trained on human-annotated preference pairs (51.56% vs 51.44% distractor, 57.18% vs 57.02% error), suggesting the mined inconsistencies carry most of the useful signal.","pith_inferences":["The exact-match preference rule implicitly teaches the model to reproduce the ground-truth wording; a similarity-based or equivalence-based negative criterion could retain the consistency gains while better tolerating paraphrase-valid distractors.","The alternating SFT/DPO schedule is a generic stabilization technique: it could be dropped into other DPO applications where models over-optimize and drift out of distribution, not just distractor generation.","The inconsistency-mining loop is architecture-agnostic: as long as a task has a ground-truth target that can be compared for equivalence, the same self-generated negative pairing could be applied to other educational generation tasks such as feedback or explanation generation.","A direct comparison with DPO-GT suggests the main barrier is not annotation quality but the design of the preference objective; future work could explore preference pairs constructed from error and distractor models jointly, as the paper itself notes."],"forward_implications":["Distractor and error generation improve together: the same preference-mined loop lifts both tasks, indicating that error–distractor consistency is a shared bottleneck.","The method matches the performance of DPO trained on human-written preference pairs, so synthetic inconsistency mining can substitute for expensive ground-truth preference annotation.","Alternating SFT with DPO outperforms joint regularization (RPO) and penalty-based regularization (DPOP), giving a simple, hyperparameter-light recipe for stabilizing DPO.","Because LookAlike consistently outperforms DiVERT and SFT across five folds with statistically significant margins, the consistency gains are not an artifact of a single train/test split."],"supporting_citations":[{"why":"Supplies the DiVERT baseline and the variational error-space approach that LookAlike must beat.","marker":"(Fernandez et al., 2024)"},{"why":"Provides the Direct Preference Optimization algorithm used to train on mined preference pairs.","marker":"(Rafailov et al., 2023)"},{"why":"Supplies MetaMath-Mistral 7B, the base LLM used for all methods in the experiments.","marker":"(Yu et al., 2024b)"},{"why":"Describes Regularized Preference Optimization (RPO), a joint SFT/DPO baseline the paper compares against for training stability.","marker":"(Liu et al., 2024b)"},{"why":"Describes DPO-Positive (DPOP), an SFT-penalty regularization baseline the paper compares against for training stability.","marker":"(Pal et al., 2024)"}],"fun_headline_variants":["LookAlike: self-mined mistakes boost math MCQ distractors by 6%","Your model's own errors train it to write better math MCQs","LookAlike mines its own slips to teach consistent math MCQ generation","6% better distractors: LookAlike learns from its own inconsistencies","Self-inconsistency becomes training signal for math MCQ accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that any generated distractor that does not exactly match the single ground-truth distractor is a dispreferred response, even if it is a mathematically valid and plausible wrong answer phrased differently.","fun_headline_variants_meta":{"raw":{"variants":["LookAlike: self-mined mistakes boost math MCQ distractors by 6%","Your model's own errors train it to write better math MCQs","LookAlike mines its own slips to teach consistent math MCQ generation","6% better distractors: LookAlike learns from its own inconsistencies","Self-inconsistency becomes training signal for math MCQ accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001461,"raw_usage":{"total_tokens":5893,"prompt_tokens":974,"completion_tokens":4919,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":4825}},"tokens_in":590,"tokens_out":4919,"duration_ms":33407,"temperature":1.0,"reasoning_tokens":4825,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:05:21.225736+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set in which each error admits two or more distinct but equally consistent distractors; if LookAlike, trained under its exact-match labeling, performs no better than SFT when graded by a similarity-based judge that accepts all valid distractors, then the reported consistency gains depend on surface-form reproduction rather than genuine error–answer alignment.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DiVERT baseline and the variational error-space approach that LookAlike must beat."}],"review_version":1}