{"id":"9a66917d-1e1b-46bc-9573-08aa7906622f","arxiv_id":"2510.13434","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"M2PO combines a QE-plus-alignment reward with a dynamic curriculum and multi-pair DPO loss, and reports WMT21-22 gains for a 7B translation model, but the abstract's WMT23/24 9B parity claims are unsupported.","lead":"A machine-translation training method, M2PO, scores candidate translations on both fluency and factual alignment, then trains the model on many ranked pairs rather than one winner-loser pair. The paper reports that a 7B open model approaches GPT-4o on WMT21-22, though the abstract's stronger WMT23/24 and 9B claims do not appear in the body.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing concern is that the faithfulness reward (WSPAlign S_align and Gemini Coverage Score) is never validated against human judgments, making the preference labels and final evaluation unreliable.","rationale":"I focused on the faithfulness-signal validity because it is the premise on which both the preference data construction (Eq. 2) and the final evaluation (§5.2) depend. The paper's contribution is specifically to correct QE models' blind spot for partial hallucinations, so if the replacement signal (WSPAlign alignment coverage) is not itself a reliable measure of that construct, the entire framework is misaligned. The abstract's unsupported claims about WMT23/24 and 9B are a serious reporting issue, but they could be fixed by revising the abstract; the metric issue cannot be fixed without re-running the experiments with a human-validated reward. The reader's weakest assumption identifies exactly this, and I agree. A human evaluation of the preference pairs and final outputs would settle whether the automatic faithfulness signals track human judgment; if they do not, the headline faithfulness gains are circular. I therefore recommend the reader's REJECT verdict remain unchanged.","tokens_in":14937,"tokens_out":6925,"duration_ms":60823,"concrete_test":"Validate S_align and the Gemini Coverage Score against human judgments on a stratified sample of 200 source sentences from the M2PO-Preference data (covering all 10 directions and including partial-error cases). Have three bilingual annotators per instance judge (1) which of each preference pair is more faithful (partial hallucinations/omissions) and (2) rate the final M2PO translations for faithfulness. Compute Pearson/Spearman correlation between the automatic signals (S_align, Gemini Coverage) and the human labels. Also compute S_align correlations on the HalOmi benchmark using the protocol of Table 1. If the correlation is below ~0.6, or if WSPAlign's correlation is substantially lower than Gemini Coverage's, the reward signal is unreliable and the paper's faithfulness claims do not stand.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that M2PO improves translation faithfulness rests on the validity of two faithfulness signals: S_align from WSPAlign (Eq. 2) used to construct preference pairs, and the Gemini-2.0-Flash 'Coverage Score' used for motivation (§3) and final evaluation (§5.2, Appendix A). Neither signal is validated against human judgments of partial hallucinations/omissions. §3 benchmarks Gemini Coverage, XCOMET, and KIWI-XXL against HalOmi, but never validates WSPAlign-based S_align, even though that is the signal that actually labels the preference data. Moreover, the evaluation metric (Gemini Coverage) is an LLM-judge from the same family used to motivate the method, so the validation is not independent. No human evaluation is reported anywhere. If WSPAlign alignment coverage or the Gemini coverage score does not track human faithfulness (especially for partial errors), then the preference pairs are mislabeled and the reported faithfulness gains are an artifact of optimizing a faulty reward. The body's claim that M2PO 'elevates the 7B model to a level highly competitive with leading proprietary systems' on WMT21-22 would then be unsupported, independent of the abstract's unsubstantiated WMT23/24/9B claims.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes M2PO, a multi-pair, multi-perspective preference optimization framework for machine translation. It constructs candidate pools from an SFT model and GPT-4o-mini, augments them with difficult negatives, and scores candidates with a static reward r_s = r_qe + λ_f S_align (Eq. 2) that combines a QE score (KIWI-XXL) with a WSPAlign-based factuality bonus. During training, the static score is fused with the model's own log-probabilities via a curriculum weight α_t (Eq. 3), and the ranked candidates are used to create K/2 head-to-tail preference pairs. The final loss (Eq. 4) combines a multi-pair DPO-style loss, a listwise ranking loss, and a behavior-cloning loss. Experiments on WMT21-22 with ALMA-7B-LoRA report improvements over DPO/CPO baselines and competitiveness with GPT-4o and GPT-4o-mini. The arXiv abstract additionally claims results on WMT23, WMT24, FLORES-200, and a 9B model with parity to Gemini-2.0-Flash, but these are not present in the body.","tokens_in":15228,"tokens_out":8159,"duration_ms":65243,"significance":"The core idea is plausible and potentially useful: constructing a diverse candidate pool with difficult negatives, combining a QE score with an alignment-based factuality bonus, fusing static and self-reward scores with a curriculum, and training on multiple pairs are all sensible design choices. Table 1 provides a useful demonstration that standard QE metrics lose correlation with human labels when partial hallucination examples are included, and the authors are careful not to reuse the training-time metric (KIWI-XXL) for evaluation. If the claims are reproducible, M2PO would be a valuable data-centric recipe for MT alignment. However, the manuscript does not validate the faithfulness signal against human judgments, lacks statistical significance testing, contains a formula error in its core loss, and includes a material mismatch between the abstract and the reported experiments. The stated plan to release code and data is a strength, but availability is not yet confirmed.","major_comments":[{"comment":"Equation (5) defines L_DM-DPO = -Σ_i w_i log σ(β Δ_i log πθ). This contains only the policy-model log-ratio; the standard DPO objective in Eq. (1) includes the reference-model term Δπ_ref = log π_ref(y_w|x) - log π_ref(y_l|x). As written, Eq. (5) is a reference-free/SimPO-like loss, not DPO. The method therefore cannot be reproduced from the equations, and the statement that M2PO wraps 'DPO-like algorithms' is misleading. Please either insert the π_ref term, or explicitly define the loss as reference-free and justify the choice. If the implemented loss differs from Eq. (5), all main experiments and ablations need to be re-examined.","section":"§4.4, Eq. (5)"},{"comment":"The faithfulness component of the preference signal is S_align from WSPAlign in Eq. (2), yet §3 validates only the Gemini Coverage score against HalOmi; WSPAlign is never evaluated against human annotations. In addition, the final faithfulness evaluation (§5.2, Appendix A) uses a Gemini-2.0-Flash coverage score from the same family as the motivating analysis. No human evaluation of the final translations is reported anywhere. If S_align, or the Gemini judge, does not track human judgments of partial hallucinations/omissions, the preference pairs are mislabeled and the reported faithfulness gains may be an artifact of optimizing a circular reward. The authors should validate S_align on HalOmi and include a human faithfulness evaluation of the M2PO outputs.","section":"§4.3, §5.2, Appendix A"},{"comment":"Tables 2-3 and Table 7 report single-run point estimates with no error bars or significance tests. Several headline differences are very small (e.g., Table 2 En→De XCOMET: M2PO 97.25 vs CPO 97.11; En→Cs COMET22: 90.40 vs 90.29), which is within typical COMET/XCOMET variability. The repeated claim that M2PO 'significantly' or 'substantially' outperforms baselines is therefore unsupported. Please provide multiple seeds and pairwise significance tests, or weaken the claims accordingly.","section":"§5.2, Tables 2-3"},{"comment":"The arXiv abstract promises experiments on WMT23, WMT24 and FLORES-200 with a 9B model that reaches parity with GPT-4o and Gemini-2.0-Flash. The full text's own abstract and §5 only report WMT21-22, ALMA-7B-LoRA, and baselines GPT-4o/GPT-4o-mini; FLORES-200 is used only to build the preference dataset (§5.1), and Gemini-2.0-Flash appears only as the name of the coverage-score judge. This is a material mismatch between the claimed scope and the evidence. Either add the promised experiments or correct the abstract to match the body; as written, the headline claim cannot be verified.","section":"Abstract vs §5.1, §5.3"}],"minor_comments":[{"comment":"The index i in Eq. (5) and the notation Δ_i are not defined; specify i = 1,...,K/2 and define Δ_i log πθ = log πθ(y_w^i) - log πθ(y_l^i). Also specify how λ_pref, λ_rank, and λ_bc are chosen.","section":"§4.4"},{"comment":"Hyperparameters λ_f, β, α_t schedule, τ_s, τ_w, and K are set ad hoc; provide at least a sensitivity analysis for λ_f and β, since the preference-pair construction depends on them.","section":"§5.1"},{"comment":"The 'volatility' claim is based on visual inspection; provide a quantitative measure (e.g., variance or quartiles of XCOMET scores within coverage bins) to support the conclusion.","section":"Figure 1"},{"comment":"'All evaluations are conducted in a zero-shot setting' is ambiguous; M2PO is fine-tuned on a preference dataset. Clarify that zero-shot refers to test-domain generalization rather than absence of task fine-tuning.","section":"§5.2"},{"comment":"Clarify what 'Orig.' means for each DPO-like algorithm (same preference data? same hyperparameters?) and whether the +M2PO setting uses the same multi-pair data for all algorithms.","section":"Table 4"},{"comment":"Provide the full coverage-score prompt and the exact computation of S_align (e.g., is it an alignment coverage ratio?); the current one-sentence description is insufficient for reproducibility.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"This is a promising empirical framework, but in its current form the manuscript has load-bearing issues: the core loss equation is not the DPO loss it claims to be, the faithfulness reward is not validated against human judgments, and the abstract promises experiments that the body does not report. These are fixable with additional experiments and careful revision, but they are not merely presentational."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the M2PO package—dynamic fusion of external QE scores with the model's own log-probs, a WSPAlign-based factuality bonus, and head-to-tail multi-pair DPO—is a plausible and reasonably well-tested integration. The WMT21-22 results on ALMA-7B are consistent: M2PO beats DPO and CPO on almost every direction, the 'student surpasses teacher' claim holds against GPT-4o-mini, and the ablations show each component contributes. The HalOmi analysis in §3 is a nice concrete demonstration that QE metrics degrade on partial hallucinations, and using that to motivate a faithfulness-aware reward is sensible.\n\nThe soft spots are real and non-trivial. Eq (5) is labeled DPO but drops the reference-model log-ratio; as written it's closer to SimPO. That's a reproducibility problem, not just a naming issue—the exact objective needs to be stated correctly. More importantly, the faithfulness half of the claim rests on two signals that are never validated against human judgment: the WSPAlign S_align bonus used to label the preference data, and the Gemini Coverage Score used for final evaluation. §3 benchmarks the Gemini score against HalOmi but not WSPAlign, and the final faithfulness metric is the same LLM-judge family used to motivate the method. Without a human evaluation of partial hallucinations and omissions, the faithfulness gains are at risk of being circular. The quality gains on COMET22/XCOMET are external and not circular, so the core quality claim is on firmer ground.\n\nThe abstract is a clear overreach. It claims WMT23/24, FLORES-200, and a 9B model; the body only reports WMT21-22 on the 7B ALMA-LoRA. That needs to be fixed before anything else. Also, all results are single runs with no variance or significance tests; the margins over CPO are small in several directions, so I'd trust the direction but not the size of the gains.\n\nBottom line: this is a competent, thoughtful paper with one clean idea and a serious measurement problem. It deserves a proper referee—the issues are fixable with a corrected objective, a corrected abstract, and a human faithfulness evaluation. I'd send it to review, but I would not cite the faithfulness result until the metric is validated.","headline":"Solid engineering with a real idea, but the abstract oversells and the faithfulness signal is not validated.","tokens_in":15802,"tokens_out":2478,"would_cite":false,"duration_ms":22053,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"M2PO enables a 7B translation model to rival GPT-4o by adding a word-alignment faithfulness penalty to quality-estimation rewards and training on multiple preference pairs from the same candidate pool.","keywords":["machine translation","preference optimization","direct preference optimization","hallucination","faithfulness","quality estimation","multi-pair learning","LLM alignment"],"falsifier":"A human evaluation on a benchmark designed for partial errors, such as HalOmi, comparing M2PO versus DPO/CPO translations on the same source sentences would settle the claim: if human raters find no faithfulness superiority for M2PO, or find it less faithful, the central claim fails. A cheaper check is to rerun the preference construction with an independent human-validated faithfulness metric in place of WSPAlign and see whether the criterion still selects the same winners.","tokens_in":14769,"feed_emoji":"🌐","tokens_out":8272,"duration_ms":62643,"temperature":0.7,"pith_summary":"The paper argues that current preference-optimization methods for machine translation fail because quality-estimation (QE) scores ignore partial hallucinations and omissions, favoring fluent but unfaithful translations. M2PO corrects this with a composite reward that adds a word-alignment-based factuality score to the QE score, a curriculum that gradually shifts from static expert scores to the model's own judgment, and a multi-pair objective that uses every candidate instead of a single best-worst pair. On WMT21-22, the 7B M2PO model outperforms all open-source baselines, matches GPT-4o, and surpasses its own data source GPT-4o-mini. If the results hold, the framework offers a data-efficient, label-free way to align small open models to high-fidelity translation.","feed_headline":"7B translator rivals GPT-4o by fixing the hallucination blind spot","feed_subtitle":"Rewards word-level faithfulness as well as fluency, closing the gap to proprietary models.","key_machinery":"The paper's load-bearing mechanism is the composite static reward r_s = r_qe + λ_f·S_align, where r_qe is a QE score (KIWI-XXL) and S_align is a word-level alignment score (WSPAlign) that penalizes content not grounded in the source. During training, z-score normalized static scores are fused with the model's own log-probabilities via r_fused = (1-α_t)·z(r_s) + α_t·z(log π_θ), with α_t increasing from 0.1 to 0.9, so the model gradually trusts its own judgment. Candidates are ranked by r_fused and split into K/2 head-to-tail pairs; the training loss combines (1) a weighted multi-pair DPO loss with softmax weights on reward differences, (2) a ListNet-style listwise ranking loss that keeps the","core_discovery":"The central claim is that QE-based rewards are systematically blind to partial errors — partial hallucinations and omissions — and that this blind spot can be removed by supplementing the QE score with a word-alignment faithfulness score (S_align from WSPAlign). The paper then shows that with this corrected reward, using the full ranked candidate list in K/2 head-to-tail pairs, weighted by the reward gap, yields better alignment than the standard single best-vs-worst comparison. The reported outcome is a 7B ALMA-based model that advances simultaneously on translation quality (XCOMET) and faithfulness (coverage score), becoming competitive with GPT-4o and beating GPT-4o-mini on WMT21-22. The","pith_inferences":["The paper evaluates faithfulness with an LLM-judge coverage score from Gemini-2.0-Flash, but no human evaluation is reported; the strongest test would be a human study on partial hallucinations to confirm that the automatic faithfulness gains reflect real fidelity improvements rather than agreement between two correlated automatic metrics.","The curriculum that shifts from external static scores to the model's own log-probabilities is reminiscent of an exploration-exploitation trade-off; it could generalize to any alignment pipeline where the reward model is known to be biased, but the paper does not test it outside MT.","The head-to-tail pairing strategy is one of several possible ways to exploit a ranked candidate list; an immediate extension would be to vary K and the pairing scheme and see where gains saturate, or to generate the entire candidate pool from a single model at multiple temperatures rather than mixing in GPT-4o-mini.","If the QE blind spot is as systematic as the paper's HalOmi analysis suggests, the same multi-perspective reward design could be applied to other faithfulness-critical generation tasks such as summarization and instruction following, where partial omission and hallucination are also under-penalized by reference-free quality metrics."],"forward_implications":["A corrected reward signal can be reused: the same composite reward (QE + alignment bonus) can be plugged into any DPO-like algorithm, and the paper reports consistent gains across DPO, KTO, SimPO, ORPO, and CPO.","Small models can beat their teachers: by learning from a richer preference signal rather than imitating outputs, a 7B model surpasses GPT-4o-mini, one of the models that generated its training candidates.","Fluency collapse is the main risk in multi-pair alignment: ablating the behavior-cloning term costs more than five XCOMET points, indicating that a simple NLL safeguard on the best candidate is essential for multi-pair DPO-style training.","Faithfulness and fluency are not in inherent conflict in preference optimization; the M2PO model rises on both XCOMET and the coverage score simultaneously, decoupling the two axes.","The QE blind-spot finding means that any preference dataset built with QE scores alone will contain systematically mislabeled winner-loser pairs for partial hallucinations, which may drive the 'fluency bias' observed in many aligned MT models."],"fun_headline_variants":["9B translator matches GPT-4o by fixing partial hallucination blind spot","Multi-pair preference optimization catches partial errors LLMs miss","Word-level faithfulness reward outperforms GPT-4o-mini","Fixing QE blind spot: partial hallucinations caught by new training"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the automatic faithfulness scores (WSPAlign's alignment score during training and Gemini's coverage score during evaluation) genuinely reflect human judgments of partial hallucinations and omissions; without human evaluation, the reported faithfulness gains could be an artifact of using similar automatic signals on both sides.","fun_headline_variants_meta":{"raw":{"variants":["9B translator matches GPT-4o by fixing partial hallucination blind spot","Multi-pair preference optimization catches partial errors LLMs miss","Word-level faithfulness reward outperforms GPT-4o-mini","Fixing QE blind spot: partial hallucinations caught by new training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001058,"raw_usage":{"total_tokens":4296,"prompt_tokens":786,"completion_tokens":3510,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":3436}},"tokens_in":530,"tokens_out":3510,"duration_ms":21070,"temperature":1.0,"reasoning_tokens":3436,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T09:45:22.746698+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A human evaluation on a benchmark designed for partial errors, such as HalOmi, comparing M2PO versus DPO/CPO translations on the same source sentences would settle the claim: if human raters find no faithfulness superiority for M2PO, or find it less faithful, the central claim fails. A cheaper check is to rerun the preference construction with an independent human-validated faithfulness metric in place of WSPAlign and see whether the criterion still selects the same winners.","supporting_citations":[],"review_version":1}