{"id":"8038bd1e-63e7-40f4-8457-406ce781ba10","arxiv_id":"2412.08473","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A reinforcement learning method that rewards both naturalness and content preservation improves lexical diversity in English-to-Dutch literary machine translation, with mixed results on translation accuracy.","lead":"This paper trains a translation system to favor natural, human-like Dutch while preserving meaning, using reinforcement learning rewards on top of a supervised model. Tested on English-to-Dutch literary translation, the best model produces more lexically diverse output, but the naturalness evidence partly relies on the same classifiers used in training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'without loss in translation accuracy' claim rests on tiny KIWI/MetricX deltas (80.6 vs 80.4; 2.63 vs 2.66) with no significance testing or repeated runs, while manual checkpoint selection on validation adds an unquantified selection effect, so the no-loss conclusion is not yet established.","rationale":"The reader's verdict (CONDITIONAL) is appropriate, and the reader's rationale correctly flags missing significance tests and checkpoint selection. However, the single most load-bearing weakness is not the reward/evaluation circularity alone: the same classifier used as a reward is admittedly used for translationese evaluation, and this does inflate the MT-HT classification column, but the paper's central claim also leans on independent lexical-diversity metrics and on held-out accuracy metrics. The accuracy claim is load-bearing because the paper explicitly claims 'without loss in translation accuracy' and supports it with two very small metric differences. Without interval estimates, multiple seeds, or a selection-robust protocol, those deltas are consistent with noise. A paired bootstrap across books or repeated training runs would settle whether the no-loss claim survives. The concrete test proposed would directly address this. Therefore the verdict should remain CONDITIONAL: the method is promising and the lexical-diversity evidence is real, but the no-loss conclusion needs stronger statistical support before acceptance as stated.","tokens_in":15376,"tokens_out":5512,"duration_ms":60330,"concrete_test":"Retrain BM and BM + COMET & MT-HT with 5 random seeds, select checkpoints using the same manual or early-stopping protocol on validation, and compute paired bootstrap 95% confidence intervals over per-book (or per-sentence) KIWI and MetricX deltas on the test set. If either interval includes zero or favors the base model, the 'without loss in translation accuracy' claim is unsupported. As a secondary check, also report the same deltas for the fixed 5K checkpoint without selection to quantify how much manual selection contributes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in Section 6.2 is that BM + COMET & MT-HT is lexically richer and more human-like without loss in translation accuracy. The naturalness half is independently supported by lexical-diversity metrics (e.g., MTLD 93.3 vs 90.4) that were not used as rewards; the classifier-based columns are partly circular but not the only evidence. The accuracy half, however, rests almost entirely on two held-out metrics that the authors emphasize were not used in reward learning: KIWI (80.6 vs 80.4) and MetricX (2.63 vs 2.66). These deltas are very small, no confidence intervals, significance tests, or multiple seeds are reported, and BLEU (32.1 vs 32.5) and COMET (82.2 vs 82.3) actually move slightly against the model. In addition, Section 6.1 describes manual checkpoint selection between 1k and 6k steps using validation loss, classifier accuracy, and MTLD; the reported test results are a single draw conditional on that selection, so the point estimates cannot be taken at face value as evidence of no loss. This is separate from, and more load-bearing than, the reward/classifier circularity: even if the naturalness classifiers are non-circular, the 'without loss' clause is not supported by the statistical evidence presented.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a reinforcement-learning alignment method for English-to-Dutch literary NMT that rewards both naturalness and content preservation. A base BART MT model is trained with supervised learning; a translationese classifier (OR vs HT, HT vs MT, or OR vs MT) and COMET are then used as reward models in a policy-gradient objective, with a negative log-likelihood term controlled by β. The authors evaluate on a 31-book literary test set using BLEU, COMET, KIWI, MetricX, the same translationese classifiers, and lexical-diversity metrics. Their best model (BM + COMET & MT-HT) improves MTLD (93.3 vs 90.4), with small favorable changes on KIWI and MetricX but small unfavorable changes on BLEU and COMET; they conclude that the model is lexically richer and more human-like without loss in translation accuracy.","tokens_in":15755,"tokens_out":7900,"duration_ms":82101,"significance":"The framework addresses a real trade-off in literary MT, and the idea of using multiple preference perspectives is a useful contribution. Strengths include the public code repository, the large realistic test set, the ablation study, and the per-book MTLD analysis, which provides a falsifiable check that the effect is not driven by a few books. If the statistical and evaluation-validity concerns are addressed, the method would be a solid contribution. As it stands, the naturalness improvement is supported by the independent MTLD signal, but the classifier-based metrics are partly circular, and the 'without loss in translation accuracy' claim is not supported by the significance evidence presented.","major_comments":[{"comment":"The headline claim that the best system improves naturalness 'without loss in translation accuracy' is not statistically supported. For BM + COMET & MT-HT versus the base MT model, BLEU decreases (32.1 vs 32.5) and COMET decreases slightly (82.2 vs 82.3), while the favorable KIWI (80.6 vs 80.4) and MetricX (2.63 vs 2.66) differences are very small. No significance tests, confidence intervals, or multiple seeds are reported, and Table 3 shows that the COMET-only ablation already attains KIWI 80.7 and MetricX 2.64, so the incremental effect of the MT-HT reward on accuracy is within noise. The authors should provide paired significance tests over the 31 books (e.g., paired bootstrap or permutation tests) and ideally multiple training seeds, or explicitly soften the claim to 'no substantial loss' with the associated uncertainty.","section":"§6.2, Table 2"},{"comment":"The naturalness evaluation is partly circular. The binary translationese classifiers trained in Section 4.2 are used as reward models in Eq. (2), and the same classifiers (as described in Section 5.3) are then used to score the outputs in Tables 2 and 3; COMET is likewise used both as the content reward in Eq. (3) and as an evaluation metric. Improvements in the HT-OR, MT-HT, and MT-OR columns can therefore reflect the MT model satisfying its own reward classifiers rather than a general increase in human-likeness, and the COMET column is not independent. The MTLD result (93.3 vs 90.4) is a genuinely independent signal and should be foregrounded; the paper would also benefit from a held-out or human naturalness evaluation before claiming 'more properties of human-written language' in the abstract.","section":"§5.3 vs §4.3"},{"comment":"The manual checkpoint selection described in Section 6.1 is a selection effect that is not accounted for in the reported results. The authors state that they manually selected checkpoints between 1k and 6k steps using validation loss, classifier accuracy, and MTLD, and then report the 5k model in all main tables. Because the test-set numbers in Tables 2 and 3 are a single draw conditional on that manual selection, the point estimates are not an unbiased comparison of the method. The authors should specify the selection criterion in advance, report the full checkpoint curves on the test set, or otherwise quantify the selection effect; otherwise the 'without loss' comparison is difficult to interpret.","section":"§6.1"}],"minor_comments":[{"comment":"The sentence introducing Eq. (3) says 'σt represents the content threshold'; this should be σ_c to match the variable used in the equation.","section":"Eq. (3)"},{"comment":"The sentence after Eq. (6) contains 'β a is a hyperparameter'; this should be simply 'β'.","section":"Eq. (6)"},{"comment":"Figure 3 is described as a confusion matrix, but the columns appear to be test sets rather than predicted classes; please clarify the axes and how the diagonal values should be read.","section":"Figure 3"},{"comment":"The definitions of B1, PTF, and CDU are very compact; for reproducibility, please specify whether these are computed per sentence or per book and how the source-word-to-translation-option counts are obtained when a source word has multiple aligned target tokens.","section":"§5.3"},{"comment":"Table 4 refers to green and blue highlights that are not visible in monochrome print; please add textual markers or boldface in addition to color.","section":"Table 4"},{"comment":"The Limitations section acknowledges the absence of large-scale human evaluation, but the abstract and conclusion describe the output as exhibiting 'more properties of human-written language'; these statements should be qualified as being based on automatic proxy metrics until human evaluation is available.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the headline 'without loss in translation accuracy' claim is too strong for the evidence. I would be comfortable with acceptance after the authors add significance testing (or clearly soften the claim) and address the circularity of the classifier-based evaluation. The contribution itself is within the journal's scope and the public code is a plus."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this: the paper is a solid empirical comparison of three translationese-perspective rewards in an RL alignment setup for EN→NL literary MT. The naturalness improvement is believable because the lexical-diversity metrics, which were never rewards, improve. But the headline claim 'without loss in translation accuracy' is not supported by the numbers at all.\n\nThe genuinely new piece is the systematic comparison of OR-vs-HT, HT-vs-MT, and OR-vs-MT classifiers as naturalness rewards, each paired with a COMET content reward, in the same policy-gradient framework. The evaluation is thorough: three strong baselines, ablations separating naturalness and content rewards, a β=0 control, per-book MTLD, and a concrete example showing the model preserving content better than the Tagging baseline. The paper is also honest in its limitations section and ships code.\n\nNow the soft spots. First, the no-loss claim. The best system does improve MTLD from 90.4 to 93.3, but the accuracy metrics are a wash: KIWI 80.6 vs 80.4, MetricX 2.63 vs 2.66, while BLEU drops 32.5→32.1 and COMET 82.3→82.2. There are no significance tests, no repeated runs, and the checkpoint is manually selected from the validation curve. Those deltas are indistinguishable from noise. The abstract should read 'without measurable loss' at most, and even that needs a second seed or a confidence interval.\n\nSecond, the circularity: the same classifiers reward the model and score the output. This is a real design flaw, although not fatal. The lexical diversity metrics (MTLD, TTR, etc.) are independent and they confirm the naturalness direction. The classifier columns alone would not convince, but they are not the only evidence.\n\nThird, the limitations are acknowledged but real: one language pair, one domain, a BART model from scratch, no human evaluation. That constrains how far the conclusions generalize, and the authors say so.\n\nWhom is this for? Anyone working on NMT naturalness or literary MT will want to read it; it is a useful reference for design choices and it is honest about trade-offs. It is not a breakthrough, but it is a solid, reproducible study.\n\nI would accept it for peer review. The right request is to add significance testing or multiple runs, and to rewrite the abstract's accuracy claim to match the evidence. With that, it is a publishable contribution.","headline":"Solid empirical comparison of RL-based naturalness alignment for literary MT, but the 'no loss in accuracy' claim is not supported by the numbers.","tokens_in":16229,"tokens_out":3909,"would_cite":true,"duration_ms":39025,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-reward alignment method can make neural machine translation output more human-like and lexically diverse without sacrificing translation accuracy.","keywords":["naturalness","machine translationese","translationese classification","reward learning","lexical diversity","literary machine translation","English-to-Dutch translation","policy gradient"],"falsifier":"Retrain the MT-vs-HT classifier from scratch on a held-out set of machine and human translations, re-score the aligned model's output with that new classifier, and compare with human ratings; if the MTLD and classification gains disappear or reverse, the naturalness improvement is an artifact of reward overfitting.","tokens_in":15207,"feed_emoji":"📖","tokens_out":9664,"duration_ms":83099,"temperature":0.7,"pith_summary":"The paper tries to solve a known failure of neural machine translation: output is impoverished compared to human-written or human-translated text, with less lexical diversity and more translationese. Its proposal is a reward-based alignment method that optimizes two things at once: a translationese classifier that rewards more human-like language, and COMET, which rewards keeping the source content. The authors test three different preference classifiers (preferring original target text over human translation, human translation over machine translation, and original over machine translation) on English-to-Dutch literary translation. Their central claim is that the best configuration, COMET plus the machine-translation-versus-human-translation classifier, raises lexical diversity (MTLD 93.3 vs 90.4) while holding or slightly improving translation quality on metrics not used in training (KIWI 80.6 vs 80.4, MetricX 2.63 vs 2.66). If true, this gives an adjustable way to make MT sound more human without paying an accuracy penalty.","feed_headline":"Two-reward alignment makes MT more natural without losing accuracy","feed_subtitle":"On English-to-Dutch literary translation, the best setup raises lexical diversity and keeps held-out quality scores steady.","key_machinery":"The central mechanism is the composite reward function r(ŷ), the harmonic mean of a naturalness reward r_t(ŷ) and a content reward r_c(ŷ), with thresholds (σ_t=0.5, σ_c=0.85) that zero out the reward if either component falls short. The naturalness reward is the probability from a binary classifier trained to prefer one text type over another (original Dutch over human translation, human translation over machine translation, or original over machine translation), and the content reward is the COMET score of the sampled translation against the source and reference. The MT model is fine-tuned with a policy-gradient loss that maximizes this reward, while a supervised loss weighted by β keeps it near the base model. This turns 'naturalness' into a tunable, trainable preference signal rather than a fixed post-processing rule.","core_discovery":"The central claim is that machine translationese can be reduced by aligning the MT model with a reward that combines content preservation and naturalness, where naturalness is operationalized through binary classifiers that distinguish original target-language texts (OR), human translations (HT), and machine translations (MT). Treating naturalness as a learnable preference rather than a fixed tag, the paper fine-tunes a base MT model with policy gradient so that sampled translations receive a harmonic-mean reward of a COMET content score and a classifier's naturalness probability. Across three preference perspectives, the model trained with the MT-vs-HT classifier performs best, producing more lexically rich output (e.g., MTLD 93.3 vs 90.4 for the base model) while retaining translation accuracy as measured by held-out metrics. The authors conclude that with the right preference signal, increased naturalness need not come at the cost of content preservation.","pith_inferences":["I infer that combining all three classifiers, or adding a penalty for classifier disagreement, could test whether the OR-based rewards fail because of target-data mismatch or because the classifier signal itself is weaker.","A stronger validation of genuine naturalness would use human evaluation or a classifier trained on entirely different data; the paper itself lists human evaluation as future work.","The method's principle is language-pair-agnostic, so I would expect it to transfer to other language pairs and domains, but the naturalness reward would need to be re-trained on that target variety."],"forward_implications":["Because the best aligned model improves or holds held-out quality metrics (KIWI 80.6 vs 80.4, MetricX 2.63 vs 2.66) that were not part of the reward, the accuracy retention is not an artifact of optimizing those exact metrics.","The comparison across reward classifiers shows the naturalness signal must match the target side of the training data: OR-based rewards underperform when the MT training target is human-translated text, so choosing the classifier is part of the method.","The hyperparameter β tunes how far the aligned model can move from the base MT model: β=0.5 preserves content while β=0.0 trades accuracy for stronger classifier agreement, making the method adaptable to faithfulness-critical or style-critical applications.","The paper's per-book MTLD analysis shows the alignment model lies between base MT and human translation in lexical diversity, so the method narrows the gap without overshooting on most books."],"supporting_citations":[{"why":"It supplies COMET, the content-based reward model that scores how well the translation preserves source meaning in Eq. 3.","marker":"(Rei et al., 2020)"},{"why":"It defines machine translationese and provides the MTLD, B1, PTF, and CDU metrics used to measure lexical diversity and naturalness.","marker":"(Vanmassenhove et al., 2021)"},{"why":"It provides the preprocessed English-to-Dutch literary corpus and the Tailored RR baseline, whose accuracy loss motivates combining naturalness with a content reward.","marker":"(Ploeger et al., 2024)"},{"why":"It is the Tagging baseline for improving MT naturalness, used as a comparison point for the accuracy-naturalness trade-off.","marker":"(Freitag et al., 2022)"},{"why":"It supplies the policy-gradient algorithm used to maximize the expected reward in the alignment objective.","marker":"(Williams, 1992)"},{"why":"It is the source of the parallel and monolingual book datasets used for training, validation, and testing.","marker":"(Toral et al., 2021)"},{"why":"It provides BERTje, the Dutch language model fine-tuned into the three translationese classifiers that act as naturalness rewards.","marker":"(de Vries et al., 2019)"},{"why":"It shows that neural classifiers can reliably distinguish translated from original text, which supports using classifier probabilities as reward signals.","marker":"(Pylypenko et al., 2021)"}],"fun_headline_variants":["Dual-reward MT alignment boosts naturalness, keeps accuracy","New MT training rewards naturalness and content, no trade-off","Lexical richness up, accuracy steady: NMT alignment advance","Aligning MT with human-preference rewards cuts translationese","Two-perspective reward makes MT more human, not less exact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the classifiers used to reward the model during training are also trustworthy yardsticks for measuring naturalness in the evaluation, so the reported gains could partly reflect the model learning to please those same classifiers.","fun_headline_variants_meta":{"raw":{"variants":["Dual-reward MT alignment boosts naturalness, keeps accuracy","New MT training rewards naturalness and content, no trade-off","Lexical richness up, accuracy steady: NMT alignment advance","Aligning MT with human-preference rewards cuts translationese","Two-perspective reward makes MT more human, not less exact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1327,"prompt_tokens":868,"completion_tokens":459,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":373}},"tokens_in":484,"tokens_out":459,"duration_ms":5820,"temperature":1.0,"reasoning_tokens":373,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:43:59.435622+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the MT-vs-HT classifier from scratch on a held-out set of machine and human translations, re-score the aligned model's output with that new classifier, and compare with human ratings; if the MTLD and classification gains disappear or reverse, the naturalness improvement is an artifact of reward overfitting.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the preprocessed English-to-Dutch literary corpus and the Tailored RR baseline, whose accuracy loss motivates combining naturalness with a content reward."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It shows that neural classifiers can reliably distinguish translated from original text, which supports using classifier probabilities as reward signals."}],"review_version":1}