Pith. sign in

REVIEW 3 major objections 6 minor 56 references

Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A two-reward alignment method can make neural machine translation output more human-like and lexically diverse without sacrificing translation accuracy.

desk verdict Solid empirical comparison of RL-based naturalness alignment for literary MT, but the 'no loss in accuracy' claim is not supported by the numbers. read the letter →

arxiv 2412.08473 v2 pith:ES65MUGP submitted 2024-12-11 cs.CL

Huiyuan Lai , Esther Ploeger , Rik van Noord , Antonio Toral This is my paper · ORCID
classification cs.CL
keywords naturalnessmachinetranslationeseclassificationrewardlearninglexicaldiversityliterarytranslationEnglish-to-Dutchpolicygradient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to solve a known failure of neural machine translation: output is impoverished compared to human-written or human-translated text, with less lexical diversity and more translationese. Its proposal is a reward-based alignment method that optimizes two things at once: a translationese classifier that rewards more human-like language, and COMET, which rewards keeping the source content. The authors test three different preference classifiers (preferring original target text over human translation, human translation over machine translation, and original over machine translation) on English-to-Dutch literary translation. Their central claim is that the best configuration, COMET plus the machine-translation-versus-human-translation classifier, raises lexical diversity (MTLD 93.3 vs 90.4) while holding or slightly improving translation quality on metrics not used in training (KIWI 80.6 vs 80.4, MetricX 2.63 vs 2.66). If true, this gives an adjustable way to make MT sound more human without paying an accuracy penalty.

What carries the argument

The central mechanism is the composite reward function r(ŷ), the harmonic mean of a naturalness reward r_t(ŷ) and a content reward r_c(ŷ), with thresholds (σ_t=0.5, σ_c=0.85) that zero out the reward if either component falls short. The naturalness reward is the probability from a binary classifier trained to prefer one text type over another (original Dutch over human translation, human translation over machine translation, or original over machine translation), and the content reward is the COMET score of the sampled translation against the source and reference. The MT model is fine-tuned with a policy-gradient loss that maximizes this reward, while a supervised loss weighted by β keeps it near the base model. This turns 'naturalness' into a tunable, trainable preference signal rather than a fixed post-processing rule.

What would settle it

Retrain the MT-vs-HT classifier from scratch on a held-out set of machine and human translations, re-score the aligned model's output with that new classifier, and compare with human ratings; if the MTLD and classification gains disappear or reverse, the naturalness improvement is an artifact of reward overfitting.

Watch

Extended reading notes

Core claim

The central claim is that machine translationese can be reduced by aligning the MT model with a reward that combines content preservation and naturalness, where naturalness is operationalized through binary classifiers that distinguish original target-language texts (OR), human translations (HT), and machine translations (MT). Treating naturalness as a learnable preference rather than a fixed tag, the paper fine-tunes a base MT model with policy gradient so that sampled translations receive a harmonic-mean reward of a COMET content score and a classifier's naturalness probability. Across three preference perspectives, the model trained with the MT-vs-HT classifier performs best, producing more lexically rich output (e.g., MTLD 93.3 vs 90.4 for the base model) while retaining translation accuracy as measured by held-out metrics. The authors conclude that with the right preference signal, increased naturalness need not come at the cost of content preservation.

Load-bearing premise

The load-bearing assumption is that the classifiers used to reward the model during training are also trustworthy yardsticks for measuring naturalness in the evaluation, so the reported gains could partly reflect the model learning to please those same classifiers.

Editorial extensions

If this is right

  • Because the best aligned model improves or holds held-out quality metrics (KIWI 80.6 vs 80.4, MetricX 2.63 vs 2.66) that were not part of the reward, the accuracy retention is not an artifact of optimizing those exact metrics.
  • The comparison across reward classifiers shows the naturalness signal must match the target side of the training data: OR-based rewards underperform when the MT training target is human-translated text, so choosing the classifier is part of the method.
  • The hyperparameter β tunes how far the aligned model can move from the base MT model: β=0.5 preserves content while β=0.0 trades accuracy for stronger classifier agreement, making the method adaptable to faithfulness-critical or style-critical applications.
  • The paper's per-book MTLD analysis shows the alignment model lies between base MT and human translation in lexical diversity, so the method narrows the gap without overshooting on most books.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I infer that combining all three classifiers, or adding a penalty for classifier disagreement, could test whether the OR-based rewards fail because of target-data mismatch or because the classifier signal itself is weaker.
  • A stronger validation of genuine naturalness would use human evaluation or a classifier trained on entirely different data; the paper itself lists human evaluation as future work.
  • The method's principle is language-pair-agnostic, so I would expect it to transfer to other language pairs and domains, but the naturalness reward would need to be re-trained on that target variety.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a reinforcement-learning alignment method for English-to-Dutch literary NMT that rewards both naturalness and content preservation. A base BART MT model is trained with supervised learning; a translationese classifier (OR vs HT, HT vs MT, or OR vs MT) and COMET are then used as reward models in a policy-gradient objective, with a negative log-likelihood term controlled by β. The authors evaluate on a 31-book literary test set using BLEU, COMET, KIWI, MetricX, the same translationese classifiers, and lexical-diversity metrics. Their best model (BM + COMET & MT-HT) improves MTLD (93.3 vs 90.4), with small favorable changes on KIWI and MetricX but small unfavorable changes on BLEU and COMET; they conclude that the model is lexically richer and more human-like without loss in translation accuracy.

Significance. The framework addresses a real trade-off in literary MT, and the idea of using multiple preference perspectives is a useful contribution. Strengths include the public code repository, the large realistic test set, the ablation study, and the per-book MTLD analysis, which provides a falsifiable check that the effect is not driven by a few books. If the statistical and evaluation-validity concerns are addressed, the method would be a solid contribution. As it stands, the naturalness improvement is supported by the independent MTLD signal, but the classifier-based metrics are partly circular, and the 'without loss in translation accuracy' claim is not supported by the significance evidence presented.

major comments (3)
  1. [§6.2, Table 2] The headline claim that the best system improves naturalness 'without loss in translation accuracy' is not statistically supported. For BM + COMET & MT-HT versus the base MT model, BLEU decreases (32.1 vs 32.5) and COMET decreases slightly (82.2 vs 82.3), while the favorable KIWI (80.6 vs 80.4) and MetricX (2.63 vs 2.66) differences are very small. No significance tests, confidence intervals, or multiple seeds are reported, and Table 3 shows that the COMET-only ablation already attains KIWI 80.7 and MetricX 2.64, so the incremental effect of the MT-HT reward on accuracy is within noise. The authors should provide paired significance tests over the 31 books (e.g., paired bootstrap or permutation tests) and ideally multiple training seeds, or explicitly soften the claim to 'no substantial loss' with the associated uncertainty.
  2. [§5.3 vs §4.3] The naturalness evaluation is partly circular. The binary translationese classifiers trained in Section 4.2 are used as reward models in Eq. (2), and the same classifiers (as described in Section 5.3) are then used to score the outputs in Tables 2 and 3; COMET is likewise used both as the content reward in Eq. (3) and as an evaluation metric. Improvements in the HT-OR, MT-HT, and MT-OR columns can therefore reflect the MT model satisfying its own reward classifiers rather than a general increase in human-likeness, and the COMET column is not independent. The MTLD result (93.3 vs 90.4) is a genuinely independent signal and should be foregrounded; the paper would also benefit from a held-out or human naturalness evaluation before claiming 'more properties of human-written language' in the abstract.
  3. [§6.1] The manual checkpoint selection described in Section 6.1 is a selection effect that is not accounted for in the reported results. The authors state that they manually selected checkpoints between 1k and 6k steps using validation loss, classifier accuracy, and MTLD, and then report the 5k model in all main tables. Because the test-set numbers in Tables 2 and 3 are a single draw conditional on that manual selection, the point estimates are not an unbiased comparison of the method. The authors should specify the selection criterion in advance, report the full checkpoint curves on the test set, or otherwise quantify the selection effect; otherwise the 'without loss' comparison is difficult to interpret.
minor comments (6)
  1. [Eq. (3)] The sentence introducing Eq. (3) says 'σt represents the content threshold'; this should be σ_c to match the variable used in the equation.
  2. [Eq. (6)] The sentence after Eq. (6) contains 'β a is a hyperparameter'; this should be simply 'β'.
  3. [Figure 3] Figure 3 is described as a confusion matrix, but the columns appear to be test sets rather than predicted classes; please clarify the axes and how the diagonal values should be read.
  4. [§5.3] The definitions of B1, PTF, and CDU are very compact; for reproducibility, please specify whether these are computed per sentence or per book and how the source-word-to-translation-option counts are obtained when a source word has multiple aligned target tokens.
  5. [Table 4] Table 4 refers to green and blue highlights that are not visible in monochrome print; please add textual markers or boldface in addition to color.
  6. [Limitations] The Limitations section acknowledges the absence of large-scale human evaluation, but the abstract and conclusion describe the output as exhibiting 'more properties of human-written language'; these statements should be qualified as being based on automatic proxy metrics until human evaluation is available.

Circularity Check

2 steps flagged · score 6.0 of 10

Classifier-based naturalness evaluation reuses the same models that serve as RL rewards, so those columns are partly forced; independent MTLD and KIWI/MetricX evidence keeps the central claim from collapsing entirely.

  1. fitted input called prediction [Section 4.2 / Section 4.3 (Eq. 2) / Section 5.3 (Translationese Evaluation)]
    "These classifiers will be used as rewards (Section 4.3) to foster naturalness. ... We apply the translationese detection models to MT outputs and report the rate (i.e. classification accuracy) at which they are classified as the target aspect, such as OR in HT-OR, with higher rates indicating that the outputs are more human-like."

    The same fitted binary classifiers define the naturalness reward in Eq. 2 (rt(y_hat) = p(t1|y_hat; phi) above threshold) and are then applied in Section 5.3 as the naturalness evaluation. The alignment procedure maximizes this classifier probability via policy-gradient RL, so higher HT-OR / MT-HT / MT-OR classification accuracy after training is expected by construction; it is not independent evidence that outputs became more human-like. The paper's own care to point to KIWI and MetricX as metrics 'not used in our reward learning' confirms that same-instrument scores are not treated as independent. Lexical-diversity metrics such as MTLD were not rewards and do provide independent support, so the circularity is partial.

  2. other [Section 4.3 (Eq. 3) / Section 5.3 (Translation Quality) / Section 7]
    "Using the evaluation model COMET and different binary translationese classifiers trained with MT, HT, and original target-language data as reward models, we approximate human preference and align the MT model with it. ... We employ three metrics to automatically calculate the content preservation of the output based on human references (and source sentences), namely BLEU (Papineni et al., 2002), COMET (Rei et al., 2020, 2022), and MetricX (Juraska et al., 2024)."

    COMET is both the content-based reward (Eq. 3, rc(y_hat) = C(x, y, y_hat) above threshold) and one of the translation-quality metrics reported in Table 2 and Table 3. Comparisons on the COMET column therefore reuse the model optimized during training, so those numbers are not independent evidence about content preservation. This is secondary: the paper's no-loss claim is based on KIWI and MetricX, which were not rewards, and BLEU is also reported; but presenting COMET as an evaluation metric despite its reward role is an instrumentation overlap.

full rationale

The most significant circularity is that the binary translationese classifiers trained in Section 4.2 are used twice: as the naturalness reward in Eq. 2 and as the naturalness evaluation in Section 5.3. Because RL maximizes p(t1|y_hat; phi), reporting the same classifier's accuracy as evidence that outputs are more human-like is partly a check that optimization worked, not an independent measurement. The paper partially insulates its headline claim: the 'without loss in translation accuracy' clause is supported by KIWI and MetricX, which the authors explicitly note were not used in reward learning, and the lexical-richness improvement (MTLD 93.3 vs 90.4) is measured with metrics that were not rewards. COMET is also used both as the content reward (Eq. 3) and as one of the translation-quality metrics in Table 2, a secondary circularity; the authors avoid relying on COMET for the no-loss conclusion. The self-citations (Ploeger et al. 2024 for data and Tailored RR; Lai et al. 2021 for style transfer) are not load-bearing because the comparisons are re-evaluated in this paper. The manual checkpoint selection described in Section 6.1 is a statistical/selection concern rather than a circularity. The Limitations section acknowledges that automatic naturalness metrics are limited and that large-scale human evaluation is missing, which is relevant but does not remove the classifier/evaluator overlap. Overall, the central claim retains independent content through MTLD and held-out KIWI/MetricX, but the naturalness-classifier columns reduce by construction, giving partial circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method does not introduce new physical or conceptual entities; it combines existing components (COMET, BERTje classifiers, BART MT model, policy gradient). The main additional load-bearing choices are the hyperparameters, the dataset label assumptions, and the circular use of classifiers as both rewards and evaluation metrics.

free parameters (5)
  • beta (NLL loss weight) = 0.5 in main experiments; 0.0 in Section 6.5
    Controls the trade-off between the supervised loss and the reward loss. Tuned by the authors, not derived. See Section 4.3.
  • sigma_t (translationese threshold) = 0.5
    Gates the naturalness reward; described as 'set based on preliminary results' in Section 4.3.
  • sigma_c (content threshold) = 0.85
    Gates the COMET content reward; described as 'set based on preliminary results' in Section 4.3.
  • Alignment checkpoint = 5k training steps
    Manually selected between 1k and 6k steps based on validation curves in Section 6.1.
  • Beam size = 5
    Inference decoding parameter set in Section 5.2.
assumptions (5)
  • domain assumption The OR/HT labels in the book dataset (Toral et al., 2021) are correct.
    The classifiers and the evaluation rely on these labels being accurate.
  • domain assumption COMET is a valid proxy for content preservation and translation quality.
    COMET is used both as the content reward and as an evaluation metric, and its validity is assumed from prior work.
  • domain assumption Binary translationese classifiers capture naturalness well enough to serve as rewards.
    The method assumes a classifier trained to distinguish OR from HT, MT from HT, or MT from OR can provide a reward signal that increases naturalness.
  • ad hoc to paper Synthetic MT data generated by the base MT model is representative of machine translation for classifier training.
    The MT vs OR and MT vs HT classifiers are trained on translations produced by the same base model that is later aligned. This is specific to the paper's setup and may create a self-referential training loop.
  • standard math Policy gradient (REINFORCE) is a valid optimization algorithm for this setting.
    The REINFORCE algorithm from Williams (1992) is used without modifications or convergence guarantees.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation." pith.science (2026). https://pith.science/paper/ES65MUGP

@misc{pith2026241208473,
  author       = {Pith},
  title        = {Pith review of: Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES65MUGP}},
  note         = {Machine review of arXiv:2412.08473}
}
read the original abstract

Neural machine translation (NMT) systems amplify lexical biases present in their training data, leading to artificially impoverished language in output translations. These language-level characteristics render automatic translations different from text originally written in a language and human translations, which hinders their usefulness in for example creating evaluation datasets. Attempts to increase naturalness in NMT can fall short in terms of content preservation, where increased lexical diversity comes at the cost of translation accuracy. Inspired by the reinforcement learning from human feedback framework, we introduce a novel method that rewards both naturalness and content preservation. We experiment with multiple perspectives to produce more natural translations, aiming at reducing machine and human translationese. We evaluate our method on English-to-Dutch literary translation, and find that our best model produces translations that are lexically richer and exhibit more properties of human-written language, without loss in translation accuracy.

Figures

Figures reproduced from arXiv: 2412.08473 by the authors.

Figure 1
Figure 1. Aligning the translation policy from both [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Evaluation results on the validation set under various settings. Notes: (i) The training step of 0K represents [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrices for the different binary [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Per-book comparison of MTLD under human translation, base MT model, and alignment model. Note [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 26 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.618 Translation artifacts in cross-lingual transfer learning . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7674--7684, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.618 2020

  4. [4]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

    arXiv 2015

  5. [5]

    Mona Baker. 1993. https://www.jbe-platform.com/content/books/9789027285874-z.64.15bak Corpus linguistics and translation studies: Implications and applications . In Text and Technology. John Benjamins

    arXiv 1993

  6. [6]

    Marco Baroni and Silvia Bernardini. 2005. https://doi.org/10.1093/llc/fqi039 A New Approach to the Study of Translationese: Machine-learning the Difference between Original and Translated Text . Literary and Linguistic Computing, 21(3):259--274

    work page doi:10.1093/llc/fqi039 2005

  7. [7]

    Yuri Bizzoni, Tom S Juzek, Cristina Espa \ n a-Bonet, Koel Dutta Chowdhury, Josef van Genabith, and Elke Teich. 2020. https://doi.org/10.18653/v1/2020.iwslt-1.34 How human is machine translationese? comparing human and machine translations of text and speech . In Proceedings of the 17th International Conference on Spoken Language Translation, pages 280--2...

    doi:10.18653/v1/2020.iwslt-1.34 2020

  8. [8]

    Malina Chichirau, Rik van Noord, and Antonio Toral. 2023. https://aclanthology.org/2023.eamt-1.21 Automatic discrimination of human and neural machine translation in multilingual scenarios . In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 217--226, Tampere, Finland. European Association for Machine T...

    work page 2023

Show all 56 references
  1. [9]

    translationese

    Miguel A Jim \'e nez Crespo. 2023. “translationese”(and “post-editese”?) no more: on importing fuzzy conceptual tools from translation studies in mt research. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 261--268

    2023

  2. [10]

    Orph \'e e de Clercq, Rudy Loock, Gert de Sutter, Bert Cappelle, and Koen Plevoets. 2020. https://hal.science/hal-02454668 Uncovering Machine Translationese: an experiment on 4 MT systems for English-French translations . In Journ \'e e d' \'e tudes \#TQ2020 ''Traduction & Qua...

    2020

  3. [11]

    Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, Tommaso Caselli, Gertjan van Noord, and Malvina Nissim. 2019. http://arxiv.org/abs/1912.09582 BERTje : A Dutch BERT Model . arXiv preprint arXiv:1912.09582

    2019 arXiv

  4. [12]

    Dirk Delabastita. 2011. Literary translation. Handbook of translation studies, 2:69--78

    2011

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  6. [14]

    Koel Dutta Chowdhury, Rricha Jalota, Cristina Espa \ n a-Bonet, and Josef Genabith. 2022. https://doi.org/10.18653/v1/2022.naacl-main.292 Towards debiasing translation artifacts . In Proceedings of the 2022 Conference of the North American Chapter of the Association for Comput...

    2022 doi

  7. [15]

    Markus Freitag, Isaac Caswell, and Scott Roy. 2019. https://doi.org/10.18653/v1/W19-5204 APE at scale and its implications on MT evaluation biases . In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 34--44, Florence, Italy. Assoc...

    2019 doi

  8. [16]

    Markus Freitag, Nitika Mathur, Daniel Deutsch, Chi-Kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Frederic Blain, Tom Kocmi, Jiayi Wang, David Ifeoluwa Adelani, Marianna Buchicchio, Chrysoula Zerva, and Alon Lavie. 2024. https://doi.org/10.18653/v1/2024.wmt-1.2 Ar...

    2024 doi

  9. [17]

    Markus Freitag, David Vilar, David Grangier, Colin Cherry, and George Foster. 2022. https://doi.org/10.18653/v1/2022.findings-acl.263 A natural diet: Towards improving naturalness of machine translation output . In Findings of the Association for Computational Linguistics: ACL...

    2022 doi

  10. [18]

    Martin Gellerstam. 1986. https://api.semanticscholar.org/CorpusID:59685951 Translationese in swedish novels translated from english . In Translation studies in Scandinavia: Poceedings from the Scandinavian Symposium on Translation Theory (SSOTT) II, 75, page 88–95

    1986

  11. [19]

    Yvette Graham, Barry Haddow, and Philipp Koehn. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.6 Statistical power and translationese in machine translation evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages...

    2020 doi

  12. [20]

    Ana Guerberof-Arenas and Antonio Toral. 2020. The impact of post-editing and machine translation on creativity and reading experience. Translation Spaces, 9(2):255--282

    2020

  13. [21]

    Zhiwei He, Xing Wang, Wenxiang Jiao, Zhuosheng Zhang, Rui Wang, Shuming Shi, and Zhaopeng Tu. 2024. https://doi.org/10.18653/v1/2024.naacl-long.451 Improving machine translation with human feedback: An exploration of quality estimation as a reward model . In Proceedings of the...

    2024 doi

  14. [22]

    Rricha Jalota, Koel Chowdhury, Cristina Espa \ n a-Bonet, and Josef van Genabith. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.438 Translating away translationese without parallel data . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proces...

    2023 doi

  15. [23]

    Juraj Juraska, Daniel Deutsch, Mara Finkelstein, and Markus Freitag. 2024. https://doi.org/10.18653/v1/2024.wmt-1.35 M etric X -24: The G oogle submission to the WMT 2024 metrics shared task . In Proceedings of the Ninth Conference on Machine Translation, pages 492--504, Miami...

    2024 doi

  16. [24]

    Moshe Koppel and Noam Ordan. 2011. https://aclanthology.org/P11-1132 Translationese and its dialects . In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1318--1326, Portland, Oregon, USA. Association ...

    2011

  17. [25]

    Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021 a . https://doi.org/10.18653/v1/2021.emnlp-main.349 Generic resources are what you need: Style transfer tasks without task-specific parallel training data . In Proceedings of the 2021 Conference on Empirical Methods in Natur...

    2021 doi

  18. [26]

    Huiyuan Lai, Antonio Toral, and Malvina Nissim. 2021 b . https://doi.org/10.18653/v1/2021.acl-short.62 Thank you BART ! rewarding pre-trained models improves formality style transfer . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics a...

    2021 doi

  19. [27]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.acl-main.703 BART : Denoising sequence-to-sequence pre-training for natural language generation, translatio...

    2020 doi

  20. [28]

    Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

    2019

  21. [29]

    Philip M McCarthy. 2005. https://scholar.google.com/citations?view_op=view_citation&hl=en&user=3uM8lA8AAAAJ&citation_for_view=3uM8lA8AAAAJ:Tyk-4Ss8FVUC An Assessment of the Range and Usefulness of Lexical Diversity Measures and the Potential of the Measure of Textual, Lexical ...

    2005

  22. [30]

    Philip M McCarthy and Scott Jarvis. 2010. Mtld, vocd-d, and hd-d: A validation study of sophisticated approaches to lexical diversity assessment. Behavior research methods, 42(2):381--392

    2010

  23. [31]

    Lili Mou and Olga Vechtomova. 2020. https://doi.org/10.18653/v1/2020.acl-tutorials.5 Stylized text generation: Approaches and applications . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 19--22, Online. As...

    2020 doi

  24. [32]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 B leu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311--3...

    2002

  25. [33]

    Prasanna Parthasarathi, Koustuv Sinha, Joelle Pineau, and Adina Williams. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.275 Sometimes we want ungrammatical translations . In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3205--3227, Punta ...

    2021 doi

  26. [34]

    Esther Ploeger, Huiyuan Lai, Rik Van Noord, and Antonio Toral. 2024. https://aclanthology.org/2024.eamt-1.24/ Towards tailored recovery of lexical diversity in literary machine translation . In Proceedings of the 25th Annual Conference of the European Association for Machine T...

    2024

  27. [35]

    Matt Post. 2018. https://doi.org/10.18653/v1/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Brussels, Belgium. Association for Computational Linguistics

    2018 doi

  28. [36]

    Daria Pylypenko, Kwabena Amponsah-Kaakyire, Koel Dutta Chowdhury, Josef van Genabith, and Cristina Espa \ n a-Bonet. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.676 Comparing feature-engineering and feature-learning approaches for multilingual translationese classificati...

    2021 doi

  29. [37]

    Miguel Ramos, Patrick Fernandes, Ant \'o nio Farinhas, and Andre Martins. 2024. https://aclanthology.org/2024.eamt-1.22 Aligning neural machine translation models: Human feedback in training and inference . In Proceedings of the 25th Annual Conference of the European Associati...

    2024

  30. [38]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.213 COMET : A neural framework for MT evaluation . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685--2702, ...

    2020 doi

  31. [39]

    Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and Andr \'e F. T. Martins. 2022. https://aclanthology.org/2022.wmt-1.60 C omet K iwi: IST -unba...

    2022

  32. [40]

    Team NLLB , Marta R. Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez,...

    2022 arXiv

  33. [41]

    M.C. Templin. 1957. https://books.google.nl/books?id=TCidAAAAMAAJ Certain Language Skills in Children: Their Development and Interrelationships . Child Welfare Monograph Series. University of Minnesota Press

    1957

  34. [42]

    Antonio Toral. 2019. https://aclanthology.org/W19-6627 Post-editese: an exacerbated translationese . In Proceedings of Machine Translation Summit XVII: Research Track, pages 273--281, Dublin, Ireland. European Association for Machine Translation

    2019

  35. [43]

    Antonio Toral, Andreas van Cranenburgh, and Tia Nutters. 2021. https://research.rug.nl/en/publications/literary-adapted-machine-translation-in-a-well-resourced-language L iterary-adapted machine translation in a well-resourced language pair . In Proceedings of the 7th Conferen...

    2021

  36. [44]

    Antonio Toral and Andy Way. 2015. https://doi.org/10.1075/ts.4.2.04tor Machine-assisted translation of literary text: A case study . Translation Spaces, 4(2):240--267

    2015 doi

  37. [45]

    Gideon Toury. 2012. Descriptive translation studies-and beyond. revised version. Amsterdam and Philadelphia: John Benjamins Publishing Company

    2012

  38. [46]

    Tobias van der Werff, Rik van Noord, and Antonio Toral. 2022. https://aclanthology.org/2022.eamt-1.19 Automatic discrimination of human and neural machine translation: A study with multiple pre-trained models and longer context . In Proceedings of the 23rd Annual Conference of...

    2022

  39. [47]

    Eva Vanmassenhove, Dimitar Shterionov, and Matthew Gwilliam. 2021. https://doi.org/10.18653/v1/2021.eacl-main.188 Machine translationese: Effects of algorithmic bias on linguistic complexity in machine translation . In Proceedings of the 16th Conference of the European Chapter...

    2021 doi

  40. [48]

    Eva Vanmassenhove, Dimitar Shterionov, and Andy Way. 2019. https://aclanthology.org/W19-6622 Lost in translation: Loss and decay of linguistic richness in machine translation . In Proceedings of Machine Translation Summit XVII: Research Track, pages 222--232, Dublin, Ireland. ...

    2019

  41. [49]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf Attention is all you need . In Advances in Ne...

    2017

  42. [50]

    Vered Volansky, Noam Ordan, and Shuly Wintner. 2015. https://doi.org/10.1093/llc/fqt031 On the features of translationese . Digital Scholarship in the Humanities, 30(1):98--118

    2015 doi

  43. [51]

    Shira Wein and Nathan Schneider. 2024. https://aclanthology.org/2024.eacl-long.45 Lost in translationese? reducing translation effect using A bstract M eaning R epresentation . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational L...

    2024

  44. [52]

    Williams

    Ronald J. Williams. 1992. https://doi.org/10.1007/BF00992696 Simple statistical gradient-following algorithms for connectionist reinforcement learning . Machine Learning, 8:229–256

    1992 doi

  45. [53]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020 doi

  46. [54]

    Sicheng Yu, Qianru Sun, Hao Zhang, and Jing Jiang. 2022. https://doi.org/10.18653/v1/2022.acl-short.40 Translate-train embracing translationese artifacts . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), page...

    2022 doi

  47. [55]

    Udnv Yule

    G. Udnv Yule. 1944. https://pure.mpg.de/rest/items/item_2407784_3/component/file_2622080/content The Statistical Study of Literary Vocabulary . Cambridge University Press

    1944

  48. [56]

    Mike Zhang and Antonio Toral. 2019. https://doi.org/10.18653/v1/W19-5208 The effect of translationese in machine translation test sets . In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 73--81, Florence, Italy. Association for C...

    2019 doi

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.