Pith. sign in

REVIEW 3 major objections 5 minor 24 references

Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Post-edited corrections of ambiguous terms can be turned into preference pairs that teach a translation model to choose the right variant, raising exact term accuracy from 53.7% to 56.3% without significant COMET loss.

desk verdict A useful but unvalidated preference-pair construction keeps the 53.7→56.3 term-accuracy gain from being fully trustworthy; the masked SFT/PO loss variants are the paper's real, publishable contribution. read the letter →

arxiv 2507.03580 v2 pith:YAOEUSJJ submitted 2025-07-04 cs.CL

Nathaniel Berger , Johannes Eschbach-Dymanus , Miriam Exel , Matthias Huck , Stefan Riezler This is my paper
classification cs.CL
keywords ambiguousterminologyneuralmachinetranslationpreferenceoptimizationpost-editingdisambiguationEnglish-Germantermaccuracysupervisedfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Real terminology dictionaries are usually one-to-many: a source term like 'transfer' can have dozens of valid translations in the target language, and only context or a corporate style guide decides which one is right. This paper tries to make a neural machine translation system learn that decision from human post-edits, where translators have already replaced a valid but wrong dictionary term with the correct one. The authors turn these corrections into preference pairs and train with a combination of supervised fine-tuning and preference optimization, masking non-term tokens so the model concentrates on terminology while full-sequence objectives protect fluency. On an English-German test set, the best configuration raises exact term accuracy from 53.7% to 56.3%, a statistically significant gain, without a significant change in COMET score. The approach needs no one-to-one dictionary and no human input at decoding time.

What carries the argument

The central object is a composite training loss, $L_{\mathrm{term}} = \mathbb{1}_{PO} L_{PO} + \mathbb{1}_{mPO} L_{mPO} + \alpha(\mathbb{1}_{SFT} L_{SFT} + \mathbb{1}_{mSFT} L_{mSFT})$, in which each component can be switched on or off. The preference-optimization term is an IPO-style loss with its squared error replaced by smooth-L1, plus a supervised fine-tuning term; both come in a masked variant that computes the geometric mean of token probabilities only over tokens belonging to dictionary terms. Full-sequence objectives keep general translation quality, while masked objectives sharpen the model's choice among semantically close variants. The training pairs are extracted automatically by fuzzy string matching of dictionary entries in the machine translation and the post-edit, accepting matches at a similarity threshold of 0.95.

What would settle it

Take a random sample of the 123,518 training pairs and manually identify the wrong term in the machine translation and the correction in the post-edit. If the fuzzy-matching pipeline disagrees with the manual labels in a substantial fraction of cases, or if randomly flipping the preferred and dispreferred terms in 10% of the training pairs does not reduce the reported term-accuracy gain on the test set, then the improvement is not being driven by the claimed term-preference signal.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that preference pairs built from machine translations and human post-edits, where the post-editor replaced one valid dictionary term with another, provide a discriminative training signal that separates semantically close term variants. The paper shows that a 7-billion-parameter translation model pre-trained on in-domain post-edits, then fine-tuned with a loss that combines supervised fine-tuning and preference optimization while masking non-term tokens, improves exact term accuracy from 53.7% to 56.3% on a held-out English-German test set with p < 0.05, while COMET scores do not significantly change. The gain is attributed to learning editor preferences and context rather than to decoding-time constraints or one-to-one dictionaries.

Load-bearing premise

The load-bearing premise is that fuzzy string matching reliably identifies which dictionary term the machine translation got wrong and which replacement term the post-editor chose, and that the post-editor's choice is the correct disambiguation.

Editorial extensions

If this is right

  • A dictionary with many valid translations per source term can be exploited without feeding the dictionary to the model or constraining decoding.
  • The best configuration, combining all four objectives (SFT, masked SFT, PO, masked PO), yields the only statistically significant term-accuracy gain with no significant COMET loss.
  • Training only on masked term tokens improves term accuracy but significantly lowers ChrF and COMET, so full-sequence supervised fine-tuning is needed to protect general translation quality.
  • Prompting a general-purpose LLM with the term dictionary can hurt disambiguation: exact term accuracy fell from 43.5% to 37.1% when the dictionary was added to the prompt, even though dictionary-valid term usage rose.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: because the preference signal is editor corrections, the same machinery should apply to other one-to-many style choices in corporate translation, such as formality levels or banned wording, whenever post-edits encode the preferred variant.
  • Beyond the paper: since 97.8% of test terms also appear in training, the reported gain may reflect memorization of term-context associations; a test split whose terms are absent from training would show whether the model truly disambiguates unseen terms.
  • Beyond the paper: the fuzzy-matching pipeline treats inflections as near-matches at threshold 0.95, so the extracted 'wrong term' and 'correction' may sometimes be the same lemma in different surface forms; lemmatization or morphological alignment could sharpen the preference signal.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes learning to disambiguate ambiguous terminology in English-to-German machine translation from post-edited corpora. It extracts preference pairs by fuzzy-matching dictionary terms in machine translations and post-edits, then fine-tunes Tower Base LLM 7B with a combination of supervised fine-tuning (SFT) and preference optimization (PO), including term-masked variants. On a held-out test set, the best configuration (setting 6: SFT + mSFT + PO + mPO) improves exact term accuracy from 53.7% to 56.3% with statistical significance, without significant COMET degradation. The authors also release terminology-annotated test sets and a dictionary.

Significance. If the result holds, the paper demonstrates a practical alternative to constrained decoding and soft-constraint prompting for terminology translation when the dictionary is ambiguous, requiring no one-to-one dictionary and no human input at inference time. The method is concrete and well-parameterized, hyperparameters are fully specified, and the use of pairwise approximate randomization testing is appropriate. The release of test sets with ambiguous terminology is a useful community resource. However, the central empirical claim depends on the reliability of the automated preference-pair construction, which is not validated, and on the absence of head-to-head comparisons with terminology-constrained baselines.

major comments (3)
  1. [Section 3.1] The criterion for accepting a preference pair is that the RapidFuzz term-match sets for the MT and PE are disjoint. This does not verify that the post-edit term is the replacement of the machine-translated term; a pair is accepted also when the MT term is dropped during rephrasing and a different dictionary term appears elsewhere in the post-edit. Because all 123,518 training pairs are produced in this way, the preference signal may be systematically mislabeled, and the reported term-accuracy gain of 53.7% to 56.3% rests on the unverified assumption that the matching identifies genuine corrections. The paper should validate a random sample of pairs manually, use a stricter alignment (e.g., requiring the matched spans to be aligned between MT and PE), or at least quantify how often the MT term is absent from the PE in accepted pairs.
  2. [Section 5 (Table 2)] The paper frames the contribution as an alternative to hard-constraint decoding and soft-constraint input, but all fine-tuning baselines are variants of the same model that do not use the dictionary at inference time. To support that claim, the paper should compare setting 6 against the same Tower baseline when the ambiguous dictionary is provided as additional input (soft constraint) or when decoding is constrained to allow any of the valid dictionary translations. The GPT-4.1 prompting results in Table 1 are not a controlled comparison because they use a different base model, so the improvement over the Tower baseline may reflect fine-tuning in general rather than the specific benefit of preference optimization on post-edits.
  3. [Section 4.3] The term-accuracy metric performs fuzzy matching of the model output against the post-edit term using partial_ratio_alignment with a threshold of 0.95. Because partial alignment of a substring counts as a match, a model output containing a shorter variant of the reference term (e.g., 'Überführung' when the reference is 'Warenüberführung') may be credited as correct, potentially inflating the reported term accuracy. The paper should either use a full-sequence or exact-match criterion, or explicitly remove overlapping term matches in evaluation exactly as is done in training pair construction in Section 3.1, and also report exact-match term accuracy as a sanity check.
minor comments (5)
  1. [Section 1] The phrase "to to learn terminology translation" contains a typo; it should read "to learn".
  2. [Section 2] In the related work, "one most know beforehand" should be "one must know beforehand".
  3. [Section 3.2] The sentence "the log of which is proportional to the arithmetic mean of log-probabilities" is imprecise; the log of the geometric mean is exactly the arithmetic mean of log-probabilities, not merely proportional.
  4. [Table 2] The column header "Init Model" is unclear because all rows report the same Tower Baseline; the text should explain that every fine-tuning run is initialized from the continued-pre-trained baseline.
  5. [Section 5] The paper does not report exact p-values or confidence intervals for the significant differences; providing these numbers would increase transparency and allow readers to assess the margin of the claimed improvement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the held-out supervised-learning claim is self-contained, and the dCPO self-citation and shared fuzzy-matching heuristic are not load-bearing reductions.

full rationale

The paper's central claim is an empirical comparison on a held-out 2000-example test set: setting 6 of the Lterm loss improves term accuracy from 53.7% to 56.3% with p < 0.05 by pairwise approximate randomization testing. The loss functions in Eqs. (3)-(7) are fully specified in the paper and do not take the test-set term accuracy as an input; training pairs come from a separate 123,518-example corpus, and the test set is released and not used for fitting. The dCPO loss is cited from Berger et al. (2024) with overlapping authors, but the paper redefines and modifies it, and the comparison among training settings is empirical; no load-bearing uniqueness or correctness claim rests on that citation alone. The shared RapidFuzz threshold between Section 3.1 pair construction and Section 4.3 term-accuracy evaluation is a metric-consistency and possible label-noise issue, not a definitional reduction: the model is not guaranteed to reproduce the post-edit term at test time, and the baseline reaches only 53.7%. The limitations section (Section 7) notes missing language pairs and alternative preference-optimization baselines, which are completeness caveats rather than circular steps. No equation reduces to its own input, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces new loss variants, masked SFT and masked PO with smooth-L1, but no new physical or conceptual entity with independent falsifiable handles. The central claim rests on three hand-set numerical parameters and four domain assumptions about post-edit reliability, dictionary authority, fuzzy-match correctness, and metric validity.

free parameters (3)
  • alpha (SFT loss weight) = 1 for SFT-only settings, 10 for PO combinations (Table 2)
    Hand-set to balance SFT against PO gradients; no tuning curve is reported, and the value changes across experimental settings.
  • beta (inverse temperature in PO loss) = 0.25 (Table 4)
    Hand-set; it controls the width of the preference margin in the PO loss and therefore the strength of term separation.
  • fuzzy match score threshold = 0.95 (Section 3.1 and Section 4.3)
    Chosen to allow minor inflections; it determines which MT/PE pairs enter training and which outputs count as correct in term accuracy, so it directly shapes the measured result.
assumptions (4)
  • domain assumption The post-edit target is the correct or preferred terminology choice, and the MT term is dispreferred.
    The whole PO objective in Section 3.2 treats PE terms as y_w and MT terms as y_l, but Section 1 admits that editors vary in their choices.
  • domain assumption The term dictionary is the authoritative set of valid translations.
    Term matching and term accuracy in Sections 3.1 and 4.3 only count dictionary terms; valid translations outside the dictionary are scored as incorrect.
  • ad hoc to paper Fuzzy matching with RapidFuzz at threshold 0.95 correctly identifies term occurrences and corrections.
    Both data construction and evaluation rely on this heuristic, and no manual audit of matched pairs is reported.
  • domain assumption Fine-tuned COMET on the authors' direct-assessment data is a valid quality measure.
    Section 4.3; the 'no significant COMET loss' claim depends on the reliability of this custom fine-tuned metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits." pith.science (2026). https://pith.science/paper/YAOEUSJJ

@misc{pith2026250703580,
  author       = {Pith},
  title        = {Pith review of: Learning to Translate Ambiguous Terminology by Preference Optimization on Post-Edits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YAOEUSJJ}},
  note         = {Machine review of arXiv:2507.03580}
}
read the original abstract

In real world translation scenarios, terminology is rarely one-to-one. Instead, multiple valid translations may appear in a terminology dictionary, but correctness of a translation depends on corporate style guides and context. This can be challenging for neural machine translation (NMT) systems. Luckily, in a corporate context, many examples of human post-edits of valid but incorrect terminology exist. The goal of this work is to learn how to disambiguate our terminology based on these corrections. Our approach is based on preference optimization, using the term post-edit as the knowledge to be preferred. While previous work had to rely on unambiguous translation dictionaries to set hard constraints during decoding, or to add soft constraints in the input, our framework requires neither one-to-one dictionaries nor human intervention at decoding time. We report results on English-German post-edited data and find that the optimal combination of supervised fine-tuning and preference optimization, with both term-specific and full sequence objectives, yields statistically significant improvements in term accuracy over a strong NMT baseline without significant losses in COMET score. Additionally, we release test sets from our post-edited data and terminology dictionary.

Figures

Figures reproduced from arXiv: 2507.03580 by the authors.

Figure 1
Figure 1. The term ’transfer’ is a highly ambiguous term [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. In order to train terminology corrected models, we begin by performing continued pre-training on [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Here we show a histogram of the number of unique source terms in the test set for a given amount of valid [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Mapping from a source term, ’transfer’, to multiple valid target terms. Our term dictionary is a one-to-many [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Sample prompts for GPT 4.1. The upper prompt shows the added terminology constraints while the lower [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 7 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Arash Ahmadian, Chris Cremer, Matthias Gall \'e , Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet \"U st \"u n, and Sara Hooker. 2024. https://doi.org/10.18653/v1/2024.acl-long.662 Back to basics: Revisiting REINFORCE -style optimization for learning from human feedback in LLM s . In Proceedings of the 62nd Annual Meeting of the Association for Co...

    doi:10.18653/v1/2024.acl-long.662 2024

  4. [4]

    Duarte Miguel Alves, Jos \'e Pombal, Nuno M Guerreiro, Pedro Henrique Martins, Jo \ a o Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, Jos \'e G. C. de Souza, and Andre Martins. 2024. https://openreview.net/forum?id=EHPns3hVkj Tower: An open multilingual large language model for translation-related tasks ....

    2024

  5. [5]

    Nathaniel Berger, Miriam Exel, Matthias Huck, and Stefan Riezler. 2024. https://doi.org/10.18653/v1/2024.wmt-1.122 Post-edits are preferences too . In Proceedings of the Ninth Conference on Machine Translation, pages 1289--1300, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.wmt-1.122 2024

  6. [6]

    Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul F. Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf Deep reinforcement learning from human preferences . In Advances in Neural Information Processing Systems (NIPS) , Long Beach, CA, USA

    work page 2017

  7. [7]

    Georgiana Dinu, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan. 2019. https://doi.org/10.18653/v1/P19-1294 Training neural machine translation to apply terminology constraints . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3063--3068, Florence, Italy. Association for Computational Linguistics

    doi:10.18653/v1/p19-1294 2019

  8. [8]

    Miriam Exel, Bianka Buschbeck, Lauritz Brandt, and Simona Doneva. 2020. https://aclanthology.org/2020.eamt-1.29/ Terminology-constrained neural machine translation at SAP . In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 271--280, Lisboa, Portugal. European Association for Machine Translation

    work page 2020

Show all 24 references
  1. [9]

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. https://proceedings.mlr.press/v238/gheshlaghi-azar24a.html A general theoretical paradigm to understand learning from human preferences . In Procee...

    2024

  2. [10]

    Guerreiro

    Hippolyte Gisserot-Boukhlef, Ricardo Rei, Emmanuel Malherbe, C \'e line Hudelot, Pierre Colombo, and Nuno M. Guerreiro. 2024. https://doi.org/10.18653/v1/2024.wmt-1.127 Is preference alignment always the best option to enhance LLM -based translation? an empirical analysis . In...

    2024 doi

  3. [11]

    Iikka Hauhio and Th \'e o Friberg. 2024. https://aclanthology.org/2024.eamt-1.12/ Mitra: Improving terminologically constrained translation quality with backtranslations and flag diacritics . In Proceedings of the 25th Annual Conference of the European Association for Machine ...

    2024

  4. [12]

    Chris Hokamp and Qun Liu. 2017. https://doi.org/10.18653/v1/P17-1141 Lexically constrained decoding for sequence generation using grid beam search . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1535--...

    2017 doi

  5. [13]

    Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A

    Nathan Lambert, Jacob Daniel Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James Validad Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris ...

    2024 arXiv

  6. [14]

    Kelleher, and Andy Way

    Yasmin Moslem, Rejwanul Haque, John D. Kelleher, and Andy Way. 2023. https://aclanthology.org/2023.eamt-1.22/ Adaptive machine translation with large language models . In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 227--...

    2023

  7. [15]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  8. [16]

    Matt Post and David Vilar. 2018. https://doi.org/10.18653/v1/N18-1119 Fast lexically constrained decoding with dynamic beam allocation for neural machine translation . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Lin...

    2018 doi

  9. [17]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. https://openreview.net/forum?id=HPuSIXJaa9 Direct preference optimization: Your language model is secretly a reward model . In Thirty-seventh Conference on Neural Infor...

    2023

  10. [18]

    Ricardo Rei, Jos \'e G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and Andr \'e F. T. Martins. 2022. https://aclanthology.org/2022.wmt-1.52/ COMET -22: Unbabel- IST 2022 submission for the metrics shared task . In Pr...

    2022

  11. [19]

    Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F

    Ricardo Rei, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F. T. Martins. 2025. https://arxiv.org/abs/2506.17080 Tower+: Bridging generality and translation specialization in multilingual llms . Preprint, arXiv:2506.17080

    2025 arXiv

  12. [20]

    Stefan Riezler and John T. Maxwell. 2005. https://aclanthology.org/W05-0908/ On some pitfalls in automatic evaluation and significance testing for MT . In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarizatio...

    2005

  13. [21]

    Amit Rozner, Barak Battash, Lior Wolf, and Ofir Lindenbaum. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.273 Knowledge editing in language models via adapted direct preference optimization . In Findings of the Association for Computational Linguistics: EMNLP 2024, pag...

    2024 doi

  14. [22]

    Kirill Semenov, Vil \'e m Zouhar, Tom Kocmi, Dongdong Zhang, Wangchunshu Zhou, and Yuchen Eleanor Jiang. 2023. https://doi.org/10.18653/v1/2023.wmt-1.54 Findings of the WMT 2023 shared task on machine translation with terminologies . In Proceedings of the Eighth Conference on ...

    2023 doi

  15. [23]

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. 2024. Contrastive preference optimization: pushing the boundaries of llm performance in machine translation. In Proceedings of the 41st International Conference...

    2024

  16. [24]

    Vil \'e m Zouhar, Shuoyang Ding, Anna Currey, Tatyana Badeka, Jenyuan Wang, and Brian Thompson. 2024. https://doi.org/10.18653/v1/2024.acl-short.45 Fine-tuned machine translation metrics struggle in unseen domains . In Proceedings of the 62nd Annual Meeting of the Association ...

    2024 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.