Pith. sign in

REVIEW 3 major objections 5 minor 29 references

Do Not Change Me: On Transferring Entities Without Modification in Neural Machine Translation -- a Multilingual Perspective

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A new 36,000-sentence benchmark measures how often machine translators preserve URLs, IBANs, emails, and emojis, and finds a large gap between the best and worst models.

desk verdict Useful new 36k no-translate entity benchmark, credible high-level rankings, but the regex-based ground truth has a missing emoji pattern and a phone pattern that masks '+' deletion — needs fixing before the numbers are trusted. read the letter →

arxiv 2505.06010 v1 pith:2IZXIMNU submitted 2025-05-09 cs.CL

Dawid Wisniewski , Mikolaj Pokrywka , Zofia Rostek This is my paper · ORCID
classification cs.CL
keywords no-translateentitiesneuralmachinetranslationentitytransferaccuracymultilingualbenchmarkemojitokenizationgranularityLevenshteindistancesyntheticdataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether popular neural machine translation models recognize text fragments that must be copied into the target language without modification—URLs, email addresses, IBANs, phone numbers, IP addresses, ISBNs, social media handles, alphanumeric codes, and emojis. To answer this, the authors generate a new synthetic dataset of 36,000 sentences in English, German, Polish, and Ukrainian, each containing exactly one such entity, and measure how often eight systems reproduce the entity character-for-character. The central finding is a wide quality gap: the best system, EuroLLM 9B, transfers entities correctly 95.89% of the time on a macro average, Google Translate 93.32%, and the OPUS family only 45.68%. The biggest failure class is emojis, where five of the eight systems score below 5.5%. If the dataset is representative, the paper supplies a working instrument for ranking models on this specific failure mode and for debugging why it happens.

What carries the argument

The load-bearing instrument is a new synthetic multilingual benchmark: 36,000 sentences, with 1,000 examples for each of nine entity categories in each of four languages, and each sentence containing exactly one entity detectable by hand-crafted regular expressions. The measurement mechanism is character-exact entity matching: a transfer counts as correct only when a regular expression for the expected category finds the identical character sequence in the source and target sentences, and mismatches are quantified by Levenshtein distance. The generation pipeline combines instruction-following sentence generation with filtering by language detection, length bucketing, grammar checking, and regex validation, which together make the dataset a controlled testbed for entity transfer.

What would settle it

Re-run the same evaluation on naturally occurring texts that contain the same entity types, or replace the regex-based ground truth with human annotations of whether each entity is unchanged; if the accuracy ordering of the eight systems changes substantially, the synthetic generation or the regexes are driving the result.

Watch

Extended reading notes

Core claim

The paper establishes that no-translate entity transfer is a separable, measurable quality axis in neural machine translation, not fully predicted by overall translation quality. Across all twelve translation directions among English, German, Polish, and Ukrainian, macro-average entity-transfer accuracy ranges from 95.89% for EuroLLM 9B to 45.68% for the OPUS family, with a global average of 78.65%. Numerical sequences such as IP addresses, phone numbers, and ISBNs are transferred well by nearly all models, while emojis are the main systemic failure; models without byte-level tokenization support almost always drop them. Error analysis shows partial translation, character substitution, repetition, and omission, with one-character edits the most common failure size, and longer entities more error-prone for some models but not others. The paper also demonstrates that a focused prompt instructing the model to keep non-translatable elements unchanged improves EuroLLM both on entity transfer and on CometKiwi translation quality.

Load-bearing premise

The rankings depend on the synthetic sentences generated by the language model being representative of how these entities actually appear in real texts, and on the regular expressions being an exact definition of an unchanged entity; if either fails, the model rankings may not transfer to production use.

Editorial extensions

If this is right

  • Entity-transfer accuracy is a distinct evaluation axis: a model can score acceptably on overall quality metrics yet still alter or drop an IBAN, URL, or email in the same sentence.
  • Adding one explicit instruction to keep non-translatable elements unchanged raised EuroLLM 9B's macro accuracy from 94.83% to 95.89%, so prompt design is a low-cost lever for this failure class.
  • Emoji transfer is a sharp diagnostic: five of the eight systems scored under 5.5% on emojis, making emoji presence enough to expose tokenizer-level limitations.
  • Tokenization granularity appears to matter: EuroLLM splits an example ISBN into 30 subtokens and transfers it well, while OPUS splits the same number into 12 chunks and struggles, suggesting that finer-grained tokenizers handle entities better.
  • The dataset gives a statistically stronger view than prior challenge sets, with 1,000 examples per language-category pair and all twelve translation directions covered rather than only translations into English.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of the prompt result: applying the same focused-instruction recipe to other instruction-tuned translation LLMs should improve their entity-transfer accuracy if the effect reflects a general property of explicit instructions rather than something specific to EuroLLM's training.
  • A consequence the paper leaves implicit is that production translation systems handling identifiers, bank details, or handles would benefit from a guardrail that detects these entities in the source and verifies their presence in the output, since even the best model still fails on about 4% of cases.
  • Because the ground truth is regex-based, the dataset measures 'string copied exactly' rather than 'entity preserved semantically'; a phone number reformatted with different spacing, or an emoji rendered as its text description, would count as wrong even if a human reader accepts the translation.
  • A concrete fix suggested by the emoji results: adding byte-level Unicode coverage for emojis to tokenizers should raise transfer accuracy without retraining the translation model, and this could be tested directly on the low-scoring systems.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript constructs a new synthetic multilingual benchmark for evaluating whether machine translation systems preserve no-translate entities. The dataset consists of 36,000 sentences in English, German, Polish, and Ukrainian, covering nine categories (alphanumeric sequences, emails, emojis, IBANs, IPs, ISBNs, phone numbers, social handles, URLs). Eight translation systems (OPUS, mBART, NLLB, M2M100, EuroLLM, MADLAD, SeamlessM4T, Google Translate) are evaluated across all 12 translation directions. Correct transfer is measured by extracting entities with hand-crafted regular expressions and comparing source and target sequences via Levenshtein distance. The main results (Table 2) show EuroLLM 9B as the best (95.89% macro-average), Google Translate second (93.32%), and OPUS worst (45.68%), with emoji being the most difficult category for most models. The authors also analyze error types, prompt effects for EuroLLM, and correlations with entity length and sentence length.

Significance. If the evaluation pipeline can be corrected and the numbers reproduced, the paper would provide a useful, larger-scale benchmark than existing entity-transfer sets such as ACES, with 1,000 examples per category/language cell and publicly released data and translations. The breadth is a clear strength: eight models, twelve directions, nine categories, and an error analysis that goes beyond binary accuracy. The emoji finding, however, rests on a detection mechanism that is not documented in the manuscript, and the phone-number regex is objectively incorrect for international prefixes. Until these issues are fixed, the headline rankings cannot be trusted. The CometKiwi sanity checks and the correlation analyses are informative additions, and the authors deserve credit for transparently reporting their generation and sampling procedures.

major comments (3)
  1. [3.2, 5.3, Appendix A, Table 2] The emoji accuracy numbers are not reproducible. Section 3.2 states that each sentence was filtered to contain exactly one entity of the expected category 'using regular expressions listed in Appendix A', and Section 5.3 states that the same expressions are used for source-target comparison. However, Appendix A lists patterns for only eight categories and contains no emoji pattern, even though emoji is one of the nine dataset categories and drives the most striking result in Table 2 (five models below 5.5%). Without a documented emoji detection rule, the emoji column, the no-match error fractions attributed to emojis in Section 6, and the corresponding Figure 4 results cannot be verified.
  2. [Appendix A (Phone regex), Table 10] The phone-number expression is defective for international numbers. The pattern starts with \b before a character class that includes '+'; because '+' is a non-word character, \b is false before '+49...', so the regex matches only from the first digit onward and never captures the leading '+'. Table 10 confirms this: srcEntity is '49 030 1234567890' while the source sentence contains '+49 030 1234567890'. A model that deletes the '+' is therefore counted as correct. Since phone numbers are among the highest-scoring categories in Table 2, this systematically inflates the phone accuracy for every model and distorts the macro-averages and the claim that IP, phone, and ISBN are the three easiest categories.
  3. [Section 6, 'Prompt selection for EuroLLM', Tables 2 and 5] The EuroLLM scores in the main comparison are obtained with a prompt chosen by evaluating two prompt variants on the same test set used for the headline evaluation. This is a form of test-set selection; the other models were evaluated only with their default configurations. The reported 95.89% for EuroLLM 9B is therefore not directly comparable to the other systems. The authors should either report both prompt conditions in the main table or select prompts on a held-out development set before computing the final rankings.
minor comments (5)
  1. [Table 3] The list of best directions repeats 'de→ pl' twice; one of the two entries should presumably be 'pl→ de'.
  2. [Table 5] The caption says 'Average change when using large models and smaller ones', but the table contains absolute accuracy values, not changes; the caption should be revised.
  3. [Table 8] Entries such as '1,225 (352)' and '1 (11)' are unexplained; please clarify what the parenthesized numbers denote.
  4. [Section 6] The text uses 'Spearmans's' where 'Spearman's' is intended.
  5. [Appendix C and Section 3.2] Appendix C acknowledges the synthetic nature of the test set, but Section 3.2 calls the resulting data a 'high-quality dataset' without reporting any human validation; the authors should state whether a manual inspection of a sample was performed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the benchmark consists of externally trained models and independent measurements, and no reported quantity is constructed from its own inputs.

full rationale

The paper makes no derivation claim that reduces to its inputs. The dataset is generated by an external model (Gemma 2), filtered by external language-tool checks, and then translated by externally trained NMT models. The ground-truth regexes in Appendix A define both category membership and the scoring rule; using the same definition to label and to score is the normal construction of a benchmark, not a circularity. The EuroLLM prompt comparison in Section 6 is a transparent configuration choice: the authors report both generic and focused prompts and openly state that the focused prompt was used. Selecting a prompt on the same test set can bias comparisons and is a validity or correctness risk, but it is not a fitted parameter renamed as a prediction, and it does not make the model's output equivalent to the test-set labels. No uniqueness theorem, no self-citation chain, and no ansatz smuggled via citation is load-bearing. Potential defects in the regex pipeline, such as the phone-pattern boundary anchoring, are measurement-validity concerns rather than circularity. The numeric rankings are externally falsifiable measurements of fixed model configurations, so the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical evaluation, so the 'axioms' are the measurement assumptions that define the ground truth and the comparison. No invented entities are introduced, and the main hand-chosen settings are the Gemma sampling parameters, the bucket sampling design, and the EuroLLM prompt selection.

free parameters (4)
  • Gemma 2 generation temperature = 1.2
    Sampling temperature used to generate the 720,000 candidate sentences (Section 3.1); affects sentence diversity and the resulting test distribution.
  • Gemma 2 top_p = 0.95
    Nucleus sampling parameter for sentence generation (Section 3.1).
  • Sentence buckets and samples per bucket = 20 buckets, 50 samples each
    Stratified length sampling in Section 3.2; defines the composition of the 1,000-example cells and thus the difficulty mix of the benchmark.
  • EuroLLM prompt template = focused prompt
    The authors selected the focused prompt after comparing it with a generic prompt on this test set (Table 5); this choice affects EuroLLM's reported accuracy and its ranking.
assumptions (5)
  • domain assumption The hand-crafted regular expressions in Appendix A correctly identify all and only the no-translate entities in source and target sentences.
    The entire accuracy metric is computed by regex matching; if a regex over- or under-matches, the reported accuracies change. This is the measurement definition and is not independently validated.
  • domain assumption The synthetic sentences generated by Gemma 2 are representative of real-world contexts for these entity types.
    The benchmark is entirely synthetic; the conclusions about model quality are assumed to transfer to production text (Section 3.1).
  • domain assumption Exact character identity between source and target entity is the correct criterion for 'without modification'; formatting changes are not allowed.
    The true positive condition in Section 6 requires the exact same character sequence; legitimate typographic variation would be counted as an error.
  • standard math Levenshtein distance is an appropriate measure of the degree of entity modification.
    Used in Section 5.3 to quantify differences; standard but a modeling choice.
  • domain assumption The language-tool filter in Section 3.2 removes ungrammatical sentences.
    Automated grammar checking is assumed sufficient; Table 10 contains a sentence that appears ungrammatical, so the filter's effectiveness is imperfect.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Not Change Me: On Transferring Entities Without Modification in Neural Machine Translation -- a Multilingual Perspective." pith.science (2026). https://pith.science/paper/2IZXIMNU

@misc{pith2026250506010,
  author       = {Pith},
  title        = {Pith review of: Do Not Change Me: On Transferring Entities Without Modification in Neural Machine Translation -- a Multilingual Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2IZXIMNU}},
  note         = {Machine review of arXiv:2505.06010}
}
read the original abstract

Current machine translation models provide us with high-quality outputs in most scenarios. However, they still face some specific problems, such as detecting which entities should not be changed during translation. In this paper, we explore the abilities of popular NMT models, including models from the OPUS project, Google Translate, MADLAD, and EuroLLM, to preserve entities such as URL addresses, IBAN numbers, or emails when producing translations between four languages: English, German, Polish, and Ukrainian. We investigate the quality of popular NMT models in terms of accuracy, discuss errors made by the models, and examine the reasons for errors. Our analysis highlights specific categories, such as emojis, that pose significant challenges for many models considered. In addition to the analysis, we propose a new multilingual synthetic dataset of 36,000 sentences that can help assess the quality of entity transfer across nine categories and four aforementioned languages.

Figures

Figures reproduced from arXiv: 2505.06010 by the authors.

Figure 1
Figure 1. Number of NLTK (Bird and Loper, 2004) tokens in sentences expressed in a given language. There is a greater variability in the number of to￾kens in sentences when analyzing texts by category. These values, presented in Appendix D, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distribution of the number of characters for each entity category. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Percentage of correctly transferred entities among different sentence lengths. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Distribution of the size of errors among models [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Percentage of entities not matched in target sentences among different lengths. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Percentage of modified entities among different lengths. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Number of NLTK (Bird and Loper, 2004) tokens per category. Sentences expressed in all languages were collected together for this analysis [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Errors distributions without emoji category [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 7 canonical work pages

  1. [1]

    Duarte Alves, Ricardo Rei, Ana C Farinha, Jos \'e G. C. de Souza, and Andr \'e F. T. Martins. 2022. https://aclanthology.org/2022.wmt-1.43/ Robust MT evaluation with sentence-level multilingual augmentation . In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 469--478, Abu Dhabi, United Arab Emirates (Hybrid). Association for Com...

    work page 2022

  2. [2]

    Alves, Jos \' e Pombal, Nuno Miguel Guerreiro, Pedro Henrique Martins, Jo \ a o Alves, M

    Duarte M. Alves, Jos \' e Pombal, Nuno Miguel Guerreiro, Pedro Henrique Martins, Jo \ a o Alves, M. Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, Jos \' e G. C. de Souza, and Andr \' e F. T. Martins. 2024. https://doi.org/10.48550/ARXIV.2402.17733 Tower: An open multilingual large language model for translation-...

    arXiv doi:10.48550/arxiv.2402.17733 2024

  3. [3]

    Chantal Amrhein, Nikita Moghe, and Liane Guillou. 2022. https://aclanthology.org/2022.wmt-1.44 ACES : Translation accuracy challenge sets for evaluating machine translation metrics . In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 479--513, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics

    work page 2022

  4. [4]

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. http://arxiv.org/abs/1409.0473 Neural machine translation by jointly learning to align and translate . In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings

    arXiv 2015

  5. [5]

    Lo \" c Barrault et al. 2023. https://doi.org/10.48550/ARXIV.2308.11596 Seamlessm4t-massively multilingual & multimodal machine translation . CoRR, abs/2308.11596

    arXiv doi:10.48550/arxiv.2308.11596 2023

  6. [6]

    Steven Bird and Edward Loper. 2004. https://aclanthology.org/P04-3031/ NLTK : The natural language toolkit . In Proceedings of the ACL Interactive Poster and Demonstration Sessions , pages 214--217, Barcelona, Spain. Association for Computational Linguistics

    2004

  7. [7]

    Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, Chang Zhou, and Jingren Zhou. 2024. https://doi.org/10.48550/ARXIV.2407.10759 Qwen2-audio technical report . CoRR, abs/2407.10759

    arXiv doi:10.48550/arxiv.2407.10759 2024

  8. [8]

    Costa - juss \` a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Y

    Marta R. Costa - juss \` a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Y. Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Lo \" c Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Ro...

    arXiv doi:10.48550/arxiv.2207.04672 2022

Show all 29 references
  1. [9]

    Abhimanyu Dubey et al. 2024. https://doi.org/10.48550/ARXIV.2407.21783 The llama 3 herd of models . CoRR, abs/2407.21783

    2024 doi arXiv

  2. [10]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L \' e lio Renard Lavaud, Marie - Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, ...

    2023 doi arXiv

  3. [11]

    Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, Andr\' e F. T. Martins, and Alexandra Birch. 2018. http://www.aclweb.org/anthology/P18-4020 Marian: Fast ne...

    2018

  4. [12]

    Marzena Karpinska, Nishant Raj, Katherine Thai, Yixiao Song, Ankita Gupta, and Mohit Iyyer. 2022. Demetr: Diagnosing evaluation metrics for translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing

    2022

  5. [13]

    Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/d49042a5d49818711c401d34172f9900-Abstract-Datasets\_and\_Benchmarks.html MADLAD-400: A m...

    2023

  6. [14]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. https://doi.org/10.1145/3600006.3613165 Efficient memory management for large language model serving with pagedattention . In Proceedings of the 29...

    2023

  7. [15]

    Samuel L \" a ubli, Rico Sennrich, and Martin Volk. 2018. https://aclanthology.org/D18-1512/ Has machine translation achieved human parity? A case for document-level evaluation . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussel...

    2018

  8. [16]

    Arle Lommel, Hans Uszkoreit, and Aljoscha Burchardt. 2014. Multidimensional quality metrics (mqm): A framework for declaring and describing translation quality metrics. Tradum \`a tica , (12):0455--463

    2014

  9. [17]

    Alves, Jos \' e Pombal, M

    Pedro Henrique Martins, Patrick Fernandes, Jo \ a o Alves, Nuno Miguel Guerreiro, Ricardo Rei, Duarte M. Alves, Jos \' e Pombal, M. Amin Farajian, Manuel Faysse, Mateusz Klimaszewski, Pierre Colombo, Barry Haddow, Jos \' e G. C. de Souza, Alexandra Birch, and Andr \' e F. T. M...

    2024 doi arXiv

  10. [18]

    Marcin Mi kowski. 2010. Developing an open-source, rule-based proofreading tool. Softw. Pract. Exper., 40(7):543–566

    2010

  11. [19]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, ...

    2002

  12. [20]

    Ricardo Rei, Jos \' e G. C. de Souza, Duarte M. Alves, Chrysoula Zerva, Ana C. Farinha, Taisiya Glushkova, Alon Lavie, Lu \' sa Coheur, and Andr \' e F. T. Martins. 2022 a . https://aclanthology.org/2022.wmt-1.52 COMET-22: unbabel-ist 2022 submission for the metrics shared tas...

    2022

  13. [21]

    Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, Jos \'e G. C. de Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, Alon Lavie, and Andr \'e F. T. Martins. 2022 b . https://aclanthology.org/2022.wmt-1.60 C omet K iwi: IST -u...

    2022

  14. [22]

    Morgane Rivi \` e re et al. 2024. https://doi.org/10.48550/ARXIV.2408.00118 Gemma 2: Improving open language models at a practical size . CoRR, abs/2408.00118

    2024 doi arXiv

  15. [23]

    Felix Stahlberg. 2020. https://doi.org/10.1613/JAIR.1.12007 Neural machine translation: A review . J. Artif. Intell. Res., 69:343--418

    2020 doi

  16. [24]

    Yuqing Tang, Chau Tran, Xian Li, Peng - Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. https://arxiv.org/abs/2008.00401 Multilingual translation with extensible multilingual pretraining and finetuning . CoRR, abs/2008.00401

    2020 arXiv

  17. [25]

    o rg Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig - Arne Gr \

    J \" o rg Tiedemann, Mikko Aulamo, Daria Bakshandaeva, Michele Boggia, Stig - Arne Gr \" o nroos, Tommi Nieminen, Alessandro Raganato, Yves Scherrer, Ra \' u l V \' a zquez, and Sami Virpioja. 2024. https://doi.org/10.1007/S10579-023-09704-W Democratizing neural machine transl...

    2024 doi

  18. [26]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Attention is all you need . In Advances in Neural I...

    2017

  19. [27]

    Yangjian Wu and Gang Hu. 2023. https://doi.org/10.18653/V1/2023.WMT-1.15 Exploring prompt engineering with GPT language models for document-level machine translation: Insights and findings . In Proceedings of the Eighth Conference on Machine Translation, WMT 2023, Singapore, D...

    2023 doi

  20. [28]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  21. [29]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.