Pith. sign in

REVIEW 3 major objections 6 minor 61 references

Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fine-tuning mBART-50 on the new Translate-with-Care dataset yields substantially more accurate and less biased pronoun translation from six genderless languages into English than GPT-4, Google Translate, and other…

desk verdict A useful new benchmark and a fine-tuning result, but the headline comparison to GPT-4 depends on a scoring rule that only accepts 'one' as neutral—so the paper needs a human-acceptability or re-scoring check before the big claims can be trusted. read the letter →

arxiv 2506.00748 v1 pith:HC4FDMRZ submitted 2025-05-31 cs.CL cs.CY

Pardis Sadat Zahraei , Ali Emami This is my paper
classification cs.CLcs.CY
keywords genderbiasinmachinetranslationgenderlesslanguagespronounresolutiongender-neutralTranslate-with-CaredatasetmBART-50fine-tuninglow-resourcecross-lingualtransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Translation from a genderless language into English forces a choice of pronoun that the source sentence does not make. The paper introduces Translate-with-Care (TWC), 3,950 short scenarios in Persian, Turkish, Finnish, Indonesian, Estonian, and Azerbaijani, to test how well systems handle that choice. It reports that every evaluated system, including GPT-4 and Google Translate, defaults to masculine pronouns in stereotype-relevant contexts and often fails at the logical reasoning needed to pick the right pronoun. Fine-tuning the small open mBART-50 model on part of TWC raises accuracy on the TWC test set to 87.6%, compared with 35.4% for GPT-4 and 22.8% for Google Translate, and makes the model choose the neutral pronoun "one" in 93.37% of Bias instances. A sympathetic reader would take the paper as showing that targeted fine-tuning on a genderless-language benchmark can beat much larger proprietary systems at gender-fair translation while remaining open-source.

What carries the argument

The load-bearing mechanism is the TWC dataset paired with the authors' correctness convention: each instance is a short English sentence with two antecedents (names, titles, or roles), machine-translated into a target genderless language and post-edited, then labeled as Bias (a stereotype may intrude), Neutrality (no information resolves gender), or Reasoning (context determines the pronoun). The ground truth for Bias and Neutrality renders the genderless pronoun as "one" rather than "he", "she", or "they". The intervention that carries the result is fine-tuning mBART-50 on the augmented personal-name training split, with a test set deliberately containing unseen languages and content to probe generalization. This design separates the three challenges and makes the pronoun the unit of measurement.

What would settle it

Run a native-speaker study on a sample of TWC Bias and Neutrality instances in which participants choose between "one", singular "they", a gendered pronoun, or "no gender implied" as the most natural accurate translation; if "they" is preferred or "one" is judged unnatural, the benchmark's ground truth, and therefore the fine-tuned model's reported advantage, would not survive that evaluation standard.

Watch

Extended reading notes

Core claim

The central discovery is that current translation systems have two systematic failure modes when the source language has no grammatical gender: they inject gender stereotypically, and they lose logical coherence. On the paper's own evaluation, all baselines prefer masculine pronouns when a stereotype is available, with Google Translate and GPT-4 using male pronouns four to six times more often than female pronouns in leadership and professional-success contexts. The paper's positive claim is that fine-tuning mBART-50 on the TWC training set reverses this: the fine-tuned model not only generalizes to unseen languages (Estonian and Azerbaijani), human-written sentences, and title/role antecedents, but also surpasses GPT-4 on the overall TWC test set, 87.6% to 35.4%, while staying open-source.

Load-bearing premise

The benchmark's gold labels treat the gender-neutral pronoun "one" as the only correct English rendering in Bias and Neutrality instances, so the reported accuracy numbers depend on that stylistic choice being the right standard.

Editorial extensions

If this is right

  • If the TWC results hold, an open-source fine-tuned model can provide gender-fair translation from genderless languages without relying on proprietary APIs.
  • The strong cross-lingual transfer measured for the fine-tuned model implies that pronoun-handling skills learned on one language family can reduce the amount of labeled data needed for another genderless language.
  • Because fine-tuning on TWC lowered general translation quality on OPUS-100, practical systems would need to decide when to route through a fairness-specialized model versus a general one.
  • The universal masculine-default behavior of baselines means that deploying current translation systems on genderless-to-English content will systematically misgender people unless a targeted intervention is applied.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the evaluation rewards only "one", the paper's advantage over GPT-4 may shrink or reverse if the standard is singular "they" or another gender-neutral strategy; this is an editorial inference, not a claim in the paper.
  • The TWC template uses short two-antecedent sentences; whether the fine-tuned model's pronoun reasoning survives longer, multi-antecedent text is an open question that follows naturally from the paper's design.
  • If the cross-lingual transfer result reproduces, a practical recipe would be to collect small high-quality genderless-pronoun instances in one language, fine-tune a compact translation model, and evaluate on a held-out language before larger data collection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces TWC, a benchmark of 3,950 English-translated instances from six genderless languages (Persian, Indonesian, Finnish, Turkish, Estonian, Azerbaijani), with three challenge types: Bias, Neutrality, and Reasoning. The authors evaluate several MT systems (GPT-4, Google Translate, mBART-50, NLLB-200 variants, SeamlessM4T) and find that all exhibit masculine bias and reasoning errors. They fine-tune mBART-50 on a subset of TWC (mBART-ft-TWC and mBART-id-ft-TWC) and report that the fine-tuned models substantially improve accuracy, reaching 87.6% overall vs. 35.4% for GPT-4 on the TWC test set, with 93.37% use of the neutral pronoun 'one' in Bias instances. The test set includes unseen languages (Estonian, Azerbaijani), unseen titles/roles, and human-generated content, and the paper also reports a slight degradation on OPUS-100, quantifying the specialization trade-off.

Significance. If the results hold, TWC is a valuable resource for a relatively neglected direction in MT evaluation: genderless-to-gendered translation with an explicit focus on bias, neutrality, and reasoning. The held-out design (unseen languages, titles/roles, human-written instances) is a genuine attempt to assess cross-lingual and content-type generalization, and the release of data, code, and models supports reproducibility. The OPUS-100 analysis honestly documents a quality trade-off. The main caveat is that the Bias/Neutrality correctness criterion is defined by the authors' choice that the only correct neutral pronoun is 'one'; the headline superiority of the fine-tuned model is therefore measured against that specific convention rather than against a broader notion of acceptable gender-neutral translation.

major comments (3)
  1. [Section 2, 'The TWC Translation Task'] The ground-truth labels for Bias and Neutrality (together 61.2% of the test set, per Appendix Table 12) encode the authors' policy that a gender-neutral pronoun must be rendered as 'one' (Section 2: 'We chose to translate gender-neutral pronouns to one for clarity and consistency'). Consequently, the reported accuracy gap between mBART-ft-TWC (87.6%) and GPT-4 (35.4%) largely measures how often a model emits the exact token 'one', not whether it produces a gender-neutral translation. Since Section 4.3 admits this strategy may 'seem unnatural,' the paper should justify this convention empirically (e.g., a human acceptability study) or provide an alternative-reference evaluation that counts singular 'they' and other neutral reformulations as correct. Without this, the claim 'surpassed proprietary LLMs' is conditional on a self-imposed labeling convention.
  2. [Section 3, 'Evaluation Metrics' and Figure 2/Tables 14–16] No confidence intervals, bootstrap estimates, or significance tests are reported for any of the accuracy comparisons. For instance, the overall 87.6% vs. 35.4% gap is large, but finer-grained comparisons (e.g., Azerbaijani Reasoning in Table 14: mBART-ft-TWC 71.60 vs. GPT-4 82.23) suggest the ranking may not be uniform. The authors should report paired significance tests (e.g., McNemar's test) or at minimum bootstrap CIs for the key comparisons.
  3. [Section 3, 'Evaluation Metrics'] The automatic pronoun-extraction script is not specified precisely enough to rule out measurement artifacts. It is unclear how the script handles 'one' when it occurs as a numeral or in 'one of them,' how it treats singular 'they,' and what happens when a model avoids a pronoun altogether (e.g., via restructuring). The sentence 'we also conducted a manual review... with no errors detected' is reassuring but should be replaced with a quantitative evaluation of the script's precision/recall on a gold-standard subset.
minor comments (6)
  1. [Footnote 1] The items 'GitHub Repository, TWC Dataset, mBART-ft-TWC Model, and mBART-id-ft-TWC Model' appear to be placeholder links; actual URLs should be provided.
  2. [Table 5 caption] The statement 'The Reasoning category has a near-balanced he/she distribution of 1.1' is not supported by the data in the table, which show widely varying he/she ratios across models (e.g., Google Translate 85.18% he vs. 12.35% she); please clarify or remove this statement.
  3. [Section 3, 'Training Data Preparation'] The training set is described as covering Persian, Turkish, Finnish, and Indonesian, but mBART-ft-TWC is fine-tuned on Turkish, Persian, and Indonesian, leaving Finnish unaccounted for. Please clarify which languages are included in each fine-tuned model and whether Finnish is treated as seen or unseen.
  4. [Table 19] The language name 'Azarbaijani' is misspelled; it should be 'Azerbaijani.'
  5. [Section 4.3, 'Qualitative Differences'] The claim that mBART-ft-TWC 'consistently outperforms other systems' in three key areas conflicts with the quantitative results in Table 14, where GPT-4 has higher Reasoning accuracy for every language (e.g., Persian: mBART-ft-TWC 84.79 vs. GPT-4 89.23); please qualify the qualitative claim.
  6. [Abstract and Conclusion] The abstract and conclusion state that the fine-tuned model 'surpassed proprietary LLMs,' but Section 4.1 notes that GPT-4 slightly outperformed in the Reasoning category; please add this qualification to the abstract or revise the claim.

Circularity Check

1 steps flagged · score 4.0 of 10

The headline mBART-ft-TWC advantage is largely measured on the authors' own 'one'-only neutral-pronoun rubric, which is both the training target and the scoring rule, making the central comparison partially self-definitional.

  1. self definitional [Section 2 (TWC task definition) and Section 4.5 (Bias results)]
    "For the Bias and Neutrality examples, R for P’s pronoun antecedent is undefined, so Hän and o are translated to the gender-neutral pronoun one. ... To evaluate correctness in the TWC task, we consider a system’s translation output, T ′, to be correct if it contains the appropriately translated pronoun R′ that aligns with the correct antecedent R. ... mBART-ft-TWC selected gender-neutral pronouns in 93.37% of bias instances ... In contrast, GPT-4 selected gender-neutral pronouns in only 2.5% of bias cases."

    The correctness rule for Bias and Neutrality instances (together 61.2% of the test set) is by definition that the only acceptable pronoun is 'one'. The fine-tuned model was trained on TWC examples whose ground-truth translations use this same 'one' convention, and the evaluation script scores exactly this convention. The reported 93.37% 'one' rate and the large gap over GPT-4 therefore largely measure how closely the model reproduces the authors' own labeling rule rather than an independently established property of gender-fair translation.

full rationale

The paper is a benchmark-and-finetuning study, and most of its pipeline is not circular: the test set is held out, includes unseen languages (Estonian, Azerbaijani), unseen antecedent types (titles, roles), and human-written instances; the OPUS-100 evaluation provides an external check of the specialization trade-off; and the reasoning category has an independently resolvable ground truth. The main circularity concern is confined to the Bias and Neutrality categories, where the definition of correctness ('one') is identical to the training target, so the headline comparison against GPT-4 and Google Translate is partially self-definitional. This is a genuine construct-validity issue rather than an internal inconsistency, and it does not invalidate the held-out generalization results. The self-citation to the authors' Tree-of-Experts work is used for dataset generation but is not load-bearing for the central evaluation claim. Overall score 4 reflects one definitional circularity in the central benchmark comparison while acknowledging the independent evidence that keeps the paper from being fully circular.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted constants appear in the paper; the method is supervised fine-tuning of a pretrained model, and the hyperparameters are standard choices rather than parameters the central claim depends on. The main assumptions are the task definition (the 'one' convention), the linguistic characterization of the source languages, and the naturalness of the machine-translated source sentences.

assumptions (4)
  • domain assumption The six source languages use a single third-person singular pronoun with no gender information (Persian 'او', Turkish 'o', Finnish 'hän', Indonesian 'dia', Estonian 'ta', Azerbaijani 'o').
    Invoked throughout Section 1.2 and Table 9; the entire task presumes the source pronoun is genderless.
  • ad hoc to paper For Bias and Neutrality instances, the only correct English rendering of the ambiguous pronoun is the singular 'one'; 'they', neopronouns, and gendered pronouns are treated as wrong.
    Adopted in Section 2 to define ground-truth translations and the accuracy metric; this is a paper-specific normalization, not a fact about English.
  • domain assumption The GPT-4-generated English sentences, after human editing, contain unambiguous contextual clues for the Reasoning instances and no usable clue for Bias and Neutrality instances.
    Underpins the labels R and T in Section 2.1; no inter-annotator agreement or external validation of this labeling is reported.
  • domain assumption Machine-translating the curated English sentences with Google Translate and then post-editing produces valid, natural source-language sentences for all six languages.
    Section 2.1 describes this pipeline; if the resulting sentences are translationese, model scores may not reflect performance on naturally occurring text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations." pith.science (2026). https://pith.science/paper/HC4FDMRZ

@misc{pith2026250600748,
  author       = {Pith},
  title        = {Pith review of: Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HC4FDMRZ}},
  note         = {Machine review of arXiv:2506.00748}
}
read the original abstract

Addressing gender bias and maintaining logical coherence in machine translation remains challenging, particularly when translating between natural gender languages, like English, and genderless languages, such as Persian, Indonesian, and Finnish. We introduce the Translate-with-Care (TWC) dataset, comprising 3,950 challenging scenarios across six low- to mid-resource languages, to assess translation systems' performance. Our analysis of diverse technologies, including GPT-4, mBART-50, NLLB-200, and Google Translate, reveals a universal struggle in translating genderless content, resulting in gender stereotyping and reasoning errors. All models preferred masculine pronouns when gender stereotypes could influence choices. Google Translate and GPT-4 showed particularly strong bias, favoring male pronouns 4-6 times more than feminine ones in leadership and professional success contexts. Fine-tuning mBART-50 on TWC substantially resolved these biases and errors, led to strong generalization, and surpassed proprietary LLMs while remaining open-source. This work emphasizes the need for targeted approaches to gender and semantic coherence in machine translation, particularly for genderless languages, contributing to more equitable and accurate translation systems.

Figures

Figures reproduced from arXiv: 2506.00748 by the authors.

Figure 1
Figure 1. Comparison of GPT-4, on TWC instances with the performance of mBART-ft-TWC, a fine-tuned version [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparative performance (accuracy) of translation models. (a) Accuracy on each category of the test set. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 27 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Ashish Agrawal, Barah Fazili, and Preethi Jyothi. 2024. https://aclanthology.org/2024.eacl-short.28 Translation errors significantly impact low-resource languages in cross-lingual learning . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 319--329, St. Julian ' ...

    work page 2024

  4. [4]

    Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2023. https://doi.org/10.18653/v1/2023.findings-acl.564 In-context examples selection for machine translation . In Findings of the Association for Computational Linguistics: ACL 2023, pages 8857--8873, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-acl.564 2023

  5. [5]

    Nur Ahmed and Muntasir Wahed. 2020. The de-democratization of ai: Deep learning and the compute divide in artificial intelligence research. arXiv preprint arXiv:2010.15581

    arXiv 2020

  6. [6]

    Murtaza Ali, Sourojit Ghosh, Prerna Rao, Raveena Dhegaskar, Sophia Jawort, Alix Medler, Mengqi Shi, and Sayamindu Dasgupta. 2023. https://doi.org/10.1145/3568813.3600120 Taking stock of concept inventories in computing education: A systematic literature review . In Proceedings of the 2023 ACM Conference on International Computing Education Research V.1, I...

    arXiv 2023

  7. [7]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  8. [8]

    Lo \" c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, et al. 2023. Seamless: Multilingual expressive and streaming speech translation. arXiv preprint arXiv:2312.05187

    arXiv 2023

Show all 61 references
  1. [9]

    Josh Barua, Sanjay Subramanian, Kayo Yin, and Alane Suhr. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.278 Using language models to disambiguate lexical choices in translation . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pag...

    2024 doi

  2. [10]

    Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. https://doi.org/10.1145/3442188.3445922 On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Tra...

    2021

  3. [11]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  4. [12]

    Bryson, and Arvind Narayanan

    Aylin Caliskan, Joanna J. Bryson, and Arvind Narayanan. 2017. https://doi.org/10.1126/science.aal4230 Semantics derived automatically from language corpora contain human-like biases . Science, 356(6334):183–186

    2017 doi

  5. [13]

    Niccol \`o Campolungo, Federico Martelli, Francesco Saina, Roberto Navigli, et al. 2022. Dibimt: A novel benchmark for measuring word sense disambiguation biases in machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ...

    2022

  6. [14]

    Marine Carpuat and Dekai Wu. 2007. https://aclanthology.org/D07-1007 Improving statistical machine translation using word sense disambiguation . In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language L...

    2007

  7. [15]

    Yee Seng Chan, Hwee Tou Ng, and David Chiang. 2007. Word sense disambiguation improves statistical machine translation. In Proceedings of the 45th annual meeting of the association of computational linguistics, pages 33--40

    2007

  8. [16]

    Heeyoul Choi, Kyunghyun Cho, and Yoshua Bengio. 2017. https://doi.org/10.1016/j.csl.2017.01.007 Context-dependent word representation for neural machine translation . Computer Speech & Language, 45:149--160

    2017 doi

  9. [17]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1--113

    2023

  10. [18]

    Marta R Costa-juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    2022 arXiv

  11. [19]

    Anna Currey, Maria Nadejde, Raghavendra Reddy Pappagari, Mia Mayer, Stanislas Lauly, Xing Niu, Benjamin Hsu, and Georgiana Dinu. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.288 MT - G en E val: A counterfactual and contextual dataset for evaluating gender accuracy in mac...

    2022 doi

  12. [20]

    Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Pr...

    2015

  13. [21]

    Denis Emelin and Rico Sennrich. 2021. Wino-x: Multilingual winograd schemas for commonsense reasoning and coreference resolution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8517--8532

    2021

  14. [22]

    Mikel L Forcada, Mireia Ginest \' -Rosell, Jacob Nordfalk, Jim O’Regan, Sergio Ortiz-Rojas, Juan Antonio P \'e rez-Ortiz, Felipe S \'a nchez-Mart \' nez, Gema Ram \' rez-S \'a nchez, and Francis M Tyers. 2011. Apertium: a free/open-source platform for rule-based machine transl...

    2011

  15. [23]

    Marjan Ghazvininejad, Hila Gonen, and Luke Zettlemoyer. 2023. Dictionary-based phrase-level prompting of large language models for machine translation. arXiv preprint arXiv:2302.07856

    2023 arXiv

  16. [24]

    Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting. In Findings of the Association for Computational Linguistics: ...

    2023

  17. [25]

    Vivek Iyer, Pinzhen Chen, and Alexandra Birch. 2023. https://doi.org/10.18653/v1/2023.wmt-1.44 Towards effective disambiguation for machine translation with large language models . In Proceedings of the Eighth Conference on Machine Translation, pages 482--495, Singapore. Assoc...

    2023 doi

  18. [26]

    Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean

    Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. https://doi.org/10.1162/tacl_a_00065 G oogle ' s multilingual neural machine translat...

    2017 doi

  19. [27]

    Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. https://doi.org/10.18653/v1/2020.acl-main.560 The state and fate of linguistic diversity and inclusion in the NLP world . In Proceedings of the 58th Annual Meeting of the Association for Co...

    2020 doi

  20. [28]

    Seza Do g ru \"o z, and En-Shiun Lee

    Eric Khiu, Hasti Toossi, Jinyu Liu, Jiaxu Li, David Anugraha, Juan Flores, Leandro Roman, A. Seza Do g ru \"o z, and En-Shiun Lee. 2024. https://aclanthology.org/2024.findings-eacl.100 Predicting machine translation performance on low-resource languages: The role of domain sim...

    2024

  21. [29]

    Yunsu Kim, Miguel Gra c a, and Hermann Ney. 2020. https://aclanthology.org/2020.eamt-1.5 When and why is unsupervised neural machine translation useless? In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 35--44, Lisboa, Por...

    2020

  22. [30]

    Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting ...

    2007

  23. [31]

    Surafel Melaku Lakew, Mauro Cettolo, and Marcello Federico. 2018. A comparison of transformer and recurrent neural networks on multilingual neural machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 641--652

    2018

  24. [32]

    Minwoo Lee, Hyukhun Koh, Minsung Kim, and Kyomin Jung. 2024. https://doi.org/10.18653/v1/2024.naacl-long.303 Fine-grained gender control in machine translation with large language models . In Proceedings of the 2024 Conference of the North American Chapter of the Association f...

    2024 doi

  25. [33]

    Chin-Yew Lin and FJ Och. 2004. Looking for a few good metrics: Rouge and its evaluation. In Ntcir workshop

    2004

  26. [34]

    Frederick Liu, Han Lu, and Graham Neubig. 2018. https://doi.org/10.18653/v1/N18-1121 Handling homographs in neural machine translation . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Techno...

    2018 doi

  27. [35]

    Kelleher, and Andy Way

    Yasmin Moslem, Rejwanul Haque, John D. Kelleher, and Andy Way. 2023 a . https://aclanthology.org/2023.eamt-1.22 Adaptive machine translation with large language models . In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 227...

    2023

  28. [36]

    Yasmin Moslem, Rejwanul Haque, and Andy Way. 2023 b . https://arxiv.org/abs/2312.12740 Fine-tuning large language models for adaptive machine translation . Preprint, arXiv:2312.12740

    2023 arXiv

  29. [37]

    Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, et al. 2020. Participatory research for low-resourced machine translation: A case...

    2020

  30. [38]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 Gpt-4 technical report . Preprint, arXiv:2303.08774

    2023 arXiv

  31. [39]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  32. [40]

    Andrea Piergentili, Beatrice Savoldi, Dennis Fucci, Matteo Negri, and Luisa Bentivogli. 2023. https://arxiv.org/abs/2310.05294 Hi guys or hi folks? benchmarking gender-neutral machine translation with the gente corpus . Preprint, arXiv:2310.05294

    2023 arXiv

  33. [41]

    Xiao Pu, Nikolaos Pappas, James Henderson, and Andrei Popescu-Belis. 2018. Integrating weakly supervised word sense disambiguation into neural machine translation. Transactions of the Association for Computational Linguistics, 6:635--649

    2018

  34. [42]

    Alessandro Raganato, Yves Scherrer, and J \"o rg Tiedemann. 2019. The mucow test suite at wmt 2019: Automatically harvested multilingual contrastive word sense disambiguation test sets for machine translation. In Proceedings of the Fourth Conference on Machine Translation (Vol...

    2019

  35. [43]

    Alessandro Raganato, Yves Scherrer, and J \"o rg Tiedemann. 2020. An evaluation benchmark for testing the word sense disambiguation capabilities of machine translation systems. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3668--3675

    2020

  36. [44]

    Spencer Rarrick, Ranjita Naik, Varun Mathur, Sundar Poudel, and Vishal Chowdhary. 2023. https://arxiv.org/abs/2303.03975 Gate: A challenge set for gender-ambiguous translation examples . Preprint, arXiv:2303.03975

    2023 arXiv

  37. [45]

    Abudurexiti Reheman, Tao Zhou, Yingfeng Luo, Di Yang, Tong Xiao, and Jingbo Zhu. 2023. https://doi.org/10.1609/aaai.v37i11.26585 Prompting neural machine translation with translation memories . Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13519--13527

    2023 doi

  38. [46]

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. https://arxiv.org/abs/2009.09025 Comet: A neural framework for mt evaluation . Preprint, arXiv:2009.09025

    2020 arXiv

  39. [47]

    Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2021. https://doi.org/10.1162/tacl_a_00401 Gender bias in machine translation . Transactions of the Association for Computational Linguistics, 9:845--874

    2021 doi

  40. [48]

    Beatrice Savoldi, Andrea Piergentili, Dennis Fucci, Matteo Negri, and Luisa Bentivogli. 2024. https://aclanthology.org/2024.eacl-short.23 A prompt response to the demand for automatic gender-neutral translation . In Proceedings of the 18th Conference of the European Chapter of...

    2024

  41. [49]

    Yves Scherrer, Alessandro Raganato, and J \"o rg Tiedemann. 2020. https://aclanthology.org/2020.wmt-1.40 The MUCOW word sense disambiguation test suite at WMT 2020 . In Proceedings of the Fifth Conference on Machine Translation, pages 365--370, Online. Association for Computat...

    2020

  42. [50]

    Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers, pages 223--231

    2006

  43. [51]

    Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. https://doi.org/10.18653/v1/P19-1355 Energy and policy considerations for deep learning in NLP . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645--3650, Florence, Ita...

    2019 doi

  44. [52]

    Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, and Tie-Yan Liu. 2018. Multilingual neural machine translation with knowledge distillation. In International Conference on Learning Representations

    2018

  45. [53]

    Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2020. https://arxiv.org/abs/2008.00401 Multilingual translation with extensible multilingual pretraining and finetuning . Preprint, arXiv:2008.00401

    2020 arXiv

  46. [54]

    Warren Weaver. 1952. Translation. In Proceedings of the Conference on Mechanical Translation

    1952

  47. [55]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. https://arxiv.org/abs/2201.11903 Chain-of-thought prompting elicits reasoning in large language models . Preprint, arXiv:2201.11903

    2023 arXiv

  48. [56]

    Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. 2023. Bigtrans: Augmenting large language models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098

    2023 arXiv

  49. [57]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601

    2023 arXiv

  50. [58]

    Pardis Zahraei and Ali Emami. 2024. https://aclanthology.org/2024.eacl-long.99 WSC +: Enhancing the W inograd schema challenge using tree-of-experts . In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long...

    2024

  51. [59]

    Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. Improving massively multilingual neural machine translation and zero-shot translation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628--1639

    2020

  52. [60]

    Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, et al. 2023. Bayling: Bridging cross-lingual alignment and instruction following through interactive translation for large language models. arX...

    2023 arXiv

  53. [61]

    Yuanhang Zheng, Zhixing Tan, Meng Zhang, Mieradilijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, and Yang Liu. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.267 Self-supervised quality estimation for machine translation . In Proceedings of the 2021 Conference on Empiri...

    2021 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.