Pith. sign in

REVIEW 2 major objections 5 minor 82 references

A Context-aware Framework for Translation-mediated Conversations

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TowerChat, a 7B model trained on bilingual chat context with context-aware MBR decoding, beats GPT-4o across all metrics on the WMT24 chat translation task.

desk verdict Solid chat-MT empirical paper; automatic metrics only, so the 'better translations' headline overreaches, but the core result and ablations hold up. read the letter →

arxiv 2412.04205 v2 pith:QJVNWXEB submitted 2024-12-05 cs.CL

José Pombal , Sweta Agrawal , Patrick Fernandes , Emmanouil Zaranis , André F. T. Martins This is my paper · ORCID
classification cs.CL MSC 68T5068T07
keywords context-awaremachinetranslationtranslation-mediatedconversationquality-awaredecodingMBRchatLLMfinetuningbilingualcontextcustomersupport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that machine translation in live bilingual conversations is best improved by making the model explicitly use the full bilingual conversational history, both during training and when choosing its output. The authors build TowerChat, a 7-billion-parameter model fine-tuned on customer-support chats with context-augmented prompts, and decode with quality-aware reranking using the context-aware metric ContextCOMET. They report that this system consistently outperforms GPT-4o and the stronger TowerInstruct baseline on the WMT24 chat translation task across five language pairs on every automatic metric they used. It also shows that the model relies on the parts of context that resolve ambiguity, with gains concentrated on segments that are hard to translate without context. The claim matters because translation glitches in conversations might be reduced if systems read the prior exchange, and better decoding can push a small model past a much larger one.

What carries the argument

The load-bearing mechanism is a two-stage framework: during training, each instance is formatted as an instruction that prepends the preceding bilingual turns as context, and the model is trained with a simple cross-entropy loss conditioned on that context; during inference, the model samples 100 candidate translations via epsilon sampling and reranks them with Minimum Bayes Risk decoding using ContextCOMET, a context-aware metric that scores a candidate by comparing it with other candidates given the same bilingual context. The paper also introduces a self-distillation step that fine-tunes TowerChat on its own context-aware MBR outputs, cutting inference cost while preserving most of the quality gain.

What would settle it

Take a random sample of 300 segments from the WMT24 chat test set across the five language pairs, have professional bilingual annotators compare TowerChat plus QAD with ContextCOMET against GPT-4o blind using MQM error annotation, and count preference; if GPT-4o wins or ties, the paper's central claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that context-augmented instruction finetuning plus quality-aware decoding with context-aware metrics is enough to make a 7B open model outperform GPT-4o on translation-mediated customer-support chat, across English to and from German, French, Portuguese, Korean, and Dutch, on every automatic metric reported. The paper isolates the training contribution with an ablation: fine-tuning on the same chat data without context-aware prompts yields notably worse translation quality, so the gain is attributed to how the model is taught to use context rather than to in-domain exposure alone. It further claims that the model uses context in a targeted way: context-aware hypotheses are better than context-free ones for low-quality segments and for segments whose references are surprising without context, and saliency analysis shows the model attends strongly to the segments that resolve ambiguity. The authors also show that most of the gain of expensive MBR decoding can be recovered by fine-tuning on the model's own MBR outputs.

Load-bearing premise

The claimed win over GPT-4o is measured only with automatic metrics, and the metric used to pick the final translation is closely related to the metric used to score it, so a human judge might not see the same gap.

Editorial extensions

If this is right

  • A 7-billion-parameter open model can match or beat a far larger closed system on a real-world translation task, suggesting model size is not the only path to quality.
  • Context-aware training and context-aware reranking provide additive gains, so the framework should transfer to other contextual generation tasks such as real-time meeting translation or medical dialogue translation.
  • The optimal context window differs by language pair, with no quality loss from adding all turns, so adaptive context selection could save computation without hurting quality.
  • Self-distillation on MBR outputs recovers most of the gain at a fraction of the inference cost, pointing to a practical path to deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because ContextCOMET is used both to select the final output and to score the final output, the reported advantage over GPT-4o may be larger than what human judges would confirm; a human MQM study would be the cleanest test.
  • The P-CXMI analysis suggests a cheap do-I-need-context predictor: gating context on the estimated surprise of the reference could halve latency while keeping quality.
  • The same two-stage recipe may transfer to other languages and domains, but the largest wins are likely where chat domain data are abundant (customer support) rather than where they are scarce.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper proposes a context-aware framework for LLM-based translation in bilingual conversations. At training time, the authors fine-tune TowerBase 7B on the WMT24 Chat training data using prompts that prepend the previous bilingual turns as context; at inference time, they apply quality-aware decoding (MBR-style reranking) with COMET or ContextCOMET over 100 epsilon-sampled candidates. Experiments on the WMT24 Chat shared task (en↔de/fr/pt/ko/nl) and on BCONTRAS T (en-de) measure chrF, COMET-22, MetricX-XL, a GPT-4-based ContextMQM proxy, and MUDA tag F1. The headline finding is that TowerChat with QAD outperforms TowerInstruct and GPT-4o on most WMT24 metrics, that bilingual context is beneficial when the model is trained with it, and that context-aware reranking reduces context-related errors. The paper also includes an ablation controlling for in-domain training (Table 5) and analyses suggesting that context helps low-quality segments and that the model relies on ambiguity-resolving context tokens.

Significance. If the results hold, the paper is a solid empirical contribution: it demonstrates that a 7B model fine-tuned with bilingual context plus context-aware MBR can match or beat GPT-4o on chat translation according to multiple automatic metrics. The training/inference ablation in Table 5 is well designed, the use of statistically significant quality clusters is appropriate, and the inclusion of MetricX, a metric not optimized by QAD, provides partial evidence against circularity. The generalization test on BCONTRAS T without any training data is also a strength. The main weaknesses are the absence of any human evaluation and an abstract that overstates the BCONTRAS T results.

major comments (2)
  1. [Abstract and Table 3] The abstract states that "Across both settings, the system produced by our framework—TowerChat—consistently results in better translations than state-of-the-art systems like GPT-4o and TowerInstruct, as measured by multiple automatic translation quality metrics on several language pairs." This is not supported for the BCONTRAS T setting. In Table 3, GPT-4o with context achieves higher chrF than every TowerChat variant in both directions (EN-DE: 70.23 vs. 67.84 for QAD+COMET; DE-EN: 72.72 vs. 69.71), and GPT-4o is also better on COMET and MetricX for DE-EN (92.81 vs. 92.34 and 0.39 vs. 0.43). Only EN-DE COMET and MetricX place TowerChat QAD in the same cluster as GPT-4o. The abstract's "consistently better" claim should be restricted to the WMT24 Chat task or rephrased to describe the strong WMT24 gains and the comparable BCONTRAS T results.
  2. [Section 4.2 and Section 7] The paper's conclusion in Section 7 claims that the framework "improves translation quality in bilingual conversations," but the supporting evidence in Section 4.2 is entirely automatic: COMET-22, MetricX-XL, chrF, a GPT-4-based ContextMQM, and MUDA tag F1. There is no human MQM or other human evaluation. This matters because one of the primary metrics, COMET-22, is also the optimization target of the QAD reranking, and ContextMQM is an LLM proxy rather than a human judgment of discourse-level appropriateness. The independent gains on chrF and MetricX in Tables 2, 9, and 11 reduce the circularity concern, but they do not establish that the automatic metrics track human judgments of formality, pronoun resolution, and communicative success in this domain. Either a human evaluation (e.g., a sample of MQM annotations per language pair) should be added, or the claims should be consistently and explicitly limited to automatic-metric improvements.
minor comments (5)
  1. [Throughout] The text contains many spacing artifacts from the PDF/LaTeX rendering, such as "T OWER", "V oita", "BC ONTRAS T", and "METRIC X". These should be normalized to "Tower", "Voita", "BCONTRAS T", and "MetricX" for readability.
  2. [Table 4] The column headers in Table 4 (e.g., "F1 XX EN") are difficult to parse, and the caption mentions ContextMQM without clearly indicating how the MQM columns are computed. Please restructure the table or caption so that the reader can see which columns report MUDA F1 and which report ContextMQM, and clarify that ContextMQM is a GPT-4-based proxy, not human MQM.
  3. [Section 3.2, Eq. (2)] In Eq. (2), the symbol yt is used both as the reference translation defined in Section 3.1 and as a candidate hypothesis inside the MBR utility sum. This is a notation clash that could confuse readers; please use a different symbol (e.g., r for the reference and c or y' for candidate hypotheses).
  4. [Section 6.3] The interpretability claim that "the model leverages context in an intended and interpretable way" is supported by only two qualitative examples (Figure 7). The PeCoRE saliency analysis is interesting, but the paper would be stronger with a quantitative aggregation over more examples or error cases.
  5. [Table 2 caption] The caption states "QAD with TOWER CHAT significantly outperforms all baselines across the board," which is accurate only for the WMT24 results in that table. The wording "across the board" could be read as applying to both datasets; consider qualifying it as "across language pairs on WMT24."

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central training and QAD claims are supported by non-optimized metrics and greedy-decoding ablations; only a minor same-group metric self-citation is present.

full rationale

The paper's derivation chain is largely self-contained. TowerChat is obtained by context-augmented instruction finetuning of TowerBase on TowerBlocks plus the WMT24 chat training set and is evaluated on held-out WMT24 test and BCONTRAS T, so the training claim is not defined in terms of the headline result. The QAD component reranks sampled candidates with a COMET-family utility (Eq. 2), and the paper explicitly acknowledges the resulting entanglement: "Considering all the metrics is crucial because COMET may favour our QAD strategies." Crucially, the main WMT24 headline (Table 2) also shows TowerChat greedy decoding outperforming GPT-4o and TowerInstruct on chrF, COMET, and MetricX, and QAD gains are reported on MetricX, "a metric not directly optimized by QAD," so the central superiority claim does not reduce by construction to optimizing the evaluation metric. The same-group metrics from Agrawal et al. (2024), namely ContextCOMET for reranking and ContextMQM for fine-grained evaluation, constitute a self-citation, but ContextCOMET is not the COMET score reported in the main tables, and context-sensitive accuracy is also assessed with MUDA, a rule-based tagger with native-speaker-validated rules. The remaining weaknesses are real but non-circular: the abstract's "consistently results in better translations" overstates Table 3, where TowerChat with QAD is still below GPT-4o on BCONTRAS T EN-DE chrF (67.84 vs. 70.23) and MetricX (0.40 vs. 0.39), and the evaluation is fully automatic, with no human MQM study. These are correctness-evidence concerns, not definitional circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The framework does not introduce new theoretical entities or fit a scientific constant. Its load-bearing assumptions are measurement-related: automatic metrics stand in for human quality, and the previous bilingual turns are treated as sufficient context. The only hand-set experimental parameters that materially affect the QAD results are the epsilon sampling threshold and the candidate pool size.

free parameters (2)
  • epsilon sampling threshold = 0.02
    Chosen in Section 4.3 for generating the 100 MBR candidates; it controls the diversity and quality of the candidate pool and therefore influences QAD gains, but it is not fitted to the test set.
  • MBR candidate pool size = 100
    Chosen in Section 4.3; larger pools generally improve MBR quality but increase compute, so this is a hand-set parameter that affects the reported QAD results.
assumptions (4)
  • domain assumption Previous bilingual turns (x<t) are a sufficient context for resolving translation ambiguities in these conversations.
    Section 3.1, 'Choice of context', restricts context to the original bilingual texts from previous turns; no external knowledge, speaker metadata, or summaries are used.
  • domain assumption Automatic metrics (COMET-22, MetricX-XL, chrF, MUDA F1, ContextMQM) are valid proxies for translation quality in conversations.
    Section 4.2 describes only automatic evaluation; no human judgments are collected. The paper's main conclusions depend on these metrics.
  • domain assumption TowerBase and TowerInstruct are suitable base models and the WMT24 chat training data covers the target domain.
    Section 4.3 finetunes TowerBase with TowerInstruct hyperparameters on TowerBlocks plus WMT24 chat training; generalization to BCONTRAS T is used as evidence the method transfers.
  • domain assumption Sampling 100 candidates with epsilon sampling gives a representative candidate pool for MBR decoding.
    Section 4.3 relies on epsilon sampling and 100 candidates; prior work (Freitag et al., 2023a) supports this choice, but the paper does not vary it for the main claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Context-aware Framework for Translation-mediated Conversations." pith.science (2026). https://pith.science/paper/QJVNWXEB

@misc{pith2026241204205,
  author       = {Pith},
  title        = {Pith review of: A Context-aware Framework for Translation-mediated Conversations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QJVNWXEB}},
  note         = {Machine review of arXiv:2412.04205}
}
read the original abstract

Automatic translation systems offer a powerful solution to bridge language barriers in scenarios where participants do not share a common language. However, these systems can introduce errors leading to misunderstandings and conversation breakdown. A key issue is that current systems fail to incorporate the rich contextual information necessary to resolve ambiguities and omitted details, resulting in literal, inappropriate, or misaligned translations. In this work, we present a framework to improve large language model-based translation systems by incorporating contextual information in bilingual conversational settings during training and inference. We validate our proposed framework on two task-oriented domains: customer chat and user-assistant interaction. Across both settings, the system produced by our framework-TowerChat-consistently results in better translations than state-of-the-art systems like GPT-4o and TowerInstruct, as measured by multiple automatic translation quality metrics on several language pairs. We also show that the resulting model leverages context in an intended and interpretable way, improving consistency between the conveyed message and the generated translations.

Figures

Figures reproduced from arXiv: 2412.04205 by the authors.

Figure 1
Figure 1. Our framework for optimizing LLMs for mediating conversations with translation. First, we train the LLM on a conversation translation dataset formatted with context-aware prompts. At inference time, we use a context-aware metric to select the best translation from a pool of candidates. original bilingual messages exchanged between the two interacting parties as context. This allows the model to attend to both contex… view at source ↗
Figure 2
Figure 2. A sample conversation illustrating help [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. COMET from TOWERCHAT w/o context. Blue: w/ context. Red: w/o context. ing to COMET and METRICX. Furthermore, TOW￾ERCHAT shows an average improvement of 4 CHRF points for WMT24 EN-XX and 1.7 points EN-DE when using context (w/ context), compared to a context-agnostic prompt (w/o context). This trend also holds when evaluating translation quality with COMET, for 8 out of 10 WMT24 language pairs, as shown in the Append… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: shows the impact of varying the number of turns included in the context during inference on COMET and CHRF. For both metrics, transla￾tion quality improves as the context window length increases. Notably, for EN-NL and PT-EN, us￾ing the full context yields improvements…
Figure 5
Figure 5. Figure 5: COMET under different prompt settings (with and without context) for different bins of log LD on the hypothesis for TOWERCHAT (left) and TOWERINSTRUCT (right). < -0.5 [-0.5, 0] [0, 0.5] > 0.5 Bin of P-CXMI on reference 80 85 90 95 100 COMET-22 202 7178 12650 308 Refere…
Figure 6
Figure 6. Figure 6: COMET under different prompts (with and without context) for different P-CXMI bins. Unlike P-CXMI, this metric does not rely on a ref￾erence translation. Instead, it directly evaluates how incorporating context affects the likelihood of the generated hypothesis. Our fi…
Figure 7
Figure 7. Figure 7: Two examples of PT→EN translations with and without context where the contextually￾informed translation is accurate, while the trans￾lation without context is lexically correct but con￾textually incorrect: saliency values are high for context segments that resolve sema…
Figure 8
Figure 8. Figure 8: Specific training instance with context for [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: MuDA F1 by language pairs. LC: Lexical Cohesion, VF: Verb Form, P: Pronouns, F: Formality. [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: COMET under different prompts (with and without context) for different P-CXMI bins for TOWERINSTRUCT trained on chat domain data but context-unaware. F Additional Examples We show concrete examples where context results in a more contextually appropriate translation i…
Figure 11
Figure 11. Figure 11: Quality Bins from COMET for TOWERCHAT w/o context on EN→ XX (top) and XX→ EN (bottom) language pairs. Blue: w/ context. Red: w/o context. < -0.5 [-0.5, 0] [0, 0.5] > 0.5 70 75 80 85 90 95 100 COMET-22 10 768 1290 6 EN DE < -0.5 [-0.5, 0] [0, 0.5] > 0.5 4 726 1390 10 E…
Figure 12
Figure 12. Figure 12: COMET under different prompt settings (with and without context) for different P-CXMI bins and language pairs [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Full conversation of first example in Figure [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Full conversation of second example in Figure [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Full conversation where context helps improve the translation quality. In [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Full conversation where context helps improve the translation quality. In [PITH_FULL_IMAGE:figures/full_fig_p025_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

82 extracted references · 24 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Sweta Agrawal, Amin Farajian, Patrick Fernandes, Ricardo Rei, and André F. T. Martins. 2024. https://doi.org/10.1162/tacl_a_00700 Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions? Transactions of the Association for Computational Linguistics, 12:1250--1267

    doi:10.1162/tacl_a_00700 2024

  4. [4]

    Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2023. https://doi.org/10.18653/v1/2023.findings-acl.564 In-context examples selection for machine translation . In Findings of the Association for Computational Linguistics: ACL 2023, pages 8857--8873, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.findings-acl.564 2023

  5. [5]

    Duarte Alves, Nuno Guerreiro, Jo \ a o Alves, Jos \'e Pombal, Ricardo Rei, Jos \'e de Souza, Pierre Colombo, and Andre Martins. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.744 Steering large language models for machine translation with finetuning and in-context learning . In Findings of the Association for Computational Linguistics: EMNLP 2023, ...

    doi:10.18653/v1/2023.findings-emnlp.744 2023

  6. [6]

    Alves, José Pombal, Nuno M

    Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. https://arxiv.org/abs/2402.17733 Tower: An open multilingual large language model for translation-related tasks . Preprint, arXiv:2402.17733

    arXiv 2024

  7. [7]

    Eleftheria Briakou, Navita Goyal, and Marine Carpuat. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.690 Explaining with contrastive phrasal highlighting: A case study in assisting humans to detect translation differences . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11220--11237, Singapore. Associat...

    doi:10.18653/v1/2023.emnlp-main.690 2023

  8. [8]

    Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Ben Goodrich, Daniel Duckworth, Semih Yavuz, Amit Dubey, Kyu-Young Kim, and Andy Cedilnik. 2019. https://doi.org/10.18653/v1/D19-1459 Taskmaster-1: Toward a realistic and diverse dialog dataset . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Proce...

    doi:10.18653/v1/d19-1459 2019

Show all 82 references
  1. [9]

    Sheila Castilho and Rebecca Knowles. 2024. https://doi.org/10.1017/nlp.2024.7 A survey of context in neural machine translation and its evaluation . Natural Language Processing, page 1–31

    2024 doi

  2. [10]

    Benjamin Cohen-Wang, Harshay Shah, Kristian Georgiev, and Aleksander Madry. 2024. https://openreview.net/forum?id=7CMNSqsZJt Contextcite: Attributing model generation to context . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  3. [11]

    Pierre Colombo, Nathan Noiry, Ekhine Irurozki, and St \'e phan Cl \'e men c on. 2022. https://arxiv.org/abs/2202.03799 What are the best systems? new perspectives on nlp benchmarking . In Advances in Neural Information Processing Systems

    2022 arXiv

  4. [12]

    Donald A DePalma, Benjamin B Sargent, and Renato S Beninatto. 2006. Can’t read, won’t buy: Why language matters on global websites. Lowell, MA: Common Sense Advisory Inc

    2006

  5. [13]

    Arid Hasan, Imran Razzak, and Usman Naseem

    Krishno Dey, Prerona Tarannum, Md. Arid Hasan, Imran Razzak, and Usman Naseem. 2024. https://arxiv.org/abs/2410.13153 Better to ask in english: Evaluation of large language models on english, low-resource and cross-lingual settings . Preprint, arXiv:2410.13153

    2024 arXiv

  6. [14]

    Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, and Mikel Artetxe. 2023. Do multilingual language models think better in english? arXiv preprint arXiv:2308.01223

    2023 arXiv

  7. [15]

    Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lacalle, and Mikel Artetxe. 2024. https://doi.org/10.18653/v1/2024.naacl-short.46 Do multilingual language models think better in E nglish? In Proceedings of the 2024 Conference of the North American Chapter of the Association for...

    2024 doi

  8. [16]

    Amin Farajian, Ant \'o nio V

    M. Amin Farajian, Ant \'o nio V. Lopes, Andr \'e F. T. Martins, Sameen Maruf, and Gholamreza Haffari. 2020. https://aclanthology.org/2020.wmt-1.3 Findings of the WMT 2020 shared task on chat translation . In Proceedings of the Fifth Conference on Machine Translation, pages 65-...

    2020

  9. [17]

    Ant \'o nio Farinhas, Jos \'e de Souza, and Andre Martins. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.733 An empirical study of translation hypothesis ensembling with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language ...

    2023 doi

  10. [18]

    Ant \'o nio Farinhas, Nuno M Guerreiro, Sweta Agrawal, Ricardo Rei, and Andr \'e FT Martins. 2025. Translate smart, not hard: Cascaded translation systems with quality-aware deferral. arXiv preprint arXiv:2502.12701

    2025 arXiv

  11. [19]

    Patrick Fernandes, Ant \'o nio Farinhas, Ricardo Rei, Jos \'e G. C. de Souza, Perez Ogayo, Graham Neubig, and Andre Martins. 2022. https://doi.org/10.18653/v1/2022.naacl-main.100 Quality-aware decoding for neural machine translation . In Proceedings of the 2022 Conference of t...

    2022 doi

  12. [20]

    Patrick Fernandes, Kayo Yin, Emmy Liu, Andr \'e Martins, and Graham Neubig. 2023. https://doi.org/10.18653/v1/2023.acl-long.36 When does translation require context? a data-driven, multilingual exploration . In Proceedings of the 61st Annual Meeting of the Association for Comp...

    2023 doi

  13. [21]

    Patrick Fernandes, Kayo Yin, Graham Neubig, and Andr \'e F. T. Martins. 2021. https://doi.org/10.18653/v1/2021.acl-long.505 Measuring and increasing context usage in context-aware machine translation . In Proceedings of the 59th Annual Meeting of the Association for Computatio...

    2021 doi

  14. [22]

    Mara Finkelstein and Markus Freitag. 2024. https://openreview.net/forum?id=bkNx3O0sND MBR and QE finetuning: Training-time distillation of the best and most expensive decoding methods . In The Twelfth International Conference on Learning Representations

    2024

  15. [23]

    Markus Freitag, Behrooz Ghorbani, and Patrick Fernandes. 2023 a . https://doi.org/10.18653/v1/2023.findings-emnlp.617 Epsilon sampling rocks: Investigating sampling strategies for minimum B ayes risk decoding for machine translation . In Findings of the Association for Computa...

    2023 doi

  16. [24]

    Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. 2022. https://doi.org/10.1162/tacl_a_00491 High quality rather than high model probability: Minimum B ayes risk decoding with neural metrics . Transactions of the Association for Computational Linguistics, 10:811--825

    2022 doi

  17. [25]

    Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frederic Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, and George Foster. 2023 b . https://aclanthology.org/2023.wmt-1.51 Results of ...

    2023

  18. [26]

    Marjan Ghazvininejad, Hila Gonen, and Luke Zettlemoyer. 2023. Dictionary-based phrase-level prompting of large language models for machine translation. arXiv preprint arXiv:2302.07856

    2023 arXiv

  19. [27]

    Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. https://doi.org/10.1162/tacl_a_00642 Exploring human-like translation strategy with large language models . Transactions of the Association for Computati...

    2024 doi

  20. [28]

    John Hewitt, Christopher Manning, and Percy Liang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.249 Truncation sampling as language model desmoothing . In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 3414--3427, Abu Dhabi, United Arab E...

    2022 doi

  21. [29]

    Songbo Hu, Han Zhou, Moy Yuan, Milan Gritta, Guchun Zhang, Ignacio Iacobacci, Anna Korhonen, and Ivan Vuli \'c . 2023 a . https://doi.org/10.18653/v1/2023.emnlp-main.422 A systematic study of performance disparities in multilingual task-oriented dialogue systems . In Proceedin...

    2023 doi

  22. [30]

    Xinyu Hu, Xunjian Yin, and Xiaojun Wan. 2023 b . https://doi.org/10.18653/v1/2023.findings-emnlp.1021 Exploring context-aware evaluation metrics for machine translation . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15291--15298, Singapore. A...

    2023 doi

  23. [31]

    Yuchen Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, and Ming Zhou. 2022. https://doi.org/10.18653/v1/2022.naacl-main.111 BlonDe : An automatic evaluation metric for document-level machine translation ....

    2022 doi

  24. [32]

    Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, and Srijan Kumar. 2024. Better to ask in english: Cross-lingual evaluation of large language models for healthcare queries. In Proceedings of the ACM on Web Conference 2024, pages 2627--2638

    2024

  25. [33]

    Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, and Markus Freitag. 2023. https://doi.org/10.18653/v1/2023.wmt-1.63 M etric X -23: The G oogle submission to the WMT 2023 metrics shared task . In Proceedings of the Eighth Conference on Machin...

    2023 doi

  26. [34]

    Marzena Karpinska and Mohit Iyyer. 2023. https://doi.org/10.18653/v1/2023.wmt-1.41 Large language models effectively leverage document-level context for literary translation, but critical errors persist . In Proceedings of the Eighth Conference on Machine Translation, pages 41...

    2023 doi

  27. [35]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR)

    2015

  28. [36]

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ond r ej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, To...

    2023 doi

  29. [37]

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, et al. 2024. Preliminary wmt24 ranking of general mt systems and llms. arXiv preprint arXiv:2407.19884

    2024 arXiv

  30. [38]

    Lea Krause, Selene Baez Santamaria, and Jan-Christoph Kalo. 2024. https://doi.org/10.18653/v1/2024.wmt-1.106 Graph representations for machine translation in dialogue settings . In Proceedings of the Ninth Conference on Machine Translation, pages 1038--1046, Miami, Florida, US...

    2024 doi

  31. [39]

    Yunlong Liang, Fandong Meng, Yufeng Chen, Jinan Xu, and Jie Zhou. 2021. https://doi.org/10.18653/v1/2021.acl-long.444 Modeling bilingual conversational characteristics for neural chat translation . In Proceedings of the 59th Annual Meeting of the Association for Computational ...

    2021 doi

  32. [40]

    Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu, and Lidong Bing. 2024. Is translation all you need? a study on solving multilingual tasks with large language models. arXiv preprint arXiv:2403.10258

    2024 arXiv

  33. [41]

    Amin Farajian, Rachel Bawden, Michael Zhang, and Andr \'e F

    Ant \'o nio Lopes, M. Amin Farajian, Rachel Bawden, Michael Zhang, and Andr \'e F. T. Martins. 2020. https://aclanthology.org/2020.eamt-1.24 Document-level neural MT : A systematic comparison . In Proceedings of the 22nd Annual Conference of the European Association for Machin...

    2020

  34. [42]

    Sameen Maruf, Andr \'e F. T. Martins, and Gholamreza Haffari. 2018. https://doi.org/10.18653/v1/W18-6311 Contextual neural model for translating bilingual multi-speaker conversations . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 101--1...

    2018 doi

  35. [43]

    Sameen Maruf, Fahimeh Saleh, and Gholamreza Haffari. 2021. https://doi.org/10.1145/3441691 A survey on document-level neural machine translation: Methods and evaluation . ACM Comput. Surv., 54(2)

    2021 doi

  36. [44]

    Nikita Mehandru, Sweta Agrawal, Yimin Xiao, Ge Gao, Elaine Khoong, Marine Carpuat, and Niloufar Salehi. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.712 Physician detection of clinical harm in machine translation: Quality estimation aids in reliance and backtranslation id...

    2023 doi

  37. [45]

    John Mendonca, Patr \' cia Pereira, Miguel Menezes, Vera Cabarr \ a o, Ana C Farinha, Helena Moniz, Alon Lavie, and Isabel Trancoso. 2023. https://aclanthology.org/2023.gem-1.2 Dialogue quality and emotion annotations for customer support conversations . In Proceedings of the ...

    2023

  38. [46]

    Wafaa Mohammed, Sweta Agrawal, Amin Farajian, Vera Cabarr \ a o, Bryan Eikema, Ana C Farinha, and Jos \'e G. C. De Souza. 2024. https://doi.org/10.18653/v1/2024.wmt-1.59 Findings of the WMT 2024 shared task on chat translation . In Proceedings of the Ninth Conference on Machin...

    2024 doi

  39. [47]

    Kelleher, Rejwanul Haque, and Andy Way

    Yasmin Moslem, Gianfranco Romani, Mahdi Molaei, John D. Kelleher, Rejwanul Haque, and Andy Way. 2023. https://doi.org/10.18653/v1/2023.wmt-1.82 Domain terminology integration into machine translation: Leveraging large language models . In Proceedings of the Eighth Conference o...

    2023 doi

  40. [48]

    Yongyu Mu, Abudurexiti Reheman, Zhiquan Cao, Yuchun Fan, Bei Li, Yinqiao Li, Tong Xiao, Chunliang Zhang, and Jingbo Zhu. 2023. https://doi.org/10.18653/v1/2023.findings-acl.653 Augmenting large language model translators via translation memories . In Findings of the Associatio...

    2023 doi

  41. [49]

    Artur Nowakowski, Gabriela Pa ka, Kamil Guttmann, and Miko aj Pokrywka. 2022. https://aclanthology.org/2022.wmt-1.26 A dam M ickiewicz U niversity at WMT 2022: NER -assisted and quality-aware neural machine translation . In Proceedings of the Seventh Conference on Machine Tran...

    2022

  42. [50]

    Jonathan Pilault, Xavier Garcia, Arthur Bra z inskas, and Orhan Firat. 2023. https://doi.org/10.18653/v1/2023.ijcnlp-main.31 Interactive-chain-prompting: Ambiguity resolution for crosslingual conditional generation with interaction . In Proceedings of the 13th International Jo...

    2023 doi

  43. [51]

    Maja Popovi \'c . 2015. https://doi.org/10.18653/v1/W15-3049 chr F : character n-gram F -score for automatic MT evaluation . In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392--395, Lisbon, Portugal. Association for Computational Linguistics

    2015 doi

  44. [52]

    Ricardo Rei, Jos \'e G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and Andr \'e F. T. Martins. 2022. https://aclanthology.org/2022.wmt-1.52 COMET -22: Unbabel- IST 2022 submission for the metrics shared task . In Pro...

    2022

  45. [53]

    Raphael Reinauer, Patrick Simianer, Kaden Uhlig, Johannes E. M. Mosig, and Joern Wuebker. 2023. https://arxiv.org/abs/2309.08590 Neural machine translation models can learn to be few-shot learners . Preprint, arXiv:2309.08590

    2023 arXiv

  46. [54]

    Samantha Robertson and Mark D\' az. 2022. https://doi.org/10.1145/3531146.3534638 Understanding and being understood: User strategies for identifying and recovering from mistranslations in machine translation-mediated chat . In Proceedings of the 2022 ACM Conference on Fairnes...

    2022

  47. [55]

    Gabriele Sarti, Grzegorz Chrupa a, Malvina Nissim, and Arianna Bisazza. 2024. https://openreview.net/forum?id=XTHfNGI3zT Quantifying the plausibility of context reliance in neural machine translation . In The Twelfth International Conference on Learning Representations

    2024

  48. [56]

    Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. https://doi.org/10.18653/v1/2024.naacl-short.69 Trusting your evidence: Hallucinate less with context-aware decoding . In Proceedings of the 2024 Conference of the North American C...

    2024 doi

  49. [57]

    Daria Sinitsyna and Konstantin Savenkov. 2024. https://aclanthology.org/2024.amta-presentations.12 Comparative evaluation of large language models for linguistic quality assessment in machine translation . In Proceedings of the 16th Conference of the Association for Machine Tr...

    2024

  50. [58]

    Felix Stahlberg, Adri \`a de Gispert, Eva Hasler, and Bill Byrne. 2017. https://aclanthology.org/E17-2058 Neural machine translation by minimising the B ayes-risk with respect to syntactic translation lattices . In Proceedings of the 15th Conference of the E uropean Chapter of...

    2017

  51. [59]

    Mingi Sung, Seungmin Lee, Jiwon Kim, and Sejoon Kim. 2024. https://doi.org/10.18653/v1/2024.wmt-1.102 Context-aware LLM translation system using conversation summarization and dialogue history . In Proceedings of the Ninth Conference on Machine Translation, pages 1011--1015, M...

    2024 doi

  52. [60]

    Firas Trabelsi, David Vilar, Mara Finkelstein, and Markus Freitag. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/626ab938fe19200324b368f5ee816868-Paper-Conference.pdf Efficient minimum bayes risk decoding using low-rank matrix completion algorithms . In Adva...

    2024

  53. [61]

    Guerreiro, Sweta Agrawal, Ricardo Rei, José Pombal, Tania Vaz, Helena Wu, Beatriz Silva, Daan van Stigt, and André F

    Marcos Treviso, Nuno M. Guerreiro, Sweta Agrawal, Ricardo Rei, José Pombal, Tania Vaz, Helena Wu, Beatriz Silva, Daan van Stigt, and André F. T. Martins. 2024. https://arxiv.org/abs/2406.19482 xtower: A multilingual llm for explaining and correcting translation errors . Prepri...

    2024 arXiv

  54. [62]

    Jun-ichi Tsujii and Makoto Nagao. 1988. https://aclanthology.org/C88-2142 Dialogue translation vs. text translation . In C oling B udapest 1988 Volume 2: I nternational C onference on C omputational L inguistics

    1988

  55. [63]

    Jannis Vamvas and Rico Sennrich. 2024. https://doi.org/10.18653/v1/2024.acl-short.71 Linear-time minimum B ayes risk decoding with reference aggregation . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages...

    2024 doi

  56. [64]

    Giorgos Vernikos, Brian Thompson, Prashant Mathur, and Marcello Federico. 2022. https://aclanthology.org/2022.wmt-1.6 Embarrassingly easy document-level MT metrics: How to convert any pretrained metric into a document-level metric . In Proceedings of the Seventh Conference on ...

    2022

  57. [65]

    Vincent, Lo \" c Barrault, and Carolina Scarton

    Sebastian T. Vincent, Lo \" c Barrault, and Carolina Scarton. 2022. https://aclanthology.org/2022.eamt-1.15 Controlling extra-textual attributes about dialogue participants: A case study of E nglish-to- P olish neural machine translation . In Proceedings of the 23rd Annual Con...

    2022

  58. [66]

    Elena Voita, Rico Sennrich, and Ivan Titov. 2019. https://doi.org/10.18653/v1/P19-1116 When a good translation is wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion . In Proceedings of the 57th Annual Meeting of the Associati...

    2019 doi

  59. [67]

    Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.1036 Document-level machine translation with large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natu...

    2023 doi

  60. [68]

    Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. https://doi.org/10.18653/v1/D17-1301 Exploiting cross-sentence context for neural machine translation . In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2826--2831, Copenhag...

    2017 doi

  61. [69]

    Longyue Wang, Xiaojun Zhang, Zhaopeng Tu, Andy Way, and Qun Liu. 2016. https://aclanthology.org/L16-1436 Automatic construction of discourse corpora for dialogue translation . In Proceedings of the Tenth International Conference on Language Resources and Evaluation ( LREC '16)...

    2016

  62. [70]

    Bonnie Webber, Andrei Popescu-Belis, Katja Markert, and J \"o rg Tiedemann, editors. 2013. https://aclanthology.org/W13-3300 Proceedings of the Workshop on Discourse in Machine Translation . Association for Computational Linguistics, Sofia, Bulgaria

    2013

  63. [71]

    Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, Tianxiang Hu, Shangjie Li, Binyuan Hui, Bowen Yu, Dayiheng Liu, Baosong Yang, Fei Huang, and Jun Xie. 2023. https://arxiv.org/abs/2307.06018 Polylm: An open sourc...

    2023 arXiv

  64. [72]

    Minghao Wu, Thuy-Trang Vu, Lizhen Qu, George Foster, and Gholamreza Haffari. 2024. Adapting large language models for document-level machine translation. arXiv preprint arXiv:2401.06468

    2024 arXiv

  65. [73]

    Kayo Yin and Graham Neubig. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.14 Interpreting language models with contrastive explanations . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 184--198, Abu Dhabi, United Arab Emira...

    2022 doi

  66. [74]

    Biao Zhang, Barry Haddow, and Alexandra Birch. 2023 a . Prompting large language model for machine translation: A case study. In International Conference on Machine Learning, pages 41092--41110. PMLR

    2023

  67. [75]

    Bryan Zhang and Amita Misra. 2022. https://doi.org/10.18653/v1/2022.emnlp-industry.8 Machine translation impact in E -commerce multilingual search . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 99--109, Abu Dh...

    2022 doi

  68. [76]

    Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, and Yang Feng. 2023 b . https://arxiv.org/abs/2306.10968 Bayling: Bridging cross-lingual alignment and instruction following through interactiv...

    2023 arXiv

  69. [77]

    Yongle Zhang, Dennis Asamoah Owusu, Marine Carpuat, and Ge Gao. 2022. https://doi.org/10.1145/3512937 Facilitating global team meetings between language-based subgroups: When and how can machine translation help? Proc. ACM Hum.-Comput. Interact., 6(CSCW1)

    2022 doi

  70. [78]

    Yongle Zhang, Dennis Asamoah Owusu, Emily Gong, Shaan Chopra, Marine Carpuat, and Ge Gao. 2021. https://doi.org/10.1145/3411763.3451837 Leveraging machine translation to support distributed teamwork between language-based subgroups: The effects of automated keyword tagging . I...

    2021

  71. [79]

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. How do large language models handle multilingualism? arXiv preprint arXiv:2402.18815

    2024 arXiv

  72. [80]

    Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024. https://doi.org/10.18653/v1/2024.findings-naacl.176 Multilingual machine translation with large language models: Empirical results and analysis . In Findings of the ...

    2024 doi

  73. [81]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  74. [82]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.