Pith. sign in

REVIEW 4 major objections 6 minor 10 cited by

A 7B translation model beats GPT-4.1 and Gemini on 30 of 31 WMT pairs

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 05:28 UTC pith:RADT2QID

load-bearing objection Useful open-source MT recipe and weights, but the SOTA claims rest on contaminated and self-referential evaluation; treat Table 4 as training-adjacent, not independent. the 4 major comments →

arxiv 2509.05209 v2 pith:RADT2QID submitted 2025-09-05 cs.CL

Hunyuan-MT Technical Report

Mao Zheng , Zheng Li , Bingxin Qu , Mingyang Song , Yang Du , Mingrui Sun , Di Wang This is my paper
classification cs.CL
keywords machine translationmultilingual LLMweak-to-strong reinforcement learningMandarin minority languagestest-time scalingGRPOXCOMETopen-source 7B model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Hunyuan-MT-7B and Hunyuan-MT-Chimera-7B are open-source 7B translation models that, according to this report, outperform all translation-specific models of comparable size and most state-of-the-art large models. The central claim is that a carefully built training pipeline—multilingual pretraining, two-stage SFT, GRPO reinforcement learning, and a weak-to-strong fusion stage—can close the gap between small open systems and much larger proprietary ones. The models place special emphasis on translating between Mandarin and ethnic minority languages and dialects, where the authors report the largest margins over baselines like Gemini-2.5-Pro. If the reported results hold, the practical consequence is that high-quality multilingual translation no longer requires giant closed models or expensive chain-of-thought inference. The reported WMT2025 performance, ranking first in 30 of 31 language pairs, is the paper's headline evidence.

Core claim

The paper claims that Hunyuan-MT-7B and its fusion variant Hunyuan-MT-Chimera-7B achieve state-of-the-art translation quality on FLORES-200, WMT24pp, and a Mandarin-minority test set, surpassing much larger models including Gemini-2.5-Pro, Claude-Sonnet-4, and GPT-4.1 on several directions. Hunyuan-MT-7B is trained through MT-oriented pretraining, two-stage supervised fine-tuning, and GRPO reinforcement learning with XCOMET-XXL, DeepSeek-V3 scoring, terminology-aware rewards, and repetition penalties. Hunyuan-MT-Chimera-7B extends this by generating multiple translation candidates and learning to synthesize them into a single refined output, which the paper presents as a more effective slow-

What carries the argument

The central mechanism is the Chimera fusion model: Hunyuan-MT-7B first produces a portfolio of translation candidates under varying parameter settings, then Hunyuan-MT-Chimera-7B, trained via GRPO with a reward combining XCOMET-XXL, DeepSeek-V3 scoring, and a repetition penalty, takes those candidates as input and outputs one unified translation. This learned weak-to-strong synthesis is the paper's proposed alternative to chain-of-thought as a test-time scaling strategy for machine translation.

Load-bearing premise

The headline benchmark scores assume that the sentences used in training, which include FLORES-200 development sets and past WMT test sets, do not also appear in the FLORES-200 and WMT24pp evaluation sets.

What would settle it

Run a fuzzy near-duplicate search over the reported SFT training corpus, including the synthetic DeepSeek-V3-0324 data, using sentences from the FLORES-200 and WMT24pp evaluation sets; if any evaluation sentence is found in training data, the claimed generalization scores cannot be accepted as out-of-distribution performance.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • A 7B open-source translation model can match or beat much larger proprietary systems on standard MT benchmarks when trained with this recipe.
  • Fusing several sampled translations through a trained synthesis model outperforms CoT-based slow thinking for translation.
  • Targeted multilingual pretraining, two-stage SFT, and RL can lift Mandarin-minority translation far above existing systems.
  • The recipe is accompanied by open weights and a code repository, so the reported results are reproducible by others.
  • Human evaluation places Hunyuan-MT-7B in the same tier as Gemini-2.5-Pro and DeepSeek-V3-0324 for Chinese-English translation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's Section 3.1 names FLORES-200 development sets and past WMT test sets as SFT training sources, so a direct implication is that the reported FLORES-200 and WMT24pp numbers should be read as in-distribution performance unless a decontamination analysis is released.
  • A clean test of the recipe's generality would be to apply the identical pipeline to another 7B base model and check whether the WMT24pp gains reproduce; the paper only demonstrates the pipeline on Hunyuan-7B-Base.
  • The weak-to-strong fusion idea may transfer to other generation tasks where multiple candidate outputs exist, such as summarization or code repair, if a learned synthesizer can be trained with reward feedback.
  • The Mandarin-minority advantage is probably driven jointly by the 1.3 trillion tokens of multilingual pretraining data and the human-annotated minority-language pairs, but the paper does not ablate those two factors separately.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Hunyuan-MT-7B, an open-source 7B multilingual translation model covering 33 languages with a focus on Mandarin-to-minority-language translation (Kazakh, Uyghur, Mongolian, Tibetan), and Hunyuan-MT-Chimera-7B, a 'weak-to-strong' fusion model that combines multiple candidate translations at test time. The training pipeline consists of general pre-training, MT-oriented pre-training with a RegMix-style data mixture, two-stage SFT, GRPO-based RL with a composite reward (XCOMET-XXL, GEMBA-style DeepSeek scoring, TAT-R1 terminology reward, repetition penalty), and a further weak-to-strong RL stage. Evaluation uses FLORES-200, WMT24pp, a self-constructed Mandarin⇔Minority test set, and a human evaluation for ZH⇄EN. The paper claims that both models significantly outperform comparable-size translation models and most large SOTA models, and that they ranked first in 30/31 language pairs at WMT2025.

Significance. If the claims are substantiated, this would be a valuable open-source contribution: a 7B model with strong translation quality, a concrete recipe for MT-oriented post-training, and a novel Chimera-style test-time fusion that avoids CoT. The release of model weights and code is a genuine service to the community. However, the headline SOTA comparisons are currently not supported by the evidence as presented: the evaluation sets overlap with training data, the primary automatic metric is also the RL reward, the minority-language claim is contradicted by the second reported metric, and the WMT2025 result is unverifiable. The human evaluation shows parity, not superiority, for the only direction it covers. The paper's value would be greatly increased if the authors addressed these issues with leakage analysis, independent evaluation, and full reporting of the constructed test sets.

major comments (4)
  1. [§3.1 vs §4.1] Training/evaluation leakage. Section 3.1 states that the SFT corpus includes 'the Flores-200 development set and past WMT test sets.' Section 4.1 evaluates on FLORES-200 and WMT24pp, and WMT24pp (Deutsch et al., 2025) is built from WMT24 test sets. Unless 'past WMT test sets' explicitly excludes WMT24, the WMT24pp column of Table 4 is not an independent generalization score. The paper also does not state which FLORES-200 split is used for evaluation; if the dev set is used in SFT and the devtest used for evaluation, that should be stated, but the current text leaves this ambiguous. I request a contamination analysis and re-evaluation on test sets that provably do not overlap with SFT or RL data.
  2. [§3.2, §3.3, §4.2, §5.3] Metric circularity and lack of independent validation. XCOMET-XXL is used as the RL reward in both RL stages (Section 3.2 and 3.3) and as the primary automatic metric in Table 4 (Section 4.2). The reported margins over baselines that were not trained with this reward may therefore partly reflect optimization of the evaluation metric itself. The only human evaluation (Table 10, Section 5.3) covers ZH⇄EN and shows Hunyuan-MT-7B at 3.189, slightly below Gemini-2.5-Pro (3.223) and DeepSeek-V3-0324 (3.219) — not the 'significant outperformance' claimed in the abstract. No confidence intervals or statistical significance tests are reported. This issue is load-bearing for the SOTA claims, and the paper should provide human evaluation on the WMT24pp and minority-language directions, or at minimum report XCOMET-XXL scores on a held-out set not used in training.
  3. [Table 4, Mand.⇔Min. column] The minority-language claim is internally contradicted by the second metric. In Table 4, Hunyuan-MT-7B's Mand.⇔Min. XCOMET-XXL is 0.6082 vs Gemini-2.5-Pro's 0.5811, but the CometKiwi score for the same models is 0.4162 vs 0.5418 — a large relative deficit. The paper does not mention this discrepancy, yet Section 4.3 uses the XCOMET-XXL margin to claim 'considerably higher' performance and 'significant superiority' in minority translation. Moreover, the Mandarin⇔Minority test set is not described: its size, sampling, domains, and whether it overlaps with the human-annotated minority SFT data are all unspecified. This is a core contribution of the paper and must be documented and the discrepancy resolved.
  4. [Abstract, §1] The WMT2025 'ranking first in 30 out of 31 language pairs' claim is unverifiable. No official WMT2025 results are cited, no test set is described, and no scores are given. This claim appears in the abstract and introduction and is central to the paper's SOTA framing. Either provide the official results with a citation and a detailed breakdown, or remove the claim and qualify the abstract accordingly.
minor comments (6)
  1. [§5.3, Table 10] The model name is inconsistently spelled 'Huanyuan-MT-7B' in the human-evaluation section and table; elsewhere it is 'Hunyuan-MT-7B'.
  2. [Table 5] The benchmark 'IINCLUDE' appears to be a typo for 'INCLUDE'.
  3. [§4.1] The wording 'We incorporate development sets from WMT-25, encompassing English-to-XX translations across 25 target languages' followed by 'WMT24pp' is confusing. Please clarify the relationship between WMT24pp and WMT-25 development sets.
  4. [References] The reference for Llama-3.1-8B-Instruct contains a URL for Llama-4-Scout-17B-16E-Instruct; the reference list should be corrected.
  5. [§3.3, Table 3] The Chimera model's candidate-generation procedure is described only as 'varying parameter settings.' Please specify the number of candidates, sampling temperatures, and any selection criteria, since the fusion model's behavior depends on the diversity of the candidate pool.
  6. [§5.3] The human evaluation section does not report the number of annotators, their language proficiency, the annotation protocol, or inter-annotator agreement. These details are needed to assess the reliability of Table 10.

Circularity Check

2 steps flagged

SOTA claim rests on Table 4, but WMT24pp evaluation overlaps SFT data and XCOMET-XXL is both the RL reward and the headline metric.

specific steps
  1. fitted input called prediction [Section 3.1 (SFT corpus) vs Section 4.1 (benchmarks); Table 4]
    "This dataset is aggregated from diverse sources, including established benchmarks like the Flores-200 development set and past WMT test sets... WMT24pp (Deutsch et al., 2025). We incorporate development sets from WMT-25, encompassing English-to-XX translations across 25 target languages. WMT24pp serves as the official development set recommended by WMT25."

    The SFT corpus is stated to include 'past WMT test sets.' The WMT24pp benchmark used in Table 4 is Deutsch et al.'s WMT24++ set, which is built from WMT24 test data. WMT24 is a 'past WMT test set' relative to WMT25, and the paper nowhere excludes it from the SFT corpus. Therefore the WMT24pp rows in Table 4 — a central basis for the abstract's 'significantly outperform' SOTA claim — are not held-out predictions but scores on data that could be in the training set by the paper's own description. The 'prediction' on WMT24pp is thus statistically forced unless the authors can show explicit exclusion, which they do not.

  2. fitted input called prediction [Sections 3.2 and 3.3 (RL rewards) vs Section 4.2 and Table 4 (evaluation)]
    "To ensure translation quality during RL training, we employ two complementary reward signals. The first is XCOMET-XXL, a widely adopted metric in translation evaluation scenarios... The reward function comprises three primary elements: XCOMET-XXL scoring, DeepSeek-V3-0324 scoring, and a repetition penalty term. For automatic evaluation, we use the neural metrics XCOMET-XXL (Guerreiro et al., 2023) and CometKiwi..."

    The model is trained by maximizing XCOMET-XXL during both RL (Section 3.2) and weak-to-strong RL (Section 3.3), and then evaluated primarily with XCOMET-XXL in Table 4 and Figure 1. The reported margins over baselines that were not trained with this reward are therefore partly an artifact of optimizing the evaluation metric itself. This is especially clear for the claimed Chimera improvement of 2.3% average XCOMET-XXL gain (Section 4.3): the fusion model was trained with XCOMET-XXL as its reward, and the same metric is used to measure the improvement. The headline comparison is thus a self-referential fit to the evaluation metric, not an independent measurement of translation quality.

full rationale

The central SOTA claim is not independently supported by the reported automatic benchmarks. Two specific reductions make Table 4 partially circular. First, Section 3.1 says the SFT corpus includes 'past WMT test sets'; Section 4.1 evaluates on WMT24pp, which is Deutsch et al.'s WMT24++ built from WMT24 test data and is the WMT25 development set. Unless WMT24 was explicitly excluded from 'past WMT test sets' — and the paper does not say so — the WMT24pp rows are training-set scores, so the headline comparison against models like Gemini-2.5-Pro on WMT24pp is not a held-out prediction. Second, XCOMET-XXL is used as the RL reward (Sections 3.2 and 3.3) and as the primary automatic metric (Section 4.2, Table 4, Figure 1). The model was trained to maximize this exact scalar, so its high XCOMET-XXL scores relative to models not trained with that reward are partly a reward-overfitting artifact; the reported Chimera improvement of 2.3% is the same metric used to train the fusion model. The paper's own human evaluation (Table 10) shows Hunyuan-MT-7B essentially tied with Gemini-2.5-Pro (3.189 vs 3.223), not the 'significantly outperform ... most of the SOTA large models' claim, and provides no human support for the Mandarin-minority SOTA claim. The WMT2025 'first in 30/31' statement is reported without official results or a test set, so it cannot be independently checked. Self-citations (TAT-R1, Song et al. 2025b) are used as technique components but are not the load-bearing justification for the SOTA claim; the circularity is the benchmark-overlap and metric-as-reward issue. The open-source release is a real contribution, but it does not validate the headline benchmark comparison.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim rests on data curation choices (mixtures, thresholds), evaluation assumptions (metric validity, non-overlap) and the unverified fairness of two privately-built test sets. No new physical or mathematical entities are introduced. The most fragile ledger items are the benchmark overlap and the use of reward metrics as evaluation metrics.

free parameters (5)
  • RegMix data mixture ratio = not specified
    Fitted on small-scale model loss to predict the optimal multilingual mixture for MT-oriented pretraining (Section 2.2).
  • Quality filtering thresholds = not specified
    CometKiwi and GEMBA score cutoffs chosen by hand to discard low-quality SFT pairs (Section 3.1).
  • Number of Chimera candidates = 6
    Six candidate translations are fed to the fusion model (Table 3); the number is chosen by hand.
  • RL composite reward weights = not specified
    Weights combining XCOMET, DeepSeek GEMBA, terminology and repetition penalty are not given (Section 3.2).
  • Two-stage SFT dataset sizes = 3M and 268K
    Dataset sizes for the two SFT stages are hand-selected (Section 3.1).
axioms (5)
  • domain assumption XCOMET-XXL and CometKiwi correlate with human translation quality
    Invoked in Section 4.2 to justify automatic metrics as substitutes for human judgment; the same metrics also appear in training rewards.
  • domain assumption DeepSeek-V3-0324 is a reliable translation quality evaluator via GEMBA
    Used both to filter SFT data (Section 3.1) and as an RL reward (Section 3.2), without validating its neutrality.
  • ad hoc to paper Training on Flores-200 dev and past WMT test sets does not invalidate benchmark evaluation
    Implied by using those sets in SFT (Section 3.1) while evaluating on FLORES-200 and WMT24pp (Section 4.1); contradicted by the construction of WMT24pp from WMT24 test sets.
  • ad hoc to paper The constructed Hunyuan-MT Challenge Testset and Mandarin-minority testset are fair, non-overlapping samples
    Sections 5.3 and 4.1 describe these sets without release or overlap analysis, so the fairness assumption is unverifiable.
  • ad hoc to paper RegMix's fitted loss function extrapolates to the full 7B model
    Section 2.2 adopts small-scale RegMix experiments to set the full-scale mixture, assuming the fitted function transfers.

pith-pipeline@v1.4.0-alltime-deepseek-medium · 22325 in / 10748 out tokens · 96399 ms · 2026-08-05T05:28:33.227211+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Hunyuan-MT Technical Report." pith.science (2026). https://pith.science/paper/RADT2QID

@misc{pith2026250905209,
  author       = {Pith},
  title        = {Pith review of: Hunyuan-MT Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RADT2QID}},
  note         = {Machine review of arXiv:2509.05209}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In this report, we introduce Hunyuan-MT-7B, our first open-source multilingual translation model, which supports bidirectional translation across 33 major languages and places a special emphasis on translation between Mandarin and several ethnic minority languages as well as dialects. Furthermore, to serve and address diverse translation scenarios and enhance model performance at test time, we introduce Hunyuan-MT-Chimera-7B, a translation model inspired by the slow thinking mode. This model integrates multiple outputs generated by the Hunyuan-MT-7B model under varying parameter settings, thereby achieving performance superior to that of conventional slow-thinking models based on Chain-of-Thought (CoT). The development of our models follows a holistic training process specifically engineered for multilingual translation, which begins with general and MT-oriented pre-training to build foundational capabilities, proceeds to Supervised Fine-Tuning (SFT) for task-specific adaptation, and culminates in advanced alignment through Reinforcement Learning (RL) and weak-to-strong RL. Through comprehensive experimentation, we demonstrate that both Hunyuan-MT-7B and Hunyuan-MT-Chimera-7B significantly outperform all translation-specific models of comparable parameter size and most of the SOTA large models, particularly on the task of translation between Mandarin and minority languages as well as dialects. In the WMT2025 shared task (General Machine Translation), our models demonstrate state-of-the-art performance, ranking first in 30 out of 31 language pairs. This result highlights the robustness of our models across a diverse linguistic spectrum, encompassing high-resource languages such as Chinese, English, and Japanese, as well as low-resource languages including Czech, Marathi, Estonian, and Icelandic.

Figures

Figures reproduced from arXiv: 2509.05209 by Bingxin Qu, Di Wang, Mao Zheng, Mingrui Sun, Mingyang Song, Yang Du, Zheng Li.

Figure 1
Figure 1. Figure 1: Benchmark performance of Hunyuan-MT models and state-of-the-art baselines. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Post-training pipeline of the Hunyuan-MT-7B and Hunyuan-MT-Chimera-7B models. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluation

    cs.CL 2026-01 conditional novelty 7.0

    MENT benchmark plus RATE agentic evaluator raise combined system- and segment-level correlation with human judgments by at least 3.2 points over prior MT metrics and LLM judges.

  2. Reasoning over Grammar: Can Synthetic Linguistic Reasoning Traces Enhance Low-Resource Machine Translation?

    cs.CL 2026-06 unverdicted novelty 6.0

    Synthetic linguistic reasoning traces improve low-resource machine translation most when supplied as reliable in-context guidance, but produce smaller and less consistent gains when used as training data for fine-tuning.

  3. NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learning

    cs.CL 2026-01 unverdicted novelty 6.0

    NeoAMT trains an RL agent with a Wiktionary toolkit, novel reward, and adaptive rollouts to translate sentences containing neologisms in 16 languages and 75 directions.

  4. NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learning

    cs.CL 2026-01 conditional novelty 6.0

    An RL-trained translation agent that searches Wiktionary during reasoning improves neologism translation, and the new Neko benchmark covers 16 languages and 75 directions.

  5. Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models

    cs.CL 2026-02 conditional novelty 5.0

    Open 12B multilingual translator MiLMMT-46, trained with up to 3B tokens of parallel data per language plus ~100K curated instruction pairs, matches proprietary systems on FLORES+ and WMT24++.

  6. UltraFlux: Data-Model Co-Design for High-quality Native 4K Text-to-Image Generation across Diverse Aspect Ratios

    cs.CV 2025-11 conditional novelty 5.0

    A data-model co-designed Flux DiT with resonance RoPE, post-trained F16 VAE, SNR-aware Huber wavelet loss, and aesthetic curriculum claims state-of-the-art open-source 4K multi-AR text-to-image generation.

  7. HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task

    cs.CL 2026-06 unverdicted novelty 4.0

    HydraQE is a new end-to-end speech translation QE system using Qwen3-ASR backbone, sparsemax layer mixing, bidirectional Transformer, and multi-task curriculum training on human and pseudo labels that outperforms casc...

  8. When Your LLM Reaches End-of-Life: A Framework for Confident Model Migration in Production Systems

    cs.AI 2026-04 unverdicted novelty 4.0

    A Bayesian calibration technique lets production teams compare replacement LLMs reliably using limited human feedback on correctness, refusal, and style.

  9. CAT-Translate: Building Compact Open-Source Models for Japanese-English Translation

    cs.CL 2026-06 unverdicted novelty 3.0

    Compact 0.8B-7B models for bidirectional Japanese-English translation outperform large multilingual models on real-world domain benchmarks.

  10. Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation

    cs.CL 2025-04 unverdicted novelty 3.0

    A literature survey that organizes prompting, fine-tuning, preference optimization, and context-aware techniques for LLM-based machine translation with emphasis on low-resource languages.

Reference graph

Works this paper leans on

56 extracted references · 22 canonical work pages · cited by 9 Pith papers

  1. [1]

    Rishabh Agarwal, Avi Singh, Lei Zhang, Bernd Bohnet, Luis Rosias, Stephanie C. Y. Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, John D. Co - Reyes, Eric Chu, Feryal M. P. Behbahani, Aleksandra Faust, and Hugo Larochelle. Many-shot in-context learning. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tom...

    work page 2024

  2. [2]

    Llama-3.1-8b-instruct

    AI@Meta. Llama-3.1-8b-instruct. https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct, 2025 a . Accessed: 2025-08-26

    work page 2025

  3. [3]

    Llama-4-scout-17b-16e-instruct

    AI@Meta. Llama-4-scout-17b-16e-instruct. https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct, 2025 b . Accessed: 2025-08-26

    work page 2025

  4. [4]

    Introducing claude 4

    Anthropic. Introducing claude 4. https://www.anthropic.com/news/claude-4, 2025. Accessed: 2025-08-26

    work page 2025

  5. [5]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. URL http://arxiv.org/abs/1409.0473

    Pith/arXiv arXiv 2015

  6. [6]

    Brown, John Cocke, Stephen Della Pietra, Vincent J

    Peter F. Brown, John Cocke, Stephen Della Pietra, Vincent J. Della Pietra, Frederick Jelinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin. A statistical approach to machine translation. Comput. Linguistics, 16 0 (2): 0 79--85, 1990

    work page 1990

  7. [7]

    Brown, Stephen Della Pietra, Vincent J

    Peter F. Brown, Stephen Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine translation: Parameter estimation. Comput. Linguistics, 19 0 (2): 0 263--311, 1993

    work page 1993

  8. [8]

    Findings of the wmt 2016 bilingual document alignment shared task

    Christian Buck and Philipp Koehn. Findings of the wmt 2016 bilingual document alignment shared task. In Proceedings of the First Conference on Machine Translation, pp.\ 554--563, Berlin, Germany, August 2016. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/W/W16/W16-2347

    work page 2016

  9. [9]

    Feldman, Arjun Guha, Michael Greenberg, and Abhinav Jangda

    Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps - Costin, Donald Pinckney, Ming - Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q. Feldman, Arjun Guha, Michael Greenberg, and Abhinav Jangda. Multipl-e: A scalable and polyglot approach to benchmarking neural code generation. IEEE Trans. Software Eng. , 49 0 (7): 0 3675--3691, 2...

    work page arXiv 2023

  10. [10]

    Seed-x: Building strong multilingual translation llm with 7b parameters, 2025

    Shanbo Cheng, Yu Bao, Qian Cao, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, Wenhao Zhu, Jingwen Chen, Zhichao Huang, Tao Li, Yifu Li, Huiying Lin, Sitong Liu, Ningxin Peng, Shuaijie She, Lu Xu, Nuo Xu, Sen Yang, Runsheng Yu, Yiming Yu, Liehao Zou, Hang Li, Lu Lu, Yuxuan Wang, and Yonghui Wu. Seed-x: Building strong multilingual translation llm with 7b ...

    Pith/arXiv arXiv 2025

  11. [11]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168

    Pith/arXiv arXiv 2021

  12. [12]

    Multilingual machine translation with open large language models at practical scale: An empirical study, 2025

    Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, and Bin Wang. Multilingual machine translation with open large language models at practical scale: An empirical study, 2025. URL https://arxiv.org/abs/2502.02481

    Pith/arXiv arXiv 2025

  13. [13]

    We’re expanding our gemini 2.5 family of models

    DeepMind. We’re expanding our gemini 2.5 family of models. https://blog.google/products/gemini/gemini-2-5-model-family-expands/, 2025. Accessed: 2025-08-26

    work page 2025

  14. [14]

    Deepseek-v3 technical report, 2024

    DeepSeek-AI. Deepseek-v3 technical report, 2024. URL https://arxiv.org/abs/2412.19437

    Pith/arXiv arXiv 2024

  15. [15]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    Pith/arXiv arXiv 2025

  16. [16]

    WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects , 2025

    Daniel Deutsch, Eleftheria Briakou, Isaac Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects , 2025. URL h...

    Pith/arXiv arXiv 2025

  17. [17]

    Cruxeval: A benchmark for code reasoning, understanding and execution

    Alex Gu, Baptiste Rozi \` e re, Hugh James Leather, Armando Solar - Lezama, Gabriel Synnaeve, and Sida Wang. Cruxeval: A benchmark for code reasoning, understanding and execution. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL https://openreview.net/forum?id=Ffpg52swvg

    work page 2024

  18. [18]

    Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F

    Nuno M. Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André F. T. Martins. xcomet: Transparent machine translation evaluation through fine-grained error detection, 2023. URL https://arxiv.org/abs/2310.10482

    Pith/arXiv arXiv 2023

  19. [19]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . OpenReview.net, 2021. URL https://openreview.net/forum?id=d7KBjmI3GmQ

    2021

  20. [20]

    Tencent minority-mandarin translation system

    Bojie Hu, Ambyer Han, Zheyang Zhang, Shen Huang, and Qi Ju. Tencent minority-mandarin translation system. In Shujian Huang and Kevin Knight (eds.), Machine Translation, pp.\ 93--104, Singapore, 2019. Springer Singapore. ISBN 978-981-15-1721-1

    work page 2019

  21. [21]

    Is chatgpt A good translator? A preliminary study

    Wenxiang Jiao, Wenxuan Wang, Jen - tse Huang, Xing Wang, and Zhaopeng Tu. Is chatgpt A good translator? A preliminary study. CoRR, abs/2301.08745, 2023. doi:10.48550/ARXIV.2301.08745. URL https://doi.org/10.48550/arXiv.2301.08745

    Pith/arXiv arXiv doi:10.48550/arxiv.2301.08745 2023

  22. [22]

    Large language models are state-of-the-art evaluators of translation quality

    Tom Kocmi and Christian Federmann. Large language models are state-of-the-art evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pp.\ 193--203, Tampere, Finland, June 2023. European Association for Machine Translation. URL https://aclanthology.org/2023.eamt-1.19

    work page 2023

  23. [23]

    Findings of the WMT24 general machine translation shared task: The LLM era is here but MT is not solved yet

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popovic, Mariya Shmatova, Steinth \' o r Steingr \' msson, and ...

    2024

  24. [24]

    Tat-r1: Terminology-aware translation with reinforcement learning and word alignment, 2025

    Zheng Li, Mao Zheng, Mingyang Song, and Wenjie Yang. Tat-r1: Terminology-aware translation with reinforcement learning and word alignment, 2025. URL https://arxiv.org/abs/2505.21172

    Pith/arXiv arXiv 2025

  25. [25]

    Assimilation over protection: rethinking mandarin language assimilation in china

    Cong Lin and Liz Jackson. Assimilation over protection: rethinking mandarin language assimilation in china. Multicultural Education Review, 13 0 (4): 0 338--361, 2021. doi:10.1080/2005615X.2021.2006117

    work page arXiv 2021

  26. [26]

    Regmix: Data mixture as regression for language model pre-training, 2025

    Qian Liu, Xiaosen Zheng, Niklas Muennighoff, Guangtao Zeng, Longxu Dou, Tianyu Pang, Jing Jiang, and Min Lin. Regmix: Data mixture as regression for language model pre-training, 2025. URL https://arxiv.org/abs/2407.01492

    Pith/arXiv arXiv 2025

  27. [27]

    Deepcoder: A fully open-source 14b coder at o3-mini level

    Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepcoder: A fully open-source 14b coder at o3-mini level. https://pretty-radio-b75.notion.site/DeepCoder-A-Fully-Open-Source-14B-Coder-at-O3-mini-Level-1cf81902c14680b3bee5...

    work page 2025

  28. [28]

    Cand \` e s, and Tatsunori Hashimoto

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei - Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel J. Cand \` e s, and Tatsunori Hashimoto. s1: Simple test-time scaling. CoRR, abs/2501.19393, 2025. doi:10.48550/ARXIV.2501.19393. URL https://doi.org/10.48550/arXiv.2501.19393

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.19393 2025

  29. [29]

    Introducing gpt-4.1 in the api

    OpenAI. Introducing gpt-4.1 in the api. https://openai.com/index/gpt-4-1/, 2025. Accessed: 2025-08-26

    work page 2025

  30. [30]

    Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures

    Pedro Javier Ortiz Su 'a rez , Benoit Sagot, and Laurent Romary. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. Cardiff, 22nd July 2019, pp.\ 9 -- 16, Mannheim, 2019. Leibniz-Institut f "u r Deutsche Sprache. doi:10.14...

    doi:10.14618/ids-pub-9021 2019

  31. [31]

    A monolingual approach to contextualized word embeddings for mid-resource languages

    Pedro Javier Ortiz Su 'a rez, Laurent Romary, and Benoit Sagot. A monolingual approach to contextualized word embeddings for mid-resource languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.\ 1703--1714, Online, July 2020. Association for Computational Linguistics. URL https://www.aclweb.org/anthology/...

    work page 2020

  32. [32]

    Salute the classic: Revisiting challenges of machine translation in the age of large language models

    Jianhui Pang, Fanghua Ye, Derek Fai Wong, Dian Yu, Shuming Shi, Zhaopeng Tu, and Longyue Wang. Salute the classic: Revisiting challenges of machine translation in the age of large language models. Trans. Assoc. Comput. Linguistics, 13: 0 73--95, 2025. doi:10.1162/TACL\_A\_00730. URL https://doi.org/10.1162/tacl\_a\_00730

    doi:10.1162/tacl 2025

  33. [33]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA , pp.\ 311--318. ACL , 2002. doi:10.3115/1073083.1073135. URL https://aclanthology.org/P02-1040/

    arXiv 2002

  34. [34]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv e-prints, 2019

    2019

  35. [35]

    Guerreiro, Chrysoula Zerva, Ana C

    Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C. Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte M. Alves, Alon Lavie, Luisa Coheur, and André F. T. Martins. Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task, 2022. URL https://arxiv.org/abs/2209.06243

    Pith/arXiv arXiv 2022

  36. [36]

    Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F

    Ricardo Rei, Nuno M. Guerreiro, José Pombal, João Alves, Pedro Teixeirinha, Amin Farajian, and André F. T. Martins. Tower+: Bridging generality and translation specialization in multilingual llms, 2025. URL https://arxiv.org/abs/2506.17080

    Pith/arXiv arXiv 2025

  37. [37]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q & a benchmark. CoRR, abs/2311.12022, 2023. doi:10.48550/ARXIV.2311.12022. URL https://doi.org/10.48550/arXiv.2311.12022

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.12022 2023

  38. [38]

    Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A. Haggag, Imanol Schlag, Marzieh Fadaee, Sara Hooker, Antoine Bosselut, Snegha A, Alfonso Amayuelas, Azril Hafizi Amirudin, Viraat Aryabumi, Danylo Boiko, Michael Chang, Jenny Chim, Gal Cohen, Aditya Kumar Dalm...

    work page 2025

  39. [39]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi:10.48550/ARXIV.2402.03300. URL https://doi.org/10.48550/arXiv.2402.03300

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.03300 2024

  40. [40]

    Language models are multilingual chain-of-thought reasoners

    Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. Language models are multilingual chain-of-thought reasoners. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.ne...

    work page 2023

  41. [41]

    Fastcurl: Curriculum reinforcement learning with stage-wise context scaling for efficient training r1-like reasoning models, 2025 a

    Mingyang Song, Mao Zheng, Zheng Li, Wenjie Yang, Xuan Luo, Yue Pan, and Feng Zhang. Fastcurl: Curriculum reinforcement learning with stage-wise context scaling for efficient training r1-like reasoning models, 2025 a . URL https://arxiv.org/abs/2503.17287

    arXiv 2025

  42. [42]

    Can many-shot in-context learning help llms as evaluators? A preliminary empirical study

    Mingyang Song, Mao Zheng, and Xuan Luo. Can many-shot in-context learning help llms as evaluators? A preliminary empirical study. In Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al - Khalifa, Barbara Di Eugenio, and Steven Schockaert (eds.), Proceedings of the 31st International Conference on Computational Linguistics, COLING 2025, Abu Dhabi, UAE, J...

    work page 2025

  43. [43]

    Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neural networks. In Zoubin Ghahramani, Max Welling, Corinna Cortes, Neil D. Lawrence, and Kilian Q. Weinberger (eds.), Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec,...

    work page 2014

  44. [44]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Bey...

    Pith/arXiv arXiv 2025

  45. [45]

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Hongcheng Gao, Peizhong Gao, Tong Gao, Xinran Gu, Longyu Guan, Haiqing Guo, Jianhang Guo, Ha...

    Pith/arXiv arXiv 2025

  46. [46]

    M. - A - P. Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, Kang Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, Chujie Zheng, Kaixin Deng, Shian Jia, Sichao Jiang, Yiyan Liao, Rui Li, Qinrui Li, Sirun Li, Yizhi Li, Yunwen Li, Dehua Ma, Yuansheng Ni, Haoran Que, Qiyao Wang, Zhoufutu Wen, Siwei Wu, Tianshun Xing, Ming Xu, Zhen...

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.14739 2025

  47. [47]

    NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe,...

    Pith/arXiv arXiv 2022

  48. [48]

    Qwen3 technical report, 2025

    Qwen Team. Qwen3 technical report, 2025. URL https://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  49. [49]

    Parallel data, tools and interfaces in opus

    J \"o rg Tiedemann. Parallel data, tools and interfaces in opus. In Lrec, volume 2012, pp.\ 2214--2218, 2012

    work page 2012

  50. [50]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  51. [51]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang, Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela ...

    work page 2024

  52. [52]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL https://arxiv.org/abs/2201.11903

    Pith/arXiv arXiv 2023

  53. [53]

    Google's neural machine translation system: Bridging the gap between human and machine translation

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google's neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016

    Pith/arXiv arXiv 2016

  54. [54]

    A survey on test-time scaling in large language models: What, how, where, and how well?, 2025

    Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. A survey on test-time scaling in large language models: What, how, where, and how well?, 2025. URL https://arxiv.org/abs/2503.24235

    Pith/arXiv arXiv 2025

  55. [55]

    Multilingual machine translation with large language models: Empirical results and analysis

    Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. Multilingual machine translation with large language models: Empirical results and analysis. In Kevin Duh, Helena G \' o mez - Adorno, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexic...

    doi:10.18653/v1/2024.findings-naacl.176 2024

  56. [56]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...