Pith. sign in

REVIEW 3 major objections 7 minor 5 cited by

Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B open-weight model claims translation parity with GPT-4o and Gemini-2.5 across 28 languages.

desk verdict A serious, well-executed engineering report whose headline parity claim is plausible but not yet airtight, because the paper never shows test-set contamination checks or error bars. read the letter →

arxiv 2507.13618 v4 pith:5D4YCYKP submitted 2025-07-18 cs.CL cs.AI

Shanbo Cheng , Yu Bao , Qian Cao , Luyang Huang , Liyan Kang , Zhicheng Liu , Yu Lu , Wenhao Zhu
show 18 more authors
Jingwen Chen Zhichao Huang Tao Li Yifu Li Huiying Lin Sitong Liu Ningxin Peng Shuaijie She Lu Xu Nuo Xu Sen Yang Runsheng Yu Yiming Yu Liehao Zou Hang Li Lu Lu Yuxuan Wang Yonghui Wu
This is my paper · ORCID
classification cs.CLcs.AI
keywords multilingualmachinetranslationlargelanguagemodelschain-of-thoughtreinforcementlearningdualpreferenceoptimizationpretrainingopen-sourcemodelevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Seed-X is a 7B-parameter, open-weight translation model family trained from scratch on 28 languages, and the paper's central claim is that this small model matches or beats the largest proprietary translators. The authors report that the final reinforcement-learned variant scores 76.25 BLEURT and 90.81 COMET on average across FLORES-200 and WMT-25 directions, placing it level with Gemini-2.5-Pro, GPT-4o, and DeepSeek-R1, and ahead of every open baseline. In their own human evaluation across 14 language directions it averages 3.73 out of 4, the highest score in the table. The paper also argues that the path to this result is a data recipe rather than scale: staged pretraining from monolingual to parallel-only data, chain-of-thought annotation by linguists, and preference learning with dual rewards. If the claim survives independent checking, it means frontier translation quality does not require frontier-scale models.

What carries the argument

The load-bearing mechanism is the staged data curriculum the paper calls the mono-parallel balance. Pretraining starts on monolingual text from high-resource languages, shifts toward multilingual and parallel data, and finishes on parallel-only data concatenated with language-tag delimiters such as <EN> or <ZH>; this ordering is what transfers knowledge from English and Chinese into distant languages. On top of that base, the instruct model is fine-tuned on 236K translation instances that include linguist-written chain-of-thought rationales, and the RL stage optimizes with PPO using two reward types: a learned human-preference reward for high-resource pairs, and a dual reward that translates A to B and back to A and scores the round-trip similarity where no preference data exists. The language tags act as the alignment signal, the CoT annotations as the reasoning scaffold, and the dual reward as the substitute for human feedback in low-resource directions.

What would settle it

Search the released training corpora for exact and near-duplicate sentences from FLORES-200 and WMT-25, then re-run the evaluation on a newly written parallel test set that is known to be absent from training; if the Seed-X scores drop below the closed models' by a clear margin, the parity claim is refuted.

Watch

Extended reading notes

Core claim

The paper's own claim is that a 7B open-weight model can translate at the level of closed-source frontier systems. The supporting results are the reported FLORES-200 and WMT-25 scores, where Seed-X-PPO lands within a decimal of Claude-3.5-Sonnet and above every open model tested, and the Seed-X-Challenge human evaluation, where it takes first place overall with 3.73/4. The authors attribute the result to the training design: a 6-trillion-token monolingual and bilingual pretraining corpus, a three-stage curriculum that ends in parallel-only data organized by language tags, instruction tuning with professionally annotated chain-of-thought explanations, and reinforcement learning using a learned preference reward for high-resource pairs plus a dual round-trip reward for the rest.

Load-bearing premise

The load-bearing premise is that the public web data used for training does not contain the FLORES-200 and WMT-25 sentences the model is later scored on; the paper reports no overlap check, so the headline automatic scores stand or fall with that assumption.

Editorial extensions

If this is right

  • Open-weight 7B models become viable replacements for closed translation APIs in production, since the reported quality is comparable without proprietary serving.
  • The three-stage recipe (monolingual base, multilingual ramp, parallel-only finish) gives a reusable template for building task-specialized LLMs on a 7B budget.
  • Chain-of-thought supervision and round-trip dual rewards can cover low-resource directions that lack human preference data, which is the usual bottleneck for RL in translation.
  • The released challenge set and reward model give the community tools to measure idiomatic, slang, and literary translation rather than news-style sentences only.
  • If the recipe generalizes, parameter count is not the main lever for translation quality; data ordering and data quality are.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe applied to a larger base model would likely widen the reported margin over closed systems, because the paper's own curves show quality still rising with tokens and the remaining average gap to Claude-3.5 is small.
  • The dual-reward mechanism is reference-free, so it could in principle be pressed into service for languages beyond the supported 28; the paper does not test that extension.
  • Until an overlap check against FLORES-200 and WMT-25 is published, the fair reading is that the human evaluation carries the argument and the automatic scores are provisional.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents Seed-X, a 7B-parameter open-weight multilingual translation LLM family trained from scratch on a curated 6T-token corpus spanning 28 languages, followed by supervised fine-tuning with chain-of-thought reasoning and reinforcement learning (PPO and DuPO). The authors report BLEURT and COMET scores on FLORES-200 and WMT-25 showing Seed-X-PPO at or near the level of frontier closed models such as Gemini-2.5-Pro, GPT-4o, and Claude-3.5-Sonnet, and above larger open models. They also introduce a human-annotated Seed-X-Challenge testset covering idioms, slang, and classical text, on which Seed-X-PPO receives the highest average 0--4 human score among the compared systems. The paper includes detailed ablations on data quality, prompt design, delimiters, and mono\u2013parallel data balance, as well as an analysis of knowledge transfer from core to secondary languages.

Significance. If the evaluation withstands scrutiny, the result is practically significant: a 7B open model matching frontier closed models on multilingual translation would be a strong community resource, and the released models, challenge testset, and training recipe would lower the cost of building specialized translation LLMs. The paper also makes a useful methodological contribution by deliberately avoiding automatic MT metrics as RL rewards and by documenting trade-offs between monolingual and parallel data. The main novelty is in the engineering and data pipeline rather than in a new learning algorithm; the value of the paper depends on whether the headline performance claims survive the evaluation-validity concerns below.

major comments (3)
  1. [§2.1, §2.2, §3.1, Table 4] The headline claim of parity with Gemini-2.5-Pro and GPT-4o rests on BLEURT/COMET scores on FLORES-200 and WMT-25 reported in Table 4, but the paper reports no decontamination analysis. The 6T-token monolingual corpus is built from publicly available web data (§2.1), and the bilingual corpus is iteratively expanded by translating monolingual web sentences with an earlier Seed-X model (§2.2), so source sentences from FLORES-200 devtest and WMT-25 sets could plausibly appear in training or augmentation. Moreover, §4.1 states that the WMT-25 evaluation uses “dev sets from WMT-25”, which are public rather than hidden, making test-set leakage more likely rather than less. Finally, §3.1 uses the public FLORES devset for SFT; although dev and devtest are nominally disjoint, the paper does not demonstrate that devtest sentences were excluded from the earlier pretraining or augmentation stages. The authors should report n-gram or embedding-level overlap between all training stages and the test sets, and ideally recompute scores after removing near-duplicate test items.
  2. [§4.2, Table 4, Figure 2] The comparison with open-source baselines is weakened by the post hoc statement that “we only evaluate the highest-performing variant from each model series.” For model families such as Qwen3, Gemma3, and Llama-4, many variant sizes and instruction versions exist, and the paper does not list which variants were tried or what scores they obtained. Choosing the best variant after seeing the results makes the claim that Seed-X “significantly outperforms larger open-source models” difficult to falsify. The authors should either report scores for all variants considered or specify a pre-registered selection rule. In addition, Table 4 reports only aggregate averages with no confidence intervals or significance tests; Seed-X-PPO and Claude-3.5-Sonnet differ by 0.38 BLEURT and 0.26 COMET on the average, which may be within evaluation noise given that per-direction and per-item variance are not reported.
  3. [§4.2, Figure 3, Table 13, Appendix D] The human evaluation is conducted on the authors’ own Seed-X-Challenge testset, but the paper reports no inter-annotator agreement, no number of sentences per direction, no number of annotators, and no confidence intervals for the 0–4 scores. The claimed advantages are small: Seed-X-PPO averages 3.73 versus 3.68 for Claude-3.5-Sonnet and 3.66 for GPT-4o and Gemini-2.5-Pro. Without reliability statistics, such differences are not established as meaningful. The authors should report agreement measures (e.g., Krippendorff’s alpha), the number of judgments per direction, and per-item score distributions, and should release the annotation guidelines and the testset to allow independent verification.
minor comments (7)
  1. [Abstract] The phrase “make the parameter public available” is ungrammatical; it should read “make the parameters publicly available.”
  2. [§4.1, Table 4] The WMT-25 column appears to use public dev sets, not the official hidden WMT-25 test sets; the column label and the claims in the abstract and figures should say “WMT-25 dev” to avoid implying evaluation on the official test sets.
  3. [§5.2] The bullet “Continue training on pure parallel data maximizes translation effectiveness” says this is “empirically validated in Table 4c”, but the cited evidence appears in Figure 4c; the cross-reference should be corrected.
  4. [References] References [34] and [35] are identical (both cite Guerreiro et al., “xcomet”); one should be removed or merged, and the reference list should be deduplicated.
  5. [Appendix A] The supported-language list includes both “Norwegian Bokmal (nb)” and “Norwegian (no)”, which is confusing; the authors should clarify whether these are two distinct codes or a duplication, and how the XX⇒XX evaluation treats them.
  6. [§2.2] The term “boostingSeed-X model” appears before any Seed-X model has been properly defined; the iterative boosting process should be described more clearly in terms of which model versions are used at each round.
  7. [§4.2] For closed baselines such as Gemini-2.5-Pro, GPT-4o, and Google Translator, the exact API versions and decoding parameters are not reported; this information should be provided for reproducibility, especially since decoding settings can affect translation quality.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the training pipeline is empirical, the headline claim is judged on external benchmarks, and same-group citations are used as implementation tools, not as load-bearing premises.

full rationale

The paper does not claim a formal derivation from assumptions to predictions; it trains a 7B translation model and measures it on external benchmarks. The evaluation anchors (FLORES-200, WMT-25, and human expert judgments) are independent of the training objective, and Section 3.2 explicitly states that automatic metrics are not used as rewards: 'we avoid using automatic evaluation metrics (e.g., BLEU, BLEURT or COMET) as rewards.' The same-group citations (G-DIG, DuPO, language-tag work) are used as data-selection, reward, or formatting tools; none of them is invoked as the authority for the headline parity claim. The weak-to-strong bilingual data loop in Section 2.2 uses a Seed-X-based temporary model to generate pseudo-parallel training data, but this is iterative self-training rather than a definitional equivalence: the final claim is still assessed on held-out public test sets and human evaluation. No equation reduces the predicted quantity to a fitted input, and no fitted parameter is relabeled as a prediction. The skeptic's leakage concern about the undecontaminated web corpus is a data-hygiene risk that could inflate automatic scores, but it is not a form of circularity under the definitions used here. Therefore no circular steps are identified.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central claim depends on several unverified assumptions: representativeness of the FLORES subset, reliability of neural metrics, absence of test-set contamination, unbiased human evaluation, and the soundness of self-generated training data. No new physical or conceptual entities are introduced; the Seed-X-Challenge is a benchmark artifact, not an invented entity.

free parameters (9)
  • Pretraining vocabulary size = 65,269 tokens
    Expanded from Mistral's 32,000 tokens to improve multilingual compression; chosen by the authors.
  • Maximum sequence length = 2,048
    Set to match translation requirements, a modeling choice.
  • Pretraining learning rate = 3e-4
    Tuned through scaling laws and experiments (Appendix B).
  • Pretraining batch size = 2M tokens
    Tuned through scaling laws and experiments (Appendix B).
  • SFT learning rate = 3e-6
    Chosen for supervised fine-tuning stability (Appendix B).
  • SFT batch size = 64 sentences
    Chosen for supervised fine-tuning (Appendix B).
  • Inference beam size = 4
    Beam search slightly outperforms greedy decoding (Appendix B).
  • Human preference data size = ~20k pairs
    Preference data collected only for high-resource language pairs due to annotation cost (Section 3.2).
  • Data mixture ratios across stages = Monolingual proportions in Table 1; three-stage schedule S1/S2/S3
    Hand-designed curriculum to prioritize English/Chinese then expand multilingual coverage (Section 2.3).
assumptions (6)
  • domain assumption FLORES-200 subset of 756 pairs is representative of 'across 28 languages' performance
    The claim covers 28 languages but only selected pairs are evaluated; not all directions are covered.
  • domain assumption BLEURT and XCOMET-XL scores are reliable proxies for translation quality
    Automatic metrics are used for headline comparisons; the paper itself notes these metrics can miss translation phenomena (Section 4.1).
  • domain assumption The pretraining corpus does not contain FLORES-200/WMT-25 test sentences
    No decontamination is reported, and the corpus is assembled from public web data (Section 2.1).
  • ad hoc to paper Human 0-4 scores by professional linguists are unbiased without inter-annotator agreement
    The challenge testset and annotations are produced by the authors' team; no agreement statistics are given (Appendix D).
  • ad hoc to paper Pseudo-parallel data generated by an earlier version of the same model family is a valid training signal
    The iterative boosting process uses Seed-X and its predecessors to create, rewrite, and filter training data (Section 2.2), which could propagate model biases.
  • ad hoc to paper Choosing the highest-performing variant of each baseline model is fair
    The authors explicitly select variants post hoc, which can favor the comparison (Section 4.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters." pith.science (2026). https://pith.science/paper/5D4YCYKP

@misc{pith2026250713618,
  author       = {Pith},
  title        = {Pith review of: Seed-X: Building Strong Multilingual Translation LLM with 7B Parameters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5D4YCYKP}},
  note         = {Machine review of arXiv:2507.13618}
}
read the original abstract

Multilingual translation stands as a challenging task for large language models (LLMs) to handle intricate language patterns and stilted translations that arise in automated translations. In this paper, we introduce Seed-X, a family of open-source LLMs comprising instruct and reasoning models, pushing the limits of translation capability with 7B parameter size. The base model is pre-trained on a diverse, high-quality dataset encompassing both monolingual and bilingual content across 28 languages, harnessing the full potential of multilingual data. The instruct model is then finetuned to translate by Chain-of-Thought (CoT) reasoning and further enhanced through reinforcement learning (RL) to achieve better generalization across diverse language pairs. Seed-X achieves performance comparable to leading closed-source models, including Gemini-2.5 and GPT-4o, across 28 languages, and significantly outperforms larger open-source models in both automatic metrics and human evaluations. We share the best practices through our optimization process, and make the parameter public available for advancing translation research and applications.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Studying quantization trade-offs for efficient inference deployment in machine translation

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Quantized Hy-MT2 models stay accurate at long context, but quantized EuroLLM 9B/22B models collapse (up to ~60% chrF++ drop) while W4A8/W8A8 plus 200–400-token chunking improves serving throughput.

  2. Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A two-stage alignment framework that first fuses visual modalities (RGB, flow, skeleton) then introduces text, achieving 21% SRCC improvement on a new clinical AQA dataset and gains on two public benchmarks.

  3. Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models

    cs.CL 2026-02 conditional novelty 5.0 of 10

    Open 12B multilingual translator MiLMMT-46, trained with up to 3B tokens of parallel data per language plus ~100K curated instruction pairs, matches proprietary systems on FLORES+ and WMT24++.

  4. Hunyuan-MT Technical Report

    cs.CL 2025-09 conditional novelty 5.0 of 10

    Hunyuan-MT and Chimera, a 7B open-source translation model and its multi-candidate fusion variant, claim state-of-the-art multilingual translation including Mandarin to minority languages, with open weights.

  5. Observation of momentum dependent charge density wave gap in EuTe4

    cond-mat.mes-hall 2025-08 unverdicted novelty 4.0 of 10

    EuTe4 shows a momentum-dependent charge density wave gap at the Fermi level, largest along Gamma-Y and smallest along Gamma-X, plus a low-temperature magnetic phase diagram near TN = 6.9 K.

Reference graph

Works this paper leans on

63 extracted references · 22 canonical work pages · cited by 5 Pith papers

  1. [1]

    Brown, Stephen A

    Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 1993. URL https:// aclanthology.org/J93-2003/

    work page 1993

  2. [2]

    Google’s neural machine translation system: Bridging the gap between human and machine translation.arXiv preprint arXiv:1609.08144, 2016

    Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation.arXiv preprint arXiv:1609.08144, 2016

    arXiv 2016

  3. [3]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017

    2017

  4. [4]

    Team NLLB, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe,...

    arXiv 2022

  5. [5]

    The unreasonable effectiveness of few-shot learning for machine translation

    Xavier Garcia, Yamini Bansal, Colin Cherry, George Foster, Maxim Krikun, Fangxiaoyu Feng, Melvin Johnson, and Orhan Firat. The unreasonable effectiveness of few-shot learning for machine translation, 2023. URL https://arxiv.org/abs/2302.01398

    work page Pith review arXiv 2023

  6. [6]

    Multilingual machine translation with large language models: Empirical results and analysis

    Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. Multilingual machine translation with large language models: Empirical results and analysis. InFindings of the Association for Computational Linguistics: NAACL 2024, 2024. URL https://aclanthology.org/2024. findings-naacl.176/

    work page 2024

  7. [7]

    A paradigm shift in machine translation: Boosting translation performance of large language models

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in machine translation: Boosting translation performance of large language models. InThe TwelfthInternational Conference on Learning Representations, 2024. URLhttps://openreview.net/pdf?id=farT6XXntP

    work page 2024

  8. [8]

    Alves, José Pombal, Nuno M

    Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. Tower: An open multilingual large language model for translation-related tasks, 2024

    work page 2024

Show all 63 references
  1. [9]

    Claude 3.5

    Anthropic. Claude 3.5. https://www.anthropic.com/claude, 2025. Accessed: 2025-04-18

    2025

  2. [10]

    Findings of the WMT24 general machine translation shared task: The LLM era is here but MT is not solved yet

    Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Ma...

    2024

  3. [11]

    Salute the classic: Revisiting challenges of machine translation in the age of large language models.Transactions of the Association for Computational Linguistics, 2025

    Jianhui Pang, Fanghua Ye, Derek Fai Wong, Dian Yu, Shuming Shi, Zhaopeng Tu, and Longyue Wang. Salute the classic: Revisiting challenges of machine translation in the age of large language models.Transactions of the Association for Computational Linguistics, 2025. URL https://...

    2025

  4. [12]

    Gemma 3 technical report.https://arxiv.org/abs/2503.19786, 2025

    Gemma Team. Gemma 3 technical report.https://arxiv.org/abs/2503.19786, 2025

    2025 arXiv

  5. [13]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innovation.https://ai

    AI Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation.https://ai. meta. com/blog/llama-4-multimodal-intelligence/, checked on, 2025

    2025

  6. [14]

    Qwen3 technical report, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

    2025 arXiv

  7. [15]

    Hello gpt-4o

    OpenAI. Hello gpt-4o. https://openai.com/index/hello-gpt-4o/, 2023. Accessed: 2025-04-18

    2023

  8. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.https://arxiv

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.https://arxiv. org/abs/2501.12948, 2025

    2025 arXiv

  9. [17]

    Gemini 2.5

    DeepMind. Gemini 2.5. https://deepmind.google/technologies/gemini/, 2025. Accessed: 2025-04-18

    2025

  10. [18]

    # instag: Instruction tagging for analyzing supervised fine-tuning of large language models

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. # instag: Instruction tagging for analyzing supervised fine-tuning of large language models. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  11. [19]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    arXiv

  12. [20]

    The llama 3 herd of models.ArXiv, abs/2407.21783, 2024

    Meta AI. The llama 3 herd of models.ArXiv, abs/2407.21783, 2024. URLhttps://api.semanticscholar.org/ CorpusID:271571434

    2024 arXiv

  13. [21]

    Qwen2.5 technical report.ArXiv, abs/2412.15115, 2024

    Qwen An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxin Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, ...

    2024 arXiv

  14. [22]

    Include: Evaluating multilingual language understanding with regional knowledge.arXiv preprint arXiv:2411.19799, 2024

    Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shivalika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A Haggag, Alfonso Amayuelas, et al. Include: Evaluating multilingual language understanding with regional knowledge.arXiv preprint ...

    2024 arXiv

  15. [23]

    Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.arXiv preprint arXiv:2009.03300, 2020

    2009 arXiv

  16. [24]

    Xcopa: A multilingual dataset for causal commonsense reasoning.arXiv preprint arXiv:2005.00333, 2020

    Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. Xcopa: A multilingual dataset for causal commonsense reasoning.arXiv preprint arXiv:2005.00333, 2020

    2005 arXiv

  17. [25]

    Crosslingual generalization through multitask finetuning (2022)

    Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, TL Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning (2022). arXiv preprint arXiv:2211.01786, 1786

    2022 arXiv

  18. [26]

    Few-shot learning with multilingual generative language models

    Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. Few-shot learning with multilingual generative language models. In Proceedings of the 2022 conference on empirical methods in natu...

    2022

  19. [27]

    Paws-x: A cross-lingual adversarial dataset for paraphrase identification

    Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. Paws-x: A cross-lingual adversarial dataset for paraphrase identification. arXiv preprint arXiv:1908.11828, 2019

    1908 arXiv

  20. [28]

    The flores-101 evaluation benchmark for low-resource and multilingual machine translation.Transactionsof the Association for Computational Linguistics, 10:522–538, 2022

    Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. The flores-101 evaluation benchmark for low-resource and multilingual machine translation.Transactionsof the Associati...

    2022

  21. [29]

    G-dig: Towards gradient- based diverse and high-quality instruction data selection for machine translation.arXiv preprint arXiv:2405.12915, 2024

    Xingyuan Pan, Luyang Huang, Liyan Kang, Zhicheng Liu, Yu Lu, and Shanbo Cheng. G-dig: Towards gradient- based diverse and high-quality instruction data selection for machine translation.arXiv preprint arXiv:2405.12915, 2024

    2024 arXiv

  22. [30]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023

    2023 arXiv

  23. [31]

    Dupo: Enabling reliable llm self-verification via dual preference optimization, 2025

    Shuaijie She, Yu Bao, Yu Lu, Lu Xu, Tao Li, Wenhao Zhu, Shujian Huang, Shanbo Cheng, Lu Lu, and Yuxuan Wang. Dupo: Enabling reliable llm self-verification via dual preference optimization, 2025. URL https://arxiv.org/abs/2508.14460. 15

    2025 arXiv

  24. [32]

    Bleu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002

    2002

  25. [33]

    Bleurt: Learning robust metrics for text generation

    Thibault Sellam, Dipanjan Das, and Ankur Parikh. Bleurt: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, 2020

    2020

  26. [35]

    xcomet: Transparent machine translation evaluation through fine-grained error detection.arXiv preprint arXiv:2310.10482, 2023

    Nuno M Guerreiro, Ricardo Rei, Daan van Stigt, Luisa Coheur, Pierre Colombo, and André FT Martins. xcomet: Transparent machine translation evaluation through fine-grained error detection.arXiv preprint arXiv:2310.10482, 2023

    2023 arXiv

  27. [36]

    Comet: A neural framework for mt evaluation

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. Comet: A neural framework for mt evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, 2020

    2020

  28. [37]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  29. [38]

    Llamax: Scaling linguistic horizons of llm by enhancing translation capabilities beyond 100 languages.arXiv preprint arXiv:2407.05975, 2024

    Yinquan Lu, Wenhao Zhu, Lei Li, Yu Qiao, and Fei Yuan. Llamax: Scaling linguistic horizons of llm by enhancing translation capabilities beyond 100 languages.arXiv preprint arXiv:2407.05975, 2024

    2024 arXiv

  30. [39]

    Internlm2 technical report, 2024

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang,...

    2024

  31. [40]

    Aya expanse: Combining research breakthroughs for a new multilingual frontier, 2024

    John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztaj...

    2024 arXiv

  32. [41]

    Mistral large

    Mistral AI. Mistral large. https://mistral.ai/news/mistral-large-2407, 2024

    2024

  33. [42]

    Deepseek llm: Scaling open-source language models with longtermism.arXiv preprint arXiv:2401.02954, 2024

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism.arXiv preprint arXiv:2401.02954, 2024

    2024 arXiv

  34. [43]

    Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv:1803.05457v1, 2018

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge.arXiv:1803.05457v1, 2018

    2018 arXiv

  35. [44]

    Language tags matter for zero-shot neural machine translation

    Liwei Wu, Shanbo Cheng, Mingxuan Wang, and Lei Li. Language tags matter for zero-shot neural machine translation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 3001–3007, 2021

    2021

  36. [45]

    Large language models as agents in two-player games.arXiv preprint arXiv:2402.08078, 2024

    Yang Liu, Peng Sun, and Hang Li. Large language models as agents in two-player games.arXiv preprint arXiv:2402.08078, 2024

    2024 arXiv

  37. [46]

    A paradigm shift in machine translation: Boosting translation performance of large language models, 2023

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. A paradigm shift in machine translation: Boosting translation performance of large language models, 2023. 16

    2023

  38. [47]

    How do large language models handle multilingualism? arXiv preprint arXiv:2402.18815, 2024

    Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. How do large language models handle multilingualism? arXiv preprint arXiv:2402.18815, 2024

    2024 arXiv

  39. [48]

    Opencompass: A universal evaluation platform for foundation models

    OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models. https: //github.com/open-compass/opencompass, 2023

    2023

  40. [49]

    Sailer: structure-aware pre-trained language model for legal case retrieval

    Haitao Li, Qingyao Ai, Jia Chen, Qian Dong, Yueyue Wu, Yiqun Liu, Chong Chen, and Qi Tian. Sailer: structure-aware pre-trained language model for legal case retrieval. InProceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retri...

    2023

  41. [50]

    Chatlaw: Open-source legal large language model with integrated external knowledge bases.arXiv preprint arXiv:2306.16092, 2023

    Jiaxi Cui, Zongjian Li, Yang Yan, Bohua Chen, and Li Yuan. Chatlaw: Open-source legal large language model with integrated external knowledge bases.arXiv preprint arXiv:2306.16092, 2023

    2023 arXiv

  42. [51]

    A survey of large language models in medicine: Progress, application, and challenge.arXiv preprint arXiv:2311.05112, 2023

    Hongjian Zhou, Boyang Gu, Xinyu Zou, Yiru Li, Sam S Chen, Peilin Zhou, Junling Liu, Yining Hua, Chengfeng Mao, Xian Wu, et al. A survey of large language models in medicine: Progress, application, and challenge.arXiv preprint arXiv:2311.05112, 2023

    2023 arXiv

  43. [52]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, YK Li, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  44. [53]

    Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning

    Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning. arXiv preprint arXiv:2310.03731, 2023

    arXiv

  45. [54]

    Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950, 2023

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code llama: Open foundation models for code.arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  46. [55]

    Starcoder: may the source be with you!arXiv preprint arXiv:2305.06161, 2023

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. Starcoder: may the source be with you!arXiv preprint arXiv:2305.06161, 2023

    2023 arXiv

  47. [56]

    Wizardcoder: Empowering code large language models with evol-instruct

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  48. [57]

    Mammoth: Building math generalist models through hybrid instruction tuning.arXiv e-prints, pages arXiv–2309, 2023

    Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mammoth: Building math generalist models through hybrid instruction tuning.arXiv e-prints, pages arXiv–2309, 2023

    2023

  49. [58]

    Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation, 2024

    Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim. Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation, 2024

    2024

  50. [59]

    Is chatgpt a good translator? yes with gpt-4 as the engine.arXiv preprint arXiv:2301.08745, 2023

    Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi, and Zhaopeng Tu. Is chatgpt a good translator? yes with gpt-4 as the engine.arXiv preprint arXiv:2301.08745, 2023

    2023 arXiv

  51. [60]

    Jiahuan Li, Hao Zhou, Shujian Huang, Shanbo Cheng, and Jiajun Chen. Eliciting the translation ability of large language models via multilingual finetuning with translation instructions.Transactions of the Association for Computational Linguistics, 12:576–592, 2024

    2024

  52. [61]

    Trans-zero: Self-play incentivizes large language models for multilingual translation without parallel data.arXiv preprint arXiv:2504.14669, 2025

    Wei Zou, Sen Yang, Yu Bao, Shujian Huang, Jiajun Chen, and Shanbo Cheng. Trans-zero: Self-play incentivizes large language models for multilingual translation without parallel data.arXiv preprint arXiv:2504.14669, 2025

    2025 arXiv

  53. [62]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 568:127063, 2024

    2024

  54. [63]

    Neural machine translation of rare words with subword units

    Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, 2016

    2016

  55. [64]

    某宝”: Taobao, “绝绝子

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024. 17 Appendix A Supported Languages Languages Abbr. Languages Abbr. Arabic ar M...

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.