Pith. sign in

REVIEW 4 major objections 4 minor 46 references

FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 13B-parameter Chinese-centric model using sparse Mixture-of-Experts translation surpasses GPT-3.5 and NLLB on low-resource pairs and translates four unseen languages without training data.

desk verdict A real engineering effort, but the zero-shot claim is contradicted by the paper's own tables and the evaluation is not reproducible; put it through a serious revision before sending to referees. read the letter →

arxiv 2505.14256 v1 pith:CMDTDYMS submitted 2025-05-20 cs.CL cs.AI

Shaolin Zhu , Tianyu Dong , Bo Li , Deyi Xiong This is my paper
classification cs.CLcs.AI
keywords Chinese-centricmachinetranslationMixture-of-ExpertssparseLLMlow-resourcecurriculumlearningzero-shotBLOOMzmultilingual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FuxiMT is a roughly 13-billion-parameter translation model built from a frozen BLOOMz backbone with sparse Mixture-of-Experts layers inserted every eight transformer blocks. The paper claims that this Chinese-first recipe—5 billion units of Chinese pre-training followed by instruction-tuned multilingual fine-tuning on 65 languages with curriculum learning and back-translation—makes a decoder-only model beat GPT-3.5, NLLB, and most open-weight LLMs on translation into Chinese, with the largest gains on low- and very-low-resource languages. It also claims zero-shot translation on language pairs never seen in training, such as Tigrinya, Tibetan, Turkmen, and Pijin. A sympathetic reader would care because if true, the result offers a parameter-efficient path to expanding LLM translation coverage beyond English-centric models without requiring parallel data for every language pair.

What carries the argument

The load-bearing mechanism is the sparse Mixture-of-Experts layer inserted into a frozen decoder-only LLM. Each MoE layer holds eight FFN experts and a gating network that computes a weighted combination of expert outputs per token, while only one expert is active every eight layers (sparse step 8). Because the original BLOOMz weights stay frozen, the pre-trained language knowledge is preserved and only the expert, router, and normalization parameters are updated during multilingual fine-tuning. A mixed initialization—some experts random, some copied from the corresponding BLOOMz FFN—plus curriculum learning and back-translation completes the machinery; the gating equation $P(y_t|X,\theta)=\sum_i g_i(X;\theta_g)P(y_t|X,\theta_{e_i})$ formalises how the router chooses among experts.

What would settle it

Run FuxiMT, NLLB, and GPT-3.5 on the same public xx-to-Chinese test set (for example, the Chinese directions of FLORES-200), using identical instruction templates, greedy decoding, and the paper's reported settings, then check whether FuxiMT keeps a low-resource BLEU advantage larger than a few points; if its margin over NLLB and GPT-3.5 collapses, the central claim fails.

Watch

Extended reading notes

Core claim

In the paper's own terms, the central discovery is that a sparsified LLM can be made Chinese-centric and multilingual at once: freeze BLOOMz-7B, splice in eight-expert MoE modules every eighth layer with mixed random-and-reused initialization, and train those modules first on Chinese text and then on xx-to-Chinese-heavy parallel data with a curriculum that raises the weight of low-resource languages as training progresses. The result, FuxiMT, reports an average BLEU of 26.15 across 64 xx-to-Chinese pairs versus 13.08 for the BLOOMz base and surpasses GPT-3.5 on low- and very-low-resource categories (20.64 and 20.66 versus 10.40 and 9.15). On four unseen pairs it reports BLEU values from 21.67 for Tigrinya to 29.10 for Pijin, above every baseline in the comparison. The paper interprets these numbers as evidence that Chinese pre-training plus sparse expert routing transfers cross-lingual knowledge to data-scarce and even unseen languages.

Load-bearing premise

The ranking assumes every model was evaluated on the same held-out test set with equally good prompts and decoding settings, but the paper does not name that test set or describe how baselines were prompted and decoded.

Editorial extensions

If this is right

  • If FuxiMT's numbers hold, low-resource languages do not need their own parallel corpora to get usable translation into Chinese; curriculum-trained expert routing transfers knowledge from related high-resource pairs.
  • Freezing the backbone while training sparse experts means a 13B-scale translation model can be specialized without catastrophic forgetting, so the same base could be reused for other language clusters by training new experts.
  • Zero-shot results on four unseen pairs imply that adding a new language to FuxiMT may require only a small amount of data or none at all, which matters for the thousands of languages currently unserved.
  • The Chinese-centric data allocation (over half of parallel data in xx-to-Chinese pairs) suggests that purposefully imbalanced training toward one anchor language can outperform broad but shallower multilingual coverage.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the recipe generalises, Chinese is not the only possible anchor: a similar frozen-backbone-plus-MoE pipeline trained with, say, Arabic or Hindi as the anchor could plausibly reproduce the low-resource gains for those language families, and that is a direct testable extension the paper does not run.
  • The zero-shot success hints that expert routing may organise by language or script rather than purely by surface token; inspecting router assignments on unseen pairs would show whether the experts are actually specialising as the paper assumes.
  • Because the evaluation protocol does not name the test set or decode settings used for baselines, a controlled re-run on a public benchmark such as FLORES-200 with identical prompting and decoding would determine how much of the reported margin is the model versus the setup.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents FuxiMT, a Chinese-centric multilingual machine translation model built on a sparsified BLOOMz backbone with inserted Mixture-of-Experts layers, trained in two stages: Chinese pre-training on 5B sentences and multilingual fine-tuning on parallel data covering 65 languages, with curriculum learning and back-translation. The authors report BLEU and chrF results on xx-to-zh pairs across four resource tiers and claim that FuxiMT significantly outperforms strong baselines (GPT-3.5, NLLB, LLaMA-3.1, Qwen-2.5, etc.), especially on low-resource languages, and that it exhibits remarkable zero-shot translation for unseen language pairs such as Tigrinya, Tibetan, Turkmen, and Pijin. Ablation studies are provided for MoE initialization and curriculum learning, along with a comparison to BigTranslate.

Significance. If the central claims were fully substantiated, a Chinese-centric sparse-MoE model that beats GPT-3.5 and NLLB on low-resource xx-to-zh translation and generalizes to unseen languages would be a useful contribution to multilingual MT. The paper proposes a reasonable architecture and a sensible two-stage training recipe, and it reports a large amount of per-language results. However, as written, the key claims are not verifiable: the zero-shot claim is internally contradicted by the paper's own tables, and the evaluation protocol is too underspecified to support the ranking against baselines. The significance of the contribution is therefore currently limited by the reliability of the evidence, not by the novelty of the idea.

major comments (4)
  1. [A.6, Tables 5-7] The zero-shot claim is internally contradicted. Appendix A.6 states that FuxiMT was evaluated on 'four unseen language pairs: Tigrinyan (ti), Tibetan (bo), Turkmen (tk), and Pijin (pis)', but Table 5 classifies ti, bo, tk, and pis as 'Very Low Resource' languages in FuxiMT's training-data-based resource taxonomy, Table 6 lists all four in the 65 supported languages, and Table 7 reports their BLEU scores alongside all other language pairs. If these languages were in the multilingual fine-tuning data, the zero-shot claim is false; if they were not, the tables misrepresent the training setup. Either way, the advertised 'remarkable zero-shot translation capabilities for unseen language pairs' is not demonstrated by the evidence in this manuscript.
  2. [Section 4.2, Tables 1 and 7] The main experimental ranking is not reproducible because the evaluation protocol is never specified. Section 4.2 reports BLEU averages by resource tier and Tables 7-8 report per-language BLEU and chrF, but the manuscript never names the test set (e.g., FLORES-200 or a custom split), never states how many segments per language were evaluated, and does not describe the prompting scheme or decoding parameters (beam size, max length, temperature) used for GPT-3.5, NLLB, or any other baseline. Without this information, the claimed 'consistent improvement' over baselines cannot be independently checked, and any mismatch in prompt templates or decoding settings could produce the observed gaps.
  3. [A.7, Table 4] The claim that 'FuxiMT consistently achieves higher BLEU scores' over BigTranslate is contradicted by the paper's own Table 4. For Tibetan (bo), BigTranslate scores 36.3 while FuxiMT scores 25.24. This is a direct counterexample to the 'consistently' claim in the text, and since bo is also one of the languages presented as a zero-shot language in A.6, this comparison deserves a careful explanation or a correction. The current presentation overstates the result.
  4. [Tables 7-8 and Figure 3] No measure of variability is reported. All BLEU and chrF numbers appear as point estimates, and Figure 3 provides bar charts without error bars or significance tests. Given that several low-resource language pairs show differences of only 1-3 BLEU points between FuxiMT and the next-best system (e.g., ti 21.67 vs. BLOOMz 18.87; tk 24.55 vs. NLLB 22.40), the reader cannot tell whether these gaps are meaningful. At minimum, the seed/test-split details and ideally bootstrap confidence intervals should be reported.
minor comments (4)
  1. [Abstract and Section 2.2] The data scale is stated inconsistently: the abstract says 'a large parallel dataset encompassing 65 languages', Section 2.2 says 'more than 2T' and 'over 1900 language pairs', while the introduction and Section 3.2 mention 'over 100 billion sentence pairs'. Please unify these numbers.
  2. [A.6 and Table 6] The language name 'Tigrinyan' in A.6 and Figure 3(a) differs from the standard English name 'Tigrinya' used in Table 6. Also, 'Tigranian' appears in Table 6's language list; the spelling should be consistent.
  3. [Tables 7 and 8] The 'ro' language row is duplicated at the bottom of both Tables 7 and 8, and 'ro' is labeled both 'Romanian' and 'Moldovan' in Table 6. This may indicate a data-processing duplication that should be resolved.
  4. [Section 4.1] The hyperparameter table (Appendix Table 3) lists 'Position Embed 4,096' and 'Vocab Size 250,752', but the model is described as BLOOMz-7B with sparse MoE; the relation between the listed hidden size and the BLOOMz-7B configuration should be stated explicitly, and the 'Position Embed' value seems abbreviated or incorrect.

Circularity Check

1 steps flagged · score 6.0 of 10

The paper's zero-shot claim is internally refuted: the four 'unseen' languages are in FuxiMT's own 65-language training list, so the flagship generalization result is an in-training evaluation relabeled as zero-shot; the rest of the experimental chain is not circular.

  1. fitted input called prediction [Appendix A.6 'Zero-Shot Translation'; Appendix Tables 5 and 6; Table 7]
    "FuxiMT's generalization ability was further evaluated through zero-shot translation on four unseen language pairs: Tigrinyan (ti), Tibetan (bo), Turkmen (tk), and Pijin (pis). ... demonstrating its remarkable capacity to transfer knowledge to unseen languages without explicit training data. Table 5 lists 'Very Low Resource ps, ha, lo, rw, mi, ug, prs, ti, bo, tk, pis' and Table 6 includes 'bo Tibetan', 'ti Tigranian', 'tk Turkmen', 'pis Pijin' among 'The list of 65 natural languages supported by FuxiMT'."

    The paper's own training corpus (Section 2.2) is built from OPUS parallel data covering 65 languages and over 1900 language pairs, and Tables 5-6 place ti, bo, tk, and pis inside that 65-language supported set, with xx->zh BLEU scores for them reported in Table 7. The A.6 evaluation therefore is not about unseen languages or pairs: the model was fine-tuned on these languages. Calling the result 'zero-shot' relabels in-training performance as generalization; the claimed capability is not derived from any transfer mechanism but from training examples the paper elsewhere says it did not use. The defining criterion 'unseen' and the stated membership in the 65-language training corpus cannot both hold for the same pair, so the zero-shot result reduces by construction to a seen-data score.

full rationale

The core training pipeline is not circular: the CLM loss (Eq. 2-3), weighted multilingual loss (Eq. 4), back-translation augmentation (Eq. 5), and MoE gating (Eq. 1) do not fit any parameter to the evaluation metric, and no reported BLEU/chrF is an algebraic identity of a fitted quantity. The comparison against BLOOMz, NLLB, GPT-3.5, and other LLMs is an independent empirical claim, although it is under-specified because no test set, prompting protocol, or decoding setup is named; that is a reproducibility and correctness risk, not a circularity. The one load-bearing circularity is the zero-shot claim in A.6: the four languages asserted to be 'unseen' are listed in the paper's own Table 5 and Table 6 as part of the 65-language training support set and have scores in Table 7. Thus the 'zero-shot prediction' is an in-training evaluation renamed as generalization, which reduces that headline claim by construction to the paper's own training input. The A.7 assertion that FuxiMT 'consistently achieves higher BLEU scores' is also contradicted by Table 4 for bo (BigTranslate 36.3 vs FuxiMT 25.24), but that is an internal inconsistency rather than a circular step. No other step qualifies: the self-citations (FuxiTranyu, MoE fusion) are descriptive antecedents, not load-bearing justifications of the reported results.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-chosen hyperparameters and on the assumption that the constructed corpora and unreported test set are valid. The most serious assumption, that the zero-shot languages were unseen, is contradicted by the paper's own language tables.

free parameters (5)
  • moe_expert_count = 8
    Number of experts per MoE layer, chosen by hand (Section 3.2).
  • sparse_step = 8
    MoE layer inserted once every 8 transformer layers, chosen by hand (Section 3.2).
  • curriculum_weight_schedule = not specified
    Equation 4 defines per-language weights wt, but the actual schedule is never given, so the curriculum is a hand-designed free component.
  • learning_rate = 5e-5
    Peak learning rate in Appendix Table 3.
  • data_filter_thresholds = length 50-250 chars; punctuation ratio >50% removed; sensitive frequency >0.5
    Hand-chosen cleaning thresholds in Sections 2.1 and 2.2.
assumptions (4)
  • domain assumption BLOOMz-7B provides a suitable frozen backbone for multilingual transfer.
    Section 3.1 assumes the frozen BLOOMz retains useful linguistic knowledge and that inserting MoE layers does not harm it.
  • domain assumption The OPUS-derived corpus, after six-stage cleaning, is of sufficient quality for training.
    Section 2.2 describes cleaning but provides no quality validation.
  • domain assumption BLEU and chrF scores on the unreported test set are valid measures of translation quality.
    Section 4 reports these metrics with no test set description.
  • ad hoc to paper Tigrinya, Tibetan, Turkmen, and Pijin were not seen during multilingual training.
    Appendix A.6 makes this assumption for zero-shot evaluation, yet Tables 5 and 6 list these languages as training languages, contradicting it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation." pith.science (2026). https://pith.science/paper/CMDTDYMS

@misc{pith2026250514256,
  author       = {Pith},
  title        = {Pith review of: FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CMDTDYMS}},
  note         = {Machine review of arXiv:2505.14256}
}
read the original abstract

In this paper, we present FuxiMT, a novel Chinese-centric multilingual machine translation model powered by a sparsified large language model (LLM). We adopt a two-stage strategy to train FuxiMT. We first pre-train the model on a massive Chinese corpus and then conduct multilingual fine-tuning on a large parallel dataset encompassing 65 languages. FuxiMT incorporates Mixture-of-Experts (MoEs) and employs a curriculum learning strategy for robust performance across various resource levels. Experimental results demonstrate that FuxiMT significantly outperforms strong baselines, including state-of-the-art LLMs and machine translation models, particularly under low-resource scenarios. Furthermore, FuxiMT exhibits remarkable zero-shot translation capabilities for unseen language pairs, indicating its potential to bridge communication gaps where parallel data are scarce or unavailable.

Figures

Figures reproduced from arXiv: 2505.14256 by the authors.

Figure 1
Figure 1. Language and data distribution in the pre [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Diagram of FuxiMT. FuxiMT is built upon BLOOMz-7B and fine-tuned on translation and general tasks. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results on zero-shot language pairs A.7 Comparison to Other MT model based on LLM BigTranslate is another MT model based on LLM which demonstrates strong performance on the FLORES-200 dataset. In this section, we compare FuxiMT with BigTranslate on FLORES-200. As shown in [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 10 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Viraat Aryabumi, John Dang, Dwarak Talupuru, Saurabh Dash, David Cairuz, Hangyu Lin, Bharat Venkitesh, Madeline Smith, Kelly Marchisio, Sebastian Ruder, Acyr Locatelli, Julia Kreutzer, Nick Frosst, Phil Blunsom, Marzieh Fadaee, Ahmet Üstün, and Sara Hooker. 2024. https://arxiv.org/abs/2405.15032 Aya 23: Open weight releases to further multilingual progres...

    arXiv 2024

  4. [4]

    Mikko Aulamo, Umut Sulubacak, Sami Virpioja, and J \"o rg Tiedemann. 2020. https://www.aclweb.org/anthology/2020.lrec-1.467 O pus T ools and parallel corpus diagnostics . In Proceedings of The 12th Language Resources and Evaluation Conference, pages 3782--3789. European Language Resources Association

    work page 2020

  5. [5]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  6. [6]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

    arXiv 2024

  7. [7]

    Jianghao Chen, Pu Jian, Tengxiao Xi, Dongyi Yi, Qianlong Du, Chenglin Ding, Guibo Zhu, Chengqing Zong, Jinqiao Wang, and Jiajun Zhang. 2023. https://arxiv.org/abs/2311.01149 Chinesewebtext: Large-scale high-quality chinese web text extracted with effective evaluation model . Preprint, arXiv:2311.01149

    arXiv 2023

  8. [8]

    Yong Cheng, Ankur Bapna, Orhan Firat, Yuan Cao, Pidong Wang, and Wolfgang Macherey. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.282 Multilingual mix: Example interpolation improves multilingual neural machine translation . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dubl...

    work page doi:10.18653/v1/2022.acl-long.282 2022

Show all 46 references
  1. [9]

    Marta R Costa-Juss \`a , James Cross, Onur C elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672

    2022 arXiv

  2. [10]

    Tri Dao. 2024. Flash A ttention-2: Faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (ICLR)

    2024

  3. [11]

    Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P

    Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen S. Meie...

    2022

  4. [12]

    Fraser, Philipp Koehn, and Hinrich Sch \" u tze

    Nadir Durrani, Helmut Schmid, Alexander M. Fraser, Philipp Koehn, and Hinrich Sch \" u tze. 2015. https://doi.org/10.1162/COLI\_A\_00218 The operation sequence model - combining n-gram-based and phrase-based statistical machine translation . Comput. Linguistics, 41(2):185--214

    2015 doi

  5. [13]

    Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, et al. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1--48

    2021

  6. [14]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1--39

    2022

  7. [15]

    Orhan Firat, Kyunghyun Cho, and Yoshua Bengio. 2016. https://doi.org/10.18653/V1/N16-1101 Multi-way, multilingual neural machine translation with a shared attention mechanism . In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computat...

    2016 doi

  8. [16]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  9. [17]

    Conghui He, Zhenjiang Jin, Chao Xu, Jiantao Qiu, Bin Wang, Wei Li, Hang Yan, Jiaqi Wang, and Dahua Lin. 2023. https://arxiv.org/abs/2308.10755 Wanjuan: A comprehensive multimodal dataset for advancing english and chinese large models . Preprint, arXiv:2308.10755

    2023 arXiv

  10. [18]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. http://proceedings.mlr.press/v97/houlsby19a.html Parameter-efficient transfer learning for NLP . In Proceedings of the 36th Int...

    2019

  11. [19]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  12. [20]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas ...

    2023 arXiv

  13. [21]

    Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B

    Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda B. Vi \' e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. https://doi.org/10.1162/TACL\_A\_00065 Google's multilingual neural machine trans...

    2017 doi

  14. [22]

    Alexander Jones, Isaac Caswell, Ishank Saxena, and Orhan Firat. 2023. https://doi.org/10.48550/ARXIV.2303.15265 Bilex rx: Lexical data augmentation for massively multilingual machine translation . CoRR, abs/2303.15265

    2023 doi arXiv

  15. [23]

    Taku Kudo and John Richardson. 2018. https://doi.org/10.18653/v1/D18-2012 S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processin...

    2018 doi

  16. [24]

    Zehui Lin, Liwei Wu, Mingxuan Wang, and Lei Li. 2021. https://doi.org/10.18653/V1/2021.ACL-LONG.25 Learning language specific sub-network for multilingual machine translation . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1...

    2021 doi

  17. [25]

    Pierre Lison and J \"o rg Tiedemann. 2016. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles

    2016

  18. [26]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  19. [27]

    Yinquan Lu, Wenhao Zhu, Lei Li, Yu Qiao, and Fei Yuan. 2024. Llamax: Scaling linguistic horizons of llm by enhancing translation capabilities beyond 100 languages. arXiv preprint arXiv:2407.05975

    2024 arXiv

  20. [28]

    Lars Nygaard and J \"o rg Tiedemann. 2003. Opus—an open source parallel corpus. In Proceedings of the 13th Nordic Conference on Computational Linguistics

    2003

  21. [29]

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1--16. IEEE

    2020

  22. [30]

    Xiaozhe Ren, Pingyi Zhou, Xinfan Meng, Xinjing Huang, Yadao Wang, Weichao Wang, Pengfei Li, Xiaoda Zhang, Alexander Podolskiy, Grigory Arshinov, Andrey Bout, Irina Piontkovskaya, Jiansheng Wei, Xin Jiang, Teng Su, Qun Liu, and Jun Yao. 2023. https://doi.org/10.48550/ARXIV.2303...

    2023 doi arXiv

  23. [31]

    Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, and Armand Joulin. 2019. Ccmatrix: Mining billions of high-quality parallel sentences on the web. arXiv preprint arXiv:1911.04944

    2019 arXiv

  24. [32]

    Le, Geoffrey E

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. https://openreview.net/forum?id=B1ckMDqlg Outrageously large neural networks: The sparsely-gated mixture-of-experts layer . In 5th International Conference on L...

    2017

  25. [33]

    Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, and Deyi Xiong. 2024. https://doi.org/10.48550/ARXIV.2408.06273 Fuxitranyu: A multilingual large language model trained with balanced da...

    2024 doi arXiv

  26. [34]

    Gemma Team. 2024 a . https://doi.org/10.34740/KAGGLE/M/3301 Gemma

    2024 doi

  27. [35]

    Qwen Team. 2024 b . https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

    2024

  28. [36]

    BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili \'c , Daniel Hesslow, Roman Castagn \'e , Alexandra Sasha Luccioni, Fran c ois Yvon, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:...

    2022 arXiv

  29. [37]

    Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2024. https://openreview.net/forum?id=farT6XXntP A paradigm shift in machine translation: Boosting translation performance of large language models . In The Twelfth International Conference on Learning Representat...

    2024

  30. [38]

    Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. 2023. Bigtranslate: Augmenting large language models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098

    2023 arXiv

  31. [39]

    Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, and Yang Feng. 2023. https://doi.org/10.48550/ARXIV.2306.10968 Bayling: Bridging cross-lingual alignment and instruction following through inte...

    2023 doi arXiv

  32. [40]

    Jiawei Zheng, Hanghai Hong, Xiaoli Wang, Jingsong Su, Yonggui Liang, and Shikai Wu. 2024. https://doi.org/10.48550/ARXIV.2402.15061 Fine-tuning large language models for domain-specific machine translation . CoRR, abs/2402.15061

    2024 doi arXiv

  33. [41]

    Fan Zhou and Chengtai Cao. 2021. https://doi.org/10.1609/AAAI.V35I5.16602 Overcoming catastrophic forgetting in graph neural networks with experience replay . In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applicati...

    2021 doi

  34. [42]

    Shaolin Zhu, Menglong Cui, and Deyi Xiong. 2024 a . https://aclanthology.org/2024.lrec-main.1444 Towards robust in-context learning for machine translation with large language models . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Lang...

    2024

  35. [43]

    Shaolin Zhu, Leiyu Pan, Dong Jian, and Deyi Xiong. 2025. Overcoming language barriers via machine translation with sparse mixture-of-experts fusion of large language models. Information Processing & Management, 62(3):104078

    2025

  36. [44]

    ShaoLin Zhu, Leiyu Pan, and Deyi Xiong. 2024 b . https://doi.org/10.1016/J.IPM.2024.103825 FEDS-ICL: enhancing translation ability and efficiency of large language model by optimizing demonstration selection . Inf. Process. Manag., 61(5):103825

    2024

  37. [45]

    Shaolin Zhu, Shaoyang Xu, Haoran Sun, Leiyu Pan, Menglong Cui, Jiangcun Du, Renren Jin, Ant \'o nio Branco, Deyi Xiong, et al. 2024 c . Multilingual large language models: A systematic survey. arXiv preprint arXiv:2411.11072

    2024 arXiv

  38. [46]

    Yaoming Zhu, Jiangtao Feng, Chengqi Zhao, Mingxuan Wang, and Lei Li. 2021. https://doi.org/10.18653/V1/2021.FINDINGS-EMNLP.240 Counter-interference adapter for multilingual machine translation . In Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual ...

    2021 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.