REVIEW 7 cited by
BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) demonstrate promising translation performance among various natural languages. However, many LLMs especially the open-sourced ones, such as BLOOM and LLaMA, are English-dominant and support only dozens of natural languages, making the potential of LLMs on language translation less explored. In this work, we present BigTranslate which adapts LLaMA that covers only 20 languages and enhances it with multilingual translation capability on more than 100 languages. BigTranslate is built upon LLaMA-13B and it is optimized in three steps. First, we continue training LLaMA with massive Chinese monolingual data. Second, we continue training the model with a large-scale parallel dataset that covers 102 natural languages. Third, we instruct-tune the foundation model with multilingual translation instructions, leading to our BigTranslate model. The preliminary experiments on multilingual translation show that BigTranslate performs comparably with ChatGPT and Google Translate in many languages and even outperforms ChatGPT in 8 language pairs. We release the BigTranslate model and hope it can advance the research progress.
Forward citations
Cited by 7 Pith papers
-
Translate With Care: Addressing Gender Bias, Neutrality, and Reasoning in Large Language Model Translations
A new genderless-to-English benchmark shows that fine-tuning mBART-50 on carefully curated examples cuts gender stereotyping and pronoun-reasoning errors, beating larger proprietary systems on that benchmark.
-
Preperiodic points, finiteness, and structures of semigroups of algebraic morphisms
The paper proves finiteness and structural results for preperiodic points of algebraic morphisms, including Burnside-type and Northcott-type theorems.
-
Improving MLLM's Document Image Machine Translation via Synchronously Self-reviewing Its OCR Proficiency
A fine-tuning paradigm that prompts MLLMs to self-generate OCR text before translating document images improves DIMT quality and reduces catastrophic forgetting of OCR.
-
Group then Scale: Dynamic Mixture-of-Experts Multilingual Language Model
A multilingual LLM training method that groups similar languages, converts high-deviation layers into mixture-of-experts layers, and assigns one expert per language group improves perplexity across 18 to 128 languages.
-
TokAlign: Efficient Vocabulary Adaptation via Token Alignment
TokAlign aligns source and target BPE token vocabularies using GloVe co-occurrence embeddings and re-initializes LLM embeddings, recovering within 5k steps and enabling token-level distillation.
-
Text2Cypher Across Languages: Evaluating and Finetuning LLMs
A new multilingual Text2Cypher benchmark shows LLMs rank English highest, Spanish next, and Turkish lowest, and multilingual finetuning narrows the language gap more than English-only finetuning.
-
FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation
FuxiMT combines a frozen BLOOMz model with sparse mixture-of-experts layers, Chinese-first pretraining, and curriculum learning to translate into Chinese from 65 languages, with claimed low-resource gains that the pap...
Discussion (0). Sign in to comment.