Pith. sign in

REVIEW 5 cited by

Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.15011 v2 pith:AA2V5E6J submitted 2023-05-24 cs.CL

Haonan Li , Fajri Koto , Minghao Wu , Alham Fikri Aji , Timothy Baldwin This is my paper
classification cs.CL
keywords modelsbactrian-xmultilinguallanguagelanguagesacrossadaptationadapters
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Instruction tuning has shown great promise in improving the performance of large language models. However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages. To bridge this gap, we present Bactrian-X, a comprehensive multilingual parallel dataset of 3.4 million instruction-response pairs across 52 languages. Leveraging this dataset, we train a set of adapters using low-rank adaptation (LoRA), which are lightweight components that seamlessly integrate with large language models. These adapters have a substantially lower parameter count than the base model, making them easily replaceable and usable as plug-ins for different languages or language groups. Extensive experiments in various multilingual evaluation settings demonstrate that models derived from LoRA-based training over Bactrian-X outperform both the vanilla models and existing instruction-tuned models. The code and models are publicly available at https://github.com/mbzuai-nlp/bactrian-x

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems

    cs.CL 2026-07 conditional novelty 7.0 of 10

    A 24,795-pair multilingual instruction dataset for Indian Knowledge Systems pedagogy brings a 7B fine-tune within 0.15 median-judge points of a strong general-purpose reference model.

  2. LuxInstruct: A Cross-Lingual Instruction Tuning Dataset For Luxembourgish

    cs.CL 2025-10 conditional novelty 6.0 of 10

    LuxInstruct is the first native-output instruction-tuning dataset for Luxembourgish (537k samples, instructions in English/French/German); its evidence that cross-lingual tuning beats monolingual tuning is directional...

  3. MELABenchv1: Benchmarking Large Language Models against Smaller Fine-Tuned Models for Low-Resource Maltese NLP

    cs.CL 2025-06 conditional novelty 6.0 of 10

    A new 11-task Maltese benchmark shows that 55 large language models lag behind small fine-tuned models, with prior Maltese exposure the strongest predictor.

  4. CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning

    cs.CL 2025-06 conditional novelty 6.0 of 10

    CC-Tuning fuses English feed-forward activations into non-English inputs during multilingual supervised fine-tuning, using a trainable Decision Maker and a least-squares Transform Matrix to simulate the connection at ...

  5. Text2Cypher Across Languages: Evaluating and Finetuning LLMs

    cs.CL 2025-06 conditional novelty 5.0 of 10

    A new multilingual Text2Cypher benchmark shows LLMs rank English highest, Spanish next, and Turkish lowest, and multilingual finetuning narrows the language gap more than English-only finetuning.

Pith tools