Adapting BigScience Multilingual Model to Unseen Languages

Vassilina Nikoulina; Zheng-Xin Yong

Adapting BigScience Multilingual Model to Unseen Languages

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2204.04873 v1 pith:DBPXOCJI submitted 2022-04-11 cs.CL cs.AIcs.LG

Zheng-Xin Yong , Vassilina Nikoulina This is my paper · ORCID

classification cs.CLcs.AIcs.LG

keywords languagesmodelbigsciencelanguagemultilingualadaptabilityadaptingadding

verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments

read the original abstract

We benchmark different strategies of adding new languages (German and Korean) into the BigScience's pretrained multilingual language model with 1.3 billion parameters that currently supports 13 languages. We investigate the factors that affect the language adaptability of the model and the trade-offs between computational costs and expected performance.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

ChocoLlama: Lessons Learned From Teaching Llamas Dutch
cs.CL 2024-12 conditional novelty 5.0 of 10

Continued pretraining with LoRA and a Dutch-specific tokenizer improves Llama-2's Dutch, but gives limited gains for already-multilingual Llama-3.

Pith tools