QLoRA on Mistral-7B and Phi-2 yields perplexity 3.79-3.81 on Bashkir, close to full fine-tuning's 3.34 but with over 40x fewer trainable parameters, though some base models degrade sharply and best-perplexity models often switch to English in generation.
Kamala Baghirova, Lutfi Kerem Senel, Benedict Ebing, Hinrich Schuetze, and Goran Glavaš
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Adapting Large Language Models to a Low-Resource Agglutinative Language: A Comparative Study of LoRA and QLoRA for Bashkir
QLoRA on Mistral-7B and Phi-2 yields perplexity 3.79-3.81 on Bashkir, close to full fine-tuning's 3.34 but with over 40x fewer trainable parameters, though some base models degrade sharply and best-perplexity models often switch to English in generation.