Feed-forward LoRA with more parameters adapts Llama-3.2-1B to Icelandic better than attention LoRA, bottleneck adapters, IA3, or prefix tuning, and final-layer placement mitigates context-length degradation.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Train More Parameters But Mind Their Placement: Insights into Language Adaptation with PEFT
Feed-forward LoRA with more parameters adapts Llama-3.2-1B to Icelandic better than attention LoRA, bottleneck adapters, IA3, or prefix tuning, and final-layer placement mitigates context-length degradation.