SSU mitigates catastrophic forgetting in low-resource LLM target-language adaptation by scoring and column-wise freezing source-critical parameters, reducing source degradation to ~3% versus ~20% for full fine-tuning while matching target performance.
AutoFreeze : Automatically freezing model blocks to accelerate fine-tuning
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
representative citing papers
MGUP augments momentum optimizers with selective larger steps on a fixed proportion of parameters per iteration, claiming convergence guarantees for MGUP-AdamW and superior empirical performance on pretraining and fine-tuning.
citing papers explorer
-
Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updates
SSU mitigates catastrophic forgetting in low-resource LLM target-language adaptation by scoring and column-wise freezing source-critical parameters, reducing source degradation to ~3% versus ~20% for full fine-tuning while matching target performance.
-
MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization
MGUP augments momentum optimizers with selective larger steps on a fixed proportion of parameters per iteration, claiming convergence guarantees for MGUP-AdamW and superior empirical performance on pretraining and fine-tuning.