A Synaptic-Intelligence-style importance penalty, scaled per layer, reduces catastrophic forgetting during LoRA fine-tuning of GPT-J and LLaMA-3 on three domain tasks.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
How to Alleviate Catastrophic Forgetting in LLMs Finetuning? Hierarchical Layer-Wise and Element-Wise Regularization
A Synaptic-Intelligence-style importance penalty, scaled per layer, reduces catastrophic forgetting during LoRA fine-tuning of GPT-J and LLaMA-3 on three domain tasks.