MFT, a corrective self-distillation objective, reports 2 to 10 times better specialization-to-degeneralization ratios than standard finetuning across many models and three specialized domains.
Distill and replay for continual language learning
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
MFT, a corrective self-distillation objective, reports 2 to 10 times better specialization-to-degeneralization ratios than standard finetuning across many models and three specialized domains.