Training the teacher a small number of steps ahead of the student and freezing it during distillation improves student generalization by up to 3.4% on image benchmarks.
Follow your path: a progressive method for knowledge distilla- tion
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Right Time to Learn:Promoting Generalization via Bio-inspired Spacing Effect in Knowledge Distillation
Training the teacher a small number of steps ahead of the student and freezing it during distillation improves student generalization by up to 3.4% on image benchmarks.