TAID improves knowledge distillation for language models by gradually shifting the student's training target from its own distribution to the teacher's distribution over time.
Model compression
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models
TAID improves knowledge distillation for language models by gradually shifting the student's training target from its own distribution to the teacher's distribution over time.