A geometric self-distillation objective using Hellinger loss and Fisher-Rao proximal regularization prevents predictive drift in LLM post-training, improving out-of-distribution reasoning by 5.7-8.6 points.
Trust-Region Behavior Blending for On-Policy Distillation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
On-policy distillation (OPD) trains a student on prefixes sampled from its own policy while matching a stronger teacher. This addresses the prefix mismatch of offline distillation, but early student rollouts can still be poor, placing teacher supervision on weak or low-quality prefixes. We propose Trust-Region behavior Blending (TRB), a warmup method that replaces the early rollout policy with the closest-to-teacher behavior policy inside a student-centered KL trust region, while keeping the per-prefix reverse-KL OPD loss unchanged. The KL budget is annealed to zero, so training returns to pure student rollouts after warmup. Across two math-reasoning distillation settings, TRB attains the strongest average among the compared methods.
fields
cs.LG 1years
2026 1verdicts
ACCEPT 1representative citing papers
citing papers explorer
-
Geometric Self-Distillation for Reasoning Generalization
A geometric self-distillation objective using Hellinger loss and Fisher-Rao proximal regularization prevents predictive drift in LLM post-training, improving out-of-distribution reasoning by 5.7-8.6 points.