Replacing the target reference solution in on-policy self-distillation with a solution to a different problem preserves most of the accuracy gains, showing target-specific privileged information is not necessary.
=== Unrelated physics Worked Example Solution End === Now solve the target mathematics problem independently
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation
Replacing the target reference solution in on-policy self-distillation with a solution to a different problem preserves most of the accuracy gains, showing target-specific privileged information is not necessary.