A preference-based knowledge distillation framework uses a confidence-set reward model in a min-max imitation game, with offline, online, and white-box variants, and outperforms prior KD baselines on LLM benchmarks.
Knowledge distillation of black-box large language models, 2024
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
baseline 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
Online Knowledge Distillation with Reward Guidance
A preference-based knowledge distillation framework uses a confidence-set reward model in a min-max imitation game, with offline, online, and white-box variants, and outperforms prior KD baselines on LLM benchmarks.