A learned transformation matrix minimizes CMI in teacher logits to degrade distillation performance while preserving task accuracy.
Distillm: Towards streamlined distillation for large language models
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
Towards Distillation-Resistant Large Language Models: An Information-Theoretic Perspective
A learned transformation matrix minimizes CMI in teacher logits to degrade distillation performance while preserving task accuracy.