Converting Qwen3-0.6B to KDA linear attention keeps perplexity close to the teacher while collapsing C-Eval accuracy to near chance; a four-permutation diagnostic shows the model is stuck on option labels, and a short format-targeted stage mostly repairs it.
Title resolution pending
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Stuck on "A": Diagnosing and Repairing Interface Injury in Attention-to-KDA Linearization of a 0.6B Language Model
Converting Qwen3-0.6B to KDA linear attention keeps perplexity close to the teacher while collapsing C-Eval accuracy to near chance; a four-permutation diagnostic shows the model is stuck on option labels, and a short format-targeted stage mostly repairs it.