On-the-fly distillation of Transformer layers to dual-state linear attention produces about 2.3x faster simulated LLM serving than Llama2-7B with roughly comparable benchmark accuracy.
https://huggingface.co/blog/bamba
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
On-the-fly distillation of Transformer layers to dual-state linear attention produces about 2.3x faster simulated LLM serving than Llama2-7B with roughly comparable benchmark accuracy.