EAGLE-2 loses most of its speedup on 4-bit quantized LLMs, and a hierarchical draft-then-sequence scheme restores 1.31x speedup over EAGLE-2 on W4A16 Llama-3-70B.
Lee, Deming Chen, and Tri Dao
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design
EAGLE-2 loses most of its speedup on 4-bit quantized LLMs, and a hierarchical draft-then-sequence scheme restores 1.31x speedup over EAGLE-2 on W4A16 Llama-3-70B.