Hierarchy Drafting accelerates LLM inference without retraining by retrieving draft tokens from three databases in order of temporal locality, beating prior database drafting methods on Spec-Bench.
Lee, Deming Chen, and Tri Dao
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding
Hierarchy Drafting accelerates LLM inference without retraining by retrieving draft tokens from three databases in order of temporal locality, beating prior database drafting methods on Spec-Bench.