Graft combines pruning and retrieval in a sequential mechanism to build hybrid draft trees for speculative decoding, delivering up to 5.41× speedup and 21.8% better average speedup than EAGLE-3 on large models.
Triforce: Lossless acceleration of long sequence generation with hierarchical speculative decoding.arXiv preprint arXiv:2404.11912
7 Pith papers cite this work. Polarity classification is still indexing.
7
Pith papers citing it
representative citing papers
Dustin reports 27.85x self-attention and 9.17x end-to-end speedups at 32k length on Qwen2.5-72B using draft-augmented sparse verification with negligible accuracy loss on PG-19 and LongBench.
DREAM-S combines neural architecture search, target-aware supernet training, and attention-entropy-guided distillation to accelerate speculative decoding in VLMs, reporting up to 3.85x speedup over standard methods.
Ada-KV is the first head-wise adaptive KV cache budget allocator for LLMs, using a theoretical loss upper bound to allocate eviction differently per attention head and yielding higher quality than uniform methods on long-context benchmarks.
K-VEC is a coverage-aware KV-cache eviction strategy using cross-head and cross-layer modules that improves performance by up to 10.35 points over prior methods on LongBench subsets at fixed memory budget.
BudgetDraft applies multi-view sparse training with an acceptance-aware full-cache loss branch to produce one budget-robust drafter that recovers acceptance rates across sparsity levels in speculative decoding for 4K-16K contexts.
DREAM-R introduces RL-based draft alignment, ratio-based verification, and parallel execution to accelerate speculative reasoning in multimodal models while preserving accuracy.
citing papers explorer
-
Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding
Graft combines pruning and retrieval in a sequential mechanism to build hybrid draft trees for speculative decoding, delivering up to 5.41× speedup and 21.8% better average speedup than EAGLE-3 on large models.
-
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
Dustin reports 27.85x self-attention and 9.17x end-to-end speedups at 32k length on Qwen2.5-72B using draft-augmented sparse verification with negligible accuracy loss on PG-19 and LongBench.
-
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
DREAM-S combines neural architecture search, target-aware supernet training, and attention-entropy-guided distillation to accelerate speculative decoding in VLMs, reporting up to 3.85x speedup over standard methods.
-
Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
Ada-KV is the first head-wise adaptive KV cache budget allocator for LLMs, using a theoretical loss upper bound to allocate eviction differently per attention head and yielding higher quality than uniform methods on long-context benchmarks.
-
Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM
K-VEC is a coverage-aware KV-cache eviction strategy using cross-head and cross-layer modules that improves performance by up to 10.35 points over prior methods on LongBench subsets at fixed memory budget.
-
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
BudgetDraft applies multi-view sparse training with an acceptance-aware full-cache loss branch to produce one budget-robust drafter that recovers acceptance rates across sparsity levels in speculative decoding for 4K-16K contexts.
-
DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Execution
DREAM-R introduces RL-based draft alignment, ratio-based verification, and parallel execution to accelerate speculative reasoning in multimodal models while preserving accuracy.