bicache enables shared-prefix KV caching in diffusion language models by dynamically selecting reusable shallow-layer depths based on prefix fraction, delivering 36.3-98.3% throughput gains with 0-1.8% accuracy difference.
Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
representative citing papers
SimSD adds a masking strategy to enable speculative decoding in diffusion LLMs, delivering up to 7.46x throughput gains on SDAR models while preserving generation quality.
DFlare replaces DFlash's shared fused representation with per-draft-layer attention to distinct target-layer combinations, enabling deeper drafts and 2.4M training samples for 5-11% higher speedups than DFlash on Qwen3 and GPT-OSS models.
A new benchmark for speculative decoding that maximizes semantic diversity and supports throughput evaluation across input lengths, exposing biases in synthetic benchmarks.
LogitSpec accelerates retrieval-based speculative decoding by speculating the next-next token from the last logit and retrieving relevant references for both next and next-next tokens, reporting up to 2.61x speedup and 3.28 mean accepted tokens.
AdaPLD adaptively mixes lexical and semantic retrieval with branched reuse to improve model-free speculative decoding and reports up to 3.10x speedup across benchmarks.
citing papers explorer
-
Enabling KV Caching of Shared Prefix for Diffusion Language Models
bicache enables shared-prefix KV caching in diffusion language models by dynamically selecting reusable shallow-layer depths based on prefix fraction, delivering 36.3-98.3% throughput gains with 0-1.8% accuracy difference.
-
SimSD: Simple Speculative Decoding in Diffusion Language Models
SimSD adds a masking strategy to enable speculative decoding in diffusion LLMs, delivering up to 7.46x throughput gains on SDAR models while preserving generation quality.
-
DFlare: Scaling Up Draft Capacity for Block Diffusion Speculative Decoding
DFlare replaces DFlash's shared fused representation with per-draft-layer attention to distinct target-layer combinations, enabling deeper drafts and 2.4M training samples for 5-11% higher speedups than DFlash on Qwen3 and GPT-OSS models.
-
SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding
A new benchmark for speculative decoding that maximizes semantic diversity and supports throughput evaluation across input lengths, exposing biases in synthetic benchmarks.
-
LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation
LogitSpec accelerates retrieval-based speculative decoding by speculating the next-next token from the last logit and retrieving relevant references for both next and next-next tokens, reporting up to 2.61x speedup and 3.28 mean accepted tokens.
-
AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decoding
AdaPLD adaptively mixes lexical and semantic retrieval with branched reuse to improve model-free speculative decoding and reports up to 3.10x speedup across benchmarks.