REVIEW 8 cited by
Multi-Candidate Speculative Decoding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Multi-Candidate Speculative Decoding
read the original abstract
Large language models have shown impressive capabilities across a variety of NLP tasks, yet their generating text autoregressively is time-consuming. One way to speed them up is speculative decoding, which generates candidate segments (a sequence of tokens) from a fast draft model that is then verified in parallel by the target model. However, the acceptance rate of candidate tokens receives limitations from several factors, such as the model, the dataset, and the decoding setup. This paper proposes sampling multiple candidates from a draft model and then organising them in batches for verification. We design algorithms for efficient multi-candidate verification while maintaining the distribution of the target model. Our approach shows significant improvements in acceptance rates on multiple datasets and models, consistently outperforming standard speculative decoding.
Forward citations
Cited by 8 Pith papers
-
When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding
Develops theory for acceptance in speculative decoding under greedy/relaxed/tree criteria, with exact KL certificates and margin bounds, evaluated on Qwen3 models.
-
UniVer: A Unified Perspective for Multi-step and Multi-draft Speculative Decoding
UniVer frames tree-based speculative decoding as conditional optimal transport, proving it is lossless with optimal acceptance rates and delivering 4.2-8.5% longer accepted sequences than standard rejection sampling.
-
Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing
Copy-as-Decode recasts LLM editing as grammar-constrained decoding over copy and generate primitives, delivering closed-form upper-bound speedups of 13x pooled on editing benchmarks via parallel prefill without any training.
-
SpecTr-GBV: Multi-Draft Block Verification Accelerating Speculative Decoding
SpecTr-GBV combines multi-draft and block verification via optimal transport to achieve the optimal expected acceptance length under i.i.d. draft generation.
-
Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes
Lossy speculative-decoding verification splits into truncation-based and collaborative methods; truncation-based methods underperform their matched baselines, and capping draft overshoot preserves quality.
-
Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting
PTD accelerates autoregressive LLM decoding up to ~2× by guiding the target model to explore multiple coherent draft paths via a progressive, pruned tree in a single forward pass.
-
Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing
Copy-as-Decode recasts LLM editing as grammar-constrained decoding over copy/generate primitives, using parallel prefill to replace autoregressive generation of verbatim spans.
-
Multi-Drafter Speculative Decoding with Alignment Feedback
MetaSD integrates multiple heterogeneous drafters into speculative decoding, dynamically selecting them via alignment feedback modeled as a multi-armed bandit to consistently outperform single-drafter baselines.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.