Pith. sign in

REVIEW 10 cited by

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.05589 v1 pith:3C3RFU6H submitted 2024-10-08 cs.CL cs.LG

Zilin Xiao , Hongming Zhang , Tao Ge , Siru Ouyang , Vicente Ordonez , Dong Yu This is my paper
classification cs.CLcs.LG
keywords speculativemodeldecodingdrafterparallelparallelspecefficienttokens
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Speculative decoding has proven to be an efficient solution to large language model (LLM) inference, where the small drafter predicts future tokens at a low cost, and the target model is leveraged to verify them in parallel. However, most existing works still draft tokens auto-regressively to maintain sequential dependency in language modeling, which we consider a huge computational burden in speculative decoding. We present ParallelSpec, an alternative to auto-regressive drafting strategies in state-of-the-art speculative decoding approaches. In contrast to auto-regressive drafting in the speculative stage, we train a parallel drafter to serve as an efficient speculative model. ParallelSpec learns to efficiently predict multiple future tokens in parallel using a single model, and it can be integrated into any speculative decoding framework that requires aligning the output distributions of the drafter and the target model with minimal training cost. Experimental results show that ParallelSpec accelerates baseline methods in latency up to 62% on text generation benchmarks from different domains, and it achieves 2.84X overall speedup on the Llama-2-13B model using third-party evaluation criteria.

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    SpecBlock achieves 8-13% higher mean speedup than EAGLE-3 at 44-52% drafting cost via block-iterative drafting with hidden-state inheritance, dynamic rank-head branching, valid-prefix masking, and optional cost-aware ...

  2. FASER: Fine-Grained Phase Management for Speculative Decoding in Dynamic LLM Serving

    cs.DC 2026-04 unverdicted novelty 7.0 of 10

    FASER delivers up to 53% higher throughput and 1.92x lower latency in dynamic LLM serving by adjusting speculative lengths per request, early pruning of rejects, and overlapping draft/verification phases via frontiers.

  3. PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    PARD-2 uses Confidence-Adaptive Token optimization to align draft model training with acceptance length in speculative decoding, enabling dual-mode operation and up to 6.94x lossless speedup on Llama3.1-8B.

  4. SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    SpecBlock achieves 8-19% higher speedup than EAGLE-3 in LLM speculative decoding by using repeated block expansions with hidden-state inheritance, a dynamic rank head, and a valid-prefix training mask.

  5. MineDraft: A Framework for Batch Parallel Speculative Decoding

    cs.CL 2026-02 conditional novelty 6.0 of 10

    MineDraft speeds up speculative decoding by keeping two request batches — drafting one batch while verifying the other — gaining up to ~40% per-GPU throughput at moderate draft lengths.

  6. SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

    cs.DC 2026-02 conditional novelty 6.0 of 10

    A new benchmark for speculative decoding that maximizes semantic diversity and supports throughput evaluation across input lengths, exposing biases in synthetic benchmarks.

  7. SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

    cs.DC 2026-02 unverdicted novelty 6.0 of 10

    SPEED-Bench is a new standardized benchmark for speculative decoding that supplies semantically diverse qualitative data and throughput-oriented splits across concurrency levels, integrated with vLLM and TensorRT-LLM.

  8. Training-Free Loosely Speculative Decoding: Accepting Semantically Correct Drafts Beyond Exact Match

    cs.CL 2025-11 unverdicted novelty 6.0 of 10

    FLy is a training-free method that speeds up LLM generation by accepting semantically correct but non-exact draft tokens via an entropy gate and deferred verification window.

  9. Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A gated LoRA and a small sampler let an autoregressive LLM draft multiple future tokens per step, and self-speculative verification converts those drafts into up to roughly 5x fewer generation steps.

  10. SpecASR: Accelerating LLM-based Automatic Speech Recognition via Speculative Decoding

    eess.AS 2025-07 reject novelty 4.0 of 10

    SpecASR accelerates LLM-based ASR by 3.04x-3.79x over autoregressive decoding using adaptive draft lengths, draft token recycling, and sparse token trees, but the speedups are simulated from Whisper proxy models rathe...

Pith tools