Pith. sign in

REVIEW 10 cited by

PEARL: Parallel Speculative Decoding with Adaptive Draft Length

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2408.11850 v3 pith:4R36P3X6 submitted 2024-08-13 cs.CL

Tianyu Liu , Yun Li , Qitan Lv , Kai Liu , Jianchen Zhu , Winston Hu , Xiao Sun This is my paper · ORCID
classification cs.CL
keywords draftdecodingmodelspeculativepearllengthphasetokens
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Speculative decoding (SD), where an extra draft model is employed to provide multiple draft tokens first, and then the original target model verifies these tokens in parallel, has shown great power for LLM inference acceleration. However, existing SD methods suffer from the mutual waiting problem, i.e., the target model gets stuck when the draft model is guessing tokens, and vice versa. This problem is directly incurred by the asynchronous execution of the draft model and the target model and is exacerbated due to the fixed draft length in speculative decoding. To address these challenges, we propose a conceptually simple, flexible, and general framework to boost speculative decoding, namely Parallel spEculative decoding with Adaptive dRaft Length (PEARL). Specifically, PEARL proposes pre-verify to verify the first draft token in advance during the drafting phase, and post-verify to generate more draft tokens during the verification phase. PEARL parallels the drafting phase and the verification phase via applying the two strategies, and achieves adaptive draft length for different scenarios, which effectively alleviates the mutual waiting problem. Experiments on various text generation benchmarks demonstrate the effectiveness of our PEARL, leading to a superior speed up performance up to 4.43$\times$ and 1.50$\times$, compared to auto-regressive decoding and vanilla speculative decoding, respectively. Our code is available at https://github.com/smart-lty/ParallelSpeculativeDecoding.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment

    cs.LG 2025-01 conditional novelty 7.0 of 10

    Judge decoding replaces strict token-alignment verification in speculative decoding with a learned correctness classifier on target embeddings, accepting about three times more tokens and enabling up to 9x speedup on ...

  2. AsymSpec: Efficient Cloud-Edge Speculative Decoding over Asymmetric Networks

    cs.DC 2026-08 conditional novelty 6.0 of 10

    AsymSpec speeds cloud-edge speculative decoding by uploading only token-probability pairs, moving correction traffic to the downlink, and overlapping independent requests instead of same-request runahead.

  3. Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting

    cs.CL 2026-07 accept novelty 6.0 of 10

    PTD accelerates autoregressive LLM decoding up to ~2× by guiding the target model to explore multiple coherent draft paths via a progressive, pruned tree in a single forward pass.

  4. Utility-Driven Speculative Decoding for Mixture-of-Experts

    cs.DC 2025-06 conditional novelty 6.0 of 10

    Cascade is a utility-driven speculation manager that makes speculative decoding practical for MoE LLMs by disabling it when expert-activation cost exceeds token gain and hill-climbing to the best speculation length.

  5. SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding

    cs.DC 2025-06 conditional novelty 6.0 of 10

    SwiftSpec uses asynchronous, disaggregated speculative decoding with parallel tree generation and fused kernels to speed up LLM decoding by 1.75x on average over baselines, reaching 348 tokens/s for Llama3-70B on 8 H800 GPUs.

  6. POSS: Position Specialist Generates Better Draft for Speculative Decoding

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Using position-specialized draft layers instead of one single draft model improves later-token acceptance in speculative decoding, yielding modest speedups on Llama-3-8B and Llama-2-13B.

  7. AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

    cs.LG 2026-07 conditional novelty 5.5 of 10

    A runtime-adaptive speculative decoder with an 11-rule hierarchy and multi-policy engine keeps wasted draft compute under ~32% and bounds latency variance on CPU-constrained GGUF inference.

  8. Advancing Decoding Strategies: Enhancements in Locally Typical Sampling for LLMs

    cs.CL 2025-06 reject novelty 5.0 of 10

    ASTS extends locally typical sampling with semantic scoring and dynamic thresholds, reporting improved perplexity, MAUVE, and diversity on story and summarization tasks.

  9. AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures

    cs.AI 2024-12 conditional novelty 5.0 of 10

    AdaEAGLE learns to predict the number of accepted draft tokens from the last hidden state and uses that prediction as the adaptive draft length in EAGLE-style speculative decoding.

  10. Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation

    cs.CL 2024-11 conditional novelty 5.0 of 10

    Speculative decoding can be accelerated by letting the draft model decide when to stop based on its prediction entropy, yielding up to 17-22% speedups in long-form generation.

Pith tools