Weaver builds conditional proposal trees from a factorized drafter’s top-K marginals and a rollback-free GDN tree-verify kernel, yielding 4.37× speedup over AR decoding.
Specinfer: Accelerating large language model serving with tree-based speculative inference and verification
13 Pith papers cite this work, alongside 75 external citations. Polarity classification is still indexing.
13
Pith papers citing it
75
external citations · external index
citation-role summary
background 1
citation-polarity summary
roles
background 1polarities
background 1representative citing papers
Develops theory for acceptance in speculative decoding under greedy/relaxed/tree criteria, with exact KL certificates and margin bounds, evaluated on Qwen3 models.
TAPS converts diffusion marginal probabilities into path-conditioned acceptance estimates to select prefix-closed subtrees under a fixed verification budget, achieving up to 7.9x end-to-end speedup over autoregressive decoding.
TIDAL recovers temporal phase signals from LLM-derived semantics of provisioning metadata to enable complementary CVD placement, reducing overload frequency by 79.1% on production traces.
RKSC delivers 3.008x mean speedup over baseline and 1.66x over vLLM prefix caching for multi-branch LLM reasoning via similarity-based KV sharing and confidence-gated early exit, with 0.37% error rate.
Draft-OPD applies on-policy distillation via target-assisted generation and error replay to train speculative draft models, yielding over 5x lossless acceleration and gains over EAGLE-3 and DFlash.
SPECTRE achieves up to 2.28x speedup for large-model LLM serving by running speculative draft generation and target verification in parallel using idle tail-model services.
KV cache reuse improves long-range draft acceptance in speculative decoding but delivers only marginal end-to-end speedups due to drafter limitations.
AReaL decouples generation and training in LLM reinforcement learning to achieve up to 2.77x speedup with matched or better performance on math and code benchmarks.
Relaxed speculative decoding methods that trade exactness for speed only produce useful capability-speed trade-offs when the drafter is a strong standalone language model; lightweight MTP drafters are largely unsuited to relaxation.
LogitSpec accelerates retrieval-based speculative decoding by speculating the next-next token from the last logit and retrieving relevant references for both next and next-next tokens, reporting up to 2.61x speedup and 3.28 mean accepted tokens.
PipeSD is a cloud-edge collaborative inference framework that overlaps token generation and communication via dynamic programming pipeline scheduling and uses Bayesian-optimized dual-threshold NAV triggering, delivering 1.16x-2.16x speedup and 14.3%-25.3% energy reduction over baselines.
citing papers explorer
-
Trees from Marginals: Autoregressive drafting with factorized priors
Weaver builds conditional proposal trees from a factorized drafter’s top-K marginals and a rollback-free GDN tree-verify kernel, yielding 4.37× speedup over AR decoding.
-
When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding
Develops theory for acceptance in speculative decoding under greedy/relaxed/tree criteria, with exact KL certificates and margin bounds, evaluated on Qwen3 models.
-
TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding
TAPS converts diffusion marginal probabilities into path-conditioned acceptance estimates to select prefix-closed subtrees under a fixed verification budget, achieving up to 7.9x end-to-end speedup over autoregressive decoding.
-
TIDAL: Recovering Temporal Phase for Cloud Block Storage Placement from LLM-Derived Semantics
TIDAL recovers temporal phase signals from LLM-derived semantics of provisioning metadata to enable complementary CVD placement, reducing overload frequency by 79.1% on production traces.
-
RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference
RKSC delivers 3.008x mean speedup over baseline and 1.66x over vLLM prefix caching for multi-branch LLM reasoning via similarity-based KV sharing and confidence-gated early exit, with 0.37% error rate.
-
Draft-OPD: On-Policy Distillation for Speculative Draft Models
Draft-OPD applies on-policy distillation via target-assisted generation and error replay to train speculative draft models, yielding over 5x lossless acceleration and gains over EAGLE-3 and DFlash.
-
SPECTRE: Hybrid Ordinary-Parallel Speculative Serving for Resource-Efficient LLM Inference
SPECTRE achieves up to 2.28x speedup for large-model LLM serving by running speculative draft generation and target verification in parallel using idle tail-model services.
-
When Hidden States Drift: Can KV Caches Rescue Long-Range Speculative Decoding?
KV cache reuse improves long-range draft acceptance in speculative decoding but delivers only marginal end-to-end speedups due to drafter limitations.
-
AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning
AReaL decouples generation and training in LLM reinforcement learning to achieve up to 2.77x speedup with matched or better performance on math and code benchmarks.
-
A Practical Investigation of Training-free Relaxed Speculative Decoding
Relaxed speculative decoding methods that trade exactness for speed only produce useful capability-speed trade-offs when the drafter is a strong standalone language model; lightweight MTP drafters are largely unsuited to relaxation.
-
LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation
LogitSpec accelerates retrieval-based speculative decoding by speculating the next-next token from the last logit and retrieving relevant references for both next and next-next tokens, reporting up to 2.61x speedup and 3.28 mean accepted tokens.
-
PipeSD: An Efficient Cloud-Edge Collaborative Pipeline Inference Framework with Speculative Decoding
PipeSD is a cloud-edge collaborative inference framework that overlaps token generation and communication via dynamic programming pipeline scheduling and uses Bayesian-optimized dual-threshold NAV triggering, delivering 1.16x-2.16x speedup and 14.3%-25.3% energy reduction over baselines.
- SimSD: Simple Speculative Decoding in Diffusion Language Models