Pith Number

pith:DCS3XIDB

pith:2023:DCS3XIDBIYGA7ABAULVZ5I3AXY

not attested not anchored not stored refs resolved

Accelerating Large Language Model Decoding with Speculative Sampling

Charlie Chen, Geoffrey Irving, Jean-Baptiste Lespiau, John Jumper, Laurent Sifre, Sebastian Borgeaud

Speculative sampling generates multiple tokens per transformer call by drafting sequences from a smaller model and verifying them with rejection sampling that matches the target distribution exactly.

arxiv:2302.01318 v1 · 2023-02-02 · cs.CL

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{DCS3XIDBIYGA7ABAULVZ5I3AXY}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

We benchmark speculative sampling with Chinchilla, a 70 billion parameter language model, achieving a 2-2.5x decoding speedup in a distributed setup, without compromising the sample quality or making modifications to the model itself.

C2weakest assumption

The latency of parallel scoring of short continuations, generated by a faster but less powerful draft model, is comparable to that of sampling a single token from the larger target model.

C3one line summary

Speculative sampling accelerates LLM decoding 2-2.5x by letting a draft model propose short sequences that the target model scores in parallel, then applies modified rejection sampling to keep the exact target distribution.

References

18 extracted · 18 resolved · 7 Pith anchors

[1] Vivit: Avideovisiontransformer 2021

[2] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in neural information processi 1901

[4] Evaluating Large Language Models Trained on Code · arXiv:2107.03374

[5] LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale · arXiv:2208.07339

[6] An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 2010 · arXiv:2010.11929

Cited by

176 papers in Pith

Trees from Marginals: Autoregressive drafting with factorized priors

DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding

TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

To Intervene or Not: Guiding Inference-time Alignment with Probabilistic Model Blending

Receipt and verification

First computed	2026-07-05T05:38:24.955160Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

18a5bba061460c0f8020a2eb9ea360be3cbc7d1831a9591876e6663cdd7f8382

Aliases

arxiv: 2302.01318 · arxiv_version: 2302.01318v1 · doi: 10.48550/arxiv.2302.01318 · pith_short_12: DCS3XIDBIYGA · pith_short_16: DCS3XIDBIYGA7ABA · pith_short_8: DCS3XIDB

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 18a5bba061460c0f8020a2eb9ea360be3cbc7d1831a9591876e6663cdd7f8382

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "fc9baa2a11d318aa437718252e11bdeedfc953d0064f81d2f08d5d0df1503fa3",
    "cross_cats_sorted": [],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2023-02-02T18:44:11Z",
    "title_canon_sha256": "909bf5c0a99d70c8f3dcf2fde04d3f3ff066c60e34ef9fe1a39da7b3bf824139"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2302.01318",
    "kind": "arxiv",
    "version": 1
  }
}