{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2023:DCS3XIDBIYGA7ABAULVZ5I3AXY","short_pith_number":"pith:DCS3XIDB","schema_version":"1.0","canonical_sha256":"18a5bba061460c0f8020a2eb9ea360be3cbc7d1831a9591876e6663cdd7f8382","source":{"kind":"arxiv","id":"2302.01318","version":1},"attestation_state":"computed","paper":{"title":"Accelerating Large Language Model Decoding with Speculative Sampling","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Speculative sampling generates multiple tokens per transformer call by drafting sequences from a smaller model and verifying them with rejection sampling that matches the target distribution exactly.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Charlie Chen, Geoffrey Irving, Jean-Baptiste Lespiau, John Jumper, Laurent Sifre, Sebastian Borgeaud","submitted_at":"2023-02-02T18:44:11Z","abstract_excerpt":"We present speculative sampling, an algorithm for accelerating transformer decoding by enabling the generation of multiple tokens from each transformer call. Our algorithm relies on the observation that the latency of parallel scoring of short continuations, generated by a faster but less powerful draft model, is comparable to that of sampling a single token from the larger target model. This is combined with a novel modified rejection sampling scheme which preserves the distribution of the target model within hardware numerics. We benchmark speculative sampling with Chinchilla, a 70 billion p"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":true,"formal_links_present":false},"canonical_record":{"source":{"id":"2302.01318","kind":"arxiv","version":1},"metadata":{"license":"http://creativecommons.org/licenses/by/4.0/","primary_cat":"cs.CL","submitted_at":"2023-02-02T18:44:11Z","cross_cats_sorted":[],"title_canon_sha256":"909bf5c0a99d70c8f3dcf2fde04d3f3ff066c60e34ef9fe1a39da7b3bf824139","abstract_canon_sha256":"fc9baa2a11d318aa437718252e11bdeedfc953d0064f81d2f08d5d0df1503fa3"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T05:38:24.955648Z","signature_b64":"Gkc7OkrD21d3HxbCaD7gme0rm4kICpdWfm/Ke/dH7EwrSwbZyp/Zfix6eOJZXHi+6yBDBYVQP0BIKdV9+75uCw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"18a5bba061460c0f8020a2eb9ea360be3cbc7d1831a9591876e6663cdd7f8382","last_reissued_at":"2026-07-05T05:38:24.955160Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T05:38:24.955160Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"Accelerating Large Language Model Decoding with Speculative Sampling","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Speculative sampling generates multiple tokens per transformer call by drafting sequences from a smaller model and verifying them with rejection sampling that matches the target distribution exactly.","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Charlie Chen, Geoffrey Irving, Jean-Baptiste Lespiau, John Jumper, Laurent Sifre, Sebastian Borgeaud","submitted_at":"2023-02-02T18:44:11Z","abstract_excerpt":"We present speculative sampling, an algorithm for accelerating transformer decoding by enabling the generation of multiple tokens from each transformer call. Our algorithm relies on the observation that the latency of parallel scoring of short continuations, generated by a faster but less powerful draft model, is comparable to that of sampling a single token from the larger target model. This is combined with a novel modified rejection sampling scheme which preserves the distribution of the target model within hardware numerics. We benchmark speculative sampling with Chinchilla, a 70 billion p"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"We benchmark speculative sampling with Chinchilla, a 70 billion parameter language model, achieving a 2-2.5x decoding speedup in a distributed setup, without compromising the sample quality or making modifications to the model itself.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The latency of parallel scoring of short continuations, generated by a faster but less powerful draft model, is comparable to that of sampling a single token from the larger target model.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Speculative sampling accelerates LLM decoding 2-2.5x by letting a draft model propose short sequences that the target model scores in parallel, then applies modified rejection sampling to keep the exact target distribution.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Speculative sampling generates multiple tokens per transformer call by drafting sequences from a smaller model and verifying them with rejection sampling that matches the target distribution exactly.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"410fdf7b93229e0572ea1e04357819a74fd374865f4023418fa69d87ee9353ee"},"source":{"id":"2302.01318","kind":"arxiv","version":1},"verdict":{"id":"6be6da2d-8aed-435c-940e-c30badc3424b","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-11T07:23:19.309503Z","strongest_claim":"We benchmark speculative sampling with Chinchilla, a 70 billion parameter language model, achieving a 2-2.5x decoding speedup in a distributed setup, without compromising the sample quality or making modifications to the model itself.","one_line_summary":"Speculative sampling accelerates LLM decoding 2-2.5x by letting a draft model propose short sequences that the target model scores in parallel, then applies modified rejection sampling to keep the exact target distribution.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The latency of parallel scoring of short continuations, generated by a faster but less powerful draft model, is comparable to that of sampling a single token from the larger target model.","pith_extraction_headline":"Speculative sampling generates multiple tokens per transformer call by drafting sequences from a smaller model and verifying them with rejection sampling that matches the target distribution exactly."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2302.01318/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":18,"sample":[{"doi":"","year":2021,"title":"Vivit: Avideovisiontransformer","work_id":"31194d14-e3e7-41f9-995f-fc580e37c0b0","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":1901,"title":"T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. Language models are few-shot learners.Advances in neural information processi","work_id":"8e180b3a-a506-460e-b38b-5d27a2fb6ab2","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":null,"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","ref_index":4,"cited_arxiv_id":"2107.03374","is_internal_anchor":true},{"doi":"","year":null,"title":"LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale","work_id":"98201f98-f4e5-4d1c-9ed7-b795e3c8f76c","ref_index":5,"cited_arxiv_id":"2208.07339","is_internal_anchor":true},{"doi":"","year":2010,"title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","work_id":"e96730e3-129b-4db6-b981-15ab7932e297","ref_index":6,"cited_arxiv_id":"2010.11929","is_internal_anchor":true}],"resolved_work":18,"snapshot_sha256":"5a32f7e842cb7f417ad0524bbac599ad5e13e5dcab2ba23d72943d8c368ed890","internal_anchors":7},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2302.01318","created_at":"2026-07-05T05:38:24.955222+00:00"},{"alias_kind":"arxiv_version","alias_value":"2302.01318v1","created_at":"2026-07-05T05:38:24.955222+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2302.01318","created_at":"2026-07-05T05:38:24.955222+00:00"},{"alias_kind":"pith_short_12","alias_value":"DCS3XIDBIYGA","created_at":"2026-07-05T05:38:24.955222+00:00"},{"alias_kind":"pith_short_16","alias_value":"DCS3XIDBIYGA7ABA","created_at":"2026-07-05T05:38:24.955222+00:00"},{"alias_kind":"pith_short_8","alias_value":"DCS3XIDB","created_at":"2026-07-05T05:38:24.955222+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":176,"internal_anchor_count":176,"sample":[{"citing_arxiv_id":"2607.06763","citing_title":"Trees from Marginals: Autoregressive drafting with factorized priors","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2607.08642","citing_title":"DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding","ref_index":11,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07388","citing_title":"TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2607.07409","citing_title":"DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2606.11201","citing_title":"To Intervene or Not: Guiding Inference-time Alignment with Probabilistic Model Blending","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25354","citing_title":"Efficient and Trainable Language Model Test-Time Scaling via Local Branch Routing","ref_index":5,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25091","citing_title":"Speculation at a Distance: Where Edge-Cloud Speculative Decoding Actually Pays Off","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2606.25097","citing_title":"Speculative Decoding at Temperature Zero: A Scoped Safety-Invariance Screen with a 48,072-Sample Expansion","ref_index":7,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24957","citing_title":"Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding","ref_index":3,"is_internal_anchor":true},{"citing_arxiv_id":"2606.24447","citing_title":"P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling","ref_index":13,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26978","citing_title":"To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2606.26744","citing_title":"HyperDFlash: Hyper-Connection-Aligned Block Speculative Decoding with Gated Residual Reduction","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2606.22840","citing_title":"RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20869","citing_title":"A3C3: AI Algorithm and Accelerator Co-design, Co-search, and Co-generation","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20295","citing_title":"Token-Operations-Oriented Inference Optimization Techniques for Large Models","ref_index":134,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20543","citing_title":"SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2606.18967","citing_title":"EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17787","citing_title":"LUMEN: Coordinated Failure Recovery for Distributed LLM Serving","ref_index":12,"is_internal_anchor":true},{"citing_arxiv_id":"2606.17518","citing_title":"SpecGen: Accelerating Agentic Kernel Optimization with Speculative Generation","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2606.18502","citing_title":"Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications","ref_index":31,"is_internal_anchor":true},{"citing_arxiv_id":"2606.18394","citing_title":"JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting","ref_index":6,"is_internal_anchor":true},{"citing_arxiv_id":"2606.13426","citing_title":"Accelerating Speculative Diffusions via Block Verification","ref_index":9,"is_internal_anchor":true},{"citing_arxiv_id":"2606.12935","citing_title":"MARS: Margin-Adversarial Risk-controlled Stopping for Parallel LLM Test-time Scaling","ref_index":4,"is_internal_anchor":true},{"citing_arxiv_id":"2607.01893","citing_title":"Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters","ref_index":2,"is_internal_anchor":true},{"citing_arxiv_id":"2606.12243","citing_title":"VIA-SD: Verification via Intra-Model Routing for Speculative Decoding","ref_index":57,"is_internal_anchor":true}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY","json":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY.json","graph_json":"https://pith.science/api/pith-number/DCS3XIDBIYGA7ABAULVZ5I3AXY/graph.json","events_json":"https://pith.science/api/pith-number/DCS3XIDBIYGA7ABAULVZ5I3AXY/events.json","paper":"https://pith.science/paper/DCS3XIDB"},"agent_actions":{"view_html":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY","download_json":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY.json","view_paper":"https://pith.science/paper/DCS3XIDB","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2302.01318&json=true","fetch_graph":"https://pith.science/api/pith-number/DCS3XIDBIYGA7ABAULVZ5I3AXY/graph.json","fetch_events":"https://pith.science/api/pith-number/DCS3XIDBIYGA7ABAULVZ5I3AXY/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY/action/timestamp_anchor","attest_storage":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY/action/storage_attestation","attest_author":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY/action/author_attestation","sign_citation":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY/action/citation_signature","submit_replication":"https://pith.science/pith/DCS3XIDBIYGA7ABAULVZ5I3AXY/action/replication_record"}},"created_at":"2026-07-05T05:38:24.955222+00:00","updated_at":"2026-07-05T05:38:24.955222+00:00"}