Pith. sign in

REVIEW 2 major objections 7 minor 29 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Cheap local head boosts speculative decoding without retraining

2026-07-09 11:39 UTC pith:6ZY5GNEI

load-bearing objection Practical speculative decoding improvement with a real theory-practice gap that the paper doesn't fully resolve the 2 major comments →

arxiv 2607.07409 v1 pith:6ZY5GNEI submitted 2026-07-08 cs.CL

DeLS-Spec: Decoupled Long-Short Contexts for Parallel Speculative Drafting

Hong-Kai Zheng , Piji Li This is my paper
classification cs.CL
keywords dels-specdflashdraftingtrainingheadlocalmodelspeculative
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters like DFlash predict an entire block of tokens in one pass, gaining speed but losing intra-block causal dependencies — each token in the block is predicted without knowing what the other tokens in the same block will be. Prior fixes (Domino, DSpark) restore this causality but require expensive joint training of the draft model from scratch. This paper proposes DeLS-Spec, which keeps DFlash fixed as a long-context expert and trains a tiny, independent RNN-based local head as a short-context expert. At inference, the two experts' logit distributions are combined after subtracting a unigram frequency prior to avoid double-counting common tokens. The local head trains in about one hour on a single GPU with under 10 GB of memory — compared to 13+ hours and 42+ GB for Domino-FT — because it only needs plain text and never touches the target model or DFlash backbone. Experiments on Qwen3-4B and Qwen3-8B show DeLS-Spec consistently improves both decoding speedup and acceptance length over DFlash across math, code, and dialogue benchmarks, recovering about 76% of the gain that end-to-end joint training (Domino-FT) achieves over DFlash, at a fraction of the cost. The local head also transfers across different DFlash checkpoints without retraining.

Core claim

The central mechanism is a product-of-experts logit fusion with unigram-prior subtraction. The paper decomposes the true conditional probability of a draft token into three parts: a long-context term from DFlash, a short-context term from the local head, and a unigram prior. Because both the long and short experts already encode the intrinsic frequency bias of each token, multiplying them double-counts that bias — so the unigram prior is divided out. The paper proves this decomposition is exact up to a residual interaction term, then deliberately drops that residual in exchange for modularity and cheap independent training. The key empirical finding is that the dropped residual is small in实践

What carries the argument

Logit fusion formula: final_logits = DFlash_long_logits + alpha * local_head_short_logits - beta * unigram_prior_logits

Load-bearing premise

The method assumes that the long context and the local block prefix are approximately conditionally independent given the candidate token, which lets it drop a residual interaction term. The paper's own comparison shows this dropped term accounts for roughly 24% of the improvement that end-to-end joint training achieves, so the approximation is good but not free.

What would settle it

If the residual interaction term is large for tasks with strong long-range-to-local dependencies (e.g., code completion where a distant function signature constrains nearby token choices), DeLS-Spec's acceptance length would fall well below joint-training methods, showing the approximation breaks down.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Existing block-parallel speculative decoding systems can be upgraded with intra-block causality without retraining their draft models, lowering the barrier to adoption for teams with limited GPU resources.
  • The local head is checkpoint-agnostic: one trained head can be reused across compatible DFlash-style models, enabling a modular ecosystem where correction heads are shared independently of backbone versions.
  • The product-of-experts-with-prior-subtraction decomposition could apply to other speculative decoding architectures beyond DFlash, since the theoretical framework does not depend on DFlash-specific properties.
  • The 24% of achievable gain left on the table by omitting the residual term quantifies a clear cost-quality tradeoff: near-full performance at 12x lower training cost and 5x lower memory.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the residual interaction term is larger for certain task types (e.g., highly structured code with strict syntactic dependencies between nearby tokens), the gap between DeLS-Spec and end-to-end methods like Domino-FT would widen, potentially making joint training worth its cost for those domains.
  • The observation that learnable fusion weights underperform fixed ones due to teacher-forcing exposure bias suggests that scheduled sampling or reinforcement-learning-based tuning of the weights could close this gap and recover additional acceptance length without retraining the backbone.
  • The position-wise acceptance rate analysis shows the largest gains in the middle of the draft block (positions 5-6), suggesting the local head's causal correction has diminishing returns for very late positions — possibly because errors compound through the autoregressive local head itself, creating a natural ceiling on block size.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 7 minor

Summary. The paper proposes DeLS-Spec, a method for improving block-parallel speculative decoding (DFlash) by introducing a lightweight, independently trained local head that models intra-block causal dependencies. The key idea is a product-of-experts logit fusion: long-context logits from DFlash are combined with short-context logits from the local head, with a unigram prior subtracted to avoid double-counting token frequency bias. The local head is trained on plain text with standard next-token prediction, requiring no access to target-model or DFlash hidden states, which reduces training cost by over 12x and VRAM by ~4.7x compared to Domino-FT. Experiments on Qwen3-4B and Qwen3-8B across 8 benchmarks show consistent improvements in speedup and average acceptance length over DFlash. The paper also provides a hyperparameter scan, transferability tests, and a comparison quantifying the cost of omitting the residual interaction term.

Significance. The paper addresses a practical and relevant problem in speculative decoding: adding intra-block causality to block-parallel drafters without expensive joint retraining. The product-of-experts factorization (Eq. 4-8) with unigram-prior subtraction is a clean and well-motivated design. The training efficiency gains are substantial and well-documented (Table 5). The experimental coverage is thorough: Table 1 spans 8 benchmarks across 2 models and 2 temperatures, Table 2 demonstrates transferability to different DFlash checkpoints, Table 3 quantifies the omitted residual, and Figure 2 provides a hyperparameter scan. The position-wise acceptance rate analysis (Table 8) is a nice addition that localizes where gains occur. The method is practical and the gains, while modest (3-5% speedup), are consistent.

major comments (2)
  1. §4.3, Eq. (10) and §5.4: The product-of-experts derivation in §4.1 (Eq. 8) corresponds to α=β=1, but Figure 2 shows that α=β=1 yields τ well below the DFlash baseline (roughly −1.73 relative to the best setting). The optimal operating point is α=β=0.3. The paper attributes this to 'different scales' across sources (§4.3), but this explanation is insufficient: if scale mismatch were the sole issue, a single temperature parameter or the learnable-α/β variant (Table 4) should recover performance. Yet Table 4 shows learnable weights (converging to ~0.5–0.76 per Table 7) still underperform fixed 0.3. The paper's alternative explanation — exposure bias from teacher forcing (§5.6, Appendix C) — is plausible but unverified. This matters because the central claim frames DeLS-Spec as a principled decoupling via logit fusion with unigram-prior subtraction. If the gains at α=β=0.3 come from a weaker
  2. §5.4: The grid search for α and β (Figure 2) is performed on Qwen3-4B, and α=β=0.3 is selected as the default for all experiments including Qwen3-8B (Table 1) and the DSpark block-7 checkpoints (Table 2). However, there is no verification that this hyperparameter transfers across model sizes or architectures. The paper should either (a) confirm that α=β=0.3 is near-optimal for Qwen3-8B and the block-7 checkpoints via a limited scan, or (b) explicitly state this as a limitation and discuss the sensitivity of results to this choice. Without this, it is unclear whether the 8B results reflect a robust operating point or a potentially suboptimal one.
minor comments (7)
  1. Title header: 'DECOUPLEDLONG-SHORTCONTEXTS FORPARALLELSPECULATIVEDRAFTING' appears to have missing spaces, likely a LaTeX formatting issue.
  2. §5.7 states 'all methods are trained with batch size 1, accumulation steps 4,' but Table 6 lists batch-size as 4 and accumulation-steps as 1. Please reconcile.
  3. §5.7 states 'we sample 256 anchors from each example,' but Table 6 lists num-anchors as 512. Please clarify which is correct.
  4. Table 3: The τ gain ratio for LCB is 105.56%, meaning DeLS-Spec (RNN) surpasses Domino-FT. This is a notable result that could be highlighted more prominently in the main text, as it suggests the residual term is not always beneficial.
  5. §4.1, Eq. (6): The term p(y)p(z_i)/p(y,z_i) is described as not depending on x_i and absorbed into normalization. This is correct for a fixed y and z_i, but during speculative decoding z_i changes as draft tokens are generated. A brief clarification that this term is constant within a single position's ranking (not across positions) would avoid confusion.
  6. Table 7 (Appendix C): The learned α and β values increase with position, which is interpreted as the local head becoming more reliable for later tokens under teacher forcing. An alternative explanation is that DFlash's parallel predictions degrade more at later positions, making the local head relatively more valuable. Both explanations are plausible; briefly acknowledging the alternative would strengthen the discussion.
  7. Figure 2: The color scale annotations (e.g., '+0.00', '-0.01') overlaid on the heatmap cells are small and may be hard to read in print. Consider enlarging or providing a separate colorbar with labeled ticks.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises two major comments, both of which concern the relationship between the theoretical motivation and the empirical operating point of DeLS-Spec. We address each below.

read point-by-point responses

  1. Referee: Comment 1 (§4.3, Eq. 10 and §5.4): The product-of-experts derivation corresponds to α=β=1, but Figure 2 shows α=β=1 is far from optimal (α=β=0.3 is best). Scale mismatch alone doesn't explain this, since learnable α/β should recover performance but doesn't. The exposure bias explanation is plausible but unverified. This matters because the central claim frames DeLS-Spec as principled decoupling via logit fusion with unigram-prior subtraction.

    Authors: The referee is correct that there is a genuine gap between the theoretical derivation (which motivates α=β=1) and the empirical optimum (α=β=0.3), and we agree that our current explanation is incomplete. We address this in two parts. First, we note that the theoretical framework does make one prediction that is empirically confirmed: the unigram-prior subtraction (β>0) is beneficial across virtually all values of α, as shown in the middle and right panels of Figure 2. This supports the core claim that dividing out the unigram prior is a principled correction, regardless of the specific weight values. Second, we acknowledge that the referee's critique of the scale-mismatch explanation is valid. If scale mismatch were the sole issue, a single temperature or learnable weights should recover performance, yet Table 4 shows they do not. The exposure bias hypothesis (Appendix C) is our best current explanation for why learnable weights converge to values (~0.5–0.76) that still underperform the fixed 0.3 setting, but we agree this is unverified — we have not run experiments (e.g., scheduled sampling, reinforcement of α/β on drafted rather than ground-truth prefixes) that would directly test it. We will revise the manuscript to be more transparent about this: (a) we will explicitly state that the theoretical derivation motivates the form of the fusion (logit addition with unigram-prior subtraction) but does not predict the optimal weight values; (b) we will reframe the α=β=1 point as a theoretical reference rather than a recommended operating point; and (c) we will acknowledge that the exposure bias explanation remains a hypothesis and identify its direct verification as future work. We believe this is an honest characterization: the product-of-experts factorization and unigram- revision: partial

  2. Referee: Comment 2 (§5.4): The grid search for α and β is performed only on Qwen3-4B, but α=β=0.3 is used for all experiments including Qwen3-8B and block-7 checkpoints. No verification that this hyperparameter transfers across model sizes or architectures.

    Authors: This is a fair concern. We will run a limited α/β scan on Qwen3-8B (using the same benchmarks as in Figure 2) to verify that α=β=0.3 is near-optimal for the 8B model as well. We will also run a smaller check on at least one of the DSpark block-7 checkpoints. If the scan confirms that 0.3 is near-optimal for 8B, we will report this in the revised manuscript. If it reveals a different optimum, we will report the 8B-optimal values and re-run Table 1 accordingly. In either case, we will add an explicit statement about the scope of the hyperparameter search and its transferability. We note that the consistency of improvements across both model sizes and the independently trained block-7 checkpoints (Tables 1 and 2) provides indirect evidence that the method is not overly sensitive to the exact α/β values, but we agree that a direct verification is needed and will add it. revision: yes

Circularity Check

0 steps flagged

No circularity found: the derivation chain is self-contained, the factorization follows from Bayes' theorem, and empirical results are measured against external baselines.

full rationale

The paper's central derivation chain proceeds as follows: (1) Start from the product-of-experts intuition (Eq. 3), (2) derive an exact factorization of p(x_i|y,z_i) via Bayes' theorem (Eq. 4, proven step-by-step in Appendix B without self-citation), (3) drop the token-independent normalization factor and the residual interaction term R (Eq. 7) to obtain the practical approximation (Eq. 8), and (4) introduce calibration weights α, β at inference (Eq. 10). None of these steps reduce to their inputs by construction. The local head is trained independently with standard next-token prediction on plain text (Eq. 9) without access to DFlash or target-model hidden states. The unigram prior is estimated from corpus statistics (Appendix A). The product-of-experts factorization cites Hinton (2002), an external reference. No self-citations by the present authors are load-bearing. The empirical results (Tables 1–4) are measured against external baselines (DFlash, EAGLE-3, DART, Domino-FT) using independently released checkpoints. The acknowledged disconnect between the theoretical prediction (α=β=1) and the empirical optimum (α=β=0.3) is a validity/correctness concern, not circularity: the paper does not fit α,β to the test data and then claim prediction of those same results. The grid search (Figure 2) is reported transparently as hyperparameter tuning, and the learnable-weights experiment (Table 4) honestly shows underperformance relative to fixed weights. The 75.69% recovery metric (Table 3) is computed from measured τ values against an external baseline, not from a fitted quantity renamed as a prediction. The derivation is self-contained and the central claim has independent empirical content.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities (particles, forces, dimensions, etc.). It introduces a method (DeLS-Spec) and an architectural component (the local head), but these are engineering constructs, not postulated physical or mathematical objects. The free parameters (alpha, beta) are calibration coefficients tuned on held-out data, and the axioms are standard probability theory plus the domain-specific approximation of dropping the residual term.

free parameters (6)
  • alpha = 0.3
    Fusion weight for local-head logits, grid-searched on Qwen3-4B (Figure 2) and applied to all settings. Theoretical value is 1.0 but practical best is 0.3.
  • beta = 0.3
    Fusion weight for unigram-prior subtraction, grid-searched on Qwen3-4B (Figure 2) and applied to all settings. Theoretical value is 1.0 but practical best is 0.3.
  • lambda (smoothing) = 1.0
    Additive smoothing parameter for unigram prior estimation (Appendix A). Default value, not tuned.
  • loss-decay-gamma = 7.0
    Decay factor for position-weighted loss in local head training (Table 6).
  • local-gru-hidden-dim = 1024
    Hidden dimension of the GRU local head (Table 6).
  • local-rank = 256
    Low-rank bottleneck dimension for the local head (Table 6).
axioms (4)
  • domain assumption Product of experts: p(x_i|y,z_i) can be approximated by p(x_i|y)p(x_i|z_i)/p(x_i)
    Section 4.1, Eq. 3-8. The factorization is exact (proven in Appendix B) but the approximation comes from dropping the residual term R(x_i;y,z_i).
  • ad hoc to paper The residual interaction term R(x_i;y,z_i) is small enough to ignore for practical speculative decoding
    Section 4.1, Eq. 7-8. The paper deliberately omits this term for modularity. Table 3 shows it accounts for ~25% of the achievable gain, validating that it is non-zero but bounded.
  • domain assumption Standard next-token prediction on plain text is sufficient to learn useful short-context causal dependencies
    Section 4.2, Eq. 9. The local head is trained without access to DFlash or target model, relying on the assumption that local token dependencies are learnable from text alone.
  • standard math Bayes' theorem and standard probability calculus
    Appendix B, Eq. 11-16. Used to derive the factorization.

pith-pipeline@v1.1.0-glm · 21860 in / 3073 out tokens · 459751 ms · 2026-07-09T11:39:22.202821+00:00 · methodology

0 comments
read the original abstract

Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel. Block-parallel drafters such as DFlash further improve drafting efficiency by predicting an entire block in one pass, but their position-wise predictions lack explicit intra-block causal conditioning. Recent methods such as Domino and DSpark attempt to introduce such causality into block-parallel drafting, but they require training the draft model from scratch, which limits their flexibility and increases training cost. We propose DeLS-Spec, a decoupled long-short context speculative decoding method. DeLS-Spec treats the fixed DFlash model as a long-context expert and introduces a lightweight local head as a short-context expert. The local head can be trained independently with a standard next-token prediction objective, without joint training with the target model or the DFlash backbone, leading to extremely low training cost. At inference time, DeLS-Spec combines long-context and short-context logits, and the local head is not tied to a specific DFlash checkpoint, making the method more modular and flexible. Experiments on Qwen3 models show that DeLS-Spec consistently improves speedup and average acceptance length over DFlash across math, code, and dialogue benchmarks.

Figures

Figures reproduced from arXiv: 2607.07409 by Hong-Kai Zheng, Piji Li.

Figure 1
Figure 1. Figure 1: Overview of DeLS-Spec. DFlash produces long-context logits in parallel, while a [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: α, β scan on Qwen3-4B. Left: average acceptance length under different α and β values. Middle: row-wise improvement after subtracting the corresponding β = 0 value. Right: improve￾ment of the best β for each α over β = 0 [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 18 internal anchors

  1. [1]

    GPT-4 Technical Report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    work page internal anchor Pith review Pith/arXiv arXiv

  2. [2]

    Pard: Accelerating llm inference with low-cost parallel draft model adaptation.arXiv preprint arXiv:2504.18583,

    Zihao An, Huajun Bai, Ziqiong Liu, Dong Li, and Emad Barsoum. Pard: Accelerating llm inference with low-cost parallel draft model adaptation.arXiv preprint arXiv:2504.18583,

    work page arXiv

  3. [3]

    Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding

    Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christopher Rinard, Jonathan Ragan- Kelley, and William Brandon. Hydra: Sequentially-dependent draft heads for medusa decoding. arXiv preprint arXiv:2402.05109,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    Block diffusion: Interpolating between autoregressive and diffusion language models

    Marianne Arriola, Aaron Gokaslan, Justin Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sahoo, and V olodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. InInternational Conference on Learning Representations, volume 2025, pp. 50726–50753,

    work page 2025

  5. [5]

    Program Synthesis with Large Language Models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXiv preprint arXiv:2108.07732,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

    Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. Medusa: Simple llm inference acceleration framework with multiple decoding heads.arXiv preprint arXiv:2401.10774,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Accelerating Large Language Model Decoding with Speculative Sampling

    Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling.arXiv preprint arXiv:2302.01318,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    DFlash: Block Diffusion for Flash Speculative Decoding

    Jian Chen, Yesheng Liang, and Zhijian Liu. Dflash: Block diffusion for flash speculative decoding. arXiv preprint arXiv:2602.06036,

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    Evaluating Large Language Models Trained on Code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation

    Shuang Cheng, Yihan Bian, Dawei Liu, Yuhua Jiang, Yihao Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, et al. Sdar: A synergistic diffusion-autoregression paradigm for scalable sequence generation. InFindings of the Association for Computational Linguistics: ACL 2026, pp. 22058–22075, 2026a. Xin Cheng, Xingkai Yu, Chenze Shao, Jiashi...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  11. [11]

    Speculative diffusion decoding: Accelerating language generation through diffusion

    Jacob K Christopher, Brian R Bartoldson, Tal Ben-Nun, Michael Cardei, Bhavya Kailkhura, and Ferdinando Fioretto. Speculative diffusion decoding: Accelerating language generation through diffusion. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (V olu...

    work page 2025

  12. [12]

    Training Verifiers to Solve Math Word Problems

    11 Preprint Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Non-Autoregressive Neural Machine Translation

    Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. Non-autoregressive neural machine translation.arXiv preprint arXiv:1711.02281,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Measuring Mathematical Problem Solving With the MATH Dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset.arXiv preprint arXiv:2103.03874,

    work page internal anchor Pith review Pith/arXiv arXiv

  15. [15]

    JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

    URLhttps://arxiv.org/ abs/2606.18394. Jianuo Huang, Yaojie Zhang, Qituan Zhang, Hao Lin, Hanlin Xu, and Linfeng Zhang. Domino: Decoupling causal modeling from autoregressive drafting in speculative decoding.arXiv preprint arXiv:2605.29707,

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

    Guanghao Li, Zhihui Fu, Min Fang, Qibin Zhao, Ming Tang, Chun Yuan, and Jun Wang. Diffuspec: Unlocking diffusion language models for speculative decoding. InFindings of the Association for Computational Linguistics: ACL 2026, pp. 20896–20910, 2026a. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feat...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  17. [17]

    Dart: Diffusion-inspired speculative decoding for fast llm inference.arXiv preprint arXiv:2601.19278,

    Fuliang Liu, Xue Li, Ketai Zhao, Yinxi Gao, Ziyan Zhou, Zhonghui Zhang, Zhibin Wang, Wanchun Dou, Sheng Zhong, and Chen Tian. Dart: Diffusion-inspired speculative decoding for fast llm inference.arXiv preprint arXiv:2601.19278,

    work page arXiv

  18. [18]

    Tidar: Think in diffusion, talk in autoregression.arXiv preprint arXiv:2511.08923,

    Jingyu Liu, Xin Dong, Zhifan Ye, Rishabh Mehta, Yonggan Fu, Vartika Singh, Jan Kautz, Ce Zhang, and Pavlo Molchanov. Tidar: Think in diffusion, talk in autoregression.arXiv preprint arXiv:2511.08923,

    work page arXiv

  19. [19]

    TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding

    Peer Rheinboldt, Fr´ed´eric Berdoz, and Roger Wattenhofer. Treeflash: Parallel ar-approximation for faster speculative decoding.arXiv preprint arXiv:2606.03819,

    work page internal anchor Pith review Pith/arXiv arXiv

  20. [20]

    Accelerating Speculative Decoding with Block Diffusion Draft Trees

    Liran Ringel and Yaniv Romano. Accelerating speculative decoding with block diffusion draft trees. arXiv preprint arXiv:2604.12989,

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Specdiff- 2: Scaling diffusion drafter alignment for faster speculative decoding.arXiv preprint arXiv:2511.00606,

    Jameson Sandler, Jacob K Christopher, Thomas Hartvigsen, and Ferdinando Fioretto. Specdiff- 2: Scaling diffusion drafter alignment for faster speculative decoding.arXiv preprint arXiv:2511.00606,

    work page arXiv

  22. [22]

    Alpaca: A strong, replicable instruction- following model.Stanford Center for Research on F oundation Models

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpaca: A strong, replicable instruction- following model.Stanford Center for Research on F oundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html, 3(6):7,

    work page 2023

  23. [23]

    HuggingFace's Transformers: State-of-the-art Natural Language Processing

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R ´emi Louf, Morgan Funtowicz, et al. Huggingface’s transformers: State-of-the-art natural language processing.arXiv preprint arXiv:1910.03771,

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  24. [24]

    D-PACE: Dynamic Position-Aware Cross-Entropy for Parallel Speculative Drafting

    Tianyu Wu, Yu Yao, Zhenting Qi, Han Zheng, Zhuohan Wang, Haoran Ma, Lawrence Liao, Himabindu Lakkaraju, Ju Li, and Yilun Du. D-pace: Dynamic position-aware cross-entropy for parallel speculative drafting.arXiv preprint arXiv:2605.18810,

    work page internal anchor Pith review Pith/arXiv arXiv

  25. [25]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    American invitational mathematics examination (aime) 2025,

    Yifan Zhang and Team Math-AI. American invitational mathematics examination (aime) 2025,

    work page 2025

  27. [27]

    Distillspec: Improving speculative decoding via knowledge distillation

    Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-Franc ¸ois Kagy, and Rishabh Agarwal. Distillspec: Improving speculative decoding via knowledge distillation. InInternational Conference on Learning Representations, volume 2024, pp. 32011–32050,

    work page 2024

  28. [28]

    13 Preprint A IMPLEMENTATIONDETAILS Local Head Implementation.We instantiatep S(xi |z i)with a lightweight local head over the local draft prefix. Since the local head is evaluated autoregressively during inference, we use a low- rank bottleneck to reduce the sequential computation cost, following the design choice in Domino and DSpark (Huang et al., 2026...

    work page 2026

  29. [29]

    For each training sequence, we count only tokens whose loss mask equals one, i.e., assistant-response tokens that contribute to the super- vised objective

    Unigram Prior Estimation.We construct a fixed unigram log-prior over target tokens using the same tokenizer and chat-template preprocessing as training. For each training sequence, we count only tokens whose loss mask equals one, i.e., assistant-response tokens that contribute to the super- vised objective. Letc(v)denote the resulting count for vocabulary...

    work page 2015