Pith. sign in

REVIEW 2 major objections 4 minor 42 references

Approximate Speculative Decoding

T0 review · 2 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Approximate Speculative Decoding replaces strict first-mismatch truncation with budgeted longest-prefix selection, reusing target-greedy suffixes to gain 3.05%–15.26% throughput over strict speculative decoding.

desk verdict A genuinely new verifier-side mechanism with robust throughput gains; the accuracy-preservation phrasing should be softened, but the central result holds. read the letter →

arxiv 2608.03447 v1 pith:NF43OYFA submitted 2026-08-04 cs.LG cs.AI

Yuannuo Feng , Zegang Peng , Yuxin Xie , Yubing Ye , Yizhe Chen , Wenshuai Yao , Wenyong Zhou , Wang Kang This is my paper
classification cs.LGcs.AI
keywords speculativedecodingapproximateverificationbudgetedprefixselectiontarget-logitregretsuffixreusethroughputoptimizationLLMinferencegreedy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that speculative decoding's standard 'stop at the first mismatch' rule is unnecessarily wasteful: the target model has already scored the whole draft block, and a later suffix can be target-greedy under the prefix that includes an accepted early mismatch. ASD therefore turns verification into a budgeted longest-prefix selection: it accepts a draft token only when its local target-logit regret is small relative to the remaining proposal length, the block's exception count is below a cap, and the request's cumulative regret ledger has room. If this is right, a training-free verifier-side change can raise fixed-workload throughput by 3.05%–15.26% (7.78% average on seven Qwen3-14B + DSpark-14B tasks) while keeping measured task accuracy roughly intact on most benchmarks, at the cost of a changed decoding trajectory. The paper presents ASD as a controlled approximation, not an output-preserving speedup.

What carries the argument

The central object is the budgeted longest-prefix verifier: a per-position feasibility mask f_i = (1-d_i) OR (r_i/q_i <= g AND cumulative regret <= B-s AND exception count <= M), with the accepted length being the longest all-true prefix. The persistent request-level regret ledger s tracks spent regret across rounds; the local regret r_i = z_i(y*_i) - z_i(x_i) is the target's conditional log-probability preference for its greedy token over the draft token; and realized-prefix suffix reuse (Proposition 1) says a committed draft token equal to its target argmax is target-greedy under the realized history and needs no additional exception. Together these turn verification into O(K) arithmetic o

What would settle it

Take a fixed benchmark with exact-match scoring, run ASD at B=8 and B=16 versus strict B=0, and plot benchmark accuracy against cumulative spent regret across requests; if accuracy falls monotonically and steeply while accepted exceptions still satisfy the local gate and cap, the regret proxy does not control downstream damage, falsifying the central premise. A natural first test is code generation where the output must compile and pass unit tests.

Watch

Extended reading notes

Core claim

ASD's central discovery is that a bounded number of explicitly accepted draft-target mismatches can unlock a contiguous already-scored suffix: once a low-regret mismatch at position i is committed, later draft tokens that coincide with the target argmax under the realized prefix are reusable without another target forward pass. The verifier selects the longest feasible prefix satisfying three conditions—local gate r_i/q_i <= g, per-block exception cap M, and request-level cumulative regret budget B—and updates a persistent ledger of spent regret. With B=0, the procedure is exactly standard token-ID greedy verification. Across DSpark, EAGLE3, and Medusa drafters, accepted length increases in

Load-bearing premise

The safety margin rests on the assumption that local target-logit regret, normalized by remaining proposal length and budgeted across the request, is a reliable enough proxy for preserving downstream task quality that a persistent budget keeps damage in check; the paper itself notes this is an accounting quantity rather than a calibrated quality guarantee.

Editorial extensions

If this is right

  • If ASD's claims hold, any greedy speculative-decoding pipeline can be accelerated by swapping the verifier only, with no drafter retraining or fine-tuning.
  • Because the budget, gate, and cap are explicit knobs, the speed–accuracy trade-off becomes a tunable operating point that can be frozen on a dev slice and audited per task.
  • The gains are not tied to one proposal architecture: positive results across DSpark, EAGLE3, and Medusa imply a verifier-side add-on that composes with future drafters.
  • Setting B=0 recovers strict greedy verification exactly, so the method can be toggled off losslessly when output fidelity matters.
  • Since trajectories change (hash divergence above 95% on GSM8K and MATH-500), ASD is appropriate where task-level accuracy, not token identity, is the contract.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The regret ledger is request-level, so long-form generation may accumulate many small exceptions; a natural test is to measure whether wall-clock gains persist and quality holds on long-context benchmarks beyond the 512-token natural-EOS limit.
  • Normalizing local regret by remaining proposal length qi implicitly rewards early mismatches; an alternative normalization by, say, expected suffix value or a learned cost model could change the Pareto frontier, though the paper does not test this.
  • Because ASD modifies only the verifier, it should stack with tree-based draft organization and exact speculative sampling; a combined implementation would show whether the 3–15% gain is additive or partially overlapping.
  • The large-model FP4-to-FP8 acceptance results suggest the mechanism transfers to low-precision serving, but the paper reports acceptance only, not end-to-end speedup, for that setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes Approximate Speculative Decoding (ASD), a training-free modification to the verifier in speculative decoding. Instead of stopping at the first draft token that differs from the target argmax, ASD accepts a bounded number of mismatches subject to a local target-logit regret gate, a per-block exception cap, and a persistent request-level regret budget. It then reuses the contiguous target-greedy suffix that was already scored by the teacher-forced target pass. The method reduces exactly to standard greedy verification when the budget is zero and adds only O(K) verifier-side arithmetic. Experiments on Qwen3-14B with DSpark-14B report fixed-workload throughput gains of 3.64%–11.73% (mean 7.78%) over matched strict speculative decoding, and cross-family experiments with DSpark, EAGLE3, and Medusa report gains of 3.05%–15.26% across all ten reported cells. Natural-EOS accuracy deltas are mostly non-negative but reported only as point estimates; hash divergence is high on GSM8K and MATH-500. The paper also reports acceptance gains on DeepSeek-V4-Flash with DSpark. The algorithmic derivation, the suffix-reuse observation, and the implementation details are clear and appear sound.

Significance. If the empirical claims hold, ASD is a simple and broadly compatible verifier-side improvement: it requires no draft-model retraining, no target training, and no extra target forward passes, and it exactly reduces to strict verification at zero budget. The paper is careful to separate throughput measurement from an accuracy audit, and it explicitly acknowledges that ASD changes the decoding trajectory and is not an output-preserving or safety-guaranteed method. The public code and the use of disjoint development slices for hyperparameter selection are strengths. The main weakness is that the accuracy-preservation claim rests on point estimates without uncertainty quantification, despite the fact that the method deliberately changes the decoding trajectory and the hash audit shows very frequent divergence.

major comments (2)
  1. [Tables 1 and 2, 'Natural-EOS audit'] The claim that ASD 'can retain task performance on most measured tasks' is supported only by point estimates of ΔAcc. No confidence intervals, standard errors, or significance tests are reported for accuracy changes, despite the stated protocol of four repetitions per task. On HumanEval, which has 164 problems, a change of −0.61 points is exactly one changed problem; MT-Bench's −0.64 is similarly within one-item noise. Since hash divergence exceeds 95% on GSM8K and MATH-500, trajectories are pervasively altered, making the natural-EOS audit the sole evidence that task quality is retained. Please report per-run accuracy distributions, bootstrap confidence intervals, or paired significance tests (e.g., McNemar), and state explicitly which deltas are distinguishable from zero. Without this, the accuracy-preservation aspect of the contribution is not statistically established.
  2. [Figure 5 / Table 3 and control sweeps] The control sweeps that motivate the frozen configuration (B=8, g=0.25, M=2) are largely single-seed and single-task. Figure 5 uses n=256 and one seed, Figure 6(b) uses n=64 and one seed, and Table 3 does not state the number of repetitions. The paper should either add repeated-run variability to these sweeps or explicitly label them as exploratory selection diagnostics rather than measured operating-characteristic curves. This is secondary to the main throughput result, but it bears on the reproducibility of the selected operating point.
minor comments (4)
  1. [Abstract / Section 'Experiments'] The model size is given as 284B in the abstract but 285B in the introduction and experiments. Please make this consistent.
  2. [Tables 1 and 2] The table captions say 'Parentheses denote confidence-interval half-widths,' but the ΔAcc. columns contain no parentheses or uncertainty intervals. Clarify that confidence intervals apply only to the TPS gains.
  3. [Section 'Budgeted Prefix Verification', Eq. (14)] Equation (14) is correct, but the phrase 'longest all-true prefix' could be misread as requiring f_i to be true independently at each position. It may help to state explicitly that C_i and N_i are cumulative, so the mask already encodes the prefix constraints.
  4. [Figure 5] The statement that g=0.25 'yields a 7.2% TPS gain without a measured GSM8K accuracy change' should be softened or supported by an uncertainty interval, since Figure 5 is based on one seed and n=256.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ASD's throughput and accuracy claims are empirical measurements against matched strict verification, with controls frozen on disjoint development slices; the regret proxy is explicitly heuristic and no load-bearing step reduces to its own inputs.

full rationale

The paper's central claims are empirical rather than derivational. Fixed-workload throughput gains are measured against matched strict speculative decoding runs in adjacent triplets, with strict-hash and token-count matching and baseline-drift filtering; hyperparameters (B, g, M) were selected on disjoint development slices and frozen before the multi-task evaluation. The accuracy audit reports point deltas under natural-EOS decoding and is explicitly separated from the throughput result. The regret proxy is acknowledged in the text as 'an accounting quantity, rather than a calibrated error probability or a guarantee about downstream task quality,' and the ledger is repeatedly stated not to imply output identity, semantic preservation, or task correctness. Proposition 1, which justifies suffix reuse, is a definitional identity: for a committed contiguous prefix, the teacher-forced history used to compute y*_j is exactly the realized ASD history, so if x_j = y*_j then x_j is target-greedy by definition. This is a mechanism lemma, not a predicted result, and it does not smuggle in the measured speedup. The paper cites no prior work by the same authors as load-bearing; all cited results are external baselines or background. The absence of confidence intervals or significance tests on accuracy deltas is a statistical-evidence limitation and a correctness risk, but it is not circularity because the deltas are not fitted inputs that are later relabeled as predictions. Thus no circular step can be exhibited from the paper's own equations or citations, and the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities beyond the budget ledger, which is a data structure. The free parameters B, g, M are user-set knobs tuned on a development set, not fitted to the reported test results. The two ad hoc axioms about proxy quality are acknowledged in the paper.

free parameters (3)
  • B (request-level regret budget) = 8
    Selected by hyperparameter search on a disjoint GSM8K slice; controls cumulative accepted regret.
  • g (local regret gate) = 0.25
    Selected on development slice; normalizes local regret by remaining proposal length.
  • M (per-block exception cap) = 2
    Selected on development slice; caps exceptions per block.
assumptions (3)
  • domain assumption Teacher-forced target rows are computed under draft history including mismatches (Eq. 1).
    Required for Proposition 1 and the entire suffix-reuse mechanism; this is how speculative decoding verification is standardly implemented.
  • ad hoc to paper Target logit regret r_i is an adequate proxy for downstream task impact.
    The method's safety depends on this proxy; the paper itself states it is 'an accounting quantity' not a guarantee, and only audits quality on limited benchmarks.
  • ad hoc to paper Remaining proposal length q_i is a useful proxy for suffix-reuse opportunity.
    Used in the gate r_i/q_i <= g; the paper admits it is a heuristic and not an optimality claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Approximate Speculative Decoding." pith.science (2026). https://pith.science/paper/NF43OYFA

@misc{pith2026260803447,
  author       = {Pith},
  title        = {Pith review of: Approximate Speculative Decoding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NF43OYFA}},
  note         = {Machine review of arXiv:2608.03447}
}
abstract

Speculative decoding accelerates autoregressive generation by verifying a draft block with a target model in parallel. Under standard greedy verification, decoding stops at the first draft token that differs from the target argmax, discarding the remaining target-scored suffix. Although accepting such a mismatch changes the decoding trajectory, it can make a contiguous suffix reusable when its tokens remain target-greedy under the realized prefix. In this paper, we introduce \textbf{Approximate Speculative Decoding (ASD)}, a training-free verifier that replaces binary first-mismatch truncation with budgeted longest-prefix selection. ASD accepts selected mismatches subject to a local target-logit regret gate, a per-block exception cap, and a persistent request-level regret budget, then reuses the contiguous target-greedy suffix without additional approximate decisions or target-model forward passes. ASD requires neither a new draft model nor fine-tuning, and exactly reduces to standard greedy verification when the budget is zero. Experiments show that ASD improves fixed-workload throughput by $3.05\%$--$15.26\%$ over matched strict verification and averages a $7.78\%$ gain across seven Qwen3-14B + DSpark-14B tasks. On DeepSeek-V4-Flash (284B) with DSpark it also raises verifier-side acceptance by roughly $10\%$--$16\%$ on GSM8K and MATH-500 in an FP4-to-FP8 compatibility setting. The source code is publicly available at: https://github.com/Kissmetothemoon/ASD

Figures

Figures reproduced from arXiv: 2608.03447 by the authors.

Figure 1
Figure 1. ASD verification. Strict verification stops at mis￾match E. ASD accepts E within budget, reuses the target￾greedy suffix F, G under the realized prefix, and recovers at H after the same target pass. whose draft tokens exactly match the target argmax, stops at the first mismatch, and appends a target recovery token. This rule exactly preserves the target greedy trajectory, but it is in￾herently binary: a nearly tied … view at source ↗
Figure 2
Figure 2. Different token paths can yield the same task-level answer. In this example, distinct reasoning trajectories produce the same normalized final answer. This motivates a separate task-quality audit, but does not imply that any approximate trajectory is individually exact or safe. and 48 × 37, and answer formats such as “1776,” “1,776,” and \boxed{1776} can preserve the same result. However, token mismatch is only an i… view at source ↗
Figure 4
Figure 4. Preliminary speed–quality frontier. Budget and cap move the discovery trade-off, motivating joint local and request-level controls. The frontier selects a conservative configuration; it is not a holdout result. where M is the set of draft–target mismatch positions over the evaluated blocks [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Regret-gate sweep on GSM8K under natural-EOS [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: ASD ablations with Qwen3-14B + DSpark-14B: [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 36 canonical work pages

  1. [1]

    Advances in Neural Information Processing Systems (

    Blockwise Parallel Decoding for Deep Autoregressive Models , author =. Advances in Neural Information Processing Systems (. 2018 , eprint =

    work page 2018

  2. [2]

    International Conference on Machine Learning (

    Fast Inference from Transformers via Speculative Decoding , author =. International Conference on Machine Learning (. 2023 , eprint =

    work page 2023

  3. [3]

    arXiv preprint arXiv:2302.01318 , year =

    Accelerating Large Language Model Decoding with Speculative Sampling , author =. arXiv preprint arXiv:2302.01318 , year =

    arXiv

  4. [4]

    2024 , doi =

    Miao, Xupeng and Oliaro, Gabriele and Zhang, Zhihao and Cheng, Xinhao and Wang, Zeyu and Zhang, Zhengxin and Wong, Rae Ying Yee and Zhu, Alan and Yang, Lijie and Shi, Xiaoxiang and Shi, Chunan and Chen, Zhuoming and Arfeen, Daiyaan and Abhyankar, Reyna and Jia, Zhihao , booktitle =. 2024 , doi =

    work page 2024

  5. [5]

    and Chen, Deming and Dao, Tri , booktitle =

    Cai, Tianle and Li, Yuhong and Geng, Zhengyang and Peng, Hongwu and Lee, Jason D. and Chen, Deming and Dao, Tri , booktitle =. 2024 , eprint =

    work page 2024

  6. [6]

    2024 , eprint =

    Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang , booktitle =. 2024 , eprint =

    2024

  7. [7]

    Break the Sequential Dependency of

    Fu, Yichao and Bailis, Peter and Stoica, Ion and Zhang, Hao , booktitle =. Break the Sequential Dependency of. 2024 , eprint =

    work page 2024

  8. [8]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =

    Speculative Streaming: Efficient and Scalable Speculative Decoding with Multi-Stream Attention , author =. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages =. 2025 , doi =

    work page 2025

Show all 42 references
  1. [9]

    Advances in Neural Information Processing Systems (

    Sequoia: Scalable and Robust Speculative Decoding , author =. Advances in Neural Information Processing Systems (. 2024 , doi =. 2402.12374 , archivePrefix =

    2024 arXiv

  2. [10]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , doi =

    2024

  3. [11]

    Hydra: Sequentially-Dependent Draft Heads for

    Ankner, Zachary and Parthasarathy, Rishab and Nrusimha, Aniruddha and Rinard, Christopher and Ragan-Kelley, Jonathan and Brandon, William , journal =. Hydra: Sequentially-Dependent Draft Heads for

  4. [12]

    arXiv preprint arXiv:2403.09919 , year =

    Recurrent Drafter for Fast Speculative Decoding in Large Language Models , author =. arXiv preprint arXiv:2403.09919 , year =

    arXiv

  5. [13]

    Kangaroo: Lossless Self-Speculative Decoding for Accelerating

    Liu, Fangcheng and Tang, Yehui and Liu, Zhenhua and Ni, Yunsheng and Tang, Duyu and Han, Kai and Wang, Yunhe , booktitle =. Kangaroo: Lossless Self-Speculative Decoding for Accelerating. 2024 , doi =. 2404.18911 , archivePrefix =

    2024 arXiv

  6. [14]

    2024 , doi =

    Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang , booktitle =. 2024 , doi =

    2024

  7. [15]

    2025 , doi =

    Wang, Jikai and Su, Yi and Li, Juntao and Xia, Qingrong and Ye, Zi and Duan, Xinyu and Wang, Zhefeng and Zhang, Min , journal =. 2025 , doi =

    2025

  8. [16]

    2025 , doi =

    Xiong, Yunfan and Zhang, Ruoyu and Li, Yanzeng and Zou, Lei , journal =. 2025 , doi =

    2025

  9. [17]

    and Keutzer, Kurt and Gholami, Amir , booktitle =

    Tiwari, Rishabh and Xi, Haocheng and Tomar, Aditya and Hooper, Coleman and Kim, Sehoon and Horton, Maxwell and Najibi, Mahyar and Mahoney, Michael W. and Keutzer, Kurt and Gholami, Amir , booktitle =. 2025 , eprint =

    2025

  10. [18]

    2025 , eprint =

    Li, Yuhui and Wei, Fangyun and Zhang, Chao and Zhang, Hongyang , booktitle =. 2025 , eprint =

    2025

  11. [19]

    International Conference on Learning Representations (

    Judge Decoding: Faster Speculative Sampling Requires Going Beyond Model Alignment , author =. International Conference on Learning Representations (. 2025 , eprint =

    2025

  12. [20]

    arXiv preprint arXiv:2510.26527 , year =

    Polybasic Speculative Decoding Through a Theoretical Perspective , author =. arXiv preprint arXiv:2510.26527 , year =

  13. [21]

    Tan, Zhendong and Zhang, Xingjun and Hu, Chaoyi and Peng, Junjie and Xia, Kun , journal =

  14. [22]

    arXiv preprint arXiv:2512.21911 , year =

    Accelerate Speculative Decoding with Sparse Computation in Verification , author =. arXiv preprint arXiv:2512.21911 , year =

  15. [23]

    Findings of the Association for Computational Linguistics:

    Speculative Decoding with a Speculative Vocabulary , author =. Findings of the Association for Computational Linguistics:. 2026 , doi =

    2026

  16. [24]

    International Conference on Learning Representations (

    Speculative Speculative Decoding , author =. International Conference on Learning Representations (. 2026 , eprint =

    2026

  17. [25]

    Cheng, Xin and Yu, Xingkai and Shao, Chenze and Li, Jiashi and Xiong, Yunfan and Qian, Yi and Zhu, Jiaqi and Ma, Shirong and Zhang, Xiaokang and Ye, Jiasheng and Chen, Qinyu and Deng, Chengqi and Yu, Jiping and Dai, Damai and Zhang, Zhengyan and Wei, Yixuan and Tan, Yixuan and...

  18. [26]

    International Conference on Learning Representations (

    Block Verification Accelerates Speculative Decoding , author =. International Conference on Learning Representations (. 2025 , eprint =

    2025

  19. [27]

    Findings of the Association for Computational Linguistics:

    Fuzzy Speculative Decoding for a Tunable Accuracy-Runtime Tradeoff , author =. Findings of the Association for Computational Linguistics:. 2025 , doi =

    2025

  20. [28]

    arXiv preprint arXiv:2512.13194 , year =

    Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models , author =. arXiv preprint arXiv:2512.13194 , year =

  21. [29]

    International Conference on Learning Representations (

    Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling , author =. International Conference on Learning Representations (. 2026 , eprint =

    2026

  22. [30]

    2026 , doi =

    Song, Jingwei and Wang, Xinyu and Wang, Hanbin and Lei, Xiaoxuan and Shi, Bill and Han, Shixin and Yang, Eric and Chang, Xiao-Wen and Ai, Lynn , booktitle =. 2026 , doi =

    2026

  23. [31]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Talon: Breaking the Synchronization Barrier in Speculative Decoding with Hybrid Model-based and Retrieve-based Drafting , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  24. [32]

    2026 , doi =

    Do, Dinh-Truong and Le, Nguyen-Khang and Nguyen, Le-Minh , booktitle =. 2026 , doi =

    2026

  25. [33]

    Shi, Luohe and Li, Zuchao and Zhang, Lefei and Qi, Baoyuan and Liu, Guoming and Zhao, Hai , booktitle =. Scaling. 2026 , doi =

    2026

  26. [34]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Steering Pretrained Drafters During Speculative Decoding , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  27. [35]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Annealed Relaxation of Speculative Decoding for Faster Autoregressive Image Generation , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  28. [36]

    2026 , doi =

    Pan, Jiayi and Xu, Jiaming and Zhou, Yongkang and Dai, Guohao , booktitle =. 2026 , doi =

    2026

  29. [37]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Inductive Generative Recommendation Via Retrieval-based Speculation , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  30. [38]

    and Chen, Haipeng , booktitle =

    Wang, Chenan and Shi, Daniel H. and Chen, Haipeng , booktitle =. 2026 , doi =

    2026

  31. [39]

    and Dou, Wanchun , booktitle =

    Tian, Hao and Lu, Sheng and Tian, Fuwen and Cui, Guangming and Li, Zheng and Zhang, Xuyun and Sheng, Quan Z. and Dou, Wanchun , booktitle =. 2026 , doi =

    2026

  32. [40]

    Dynamic-Width Speculative Beam Decoding for

    Qin, Zongyue and He, Zifan and Prakriya, Neha and Cong, Jason and Sun, Yizhou , booktitle =. Dynamic-Width Speculative Beam Decoding for. 2025 , doi =

    2025

  33. [41]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Adaptive Draft-Verification for Efficient Large Language Model Decoding , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2025 , doi =

    2025

  34. [42]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Falcon: Faster and Parallel Inference of Large Language Models Through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2025 , doi =

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.