Decoupled Contrastive Decoding via Expert-Aligned Drafting

Chao Yang; Yuanfu Wang; Zhichen Dong; Zhixuan Liu

REVIEW 2 major objections 6 minor 65 references

Decoupled Contrastive Decoding via Expert-Aligned Drafting

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Decoupled Contrastive Decoding achieves vanilla contrastive-decoding output at 1.65–1.95x lower serial cost by keeping the amateur model out of the draft loop.

desk verdict Useful systems paper with a clean decoupling idea, but the load-bearing losslessness claim is not proven at the greedy operating point that produces the headline 1.65–1.95x speedups. read the letter →

arxiv 2608.12913 v1 pith:PTVKMAZC submitted 2026-08-13 cs.CL

Zhixuan Liu , Zhichen Dong , Yuanfu Wang , Chao Yang This is my paper · ORCID

classification cs.CL

keywords contrastivedecodingspeculativedecoupledexpert-aligneddraftinglightweightproposerEAGLE3draft-then-verifyacceptedlength

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Contrastive decoding improves LLM output by penalizing tokens the large "expert" model dislikes and the small "amateur" likes, but it costs an extra model pass per token. This paper asks whether, when accelerating CD with speculative decoding, the contrastive signal should shape the draft or only score it. Using two controlled diagnostics, the paper argues that a lightweight drafter trained on the contrastive distribution gains nothing consistent over one trained on the expert, because the contrastive correction is usually smaller than the drafter's own error. The proposed method, Decoupled Contrastive Decoding (DCD), drafts with an expert-aligned proposer and applies the amateur only in unchanged verification, so standard speculative verification preserves vanilla CD's output distribution. The reported payoff is a 1.65–1.95x greedy speedup over vanilla CD across three 8B families, with 5–12x lower proposal-path latency on MMLU.

What carries the argument

The carrying object is the decoupled proposal–verification split, formalized as DCD's one-round loop: an amateur-independent proposer $E$ (EAGLE3, a feature-level drafter that predicts hidden states rather than tokens) draws $\gamma$ candidate tokens, and the expert and amateur models run parallel forward passes so each candidate is accepted with probability $\min(1, \pi_{\mathrm{CD}}(y_i)/\pi_e(y_i))$, with a fallback sample from the normalized contrastive distribution on rejection. The contrastive target is the standard CD product $\pi_{\mathrm{CD}}(x|h) \propto \pi_p(x|h)^{1+\alpha} \pi_q(x|h)^{-\alpha}$, which separates into an expert term and the contrastive factor $(\pi_p/\pi_q)^\alpha$. Two theoretical pieces carry the argument: the reconstruction-error decomposition $\Delta(x) = (1+\alpha)\epsilon_p(x) - \alpha\epsilon_q(x)$, showing why inference-time recombination amplifies drafter errors, and the KL-slope inequality proving that under effective draft alignment—the drafter places more mass than the amateur on tokens where the expert has a log-likelihood advantage—an expert-aligned drafter's distance to $\pi_{\mathrm{CD}}$ grows more slowly with $\alpha$ than the amateur's.

What would settle it

Train a higher-capacity contrastive-aware drafter on the CD target and compare mean accepted length against expert-aligned EAGLE3 in an online rollout on GSM8K and MMLU; if the contrastive-aware drafter wins in a majority of settings at 1000 samples, DCD's central diagnosis is overturned.

Watch

Extended reading notes

Core claim

The central claim is that the contrastive signal should stay in verification and never enter the lightweight proposal path. Concretely, DCD runs an expert-aligned feature-level proposer (EAGLE3) to draft $\gamma$ tokens, then verifies those tokens in parallel against the unchanged contrastive distribution $\pi_{\mathrm{CD}}(x) \propto \pi_p(x)^{1+\alpha} \pi_q(x)^{-\alpha}$; the proposer changes only the acceptance rate, never the output distribution. The paper supports the decoupling with a position-level diagnosis: across 24 configurations, 81.1% of positions have a contrastive signal below 1.0 while 48.7% have expert-side proposal KL error at or above 2.0, so the correction a contrastive-aware drafter must model is usually weaker than the drafter error it must overcome. In matched training, neither direct training on the contrastive target nor an approximate dual-drafter decomposition beats expert-aligned drafting in accepted length, and the decomposed route amplifies expert-side error by $1+\alpha$. The deployment result follows: EAGLE3-based DCD reaches 1.65–1.95x average greedy speedup over vanilla CD across three 8B families, stays close to 2x in a greedy 70B extension, and preserves the vanilla-CD output distribution exactly.

Load-bearing premise

The load-bearing premise is that no lightweight contrastive-aware drafter can consistently beat an expert-aligned drafter, which the paper supports with single-run matched experiments on 200 samples and an offline diagnosis using true amateur logits rather than an online rollout.

Editorial extensions

If this is right

Contrastive decoding can be made cheap without retraining the amateur: any amateur-independent proposer—EAGLE3, an N-gram matcher, or a future expert-aligned drafter—can be dropped into DCD's verification rule and the vanilla-CD output distribution is preserved.
If DCD's diagnostics hold, the amateur's role in CD is verification-time correction, not generation, so the serial draft path no longer depends on the amateur model's latency.
At $\alpha=0.5$, where amateur-coupled SCD and CoS fall below 1.0x on hard settings such as Qwen3 MMLU and CNN/DM, DCD EAGLE3 still delivers 1.39–1.60x, so the speed advantage widens as the contrastive penalty strengthens.
Because the target distribution is unchanged, DCD's task-level accuracy tracks vanilla CD across $\alpha$ (GSM8K greedy sweep), so the speedup is pure latency reduction rather than a new decoding objective.
A greedy-only 70B extension maintains the trend, with DCD EAGLE3 averaging 2.02x at $\alpha=0.1$ and 1.96x at $\alpha=0.5$ over vanilla CD.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The 81.1% low-signal statistic suggests a design space the paper does not explore: a drafter that conditions on whether the current position is a high-signal, low-error cell—e.g., via a learned gate or position-adaptive $\alpha$—could outperform both expert-aligned and contrastive-aware routes.
Since losslessness holds for any proposer, DCD's gains compound with future expert-aligned proposers; tree-based or self-speculative drafters could plausibly push speedups beyond the reported 1.95x without touching the amateur.
The paper's framing implies the amateur in CD might be replaceable by a model that is not a good generator but a good scorer of expert-preferred tokens, potentially shrinking the memory and KV-cache footprint of contrastive verification.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a paper worth arguing with. The DCD design is clean and the deployment numbers look real, but the headline losslessness claim is not proven at the operating point they actually run. The experiments use greedy chain drafting (topk=1), which selects the argmax of the EAGLE3 head, while Algorithm 1 and Appendix B.3 assume the draft token was sampled from π_e. The acceptance rule in lines 10–13 is the correct one only for a sample from π_e; applied to an argmax token it can accept a non-CD-argmax token or over-accept, so the output distribution need not match vanilla CD greedy. If the implementation uses a different greedy verification rule—say accept only when the draft token is the argmax of π_CD—they need to state and prove it. As written, the proof in B.3 is about sampling from π_CD, not about the greedy regime that carries the 1.65–1.95x numbers.

What is genuinely good: the proposal-alignment question is well framed and the two diagnostics, matched Cross-α and approximate dual-drafter, are honest attempts to test whether contrastive-aware lightweight drafting pays off. The negative result is useful even if it only covers a narrow regime. The deployment methodology is careful: three-run averages, standard deviations, matched hardware, and a best-γ check for baselines. DCD-NGRAM as a proposer-agnostic sanity check is a nice touch.

The other soft spots are minor by comparison. The 81.1% signal statistic comes from 200-sample single-run matched controls with no error bars, so the general claim that contrastive-aware drafting never helps should be read as conditional on these specific lightweight drafters. The 3B stress test is offline, as the paper says.

Bottom line: the speedups are plausible, but the central correctness claim is not yet established for the setting that matters. A serious referee should engage; the fix is probably not hard—either sample from π_e during deployment, or define and prove a greedy-specific verification rule. Right now I would be cautious about citing the paper as a lossless accelerator. The reader's conditional verdict is fair but underplays this greedy issue; I think it is the load-bearing point.

Referee Report

2 major / 6 minor

Summary. The paper studies how to accelerate Contrastive Decoding (CD) with speculative decoding, focusing on where the amateur model's signal should enter the pipeline: the proposal path, the training target of a lightweight drafter, or only the verification step. Two controlled diagnostics are presented—a dual-input EAGLE variant trained on the contrastive residual (Cross-alpha) and an Approximate Dual-Drafter that recombines separately learned expert and amateur heads—and the authors report that contrastive-aware lightweight drafting does not consistently improve accepted length over expert-aligned drafting. They then propose Decoupled Contrastive Decoding (DCD), which keeps an expert-aligned lightweight proposer in the serial path and applies the amateur only in unchanged verification against the normalized CD distribution pi_CD, claiming the standard speculative-sampling losslessness guarantee. Deployment experiments instantiate DCD with EAGLE3 and a matched N-gram proposer, reporting average greedy speedups of 1.65-1.95x over vanilla CD across three 8B-class model families and a 5-12x reduction in MMLU proposal-path latency; appendices include T=1 tables, a 70B greedy extension, multi-request serving, Spec-Bench, and a latency decomposition. A short algebraic analysis (Theorem B.3) shows that, under an empirically checked 'effective draft alignment' condition, an expert-aligned drafter's KL distance to pi_CD grows more slowly with alpha than the amateur's.

Significance. If the losslessness claim holds at the reported operating points, DCD is a practically valuable and conceptually clean result: it shows that vanilla CD can be accelerated with off-the-shelf expert-aligned drafters and no per-amateur retraining, and it makes explicit a proposal-alignment design choice (amateur-coupled versus contrastive-aware versus expert-aligned routes) that prior CD-acceleration work did not isolate. The experimental discipline is a genuine strength: three-run averages with standard deviations, a matched SGLang stack and hardware, a best-gamma check for the baselines, released code, and an honest limitations section. Theorem B.3 is a falsifiable, empirically checked condition rather than a fitted one, and Table 9 provides direct evidence for it. The principal correctness risk is the gap between the sampling-based losslessness proof and the greedy deployment mode that carries the headline speedups; this is addressable in revision and does not undermine the diagnostic evidence or the latency-decomposition argument.

major comments (2)

[Section 3 (Shared setting), Algorithm 1, Appendix B.3, Appendix H] Algorithm 1 computes the acceptance probability min(1, pi_CD(y_i)/pi_e(y_i)) and the residual Normalize(max(0, pi_CD - pi_e)), which is the correct lossless procedure only when the draft token y_i is actually drawn from pi_e (lines 4, 10, and 13). The deployment that produces the headline greedy speedups uses 'chain-style, greedy drafting' with speculative-eagle-topk=1 (Section 3, Shared setting; Appendix H, text and Table 20), meaning drafts are the argmax of the EAGLE3 head, a point mass, rather than samples from pi_e. Applied to argmax-selected tokens, the ratio can accept a token that is not the argmax of pi_CD and can reject the true CD argmax and replace it with a sample from the residual, so the outputs need not equal vanilla-CD greedy outputs. Theorem B.4 and Remark B.5 prove losslessness only for the sampling-based procedure, not for this greedy operating point, so the paper's central claim that standard speculative verification preserves the vanilla-CD output distribution is currently unsupported for the reported 1.65-1.95x greedy speedups. Please state the exact verification rule executed in the deployed greedy mode; if it is a deterministic rule (e.g., accept only when the draft token equals argmax pi_CD), state and prove that rule, and if it is Algorithm 1 as written, either report speedups with sampling-based drafting or explicitly restrict the lossless claim to T=1 and re-characterize the greedy results as approximate, supported by the consistency check in Figure 4.
[Section 3.1, Tables 1 and 6, Figure 9] The negative diagnosis—that contrastive-aware lightweight drafting does not reliably beat expert-aligned drafting—is the stated basis for the decoupled design and for contribution (2), but the supporting measurements are single-run traces on 200 samples with no variance estimates. In Table 1 the only favorable cell is an average Delta-L of +0.015 (0.77%) on Llama-3/GSM8K, which is within plausible run-to-run noise, and the Approximate Dual-Drafter gaps in Table 6 are likewise unreplicated. Figure 9's stronger-3B stress test is explicitly offline (true amateur logits on shared histories) and gives setting-level mean Delta-Top-1 values between -0.0144 and -0.0028, essentially indistinguishable from zero. The 'tested regime' hedge in the Section 3.1 takeaway is appropriate, but the abstract and Section 1 present the result as settled. Please add error bars or a small number of independent seeds for the matched experiments, or soften the abstract and introduction to match the strength of the evidence.

minor comments (6)

[Abstract] The phrase 'We introduceDecoupled Contrastive Decoding' is missing a space and should read 'We introduce Decoupled Contrastive Decoding'.
[Section 3.1, Figures 2 and 9, Table 7] The 'signal is weaker than the error' comparison mixes a pointwise absolute log-ratio, |log(pi_p(x*)/pi_q(x*))|, with a full-vocabulary KL divergence, D_KL(pi_e || pi_p); these are not unit-comparable quantities, so the 81.1%-versus-48.7% contrast should be labeled as a heuristic rather than as a measured comparison of comparable magnitudes.
[Appendix B.1, Theorem B.3] The text under 'Proof of Theorem B.3' begins with the informal remark 'Empirically, SCD often has a higher acceptance rate at alpha=0 ...' before the formal derivation begins; move this narrative out of the proof block so that the theorem statement and proof are cleanly separated.
[Section 3.4, Figure 4] The caption of Figure 4 does not state the sample size; the underlying sweep uses 1000 examples per setting (Appendix E.2), which should be noted in the caption.
[Appendix H] The terms 'chain-style, greedy drafting' and speculative-eagle-topk=1 are used throughout the main text but are defined only in Appendix H; a one-sentence definition should appear in the Section 3 shared settings so the main text is self-contained.
[Section 3, Related Work] The abbreviation CoS is used without expansion at first use; the method name (Collaborative Speculation, per Fu et al., 2025) should be introduced explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DCD's lossless claim applies an external speculative-decoding theorem to an independently defined CD target, and the supporting diagnostics are matched experiments rather than fitted predictions.

full rationale

The paper's central claim is that DCD is a lossless accelerator of vanilla CD because verification still targets the independently defined distribution pi_CD(x) proportional to pi_p(x)^(1+alpha) pi_q(x)^(-alpha), while only the serial proposal path changes. Appendix B.3 (Theorem B.4) invokes the externally proven standard speculative-decoding losslessness result (Leviathan et al., 2023; Chen et al., 2023) and instantiates it with pi_target = pi_CD; this is an application of an external theorem to a target that is not defined in terms of the drafter, so it is not circular. The Section 3.1 negative result on contrastive-aware drafting rests on matched single-run experiments and an explicitly labeled offline diagnostic, not on a fitted parameter that is later reported as a prediction. The alpha-robustness theorem (Theorem B.3) is a conditional algebraic consequence of the explicitly stated Effective Draft Alignment condition (Definition B.2), which is separately measured in Appendix C; the condition is not chosen to force the conclusion. No parameter is fitted to a subset and then renamed as a prediction, and no load-bearing self-citation or author-imported uniqueness theorem appears. The one flagged concern, that Algorithm 1 samples draft tokens from pi_e while deployment uses chain-style greedy drafting with speculative-eagle-topk=1, is a correctness and assumptions gap in the lossless proof's conditions rather than a circular reduction: the target distribution is defined independently of the proposer, and the reported speedups are measured empirically rather than derived from the lossless claim.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard speculative decoding theory and empirical measurements. No target-fitted constants are introduced; alpha and draft length are operating hyperparameters. The only hand-set constant is the CoS calibration parameter rho_p=0.9 in an appendix throughput model, which does not feed into any central result.

free parameters (1)

CoS calibration constant rho_p = 0.9 (hand-set)
Used only in the analytic CoS throughput surrogate in Appendix D.3; does not affect any measured speedup or the lossless proof.

assumptions (4)

standard math Standard speculative decoding is lossless for any valid normalized target distribution.
Invoked in Appendix B.3 (Theorem B.4) to establish that DCD preserves the vanilla-CD distribution; the result is cited to Leviathan et al. (2023) and Chen et al. (2023).
domain assumption The contrastive target pi_CD defined in Eq. (1) is a valid normalized probability distribution for the softmax experts and amateurs used.
Section 2.1 defines pi_CD with normalization Z_CD; the lossless proof relies on it being a valid target, and Appendix B.3 explicitly checks positivity.
domain assumption Effective Draft Alignment (Definition B.2) holds in the evaluated settings.
Theorem B.3's slope comparison depends on this condition; the paper estimates the gap in Table 9 and finds positive gaps in four settings, but it is an assumption about the draft distributions.
domain assumption The matched-control diagnostics isolate the effect of train-alpha because architecture, data, and training recipe are fixed.
Section 3.1 and Appendix A.2 use this to attribute differences in accepted length to the training target rather than to other variables.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Decoupled Contrastive Decoding via Expert-Aligned Drafting." pith.science (2026). https://pith.science/paper/PTVKMAZC

@misc{pith2026260812913,
  author       = {Pith},
  title        = {Pith review of: Decoupled Contrastive Decoding via Expert-Aligned Drafting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PTVKMAZC}},
  note         = {Machine review of arXiv:2608.12913}
}

read the original abstract

Contrastive Decoding (CD) improves generation quality, but its amateur-model pass makes decoding expensive. Accelerating CD with speculative decoding raises a proposal-alignment question: should the contrastive signal shape the drafter, or should it remain only in verification? We study this question in the lightweight feature-level drafter regime. Two controlled diagnostics, matched Cross-alpha training and an Approximate Dual-Drafter decomposition, give the same diagnosis: contrastive-aware drafting does not consistently improve over expert-aligned drafting because the contrastive correction is usually weaker than drafter error, and reconstruction can amplify that error. We introduce Decoupled Contrastive Decoding (DCD), which drafts with an expert-aligned lightweight proposer and applies the amateur only in unchanged CD verification. Standard speculative verification preserves the vanilla-CD output distribution. Across the main 8B settings, EAGLE3-based DCD achieves average greedy speedups of 1.65 to 1.95x over vanilla CD and reduces MMLU proposal-path latency by about 5 to 12x relative to amateur-coupled proposal paths.

Figures

Figures reproduced from arXiv: 2608.12913 by the authors.

**Figure 2.** Contrastive-aware lightweight drafting has only a small favorable region to exploit. Most positions have [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Even a stronger same-family proposer leaves [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: DCDEAGLE3 tracks vanilla CD accuracy across α on Llama-3 GSM8K (γ=5). Error bars are 95% confidence intervals; the dashed line is AR. rate: SCD or CoS can start from a higher L at α=0, while the diagnostic tracks how quickly each proposal distribution drifts away fro…

**Figure 5.** Figure 5: DCDEAGLE3 loses accepted length more mildly as contrastive strength increases. Relative mean accepted length is normalized to α=0 in each panel. Method L te tp tq Speed Theo. Llama-3 (8B-Ins / 1B-Ins) CD 0.00 - 6.95 3.83 1.00x 1.00x SCD 2.45 - 6.84 3.47 1.34x 1.48x CoS…

**Figure 6.** Figure 6: Configuration-level ∆Top-1 across KL(πf ∥ πq) buckets. The association is heterogeneous across model groups and contrastive strengths: for α ≤ 0.5, KL-axis patterns are weak or mixed, whereas at α = 1.0 the gap often becomes less negative at higher KL. No configuration…

**Figure 7.** Figure 7: Compact regime heatmap for the expanded root-level contrastive-signal bucket analysis. Here contrastive [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

**Figure 8.** Figure 8: Per-group ∆Top-1 trends across contrastive-signal buckets. Here contrastive signal denotes the perposition expert–amateur absolute log-ratio [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

**Figure 9.** Figure 9: Full offline 3B proposer diagnostic. ∆Top-1 is the CD-aware reconstruction’s Top-1 agreement gain over the unmodified 3B proposer, measured against arg maxx πCD(x | h). Low-signal rows remain mostly negative; setting-level mean ∆Top-1 values are slightly negative in al…

**Figure 10.** Figure 10: Full task-accuracy sweep across contrastive strengths [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

**Figure 11.** Figure 11: Appendix-only γ ablation. The y-axis reports speedup relative to vanilla CD. DCDEAGLE3 stays efficient across a broader draft-length range, while the amateur-coupled baselines usually peak earlier as larger draft lengths make their heavier proposal paths less worthwhi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 30 canonical work pages

[1]

Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. 2024. MEDUSA : Simple LLM inference acceleration framework with multiple decoding heads. In Proceedings of the 41st International Conference on Machine Learning, pages 5209--5235

work page 2024
[3]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. https://arxiv.org/abs/2107.03374 Evaluating large lang...

arXiv 2021
[4]

Glass, and Pengcheng He

Yung - Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. https://openreview.net/forum?id=Th6NyL07na DoLa : Decoding by contrasting layers improves factuality in large language models . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

work page 2024
[6]

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole - Jean Wu. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.681 LayerSkip : Enabling early exit inference and self-speculative decoding . In Proceedings of the 62nd Annua...

doi:10.18653/v1/2024.acl-long.681 2024
[7]

Jiale Fu, Yuchu Jiang, Junkai Chen, Jiaming Fan, Xin Geng, and Xu Yang. 2025. Fast large language model collaborative decoding via speculation. In Forty-second International Conference on Machine Learning

work page 2025
[8]

Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. 2024. https://openreview.net/forum?id=eDjvSFOkXw Break the sequential dependency of LLM inference using lookahead decoding . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net

2024
[9]

Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, and Rui Yan. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.677 Graph-structured speculative decoding . In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024 , pages 11404--11415. As...

work page doi:10.18653/v1/2024.findings-acl.677 2024
[10]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

2021

Show all 65 references

[11]

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. https://doi.org/10.1145/3703155 A survey on hallucination in large language models: Principles, taxonomy, challenges, and o...

2025 doi
[12]

Youna Kim, Hyuhng Joon Kim, Cheonbok Park, Choonghyun Park, Hyunsoo Cho, Junyeob Kim, Kang Min Yoo, Sang - goo Lee, and Taeuk Kim. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-EMNLP.136 Adaptive contrastive decoding in retrieval-augmented generation for handling noisy conte...

2024 doi
[13]

Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

2024
[14]

Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274--19286. PMLR

2023
[15]

Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori B Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. Contrastive decoding: Open-ended text generation as optimization. In Proceedings of the 61st Annual Meeting of the Association for Computational...

2023
[16]

Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024 a . EAGLE-2 : Faster inference of language models with dynamic draft trees. In Empirical Methods in Natural Language Processing

2024
[17]

Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024 b . EAGLE : Speculative sampling requires rethinking feature uncertainty. In International Conference on Machine Learning

2024
[18]

Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025. EAGLE-3 : Scaling up inference acceleration of large language models via training-time test. In Annual Conference on Neural Information Processing Systems

2025
[20]

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, Chunan Shi, Zhuoming Chen, Daiyaan Arfeen, Reyna Abhyankar, and Zhihao Jia. 2024. SpecInfer : Accelerating large language model serving ...

2024
[21]

Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D. Manning. 2024. https://openreview.net/forum?id=Eo7kv0sllr An emulator for fine-tuning large language models using small language models . In The Twelfth International Conference on Learning Represe...

2024
[25]

Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.937 Mitigating hallucinations in large vision-language models with instruction contrastive decoding . In Findings of the Association for Computational Linguistics, AC...

2024 doi
[27]

Hongyi Yuan, Keming Lu, Fei Huang, Zheng Yuan, and Chang Zhou. 2024. Speculative contrastive decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 56--64

2024
[28]

Jun Zhang, Jue Wang, Huan Li, Lidan Shou, Ke Chen, Gang Chen, and Sharad Mehrotra. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.607 Draft & verify: Lossless large language model acceleration via self-speculative decoding . In Proceedings of the 62nd Annual Meeting of the As...

2024 doi
[29]

Zheng Zhao, Emilio Monti, Jens Lehmann, and Haytham Assem. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.237 Enhancing contextual understanding in large language models through contrastive decoding . In Proceedings of the 2024 Conference of the North American Chapter of th...

2024 doi
[30]

Gonzalez, Clark W

Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. 2024. SGLang : Efficient execution of structured language model programs. Advances in Neural Infor...

2024
[31]

Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean - Fran c ois Kagy, and Rishabh Agarwal. 2024. https://openreview.net/forum?id=rsY6J3ZaTF DistillSpec : Improving speculative decoding via knowledge distillation . In Th...

2024
[32]

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

Xia, Heming and Yang, Zhe and Dong, Qingxiu and Wang, Peiyi and Li, Yongqi and Ge, Tao and Liu, Tianyu and Li, Wenjie and Sui, Zhifang. Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding. Findings of the Association for Compu...

2024 doi
[33]

International Conference on Machine Learning , pages=

Fast inference from transformers via speculative decoding , author=. International Conference on Machine Learning , pages=. 2023 , organization=

2023
[34]

arXiv preprint arXiv:2302.01318 , year=

Accelerating large language model decoding with speculative sampling , author=. arXiv preprint arXiv:2302.01318 , year=

arXiv
[35]

Cai, Tianle and Li, Yuhong and Geng, Zhengyang and Peng, Hongwu and Lee, Jason D and Chen, Deming and Dao, Tri , booktitle=
[36]

Miao, Xupeng and Oliaro, Gabriele and Zhang, Zhihao and Cheng, Xinhao and Wang, Zeyu and Zhang, Zhengxin and Wong, Rae Ying Yee and Zhu, Alan and Yang, Lijie and Shi, Xiaoxiang and Shi, Chunan and Chen, Zhuoming and Arfeen, Daiyaan and Abhyankar, Reyna and Jia, Zhihao , booktitle=
[37]

Forty-first International Conference on Machine Learning,

Yichao Fu and Peter Bailis and Ion Stoica and Hao Zhang , title =. Forty-first International Conference on Machine Learning,. 2024 , url =

2024
[38]

Advances in Neural Information Processing Systems , volume=

Confident adaptive language modeling , author=. Advances in Neural Information Processing Systems , volume=
[39]

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

Speculative contrastive decoding , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=
[40]

Forty-second International Conference on Machine Learning , year=

Fast Large Language Model Collaborative Decoding via Speculation , author=. Forty-second International Conference on Machine Learning , year=
[41]

Proceedings of machine learning and systems , volume=

Awq: Activation-aware weight quantization for on-device llm compression and acceleration , author=. Proceedings of machine learning and systems , volume=
[42]

International conference on machine learning , pages=

Smoothquant: Accurate and efficient post-training quantization for large language models , author=. International conference on machine learning , pages=. 2023 , organization=

2023
[43]

2023 , journal=

A Simple and Effective Pruning Approach for Large Language Models , author=. 2023 , journal=

2023
[44]

arXiv preprint arXiv:1503.02531 , year=

Distilling the knowledge in a neural network , author=. arXiv preprint arXiv:1503.02531 , year=

arXiv
[45]

Reasoning with Large Language Models, a Survey , journal =

Aske Plaat and Annie Wong and Suzan Verberne and Joost Broekens and Niki van Stein and Thomas B. Reasoning with Large Language Models, a Survey , journal =. 2024 , url =. doi:10.48550/ARXIV.2407.11511 , eprinttype =

2024 doi
[46]

2025 , url =

Lei Huang and Weijiang Yu and Weitao Ma and Weihong Zhong and Zhangyin Feng and Haotian Wang and Qianglong Chen and Weihua Peng and Xiaocheng Feng and Bing Qin and Ting Liu , title =. 2025 , url =

2025
[47]

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),

Mostafa Elhoushi and Akshat Shrivastava and Diana Liskovich and Basil Hosmer and Bram Wasti and Liangzhen Lai and Anas Mahmoud and Bilge Acun and Saurabh Agarwal and Ahmed Roman and Ahmed A Aly and Beidi Chen and Carole. Proceedings of the 62nd Annual Meeting of the Associatio...

2024
[48]

The Twelfth International Conference on Learning Representations,

Yongchao Zhou and Kaifeng Lyu and Ankit Singh Rawat and Aditya Krishna Menon and Afshin Rostamizadeh and Sanjiv Kumar and Jean. The Twelfth International Conference on Learning Representations,. 2024 , url =

2024
[49]

Decoding-time Realignment of Language Models , booktitle =

Tianlin Liu and Shangmin Guo and Leonardo Bianco and Daniele Calandriello and Quentin Berthet and Felipe Llinares. Decoding-time Realignment of Language Models , booktitle =. 2024 , url =

2024
[50]

Jun Zhang and Jue Wang and Huan Li and Lidan Shou and Ke Chen and Gang Chen and Sharad Mehrotra , editor =. Draft. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),. 2024 , url =

2024
[51]

Graph-Structured Speculative Decoding , booktitle =

Zhuocheng Gong and Jiahao Liu and Ziyue Wang and Pengfei Wu and Jingang Wang and Xunliang Cai and Dongyan Zhao and Rui Yan , editor =. Graph-Structured Speculative Decoding , booktitle =. 2024 , url =

2024
[52]

Smith and Simon S

Ruizhe Shi and Yifang Chen and Yushi Hu and Alisa Liu and Hanna Hajishirzi and Noah A. Smith and Simon S. Du , editor =. Decoding-Time Language Model Alignment with Multiple Objectives , booktitle =. 2024 , url =

2024
[53]

arXiv preprint arXiv:2307.09288 , year=

Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

arXiv
[54]

arXiv e-prints , pages=

The llama 3 herd of models , author=. arXiv e-prints , pages=
[55]

arXiv preprint arXiv:2505.09388 , year=

Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

arXiv
[56]

and Barrett, Clark W

Zheng, Lianmin and Yin, Liangsheng and Xie, Zhiqiang and Sun, Chuyue and Huang, Jeff and Yu, Cody Hao and Cao, Shiyi and Kozyrakis, Christos and Stoica, Ion and Gonzalez, Joseph E. and Barrett, Clark W. and Sheng, Ying , journal=
[57]

2021 , eprint=

Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

2021
[58]

arXiv preprint arXiv:2110.14168 , year=

Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year=

arXiv
[59]

Proceedings of the International Conference on Learning Representations (ICLR) , year=

Measuring Massive Multitask Language Understanding , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=
[60]

and Manning, Christopher D

See, Abigail and Liu, Peter J. and Manning, Christopher D. Get To The Point: Summarization with Pointer-Generator Networks. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2017. doi:10.18653/v1/P17-1099

2017 doi
[61]

International Conference on Machine Learning , year =

Yuhui Li and Fangyun Wei and Chao Zhang and Hongyang Zhang , title =. International Conference on Machine Learning , year =
[62]

Empirical Methods in Natural Language Processing , year =

Yuhui Li and Fangyun Wei and Chao Zhang and Hongyang Zhang , title =. Empirical Methods in Natural Language Processing , year =
[63]

Annual Conference on Neural Information Processing Systems , year =

Yuhui Li and Fangyun Wei and Chao Zhang and Hongyang Zhang , title =. Annual Conference on Neural Information Processing Systems , year =
[64]

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

Contrastive decoding: Open-ended text generation as optimization , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
[65]

CoRR , volume =

Sean O'Brien and Mike Lewis , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2309.09117 , eprinttype =

2023 doi arXiv
[66]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

Mitigating object hallucinations in large vision-language models through visual contrastive decoding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=
[67]

Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding , booktitle =

Xintong Wang and Jingheng Pan and Liang Ding and Chris Biemann , editor =. Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding , booktitle =. 2024 , url =

2024
[68]

Smith , title =

Alisa Liu and Xiaochuang Han and Yizhong Wang and Yulia Tsvetkov and Yejin Choi and Noah A. Smith , title =. CoRR , volume =. 2024 , url =. doi:10.48550/ARXIV.2401.08565 , eprinttype =

2024 doi arXiv
[69]

Manning , title =

Eric Mitchell and Rafael Rafailov and Archit Sharma and Chelsea Finn and Christopher D. Manning , title =. The Twelfth International Conference on Learning Representations,. 2024 , url =

2024
[70]

The Twelfth International Conference on Learning Representations,

Yung. The Twelfth International Conference on Learning Representations,. 2024 , url =

2024
[71]

Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding , booktitle =

Zheng Zhao and Emilio Monti and Jens Lehmann and Haytham Assem , editor =. Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding , booktitle =. 2024 , url =

2024
[72]

Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts , booktitle =

Youna Kim and Hyuhng Joon Kim and Cheonbok Park and Choonghyun Park and Hyunsoo Cho and Junyeob Kim and Kang Min Yoo and Sang. Adaptive Contrastive Decoding in Retrieval-Augmented Generation for Handling Noisy Contexts , booktitle =. 2024 , url =

2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. 2024. MEDUSA : Simple LLM inference acceleration framework with multiple decoding heads. In Proceedings of the 41st International Conference on Machine Learning, pages 5209--5235

work page 2024

[2] [3]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. https://arxiv.org/abs/2107.03374 Evaluating large lang...

arXiv 2021

[3] [4]

Glass, and Pengcheng He

Yung - Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. https://openreview.net/forum?id=Th6NyL07na DoLa : Decoding by contrasting layers improves factuality in large language models . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

work page 2024

[4] [6]

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed A Aly, Beidi Chen, and Carole - Jean Wu. 2024. https://doi.org/10.18653/V1/2024.ACL-LONG.681 LayerSkip : Enabling early exit inference and self-speculative decoding . In Proceedings of the 62nd Annua...

doi:10.18653/v1/2024.acl-long.681 2024

[5] [7]

Jiale Fu, Yuchu Jiang, Junkai Chen, Jiaming Fan, Xin Geng, and Xu Yang. 2025. Fast large language model collaborative decoding via speculation. In Forty-second International Conference on Machine Learning

work page 2025

[6] [8]

Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. 2024. https://openreview.net/forum?id=eDjvSFOkXw Break the sequential dependency of LLM inference using lookahead decoding . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net

2024

[7] [9]

Zhuocheng Gong, Jiahao Liu, Ziyue Wang, Pengfei Wu, Jingang Wang, Xunliang Cai, Dongyan Zhao, and Rui Yan. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.677 Graph-structured speculative decoding . In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024 , pages 11404--11415. As...

work page doi:10.18653/v1/2024.findings-acl.677 2024

[8] [10]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

2021