{"id":"944225df-a5b2-4864-84fb-ee9822f0836e","arxiv_id":"2608.12913","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"DCD, an expert-aligned proposal path with unchanged contrastive verification, speeds up contrastive decoding by 1.65 to 1.95x on 8B models while preserving the vanilla-CD output distribution.","lead":"This paper asks whether the smaller model in contrastive decoding should help draft tokens or only check them. It finds that keeping the drafter expert-aligned and using the smaller model only in verification is faster and preserves the original output distribution.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Greedy-mode verification in Algorithm 1 is not the lossless check for argmax-drafted tokens, so the headline lossless claim is unproven for the reported greedy speedups.","rationale":"The reader's conditional verdict is reasonable, but I do not think the weakest assumption is the generality of the negative diagnostic in Section 3.1. Even if a stronger contrastive-aware drafter existed, DCD would still be a lossless accelerator of vanilla CD; it might simply not be the fastest possible proposer. The more load-bearing issue is internal to the central losslessness claim itself. The paper explicitly reports greedy speedups as its headline result, and it describes the deployment as using chain-style greedy drafting with top-k=1. Under greedy drafting, the EAGLE3 proposer emits argmax tokens, so the effective proposal distribution is degenerate. Algorithm 1's stochastic acceptance and residual sampling are valid only when draft tokens are drawn from the softmax distribution pi_e used in the acceptance ratio. If the released implementation follows Algorithm 1 in greedy mode, the outputs need not match vanilla CD greedy, and the 'standard speculative verification preserves the vanilla-CD output distribution' assertion is not established for the reported greedy numbers. If, instead, the implementation uses a deterministic argmax-match verification for T=0, then the pseudocode and proof are missing that rule, and the claim is unproven as written. This is directly testable: a lossless greedy accelerator must produce exactly the same output sequence as autoregressive vanilla CD greedy, and the code can reveal which acceptance rule is used. I therefore keep the reader's CONDITIONAL verdict, but the condition should be the greedy-verification consistency rather than only the sample size of the contrastive-aware diagnostic. I credit the paper for the clear sampling-mode losslessness argument, the reproducible deployment setup, and the extensive system measurements; the concern is not that the speedups are fabricated, but that the losslessness guarantee may not cover the exact operating point used for the headline claims.","tokens_in":31902,"tokens_out":10321,"duration_ms":114523,"concrete_test":"Inspect the released code's greedy branch and run an exact-output check. Concretely: (1) Determine whether EAGLE3 draft tokens in T=0 runs are generated by argmax or by sampling from the EAGLE3 softmax, and whether the verification step uses the same distribution in the acceptance ratio; (2) on 500 GSM8K prompts with a fixed seed, run DCD greedy and vanilla CD greedy with the same alpha and model pair, and compare the generated sequences token-by-token. A lossless greedy accelerator must produce identical sequences. Any divergence would mean the greedy speedups in Table 2 are not accompanied by the claimed losslessness; if the sequences match, report the exact greedy verification rule used and add it explicitly to Algorithm 1 and to the proof in Appendix B.3.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that DCD is lossless because verification targets pi_CD. That is true only if draft tokens are actually sampled from the proposer distribution pi_e used in the acceptance ratio, which is the standard speculative-sampling condition invoked in Appendix B.3. The deployment protocol, however, reports greedy speedups with 'chain-style, greedy drafting' and `speculative-eagle-topk=1` (Section 3, Appendix H). Greedy drafting selects the argmax of the EAGLE3 head, making the true proposal distribution a point mass rather than the softmax pi_e. Algorithm 1 nevertheless computes the stochastic acceptance probability min(1, pi_CD(y)/pi_e(y)) and the residual max(0, pi_CD - pi_e). This is the correct lossless procedure only when y is actually drawn from pi_e. Applied to argmax-selected tokens, it can accept a token that is not the CD argmax, or over-accept relative to the degenerate proposal, so the output distribution need not equal vanilla CD greedy. The proof in Appendix B.3 proves losslessness for sampling from pi_CD, not for the greedy operating point that carries the headline 1.65-1.95x speedups. If the implementation actually uses a separate greedy verification rule (e.g., acceptance only when the draft token equals argmax pi_CD), the paper should state and prove that rule; if it uses Algorithm 1 as written, the greedy losslessness claim is false.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies how to accelerate Contrastive Decoding (CD) with speculative decoding, focusing on where the amateur model's signal should enter the pipeline: the proposal path, the training target of a lightweight drafter, or only the verification step. Two controlled diagnostics are presented—a dual-input EAGLE variant trained on the contrastive residual (Cross-alpha) and an Approximate Dual-Drafter that recombines separately learned expert and amateur heads—and the authors report that contrastive-aware lightweight drafting does not consistently improve accepted length over expert-aligned drafting. They then propose Decoupled Contrastive Decoding (DCD), which keeps an expert-aligned lightweight proposer in the serial path and applies the amateur only in unchanged verification against the normalized CD distribution pi_CD, claiming the standard speculative-sampling losslessness guarantee. Deployment experiments instantiate DCD with EAGLE3 and a matched N-gram proposer, reporting average greedy speedups of 1.65-1.95x over vanilla CD across three 8B-class model families and a 5-12x reduction in MMLU proposal-path latency; appendices include T=1 tables, a 70B greedy extension, multi-request serving, Spec-Bench, and a latency decomposition. A short algebraic analysis (Theorem B.3) shows that, under an empirically checked 'effective draft alignment' condition, an expert-aligned drafter's KL distance to pi_CD grows more slowly with alpha than the amateur's.","tokens_in":32182,"tokens_out":16776,"duration_ms":158946,"significance":"If the losslessness claim holds at the reported operating points, DCD is a practically valuable and conceptually clean result: it shows that vanilla CD can be accelerated with off-the-shelf expert-aligned drafters and no per-amateur retraining, and it makes explicit a proposal-alignment design choice (amateur-coupled versus contrastive-aware versus expert-aligned routes) that prior CD-acceleration work did not isolate. The experimental discipline is a genuine strength: three-run averages with standard deviations, a matched SGLang stack and hardware, a best-gamma check for the baselines, released code, and an honest limitations section. Theorem B.3 is a falsifiable, empirically checked condition rather than a fitted one, and Table 9 provides direct evidence for it. The principal correctness risk is the gap between the sampling-based losslessness proof and the greedy deployment mode that carries the headline speedups; this is addressable in revision and does not undermine the diagnostic evidence or the latency-decomposition argument.","major_comments":[{"comment":"Algorithm 1 computes the acceptance probability min(1, pi_CD(y_i)/pi_e(y_i)) and the residual Normalize(max(0, pi_CD - pi_e)), which is the correct lossless procedure only when the draft token y_i is actually drawn from pi_e (lines 4, 10, and 13). The deployment that produces the headline greedy speedups uses 'chain-style, greedy drafting' with speculative-eagle-topk=1 (Section 3, Shared setting; Appendix H, text and Table 20), meaning drafts are the argmax of the EAGLE3 head, a point mass, rather than samples from pi_e. Applied to argmax-selected tokens, the ratio can accept a token that is not the argmax of pi_CD and can reject the true CD argmax and replace it with a sample from the residual, so the outputs need not equal vanilla-CD greedy outputs. Theorem B.4 and Remark B.5 prove losslessness only for the sampling-based procedure, not for this greedy operating point, so the paper's central claim that standard speculative verification preserves the vanilla-CD output distribution is currently unsupported for the reported 1.65-1.95x greedy speedups. Please state the exact verification rule executed in the deployed greedy mode; if it is a deterministic rule (e.g., accept only when the draft token equals argmax pi_CD), state and prove that rule, and if it is Algorithm 1 as written, either report speedups with sampling-based drafting or explicitly restrict the lossless claim to T=1 and re-characterize the greedy results as approximate, supported by the consistency check in Figure 4.","section":"Section 3 (Shared setting), Algorithm 1, Appendix B.3, Appendix H"},{"comment":"The negative diagnosis—that contrastive-aware lightweight drafting does not reliably beat expert-aligned drafting—is the stated basis for the decoupled design and for contribution (2), but the supporting measurements are single-run traces on 200 samples with no variance estimates. In Table 1 the only favorable cell is an average Delta-L of +0.015 (0.77%) on Llama-3/GSM8K, which is within plausible run-to-run noise, and the Approximate Dual-Drafter gaps in Table 6 are likewise unreplicated. Figure 9's stronger-3B stress test is explicitly offline (true amateur logits on shared histories) and gives setting-level mean Delta-Top-1 values between -0.0144 and -0.0028, essentially indistinguishable from zero. The 'tested regime' hedge in the Section 3.1 takeaway is appropriate, but the abstract and Section 1 present the result as settled. Please add error bars or a small number of independent seeds for the matched experiments, or soften the abstract and introduction to match the strength of the evidence.","section":"Section 3.1, Tables 1 and 6, Figure 9"}],"minor_comments":[{"comment":"The phrase 'We introduceDecoupled Contrastive Decoding' is missing a space and should read 'We introduce Decoupled Contrastive Decoding'.","section":"Abstract"},{"comment":"The 'signal is weaker than the error' comparison mixes a pointwise absolute log-ratio, |log(pi_p(x*)/pi_q(x*))|, with a full-vocabulary KL divergence, D_KL(pi_e || pi_p); these are not unit-comparable quantities, so the 81.1%-versus-48.7% contrast should be labeled as a heuristic rather than as a measured comparison of comparable magnitudes.","section":"Section 3.1, Figures 2 and 9, Table 7"},{"comment":"The text under 'Proof of Theorem B.3' begins with the informal remark 'Empirically, SCD often has a higher acceptance rate at alpha=0 ...' before the formal derivation begins; move this narrative out of the proof block so that the theorem statement and proof are cleanly separated.","section":"Appendix B.1, Theorem B.3"},{"comment":"The caption of Figure 4 does not state the sample size; the underlying sweep uses 1000 examples per setting (Appendix E.2), which should be noted in the caption.","section":"Section 3.4, Figure 4"},{"comment":"The terms 'chain-style, greedy drafting' and speculative-eagle-topk=1 are used throughout the main text but are defined only in Appendix H; a one-sentence definition should appear in the Section 3 shared settings so the main text is self-contained.","section":"Appendix H"},{"comment":"The abbreviation CoS is used without expansion at first use; the method name (Collaborative Speculation, per Fu et al., 2025) should be introduced explicitly.","section":"Section 3, Related Work"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is well within scope and is, on the whole, carefully executed: matched hardware, three-run averages with standard deviations, a best-gamma check for the baselines, and honest appendices. The decisive point for the final decision is the verification rule actually executed in the deployed greedy mode; I would ask the authors to confirm whether the SGLang greedy path uses argmax-equality acceptance rather than the stochastic ratio in Algorithm 1. If the former, the losslessness claim for the greedy speedups is true but must be stated and proved; if the latter, the claim must be restricted or the experiments rerun. The secondary concern about the variance of the Section 3.1 diagnostic is also worth requesting in revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a paper worth arguing with. The DCD design is clean and the deployment numbers look real, but the headline losslessness claim is not proven at the operating point they actually run. The experiments use greedy chain drafting (topk=1), which selects the argmax of the EAGLE3 head, while Algorithm 1 and Appendix B.3 assume the draft token was sampled from π_e. The acceptance rule in lines 10–13 is the correct one only for a sample from π_e; applied to an argmax token it can accept a non-CD-argmax token or over-accept, so the output distribution need not match vanilla CD greedy. If the implementation uses a different greedy verification rule—say accept only when the draft token is the argmax of π_CD—they need to state and prove it. As written, the proof in B.3 is about sampling from π_CD, not about the greedy regime that carries the 1.65–1.95x numbers.\n\nWhat is genuinely good: the proposal-alignment question is well framed and the two diagnostics, matched Cross-α and approximate dual-drafter, are honest attempts to test whether contrastive-aware lightweight drafting pays off. The negative result is useful even if it only covers a narrow regime. The deployment methodology is careful: three-run averages, standard deviations, matched hardware, and a best-γ check for baselines. DCD-NGRAM as a proposer-agnostic sanity check is a nice touch.\n\nThe other soft spots are minor by comparison. The 81.1% signal statistic comes from 200-sample single-run matched controls with no error bars, so the general claim that contrastive-aware drafting never helps should be read as conditional on these specific lightweight drafters. The 3B stress test is offline, as the paper says.\n\nBottom line: the speedups are plausible, but the central correctness claim is not yet established for the setting that matters. A serious referee should engage; the fix is probably not hard—either sample from π_e during deployment, or define and prove a greedy-specific verification rule. Right now I would be cautious about citing the paper as a lossless accelerator. The reader's conditional verdict is fair but underplays this greedy issue; I think it is the load-bearing point.","headline":"Useful systems paper with a clean decoupling idea, but the load-bearing losslessness claim is not proven at the greedy operating point that produces the headline 1.65–1.95x speedups.","tokens_in":32714,"tokens_out":3366,"would_cite":false,"duration_ms":33971,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Decoupled Contrastive Decoding achieves vanilla contrastive-decoding output at 1.65–1.95x lower serial cost by keeping the amateur model out of the draft loop.","keywords":["contrastive decoding","speculative decoding","decoupled contrastive decoding","expert-aligned drafting","lightweight proposer","EAGLE3","draft-then-verify","accepted length"],"falsifier":"Train a higher-capacity contrastive-aware drafter on the CD target and compare mean accepted length against expert-aligned EAGLE3 in an online rollout on GSM8K and MMLU; if the contrastive-aware drafter wins in a majority of settings at 1000 samples, DCD's central diagnosis is overturned.","tokens_in":31718,"feed_emoji":"⚡","tokens_out":11830,"duration_ms":99666,"temperature":0.7,"pith_summary":"Contrastive decoding improves LLM output by penalizing tokens the large \"expert\" model dislikes and the small \"amateur\" likes, but it costs an extra model pass per token. This paper asks whether, when accelerating CD with speculative decoding, the contrastive signal should shape the draft or only score it. Using two controlled diagnostics, the paper argues that a lightweight drafter trained on the contrastive distribution gains nothing consistent over one trained on the expert, because the contrastive correction is usually smaller than the drafter's own error. The proposed method, Decoupled Contrastive Decoding (DCD), drafts with an expert-aligned proposer and applies the amateur only in unchanged verification, so standard speculative verification preserves vanilla CD's output distribution. The reported payoff is a 1.65–1.95x greedy speedup over vanilla CD across three 8B families, with 5–12x lower proposal-path latency on MMLU.","feed_headline":"Contrastive decoding runs 1.65–1.95x faster with the same output","feed_subtitle":"Decoupling the amateur from drafting preserves contrastive quality while cutting serial proposal cost.","key_machinery":"The carrying object is the decoupled proposal–verification split, formalized as DCD's one-round loop: an amateur-independent proposer $E$ (EAGLE3, a feature-level drafter that predicts hidden states rather than tokens) draws $\\gamma$ candidate tokens, and the expert and amateur models run parallel forward passes so each candidate is accepted with probability $\\min(1, \\pi_{\\mathrm{CD}}(y_i)/\\pi_e(y_i))$, with a fallback sample from the normalized contrastive distribution on rejection. The contrastive target is the standard CD product $\\pi_{\\mathrm{CD}}(x|h) \\propto \\pi_p(x|h)^{1+\\alpha} \\pi_q(x|h)^{-\\alpha}$, which separates into an expert term and the contrastive factor $(\\pi_p/\\pi_q)^\\alpha$. Two theoretical pieces carry the argument: the reconstruction-error decomposition $\\Delta(x) = (1+\\alpha)\\epsilon_p(x) - \\alpha\\epsilon_q(x)$, showing why inference-time recombination amplifies drafter errors, and the KL-slope inequality proving that under effective draft alignment—the drafter places more mass than the amateur on tokens where the expert has a log-likelihood advantage—an expert-aligned drafter's distance to $\\pi_{\\mathrm{CD}}$ grows more slowly with $\\alpha$ than the amateur's.","core_discovery":"The central claim is that the contrastive signal should stay in verification and never enter the lightweight proposal path. Concretely, DCD runs an expert-aligned feature-level proposer (EAGLE3) to draft $\\gamma$ tokens, then verifies those tokens in parallel against the unchanged contrastive distribution $\\pi_{\\mathrm{CD}}(x) \\propto \\pi_p(x)^{1+\\alpha} \\pi_q(x)^{-\\alpha}$; the proposer changes only the acceptance rate, never the output distribution. The paper supports the decoupling with a position-level diagnosis: across 24 configurations, 81.1% of positions have a contrastive signal below 1.0 while 48.7% have expert-side proposal KL error at or above 2.0, so the correction a contrastive-aware drafter must model is usually weaker than the drafter error it must overcome. In matched training, neither direct training on the contrastive target nor an approximate dual-drafter decomposition beats expert-aligned drafting in accepted length, and the decomposed route amplifies expert-side error by $1+\\alpha$. The deployment result follows: EAGLE3-based DCD reaches 1.65–1.95x average greedy speedup over vanilla CD across three 8B families, stays close to 2x in a greedy 70B extension, and preserves the vanilla-CD output distribution exactly.","pith_inferences":["The 81.1% low-signal statistic suggests a design space the paper does not explore: a drafter that conditions on whether the current position is a high-signal, low-error cell—e.g., via a learned gate or position-adaptive $\\alpha$—could outperform both expert-aligned and contrastive-aware routes.","Since losslessness holds for any proposer, DCD's gains compound with future expert-aligned proposers; tree-based or self-speculative drafters could plausibly push speedups beyond the reported 1.95x without touching the amateur.","The paper's framing implies the amateur in CD might be replaceable by a model that is not a good generator but a good scorer of expert-preferred tokens, potentially shrinking the memory and KV-cache footprint of contrastive verification."],"forward_implications":["Contrastive decoding can be made cheap without retraining the amateur: any amateur-independent proposer—EAGLE3, an N-gram matcher, or a future expert-aligned drafter—can be dropped into DCD's verification rule and the vanilla-CD output distribution is preserved.","If DCD's diagnostics hold, the amateur's role in CD is verification-time correction, not generation, so the serial draft path no longer depends on the amateur model's latency.","At $\\alpha=0.5$, where amateur-coupled SCD and CoS fall below 1.0x on hard settings such as Qwen3 MMLU and CNN/DM, DCD EAGLE3 still delivers 1.39–1.60x, so the speed advantage widens as the contrastive penalty strengthens.","Because the target distribution is unchanged, DCD's task-level accuracy tracks vanilla CD across $\\alpha$ (GSM8K greedy sweep), so the speedup is pure latency reduction rather than a new decoding objective.","A greedy-only 70B extension maintains the trend, with DCD EAGLE3 averaging 2.02x at $\\alpha=0.1$ and 1.96x at $\\alpha=0.5$ over vanilla CD."],"supporting_citations":[{"why":"Establishes the lossless draft-then-verify framework whose distribution-preserving guarantee DCD inherits.","marker":"Leviathan et al., 2023"},{"why":"Supplies the speculative sampling verification rule used unchanged in DCD's Algorithm 1.","marker":"Chen et al., 2023"},{"why":"Defines the contrastive decoding target distribution that DCD keeps in verification.","marker":"Li et al., 2023"},{"why":"Gives the contrastive decoding formulation for reasoning tasks that motivates the expert–amateur setup.","marker":"O'Brien and Lewis, 2023"},{"why":"Supplies the EAGLE3 feature-level proposer used as the expert-aligned drafter in the main deployment experiments.","marker":"Li et al., 2025"},{"why":"Defines the SCD baseline that keeps the amateur in the proposal loop, the main comparison point for DCD's speedups.","marker":"Yuan et al., 2024"},{"why":"Defines the CoS baseline and the benchmark/model-pair settings used in the deployment evaluation.","marker":"Fu et al., 2025"}],"fun_headline_variants":["Draft with expert, verify with amateur: 1.65–1.95x CD speedup","Decoupled Contrastive Decoding: expert drafts, amateur verifies, 1.65–1.95x faster","Keep contrastive signal out of drafting for 1.65–1.95x CD speedup","Decoupling drafting from contrastive verification gives 1.65–1.95x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that no lightweight contrastive-aware drafter can consistently beat an expert-aligned drafter, which the paper supports with single-run matched experiments on 200 samples and an offline diagnosis using true amateur logits rather than an online rollout.","fun_headline_variants_meta":{"raw":{"variants":["Draft with expert, verify with amateur: 1.65–1.95x CD speedup","Decoupled Contrastive Decoding: expert drafts, amateur verifies, 1.65–1.95x faster","Keep contrastive signal out of drafting for 1.65–1.95x CD speedup","Decoupling drafting from contrastive verification gives 1.65–1.95x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001141,"raw_usage":{"total_tokens":4772,"prompt_tokens":1017,"completion_tokens":3755,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":3650}},"tokens_in":633,"tokens_out":3755,"duration_ms":26240,"temperature":1.0,"reasoning_tokens":3650,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:41:52.209187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a higher-capacity contrastive-aware drafter on the CD target and compare mean accepted length against expert-aligned EAGLE3 in an online rollout on GSM8K and MMLU; if the contrastive-aware drafter wins in a majority of settings at 1000 samples, DCD's central diagnosis is overturned.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the SCD baseline that keeps the amateur in the proposal loop, the main comparison point for DCD's speedups."}],"review_version":1}