{"id":"a809032e-e7eb-417c-ab4b-bfae5c79175c","arxiv_id":"2607.21804","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"ADSD shows that a short adversarial suffix appended to a prompt can collapse the token-acceptance rate in speculative decoding, increasing latency by 62.3% on GSM8K while preserving answer accuracy.","lead":"This paper introduces ADSD, a prompt-suffix attack that makes speculative decoding's verifier reject most draft tokens, slowing generation by up to 62% on a math benchmark while keeping answers mostly correct. It is the first attack aimed at the internal draft-verification mechanism of speculative decoding rather than at output length or serving infrastructure.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Soft-Collapse surrogate is derived for tokenwise acceptance (Eq. 1); the paper's claim that ADSD 'collapses verifier acceptance' across production schemes rests on unverified transfer to tree/blockwise rules and requires white-box distributions that serving stacks do not expose.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the attack's reliance on the exact tokenwise acceptance rule and white-box access. My analysis agrees with that and adds a specific technical gap: the paper derives the surrogate for tokenwise acceptance but applies it to blockwise/tree mechanisms without theoretical alignment, and the only tree-based evaluation (EAGLE-3) shows substantially degraded task quality, weakening the unqualified 'preserving task quality' claim. This concern is significant enough to keep the verdict at CONDITIONAL, but it does not invalidate the core contribution within its declared threat model. Since the reader already reached CONDITIONAL and my concern is aligned, no verdict change is warranted.","tokens_in":13796,"tokens_out":13353,"duration_ms":139779,"concrete_test":"Implement a tree-based speculative verification (e.g., Medusa-style) where acceptance is determined by the joint probability of the candidate set, and re-evaluate the GSM8K-optimized suffix without white-box access to q/p (only sampled outputs). If the mean latency increase over 1000 prompts is not significantly >0 (e.g., 95% CI excludes >20%) or accuracy degrades by >5 points, the central claim's external validity fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that ADSD is 'the first prompt-suffix attack that collapses verifier acceptance' (abstract) rests on the validity of the Soft-Collapse surrogate (Eq. 3), which is derived from the tokenwise acceptance rule α=min(1,p/q) in Eq. (1). The paper's theoretical justification is therefore specific to the tokenwise scheme. Yet the evaluation extends the claim to Blockwise and HSD (Table 2) without specifying the acceptance rules used in those simulations or re-deriving a surrogate for them. If those schemes use a block-level or tree-level acceptance decision (as in Medusa/EAGLE), the per-token margin SC may not track the true rejection signal. The paper's own limitation statement (§6) concedes the method is 'limited to white-box optimization', and the attacker needs access to both q and p at deployment time. In a real serving stack (vLLM/TGI), these internal distributions are not exposed; a user only receives sampled tokens. Moreover, the EAGLE-3 result (Table 5) shows the attack degrades GSM8K accuracy from 0.638 to 0.575, contradicting the unqualified 'preserving task quality' claim. Thus the headline result is established only for a narrow, white-box, tokenwise setting; the abstract overstates the scope.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADSD, a prompt-suffix attack against speculative decoding. The attacker appends a short discrete suffix to a benign prompt to push the draft distribution toward tokens the target verifier is unlikely to accept, thereby reducing acceptance rate and increasing end-to-end latency while attempting to keep final answer quality close to benign. The method combines Soft-Collapse (Eq. 3), derived from the tokenwise acceptance rule α=min(1,p/q) in Eq. (1), with a forward-KL target-preservation penalty (Eq. 4), optimized via gradient-guided beam search over discrete tokens. Evaluations on GSM8K with Qwen2.5 models report a 62.3% mean-time increase with accuracy dropping from 0.821 to 0.802; additional experiments cover Blockwise/HSD schemes, cross-domain transfer (HumanEval, CNN/DailyMail), larger Qwen and LLaMA models, and EAGLE-3. The paper also includes an ablation showing both objective terms are needed.","tokens_in":14260,"tokens_out":5065,"duration_ms":51954,"significance":"If the results hold, ADSD identifies a new availability attack surface inside the speculative-decoding verification loop, distinct from output-length or guardrail attacks. The core derivation of Soft-Collapse from the tokenwise acceptance rule is mathematically sound, and the ablation in Table 6 supports the necessity of both objective terms. The paper ships detailed reproducibility settings for the main runs, which is a strength. However, the significance is currently narrowed by several unaddressed scope and evidence gaps: the transfer to Blockwise/HSD is asserted without specifying their acceptance rules, the abstract's unqualified 'preserving task quality' is contradicted by the EAGLE-3 accuracy drop, and most effect sizes lack confidence intervals. These issues do not invalidate the central idea but prevent the paper from supporting its broadest claims.","major_comments":[{"comment":"The Soft-Collapse surrogate is derived from the tokenwise acceptance rule α=min(1,p/q) in Eq. (1). However, Table 2 claims transfer to Blockwise and HSD without specifying the acceptance rules of those schemes or justifying that SC remains a valid surrogate under block- or tree-level verification. If these schemes accept based on joint/tree probabilities or token-set membership, the per-token ReLU(log q − log p) margin in Eq. (3) need not track the actual rejection decision. Please state the verification algorithms used in the simulations and either derive the corresponding surrogate or restrict the theoretical claim to tokenwise schemes. As written, the cross-scheme claim is not supported.","section":"§5.3 / Table 2 and Eq. (3)"},{"comment":"The abstract states the attack 'preserves the task quality' on GSM8K. Yet Table 5 shows EAGLE-3 GSM8K accuracy dropping from 0.638 to 0.575 (a relative drop of about 9.9%) under ADSD. Since this is the same benchmark and a different architecture, the unqualified claim is contradicted by the paper's own data. Please qualify the quality-preservation claim (e.g., to the tokenwise Qwen settings) or provide an explanation for the EAGLE-3 accuracy loss and its implications for stealth. The phrase 'while preserving the task quality' in Section 1 has the same issue.","section":"Abstract vs. Table 5"},{"comment":"The headline results (e.g., +62.3% time, −38.9% block efficiency) are reported as point estimates without confidence intervals, number of repeated runs, or seed variation. Table 2 includes error bars only on the ADSD rows, not on the benign baselines. Because latency and efficiency are hardware- and scheduling-sensitive, the reader cannot assess whether the reported differences are within run-to-run noise. Please report standard errors or 95% confidence intervals over at least three independent evaluations for every setting, or justify determinism of the measurement and state it explicitly.","section":"Tables 1, 3, 4, 5"},{"comment":"The introduction and systemic-impact discussion frame ADSD as a practical denial-of-wallet threat to serving stacks such as vLLM/TGI, but the limitation statement in §6 concedes the method is limited to white-box optimization with access to q and p at deployment time. Production engines generally do not expose these internal per-token distributions to user-space prompts, and may use tree or block verification. Please add an explicit statement in the abstract and introduction that the demonstrated attack requires a white-box deployment replica with the exact tokenwise acceptance rule, and indicate how this affects the practical threat assessment. This is a scope caveat rather than a correctness error in the optimization itself.","section":"§1, §3.1, §6"}],"minor_comments":[{"comment":"The text in §5.4 acknowledges that cross-domain transfer harms task quality, but the abstract's phrase 'the vulnerability exists across different domains' could be misread as including stealth. Consider adding 'without preserving stealth' or similar qualification.","section":"§5.4 / Abstract"},{"comment":"The uncertainty notation '(±)' over the GSM8K questions for attack results is undefined; specify whether it is a standard error, standard deviation, or 95% confidence interval, and how it was computed.","section":"Table 2"},{"comment":"The hyperparameter description says 'the main revised Qwen and EAGLE runs' but there is no earlier mention of 'revised' runs; clarify what distinguishes these from other runs.","section":"Appendix A"},{"comment":"Reference [41] appears to have a garbled author list ('Woosuk Kwon, Siheon Min, Sainbayar Sukhbaatar Seo, ...'); please verify the citation.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a timely and interesting attack study. The core derivation and ablation are sound, and the reproducibility appendix is a plus. The main changes needed are to qualify the scope of the claims (tokenwise, white-box), address the EAGLE-3 quality contradiction, and add statistical uncertainty to the performance measurements. I do not see a fundamental correctness error in the core optimization, but the current manuscript overstates its breadth."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. It identifies a new attack surface — turning the verifier's acceptance rule in speculative decoding into an availability bug — and backs the idea with a simple, correct surrogate (Soft-Collapse) and a sensible target-preservation term. The 62.3% latency increase on GSM8K with only a 0.821→0.802 accuracy drop is a real end-to-end result, not an artifact of the optimization objective. The ablations (Table 6) do what ablations should: removing the collapse term kills the attack, removing the KL term kills stealth. That is honest evidence.\n\nWhat's new: to my knowledge, no prior work attacks the draft-verification mechanism with a prompt suffix. The related-work coverage (side channels, safety decoding, output-length attacks) is fair, and the distinction is real. The evaluation across two target scales, LLaMA, and EAGLE-3 gives some breadth, even if it is all in a research harness.\n\nSoft spots, in proportion. The abstract says the attack preserves task quality. That is only true for the matched GSM8K setting. On EAGLE-3, accuracy drops from 0.638 to 0.575; on transfer to HumanEval, pass@1 goes 0.829→0.683; on CNN/DailyMail, R-1 collapses from 0.227 to 0.048. The body acknowledges this (cross-domain latency transfer does not necessarily imply cross-domain stealth), but the abstract overstates it. Second, the attack is white-box and assumes the exact tokenwise acceptance rule (Eq. 1). The paper evaluates Blockwise and HSD with the same suffix but never specifies the acceptance rules for those simulations. If they use tree or block verification, the Soft-Collapse surrogate may not track the rejection signal, so the 'across speculative decoding strategies' claim is stronger than the evidence. Third, no code or artifacts are released, and only Table 2 has error bars. For a security paper, that is a reproducibility gap. The limitation section is honest about white-box and a representative set of pairs, and I take that at face value.\n\nThe stress-test note about production stacks not exposing distributions is fair, but it is a limitation of the threat model, not a flaw in the attack. The paper is clear about attacker knowledge. If a serving stack uses a different verifier, the suffix may not transfer; that is an open question, not a refutation.\n\nVerdict: accept for peer review. The central result is solid in the claimed white-box tokenwise setting. The scope claims need tightening and artifacts would help, but the paper is a legitimate contribution. I would cite it if I worked on speculative decoding security.","headline":"ADSD is a genuinely new attack surface — prompt-side verifier collapse — with a clean surrogate objective and convincing ablations, but the 'preserving task quality' claim only holds for the matched GSM8K setting and production relevance is unproven.","tokens_in":797,"tokens_out":1677,"would_cite":true,"duration_ms":41799,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Short prompt suffixes can systematically collapse the acceptance rate inside speculative decoding, turning a lossless accelerator into a latency and cost amplifier while leaving final answers mostly correct.","keywords":["speculative decoding","adversarial prompt suffix","verifier acceptance collapse","availability attack","denial-of-wallet","draft-target alignment","latency inflation","white-box attack"],"falsifier":"Run ADSD-style suffix optimization against a production speculative-decoding engine that uses tree-based verification or a modified acceptance rule (e.g., typical sampling or entropy-based acceptance) and measure whether block efficiency still falls by roughly 39% and latency rises by ~60%; alternatively, measure the same suffix in a continuous-batching server under load to see whether batching amortizes the wasted target passes and shrinks the per-request slowdown.","tokens_in":13742,"feed_emoji":"⏱️","tokens_out":8365,"duration_ms":84190,"temperature":0.7,"pith_summary":"This paper tries to establish that a short text suffix appended to a benign prompt can systematically collapse the acceptance rate of speculative decoding, a lossless acceleration technique in which a small draft model proposes tokens and a larger target model verifies them in parallel. The attack works because the tokenwise acceptance rule α = min(1, p/q) rejects a draft token whenever the draft probability q exceeds the target probability p; by pushing draft probability mass into that rejection region, an attacker forces the verifier into repeated wasted rounds. On a math word-problem benchmark, the optimized suffix raises mean sample time from 26.05 to 42.29 seconds (+62.3%) while GSM8K accuracy only slips from 0.821 to 0.802, so the slowdown is not easily visible in answer quality. If correct, this means the 'lossless' guarantee of speculative decoding concerns only output semantics: the operational cost of a request is attacker-controllable, and one adversarial request can amplify tail latency for co-scheduled benign users in batched serving.","feed_headline":"Five-token suffix makes speculative decoding 62% slower","feed_subtitle":"A 5-token prompt addition can turn a lossless accelerator into a cost sink—without corrupting the visible answer.","key_machinery":"The load-bearing identity is the tokenwise acceptance rule α(v) = min(1, p(v)/q(v)) = exp(−ReLU(log q(v) − log p(v))), where q is the draft's next-token distribution and p is the target's. Rejection pressure appears exactly when the draft is overconfident, i.e., the positive log-margin Δ(v) = log q(v) − log p(v) > 0. The Soft-Collapse objective SC(q, p) = Σ_v q(v)·ReLU(Δ(v)) sums draft-weighted rejection risk over the vocabulary, and the target-preservation penalty KL(p0 || p^u) keeps the attacked target distribution close to the benign one. Gradient-guided sequential beam search over suffix token embeddings solves the discrete optimization, yielding a universal suffix of length m=5 with lin","core_discovery":"The paper's central claim is that the draft-verifier acceptance mechanism inside speculative decoding is a usable attack surface. The authors introduce ADSD, a prompt-suffix optimization that maximizes the verifier's expected rejection by optimizing a verifier-aligned surrogate, Soft-Collapse, derived directly from the acceptance rule, and it anchors the target model's distribution to its benign behavior with a forward-KL term so the final answer remains coherent. They report that a single 5-token universal suffix, calibrated on 10 GSM8K prompts, increases mean sample time by 62.3% on the matching benchmark, that the same suffix transfers to code and summarization (with larger latency effect","pith_inferences":["I would expect the same Soft-Collapse logic to transfer to any verification rule that rejects when the drafter is overconfident relative to a reference distribution; changing to a symmetric rule that also penalizes target-overconfidence would likely shrink the attack, but the paper does not test that variant.","The attack's white-box requirement could be relaxed: a suffix optimized against a distilled surrogate for the target distribution, or against the draft model alone plus a cheap proxy of p, might produce a transferable black-box attack; the paper only evaluates white-box optimization.","A practical defense the paper leaves implicit is to calibrate the drafter's confidence (e.g., temperature scaling on the draft head) so that overconfident proposals are less frequent benignly; this is testable and would reduce the ReLU margin the attack exploits, but it could also reduce benign acceptance.","The transfer results suggest latency transfer and quality preservation decouple: the GSM8K-optimized suffix slows code generation dramatically but visibly damages it, so in practice an adaptive attacker would likely generate per-domain suffices, and this is a testable extension of the threat model."],"forward_implications":["A deployed speculative-decoding service using tokenwise acceptance becomes a latency-amplification target: one 5-token universal suffix raises mean per-request time by roughly 60% on arithmetic reasoning tasks without visibly corrupting the answer.","The vulnerability holds across verification layouts (tokenwise, blockwise, hierarchical) and across draft-target architectures (independent small-to-large draft pairs, LLaMA-3 70B→8B, and shared-head EAGLE-3), so redesigning the verification procedure alone does not remove the attack.","Because the target model can repair the draft's corrupted prefix, the final user-visible answer can remain correct while internal verifier cost has already exploded—so post-hoc answer-quality checks are insufficient to detect the attack (paper's Appendix B.1 case study).","The authors' proposed mitigation, acceptance-rate monitoring with fallback to target-only decoding, can bound repeated cost amplification but cannot protect the first slowed request; a batch neighbor's latency can still be amplified through shared GPU execution."],"fun_headline_variants":["5-token suffix makes speculative decoding 62% slower","62% slower decoding from a 5-token adversarial suffix","Tiny 5-token attack slows speculative decoding 62% without harming answers","Crafty suffix exploits verifier to slow decoding 62%—answers intact","5-token suffix yields 62% slowdown while preserving task quality"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The attack assumes the target service uses the exact tokenwise acceptance rule α(v) = min(1, p(v)/q(v)) and that the attacker has white-box access to both the draft distribution q and the target distribution p at optimization time; a different verification rule or hidden internals would void the claimed collapse.","fun_headline_variants_meta":{"raw":{"variants":["5-token suffix makes speculative decoding 62% slower","62% slower decoding from a 5-token adversarial suffix","Tiny 5-token attack slows speculative decoding 62% without harming answers","Crafty suffix exploits verifier to slow decoding 62%—answers intact","5-token suffix yields 62% slowdown while preserving task quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000737,"raw_usage":{"total_tokens":3100,"prompt_tokens":685,"completion_tokens":2415,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":429,"completion_tokens_details":{"reasoning_tokens":2335}},"tokens_in":429,"tokens_out":2415,"duration_ms":18538,"temperature":1.0,"reasoning_tokens":2335,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T06:36:49.633412+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ADSD-style suffix optimization against a production speculative-decoding engine that uses tree-based verification or a modified acceptance rule (e.g., typical sampling or entropy-based acceptance) and measure whether block efficiency still falls by roughly 39% and latency rises by ~60%; alternatively, measure the same suffix in a continuous-batching server under load to see whether batching amortizes the wasted target passes and shrinks the per-request slowdown.","supporting_citations":[],"review_version":1}