{"id":"f895f719-2c22-4b77-9fdc-816add6d9e28","arxiv_id":"2507.21522","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Token Map Drafting speeds up transformer ASR decoding on CPU by using a precomputed n-gram token map as a model-free draft for speculative decoding.","lead":"This paper replaces the small draft model used in speculative decoding with a precomputed n-gram token map, speeding up Whisper transcription on CPUs by about 1.3x in structured domains. Because it avoids a second model, it may help deploy ASR on phones and embedded devices.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim of accuracy preservation is unsupported: no WER is reported, and the threshold-based acceptance in Section II.3 can change the main model's output, so the claim of 'without sacrificing transcription accuracy' is not established.","rationale":"Both the reader and I identify the same gap: the central claim promises accuracy preservation but provides no accuracy measurement, and the threshold-based verification lacks the distribution-preserving guarantee of standard speculative decoding. This is not merely a missing nice-to-have; it is the load-bearing part of the contribution. A method that speeds up decoding by changing outputs is not 'without sacrificing transcription accuracy.' The paper's own Section II describes acceptance based on 'a predefined threshold' without specifying it, and Section IV-C discusses lower acceptance rate and conservative predictions, implying output changes may occur. The lack of WER means even the empirical question is unanswered. I would not raise a separate data leakage concern as the primary issue because the manuscript says the token map is derived from training transcriptions, so train/test separation may exist, but it is underspecified. The accuracy gap alone justifies REJECT. The concrete test is straightforward and would settle the concern: compute WER with standard splits. If the authors can show no degradation, the central claim could be repaired; until then it is unsupported.","tokens_in":6492,"tokens_out":4464,"duration_ms":55155,"concrete_test":"Run Whisper-large-v3 on the CI-AVSR test split and the internal test set using (a) standard autoregressive greedy decoding and (b) Token Map Drafting with the reported configuration; compute WER/CER and the fraction of utterances whose transcripts differ. If WER with Token Map Drafting is not statistically indistinguishable from the baseline (e.g., within a 95% CI of zero difference), the accuracy-preservation claim fails. Additionally, report the acceptance threshold used; if the threshold is set so high that accepted tokens are always the greedy argmax, WER will match by construction but speedup may vanish, revealing the accuracy/speed trade-off.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim is that Token Map Drafting accelerates Whisper decoding 'without sacrificing transcription accuracy.' No word error rate or any accuracy metric appears anywhere in the paper; only speedup, acceptance rate, and acceptance length are reported (Table I). The verification step described in Section II (steps 2–3) accepts a draft token if its conditional probability exceeds a threshold, and otherwise lets the main model generate a replacement token. This is not the rejection-sampling scheme of Leviathan et al. [12], which guarantees output-distribution preservation. With greedy decoding (the typical mode for CTranslate2 Whisper), the main model's output at each position is the argmax token; accepting a non-argmax token whose probability merely exceeds an unreported threshold can change the final transcription. The paper also never reports the threshold value, so the reader cannot tell whether the method is configured to preserve outputs or to trade accuracy for speed. Therefore the 'without degrading recognition accuracy' part of the central claim is unverified, and the mechanism described gives a concrete way that claim could fail. This is load-bearing because the method's value proposition depends on both speed and fidelity; speed alone is not sufficient.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Token Map Drafting, a model-free speculative decoding method for transformer-based ASR (Whisper). Instead of a separate draft model, it builds a precomputed n-gram token map from domain-specific transcriptions and uses it to propose candidate token sequences. The main model verifies these candidates by thresholding their conditional probabilities, and accepted tokens are kept while mismatches trigger autoregressive correction. Experiments on the CI-AVSR dataset and an internal maintenance-task dataset report decoding speedups of 1.27x and 1.37x over autoregressive decoding, and a comparison to a Distil-Whisper-based speculative decoding baseline. The paper claims these speedups are obtained 'without sacrificing transcription accuracy', but no word error rate or any accuracy metric is reported anywhere.","tokens_in":6769,"tokens_out":5557,"duration_ms":64210,"significance":"If the accuracy-preservation claim could be substantiated, the method would be a practical contribution for CPU-based ASR in structured, low-perplexity domains, since it removes the need for an auxiliary draft model and exploits domain-specific regularities. The core idea is simple and plausible, and the reported speedup numbers, taken at face value, are modest but potentially useful. However, the significance is almost entirely contingent on empirical validation that is currently missing: there are no WER/CER numbers, the acceptance threshold is not disclosed, and the hyperparameters are selected and evaluated on the same internal dataset. Without these components, the paper does not establish its central value proposition.","major_comments":[{"comment":"The claim 'without sacrificing transcription accuracy' is unsupported because no accuracy metric is reported. Table I reports only speedup (S), acceptance rate (A_r), and average acceptance length (A_l); no WER, CER, or any recognition quality metric appears for the baseline or the proposed method. The abstract and conclusion assert accuracy preservation, but the evaluation metrics in Section IV-B do not include accuracy. The paper must add a direct comparison of transcription accuracy (e.g., WER/CER) for both the baseline and the proposed method before this claim can be accepted.","section":"Abstract; Table I; Section IV-B"},{"comment":"The verification rule described in Section II (and reflected in Figure 2) accepts a draft token if its conditional probability exceeds a predefined threshold, otherwise it generates a replacement token from the main model. This is not the distribution-preserving rejection scheme of Leviathan et al. [12]. With greedy decoding, which is the typical mode for CTranslate2 Whisper, the main model's output at each position is its argmax; accepting a non-argmax token whose probability merely exceeds an unreported threshold changes the final transcription. The threshold value is never given, so the reader cannot tell whether the method is configured to preserve outputs or to trade accuracy for speed. The authors should either specify the threshold and demonstrate empirically that WER is unchanged, or change the acceptance rule to one that guarantees output equivalence.","section":"Section II, Step 3; Section III"},{"comment":"The hyperparameter selection is circular with respect to the evaluation. Section III-B states that the optimal n-gram length N is chosen by measuring speedup on 'an internal dataset' (Figure 4), and the pruning thresholds shown in Figure 3 appear to be derived from similar speedup measurements. The final speedup for the proposed method on the internal dataset (Table I) is then reported on the same data used to select N and the thresholds. This means the reported 1.37x speedup is partially optimized on the evaluation set. The authors should use a held-out validation split or cross-validation for hyperparameter selection and report speedups on a separate test set.","section":"Section III-B, Figure 4, Table I"},{"comment":"The runtime decoding algorithm is not fully specified. Algorithm 1 only describes token-map construction; the decoding-time steps of sequence matching, validation, candidate merging, and acceptance are only described informally in the text and Figure 2. In particular, the acceptance threshold, the procedure for 'iteratively merge candidates with their nearest matches', and the condition under which decoder state is truncated are not formalized. Without a precise algorithmic statement, the method is not reproducible and the behavior of the verification step cannot be audited. A complete pseudo-code for the decoding loop should be added.","section":"Algorithm 1; Section III; Figure 2"}],"minor_comments":[{"comment":"There is a typographical artifact in the first line: 'we proposeT oken Map Drafting' should be 'we propose Token Map Drafting'.","section":"Abstract"},{"comment":"The phrase '10% absolute improvement in decoding speed' is misleading. Table I shows speedups of 1.27 for the baseline and 1.37 for the proposed method, a difference of 0.10 in the speedup factor. This is a 0.10x speedup difference, or about 7.9% relative improvement, not '10% absolute improvement' in decoding speed.","section":"Abstract; Conclusion"},{"comment":"The sentence 'Beyond 3 candidates, SD show less effective' contains a subject-verb agreement error and should be rewritten, e.g., 'Beyond 3 candidates, SD is less effective'.","section":"Section III-B"},{"comment":"The table header formatting is inconsistent ('CI-A VSR' versus 'Internal Dataset'), and the column alignment is off in the rendered version. Please format the table cleanly and ensure the dataset names match the text.","section":"Table I"},{"comment":"The sentence 'While our experimental results suggest that the proposed method maintains transcription accuracy' is not supported by any reported accuracy result. This sentence should be revised or removed until actual WER/CER measurements are reported.","section":"Conclusion"},{"comment":"The baseline configuration should be described more carefully. The paper cites Distil-Whisper [15] as the basis of the 'Distill-spec' baseline, but uses Whisper-large-v3-turbo as the draft model. It is not clear whether this exact setup is prescribed by [15] or is a modified configuration; please clarify the baseline hyperparameters and decoding settings.","section":"Section IV-A"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is currently unverifiable because no accuracy metric is reported. The described threshold-based acceptance rule can in principle deviate from the main model's greedy argmax, so the claim 'without sacrificing transcription accuracy' is not just unmeasured but mechanistically questionable. The authors need to add WER/CER experiments, disclose the acceptance threshold, and remove the circular hyperparameter selection. If the authors can provide those, the method could be a reasonable empirical contribution. If the accuracy experiments show degradation, the paper's premise collapses. Given the current state, I would not accept the manuscript without substantial revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a neat application—n-gram lookup as a draft for speculative decoding in Whisper—and the CPU speedups look plausible. But the paper's central promise, 'without sacrificing transcription accuracy,' is not supported: there is no WER or any accuracy metric in the paper. That's a load-bearing hole, and it's fixable.\n\nWhat's new: model-free drafting via a precomputed n-gram token map is a legitimate, useful twist on speculative decoding. The paper correctly identifies that distilled draft models are impractical on CPU-only devices, and it shows a real speedup (1.27x on CI-AVSR, 1.37x on an internal maintenance-task dataset) with a simple C++ implementation on CTranslate2. The analysis of when candidate sequence length makes SD worthwhile (Figure 3) is a good practical contribution. The speedup measurements are specific enough to reproduce.\n\nThe soft spots: first, no accuracy metric. Acceptance rate and average acceptance length are not accuracy. The threshold-based verification (Section II, step 2) accepts tokens whose probability exceeds an unreported threshold, so it can accept a non-argmax token and change the transcription. Standard speculative decoding with rejection sampling preserves the output distribution; this method does not. Second, N=3 is selected by measuring speedup on the same internal dataset used for the final numbers, so the headline speedup is partially tuned on the evaluation set. Third, the internal dataset is described only vaguely, so it's hard to judge how representative the 1.37x is.\n\nNone of these flaws kills the idea. The method will very likely preserve WER in low-perplexity domains if the threshold is set conservatively (e.g., only accept when the token is the argmax), but the paper doesn't say that and doesn't measure it. The authors should add WER on both datasets, report the threshold, and either move the N selection to a validation set or show it is insensitive.\n\nBottom line: this deserves referee time—the idea is real and the speedup data is worth checking—but the current version's main claim needs support. If you're working on on-device ASR, it's worth a read for the token-map idea alone, but don't cite it as evidence that accuracy is preserved.","headline":"Model-free n-gram drafting for ASR spec decoding is a nice idea with plausible CPU speedups, but the missing WER numbers leave the accuracy-preservation claim unproven.","tokens_in":7255,"tokens_out":2416,"would_cite":false,"duration_ms":27919,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Token Map Drafting replaces the draft model with an n-gram token map, claiming 1.27x–1.37x CPU decoding speedups for Whisper without accuracy loss.","keywords":["speculative decoding","automatic speech recognition","Whisper","n-gram token map","model-free drafting","CPU inference","on-device ASR","low-perplexity domains"],"falsifier":"Measure word error rate on both datasets with Token Map Drafting versus unmodified Whisper decoding on the same audio; if the WER rises, the central claim of no accuracy loss falls. The paper provides speed and acceptance metrics but no accuracy table, so this measurement is the decisive check.","tokens_in":6330,"feed_emoji":"⚡","tokens_out":6592,"duration_ms":67726,"temperature":0.7,"pith_summary":"This paper proposes Token Map Drafting, a speculative decoding method for transformer ASR that replaces the usual small draft model with a precomputed n-gram token map built from domain-specific transcripts. The aim is to accelerate autoregressive decoding of Whisper-family models on CPU-only devices, where running a second model would add overhead. The paper claims 1.27x and 1.37x decoding speedups on the CI-AVSR dataset and an internal maintenance-task dataset, with no reported degradation in transcription accuracy, plus a 10% absolute speed improvement over a distilled-draft-model baseline. The central contribution is showing that, for structured low-perplexity speech, domain statistics can substitute for a learned drafter.","feed_headline":"Token map drafting speeds Whisper decoding 1.37x on CPU","feed_subtitle":"Structured speech gets 1.27x–1.37x faster CPU decoding, with no reported accuracy loss.","key_machinery":"The central object is the n-gram token map: a dictionary whose keys are token n-grams (length 1 to N) and whose values are ranked lists of candidate token sequences that followed those n-grams in the domain training transcripts. It replaces the learned draft model of standard speculative decoding; decoding runs by matching the current token context to a key, proposing the stored continuation as a draft, verifying all proposed tokens with one parallel pass of the main decoder, and keeping tokens until the first mismatch. The matching and verification loop, together with the pruning rule derived from the paper's timing analysis (only propose when candidate sequences are long enough to beat autoregressive cost), carries the argument.","core_discovery":"Token Map Drafting is a model-free speculative decoding algorithm for Whisper-family ASR. A token map is constructed by tokenizing domain training transcripts, extracting n-grams of lengths 1 through N as keys, recording the token sequences that follow each n-gram as candidate drafts, and pruning candidates by length and frequency. At decode time, the current decoder state's last tokens are matched against map keys; matching candidate continuations are proposed, verified in parallel by the main model, and accepted or truncated back to the first unmatched token. The paper reports 1.27x and 1.37x decoding speedups on CI-AVSR and an internal maintenance-command dataset, and claims these come without degrading transcription accuracy and with a 10% absolute speed advantage over a distilled-draft-model baseline on CPU.","pith_inferences":["Editorial extension: because the token map is built from domain training transcripts, the same drafting idea should transfer to other autoregressive sequence models in closed-vocabulary, structured domains, but only if the tokenizer and domain text statistics are stable enough that n-gram continuations are reliable.","Editorial extension: the headline speedups are averages over utterances; in a long tail of utterances with no matching n-gram, the method falls back to autoregressive decoding, so tail latency could be much worse than the mean suggests.","Editorial extension: the optimal n-gram length and pruning threshold are likely domain-dependent; the paper measures a peak at N=3 on one internal dataset, and a different domain perplexity could shift that optimum."],"forward_implications":["On CI-AVSR, Token Map Drafting speeds Whisper-large-v3 decoding by 1.27x compared with autoregressive decoding, with the paper reporting no transcription accuracy degradation.","On the internal maintenance-task dataset, the speedup reaches 1.37x with Whisper-large-v3, and 1.35x to 1.36x with Whisper-medium and Whisper-small, so the gain is not tied to the largest model.","Compared with a distilled Whisper draft model on CPU, the token map gives roughly a 10 percentage point speedup advantage on the internal dataset, despite a shorter average acceptance length, showing that the draft model's overhead was a major bottleneck.","Because no draft model needs to be trained or loaded, the method is directly deployable on CPU-only and memory-constrained devices for domain-specific ASR.","The method is designed for structured, low-perplexity domains such as in-car commands or measurement reports; its benefit shrinks on diverse, spontaneous speech, where the reported CI-AVSR acceptance rate is lower."],"supporting_citations":[{"why":"Introduces speculative decoding, the framework the paper adapts by replacing the draft model with a token map.","marker":"[12]"},{"why":"Introduces Whisper, the transformer ASR model family whose decoder the paper accelerates.","marker":"[2]"},{"why":"Supplies the Distill-spec baseline (a distilled draft model) that Token Map Drafting is compared against on CPU.","marker":"[15]"},{"why":"Represents the multi-head speculative decoding extension that the paper positions against as requiring architecture changes.","marker":"[16]"},{"why":"Provides the CI-AVSR dataset used for the 1.27x speedup result.","marker":"[19]"}],"fun_headline_variants":["Whisper decoding hits 1.37x speedup on CPU with token maps","Model-free draft speeds ASR 1.37x, no accuracy cost","Token map drafting gives 1.37x faster CPU speech decoding","N-gram token maps accelerate Whisper on CPU by 1.37x","Draftless speculative decoding accelerates ASR on CPU 1.37x"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central premise is that accepting draft tokens whose conditional probability clears a fixed threshold yields exactly the tokens the main model would have produced; the paper does not report any accuracy metric, so the claim that accuracy is preserved depends on that output-equivalence assumption.","fun_headline_variants_meta":{"raw":{"variants":["Whisper decoding hits 1.37x speedup on CPU with token maps","Model-free draft speeds ASR 1.37x, no accuracy cost","Token map drafting gives 1.37x faster CPU speech decoding","N-gram token maps accelerate Whisper on CPU by 1.37x","Draftless speculative decoding accelerates ASR on CPU 1.37x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1444,"prompt_tokens":930,"completion_tokens":514,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":412}},"tokens_in":546,"tokens_out":514,"duration_ms":5563,"temperature":1.0,"reasoning_tokens":412,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T12:39:23.600421+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure word error rate on both datasets with Token Map Drafting versus unmodified Whisper decoding on the same audio; if the WER rises, the central claim of no accuracy loss falls. The paper provides speed and acceptance metrics but no accuracy table, so this measurement is the decisive check.","supporting_citations":[{"cited_title":"Fast Inference from Transformers via Speculative Decoding,","cited_arxiv_id":null,"evidence_quote":"Introduces speculative decoding, the framework the paper adapts by replacing the draft model with a token map."},{"cited_title":"Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR","cited_arxiv_id":"2409.15869","evidence_quote":"Represents the multi-head speculative decoding extension that the paper positions against as requiring architecture changes."}],"review_version":1}