{"id":"29c75420-4292-4671-aeee-b8060058703a","arxiv_id":"2607.05364","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage replay-based post-training method corrects ASR timestamp drift across non-speech gaps while preserving recognition far better than ordinary timestamp fine-tuning.","lead":"Speech-AI models that write their own timestamps can place correctly recognized words at the wrong times after long silences, and ordinary fine-tuning that fixes this wrecks their other skills. The authors' REDDIT method edits timestamps under the model's own replay context while anchoring the rest of its output, cutting average timing error from about 2.75 seconds to 0.22 seconds in held-out tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Anti-forgetting claim hinges on teacher-forced KL; own no-gap OOD numbers show CV-en 37.0→41.3 and ASCEND-zh 53.9→63.5 on tiny single-run sets — free-decoding transfer is unproven.","rationale":"The reader's weakest assumption precisely identifies the same load-bearing concern: the anti-forgetting guarantee is formulated under teacher-forced contexts, but the deployment setting is free decoding. My analysis confirms this is the critical soft spot because it targets the 'without forgetting' half of the central claim, which is the paper's novel contribution over naive timestamp fine-tuning. The paper's own OOD no-gap table already shows non-trivial degradation (CV-en +11.6% relative, ASCEND-zh +17.8% relative) on small, single-run sets, so the empirical support is thin. The concern is not that the method fails outright—the retention is far better than SFT—but that the distribution-preservation mechanism is not validated under self-generated contexts. A straightforward multi-seed, large-scale OOD evaluation with a rollout-KL diagnostic would settle whether the teacher-forced KL anchor transfers. Given this, the reader's CONDITIONAL verdict remains appropriate; no stronger rejection is warranted because the timestamp-correction results are strong and the forgetting is bounded relative to baselines.","tokens_in":16079,"tokens_out":5796,"duration_ms":58051,"concrete_test":"Run REDDIT Full and the frozen base on a large no-gap OOD benchmark (e.g., LibriSpeech test-clean or FLEURS English, >5k utterances) with at least 5 random seeds. Report mean±std MER and the per-token KL between base and REDDIT predictive distributions under each model's own free-decoding rollouts. If the rollout KL substantially exceeds the training-time teacher-forced KL (e.g., >2x) or the relative MER degradation exceeds 10%, the 'without forgetting' claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two halves: timestamp correction and no ASR forgetting. The correction half is well supported on synthetic in-domain tests. The 'without forgetting' half relies on L_text (Eq. 7) matching the frozen base distribution on non-timestamp tokens. But this KL is applied per-position under teacher-forced contexts only: cached replay prefixes in Stage 1, edited prefixes in Stage 2 (Eq. 9). It does not constrain the student under its own free-decoding rollouts, where the generated timestamps are neither the cached drifted ones nor the exact edited targets. Since the KL is not evaluated on those self-generated contexts, the divergence in free decoding is only indirectly bounded. The paper's own Table VII shows no-gap OOD MER degradation: CV-en 37.0→41.3 and ASCEND-zh 53.9→63.5, on small single-run sets (214–2997 samples, Table II) with no error bars. These are not catastrophic like SFT decoder's 524.2%, but they contradict a strict 'without forgetting' reading. If free-decoding transfer fails more broadly on other domains or longer audio, the distribution-editing guarantee collapses, even though the timestamp-correction half survives.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies a failure mode in autoregressive ASR systems that emit timestamps as decoded tokens: across long non-speech spans, the decoded timeline can drift while the transcript remains plausible. It introduces a controlled gap/long-gap benchmark across 15 timestamp-producing ASR/audio-language systems and shows that ordinary timestamp-corrected fine-tuning repairs alignment but catastrophically degrades non-target recognition. The proposed method, REDDIT, is a two-stage post-training procedure: Stage 1 edits timestamp targets under a cached base-model replay context while KL-matching the frozen base distribution on non-timestamp positions (Eqs. 6–8); Stage 2 refines under edited-prefix context (Eq. 9), using the Stage-1 checkpoint as the KL teacher. Training data are constructed by VAD-trimming speech spans and inserting non-speech gaps, with exact timestamps derived from the splice offsets. On Whisper-tiny, updating only 1.6% of parameters, REDDIT improves long-gap mIoU from 38.7% to 95.0% and mixed-gap OOD AAS from 2752 ms to 223 ms, while CV-en MER rises to 41.3% compared to 524.2% for full-decoder SFT. The central claim is that timestamp drift can be corrected without ASR forgetting, using no inference-time VAD, forced alignment, or post-processing.","tokens_in":16202,"tokens_out":6312,"duration_ms":78321,"significance":"If the results hold, this is a useful and practical contribution: it offers a parameter-efficient, annotation-free way to repair a model's native timestamp outputs without adding inference-time alignment modules. The controlled synthetic construction with exact reference timestamps is a strength, as is the breadth of the diagnostic benchmark across 15 systems, which cleanly separates temporal drift from lexical hallucination. The paper is also transparent about several limitations, including checkpoint selection by validation behavior and the small single-run OOD evaluation sets. However, the 'without forgetting' half of the central claim is only partially supported by the paper's own numbers: Table VII shows measurable no-gap recognition degradation on CV-en and ASCEND-zh, so the claim needs to be reframed and the retention mechanism tested under free-decoding contexts.","major_comments":[{"comment":"The paper's headline and conclusion state that REDDIT preserves non-target ASR behavior, but Table VII shows no-gap CV-en MER rising from 37.0% (base) to 41.3% (REDDIT Full) and ASCEND-zh from 53.9% to 63.5%. These are the authors' own numbers on sets of 2,997 and 578 samples, respectively, with no error bars or repeated runs. The abstract's 'preserving CV-en MER at 41.3%' is therefore better described as 'substantially less forgetting than full-decoder SFT' than as absolute preservation. Please report base-relative degradations, confidence intervals, and a direct statistical comparison; otherwise the 'without forgetting' claim is overstated.","section":"Section V-C / Table VII"},{"comment":"The anti-forgetting anchor is a per-position KL divergence computed under teacher-forced contexts: cached replay prefixes in Stage 1 and edited prefixes in Stage 2. It does not constrain the student under its own free-decoding rollouts, where the context includes self-generated timestamp tokens that are neither the cached drifted ones nor the edited targets. Moreover, in Stage 2 the KL teacher is the Stage-1 checkpoint rather than the frozen base, so any Stage-1 drift is not corrected. Since the KL is not evaluated on self-generated contexts, the free-decoding divergence is only indirectly bounded; Table VII's degradation is consistent with this gap. A concrete test would be to compute a rollout-based KL or report OOD no-gap MER with self-generated decoding on longer audio.","section":"Eqs. (7), (9) / Sec. III-D"},{"comment":"The paper states: 'We select the reported full-pipeline checkpoint from this short refinement trajectory by validation behavior rather than treating a fixed step count as part of the method.' This introduces a best-of-trajectory selection for REDDIT Full, but the manuscript does not define what 'validation behavior' means nor whether the validation set overlaps with the reported Gap/Long-Gap test sets. If the validation set includes any of the reported evaluation splits, the headline numbers are not a fixed-procedure estimate. Please specify the selection rule, the validation data, and the sensitivity of the results to the chosen checkpoint.","section":"Sec. IV-C"},{"comment":"Correction examples are pre-filtered to remove 'hallucinations, repetitions, boilerplate, empty or unusable text, and structurally inconsistent timestamps,' retaining examples with exactly 2K well-ordered timestamp tokens. The paper does not report how many cached teacher replays are discarded or the filtering rate per split. If a large fraction of base-model outputs is filtered out, the method may only be demonstrated on already well-structured trajectories, and the '34.9 hours of targeted correction audio' is not reproducible without this rate. Please report filtering counts and characterize the discarded examples.","section":"Sec. III-F"}],"minor_comments":[{"comment":"The caption states that 'DRIFT AND HALLUCINATION COLUMNS ARE FILE-LEVEL RATES,' but the metric definitions in Sec. IV-D.d describe Drift>5s and Drift>10s as percentages of matched segments. This is inconsistent and should be corrected.","section":"Table IV caption"},{"comment":"The abstract's 'preserving CV-en MER at 41.3%' is misleading without an explicit base of 37.0%; please give the base value as well, e.g., '41.3% vs. 37.0% base and 524.2% SFT decoder.'","section":"Abstract / Table VII"},{"comment":"No code, data, or trained checkpoints are linked. Given the synthetic data construction and replay-cache pipelines, a release of the construction code would materially improve reproducibility.","section":"Sec. IV-C"},{"comment":"The sentence 'REDDIT Full stays close to the base model on CV-en and ASCEND-en' is too strong given the 4.3-point CV-en and 9.6-point ASCEND-zh degradations in Table VII; consider qualifying with 'closer than the baselines' or reporting a paired significance test.","section":"Sec. V-C"}],"recommendation":"major_revision","confidential_remarks":"The empirical core is solid and publishable after revision: the timestamp-correction results are strong, the synthetic data design is clean, and the comparison against full-decoder SFT is persuasive. The main risk is overclaiming the 'without forgetting' half; the authors should rephrase the central claim, add uncertainty quantification, and tighten the checkpoint-selection protocol. I do not see a fundamental flaw in the method itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent and useful paper. It documents a real failure mode—autoregressive ASR models can output plausible transcripts on the wrong part of the timeline when non-speech gaps are long—and it gives the field a reusable benchmark (the GAP/LONG-GAP splits, 15-system comparison) plus a parameter-efficient post-training recipe that fixes timestamps without destroying recognition. The numbers are internally consistent, the method description is coherent, and the timestamp supervision is genuinely external: targets are exact splice offsets, so there is no circularity problem.\n\nThe strongest evidence is the Whisper-tiny result. 34.9 hours of synthetic correction audio, 1.6% of parameters, long-gap mIoU from 38.7 to 95.0, mixed-gap OOD AAS from 2752 ms to 223 ms, and no-gap CV-en MER at 41.3% versus 524.2% for full-decoder SFT. That contrast is the paper's core point and it lands.\n\nThe soft spots are real but not fatal. First, the 'without forgetting' half is only partially supported. The KL anchor (Eq. 7) is computed under teacher-forced, cached replay contexts, not under the student's own free-decoding rollouts. The paper's own no-gap OOD numbers show CV-en MER rising from 37.0 to 41.3 and ASCEND-zh from 53.9 to 63.5, on small, single-run sets (214–2997 samples) with no error bars. That is not catastrophic forgetting, but it is not 'without forgetting' in the strict sense. The authors should either soften the wording or supply more transfer evidence. Second, the reported REDDIT Full checkpoint is selected by validation behavior (Sec. IV-C), so the headline numbers are best-of-trajectory; full curves or seed averaging would be more honest. Third, the replay pre-filter survival rate is undisclosed, and the target evaluation is same-distribution synthetic data. These are all addressable in revision.\n\nThe citation pattern and related work look reasonable. The self-citations are to the authors' own prior speech-adaptation work and are used as background, not as a load-bearing assumption. The minor textual inconsistency in the header author name ('Ming-Douo Tchouang') is odd but scientifically irrelevant.\n\nThis paper is for speech/ASR researchers who care about timestamped transcription, subtitling, or long-form captioning. It deserves a serious referee: the diagnostic is reusable, the method is simple and clearly explained, and the failure mode is real. I'd send it to peer review with a request that the authors address the retention-transfer and checkpoint-selection issues.","headline":"A genuinely useful diagnostic and a clever lightweight fix for ASR timestamp drift, with the 'no forgetting' half of the claim only partially proven.","tokens_in":16899,"tokens_out":2329,"would_cite":true,"duration_ms":23286,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that timestamp drift in autoregressive ASR can be repaired post-training by editing only timestamp tokens while anchoring the lexicon, so corrected timestamps come natively at decoding time.","keywords":["automatic speech recognition","timestamp drift","non-speech gaps","replay-based distribution editing","catastrophic forgetting","Whisper","timestamp tokens","post-training model editing"],"falsifier":"Run the REDDIT-trained Whisper-tiny on a large, previously unseen no-gap corpus and measure MER against the base model; if the MER gap exceeds the small degradation reported on the paper's out-of-domain sets, the 'without forgetting' claim fails. More directly, compare the student's non-timestamp token distribution under free decoding (no teacher forcing) with the frozen base's distribution: if that divergence is large while the teacher-forced KL is small, the anchor does not bound free decoding.","tokens_in":15781,"feed_emoji":"⏱️","tokens_out":9235,"duration_ms":89120,"temperature":0.7,"pith_summary":"Autoregressive ASR that emits timestamps as decoded tokens can produce a plausible transcript while placing it on the wrong part of the audio timeline, especially after long silences. The paper argues this drift is a distinct failure mode, separate from word error and hallucination, and shows it across 15 timestamp-producing systems. It then proposes REDDIT, a two-stage post-training method that corrects timestamp tokens while preserving the recognizer's non-timestamp behavior by replaying the frozen model's own outputs as context and matching its distribution on ordinary text tokens. With 34.9 hours of automatically spliced correction audio and 1.6% of parameters updated, the method raises long-gap temporal overlap on Whisper-tiny from 38.7% to 95.0% and cuts out-of-domain timestamp error from 2752 ms to 223 ms, while avoiding the severe recognition loss that naive timestamp fine-tuning causes. If correct, this means generated timestamps can be repaired natively, with no inference-time aligner, VAD, or DTW.","feed_headline":"ASR timestamp drift fixed with 1.6% of weights","feed_subtitle":"Replay-based edit lifts long-gap timestamp accuracy from 38.7% to 95.0% while keeping recognition close to base.","key_machinery":"The load-bearing object is the replay context: the frozen base model's own decoded token sequence, cached once and used as the teacher-forcing prefix for both losses. Timestamp tokens in that replay are replaced by corrected targets, computed from VAD-trimmed speech spans and inserted non-speech gaps with known offsets, and trained with cross-entropy; non-timestamp positions are trained to match the frozen teacher's distribution through KL divergence under the same context. This decouples temporal editing from lexical preservation. Stage 2 swaps the context to the edited prefix and uses the Stage-1 model as teacher.","core_discovery":"The central claim is that timestamp drift across non-speech gaps is an edit of the time axis, not a failure of speech recognition, and that this axis can be corrected in place: REDDIT overwrites each replayed timestamp token with an exact boundary target derived from a synthetic splicing schedule, trains the student under the frozen model's cached decoder prefix, and uses KL divergence on every non-timestamp position to keep the lexical distribution anchored to the base model. A short second stage re-trains the same objective under the edited prefix to consolidate the corrected transitions. On Whisper-tiny the intervention updates only the last cross-attention and layer norms, using no human","pith_inferences":["If the decoupling holds beyond Whisper, the replay-edit recipe is a general model-editing pattern: pick one output axis to rewrite, replay the base model's context, and distill everything else. A direct test is applying it to word-level timestamps or to punctuation and speaker-turn tokens.","Because the synthetic splice construction is label-free, the pipeline can be pointed at any language or domain the base model already transcribes; a testable extension is multilingual timestamp correction with per-language drift measurement.","The fact that only the last cross-attention and layer norms need updating suggests timestamp placement is a localized readout in the decoder; a falsifiable corollary is that drift severity is predictable from cross-attention statistics over silent regions, which could guide where to collect correction data."],"forward_implications":["Corrected timestamps are produced natively at decoding time, so a REDDIT-processed model needs no VAD, forced alignment, DTW, or attention-head selection.","The 34.9-hour, 0.59M-parameter intervention on Whisper-tiny and the 0.43% parameter update on Whisper-large-v3 indicate timestamp repair can be a small post-training step rather than a full retraining.","Because correction targets come from splicing offsets, training data require no human transcripts or human timestamp labels; the frozen model supplies pseudo-text and replay distributions.","The OOD mixed-gap result (AAS from 2752 ms to 223 ms) implies the correction transfers beyond the synthetic training distribution to unseen gap patterns and domains.","The contrast with SFT decoder tuning (41.3% vs 524.2% MER on English Common Voice no-gap) shows that without the non-timestamp KL anchor, timestamp fine-tuning collapses recognition, making forgetting a central part of the timestamp-correction objective."],"fun_headline_variants":["ASR timestamp drift fixed with just 1.6% of weights","Timestamp drift in ASR corrected without forgetting","Fix ASR time drift: 38.7% to 95% mIoU with 1.6% weights","Replay-based editing fixes ASR timestamp drift, keeps recognition","1.6% parameter update fixes ASR timestamp drift, preserves ASR"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that matching the frozen teacher's word-output distribution on non-timestamp positions under replayed contexts transfers to free-running decoding on unseen domains—something the paper only probes on small out-of-domain sets, where its own tables show CV-en no-gap MER degrading from 37.0% to 41.3% and ASCEND-zh from 53.9% to 63.5%, and where the KL anchor is measured under teacher-forced contexts rather than multi-step rollout.","fun_headline_variants_meta":{"raw":{"variants":["ASR timestamp drift fixed with just 1.6% of weights","Timestamp drift in ASR corrected without forgetting","Fix ASR time drift: 38.7% to 95% mIoU with 1.6% weights","Replay-based editing fixes ASR timestamp drift, keeps recognition","1.6% parameter update fixes ASR timestamp drift, preserves ASR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1504,"prompt_tokens":828,"completion_tokens":676,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":586}},"tokens_in":572,"tokens_out":676,"duration_ms":6664,"temperature":1.0,"reasoning_tokens":586,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:28:35.135680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the REDDIT-trained Whisper-tiny on a large, previously unseen no-gap corpus and measure MER against the base model; if the MER gap exceeds the small degradation reported on the paper's out-of-domain sets, the 'without forgetting' claim fails. More directly, compare the student's non-timestamp token distribution under free decoding (no teacher forcing) with the frozen base's distribution: if that divergence is large while the teacher-forced KL is small, the anchor does not bound free decoding.","supporting_citations":[],"review_version":2}