{"id":"a4db973f-c72d-4fd7-8bdf-696b661b534b","arxiv_id":"2501.03184","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Denoising autoregressive predictive coding pretraining improves target-speaker voice activity detection in noise by roughly two percentage points, with FiLM conditioning performing best.","lead":"This paper tests whether pretraining a speech model to predict clean future audio from noisy audio, without labels, makes a later target-speaker voice activity detector more robust to noise. It reports roughly two percentage points better accuracy in noisy conditions and finds that one conditioning method, FiLM, works best.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table III's claimed ~2% improvement is not reproducible from the per-model scores in Tables IV/V; the discrepancy (~0.6 pp) directly undermines the headline.","rationale":"The reader's weakest_assumption was the data-quantity confound: pretrained models see 960 hours of audio versus 100 hours for the supervised baseline, so the improvement could stem from data volume rather than the SSL objective. That is a valid concern, but the more immediate, load-bearing problem is an internal numerical inconsistency: Table III's claimed improvements cannot be reproduced from the per-model scores in Tables IV and V using standard averaging. This discrepancy is independent of any external confound and directly affects the credibility of the paper's headline result. The reader's rationale did mention reconciling Table III with Tables IV/V, but did not elevate it to the primary weak point; hence 'partial' agreement. A correct verdict remains CONDITIONAL: the paper should be accepted only if the authors either fix the table inconsistency or provide the missing per-SNR breakdown and show that the claimed effect survives both the arithmetic reconciliation and a controlled comparison that isolates SSL pretraining from additional data. No code or data were released, so independent verification is not currently possible, but the internal arithmetic check is a first necessary step.","tokens_in":17462,"tokens_out":6526,"duration_ms":56993,"concrete_test":"Recompute Table III from Tables IV and V: for each of the five conditioning methods, compute DN-APC minus Baseline for mAP and for each class AP, averaging over the listed SNR levels and noise conditions; then average these differences across the five methods. Compare the resulting clean/seen/unseen mAP and class AP averages to Table III's 2.15/2.18/2.21 and 2.06/2.26/2.11 values. If the recomputed values are approximately 1.3-1.7 pp rather than 2.1-2.2 pp, the central numeric claim is contradicted by the paper's own tables and requires correction or explicit reconciliation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim rests on Table III, which reports average DN-APC improvements of 2.15% (clean), 2.18% (seen noise), and 2.21% (unseen noise) in mAP, with class-level seen-noise averages of 2.06/2.26/2.11 for ns/ts/nts. However, these numbers are not consistent with the detailed per-conditioning scores in Tables IV and V. Averaging the five model-specific mAP differences from Table IV gives only about 1.61 pp for seen noise and 1.63 pp for unseen noise; class AP differences average about 1.56/1.72/1.55 pp (seen), versus the 2.06/2.26/2.11 pp in Table III. Clean mAP differences from Table IV average about 1.34 pp, versus 2.15 pp claimed. Since both tables state they are averages over the same five conditioning methods and the same SNR levels, the arithmetic is internally inconsistent. This is load-bearing because the headline 'approx. 2% improvement' is not a faithful summary of the paper's own results. The discrepancy is about 0.5-0.8 percentage points, which is roughly a quarter to a third of the claimed effect, and it appears before considering the data-quantity confound. The authors must either correct Table III or specify the exact averaging (e.g., per-SNR tables, different baseline, or another weighting) that reconciles the two presentations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes Denoising Autoregressive Predictive Coding (DN-APC), a causal self-supervised pretraining objective, for target-speaker voice activity detection (TS-VAD) in noisy conditions. The authors pretrain a small causal Conformer encoder on LibriSpeech data with a denoising future-frame prediction loss, then fine-tune it for TS-VAD under five speaker-conditioning schemes (concatenation, addition, multiplication, FiLM, and FiLM with embedding preprocessing). They report that DN-APC pretraining improves TS-VAD mean average precision by about 2% in clean, seen-noise, and unseen-noise conditions, and that FiLM conditioning performs best overall. They also present t-SNE visualizations of the hidden representations, arguing that pretraining provides better speech/non-speech separation before fine-tuning.","tokens_in":17820,"tokens_out":5351,"duration_ms":46778,"significance":"If the reported effect is real, the paper makes a useful contribution to an underexplored area: self-supervised pretraining for low-complexity, task-specific audio models rather than large general-purpose SSL backbones. The causal formulation of the pretext task is appropriate for streaming TS-VAD, and the systematic comparison of five conditioning methods under matched architecture and training settings is informative. The paper also provides an unusually detailed experimental protocol, including data construction, augmentation, and model configurations, which aids reproducibility. However, the central quantitative claim currently has internal arithmetic inconsistencies and an experimental confound that must be resolved before the contribution can be assessed.","major_comments":[{"comment":"Section VI-A and the abstract claim average DN-APC improvements of 2.18% (seen noise) and 2.21% (unseen noise) in mAP, with class-level averages of 2.06/2.26/2.11 and 1.89/2.57/2.18 for ns/ts/nts. These numbers are not reproducible from Tables IV and V, which are stated to be averages over the same five conditioning methods and the same SNR levels. Averaging the five per-conditioning mAP differences from Table IV gives about 1.61 pp for seen noise and 1.63 pp for unseen noise from Table V, not 2.18 and 2.21. The class AP averages also differ: seen ns/ts/nts average about 1.56/1.72/1.55 pp versus the reported 2.06/2.26/2.11 pp, and clean mAP averages about 1.34 pp versus the reported 2.15 pp. Since both presentations purport to be averages over the identical set of experiments, this is an internal inconsistency in the headline result. Please correct Table III and the abstract, or explicitly state the different aggregation rule used in each table.","section":"Section VI-A, Tables III-V"},{"comment":"The comparison does not isolate the effect of self-supervised pretraining. Pretrained models are trained on train-clean-100, train-clean-360, and train-other-500 (approximately 960 h of audio), while the supervised baseline is trained only on train-clean-100 (100 h). The paper attributes the improvement to the DN-APC objective, but the pretrained models also see roughly ten times more data, so the gain could come from data quantity rather than the self-supervised pretext. Please add a control experiment where the supervised baseline is trained on the same 960 h of labeled data, or where DN-APC pretraining is restricted to train-clean-100, and report whether the improvement persists.","section":"Section V-A, Table I"},{"comment":"No statistical significance tests or confidence intervals are reported for the key comparisons, and several improvements are small relative to the displayed variation. For example, the Concat model shows a seen-noise mAP gain of 1.04 pp while the DN-APC standard deviation is 0.95, and the FiLM+Pre model shows a clean mAP gain of 0.64 pp with base standard deviation 0.43. Given that the central claim is a general ~2% improvement, the authors should report paired significance tests (e.g., across SNR levels or across conditioning methods) or otherwise quantify the uncertainty of the average differences.","section":"Tables IV and V"}],"minor_comments":[{"comment":"There is a typo in the gradient-clipping sentence: \"nornalizing\" should be \"normalizing.\"","section":"Section V-C"},{"comment":"The sentence \"After embedding the representations in a two-dimensional space\" is incomplete and should be finished or merged with the following sentence.","section":"Section VI-C"},{"comment":"The claim of robustness to unseen noise is based on a single held-out noise type (café). Please qualify the conclusion as applying to this unseen condition, or evaluate on additional unseen noise types to support the broader generalization claim.","section":"Section V-A"},{"comment":"In the DN-APC objective, the notation uses y_n for both the clean features used as prediction targets and, via y-hat_n, the noisy input features; this is understandable but could be clarified by defining the augmented-feature sequence explicitly.","section":"Equation (18)"},{"comment":"The entry \"149k (+263k)\" for the FiLM+Pre model is unclear; please specify whether the parenthetical is the size of the preprocessing module or the total with the frozen embedding model.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an extension of the authors' ICASSP 2024 work, which is cited as [27]. The novelty lies mainly in the Conformer architecture, the systematic conditioning-method comparison, and the representation analysis. The central empirical claim needs to be made internally consistent and the data-size confound addressed; neither issue seems impossible to fix within the scope of a revision. I did not find evidence of a circular derivation: Eq. (18) is an adopted objective, and the downstream metrics are measured empirically. The self-citation pattern is concentrated in [27] and [38] and is not excessive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read. The paper's headline claim—about 2% mAP improvement from DN-APC pretraining—does not survive contact with its own tables. I recomputed the per-conditioning differences from Tables IV and V. Averaged over the five conditioning methods, the seen-noise mAP gain is about 1.61 pp, unseen about 1.63 pp, and clean about 1.34 pp. Table III claims 2.18, 2.21, and 2.15. The class-level numbers are similarly off (e.g., seen ns 1.56 vs 2.06, ts 1.72 vs 2.26, nts 1.55 vs 2.11). That's a quarter to a third of the claimed effect. Either Table III is computed differently than stated, or there's an arithmetic error. The authors need to reconcile this, ideally by releasing the per-SNR and per-conditioning means.\n\nThe second issue is the data-quantity confound. Pretrained models see 960 h of unlabelled audio; the supervised baseline sees only 100 h. The paper attributes the gain to SSL, but it could easily be mostly a data effect. A controlled ablation—same training data volume for both—would settle this. Without it, the core claim is not yet established.\n\nWhat's genuinely good: the systematic comparison of five speaker-conditioning methods (concat, add, multiply, FiLM, FiLM+pre) is useful and clearly presented; the causal Conformer setup is described with enough detail to reproduce; and the t-SNE analysis, though qualitative, hints at why pretraining helps noise robustness. The authors also honestly credit their prior work on DN-APC, so the self-citation is appropriate.\n\nOther soft spots: no significance tests or confidence intervals on the differences, no code or trained models released, and the t-SNE evidence is not quantified. These are common in the subfield, but they matter more here because the effect size is modest and the tables disagree.\n\nWho should read this? People working on TS-VAD or low-complexity SSL for audio. It's a worthwhile empirical study, but the headline number needs correcting. Send it out for review—the experiments are worth refereeing—but with a clear request to fix the table inconsistency, add a data-amount ablation, and report uncertainty. As is, I would not cite the 2% figure.","headline":"The headline \"~2% improvement\" is a misreading of the paper's own tables; the real gain is ~1.3-1.6 pp, and a data-amount confound blurs the SSL attribution.","tokens_in":18348,"tokens_out":4897,"would_cite":false,"duration_ms":37584,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretraining a causal Conformer encoder with Denoising Autoregressive Predictive Coding makes target-speaker voice activity detection models roughly 2% more accurate in both seen and unseen background noise, while also reducing the need…","keywords":["self-supervised learning","target-speaker voice activity detection","noise robustness","autoregressive predictive coding","speaker conditioning","conformer","denoising pretraining"],"falsifier":"Train the supervised baseline on the same 960 hours of audio used for pretraining and compare its noisy-condition accuracy to the DN-APC-pretrained model; if the full-data supervised model matches the pretrained model, the gain is a data effect rather than an SSL effect.","tokens_in":17336,"feed_emoji":"🎙️","tokens_out":6307,"duration_ms":49385,"temperature":0.7,"pith_summary":"The paper argues that a causal self-supervised pretraining objective, Denoising Autoregressive Predictive Coding (DN-APC), makes target-speaker voice activity detection (TS-VAD) models more robust to background noise, improving average precision by roughly 2% on both noise types seen during training and held-out noise. The motivation is that TS-VAD models normally need expensive labelled data, while unlabelled audio is cheap; DN-APC pretrains the encoder to predict clean future speech features from noisy, reverberated input. The authors also compare five speaker-conditioning methods and find that FiLM conditioning gives the best overall performance, while multiplication conditioning is best for detecting target-speech frames. A representation analysis shows that the pretrained encoder already separates speech from non-speech in noise before fine-tuning, while target-versus-non-target speaker discrimination only emerges during supervised fine-tuning.","feed_headline":"Pretraining boosts target-speaker VAD by ~2% in noise","feed_subtitle":"A denoising pretraining step makes voice-activity detection for a known speaker more robust to unseen background noise.","key_machinery":"The central object is the DN-APC learning objective, $\\mathcal{L}_{\\text{DN-APC}} = \\sum_{n=0}^{N-k-1} \\lVert g(h(\\hat{y}_n)) - y_{n+k} \\rVert_1$, where $h$ is the causal Conformer encoder, $\\hat{y}_n$ are logMel features computed from noise- and reverb-augmented audio, $g$ is a 1D convolution with kernel size 1, and $y_{n+k}$ are clean logMel features $k=3$ frames ahead. This objective combines autoregressive predictive coding with denoising, forcing the encoder to extract noise-robust representations that are useful for downstream TS-VAD. The supporting mechanism for injecting target-speaker information is speaker conditioning, implemented as concatenation, addition, multiplication, FiLM, or FiLM with a nonlinear embedding preprocessing step; FiLM performs best overall in the paper's experiments.","core_discovery":"The central claim is that pretraining a causal Conformer encoder with DN-APC, which predicts clean logMel features three frames ahead from noise- and reverb-augmented input using an $\\ell^1$ loss, improves the noise robustness of TS-VAD models over the same architecture trained from scratch, with no need for labels during pretraining. Across all conditioning methods and SNR levels from -5 dB to 20 dB, pretrained models improve over supervised baselines by about 2.15% in clean conditions, 2.18% on seen noise, and 2.21% on unseen noise. The paper further claims that FiLM conditioning yields the best overall mean average precision, and that pretraining creates initial representations that separate speech from non-speech even at low SNR, while speaker identity discrimination is learned during fine-tuning.","pith_inferences":["If the improvement is driven by the SSL objective rather than the larger pretraining corpus, similar gains should appear when the supervised baseline is trained on the same 960 hours of data; the paper does not run this control, so the data-quantity confound remains open.","The causal, streaming-friendly nature of DN-APC suggests it could transfer to other causal speech tasks such as keyword spotting or streaming speaker diarization with minimal modification.","A natural extension would be to make the pretraining objective speaker-aware, since the current DN-APC only separates speech from non-speech and leaves target-speaker discrimination to fine-tuning, as the tSNE analysis shows.","The tSNE analysis is qualitative; a quantitative measure of class separation, such as linear probing or cluster purity, could confirm whether the pretrained representations are genuinely more separable in noise."],"forward_implications":["TS-VAD models pretrained with DN-APC generalize better to unseen noise types than models trained from scratch on the same labelled subset.","The improvement holds across all three output classes (non-speech, target-speech, non-target-speech), with target-speech detection gaining most in clean conditions and non-speech detection gaining most at low SNR.","FiLM speaker conditioning is the most accurate overall conditioning method, while multiplication conditioning yields the best target-speech detection, suggesting that simple conditioning is sufficient for this task.","Because pretraining uses unlabelled audio, the labelled-data requirement for training noise-robust TS-VAD models can be reduced, which matters for hearing aids and streaming speech recognition where labels are scarce.","The pretrained encoder's internal representations already separate speech from noise before fine-tuning, which explains its robustness at low SNR and indicates that SSL pretraining can supply a useful speech/noise prior."],"supporting_citations":[{"why":"Supplies the DN-APC framework and previous evidence that SSL pretraining improves noise robustness of an LSTM-based TS-VAD model, which this paper extends to a Conformer.","marker":"[27]"},{"why":"Provides the speaker-conditioned TS-VAD baseline (PersonalVAD) and the method for generating multi-speaker utterances from single-speaker corpora.","marker":"[8]"},{"why":"Supplies the causal Conformer encoder and FiLM conditioning that the paper adopts and compares against the other conditioning methods.","marker":"[9]"},{"why":"Defines Autoregressive Predictive Coding, the causal SSL objective that DN-APC extends with denoising and reverberation augmentation.","marker":"[41]"},{"why":"Provides evidence that SSL representations are more robust to input corruptions such as noise, motivating the robustness claim.","marker":"[37]"},{"why":"Supplies the LibriSpeech corpus used for both supervised training and SSL pretraining, including the clean/other splits and speaker metadata.","marker":"[44]"}],"fun_headline_variants":["Self-supervised pretraining sharpens target-speaker VAD in noise","Pretraining makes speaker-aware VAD robust to unseen noise","Tiny pretraining lift: 2% better VAD in noise","DN-APC pretraining boosts noisy TS-VAD by 2%","FiLM conditioning wins for noise-robust TS-VAD"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claimed benefit of self-supervised pretraining is not cleanly separated from the fact that pretrained models are trained on roughly ten times more audio data than the supervised baselines, so the improvement could come from data quantity rather than the pretraining objective itself.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised pretraining sharpens target-speaker VAD in noise","Pretraining makes speaker-aware VAD robust to unseen noise","Tiny pretraining lift: 2% better VAD in noise","DN-APC pretraining boosts noisy TS-VAD by 2%","FiLM conditioning wins for noise-robust TS-VAD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000504,"raw_usage":{"total_tokens":2447,"prompt_tokens":915,"completion_tokens":1532,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":531,"completion_tokens_details":{"reasoning_tokens":1440}},"tokens_in":531,"tokens_out":1532,"duration_ms":10051,"temperature":1.0,"reasoning_tokens":1440,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:43.962882+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the supervised baseline on the same 960 hours of audio used for pretraining and compare its noisy-condition accuracy to the DN-APC-pretrained model; if the full-data supervised model matches the pretrained model, the gain is a data effect rather than an SSL effect.","supporting_citations":[{"cited_title":"Self-supervised pretraining for robust personalized voice activity detection in adverse conditions,","cited_arxiv_id":null,"evidence_quote":"Supplies the DN-APC framework and previous evidence that SSL pretraining improves noise robustness of an LSTM-based TS-VAD model, which this paper extends to a Conformer."},{"cited_title":"Per- sonal V AD: Speaker-conditioned voice activity detection,","cited_arxiv_id":null,"evidence_quote":"Provides the speaker-conditioned TS-VAD baseline (PersonalVAD) and the method for generating multi-speaker utterances from single-speaker corpora."},{"cited_title":"Personal V AD 2.0: Optimizing personal voice activity detection for on-device speech recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies the causal Conformer encoder and FiLM conditioning that the paper adopts and compares against the other conditioning methods."},{"cited_title":"Autore- gressive predictive coding: A comprehensive study,","cited_arxiv_id":null,"evidence_quote":"Defines Autoregressive Predictive Coding, the causal SSL objective that DN-APC extends with denoising and reverberation augmentation."},{"cited_title":"Using self- supervised learning can improve model robustness and uncertainty,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that SSL representations are more robust to input corruptions such as noise, motivating the robustness claim."}],"review_version":1}