Pith. sign in

REVIEW 3 major objections 5 minor 53 references

Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pretraining a causal Conformer encoder with Denoising Autoregressive Predictive Coding makes target-speaker voice activity detection models roughly 2% more accurate in both seen and unseen background noise, while also reducing the need…

desk verdict The headline "~2% improvement" is a misreading of the paper's own tables; the real gain is ~1.3-1.6 pp, and a data-amount confound blurs the SSL attribution. read the letter →

arxiv 2501.03184 v1 pith:JOMNKNS2 submitted 2025-01-06 eess.AS cs.LGcs.SD

Holger Severin Bovbjerg (1) , Jan {O}stergaard (1) , Jesper Jensen (1 , 2) , Zheng-Hua Tan (1) ((1) Aalborg University , (2) Oticon A/S) This is my paper · ORCID
classification eess.AScs.LGcs.SD
keywords self-supervisedlearningtarget-speakervoiceactivitydetectionnoiserobustnessautoregressivepredictivecodingspeakerconditioningconformerdenoisingpretraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a causal self-supervised pretraining objective, Denoising Autoregressive Predictive Coding (DN-APC), makes target-speaker voice activity detection (TS-VAD) models more robust to background noise, improving average precision by roughly 2% on both noise types seen during training and held-out noise. The motivation is that TS-VAD models normally need expensive labelled data, while unlabelled audio is cheap; DN-APC pretrains the encoder to predict clean future speech features from noisy, reverberated input. The authors also compare five speaker-conditioning methods and find that FiLM conditioning gives the best overall performance, while multiplication conditioning is best for detecting target-speech frames. A representation analysis shows that the pretrained encoder already separates speech from non-speech in noise before fine-tuning, while target-versus-non-target speaker discrimination only emerges during supervised fine-tuning.

What carries the argument

The central object is the DN-APC learning objective, $\mathcal{L}_{\text{DN-APC}} = \sum_{n=0}^{N-k-1} \lVert g(h(\hat{y}_n)) - y_{n+k} \rVert_1$, where $h$ is the causal Conformer encoder, $\hat{y}_n$ are logMel features computed from noise- and reverb-augmented audio, $g$ is a 1D convolution with kernel size 1, and $y_{n+k}$ are clean logMel features $k=3$ frames ahead. This objective combines autoregressive predictive coding with denoising, forcing the encoder to extract noise-robust representations that are useful for downstream TS-VAD. The supporting mechanism for injecting target-speaker information is speaker conditioning, implemented as concatenation, addition, multiplication, FiLM, or FiLM with a nonlinear embedding preprocessing step; FiLM performs best overall in the paper's experiments.

What would settle it

Train the supervised baseline on the same 960 hours of audio used for pretraining and compare its noisy-condition accuracy to the DN-APC-pretrained model; if the full-data supervised model matches the pretrained model, the gain is a data effect rather than an SSL effect.

Watch

Extended reading notes

Core claim

The central claim is that pretraining a causal Conformer encoder with DN-APC, which predicts clean logMel features three frames ahead from noise- and reverb-augmented input using an $\ell^1$ loss, improves the noise robustness of TS-VAD models over the same architecture trained from scratch, with no need for labels during pretraining. Across all conditioning methods and SNR levels from -5 dB to 20 dB, pretrained models improve over supervised baselines by about 2.15% in clean conditions, 2.18% on seen noise, and 2.21% on unseen noise. The paper further claims that FiLM conditioning yields the best overall mean average precision, and that pretraining creates initial representations that separate speech from non-speech even at low SNR, while speaker identity discrimination is learned during fine-tuning.

Load-bearing premise

The claimed benefit of self-supervised pretraining is not cleanly separated from the fact that pretrained models are trained on roughly ten times more audio data than the supervised baselines, so the improvement could come from data quantity rather than the pretraining objective itself.

Editorial extensions

If this is right

  • TS-VAD models pretrained with DN-APC generalize better to unseen noise types than models trained from scratch on the same labelled subset.
  • The improvement holds across all three output classes (non-speech, target-speech, non-target-speech), with target-speech detection gaining most in clean conditions and non-speech detection gaining most at low SNR.
  • FiLM speaker conditioning is the most accurate overall conditioning method, while multiplication conditioning yields the best target-speech detection, suggesting that simple conditioning is sufficient for this task.
  • Because pretraining uses unlabelled audio, the labelled-data requirement for training noise-robust TS-VAD models can be reduced, which matters for hearing aids and streaming speech recognition where labels are scarce.
  • The pretrained encoder's internal representations already separate speech from noise before fine-tuning, which explains its robustness at low SNR and indicates that SSL pretraining can supply a useful speech/noise prior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the improvement is driven by the SSL objective rather than the larger pretraining corpus, similar gains should appear when the supervised baseline is trained on the same 960 hours of data; the paper does not run this control, so the data-quantity confound remains open.
  • The causal, streaming-friendly nature of DN-APC suggests it could transfer to other causal speech tasks such as keyword spotting or streaming speaker diarization with minimal modification.
  • A natural extension would be to make the pretraining objective speaker-aware, since the current DN-APC only separates speech from non-speech and leaves target-speaker discrimination to fine-tuning, as the tSNE analysis shows.
  • The tSNE analysis is qualitative; a quantitative measure of class separation, such as linear probing or cluster purity, could confirm whether the pretrained representations are genuinely more separable in noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This manuscript proposes Denoising Autoregressive Predictive Coding (DN-APC), a causal self-supervised pretraining objective, for target-speaker voice activity detection (TS-VAD) in noisy conditions. The authors pretrain a small causal Conformer encoder on LibriSpeech data with a denoising future-frame prediction loss, then fine-tune it for TS-VAD under five speaker-conditioning schemes (concatenation, addition, multiplication, FiLM, and FiLM with embedding preprocessing). They report that DN-APC pretraining improves TS-VAD mean average precision by about 2% in clean, seen-noise, and unseen-noise conditions, and that FiLM conditioning performs best overall. They also present t-SNE visualizations of the hidden representations, arguing that pretraining provides better speech/non-speech separation before fine-tuning.

Significance. If the reported effect is real, the paper makes a useful contribution to an underexplored area: self-supervised pretraining for low-complexity, task-specific audio models rather than large general-purpose SSL backbones. The causal formulation of the pretext task is appropriate for streaming TS-VAD, and the systematic comparison of five conditioning methods under matched architecture and training settings is informative. The paper also provides an unusually detailed experimental protocol, including data construction, augmentation, and model configurations, which aids reproducibility. However, the central quantitative claim currently has internal arithmetic inconsistencies and an experimental confound that must be resolved before the contribution can be assessed.

major comments (3)
  1. [Section VI-A, Tables III-V] Section VI-A and the abstract claim average DN-APC improvements of 2.18% (seen noise) and 2.21% (unseen noise) in mAP, with class-level averages of 2.06/2.26/2.11 and 1.89/2.57/2.18 for ns/ts/nts. These numbers are not reproducible from Tables IV and V, which are stated to be averages over the same five conditioning methods and the same SNR levels. Averaging the five per-conditioning mAP differences from Table IV gives about 1.61 pp for seen noise and 1.63 pp for unseen noise from Table V, not 2.18 and 2.21. The class AP averages also differ: seen ns/ts/nts average about 1.56/1.72/1.55 pp versus the reported 2.06/2.26/2.11 pp, and clean mAP averages about 1.34 pp versus the reported 2.15 pp. Since both presentations purport to be averages over the identical set of experiments, this is an internal inconsistency in the headline result. Please correct Table III and the abstract, or explicitly state the different aggregation rule used in each table.
  2. [Section V-A, Table I] The comparison does not isolate the effect of self-supervised pretraining. Pretrained models are trained on train-clean-100, train-clean-360, and train-other-500 (approximately 960 h of audio), while the supervised baseline is trained only on train-clean-100 (100 h). The paper attributes the improvement to the DN-APC objective, but the pretrained models also see roughly ten times more data, so the gain could come from data quantity rather than the self-supervised pretext. Please add a control experiment where the supervised baseline is trained on the same 960 h of labeled data, or where DN-APC pretraining is restricted to train-clean-100, and report whether the improvement persists.
  3. [Tables IV and V] No statistical significance tests or confidence intervals are reported for the key comparisons, and several improvements are small relative to the displayed variation. For example, the Concat model shows a seen-noise mAP gain of 1.04 pp while the DN-APC standard deviation is 0.95, and the FiLM+Pre model shows a clean mAP gain of 0.64 pp with base standard deviation 0.43. Given that the central claim is a general ~2% improvement, the authors should report paired significance tests (e.g., across SNR levels or across conditioning methods) or otherwise quantify the uncertainty of the average differences.
minor comments (5)
  1. [Section V-C] There is a typo in the gradient-clipping sentence: "nornalizing" should be "normalizing."
  2. [Section VI-C] The sentence "After embedding the representations in a two-dimensional space" is incomplete and should be finished or merged with the following sentence.
  3. [Section V-A] The claim of robustness to unseen noise is based on a single held-out noise type (café). Please qualify the conclusion as applying to this unseen condition, or evaluate on additional unseen noise types to support the broader generalization claim.
  4. [Equation (18)] In the DN-APC objective, the notation uses y_n for both the clean features used as prediction targets and, via y-hat_n, the noisy input features; this is understandable but could be clarified by defining the augmented-feature sequence explicitly.
  5. [Table II] The entry "149k (+263k)" for the FiLM+Pre model is unclear; please specify whether the parenthetical is the size of the preprocessing module or the total with the frozen embedding model.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: DN-APC is defined in-paper and the downstream TS-VAD evaluation is independent; the only self-citation to prior DN-APC work is background motivation, not a load-bearing reduction.

full rationale

The claimed derivation chain is: Eq. (18) defines the DenoisingAPC objective as predicting clean future logMel features from noisy/reverberant input under an l1 loss; the Conformer encoder trained on this pretext task is copied into five TS-VAD models; those models are fine-tuned with cross-entropy on LibriSpeech-derived multi-speaker data; and improvements are measured as mAP differences on held-out test-clean with seen and unseen noise. No equation in this chain is fitted to the target quantity: the DN-APC labels are clean features, not TS-VAD labels, and the mAP scores come from test data not used in Eq. (18). The use of DN-APC is motivated by the authors' prior ICASSP paper [27], and the text says DN-APC 'proved to outperform standard APC' there, but the current paper restates the objective and reports new experiments with a different architecture, different conditioning methods, and different test conditions. The prior result is therefore not the evidence for the headline claim, so the self-citation is not load-bearing. The main validity concerns are experimental rather than circular: pretraining uses train-clean-100 plus train-clean-360 plus train-other-500, while the supervised baseline uses only train-clean-100, making the SSL and data-quantity effects entangled; and Table III's aggregate deltas are not fully reproducible by averaging the per-model deltas in Tables IV and V. Both are correctness/consistency issues, not definitional reductions. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation. Score 2 reflects one minor self-citation that is not load-bearing; the central empirical result is not circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The main assumptions are experimental: label quality, speaker embedding quality, and the transferability of the pretraining task. No free parameter is fitted to the test set; the fitted quantities are model hyperparameters and the prediction horizon.

free parameters (5)
  • DN-APC prediction horizon k = 3
    Number of frames ahead the model is trained to predict (Eq. 18). Chosen from prior work [27]; not ablated in this paper.
  • Conformer hidden dimension = 64
    Model width chosen by hand; no sweep reported (Section V-B).
  • Conformer context span = 31 frames (310 ms)
    Causal context length fixed by kernel size and relative attention span; chosen to match prior work [9].
  • Number of Conformer layers = 2
    Depth chosen by hand; no ablation.
  • Training SNR range = -5 to 20 dB
    Range selected to match test conditions; augmentation probability 50% (Section V-A).
assumptions (5)
  • domain assumption LibriSpeech forced alignment provides accurate VAD labels for training and fine-tuning
    Labels are generated with Montreal Forced Aligner (Section V-A); errors would affect both baseline and pretrained models but are not quantified.
  • domain assumption d-vector embeddings from VoxCeleb and LibriSpeech capture target speaker identity well enough for conditioning
    Speaker embeddings are used as the only target information (Section II-B, V-B); no analysis of embedding errors.
  • domain assumption Additive noise plus room impulse response augmentation is representative of real noisy conditions
    Noise from [46] and RIR from SLR28 are used for training; cafe noise is held out as unseen (Section V-A).
  • domain assumption Predicting clean future logMel features from noisy past context is a useful pretext task for TS-VAD
    This is the central SSL hypothesis (Eq. 18); the paper tests it empirically but does not prove transfer.
  • standard math The signal model x[t] = st[t] + snt[t] + v[t] with additive sources holds
    Used in the problem formulation (Eq. 1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining." pith.science (2026). https://pith.science/paper/JOMNKNS2

@misc{pith2026250103184,
  author       = {Pith},
  title        = {Pith review of: Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JOMNKNS2}},
  note         = {Machine review of arXiv:2501.03184}
}
read the original abstract

Target-Speaker Voice Activity Detection (TS-VAD) is the task of detecting the presence of speech from a known target-speaker in an audio frame. Recently, deep neural network-based models have shown good performance in this task. However, training these models requires extensive labelled data, which is costly and time-consuming to obtain, particularly if generalization to unseen environments is crucial. To mitigate this, we propose a causal, Self-Supervised Learning (SSL) pretraining framework, called Denoising Autoregressive Predictive Coding (DN-APC), to enhance TS-VAD performance in noisy conditions. We also explore various speaker conditioning methods and evaluate their performance under different noisy conditions. Our experiments show that DN-APC improves performance in noisy conditions, with a general improvement of approx. 2% in both seen and unseen noise. Additionally, we find that FiLM conditioning provides the best overall performance. Representation analysis via tSNE plots reveals robust initial representations of speech and non-speech from pretraining. This underscores the effectiveness of SSL pretraining in improving the robustness and performance of TS-VAD models in noisy environments.

Figures

Figures reproduced from arXiv: 2501.03184 by the authors.

Figure 1
Figure 1. Illustration of the DN-APC framework also used in [27]. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Histograms showing the distributions of the concatenated LibriSpeech training and test set utterances. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Various TS-VAD model setups for speaker conditioning. Blue boxes designate the pretrained encoder, and green boxes represent elements related to [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Hidden representation tSNE plot of hidden representations of speech in [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 38 canonical work pages

  1. [1]

    rV AD: An unsupervised segment-based robust voice activity detection method,

    Z.-H. Tan, A. kr. Sarkar, and N. Dehak, “rV AD: An unsupervised segment-based robust voice activity detection method,” Computer Speech & Language , vol. 59, pp. 1–21, 2020

    work page 2020

  2. [2]

    V oice activity detection in the wild: A data-driven approach using teacher-student training,

    H. Dinkel, S. Wang, X. Xu, M. Wu, and K. Yu, “V oice activity detection in the wild: A data-driven approach using teacher-student training,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 1542–1555, 2021

    2021

  3. [3]

    Robust voice activity detection using an auditory-inspired masked modulation encoder based convolutional attention network,

    N. Li, L. Wang, M. Ge, M. Unoki, S. Li, and J. Dang, “Robust voice activity detection using an auditory-inspired masked modulation encoder based convolutional attention network,” Speech Communication , vol. 157, p. 103024, 2024

    work page 2024

  4. [4]

    V oice biometric system security: Design and analysis of countermeasures for replay attacks

    B. Chettri, “V oice biometric system security: Design and analysis of countermeasures for replay attacks.” Ph.D. dissertation, Queen Mary University of London, 2020

    work page 2020

  5. [5]

    Audiovisual speaker indexing for web-tv automations,

    N. Vryzas, L. Vrysis, and C. Dimoulas, “Audiovisual speaker indexing for web-tv automations,” Expert Systems with Applications , vol. 186, p. 115833, 2021

    work page 2021

  6. [6]

    Neural target speech extraction: An overview,

    K. Zmolikova, M. Delcroix, T. Ochiai, K. Kinoshita, J. ˇCernock´y, and D. Yu, “Neural target speech extraction: An overview,” IEEE Signal Processing Magazine, vol. 40, no. 3, pp. 8–29, 2023

    work page 2023

  7. [7]

    Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,

    K. ˇZmol´ıkov´a, M. Delcroix, K. Kinoshita, T. Ochiai, T. Nakatani, L. Bur- get, and J. ˇCernock`y, “Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,” IEEE Journal of Selected Topics in Signal Processing , vol. 13, no. 4, pp. 800–814, 2019

    work page 2019

  8. [8]

    Per- sonal V AD: Speaker-conditioned voice activity detection,

    S. Ding, Q. Wang, S.-Y . Chang, L. Wan, and I. Lopez Moreno, “Per- sonal V AD: Speaker-conditioned voice activity detection,” in Proc. The Speaker and Language Recognition Workshop (Odyssey 2020) , 2020, pp. 433–439. Baseline -5 dB DN APC frozen DN APC finetuned ns ts nts 0 dB 5 dB 10 dB 15 dB 20 dB Clean Fig. 4. Hidden representation tSNE plot of hidden ...

    work page 2020

Show all 53 references
  1. [9]

    Personal V AD 2.0: Optimizing personal voice activity detection for on-device speech recognition,

    S. Ding, R. Rikhye, Q. Liang, Y . He, Q. Wang, A. Narayanan, T. O’Malley, and I. McGraw, “Personal V AD 2.0: Optimizing personal voice activity detection for on-device speech recognition,” in Proc. Interspeech 2022, 2022, pp. 3744–3748

    2022

  2. [10]

    Target speech extraction with pre-trained self-supervised learning models,

    J. Peng, M. Delcroix, T. Ochiai, O. Plchot, S. Araki, and J. ˇCernock´y, “Target speech extraction with pre-trained self-supervised learning models,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 10 421– 11 10 425

    2024

  3. [11]

    Target-speaker voice activity detection: a novel approach for multi-speaker diarization in a dinner party scenario,

    I. Medennikov, M. Korenevsky, T. Prisyach, Y . Y . Khokhlov, M. Ko- renevskaya, I. Sorokin, T. Timofeeva, A. Mitrofanov, A. Andrusenko, I. Podluzhny, A. Laptev, and A. Romanenko, “Target-speaker voice activity detection: a novel approach for multi-speaker diarization in a dinn...

    2020

  4. [12]

    Target- speaker voice activity detection with improved i-vector estimation for unknown number of speaker,

    M. He, D. Raj, Z. Huang, J. Du, Z. Chen, and S. Watanabe, “Target- speaker voice activity detection with improved i-vector estimation for unknown number of speaker,” in Proc. Interspeech 2021 , 2021, pp. 2523–2527

    2021

  5. [13]

    Target speaker voice activity detection with transformers and its integration with end- to-end neural diarization,

    D. Wang, X. Xiao, N. Kanda, T. Yoshioka, and J. Wu, “Target speaker voice activity detection with transformers and its integration with end- to-end neural diarization,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 202...

    2023

  6. [14]

    Profile-error-tolerant target-speaker voice activity detection,

    D. Wang, X. Xiao, N. Kanda, M. Yousefi, T. Yoshioka, and J. Wu, “Profile-error-tolerant target-speaker voice activity detection,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2024, pp. 11 906–11 910

    2024

  7. [15]

    Multimodal attention fusion for target speaker extraction,

    H. Sato, T. Ochiai, K. Kinoshita, M. Delcroix, T. Nakatani, and S. Araki, “Multimodal attention fusion for target speaker extraction,” in 2021 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2021, pp. 778–784

    2021

  8. [16]

    Multimodal speakerbeam: Single channel target speech extraction with audio-visual speaker clues

    T. Ochiai, M. Delcroix, K. Kinoshita, A. Ogawa, and T. Nakatani, “Multimodal speakerbeam: Single channel target speech extraction with audio-visual speaker clues.” in INTERSPEECH, 2019, pp. 2718–2722

    2019

  9. [17]

    Av-sepformer: Cross-attention sepformer for audio-visual target speaker extraction,

    J. Lin, X. Cai, H. Dinkel, J. Chen, Z. Yan, Y . Wang, J. Zhang, Z. Wu, Y . Wang, and H. Meng, “Av-sepformer: Cross-attention sepformer for audio-visual target speaker extraction,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICA...

    2023

  10. [18]

    Usev: Universal speaker extraction with visual cue,

    Z. Pan, M. Ge, and H. Li, “Usev: Universal speaker extraction with visual cue,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 3032–3045, 2022

    2022

  11. [19]

    Brain-informed speech separation (biss) for enhancement of target speaker in multitalker speech perception,

    E. Ceolini, J. Hjortkjær, D. D. Wong, J. O’Sullivan, V . S. Raghavan, J. Herrero, A. D. Mehta, S.-C. Liu, and N. Mesgarani, “Brain-informed speech separation (biss) for enhancement of target speaker in multitalker speech perception,” NeuroImage, vol. 223, p. 117282, 2020

    2020

  12. [20]

    Target speaker detection with concealed eeg around the ear,

    B. Mirkovic, M. G. Bleichner, M. De V os, and S. Debener, “Target speaker detection with concealed eeg around the ear,” Frontiers in neuroscience, vol. 10, p. 206084, 2016

    2016

  13. [21]

    Eeg decoding of the target speaker in a cocktail party scenario: considerations regarding dynamic switching of talker location,

    E. S. Teoh and E. C. Lalor, “Eeg decoding of the target speaker in a cocktail party scenario: considerations regarding dynamic switching of talker location,” Journal of neural engineering, vol. 16, no. 3, p. 036017, 2019

    2019

  14. [22]

    Cortical auditory at- tention decoding during music and speech listening,

    A. Simon, G. Loquet, J. Østergaard, and S. Bech, “Cortical auditory at- tention decoding during music and speech listening,” IEEE Transactions on Neural Systems and Rehabilitation Engineering , 2023

    2023

  15. [23]

    Self-supervised speech representation learning: A review,

    A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløe, T. N. Sainath, and S. Watanabe, “Self-supervised speech representation learning: A review,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6...

    2022

  16. [24]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  17. [25]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. rahman Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 3451–3460, 2021

    2021

  18. [26]

    WavLM: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu et al. , “WavLM: Large-scale self- supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, pp. 1505–1518, 2021

    2021

  19. [27]

    Self-supervised pretraining for robust personalized voice activity detection in adverse conditions,

    H. Bovbjerg, J. Jensen, J. Østergaard, and Z.-H. Tan, “Self-supervised pretraining for robust personalized voice activity detection in adverse conditions,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE Signal Pro...

    2024

  20. [28]

    Features for voice activity detection: a comparative analysis,

    S. Graf, T. Herbig, M. Buck, and G. Schmidt, “Features for voice activity detection: a comparative analysis,” EURASIP Journal on Advances in Signal Processing, vol. 2015, pp. 1–15, 2015

    2015

  21. [29]

    I-vector-based speaker adaptation of deep neural networks for french broadcast audio transcription,

    V . Gupta, P. Kenny, P. Ouellet, and T. Stafylakis, “I-vector-based speaker adaptation of deep neural networks for french broadcast audio transcription,” in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2014, pp. 6334–6338

    2014

  22. [30]

    Deep neural networks for small footprint text-dependent speaker verification,

    E. Variani, X. Lei, E. McDermott, I. L. Moreno, and J. Gonzalez- Dominguez, “Deep neural networks for small footprint text-dependent speaker verification,” in 2014 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP) , 2014, pp. 4052–4056

    2014

  23. [31]

    Generalized end-to-end loss for speaker verification,

    L. Wan, Q. Wang, A. Papir, and I. L. Moreno, “Generalized end-to-end loss for speaker verification,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2018, pp. 4879– 4883

    2018

  24. [32]

    X- vectors: Robust dnn embeddings for speaker recognition,

    D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X- vectors: Robust dnn embeddings for speaker recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 5329–5333

    2018

  25. [33]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , vol. 30. Curran Associates, Inc., 2017

    2017

  26. [34]

    Film: Visual reasoning with a general conditioning layer,

    E. Perez, F. Strub, H. De Vries, V . Dumoulin, and A. Courville, “Film: Visual reasoning with a general conditioning layer,” in Proceedings of the AAAI conference on artificial intelligence , vol. 32, no. 1, 2018

    2018

  27. [35]

    Conditional conformer: Improving speaker modulation for single and multi-user speech enhancement,

    T. O’Malley, S. Ding, A. Narayanan, Q. Wang, R. Rikhye, Q. Liang, Y . He, and I. McGraw, “Conditional conformer: Improving speaker modulation for single and multi-user speech enhancement,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Proc...

    2023

  28. [36]

    Searching for activation functions,

    P. Ramachandran, B. Zoph, and Q. V . Le, “Searching for activation functions,” ArXiv, vol. abs/1710.05941, 2018

    2018 arXiv

  29. [37]

    Using self- supervised learning can improve model robustness and uncertainty,

    D. Hendrycks, M. Mazeika, S. Kadavath, and D. Song, “Using self- supervised learning can improve model robustness and uncertainty,” in Advances in Neural Information Processing Systems , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch ´e-Buc, E. Fox, and R. Garnett, Eds.,...

    2019

  30. [38]

    Noise-robust keyword spotting through self-supervised pretraining,

    J. Mørk, H. S. Bovbjerg, G. Kiss, and Z.-H. Tan, “Noise-robust keyword spotting through self-supervised pretraining,” arXiv preprint arXiv:2403.18560, 2024

    2024 arXiv

  31. [39]

    Bootstrap predictive coding: Investigating a non-contrastive self-supervised learning approach,

    Y . Mohamied and P. Bell, “Bootstrap predictive coding: Investigating a non-contrastive self-supervised learning approach,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 11 541–11 545

    2024

  32. [40]

    Representation learning with contrastive predictive coding,

    A. van den Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” ArXiv, vol. abs/1807.03748, 2018

    2018 arXiv

  33. [41]

    Autore- gressive predictive coding: A comprehensive study,

    G.-P. Yang, S.-L. Yeh, Y .-A. Chung, J. Glass, and H. Tang, “Autore- gressive predictive coding: A comprehensive study,” IEEE Journal of Selected Topics in Signal Processing , pp. 1–12, 2022

    2022

  34. [42]

    Generative pre-training for speech with au- toregressive predictive coding,

    Y .-A. Chung and J. Glass, “Generative pre-training for speech with au- toregressive predictive coding,” in 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2020, pp. 3497– 3501

    2020

  35. [43]

    Pytorch: An imperative style, high- performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high- p...

    2019

  36. [44]

    Librispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  37. [45]

    Montreal Forced Aligner: Trainable text-speech alignment using kaldi,

    M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal Forced Aligner: Trainable text-speech alignment using kaldi,” in Proc. Interspeech 2017 , 2017, pp. 498–502

    2017

  38. [46]

    Speech enhancement using long short-term memory based recurrent neural networks for noise robust speaker verification,

    M. Kolbœk, Z.-H. Tan, and J. Jensen, “Speech enhancement using long short-term memory based recurrent neural networks for noise robust speaker verification,” in 2016 IEEE Spoken Language Technology Workshop (SLT), 2016, pp. 305–311

    2016

  39. [47]

    A study on data augmentation of reverberant speech for robust speech recognition,

    T. Ko, V . Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” in 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2017, pp. 5220–5224

    2017

  40. [48]

    audiomentations,

    I. Jordal, “audiomentations,” https://iver56.github.io/audiomentations/, 2024

    2024

  41. [49]

    Transformer-xl: Attentive language models beyond a fixed- length context,

    Z. Dai, Z. Yang, Y . Yang, J. G. Carbonell, Q. V . Le, and R. Salakhut- dinov, “Transformer-xl: Attentive language models beyond a fixed- length context,” in Annual Meeting of the Association for Computational Linguistics, 2019. 12

    2019

  42. [50]

    V oxceleb: Large- scale speaker verification in the wild,

    A. Nagrani, J. S. Chung, W. Xie, and A. Zisserman, “V oxceleb: Large- scale speaker verification in the wild,” Computer Speech & Language , vol. 60, p. 101027, 2020

    2020

  43. [51]

    SGDR: Stochastic gradient descent with warm restarts,

    I. Loshchilov and F. Hutter, “SGDR: Stochastic gradient descent with warm restarts,” in International Conference on Learning Representa- tions, 2017

    2017

  44. [52]

    On batching variable size inputs for training end-to-end speech enhancement systems,

    P. Gonzalez, T. Sonne Alstrøm, and T. May, “On batching variable size inputs for training end-to-end speech enhancement systems,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2023, pp. 1–5

    2023

  45. [53]

    Visualizing data using t-sne,

    L. van der Maaten and G. Hinton, “Visualizing data using t-sne,” Journal of Machine Learning Research , vol. 9, no. 86, pp. 2579–2605, 2008

    2008

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.