Pith. sign in

REVIEW 3 major objections 7 minor 73 references

Generic Speech Enhancement with Self-Supervised Representation Space Loss

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training a speech-enhancement model to match clean speech inside a frozen self-supervised representation space yields a single front-end that improves several downstream tasks at once.

desk verdict A useful empirical extension of the authors' earlier SSL-MSE idea, but the generic black-box front-end claim is not supported by the Whisper results without the separate OA post-process. read the letter →

arxiv 2507.07631 v1 pith:LDQ5VDPA submitted 2025-07-10 eess.AS cs.SDeess.SP

Hiroshi Sato , Tsubasa Ochiai , Marc Delcroix , Takafumi Moriya , Takanori Ashihara , Ryo Masumura This is my paper · ORCID
classification eess.AScs.SDeess.SP
keywords speechenhancementself-supervisedlearningSSL-MSElossfront-enddownstreamtasksWhisperASRSUPERBbenchmarkWavLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to build a single speech enhancement (SE) front-end that improves many downstream tasks without being retuned for each one. The training criterion, called SSL-MSE, minimizes the mean squared distance between the enhanced and clean waveforms inside the representation space of a frozen self-supervised speech model. Because SSL representations are known to carry phonetic, semantic, and speaker information, the hope is that an SE trained this way preserves what downstream back-ends need rather than just maximizing signal-to-noise ratio. The experiments show one Conv-TasNet front-end improving SSL-based ASR, speaker verification, and intent classification, and also an off-the-shelf Whisper model, while keeping perceptual quality roughly at the level of SNR-trained enhancement.

What carries the argument

The mechanism is the SSL-MSE loss, defined as the mean squared error between enhanced and clean feature sequences extracted by a frozen SSL model, computed on a weighted sum of layer outputs with zero weight on the first floor(N/2) layers and uniform weight on the remaining layers (Eqs. 14-20), combined with the SNR loss in a multitask objective L = L_SSL + alpha L_SNR. The layer weighting matters: it pushes the SE to preserve higher-level linguistic and semantic content while the SNR term keeps acoustic fidelity. The experiments use Conv-TasNet as the SE network, WavLM Large as the teacher SSL model, and optional observation-adding (OA) post-processing that interpolates enhanced and observed waveforms.

What would settle it

Run the SSL-MSE-trained front-end on a back-end that does not use SSL features and is sensitive to processing artifacts, such as a phase-only or raw-waveform ASR system. If it fails to beat the SNR-trained baseline on any such task, or if it degrades a non-SSL back-end's performance as Whisper does before OA is added, then the claim of a truly generic front-end is falsified for that combination.

Watch

Extended reading notes

Core claim

The paper's central claim is that training an SE model to bring the SSL features of the enhanced signal close to those of the clean signal transfers the generalizability of the SSL model (WavLM Large in the best configuration) into the SE model itself. Concretely, the SSL-MSE loss is added to the conventional SNR loss as a multitask objective, with the SSL teacher frozen, and the best result in Table 1 improves SSL downstream ASR WER from 14.6% to 11.3%, ASV EER from 8.6% to 7.4%, intent classification accuracy from 84.8% to 88.6%, and Whisper WER from 12.2% to 9.6% compared with SNR-only training. Adding observation-adding post-processing pushes Whisper further to 9.1%, which beats the no-enhancement baseline of 9.3%; the paper reports that, without that post-processing, Whisper still reads 9.6% and thus remains worse than doing no enhancement. The same model preserves or slightly changes PESQ and DNSMOS, and the claim is that the method therefore yields a generic front-end that works for both machine back-ends and human listeners, including black-box systems that cannot be retrained.

Load-bearing premise

The central claim rests on the premise that a frozen, noise-robust self-supervised model's feature space (WavLM Large in the best results) captures the speech information every downstream back-end needs, including non-SSL systems like Whisper and human listeners; if that premise fails, the SE front-end would only help SSL-based tasks, and the paper's own results with wav2vec 2.0 and HuBERT teachers already show that the proxy is teacher-dependent.

Editorial extensions

If this is right

  • A single SE model trained with SSL-MSE can be dropped in front of multiple SSL-based back-ends without per-task fine-tuning, improving WER, EER, and accuracy simultaneously.
  • The benefit extends to at least one non-SSL, black-box back-end: Whisper WER improves relative to SNR-trained SE, and with OA post-processing it beats the no-enhancement baseline.
  • The teacher choice matters: only noise-robust SSL models such as WavLM Base+ and WavLM Large transfer gains, while wav2vec 2.0 Base and HuBERT Base teachers do not improve over SNR training.
  • SSL-MSE also helps when both the upstream SSL model and the downstream task model are themselves trained to be noise-robust.
  • The SSL-MSE loss improves downstream task performance while keeping perceptual quality (PESQ and DNSMOS) on par with SNR-trained SE, so the front-end remains usable for human listening.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the SSL representation space is a good universal target, the same training recipe could be transferred to other SE architectures such as real-time or larger models, though the paper tests only Conv-TasNet.
  • The approach implicitly assumes WavLM Large's features are a stable proxy for whatever information future back-ends need; a back-end that relies on spectral fine-structure or phase may not be covered by this proxy.
  • The success of combining SSL-MSE with OA suggests a possible extension: treat the SSL-MSE objective as a regularizer controlling the artifact-versus-noise trade-off, and tune the OA ratio at deployment time per task without retraining the SE.
  • Because SSL-MSE keeps SDR roughly unchanged but changes the nature of residual errors, actual human listening tests, not just DNSMOS, would show whether the human-listener back-end truly benefits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes a training criterion for single-channel speech enhancement (SE): SSL-MSE, an MSE loss between enhanced and clean speech in the feature space of a frozen self-supervised speech model, combined with an SNR loss in a multitask objective (Eqs. 14-21), with layer weights emphasizing the latter half of teacher layers. Using Conv-TasNet trained on simulated LibriSpeech plus DNS noise mixtures, the authors evaluate the resulting front-end on (i) perceptual metrics PESQ and DNSMOS, (ii) SUPERB-style SSL downstream tasks ASR/ASV/intent classification with four SSL back-ends (WavLM Base+/Large, wav2vec 2.0 Base, HuBERT Base), and (iii) an off-the-shelf Whisper Medium ASR, comparing against SNR-only training, LMFB-loss training, and ASR-loss training, with and without observation-adding (OA) post-processing. Headline results: relative to the SNR baseline, SSL-MSE with a WavLM Large teacher improves SSL-back-end ASR WER from 14.6% to 11.3%, ASV EER from 8.6% to 7.4%, and IC accuracy from 84.8% to 88.6%, and Whisper WER from 12.2% to 9.6%; however, the Whisper result is worse than no enhancement (9.3%), improving to 9.1% only when OA with beta=0.1 is added. Additional experiments show cross-SSL-model transfer for WavLM teachers and gains when the downstream model is trained to be noise-robust.

Significance. The core idea---aligning SE output to a generic SSL representation space to decouple the front-end from specific back-ends---is appealing, and the paper assembles unusually broad evidence for it: four possible teachers, four SSL inference back-ends, three SSL downstream tasks, a non-SSL ASR, perceptual metrics, and comparisons against LMFB- and ASR-based multitask losses. The cross-model results of Table 2, the noise-robust downstream results of Table 3, and the internal consistency of the improvements (larger gains with larger, noise-robust teachers) indicate that the SSL-downstream improvements are real and worth publishing. The significance as stated, however, is overstated: the 'generic black-box front-end' claim rests on a single non-SSL back-end (Whisper), on which the trained model alone fails, and the mechanism by construction favors SSL feature-space back-ends. Scoping the claims and reporting uncertainty would make the paper's contribution both accurate and reproducible; no code or data release is mentioned.

major comments (3)
  1. [§5.2.1, Table 1 (a2/b1/c2/c3), Eq. (22)] The central claim that SSL-MSE produces a generic front-end that improves black-box back-ends is not supported by the only non-SSL back-end tested. Table 1 shows Whisper WER of 9.6% for the SSL-MSE model (c2) versus 9.3% for the unprocessed signal (a2), and Section 5.2.1 explicitly concedes that (c2) 'still performs worse than the system without SE.' The 9.1% figure that beats no enhancement is obtained only in (c3), which adds OA post-processing with beta=0.1 from Eq. (22); Section 4 states that the OA ratio may be modified or omitted depending on the downstream task, so the improvement is not a property of the trained front-end alone and introduces a per-task tuning decision. In addition, the SSL-based gains in Table 1 are partly by construction, since the training loss minimizes the distance to the clean signal in the same WavLM feature space that the SSL back-ends consume. Please restrict the abstract and contribution-1 claims to SSL-based back-ends, or report the black-box recipe as the SSL-MSE+OA pipeline with the beta selection procedure specified (e.g., chosen on a held-out development set).
  2. [§5.2.3, Table 2, and Conclusion] The claimed transfer of 'the generalizability of SSL models over various downstream tasks' (Contribution 1) is empirically restricted to noise-robust WavLM teachers. With wav2vec 2.0 Base and HuBERT Base as SSL-MSE teachers, Table 2 shows no consistent improvement over the SNR-loss baseline on the WavLM Base+ downstream (rows d1/d2: 14.4% and 14.7% vs 14.6% WER) and no gain in matched teacher/inference conditions (e5: 18.5% vs 18.8%; f5: 18.8% vs 18.5%), consistent with the Conclusion's own statement that 'SSL-MSE loss training requires the use of a noise-robust SSL model.' Because the paper's mechanism claim is that SSL representations generally carry transferable information, the finding that only a noise-robust teacher family works changes the scope of the claim; the abstract and introduction should either be reworded accordingly or the paper should explain why the mechanism fails for non-noise-robust teachers.
  3. [Table 1 caption and §5.2.1] The significance evidence is currently unverifiable. The Table 1 caption states that bold values indicate 't-tests conducted at a 5% significance level,' but the paper reports no test statistic, p-values, number of evaluation utterances, or run-to-run variance, and key comparisons are small in absolute terms (Whisper 9.6 vs 9.3 in (c2)/(a2); 9.1 vs 9.3 in (c3)/(a2); SSL ASR 14.4 vs 14.6 in row d1). Report means with variance (e.g., over test-utterance bootstrap or multiple training seeds) for the headline comparisons, and either supply the t-test details or remove the significance claim.
minor comments (7)
  1. [Eqs. (19)-(20)] The floor expression is written as ⌊2/N⌋, which appears to be a typo for ⌊N/2⌋; as printed, the layer-weight formula is undefined or wrong for the network sizes used (e.g., N=12 for WavLM Base+), and it contradicts the stated intention of uniformly emphasizing the latter half of the layers.
  2. [Table 1] The DNSMOS columns (SIG, BAK, OVRL) are not clearly labeled in the table body, making it hard to verify the claims about which perceptual sub-scores improve or degrade; please reorder or label the columns explicitly.
  3. [§5.2.1] The statement that SSL-MSE 'improved the WER of the Whisper ASR model by 21%' reports only a relative change; please state absolute and relative values together, and do the same for the other percentage improvements.
  4. [§1, §4] Since the paper positions itself against Hsieh et al.'s PFPL (final-layer wav2vec 2.0 loss), a direct experimental comparison with that training criterion would be valuable; at minimum, a discussion of why multi-layer WavLM-based loss differs mechanistically from PFPL would let readers assess the claimed advantage.
  5. [§5.1.2, Figure 3, Table 1] The choice of alpha=0.1 for Table 1 is not justified even though Figure 3 shows the best downstream-task performance around alpha=0.01; please state the selection criterion (e.g., balancing PESQ/DNSMOS against downstream metrics) so that the headline configuration is reproducible.
  6. [Eq. (14)] It is unclear whether the SSL features are normalized (e.g., per-frame or per-dimension) before computing the MSE; if not, the uniform layer weighting in Eq. (20) may be dominated by high-variance layers, and the paper should state whether any normalization was applied.
  7. [Figure 3 caption] The figure caption contains stray citation-bracket artifacts ('[28] [45] [66]'), and the curves are not accompanied by numeric values at the operating point alpha=0.1 used in Table 1; please clean up the caption and report the key values in the text for reproducibility.

Circularity Check

1 steps flagged · score 5.0 of 10

SSL downstream gains are partly the SSL-MSE objective itself; the non-SSL Whisper result is negative without OA, so the generic black-box claim is not independently established.

  1. self definitional [Section 4, Eqs. (14)-(21); Section 3.3.1, Eq. (7); Table 1]
    "Although it is possible to calculate the loss on each layer and sum the losses instead of calculating the loss on the weighted sum of the representations, we adopted the latter way to maintain consistency with the feature extraction process of the SSL pipeline as in Eq. (7)."

    For the SSL-based back-ends, the input features are WS(SSL(SE(y); theta_ssl); theta_tau_w) (Eq. 7), while SSL-MSE minimizes ||F_enh - F_clean||^2_F with F_enh = sum_n tilde_w_n SSL_n(SE(y)) and F_clean = sum_n tilde_w_n SSL_n(x) (Eqs. 14-18). Thus training directly pulls the exact feature vector consumed by the evaluated SSL back-ends toward the clean feature vector. Since the SUPERB downstream models were trained on relatively clean SSL features, the WER/EER/Acc gains in Table 1 are, to a large extent, the immediate objective of the loss rather than a test of transfer to arbitrary back-ends. Independent but weaker support comes from cross-SSL results (Table 2); the non-SSL Whisper result is negative without OA post-processing, as the paper admits in Section 5.2.1.

full rationale

The defensible circularity is partial: SSL-MSE is defined on the same SSL feature space that the SSL-based back-ends consume, and the paper explicitly chose the weighted-sum form to match Eq. (7). This makes the SSL downstream-task improvements partly a direct consequence of the training objective. The circularity is bounded, however: the loss is not fit to test labels, the downstream models are independently trained, and the paper includes cross-SSL-model and Whisper evaluations, so the central claim still has independent content. The Whisper result (9.6% WER vs 9.3% without SE for SSL-MSE alone; 9.1% only with OA post-processing) is an honest, explicitly reported empirical limitation of the generic black-box claim, not an additional circular step. Self-citations (Sato et al. 2023, Ochiai et al. 2024) are used for context and for OA, not as load-bearing uniqueness arguments. Score 5 reflects the partial reduction by construction while acknowledging the independent evidence.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several domain assumptions: SSL representations are a universal proxy for downstream back-ends, MSE in SSL space avoids artifacts, and simulated DNS-noise mixtures represent evaluation. Free parameters alpha, beta, and layer weights are selected based on evaluation performance. No new physical entities are introduced.

free parameters (3)
  • SNR loss weight alpha in SSL-MSE multitask loss = alpha=0.1 for Table 1; best downstream performance at alpha=0.01 in Figure 3
    Searched over {0, 0.0001, 0.001, 0.01, 0.1, 1, 10} and chosen based on downstream evaluation results; the main table uses alpha=0.1, which is not the value that maximizes downstream ASR and Whisper gains.
  • Observation adding ratio beta = 0.1 and 0.5
    Selected from two candidates; final best results use beta=0.1, and the value affects Whisper and SSL task gains.
  • SSL layer weights for latter-half emphasis = Uniform over latter half (0 for earlier layers)
    Hand-chosen hyperparameter; the printed formula (19)-(20) appears to have a typo (floor(2/N) rather than floor(N/2)), so the actual weights used are ambiguous.
assumptions (4)
  • domain assumption A frozen SSL model's representation space is a sufficient proxy for all downstream back-end input requirements, including non-SSL systems and human listeners.
    Central premise of SSL-MSE, used in Eq. (14)-(18) and Sections 1 and 4. Table 2 shows it fails for non-noise-robust teachers, so it is load-bearing.
  • domain assumption MSE in SSL feature space between enhanced and clean signals preserves downstream task-relevant information without harmful artifacts.
    The loss optimizes feature-space distance; the paper relies on empirical PESQ and DNSMOS results rather than a proof that feature MSE avoids distortion.
  • domain assumption Simulated noisy mixtures with LibriSpeech speech and DNS noise represent evaluation conditions.
    Training and evaluation both draw from DNS noise (different samples); generalization to other noise types is untested and explicitly left to future work.
  • ad hoc to paper The layer-weight formula in Eq. (19) implements uniform emphasis of the latter half of SSL layers.
    As printed, floor(2/N) is 0 or 1 for typical N (e.g., 12 or 24), not N/2; the text describes latter-half emphasis, so either the equation or the description is wrong.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generic Speech Enhancement with Self-Supervised Representation Space Loss." pith.science (2026). https://pith.science/paper/LDQ5VDPA

@misc{pith2026250707631,
  author       = {Pith},
  title        = {Pith review of: Generic Speech Enhancement with Self-Supervised Representation Space Loss},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LDQ5VDPA}},
  note         = {Machine review of arXiv:2507.07631}
}
read the original abstract

Single-channel speech enhancement is utilized in various tasks to mitigate the effect of interfering signals. Conventionally, to ensure the speech enhancement performs optimally, the speech enhancement has needed to be tuned for each task. Thus, generalizing speech enhancement models to unknown downstream tasks has been challenging. This study aims to construct a generic speech enhancement front-end that can improve the performance of back-ends to solve multiple downstream tasks. To this end, we propose a novel training criterion that minimizes the distance between the enhanced and the ground truth clean signal in the feature representation domain of self-supervised learning models. Since self-supervised learning feature representations effectively express high-level speech information useful for solving various downstream tasks, the proposal is expected to make speech enhancement models preserve such information. Experimental validation demonstrates that the proposal improves the performance of multiple speech tasks while maintaining the perceptual quality of the enhanced signal.

Figures

Figures reproduced from arXiv: 2507.07631 by the authors.

Figure 1
Figure 1. Overview of the combination of SE front-end with back-end tasks. The backend task can either be 1) a speech processing task with SSL upstream and downstream models, 2) a speech processing task without SSL models, or 3) a human listening task. 3.3 Combination of SE front-end with back-end tasks The introduction of an SE front-end can improve the performance of subsequent tasks. The subsequent task can either be 1) a … view at source ↗
Figure 2
Figure 2. Overview of the proposed SSL-MSE loss. Proposed SSL-MSE loss calculate the distance between enhanced and clean signals on the feature domain extracted by the SSL model. of downstream tasks, even when those tasks are black-box systems that cannot be retrained. It is also beneficial for tasks with a very large number of parameters, where retraining would be impractical. To further reduce the mismatch between SE front-… view at source ↗
Figure 3
Figure 3. Comparison of SSL-MSE loss with ASR loss and LMFB loss joint training. The X-axis represents the multitask loss weight α in the joint training with each loss, and the Y-axis represents the performance in each evaluation metric. For SDR [dB] DNSMOS OVRL, higher is better, and for WER [%], lower is better. ASR performance. For each graph, the X-axis represents the multitask loss weight α in the multitask training, whi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 70 canonical work pages

  1. [1]

    WebRTC V oice Activity Detector

    [Dataset] (Cited June 1 2025). WebRTC V oice Activity Detector. https://github.com/ wiseman/py-webrtcvad [Dataset] (Cited October 7 2024). s3prl toolkit. https://github.com/s3prl/s3prl [Dataset] (Cited October 7 2024). Speech to text - OpenAI API. https://platform.openai. com/docs/guides/speech-to-text [Dataset] (Cited October 7 2024). Speech-to-Text AI. ...

    work page 2025

  2. [2]

    N., Brutti, A., and Falavigna, D

    Ali, M. N., Brutti, A., and Falavigna, D. (2023). Direct enhancement of pre-trained speech embeddings for speech processing in noisy conditions. Computer Speech & Language 81, 101501

    work page 2023

  3. [3]

    N., Brutti, A., Falavigna, D., et al

    Ali, M. N., Brutti, A., Falavigna, D., et al. (2022). Enhancing embeddings for speech classification in noisy conditions. In Proc. Annual Conference of the International Speech Communication Association. 2933–2937

    work page 2022

  4. [4]

    R., Alam, M

    Avila, A. R., Alam, M. J., O’Shaughnessy, D., and Falk, T. (2018). Investigating Speech Enhancement and Perceptual Quality for Speech Emotion Recognition. In Proc. Annual Conference of the International Speech Communication Association. 3663–3667

    work page 2018

  5. [5]

    Bredin, H. (2023). pyannote. audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe. In Proc. Annual Conference of the International Speech Communication Association(ISCA), 1983–1987

    work page 2023

  6. [6]

    Chen, S., Wang, C., Chen, Z., Wu, Y ., Liu, S., Chen, Z., et al. (2022). Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing16, 1505–1518

    work page 2022

  7. [7]

    S., Xu, H., and Watanabe, S

    Chen, S.-J., Subramanian, A. S., Xu, H., and Watanabe, S. (2018). Building state-of-the-art distant speech recognition using the chime-4 challenge with a setup of speech enhancement baseline. In Proc. Annual Conference of the International Speech Communication Association. 1571–1575

    work page 2018

  8. [8]

    Chen, W., Shi, J., Yan, B., Berrebbi, D., Zhang, W., Peng, Y ., et al. (2023). Joint prediction and denoising for large-scale multilingual self-supervised learning. In IEEE Automatic Speech Recognition and Understanding Workshop (IEEE), 1–8

    work page 2023

Show all 73 references
  1. [9]

    Delcroix, M., Ochiai, T., Zmolikova, K., Kinoshita, K., Tawara, N., Nakatani, T., et al. (2020). Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. 691–695

    2020

  2. [10]

    Delcroix, M., ˇZmol´ıkov´a, K., Kinoshita, K., Ogawa, A., and Nakatani, T. (2018). Single channel target speaker extraction and recognition with speaker beam. In Proc. IEEE International Conference on

    2018

  3. [11]

    Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn...

    2019

  4. [12]

    Dubey, H., Aazami, A., Gopal, V ., Naderi, B., Braun, S., Cutler, R., et al. (2024). Icassp 2023 deep noise suppression challenge. IEEE Open Journal of Signal Processing

    2024

  5. [13]

    Dubey, H., Gopal, V ., Cutler, R., Aazami, A., Matusevych, S., Braun, S., et al. (2022). ICASSP 2022 deep noise suppression challenge. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. 9271–9275

    2022

  6. [14]

    A., de Seyssel, M., Roz ´e, P., et al

    Dunbar, E., Bernard, M., Hamilakis, N., Nguyen, T. A., de Seyssel, M., Roz ´e, P., et al. (2021). The Zero Resource Speech Challenge 2021: Spoken Language Modelling. In Proc. Annual Conference of the International Speech Communication Association. 1574–1578 D´efossez, A., Synn...

    2021

  7. [15]

    E., Soufleris, P., Duan, Z., and Heinzelman, W

    Eskimez, S. E., Soufleris, P., Duan, Z., and Heinzelman, W. (2018). Front-end speech enhancement for commercial speaker verification systems. Speech Communication 99, 101–113

    2018

  8. [16]

    and Kawai, H

    Fujimoto, M. and Kawai, H. (2019). One-pass single-channel noisy speech recognition using a combination of noisy and enhanced features. In Proc. Annual Conference of the International Speech Communication Association. 486–490

    2019

  9. [17]

    Fujita, K., Sato, H., Ashihara, T., Kanagawa, H., Delcroix, M., Moriya, T., et al. (2024). Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with Frontiers 18 Hiroshi Sato et al. Generic Speech Enhancement with SSL Loss ...

    2024

  10. [18]

    Gao, T., Du, J., Dai, L.-R., and Lee, C.-H. (2015). Joint training of front-end and back-end deep neural networks for robust speech recognition. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 4375–4379

    2015

  11. [19]

    Graves, A., Fernandez, S., Gomez, F., and Schmidhuber, J. (2006). Connectionist temporal classification : Labelling unsegmented sequence data with recurrent neural networks. In International conference on machine learning (PMLR), 369–376

    2006

  12. [20]

    Hao, X., Su, X., Horaud, R., and Li, X. (2021). Fullsubnet: A full-band and sub-band fusion model for real-time single-channel speech enhancement. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 6633–6637

    2021

  13. [21]

    Hsieh, T.-A., Yu, C., Fu, S.-W., Lu, X., and Tsao, Y . (2021). Improving Perceptual Quality by Phone- Fortified Perceptual Loss Using Wasserstein Distance for Speech Enhancement. In Proc. Annual Conference of the International Speech Communication Association. 196–200

    2021

  14. [22]

    H., Lakhotia, K., Salakhutdinov, R., and Mohamed, A

    Hsu, W.-N., Bolte, B., Tsai, Y .-H. H., Lakhotia, K., Salakhutdinov, R., and Mohamed, A. (2021). Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing29, 3451–3460

    2021

  15. [23]

    Hu, Y ., Liu, Y ., Lv, S., Xing, M., Zhang, S., Fu, Y ., et al. (2020). DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement. In Proc. Annual Conference of the International Speech Communication Association. 2472–2476

    2020

  16. [24]

    S., Nagrani, A., Garcia-Romero, D., et al

    Huh, J., Brown, A., Jung, J.-w., Chung, J. S., Nagrani, A., Garcia-Romero, D., et al. (2023). V oxsrc 2022: The fourth voxceleb speaker recognition challenge. arXiv preprint arXiv:2302.10248

    2023 arXiv

  17. [25]

    Iwamoto, K., Ochiai, T., Delcroix, M., Ikeshita, R., Sato, H., Araki, S., et al. (2022). How bad are artifacts?: Analyzing the impact of speech enhancement errors on ASR. In Proc. Annual Conference of the International Speech Communication Association. 5418–5422

    2022

  18. [26]

    R., Zadeh, M

    Jaiswal, A., Babu, A. R., Zadeh, M. Z., Banerjee, D., and Makedon, F. (2020). A survey on contrastive self-supervised learning. Technologies 9, 2

    2020

  19. [27]

    Ju, Y ., Zhang, S., Rao, W., Wang, Y ., Yu, T., Xie, L., et al. (2023). Tea-pse 2.0: Sub-band network for real-time personalized speech enhancement. In Proc. SLT 2022 (IEEE), 472–479

    2023

  20. [28]

    Kandagatla, R. K. and Potluri, V . S. (2020). Performance analysis of neural network, nmf and statistical approaches for speech enhancement. International Journal of Speech Technology23, 917–937

    2020

  21. [29]

    Karita, S., Soplin, N. E. Y ., Watanabe, S., Delcroix, M., Ogawa, A., and Nakatani, T. (2019). Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration. In Proc. Annual Conference of the International S...

    2019

  22. [30]

    Kingma, D. P. and Ba, J. (2015). Adam: A method for stochastic optimization. In Proc. International Conference on Learning Representations

    2015

  23. [31]

    Kinoshita, K., Ochiai, T., Delcroix, M., and Nakatani, T. (2020). Improving noise robust automatic speech recognition with single-channel time-domain enhancement network. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 7009–7013 Le Rou...

    2020

  24. [32]

    T., Yang, S.-w., Chi, P.-H., Hsu, P.-c., and Lee, H.-y

    Liu, A. T., Yang, S.-w., Chi, P.-H., Hsu, P.-c., and Lee, H.-y. (2020b). Mockingjay: Unsupervised speech representation learning with deep bidirectional transformer encoders. Proc. IEEE international conference on acoustics, speech and signal processing doi:10.1109/icassp40776...

    2020

  25. [33]

    Liu, X., Zhang, F., Hou, Z., Mian, L., Wang, Z., Zhang, J., et al. (2021). Self-supervised learning: Generative or contrastive. IEEE Transactions on Knowledge and Data Engineering 35, 857–876

    2021

  26. [34]

    Lu, Y .-J., Chang, X., Li, C., Zhang, W., Cornell, S., Ni, Z., et al. (2022). ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding. In Proc. Annual Conference of the International Speech Communication Association. 5458–5462

    2022

  27. [35]

    and Mesgarani, N

    Luo, Y . and Mesgarani, N. (2019). Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation. IEEE/ACM transactions on audio, speech, and language processing27, 1256–1266

    2019

  28. [36]

    Masuyama, Y ., Chang, X., Cornell, S., Watanabe, S., and Ono, N. (2023). End-to-end integration of speech recognition, dereverberation, beamforming, and self-supervised learning representation. In Proc. IEEE Spoken Language Technology Workshop(IEEE), 260–265

    2023

  29. [37]

    Menne, T., Schl ¨uter, R., and Ney, H. (2019). Investigation into joint optimization of single channel speech enhancement and acoustic modeling for robust asr. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 6660–6664

    2019

  30. [38]

    D., Edin, J., Igel, C., et al

    Mohamed, A., Lee, H.-y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., et al. (2022). Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal Processing

    2022

  31. [39]

    Ochiai, T., Iwamoto, K., Delcroix, M., Ikeshita, R., Sato, H., Araki, S., et al. (2024). Rethinking processing distortions: Disentangling the impact of speech enhancement errors on speech recognition performance. arXiv preprint arXiv:2404.14860 O’Malley, T., Narayanan, A., Wan...

    2024 arXiv

  32. [40]

    Panayotov, V ., Chen, G., Povey, D., and Khudanpur, S. (2015). LibriSpeech: An ASR corpus based on public domain audio books. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing. 5206–5210

    2015

  33. [41]

    Pandey, A., Liu, C., Wang, Y ., and Saraf, Y . (2021). Dual application of speech enhancement for automatic speech recognition. In Proc. SLT 2021 (IEEE), 223–228

    2021

  34. [42]

    and Wang, D

    Pandey, A. and Wang, D. (2020). Densely connected neural network with dilated convolutions for real- time speech enhancement in the time domain. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 6629–6633

    2020

  35. [43]

    Pasad, A., Chou, J.-C., and Livescu, K. (2021). Layer-wise analysis of a self-supervised speech representation model. In Proc. IEEE Automatic Speech Recognition and Understanding Workshop (IEEE), 914–921

    2021

  36. [44]

    N., Schl ¨uter, R., and Watanabe, S

    Prabhavalkar, R., Hori, T., Sainath, T. N., Schl ¨uter, R., and Watanabe, S. (2023). End-to-end speech recognition: A survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2023

  37. [45]

    W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I

    Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023). Robust speech recognition via large-scale weak supervision. In International conference on machine learning (PMLR), 28492–28518

    2023

  38. [46]

    K., Dubey, H., Koishida, K., Nair, A., Gopal, V ., Cutler, R., et al

    Reddy, C. K., Dubey, H., Koishida, K., Nair, A., Gopal, V ., Cutler, R., et al. (2021). INTERSPEECH 2021 deep noise suppression challenge. In Proc. Annual Conference of the International Speech Communication Association. 2796–2800 Frontiers 20 Hiroshi Sato et al. Generic Speec...

    2021

  39. [47]

    K., Gopal, V ., and Cutler, R

    Reddy, C. K., Gopal, V ., and Cutler, R. (2022). Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In Proc. IEEE International Conference on

    2022

  40. [48]

    K., Gopal, V ., Cutler, R., Beyrami, E., Cheng, R., Dubey, H., et al

    Reddy, C. K., Gopal, V ., Cutler, R., Beyrami, E., Cheng, R., Dubey, H., et al. (2020). The interspeech 2020 deep noise suppression challenge: Datasets, subjective testing framework, and challenge results. arXiv preprint arXiv:2005.13981

    2020 arXiv

  41. [49]

    W., Beerends, J

    Rix, A. W., Beerends, J. G., Hollier, M. P., and Hekstra, A. P. (2001). Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs. In 2001 IEEE international conference on acoustics, speech, and signal processing...

    2001

  42. [50]

    Sato, H., Masumura, R., Ochiai, T., Delcroix, M., Moriya, T., Ashihara, T., et al. (2023). Downstream Task Agnostic Speech Enhancement with Self-Supervised Representation Loss. In Proc. Annual Conference of the International Speech Communication Association. 854–858

    2023

  43. [51]

    Sato, H., Moriya, T., Mimura, M., Horiguchi, S., Ochiai, T., Ashihara, T., et al. (2024). Speakerbeam- ss: Real-time target speaker extraction with lightweight conv-tasnet and state space modeling. arXiv preprint arXiv:2407.01857

    2024 arXiv

  44. [52]

    Sato, H., Ochiai, T., Delcroix, M., Kinoshita, K., Kamo, N., and Moriya, T. (2022). Learning to enhance or not: Neural network-based switching of enhanced and observed signals for overlapping speech recognition. In Proc. IEEE International Conference on Acoustics, Speech and S...

    2022

  45. [53]

    Sato, H., Ochiai, T., Delcroix, M., Kinoshita, K., Moriya, T., and Kamo, N. (2021). Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition. In Proc. Annual Conference of the International Speech Communication Association....

    2021

  46. [54]

    Schneider, S., Baevski, A., Collobert, R., and Auli, M. (2019). wav2vec: Unsupervised Pre-Training for Speech Recognition. In Proc. Annual Conference of the International Speech Communication Association. 3465–3469

    2019

  47. [55]

    Shi, J., Inaguma, H., Ma, X., Kulikov, I., and Sun, A. Y . (2024). Multi-resolution hubert: Multi-resolution speech self-supervised learning with masked unit prediction. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, ...

    2024

  48. [56]

    Shon, S., Tang, H., and Glass, J. (2019). V oiceID Loss: Speech Enhancement for Speaker Verification. In Proc. Annual Conference of the International Speech Communication Association. 2888–2892

    2019

  49. [57]

    S., Wang, X., Baskar, M

    Subramanian, A. S., Wang, X., Baskar, M. K., Watanabe, S., Taniguchi, T., Tran, D., et al. (2019). Speech enhancement using end-to-end speech recognition objectives. InProc. WASPAA 2019(IEEE), 234–238

    2019

  50. [58]

    E., Yoshioka, T., and Wang, H

    Thakker, M., Eskimez, S. E., Yoshioka, T., and Wang, H. (2022). Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation. In Proc. Annual Conference of the International Speech Communication Association. 991–995

    2022

  51. [59]

    Tsai, H.-S. et al. (2022). SUPERB-SG: Enhanced speech processing universal PERformance benchmark for semantic and generative capabilities. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 8479–8492

    2022

  52. [60]

    Vincent, E., Gribonval, R., and F ´evotte, C. (2006). Performance measurement in blind audio source separation. IEEE Transactions on Audio, Speech, and Language Processing14, 1462–1469

    2006

  53. [61]

    Wang, K., He, B., and Zhu, W.-P. (2021). Tstnn: Two-stage transformer based neural network for speech enhancement in the time domain. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 7098–7102 Frontiers 21 Hiroshi Sato et al. Generic Sp...

    2021

  54. [62]

    L., Saglam, M., Wilson, K., Chiao, A., Liu, R., et al

    Wang, Q., Moreno, I. L., Saglam, M., Wilson, K., Chiao, A., Liu, R., et al. (2020). V oiceFilter-Lite: Streaming targeted voice separation for on-device speech recognition. In Proc. Annual Conference of the International Speech Communication Association. 2677–2681

    2020

  55. [63]

    R., et al

    Wang, Q., Muckenhirn, H., Wilson, K., Sridhar, P., Wu, Z., Hershey, J. R., et al. (2019). V oiceFilter: Targeted V oice Separation by Speaker-Conditioned Spectrogram Masking. InProc. Annual Conference of the International Speech Communication Association. 2728–2732

    2019

  56. [64]

    Wang, Z.-Q., Cornell, S., Choi, S., Lee, Y ., Kim, B.-Y ., and Watanabe, S. (2023). Tf-gridnet: Making time-frequency domain models great again for monaural speaker separation. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 1–5

    2023

  57. [65]

    and Wang, D

    Wang, Z.-Q. and Wang, D. (2016). A joint training framework for robust automatic speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing24, 796–806

    2016

  58. [66]

    M., Gunawan, T

    Wani, T. M., Gunawan, T. S., Qadri, S. A. A., Kartiwi, M., and Ambikairajah, E. (2021). A comprehensive review of speech emotion recognition systems. IEEE access 9, 47795–47814

    2021

  59. [67]

    Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y ., et al. (2018). Espnet: End-to-end speech processing toolkit. In Proc. Annual Conference of the International Speech Communication Association. 2207–2211

    2018

  60. [68]

    Watanabe, S., Hori, T., Kim, S., Hershey, J., and Hayashi, T. (2017). Hybrid CTC/Attention architecture for end-to-end speech recognition. IEEE Journal on Selected Topics in Signal Processing 11, 1240– 1253

    2017

  61. [69]

    K., Dubey, H., Cutler, R., and Tashev, I

    Xia, Y ., Braun, S., Reddy, C. K., Dubey, H., Cutler, R., and Tashev, I. (2020). Weighted speech distortion losses for neural-network-based real-time speech enhancement. InProc. IEEE international conference on acoustics, speech and signal processing (IEEE), 871–875

    2020

  62. [70]

    Yadav, H., Sitaram, S., and Shah, R. R. (2024). Ms-hubert: Mitigating pre-training and inference mismatch in masked language modelling methods for learning speech representations. In Proc. Annual Conference of the International Speech Communication Association. 5053–5057

    2024

  63. [71]

    J., Lakhotia, K., Lin, Y

    Yang, S.-w., Chi, P.-H., Chuang, Y .-S., Lai, C.-I. J., Lakhotia, K., Lin, Y . Y ., et al. (2021). SUPERB: Speech Processing Universal PERformance Benchmark. In Proc. Annual Conference of the International Speech Communication Association. 1194–1198

    2021

  64. [72]

    Yoshioka, T., Ito, N., Delcroix, M., Ogawa, A., Kinoshita, K., Fujimoto, M., et al. (2015). The ntt chime- 3 system: Advances in speech enhancement and recognition for mobile multi-microphone devices. In Proc. IEEE Workshop on Automatic Speech Recognition and Understanding. 436–443

    2015

  65. [73]

    N., and Gan, W.-S

    Zhao, S., Ma, B., Watcharasupat, K. N., and Gan, W.-S. (2022). Frcrn: Boosting feature representation using frequency recurrence for monaural speech enhancement. In Proc. IEEE international conference on acoustics, speech and signal processing (IEEE), 9281–9285 ˇZmol´ıkov´a, K...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.