Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Lessons Learned from the URGENT 2024 Speech Enhancement Challenge

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The URGENT 2024 challenge analysis shows that speech enhancement systems are most defeated by overlapped speech, strong wideband or instantaneous noise, and high reverberation, and that combining multiple evaluation metrics into an…

desk verdict Useful measurement paper with real data findings, but the headline KRCC=0.73 is likely self-correlation because MOS is part of the overall ranking score; fix before treating it as evidence. read the letter →

arxiv 2506.01611 v1 pith:SAHNQ34J submitted 2025-06-02 eess.AS cs.SDeess.SP

Wangyou Zhang , Kohei Saijo , Samuele Cornell , Robin Scheibler , Chenda Li , Zhaoheng Ni , Anurag Kumar , Marvin Sach
show 5 more authors
Wei Wang Yihui Fu Shinji Watanabe Tim Fingscheidt Yanmin Qian
This is my paper
classification eess.AScs.SDeess.SP
keywords speechenhancementURGENT2024challengedatacleaningbandwidthmismatchlabelnoiseevaluationmetricsperceptualcorrelationnon-intrusive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper analyzes the URGENT 2024 speech enhancement challenge to learn where current SE systems fail and why evaluation misleads. It argues that standard public speech corpora are far less clean than assumed: many 'clean' samples have a lower effective bandwidth than their sampling frequency suggests, and audible noise survives in labels from WSJ, VCTK, and LibriTTS. It shows that modern enhancement systems most often fail on overlapped speech, strong wideband or instantaneous noise, and high reverberation, and that SNR does not predict which samples are hard. It then demonstrates that combining multiple objective metrics into an overall rank tracks human listening scores closely (Kendall rank correlation 0.73), whereas single-category metrics, especially non-intrusive ones, can be misleading.

What carries the argument

The argument is carried by two mechanisms. The first is the challenge's manually curated and vetted test data: 1000 non-blind samples and 1000 blind-test samples (half simulated, half real recordings) tagged for speech overlap, unison, voice style, noise type, SNR, and reverberation level, which lets the authors locate where systems fail. The second is the perceptual-correlation machinery: each objective metric's scores are ranked against human MOS via Kendall and linear correlation coefficients, and the overall ranking score produced by the challenge's rank-based aggregation rule is tested the same way. That aggregation rule, a weighted sum of individual metric ranks across 13 metrics, is the specific object shown to match human judgment at KRCC 0.73.

What would settle it

Apply the hard-sample rule from the linked repository to the public URGENT 2024 test sets and recount tag occurrences; the central lessons would weaken if overlapped speech, strong wideband or instantaneous noise, and high reverberation no longer dominate the hard set, or if the overall ranking's Kendall correlation with MOS falls well below 0.73 when the threshold rule is fully specified and reproducible.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck in speech enhancement is not only model design but the data and the measuring stick. Training-data inspection reveals widespread bandwidth mismatches — roughly a quarter of LibriTTS and essentially all DNS5 LibriVox and CommonVoice English samples carry a lower effective bandwidth than their sampling frequency implies — and label noise in supposedly clean corpora, which means much published SE work is effectively doing noisy-target training. On the challenge's blind test set, a tag-based analysis defines samples as hard when at least two teams enhance them poorly, and the hard samples cluster around overlapped speech, strong wideband or instantaneous noise, and high reverberation, even at high SNR. Finally, correlating each objective metric with human MOS shows the overall multi-metric ranking reaches KRCC 0.73, while individual metrics such as DNSMOS or WADA-SNR correlate poorly, supporting the paper's recommendation to evaluate with a broad metric set rather than any single score.

Load-bearing premise

The conclusions about which acoustic conditions are hardest depend on a 'hard sample' definition whose metric thresholds and weights are shared only through an online repository link, not specified in the paper, so different threshold choices could shift the difficulty ranking.

Editorial extensions

If this is right

  • SE data pipelines should verify effective bandwidth, not just nominal sampling frequency, and use stronger cleaning before treating corpus speech as clean targets.
  • Benchmarks should balance difficulty across overlapped speech, strong wideband or instantaneous noise, and high reverberation instead of using SNR as the proxy for difficulty.
  • SE evaluation should report a combination of intrusive, non-intrusive, and downstream-task metrics; the overall rank, not any single metric, is the best predictor of human opinion.
  • Newer non-intrusive metrics such as UTMOS and SCOREQ may be safer stand-ins for human quality than legacy ones like DNSMOS, but still should not be used alone.
  • Top ranks on non-intrusive metrics alone do not guarantee good perceived quality, as teams T17 and T18 show in the leaderboard.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the KRCC 0.73 result generalizes beyond this challenge, composite metric ranks could become the default leaderboard style for SE, nudging the field away from single-metric optimization.
  • The bandwidth-mismatch finding implies that TTS and voice-conversion models trained on LibriTTS or CommonVoice may have silently learned band-limited targets; re-evaluating them with bandwidth-verified data could alter reported quality.
  • A testable extension would build a difficulty predictor from the acoustic tags and compare it against SNR on the blind test set; the paper's data suggest such a predictor would match human perception better.
  • Because the hard-sample rule relies on metric thresholds shared only through an online link, the exact list of hardest conditions is a strong hypothesis rather than a fully specified measurement until those thresholds are published.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper reports lessons from the URGENT 2024 speech enhancement challenge, focusing on two aspects: data cleaning and evaluation metrics. On data, it documents mismatches between declared and effective audio bandwidths in public corpora, label noise in supposedly clean speech data, and the limitations of WADA-SNR as a difficulty measure. It then uses manual tags and a hard-sample classification to argue that overlapped speech, strong wideband/instantaneous noise, and high reverberation remain the most challenging conditions. On metrics, the paper compares objective metrics against subjective MOS scores, reports that UTMOS and SCOREQ correlate best with MOS, that non-intrusive metrics used alone can be misleading, and that the challenge's combined overall ranking score correlates strongly with MOS (KRCC 0.73). It concludes with recommendations for future SE system evaluation.

Significance. If the claims hold, the paper provides useful, concrete evidence for the SE community: the bandwidth-mismatch statistics and label-noise observations are actionable for data curation, and the comparison of intrusive, non-intrusive, and downstream-task metrics on a common challenge platform is a valuable community resource. The public release of analysis scripts and tagging metadata is a strength. However, the two most prominent quantitative conclusions—which conditions are hardest and which metrics best predict human judgment—currently rest on details that are either undisclosed (hard-sample thresholds and weights) or potentially circular (the MOS-vs-overall-rank correlation). These need to be addressed before the paper's recommendations can be fully trusted.

major comments (3)
  1. [Section 3.1, Table 1, Section 3.2, Fig. 4(a)] The headline KRCC of 0.73 between MOS rank and overall rank is potentially inflated by self-correlation. Section 3.1 states that MOS was one of the added challenge metrics and that the overall ranking score is the weighted sum of all metrics' individual rankings, so the overall rank includes MOS itself. Correlating MOS rank with a rank that contains MOS is not an independent validation, and the paper does not state that the overall ranking used for Fig. 4(a) was recomputed after excluding MOS. Please recompute the overall rank without MOS (and ideally without all subjective metrics) and report the resulting KRCC, along with confidence intervals. If the recomputed correlation is substantially lower, the conclusion that combining objective metrics yields strong agreement with human listening scores is unsupported as stated.
  2. [Section 2.3 and Fig. 3] The hard-sample classification is load-bearing for the 'hardest conditions' conclusion, but its definition is not reproducible from the paper. A sample is declared hard if it is poorly enhanced by at least two teams, and 'poorly enhanced' is determined by thresholding all evaluation metrics with metric-wise weights; neither the thresholds nor the weights are given, and the only reference is a GitHub link. The tag-occurrence analysis in Fig. 3 and the resulting ranking of acoustic conditions (overlapped speech, strong noise, high reverberation) depend directly on these free parameters. Please report the exact thresholds and weights in the paper, and include a sensitivity analysis showing that the main conclusions are stable under reasonable variations.
  3. [Section 3.2, Fig. 4] The metric-comparison evidence is statistically fragile and may also suffer from an evaluation-set mismatch. The correlations are computed across only 23 systems (21 teams plus baseline and noisy input), yet no confidence intervals or significance tests are reported, making it hard to judge whether the ordering of metrics (e.g., UTMOS/SCOREQ above NISQA/DNSMOS) is meaningful. In addition, MOS was measured on a 300-sample subset, while several objective metrics in Table 1 are computed on the full blind test set or on the simulated half only; the paper does not state whether the objective metrics were recomputed on the same 300-sample MOS subset before correlation. Please report confidence intervals and clarify or correct the evaluation-set alignment.
minor comments (4)
  1. [Fig. 3 caption] The caption says the detailed hard-sample rule 'can be found in Section 2.2', but Section 2.2 describes the non-blind test data and does not define the rule; the definition appears only in Section 2.3 and is incomplete there. Please fix the cross-reference and move the full rule into the main text.
  2. [Section 3.2, Fig. 4(b)] For metrics where lower values are better (e.g., MCD, LSD, WADA-SNR), it is unclear whether the reported LCC is the raw correlation or its absolute value, and whether the sign convention is consistent with the 'higher is better' interpretation. Please state the sign convention used.
  3. [Section 2.3] The phrase 'strong wideband background noise and instantaneous noise' is not precisely defined in the tag taxonomy; please clarify how the 'instantaneous noise' tag differs from other noise-type tags and whether the conclusion is driven by co-occurring tags.
  4. [Table 1] The table includes 'Noisy input' as a row, which is useful, but the reader must infer that the comparison set is 23 systems rather than the 21 participating teams. Please state the total number of systems used in the correlation analysis explicitly in the text.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported KRCC=0.73 between MOS and the overall ranking score is inflated because MOS is one of the metrics inside that overall score.

  1. self definitional [Sec. 3.1 (overall-ranking definition; MOS added to challenge) and Sec. 3.2 (KRCC vs MOS), Table 1 / Fig. 4(a).]
    "Compared to the initial design in [1], two new metrics (i.e., POLQA and MOS) are added to better reflect the quality of the enhanced audio, and the same ranking rule as in [1] is adopted to obtain the final results. ... The final results are summarized in Table 1, where the overall ranking score in the last column is the weighted sum of all metrics’ individual rankings [1]. ... The KRCC between the MOS rank and the overall rank is 0.73, showing strong consistency between the MOS and the overall ranking score."

    The overall ranking score is defined as a weighted sum of the individual rankings of all challenge metrics, and MOS was explicitly added as one of those metrics. Consequently the MOS rank is an input to the overall rank. The paper then reports KRCC=0.73 between MOS rank and this overall rank as evidence that a multifaceted evaluation correlates with human judgment. This is a partial self-correlation: a component of a weighted average is correlated with the average by construction, even if the other components carried no signal. The paper does not recompute the overall ranking with MOS excluded, so the headline statistic is inflated and cannot serve as an independent validation that objective metric combinations track MOS.

full rationale

The data-cleaning findings (bandwidth mismatch, label noise) are self-contained empirical analyses and are not circular. The hard-sample difficulty ranking depends on the authors' own metric thresholds, which is a transparency and robustness concern rather than a circularity, because the acoustic tags are manually labeled independently of those thresholds. The one load-bearing circular step is the perceptual-correlation result in Section 3.2: the overall ranking score is defined as a weighted sum of all metrics' rankings, including the subjective MOS, and the paper then uses KRCC between MOS and that overall score as evidence that combining metrics matches human judgment. This is a self-correlation; a leave-MOS-out overall rank would be needed to support the claim. The KRCC values for individual objective metrics in Fig. 4 are not affected, since those metrics are not part of the aggregate. Score 6 reflects that the central quantitative evidence for metric combination is partially circular, while the rest of the paper's analysis remains independent.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted physical parameters. Its load-bearing analytical choices are the undisclosed hard-sample thresholds and weights, alongside assumptions about the reliability of WADA-SNR, the representativeness of the MOS subset, and manual listening as ground truth.

free parameters (2)
  • Hard-sample metric thresholds
    Section 2.3 defines hard samples by thresholding all evaluation metrics with metric-wise weights; the actual threshold values are not reported in the paper and are only linked on GitHub, and they directly affect the difficulty findings.
  • Metric-wise weights for hard-sample classification
    Footnote 5 points to a GitHub repository for the weights used to combine metrics into the hard/other split. These weights are hand-chosen analysis parameters, not justified in the text, and they shape the conclusion about which acoustic conditions are hardest.
assumptions (4)
  • domain assumption Relative WADA-SNR values are indicative of label noise levels despite acknowledged absolute inaccuracy.
    Section 2.1 uses WADA-SNR histograms to claim that WSJ, VCTK, and LibriTTS contain noisy labels, while noting the algorithm is imprecise.
  • domain assumption The 300-sample MOS subset is representative of the full blind test distribution.
    Section 3.1 states the subset was carefully selected to preserve distribution in domains, difficulty, and degradations, but no quantitative verification is provided.
  • domain assumption Manual listening verification of low WADA-SNR labels is ground truth for label noisiness.
    Section 2.1 says negative-WADA-SNR labels were 'manually verified to be mostly noisy'; human listening is treated as the reference without an inter-rater reliability check.
  • ad hoc to paper The pre-defined hard-sample thresholds and metric weights are a valid proxy for task difficulty.
    Section 2.3 defines hard samples using thresholds and weights hosted on GitHub; the paper does not justify these choices, making the difficulty analysis dependent on an ad-hoc classification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lessons Learned from the URGENT 2024 Speech Enhancement Challenge." pith.science (2026). https://pith.science/paper/SAHNQ34J

@misc{pith2026250601611,
  author       = {Pith},
  title        = {Pith review of: Lessons Learned from the URGENT 2024 Speech Enhancement Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SAHNQ34J}},
  note         = {Machine review of arXiv:2506.01611}
}
read the original abstract

The URGENT 2024 Challenge aims to foster speech enhancement (SE) techniques with great universality, robustness, and generalizability, featuring a broader task definition, large-scale multi-domain data, and comprehensive evaluation metrics. Nourished by the challenge outcomes, this paper presents an in-depth analysis of two key, yet understudied, issues in SE system development: data cleaning and evaluation metrics. We highlight several overlooked problems in traditional SE pipelines: (1) mismatches between declared and effective audio bandwidths, along with label noise even in various "high-quality" speech corpora; (2) lack of both effective SE systems to conquer the hardest conditions (e.g., speech overlap, strong noise / reverberation) and reliable measure of speech sample difficulty; (3) importance of combining multifaceted metrics for a comprehensive evaluation correlating well with human judgment. We hope that this endeavor can inspire improved SE pipeline designs in the future.

Figures

Figures reproduced from arXiv: 2506.01611 by the authors.

Figure 1
Figure 1. Sampling frequency distribution of speech and noise data in the training set. The original noise data are all in 48 kHz. (a) Training + validation sets (b) Non-blind test set (c) Blind test set (simulated part) < -10 0.0 0.5 -10~0 1.0 1.5 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Histogram of estimated SNRs of the “clean” speech labels in each dataset based on the WADA algorithm [19]. et al. [1] managed to address the first one, the other remains an open question that requires more investigations in future work. Below we discuss each problem in detail. 1) Effective bandwidth vs. actual sampling frequency: We found that various public speech corpora contain “bad” sam￾ples with a lower effecti… view at source ↗
Figure 3
Figure 3. Tag occurrences of different samples in the non-blind and blind test sets. The “hard samples” are defined as samples with poor metric scores below pre-defined thresholds. The detailed rule for determining hard samples can be found in Section 2.2. all-positive WADA-SNR values in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Correlations (KRCC and LCC) between MOS and other objective metrics on the blind test data. the remaining teams often show mixed rankings among differ￾ent metrics. For example, teams T17 and T18 show excellent performance on non-intrusive SE metrics (i.e., DNSMOS and N…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Improving Speech Enhancement with Multi-Metric Supervision from Learned Quality Assessment

    cs.SD 2025-06 conditional novelty 6.0 of 10

    A multi-metric learned quality model (Uni-VERSA-Ext) is used as a differentiable training loss for speech enhancement, with a regularization term to prevent adversarial exploitation.

Reference graph

Works this paper leans on

72 extracted references · 71 canonical work pages · cited by 1 Pith paper

  1. [1]

    It features a broader SE task definition, large-scale and diverse source data, and extensive evaluation metrics, complementing existing SE challenges [2–9]

    Introduction The URGENT 2024 Challenge [1] is a newly launched competi- tion that aims to assess and advance the universality, robustness, and generalizability of speech enhancement (SE) techniques. It features a broader SE task definition, large-scale and diverse source data, and extensive evaluation metrics, complementing existing SE challenges [2–9]. S...

    work page 2024

  2. [2]

    For example, ~25% LibriTTS [11], ~100% DNS5 LibriV ox [3], and ~100% CommonV oice 11.0 [12] English speech data have mismatched audio bandwidths

    A mismatch between declared and effective audio band- widths commonly exist in both public audio corpora and in-the- wild data. For example, ~25% LibriTTS [11], ~100% DNS5 LibriV ox [3], and ~100% CommonV oice 11.0 [12] English speech data have mismatched audio bandwidths. Furthermore, the label noisiness issue is also surprisingly common, even in suppose...

    work page

  3. [3]

    Moreover, there is no good objective measure of the sample difficulty for the SE task, and conventional metrics such as signal-to-noise ratio (SNR) are often misleading

    Overlapped speech, strong wideband / instantaneous noise, and high reverberation remain the hardest challenges in SE tasks. Moreover, there is no good objective measure of the sample difficulty for the SE task, and conventional metrics such as signal-to-noise ratio (SNR) are often misleading

    work page

  4. [4]

    Interestingly, some newly proposed non-intrusive met- rics such as UTMOS [16] and VQScore [17] show better per- ceptual correlations than traditional ones (e.g., DNSMOS [18]), despite that they may not be designed directly for the SE task

    work page

  5. [5]

    Section 3.1) of the SE performance, which leads to high correlations with human judgments

    It is beneficial to combine various metrics for a mul- tifaceted evaluation (cf. Section 3.1) of the SE performance, which leads to high correlations with human judgments. On the other hand, using solely asinglecategory of metrics, especially non-intrusive metrics, for SE performance assessment can be unreliable or even misleading

    work page

  6. [6]

    hard samples

    Data Analysis 2.1. Training and validation data Both training and validation datasets of the URGENT 2024 Challenge are fully simulated based on 5 public speech corpora (DNS5 LibriV ox speech [3], LibriTTS, CommonV oice 11.0 En- glish portion [12], VCTK, and WSJ), 2 noise corpora (Au- dioset+FreeSound noise in DNS5 challenge [3], and WHAM! noise recordings...

    arXiv 2024

  7. [7]

    Modern SE systems tend to be robust against differ- ent voice styles, speaker genders, and different distortion types (e.g., bandwidth limitation, clipping). However, the follow- ing three acoustic conditions remain the most challenges for SE models to cope with: overlapped speech (denoted by thespeech_overlapandunisontags), strong wideband background noi...

    work page

  8. [8]

    How- ever, a reliable objective measure of speech sample difficulty is 5 https://github.com/urgent-challenge/ urgent2024_analysis/blob/main/tagging/

    Samples with perceptually low intelligibility (often caused by strong degradations) are often hard to process. How- ever, a reliable objective measure of speech sample difficulty is 5 https://github.com/urgent-challenge/ urgent2024_analysis/blob/main/tagging/. still unavailable. Conventional metrics such as SNR fail to cor- relate well with the difficulty...

    work page

Show all 72 references
  1. [9]

    Exploring better objective metrics that can reflect the speech sample difficulty more accurately, thus improving the design of SE corpora with balanced difficulty distribution

  2. [10]

    Designing more universal SE systems that can handle a wide variety of distortion types

  3. [11]

    Improving SE methods to tackle overlapped speech, strong noise and high reverberation

  4. [12]

    Metric Analysis In this section, we analyze and discuss the participants’ results on the blind test dataset to evaluate and reflect on the design of the challenge’s evaluation metrics. 3.1. Overview of evaluation metrics This section first provides an overview of the evaluatio...

  5. [13]

    Non-intrusive SE metrics: DNSMOS and NISQA [30]

  6. [14]

    Intrusive SE metrics: POLQA [31], PESQ [32], ESTOI [33], signal-to-distortion ratio (SDR) [34], mel cepstral distortion (MCD) [35], log-spectral distance (LSD) [36]

  7. [15]

    Downstream-task-independent metrics: Levenshtein phoneme similarity (LPS) [37] and SpeechBERTScore [38]

  8. [16]

    Downstream-task-dependent metrics: speaker similarity (SpkSim) and word accuracy (W Acc)

  9. [17]

    Subjective metric: mean opinion score (MOS). Compared to the initial design in [1], two new metrics (i.e., POLQA and MOS) are added to better reflect the quality of the enhanced audio, and the same ranking rule as in [1] is adopted to obtain the final results. Note that the MO...

  10. [18]

    In contrast, W ADA-SNR shows the lowest correlations with MOS, suggest- ing that it is unsuitable for measuring enhanced audio quality

    Among all objective metrics, UTMOS and SCOREQ demonstrate the highest correlations with MOS. In contrast, W ADA-SNR shows the lowest correlations with MOS, suggest- ing that it is unsuitable for measuring enhanced audio quality

  11. [19]

    Among all challenge metrics, LSD and DNSMOS lag be- hind others on KRCC, indicating that they may not accurately reflect the overall SE performance when used alone. Mean- while, the relatively high LCC value of DNSMOS implies that the rank mismatch between MOS and DNSMOS can b...

  12. [20]

    Based on the above analysis, the best practice of SE evalu- ation can be summarized as follows:

    SpkSim and W Acc show relatively high KRCCs with much lower LCCs, suggesting that downstream-task-dependent met- ric values are likely to correlate non-linearly with the audio per- ceptual quality. Based on the above analysis, the best practice of SE evalu- ation can be summar...

  13. [21]

    Adopting advanced non-intrusive metrics (e.g., UTMOS and SCOREQ) in lieu of / along with traditional ones (e.g., DNS- MOS) for more accurate SE performance assessment

  14. [22]

    Utilizing multifaceted metrics for comprehensive SE evalua- tions rather than using a single category of metrics, especially purely non-intrusive ones

  15. [23]

    Conclusion In this paper, we have presented an in-depth analysis of the recent URGENT 2024 Challenge from data and metric per- spectives. By looking into the distributions of various data at- tributes (e.g., sampling frequency, estimated SNR, tag occur- rence), we have reveale...

    2024

  16. [24]

    The subjective listening test was funded and executed by Technische Universität Braun- schweig, strictly following ITU-T Recommendation P.808 [39, 40]

    Acknowledgment The leaderboard evaluation has been supported by the PSC Bridges2 system via ACCESS allocation CIS210014, supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. The subjective listening test was funded and executed ...

  17. [25]

    URGENT challenge: Universality, robustness, and generalizability for speech enhancement,

    W. Zhanget al., “URGENT challenge: Universality, robustness, and generalizability for speech enhancement,” inInterspeech, 2024, pp. 4868–4872

    2024

  18. [26]

    ConferencingSpeech challenge: Towards far-field multi-channel speech enhancement for video conferencing,

    W. Raoet al., “ConferencingSpeech challenge: Towards far-field multi-channel speech enhancement for video conferencing,” in Proc. IEEE ASRU, 2021, pp. 679–686

    2021

  19. [27]

    ICASSP 2023 deep noise suppression chal- lenge,

    H. Dubeyet al., “ICASSP 2023 deep noise suppression chal- lenge,”IEEE Open Journal of Signal Processing, pp. 1–13, 2024

    2023

  20. [28]

    ICASSP 2024 speech signal improve- ment challenge,

    N.-C. Risteaet al., “ICASSP 2024 speech signal improve- ment challenge,” inIEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2024, pp. 15–16

    2024

  21. [29]

    INTERSPEECH 2022 audio deep packet loss concealment challenge,

    L. Dieneret al., “INTERSPEECH 2022 audio deep packet loss concealment challenge,” inInterspeech, 2022, pp. 580–584

    2022

  22. [30]

    ICASSP 2023 acoustic echo cancellation chal- lenge,

    R. Cutleret al., “ICASSP 2023 acoustic echo cancellation chal- lenge,”IEEE Open Journal of Signal Processing, 2024

    2023

  23. [31]

    The 2nd clarity enhancement challenge for hearing aid speech intelligibility enhancement: Overview and outcomes,

    M. A. Akeroydet al., “The 2nd clarity enhancement challenge for hearing aid speech intelligibility enhancement: Overview and outcomes,” inICASSP, 2023

    2023

  24. [32]

    Overview of the L3DAS23 challenge on audio-visual extended reality,

    C. Marinoniet al., “Overview of the L3DAS23 challenge on audio-visual extended reality,” inICASSP, 2023

    2023

  25. [33]

    The CHiME-7 UDASE task: Unsupervised do- main adaptation for conversational speech enhancement,

    S. Leglaiveet al., “The CHiME-7 UDASE task: Unsupervised do- main adaptation for conversational speech enhancement,” inProc. CHiME, 2023

    2023

  26. [34]

    The use of ranks to avoid the assumption of nor- mality implicit in the analysis of variance,

    M. Friedman, “The use of ranks to avoid the assumption of nor- mality implicit in the analysis of variance,”Journal of the Ameri- can Statistical Association, vol. 32, no. 200, pp. 675–701, 1937

    1937

  27. [35]

    LibriTTS: A corpus derived from LibriSpeech for text-to-speech,

    H. Zenet al., “LibriTTS: A corpus derived from LibriSpeech for text-to-speech,” inInterspeech, 2019, pp. 1526–1530

    2019

  28. [36]

    Common voice: A massively-multilingual speech corpus,

    R. Ardilaet al., “Common voice: A massively-multilingual speech corpus,” inProc. LREC, 2020, pp. 4218–4222

    2020

  29. [37]

    LDC,LDC Catalog: CSR-I (WSJ0) Complete, University of Pennsylvania, 1993

    1993

  30. [38]

    Philadelphia: Linguistic Data Consortium,LDC Catalog: CSR-II (WSJ1) Complete LDC94S13A, 1994

    1994

  31. [39]

    The V oice Bank corpus: Design, collection and data analysis of a large regional accent speech database,

    C. Veaux, J. Yamagishi, and S. King, “The V oice Bank corpus: Design, collection and data analysis of a large regional accent speech database,” inProc. O-COCOSDA/CASLRE, 2013, pp. 1–4

    2013

  32. [40]

    UTMOS: UTokyo-SaruLab system for V oiceMOS challenge 2022,

    T. Saekiet al., “UTMOS: UTokyo-SaruLab system for V oiceMOS challenge 2022,” inInterspeech, 2022, pp. 4521–4525

    2022

  33. [41]

    Self-supervised speech quality estimation and enhancement using only clean speech,

    S.-W. Fuet al., “Self-supervised speech quality estimation and enhancement using only clean speech,” inProc. ICLR, 2024

    2024

  34. [42]

    DNSMOS P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” inICASSP, 2022, pp. 886–890

    2022

  35. [43]

    Robust signal-to-noise ratio estima- tion based on waveform amplitude distribution analysis,

    C. Kim and R. M. Stern, “Robust signal-to-noise ratio estima- tion based on waveform amplitude distribution analysis,” inInter- speech, 2008, pp. 2598–2601

    2008

  36. [44]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichernet al., “WHAM!: Extending speech separation to noisy environments,” inInterspeech, 2019, pp. 1368–1372

    2019

  37. [45]

    Hi-Fi multi-speaker English TTS dataset,

    E. Bakhturinaet al., “Hi-Fi multi-speaker English TTS dataset,” inInterspeech, 2021, pp. 2776–2780

    2021

  38. [46]

    Noisy-target training: A training strategy for DNN-based speech enhancement without clean speech,

    T. Fujimuraet al., “Noisy-target training: A training strategy for DNN-based speech enhancement without clean speech,” in29th European Signal Processing Conference (EUSIPCO). IEEE, 2021, pp. 436–440

    2021

  39. [47]

    Analysis of noisy-target training for DNN-based speech enhancement,

    T. Fujimura and T. Toda, “Analysis of noisy-target training for DNN-based speech enhancement,” inICASSP, 2023

    2023

  40. [48]

    TF-GridNet: Integrating full-and sub- band modeling for speech separation,

    Z.-Q. Wanget al., “TF-GridNet: Integrating full-and sub- band modeling for speech separation,”IEEE/ACM Trans. ASLP ., vol. 31, pp. 3221–3236, 2023

    2023

  41. [49]

    Interspeech 2025 URGENT speech enhancement challenge,

    K. Saijoet al., “Interspeech 2025 URGENT speech enhancement challenge,”Accepted by Interspeech, 2025

    2025

  42. [50]

    The diverse environments multi-channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,

    J. Thiemann, N. Ito, and E. Vincent, “The diverse environments multi-channel acoustic noise database (DEMAND): A database of multichannel environmental noise recordings,” inProceedings of Meetings on Acoustics, vol. 19, no. 1, 2013

    2013

  43. [51]

    A multi-device dataset for urban acoustic scene classification,

    A. Mesaros, T. Heittola, and T. Virtanen, “A multi-device dataset for urban acoustic scene classification,” inProceedings of the De- tection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE2018), 2018, pp. 9–13

    2018

  44. [52]

    A study on data augmentation of reverberant speech for robust speech recognition,

    T. Koet al., “A study on data augmentation of reverberant speech for robust speech recognition,” inICASSP, 2017, pp. 5220–5224

    2017

  45. [53]

    MYRiAD: A multi-array room acoustic database,

    T. Dietzenet al., “MYRiAD: A multi-array room acoustic database,”EURASIP Journal on Audio, Speech, and Music Pro- cessing, vol. 2023, no. 1, p. 17, 2023

    2023

  46. [54]

    NISQA: A deep CNN-self-attention model for multidimensional speech quality prediction with crowdsourced datasets,

    G. Mittaget al., “NISQA: A deep CNN-self-attention model for multidimensional speech quality prediction with crowdsourced datasets,” inInterspeech, 2021, pp. 2127–2131

    2021

  47. [55]

    Perceptual objective listening quality as- sessment (POLQA), the third generation ITU-T standard for end- to-end speech quality measurement part I–—temporal alignment,

    J. G. Beerendset al., “Perceptual objective listening quality as- sessment (POLQA), the third generation ITU-T standard for end- to-end speech quality measurement part I–—temporal alignment,” AES Journal, vol. 61, no. 6, pp. 366–384, 2013

    2013

  48. [56]

    Perceptual evaluation of speech quality (PESQ)—a new method for speech quality assessment of tele- phone networks and codecs,

    A. W. Rixet al., “Perceptual evaluation of speech quality (PESQ)—a new method for speech quality assessment of tele- phone networks and codecs,” inICASSP, 2001, pp. 749–752

    2001

  49. [57]

    An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,”IEEE/ACM Trans. ASLP ., vol. 24, no. 11, pp. 2009–2022, 2016

    2009

  50. [58]

    Performance mea- surement in blind audio source separation,

    E. Vincent, R. Gribonval, and C. Févotte, “Performance mea- surement in blind audio source separation,”IEEE Trans. ASLP ., vol. 14, no. 4, pp. 1462–1469, 2006

    2006

  51. [59]

    Mel-cepstral distance measure for objective speech quality assessment,

    R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” inProc. PACRIM, 1993, pp. 125–128

    1993

  52. [60]

    Distance measures for speech process- ing,

    A. Gray and J. Markel, “Distance measures for speech process- ing,”IEEE Transactions on Acoustics, Speech, and Signal Pro- cessing, vol. 24, no. 5, pp. 380–391, 1976

    1976

  53. [61]

    Evaluation metrics for generative speech en- hancement methods: Issues and perspectives,

    J. Pirklbaueret al., “Evaluation metrics for generative speech en- hancement methods: Issues and perspectives,” inSpeech Commu- nication; 15th ITG Conference, 2023, pp. 265–269

    2023

  54. [62]

    SpeechBERTScore: Reference-aware automatic evaluation of speech generation leveraging NLP evaluation met- rics,

    T. Saekiet al., “SpeechBERTScore: Reference-aware automatic evaluation of speech generation leveraging NLP evaluation met- rics,” inInterspeech, 2024, pp. 4943–4947

    2024

  55. [63]

    ITU-T recommendation P.808, subjective evaluation of speech quality with a crowdsourcing approach,

    “ITU-T recommendation P.808, subjective evaluation of speech quality with a crowdsourcing approach,” ITU-T, Jun. 2018

    2018

  56. [64]

    An open source implementation of ITU- T recommendation P.808 with validation,

    B. Naderi and R. Cutler, “An open source implementation of ITU- T recommendation P.808 with validation,” inInterspeech, 2020, pp. 2862–2866

    2020

  57. [65]

    RaD-Net 2: A causal two-stage repairing and de- noising speech enhancement network with knowledge distillation and complex axial self-attention,

    M. Liuet al., “RaD-Net 2: A causal two-stage repairing and de- noising speech enhancement network with knowledge distillation and complex axial self-attention,” inInterspeech, 2024, pp. 1700– 1704

    2024

  58. [66]

    Personalized speech enhancement without a separate speaker embedding model,

    T. Pärnamaa and A. Saabas, “Personalized speech enhancement without a separate speaker embedding model,” inInterspeech, 2024, pp. 4863–4867

    2024

  59. [67]

    The PESQetarian: On the relevance of Goodhart’s law for speech enhancement,

    D. de Oliveiraet al., “The PESQetarian: On the relevance of Goodhart’s law for speech enhancement,” inInterspeech, 2024, pp. 3854–3858

    2024

  60. [68]

    DNSMOS Pro: A reduced-size DNN for proba- bilistic MOS of speech,

    F. Cumlinet al., “DNSMOS Pro: A reduced-size DNN for proba- bilistic MOS of speech,” inInterspeech, 2024, pp. 4818–4822

    2024

  61. [69]

    HiFi++: A unified framework for bandwidth extension and speech enhancement,

    P. Andreevet al., “HiFi++: A unified framework for bandwidth extension and speech enhancement,” inICASSP, 2023

    2023

  62. [70]

    SCOREQ: Speech qual- ity assessment with contrastive regression,

    A. Ragano, J. Skoglund, and A. Hines, “SCOREQ: Speech qual- ity assessment with contrastive regression,” inAdvances in Neu- ral Information Processing Systems, vol. 37, 2024, pp. 105 702– 105 729

    2024

  63. [71]

    A new measure of rank correlation,

    M. G. Kendall, “A new measure of rank correlation,”Biometrika, vol. 30, no. 1–2, pp. 81–93, 1938

    1938

  64. [72]

    Notes on the history of correlation,

    K. Pearson, “Notes on the history of correlation,”Biometrika, vol. 13, no. 1, pp. 25–45, 1920

    1920

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.