Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Uni-VERSA: Versatile Speech Assessment with a Unified Network

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single network can predict 11 speech-quality metrics at once

desk verdict A useful benchmark-plus-baseline paper whose headline claims outrun the evidence; needs loss normalization, error bars, and a comparison with Torchaudio-Squim. read the letter →

arxiv 2505.20741 v1 pith:65BCW2X7 submitted 2025-05-27 cs.SD eess.AS

Jiatong Shi , Hye-Jin Shim , Shinji Watanabe This is my paper · ORCID
classification cs.SDeess.AS
keywords speechqualityassessmentmulti-tasklearningobjectivemetricsself-supervisedrepresentationsmeanopinionscoreenhancementevaluationsynthesisunifiednetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Uni-VERSA, a single neural network that predicts eleven objective speech-quality metrics simultaneously, covering noise level, prosody, naturalness, intelligibility, and speaker characteristics. The authors argue that speech quality is naturally multidimensional, so evaluating it with one metric at a time risks fragmented or biased results. They construct a large benchmark from a speech-enhancement challenge, annotate it with a versatile evaluation toolkit, and train a baseline that takes self-supervised speech representations as input. Their experiments show that joint prediction across all five domains is a viable alternative to running separate metric tools, that it aligns closely with human perception on a human-rated subset, and that it can be much faster than computing metrics individually. If correct, a single forward pass could replace a toolbox of separate evaluations for enhancement, synthesis, and quality control.

What carries the argument

The central object is the Uni-VERSA architecture: a frozen self-supervised speech feature extractor (a large pre-trained speech model) feeding three four-layer transformer encoders—one for the target audio, one for reference audio, and one for reference text—with two cross-attention modules aligning reference information to the target speech, followed by one mean-pooled linear predictor per metric. The training objective is simply the sum of absolute errors between each predicted metric and its ground-truth value, with predictors masked when a label is missing and zero-padded placeholders when a reference signal is absent. This machinery lets a single model share representations across very different metric scales and domains, and the reference branches make the model optionally intrusive or non-intrusive.

What would settle it

Collect a new set of human mean-opinion ratings on a random sample of the benchmark utterances (or on a held-out set of enhanced or synthesised speech), then compute the correlation between Uni-VERSA's predicted metrics and those human ratings. If the correlations are no better than a single-metric baseline or drop sharply in out-of-domain material such as conversational speech, the claim that one network provides a human-aligned, multidimensional assessment would be refuted.

Watch

Extended reading notes

Core claim

The paper claims that one multi-task network, Uni-VERSA, can accurately predict eleven established objective metrics spanning five domains—noise level (SI-SNR, PESQ, DNSMOS), prosody (F0 correlation), naturalness (MOS, UTMOS, SHEET-base), intelligibility (WER, STOI, Speech BERT Score), and speaker characteristics (speaker embedding similarity)—from a single forward pass. The model takes the target speech plus optional reference audio and reference text as inputs, uses a frozen self-supervised speech encoder with three lightweight encoders and cross-attention, and outputs one number per metric through simple pooled linear predictors. It is trained on a benchmark built from a speech-enhancement challenge, using a semi-supervised scheme that masks predictors when their labels are unavailable. The authors report that this unified predictor matches or beats single-aspect evaluation methods, aligns closely with human MOS on a subset of 6,900 human-rated utterances, and achieves up to a 109× speedup over computing the metrics directly.

Load-bearing premise

The ground-truth labels for ten of the eleven metrics are computer-generated by the authors' own evaluation toolkit rather than rated by human listeners, so the claim of close alignment with human perception depends on those computed metrics being valid proxies for what humans hear.

Editorial extensions

If this is right

  • A single forward pass through Uni-VERSA replaces running eleven separate metric tools, making multidimensional evaluation practical at scale, with the authors reporting up to a 109× speedup on their test set.
  • The framework can be applied across speech enhancement, speech synthesis, and conversational quality control without retraining, though the paper shows out-of-domain performance degrades for TTS and conversation.
  • The reference-free variant performs competitively with the full-reference variant, suggesting that intrusive metrics like PESQ and STOI can be approximated without reference signals once trained on enough in-domain data.
  • The shared multi-task representation lets one model serve both reference-based and non-intrusive evaluation, depending on whether reference audio and text are supplied.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the metric-proxy assumption holds, the same architecture could be extended to other acoustic domains such as music or general audio by swapping the label set for domain-appropriate metrics, which the paper lists as future work.
  • The ablation shows joint prediction helps Fbank-based models but slightly hurts a strong SSL-based MOS-only model, suggesting that the benefit of multi-task training depends on the capacity of the feature extractor; a larger human-labeled set would clarify whether joint prediction truly improves perceptual alignment.
  • The paper's 109× speedup compares Uni-VERSA to recomputing all metrics from scratch; in practice a user who only needs one or two metrics might still prefer the dedicated single-metric model, so the appeal is strongest for comprehensive profiling workflows.
  • Because the ground truth for most metrics is itself generated by models, Uni-VERSA is in effect a distilled approximation of a metric toolbox; validating it against human listeners on a broader range of input types would be the natural next test beyond the 6,900-utterance MOS subset.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces Uni-VERSA, a unified multi-task neural network that predicts eleven objective speech metrics spanning five domains (noise level, prosody, naturalness, intelligibility, and speaker characteristics) from a single forward pass. The authors formalize the framework, construct a benchmark from the URGENT24 challenge blind test set using the VERSA toolkit to generate labels (plus a human MOS subset), and evaluate a baseline built on frozen self-supervised speech representations with optional reference audio/text encoders and cross-attention. Experiments report LCC/SRCC per metric across feature extractors (Fbank, HuBERT, MR-HuBERT, WavLM), ablations on reference input availability, a comparison of single-task versus joint MOS prediction, and out-of-domain evaluations on TTS and conversational speech, along with a claimed 109x speedup over direct metric computation. The paper's central claims are that Uni-VERSA provides a viable alternative to single-aspect evaluation methods and aligns closely with human perception.

Significance. If the central claims were fully supported, Uni-VERSA would be a practically valuable contribution: a single model producing a multidimensional quality profile with one inference pass, with an open model collection and benchmark, would be useful to the speech enhancement, synthesis, and quality-control communities. The paper also contributes a clear formalization of the multi-metric prediction problem and a systematic comparison of SSL features for this task. However, the claimed advantage over single-aspect methods is not established by the evidence presented: the only human-labeled metric (MOS) shows degraded joint-task performance for the best feature extractor, and the perceptual-alignment claim rests on a small human-labeled subset plus an assumption that VERSA-computed metrics are perceptual proxies. The efficiency gain is credible and well demonstrated.

major comments (3)
  1. [Eq. (2) and Table 4] The claim in the abstract and Section 1 that Uni-VERSA 'outperforms methods focusing on a single aspect' is not supported by Table 4 for the only human-labeled metric. For WavLM features, joint prediction achieves MOS LCC/SRCC of 0.77/0.77 while single-task MOS prediction achieves 0.80/0.80. This is consistent with the unnormalized sum-of-L1-loss formulation in Eq. (2): Table 1 lists SI-SNR and WER as unbounded, F0-CORR and SPK-SIM in [-1,1], and MOS/PESQ in [1,5], so the loss is dominated by large-magnitude metrics and provides no incentive to preserve MOS accuracy. No confidence intervals or significance tests are reported, so the 0.03 gap may be noise, but the direction contradicts the stated claim. The authors should either introduce a properly normalized or weighted multi-task loss, report significance/confidence, and re-evaluate, or substantially temper the claim that joint prediction outperforms single-aspect methods on human-perceived quality.
  2. [Sec. 3.1 and Tables 1-3] Ten of the eleven ground-truth metrics are produced by the authors' own VERSA toolkit, which wraps the same individual metrics Uni-VERSA is trained to predict. High correlations on those metrics are therefore partly a measure of how well the model reproduces the toolkit outputs, not independent evidence of multi-dimensional perceptual alignment. The only human-labeled component is the URGENT24 MOS subset (6,900 utterances of 114,000), and no human labels are provided for prosody, intelligibility, or speaker characteristics. The paper should explicitly separate the two claims: (a) Uni-VERSA efficiently predicts existing objective metrics, and (b) Uni-VERSA aligns with human perception. Claim (b) is currently anchored only to one small subset and to the unvalidated assumption that VERSA metrics are perceptual proxies. The authors should either evaluate against human labels in more domains or restrict the perceptual claim to naturalness/MOS.
  3. [Table 5] The out-of-domain results undermine the claim that the model 'maintains consistent quality estimation ... and aligns well with human preferences in TTS evaluation.' On TTS data, DNSMOS correlation drops to 0.53/0.49 LCC/SRCC and MOS to 0.61/0.60; on conversational data, DNSMOS is 0.45/0.45. These are low for a method claimed to align closely with human perception, and no comparison is given to specialized single-aspect predictors (e.g., a TTS-trained MOS model) on the same data. Without such a baseline or an error analysis, the out-of-domain claim is not supported. The authors should either provide stronger evidence of competitive out-of-domain performance or reframe the contribution as an in-domain multi-metric predictor with known degradation outside the enhancement domain.
minor comments (6)
  1. [Abstract] The phrase 'golden standard' should be 'gold standard'.
  2. [Eq. (2)] The notation 'n-norm' is ambiguous; the paper later sets n=1, so it is clearer to write 'the L1 loss' directly.
  3. [Sec. 2.4] The description of the semi-supervised strategy says that a 1-second zero-padded audio clip is used as a placeholder for missing reference audio, but it is not explained how this placeholder interacts with the cross-attention module or whether the model learns to ignore it. A short clarification would help reproducibility.
  4. [Tables 2-5] The tables report LCC/SRCC but not confidence intervals or the number of evaluation utterances per cell; given the large variation across metrics and domains, at least a mention of statistical significance or bootstrap intervals would strengthen the conclusions.
  5. [Sec. 3.2] The abbreviation 'MR-HuBERT' is used before it is defined in the ablation text; define it at first use, e.g., 'multi-resolution HuBERT'.
  6. [Table 1] The range column lists PESQ as [1, 4.5], while the text in Sec. 2.1 says PESQ focuses on quality comparisons without specifying a range; make these consistent.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: Uni-VERSA is a supervised regressor onto standard metrics; the only notable self-citation (VERSA) is a code toolkit, and human MOS from URGENT24 provides an external anchor.

full rationale

The derivation chain is a supervised multi-task regression, not a derivation that reduces to its inputs. The target quantities are the metrics in Table 1, and Eq. (2) is a standard L1 regression loss; evaluation in Sec. 2.3 compares held-out predictions to ground-truth labels, so there is no equation-level equivalence between the predictions and the inputs. The main self-citation is VERSA [10], used in Sec. 3.1 to annotate the benchmark: "we annotate the curated dataset using VERSA [10] with the metrics described in Section 2, adhering to their default configuration in VERSA." Because VERSA is an open-source wrapper around established metrics (PESQ, DNSMOS, STOI, WER, etc.) and because the paper separately incorporates human MOS from URGENT24, the benchmark is not purely self-referential; the human-MOS result provides independent content. The abstract claim that Uni-VERSA "outperforms methods focusing on a single aspect of human-perceived speech quality" is, however, in tension with Table 4, where the WavLM joint model reaches MOS LCC/SRCC 0.77/0.77 versus 0.80/0.80 for the single-task model; this is a correctness and significance concern, not a circularity. Eq. (2)'s unnormalized sum of L1 losses across metrics of very different scales may explain that degradation, but again it is a training-objective flaw rather than a reduction of the claimed result to the input. No circular step is identifiable at the level of the paper's equations or citations.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

This paper contains no derivation; it is an empirical machine-learning study. The central results depend on dataset construction and label generation rather than on explicit free parameters. All neural network weights are learned from data, so no hand-fitted constants are hidden. The main assumptions are about label validity and domain representativeness, listed above.

assumptions (4)
  • domain assumption The URGENT24 challenge test set is representative of speech enhancement outputs and suitable for training a general-purpose assessor.
    Sec. 3.1 constructs the benchmark from 1,000 blind test samples across 112 systems; the paper does not analyze representativeness for TTS or conversational domains.
  • domain assumption Labels produced by VERSA (PESQ, DNSMOS, WER from an ASR model, SBERT, SPK-SIM, etc.) are accurate ground truth for the respective metrics.
    Sec. 3.1 annotates the dataset using VERSA with default configuration; the paper treats these computed labels as supervision targets.
  • domain assumption Frozen WavLM features provide sufficient representations for all five speech assessment dimensions.
    Sec. 2.4 adopts frozen WavLM-large via layer-wise weighted summation; the paper tests HuBERT, MR-HuBERT, and Fbank, but does not justify transfer to prosody or speaker characteristics.
  • domain assumption The random 85:5:10 split on source speech samples prevents leakage between training and test.
    Sec. 3.1 states the split on source speech samples but does not report speaker or recording overlap.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Uni-VERSA: Versatile Speech Assessment with a Unified Network." pith.science (2026). https://pith.science/paper/65BCW2X7

@misc{pith2026250520741,
  author       = {Pith},
  title        = {Pith review of: Uni-VERSA: Versatile Speech Assessment with a Unified Network},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/65BCW2X7}},
  note         = {Machine review of arXiv:2505.20741}
}
read the original abstract

Subjective listening tests remain the golden standard for speech quality assessment, but are costly, variable, and difficult to scale. In contrast, existing objective metrics, such as PESQ, F0 correlation, and DNSMOS, typically capture only specific aspects of speech quality. To address these limitations, we introduce Uni-VERSA, a unified network that simultaneously predicts various objective metrics, encompassing naturalness, intelligibility, speaker characteristics, prosody, and noise, for a comprehensive evaluation of speech signals. We formalize its framework, evaluation protocol, and applications in speech enhancement, synthesis, and quality control. A benchmark based on the URGENT24 challenge, along with a baseline leveraging self-supervised representations, demonstrates that Uni-VERSA provides a viable alternative to single-aspect evaluation methods. Moreover, it aligns closely with human perception, making it a promising approach for future speech quality assessment.

Figures

Figures reproduced from arXiv: 2505.20741 by the authors.

Figure 1
Figure 1. The architecture of the Uni-VERSA base model with two metrics as an example. Detailed implementation is dis￾cussed in Sec. 2.4. 2.4. Baseline Model Base Model Architecture. Following the formulation in Eq. (1), we design the base model architecture (i.e., UniVERSA(·)) following the self-supervised learning (SSL)- based speech quality predictor [33]. The full-reference architec￾ture, shown in [PITH_FULL_IMAGE:figure… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Improving Speech Enhancement with Multi-Metric Supervision from Learned Quality Assessment

    cs.SD 2025-06 conditional novelty 6.0 of 10

    A multi-metric learned quality model (Uni-VERSA-Ext) is used as a differentiable training loss for speech enhancement, with a regularization term to prevent adversarial exploitation.

Reference graph

Works this paper leans on

43 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Introduction Speech profiling or speech quality assessment becomes a nec- essary function to evaluate various tasks, including speech syn- thesis, enhancement, separation, and coding [1–4]. The task is now getting more attention due to the growing demand for reli- able and scalable evaluation methods in real-world applications, driven by rapid advancement...

    work page

  2. [2]

    Reference Type

    Uni-VERSA 2.1. Metrics in Uni-VERSA Setup Previous studies in speech profiling have often focused on a single target domain, such as noise level, naturalness, or emo- tion [5, 23], largely due to the scarcity of comprehensive meta- data and labels. To overcome this limitation and achieve univer- sality in speech profiling, Uni-VERSA leverages a diverse se...

    arXiv 2025

  3. [3]

    URGENT24 Benchmark The Universality, Robustness, and Generalizability for EnhancemeNT (URGENT) 2024 challenge, hosted at NeurIPS2024, is a speech enhancement competition

    Experiments 3.1. URGENT24 Benchmark The Universality, Robustness, and Generalizability for EnhancemeNT (URGENT) 2024 challenge, hosted at NeurIPS2024, is a speech enhancement competition. We construct the Uni-VERSA benchmark by curating a new dataset from submissions to the URGENT24 challenge [22]. To mitigate over-tuning contamination, we exclusively use...

    work page 2024

  4. [4]

    Our experiments on the UR- GENT24 benchmark and out-of-domain scenarios demonstrate its effectiveness for various applications

    Conclusion In this work, we introduced Uni-VERSA for multi-dimensional speech quality analysis, detailing its formulation, evaluation criteria, and neural architecture. Our experiments on the UR- GENT24 benchmark and out-of-domain scenarios demonstrate its effectiveness for various applications. Future work will ex- tend Uni-VERSA to broader speech profil...

    work page

  5. [5]

    We would like to thank Daniel Leong and Megan Choo for their valuable com- ments

    Acknowledgments This work is supported by the Defence Science and Tech- nology Agency (DSTA) in Singapore. We would like to thank Daniel Leong and Megan Choo for their valuable com- ments. Experiments of this work used the Bridges2 at PSC and Delta/DeltaAI NCSA computing systems through alloca- tion CIS210014 from the Advanced Cyberinfrastructure Coor- di...

    work page

  6. [6]

    On some biases encoun- tered in modern audio quality listening tests-a review,

    S. Zielinski, F. Rumsey, and S. Bech, “On some biases encoun- tered in modern audio quality listening tests-a review,”Journal of the Audio Engineering Society, vol. 56, no. 6, pp. 427–451, 2008

    2008

  7. [7]

    The V oiceMOS challenge 2022,

    W. C. Huang, E. Cooper, Y . Tsao, H.-M. Wang, T. Toda, and J. Ya- magishi, “The V oiceMOS challenge 2022,” in Proc. ISCA Inter- speech, 2022, pp. 4536–4540

    work page 2022

  8. [8]

    Conferencingspeech 2022 challenge: Non-intrusive objective speech quality assessment (NISQA) challenge for on- line conferencing applications,

    G. Yi, W. Xiao, Y . Xiao, B. Naderi, S. M¨oller, W. Wardah, G. Mit- tag, R. Culter, Z. Zhang, D. S. Williamson, F. Chen, F. Yang, and S. Shang, “Conferencingspeech 2022 challenge: Non-intrusive objective speech quality assessment (NISQA) challenge for on- line conferencing applications,” in Proc. ISCA Interspeech, 2022, pp. 3308–3312

    work page 2022

Show all 43 references
  1. [9]

    ESPnet-Codec: Com- prehensive training and evaluation of neural codecs for audio, mu- sic, and speech,

    J. Shi, J. Tian, Y . Wu, J.-w. Jung, J. Q. Yip, Y . Masuyama, W. Chen, Y . Wu, Y . Tang, M. Baaliet al., “ESPnet-Codec: Com- prehensive training and evaluation of neural codecs for audio, mu- sic, and speech,” in Proc. IEEE SLT. IEEE, 2024, pp. 562–569

    2024

  2. [10]

    Objective measures of per- ceptual audio quality reviewed: An evaluation of their application domain dependence,

    M. Torcoli, T. Kastner, and J. Herre, “Objective measures of per- ceptual audio quality reviewed: An evaluation of their application domain dependence,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1530–1541, 2021

    2021

  3. [11]

    A review on subjective and objective evaluation of syn- thetic speech,

    E. Cooper, W.-C. Huang, Y . Tsao, H.-M. Wang, T. Toda, and J. Ya- magishi, “A review on subjective and objective evaluation of syn- thetic speech,” Acoustical Science and Technology , pp. e24–12, 2024

    2024

  4. [12]

    Per- ceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Per- ceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” in Proc. IEEE ICASSP, vol. 2. IEEE, 2001, pp. 749–752

    2001

  5. [13]

    Speech quality assessment,

    P. C. Loizou, “Speech quality assessment,” in Multimedia analy- sis, processing and communications . Springer, 2011, pp. 623– 654

    2011

  6. [14]

    Removing the bias in speech quality scores collected in noisy crowdsourcing environ- ments,

    R. Z. Jim ´enez, G. Mittag, and S. M ¨oller, “Removing the bias in speech quality scores collected in noisy crowdsourcing environ- ments,” in Proc. IEEE QoMEX. IEEE, 2021, pp. 49–54

    2021

  7. [15]

    Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods,

    B. Naderi, R. Zequeira Jim ´enez, M. Hirth, S. M ¨oller, F. Met- zger, and T. Hoßfeld, “Towards speech quality assessment using a crowdsourcing approach: evaluation of standardized methods,” Quality and User Experience, vol. 6, pp. 1–21, 2020

    2020

  8. [16]

    Ablation Studies

    We use the AdamW optimizer with a learning rate of 0.001 and a linear warm-up scheduler with 25,000 warm-up steps. Ablation Studies . In addition to baseline training, we con- duct several ablation experiments to evaluate the impact of each module and the usefulness of differe...

  9. [17]

    VERSA: A versatile evaluation toolkit for speech, audio, and music,

    J. Shi, H.-j. Shim, J. Tian, S. Arora, H. Wu, D. Petermann, J. Q. Yip, Y . Zhang, Y . Tang, W. Zhanget al., “VERSA: A versatile evaluation toolkit for speech, audio, and music,” arXiv preprint arXiv:2412.17667, 2024

    2024 arXiv

  10. [18]

    Mel-cepstral distance measure for objective speech quality assessment,

    R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” in Proc. IEEE Pacific Rim conference on Communications Computers and Signal Processing, vol. 1, 1993, pp. 125–128

    1993

  11. [19]

    A short- time objective intelligibility measure for time-frequency weighted noisy speech,

    C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “A short- time objective intelligibility measure for time-frequency weighted noisy speech,” in Proc. IEEE ICASSP. IEEE, 2010, pp. 4214– 4217

    2010

  12. [20]

    DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise sup- pressors,

    C. K. Reddy, V . Gopal, and R. Cutler, “DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise sup- pressors,” in Proc. IEEE ICASSP. IEEE, 2021, pp. 6493–6497

    2021

  13. [21]

    UTMOS: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: Utokyo-sarulab system for voicemos challenge 2022,” in Proc. ISCA Interspeech , 2022, pp. 4521– 4525

    2022

  14. [22]

    K ¨oster, Multidimensional analysis of conversational telephone speech

    F. K ¨oster, Multidimensional analysis of conversational telephone speech. Springer, 2018

    2018

  15. [23]

    Multi-dimensional speech quality assessment in crowdsourcing,

    B. Naderi, R. Cutler, and N.-C. Ristea, “Multi-dimensional speech quality assessment in crowdsourcing,” in Proc. IEEE ICASSP . IEEE, 2024, pp. 696–700

    2024

  16. [24]

    ESPnet: End-to-end speech processing toolkit,

    S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. Enrique Yalta Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai, “ESPnet: End-to-end speech processing toolkit,” in Proc. ISCA Interspeech, 2018, pp. 2207– 2211

    2018

  17. [25]

    Amphion: An open-source audio, music and speech generation toolkit,

    X. Zhang, L. Xue, Y . Gu, Y . Wang, J. Li, H. He, C. Wang, S. Liu, X. Chen, J. Zhanget al., “Amphion: An open-source audio, music and speech generation toolkit,” arXiv preprint arXiv:2312.09911, 2023

    2023 arXiv

  18. [26]

    AudioLDM: Text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “AudioLDM: Text-to-audio generation with latent diffusion models,” Proc. ICML, pp. 21 450–21 474, 2023

    2023

  19. [27]

    Adapting frechet audio distance for generative music evaluation,

    A. Gui, H. Gamper, S. Braun, and D. Emmanouilidou, “Adapting frechet audio distance for generative music evaluation,” in Proc. IEEE ICASSP. IEEE, 2024, pp. 1331–1335

    2024

  20. [28]

    UR- GENT challenge: Universality, robustness, and generalizability for speech enhancement,

    W. Zhang, R. Scheibler, K. Saijo, S. Cornell, C. Li, Z. Ni, J. Pirkl- bauer, M. Sach, S. Watanabe, T. Fingscheidt, and Y . Qian, “UR- GENT challenge: Universality, robustness, and generalizability for speech enhancement,” in Proc. ISCA Interspeech , 2024, pp. 4868–4872

    2024

  21. [29]

    Torchaudio-Squim: Reference-less speech qual- ity and intelligibility measures in TorchAudio,

    A. Kumar, K. Tan, Z. Ni, P. Manocha, X. Zhang, E. Hender- son, and B. Xu, “Torchaudio-Squim: Reference-less speech qual- ity and intelligibility measures in TorchAudio,” in Proc. IEEE ICASSP. IEEE, 2023, pp. 1–5

    2023

  22. [30]

    SDR– half-baked or well done?

    J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR– half-baked or well done?” in Proc. IEEE ICASSP. IEEE, 2019, pp. 626–630

    2019

  23. [31]

    TASnet: time-domain audio separation network for real-time, single-channel speech separation,

    Y . Luo and N. Mesgarani, “TASnet: time-domain audio separation network for real-time, single-channel speech separation,” inProc. IEEE ICASSP. IEEE, 2018, pp. 696–700

    2018

  24. [32]

    MOS-Bench: Bench- marking generalization abilities of subjective speech quality as- sessment models,

    W.-C. Huang, E. Cooper, and T. Toda, “MOS-Bench: Bench- marking generalization abilities of subjective speech quality as- sessment models,” arXiv preprint arXiv:2411.03715, 2024

    2024 arXiv

  25. [33]

    SpeechBERTScore: Reference-aware automatic evaluation of speech generation leveraging nlp evaluation metrics,

    T. Saeki, S. Maiti, S. Takamichi, S. Watanabe, and H. Saruwatari, “SpeechBERTScore: Reference-aware automatic evaluation of speech generation leveraging nlp evaluation metrics,” in Proc. ISCA Interspeech, 2024, pp. 4943–4947

    2024

  26. [34]

    Speaker recognition based on deep learning: An overview,

    Z. Bai and X.-L. Zhang, “Speaker recognition based on deep learning: An overview,” Neural Networks, vol. 140, pp. 65–99, 2021

    2021

  27. [35]

    NOMAD: Unsupervised learning of perceptual embeddings for speech enhancement and non-matching reference audio quality assessment,

    A. Ragano, J. Skoglund, and A. Hines, “NOMAD: Unsupervised learning of perceptual embeddings for speech enhancement and non-matching reference audio quality assessment,” in Proc. IEEE ICASSP. IEEE, 2024, pp. 1011–1015

    2024

  28. [36]

    SCOREQ: Speech quality assessment with contrastive re- gression,

    ——, “SCOREQ: Speech quality assessment with contrastive re- gression,” in Proc. ICLR, 2024

    2024

  29. [37]

    The V oiceMOS challenge 2023: zero-shot subjective speech quality prediction for multiple domains,

    E. Cooper, W.-C. Huang, Y . Tsao, H.-M. Wang, T. Toda, and J. Ya- magishi, “The V oiceMOS challenge 2023: zero-shot subjective speech quality prediction for multiple domains,” in Proc. IEEE ASRU. IEEE, 2023, pp. 1–7

    2023

  30. [38]

    The V oiceMOS challenge 2024: Beyond speech quality prediction,

    W.-C. Huang, S.-W. Fu, E. Cooper, R. E. Zezario, T. Toda, H.- M. Wang, J. Yamagishi, and Y . Tsao, “The V oiceMOS challenge 2024: Beyond speech quality prediction,” in Proc. IEEE SLT . IEEE, 2024, pp. 803–810

    2024

  31. [39]

    Generaliza- tion ability of MOS prediction networks,

    E. Cooper, W.-C. Huang, T. Toda, and J. Yamagishi, “Generaliza- tion ability of MOS prediction networks,” inProc. IEEE ICASSP. IEEE, 2022, pp. 8442–8446

    2022

  32. [40]

    WavLM: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “WavLM: Large-scale self- supervised pre-training for full stack speech processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  33. [41]

    SUPERB: Speech processing universal performance benchmark,

    S. wen Yang, P.-H. Chi, Y .-S. Chuang, C.-I. J. Lai, K. Lakhotia, Y . Y . Lin, A. T. Liu, J. Shiet al., “SUPERB: Speech processing universal performance benchmark,” in Proc. ISCA Interspeech , 2021, pp. 1194–1198

    2021

  34. [42]

    HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhut- dinov, and A. Mohamed, “HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language process- ing, vol. 29, pp. 3451–3460, 2021

    2021

  35. [43]

    Multi- resolution HuBERT: Multi-resolution speech self-supervised learning with masked unit prediction,

    J. Shi, H. Inaguma, X. Ma, I. Kulikov, and A. Sun, “Multi- resolution HuBERT: Multi-resolution speech self-supervised learning with masked unit prediction,” in Proc. ICLR, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.