Pith. sign in

REVIEW 3 major objections 4 minor 36 references

FUSE: Universal Speech Enhancement using Multi-Stage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fusing a discriminative separation network with a codec-token generative model yields balanced speech enhancement, ranking third in the URGENT 2025 challenge.

desk verdict A credible challenge-system paper whose fusion story is supported by its own ablations; two formulation gaps (44/48 kHz fusion input, shift trick as a no-op) need fixing before I'd trust the architecture as written. read the letter →

arxiv 2506.00809 v1 pith:KCQHEKNR submitted 2025-06-01 cs.SD eess.AS

Nabarun Goswami , Tatsuya Harada This is my paper
classification cs.SDeess.AS
keywords speechenhancementuniversalsourceseparationneuralaudiocodecmaskedlanguagemodelinggenerativemulti-stagefusionURGENT2025
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a three-stage speech enhancement pipeline that combines a discriminative separation network, a generative model operating on neural codec tokens, and a learned fusion network. The authors set out to show that the discriminative stage supplies signal fidelity, the generative stage improves perceptual quality, and the final fusion stage combines both so that the whole system scores well on both kinds of metrics. The complete FUSE system ranked third in the URGENT 2025 speech enhancement challenge and, among the top three systems, recorded the highest DNSMOS, NISQA, UTMOS, and subjective MOS scores. If the claim holds, it demonstrates that token-based generative refinement can be grafted onto a separation model to improve listening quality without a large cost in objective signal fidelity.

What carries the argument

Three trained components are chained: Stage 1 is a Sparse Compression Network with conformer layers that estimates clean speech from the noisy input; Stage 2 is an Injection-Conformer generative model that predicts masked acoustic tokens of a DAC neural audio codec, conditioned on XEUS self-supervised features plus codec features of the noisy input and Stage 1 output, then decodes tokens to a waveform; Stage 3 is a fusion network that takes the channel-wise concatenation $[x; \hat{s}_1; \hat{s}_2]$ as input. The fusion network is the load-bearing object: it uses the same architecture as Stage 1 but is trained with additional speaker, phoneme, and UTMOS losses while Stages 1 and 2 stay frozen. Two inference-time mechanisms carry additional weight: a shift trick that averages predictions over multiple time offsets, and network blending that averages selected stage outputs, with the Stage 2 plus Stage 3 blend giving the best overall results.

What would settle it

Run the released system on the URGENT 2025 non-blind test set and compare Stage 3 against Stage 1 and Stage 2 on both an intrusive metric (SDR or PESQ) and a non-intrusive metric (NISQA or DNSMOS); the central fusion claim fails if Stage 3 does not beat the better of its two inputs on both kinds of metrics on average. A second check is pipeline inspection: if no 44-to-48 kHz resampling and time alignment of $\hat{s}_2$ exists before the concatenation in Eq. (6), the fusion input is incoherent and the reported improvement cannot be attributed to the described mechanism.

Watch

Extended reading notes

Core claim

The central discovery, stated in Section 3.5, is that the three-stage fusion achieves a well-balanced performance across all metrics by combining the high SDR of Stage 1 with the perceptual improvements of Stage 2. Stage 1 alone gives the best signal-level reconstruction (SDR 12.62 on the non-blind test set), Stage 2 alone raises perceptual metrics such as UTMOS, DNSMOS, and NISQA but drops SDR to 9.03, and Stage 3, trained on a loss that includes mel-spectrogram, SI-SDR, speaker similarity, phoneme feature matching, and UTMOS, lands at SDR 12.53 with the best non-intrusive scores of the three stages. On the blind test set, the full system with shift trick and blending achieves the best DNSMOS (2.94), NISQA (3.25), UTMOS (2.19), and subjective MOS (3.44) among the three leading challenge systems, while staying competitive on signal-level metrics.

Load-bearing premise

Stage 2 outputs a 44 kHz waveform through the DAC codec while Stages 1 and 3 are trained at 48 kHz, and the paper never states how $\hat{s}_2$ is resampled or time-aligned before it is concatenated into the fusion input, so all reported Stage 3 gains rest on that unstated alignment step being done correctly.

Editorial extensions

If this is right

  • A hybrid discriminative-generative pipeline can outperform a pure discriminative system on perceptual metrics while retaining most of its SDR: Stage 3 keeps SDR near Stage 1 while raising NISQA and UTMOS well above it.
  • Network blending is a cheap, effective post-hoc knob: averaging Stage 2 and Stage 3 outputs gives the best balance across most metrics in Table 1, and it requires no retraining.
  • The shift trick is worth roughly 2 dB SDR for the generative Stage 2 (9.03 to 11.21 with 10 shifts) and also lifts NISQA, so time-shift ensembling is a practical inference-time gain.
  • Adding a UTMOS-based loss term in Stage 3 specifically boosts UTMOS, showing that perceptual metric losses can be injected into a fusion network without destroying signal-level metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fusion recipe is largely codec-agnostic; substituting a different residual vector-quantization codec or a different self-supervised feature extractor would test whether the balanced gain comes from the fusion architecture itself or from the specific DAC/XEUS combination.
  • Because Stage 2 operates at 44 kHz while Stages 1 and 3 are 48 kHz, the paper leaves the resampling and time-alignment of $\hat{s}_2$ implicit; a reproducible implementation must fix this step explicitly.
  • The same multi-stage pattern could apply to other tasks where discriminative and generative models have complementary failure modes, such as dereverberation or bandwidth extension, with fusion arbitrating between a faithful but dull estimate and a pleasant but distorted one.
  • The 10-shift averaging implies a tunable compute-quality trade-off at inference, which matters because the paper notes the sequential pipeline is not yet real-time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FUSE, a three-stage universal speech enhancement system for the URGENT 2025 challenge. Stage 1 uses a Sparse Compression Network with Conformer layers for discriminative source separation. Stage 2 adapts an Injection-Conformer generative model, conditioned on SSL features, to perform masked language modeling over DAC codec tokens. Stage 3 is a learned fusion network that concatenates the noisy input with the outputs of Stages 1 and 2. A time-shift aggregation and network blending are also applied. Experiments on the challenge's non-blind and blind test sets report that the fused system achieves a balance between intrusive metrics (SDR, PESQ) and perceptual metrics (NISQA, UTMOS, DNSMOS), and the system obtained third rank in the challenge.

Significance. If the reported results are reproducible, the paper makes a useful contribution by demonstrating that a hybrid discriminative/generative pipeline with a learned fusion stage can be competitive in a multi-metric, multilingual, variable-sample-rate enhancement benchmark. The system attains the highest DNSMOS, NISQA, UTMOS, and subjective MOS among the top three challenge submissions, which is a meaningful empirical result. The manuscript also contains several positive features: it specifies the training losses, model sizes, and training schedule; it uses publicly available components (SCNET, XEUS, DAC); and it reports stage-wise ablation in Table 1 that supports the central claim that Stage 3 balances the strengths of Stages 1 and 2 on metrics not used as training losses (e.g., NISQA, PESQ, SDR). These strengths make the paper a credible challenge report, provided that the missing implementation details are clarified.

major comments (3)
  1. [§3.1 and §2.3, Eq. (6)] The sample-rate mismatch between Stage 2 and Stages 1/3 is not addressed. Section 3.1 states that Stages 1 and 3 are trained at 48 kHz and Stage 2 uses the DAC-44kHz codec, whose decoder outputs waveforms at 44 kHz. Equation (6) concatenates x, s1, and s2 along the channel dimension, which requires all three inputs to have the same length and sample rate. The paper never specifies how s2 is resampled to 48 kHz or how time alignment is ensured before concatenation. Since Stage 3 is exactly the component that produces the balanced results in Table 1, the reported Stage 3 numbers cannot be independently reproduced without this missing step. This is a load-bearing gap in the description.
  2. [§2.4, Eqs. (8)–(9)] The shift trick is described in a way that cannot produce the reported gains. Equation (8) defines the shifted output as s_i(t+τ), i.e., a time shift of the stage output, and Eq. (9) then inverse-shifts and averages. As written, shifting the output and then inverse-shifting it is the identity, so the reported improvements from '10 shifts' (e.g., SDR boosting from 9.03 to 11.21 for Stage 2 in Table 1) could not arise from the stated operation. The intended trick is presumably to shift the input signal to the stage, apply the model, and then inverse-shift the resulting output; the equations must be redefined accordingly, or the claimed gains will appear unsupported.
  3. [§3.5 and §4] The evaluation is partially circular for metrics that coincide with training losses. Stage 3 is trained with L_spk, L_phoneme, and L_UTMOS in Eq. (7), and Table 1 reports SpkSim, LPS, and UTMOS as evaluation metrics. The claim that Stage 3 'excels' in UTMOS is therefore expected by construction, and the LPS/CER results are influenced by the phoneme feature matching loss. The central balanced-performance claim is nonetheless independently supported by NISQA, DNSMOS, PESQ, and SDR, which are not training objectives. The paper should explicitly acknowledge this circularity and separate the two sets of metrics when interpreting the stage-wise comparison.
minor comments (4)
  1. [§3.1 and references] Reference [14] is titled 'EDM-TTS: Efficient Dual-Stage Masked Modeling for Alignment-Free Text-to-Speech Synthesis,' but the text refers to it as 'Injection-Conformer.' The paper should either use the actual model name or cite the original Injection-Conformer source if [14] is not the primary reference for that architecture.
  2. [§3.4] There is a typo in 'Speaker Encoder Cosine Similarty' — it should read 'Similarity.'
  3. [Table 2 caption] The superscripts † and * are used in the table caption but the corresponding methods are not marked in the table rows; please mark System A and System B accordingly or remove the notation.
  4. [§2.1] The paper says SCNET 'partitions the input spectrogram into subbands' but does not specify the STFT parameters for Stage 1; those are given only in Section 3.1. Consider moving the STFT configuration forward to Section 2.1 for clarity.

Circularity Check

2 steps flagged · score 4.0 of 10

Stage 3 is trained against UTMOS, speaker-cosine, and phoneme losses that are then reported as evaluation metrics; the central fusion claim retains independent support from NISQA, DNSMOS, PESQ, and SDR.

  1. fitted input called prediction [Section 2.3, Eq. (7); Section 3.4; Section 3.5, Table 1]
    "The overall training loss for Stage 3 is the sum of the primary reconstruction losses (Eq. 2) and additional task-specific losses. ... L3 = L mel + L SI-SDR + L spk + L phoneme + L UTMOS.(7) ... with the exception of UTMOS where Stage 3 excels due to its dedicated L UTMOS."

    Stage 3 is explicitly optimized to minimize 5-UTMOS through Eq. (7), and then Table 1 and Section 3.5 report UTMOS as evidence that Stage 3 achieves perceptual improvement over Stages 1 and 2. The UTMOS advantage of Stage 3 (2.64 vs. 2.38 for Stage 2 and 2.18 for Stage 1) is therefore partly produced by the training objective rather than independently measured. The authors' own sentence admits this reduction, confirming that this particular metric is not an external check on the fusion claim.

  2. fitted input called prediction [Section 2.3, Eq. (7); Section 3.4; Table 1 and Table 2]
    "In addition, we incorporate the speaker encoder cosine similarity loss, the phoneme encoder feature matching loss [21], and the UTMOS loss [22] (defined as 5-UTMOS(·)). ... downstream-task-dependent measures: Speaker Encoder Cosine Similarty (SpkSim) and Character Error Rate (CER)"

    The Stage 3 loss L_spk is a speaker-encoder cosine-similarity loss, and the reported SpkSim metric is a speaker-encoder cosine similarity, so the SpkSim column in Table 1 measures the same objective that Stage 3 is trained to optimize. Likewise L_phoneme is a feature-matching loss on a phoneme recognizer, closely aligned with the reported LPS and CER phoneme metrics. Reporting these as independent downstream evaluations double-counts the training objective, although the effect is partial because Stage 3's main SDR and PESQ/NISQA/DNSMOS gains are not direct training targets.

full rationale

The central derivation chain is not circular in the strong sense: Stage 3's balancing claim is supported by metrics that do not appear in L3 (NISQA, DNSMOS, PESQ, SDR), and the Stage 1/Stage 2 comparison is an architectural pipeline, not a definitional identity. However, the evaluation is partially circular because Eq. (7) includes L_UTMOS, L_spk, and L_phoneme as training objectives, and the same paper then reports UTMOS, SpkSim, and LPS/CER as evaluation evidence, with the paper itself noting that Stage 3's UTMOS edge comes 'due to its dedicated L_UTMOS'. This is a fitted-input-called-prediction pattern for those metrics. The self-citation to [14] for the Injection-Conformer architecture is not load-bearing circularity: it is a component choice from the authors' own prior published TTS work, reproduced here, and does not by itself force the enhancement results. The unsampled 44 kHz vs 48 kHz mismatch in Eq. (6) and the identity form of Eq. (8) are reproducibility/correctness problems, not circularity, so they do not raise the circularity score. Overall the paper has partial evaluation circularity but an independent core result, giving a score of 4.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a stack of pretrained models (XEUS, DAC, ECAPA2, wav2vec2, Injection-Conformer) and several hand-chosen training and inference hyperparameters. No new physical or conceptual entities are introduced.

free parameters (4)
  • Loss weighting in L3 = equal/unstated
    Eq. (7) sums five losses without coefficients; no weighting schedule is reported, so the balance between fidelity and perceptual losses is effectively a free choice.
  • MLM masking ratio = not stated
    Stage 2 training uses masked language modeling; the masking ratio determines training behavior and is not reported, making the training procedure underspecified.
  • Shift set size |T| = 10
    The shift trick uses 10 shifts in experiments; no ablation over the count is given and it acts as a test-time averaging parameter.
  • Sliding window overlap = 50%
    Inference uses 50% window overlap; no ablation or justification is provided, and it affects how boundary artifacts are handled.
assumptions (4)
  • domain assumption XEUS SSL features provide a sufficient conditioning signal for the generative token model.
    Stage 2 relies on phi(x) from XEUS to guide codec token prediction; no comparison to alternative SSL features is provided, so the choice is assumed beneficial.
  • domain assumption DAC codec tokens are an adequate discrete acoustic representation for enhancement.
    The generative stage predicts and decodes DAC tokens; if the codec lacks detail, the enhanced output is bounded by codec quality.
  • domain assumption The Injection-Conformer architecture from the authors' prior TTS work transfers to enhancement with the described conditioning.
    The model is 'reproduced based on [14]' with no enhancement-specific architectural validation or ablations, so transfer is assumed.
  • ad hoc to paper Shift-and-average and network blending improve or preserve all relevant metrics.
    These test-time heuristics are applied globally; the paper reports their effect only for a fixed 10-shift set, so boundary artifacts and shift values are unspecified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FUSE: Universal Speech Enhancement using Multi-Stage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge." pith.science (2026). https://pith.science/paper/KCQHEKNR

@misc{pith2026250600809,
  author       = {Pith},
  title        = {Pith review of: FUSE: Universal Speech Enhancement using Multi-Stage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KCQHEKNR}},
  note         = {Machine review of arXiv:2506.00809}
}
read the original abstract

We propose a multi-stage framework for universal speech enhancement, designed for the Interspeech 2025 URGENT Challenge. Our system first employs a Sparse Compression Network to robustly separate sources and extract an initial clean speech estimate from noisy inputs. This is followed by an efficient generative model that refines speech quality by leveraging self-supervised features and optimizing a masked language modeling objective on acoustic tokens derived from a neural audio codec. In the final stage, a fusion network integrates the outputs of the first two stages with the original noisy signal, achieving a balanced improvement in both signal fidelity and perceptual quality. Additionally, a shift trick that aggregates multiple time-shifted predictions, along with output blending, further boosts performance. Experimental results on challenging multilingual datasets with variable sampling rates and diverse distortion types validate the effectiveness of our approach.

Figures

Figures reproduced from arXiv: 2506.00809 by the authors.

Figure 1
Figure 1. Model architectures and training and inference pipeline illustration. estimate sˆ1 ∈ R T from x via the transformation: sˆ1 = fθ1 (x), (1) where fθ1 (·) is parameterized by θ1. SCNET partitions the in￾put spectrogram into subbands and employs a sparsity-based en￾coder with adaptive compression to enhance separation perfor￾mance while reducing computational cost. It employs a dual￾path BLSTM stack as the backbone. To… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 22 canonical work pages

  1. [1]

    Introduction Speech enhancement is a fundamental task in audio process- ing with applications ranging from telecommunications to as- sistive technologies [1]. The increasing demand for systems that perform robustly in real-world conditions has driven re- search into models that generalize across multiple distortion types, languages, and sampling rates. In...

    work page 2025

  2. [2]

    FUSE: Universal Speech Enhancement using Multi-Stage Fusion of Sparse Compression and Token Generation Models for the URGENT 2025 Challenge

    Method In this section, we detail our multi-stage speech enhancement framework that progressively refines noisy speech through three key stages: a Sparse Compression Network, a Token Sampling Generative Model, and a Fusion Network. The following sub- sections describe each component of our method in detail and Fig. 1 provides a visual overview of the trai...

    work page Pith review arXiv 2025

  3. [3]

    Experiments 3.1. Model architecture For Stage 1, we adopt the same configuration as the SCNET large2, with the BLSTM modules replaced by conformer lay- ers with 8 attention heads, a convolution kernel size of 3, and a feedforward multiplier of 2. For stage 2, we use the XEUS[15] SSL model 3[15] and the DAC[11] 44Khz codec 4. The Injection-Conformer (repro...

    work page 2025

  4. [4]

    Conclusion and Limitations In this paper, we introduced a multi-stage fusion framework for universal speech enhancement, tailored for Track 1 of the Inter- speech 2025 URGENT Challenge. Our approach integrates a sparse compression network with dual-path Conformer blocks, a token sampling generative model based on the Injection Con- former architecture, an...

    work page 2025

  5. [5]

    Acknowledgements This work was partially supported by JST AIP Acceleration Research JPMJCR20U3, Moonshot R&D Grant Number JP- MJPS2011, JSPS KAKENHI Grant Number JP19H01115, and JP20H05556 and Basic Research Grant (Super AI) of Institute for AI and Beyond of the University of Tokyo

    work page

  6. [6]

    P. C. Loizou,Speech Enhancement: Theory and Practice, 2nd ed. USA: CRC Press, Inc., 2013

    work page 2013

  7. [7]

    Interspeech 2025 URGENT speech enhancement challenge,

    K. Saijo, W. Zhang, S. Cornell, R. Scheibler, C. Li, Z. Ni, A. Ku- mar, M. Sach, Y . Fu, W. Wang, T. Fingscheidt, and S. Watanabe, “Interspeech 2025 URGENT speech enhancement challenge,”Ac- cepted by Interspeech, 2025

    work page 2025

  8. [8]

    Music source separation in the waveform domain,

    A. Defossez, N. Usunier, L. Bottou, and F. Bach, “Music source separation in the waveform domain,” 2020. [Online]. Available: https://openreview.net/forum?id=HJx7uJStPH

    work page 2020

Show all 36 references
  1. [9]

    Scnet: Sparse compression network for music source separation,

    W. Tong, J. Zhu, J. Chen, S. Kang, T. Jiang, Y . Li, Z. Wu, and H. Meng, “Scnet: Sparse compression network for music source separation,” inICASSP 2024-2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1276–1280

    2024

  2. [10]

    Tf-gridnet: Integrating full-and sub-band modeling for speech separation,

    Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee, B.-Y . Kim, and S. Watan- abe, “Tf-gridnet: Integrating full-and sub-band modeling for speech separation,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023

    2023

  3. [11]

    Speech enhancement and dereverberation with diffusion-based generative models,

    J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based generative models,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 2351–2364, 2023

    2023

  4. [12]

    Storm: A diffusion-based stochastic regeneration model for speech en- hancement and dereverberation,

    J.-M. Lemercier, J. Richter, S. Welker, and T. Gerkmann, “Storm: A diffusion-based stochastic regeneration model for speech en- hancement and dereverberation,”IEEE/ACM Transactions on Au- dio, Speech, and Language Processing, 2023

    2023

  5. [13]

    Hifi-gan-2: Studio-quality speech enhancement via generative adversarial networks condi- tioned on acoustic features,

    J. Su, Z. Jin, and A. Finkelstein, “Hifi-gan-2: Studio-quality speech enhancement via generative adversarial networks condi- tioned on acoustic features,” in2021 IEEE Workshop on Appli- cations of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2021, pp. 166–170

    2021

  6. [14]

    Finally: fast and universal speech enhancement with studio-like quality,

    N. Babaev, K. Tamogashev, A. Saginbaev, I. Shchekotov, H. Bae, H. Sung, W. Lee, H.-Y . Cho, and P. Andreev, “Finally: fast and universal speech enhancement with studio-like quality,”Advances in Neural Information Processing Systems, vol. 37, pp. 934–965, 2025

    2025

  7. [15]

    Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,

    H. Yang, J. Su, M. Kim, and Z. Jin, “Genhancer: High-fidelity speech enhancement via generative modeling on discrete codec tokens,” inInterspeech 2024, 2024, pp. 1170–1174

    2024

  8. [16]

    High-fidelity audio compression with improved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with improved rvqgan,”Ad- vances in Neural Information Processing Systems, vol. 36, 2024

    2024

  9. [17]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Transactions on Machine Learning Research, 2023, featured Certification, Reproducibility Certification. [Online]. Available: https://openreview.net/forum? id=ivCd8z8zR2

    2023

  10. [18]

    Soundstorm: Efficient parallel audio gen- eration,

    Z. Borsos, M. Sharifi, D. Vincent, E. Kharitonov, N. Zeghidour, and M. Tagliasacchi, “Soundstorm: Efficient parallel audio gen- eration,”arXiv preprint arXiv:2305.09636, 2023

    2023 arXiv

  11. [19]

    EDM-TTS: Efficient Dual-Stage Masked Modeling for Alignment-Free Text-to- Speech Synthesis,

    N. Goswami, H. Wang, and T. Harada, “EDM-TTS: Efficient Dual-Stage Masked Modeling for Alignment-Free Text-to- Speech Synthesis,”Transactions on Machine Learning Research (TMLR), 2025. [Online]. Available: https://openreview.net/ forum?id=c7vkDg558Z

    2025

  12. [20]

    Towards robust speech representation learning for thousands of languages,

    W. Chen, W. Zhang, Y . Peng, X. Li, J. Tian, J. Shi, X. Chang, S. Maiti, K. Livescu, and S. Watanabe, “Towards robust speech representation learning for thousands of languages,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y . Al-O...

    2024

  13. [21]

    Wavlm: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self- supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  14. [22]

    Tokensplit: Using dis- crete speech representations for direct, refined, and transcript- conditioned speech separation and recognition,

    H. Erdogan, S. Wisdom, X. Chang, Z. Borsos, M. Tagliasac- chi, N. Zeghidour, and J. R. Hershey, “Tokensplit: Using dis- crete speech representations for direct, refined, and transcript- conditioned speech separation and recognition,” inInterspeech 2023, 2023, pp. 3462–3466

    2023

  15. [23]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” in Interspeech 2020, 2020, pp. 5036–5040

    2020

  16. [24]

    Improving music source separation based on deep neural networks through data augmentation and network blending,

    S. Uhlich, M. Porcu, F. Giron, M. Enenkl, T. Kemp, N. Takahashi, and Y . Mitsufuji, “Improving music source separation based on deep neural networks through data augmentation and network blending,” in2017 IEEE International Conference on Acoustics, Speech and Signal Processing...

    2017

  17. [25]

    Sdr– half-baked or well done?

    J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “Sdr– half-baked or well done?” inICASSP 2019-2019 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 626–630

    2019

  18. [26]

    Simple and effective zero-shot cross-lingual phoneme recognition,

    Q. Xu, A. Baevski, and M. Auli, “Simple and effective zero-shot cross-lingual phoneme recognition,”arXiv preprint arXiv:2109.11680, 2021

    2021 arXiv

  19. [27]

    Utmos: Utokyo-sarulab system for voicemos challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “Utmos: Utokyo-sarulab system for voicemos challenge 2022,” inInterspeech 2022, 2022, pp. 4521–4525

    2022

  20. [28]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y . Bengio and Y . LeCun, Eds., 2015. [Online]. Available: http://arxiv...

    2015 arXiv

  21. [29]

    Simulating wind noise with airflow speed-dependent characteristics,

    D. Mirabilii, A. Lodermeyer, F. Czwielong, S. Becker, and E. A. Habets, “Simulating wind noise with airflow speed-dependent characteristics,” in2022 International Workshop on Acoustic Sig- nal Enhancement (IWAENC), 2022, pp. 1–5

    2022

  22. [30]

    Perceptual objective listening quality assessment (polqa), the third generation itu-t standard for end-to-end speech quality measurement part i—temporal align- ment,

    J. G. Beerends, C. Schmidmer, J. Berger, M. Obermann, R. Ull- mann, J. Pomy, and M. Keyhl, “Perceptual objective listening quality assessment (polqa), the third generation itu-t standard for end-to-end speech quality measurement part i—temporal align- ment,”journal of the audi...

    2013

  23. [31]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” in 2001 IEEE international conference on acoustics, speech, and signal processing. Proceed...

    2001

  24. [32]

    An algorithm for predicting the in- telligibility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the in- telligibility of speech masked by modulated noise maskers,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 24, no. 11, pp. 2009–2022, 2016

    2009

  25. [33]

    Mel-cepstral distance measure for objective speech quality assessment,

    R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” inProceedings of IEEE Pacific Rim Con- ference on Communications Computers and Signal Processing, vol. 1, 1993, pp. 125–128 vol.1

    1993

  26. [34]

    Distance measures for speech process- ing,

    A. Gray and J. Markel, “Distance measures for speech process- ing,”IEEE Transactions on Acoustics, Speech, and Signal Pro- cessing, vol. 24, no. 5, pp. 380–391, 1976

    1976

  27. [35]

    Speechbertscore: Reference-aware automatic evaluation of speech generation leveraging nlp evaluation metrics,

    T. Saeki, S. Maiti, S. Takamichi, S. Watanabe, and H. Saruwatari, “Speechbertscore: Reference-aware automatic evaluation of speech generation leveraging nlp evaluation metrics,”arXiv preprint arXiv:2401.16812, 2024

    2024 arXiv

  28. [36]

    Evaluation metrics for generative speech en- hancement methods: Issues and perspectives,

    J. Pirklbauer, M. Sach, K. Fluyt, W. Tirry, W. Wardah, S. Moeller, and T. Fingscheidt, “Evaluation metrics for generative speech en- hancement methods: Issues and perspectives,” inSpeech Commu- nication; 15th ITG Conference, 2023, pp. 265–269

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.