Pith. sign in

REVIEW 3 major objections 6 minor 83 references

A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A composite predictive-generative model, PGUSE, outperforms both predictive and diffusion baselines on universal speech enhancement while using far less compute.

desk verdict Solid composite predictive-generative SE paper with real efficiency gains; the only significant caveat is ambiguous hyperparameter selection on WSJ0-UNI. read the letter →

arxiv 2505.24576 v1 pith:2AMFALC4 submitted 2025-05-30 eess.AS

Jie Zhang , Haoyin Yan , Xiaofei Li This is my paper
classification eess.AS
keywords universalspeechenhancementscore-baseddiffusionpredictiveandgenerativemodelingoutputfusiontruncateddenoisingdereverberationsuper-resolution
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that predictive and generative speech enhancement are complementary and should be combined rather than pitted against each other. It introduces PGUSE, a single universal speech enhancement model with two parallel branches: one directly predicts the clean complex spectrum, and the other runs a score-based diffusion process on the magnitude spectrum to generate plausible speech. A weighted spectral fusion merges the two outputs, and a truncated diffusion scheme starts the reverse process from the predictive estimate, cutting sampling from 25 reverse steps to 3. On the WSJ0-UNI benchmark and other distortion datasets, PGUSE reports the best quality scores while using roughly 26.3 billion multiply-accumulate operations, two to three orders of magnitude less than diffusion baselines. The significance is a practical efficiency-quality frontier: generative enhancement need not be prohibitively slow.

What carries the argument

The load-bearing mechanism is the pairing of output fusion with truncated diffusion inside a two-branch encoder-decoder. Output fusion weights the predictive and generative magnitude estimates by \(\$\alpha$=0.4\) in the spectral domain and takes the phase entirely from the predictive branch, which compensates for the magnitude-only diffusion. Truncated diffusion replaces the usual Gaussian start of the reverse process with the predictive magnitude estimate, then solves the reverse BBED SDE for only three Euler-Maruyama steps of width 0.04; this is what converts a diffusion model that normally requires dozens of score-network evaluations into a single predictive pass plus three light refinement steps. Supporting components are the sub-band downsampling/upsampling blocks, the dual-path recurrent attention bottleneck, and interaction modules that gate predictive features into the score estimator, but the efficiency claim rests specifically on the truncated start and the fusion weight.

What would settle it

Take the trained PGUSE and run it on a set of distortion types and SNRs not in WSJ0-UNI, e.g., codec artifacts at low bitrates, overlapping speakers, or RIRs with long T60, comparing the 3-step truncated inference against the 25-step full reverse process and against the predictive branch alone. If the 3-step version's PESQ or listening quality drops substantially relative to the full process, the truncated-output assumption fails; if it stays close, the claim transfers.

Watch

Extended reading notes

Core claim

PGUSE's central claim is that a deterministic predictive branch and a stochastic generative branch can be jointly trained and integrated so that each covers the other's weaknesses: the predictive branch supplies accurate phase and suppresses over-smoothing artifacts, while the generative branch restores plausible spectral detail in severely degraded conditions. The generative branch performs denoising score matching over a magnitude STFT spectrum using the BBED stochastic differential equation, and an interaction module feeds predictive features into the score estimator. At inference, the reverse diffusion starts at a short time \(T_{rs}=0.12\) from a state approximated by the predictive magnitude estimate, runs three Euler-Maruyama steps, and the result is fused with the predictive magnitude before phase from the predictive branch is applied. The paper reports PESQ 3.53 on WSJ0-UNI, above the best predictive baseline CMGAN (3.43) and the best generative baseline UNIVERSE++ (3.20), with 26.3 G MACs compared with 8.0–15.8 T for diffusion baselines, and also reports leading scores on denoising, dereverberation, and super-resolution evaluations.

Load-bearing premise

The efficiency gain rests on the assumption that starting the reverse diffusion at \(T_{rs}=0.12\) and taking only three coarse Euler-Maruyama steps still produces properly distributed enhanced speech across every distortion type, which was validated only on the same WSJ0-UNI test set used to tune those hyperparameters.

Editorial extensions

If this is right

  • A single PGUSE model can handle additive noise, reverberation, bandwidth limitation, clipping, and other distortions from the WSJ0-UNI pipeline without task-specific retraining.
  • Diffusion-based enhancement can be made practical: 3 reverse steps instead of 25 keep quality on par, cutting inference cost to 26.3 G MACs.
  • Predictive and generative branches complement each other: fusion improves over either branch alone, and the interaction module's gradient flow trains the predictive branch to help score estimation.
  • The model generalizes across datasets: it transfers from WSJ0-UNI training to VBDMD denoising, VBDMD-REVERB dereverberation, and VBDMD-SR super-resolution with leading scores.
  • Combined with ASR, PGUSE lowers word error rate on TIMIT-UNI compared with generative baselines, making it usable as a front-end for speech recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the truncated start is as safe as the WSJ0-UNI results suggest, the same device could be applied to any conditional diffusion task where a cheap point estimate of the target exists, not just speech enhancement.
  • Beyond the paper: the magnitude-domain diffusion with predictive phase implies that phase recovery, not spectral detail, may be the main bottleneck of the fused output; testing on phase-sensitive metrics would clarify how much headroom remains.
  • Beyond the paper: a streaming variant would need to bound the receptive field of the dual-path recurrent attention bottleneck; the current 32 ms window and 12 ms shift suggest a path but the paper does not report real-time factor on a device.
  • Beyond the paper: the hyperparameter choice \(T_{rs}=0.12\) and \(N=3\) was tuned on WSJ0-UNI; an independent evaluation on held-out distortion distributions would reveal whether the 3-step approximation holds generally.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes PGUSE, a two-branch universal speech enhancement model. The predictive branch performs complex spectral mapping, while the generative branch performs score-based diffusion in the magnitude STFT domain, with an interaction module connecting the two branches. At inference the outputs are fused in the spectral domain (Eq. 17) and the reverse diffusion is truncated, starting from an initialization provided by the predictive branch (Eq. 20); this reduces the reverse process from 25 full-range steps to three steps for the reported configuration. Experiments cover WSJ0-UNI, VBDMD, VBDMD-REVERB, VBDMD-SR, and TIMIT-UNI, with PESQ, ESTOI, composite MOS, WV-MOS, ViSQOL, LSD, SSIM, and WER. On WSJ0-UNI the paper reports PESQ 3.53 at 26.3 GMACs, compared with 3.43 for CMGAN and 3.20 for UNIVERSE++, and it also reports leading cross-dataset results for dereverberation and super-resolution.

Significance. If the headline result holds, PGUSE establishes a practical efficiency-quality frontier for diffusion-based universal speech enhancement: it substantially reduces the inference cost of score-based models while improving objective quality over strong predictive baselines. The paper has concrete strengths: all main baselines are re-trained, the evaluation uses multiple datasets and metrics, code and audio examples are promised, and the conclusion acknowledges remaining artifacts. The cross-dataset experiments with a fixed configuration provide external support for the approach. The main reservation is that the hyperparameters alpha, N, and Trs are selected on the same WSJ0-UNI benchmark used for the headline comparison, so the exact size of the reported margin needs to be confirmed with a properly held-out tuning protocol.

major comments (3)
  1. [Section IV-D and Table II] The hyperparameter search for alpha, N, and Trs is described as conducted on the WSJ0-UNI dataset, but the paper does not state whether this search used the si_dt_05 validation split or the si_et_05 test split. Because Table II reports results on the test split, if the same test utterances informed the selection of alpha=0.4 and Trs=0.12 with Delta_t=0.04, the 0.10 PESQ margin over CMGAN (3.53 vs 3.43) is partly a selection artifact; the per-utterance standard deviations in Table II are near 0.87, so this margin is not large relative to utterance-level variability. Please either confirm that all hyperparameters were chosen on the validation split or retune on the validation split and report the resulting fixed configuration on the test split. The cross-dataset results in Tables III-V do not remove this concern for the WSJ0-UNI benchmark itself.
  2. [Section III-D and Algorithm 2] The efficiency claim rests on the assumption that a reverse process truncated to Trs=0.12 and discretized with only three Euler-Maruyama steps still produces a useful sample from the conditional magnitude distribution. This is an approximation, and the only evidence is Fig. 4c, which is measured on the same WSJ0-UNI set used to set the hyperparameters. Please add a validation-split version of Fig. 4c and, ideally, report the WSJ0-UNI and cross-dataset metrics for at least one neighboring operating point (e.g., Trs=0.16 or Delta_t=0.08) to demonstrate that the reported operating point is not a sharp peak that would make the complexity savings brittle.
  3. [Tables II and III] No statistical significance or confidence intervals are reported for the metric differences. Given that the headline PESQ margin over CMGAN is 0.10 with per-utterance standard deviations near 0.9, and that the matched VBDMD condition in Table III actually favors CMGAN on PESQ (3.38 vs 3.30), the paper should report paired significance tests or bootstrap confidence intervals for the WSJ0-UNI PESQ and ESTOI differences to support the state-of-the-art claim. This is especially important because multiple metrics and variants are compared in the same table and the selection of alpha also involved a trade-off between PESQ and WV-MOS in Fig. 4a.
minor comments (6)
  1. [Throughout] There are several typos and formatting artifacts, including 'Predictive andGenerative' in Section I, 'Dereveberation' in Section VI, 'V oiceBank' in Section IV-A.2, and 'Reverb' in the Table IV caption; these should be corrected.
  2. [Section II-A, Eq. (8)-(9)] The definition of the exponential integral term E is hard to parse in the typeset equation; please rewrite it with clearer notation and state explicitly that Ei(.) is the exponential integral function.
  3. [Section IV-D] Clarify the relationship between N and the actual number of reverse steps after truncation: N=25 is the full-range discretization with Delta_t=0.04, while the final model uses Trs/Delta_t=3 reverse steps. The current wording 'we set N=25 in the sequel' can be misread as saying the final model uses 25 reverse steps.
  4. [Table II] State whether each metric is averaged over one random sample of the reverse process or over multiple stochastic samples and/or seeds; if only one sample is used, note this in the table caption because the diffusion sampler is stochastic.
  5. [Table III] For the models marked with a dagger, please state explicitly whether the pre-trained checkpoints were evaluated under exactly the same conditions as the re-trained models, including the number of reverse steps and any task-specific settings, so that the matched-condition comparison is fair.
  6. [Section V-F, Table VI] In the 'Complex' ablation row, specify precisely which components of the generative branch are changed relative to the default configuration; the sentence 'without modifying the predictive branch' is not enough to determine whether the interaction module still receives the same predictive features in both configurations.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: PGUSE's gains are empirical and externally benchmarked; the hyperparameter search is a benchmark-hygiene caveat, not a definitional reduction.

full rationale

PGUSE is an empirical architecture evaluated against external baselines; its central claims do not reduce by construction to its inputs. The predictive branch minimizes magnitude and complex MSE losses (Eqs. 21-22), the generative branch minimizes denoising score matching (Eq. 13), and inference fuses the branches via Eq. 17 and initializes truncated reverse diffusion with Eq. 20. The fusion weight, step count, and truncation start are hyperparameters tuned in Sec. IV-D; the paper defines a WSJ0 validation split (si_dt_05) and reports the headline WSJ0-UNI scores in Table II. If the test partition were used for selection, the SOTA margin would be optimistic, but that would be benchmark leakage, not circularity: Eq. (17) does not force the fused output to beat both branches, and the same fixed configuration also produces leading results on VBDMD, VBDMD-REVERB, and VBDMD-SR, which are not used for tuning. The only self-citation is architectural reuse of sub-band and dual-path modules from the authors' prior LiSenNet paper [43]; it is not load-bearing because the composite predictive-generative claim is tested against external predictive and generative baselines rather than derived from [43]. No self-definitional equation, imported uniqueness theorem, ansatz smuggled in via citation, or renaming of a known result was found. The conclusion's caveat about residual artifacts from phase estimation is an ordinary limitation statement and does not reveal circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the validity of score-based diffusion in the magnitude STFT domain, the accuracy of the predictive branch's phase estimates, and the representativeness of the synthetic distortion pipeline. No new physical entities are introduced. Four free parameters are tuned by hand or on the test set, which affects the strength of the empirical conclusions.

free parameters (4)
  • alpha (output fusion weight) = 0.4
    Weight in Eq. 17, tuned on WSJ0-UNI (Fig. 4a) to balance PESQ and WV-MOS.
  • N (number of reverse steps) = 25
    Selected on WSJ0-UNI (Fig. 4b) as the point where performance plateaus.
  • Trs (truncated diffusion start time) = 0.12
    Chosen on WSJ0-UNI (Fig. 4c) to trade off quality and computational cost.
  • beta1, beta2 (amplitude compression) = 0.3, 0.3
    Hand-chosen constants in Eq. 15 for data range and perceptual weighting; not tuned on test set.
assumptions (4)
  • standard math Score-based diffusion framework with denoising score matching is a valid generative model for speech magnitudes.
    Relies on established SDE theory and score matching from [33], [34].
  • domain assumption Magnitude STFT coefficients can be treated as real-valued data for diffusion, with negative values clipped to zero after generation.
    Assumed in Section III-A and Algorithm 2; validity depends on the clipping not degrading perceptual quality.
  • domain assumption The phase estimate from the predictive branch is accurate enough to combine with the diffusion-generated magnitude.
    Central to the fusion strategy (Eq. 17-19); the authors admit high-frequency phase imprecision can cause artifacts (Section VI).
  • domain assumption The synthetic distortion pipeline adapted from the Speech Signal Improvement Challenge is representative of universal speech degradation.
    Used to create WSJ0-UNI and TIMIT-UNI; if the pipeline omits important real-world distortions, the universality claim is limited.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement." pith.science (2026). https://pith.science/paper/2AMFALC4

@misc{pith2026250524576,
  author       = {Pith},
  title        = {Pith review of: A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2AMFALC4}},
  note         = {Machine review of arXiv:2505.24576}
}
read the original abstract

It is promising to design a single model that can suppress various distortions and improve speech quality, i.e., universal speech enhancement (USE). Compared to supervised learning-based predictive methods, diffusion-based generative models have shown greater potential due to the generative capacities from degraded speech with severely damaged information. However, artifacts may be introduced in highly adverse conditions, and diffusion models often suffer from a heavy computational burden due to many steps for inference. In order to jointly leverage the superiority of prediction and generation and overcome the respective defects, in this work we propose a universal speech enhancement model called PGUSE by combining predictive and generative modeling. Our model consists of two branches: the predictive branch directly predicts clean samples from degraded signals, while the generative branch optimizes the denoising objective of diffusion models. We utilize the output fusion and truncated diffusion scheme to effectively integrate predictive and generative modeling, where the former directly combines results from both branches and the latter modifies the reverse diffusion process with initial estimates from the predictive branch. Extensive experiments on several datasets verify the superiority of the proposed model over state-of-the-art baselines, demonstrating the complementarity and benefits of combining predictive and generative modeling.

Figures

Figures reproduced from arXiv: 2505.24576 by the authors.

Figure 1
Figure 1. Visualization of the forward process using OUVE and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) The proposed PGUSE model, where the predictive branch (top) and the generative branch (bottom) are linked by [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Visualization of the reverse process of BBED SDE, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The performance analysis under different conditions of hyper-parameters. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: The PESQ of SGMSE+, StoRM, PGUSE-G and PGUSE-F in terms of different reverse steps. 5) SGMSE+: A score-based diffusion model defined in the complex spectrum domain. It adopts NCSN++ network [33] and follows the OUVE SDE formulation. 6) StoRM: The stochastic regeneratio…
Figure 6
Figure 6. Figure 6: SE and ASR results on the TIMIT-UNI dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

83 extracted references · 73 canonical work pages

  1. [1]

    Speech en- hancement and recognition using multi-task learning of long short-term memory recurrent neural networks,

    Z. Chen, S. Watanabe, H. Erdogan, and J. R. Hershey, “Speech en- hancement and recognition using multi-task learning of long short-term memory recurrent neural networks,” inProc. Interspeech, pp. 3274– 3278, 2015

    2015

  2. [2]

    Design of matrix Wiener filter for noise reduction and speech enhancement in hearing aids,

    N. Modhave, Y . Karuna, and S. Tonde, “Design of matrix Wiener filter for noise reduction and speech enhancement in hearing aids,” inProc. RTEICT, pp. 843–847, 2016

    2016

  3. [3]

    Real-time speech enhancement using an efficient convolutional recurrent network for dual-microphone mobile phones in close-talk scenarios,

    K. Tan, X.-L. Zhang, and D.-L. Wang, “Real-time speech enhancement using an efficient convolutional recurrent network for dual-microphone mobile phones in close-talk scenarios,” inProc. ICASSP, pp. 5751–5755, 2019

    2019

  4. [4]

    Towards efficient models for real-time deep noise suppression,

    S. Braun, H. Gamper, C. K. Reddy, and I. Tashev, “Towards efficient models for real-time deep noise suppression,” inProc. ICASSP, pp. 656– 660, 2021

    work page 2021

  5. [5]

    Speech dereverberation with frequency domain autoregressive modeling,

    A. Purushothaman, D. Dutta, R. Kumar, and S. Ganapathy, “Speech dereverberation with frequency domain autoregressive modeling,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 32, pp. 29–38, 2024

    work page 2024

  6. [6]

    Towards robust speech super-resolution,

    H.-M. Wang and D.-L. Wang, “Towards robust speech super-resolution,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 2058– 2066, 2021

    work page 2021

  7. [7]

    Towards generalized speech en- hancement with generative adversarial networks,

    S. Pascual, J. Serr `a, and A. Bonafonte, “Towards generalized speech en- hancement with generative adversarial networks,” inProc. Interspeech, pp. 1791–1795, 2019

    work page 2019

  8. [8]

    Cascaded time + time-frequency unet for speech enhancement: Jointly addressing clipping, codec distortions, and gaps,

    A. A. Nair and K. Koishida, “Cascaded time + time-frequency unet for speech enhancement: Jointly addressing clipping, codec distortions, and gaps,” inProc. ICASSP, pp. 7153–7157, 2021

    work page 2021

Show all 83 references
  1. [9]

    Univer- sal speech enhancement with score-based diffusion,

    J. Serr `a, S. Pascual, J. Pons, R. O. Araz, and D. Scaini, “Univer- sal speech enhancement with score-based diffusion,”arXiv preprint arXiv:2206.03065, 2022

    2022 arXiv

  2. [10]

    Universal score- based speech enhancement with high content preservation,

    R. Scheibler, Y . Fujita, Y . Shirahata, and T. Komatsu, “Universal score- based speech enhancement with high content preservation,” inProc. Interspeech, pp. 1165–1169, 2024

    2024

  3. [11]

    SDW-SWF: Speech distortion weighted single-channel Wiener filter for noise reduction,

    J. Zhang, R. Tao, J. Du, and L.-R. Dai, “SDW-SWF: Speech distortion weighted single-channel Wiener filter for noise reduction,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 3176–3189, 2023

    2023

  4. [12]

    Supervised speech separation based on deep learning: An overview,

    D.-L. Wang and J.-T. Chen, “Supervised speech separation based on deep learning: An overview,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no. 10, pp. 1702–1726, 2018

    2018

  5. [13]

    The unimportance of phase in speech enhance- ment,

    D.-L. Wang and J. Lim, “The unimportance of phase in speech enhance- ment,”IEEE Trans. Acoust., Speech, Signal Process., vol. 30, no. 4, pp. 679–681, 1982

    1982

  6. [14]

    STFT phase reconstruction in voiced speech for an improved single-channel speech enhancement,

    M. Krawczyk and T. Gerkmann, “STFT phase reconstruction in voiced speech for an improved single-channel speech enhancement,” IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 12, pp. 1931–1940, 2014

    1931

  7. [15]

    Complex ratio masking for monaural speech separation,

    D. S. Williamson, Y .-X. Wang, and D.-L. Wang, “Complex ratio masking for monaural speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 3, pp. 483–492, 2016

    2016

  8. [16]

    Complex spectrogram enhancement by convolutional neural network with multi-metrics learn- ing,

    S.-W. Fu, T.-Y . Hu, Y . Tsao, and X.-G. Lu, “Complex spectrogram enhancement by convolutional neural network with multi-metrics learn- ing,” inProc. MLSP, pp. 1–6, 2017

    2017

  9. [17]

    On the compensation between magnitude and phase in speech separation,

    Z.-Q. Wang, G. Wichern, and J. Le Roux, “On the compensation between magnitude and phase in speech separation,”IEEE Signal Proc. Let., vol. 28, pp. 2018–2022, 2021

    2018

  10. [18]

    CMGAN: Conformer-based metric-gan for monaural speech enhancement,

    S. Abdulatif, R.-Z. Cao, and B. Yang, “CMGAN: Conformer-based metric-gan for monaural speech enhancement,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 32, pp. 2477–2493, 2024

    2024

  11. [19]

    SEGAN: Speech enhancement generative adversarial network,

    S. Pascual, A. Bonafonte, and J. Serr `a, “SEGAN: Speech enhancement generative adversarial network,” inProc. Interspeech, pp. 3642–3646, 2017

    2017

  12. [20]

    SE-Conformer: Time-domain speech enhancement using conformer,

    E. Kim and H. Seo, “SE-Conformer: Time-domain speech enhancement using conformer,” inProc. Interspeech, pp. 2736–2740, 2021

    2021

  13. [21]

    WSRGlow: A glow- based waveform generative model for audio super-resolution,

    K.-X. Zhang, Y . Ren, C.-L. Xu, and Z. Zhao, “WSRGlow: A glow- based waveform generative model for audio super-resolution,” inProc. Interspeech, pp. 1649–1653, 2021

    2021

  14. [22]

    Auto-encoding variational Bayes,

    D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in Proc. ICLR, 2014

    2014

  15. [23]

    Generative adversarial nets,

    I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in Proc. NeurIPS, vol. 27, p. 2672–2680, 2014

    2014

  16. [24]

    Variational inference with normalizing flows,

    D. J. Rezende and S. Mohamed, “Variational inference with normalizing flows,” inProc. ICML, vol. 37, p. 1530–1538, 2015

    2015

  17. [25]

    Deep unsupervised learning using nonequilibrium thermodynamics,

    J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in Proc. ICML, vol. 37, p. 2256–2265, 2015

    2015

  18. [26]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” inProc. NeurIPS, vol. 33, pp. 6840–6851, 2020

    2020

  19. [27]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inProc. CVPR, pp. 10674–10685, 2022

    2022

  20. [28]

    Diffusion models beat GANs on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat GANs on image synthesis,” inProc. NeurIPS(M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, eds.), vol. 34, pp. 8780–8794, 2021

    2021

  21. [29]

    Conditional diffusion probabilistic model for speech enhancement,

    Y .-J. Lu, Z.-Q. Wang, S. Watanabe, A. Richard, C. Yu, and Y . Tsao, “Conditional diffusion probabilistic model for speech enhancement,” in Proc. ICASSP, pp. 7402–7406, 2022

    2022

  22. [30]

    Speech enhancement with score-based generative models in the complex STFT domain,

    S. Welker, J. Richter, and T. Gerkmann, “Speech enhancement with score-based generative models in the complex STFT domain,” inProc. Interspeech, pp. 2928–2932, 2022

    2022

  23. [31]

    Speech enhancement and dereverberation with diffusion-based genera- tive models,

    J. Richter, S. Welker, J.-M. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based genera- tive models,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 2351–2364, 2023

    2023

  24. [32]

    StoRM: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,

    J.-M. Lemercier, J. Richter, S. Welker, and T. Gerkmann, “StoRM: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 2724–2737, 2023

    2023

  25. [33]

    Score-based generative modeling through stochastic differ- ential equations,

    Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,” inProc. ICLR, 2021

    2021

  26. [34]

    Estimation of non-normalized statistical models by score matching,

    A. Hyv ¨arinen, “Estimation of non-normalized statistical models by score matching,”Journal of Machine Learning Research, vol. 6, no. 24, pp. 695–709, 2005

    2005

  27. [35]

    SRTNet: Time domain speech enhancement via stochastic refinement,

    Z.-B. Qiu, M.-F. Fu, Y .-F. Yu, L.-L. Yin, F.-C. Sun, and H. Huang, “SRTNet: Time domain speech enhancement via stochastic refinement,” inProc. ICASSP, pp. 1–5, 2023

    2023

  28. [36]

    Reducing the prior mismatch of stochastic differential equations for diffusion-based speech enhancement,

    B. Lay, S. Welker, J. Richter, and T. Gerkmann, “Reducing the prior mismatch of stochastic differential equations for diffusion-based speech enhancement,” inProc. Interspeech, pp. 3809–3813, 2023

    2023

  29. [37]

    Schr ¨odinger bridge for generative speech enhancement,

    A. Juki ´c, R. Korostik, J. Balam, and B. Ginsburg, “Schr ¨odinger bridge for generative speech enhancement,” inProc. Interspeech, pp. 1175– 1179, 2024

    2024

  30. [38]

    Investigating train- ing objectives for generative speech enhancement,

    J. Richter, D. de Oliveira, and T. Gerkmann, “Investigating train- ing objectives for generative speech enhancement,”arXiv preprint arXiv:2409.10753, 2024

    2024 arXiv

  31. [39]

    Reverse-time diffusion equation models,

    B. D. Anderson, “Reverse-time diffusion equation models,”Stoch. Proc. Appl., vol. 12, no. 3, pp. 313–326, 1982

    1982

  32. [40]

    A connection between score matching and denoising au- toencoders,

    P. Vincent, “A connection between score matching and denoising au- toencoders,”Neural Comput., vol. 23, no. 7, pp. 1661–1674, 2011

    2011

  33. [41]

    P. E. Kloeden and E. Platen,Numerical Solution of Stochastic Differen- tial Equations. Springer Berlin, Heidelberg, 1992

    1992

  34. [42]

    Correlation functions and computer simulations,

    G. Parisi, “Correlation functions and computer simulations,”Nucl. Phys. B, vol. 180, no. 3, pp. 378–384, 1981

    1981

  35. [43]

    LiSenNet: Lightweight sub-band and dual-path modeling for real-time speech enhancement,

    H.-Y . Yan, J. Zhang, C.-H. Fan, Y .-P. Zhou, and P.-Q. Liu, “LiSenNet: Lightweight sub-band and dual-path modeling for real-time speech enhancement,”arXiv preprint arXiv:2409.13285, 2024

    2024 arXiv

  36. [44]

    Perceptual linear predictive (PLP) analysis of speech.,

    H. Hermansky, “Perceptual linear predictive (PLP) analysis of speech.,” The Journal of the Acoustical Society of America, vol. 87, pp. 1738– 1752, 1990

    1990

  37. [45]

    MetricGAN: Generative adversarial networks based black-box metric scores optimization for speech enhancement,

    S.-W. Fu, C.-F. Liao, Y . Tsao, and S.-D. Lin, “MetricGAN: Generative adversarial networks based black-box metric scores optimization for speech enhancement,” inProc. ICML, vol. 97, pp. 2031–2041, 2019

    2019

  38. [46]

    /spl beta/-order MMSE spectral amplitude estimation for speech enhancement,

    C. H. You, S. N. Koh, and S. Rahardja, “/spl beta/-order MMSE spectral amplitude estimation for speech enhancement,”IEEE Trans. Speech Audio Process., vol. 13, no. 4, pp. 475–486, 2005

    2005

  39. [47]

    A consolidated view of loss functions for supervised deep learning-based speech enhancement,

    S. Braun and I. Tashev, “A consolidated view of loss functions for supervised deep learning-based speech enhancement,” inProc. TSP, pp. 72–76, 2021

    2021

  40. [48]

    U-Net: Convolutional net- works for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional net- works for biomedical image segmentation,” inProc. MICCAI, pp. 234– 241, 2015

    2015

  41. [49]

    Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,

    W.-Z. Shi, J. Caballero, F. Husz ´ar, J. Totz, A. P. Aitken, R. Bishop,et al., “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” inProc. CVPR, pp. 1874–1883, 2016

    2016

  42. [50]

    Fourier features let networks learn high frequency functions in low dimensional domains,

    M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng, “Fourier features let networks learn high frequency functions in low dimensional domains,” inProc. NeurIPS, vol. 33, pp. 7537–7547, 2020

    2020

  43. [51]

    Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,

    Y . Luo, Z. Chen, and T. Yoshioka, “Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech separation,” inProc. ICASSP, pp. 46–50, 2020

    2020

  44. [52]

    DPCRN: Dual-path convolution recurrent network for single channel speech enhancement,

    X.-H. Le, H.-S. Chen, K. Chen, and J. Lu, “DPCRN: Dual-path convolution recurrent network for single channel speech enhancement,” inProc. Interspeech, pp. 2811–2815, 2021

    2021

  45. [53]

    DPT-FSNet: Dual-path trans- former based full-band and sub-band fusion network for speech enhance- ment,

    F. Dang, H.-T. Chen, and P.-Y . Zhang, “DPT-FSNet: Dual-path trans- former based full-band and sub-band fusion network for speech enhance- ment,” inProc. ICASSP, pp. 6857–6861, 2022

    2022

  46. [54]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inProc. NeurIPS, vol. 30, 2017

    2017

  47. [55]

    TransNeXt: Robust foveal visual perception for vision trans- formers,

    D. Shi, “TransNeXt: Robust foveal visual perception for vision trans- formers,” inProc. CVPR, pp. 17773–17783, 2024

    2024

  48. [56]

    Xception: Deep learning with depthwise separable convo- lutions,

    F. Chollet, “Xception: Deep learning with depthwise separable convo- lutions,” inProc. CVPR, pp. 1800–1807, 2017

    2017

  49. [57]

    Mish: A self regularized non-monotonic neural activation function,

    D. Misra, “Mish: A self regularized non-monotonic neural activation function,”arXiv preprint arXiv:1908.08681, 2019

    1908 arXiv

  50. [58]

    A novel noise-aware deep learning model for underwater acoustic denoising,

    A.-L. Zhou, W. Zhang, X.-Y . Li, G.-J. Xu, B.-B. Zhang, Y .-X. Ma, and J.-Q. Song, “A novel noise-aware deep learning model for underwater acoustic denoising,”IEEE Trans. Geosci. Remote Sens., vol. 61, pp. 1– 13, 2023

    2023

  51. [59]

    V oiceFilter-Lite: Streaming targeted voice separation for on-device speech recognition,

    Q. Wang, I. L. Moreno, M. Saglam, K. Wilson, A. Chiao, R.-J. Liu, Y .-Z. He, W. Li, J. Pelecanos, M. Nika, and A. Gruenstein, “V oiceFilter-Lite: Streaming targeted voice separation for on-device speech recognition,” arXiv preprint arXiv:2009.04323, 2020

    2009 arXiv

  52. [60]

    Accelerating diffusion models via early stop of the diffusion process,

    Z.-Y . Lyu, X.-D. XU, C.-Y . Yang, D.-H. Lin, and B. Dai, “Accelerating diffusion models via early stop of the diffusion process,”arXiv preprint arXiv:2205.12524, 2022

    2022 arXiv

  53. [61]

    Truncated diffu- sion probabilistic models and diffusion-based adversarial auto-encoders,

    H.-J. Zheng, P.-C. He, W.-Z. Chen, and M.-Y . Zhou, “Truncated diffu- sion probabilistic models and diffusion-based adversarial auto-encoders,” Proc. ICLR, 2023

    2023

  54. [62]

    ICASSP 2024 speech signal improvement challenge,

    N.-C. Ristea, A. Saabas, R. Cutler, B. Naderi, S. Braun, and S. Branets, “ICASSP 2024 speech signal improvement challenge,” inProc. ICAS- SPW, pp. 15–16, 2024

    2024

  55. [63]

    CSR-I (WSJ0) Com- plete

    J. S. Garofolo, D. Graff, D. Paul, and D. Pallett, “CSR-I (WSJ0) Com- plete.” [Online]. Available: https://catalog.ldc.upenn.edu/LDC93S6A

  56. [64]

    WHAM!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “WHAM!: Extending speech separation to noisy environments,” inProc. Interspeech, pp. 1368–1372, 2019

    2019

  57. [65]

    Investi- gating RNN-based speech enhancement methods for noise-robust Text- to-Speech,

    C. Valentini-Botinhao, X. Wang, S. Takaki, and J. Yamagishi, “Investi- gating RNN-based speech enhancement methods for noise-robust Text- to-Speech,” inProc. SSW, pp. 146–152, 2016

    2016

  58. [66]

    The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,

    C. Veaux, J. Yamagishi, and S. King, “The voice bank corpus: Design, collection and data analysis of a large regional accent speech database,” inProc. O-COCOSDA/CASLRE, pp. 1–4, 2013

    2013

  59. [67]

    The diverse environments multi- channel acoustic noise database: A database of multichannel environ- mental noise recordings,

    J. Thiemann, N. Ito, and E. Vincent, “The diverse environments multi- channel acoustic noise database: A database of multichannel environ- mental noise recordings,”J. Acoust. Soc. Am., vol. 133, pp. 3591–3591, 2013

    2013

  60. [68]

    “Colorless

    M. Schroeder and B. Logan, ““Colorless” artificial reverberation,”IRE Trans. Audio, vol. AU-9, no. 6, pp. 209–214, 1961

    1961

  61. [69]

    TIMIT Acoustic-Phonetic Continuous Speech Corpus

    J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, N. L. Dahlgren, and V . Zue, “TIMIT Acoustic-Phonetic Continuous Speech Corpus.” [Online]. Available: https://catalog.ldc.upenn.edu/LDC93S1

  62. [70]

    Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,

    A. Rix, J. Beerends, M. Hollier, and A. Hekstra, “Perceptual evaluation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” inProc. ICASSP, vol. 2, pp. 749–752, 2001

    2001

  63. [71]

    An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 24, no. 11, pp. 2009–2022, 2016

    2009

  64. [72]

    Evaluation of objective quality measures for speech enhancement,

    Y . Hu and P. C. Loizou, “Evaluation of objective quality measures for speech enhancement,”IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 1, pp. 229–238, 2008

    2008

  65. [73]

    HIFI++: A unified framework for bandwidth extension and speech enhancement,

    P. Andreev, A. Alanov, O. Ivanov, and D. Vetrov, “HIFI++: A unified framework for bandwidth extension and speech enhancement,” inProc. ICASSP, pp. 1–5, 2023

    2023

  66. [74]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y .-H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” in Proc. NeurIPS, vol. 33, pp. 12449–12460, 2020

    2020

  67. [75]

    ViSQOL v3: An open source production ready objective speech and audio metric,

    M. Chinen, F. S. C. Lim, J. Skoglund, N. Gureev, F. O’Gorman, and A. Hines, “ViSQOL v3: An open source production ready objective speech and audio metric,” inProc. QoMEX, pp. 1–6, 2020

    2020

  68. [76]

    Distance measures for speech processing,

    A. Gray and J. Markel, “Distance measures for speech processing,”IEEE Trans. Acoust., Speech, Signal Process., vol. 24, no. 5, pp. 380–391, 1976

    1976

  69. [77]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,”IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, 2004

    2004

  70. [78]

    Squeezeformer: An efficient transformer for automatic speech recognition,

    S. Kim, A. Gholami, A. Shaw, N. Lee, K. Mangalam, J. Malik, M. W. Mahoney, and K. Keutzer, “Squeezeformer: An efficient transformer for automatic speech recognition,” inProc. NeurIPS, vol. 35, pp. 9361– 9373, 2022

    2022

  71. [79]

    Improved techniques for training score-based generative models,

    Y . Song and S. Ermon, “Improved techniques for training score-based generative models,” inProc. NeurIPS, vol. 33, pp. 12438–12448, 2020

    2020

  72. [80]

    Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-TasNet: Surpassing ideal time- frequency magnitude masking for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  73. [81]

    MANNER: Multi-view attention network for noise erasure,

    H. J. Park, B. H. Kang, W. Shin, J. S. Kim, and S. W. Han, “MANNER: Multi-view attention network for noise erasure,” inProc. ICASSP, pp. 7842–7846, 2022

    2022

  74. [82]

    Conformer: Convolution-augmented transformer for speech recognition,

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J.-H. Yu, W. Han, S.-B. Wang, Z.-D. Zhang, Y .-H. Wu, and R.-M. Pang, “Conformer: Convolution-augmented transformer for speech recognition,” inProc. Interspeech, pp. 5036–5040, 2020

    2020

  75. [83]

    Neural vocoder is all you need for speech super-resolution,

    H.-H. Liu, W. Choi, X.-B. Liu, Q.-Q. Kong, Q. Tian, and D.-L. Wang, “Neural vocoder is all you need for speech super-resolution,” inProc. Interspeech, pp. 4227–4231, 2022. Jie Zhang(Senior Member, IEEE) received the B.Sc. (with honors from Yunnan University, Yunnan, China), M....

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.