Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read UniFlow claims that one task-ID-conditioned Diffusion Transformer over a shared waveform-VAE latent space can do speech enhancement, target speaker extraction, acoustic echo cancellation, and language-queried separation at levels matching o

desk verdict A real unified continuous-latent front-end worth reading, but the abstract's 'consistent gains' claim is contradicted by the paper's own tables; the honest result is 'competitive, sometimes behind.' read the letter →

arxiv 2508.07558 v1 pith:ELEPSV6N submitted 2025-08-11 eess.AS

Ziqian Wang , Zikai Liu , Yike Zhu , Xingchen Li , Boyi Kang , Jixun Yao , Xianjun Xia , Chuanzeng Huang
show 1 more author
Lei Xie
This is my paper
classification eess.AS
keywords speechfront-endtaskscontinuousgenerativemodelingdiffusiontransformerflowmatchingmeantask-IDconditioningtargetspeakerextractionlanguage-queriedseparation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

UniFlow argues that four speech front-end tasks—speech enhancement, target speaker extraction, acoustic echo cancellation, and language-queried source separation—do not need separate models. It proposes a single generative model that maps raw audio into a shared continuous latent space with a frozen waveform variational autoencoder, then uses one large Diffusion Transformer to transform noisy latents into task-specific outputs, told what task to perform by a learnable task-ID embedding. Trained once, the same checkpoint reports scores that match or exceed dedicated state-of-the-art baselines on all four benchmarks, while a one-step 'mean flow' variant runs at a real-time factor of 0.02. The paper also tests three generative objectives—denoising diffusion, flow matching, and mean flow—and shows they form a quality-versus-speed ladder inside one architecture. If the reported comparisons hold, UniFlow offers a route to one extensible speech front-end rather than a stack of task-specific systems.

What carries the argument

The machinery is a frozen waveform variational autoencoder plus a 32-layer, 1.7-billion-parameter conditional Diffusion Transformer. Conditioning flows through three routes—concatenating VAE-encoded condition audio to the target latent sequence, cross-attending to frozen auxiliary extractor embeddings, and globally injecting a learnable task-ID embedding with the timestep embedding. The same DiT can be trained under denoising diffusion, flow matching, or mean flow; mean flow collapses inference to a single forward pass.

What would settle it

Re-evaluate UniFlow and the baseline systems on the same test clips with bootstrap confidence intervals around DNSMOS OVRL and AECMOS EMOS/DMOS, or run a small subjective listening test; for example, the reported SE gap between UniFlow-DDPM (OVRL 3.48) and FlowSE (3.45) would be checked against a 600-clip bootstrap. If the intervals overlap or listeners show no consistent preference, the claimed consistent gains are not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that continuous generative modeling in a shared latent space can unify speech front-end tasks. UniFlow pretrains and freezes a fully convolutional waveform VAE (48 kHz input, 256-dimensional continuous latents at 50 Hz, trained with an ELBO plus multi-resolution spectral and adversarial losses). During task training, a conditional Diffusion Transformer receives the noise-perturbed latent of the ground-truth output plus three conditioning routes: the VAE-encoded noisy or mixed audio concatenated to the latent sequence; frozen high-level embeddings of auxiliary inputs through cross-attention; and a learnable task-ID embedding with a timestep embedding injected glob

Load-bearing premise

The headline result rests on treating the learned no-reference quality scores used for each benchmark as directly comparable across systems, with differences of a few hundredths treated as meaningful even though the paper reports no error bars or significance tests.

Editorial extensions

If this is right

  • A single UniFlow checkpoint can serve speech enhancement, target speaker extraction, echo cancellation, and language-queried separation, with DDPM offering the best perceptual scores, flow matching a 32-step middle ground, and mean flow real-time one-step generation.
  • Task ID is critical rather than cosmetic: removing it causes 35–42% output confusion and substantial metric degradation, so task-specific conditioning, not task-specific weights, carries the multitask behavior.
  • Adding a front-end task reduces to assigning a new task ID and attaching an appropriate feature extractor; the paper demonstrates the point by extending the same backbone to text-to-speech.
  • The VAE is kept frozen deliberately: joint fine-tuning improves single-task enhancement quality (DNSMOS OVRL 3.48 to 3.55) but is avoided because it harms multi-task generalization.
  • Latent resolution and VAE losses set the fidelity-efficiency operating point: 50 Hz, 256 dimensions, and both spectral and adversarial losses beat lower-dimension or lower-rate configurations.
  • If the metric comparisons hold, a single 1.7B-parameter generative front-end can replace several task-specific models, with mean flow making it fast enough for real-time use.
  • The shared continuous latent space may transfer acoustic knowledge between restoration and extraction tasks; a direct experiment would be fine-tuning on one task and measuring zero-shot gains on another, which the paper does not report.
  • The task-ID mechanism suggests a scaling path: as more task IDs are added, the framework could be probed for whether confusion rates stay low or grow with task count, a question the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents UniFlow, a unified generative model for four speech front-end tasks (SE, TSE, AEC, LASS) plus a TTS extension. It uses a frozen waveform VAE to map 48 kHz audio into a 256-dimensional 50 Hz continuous latent space, and a 1.7B-parameter DiT conditioned on task ID, input audio, and frozen task-specific embeddings. Three generative objectives are compared: DDPM, flow matching, and mean flow. Experiments on DNS, AEC, TSE, and AudioCaps report DNSMOS, AECMOS, pDNSMOS, and FAD/CLAP metrics, with DDPM best on quality and mean flow fastest (RTF 0.02). The paper claims consistent gains over state-of-the-art baselines.

Significance. If borne out, a single frozen-VAE DiT checkpoint that handles SE, TSE, AEC, and LASS with one-step mean-flow at RTF 0.02 would be a practically useful step toward unified front-end processing. The paper's strengths are the breadth of tasks considered, the shared-continuous-latent formulation, the task-ID ablation (Table 6), and the efficiency comparison (Table 9). I found no circularity in the evaluation: the metrics are external, and task conditioning is verified rather than assumed. However, because the headline claim of consistent SOTA gains is contradicted by several of the paper's own tables and no statistical support is given, the current evidence supports a 'competitive unified system' claim, not a SOTA claim. The work is worth publishing after the claims and evaluation are brought in line with the data.

major comments (3)
  1. [Abstract; Main Results (Tables 2–5)] The abstract claims 'consistent gains over state-of-the-art baselines,' and the SE section says UniFlow 'consistently outperforms all baselines.' The paper's own tables do not support this. In Table 2, with reverb, FlowSE yields SIG 3.61 and OVRL 3.34, while UniFlowDDPM yields 3.59 and 3.32; LLaSE-G1 OVRL 3.33 also exceeds UniFlowDDPM. In Table 3, DeepVQE exceeds every UniFlow variant on every reported AEC condition (e.g., DT DMOS 4.29 vs. 3.93; FEST DMOS 4.41 vs. 3.70), and Align-ULCNet FEST DMOS is 4.28 vs. 3.70. In Table 5, FlowSep has CLAPScore A 81.7 vs. UniFlowDDPM 81.2, and in Table 4 Track 2 UniFlowDDPM OVRL ties NAPSE at 3.56. The headline claim should be re-scoped to 'competitive across tasks, with wins on selected metrics'; otherwise the central result is overstated.
  2. [Evaluation Metrics; Tables 2–5; Table 8] All performance comparisons lack error bars, confidence intervals, or significance tests. Many reported differences are 0.01–0.08 (e.g., Table 4 Track 1 OVRL 3.73 vs. LLaSE-G1 3.72; no-reverb OVRL 3.48 vs. FlowSE 3.45), which is within the known utterance-level variability of DNSMOS/pDNSMOS/AECMOS. In addition, the pipeline resamples everything to 48 kHz (Data Augmentation; Table 8 caption), whereas DNSMOS and AECMOS are commonly calibrated for 16 kHz; the paper does not state the test-time sample rate, DNSMOS version, or provide evidence that the scores are comparable across systems trained and evaluated at different rates. Without this information, the claimed wins—and even the direction of some comparisons—are not established.
  3. [Continuous Generative Modeling, Eqs. (11)–(14)] The flow-matching and mean-flow training targets are under-specified. In Eq. (11), the target (z0 - zt)/λ(t) is not the standard rectified-flow velocity for the usual interpolation z_t = (1-t)z0 + t z1 (which is z1 - z0), and neither the forward kernel nor λ(t) is defined; as written, the target is singular or ill-defined at t=0. In Eq. (13), u_tgt is defined in terms of the same uθ and its Jacobian, so the training procedure needs an explicit stop-gradient/EMA treatment; merely writing sg(·) around u_tgt is insufficient for reproduction. Because the comparison of DDPM, FM, and MF is a stated contribution, these definitions are load-bearing and should be aligned with the cited references.
minor comments (5)
  1. [Table 3] The table formatting is confusing: some NEST entries are missing, and it is unclear which metric applies to each column. Please make the row/column labels explicit and complete.
  2. [Conclusion] The conclusion says 'matches or exceeds task-specific baselines,' which is more cautious than the abstract. Make the wording consistent throughout the paper.
  3. [Appendix, Text-to-Speech] There is a typo in 'using m the test-clean set.' Also clarify whether the TTS baselines were evaluated under identical resampling/conditions and whether any significance testing was performed.
  4. [Eqs. (8)–(10)] Specify the noise schedule, the range of t, and whether the model predicts ε or v. Eq. (9) trains ε while Eq. (10) introduces v, so the exact prediction target at inference should be stated.
  5. [Table 6] The 'confusion rate' is not defined. Please state how outputs are classified into tasks (e.g., which classifier or metric is used) and how the confusion counts are computed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: UniFlow composes externally cited building blocks (VAE, DiT, DDPM/FM/MF) and evaluates on external blind-test metrics; the abstract's 'consistent gains' claim is contradicted by the paper's own tables, but that is a correctness/robustness issue, not circular reasoning.

full rationale

The paper's derivation chain is not circular. UniFlow is a composition of a frozen waveform VAE (inspired by SoundStream), a conditional Diffusion Transformer (adapted from Peebles & Xie), and three generative objectives (DDPM, Flow Matching, Mean Flow) imported from prior work with explicit citations. No learnable parameter is fitted to a benchmark and then relabeled as a prediction; all reported scores come from external non-intrusive metrics (DNSMOS, pDNSMOS, AECMOS, FAD, CLAPScore) on public blind test sets. The task-ID conditioning is tested by an ablation rather than assumed, and the Mean Flow objective is adopted from Geng et al. (2025) as a building block, not invoked as a self-referential uniqueness theorem. Several baselines (FlowSE, LLaSE-G1, SELM) originate from the same author group, which weakens the independence of the comparison but does not make any step reduce to its own inputs. In fact, the paper's own Tables 2 and 3 contradict the abstract's 'consistent gains' (FlowSE beats UniFlow on SE with-reverb SIG/OVRL; DeepVQE beats all UniFlow variants on AEC metrics), and the reported differences lack error bars/significance testing. These are empirical and statistical concerns, not circularity. No equation, fitted parameter, or citation chain in the paper exhibits the specific reduction required to establish circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is an empirical system result, not a derivation, so the ledger records hand-chosen hyperparameters and domain assumptions rather than fitted constants. The main unstated premises are that the continuous 50 Hz latent captures enough detail for all four tasks, that non-intrusive metrics transfer across papers, and that baselines from overlapping author groups are reliable anchors. No new physical entities are introduced.

free parameters (4)
  • VAE loss weights (beta, lambda_KL, lambda_adv, lambda_spec) = not reported
    Hand-tuned in VAE pretraining; the paper states beta and lambdas exist but gives no values. Affects latent quality and hence all downstream task scores.
  • Latent dimensionality and frame rate (256-dim, 50 Hz) = dim=256, downsample factor 960
    Chosen by design and ablated in Table 8; the central claim depends on this operating point, and 100 Hz improves quality at higher cost.
  • Generative sampling steps (DDPM=200, FM=32, MF=1) = RTF 3.49/0.31/0.02
    Selected for the efficiency trade-off table; not fitted to data but determines which variant is compared to SOTA.
  • Training data simulation ranges (SNR, SER, probabilities) = SNR [-5,20] dB SE, [-15,15] dB AEC/TSE, SER [-15,15] dB, 20%/10% noise probs
    Hand-chosen augmentation recipe; the specific values change the task difficulty and are not justified against alternatives.
assumptions (4)
  • domain assumption A continuous latent space preserves fine-grained speech details (timbre, intelligibility) better than discrete tokenized representations.
    Invoked in the Introduction and Related Work to justify the waveform VAE without VQ; not demonstrated within this paper. If false, the main distinction from AnyEnhance/LLaSE-G1 collapses.
  • domain assumption A frozen 50 Hz, 256-dim VAE latent is sufficient to represent inputs and outputs for all four front-end tasks.
    Method, Waveform VAE; ablations in Table 8 show 25 Hz degrades and 100 Hz slightly helps, but the model fixes 50 Hz globally to keep multitask generalization.
  • domain assumption DNSMOS, pDNSMOS, AECMOS, FAD, and CLAP scores are comparable across systems trained on different data and sample rates, and differences of 0.01-0.08 are meaningful.
    Experiments, Evaluation Metrics; the headline "consistent gains" rests entirely on this comparability, yet no error bars or matched-condition re-runs of baselines are provided.
  • standard math The diffusion/flow/mean-flow objectives as implemented follow the cited formulations (Ho et al. 2020; Lipman et al. 2023; Geng et al. 2025) without additional assumptions.
    Eqs. (8)-(14); the paper does not re-derive these, so correctness inherits the cited work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling." pith.science (2026). https://pith.science/paper/ELEPSV6N

@misc{pith2026250807558,
  author       = {Pith},
  title        = {Pith review of: UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ELEPSV6N}},
  note         = {Machine review of arXiv:2508.07558}
}
read the original abstract

Generative modeling has recently achieved remarkable success across image, video, and audio domains, demonstrating powerful capabilities for unified representation learning. Yet speech front-end tasks such as speech enhancement (SE), target speaker extraction (TSE), acoustic echo cancellation (AEC), and language-queried source separation (LASS) remain largely tackled by disparate, task-specific solutions. This fragmentation leads to redundant engineering effort, inconsistent performance, and limited extensibility. To address this gap, we introduce UniFlow, a unified framework that employs continuous generative modeling to tackle diverse speech front-end tasks in a shared latent space. Specifically, UniFlow utilizes a waveform variational autoencoder (VAE) to learn a compact latent representation of raw audio, coupled with a Diffusion Transformer (DiT) that predicts latent updates. To differentiate the speech processing task during the training, learnable condition embeddings indexed by a task ID are employed to enable maximal parameter sharing while preserving task-specific adaptability. To balance model performance and computational efficiency, we investigate and compare three generative objectives: denoising diffusion, flow matching, and mean flow within the latent domain. We validate UniFlow on multiple public benchmarks, demonstrating consistent gains over state-of-the-art baselines. UniFlow's unified latent formulation and conditional design make it readily extensible to new tasks, providing an integrated foundation for building and scaling generative speech processing pipelines. To foster future research, we will open-source our codebase.

Figures

Figures reproduced from arXiv: 2508.07558 by the authors.

Figure 1
Figure 1. The overall architecture of UniFlow. It includes a waveform VAE for latent encoding/decoding, a conditional Diffusion [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Post-Training Speech Enhancement Language Models with Perceptual Rewards

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Post-training autoregressive speech enhancement LMs via GSPO with composite perceptual rewards from DNSMOS, WER, and UTMOS reaches SOTA on DNS2020 and outperforms single-metric variants in human evaluation.

Reference graph

Works this paper leans on

63 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    2noise. 2024. ChatTTS. https://github.com/2noise/ChatTTS

    work page 2024

  4. [4]

    Cutler, R.; Saabas, A.; Parnamaa, T.; Purin, M.; Indenbom, E.; Ristea, N.-C.; Gužvin, J.; Gamper, H.; Braun, S.; and Aichner, R. 2023. ICASSP 2023 Acoustic Echo Cancellation Challenge. arXiv:2309.12553

    arXiv 2023

  5. [5]

    D \'e fossez, A.; Usunier, N.; Bottou, L.; and Bach, F. 2019. Demucs: Deep extractor for music sources with extra unlabeled data remixed. arXiv preprint arXiv:1909.01174

    arXiv 2019

  6. [6]

    Desplanques, B.; Thienpondt, J.; and Demuynck, K. 2020. Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. arXiv preprint arXiv:2005.07143

    arXiv 2020

  7. [7]

    Du, C.; Guo, Y.; Shen, F.; Liu, Z.; Liang, Z.; Chen, X.; Wang, S.; Zhang, H.; and Yu, K. 2024 a . Unicats: A unified context-aware text-to-speech framework with contextual vq-diffusion and vocoding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 17924--17932

    work page 2024

  8. [8]

    Du, J.; Na, X.; Liu, X.; and Bu, H. 2018. Aishell-2: Transforming mandarin asr research into industrial scale. arXiv preprint arXiv:1808.10583

    arXiv 2018

Show all 63 references
  1. [9]

    Du, Z.; Wang, Y.; Chen, Q.; Shi, X.; Lv, X.; Zhao, T.; Gao, Z.; Yang, Y.; Gao, C.; Wang, H.; Yu, F.; Liu, H.; Sheng, Z.; Gu, Y.; Deng, C.; Wang, W.; Zhang, S.; Yan, Z.; and Zhou, J. 2024 b . CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models. arXiv:2412.10117

    2024 arXiv

  2. [10]

    Dubey, H.; Aazami, A.; Gopal, V.; Naderi, B.; Braun, S.; Cutler, R.; Ju, A.; Zohourian, M.; Tang, M.; Gamper, H.; Golestaneh, M.; and Aichner, R. 2023. ICASSP 2023 Deep Noise Suppression Challenge. arXiv:2303.11510

    2023 arXiv

  3. [11]

    Défossez, A.; Copet, J.; Synnaeve, G.; and Adi, Y. 2022. High Fidelity Neural Audio Compression. arXiv:2210.13438

    2022 arXiv

  4. [12]

    Elizalde, B.; Deshmukh, S.; Al Ismail, M.; and Wang, H. 2023. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5. IEEE

    2023

  5. [13]

    D.; Carr, C.; Zukowski, Z.; Taylor, J.; and Pons, J

    Evans, Z.; Parker, J. D.; Carr, C.; Zukowski, Z.; Taylor, J.; and Pons, J. 2025. Stable audio open. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5. IEEE

    2025

  6. [14]

    Z.; and He, K

    Geng, Z.; Deng, M.; Bai, X.; Kolter, J. Z.; and He, K. 2025. Mean Flows for One-step Generative Modeling. arXiv:2505.13447

    2025 arXiv

  7. [15]

    Ghosal, D.; Majumder, N.; Mehrish, A.; and Poria, S. 2023. Text-to-audio generation using instruction guided latent diffusion model. In Proceedings of the 31st ACM International Conference on Multimedia, 3590--3598

    2023

  8. [16]

    He, H.; Shang, Z.; Wang, C.; Li, X.; Gu, Y.; Hua, H.; Liu, L.; Yang, C.; Li, J.; Shi, P.; et al. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. In 2024 IEEE Spoken Language Technology Workshop (SLT), 885--890. IEEE

    2024

  9. [17]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising Diffusion Probabilistic Models. arXiv:2006.11239

    2020 arXiv

  10. [18]

    Hu, Y.; Liu, Y.; Lv, S.; Xing, M.; Zhang, S.; Fu, Y.; Wu, J.; Zhang, B.; and Xie, L. 2020. DCCRN: Deep complex convolution recurrent network for phase-aware speech enhancement. arXiv preprint arXiv:2008.00264

    2020 arXiv

  11. [19]

    Huang, J.; Ren, Y.; Huang, R.; Yang, D.; Ye, Z.; Zhang, C.; Liu, J.; Yin, X.; Ma, Z.; and Zhao, Z. 2023. Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation. arXiv:2305.18474

    2023 arXiv

  12. [20]

    Huang, K.; Li, Y.; Wang, Z.; Wang, H.; Rao, W.; Sun, Z.; Tang, Z.; Huang, S.; Wang, Y.; Yu, T.; Xie, L.; and Shang, S.-d. 2024. The NPU-TEA System for the CHiME-8 NOTSOFAR-1 Challenge. 45--48

    2024

  13. [21]

    Indenbom, E.; Ristea, N.-C.; Saabas, A.; Pärnamaa, T.; and Gužvin, J. 2023. Deep model with built-in cross-attention alignment for acoustic echo cancellation. arXiv:2208.11308

    2023 arXiv

  14. [22]

    Ju, Y.; Chen, J.; Zhang, S.; He, S.; Rao, W.; Zhu, W.; Wang, Y.; Yu, T.; and Shang, S. 2023. TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 Dns-Challenge. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech a...

    2023

  15. [23]

    Kang, B.; Zhu, X.; Zhang, Z.; Ye, Z.; Liu, M.; Wang, Z.; Zhu, Y.; Ma, G.; Chen, J.; Xiao, L.; Weng, C.; Xue, W.; and Xie, L. 2025. LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement. arXiv:2503.00493

    2025 arXiv

  16. [24]

    Kilgour, K.; Zuluaga, M.; Roblek, D.; and Sharifi, M. 2018. Fr 'echet audio distance: A metric for evaluating music enhancement algorithms. arXiv preprint arXiv:1812.08466

    2018 arXiv

  17. [25]

    D.; Kim, B.; Lee, H.; and Kim, G

    Kim, C. D.; Kim, B.; Lee, H.; and Kim, G. 2019. Audiocaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Pa...

    2019

  18. [26]

    Kim, S.; Lee, S.-g.; Song, J.; Kim, J.; and Yoon, S. 2018. FloWaveNet: A generative flow for raw audio. arXiv preprint arXiv:1811.02155

    2018 arXiv

  19. [27]

    P.; and Welling, M

    Kingma, D. P.; and Welling, M. 2022. Auto-Encoding Variational Bayes. arXiv:1312.6114

    2022 arXiv

  20. [28]

    L.; and Khudanpur, S

    Ko, T.; Peddinti, V.; Povey, D.; Seltzer, M. L.; and Khudanpur, S. 2017. A study on data augmentation of reverberant speech for robust speech recognition. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), 5220--5224. IEEE

    2017

  21. [29]

    Kong, Z.; Ping, W.; Huang, J.; Zhao, K.; and Catanzaro, B. 2020. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761

    2020 arXiv

  22. [30]

    Le, M.; Vyas, A.; Shi, B.; Karrer, B.; Sari, L.; Moritz, R.; Williamson, M.; Manohar, V.; Adi, Y.; Mahadeokar, J.; et al. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36: 14005--14034

    2023

  23. [31]

    Lee, S.; Cheong, S.; Han, S.; and Shin, J. W. 2025. FlowSE: Flow Matching-based Speech Enhancement. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5. IEEE

    2025

  24. [32]

    Lemercier, J.-M.; Richter, J.; Welker, S.; and Gerkmann, T. 2023. Storm: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31: 2724--2737

    2023

  25. [33]

    Lipman, Y.; Chen, R. T. Q.; Ben-Hamu, H.; Nickel, M.; and Le, M. 2023. Flow Matching for Generative Modeling. arXiv:2210.02747

    2023 arXiv

  26. [34]

    D.; and Wang, W

    Liu, X.; Kong, Q.; Zhao, Y.; Liu, H.; Yuan, Y.; Liu, Y.; Xia, R.; Wang, Y.; Plumbley, M. D.; and Wang, W. 2024. Separate anything you describe. IEEE/ACM Transactions on Audio, Speech, and Language Processing

    2024

  27. [35]

    D.; and Wang, W

    Liu, X.; Liu, H.; Kong, Q.; Mei, X.; Zhao, J.; Huang, Q.; Plumbley, M. D.; and Wang, W. 2022. Separate What You Describe: Language-Queried Audio Source Separation. arXiv:2203.15147

    2022 arXiv

  28. [36]

    Luo, Y.; and Mesgarani, N. 2019. Conv-tasnet: Surpassing ideal time--frequency magnitude masking for speech separation. IEEE/ACM transactions on audio, speech, and language processing, 27(8): 1256--1266

    2019

  29. [37]

    Lv, S.; Wang, X.; Sun, S.; Ma, L.; and Xie, L. 2023. DCCRN-KWS: An audio bias based model for noise robust small-footprint keyword spotting. arXiv preprint arXiv:2305.12331

    2023 arXiv

  30. [38]

    Peebles, W.; and Xie, S. 2023. Scalable Diffusion Models with Transformers. arXiv:2212.09748

    2023 arXiv

  31. [39]

    Popov, V.; Vovk, I.; Gogoryan, V.; Sadekova, T.; and Kudinov, M. 2021. Grad-tts: A diffusion probabilistic model for text-to-speech. In International conference on machine learning, 8599--8608. PMLR

    2021

  32. [40]

    Prenger, R.; Valle, R.; and Catanzaro, B. 2019. Waveglow: A flow-based generative network for speech synthesis. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 3617--3621. IEEE

    2019

  33. [41]

    Purin, M.; Sootla, S.; Sponza, M.; Saabas, A.; and Cutler, R. 2022. AECMOS: A speech quality assessment metric for echo impairment. arXiv:2110.03010

    2022 arXiv

  34. [42]

    Reddy, C. K. A.; Gopal, V.; and Cutler, R. 2022. DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors. arXiv:2110.01763

    2022 arXiv

  35. [43]

    Reddy, C. K. A.; Gopal, V.; Cutler, R.; Beyrami, E.; Cheng, R.; Dubey, H.; Matusevych, S.; Aichner, R.; Aazami, A.; Braun, S.; Rana, P.; Srinivasan, S.; and Gehrke, J. 2020. The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Chal...

    2020 arXiv

  36. [44]

    Richter, J.; Welker, S.; Lemercier, J.-M.; Lay, B.; and Gerkmann, T. 2023. Speech enhancement and dereverberation with diffusion-based generative models. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31: 2351--2364

    2023

  37. [45]

    C.; Indenbom, E.; Saabas, A.; P \"a rnamaa, T.; Guzhvin, J.; and Cutler, R

    Ristea, N. C.; Indenbom, E.; Saabas, A.; P \"a rnamaa, T.; Guzhvin, J.; and Cutler, R. 2023. DeepVQE: Real time deep voice quality enhancement for joint acoustic echo cancellation, noise suppression and dereverberation. In Proc. Interspeech 2023, 3819--3823

    2023

  38. [46]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752

    2022 arXiv

  39. [47]

    RVC-Boss. 2024. GPT-SoVITS. https://github.com/RVC-Boss/GPT-SoVITS

    2024

  40. [48]

    Salamon, J.; Jacoby, C.; and Bello, J. P. 2014. A dataset and taxonomy for urban sound research. In Proceedings of the 22nd ACM international conference on Multimedia, 1041--1044

    2014

  41. [49]

    Salimans, T.; and Ho, J. 2022. Progressive Distillation for Fast Sampling of Diffusion Models. arXiv:2202.00512

    2022 arXiv

  42. [50]

    R.; Doucet, A.; and Li, H

    Scheibler, R.; Hershey, J. R.; Doucet, A.; and Li, H. 2025. Source Separation by Flow Matching. arXiv:2505.16119

    2025 arXiv

  43. [51]

    S.; Desiraju, N

    Shetu, S. S.; Desiraju, N. K.; Mack, W.; and Habets, E. A. P. 2024. Align-ULCNet: Towards Low-Complexity and Robust Acoustic Echo and Noise Reduction. arXiv:2410.13620

    2024 arXiv

  44. [52]

    Shi, Y.; Bu, H.; Xu, X.; Zhang, S.; and Li, M. 2020. Aishell-3: A multi-speaker mandarin tts corpus and the baselines. arXiv preprint arXiv:2010.11567

    2020 arXiv

  45. [53]

    Su, J.; Lu, Y.; Pan, S.; Murtadha, A.; Wen, B.; and Liu, Y. 2023. RoFormer: Enhanced Transformer with Rotary Position Embedding. arXiv:2104.09864

    2023 arXiv

  46. [54]

    A.; Weiss, R

    Wang, Q.; Muckenhirn, H.; Wilson, K.; Sridhar, P.; Wu, Z.; Hershey, J.; Saurous, R. A.; Weiss, R. J.; Jia, Y.; and Moreno, I. L. 2018. Voicefilter: Targeted voice separation by speaker-conditioned spectrogram masking. arXiv preprint arXiv:1810.04826

    2018 arXiv

  47. [55]

    Wang, Z.; Liu, Z.; Zhu, X.; Zhu, Y.; Liu, M.; Chen, J.; Xiao, L.; Weng, C.; and Xie, L. 2025. FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching. arXiv preprint arXiv:2505.19476

    2025 arXiv

  48. [56]

    Wang, Z.; Zhu, X.; Zhang, Z.; Lv, Y.; Jiang, N.; Zhao, G.; and Xie, L. 2024. Selm: Speech enhancement using discrete tokens and language models. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 11561--11565. IEEE

    2024

  49. [57]

    R.; McQuinn, E.; Crow, D.; Manilow, E.; and Roux, J

    Wichern, G.; Antognini, J.; Flynn, M.; Zhu, L. R.; McQuinn, E.; Crow, D.; Manilow, E.; and Roux, J. L. 2019. WHAM!: Extending Speech Separation to Noisy Environments. arXiv:1907.01160

    2019 arXiv

  50. [58]

    Yang, D.; Tian, J.; Tan, X.; Huang, R.; Liu, S.; Guo, H.; Chang, X.; Shi, J.; Bian, J.; Zhao, Z.; et al. 2024. Uniaudio: Towards universal audio generation with large language models. In Forty-first International Conference on Machine Learning

    2024

  51. [59]

    D.; and Wang, W

    Yuan, Y.; Liu, X.; Liu, H.; Plumbley, M. D.; and Wang, W. 2025. Flowsep: Language-queried sound separation with rectified flow matching. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1--5. IEEE

    2025

  52. [60]

    Zeghidour, N.; Luebs, A.; Omran, A.; Skoglund, J.; and Tagliasacchi, M. 2021. Soundstream: An end-to-end neural audio codec. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30: 495--507

    2021

  53. [61]

    Zhang, J.; Yang, J.; Fang, Z.; Wang, Y.; Zhang, Z.; Wang, Z.; Fan, F.; and Wu, Z. 2025. Anyenhance: A unified generative model with prompt-guidance and self-critic for voice enhancement. arXiv preprint arXiv:2501.15417

    2025

  54. [62]

    Ziyin, L.; Hartwig, T.; and Ueda, M. 2020. Neural Networks Fail to Learn Periodic Functions and How to Fix It. arXiv:2006.08195

    2020 arXiv

  55. [63]

    Z mol \' kov \'a , K.; Delcroix, M.; Kinoshita, K.; Ochiai, T.; Nakatani, T.; Burget, L.; and C ernock \`y , J. 2019. Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures. IEEE Journal of Selected Topics in Signal Processing, 13(4): 800--814

    2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.