Pith. sign in

REVIEW 4 major objections 4 minor 3 cited by

SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SoloSpeech proposes a fully generative, four-stage pipeline for target speech extraction, claiming state-of-the-art intelligibility and quality on Libri2Mix while also generalizing to unseen noise and real recordings.

desk verdict Solid generative TSE pipeline with a credible core, but the SOTA headline rests on a mixed-provenance table that needs re-evaluation before it can be taken at face value. read the letter →

arxiv 2505.19314 v3 pith:2BJ2QB4F submitted 2025-05-25 eess.AS cs.AIcs.SD

Helin Wang , Jiarui Hai , Dongchao Yang , Chen Chen , Kai Li , Junyi Peng , Thomas Thebaud , Laureano Moro Velazquez
show 2 more authors
Jesus Villalba Najim Dehak
This is my paper
classification eess.AScs.AIcs.SD
keywords targetspeechextractioncocktail-partyeffectgenerativemodellatentdiffusioncascadedpipelinespeakerembedding-freeconditioningsimilarityout-of-domaingeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Target speech extraction pulls one speaker's voice out of a multi-speaker mixture, guided by a short cue utterance from that speaker. SoloSpeech is a fully generative answer to this task: a time-frequency variational autoencoder compresses audio into a compact latent space, a latent diffusion model extracts the target speaker's latents from the mixture latents using cross-attention on the cue latents, and a diffusion corrector cleans up the resulting waveform. The paper claims that this cascade outperforms both discriminative and generative baselines on perceptual quality, naturalness, and intelligibility, with the best numbers in Table I on Libri2Mix (PESQ 1.89, ESTOI 0.78, SI-SNR 11.12 dB, DNSMOS 3.76, WER 0.15, SIM 0.96). It also reports that the same model, trained only on Libri2Mix, transfers without fine-tuning to three out-of-domain noise conditions and two real-world datasets. A sympathetic reader should care because the result challenges the standing assumption that generative TSE cannot match discriminative extractors on reference-based metrics.

What carries the argument

The load-bearing machinery is the combination of latent-space cross-attention conditioning and masked-reference correction. The target extractor is a latent diffusion model built from a diffusion transformer with long skip connections; it predicts the velocity of the reverse diffusion process and reconstructs the target latents from noise given the mixture latents, with the cue latents injected via multi-head cross-attention and adaptive layer norm. Because no fixed-length speaker embedding is used, the conditioning preserves temporal structure and cannot drift out of the VAE latent manifold. The corrector is a single-step diffusion model on complex STFT coefficients; it receives a noised mixture plus a reference whose random continuous span is masked, and it is trained with an SI-SNR loss. The masking ratio, set to 30%, is the main tuning knob that balances reliance on the cue against reliance on the mixture.

What would settle it

Re-run every row of Table I under one shared protocol, using identical mixture and cue files, identical metric implementations, and the same ASR model for WER, and check whether SoloSpeech still holds the best value on all six metrics; any baseline that exceeds PESQ 1.89, ESTOI 0.78, SI-SNR 11.12 dB, DNSMOS 3.76, or WER 0.15, or that beats SIM 0.96, would falsify the unconditional claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that the historical weaknesses of generative target speech extraction are not inherent to generation but consequences of trying to do everything in one model. It reports that decomposing the task into compression, extraction, reconstruction, and correction lets a fully generative system beat discriminative systems on every reported metric. The extractor is deliberately speaker-embedding-free: the cue audio is encoded into the same VAE latent space as the mixture, transformed by a condition transformer, and injected into the diffusion transformer through cross-attention, which the paper argues preserves sequential speaker detail and avoids mismatches between the speaker-embedding space and the latent space. The final corrector is a single-step time-frequency diffusion model trained from scratch with an SI-SNR objective and a randomly masked reference; the masking forces it to use the mixture rather than just enhance the reference, which is what converts it from a speech-enhancement module into a target-extraction module. With both mechanisms in place, Table I puts SoloSpeech ahead of every listed discriminative and generative baseline on PESQ, ESTOI, SI-SNR, DNSMOS, WER, and speaker similarity.

Load-bearing premise

The state-of-the-art claim rests on the comparability of Table I, where some baselines contribute numbers from their original papers while WER is measured with the authors' own ASR setup.

Editorial extensions

If this is right

  • A fully generative TSE pipeline can lead on perceptual quality, naturalness, and intelligibility simultaneously, removing the main argument for sticking with discriminative extractors.
  • The single-step corrector generalizes across extractors: applied to the USEF-TSE and SoloAudio baselines, it improves their SI-SNR, DNSMOS, and WER.
  • Training only on Libri2Mix transfers to unseen noise types (WHAM!, MUSAN, DEMAND) and to real recordings (CHiME-5, RealSEP) without fine-tuning, including mixtures with up to four speakers and 20-second audio.
  • Optimal corrector behavior needs a balance between cue and mixture: a 30% masking ratio beats both 0% and 50%, so the module is genuinely doing target extraction rather than enhancement.
  • Classifier-free guidance is not useful here; generating eight candidates and keeping the one with the highest speaker similarity is the chosen inference-time strategy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Table I lead survives a shared evaluation protocol, the practical consequence is that generative TSE becomes viable for hearing assistance and telephony, where naturalness and robustness to unseen acoustic conditions matter as much as waveform-level fidelity.
  • The paper's WER column is computed with one ASR model for all methods, but several baselines enter with metrics from their original papers; a fully shared pipeline would clarify whether the intelligibility gain is as large as reported.
  • A natural follow-up experiment is cue degradation: progressively shortening, noisy, or partially masked cues to see where cross-attention conditioning breaks, since the method's advantage is claimed to come from preserving sequential cue detail.
  • The eight-candidate selection step suggests a compute-versus-quality trade-off; distilling the selection into the model or learning a lightweight reranker could make the pipeline faster without losing the reported gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents SoloSpeech, a target-speech-extraction system built as a cascade of four generative components: a time-frequency-domain variational autoencoder that compresses waveforms into 50 Hz latent representations; a latent diffusion transformer that extracts target latents from mixture latents using cross-attention on cue latents without a separate speaker embedding; a decoder; and a single-step diffusion corrector trained with an SI-SNR objective on masked reference signals. The authors report on Libri2Mix that SoloSpeech outperforms prior discriminative and generative systems on PESQ, ESTOI, SI-SNR, DNSMOS, WER, and SIM (Table I), and report stronger out-of-domain and real-world generalization than USEF-TSE and SoloAudio (Table II), supported by module ablations in Tables III-V and Figure 5.

Significance. If the reported numbers hold, the paper demonstrates that a fully generative cascade can exceed discriminative TSE systems on both reference-based quality and ASR-derived intelligibility, which would be an important counterexample to the usual generative-versus-discriminative trade-off. The ablation design is a genuine strength: the authors isolate the contribution of the T-F compressor, the latent-space condition, and the corrector, and they show that the corrector transfers to two existing systems. The out-of-domain and real-world evaluation, including a human MOS test, is also valuable. The main caveats are that the state-of-the-art claim in Table I rests on a mix of author-reproduced and externally reported numbers, and the reported SIM is optimized at inference by the same model used to measure it.

major comments (4)
  1. [Section V-D/V-E, Table I] The state-of-the-art claim is not fully established because Table I mixes evaluation protocols. Section V-D states that for DPCCN, DiscreteTSE, and DDTSE the paper uses results reported in external papers, while Section V-E says values are reported 'as presented in their original papers.' The remaining rows are evaluated with the authors' pipeline, including their Whisper-based WER and WavLM-based SIM. Since the margins over DPCCN (0.15 PESQ) and SSL-MHFA (0.13 PESQ, 0.52 dB SI-SNR) are small, protocol differences in cue selection, mixture construction, SNR sampling, or metric versions could change the ranking. Please re-evaluate these baselines under a single common protocol, or restrict the state-of-the-art claim to the reproduced subset.
  2. [Section V-C2/V-B, Table I] The reported SIM=0.96 is directly optimized at inference. Section V-C2 says that eight candidates are generated and the one with the highest WavLM-based speaker similarity is selected, and Section V-B says SIM is computed with the same WavLM-based speaker verification network. Thus the selection criterion is the evaluation metric itself, and the number is not comparable to baseline systems that do not use this selection procedure. Please report SIM and WER for a single candidate as well, or use a selection criterion that is independent of the reported metrics.
  3. [Section V-F4, Figure 5] Figure 5 tunes the corrector masking ratio θ by SI-SNR and WER on Libri2Mix, but the text does not state whether this is the test split or a held-out validation split. If θ=30% was selected using the same test set that produces Tables I-III, the reported gains are optimistically biased. Please specify the split used for this tuning and, if it is the test set, move the selection to a validation set or adjust the reported protocol.
  4. [Table II, MUSAN row] The DNSMOS value listed for USEF-TSE on MUSAN is 0.20, which is outside the plausible operating range of DNSMOS (approximately 1-5). This suggests an error in an externally sourced table entry and reinforces that the non-reproduced numbers in Tables I and II need verification before they can support a state-of-the-art claim.
minor comments (4)
  1. [Introduction and Section VII] There are typos such as 'In this wor' in the Introduction and 'real-word datasets' in Section VII, and the appendix cross-references are broken ('See appendix??for details' in Section IV-B; references to appendix IV-B, IV-C, IV-D in Section IV-A).
  2. [Section IV-D2] The masking augmentation is described only as 'a random continuous span of the reference signal is masked' with ratio θ; the paper should specify whether masking is applied in the waveform or latent domain, how the span boundaries are sampled, and how the mask is handled when the reference is shorter than the requested span.
  3. [Section V-C2] The best-of-8 inference multiplies the diffusion sampling cost by eight, but the paper does not report single-candidate performance or total inference time, so the cost-benefit of this design choice cannot be assessed.
  4. [Section V-A] The sentence 'Libri2Mix includes different speakers and background noises in its training and test sets' is imprecise about the noise source; since noise is drawn from WHAM!, the text should distinguish speaker overlap from noise conditions.

Circularity Check

1 steps flagged · score 3.0 of 10

One evaluation metric (SIM) is selected with the same WavLM model used to score it, but the core quality/intelligibility comparisons are not circular.

  1. fitted input called prediction [Section IV-C (inference candidate selection) and Section V-B (SIM metric definition)]
    "Instead, following [78], we generate 8 candidates and select the one with the highest speaker similarity score 6. ... we measured the cosine similarity between the estimated speech and the ground truth using a pretrained WavLM-based speaker verification network4 [75] to assess speaker similarity (SIM)."

    Footnotes 6 and 4 both point to the same pretrained WavLM speaker-verification model (microsoft/wavlm-base-plus-sv). Let S denote that model's cosine similarity. The paper selects candidate i* = argmax_i S(est_i, gt) during inference and then reports SIM = S(est_{i*}, gt). The reported SIM is therefore by construction the maximum of the same scoring function over the eight generated candidates, rather than an unbiased single-sample evaluation. Baselines that did not use this selection rule are compared against a max-of-8 value, so the SIM comparison is partially circular. This affects only the SIM column; PESQ, ESTOI, SI-SNR, DNSMOS, and WER are not affected by this step.

full rationale

The paper's central claim is an empirical comparison on Libri2Mix, and the components (T-F VAE compressor, latent-diffusion target extractor, single-step corrector) are trained from scratch with stated losses and ablations. The corrector is explicitly trained from scratch rather than copied from Fast-GeCo, and the target extractor is ablated against speaker-embedding and SSL-condition variants, so no main-table result is equivalent to an input by construction. Citations to the authors' own prior work (SoloAudio, DPM-TSE, Fast-GeCo) are used as architectural inspiration or baseline components, but they are not invoked as a uniqueness theorem or as the sole justification for the state-of-the-art claim. The heterogeneous provenance of baseline numbers in Table I (some reproduced, some taken from external papers) is a comparability and correctness concern, not circularity. The one genuinely circular element is the SIM metric: the same WavLM speaker-verification model is used both to select the best of eight generated candidates and to compute the reported SIM, so the SIM value is the maximum of the scoring function over those candidates by construction. This affects only the SIM column and does not undermine the PESQ, ESTOI, SI-SNR, DNSMOS, or WER comparisons.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical benchmark result. The main load-bearing choices are experimental parameters (masking ratio, candidate count, steps, compressor configuration), each justified by sweeps or prior work. The correctness of the pipeline rests on standard diffusion math and on several domain assumptions about latent-space conditioning and masking, which are tested only indirectly through the reported metrics.

free parameters (5)
  • Reference masking ratio theta = 0.30
    Selected by sweeping masking ratios 0-50% (Figure 5); the optimum at 30% affects both SI-SNR and WER of the final system.
  • Number of inference candidates = 8
    The target extractor draws 8 candidate latent outputs at inference and picks the one with the highest WavLM speaker similarity; the count is taken from [78] without an ablation in this paper.
  • Diffusion inference steps = 50
    The reverse process runs 50 sampling steps during inference versus 1000 training steps, following standard accelerated diffusion; lower step counts trade quality for speed.
  • Audio compressor frame rate and latent dimension = 50 Hz, D=128
    The compressor uses 50 Hz frame rate and latent dimension 128, chosen as a trade-off after sweeping 25/50/100 Hz and 64/128 dims (Table V); this sets the reconstruction ceiling for the whole pipeline.
  • Corrector reverse starting point tau = 0.5
    The corrector's reverse starting point tau = 0.5 is taken from Fast-GeCo [20]; it determines how much of the diffusion process is distilled into the single-step model.
assumptions (5)
  • standard math Zero-terminal-SNR schedule and velocity prediction define a valid reverse diffusion process for latent target extraction.
    Section III restates the DPM framework from [48] and [49]; the paper relies on these as established background without re-deriving them.
  • domain assumption The T-F VAE latent space is a sufficient representation for target speech, and compressor reconstruction quality bounds the final extraction quality.
    Section IV-B and Table IV: the compressor is the first stage of the cascade and the paper measures reconstruction quality but does not prove the bound in general.
  • domain assumption Cross-attention over the cue audio's latent features carries enough speaker identity that explicit speaker embeddings are unnecessary.
    Section IV-C and Table III ablation show it outperforms speaker-embedding variants on the tested data; this is an empirical hypothesis about generalization to unseen speakers.
  • ad hoc to paper Masking a random span of the reference signal during corrector training forces the corrector to use the mixture signal and improves intelligibility.
    Section IV-D2 introduces the masking heuristic; Figure 5 shows an optimum at 30%, but the mechanism is asserted, not derived.
  • domain assumption The Whisper and WavLM models used for scoring give reliable WER and speaker-similarity measurements across systems.
    Section V-B: all WER and SIM scores depend on these two pretrained models; their behavior affects every system, but only the authors' reproduced baselines were measured with this exact setup.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline." pith.science (2026). https://pith.science/paper/2BJ2QB4F

@misc{pith2026250519314,
  author       = {Pith},
  title        = {Pith review of: SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2BJ2QB4F}},
  note         = {Machine review of arXiv:2505.19314}
}
read the original abstract

Target Speech Extraction (TSE) aims to isolate a target speaker's voice from a mixture of multiple speakers by leveraging speaker-specific cues, typically provided as auxiliary audio (a.k.a. cue audio). Although recent advancements in TSE have primarily employed discriminative models that offer high perceptual quality, these models often introduce unwanted artifacts, reduce naturalness, and are sensitive to discrepancies between training and testing environments. On the other hand, generative models for TSE lag in perceptual quality and intelligibility. To address these challenges, we present SoloSpeech, a novel cascaded generative pipeline that integrates compression, extraction, reconstruction, and correction processes. SoloSpeech features a speaker-embedding-free target extractor that utilizes conditional information from the cue audio's latent space, aligning it with the mixture audio's latent space to prevent mismatches. Evaluated on the widely-used Libri2Mix dataset, SoloSpeech achieves the new state-of-the-art intelligibility and quality in target speech extraction while demonstrating exceptional generalization on out-of-domain data and real-world scenarios.

Figures

Figures reproduced from arXiv: 2505.19314 by the authors.

Figure 1
Figure 1. Overall pipeline of SoloSpeech. IV. SOLOSPEECH In this section, we present a cascaded generative pipeline that operates through a sequence of compression, extraction, reconstruction, and correction stages, which is shown in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The audio compressor architecture. The decoder mirrors the encoder structure, reconstructing the audio waveform using the inverse STFT (iSTFT). The VAE model is trained in a generative and adversarial manner [51]. The loss functions include: (i) a perceptually weighted multi￾resolution STFT reconstruction loss [56]; (ii) an adversarial loss term with feature matching, utilizing five convolutional discriminators as i… view at source ↗
Figure 3
Figure 3. Architectures of the target extractor (a), Diffusion Transformer backbone (b) and uDiT block (c). [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Diagrams of Fast-GeCo corrector (a) and SoloSpeech corrector (b). [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Results of different masking ratios (θ) for the corrector. of the conditional features and prevent potential mismatching between VAE latents and other types of features representing the target speech. 3) Impact of the corrector: We investigated the impact of the correc…
Figure 6
Figure 6. Figure 6: Comparison of the spectrograms of the ground truth, audio extracted by SoloSpeech, and by USEF-TSE. Rows: Sample I–V. Columns: (a,d,g,j,m) [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Acoustic Prefixes: Persistent Grounding in Serialized Acoustic Memory for LLM-Based Multi-Talker Speech Recognition

    cs.SD 2026-03 accept novelty 6.0 of 10

    Persistent gated residual cross-attention over onset-ordered talker acoustic memory, refined with LoRA, substantially improves LLM-SOT multi-talker ASR especially on three-talker mixtures.

  2. Enroll-on-Wakeup: A First Comparative Study of Target Speech Extraction for Seamless Interaction in Real Noisy Human-Machine Dialogue Scenarios

    eess.AS 2026-02 conditional novelty 6.0 of 10

    Using the wake-up word as enrollment degrades current target-speech-extraction models; TTS cleanup improves perceived quality but not ASR accuracy.

  3. GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model

    eess.AS 2025-12 conditional novelty 5.0 of 10

    A two-stage decoder-only language model with continuous embeddings and UTMOS-based preference fine-tuning reports improved target-speaker-extraction scores on Libri2Mix.

Reference graph

Works this paper leans on

80 extracted references · 62 canonical work pages · cited by 3 Pith papers

  1. [1]

    The cocktail-party problem revisited: early pro- cessing and selection of multi-talker speech,

    A. W. Bronkhorst, “The cocktail-party problem revisited: early pro- cessing and selection of multi-talker speech,”Attention, Perception, & Psychophysics, vol. 77, no. 5, pp. 1465–1487, 2015. 10 (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) (m) (n) (o) Fig. 6. Comparison of the spectrograms of the ground truth, audio extracted by SoloSpeech, and by USEF-...

    work page 2015

  2. [2]

    Neural target speech extraction: An overview,

    K. Zmol´ıkov´a, M. Delcroix, T. Ochiai, K. Kinoshita, J. Cernock ´y, and D. Yu, “Neural target speech extraction: An overview,”IEEE Signal Process. Mag., vol. 40, no. 3, pp. 8–29, 2023

    work page 2023

  3. [3]

    Neural spatial filter: Target speaker speech separation assisted with directional information,

    R. Gu, L. Chen, S. Zhang, J. Zheng, Y . Xu, M. Yu, D. Su, Y . Zou, and D. Yu, “Neural spatial filter: Target speaker speech separation assisted with directional information,” in20th Annual Conference of the International Speech Communication Association, Interspeech 2019, Graz, Austria, September 15-19, 2019, G. Kubin and Z. Kacic, Eds. ISCA, 2019, pp. 4290–4294

    work page 2019

  4. [4]

    Far-field location guided target speech extraction using end-to-end speech recognition objectives,

    A. S. Subramanian, C. Weng, M. Yu, S. Zhang, Y . Xu, S. Watanabe, and D. Yu, “Far-field location guided target speech extraction using end-to-end speech recognition objectives,” in2020 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020. IEEE, 2020, pp. 7299–7303

    work page 2020

  5. [5]

    Looking to listen at the cocktail party: a speaker-independent audio-visual model for speech separation,

    A. Ephrat, I. Mosseri, O. Lang, T. Dekel, K. Wilson, A. Hassidim, W. T. Freeman, and M. Rubinstein, “Looking to listen at the cocktail party: a speaker-independent audio-visual model for speech separation,”ACM Trans. Graph., vol. 37, no. 4, p. 112, 2018

    work page 2018

  6. [6]

    Conceptbeam: Concept driven target speech extraction,

    Y . Ohishi, M. Delcroix, T. Ochiai, S. Araki, D. Takeuchi, D. Niizumi, A. Kimura, N. Harada, and K. Kashino, “Conceptbeam: Concept driven target speech extraction,” inMM ’22: The 30th ACM International Conference on Multimedia, Lisboa, Portugal, October 10 - 14, 2022, J. Magalh ˜aes, A. D. Bimbo, S. Satoh, N. Sebe, X. Alameda-Pineda, Q. Jin, V . Oria, and...

    work page 2022

  7. [7]

    V oicefilter: Targeted voice separation by speaker-conditioned spectrogram masking,

    Q. Wang, H. Muckenhirn, K. W. Wilson, P. Sridhar, Z. Wu, J. R. Hershey, R. A. Saurous, R. J. Weiss, Y . Jia, and I. L ´opez-Moreno, “V oicefilter: Targeted voice separation by speaker-conditioned spectrogram masking,” in20th Annual Conference of the International Speech Communication Association, Interspeech 2019, Graz, Austria, September 15-19, 2019, G. ...

    work page 2019

  8. [8]

    Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,

    K. Zmol ´ıkov´a, M. Delcroix, K. Kinoshita, T. Ochiai, T. Nakatani, L. Burget, and J. Cernock´y, “Speakerbeam: Speaker aware neural network for target speaker extraction in speech mixtures,”IEEE J. Sel. Top. Signal Process., vol. 13, no. 4, pp. 800–814, 2019

    work page 2019

Show all 80 references
  1. [9]

    Wesep: A scalable and flexible toolkit towards generalizable target speaker extraction,

    S. Wang, K. Zhang, S. Lin, J. Li, X. Wang, M. Ge, J. Yu, Y . Qian, and H. Li, “Wesep: A scalable and flexible toolkit towards generalizable target speaker extraction,” in25th Annual Conference of the International Speech Communication Association, Interspeech 2024, Kos, Greece...

    2024

  2. [10]

    Target confusion in end-to-end speaker extraction: Analysis and approaches,

    Z. Zhao, D. Yang, R. Gu, H. Zhang, and Y . Zou, “Target confusion in end-to-end speaker extraction: Analysis and approaches,” in23rd Annual Conference of the International Speech Communication Association, Interspeech 2022, Incheon, Korea, September 18-22, 2022, H. Ko and J. H...

    2022

  3. [11]

    Dpccn: Densely-connected pyramid complex convolutional network for robust speech separation 11 and extraction,

    J. Han, Y . Long, L. Burget, and J. ˇCernock`y, “Dpccn: Densely-connected pyramid complex convolutional network for robust speech separation 11 and extraction,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp...

    2022

  4. [12]

    Improving target sound extraction with timestamp information,

    H. Wang, D. Yang, C. Weng, J. Yu, and Y . Zou, “Improving target sound extraction with timestamp information,” in23rd Annual Conference of the International Speech Communication Association, Interspeech 2022, Incheon, Korea, September 18-22, 2022, H. Ko and J. H. L. Hansen, Ed...

    2022

  5. [13]

    Wesep: A scalable and flexible toolkit towards generalizable target speaker extraction,

    S. Wang, K. Zhang, S. Lin, J. Li, X. Wang, M. Ge, J. Yu, Y . Qian, and H. Li, “Wesep: A scalable and flexible toolkit towards generalizable target speaker extraction,”CoRR, vol. abs/2409.15799, 2024

    2024 arXiv

  6. [14]

    Spex: Multi-scale time domain speaker extraction network,

    C. Xu, W. Rao, E. S. Chng, and H. Li, “Spex: Multi-scale time domain speaker extraction network,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 28, pp. 1370–1384, 2020

    2020

  7. [15]

    Spex+: A complete time domain speaker extraction network,

    M. Ge, C. Xu, L. Wang, E. S. Chng, J. Dang, and H. Li, “Spex+: A complete time domain speaker extraction network,” in21st Annual Conference of the International Speech Communication Association, Interspeech 2020, Virtual Event, Shanghai, China, October 25-29, 2020, H. Meng, B....

    2020

  8. [16]

    X-SEPFORMER: end-to- end speaker extraction network with explicit optimization on speaker confusion,

    K. Liu, Z. Du, X. Wan, and H. Zhou, “X-SEPFORMER: end-to- end speaker extraction network with explicit optimization on speaker confusion,” inIEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023. IEEE, 2023...

    2023

  9. [17]

    X-tf-gridnet: A time-frequency domain target speaker extraction network with adaptive speaker embedding fusion,

    F. Hao, X. Li, and C. Zheng, “X-tf-gridnet: A time-frequency domain target speaker extraction network with adaptive speaker embedding fusion,”Inf. Fusion, vol. 112, p. 102550, 2024

    2024

  10. [18]

    USEF-TSE: universal speaker embedding free target speaker extraction,

    B. Zeng and M. Li, “USEF-TSE: universal speaker embedding free target speaker extraction,”CoRR, vol. abs/2409.02615, 2024

    2024 arXiv

  11. [19]

    Target speech extraction with conditional diffusion model,

    N. Kamo, M. Delcroix, and T. Nakatani, “Target speech extraction with conditional diffusion model,” inINTERSPEECH 2023, 2023, pp. 176– 180

    2023

  12. [20]

    Noise-robust speech separation with fast generative correction,

    H. Wang, J. Villalba, L. Moro-Velazquez, J. Hai, T. Thebaud, and N. De- hak, “Noise-robust speech separation with fast generative correction,” arXiv preprint arXiv:2406.07461, 2024

    2024 arXiv

  13. [21]

    Speech enhancement and dereverberation with diffusion-based generative models,

    J. Richter, S. Welker, J. Lemercier, B. Lay, and T. Gerkmann, “Speech enhancement and dereverberation with diffusion-based generative models,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 31, pp. 2351–2364, 2023

    2023

  14. [22]

    Diffusion-based generative speech source separation,

    R. Scheibler, Y . Ji, S.-W. Chung, J. Byun, S. Choe, and M.-S. Choi, “Diffusion-based generative speech source separation,” inICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  15. [23]

    Generative pre-training for speech with flow matching,

    A. H. Liu, M. Le, A. Vyas, B. Shi, A. Tjandra, and W. Hsu, “Generative pre-training for speech with flow matching,” inThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024

    2024

  16. [24]

    Metis: A foundation speech generation model with masked generative pre-training,

    Y . Wang, J. Zheng, J. Zhang, X. Zhang, H. Liao, and Z. Wu, “Metis: A foundation speech generation model with masked generative pre-training,”CoRR, vol. abs/2502.03128, 2025. [Online]. Available: https://doi.org/10.48550/arXiv.2502.03128

    2025 doi arXiv

  17. [25]

    Soloau- dio: Target sound extraction with language-oriented audio diffusion transformer,

    H. Wang, J. Hai, Y . Lu, K. Thakkar, M. Elhilali, and N. Dehak, “Soloau- dio: Target sound extraction with language-oriented audio diffusion transformer,”CoRR, vol. abs/2409.08425, 2024

    2024 arXiv

  18. [26]

    Separate and diffuse: Using a pretrained diffusion model for improving source separation,

    S. Lutati, E. Nachmani, and L. Wolf, “Separate and diffuse: Using a pretrained diffusion model for improving source separation,”CoRR, vol. abs/2301.10752, 2023

    2023 arXiv

  19. [27]

    Attention is all you need,

    A. Vaswani, “Attention is all you need,”Advances in Neural Information Processing Systems, 2017

    2017

  20. [28]

    Large language model based generative error correction: A challenge and baselines for speech recognition, speaker tagging, and emotion recognition,

    C. H. Yang, T. Park, Y . Gong, Y . Li, Z. Chen, Y . Lin, C. Chen, Y . Hu, K. Dhawan, P. Zelasko, C. Zhang, Y . Chen, Y . Tsao, J. Balam, B. Ginsburg, S. M. Siniscalchi, E. S. Chng, P. Bell, C. Lai, S. Watanabe, and A. Stolcke, “Large language model based generative error corre...

    2024

  21. [29]

    Sepalm: Audio language models are error correctors for robust speech separation,

    Z. Mu, X. Yang, and G. Wang, “Sepalm: Audio language models are error correctors for robust speech separation,”arXiv preprint arXiv:2505.03273, 2025

    2025 arXiv

  22. [30]

    Librimix: An open-source dataset for generalizable speech separation,

    J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, “Librimix: An open-source dataset for generalizable speech separation,” arXiv preprint arXiv:2005.11262, 2020

    2005 arXiv

  23. [31]

    Target speech extraction with conditional diffusion model,

    N. Kamo, M. Delcroix, and T. Nakatani, “Target speech extraction with conditional diffusion model,” in24th Annual Conference of the International Speech Communication Association, Interspeech 2023, Dublin, Ireland, August 20-24, 2023, N. Harte, J. Carson-Berndsen, and G. Jones...

    2023

  24. [32]

    Dpm-tse: A diffusion probabilistic model for target sound extraction,

    J. Hai, H. Wang, D. Yang, K. Thakkar, N. Dehak, and M. Elhilali, “Dpm-tse: A diffusion probabilistic model for target sound extraction,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1196–1200

    2024

  25. [33]

    Diffusion- based generative speech source separation,

    R. Scheibler, Y . Ji, S. Chung, J. Byun, S. Choe, and M. Choi, “Diffusion- based generative speech source separation,” inIEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023. IEEE, 2023, pp. 1–5

    2023

  26. [34]

    Generative speech foundation model pretraining for high-quality speech extraction and restoration,

    P. Ku, A. H. Liu, R. Korostik, S. Huang, S. Fu, and A. Jukic, “Generative speech foundation model pretraining for high-quality speech extraction and restoration,”CoRR, vol. abs/2409.16117, 2024

    2024 arXiv

  27. [35]

    Generation- based target speech extraction with speech discretization and vocoder,

    L. Yu, W. Zhang, C. Du, L. Zhang, Z. Liang, and Y . Qian, “Generation- based target speech extraction with speech discretization and vocoder,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 12 612–12 616

    2024

  28. [36]

    Enhancing intelligibility for generative target speech extraction via joint optimization with target speaker ASR,

    H. Ma, R. Chen, R. Jing, X. Zhang, J. Liu, and X. Li, “Enhancing intelligibility for generative target speech extraction via joint optimization with target speaker ASR,”CoRR, vol. abs/2501.14477, 2025

    2025 arXiv

  29. [37]

    Diffusion-based signal refiner for speech separation,

    M. Hirano, K. Shimada, Y . Koyama, S. Takahashi, and Y . Mitsufuji, “Diffusion-based signal refiner for speech separation,”CoRR, vol. abs/2305.05857, 2023

    2023

  30. [38]

    Storm: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,

    J. Lemercier, J. Richter, S. Welker, and T. Gerkmann, “Storm: A diffusion-based stochastic regeneration model for speech enhancement and dereverberation,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 31, pp. 2724–2737, 2023

    2023

  31. [39]

    Ddtse: Discriminative diffusion model for target speech extraction,

    L. Zhang, L. Y . Qian, L. Yu, H. Wang, H. Yang, S. Liu, L. Zhou, and Y . Qian, “Ddtse: Discriminative diffusion model for target speech extraction,” inIEEE Spoken Language Technology Workshop, 2024

    2024

  32. [40]

    Speaker-aware neural network based beamformer for speaker extraction in speech mixtures,

    K. Zmol´ıkov´a, M. Delcroix, K. Kinoshita, T. Higuchi, A. Ogawa, and T. Nakatani, “Speaker-aware neural network based beamformer for speaker extraction in speech mixtures,” in18th Annual Conference of the International Speech Communication Association, Interspeech 2017, Stockh...

    2017

  33. [41]

    X- vectors: Robust DNN embeddings for speaker recognition,

    D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X- vectors: Robust DNN embeddings for speaker recognition,” in2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada, April 15-20, 2018. IEEE, 2018, pp. ...

    2018

  34. [42]

    Probing self-supervised learning models with target speech extraction,

    J. Peng, M. Delcroix, T. Ochiai, O. Plchot, T. Ashihara, S. Araki, and J. ˇCernock´y, “Probing self-supervised learning models with target speech extraction,” in2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), 2024, pp. 535–539

    2024

  35. [43]

    Target speech extraction with pre-trained self-supervised learning models,

    J. Peng, M. Delcroix, T. Ochiai, O. Plchot, S. Araki, and J. ˇCernock´y, “Target speech extraction with pre-trained self-supervised learning models,” inICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 10 421–10 425

    2024

  36. [44]

    Smma-net: An audio clue-based target speaker extraction network with spectrogram matching and mutual attention,

    Y . Hu, H. Xu, Z. Guo, H. Huang, and L. He, “Smma-net: An audio clue-based target speaker extraction network with spectrogram matching and mutual attention,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, Apri...

    2024

  37. [45]

    Target speaker extraction by directly exploiting contextual information in the time-frequency domain,

    X. Yang, C. Bao, J. Zhou, and X. Chen, “Target speaker extraction by directly exploiting contextual information in the time-frequency domain,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2024, Seoul, Republic of Korea, April 14-19, 2024. ...

    2024

  38. [46]

    Target speaker extraction with ultra-short reference speech by VE-VE framework,

    L. Yang, W. Liu, L. Tan, J. Yang, and H. Moon, “Target speaker extraction with ultra-short reference speech by VE-VE framework,” inIEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023. IEEE, 2023, pp. 1–5

    2023

  39. [47]

    Sef-net: Speaker embedding free target speaker extraction network,

    B. Zeng, H. Suo, Y . Wan, and M. Li, “Sef-net: Speaker embedding free target speaker extraction network,” in24th Annual Conference of the International Speech Communication Association, Interspeech 2023, Dublin, Ireland, August 20-24, 2023, N. Harte, J. Carson-Berndsen, and G....

    2023

  40. [48]

    Common diffusion noise schedules and sample steps are flawed,

    S. Lin, B. Liu, J. Li, and X. Yang, “Common diffusion noise schedules and sample steps are flawed,” inIEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2024, Waikoloa, HI, USA, January 3-8, 2024. IEEE, 2024, pp. 5392–5399

    2024

  41. [49]

    Progressive distillation for fast sampling of diffusion models,

    T. Salimans and J. Ho, “Progressive distillation for fast sampling of diffusion models,” inThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. Open- Review.net, 2022

    2022

  42. [50]

    High- fidelity audio compression with improved RVQGAN,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High- fidelity audio compression with improved RVQGAN,” inAdvances in 12 Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, ...

    2023

  43. [51]

    Stable audio open,

    Z. Evans, J. D. Parker, C. Carr, Z. Zukowski, J. Taylor, and J. Pons, “Stable audio open,”CoRR, vol. abs/2407.14358, 2024

    2024 arXiv

  44. [52]

    Ezaudio: Enhancing text-to-audio generation with efficient diffusion transformer,

    J. Hai, Y . Xu, H. Zhang, C. Li, H. Wang, M. Elhilali, and D. Yu, “Ezaudio: Enhancing text-to-audio generation with efficient diffusion transformer,” arXiv preprint arXiv:2409.10819, 2024

    2024 arXiv

  45. [53]

    Tf- gridnet: Integrating full- and sub-band modeling for speech separation,

    Z. Wang, S. Cornell, S. Choi, Y . Lee, B. Kim, and S. Watanabe, “Tf- gridnet: Integrating full- and sub-band modeling for speech separation,” IEEE ACM Trans. Audio Speech Lang. Process., vol. 31, pp. 3221–3236, 2023

    2023

  46. [54]

    Spmamba: State-space model is all you need in speech separation,

    K. Li and G. Chen, “Spmamba: State-space model is all you need in speech separation,”CoRR, vol. abs/2404.02063, 2024

    2024 arXiv

  47. [55]

    Complex ratio masking for monaural speech separation,

    D. S. Williamson, Y . Wang, and D. Wang, “Complex ratio masking for monaural speech separation,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 24, no. 3, pp. 483–492, 2016

    2016

  48. [56]

    auraloss: Audio focused loss functions in pytorch,

    C. J. Steinmetz and J. D. Reiss, “auraloss: Audio focused loss functions in pytorch,” inDigital music research network one-day workshop (DMRN+ 15), 2020

    2020

  49. [57]

    High fidelity neural audio compression,

    A. D ´efossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,”Trans. Mach. Learn. Res., vol. 2023, 2023

    2023

  50. [58]

    Scalable diffusion models with transformers,

    W. Peebles and S. Xie, “Scalable diffusion models with transformers,” inIEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023. IEEE, 2023, pp. 4172–4182

    2023

  51. [59]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in9th International Conference on Learn...

    2021

  52. [60]

    Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,

    S. Elfwing, E. Uchibe, and K. Doya, “Sigmoid-weighted linear units for neural network function approximation in reinforcement learning,” Neural Networks, vol. 107, pp. 3–11, 2018

    2018

  53. [61]

    Roformer: Enhanced transformer with rotary position embedding,

    J. Su, M. H. M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: Enhanced transformer with rotary position embedding,”Neurocomputing, vol. 568, p. 127063, 2024

    2024

  54. [62]

    Single- channel multi-speaker separation using deep clustering,

    Y . Z. Isik, J. L. Roux, Z. Chen, S. Watanabe, and J. R. Hershey, “Single- channel multi-speaker separation using deep clustering,” in17th Annual Conference of the International Speech Communication Association, Interspeech 2016, San Francisco, CA, USA, September 8-12, 2016, N...

    2016

  55. [63]

    Conv-tasnet: Surpassing ideal time-frequency magnitude masking for speech separation,

    Y . Luo and N. Mesgarani, “Conv-tasnet: Surpassing ideal time-frequency magnitude masking for speech separation,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 27, no. 8, pp. 1256–1266, 2019

    2019

  56. [64]

    Wham!: Extending speech separation to noisy environments,

    G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. L. Roux, “Wham!: Extending speech separation to noisy environments,” in20th Annual Conference of the International Speech Communication Association, Interspeech 2019, Graz, Austria, Septembe...

    2019

  57. [65]

    Librispeech: An ASR corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in2015 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2015, South Brisbane, Queensland, Australia, April 19-24, 2015. IEEE, 20...

    2015

  58. [66]

    Improving speaker discrimination of target speech extraction with time-domain speakerbeam,

    M. Delcroix, T. Ochiai, K. Zmolikova, K. Kinoshita, N. Tawara, T. Nakatani, and S. Araki, “Improving speaker discrimination of target speech extraction with time-domain speakerbeam,” inICASSP 2020- 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (...

    2020

  59. [67]

    MUSAN: A music, speech, and noise corpus,

    D. Snyder, G. Chen, and D. Povey, “MUSAN: A music, speech, and noise corpus,”CoRR, vol. abs/1510.08484, 2015

    2015 arXiv

  60. [68]

    Multichannel audio database in various acoustic environments,

    E. Hadad, F. Heese, P. Vary, and S. Gannot, “Multichannel audio database in various acoustic environments,” in14th International Workshop on Acoustic Signal Enhancement, IWAENC 2014, Juan-les-Pins, France, September 8-11, 2014. IEEE, 2014, pp. 313–317

    2014

  61. [69]

    The fifth ’chime’ speech separation and recognition challenge: Dataset, task and baselines,

    J. Barker, S. Watanabe, E. Vincent, and J. Trmal, “The fifth ’chime’ speech separation and recognition challenge: Dataset, task and baselines,” in19th Annual Conference of the International Speech Communication Association, Interspeech 2018, Hyderabad, India, September 2-6, 20...

    2018

  62. [70]

    Sonicsim: A customizable simulation platform for speech processing in moving sound source scenarios,

    K. Li, W. Sang, C. Zeng, R. Yang, G. Chen, and X. Hu, “Sonicsim: A customizable simulation platform for speech processing in moving sound source scenarios,”CoRR, vol. abs/2410.01481, 2024

    2024 arXiv

  63. [71]

    Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,

    A. W. Rix, J. G. Beerends, M. P. Hollier, and A. P. Hekstra, “Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs,” inIEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2001, ...

    2001

  64. [72]

    An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,

    J. Jensen and C. H. Taal, “An algorithm for predicting the intelligibility of speech masked by modulated noise maskers,”IEEE ACM Trans. Audio Speech Lang. Process., vol. 24, no. 11, pp. 2009–2022, 2016

    2009

  65. [73]

    Dnsmos P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,

    C. K. A. Reddy, V . Gopal, and R. Cutler, “Dnsmos P.835: A non- intrusive perceptual objective speech quality metric to evaluate noise suppressors,” inIEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singapore, 23-27 May

    2022

  66. [74]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, ser. Proceedings of Machine Learning Resea...

    2023

  67. [75]

    Wavlm: Large-scale self-supervised pre- training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “Wavlm: Large-scale self-supervised pre- training for full stack speech processing,”IEEE J. Sel. Top. Signa...

    2022

  68. [76]

    Improving generalization of speech separation in real-world scenarios: Strategies in simulation, optimization, and evaluation,

    K. Chen, J. Su, T. Berg-Kirkpatrick, S. Dubnov, and Z. Jin, “Improving generalization of speech separation in real-world scenarios: Strategies in simulation, optimization, and evaluation,”CoRR, vol. abs/2408.16126, 2024

    2024 arXiv

  69. [77]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,”CoRR, vol. abs/2207.12598, 2022

    2022 arXiv

  70. [78]

    Photorealistic text-to-image diffusion models with deep language understanding,

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, S. K. S. Ghasemipour, R. Gontijo-Lopes, B. K. Ayan, T. Salimans, J. Ho, D. J. Fleet, and M. Norouzi, “Photorealistic text-to-image diffusion models with deep language understanding,” inAdvances in Neural Information P...

  71. [80]

    Available: https://openreview.net/forum?id=08Yk-n5l2Al

    [Online]. Available: https://openreview.net/forum?id=08Yk-n5l2Al

  72. [2022]

    IEEE, 2022, pp. 886–890

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.