Attractor-Based Speech Separation of Multiple Utterances by Unknown Number of Speakers

Archontis Politis; Konstantinos Drossos; Tuomas Virtanen; Yuzhu Wang

REVIEW 4 major objections 5 minor 1 cited by

Attractor-Based Speech Separation of Multiple Utterances by Unknown Number of Speakers

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single model separates overlapping speech, counts the speakers, and tracks who speaks when.

desk verdict Solid fixed-count separation results, but the unknown-count claim is undermined by a train/inference mismatch in the attractor module that the paper never addresses. read the letter →

arxiv 2505.16607 v1 pith:GWSCCT2P submitted 2025-05-22 eess.AS cs.SD

Yuzhu Wang , Archontis Politis , Konstantinos Drossos , Tuomas Virtanen This is my paper · ORCID

classification eess.AScs.SD

keywords speechseparationspeakercountingdiarizationattractornetworksmulti-utterancesingle-channelunknownnumberofspeakersFiLMconditioning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper takes on a version of the cocktail-party problem that most separation systems avoid: a recording can contain an unknown number of speakers, and each speaker can talk several times, with utterances overlapping unevenly. The authors propose A-DCSS, one neural model that simultaneously separates the mixture into one output stream per speaker, estimates how many speakers are present, and produces per-speaker activity labels over time. Their experiments on synthesized two- to three-speaker mixtures, under anechoic, noisy, reverberant, and noisy-plus-reverberant conditions, report higher separation quality than the compared baselines in both fixed-count and varying-count settings. The point of the claim is that all three tasks reinforce each other: knowing who is active when helps pull that speaker's scattered utterances into a single clean stream, and the system is designed for cases where no one tells it the speaker count in advance.

What carries the argument

The load-bearing object is the RNN attractor module: a bidirectional LSTM encoder reads the mixture embedding and an LSTM decoder emits $J+1$ attractor vectors, the last one representing the absence of a speaker. Existence probabilities for these attractors give the speaker count; inner products between the first $J$ attractors and the embedding are sigmoid-mapped into per-frame activity probabilities; and the same attractors condition the separator through FiLM (feature-wise linear modulation), which shifts and scales the features according to which speaker is being extracted. This module sits between a dual-path transformer embedding that captures local and global temporal structure and a triple-path separator whose extra path runs across the estimated speaker dimension.

What would settle it

Present the trained system with a real single-channel recording containing four or more speakers over a minute or more; if the estimated speaker count sticks at the preset maximum or the separated streams blur utterances from different speakers into one output, the central generalization claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that an attractor-based architecture can take a single-channel mixture of several multi-utterance speakers and produce the right number of separated streams, each containing all utterances of one speaker, while also outputting that speaker's activity pattern. In A-DCSS the attractor module is a BLSTM encoder-decoder that emits $J+1$ attractor vectors: $J$ candidate speakers plus an explicit "no speaker" vector. The estimated speaker count $\hat{C}$ is the number of attractors whose existence probability passes a threshold, the per-frame activity matrix $\hat{P}$ comes from matching attractors against the mixture embedding, and the same attractors are fused into the separator with feature-wise linear modulation so that each output stream is tied to an identified speaker. On the paper's synthesized corpus the system reports the best separation quality among the compared methods, the lowest diarization error against the joint baseline, and counting accuracy close to the best, with the ablations showing that the diarization branch is what raises separation quality by roughly 1.1 to 1.5 dB.

Load-bearing premise

The whole evaluation rests on synthetic two-to-three-speaker mixtures cut to ten-second segments, and the model is built around a preset maximum speaker count whose value the paper never reports, so its performance with more speakers, longer recordings, or real room conditions is not established.

Editorial extensions

If this is right

A single-channel recording with overlapping multi-utterance speech can be converted directly into as many clean speaker streams as the recording contains, without the number of speakers being supplied in advance.
Because the activity branch improves separation, systems that only separate or only diarize are leaving useful cross-task signal on the table; a joint objective is the direct corollary of the reported ablation results.
The reported ranking of attractor designs--RNN over transformer for separation, transformer over RNN for counting--gives future work a concrete design choice to test rather than assume.
The method transfers from a fixed two-speaker setting to a varying two-to-three-speaker setting with modest degradation, so the same architecture can be trained in two phases for increasingly open conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Extension the paper does not make: because counting and activity both come from thresholded attractor outputs, the architecture could be adapted to streaming or long-form recordings; the paper's ten-second segments do not test how the thresholds or the LSTM states behave over minutes.
Untested pressure point: the model's ceiling is the preset $J_{\max}$ on the number of attractors, and the paper never reports its value; a recording with more speakers than $J_{\max}$ would force the count estimate to saturate.
A natural next experiment would be to train on two-to-three-speaker mixtures and evaluate on four-to-five-speaker mixtures; the paper's counting generalization argument depends on that gap closing, which is not shown.
If the diarization-helping-separation finding holds, then deliberately corrupting or removing activity labels at test time should measurably degrade separation, giving an inexpensive causal check of the claimed mechanism.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The paper is a reasonable step in single-channel separation for unknown speaker counts, but the unknown-count claim has a real hole. The architecture (dual-path encoder, RNN attractor, FiLM, triple-path separator) is a sensible combination of known parts, and the fixed-count results on 2-speaker mixtures are consistently strong: A-DCSS beats SepEDA by about 1 dB in anechoic and noisy-reverb conditions. The ablation showing that adding diarization output improves separation by 1.1-1.5 dB is a useful empirical point. Credit also for retraining all baselines on the same synthetic multi-utterance dataset.

The soft spot is the one the stress-test note flags, and I think it lands. Section 3 says training uses J = true number of speakers C, but inference generates J_max+1 attractors. So for a test mixture with fewer than J_max speakers, the decoder is unrolled to a length it never saw for that C, and the attractor slots beyond C+1 receive no existence, diarization, or separation supervision at all. There is no training signal forcing their existence probabilities below threshold. Table 2 pools 2- and 3-speaker mixtures, so you cannot tell whether the reported SCA and delta-SI-SDR are carried by the C=3 subset. J_max is never stated, so the size of the mismatch is unknown. That is not a generalization limitation; it is a mismatch between the training objective and the inference procedure for the variable-count case. The fix is straightforward—train with a fixed J_max and pad existence labels with zeros—but as it stands, the dynamic counting contribution is not established by the experiments.

Other issues are smaller. No error bars or significance tests, so the gaps over baselines could be noise, though the consistent ordering helps. The dataset is synthetic, 2-3 speakers, 10-second segments, so real-world reach is unproven. The novelty claim is a bit strong: combining known attractor and transformer blocks is incremental, but the multi-utterance scenario and the joint objective are new enough.

This paper deserves peer review—the problem is real, the architecture is plausible, and the main flaw is fixable with a training change and a more careful evaluation. I would send it out, but with a strong request to address the J_max mismatch, report J_max, and add per-C counts or error bars before acceptance.

Recommendation: engage, but expect major revision.

Referee Report

4 major / 5 minor

Summary. The paper proposes A-DCSS, a single-channel system that jointly performs speech separation, speaker counting, and speaker activity detection for mixtures in which an unknown number of speakers each contribute multiple utterances. The architecture uses an encoder–decoder with a dual-path transformer embedding, a BLSTM-based attractor module that outputs attractors plus a non-existence attractor, and a triple-path separator conditioned on the attractors via FiLM. Training uses the true number of speakers as the number of attractors, while inference uses a preset maximum J_max. The system is evaluated on synthesized 2- and 3-speaker mixtures derived from LibriSpeech and WHAM! noise under anechoic, noisy, reverberant, and noisy-plus-reverberant conditions, against retrained baselines including Recursive-SS, EEND-SS, SepEDA, and SepTDA. Results show the proposed method achieves the best delta-SI-SDR in all conditions, competitive DER, and second-best speaker counting accuracy (SCA). An ablation study indicates that both the attractor module and the diarization branch contribute to separation quality.

Significance. If the reported results hold, the paper contributes a useful integration of attractor-based counting, diarization, and separation for multi-utterance recordings, and it provides a synthetic benchmark that other researchers can reuse. The strengths are that all baselines are retrained on the same data, the evaluation covers four acoustic conditions, and the ablation experiments are internally consistent with the proposed design. The main weakness is that the central unknown-count claim rests on an inference procedure that is not fully matched to training: attractor positions beyond the training-time source count receive no direct supervision, J_max is never reported, and the pooling of 2- and 3-speaker results in Table 2 prevents the reader from assessing behavior on the C=2 subset where the mismatch is largest. In addition, no error bars or statistical tests are provided, so differences of 0.1-1 dB between systems cannot be judged for significance. The work is a solid empirical contribution in need of revision rather than a definitive demonstration of state-of-the-art unknown-number multi-utterance separation.

major comments (4)

[Section 3 (Attractor module) and Section 5.2, Table 2] The training/inference mismatch for the unknown-count branch is load-bearing. Training always sets J to the true number C and generates C+1 attractors, with existence targets q=[1,...,1,0] of length C+1. Inference generates J_max+1 attractors, so for any test mixture with C<J_max there are attractor positions that were never trained with the corresponding sequence length and target pattern. In particular, the claim that the system 'dynamically estimates the number of speakers' depends on existence probabilities for these untrained slots, which have no training signal forcing them below tau_exist. Because J_max is never reported and Table 2 pools 2- and 3-speaker mixtures, the reader cannot tell how large the mismatch is or whether the C=2 subset drives the reported SCA/DER/delta-SI-SDR. Please report J_max, report results per number of speakers, and either train with the same J_max+1 output length (e.g., by padding training targets and masking the excess positions) or otherwise show that the untrained positions behave correctly.
[Section 5.1, Tables 1-3] No error bars, confidence intervals, or statistical significance tests are reported. Several claimed improvements are small (e.g., 0.2-1.0 dB in Table 1 and 0.3-0.6 dB SCA differences in Table 2), and without variance estimates it is not possible to determine whether the proposed system is actually better than SepEDA or SepTDA in fixed-count conditions or second-best in counting accuracy. Please report results over multiple seeds or bootstrap confidence intervals, and if possible a paired test for the main comparisons.
[Section 4.1 and Section 5.2] The unknown-number scenario is evaluated only on mixtures of 2-3 speakers, 1-5 utterances, silences up to 3 s, RT60 0.2-0.6 s, and 10-second segments. Since J_max is not reported, it is unclear even what 'maximum possible number of sources' is assumed. The claim of handling 'an unknown number of speakers' is therefore limited to a narrow range, and the paper should either test more speakers and longer recordings or explicitly scope the claim. The 22.5% overlap ratio is given only for the 2-speaker case; please report it for the 3-speaker case as well.
[Section 5.3, Table 3] The ablation study is conducted only on the two-speaker anechoic dataset. The conclusion that the diarization branch 'significantly improves separation performance' is supported only for that condition; the contribution statement in the introduction claims this importance generally, but no ablations are shown for noisy, reverberant, or three-speaker conditions. Please either extend the ablation or soften the claim.

minor comments (5)

[Section 3 (Loss Function)] The loss text says 'lambda_s, lambda_d, and lambda_d denote the weights' but the third weight is lambda_e; please correct this typo.
[Section 4.3] SCA is cited to [36] but never defined in the text; a one-sentence definition would improve readability.
[Table 2] DER is reported only for EEND-SS and A-DCSS; for Recursive-SS, SepEDA, and SepTDA the table uses '–'. Please state whether these baselines do not produce diarization outputs or whether the metric was not evaluated.
[Section 4.2] The two-phase training (first two-speaker, then varying) is described, but the duration of each phase and the criterion for switching phases are not specified; please clarify.
[References] Reference [13] contains a garbled author string ('N. G. Naoya Takahashi, Sudarsanam Parthasaarathy & Y. Mitsufuji'); please verify the author list.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claims rest on held-out evaluation against external baselines and independent components.

full rationale

The paper's derivation chain is empirical rather than circular. The proposed A-DCSS is trained with a joint loss (SI-SDR, diarization BCE, and existence BCE) on synthesized mixtures, and the reported claims of separation quality, counting accuracy, and diarization accuracy are measured on held-out test sets whose speakers, noise signals, and rooms do not overlap with training. The baselines (Conv-TasNet, Sepformer, Recursive-SS, EEND-SS, SepEDA, SepTDA) are external prior systems, and the ablation study compares architectural variants under fixed data and hyperparameters. No predicted quantity in the evaluation is defined in terms of a fitted parameter from that same evaluation, and no load-bearing uniqueness theorem or result is imported solely from the present authors' prior work. The self-citations present (e.g., refs [1], [2], [16]) are ordinary methodological references and do not carry the central claim. The reviewer-flagged train/inference mismatch regarding J_max is a potential generalization or evaluation weakness, but it is not circularity: it does not make any reported result true by construction, and the unknown-count evaluation still uses held-out mixtures with ground-truth counts. The paper's honest limitations are not hidden by a self-referential argument. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on several domain assumptions: the synthetic dataset is representative of real-world multi-speaker multi-utterance mixtures; the maximum speaker count J_max is fixed and sufficient; 10-second training segments generalize to the full recordings; and binary activity labels can be learned via BCE with PIT. No invented physical entities are introduced. The main free parameters are loss weights, thresholds, and architectural hyperparameters tuned on the validation set, plus the unreported J_max.

free parameters (4)

Loss weights (lambda_s, lambda_d, lambda_e) = 0.8, 0.1, 0.1
Chosen on the validation set; they trade off reconstruction, diarization, and existence losses and influence counting and separation outcomes (Section 4.2).
Thresholds (tau_exist, tau_diar) = 0.5, 0.5
Binarize attractor existence and speaker activity predictions; set by hand based on validation experiments (Section 4.2).
Architectural hyperparameters (kernel L=16, stride 8, D=F=256, chunk K=96, heads 4, N_triple=6) = as listed
Selected through validation experiments; they define model capacity and directly affect the reported SI-SDR, DER, and SCA numbers (Section 4.2).
Maximum source count J_max = not reported
Bounding the number of generated attractors at inference; the paper states J_max+1 attractors are generated but never states J_max, so counting capacity is undefined (Section 3).

assumptions (4)

domain assumption The synthesized mixtures from LibriSpeech and WHAM! noise, with 2-3 speakers, 1-5 utterances per speaker, silences up to 3 s, RT60 0.2-0.6 s, and SNR 0-10 dB, are representative of realistic multi-speaker multi-utterance recordings.
Section 4.1 defines the dataset; Section 6 concludes robustness in 'realistic scenarios' without evaluation on real recordings.
ad hoc to paper During inference, the true number of speakers never exceeds the preset maximum J_max, and generating J_max+1 attractors is sufficient.
Section 3, Attractor module: the model can output at most J_max speakers; the value of J_max is never disclosed or varied.
domain assumption Training on 10-second segments transfers to the full multi-utterance test mixtures, which can be longer.
Section 4.2 states each training sample is split into 10-second segments; no experiment verifies behavior on full-length inputs.
domain assumption A single binary cross-entropy loss with permutation invariant training can correctly align attractors to speakers over time, enabling joint diarization and separation.
Section 3, Loss Function: the diarization loss (Eq. 2) assumes frame-level speaker activities are sufficient supervision for speaker separation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Attractor-Based Speech Separation of Multiple Utterances by Unknown Number of Speakers." pith.science (2026). https://pith.science/paper/GWSCCT2P

@misc{pith2026250516607,
  author       = {Pith},
  title        = {Pith review of: Attractor-Based Speech Separation of Multiple Utterances by Unknown Number of Speakers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GWSCCT2P}},
  note         = {Machine review of arXiv:2505.16607}
}

read the original abstract

This paper addresses the problem of single-channel speech separation, where the number of speakers is unknown, and each speaker may speak multiple utterances. We propose a speech separation model that simultaneously performs separation, dynamically estimates the number of speakers, and detects individual speaker activities by integrating an attractor module. The proposed system outperforms existing methods by introducing an attractor-based architecture that effectively combines local and global temporal modeling for multi-utterance scenarios. To evaluate the method in reverberant and noisy conditions, a multi-speaker multi-utterance dataset was synthesized by combining Librispeech speech signals with WHAM! noise signals. The results demonstrate that the proposed system accurately estimates the number of sources. The system effectively detects source activities and separates the corresponding utterances into correct outputs in both known and unknown source count scenarios.

Figures

Figures reproduced from arXiv: 2505.16607 by the authors.

**Figure 1.** The illustration of the A-DCSS in multi-speaker with multi-utterance scenarios. The aforementioned studies focus only on single-utterance scenarios. For situations with an unknown number of speakers and multiple utterances, several methods have shown potential effectiveness, including recurrent attention networks [23], spatial filtering [24], and dual-path model consisting of LSTM and transformers [25]. However, m… view at source ↗

**Figure 2.** A-DCSS architecture. Here, the observed signal y consists of speech signals xc from C sources and noise n. The signal xc from the c-th speaker, consists of multiple utterances and potential pauses between them. The speaker counting task involves estimating the number of speakers C. We assume that by extracting features from the time-domain signals, we can predict binary speaker activities Pˆ of size C × T, where T… view at source ↗

**Figure 4.** Attractor module. speech sequence modeling [10], comprises a multi-head attention layer, an LSTM layer, and a linear layer. Furthermore, the three-path module is stacked Ntriple times, which enhances the capacity to capture complex patterns. The output of the triple-path module Tout ∈ R J×K×S×D is first processed by overlap-add to obtain T ′ out ∈ R J×T ×D. Then, a linear output layer transforms the feature dimensi… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Multi-Utterance Speech Separation and Association Trained on Short Segments
eess.AS 2025-07 conditional novelty 6.0 of 10

A lightweight recurrent network trained on 10-second segments can directly process 21 to 121 second mixtures and keep each speaker's utterances in a consistent output stream across silences up to about 40 seconds.

Reference graph

Works this paper leans on

42 extracted references · 34 canonical work pages · cited by 1 Pith paper

[1]

Introduction Speech separation aims to isolate individual speech signals from mixtures containing multiple speakers. This task is particularly challenging in real-world environments due to noise, reverber- ation, and the highly time-varying nature of source activities, especially in single-channel scenarios [1, 2, 3, 4]. Most existing research assumes a k...

work page
[2]

Here, the observed signalyconsists of speech signalsx c from Csources and noisen

Signal Model The observed signalyof lengthT ∗ can be represented as y= CX c=1 xc +n.(1) arXiv:2505.16607v1 [eess.AS] 22 May 2025 Figure 2:A-DCSS architecture. Here, the observed signalyconsists of speech signalsx c from Csources and noisen. The signalx c from thec-th speaker, consists of multiple utterances and potential pauses between them. The speaker c...

arXiv 2025
[3]

The system architecture is illustrated in Fig

Attractor-based joint diarization, counting, and separation system (A-DCSS) The proposed joint separation system consists of three core pro- cesses: feature embedding, attractor generation, and speech separation. The system architecture is illustrated in Fig. 2. An encoder and a decoder are used to transform the signal between the time domain and the time...

work page
[4]

Datasets In our research scenario, multiple speakers coexist in a mixed signal, each contributing multiple utterances

Experimental settings 4.1. Datasets In our research scenario, multiple speakers coexist in a mixed signal, each contributing multiple utterances. To synthesize data consistent with this scenario, we use speech from the Lib- riSpeech corpus [30], which provides the diarization labels for model training and evaluation. Noise signals are from the WHAM! datas...

work page
[5]

Besides, an ablation study was conducted Table 2:Evaluation on 2-speaker & 3-speaker dataset

Experimental Results We evaluated the performance of the A-DCSS in two scenarios: one with a fixed number of speakers and the other with a varying number of speakers. Besides, an ablation study was conducted Table 2:Evaluation on 2-speaker & 3-speaker dataset. DER (%)(↓) is diarization error rate. SCA(%)(↑) is speaker counting accuracy. Anechoic Noisy Rev...

work page
[6]

The system employs an RNN attractor module to estimate the number of sources and detect source ac- tivity, resulting in significant improvements in separation per- formance

Conclusions In this paper, we proposed a novel single-channel joint speech separation system that effectively handles scenarios with an un- known number of speakers, where each speaker may contribute multiple utterances. The system employs an RNN attractor module to estimate the number of sources and detect source ac- tivity, resulting in significant impr...

work page
[7]

Vincent, T

E. Vincent, T. Virtanen, and S. Gannot,Audio Source Separation and Speech Enhancement. John Wiley & Sons, 2018

work page 2018
[8]

Deep learning for audio signal processing,

H. Purwins, B. Li, T. Virtanen, J. Schl ¨uter, S.-Y . Chang, and T. Sainath, “Deep learning for audio signal processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 13, no. 2, pp. 206–219, 2019

work page 2019

Show all 42 references

[9]

An overview of machine learning and other data-based methods for spatial au- dio capture, processing, and reproduction,

M. Cobos, J. Ahrens, K. Kowalczyk, and A. Politis, “An overview of machine learning and other data-based methods for spatial au- dio capture, processing, and reproduction,”EURASIP Journal on Audio, Speech, and Music Processing, vol. 2022, no. 1, p. 10, 2022

2022
[10]

Supervised speech separation based on deep learning: An overview,

D. Wang and J. Chen, “Supervised speech separation based on deep learning: An overview,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no. 10, pp. 1702–1726, 2018

2018
[11]

Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,

Y . Luo and N. Mesgarani, “Conv-tasnet: Surpassing ideal time– frequency magnitude masking for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 27, no. 8, pp. 1256– 1266, 2019

2019
[12]

Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech sepa- ration,

Y . Luo, Z. Chen, and T. Yoshioka, “Dual-Path RNN: Efficient long sequence modeling for time-domain single-channel speech sepa- ration,” inProc. ICASSP, 2020, pp. 46–50

2020
[13]

Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,

D. Yu, M. Kolbæk, Z.-H. Tan, and J. Jensen, “Permutation invari- ant training of deep models for speaker-independent multi-talker speech separation,” inProc. ICASSP, 2017, pp. 241–245

2017
[14]

Multitalker speech separation with utterance-level permutation invariant train- ing of deep recurrent neural networks,

M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, “Multitalker speech separation with utterance-level permutation invariant train- ing of deep recurrent neural networks,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 25, no. 10, pp. 1901–1913, 2017

1901
[15]

Attention is all you need in speech separation,

C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, and J. Zhong, “Attention is all you need in speech separation,” inProc. ICASSP, 2021, pp. 21–25

2021
[16]

Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,

J. Chen, Q. Mao, and D. Liu, “Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation,” inProc. Interspeech, 2020, pp. 2642–2646

2020
[17]

TF-GridNet: Making time-frequency domain models great again for monaural speaker separation,

Z.-Q. Wang, S. Cornell, S. Choi, Y . Lee, B.-Y . Kim, and S. Watan- abe, “TF-GridNet: Making time-frequency domain models great again for monaural speaker separation,” inProc. ICASSP, 2023, pp. 1–5

2023
[18]

TF-GridNet: Integrating full- and sub-band modeling for speech separation,

——, “TF-GridNet: Integrating full- and sub-band modeling for speech separation,”IEEE/ACM Trans. Audio, Speech, Lang. Pro- cess., vol. 31, pp. 3221–3236, 2023

2023
[19]

Recursive speech separation for unknown number of speak- ers,

N. G. Naoya Takahashi, Sudarsanam Parthasaarathy and Y . Mitsu- fuji, “Recursive speech separation for unknown number of speak- ers,” inProc. Interspeech, 2019, pp. 1348–1352

2019
[20]

Listen- ing to each speaker one by one with recurrent selective hearing networks,

K. Kinoshita, L. Drude, M. Delcroix, and T. Nakatani, “Listen- ing to each speaker one by one with recurrent selective hearing networks,” inProc. ICASSP, 2018, pp. 5064–5068

2018
[21]

Deep Clus- tering: Discriminative embeddings for segmentation and separa- tion,

J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep Clus- tering: Discriminative embeddings for segmentation and separa- tion,” inProc. ICASSP, 2016, pp. 31–35

2016
[22]

Low-latency deep clus- tering for speech separation,

S. Wang, G. Naithani, and T. Virtanen, “Low-latency deep clus- tering for speech separation,” inProc. ICASSP, 2019, pp. 76–80

2019
[23]

Wavesplit: End-to-end speech separation by speaker clustering,

N. Zeghidour and D. Grangier, “Wavesplit: End-to-end speech separation by speaker clustering,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 29, pp. 2840–2849, 2021

2021
[24]

Deep attractor network for single-microphone speaker separation,

Z. Chen, Y . Luo, and N. Mesgarani, “Deep attractor network for single-microphone speaker separation,” inProc. ICASSP, 2017, pp. 246–250

2017
[25]

Speaker-independent speech separation with deep attractor network,

Y . Luo, Z. Chen, and N. Mesgarani, “Speaker-independent speech separation with deep attractor network,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 26, no. 4, pp. 787–796, 2018

2018
[26]

EEND-SS: Joint end-to-end neural speaker diariza- tion and speech separation for flexible number of speakers,

S. Maiti, Y . Ueda, S. Watanabe, C. Zhang, M. Yu, S.-X. Zhang, and Y . Xu, “EEND-SS: Joint end-to-end neural speaker diariza- tion and speech separation for flexible number of speakers,” in Proc. SLT, 2023, pp. 480–487

2023
[27]

Speech separation for an unknown number of speakers using transformers with encoder- decoder attractors,

S. R. Chetupalli and E. A. P. Habets, “Speech separation for an unknown number of speakers using transformers with encoder- decoder attractors,” inProc. Interspeech, 2022, pp. 5393–5397

2022
[28]

Boosting unknown-number speaker separation with transformer decoder-based attractor,

Y . Lee, S. Choi, B.-Y . Kim, Z.-Q. Wang, and S. Watanabe, “Boosting unknown-number speaker separation with transformer decoder-based attractor,” inProc. ICASSP, 2024, pp. 446–450

2024
[29]

Continuous speech separation with recurrent selective at- tention network,

Y . Zhang, Z. Chen, J. Wu, T. Yoshioka, P. Wang, Z. Meng, and J. Li, “Continuous speech separation with recurrent selective at- tention network,” inProc. ICASSP, 2022, pp. 6017–6021

2022
[30]

All-neural beamformer for continuous speech separation,

Z. Zhang, T. Yoshioka, N. Kanda, Z. Chen, X. Wang, D. Wang, and S. E. Eskimez, “All-neural beamformer for continuous speech separation,” inProc. ICASSP, 2022, pp. 6032–6036

2022
[31]

Continuous stream- ing multi-talker asr with dual-path transducers,

D. Raj, L. Lu, Z. Chen, Y . Gaur, and J. Li, “Continuous stream- ing multi-talker asr with dual-path transducers,” inProc. ICASSP, 2022, pp. 7317–7321

2022
[32]

Continuous speech separation: Dataset and analysis,

Z. Chen, T. Yoshioka, L. Lu, T. Zhou, Z. Meng, Y . Luo, J. Wu, X. Xiao, and J. Li, “Continuous speech separation: Dataset and analysis,” inProc. ICASSP. IEEE, 2020, pp. 7284–7288

2020
[33]

Dual-path rnn for long recording speech separation,

C. Li, Y . Luo, C. Han, J. Li, T. Yoshioka, T. Zhou, M. Delcroix, K. Kinoshita, C. Boeddeker, Y . Qianet al., “Dual-path rnn for long recording speech separation,” inProc. SLT. IEEE, 2021, pp. 865–872

2021
[34]

FiLM: Visual reasoning with a general conditioning layer,

E. Perez, F. Strub, H. de Vries, V . Dumoulin, and A. C. Courville, “FiLM: Visual reasoning with a general conditioning layer,” in Proc. AAAI, 2018

2018
[35]

End-to-end speaker diarization for an unknown number of speak- ers with encoder-decoder based attractors,

S. Horiguchi, Y . Fujita, S. Watanabe, Y . Xue, and K. Nagamatsu, “End-to-end speaker diarization for an unknown number of speak- ers with encoder-decoder based attractors,”arXiv:2005.09921, 2020

2005 arXiv
[36]

Lib- riSpeech: an ASR corpus based on public domain audio books,

V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- riSpeech: an ASR corpus based on public domain audio books,” inProc. ICASSP, 2015

2015
[37]

WHAM!: Extending speech separation to noisy environments,

G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn, D. Crow, E. Manilow, and J. Le Roux, “WHAM!: Extending speech separation to noisy environments,” inProc. Interspeech, 2019

2019
[38]

gpuRIR: A python library for room impulse response simulation with gpu accelera- tion,

D. Diaz-Guerra, A. Miguel, and J. R. Beltran, “gpuRIR: A python library for room impulse response simulation with gpu accelera- tion,”Multimedia Tools and Applications, vol. 80, pp. 5653–5671, 2021

2021
[39]

ESPnet: End-to-end speech processing toolkit,

S. Watanabe, T. Hori, S. Karita, T. Hayashi, J. Nishitoba, Y . Unno, N. Enrique Yalta Soplin, J. Heymann, M. Wiesner, N. Chen, A. Renduchintala, and T. Ochiai, “ESPnet: End-to-end speech processing toolkit,” inProc. Interspeech, 2018, pp. 2207–2211

2018
[40]

SDR – half-baked or well done?

J. Le Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR – half-baked or well done?” inProc. ICASSP, 2019, pp. 626–630

2019
[41]

The rich tran- scription 2006 spring meeting recognition evaluation,

J. G. Fiscus, J. Ajot, M. Michel, and J. S. Garofolo, “The rich tran- scription 2006 spring meeting recognition evaluation,” inProc. MLMI, 2006, pp. 309–322

2006
[42]

Speaker counting and sep- aration from single-channel noisy mixtures,

S. R. Chetupalli and E. A. Habets, “Speaker counting and sep- aration from single-channel noisy mixtures,”IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 31, pp. 1681–1692, 2023

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Introduction Speech separation aims to isolate individual speech signals from mixtures containing multiple speakers. This task is particularly challenging in real-world environments due to noise, reverber- ation, and the highly time-varying nature of source activities, especially in single-channel scenarios [1, 2, 3, 4]. Most existing research assumes a k...

work page

[2] [2]

Here, the observed signalyconsists of speech signalsx c from Csources and noisen

Signal Model The observed signalyof lengthT ∗ can be represented as y= CX c=1 xc +n.(1) arXiv:2505.16607v1 [eess.AS] 22 May 2025 Figure 2:A-DCSS architecture. Here, the observed signalyconsists of speech signalsx c from Csources and noisen. The signalx c from thec-th speaker, consists of multiple utterances and potential pauses between them. The speaker c...

arXiv 2025

[3] [3]

The system architecture is illustrated in Fig

Attractor-based joint diarization, counting, and separation system (A-DCSS) The proposed joint separation system consists of three core pro- cesses: feature embedding, attractor generation, and speech separation. The system architecture is illustrated in Fig. 2. An encoder and a decoder are used to transform the signal between the time domain and the time...

work page

[4] [4]

Datasets In our research scenario, multiple speakers coexist in a mixed signal, each contributing multiple utterances

Experimental settings 4.1. Datasets In our research scenario, multiple speakers coexist in a mixed signal, each contributing multiple utterances. To synthesize data consistent with this scenario, we use speech from the Lib- riSpeech corpus [30], which provides the diarization labels for model training and evaluation. Noise signals are from the WHAM! datas...

work page

[5] [5]

Besides, an ablation study was conducted Table 2:Evaluation on 2-speaker & 3-speaker dataset

Experimental Results We evaluated the performance of the A-DCSS in two scenarios: one with a fixed number of speakers and the other with a varying number of speakers. Besides, an ablation study was conducted Table 2:Evaluation on 2-speaker & 3-speaker dataset. DER (%)(↓) is diarization error rate. SCA(%)(↑) is speaker counting accuracy. Anechoic Noisy Rev...

work page

[6] [6]

The system employs an RNN attractor module to estimate the number of sources and detect source ac- tivity, resulting in significant improvements in separation per- formance

Conclusions In this paper, we proposed a novel single-channel joint speech separation system that effectively handles scenarios with an un- known number of speakers, where each speaker may contribute multiple utterances. The system employs an RNN attractor module to estimate the number of sources and detect source ac- tivity, resulting in significant impr...

work page

[7] [7]

Vincent, T

E. Vincent, T. Virtanen, and S. Gannot,Audio Source Separation and Speech Enhancement. John Wiley & Sons, 2018

work page 2018

[8] [8]

Deep learning for audio signal processing,

H. Purwins, B. Li, T. Virtanen, J. Schl ¨uter, S.-Y . Chang, and T. Sainath, “Deep learning for audio signal processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 13, no. 2, pp. 206–219, 2019

work page 2019