An Investigation on Speaker Augmentation for End-to-End Speaker Extraction

Dong Wang; Lantian Li; Zhenghai You; Zhenyu Zhou

REVIEW 3 major objections 4 minor 35 references

An Investigation on Speaker Augmentation for End-to-End Speaker Extraction

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-only resampling-and-rescaling pipeline that manufactures pseudo-speakers and hard mixtures consistently reduces target confusion and improves extraction quality across two architectures and two benchmark datasets.

desk verdict A useful, honest empirical study of speaker augmentation for E2E extraction, but the headline gains may partly be a training-budget effect that needs a matched-budget rerun. read the letter →

arxiv 2505.21805 v1 pith:BAHZQNRE submitted 2025-05-27 cs.SD eess.AS

Zhenghai You , Zhenyu Zhou , Lantian Li , Dong Wang This is my paper

classification cs.SDeess.AS

keywords speakerextractiontargetconfusionaugmentationspeedperturbationpseudo-speakershardsamplesembeddingnegativeSI-SDRirate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Target confusion — the failure where an end-to-end speaker extraction model locks onto the interfering voice instead of the enrolled speaker — is traced here to speaker embeddings that are neither generalizable nor discriminative enough. The paper's diagnosis is that benchmark training sets carry only a few hundred voices, so the embedding space is sparse and the model can latch onto spurious cues such as textual similarity instead of genuine voice identity. The proposed remedy, speaker augmentation, is a training-only resampling-and-rescaling pipeline: resampling shifts pitch and formants to create a pseudo-speaker, and a WSOLA-based rescaling restores the original tempo so content, prosody, and duration survive unchanged. Mixing an utterance with its augmented twin produces hard samples solvable only by comparing true speaker traits, and expanding the speaker pool fivefold densifies the embedding space. On WSJ0-2Mix and Libri2Mix, with both DPRNN and SpEx+, the method consistently raises SI-SDRi (how close the extraction is to the clean target) and lowers the negative-SI-SDRi rate (the share of clips where the wrong speaker is extracted), and it combines with a triplet-loss objective for further gains.

What carries the argument

The carrying mechanism is a two-step time-domain pipeline. First, resampling with $y(t) = x(\alpha t)$ stretches or compresses the spectrogram along both axes, shifting the fundamental frequency and the spectral envelope (formants) and thereby creating a different perceived speaker while content is untouched. Second, a WSOLA-based rescaling chops the audio into overlapping segments and cross-fades them, restoring the original duration without altering pitch, so tempo, prosody, and content survive while only speaker traits are changed. With $\alpha$ drawn from $\{0.8, 0.9, 1.0, 1.1, 1.2\}$, the speaker inventory grows fivefold, which is argued to improve the generalizability of the embedding space; mixing an augmented utterance with its original twin yields the hard samples that force the speaker encoder and extractor to use genuine characteristics. The ablation isolates three hard-sample types — same content, same speaker, and same tempo — and shows that removing any of them degrades either extraction quality or the confusion rate.

What would settle it

Train SpEx+ or DPRNN on a mixture corpus with thousands of speakers, with and without the augmentation: if SI-SDRi and NSR do not improve, the claim that augmentation fixes embedding generalizability collapses. A mechanism-level check mirrors the paper's own Section 4.4 removal experiments but outside the benchmark: if hard samples are active, then forbidding every augmented-original pairing while keeping the fivefold pseudo-speaker expansion should erase most of the gain, whereas unchanged gains would mean speaker diversity alone carries the effect.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the target-confusion problem in end-to-end speaker extraction is substantially a data problem, not only an architecture problem: when the training corpus contains only a few hundred speakers, the speaker-embedding space is too sparse, and models can satisfy their training objectives using non-speaker cues. Resampling an utterance by a factor $\alpha$ drawn from $\{0.8, 0.9, 1.0, 1.1, 1.2\}$ and rescaling it back to its original duration produces a recognizable utterance from a 'new' speaker whose content, tempo, and prosody are unchanged; the only altered property is the voice. Adding these pseudo-speakers to training both densifies the speaker space and, when an original utterance is mixed with its augmented twin, creates hard samples the model can solve only by attending to genuine vocal traits. The experiments support the claim: for example, DPRNN on WSJ0-2Mix improves from 18.62 to 20.03 dB SI-SDRi while the confusion rate drops from 3.78% to 1.42%, and the ablation that removes hard-sample mixtures shows each type contributes to the gain. The paper is explicit that these results are established on standard benchmarks with limited speaker counts and that confirming the value of hard-sample augmentation on datasets with thousands of speakers requires a new benchmark.

Load-bearing premise

The load-bearing premise is that resampled-and-rescaled copies are genuinely the 'same speech from a different speaker' — that changing F0 and formants while preserving content, tempo, and prosody creates hard mixtures whose only usable difference is voice identity, and that this dynamic, demonstrated on benchmarks with a few hundred speakers, will generalize to the larger-speaker regimes the paper has not yet tested.

Editorial extensions

If this is right

Because the gains appear in both DPRNN and SpEx+, the augmentation is architecture-agnostic: any E2E-SE system could adopt it without changing its network or loss.
The benefit is larger in harder conditions: on Libri2Mix noisy, SpEx+ gains 5.84% relative SI-SDRi and cuts the confusion rate by 21.44% relative, versus smaller gains on Libri2Mix clean.
Speaker augmentation and metric learning are complementary: adding the triplet loss on top of augmentation further improves SI-SDRi (13.73 to 13.79 clean, 11.60 to 11.67 noisy) and lowers NSR (3.98% to 3.73%, 3.81% to 3.58%).
Pseudo-speakers behave like real ones: expanding 125 real speakers to 250 with half pseudo-speakers nearly matches training on 251 real speakers (10.85 versus 10.96 dB SI-SDRi on Libri2Mix noisy).
Hard samples, not just extra data volume, drive the improvement: removing same-content or same-speaker hard mixtures from training degrades performance even though they account for about 1% and 0.08% of samples respectively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An untested consequence of the paper's two rationales: on a corpus with thousands of speakers, the sparsity rationale predicts the augmentation gains should shrink, while the hard-sample rationale predicts they should persist — measuring which one wins would separate the two mechanisms.
A variant the paper does not try is enrollment-side augmentation: applying the same resampling-and-rescaling to the enrollment utterance would alter its voice traits relative to the target, testing whether confusion is driven by enrollment-target mismatch rather than training diversity.
The ablations suggest a sharper mechanism test the paper leaves open: keep the fivefold pseudo-speaker expansion but forbid every augmented-original pairing; vanishing gains would confirm hard samples as the active ingredient, while persistent gains would credit speaker diversity alone.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zhenghai and colleagues show that the resampling+rescaling speed-perturbation pipeline from speaker recognition also helps end-to-end speaker extraction. That is a sensible transfer, and the paper does it cleanly: two architectures (SpEx+, DPRNN), two datasets (WSJ0-2Mix, Libri2Mix clean/noisy), code released, and the NSR metric targets the actual failure mode. The ablation taxonomy (same content, same speaker, same tempo) is new, and the finding that most gains come from removing same-speaker samples is informative. I also credit the authors for explicitly saying, in the conclusion, that the value on thousands-of-speaker datasets is untested; that is the right framing.

The soft spots are real, and the big one is training budget. With the default five-factor augmentation set, the +SpkAug models see roughly five times as many target utterances per epoch as the baseline. The paper says all models are trained for max 200 epochs, with no statement that total samples or gradient steps were matched. That means Table 1's consistent gains could be, at least in part, a 'more data, more steps' effect rather than evidence for the hard-sample mechanism. The paper explicitly distinguishes its approach from 'simply increasing overall data volume,' but the current experiments do not actually enforce that comparison. Table 3 attempts a matched-sample ablation, but it is limited to a twofold expansion, the duplicate row labels make it hard to read, and the tiny proportions of S.C. (~1%) and S.S. (~0.08%) make the removal effects weak evidence. A matched-budget rerun -- e.g., baseline trained for 5x epochs or on 5x repeated samples, or augmentation with a single factor at matched sample count -- is the missing control.

The other concerns are minor by comparison: no error bars or significance tests, and the gains are modest. The self-citation issue (alpha from [17]) is not a problem, since the setting comes directly from prior work and is not tuned to the target result.

Who is this for? Anyone training E2E-SE systems on small-speaker benchmarks. The paper deserves a serious referee. If I were the editor, I'd send it out, with the matched-budget question as the central referee point. I'd cite it in my own work for the pipeline and the ablation results, though the headline claim needs the control before I'd trust the mechanism.

Referee Report

3 major / 4 minor

Summary. The paper proposes a speaker augmentation strategy for end-to-end speaker extraction (E2E-SE) to mitigate target confusion. The method resamples speech in the time domain (speed perturbation) and then rescales it with WSOLA to restore tempo, producing pseudo-speakers that differ from original utterances only in speaker traits while preserving content, tempo, and prosody. Mixing an augmented utterance with the original is argued to create "hard samples" that force the model to rely on genuine speaker characteristics. Experiments with SpEx+ and DPRNN on WSJ0-2Mix and Libri2Mix (clean and noisy) report consistent improvements in SI-SDRi and reductions in NSR. Ablations examine the effect of the number of pseudo-speakers, removal of different types of hard samples, and combination with a triplet metric-learning loss. The authors acknowledge that the benchmarks contain only a few hundred speakers and that validation on datasets with thousands of speakers would require a new benchmark.

Significance. If the causal interpretation holds, this is a valuable and simple contribution: an architecture-agnostic training technique that reduces a known failure mode without changing inference. The paper includes public code, tests two architectures on two datasets, and provides a plausible mechanism. The main risk is that the primary comparison (Table 1) does not separate the speaker-augmentation mechanism from a general increase in training data and gradient updates, and the ablation intended to address this (Table 3) is under-specified. All results are single-run point estimates with no variance or significance testing, which is concerning for effects of about 0.1 dB. With matched-budget controls and clarified ablations, the central claim would be substantially strengthened.

major comments (3)

[§4.1.3, Table 1] The augmented models are trained on a set of target utterances expanded fivefold via α∈{0.8,0.9,1.0,1.1,1.2}, while all models are limited to a maximum of 200 epochs. Thus, per epoch the +SpkAug models see roughly five times more mixtures and take roughly five times more gradient steps than the baseline. Since Section 2 explicitly distinguishes the proposed method from 'simply increasing overall data volume,' the consistent improvements in Table 1 may be due to increased training data and updates rather than the pseudo-speaker/hard-sample mechanism. Please provide matched-budget comparisons, e.g., train the baseline for more epochs or subsample the augmented set to equalize the total number of training mixtures and gradient steps.
[§4.4, Table 3] The hard-sample ablation is difficult to interpret. Table 3 contains duplicate row labels for '- S.C.' and '- S.S.' with different values, and the stated proportions of removed samples are tiny (about 1% and 0.08%). The observed differences (e.g., 10.85 vs 10.82 dB SI-SDRi) are likely within run-to-run variation, yet no error bars or significance tests are provided. Additionally, the ablation is limited to the twofold α={0.9,1.0} subset rather than the default fivefold setting. Please clarify the duplicate rows, report standard deviations or significance tests, and repeat the ablation under the default augmentation setting.
[§4.2 and all tables] All experimental results are single-run point estimates. The paper's central claim that the method 'consistently improves performance under all test conditions' rests on small absolute differences in NSR (e.g., 4.26% vs 3.98%) and SI-SDRi (e.g., 13.23 vs 13.73 dB). Without standard deviations across multiple seeds or significance tests, the consistency claim is not fully supported. Please report variance or statistical significance at least for the main comparisons in Table 1 and Table 4.

minor comments (4)

[§4.1.2] In the DPRNN description, the sentence 'We adopt the DPRNN model from [27] to design our E2E-SE system. which demonstrates strong performance...' has a lowercase 'which' after a period; revise for clarity.
[§4.4] The definition of 'Remove Same Tempo (S.T.) samples' is confusing: the text says augmented speech will only undergo resampling without rescaling, causing a tempo misalignment, which means the 'same tempo' hard samples are removed. Please reword to clarify the relationship.
[Table 3] The duplicate row labels for '- S.C.' and '- S.S.' should be distinguished (e.g., by specifying the exact removal condition) so that the results are reproducible and interpretable.
[§5] The conclusion appropriately acknowledges the limitation of small speaker counts in current benchmarks, but it would be helpful to state explicitly whether the proposed method is expected to provide gains independent of data volume when evaluated on larger datasets in future work.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical gains are measured against baselines, and the augmentation hyperparameters come from external prior work rather than being fitted to the target result.

full rationale

The paper's central claim is an empirical comparison: adding pseudo-speaker augmented utterances during training improves SI-SDRi and reduces NSR across DPRNN and SpEx+ on WSJ0-2Mix and Libri2Mix. No equation in the paper reduces to a fitted quantity: the augmentation factors alpha = {0.8, 0.9, 1.0, 1.1, 1.2} are taken from prior speaker-recognition work [17], not optimized on the evaluation sets, and Eq. (3) merely defines NSR via the SI-SDRi threshold rather than encoding a result. The hard-sample hypothesis is tested by controlled ablations that remove specific hard-sample types, and the observed degradations are empirical, not tautological: 'hard samples' are defined by construction (augmented original mixed with the original), but their contribution to training is measured, not assumed. The only self-citation is [17], which shares three authors with the present paper and supplies the perturbation-factor setting and the speed-perturbation idea; this is a provenance/hyperparameter citation, not load-bearing evidence for the target-confusion claim. The results are self-contained against external benchmarks with baselines trained under the same architecture and protocol, so the derivation is not circular. The reviewer's concern about unmatched training budget or gradient steps is an experimental-control issue, not a circularity issue, and the paper's own limitation about speaker diversity in existing benchmarks is an honest scope statement.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or architectural entities; 'pseudo-speakers' are augmented training samples, not a new ontological category. The central load-bearing assumptions are behavioral: that the augmentation preserves everything except speaker traits, and that pseudo-speakers stand in for real unseen speakers.

free parameters (2)

Speed perturbation factor set α = {0.8, 0.9, 1.0, 1.1, 1.2}
Selected following the setting in [17] (Section 4.1.1), not fitted to the target result; it determines the number of pseudo-speakers and the degree of speaker-trait alteration.
SNR sampling distributions = WSJ0: U[-5,5]; LibriMix: per [19]
Dataset conventions from Section 4.1.3; not fitted to the target result.

assumptions (3)

domain assumption Target confusion is largely caused by lack of generalizability and discrimination of the speaker embeddings.
Section 1 states this as the paper's framing; the link is not directly measured, but the hard-sample ablation provides indirect support.
domain assumption Time-domain resampling shifts F0 and formants while WSOLA restores tempo, so the augmented speech differs from the original only in speaker traits.
Section 3.1.2 asserts this property of the pipeline; it is standard signal processing, but the exact degree of speaker-trait preservation is not measured.
ad hoc to paper Pseudo-speakers emulate real unseen speakers well enough to improve the speaker embedding space.
Section 3.2.1 and Table 2 give indirect evidence (125 real + 125 pseudo close to 251 real), but Section 5 admits this is untested on large-speaker datasets.

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Investigation on Speaker Augmentation for End-to-End Speaker Extraction." pith.science (2026). https://pith.science/paper/BAHZQNRE

@misc{pith2026250521805,
  author       = {Pith},
  title        = {Pith review of: An Investigation on Speaker Augmentation for End-to-End Speaker Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BAHZQNRE}},
  note         = {Machine review of arXiv:2505.21805}
}

read the original abstract

Target confusion, defined as occasional switching to non-target speakers, poses a key challenge for end-to-end speaker extraction (E2E-SE) systems. We argue that this problem is largely caused by the lack of generalizability and discrimination of the speaker embeddings, and introduce a simple yet effective speaker augmentation strategy to tackle the problem. Specifically, we propose a time-domain resampling and rescaling pipeline that alters speaker traits while preserving other speech properties. This generates a variety of pseudo-speakers to help establish a generalizable speaker embedding space, while the speaker-trait-specific augmentation creates hard samples that force the model to focus on genuine speaker characteristics. Experiments on WSJ0-2Mix and LibriMix show that our method mitigates the target confusion and improves extraction performance. Moreover, it can be combined with metric learning, another effective approach to address target confusion, leading to further gains.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 33 canonical work pages

[17]

On the effectiveness of enrollment speech augmentation fo r tar- get speaker extraction,

J. Li, K. Zhang, S. Wang, H. Li, M.-W. Mak, and K. A. Lee, “On the effectiveness of enrollment speech augmentation fo r tar- get speaker extraction,” in 2024 IEEE Spoken Language Technol- ogy W orkshop (SLT). IEEE, 2024, pp. 325–332

work page 2024
[1]

Introduction Extracting a speciﬁc speaker’s voice from multi-talker spe ech signals is a fundamental challenge in speech signal process ing, commonly referred to as Speaker Extraction (SE) [1, 2, 3]. Un- like Speaker Separation (SS) [4, 5, 6], which aims to separat e all speakers in a mixed speech, SE refers to an enrollment utt er- ance of the target spe...

work page
[2]

hard samples

Related Work Speaker augmentation techniques have been widely adopted i n speaker recognition tasks [15, 16, 22]. Among these, speed perturbation is perhaps the most popular. It modiﬁes the pit ch and formants of a speech utterance while keeping the linguis - tic content unchanged. This simple yet effective strategy h as been shown to be highly effective ...

work page
[3]

Speaker Augmentation 3.1.1

Our Method 3.1. Speaker Augmentation 3.1.1. Step 1: Resampling We ﬁrst perform speed perturbation (SP) via time-domain re- sampling [17]. Given a speech signal x(t), we modify its time axis using a perturbation factor α , resulting in the output signal y(t): y(t) = x (αt ) . (1) This time-domain modiﬁcation induces a corresponding tran s- formation in the...

work page
[4]

Experimental Setup 4.1.1

Experiments 4.1. Experimental Setup 4.1.1. Data We conduct experiments on two benchmark datasets: WSJ0- 2Mix [18] and Libri2Mix [19]. Both datasets are used in their 8kHz versions. WSJ0-2Mix: This dataset is derived from the Wall Street Journal (WSJ0) corpus [24] and consists of 2-speaker mixtur es. The training set comprises 40,000 clean utterances from ...

work page
[5]

By applying a resampling-and-rescaling pipeline, we can gene rate vast pseudo-speakers, enriching speaker diversity in the t rain- ing data

Conclusion In this paper, we propose a simple yet effective speaker augmen- tation strategy for end-to-end speaker extraction (E2E-SE ). By applying a resampling-and-rescaling pipeline, we can gene rate vast pseudo-speakers, enriching speaker diversity in the t rain- ing data. Moreover, the generated data preserved the same te xt, tempo, and prosody of th...

work page
[6]

Spex: Multi-scale tim e domain speaker extraction network,

C. Xu, W. Rao, E. S. Chng, and H. Li, “Spex: Multi-scale tim e domain speaker extraction network,” IEEE/ACM transactions on audio, speech, and language processing , vol. 28, pp. 1370–1384, 2020

work page 2020
[7]

New insights on target speaker extraction,

M. Elminshawi, W. Mack, S. R. Chetupalli, S. Chakrabarty , and E. A. Habets, “New insights on target speaker extraction,” arXiv preprint arXiv:2202.00733, 2022

arXiv 2022

Show all 35 references

[8]

Neural target speech extraction: An overview,

K. Zmolikova, M. Delcroix, T. Ochiai, K. Kinoshita, J. ˇCernock` y, and D. Y u, “Neural target speech extraction: An overview,” IEEE Signal Processing Magazine, vol. 40, no. 3, pp. 8–29, 2023

2023
[9]

Blind so urce separation and independent component analysis: A review,

S. Choi, A. Cichocki, H.-M. Park, and S.-Y . Lee, “Blind so urce separation and independent component analysis: A review,” Neu- ral Information Processing-Letters and Reviews, vol. 6, no. 1, pp. 1–57, 2005

2005
[10]

Blind source separation,

G. R. Naik, W. Wang et al. , “Blind source separation,” Berlin: Springer, vol. 10, pp. 978–3, 2014

2014
[11]

Blind source sep - aration: A review and analysis,

M. Pal, R. Roy, J. Basu, and M. S. Bepari, “Blind source sep - aration: A review and analysis,” in 2013 International Con- ference Oriental COCOSDA held jointly with 2013 Confer- ence on Asian Spoken Language Research and Evaluation (O- COCOSDA/CASLRE). IEEE, 2013, pp. 1–5

2013
[12]

Improving speaker discriminati on of target speech extraction with time-domain speakerbeam,

M. Delcroix, T. Ochiai, K. Zmolikova, K. Kinoshita, N. Ta wara, T. Nakatani, and S. Araki, “Improving speaker discriminati on of target speech extraction with time-domain speakerbeam, ” in ICASSP 2020-2020 IEEE International Conference on Acousti cs, Speech and Signal Processi...

2020
[13]

X-SepFormer: End-to-e nd speaker extraction network with explicit optimization on s peaker confusion,

K. Liu, Z. Du, X. Wan, and H. Zhou, “X-SepFormer: End-to-e nd speaker extraction network with explicit optimization on s peaker confusion,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

2023
[14]

MC-SpEx: Towards effective speaker extraction with multi-scale interfusion and conditional speaker modulati on,

J. Chen, W. Rao, Z. Wang, J. Lin, Y . Ju, S. He, Y . Wang, and Z. Wu, “MC-SpEx: Towards effective speaker extraction with multi-scale interfusion and conditional speaker modulati on,” in INTERSPEECH, 2023

2023
[15]

Target confusion in end-to-end speaker extraction: Analysis and approaches ,

Z. Zhao, D. Y ang, R. Gu, H. Zhang, and Y . Zou, “Target confusion in end-to-end speaker extraction: Analysis and approaches ,” in INTERSPEECH, 2022, pp. 5333–5337

2022
[16]

A hybrid continuity loss to redu ce over-suppression for time-domain target speaker extracti on,

Z. Pan, M. Ge, and H. Li, “A hybrid continuity loss to redu ce over-suppression for time-domain target speaker extracti on,” in INTERSPEECH, 2022, pp. 1786–1790

2022
[18]

Multi - stage speaker extraction with utterance and frame-level re ference signals,

M. Ge, C. Xu, L. Wang, E. S. Chng, J. Dang, and H. Li, “Multi - stage speaker extraction with utterance and frame-level re ference signals,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 6109–6113

2021
[19]

Robust speaker extraction network based on iterativ e re- ﬁned adaptation,

C. Deng, S. Ma, Y . Zhang, Y . Sha, H. Zhang, H. Song, and X. Li, “Robust speaker extraction network based on iterativ e re- ﬁned adaptation,” in INTERSPEECH, 2021, pp. 3530–3534

2021
[20]

Spea ker augmentation and bandwidth extension for deep speaker embe d- ding

H. Y amamoto, K. A. Lee, K. Okabe, and T. Koshinaka, “Spea ker augmentation and bandwidth extension for deep speaker embe d- ding.” in INTERSPEECH, 2019, pp. 406–410

2019
[21]

B uild a SRE challenge system: Lessons from V oxSRC 2022 and CN- SRC 2022,

Z. Chen, B. Han, X. Xiang, H. Huang, B. Liu, and Y . Qian, “B uild a SRE challenge system: Lessons from V oxSRC 2022 and CN- SRC 2022,” pp. 3202–3206, 2023

2022
[22]

A comprehensiv e investigation on speaker augmentation for speaker recogni tion,

Z. Zhou, S. Xu, S. Yin, L. Li, and D. Wang, “A comprehensiv e investigation on speaker augmentation for speaker recogni tion,” in INTERSPEECH, 2024, pp. 2160–2164

2024
[23]

Deep clus- tering: Discriminative embeddings for segmentation and se para- tion,

J. R. Hershey, Z. Chen, J. Le Roux, and S. Watanabe, “Deep clus- tering: Discriminative embeddings for segmentation and se para- tion,” in 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2016, pp. 31–35

2016
[24]

Librimix: An open-source dataset for generalizable speech separation,

J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, an d E. Vin- cent, “Librimix: An open-source dataset for generalizable speech separation,” arXiv preprint arXiv:2005.11262, 2020

2005 arXiv
[25]

Dual-path RNN: efﬁcie nt long sequence modeling for time-domain single-channel speech s epa- ration,

Y . Luo, Z. Chen, and T. Y oshioka, “Dual-path RNN: efﬁcie nt long sequence modeling for time-domain single-channel speech s epa- ration,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 46–50

2020
[26]

Spex+ : A complete time domain speaker extraction network,

M. Ge, C. Xu, L. Wang, E. S. Chng, J. Dang, and H. Li, “Spex+ : A complete time domain speaker extraction network,” in INTER- SPEECH, 2020, pp. 1406–1410

2020
[27]

Speakeraug- ment: Data augmentation for generalizable source separati on via speaker parameter manipulation,

K. Wang, Y . Y ang, H. Huang, Y . Hu, and S. Li, “Speakeraug- ment: Data augmentation for generalizable source separati on via speaker parameter manipulation,” in ICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Signal Pro cess- ing (ICASSP). IEEE, 202...

2023
[28]

An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale mo d- iﬁcation of speech,

W. V erhelst and M. Roelands, “An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale mo d- iﬁcation of speech,” in 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing , vol. 2. IEEE, 1993, pp. 554–557

1993
[29]

CSR- I (WSJ0) Complete LDC93S6A,

J. S. Garofolo, D. Graff, D. Paul, and D. Pallett, “CSR- I (WSJ0) Complete LDC93S6A,” 1993. [Online]. Available: https://catalog.ldc.upenn.edu/LDC93S6A

1993
[30]

Lib- rispeech: an ASR corpus based on public domain audio books,

V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: an ASR corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech an d signal processing (ICASSP). IEEE, 2015, pp. 5206–5210

2015
[31]

Wham!: Extending speech separation to noisy environments,

G. Wichern, J. Antognini, M. Flynn, L. R. Zhu, E. McQuinn , D. Crow, E. Manilow, and J. L. Roux, “Wham!: Extending speech separation to noisy environments,” in INTERSPEECH, 2019, pp. 1368–1372

2019
[32]

Asteroid: the PyTorch-based audio source separa- tion toolkit for researchers,

M. Pariente, S. Cornell, J. Cosentino, S. Sivasankaran , E. Tzinis, J. Heitkaemper, M. Olvera, F.-R. St¨ oter, M. Hu, J. M. Mart´ ı n- Do˜ naset al. , “Asteroid: the PyTorch-based audio source separa- tion toolkit for researchers,” in INTERSPEECH, 2020, pp. 2637– 2641

2020
[33]

TaSNet: Time-domain audio sep ara- tion network for real-time, single-channel speech separat ion,

Y . Luo and N. Mesgarani, “TaSNet: Time-domain audio sep ara- tion network for real-time, single-channel speech separat ion,” in 2018 IEEE International Conference on Acoustics, Speech an d Signal Processing (ICASSP). IEEE, 2018, pp. 696–700

2018
[34]

Data au gmen- tation for speech separation,

A. Alex, L. Wang, P . Gastaldo, and A. Cavallaro, “Data au gmen- tation for speech separation,” Speech Communication, vol. 152, p. 102949, 2023

2023
[35]

X-TaSNet: Robust and accu rate time-domain speaker extraction network,

Z. Zhang, B. He, and Z. Zhang, “X-TaSNet: Robust and accu rate time-domain speaker extraction network,” in INTERSPEECH, 2020, pp. 1421–1425

2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [17]

On the effectiveness of enrollment speech augmentation fo r tar- get speaker extraction,

J. Li, K. Zhang, S. Wang, H. Li, M.-W. Mak, and K. A. Lee, “On the effectiveness of enrollment speech augmentation fo r tar- get speaker extraction,” in 2024 IEEE Spoken Language Technol- ogy W orkshop (SLT). IEEE, 2024, pp. 325–332

work page 2024

[2] [1]

Introduction Extracting a speciﬁc speaker’s voice from multi-talker spe ech signals is a fundamental challenge in speech signal process ing, commonly referred to as Speaker Extraction (SE) [1, 2, 3]. Un- like Speaker Separation (SS) [4, 5, 6], which aims to separat e all speakers in a mixed speech, SE refers to an enrollment utt er- ance of the target spe...

work page

[3] [2]

hard samples

Related Work Speaker augmentation techniques have been widely adopted i n speaker recognition tasks [15, 16, 22]. Among these, speed perturbation is perhaps the most popular. It modiﬁes the pit ch and formants of a speech utterance while keeping the linguis - tic content unchanged. This simple yet effective strategy h as been shown to be highly effective ...

work page

[4] [3]

Speaker Augmentation 3.1.1

Our Method 3.1. Speaker Augmentation 3.1.1. Step 1: Resampling We ﬁrst perform speed perturbation (SP) via time-domain re- sampling [17]. Given a speech signal x(t), we modify its time axis using a perturbation factor α , resulting in the output signal y(t): y(t) = x (αt ) . (1) This time-domain modiﬁcation induces a corresponding tran s- formation in the...

work page

[5] [4]

Experimental Setup 4.1.1

Experiments 4.1. Experimental Setup 4.1.1. Data We conduct experiments on two benchmark datasets: WSJ0- 2Mix [18] and Libri2Mix [19]. Both datasets are used in their 8kHz versions. WSJ0-2Mix: This dataset is derived from the Wall Street Journal (WSJ0) corpus [24] and consists of 2-speaker mixtur es. The training set comprises 40,000 clean utterances from ...

work page

[6] [5]

By applying a resampling-and-rescaling pipeline, we can gene rate vast pseudo-speakers, enriching speaker diversity in the t rain- ing data

Conclusion In this paper, we propose a simple yet effective speaker augmen- tation strategy for end-to-end speaker extraction (E2E-SE ). By applying a resampling-and-rescaling pipeline, we can gene rate vast pseudo-speakers, enriching speaker diversity in the t rain- ing data. Moreover, the generated data preserved the same te xt, tempo, and prosody of th...

work page

[7] [6]

Spex: Multi-scale tim e domain speaker extraction network,

C. Xu, W. Rao, E. S. Chng, and H. Li, “Spex: Multi-scale tim e domain speaker extraction network,” IEEE/ACM transactions on audio, speech, and language processing , vol. 28, pp. 1370–1384, 2020

work page 2020

[8] [7]

New insights on target speaker extraction,

M. Elminshawi, W. Mack, S. R. Chetupalli, S. Chakrabarty , and E. A. Habets, “New insights on target speaker extraction,” arXiv preprint arXiv:2202.00733, 2022

arXiv 2022