Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios

Jakob Kienegger; Timo Gerkmann

REVIEW 4 major objections 5 minor 48 references

Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Given only the target speaker's initial direction, a jointly trained tracker and deep spatial filter extract a moving speaker through crossings and beat a mismatched strongly guided system.

desk verdict Modest, honest empirical claim that weak initial-DoA guidance plus joint training beats a mismatched strong-guidance oracle; the headline margin lacks error bars, so statistical support is not yet established. read the letter →

arxiv 2505.14517 v1 pith:OXB7E4TA submitted 2025-05-20 eess.AS cs.LGcs.SD

Jakob Kienegger , Timo Gerkmann This is my paper · ORCID

classification eess.AScs.LGcs.SD

keywords targetspeakerextractionmovingtrackingspatiallyselectivefilteringdirectionofarrivalestimationjointtrainingdynamicacousticscenesmulti-channelspeechenhancementcocktailpartyproblem

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that continuous, accurate directional cues are not necessary for extracting a moving target speaker from a two-speaker mixture. The authors propose a weakly guided pipeline that receives only the target's initial direction of arrival, estimates the trajectory with a deep tracker, and feeds those estimates to the FT-JNF spatially selective filter. With the tracker and filter trained jointly, the system resolves crossing ambiguities and, at the strongest tested motion, outperforms a strongly guided filter that was trained on static data. The point matters because manually following a moving speaker with a pointer is impractical; a single starting direction would suffice.

What carries the argument

The load-bearing components are the FT-JNF, a deep non-linear spatially selective filter that computes a complex time-frequency mask from a multi-channel STFT input conditioned on a one-hot encoded direction of arrival, and a causal CNN/LSTM tracker that outputs a Softmax posterior over $180$ azimuth bins of $2^\circ$. The tracker receives the initial direction $\theta_0$ by having a linear projection of its one-hot encoding initialize the LSTM state. Joint training is the critical mechanism: the argmax in the tracker is dropped so gradients from the filter's extraction loss flow into the tracker, making the filter robust to imperfect trajectory estimates. Training data come from an image-method room impulse response simulator with a constant-velocity motion model, producing continuous trajectories that can cross.

What would settle it

Record two moving speakers with a three-microphone circular array in a real reverberant room, provide only the target's initial azimuth, and compare the jointly trained pipeline against a static-trained oracle on SI-SDR; if the margin seen in the paper does not appear, the central claim is not transferable to real conditions.

Watch

Extended reading notes

Core claim

On synthetic reverberant two-speaker mixtures with continuously moving azimuth trajectories, the paper claims that joint end-to-end training of a deep target tracker and the FT-JNF filter makes weak guidance work. The jointly trained pipeline, which is given only the initial direction $\theta_0$ and estimates the time-varying direction of arrival itself, reaches $2.21$ dB SI-SDR at an expected motion of $360^\circ$ per $5$ s, compared with $1.72$ dB for an oracle-strongly-guided FT-JNF trained on stationary data and $4.02$ dB for the same oracle trained on dynamic data. Joint training also repairs a very inaccurate particle-filter tracker, lifting its SI-SDR from $-4.20$ dB to $0.39$ dB at the same motion level. The authors interpret this as evidence that a spatially selective filter can learn temporal-spectral speaker patterns to resolve crossing ambiguities rather than relying solely on spatial position.

Load-bearing premise

The entire demonstration rests on the assumption that a synthetic simulation—image-method rooms, a three-microphone circular array, two speakers moving at constant speed and distance, and no added noise—is representative enough of real acoustic scenes that the joint-training gains transfer.

Editorial extensions

If this is right

A user would need to point once at the target at the start; no continuous steering or per-frame direction estimates are required.
Joint training turns even a very inaccurate tracker (a particle filter) into a usable extractor, so the approach does not depend on a perfect upstream localizer.
Because static-trained strongly guided filters degrade sharply under motion, dynamic training data, not just better cues, are needed for moving-speaker extraction.
The system can resolve crossing events by learning temporal-spectral speaker patterns, going beyond what a purely spatial cue offers.
The causal architecture (unidirectional LSTM, causal convolutions) supports online operation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the simulation-to-real transfer holds, the same one-point guidance could be paired with an enrollment utterance or speaker embedding to also handle identity ambiguity when two speakers share nearly identical trajectories for long stretches, a case the paper does not test.
A natural stress test the paper does not run is perturbing the given initial direction $\theta_0$; the pipeline's robustness to tracking errors may not extend to a biased or noisy starting point, since the tracker is conditioned directly on that value.
The synthetic domain lacks noise, has only two speakers, and fixes speaker distance and height, so the reported margins are likely upper bounds for real rooms; the joint-training benefit could shrink or vanish under array mismatch or head movement.
The closed-form relation between motion-model variance and expected angular displacement could support a curriculum that starts with slow speakers and speeds up during training, an extension the paper leaves implicit.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper for what it is: a carefully scoped empirical study showing that a target-speaker extraction pipeline can get by with only the target's initial direction, if the upstream tracker and the spatial filter are trained jointly. The setup is sensible, the baselines are reasonable, and the authors do not oversell — they explicitly say the matched dynamic oracle still beats them.

What's actually new is the combination: a CNN/LSTM tracker modified to be conditioned on the initial DoA, concatenated with the FT-JNF spatial filter, trained end-to-end. I don't see that exact weakly guided TSE pipeline in the cited literature, and the synthetic dataset with continuous motion is a step up from the piecewise-constant trajectories used in prior work. The motion model and the closed-form relation between perturbation variance and expected angular displacement are nice touches, and the ablation across tracking algorithms is informative. The paper gives real credit where it is due.

Now the soft spots. The stress-test concern is legitimate. The central comparison at 360°/5s is 2.21 vs 1.72 dB SI-SDR — a 0.49 dB margin — and Table 1 has no error bars or significance tests. Figure 4 shows standard-deviation shading for some configurations but conspicuously omits the two proposed rows that carry the headline result. A 0.5 dB SI-SDR difference can easily be seed-to-seed noise in this kind of training. The margin at 180°/5s is larger (3.44 vs 2.18 dB), which helps, but the paper's strongest point is the hardest case, and that is exactly where the missing statistics matter most. Also note that at 360°/5s the proposed method's ESTOI is actually slightly below the static oracle (66.5 vs 67.9), so the 'beat the oracle' claim is true for SI-SDR and PESQ but not for intelligibility; the text is careful about this, but readers may over-read.

The other soft spot is the evaluation: everything shares the same synthetic generator. That's an addressable limitation, not a fatal one, but it means the transfer claim is untested. No code or data is released, so I cannot check the numbers independently.

Overall: the core idea is sound, the experiments are thoughtfully designed, and the claims are stated honestly. The missing error bars should be fixable and are the main thing I would demand before trusting the headline. This deserves a serious referee, not a desk reject, but the referee should ask for uncertainty quantification and ideally one cross-domain or real-recording check.

Recommendation: send it to review; require the authors to report variance or confidence intervals on Table 1, and to make code or data available if they want the result to be credible.

Referee Report

4 major / 5 minor

Summary. The paper addresses target speaker extraction in dynamic multi-speaker scenarios where the target speaker moves during the recording. Rather than requiring continuous oracle direction-of-arrival information ('strong guidance'), it proposes a weakly guided pipeline that receives only the target's initial azimuth angle, uses a deep recurrent tracker to estimate the time-varying direction, and feeds these estimates into the FT-JNF spatially selective filter. The authors generate a synthetic two-speaker dataset with continuous constant-velocity azimuth motion and compare several training strategies: independent training, training on dynamic data, and joint end-to-end training of the tracker and filter. On this dataset, the jointly trained weakly guided pipeline reaches an SI-SDR of 2.21 dB at an expected motion of 360 degrees per 5 seconds, which is above a static-trained, strongly guided oracle (1.72 dB) and below a dynamic-trained oracle (4.02 dB).

Significance. If the central result is reliable, the paper makes a practically relevant contribution: a user would only need to point once at the target, and the system would track and extract the moving speaker, including through crossing ambiguities, without an enrollment utterance. The technical core is the adaptation of a tracking network to be conditioned on the initial direction and the joint training strategy that makes the spatially selective filter robust to tracker errors. The experiments are carefully constructed: the motion model in Eqs. (4)-(6) is physically motivated, multiple metrics are reported, and both a neural tracker and a particle-filter baseline are included. However, the current evidence is insufficient on three points: the headline 0.49 dB margin is reported without variance or significance information; one of the three metrics does not improve at the most dynamic condition; and all conclusions come from a single synthetic generator. The joint-training principle is nevertheless a reasonable and falsifiable claim, and the paper does not claim to beat a matched dynamic-trained oracle.

major comments (4)

[Table 1 and Figure 4] The central claim that the weakly guided joint pipeline outperforms the strongly guided static-trained oracle rests on a 0.49 dB SI-SDR margin at the most dynamic condition (2.21 vs. 1.72 dB, row (8) vs. row (1)), but no error bars, confidence intervals, or significance tests are reported anywhere in Table 1. Figure 4 is the only place where variance information appears, and it omits the proposed weakly guided rows (6)-(8) that carry the headline result. Since training uses random rooms, trajectories, and speaker pairs, this margin may be within run-to-run variability. Please report the number of independent training runs and test utterances, add error bars for all rows in Table 1 (and include rows (6)-(8) in Figure 4), and provide a paired significance test for the row (8) vs. row (1) comparison at the non-stationary motion levels.
[Table 1, rows (8) vs. (1); Abstract and Section 5] At 360 degrees per 5 seconds, the proposed system improves SI-SDR (2.21 vs. 1.72 dB) and PESQ (1.66 vs. 1.56), but ESTOI is lower (66.5% vs. 67.9%). Thus the unqualified 'even outperform' in the abstract and conclusion is not supported on all reported metrics. Section 5 is careful to say 'regarding distortion and speech quality metrics,' but the abstract and conclusion should carry the same qualification, and the ESTOI decrease at the highest motion level should be discussed, for example as a possible trade-off between target preservation and interference suppression.
[Section 4.2 (Joint training strategy)] The description of joint training says that the argmax in Eq. (3) is dropped to retain differentiability, which means the SSF is trained with the soft posterior p(theta_t | Y_{1:t}, theta_0) as its spatial input. The inference-time pipeline in Figure 1 uses the argmax theta-hat_t, which would produce a hard one-hot input. This is a train/test input distribution mismatch that is neither stated nor analyzed. Please specify exactly what is fed to the SSF during joint training and at inference; if these quantities differ, argue why a hard one-hot at test time is still valid, or evaluate the system with the soft posterior at inference.
[Sections 4.1 and 5 (Dataset and Evaluation)] All training and evaluation examples are generated by the same simulator: image-method RIRs, a 10 cm three-microphone circular array, two speakers, no additive noise, and constant-velocity azimuth motion. The paper's stated motivation includes real-world dynamic scenarios, but no cross-domain evaluation, real recordings, or external benchmark is provided. Consequently, the robustness of the joint-training benefit to conditions outside this generator (more speakers, additive noise, array mismatch, head movement, non-constant-velocity motion) is an assumption rather than a demonstrated property. Please add a domain-shift experiment or at least a detailed discussion of the expected failure modes; without one, the generalizability claims in the introduction and conclusion should be tempered.

minor comments (5)

[Figure 4] The y-axis label 'Delta SI-SDR [dB]' is not defined in the text; please state the reference quantity, for example the improvement over the unprocessed mixture or over the reference-channel SI-SDR.
[Equation (4) / Section 4.1] The initial angular velocity used in the constant-velocity motion model is not specified; please state whether it is zero, sampled from a distribution, or derived from the initialization.
[Section 4.2] The replacement of the last B-LSTM in FT-JNF by a unidirectional LSTM 'of twice its initial parameters' is not accompanied by an ablation against the original FT-JNF architecture; since the paper repeatedly refers to [3], a comparison would clarify how much of the reported performance is due to this modification.
[Reproducibility] The manuscript does not mention code or dataset availability. Because the dataset is synthetic and built from LibriSpeech and gpuRIR, releasing the generation scripts would substantially improve reproducibility.
[Table 1 header] The header '0 deg/5 s, 180 deg/5 s, 360 deg/5 s' is rendered in a visually compressed way; consider writing '0, 180, and 360 degrees per 5 s' or using separate columns for each condition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the weakly guided pipeline is a measured end-to-end training result, not a reduction to its inputs.

full rationale

This is an empirical machine-learning paper rather than an analytic derivation, so there is no equation chain whose output is equivalent to its inputs by construction. The central comparison, weakly guided joint-trained FT-JNF extraction versus a static-trained oracle-guided FT-JNF, is reported in Table 1 rows (8) and (1), and the improvement is an experimental outcome of training on a synthetic data distribution, not a fitted parameter renamed as a prediction. The tracking network is based on published components [3, 17], but the paper modifies and retrains it, and the downstream filter is trained jointly with it; no step in the paper reduces the reported SI-SDR to the training loss or to the initial DoA input. Self-citations to Tesch and Gerkmann [3] supply the baseline architecture and loss function, but that baseline is also the comparison system and is not invoked as an unverified premise that forces the headline result. The absence of error bars and real-recorded benchmarks is a statistical-validity and generalization concern, not a circularity, and the paper explicitly frames its demonstration as being on a synthetic dataset. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The central claim is an empirical result, so the ledger records the modeling and dataset assumptions the result rests on rather than fitted physical constants. No new physical or conceptual entities are introduced; the deep tracker is a learned model, not an invented entity.

free parameters (3)

Spatial discretization step = 2 degrees (Theta=180)
Chosen by hand; defines the granularity of the one-hot DoA encoding for both the tracker and the spatial filter. Finer or coarser steps would affect tracking accuracy and extraction performance, but it is not fitted to optimize the reported metrics.
Expected absolute angular displacement E{|Delta_theta_t|} = 0, 180, 360 degrees per 5 seconds
Controls the difficulty of the synthetic scenarios; the perturbation variance sigma is set from this quantity via Eq. (6). It is an experimental condition chosen by the authors, not a fitted free parameter.
Motion model perturbation variance sigma = Derived from Eq. (6) for each chosen expected displacement
Determines the smoothness of generated speaker trajectories. It is computed from the chosen expected displacement using the closed-form Gaussian result, not optimized to improve metrics.

assumptions (7)

domain assumption Additive mixture of target speech, interfering speech, and reverberation in the STFT domain (Eq. 1)
Standard TSE model; the paper assumes noise V captures all non-target content and that dry speech recovery is the objective.
domain assumption Far-field approximation and azimuth-only cue; distance and height changes are negligible
Invoked in Section 2.2 to justify using only the azimuth angle as the spatial cue.
domain assumption Synthetic room impulse responses from the image method with RT60 between 0.2 s and 0.5 s adequately represent real reverberant environments
Used to generate all training and evaluation data; no real-RIR validation is provided.
domain assumption Constant-velocity motion model with Gaussian perturbation (Eq. 4) describes realistic speaker movement, including crossings
Defines the trajectory distribution; the authors derive Eq. (5) and Eq. (6) to set sigma from a chosen expected displacement.
domain assumption Initial DoA theta_0 is known to the user and provided as the only guidance
The premise of weak guidance; the paper does not address errors in the initial cue itself.
domain assumption No additive environmental noise; the mixture contains only reverberant speech
Dataset construction in Section 4.1; limits generality to clean reverberant two-speaker cases.
standard math The expected absolute displacement formula in Eq. (6) correctly integrates the Gaussian displacement distribution
Used to map a chosen E{|Delta_theta_t|} to the perturbation variance sigma; standard property of Gaussian distributions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios." pith.science (2026). https://pith.science/paper/OXB7E4TA

@misc{pith2026250514517,
  author       = {Pith},
  title        = {Pith review of: Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OXB7E4TA}},
  note         = {Machine review of arXiv:2505.14517}
}

read the original abstract

Recent speaker extraction methods using deep non-linear spatial filtering perform exceptionally well when the target direction is known and stationary. However, spatially dynamic scenarios are considerably more challenging due to time-varying spatial features and arising ambiguities, e.g. when moving speakers cross. While in a static scenario it may be easy for a user to point to the target's direction, manually tracking a moving speaker is impractical. Instead of relying on accurate time-dependent directional cues, which we refer to as strong guidance, in this paper we propose a weakly guided extraction method solely depending on the target's initial position to cope with spatial dynamic scenarios. By incorporating our own deep tracking algorithm and developing a joint training strategy on a synthetic dataset, we demonstrate the proficiency of our approach in resolving spatial ambiguities and even outperform a mismatched, but strongly guided extraction method.

Figures

Figures reproduced from arXiv: 2505.14517 by the authors.

**Figure 1.** Proposed weakly guided target speaker extraction (TSE) pipeline. In contrast to strongly guided TSE, our method only requires the initial DoA θ0 to extract the speech signal Sbtk. gorithm to estimate the evolution of the target’s DoA θt based on θ0. In analogy to TSE, we will refer to this task as target speaker tracking (TST) and avoid the term target speaker localization, as it is commonly paired with audio cues … view at source ↗

**Figure 2.** Proposed deep target speaker tracking (TST) conditioned on the initial DoA θ0. The network architecture is based on [3] and [17]. Colored layers indicate learnable parameters [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** DoA estimation with target speaker tracking (TST) methods presented in section 4.2. Selected trajectories correspond to (4) with an expected displacement E{|∆θt|} of 180◦ 5s . In the following, we will determine the standard deviation σ of the motion model’s perturbation νt in (4) from a given mean absolute deviation E{|∆θt|} after t frames using (6). The trajectory time step ∆T of the RIR simulations is aligned w… view at source ↗

Figures from the paper (3 more)

**Figure 5.** Figure 5: Accuracy (ACC) and median angular error (AE) of TST algorithms. Shaded areas indicate 25% and 75% quartiles. ries in fig. 3 and report frame-wise accuracy (ACC) with a 5° margin [15, 16] and angular error (AE) in fig. 5. Although the DaS-PF utilizes the motion model wi…

**Figure 6.** Figure 6: Strongly (1, 2) and weakly (6, 7, 8) guided extraction results with FT-JNF [3] and varying tracking/training configurations during crossing of target ( ) and interfering speaker ( ). 6. Conclusion In this work we proposed a weakly guided TSE pipeline which solely requ…

**Figure 4.** Figure 4: Selected TSE configurations evaluated with a metric sensitive to distortions. Shaded areas indicate std. deviation. 0 90 180 270 360 0 25 50 75 100 exp. DoA shift E{|∆θt|} ◦ 5s ACC [%] → DaS-PF[21] Proposed 0 90 180 270 360 0.1 1 10 100 exp. DoA shift E{|∆θt|} ◦ 5s AE …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 46 canonical work pages

[1]

Introduction Given a real world recording containing a speech signal, the field of speech enhancement focuses on improving its quality and intelligibility by suppressing noise and reverberation arti- facts. Noise sources in form of additional interfering speech signals, such as in the so-called cocktail-party problem [1], are especially challenging due to...

work page Pith review arXiv 2025
[2]

Target speaker extraction Let the multi-channel observation Y be modeled as additive mixture of reverberant target speech X with noise V

Problem definition 2.1. Target speaker extraction Let the multi-channel observation Y be modeled as additive mixture of reverberant target speech X with noise V. The lat- ter contains interfering speech sources as well as additive en- vironmental and measurement noise. In the short-time Fourier transform (STFT) domain, this acoustic setup is resembled by ...

work page
[3]

Proposed method 3.1. Weakly guided target speaker extraction Instead of relying on accurate continuous prior directional in- formation, we relax this constraint and only assume knowl- edge about the initial starting direction θ0 of the target speaker. In order to retain compatibility with conventional SSFs like [3, 4, 5, 6], we propose to incorporate an u...

work page
[4]

Dataset For training and evaluation we create a synthetic dataset with reverberant two-speaker mixtures

Experiments 4.1. Dataset For training and evaluation we create a synthetic dataset with reverberant two-speaker mixtures. Specifically, we use utter- ances from the LibriSpeech [29] corpus, convolve them with simulated room impulse responses (RIRs) based on the image method [30] and then combine them according to the recipe in Libri2Mix [31]. Since no add...

work page
[5]

distortions, perceptual quality and intelligibility via the metrics SI-SDR [38], PESQ

Results We evaluate the TSE performance w.r.t. distortions, perceptual quality and intelligibility via the metrics SI-SDR [38], PESQ

work page
[6]

Conclusion In this work we proposed a weakly guided TSE pipeline which solely requires information about the target’s initial direction. By developing a synthetic dataset with continuous speaker movement, we could not only demonstrate the significant per- formance improvement of a commonly used SSF in dynamic environments, but also its capability to resol...

work page
[7]

Acknowledgments This work was supported by the Deutsche Forschungsge- meinschaft (DFG, German Research Foundation) under grant 508337379. Computational resources were provided by Re- gional Computer Center (RRZ) of the University of Hamburg (UHH) and Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universit ¨at Erl...

work page
[8]

Vary and R

P. Vary and R. Martin, Digital Speech Transmission: Enhance- ment, Coding and Error Concealment. Hoboken, NY , USA: Wi- ley, 2006

work page 2006

Show all 48 references

[9]

Some experiments on the recognition of speech, with one and two ears,

E. C. Cherry, “Some experiments on the recognition of speech, with one and two ears,” J. Acoust. Soc. Am., vol. 25, 1953

1953
[10]

Neural target speech extraction: An overview,

K. Zmolikova, M. Delcroix, T. Ochiai, K. Kinoshita, J. ˇCernock´y, and D. Yu, “Neural target speech extraction: An overview,”IEEE Signal Proc. Magazine, vol. 40, 2023

2023
[11]

Multi-channel speech separation us- ing spatially selective deep non-linear filters,

K. Tesch and T. Gerkmann, “Multi-channel speech separation us- ing spatially selective deep non-linear filters,”IEEE/ACM TASLP, vol. 32, 2024

2024
[12]

Exploiting spatial information with the informed complex-valued spatial au- toencoder for target speaker extraction,

A. Briegleb, M. M. Halimeh, and W. Kellermann, “Exploiting spatial information with the informed complex-valued spatial au- toencoder for target speaker extraction,” in IEEE ICASSP, 2023

2023
[13]

Spatially se- lective speaker separation using a DNN with a location dependent feature extraction,

A. Bohlender, A. Spriet, W. Tirry, and N. Madhu, “Spatially se- lective speaker separation using a DNN with a location dependent feature extraction,” IEEE/ACM TASLP, vol. 32, 2024

2024
[14]

All neural low-latency directional speech extraction,

A. Pandey, S. Lee, J. Azcarreta, D. Wong, and B. Xu, “All neural low-latency directional speech extraction,” inInterspeech, 2024

2024
[15]

ReZero: Region-customizable sound extrac- tion,

R. Gu and Y . Luo, “ReZero: Region-customizable sound extrac- tion,” IEEE/ACM TASLP, 2024

2024
[16]

LocSelect: Tar- get speaker localization with an auditory selective hearing mech- anism,

Y . Chen, X. Qian, Z. Pan, K. Chen, and H. Li, “LocSelect: Tar- get speaker localization with an auditory selective hearing mech- anism,” in IEEE ICASSP, 2024

2024
[17]

Continuous speech separation: Dataset and analysis,

Z. Chen, T. Yoshioka, L. Lu, T. Zhou, Z. Meng, Y . Luo, J. Wu, X. Xiao, and J. Li, “Continuous speech separation: Dataset and analysis,” in IEEE ICASSP, 2020

2020
[18]

The Fifth ’CHiME’ Speech Separation and Recognition Challenge: Dataset, Task and Baselines,

J. Barker, S. Watanabe, E. Vincent, and J. Trmal, “The Fifth ’CHiME’ Speech Separation and Recognition Challenge: Dataset, Task and Baselines,” inInterspeech, 2018

2018
[19]

3S-TSE: Ef- ficient three-stage target speaker extraction for real-time and low- resource applications,

S. He, J. Liu, H. Li, Y . Yang, F. Chen, and X. Zhang, “3S-TSE: Ef- ficient three-stage target speaker extraction for real-time and low- resource applications,” in IEEE ICASSP, 2024

2024
[20]

Improved deep speaker localization and tracking: Revised training paradigm and con- trolled latency,

A. Bohlender, L. Roelens, and N. Madhu, “Improved deep speaker localization and tracking: Revised training paradigm and con- trolled latency,” inIEEE ICASSP, 2023

2023
[21]

L-SpEx: Localized target speaker extraction,

M. Ge, C. Xu, L. Wang, E. S. Chng, J. Dang, and H. Li, “L-SpEx: Localized target speaker extraction,” in IEEE ICASSP, 2022

2022
[22]

Direction-aware speaker beam for multi-channel speaker extraction

G. Li, S. Liang, S. Nie, W. Liu, M. Yu, L. Chen, S. Peng, and C. Li, “Direction-aware speaker beam for multi-channel speaker extraction.” in Interspeech, 2019

2019
[23]

GCC-Speaker: Target speaker localization with optimal speaker-dependent weighting in multi-speaker scenarios,

G. Li, W. Xue, W. Liu, J. Yi, and J. Tao, “GCC-Speaker: Target speaker localization with optimal speaker-dependent weighting in multi-speaker scenarios,” in IEEE ICASSP, 2023

2023
[24]

Differentiable particle filters through conditional normalizing flow,

X. Chen, H. Wen, and Y . Li, “Differentiable particle filters through conditional normalizing flow,” inIEEE FUSION, 2021

2021
[25]

Exploiting temporal context in CNN based multisource DoA estimation,

A. Bohlender, A. Spriet, W. Tirry, and N. Madhu, “Exploiting temporal context in CNN based multisource DoA estimation,” IEEE/ACM TASLP, vol. 29, 2021

2021
[26]

A wrapped kalman filter for azimuthal speaker tracking,

J. Traa and P. Smaragdis, “A wrapped kalman filter for azimuthal speaker tracking,” IEEE Signal Proc. Letters , vol. 20, no. 12, 2013

2013
[27]

Multiple moving speaker tracking by microphone array on mobile robot,

M. Murase, S. Yamamoto, J.-M. Valin, K. Nakadai, K. Yamada, K. Komatani, T. Ogata, and H. Okuno, “Multiple moving speaker tracking by microphone array on mobile robot,” in Interspeech, 2005

2005
[28]

Particle filter algorithm for DoA track- ing using co-prime array,

F. Dong, L. Xu, and X. Li, “Particle filter algorithm for DoA track- ing using co-prime array,”IEEE Comm. Letters, vol. 24, 2020

2020
[29]

Particle filtering al- gorithms for tracking an acoustic source in a reverberant environ- ment,

D. Ward, E. Lehmann, and R. Williamson, “Particle filtering al- gorithms for tracking an acoustic source in a reverberant environ- ment,” IEEE Trans. on Speech and Audio Proc., vol. 11, 2003

2003
[30]

Speaker tracking based on dis- tributed particle filter in distributed microphone networks,

Q. Zhang, Z. Chen, and F. Yin, “Speaker tracking based on dis- tributed particle filter in distributed microphone networks,” IEEE Trans. on Systems, Man, and Cybern.: Systems, vol. 47, 2017

2017
[31]

KalmanNet: Neural network aided kalman filtering for partially known dynamics,

G. Revach, N. Shlezinger, X. Ni, A. L. Escoriza, R. J. G. van Sloun, and Y . C. Eldar, “KalmanNet: Neural network aided kalman filtering for partially known dynamics,” IEEE Trans. on Signal Proc., 2022

2022
[32]

Survey of maneuvering target tracking. Part I. Dynamic models,

X. Rong Li and V . Jilkov, “Survey of maneuvering target tracking. Part I. Dynamic models,” IEEE Trans. on Aerospace and Elec- tronic Systems, vol. 39, 2003

2003
[33]

SRP-DNN: Learning direct-path phase difference for multiple moving sound source localization,

B. Yang, H. Liu, and X. Li, “SRP-DNN: Learning direct-path phase difference for multiple moving sound source localization,” in IEEE ICASSP, 2022

2022
[34]

MIMO-DoAnet: Multi-channel input and multiple outputs DoA network with unknown number of sound sources,

H. Yin, M. Ge, Y . Fu, G. Zhang, L. Wang, L. Zhang, L. Qiu, and J. Dang, “MIMO-DoAnet: Multi-channel input and multiple outputs DoA network with unknown number of sound sources,” in Interspeech, 2022

2022
[35]

A learning-based approach to direction of arrival estimation in noisy and reverberant environments,

X. Xiao, S. Zhao, X. Zhong, D. L. Jones, E. S. Chng, and H. Li, “A learning-based approach to direction of arrival estimation in noisy and reverberant environments,” inIEEE ICASSP, 2015

2015
[36]

Deep net- works for DoA estimation in low SNR,

G. K. Papageorgiou, M. Sellathurai, and Y . C. Eldar, “Deep net- works for DoA estimation in low SNR,” IEEE Trans. on Signal Proc., vol. 69, 2021

2021
[37]

Lib- rispeech: An ASR corpus based on public domain audio books,

V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An ASR corpus based on public domain audio books,” in IEEE ICASSP, 2015

2015
[38]

Image method for efficiently simulating small-room acoustics,

J. Allen and D. Berkley, “Image method for efficiently simulating small-room acoustics,” J. Acoust. Soc. Am., vol. 65, 1979

1979
[39]

Table 1 displays the results using FT-JNF with different tracking and training strategies for three levels of speaker dynamics, complemented by fig

and ESTOI [40] respectively. Table 1 displays the results using FT-JNF with different tracking and training strategies for three levels of speaker dynamics, complemented by fig. 4 re- garding SI-SDR improvement with a refined resolution. Strongly guided speaker extraction To i...
[40]

Librimix: An open-source dataset for generalizable speech separation,

J. Cosentino, M. Pariente, S. Cornell, A. Deleforge, and E. Vincent, “Librimix: An open-source dataset for generalizable speech separation,” 2020. [Online]. Available: https://arxiv.org/ abs/2005.11262

2020 arXiv
[41]

Accurate adaptive filtering in square-root Hann windowed short-time fourier transform do- main,

S. Shimauchi and H. Ohmuro, “Accurate adaptive filtering in square-root Hann windowed short-time fourier transform do- main,” in IEEE ICASSP, 2014

2014
[42]

Neural directional filtering: Far-field directivity control with a small microphone array,

J. Wechsler, S. R. Chetupalli, M. M. Halimeh, O. Thiergart, and E. A. P. Habets, “Neural directional filtering: Far-field directivity control with a small microphone array,” inIWAENC, 2024

2024
[43]

Spatially constrained vs. uncon- strained filtering in neural spatiospectral filters for multichannel speech enhancement,

A. Briegleb and W. Kellermann, “Spatially constrained vs. uncon- strained filtering in neural spatiospectral filters for multichannel speech enhancement,” in EUSIPCO, 2024

2024
[44]

Utilizing head rotation data in DNN- based multi-channel speech enhancement for hearing aids,

B. Lentz and R. Martin, “Utilizing head rotation data in DNN- based multi-channel speech enhancement for hearing aids,” in IWAENC, 2024

2024
[45]

gpuRIR: A Python library for room impulse response simulation with GPU accelera- tion,

D. Diaz-Guerra, A. Miguel, and J. R. Beltr ´an, “gpuRIR: A Python library for room impulse response simulation with GPU accelera- tion,” Multimedia Tools and Applications, vol. 80, 2018

2018
[46]

SDR – Half-baked or well done?

J. L. Roux, S. Wisdom, H. Erdogan, and J. R. Hershey, “SDR – Half-baked or well done?” in IEEE ICASSP, 2019

2019
[47]

Perceptual eval- uation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,

A. Rix, J. Beerends, M. Hollier, and A. Hekstra, “Perceptual eval- uation of speech quality (PESQ)-a new method for speech quality assessment of telephone networks and codecs,” in IEEE ICASSP, 2001

2001
[48]

An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,

J. Jensen and C. H. Taal, “An algorithm for predicting the intelligi- bility of speech masked by modulated noise maskers,”IEEE/ACM TASLP, vol. 24, 2016

2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Introduction Given a real world recording containing a speech signal, the field of speech enhancement focuses on improving its quality and intelligibility by suppressing noise and reverberation arti- facts. Noise sources in form of additional interfering speech signals, such as in the so-called cocktail-party problem [1], are especially challenging due to...

work page Pith review arXiv 2025

[2] [2]

Target speaker extraction Let the multi-channel observation Y be modeled as additive mixture of reverberant target speech X with noise V

Problem definition 2.1. Target speaker extraction Let the multi-channel observation Y be modeled as additive mixture of reverberant target speech X with noise V. The lat- ter contains interfering speech sources as well as additive en- vironmental and measurement noise. In the short-time Fourier transform (STFT) domain, this acoustic setup is resembled by ...

work page

[3] [3]

Proposed method 3.1. Weakly guided target speaker extraction Instead of relying on accurate continuous prior directional in- formation, we relax this constraint and only assume knowl- edge about the initial starting direction θ0 of the target speaker. In order to retain compatibility with conventional SSFs like [3, 4, 5, 6], we propose to incorporate an u...

work page

[4] [4]

Dataset For training and evaluation we create a synthetic dataset with reverberant two-speaker mixtures

Experiments 4.1. Dataset For training and evaluation we create a synthetic dataset with reverberant two-speaker mixtures. Specifically, we use utter- ances from the LibriSpeech [29] corpus, convolve them with simulated room impulse responses (RIRs) based on the image method [30] and then combine them according to the recipe in Libri2Mix [31]. Since no add...

work page

[5] [5]

distortions, perceptual quality and intelligibility via the metrics SI-SDR [38], PESQ

Results We evaluate the TSE performance w.r.t. distortions, perceptual quality and intelligibility via the metrics SI-SDR [38], PESQ

work page

[6] [6]

Conclusion In this work we proposed a weakly guided TSE pipeline which solely requires information about the target’s initial direction. By developing a synthetic dataset with continuous speaker movement, we could not only demonstrate the significant per- formance improvement of a commonly used SSF in dynamic environments, but also its capability to resol...

work page

[7] [7]

Acknowledgments This work was supported by the Deutsche Forschungsge- meinschaft (DFG, German Research Foundation) under grant 508337379. Computational resources were provided by Re- gional Computer Center (RRZ) of the University of Hamburg (UHH) and Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universit ¨at Erl...

work page

[8] [8]

Vary and R

P. Vary and R. Martin, Digital Speech Transmission: Enhance- ment, Coding and Error Concealment. Hoboken, NY , USA: Wi- ley, 2006

work page 2006