Pith. sign in

REVIEW 3 major objections 5 minor 50 references

AVFSNet: Audio-Visual Speech Separation for Flexible Number of Speakers with Multi-Scale and Multi-Task Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AVFSNet separates speech from mixtures with unknown speaker counts by separating every visible candidate in parallel and then counting which outputs are real.

desk verdict A solid fixed-count separation system whose 'unknown number of speakers' claim is untested: every experiment feeds as many lip tracks as active speakers. read the letter →

arxiv 2507.12972 v1 pith:IMAPATQD submitted 2025-07-17 eess.AS cs.SD

Daning Zhang , Ying Wei This is my paper
classification eess.AScs.SD
keywords audio-visualspeechseparationunknownnumberofspeakersspeakercountingmulti-scaleaudioencoderparallelarchitecturemulti-tasklearningtargetextractionnoiserobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AVFSNet is an audio-visual speech separation system designed for mixtures where the number of active speakers is not known in advance. It proposes to separate every visible candidate speaker in parallel and then use a learned classifier to decide which separated outputs are real speech, a strategy the paper calls separation-before-counting. Under this design, the model reports 13.56 dB SI-SDR improvement on LRS2 three-speaker mixtures and 12.95 dB on VoxCeleb2 three-speaker mixtures after transfer training, exceeding its transformer backbone by roughly 1.3 dB, and it retains positive improvement on mixtures with up to ten speakers. The paper argues that such a system matters because real cocktail-party scenes rarely come with a known head count, and audio-only flexible-count separators degrade sharply in noise.

What carries the argument

The load-bearing mechanism is the parallel separator plus a mask-validity classifier. The audio encoder is a multi-scale Branchformer-style encoder with two branches—multi-head self-attention for global context and a convolutional-gated MLP for local patterns—whose outputs are weighted and summed. The mixture feature is chunked, replicated M times, and fed to M shared-weight audio-visual separation modules; each module uses visual features as attention queries against audio keys and values, then reconstructs a full-length mask by overlap-add. The speaker counting module takes each mask through convolutional pooling, self-attention, and an MLP with sigmoid output to produce a per-candidate presence probability. The paper's separation-before-counting ordering is what lets the system avoid recursive error accumulation and avoid decoders that scale with speaker count.

What would settle it

Build a test set with three visible lip tracks but only two active speakers (or, more generally, M > N) from VoxCeleb2 or LRS2; if the counting module cannot reject the silent candidate's phantom mask, so that speaker-count accuracy drops sharply or separation metrics fall to zero for misjudged speakers, then the system is not truly handling unknown speaker counts and is instead a fixed-count separator with an extra classifier.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an unknown-number audio-visual separator can be built without estimating the speaker count in advance. The model accepts the lip movements of all M candidate speakers plus the mixed audio, runs the same shared-weight separation module once per candidate in parallel, and only then estimates a presence probability for each candidate's mask. The counting module treats the task as binary classification over separator outputs: masks produced from a real speaker should look different from 'phantom' masks produced when the input mixture does not contain that speaker's voice. Because every branch is independent, later speakers do not inherit errors from earlier iterations, and adding more candidates only means instantiating more copies of the shared module, not redesigning the decoder. The reported numbers—12.93 dB and 14.34 dB SI-SDRi on two-speaker VoxCeleb2 and LRS2 mixtures, 12.95 dB and 13.56 dB after transfer to three-speaker mixtures, and 84.74% speaker-count accuracy on mixed 2/3-speaker data—support the claim that count-agnostic separation is achievable with visual guidance.

Load-bearing premise

The argument assumes that the video supplies synchronized lip tracks for every speaker who might appear, that the actual number of speakers never exceeds the number of visible candidates, and that masks produced for absent speakers are reliably distinguishable from real speech.

Editorial extensions

If this is right

  • A system trained with this design can be applied to a mixture without being told how many speakers are in it, as long as synchronized lip tracks for all potential speakers are supplied.
  • Because the separation branches share weights, the architecture has no built-in upper limit on the number of candidates; the reported 10-speaker test is a direct demonstration of that extrapolation.
  • Counting accuracy improves when counting is done on separated masks rather than on the mixture: the paper's module reaches 84.74% speaker-count accuracy on mixed 2/3-speaker test data, well above the 59.87% of the convolutional-encoder baseline.
  • Transfer learning from 2-speaker to 3-speaker mixtures is the recommended training schedule: directly mixing speaker counts during training gives worse separation (6.43 vs 13.56 dB SI-SDRi on LRS2), while fine-tuning preserves earlier knowledge.
  • In noisy conditions, improvements over the dual-path RNN baseline grow as SNR drops: at -20 to -10 dB input SNR AVFSNet's SI-SDRi advantage is about 4.6 dB, and it stays usable where the baseline collapses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: every evaluation in the paper uses as many visible candidates as active speakers, so the decisive untested case is a video with more visible candidates than active speakers; if the counting module cannot reject a silent candidate's phantom mask, the 'unknown count' claim reduces to a fixed-count separator with a classifier.
  • Editorial inference: the counting module's performance depends on separated masks being statistically distinguishable for real and phantom speakers, so a useful diagnostic the paper does not report is the distribution of presence probabilities for silent candidates; if those probabilities overlap heavily with real speakers, the fixed threshold in the inference rule will trade off false positives a
  • Editorial inference: since the method requires lip tracks for every candidate, it inherits the limitations of face and lip detection; a natural next test is mixtures where one speaker is off-screen or has occluded lips, and the model should fail to separate that speaker, which would bound the method's applicability to visible talkers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes AVFSNet, an audio-visual speech separation model for mixtures with a flexible (unknown) number of speakers. The architecture comprises a Branchformer-based multi-scale audio encoder, a visual encoder for lip tracks, a parallel independent separator with shared-parameter AV-Sepformer-style modules, and a speaker counting module that classifies each produced mask as valid or phantom. Training is staged: the backbone is first trained on two-speaker mixtures and fine-tuned on three-speaker mixtures; the counting module is trained on positive/negative audio-visual pairs using the frozen backbone; and the whole network is then fine-tuned with a multi-task loss. Evaluations on VoxCeleb2 and LRS2 with two- and three-speaker mixtures show consistent SI-SDRi improvements over the AV-Sepformer backbone, and an extrapolation study extends to ten speakers. The central claim is that the model handles an unknown number of speakers without prior knowledge of the count.

Significance. The manuscript has clear strengths: the overall architecture is well motivated, the staged training procedure is sensible, and the reported separation gains are substantial (e.g., about 1.3 dB SI-SDRi over the AV-Sepformer backbone on three-speaker mixtures after transfer learning). The ablation studies usefully isolate the contributions of the multi-scale encoder and the counting module. However, the central contribution—separation for an unknown number of speakers—is not convincingly substantiated. No evaluation condition presents the model with more candidate speakers than active speakers, so the counting module's rejection of phantom masks is never tested at inference. If the missing M>N evaluation is added and confirms the approach, the work would be a solid contribution to audio-visual separation; as it stands, the evidence mainly demonstrates a strong all-candidates-active separator.

major comments (3)
  1. [5.1, 5.3, 4.1] All test conditions set the number of candidate speakers M equal to the number of active speakers N. Specifically, VoxCeleb2-2mix, VoxCeleb2-3mix, LRS2-2mix, LRS2-3mix, and the extrapolation study of Table 9 supply exactly the lip tracks of the speakers present in each mixture. Consequently, the speaker counting module is never asked to reject a phantom mask at inference, yet this rejection capability is the load-bearing component of the 'unknown number of speakers' claim (Contribution 1). Without such a condition, the system is, in practice, a fixed-count separator with an untested classifier. Please add test conditions with M>N (e.g., two active speakers with three provided lip tracks) and report both separation metrics and counting accuracy on those conditions.
  2. [3.7, 4.2.1] The counting module assumes (Section 3.7) that masks generated by the separator for non-existent speakers are statistically distinguishable from those for active speakers. The negative pairs used in stage-2 training (Section 4.2.1) are constructed by pairing the mixture with a lip track of a speaker absent from the mixture, which tests cross-mixture rejection. This does not cover the within-mixture case where a candidate face is visible but inactive, and the two cases may differ because the separator is conditioned on visual features of every candidate. Please train on within-mixture negative examples or justify that the two scenarios produce indistinguishable mask statistics, and add test data with visible-but-silent candidates.
  3. [Table 7] The speaker counting accuracy (SCA) in Table 7 is computed on VoxCeleb2-2&3mix, where every candidate speaker is active in every mixture. In that setting, a classifier that always predicts 'present' would achieve perfect SCA, so the reported numbers do not demonstrate the counting module's ability to detect inactive candidates. Report SCA on mixtures containing phantom candidates (M>N), and include the false-alarm rate or precision/recall for the phantom class.
minor comments (5)
  1. [1, Contribution 1] The phrase 'without relying on any visual prior information' is imprecise: the method requires lip tracks of all M candidate speakers, which is visual information. The intended meaning (no prior knowledge of the speaker count) should be stated explicitly to avoid misleading readers.
  2. [4.4] The comparison methods AV-DPRNN and AV-Sepformer are said to be reimplemented with identical transfer learning configurations, but no implementation details are given. Please describe the reimplementations or provide code to make the results reproducible.
  3. [4.2.3, Eq. (20)] The presence threshold tau is a free parameter, but its value and selection procedure are never specified. State the chosen value and report sensitivity of the final output to tau.
  4. [5.3.2, Table 9] The tables in Section 5.3.2 mix the notations SDR and SDRi; for multi-speaker mixtures, please clarify whether the reported SDR is computed with an oracle assignment of estimated signals to references and unify the notation.
  5. [Figure 1, Section 3.2] There are minor typos: 'Av-Separatoion-Module' in Figure 1 and 'Framwork' in Section 3.2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the flexible-count pipeline is a supervised classifier over a shared-weight backbone with external ground-truth labels, and the only overlapping-author citation is a non-load-bearing ablation baseline.

full rationale

The paper's claimed derivations are not circular. AVFSNet separates M candidate speakers in parallel (Eq. 10), and the speaker counting module is a learned binary classifier on the separator's mask outputs (Sec. 3.7, Eq. 15), with final sources selected by a threshold on predicted probabilities (Eq. 20). This is a supervised pipeline, not an identity: the counting module's labels are external ground-truth speaker-presence labels from positive/negative audiovisual pairs (Sec. 4.2.1), and its accuracy (SCA 84.74%, Table 7) is measured against those labels rather than derived from the separation loss. Training the counting module on masks from the frozen backbone and then fine-tuning jointly is a forward dependency — the classifier is matched to its input distribution — not circular reasoning, and no equation reduces the reported SI-SDRi or SCA to a fitted parameter. The separation objective (SI-SNR, Eq. 16) contains no speaker-count term, so the flexible-count behavior is not hard-coded by construction, and the misjudged-speaker metric convention (Sec. 4.5, setting the metric to 0 on counting errors) is a penalty that cannot inflate the results. The one overlapping-author citation, [33] (Liu, Shi, Wei; co-author Ying Wei), appears only as an ablation baseline (A2 in Fig. 7) and as one of several references in the multi-scale-encoding motivation; its comparison is measured in this paper, so the citation is not load-bearing. External benchmarks (AV-Sepformer, AV-DPRNN, MuSE, SEANet, all reimplemented under identical transfer settings) provide independent comparison. The genuine weakness is a validation gap, not circularity: every experiment sets M = N, so the counting module is never tested on visible-but-silent candidates (within-mixture phantom masks), leaving the central 'unknown number' claim only partially verified. That concern belongs to correctness risk, not to the circularity definition (Hard Rule 5).

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper contributes no new mathematical derivation; it relies on standard deep learning assumptions. The main load-bearing assumptions are availability of candidate lip tracks, the bound N<=M, the linear mixing model, and the learnability of the phantom-mask distinction. The threshold tau is a free operational parameter not specified in the paper.

free parameters (1)
  • Presence probability threshold tau = Not reported
    Used in Eq. (20) to retain speakers with p_i >= tau. It controls false positives and false negatives in the count and is not fixed or justified in the paper.
assumptions (5)
  • domain assumption N <= M and no new speakers join during the activity (Section 3.1).
    Binds the active speaker count to the number of detected faces; the system cannot handle a speaker who is off-screen or joins mid-recording.
  • domain assumption Synchronized lip tracks are available for every candidate speaker M (Section 3.2).
    The entire pipeline is keyed on these tracks; there is no fallback if face detection, tracking, or lip extraction fails.
  • domain assumption Mixtures are instantaneous linear combinations of source signals plus noise (Eq. 1).
    Underlies the waveform reconstruction through mask application; real reverberation and nonlinear distortions violate this model.
  • domain assumption Separator outputs for phantom speakers are statistically distinguishable from real-speaker outputs (Section 3.7).
    The counting module reduces to binary classification on masks; if this learnability property fails, counting cannot work.
  • ad hoc to paper Training first on 2-speaker mixtures and then fine-tuning on 3-speaker mixtures yields robust generalization to other counts (Section 4.2.1).
    This schedule is adopted because direct mixing gave worse results; it is not derived and its extrapolation to unseen counts is only tested in Table 9.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AVFSNet: Audio-Visual Speech Separation for Flexible Number of Speakers with Multi-Scale and Multi-Task Learning." pith.science (2026). https://pith.science/paper/IMAPATQD

@misc{pith2026250712972,
  author       = {Pith},
  title        = {Pith review of: AVFSNet: Audio-Visual Speech Separation for Flexible Number of Speakers with Multi-Scale and Multi-Task Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IMAPATQD}},
  note         = {Machine review of arXiv:2507.12972}
}
read the original abstract

Separating target speech from mixed signals containing flexible speaker quantities presents a challenging task. While existing methods demonstrate strong separation performance and noise robustness, they predominantly assume prior knowledge of speaker counts in mixtures. The limited research addressing unknown speaker quantity scenarios exhibits significantly constrained generalization capabilities in real acoustic environments. To overcome these challenges, this paper proposes AVFSNet -- an audio-visual speech separation model integrating multi-scale encoding and parallel architecture -- jointly optimized for speaker counting and multi-speaker separation tasks. The model independently separates each speaker in parallel while enhancing environmental noise adaptability through visual information integration. Comprehensive experimental evaluations demonstrate that AVFSNet achieves state-of-the-art results across multiple evaluation metrics and delivers outstanding performance on diverse datasets.

Figures

Figures reproduced from arXiv: 2507.12972 by the authors.

Figure 1
Figure 1. Overall architecture of the proposed AVFSNet. The system employs audiovisual multimodal [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Proposed Branchformer-based audio encoder. The architecture contains two parallel branches: a [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Convolutional spatial gating unit in au [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Separator architecture schematic. The parallel-structured separator employs flexible-quantity [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Schematic of speaker counting module 4. Experiments 4.1. Datasets 4.1.1. VoxCeleb2 VoxCeleb2 [44] comprises approximately 1 million speech clips, totaling over 2,000 hours. It includes nearly 6,000 speakers with diverse ages, genders, accents, and nationalities, and ca…
Figure 7
Figure 7. Figure 7: Seven kinds of architecture for ablation study. A1 incorporates the traditional convolutional audio [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Visualization of 8 target speaker separation samples by AVFSNet on VoxCeleb2. Compared with [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

50 extracted references · 49 canonical work pages

  1. [1]

    D. Wang, J. Chen, Supervised speech separation based on deep learning: An overview, IEEE/ACM transactions on audio, speech, and language processing 26 (10) (2018) 1702–1726

    work page 2018

  2. [2]

    Y. Liu, D. Wang, Divide and conquer: A deep casa approach to talker-independent monaural speaker separation, IEEE/ACM Transactions on audio, speech, and language processing 27 (12) (2019) 2092– 2102

    work page 2019

  3. [3]

    Chandrakala, S

    S. Chandrakala, S. Jayalakshmi, Generative model driven representation learning in a hybrid framework for environmental audio scene and sound event recognition, IEEE Transactions on Multimedia 22 (1) (2019) 3–14

    work page 2019

  4. [4]

    X. Liu, X. Liu, D. Guo, H. Liu, F. Sun, H. Min, Self-supervised learning for alignment of objects and sound, in: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2020, pp. 1588–1594

    work page 2020

  5. [5]

    Y. Luo, N. Mesgarani, Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation, IEEE/ACM transactions on audio, speech, and language processing 27 (8) (2019) 1256– 1266. 23

    work page 2019

  6. [6]

    R. Gao, K. Grauman, Visualvoice: Audio-visual speech separation with cross-modal consistency, in: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2021, pp. 15490–15500

    work page 2021

  7. [7]

    Subakan, M

    C. Subakan, M. Ravanelli, S. Cornell, M. Bronzi, J. Zhong, Attention is all you need in speech separa- tion, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 21–25

    work page 2021

  8. [8]

    K. Li, F. Xie, H. Chen, K. Yuan, X. Hu, An audio-visual speech separation model inspired by cortico- thalamo-cortical circuits, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    work page 2024

Show all 50 references
  1. [9]

    Subakan, M

    C. Subakan, M. Ravanelli, S. Cornell, F. Grondin, Real-m: Towards speech separation on real mixtures, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 6862–6866

    2022

  2. [10]

    Saijo, W

    K. Saijo, W. Zhang, S. Cornell, R. Scheibler, C. Li, Z. Ni, A. Kumar, M. Sach, Y. Fu, W. Wang, et al., Interspeech 2025 urgent speech enhancement challenge, arXiv preprint arXiv:2505.23212 (2025)

    2025 arXiv

  3. [11]

    J. Wu, Y. Xu, S.-X. Zhang, L.-W. Chen, M. Yu, L. Xie, D. Yu, Time domain audio visual speech separation, in: 2019 IEEE automatic speech recognition and understanding workshop (ASRU), IEEE, 2019, pp. 667–673

    2019

  4. [12]

    J. R. Hershey, Z. Chen, J. Le Roux, S. Watanabe, Deep clustering: Discriminative embeddings for segmentation and separation, in: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2016, pp. 31–35

    2016

  5. [13]

    J. Chen, Q. Mao, D. Liu, Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation, in: Proc. Interspeech 2020, 2020, pp. 2642–2646

    2020

  6. [14]

    Martel, J

    H. Martel, J. Richter, K. Li, X. Hu, T. Gerkmann, Audio-visual speech separation in noisy environments with a lightweight iterative model, in: INTERSPEECH, 2023

    2023

  7. [15]

    K. Li, R. Yang, F. Sun, X. Hu, Iianet: An intra-and inter-modality attention network for audio-visual speech separation, in: International Conference on Machine Learning, PMLR, 2024, pp. 29181–29200

    2024

  8. [16]

    Vasantha, B

    S. Vasantha, B. Kiranmai, M. A. Hussain, S. S. Hashmi, L. Nelson, S. Hariharan, Face and object detec- tion algorithms for people counting applications, in: 2023 2nd International Conference on Automation, Computing and Renewable Systems (ICACRS), IEEE, 2023, pp. 1188–1193

    2023

  9. [17]

    P. Ma, Y. Wang, J. Shen, S. Petridis, M. Pantic, Lip-reading with densely connected temporal convo- lutional networks, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 2857–2866

    2021

  10. [18]

    Nachmani, Y

    E. Nachmani, Y. Adi, L. Wolf, Voice separation with an unknown number of multiple speakers, in: International Conference on Machine Learning, PMLR, 2020, pp. 7164–7175

    2020

  11. [19]

    Kinoshita, L

    K. Kinoshita, L. Drude, M. Delcroix, T. Nakatani, Listening to each speaker one by one with recurrent selective hearing networks, in: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2018, pp. 5064–5068

    2018

  12. [20]

    J. Shi, X. Chang, P. Guo, S. Watanabe, Y. Fujita, J. Xu, B. Xu, L. Xie, Sequence to multi-sequence learning via conditional chain mapping for mixture signals, Advances in Neural Information Processing Systems 33 (2020) 3735–3747

    2020

  13. [21]

    S. R. Chetupalli, E. A. Habets, Speech separation for an unknown number of speakers using transformers with encoder-decoder attractors., in: INTERSPEECH, 2022, pp. 5393–5397

    2022

  14. [22]

    Takahashi, S

    N. Takahashi, S. Parthasaarathy, N. Goswami, Y. Mitsufuji, Recursive speech separation for unknown number of speakers, in: Proc. Interspeech 2019, 2019, pp. 1348–1352

    2019

  15. [23]

    J. Zhu, R. A. Yeh, M. Hasegawa-Johnson, Multi-decoder dprnn: Source separation for variable number of speakers, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 3420–3424

    2021

  16. [24]

    Maiti, Y

    S. Maiti, Y. Ueda, S. Watanabe, C. Zhang, M. Yu, S.-X. Zhang, Y. Xu, Eend-ss: Joint end-to-end neural speaker diarization and speech separation for flexible number of speakers, in: 2022 IEEE Spoken Language Technology Workshop (SLT), IEEE, 2023, pp. 480–487

    2022

  17. [25]

    J. Ao, M. S. Yıldırım, R. Tao, M. Ge, S. Wang, Y. Qian, H. Li, Used: Universal speaker extraction and diarization, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2024). 24

    2024

  18. [26]

    S. R. Chetupalli, E. A. Habets, Speaker counting and separation from single-channel noisy mixtures, IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023) 1681–1692

    2023

  19. [27]

    Y. Luo, Z. Chen, T. Yoshioka, Dual-path rnn: efficient long sequence modeling for time-domain single- channel speech separation, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 46–50

    2020

  20. [28]

    J. Lin, X. Cai, H. Dinkel, J. Chen, Z. Yan, Y. Wang, J. Zhang, Z. Wu, Y. Wang, H. Meng, Av- sepformer: Cross-attention sepformer for audio-visual target speaker extraction, in: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ...

    2023

  21. [29]

    J. Lei, Y. He, Y. Wang, Mffn: Multi-level feature fusion network for monaural speech separation, Speech Communication 171 (2025) 103229

    2025

  22. [30]

    M. W. Lam, J. Wang, D. Su, D. Yu, Effective low-cost time-domain audio separation using globally attentive locally recurrent networks, in: 2021 IEEE Spoken Language Technology Workshop (SLT), IEEE, 2021, pp. 801–808

    2021

  23. [31]

    K. Tan, B. Xu, A. Kumar, E. Nachmani, Y. Adi, Sagrnn: Self-attentive gated rnn for binaural speaker separation with interaural cue preservation, IEEE Signal Processing Letters 28 (2020) 26–30

    2020

  24. [32]

    J. Chen, Z. Wang, D. Tuo, Z. Wu, S. Kang, H. Meng, Fullsubnet+: Channel attention fullsubnet with complex spectrograms for speech enhancement, in: ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2022, pp. 7857–7861

    2022

  25. [33]

    Z. Liu, D. Shi, Y. Wei, Multi-kernel attention encoder for time-domain speech separation, in: 2024 IEEE International Symposium on Circuits and Systems (ISCAS), IEEE, 2024, pp. 1–5

    2024

  26. [34]

    Gulati, J

    A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu, et al., Conformer: Convolution-augmented transformer for speech recognition, in: Proc. Interspeech 2020, 2020, pp. 5036–5040

    2020

  27. [35]

    Y. Peng, S. Dalmia, I. Lane, S. Watanabe, Branchformer: Parallel mlp-attention architectures to capture local and global context for speech recognition and understanding, in: International Conference on Machine Learning, PMLR, 2022, pp. 17627–17643

    2022

  28. [36]

    Y. Li, F. Wang, Y. Chen, A. Cichocki, T. Sejnowski, The effects of audiovisual inputs on solving the cocktail party problem in the human brain: An fmri study, Cerebral Cortex 28 (10) (2018) 3623–3637

    2018

  29. [37]

    Stenzel, J

    H. Stenzel, J. Francombe, P. J. Jackson, Limits of perceived audio-visual spatial coherence as defined by reaction time measurements, Frontiers in neuroscience 13 (2019) 451

    2019

  30. [38]

    Lee, S.-W

    J. Lee, S.-W. Chung, S. Kim, H.-G. Kang, K. Sohn, Looking into your speech: Learning cross-modal affinity for audio-visual speech separation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1336–1345

    2021

  31. [39]

    Ephrat, I

    A. Ephrat, I. Mosseri, O. Lang, T. Dekel, K. Wilson, A. Hassidim, W. T. Freeman, M. Rubinstein, Looking to listen at the cocktail party: a speaker-independent audio-visual model for speech separation, ACM Transactions on Graphics (TOG) 37 (4) (2018) 1–11

    2018

  32. [40]

    Z. Pan, R. Tao, C. Xu, H. Li, Muse: Multi-modal target speaker extraction with visual cues, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 6678–6682

    2021

  33. [41]

    Z. Pan, M. Ge, H. Li, Usev: Universal speaker extraction with visual cue, IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2022) 3032–3045

    2022

  34. [42]

    St¨ oter, S

    F.-R. St¨ oter, S. Chakrabarty, B. Edler, E. A. Habets, Countnet: Estimating the number of concur- rent speakers using supervised learning, IEEE/ACM Transactions on Audio, Speech, and Language Processing 27 (2) (2018) 268–282

    2018

  35. [43]

    Horiguchi, Y

    S. Horiguchi, Y. Fujita, S. Watanabe, Y. Xue, K. Nagamatsu, End-to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors, in: 21st Annual Conference of the International Speech Communication Association, INTERSPEECH 2020, International...

    2020

  36. [44]

    J. S. Chung, A. Nagrani, A. Zisserman, Voxceleb2: Deep speaker recognition, in: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Vol. 25 2018, 2018, pp. 1086–1090

    2018

  37. [45]

    Afouras, J

    T. Afouras, J. S. Chung, A. Senior, O. Vinyals, A. Zisserman, Deep audio-visual speech recognition, IEEE transactions on pattern analysis and machine intelligence 44 (12) (2018) 8717–8727

    2018

  38. [46]

    Le Roux, S

    J. Le Roux, S. Wisdom, H. Erdogan, J. R. Hershey, Sdr–half-baked or well done?, in: ICASSP 2019- 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 626–630

    2019

  39. [47]

    Kendall, Y

    A. Kendall, Y. Gal, R. Cipolla, Multi-task learning using uncertainty to weigh losses for scene geometry and semantics, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7482–7491

    2018

  40. [48]

    R. Tao, X. Qian, Y. Jiang, J. Li, J. Wang, H. Li, Audio-visual target speaker extraction with selective auditory attention, IEEE Transactions on Audio, Speech and Language Processing (2025)

    2025

  41. [49]

    A. W. Rix, J. G. Beerends, M. P. Hollier, A. P. Hekstra, Perceptual evaluation of speech quality (pesq)-a new method for speech quality assessment of telephone networks and codecs, in: 2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings ...

    2001

  42. [50]

    C. H. Taal, R. C. Hendriks, R. Heusdens, J. Jensen, A short-time objective intelligibility measure for time-frequency weighted noisy speech, in: 2010 IEEE international conference on acoustics, speech and signal processing, IEEE, 2010, pp. 4214–4217. 26

    2010

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.