Pith. sign in

REVIEW 3 major objections 4 minor 56 references

Identity-Faithful Audio-Visual Target Speaker Extraction with QIANGDA and VOXBLINK2-AVSE

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that audio-visual target speaker extraction should be tested as a paired identity test: the same mixture must yield speaker A under A's lip cue and speaker B under B's, and provides a benchmark and model that meets it on 6

desk verdict A genuinely useful new benchmark and corpus for AV-TSE, with honest caveats; the headline numbers rest on an unvalidated overlap/identity proxy, but the paper deserves refereeing. read the letter →

arxiv 2608.03964 v1 pith:SANUBS7C submitted 2026-08-04 eess.AS

Peijun Yang , Zhan Jin , Juan Liu , Ming Li This is my paper
classification eess.AS
keywords audio-visualtargetspeakerextractionidentityfaithfulnesspairedtestreal-scenebenchmarkMandarinspeechmixturesAV-HuBERTTF-GridNetFiLMconditioning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that audio-visual target speaker extraction should be judged by identity faithfulness: whether the separator follows the requested speaker, not merely whether it outputs clean speech. To make that testable, it records real Mandarin two-speaker mixtures in which the same overlap is queried twice, once with speaker A's lip cue and once with speaker B's; a valid extractor must return A in the first query and B in the second, otherwise it is visually bypassing the cue. The paper contributes QIANGDA, a benchmark of 77 scenes and 6,038 dual-annotated mixtures with in-scene solo references, and VOXBLINK2-AVSE, a curated 766-hour, identity-diverse training set built by audio-visual validation and DNSMOS quality filtering. On the complete manifest, the best checkpoint reaches 0.2261 character error rate, 82.22% strict output correctness, and 69.53% both-output strict success, where strict means the output passes an overlap detector and has the correct speaker identity. The paper notes that the strongest checkpoint's exact training command was not fully preserved, so the headline numbers are checkpoint-level observations rather than controlled causal claims.

What carries the argument

The load-bearing object is the paired identity test (Eq. 1): f(x, v_A) approximately s_A and f(x, v_B) approximately s_B on the same mixture x. It converts identity faithfulness from a property of one output into a relation between two visual queries of one recording, and it is what exposes visual bypass. The evaluation side uses an OSD-first protocol: a WavLM overlap detector marks outputs with residual overlap, and only single-speaker outputs are scored by the WeSpeaker margin for identity; strict correctness requires no detected overlap plus positive target margin. The extractor carries the argument with frozen 1,280-dimensional AV-HuBERT visual features, a TF-GridNet separator, input-lev

What would settle it

Take a random sample of outputs from the manifest and have trained listeners label residual overlap and speaker identity; if the human labels disagree with the WavLM-plus-margin pipeline beyond a small tolerance, the reported 82.22% and 69.53% rates would need to be recomputed. A second check: record additional scenes with close-talk microphones and use those channels as approximate clean references to measure actual source-to-distortion ratio for the non-overlap portion.

Watch

Extended reading notes

Core claim

The central claim is that identity-faithful audio-visual target speaker extraction can be measured on real two-speaker recordings without isolated clean sources, by making every mixture a paired identity test. For a jointly recorded overlap x = s_A + s_B + n, the extractor must satisfy f(x, v_A) approximately equal to s_A and f(x, v_B) approximately equal to s_B: the same audio, queried with A's visual cue, must come back as A, and queried with B's cue, must come back as B. This paired constraint turns the failure mode called visual bypass—a separator ignoring the video and always emitting the acoustically dominant voice—into a directly measurable quantity. The paper reports that the best ch

Load-bearing premise

The evaluation assumes that the automatic overlap detector plus speaker-identity margin correctly labels each output as single-speaker and identity-correct, even though the real overlap itself has no ground-truth separation to check against.

Editorial extensions

If this is right

  • Any benchmark that scores each visual query in isolation can miss visual bypass; the paired both-output strict success rate is the metric that catches it.
  • Matched training data outweigh capacity: the VOXBLINK2-trained six-block checkpoint beats the LRS3-trained eight-block and HUGE configurations on both CER and strict correctness.
  • Layer-wise FiLM is associated with substantially less residual overlap than input-only fusion in the reported checkpoints (12.45% vs 63.85%), even though those checkpoints are not a controlled ablation.
  • The auxiliary speaker-margin loss did not produce a clear benefit in this run; the paper explicitly leaves its causal effect to matched retraining.
  • QIANGDA's evaluation protocol can be applied to any AV-TSE model without needing target clean waveforms, which makes real-scene benchmarking feasible outside synthetic mixtures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same paired-query design could generalize to enrollment-based or text-conditioned extraction: if two different cues on the same mixture must return two different speakers, it directly tests whether the cue, not the acoustic prominence, drives the output.
  • A mismatch test—presenting A's lip cue while the target transcript is B's—would stress whether identity is controlled by video or by what is acoustically easiest; QIANGDA's fixed mapping could be extended to score that.
  • Because the OSD threshold and detector choice shift absolute rates, future benchmark versions might report results as a function of threshold and detector, or with human-labeled subsets, rather than as single numbers.
  • The curated training corpus opens a testable question about scale versus quality: whether 766 hours of DNSMOS-filtered, identity-diverse pairs is a better operating point than more raw hours for real-scene generalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces QIANGDA, a Mandarin audio-visual target speaker extraction (AV-TSE) benchmark based on 77 real jointly recorded two-speaker scenes (7,598 clips, 11.84 hours) with synchronized multi-view video and A-only/B-only in-scene references, yielding 6,038 evaluable mixtures and 12,076 target-speaker evaluation rows. It also curates VOXBLINK2-AVSE, a 766-hour, 28,421-identity audio–lip-ROI training corpus from VoxBlink2 after audio-visual validation and DNSMOS filtering. The proposed extractor combines frozen AV-HuBERT visual features with a TF-GridNet separator and layer-wise FiLM, optionally aided by a speaker-margin loss. Evaluation uses Qwen3-ASR CER and a compound identity proxy (WavLM OSD plus WeSpeaker ResNet34 margin). The best archived checkpoint is reported to achieve 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success.

Significance. If the benchmark and evaluation are valid, QIANGDA addresses a real gap: synthetic mixtures cannot test whether an AV-TSE model actually follows the visual cue on real two-speaker recordings, and the paired A/B visual-query design is a sensible way to detect visual bypass. The release of VOXBLINK2-AVSE with speaker-disjoint splits is also a useful training resource. The paper is appropriately cautious in several places, acknowledges confounds, and uses external independently trained evaluation models, which limits circularity concerns. However, the headline quantitative claims rest on an unvalidated automatic proxy, and the best model's training configuration is not fully reproducible. These issues are load-bearing rather than cosmetic, so the paper needs substantial revision before the numerical results can be accepted as stated.

major comments (3)
  1. [§6.4, Eq. (5), Table 5] The headline metrics (82.22% strict correctness, 69.53% both-output strict success) are defined entirely by an unvalidated compound proxy: an output is 'strict correct' only if the WavLM OSD mean posterior over 2.0-s windows is below 0.35 and the WeSpeaker ResNet34 margin is positive. No ground-truth separation exists for QIANGDA, and no calibration of either component is reported on the QIANGDA domain. The OSD model and WeSpeaker encoder are external, trained largely on English data, and are applied to 16-kHz reverberant Mandarin; the reported threshold stability (0.25–0.45) is not a validity check. If OSD misses brief residual intrusions diluted by window averaging, or assigns identity incorrectly in reverberation, the strict-correctness and both-success rates overstate identity faithfulness. Please add a validation study (e.g., using synthetic mixtures from the A-only/B-only stages, o
  2. [§6.5, Table 5, row 8 (†)] The best checkpoint—the one giving 0.2261 CER, 82.22% strict correctness, and 69.53% both-strict success—is stated to have a launch CLI that was not fully preserved. This undermines the central claim that the proposed extractor achieves these results, because the reader cannot reproduce the model, and the absence of exact training configuration also prevents a clean attribution of the speaker-loss ablation. Please archive the complete training command, data sampling order/seed, and package versions, or re-run the configuration and report reproducible numbers. As written, the paper's headline result is not verifiable.
  3. [§6.5, Tables 4 and 5] The speaker-margin loss ablation is not matched: the no-speaker-loss checkpoint is the one whose CLI was not preserved, and all rows are single checkpoints with no variance estimate. The paper correctly says 'the causal effect of the loss requires matched retraining,' but the abstract and conclusion still present the no-loss checkpoint as the 'best,' and Table 5 labels it without a caveat. The FiLM comparisons also mix configurations: the LRS3 pair is controlled, but the VB2 FiLM comparison (rows 7–8) is not documented to be a matched pair with identical training schedules and seeds. Please provide matched retraining runs (with seeds and standard deviations) for the speaker-loss and FiLM comparisons, or explicitly report these as diagnostic rather than causal.
minor comments (4)
  1. [§3.3] Typo: 'After filtering there leaving 6,038 evaluable mixtures' should read 'after filtering, 6,038 evaluable mixtures remain'.
  2. [§4.3] Malformed sentence: 'Causing that we mainly using QIANGDA for testing, we don't dividing testing partition here.' Please rewrite, e.g., 'Because we use QIANGDA for testing, we do not create a testing partition here.'
  3. [§6.4] The OSD threshold (0.35) and the decision to discard 'low-energy windows' are not defined precisely. Please state the energy criterion and the fraction of windows discarded; this affects the proxy and its reproducibility.
  4. [Table 5] The footnote marker '†' is used in the table caption but the corresponding footnote text appears only in the body text. Add a proper table footnote explaining that this checkpoint's CLI was not fully preserved.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the benchmark and evaluation are self-contained; only minor self-citations as data/component resources.

full rationale

The paper's central claim is that QIANGDA enables a paired identity test (Eq. 1: f_theta(x,v_A)≈s_A and f_theta(x,v_B)≈s_B) and that the best checkpoint achieves 0.2261 CER, 82.22% strict correctness, and 69.53% both-output success. This claim is evaluated on a newly recorded test set (QIANGDA) disjoint from the training corpus (VOXBLINK2-AVSE), with disjoint speakers. The evaluation uses external, independently trained models: WeSpeaker ResNet34 (VoxCeleb), Qwen3-ASR, and WavLM OSD. The OSD threshold 0.35 is not fitted to the test set; the paper reports stability across 0.25–0.45 and explicitly cautions that 'OSD errors, separation artifacts, and brief residual interference can affect this proxy.' That is a validity limitation, not a circular reduction. The only self-citations are data/component resources: VOXBLINK2-AVSE is curated from the authors' VoxBlink2 corpus, and the visual encoder is taken from the authors' M2S-AVSR work. These are inputs to training/architecture, not conclusions derived from the benchmark; the benchmark's identity-faithfulness measurement does not depend on them being true. Hence no step reduces by construction to its own input.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

No new physical or theoretical entities are postulated; the datasets are constructed measurement artifacts. The free parameters are hand-set thresholds and loss weights that shape the corpus, the evaluation, and the training objective.

free parameters (5)
  • DNSMOS OVRL threshold = 3.3
    Hand-chosen operating point to balance data quality and speaker coverage (Section 4.2). It determines which utterances enter VOXBLINK2-AVSE and thus shapes the training distribution.
  • OSD mean posterior threshold = 0.35
    Hand-chosen threshold above which an output is labeled residual overlap in the strict identity evaluation (Section 6.4). Affects absolute correctness rates, though the authors report stable ordering between 0.25 and 0.45.
  • Speaker-margin loss weight (lambda_spk) = 0.1
    Set for the margin-loss baseline (Section 5.3); no tuning study reported, but it changes the training objective.
  • Speaker-margin m = 0.2
    Margin in Eq. (3) for the auxiliary speaker loss (Section 5.3).
  • Viterbi keep-or-swap transition penalty = 0.35
    Weight on state-change penalty in the identity-alignment objective Eq. (7), Appendix A; hand-set for face track identity assignment.
assumptions (4)
  • domain assumption The WeSpeaker ResNet34 speaker embedding and WavLM OSD posterior are valid proxies for output identity and residual overlap on QIANGDA's real recordings, in the absence of clean references.
    Load-bearing for the strict correctness metric (Section 6.4); if false, the headline percentages do not measure what they claim.
  • domain assumption DNSMOS OVRL>=3.3 selects training speech that is intelligible and structurally intact for separator training.
    Underlies the curation of VOXBLINK2-AVSE (Section 4.2).
  • domain assumption The RetinaFace/FAN/Viterbi/TalkNet identity-alignment pipeline correctly assigns the two face tracks to the A and B speaker labels in every scene.
    Needed for the paired identity test to be valid (Section 3.4). The paper says uncertain scenes are manually reviewed, but the automatic alignment is still an assumption for non-reviewed scenes.
  • domain assumption Headset and camera-synchronized microphone audio are synchronized with the video within a scene.
    The entire AV-TSE task depends on audio-visual synchronization (Section 3.1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identity-Faithful Audio-Visual Target Speaker Extraction with QIANGDA and VOXBLINK2-AVSE." pith.science (2026). https://pith.science/paper/SANUBS7C

@misc{pith2026260803964,
  author       = {Pith},
  title        = {Pith review of: Identity-Faithful Audio-Visual Target Speaker Extraction with QIANGDA and VOXBLINK2-AVSE},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SANUBS7C}},
  note         = {Machine review of arXiv:2608.03964}
}
read the original abstract

Audio-visual target speaker extraction should return the speaker indicated by the video, yet a separator can ignore the visual cue and repeatedly output the acoustically dominant voice. We introduce QIANGDA, a Mandarin AV-TSE benchmark of jointly recorded real two-speaker mixtures with synchronized multi-view video. Each scene also contains preceding A-only and B-only stages that provide in-scene speaker references. It contains 77 scenes and 7,598 clips (11.84 hours), including 6,042 dual-annotated mixtures. After processing there leave 6,038 evaluable mixtures and 12,076 target-speaker rows. We additionally curate VOXBLINK2-AVSE from VoxBlink2, comprising 250,828 synchronized audio--lip-ROI pairs from 28,421 identities and 766.17 hours of speech. Our extractor uses frozen, 1,280-dimensional projected AV-HuBERT features, target-conditioned training, and layer-wise feature modulation. We jointly evaluate content with Qwen3-ASR-1.7B CER and target identity with WeSpeaker ResNet34 plus Overlapped Speech Detection (OSD). On the complete manifest, the best archived checkpoint obtains 0.2261 CER, 82.22% strict output correctness, and 69.53% both-output strict success.

Figures

Figures reproduced from arXiv: 2608.03964 by the authors.

Figure 1
Figure 1. QIANGDA recording setup and a representative frontal-view frame. Panel (a) shows the camera geometry and microphone arrangement; panel (b) shows a 0.5-m example with the two speaker regions marked by dashed boxes. 3.2 Geometry and Acquisition [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Overview of the proposed AV-HuBERT-conditioned TF-GridNet. The frozen visual branch conditions [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 40 canonical work pages

  1. [1]

    Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. 2018. https://doi.org/10.48550/arXiv.1809.00496 LRS3-TED : A large-scale dataset for visual speech recognition . arXiv preprint

    arXiv doi:10.48550/arxiv.1809.00496 2018

  2. [2]

    Adrian Bulat and Georgios Tzimiropoulos. 2017. https://doi.org/10.1109/ICCV.2017.116 How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks) . In IEEE International Conference on Computer Vision, pages 1021--1030

    doi:10.1109/iccv.2017.116 2017

  3. [3]

    Parkhi, and Andrew Zisserman

    Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. 2018. https://doi.org/10.1109/FG.2018.00020 VGGFace2 : A dataset for recognising faces across pose and age . In IEEE International Conference on Automatic Face & Gesture Recognition, pages 67--74

    arXiv 2018

  4. [4]

    Joon Son Chung, Andrew Senior, Oriol Vinyals, and Andrew Zisserman. 2017. https://doi.org/10.1109/CVPR.2017.367 Lip reading sentences in the wild . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3444--3453

    doi:10.1109/cvpr.2017.367 2017

  5. [5]

    Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou. 2020. https://doi.org/10.1109/CVPR42600.2020.00525 RetinaFace : Single-shot multi-level face localisation in the wild . In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5202--5211

    arXiv 2020

  6. [6]

    Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. https://doi.org/10.21437/Interspeech.2020-2650 ECAPA-TDNN : Emphasized channel attention, propagation and aggregation in TDNN based speaker verification . In Interspeech, pages 3830--3834

    doi:10.21437/interspeech.2020-2650 2020

  7. [7]

    Bradley Efron. 1979. https://doi.org/10.1214/aos/1176344552 Bootstrap methods: Another look at the jackknife . The Annals of Statistics, 7(1):1--26

    arXiv 1979

  8. [8]

    Freeman, and Michael Rubinstein

    Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T. Freeman, and Michael Rubinstein. 2018. https://doi.org/10.1145/3197517.3201357 Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separation . ACM Transactions on Graphics, 37(4):1--11

    arXiv 2018

Show all 56 references
  1. [9]

    Ruohan Gao and Kristen Grauman. 2021. https://doi.org/10.1109/CVPR46437.2021.01524 Visualvoice: Audio-visual speech separation with cross-modal consistency . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15490--15500

    2021

  2. [10]

    Levenshtein

    Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8):707--710

    1966

  3. [11]

    Shaole Li, Shuai Wang, Jiangyu Han, Ke Zhang, Wupeng Wang, and Haizhou Li. 2025. https://doi.org/10.21437/Interspeech.2025-2662 REAL-T : Real conversational mixtures for target speaker extraction . In Interspeech 2025, pages 1923--1927

    2025 doi

  4. [12]

    Yuke Lin, Ming Cheng, Fulin Zhang, Yingying Gao, Shilei Zhang, and Ming Li. 2024. https://doi.org/10.21437/Interspeech.2024-1490 Voxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark . In Proc. Interspeech 2024, pages 4263--4267

    2024 doi

  5. [13]

    Yi Luo and Nima Mesgarani. 2019. https://doi.org/10.1109/TASLP.2019.2915167 Conv-tasnet: Surpassing ideal time--frequency magnitude masking for speech separation . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(8):1256--1266

    2019

  6. [14]

    Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. https://doi.org/10.21437/Interspeech.2017-950 Voxceleb: A large-scale speaker identification dataset . In Interspeech, pages 2616--2620

    2017 doi

  7. [15]

    Tsubasa Ochiai, Marc Delcroix, Keisuke Kinoshita, Atsunori Ogawa, and Tomohiro Nakatani. 2019. https://doi.org/10.21437/Interspeech.2019-1513 Multimodal speakerbeam: Single channel target speech extraction with audio-visual speaker clues . In Interspeech, pages 2718--2722

    2019 doi

  8. [16]

    Zexu Pan, Meng Ge, and Haizhou Li. 2022. https://doi.org/10.1109/TASLP.2022.3205759 USEV : Universal speaker extraction with visual cue . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:3032--3045

    2022

  9. [17]

    Germain, Sameer Khurana, Chiori Hori, and Jonathan Le Roux

    Zexu Pan, Gordon Wichern, Yoshiki Masuyama, Fran c ois G. Germain, Sameer Khurana, Chiori Hori, and Jonathan Le Roux. 2023. https://doi.org/10.1109/ASRU57964.2023.10389618 Scenario-aware audio-visual TF-GridNet for target speech extraction . In IEEE Automatic Speech Recognitio...

    2023

  10. [18]

    Ethan Perez, Florian Strub, Harm de Vries, Vincent Dumoulin, and Aaron Courville. 2018. https://doi.org/10.1609/aaai.v32i1.11671 FiLM : Visual reasoning with a general conditioning layer . In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32

    2018 doi

  11. [19]

    Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2021. https://doi.org/10.1109/ICASSP39728.2021.9414878 Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors . In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and...

    2021

  12. [20]

    Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. https://doi.org/10.1109/CVPR.2015.7298682 FaceNet : A unified embedding for face recognition and clustering . In IEEE Conference on Computer Vision and Pattern Recognition, pages 815--823

    2015

  13. [22]

    Xian Shi, Xiong Wang, Zhifang Guo, Yongqi Wang, Pei Zhang, Xinyu Zhang, Zishan Guo, Hongkun Hao, Yu Xi, Baosong Yang, Jin Xu, Jingren Zhou, and Junyang Lin. 2026. https://doi.org/10.48550/arXiv.2601.21337 Qwen3-ASR technical report . arXiv preprint

    2026 doi arXiv

  14. [24]

    Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, and Haizhou Li. 2021. https://doi.org/10.1145/3474085.3475587 Is someone speaking? exploring long-term temporal features for audio-visual active speaker detection . In ACM Multimedia, pages 3927--3935

    2021

  15. [25]

    Andrew J. Viterbi. 1967. https://doi.org/10.1109/TIT.1967.1054010 Error bounds for convolutional codes and an asymptotically optimum decoding algorithm . IEEE Transactions on Information Theory, 13(2):260--269

    1967

  16. [26]

    Zhong-Qiu Wang, Samuele Cornell, Shukjae Choi, Younglo Lee, Byeong-Yeol Kim, and Shinji Watanabe. 2023. https://doi.org/10.1109/TASLP.2023.3304482 TF-GridNet : Integrating full- and sub-band modeling for speech separation . IEEE/ACM Transactions on Audio, Speech, and Language ...

    2023

  17. [27]

    Bang Zeng and Ming Li. 2025. https://doi.org/10.1109/TASLPRO.2025.3572756 USEF-TSE : Universal speaker embedding free target speaker extraction . IEEE Transactions on Audio, Speech and Language Processing, 33:2110--2124

    2025

  18. [28]

    Kate r ina Z mol \' kov \'a , Marc Delcroix, Keisuke Kinoshita, Tsubasa Ochiai, Tomohiro Nakatani, Luk \'a s Burget, and Jan C ernock \`y . 2019. https://doi.org/10.1109/JSTSP.2019.2922820 SpeakerBeam : Speaker aware neural network for target speaker extraction in speech mixtu...

    2019

  19. [29]

    ACM Transactions on Graphics , volume =

    Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation , author =. ACM Transactions on Graphics , volume =. 2018 , doi =

    2018

  20. [30]

    2025 , publisher=

    Zeng, Bang and Li, Ming , journal=. 2025 , publisher=

    2025

  21. [31]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =. 2021 , doi =

    2021

  22. [32]

    Interspeech , pages =

    Multimodal SpeakerBeam: Single Channel Target Speech Extraction with Audio-Visual Speaker Clues , author =. Interspeech , pages =. 2019 , doi =

    2019

  23. [33]

    2022 , doi =

    Pan, Zexu and Ge, Meng and Li, Haizhou , journal =. 2022 , doi =

    2022

  24. [34]

    2019 , publisher=

    IEEE Journal of Selected Topics in Signal Processing , volume=. 2019 , publisher=

    2019

  25. [35]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume =

    Conv-TasNet: Surpassing Ideal Time--Frequency Magnitude Masking for Speech Separation , author =. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume =. 2019 , doi =

    2019

  26. [36]

    Interspeech , pages =

    VoxCeleb: A Large-Scale Speaker Identification Dataset , author =. Interspeech , pages =. 2017 , doi =

    2017

  27. [37]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =

    Lip Reading Sentences in the Wild , author =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages =. 2017 , doi =

    2017

  28. [38]

    2018 , doi =

    Afouras, Triantafyllos and Chung, Joon Son and Zisserman, Andrew , publisher =. 2018 , doi =

    2018

  29. [39]

    , booktitle =

    Le Roux, Jonathan and Wisdom, Scott and Erdogan, Hakan and Hershey, John R. , booktitle =. 2019 , doi =

    2019

  30. [40]

    2017 , doi =

    Bu, Hui and Du, Jiayu and Na, Xingyu and Wu, Bengu and Zheng, Hao , booktitle =. 2017 , doi =

    2017

  31. [41]

    2023 , doi =

    Wang, Zhong-Qiu and Cornell, Samuele and Choi, Shukjae and Lee, Younglo and Kim, Byeong-Yeol and Watanabe, Shinji , journal =. 2023 , doi =

    2023

  32. [42]

    Scenario-Aware Audio-Visual

    Pan, Zexu and Wichern, Gordon and Masuyama, Yoshiki and Germain, Fran. Scenario-Aware Audio-Visual. IEEE Automatic Speech Recognition and Understanding Workshop , pages =. 2023 , doi =

    2023

  33. [43]

    , title =

    Levenshtein, Vladimir I. , title =. Soviet Physics Doklady , volume =

  34. [44]

    2018 , doi =

    Perez, Ethan and Strub, Florian and de Vries, Harm and Dumoulin, Vincent and Courville, Aaron , booktitle =. 2018 , doi =

    2018

  35. [45]

    2020 , doi =

    Desplanques, Brecht and Thienpondt, Jenthe and Demuynck, Kris , booktitle =. 2020 , doi =

    2020

  36. [46]

    2020 , doi =

    Deng, Jiankang and Guo, Jia and Ververas, Evangelos and Kotsia, Irene and Zafeiriou, Stefanos , booktitle =. 2020 , doi =

    2020

  37. [47]

    IEEE International Conference on Computer Vision , pages =

    How Far Are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , author =. IEEE International Conference on Computer Vision , pages =. 2017 , doi =

    2017

  38. [48]

    2015 , doi =

    Schroff, Florian and Kalenichenko, Dmitry and Philbin, James , booktitle =. 2015 , doi =

    2015

  39. [49]

    and Zisserman, Andrew , booktitle =

    Cao, Qiong and Shen, Li and Xie, Weidi and Parkhi, Omkar M. and Zisserman, Andrew , booktitle =. 2018 , doi =

    2018

  40. [50]

    IEEE Transactions on Information Theory , volume =

    Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , author =. IEEE Transactions on Information Theory , volume =. 1967 , doi =

    1967

  41. [51]

    ACM Multimedia , pages =

    Is Someone Speaking? Exploring Long-Term Temporal Features for Audio-Visual Active Speaker Detection , author =. ACM Multimedia , pages =. 2021 , doi =

    2021

  42. [52]

    The Annals of Statistics , volume =

    Bootstrap Methods: Another Look at the Jackknife , author =. The Annals of Statistics , volume =. 1979 , doi =

    1979

  43. [53]

    International Conference on Learning Representations , year=

    Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction , author=. International Conference on Learning Representations , year=. doi:10.48550/arXiv.2201.02184 , url=

    doi arXiv

  44. [54]

    arXiv preprint arXiv:2606.05763 , year=

    M2S-AVSR: Modality-aware Multi-view Self-supervised Representation for Robust Audio-Visual Speech Recognition , author=. arXiv preprint arXiv:2606.05763 , year=. doi:10.48550/arXiv.2606.05763 , url=

    doi arXiv

  45. [55]

    VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark , author=. Proc. Interspeech 2024 , pages=. 2024 , doi=

    2024

  46. [56]

    2026 , doi =

    Shi, Xian and Wang, Xiong and Guo, Zhifang and Wang, Yongqi and Zhang, Pei and Zhang, Xinyu and Guo, Zishan and Hao, Hongkun and Xi, Yu and Yang, Baosong and Xu, Jin and Zhou, Jingren and Lin, Junyang , publisher =. 2026 , doi =

    2026

  47. [57]

    2025 , pages =

    Li, Shaole and Wang, Shuai and Han, Jiangyu and Zhang, Ke and Wang, Wupeng and Li, Haizhou , booktitle =. 2025 , pages =

    2025

  48. [58]

    ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    DNSMOS: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors , author=. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2021 , organization=

    2021

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.