REVIEW 3 major objections 7 minor 6 cited by
Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation
T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A single sequence-to-sequence network performs speaker diarization online and offline, discovering new speakers without a prior enrollment stage.
desk verdict Novel dual-decoder diarization architecture with strong but inflated SOTA numbers due to eval-set model selection. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core machinery is a pair of inverse sequence-to-sequence decoders with a shared encoder and a learnable speaker-embedding matrix. The detection decoder takes speaker embeddings (or the pseudo-speaker embedding for unknown voices) as auxiliary queries and predicts binary voice activities; the representation decoder takes those predicted voice activities as auxiliary queries and extracts speaker embeddings, trained by an ArcFace loss against the same embedding matrix. Masked speaker prediction during training randomly erases one speaker's embedding so the model learns to associate missing voices with the pseudo-speaker embedding, and blockwise inference with a weighted speaker-embedding buffer propagates identities across long recordings. This design avoids unsupervised clustering and permutation-invariant training while keeping speaker identities consistent across blocks.
What would settle it
Take a test set of conversations where, by design, two previously unenrolled speakers begin speaking within the same chunk while all other conditions match DIHARD-III, and measure the online DER of S2SND; a large error spike on those recordings would confirm that the single-pseudo-speaker slot is the bottleneck. A simpler probe: run the model online on DIHARD-III but force the block shift to align with moments of simultaneous speaker onsets and compare DER against the reported 17.12%.
Extended reading notes
Core claim
The central claim is that speaker detection and speaker representation can be jointly learned in one sequence-to-sequence architecture, so that a diarization system no longer needs an external enrollment or clustering stage. Two decoders share one encoder: a detection decoder maps speaker embeddings to voice activities, and a representation decoder maps voice activities back to speaker embeddings, with a learnable embedding matrix and ArcFace loss tying the two spaces together. During training, randomly masking one speaker's embedding teaches the model to route unknown speakers through a fixed pseudo-speaker embedding; during inference, a single pseudo-speaker slot lets the model enroll at most one new speaker per block shift, after which the buffer update feeds the next block. When the audio ends, the accumulated embeddings re-decode the whole signal for offline use. The paper reports that this design surpasses previous online and offline systems on the DIHARD-II and DIHARD-III evaluation sets.
Load-bearing premise
The online pipeline assumes that at most one new speaker can appear in any small block shift, because only a single pseudo-speaker embedding is available per block; if two unseen speakers start speaking in the same chunk, only one can be enrolled in that block and the other may be missed.
Editorial extensions
If this is right
- The same S2SND model, without any architectural change, serves as both an online streamer (blockwise, low latency) and an offline system (full re-scoring), so one deployment covers both use cases.
- New best published DERs on DIHARD-II are reported: 24.41% online and 21.95% offline without oracle VAD, beating prior systems that sometimes rely on oracle VAD.
- On DIHARD-III the model reaches 17.12% online and 15.13% offline, approaching the previous best offline systems while running online.
- Speaker counting accuracy on DIHARD-III reaches 79.54% without oracle VAD, the highest among the systems the paper compares against.
- The masked-speaker training recipe combined with knowledge distillation from a larger teacher extractor is what unlocks the gains for the medium model with the large VoxBlink2 corpus.
Reading between the lines
- If the one-new-speaker-per-block assumption fails in a recording—two previously unseen speakers first talk in the same chunk—the second speaker can only be enrolled later and may be missed entirely if their turn is short; a natural extension is to add multiple pseudo-speaker slots or a delayed enrollment pass.
- The same inverse-decoder trick (voice activities as queries to extract embeddings) could transfer to other tasks that need to track an open set of identities, such as audio-visual diarization or meeting transcription with name enrollment.
- Because the embedding space is learned jointly with detection rather than frozen from an external speaker-verification model, the framework's gains may depend on the diversity of the simulation corpus; the paper's distillation results suggest scale matters when the model is large enough to use it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes S2SND, a sequence-to-sequence neural diarization framework that performs online and offline speaker diarization with a single model. It augments a TSVAD-style encoder-decoder with two decoders: a detection decoder that predicts voice activities from partially given speaker embeddings, and a representation decoder that extracts speaker embeddings from predicted voice activities. A pseudo-speaker embedding is used to detect and enroll one unknown speaker per block, and a buffer stores the extracted embeddings for subsequent blocks. The model is trained with a masked-speaker prediction objective and an ArcFace loss on the representation branch, plus a knowledge-distillation strategy from a larger teacher extractor. Experiments on DIHARD-II and DIHARD-III report new state-of-the-art DERs: online without oracle VAD 24.41%/17.12%, offline without oracle VAD 21.95%/15.13%, with additional gains when oracle VAD post-processing is applied. The paper also reports speaker-counting accuracy and computing efficiency for the proposed models.
Significance. If the reported results are robust, the paper makes a significant contribution: it unifies online and offline diarization in a clustering-free, PIT-free architecture with automatic speaker enrollment, a property that previous TSVAD/EEND hybrid systems lack. The masked-speaker prediction and target-voice embedding extraction are novel and reasonably motivated, and the paper provides a detailed ablation study in Table II as well as comparisons on two standard benchmarks. The strengths include a clear description of the architecture, training objectives, and inference protocol, as well as a pseudocode specification of the online algorithm. However, the headline SOTA claim currently rests on model selection performed on the evaluation sets, which can materially inflate the reported gains, and there is a train/test mismatch in the representation decoder's inputs. These issues must be addressed before the central claim can be accepted.
major comments (3)
- [Sec. V-A, Table II, Tables III and IV] The headline SOTA numbers are selected on the evaluation sets themselves. Table II reports DERs for 24 configurations (S1-S24) on the DIHARD-II and DIHARD-III evaluation sets, and the text says 'We select the lowest online and offline DERs for each model size on DIHARD-II and DIHARD-III datasets as the representative results.' The configuration selection (model size, simulation corpus, training strategy, chunk length, right-context length) is therefore not independent of the test data. The only explicitly validation-tuned hyperparameters are the thresholds tau_1 and tau_2 (Sec. IV-C2). Because the spread across configurations is material (e.g., Medium online DIHARD-II ranges from 24.41% in S22 to 27.79% in S17; offline from 21.95% in S21 to 24.09% in S17), the gap between the evaluation-selected best and a validation-selected configuration could be several DER points, which is the same order as the claimed improvement over prior SOTA. Please repeat configuration selection on the validation splits (dev39/dev51) and report both the validation-selected configuration and its evaluation-set DER, or report all configurations on the validation sets so the reader can assess the optimism.
- [Sec. III-B2, Eq. (4), Sec. III-C2] The representation decoder is trained with oracle voice activities as auxiliary queries but deployed with predicted voice activities. In Sec. III-B2 the representation decoder 'utilizes extractor output X as feature embeddings and voice activities Y as auxiliary queries', where Y is ground truth, and the ArcFace loss in Eq. (4) is computed under this oracle-query condition. During inference (Sec. III-C2), the same decoder receives the predicted activities Y_hat as queries. The paper does not describe any mechanism to close this train/test gap, such as scheduled sampling, query dropout/noising, or a loss that also evaluates the representation branch on predicted activities. This exposure bias is particularly concerning for the pseudo-speaker channel, whose predicted activity is used both to enroll new speakers and to extract their embeddings, so errors can accumulate blockwise. Please either train the representation decoder with the detection decoder's outputs (with teacher forcing), or provide a validation-set analysis quantifying the effect of this mismatch.
- [Abstract, Sec. III-C2, Sec. III-C3] The online pipeline relies on the assumption that at most one new speaker appears in a small block shift, because there is only a single pseudo-speaker embedding slot. If two or more unseen speakers first appear in the same block, only one can be enrolled in that block; the others are delayed until later blocks and may be missed if their speech is short. This is a stated limitation, but the paper does not quantify how often the assumption is violated in the DIHARD-II/III evaluation sets, nor how violations affect DER. Please report the distribution of the number of new speakers per block on these evaluation sets and, if possible, compare with a variant that uses multiple pseudo-speaker slots or that delays enrollment decisions by one block, so that the robustness of the SOTA claim to this assumption can be assessed.
minor comments (7)
- [Sec. III-C2] The definition of the weight function W(hat y) uses the notation t' not in Overlap, but the set Overlap is not formally defined. Please specify how overlapping frames are determined from the predicted voice activities of all speakers.
- [Algorithm 1] The loop 'for n in range(1, len(S))' appears to use the embedding dimension S as the loop bound; the intended bound is likely the number of enrolled speaker slots (len(spk_list) or N). Please correct the pseudocode.
- [Sec. IV-C2] The text states that tau_1 and tau_2 are determined by grid search on the validation set, but it does not report the grid range or the resulting values. Please include these values for reproducibility.
- [Table II] The table caption says the lowest DERs of each model size are highlighted, but the selection is done on the evaluation set. In addition to the suggested validation-based selection, it would be helpful to mark the validation-best configuration separately in the table.
- [Sec. V-B] The sentence 'our best online DER (24.41%) is even lower than the previous best offline system (24.64%)' mixes online and offline conditions; the comparison is only valid under the no-oracle-VAD setting and with the selected configuration. Please clarify this phrasing.
- [Sec. III-C2] There is a typo: 'represnets' should be 'represents' in the sentence describing the pseudo-speaker output.
- [Sec. III-C3] The offline rescoring mechanism is described as re-decoding the audio with the final speaker-embedding buffer, but the paper does not specify whether the rescoring pass uses the same block-wise chunking with right-context or a full-sequence decode. Please clarify the inference setup for the offline mode.
Circularity Check
No significant circularity: the S2SND claims are empirical benchmark results with external comparisons; self-citations are architectural lineage and dataset sources, not load-bearing derivations.
full rationale
The central claim is an empirical SOTA benchmark result, not a derivation from an assumed target. The S2SND architecture is an extension of the authors' own Seq2Seq-TSVAD, but that lineage is not load-bearing support for the reported DER numbers: the numbers come from training on simulated data, adapting on DIHARD dev splits, grid-searching thresholds tau_1 and tau_2 on dev39/dev51, and comparing against external systems on DIHARD-II/III evaluation sets. The detection/representation decoder loop is an architectural feedback mechanism, not a logical definitional equivalence. Self-citations to [16], [55], and [75] provide architecture details, training-strategy choices, and a speech corpus, but none are used to forbid alternatives or to define the target metric into existence. The notable methodological caveat is that the 'best' configurations in Table II appear to be selected on the evaluation sets themselves, which is a test-set selection bias—an experimental-design/correctness risk, not a circularity. Because this concern concerns evaluation protocol rather than the derivation reducing to its own inputs, the circularity score is 0.
Assumptions & free parameters
free parameters (5)
- tau_1 =
grid-searched per dataset (exact value not reported)
- tau_2 =
grid-searched per dataset (exact value not reported)
- speaker capacity N =
30
- chunk and right-context lengths =
0.48s/0.64s chunk, 0/0.16s right context; best per dataset selected
- ArcFace scale alpha and margin m =
alpha=32, m=0.2
assumptions (4)
- domain assumption At most one new speaker may appear in a small block shift.
- domain assumption Voice activity patterns are sufficiently discriminative to extract speaker embeddings, since single-speaker speech usually occupies most of the conversation.
- domain assumption The learnable embedding matrix E_all provides valid supervised targets for the representation decoder.
- domain assumption Simulated mixtures of 1-3 speakers from VoxCeleb2/VoxBlink2 transfer to real DIHARD domains.
invented entities (2)
-
pseudo-speaker embedding e_pse
independent evidence
-
non-speech embedding e_non
Cite this review
Pith. "Pith review of Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation." pith.science (2026). https://pith.science/paper/BCVXU3JH
@misc{pith2026241113849,
author = {Pith},
title = {Pith review of: Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation},
year = {2026},
howpublished = {\url{https://pith.science/paper/BCVXU3JH}},
note = {Machine review of arXiv:2411.13849}
}
read the original abstract
This paper proposes a novel Sequence-to-Sequence Neural Diarization (S2SND) framework to perform online and offline speaker diarization. It is developed from the sequence-to-sequence architecture of our previous target-speaker voice activity detection system and then evolves into a new diarization paradigm by addressing two critical problems. 1) Speaker Detection: The proposed approach can utilize partially given speaker embeddings to discover the unknown speaker and predict the target voice activities in the audio signal. It does not require a prior diarization system for speaker enrollment in advance. 2) Speaker Representation: The proposed approach can adopt the predicted voice activities as reference information to extract speaker embeddings from the audio signal simultaneously. The representation space of speaker embedding is jointly learned within the whole diarization network without using an extra speaker embedding model. During inference, the S2SND framework can process long audio recordings blockwise. The detection module utilizes the previously obtained speaker-embedding buffer to predict both enrolled and unknown speakers' voice activities for each coming audio block. Next, the speaker-embedding buffer is updated according to the predictions of the representation module. Assuming that up to one new speaker may appear in a small block shift, our model iteratively predicts the results of each block and extracts target embeddings for the subsequent blocks until the signal ends. Finally, the last speaker-embedding buffer can re-score the entire audio, achieving highly accurate diarization performance as an offline system. Experimental results show that ...
Figures
Figures from the paper (4 more)
Forward citations
Cited by 6 Pith papers
-
Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering
A streaming Sortformer with an arrival-ordered speaker cache achieves lower diarization error than prior online systems on DIHARD III and CALLHOME, even at 0.32 second latency.
-
Diarization-Aware Multi-Speaker Automatic Speech Recognition via Large Language Models
An LLM conditioned on speaker embeddings and utterance time boundaries jointly transcribes and timestamps overlapping multi-speaker speech.
-
Dissecting the Segmentation Model of End-to-End Diarization with Vector Clustering
A controlled 120-configuration study of EEND-VC speaker diarization finds finetuned WavLM encoders, Conformer/Mamba decoders, and longer chunks give the biggest gains, with the best system reaching state-of-the-art on...
-
The Multimodal Information Based Speech Processing (MISP) 2025 Challenge: Audio-Visual Diarization and Recognition
The top MISP 2025 systems achieve DER 8.09%, CER 9.48%, and cpCER 11.56%, far outperforming the provided audio-visual baselines.
-
The DKU System for Multi-Speaker Automatic Speech Recognition in MLC-SLM Challenge
A challenge system combining speaker diarization, speaker embeddings, and a Qwen2.5 LLM adapter architecture reports 18.08% tcpWER on multilingual multi-speaker ASR, far below the 60.39% baseline.
-
Multi-Channel Sequence-to-Sequence Neural Diarization: Experimental Results for The MISP 2025 Challenge
Extending S2SND with a channel-attention module for multi-channel audio achieves an 8.09% diarization error rate, first place in the MISP 2025 speaker diarization task.
Reference graph
Works this paper leans on
-
[1]
A review of speaker diarization: Recent advances with deep learning,
T. J. Park, N. Kanda, D. Dimitriadis, K. J. Han, S. Watanabe, and S. Narayanan, “A review of speaker diarization: Recent advances with deep learning,”Computer Speech & Language, vol. 72, p. 101317, 2022
2022
-
[2]
Joint speaker counting, speech recognition, and speaker identification for overlapped speech of any number of speakers,
N. Kanda, Y . Gaur, X. Wang, Z. Meng, Z. Chen, T. Zhou, and T. Yoshioka, “Joint speaker counting, speech recognition, and speaker identification for overlapped speech of any number of speakers,” inProc. INTERSPEECH, 2020, pp. 36–40
2020
-
[3]
Unsupervised methods for speaker diarization: An integrated and iterative approach,
S. H. Shum, N. Dehak, R. Dehak, and J. R. Glass, “Unsupervised methods for speaker diarization: An integrated and iterative approach,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 10, pp. 2015–2028, 2013
2015
-
[4]
A study of the cosine distance-based mean shift for telephone speech diarization,
M. Senoussaoui, P. Kenny, T. Stafylakis, and P. Dumouchel, “A study of the cosine distance-based mean shift for telephone speech diarization,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 22, no. 1, pp. 217–227, 2014
2014
-
[5]
Speaker diarization with plda i-vector scoring and unsupervised calibration,
G. Sell and D. Garcia-Romero, “Speaker diarization with plda i-vector scoring and unsupervised calibration,” inProc. SLT, 2014, pp. 413–417
2014
-
[6]
Speaker diarization with lstm,
Q. Wang, C. Downey, L. Wan, P. A. Mansfield, and I. L. Moreno, “Speaker diarization with lstm,” inProc. ICASSP, 2018, pp. 5239–5243
2018
-
[7]
Lstm based similarity measurement with spectral clustering for speaker diarization,
Q. Lin, R. Yin, M. Li, H. Bredin, and C. Barras, “Lstm based similarity measurement with spectral clustering for speaker diarization,” inProc. INTERSPEECH, 2019, pp. 366–370
2019
-
[8]
Bayesian hmm clustering of x-vector sequences (vbx) in speaker diarization: Theory, implemen- tation and analysis on standard tasks,
F. Landini, J. Profant, M. Diez, and L. Burget, “Bayesian hmm clustering of x-vector sequences (vbx) in speaker diarization: Theory, implemen- tation and analysis on standard tasks,”Computer Speech & Language, vol. 71, p. 101254, 2022
2022
Show all 90 references
-
[9]
Similarity measurement of segment-level speaker embeddings in speaker diarization,
W. Wang, Q. Lin, D. Cai, and M. Li, “Similarity measurement of segment-level speaker embeddings in speaker diarization,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 2645–2658, 2022
2022
-
[10]
End-to-end neural speaker diarization with permutation-free objec- tives,
Y . Fujita, N. Kanda, S. Horiguchi, K. Nagamatsu, and S. Watanabe, “End-to-end neural speaker diarization with permutation-free objec- tives,” inProc. INTERSPEECH, 2019, pp. 4300–4304
2019
-
[11]
End-to-end neural speaker diarization with self-attention,
Y . Fujita, N. Kanda, S. Horiguchi, Y . Xue, K. Nagamatsu, and S. Watan- abe, “End-to-end neural speaker diarization with self-attention,” inProc. ASRU, 2019, pp. 296–303
2019
-
[12]
End- to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors,
S. Horiguchi, Y . Fujita, S. Watanabe, Y . Xue, and K. Nagamatsu, “End- to-end speaker diarization for an unknown number of speakers with encoder-decoder based attractors,” inProc. INTERSPEECH, 2020, pp. 269–273
2020
-
[13]
Encoder- decoder based attractors for end-to-end neural diarization,
S. Horiguchi, Y . Fujita, S. Watanabe, Y . Xue, and P. Garc ´ıa, “Encoder- decoder based attractors for end-to-end neural diarization,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 1493–1507, 2022
2022
-
[14]
Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,
M. Kolbæk, D. Yu, Z.-H. Tan, and J. Jensen, “Multitalker speech separation with utterance-level permutation invariant training of deep recurrent neural networks,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, no. 10, pp. 1901–1913, 2017
1901
-
[15]
Target-speaker voice activity detection: A novel approach for multi-speaker diarization in a dinner party scenario,
I. Medennikov, M. Korenevsky, T. Prisyach, Y . Khokhlov, M. Ko- renevskaya, I. Sorokin, T. Timofeeva, A. Mitrofanov, A. Andrusenko, I. Podluzhny, A. Laptev, and A. Romanenko, “Target-speaker voice activity detection: A novel approach for multi-speaker diarization in a dinner p...
2020
-
[16]
Target-speaker voice activity detection via sequence-to-sequence prediction,
M. Cheng, W. Wang, Y . Zhang, X. Qin, and M. Li, “Target-speaker voice activity detection via sequence-to-sequence prediction,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[17]
Ustc-nelslip system description for dihard-iii challenge,
Y . Wang, M. He, S. Niu, L. Sun, T. Gao, X. Fang, J. Pan, J. Du, and C.- H. Lee, “Ustc-nelslip system description for dihard-iii challenge,”arXiv preprint arXiv:2103.10661, 2021
2021 arXiv
-
[18]
The dku-dukeece-lenovo system for the diarization task of the 2021 voxceleb speaker recognition challenge,
W. Wang, D. Cai, Q. Lin, L. Yang, J. Wang, J. Wang, and M. Li, “The dku-dukeece-lenovo system for the diarization task of the 2021 voxceleb speaker recognition challenge,”arXiv preprint arXiv:2109.02002, 2021
2021 arXiv
-
[19]
The dku-dukeece diarization system for the voxceleb speaker recognition challenge 2022,
W. Wang, X. Qin, M. Cheng, Y . Zhang, K. Wang, and M. Li, “The dku-dukeece diarization system for the voxceleb speaker recognition challenge 2022,”arXiv preprint arXiv:2210.01677, 2022
2022 arXiv
-
[20]
The dku-msxf diarization system for the voxceleb speaker recognition challenge 2023,
M. Cheng, W. Wang, X. Qin, Y . Lin, N. Jiang, G. Zhao, and M. Li, “The dku-msxf diarization system for the voxceleb speaker recognition challenge 2023,” inProc. NCMMSC, J. Jia, Z. Ling, X. Chen, Y . Li, and Z. Zhang, Eds. Springer Nature Singapore, 2024, pp. 330–337
2023
-
[21]
The voxceleb speaker recognition chal- lenge: A retrospective,
J. Huh, J. S. Chung, A. Nagrani, A. Brown, J.-w. Jung, D. Garcia- Romero, and A. Zisserman, “The voxceleb speaker recognition chal- lenge: A retrospective,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 3850–3866, 2024
2024
-
[22]
Developing on-line speaker diarization system,
D. Dimitriadis and P. Fousek, “Developing on-line speaker diarization system,” inProc. INTERSPEECH, 2017, pp. 2739–2743
2017
-
[23]
Low-latency online speaker diarization with graph-based label genera- tion,
Y . Zhang, Q. Lin, W. Wang, L. Yang, X. Wang, J. Wang, and M. Li, “Low-latency online speaker diarization with graph-based label genera- tion,” inProc. Odyssey, 2022, pp. 162–169
2022
-
[24]
Bw-eda-eend: streaming end-to-end neural speaker diarization for a variable number of speakers,
E. Han, C. Lee, and A. Stolcke, “Bw-eda-eend: streaming end-to-end neural speaker diarization for a variable number of speakers,” inProc. ICASSP, 2021, pp. 7193–7197
2021
-
[25]
Online end-to-end neural diarization with speaker-tracing buffer,
Y . Xue, S. Horiguchi, Y . Fujita, S. Watanabe, P. Garc ´ıa, and K. Naga- matsu, “Online end-to-end neural diarization with speaker-tracing buffer,” inProc. SLT, 2021, pp. 841–848
2021
-
[26]
Online streaming end-to-end neural diariza- tion handling overlapping speech and flexible numbers of speakers,
Y . Xue, S. Horiguchi, Y . Fujita, Y . Takashima, S. Watanabe, L. P. G. Perera, and K. Nagamatsu, “Online streaming end-to-end neural diariza- tion handling overlapping speech and flexible numbers of speakers,” in Proc. INTERSPEECH, 2021, pp. 3116–3120
2021
-
[27]
Online neural diarization of unlimited numbers of speakers using global and local attractors,
S. Horiguchi, S. Watanabe, P. Garc ´ıa, Y . Takashima, and Y . Kawaguchi, “Online neural diarization of unlimited numbers of speakers using global and local attractors,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 706–720, 2023
2023
-
[28]
Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors,
D. Liang, N. Shao, and X. Li, “Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based attractors,” in Proc. ICASSP, 2024, pp. 10 521–10 525
2024
-
[29]
Online target speaker voice activity detection for speaker diarization,
W. Wang, M. Li, and Q. Lin, “Online target speaker voice activity detection for speaker diarization,” inProc. INTERSPEECH, 2022, pp. 1441–1445
2022
-
[30]
Online neural speaker diarization with target speaker tracking,
W. Wang and M. Li, “Online neural speaker diarization with target speaker tracking,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024
2024
-
[31]
Temporal modeling using dilated convolution and gating for voice-activity-detection,
S.-Y . Chang, B. Li, G. Simko, T. N. Sainath, A. Tripathi, A. van den Oord, and O. Vinyals, “Temporal modeling using dilated convolution and gating for voice-activity-detection,” inProc. ICASSP, 2018, pp. 5549– 5553
2018
-
[32]
Convolutional neural network for speaker change detection in telephone speaker diarization system,
M. Hr ´uz and Z. Zaj´ıc, “Convolutional neural network for speaker change detection in telephone speaker diarization system,” inProc. ICASSP, 2017, pp. 4945–4949
2017
-
[33]
Diarization is hard: Some experiences and lessons learned for the jhu team in the inaugural dihard challenge,
G. Sell, D. Snyder, A. McCree, D. Garcia-Romero, J. Villalba, M. Ma- ciejewski, V . Manohar, N. Dehak, D. Povey, S. Watanabe, and S. Khu- danpur, “Diarization is hard: Some experiences and lessons learned for the jhu team in the inaugural dihard challenge,” inProc. INTERSPEECH...
2018
-
[34]
Front- end factor analysis for speaker verification,
N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front- end factor analysis for speaker verification,”IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 4, pp. 788–798, 2011. 15
2011
-
[35]
X-vectors: Robust dnn embeddings for speaker recognition,
D. Snyder, D. Garcia-Romero, G. Sell, D. Povey, and S. Khudanpur, “X-vectors: Robust dnn embeddings for speaker recognition,” inProc. ICASSP, 2018, pp. 5329–5333
2018
-
[36]
But system for the second dihard speech diarization challenge,
F. Landini, S. Wang, M. Diez, L. Burget, P. Mat ˇejka, K. ˇZmol´ıkov´a, L. Moˇsner, A. Silnova, O. Plchot, O. Novotn´y, H. Zeinali, and J. Rohdin, “But system for the second dihard speech diarization challenge,” inProc. ICASSP, 2020, pp. 6529–6533
2020
-
[37]
End-to-end speaker segmentation for overlap-aware resegmentation,
H. Bredin and A. Laurent, “End-to-end speaker segmentation for overlap-aware resegmentation,” inProc. INTERSPEECH, 2021, pp. 3111–3115
2021
-
[38]
End-to-end speaker diarization conditioned on speech activity and overlap detection,
Y . Takashima, Y . Fujita, S. Watanabe, S. Horiguchi, P. Garc ´ıa, and K. Nagamatsu, “End-to-end speaker diarization conditioned on speech activity and overlap detection,” inProc. SLT, 2021, pp. 849–856
2021
-
[39]
Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds,
K. Kinoshita, M. Delcroix, and N. Tawara, “Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds,” inProc. ICASSP, 2021, pp. 7198–7202
2021
-
[40]
Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech,
——, “Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech,” inProc. INTERSPEECH, 2021, pp. 3565–3569
2021
-
[41]
Tight integration of neural- and clustering-based diarization through deep unfolding of infinite gaussian mixture model,
K. Kinoshita, M. Delcroix, and T. Iwata, “Tight integration of neural- and clustering-based diarization through deep unfolding of infinite gaussian mixture model,” inProc. ICASSP, 2022, pp. 8382–8386
2022
-
[42]
Towards neural diarization for unlimited numbers of speakers using global and local attractors,
S. Horiguchi, S. Watanabe, P. Garc ´ıa, Y . Xue, Y . Takashima, and Y . Kawaguchi, “Towards neural diarization for unlimited numbers of speakers using global and local attractors,” inProc. ASRU, 2021, pp. 98–105
2021
-
[43]
End-to-end neu- ral speaker diarization with an iterative refinement of non-autoregressive attention-based attractors,
M. Rybicka, J. Villalba, N. Dehak, and K. Kowalczyk, “End-to-end neu- ral speaker diarization with an iterative refinement of non-autoregressive attention-based attractors,” inProc. INTERSPEECH, 2022, pp. 5090– 5094
2022
-
[44]
Neural diarization with non-autoregressive intermediate attractors,
Y . Fujita, T. Komatsu, R. Scheibler, Y . Kida, and T. Ogawa, “Neural diarization with non-autoregressive intermediate attractors,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[45]
Diaper: End- to-end neural diarization with perceiver-based attractors,
F. Landini, M. Diez, T. Stafylakis, and L. Burget, “Diaper: End- to-end neural diarization with perceiver-based attractors,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 3450–3465, 2024
2024
-
[46]
Improving transformer-based end-to-end speaker diarization by assigning auxiliary losses to attention heads,
Y .-R. Jeoung, J.-Y . Yang, J.-H. Choi, and J.-H. Chang, “Improving transformer-based end-to-end speaker diarization by assigning auxiliary losses to attention heads,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[47]
Improving neural diarization through speaker attribute attractors and local dependency modeling,
D. Palzer, M. Maciejewski, and E. Fosler-Lussier, “Improving neural diarization through speaker attribute attractors and local dependency modeling,” inProc. ICASSP, 2024, pp. 11 911–11 915
2024
-
[48]
Self-supervised speaker diarization,
Y . Dissen, F. Kreuk, and J. Keshet, “Self-supervised speaker diarization,” inProc. INTERSPEECH, 2022, pp. 4013–4017
2022
-
[49]
Semi-supervised training with pseudo-labeling for end- to-end neural diarization,
Y . Takashima, Y . Fujita, S. Horiguchi, S. Watanabe, L. P. G. Perera, and K. Nagamatsu, “Semi-supervised training with pseudo-labeling for end- to-end neural diarization,” inProc. INTERSPEECH, 2021, pp. 3096– 3100
2021
-
[50]
Target- speaker voice activity detection with improved i-vector estimation for unknown number of speaker,
M. He, D. Raj, Z. Huang, J. Du, Z. Chen, and S. Watanabe, “Target- speaker voice activity detection with improved i-vector estimation for unknown number of speaker,” inProc. INTERSPEECH, 2021, pp. 3555– 3559
2021
-
[51]
Multi-target extractor and detector for unknown-number speaker diarization,
C.-Y . Cheng, H.-S. Lee, Y . Tsao, and H.-M. Wang, “Multi-target extractor and detector for unknown-number speaker diarization,”IEEE Signal Processing Letters, vol. 30, pp. 638–642, 2023
2023
-
[52]
Target speaker voice activity detection with transformers and its integration with end- to-end neural diarization,
D. Wang, X. Xiao, N. Kanda, T. Yoshioka, and J. Wu, “Target speaker voice activity detection with transformers and its integration with end- to-end neural diarization,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[53]
Cross-channel attention-based target speaker voice activity detection: Experimental results for the m2met challenge,
W. Wang, X. Qin, and M. Li, “Cross-channel attention-based target speaker voice activity detection: Experimental results for the m2met challenge,” inProc. ICASSP, 2022, pp. 9171–9175
2022
-
[54]
The whu-alibaba audio-visual speaker diarization system for the misp 2022 challenge,
M. Cheng, H. Wang, Z. Wang, Q. Fu, and M. Li, “The whu-alibaba audio-visual speaker diarization system for the misp 2022 challenge,” inProc. ICASSP, 2023, pp. 1–2
2022
-
[55]
Multi-input multi-output target-speaker voice activity detection for unified, flexible, and robust audio-visual speaker diarization,
M. Cheng and M. Li, “Multi-input multi-output target-speaker voice activity detection for unified, flexible, and robust audio-visual speaker diarization,”arXiv preprint arXiv:2401.08052, 2024
2024 arXiv
-
[56]
Target speech diarization with multimodal prompts,
Y . Jiang, R. Tao, Z. Chen, Y . Qian, and H. Li, “Target speech diarization with multimodal prompts,”arXiv preprint arXiv:2406.07198, 2024
2024 arXiv
-
[57]
Joint inference of speaker diarization and asr with multi-stage information sharing,
W. Wang, D. Cai, M. Cheng, and M. Li, “Joint inference of speaker diarization and asr with multi-stage information sharing,” inProc. ICASSP, 2024, pp. 11 011–11 015
2024
-
[58]
Flow-tsvad: Target- speaker voice activity detection via latent flow matching,
Z. Chen, B. Han, S. Wang, Y . Jiang, and Y . Qian, “Flow-tsvad: Target- speaker voice activity detection via latent flow matching,”arXiv preprint arXiv:2409.04859, 2024
2024 arXiv
-
[59]
Guided speaker embedding,
S. Horiguchi, T. Moriya, A. Ando, T. Ashihara, H. Sato, N. Tawara, and M. Delcroix, “Guided speaker embedding,” inProc. ICASSP, 2025, pp. 1–5
2025
-
[60]
Ansd-ma-mse: Adaptive neural speaker diarization using memory-aware multi-speaker embed- ding,
M.-K. He, J. Du, Q.-F. Liu, and C.-H. Lee, “Ansd-ma-mse: Adaptive neural speaker diarization using memory-aware multi-speaker embed- ding,”IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, vol. 31, pp. 1561–1573, 2023
2023
-
[61]
Fully supervised speaker diarization,
A. Zhang, Q. Wang, Z. Zhu, J. Paisley, and C. Wang, “Fully supervised speaker diarization,” inProc. ICASSP, 2019, pp. 6301–6305
2019
-
[62]
Supervised online diarization with sample mean loss for multi-domain data,
E. Fini and A. Brutti, “Supervised online diarization with sample mean loss for multi-domain data,” inProc. ICASSP, 2020, pp. 7134–7138
2020
-
[63]
Overlap-aware low-latency online speaker diarization based on end-to-end local seg- mentation,
J. M. Coria, H. Bredin, S. Ghannay, and S. Rosset, “Overlap-aware low-latency online speaker diarization based on end-to-end local seg- mentation,” inProc. ASRU, 2021, pp. 1139–1146
2021
-
[64]
Probabilistic back-ends for online speaker recognition and clustering,
A. Sholokhov, N. Kuzmin, K. A. Lee, and E. S. Chng, “Probabilistic back-ends for online speaker recognition and clustering,” inProc. ICASSP, 2023, pp. 1–5
2023
-
[65]
Interrelate training and clustering for online speaker diarization,
Y . Chen, G. Cheng, R. Yang, P. Zhang, and Y . Yan, “Interrelate training and clustering for online speaker diarization,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 1352–1364, 2024
2024
-
[66]
Transformer-XL: Attentive language models beyond a fixed-length context,
Z. Dai, Z. Yang, Y . Yang, J. Carbonell, Q. Le, and R. Salakhutdinov, “Transformer-XL: Attentive language models beyond a fixed-length context,” inProc. ACL. Association for Computational Linguistics, 2019, pp. 2978–2988
2019
-
[67]
Enhancing low-latency speaker diarization with spatial dictionary learning,
W. Chen, T. T. Anh, X. Zhong, and E. S. Chng, “Enhancing low-latency speaker diarization with spatial dictionary learning,” inProc. ICASSP, 2024, pp. 11 371–11 375
2024
-
[68]
Overview of speaker modeling and its applications: From the lens of deep speaker representation learning,
S. Wang, Z. Chen, K. A. Lee, Y . Qian, and H. Li, “Overview of speaker modeling and its applications: From the lens of deep speaker representation learning,”arXiv preprint arXiv:2407.15188, 2024
2024 arXiv
-
[69]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProc. CVPR, 2016
2016
-
[70]
Conformer: Convolution- augmented transformer for speech recognition,
A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y . Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y . Wu, and R. Pang, “Conformer: Convolution- augmented transformer for speech recognition,” inProc. INTER- SPEECH, 2020, pp. 5036–5040
2020
-
[71]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” inProc. NeurIPS, vol. 30, 2017
2017
-
[72]
BERT: pre-training of deep bidirectional transformers for language understanding,
J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” inProc. NAACL, 2019, pp. 4171–4186
2019
-
[73]
Arcface: Additive angular margin loss for deep face recognition,
J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” inProc. CVPR, 2019
2019
-
[74]
V oxceleb2: Deep speaker recognition,
J. S. Chung, A. Nagrani, and A. Zisserman, “V oxceleb2: Deep speaker recognition,” inProc. INTERSPEECH, 2018, pp. 1086–1090
2018
-
[75]
V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,
Y . Lin, M. Cheng, F. Zhang, Y . Gao, S. Zhang, and M. Li, “V oxblink2: A 100k+ speaker recognition corpus and the open-set speaker-identification benchmark,” inProc. INTERSPEECH, 2024, pp. 4263–4267
2024
-
[76]
Funasr: A fundamental end-to-end speech recognition toolkit,
Z. Gao, Z. Li, J. Wang, H. Luo, X. Shi, M. Chen, Y . Li, L. Zuo, Z. Du, and S. Zhang, “Funasr: A fundamental end-to-end speech recognition toolkit,” inProc. INTERSPEECH, 2023, pp. 1593–1597
2023
-
[77]
The second dihard diarization challenge: Dataset, task, and baselines,
N. Ryant, K. Church, C. Cieri, A. Cristia, J. Du, S. Ganapathy, and M. Liberman, “The second dihard diarization challenge: Dataset, task, and baselines,” inProc. INTERSPEECH, 2019, pp. 978–982
2019
-
[78]
The third dihard diarization challenge,
N. Ryant, P. Singh, V . Krishnamohan, R. Varma, K. Church, C. Cieri, J. Du, S. Ganapathy, and M. Liberman, “The third dihard diarization challenge,” inProc. INTERSPEECH, 2021, pp. 3570–3574
2021
-
[79]
V oxceleb: A large- scale speaker identification dataset,
A. Nagrani, J. S. Chung, and A. Zisserman, “V oxceleb: A large- scale speaker identification dataset,” inProc. INTERSPEECH, 2017, pp. 2616–2620
2017
-
[80]
Distilling the knowledge in a neural network,
G. Hinton, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015
2015 arXiv
-
[81]
Musan: A music, speech, and noise corpus,
D. Snyder, G. Chen, and D. Povey, “Musan: A music, speech, and noise corpus,”arXiv preprint arXiv:1510.08484, 2015
2015 arXiv
-
[82]
A study on data augmentation of reverberant speech for robust speech recognition,
T. Ko, V . Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” inProc. ICASSP, 2017, pp. 5220–5224
2017
-
[83]
Decoupled weight decay regularization,
I. Loshchilov, “Decoupled weight decay regularization,”arXiv preprint arXiv:1711.05101, 2017
2017 arXiv
-
[84]
Online speaker diarization with core samples selection,
Y . Yue, J. Du, M.-K. He, Y . Yeung, and R. Wang, “Online speaker diarization with core samples selection,” inProc. INTERSPEECH, 2022, pp. 1466–1470
2022
-
[85]
Absolute decision corrupts absolutely: Conservative online speaker diarisation,
Y . Kwon, H.-S. Heo, B.-J. Lee, Y . J. Kim, and J.-W. Jung, “Absolute decision corrupts absolutely: Conservative online speaker diarisation,” inProc. ICASSP, 2023, pp. 1–5. 16
2023
-
[86]
End-to-end speaker diarization as post-processing,
S. Horiguchi, P. Garc ´ıa, Y . Fujita, S. Watanabe, and K. Nagamatsu, “End-to-end speaker diarization as post-processing,” inProc. ICASSP, 2021, pp. 7188–7192
2021
-
[87]
Attention-based encoder- decoder end-to-end neural diarization with embedding enhancer,
Z. Chen, B. Han, S. Wang, and Y . Qian, “Attention-based encoder- decoder end-to-end neural diarization with embedding enhancer,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 32, pp. 1636–1649, 2024
2024
-
[88]
Powerset multi-class cross entropy loss for neural speaker diarization,
A. Plaquet and H. Bredin, “Powerset multi-class cross entropy loss for neural speaker diarization,” inProc. INTERSPEECH, 2023, pp. 3222– 3226
2023
-
[89]
Eend-m2f: Masked- attention mask transformers for speaker diarization,
M. H ¨ark¨onen, S. J. Broughton, and L. Samarakoon, “Eend-m2f: Masked- attention mask transformers for speaker diarization,” inProc. INTER- SPEECH, 2024, pp. 37–41
2024
-
[90]
The hitachi- jhu dihard iii system: Competitive end-to-end neural diarization and x-vector clustering systems combined by dover-lap,
S. Horiguchi, N. Yalta, P. Garcia, Y . Takashima, Y . Xue, D. Raj, Z. Huang, Y . Fujita, S. Watanabe, and S. Khudanpur, “The hitachi- jhu dihard iii system: Competitive end-to-end neural diarization and x-vector clustering systems combined by dover-lap,”arXiv preprint arXiv:21...
2021 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.