Pith. sign in

REVIEW 4 major objections 5 minor 43 references

Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Simple addition of MFCC and Whisper features beats attention-based fusion for voice activity detection, and the resulting lightweight model beats Pyannote by 2.04% average DER.

desk verdict Useful VAD fusion benchmark, but the addition-over-attention claim is overstated and needs seeds and a tuned CA baseline. read the letter →

arxiv 2506.01365 v1 pith:GQLVZV55 submitted 2025-06-02 cs.SD cs.CLeess.AS

Kumud Tripathi , Chowdam Venkata Kumar , Pankaj Wasnik This is my paper
classification cs.SDcs.CLeess.AS
keywords voiceactivitydetectionfeaturefusionpre-trainedspeechmodelsMFCCcross-attentionWhispererrorratelightweightmodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that voice activity detection (VAD) does not need an attention-based fusion stage: adding a traditional MFCC stream directly to features extracted from a frozen pretrained speech model is enough to get accurate, lightweight detection. The authors build FusionVAD, a single compact architecture that projects MFCC and pretrained-model features and combines them by concatenation, element-wise addition, or cross-attention, and they test six pretrained encoders (wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, Whisper) on AMI, Callhome, and VoxConverse. In their results, every fusion variant beats its single-feature counterpart, cross-attention never yields the best result for any encoder, and addition gives the best detection error rate (DER) in four of six cases. The best model, Whisper-MFCC-Addition, improves detection error rate by an average of 2.04% absolute over the Pyannote baseline across the three datasets. The practical point is that a cheap fusion recipe can replace a heavier attention mechanism for a task that sits in front of almost every speech system.

What carries the argument

The load-bearing object is FusionVAD's fusion stage: after projection layers map MFCCs and frozen pretrained-model features to 128 dimensions, the two vectors are combined either by concatenation, by element-wise addition, or by cross-attention in which MFCC features serve as queries and pretrained features as keys and values. The argument is carried by the observed complementarity of the two feature families: MFCCs alone give low miss rates but high false-alarm rates, while each pretrained encoder gives the opposite, so additive combination reduces both error directions. The equal projection to 128 dimensions makes the feature types directly comparable before fusion, and the fixed lightweight backbone of two bidirectional LSTMs plus feed-forward layers isolates the effect of the fusion choice.

What would settle it

Re-run the cross-attention fusion variant with a modest hyperparameter search over heads, hidden sizes, dropout, and learning rate on the same three datasets and training budget; if any tuned attention configuration matches or beats Whisper-MFCC-Addition in DER, the paper's claim that attention is not the answer for VAD collapses. Repeating each Table 1 entry over five random seeds and reporting the spread would also show whether the reported gaps exceed run-to-run noise.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that simple feature fusion is not a fallback but the better option for VAD: the complementary error patterns of MFCCs and pretrained speech features can be canceled by projecting both into a shared 128-dimensional space and adding them, whereas a multi-head cross-attention fusion block adds parameters and training time without ever producing the best result. The evidence is a systematic comparison over six frozen encoders in which fusion always improves on the single-feature baseline and the cross-attention variant is consistently worse than the best simple fusion. The headline system, Whisper-MFCC-Addition, beats the Pyannote VAD baseline on each of the three datasets, with an average absolute DER reduction of 2.04%, while using fewer trainable parameters and up to about 10% less training time than the cross-attention variant. The paper interprets this as evidence that VAD is a comparatively simple frame-level task that does not need the long-range contextual modeling attention provides.

Load-bearing premise

The central conclusion rests on assuming that one untuned cross-attention setup (two heads, 128-dimensional hidden space, no hyperparameter search) fairly represents attention-based fusion, and that single-run results are stable enough for the reported 2.04% gap to be meaningful.

Editorial extensions

If this is right

  • Every fusion model beats its single-feature base, so MFCC and pretrained speech features are complementary for VAD and are better used together than alone.
  • Addition is the strongest simple fusion: it beats concatenation in four of six encoders and, paired with Whisper, yields the best overall model, making it a natural default before trying learned fusion.
  • Cross-attention costs more parameters and up to about 10% more training time than addition while never producing the best DER for any encoder, so its extra complexity is hard to justify on this task.
  • Whisper-MFCC-Addition surpasses the Pyannote baseline on AMI, Callhome, and VoxConverse, averaging an absolute 2.04% DER improvement, which positions lightweight fusion as a competitive alternative to a purpose-built VAD system.
  • Using multi-resolution cochleagram features in place of MFCC hurts the best model by about 2% DER, indicating that MFCC is the better complementary hand-crafted feature in this recipe.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The result is best read as a comparison against one specific cross-attention configuration (two heads, 128-dimensional hidden space, untuned); it supports 'simple fusion is sufficient here' more strongly than 'attention can never help VAD.'
  • The complementary-error pattern suggests a transferable recipe for other speech front-ends: identify the error direction each feature family is biased toward, then fuse with a feature biased the opposite way, with addition as the cheap first attempt.
  • The efficiency advantage is demonstrated for trainable parameters and training time; the natural extension is measuring inference latency and memory on-device, where lightweight VAD is most valuable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FusionVAD, a voice activity detection framework that fuses hand-crafted MFCC features with features from six frozen pre-trained speech models (wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper) using three fusion strategies: concatenation, element-wise addition, and cross-attention. All models are trained and evaluated with the Pyannote toolkit on AMI, Callhome, and VoxConverse, with DER, FAR, and MR as metrics. The central claims are that simple fusion techniques, particularly addition, outperform cross-attention in both accuracy and efficiency, that fusion improves over single-feature models, and that the best model, Whisper-MFCC-Addition, beats the Pyannote VAD baseline by an average of 2.04% absolute DER.

Significance. If fully supported, the result would be practically valuable: it would show that lightweight feature fusion can improve VAD performance without the parameter and training-time overhead of attention-based fusion, and it would provide an empirical counterpoint to the trend of using attention mechanisms as default components. The paper's strengths include a systematic comparison across six PTMs, a consistent training architecture and protocol for all models, evaluation on three public datasets, and an efficiency analysis in terms of trainable parameters and training time. However, the headline claim is currently overgeneralized because the cross-attention baseline is a single untuned configuration and because the paper's own Table 1 contains numerical contradictions that undermine the 'addition particularly outperforms CA' statement.

major comments (4)
  1. [Section 4.2, Table 1] The claim that simple fusion, particularly addition, outperforms cross-attention is internally contradicted by Table 1. For wav2vec 2.0, addition gives DER 6.74 while cross-attention gives 6.12; for HuBERT, addition gives 6.82 while cross-attention gives 6.39. In addition, wav2vec 2.0 with addition (6.74) is worse than the base wav2vec 2.0 model (6.70), which directly contradicts the statement in Section 4.2 that 'all fusion-based models outperform their respective base models.' The authors should either restrict the claim to concatenation, report a more nuanced comparison, or provide a plausible explanation for these exceptions.
  2. [Section 2.2, Section 4.2] The cross-attention implementation is a single untuned configuration: two attention heads, a 128-dimensional hidden space, and MFCC features always serving as queries with PTM features as keys/values. No hyperparameter search, no alternative query-key assignment, and no ablations are reported. The paper's broader conclusion that 'CA may not be optimal for this task' and the title's 'Attention Is Not Always the Answer' are therefore not supported by the evidence. A properly tuned cross-attention variant, or a bidirectional assignment with PTM as queries, could plausibly close or reverse the 0.7-1.5 DER gaps observed in Table 1. The authors should either add a systematic study of the CA hyperparameters and query-key assignment, or explicitly reframe the conclusion as 'our cross-attention configuration underperforms.'
  3. [Section 3.2, Table 1] All results appear to come from a single training run, with no repeated seeds, confidence intervals, or statistical significance tests. This is especially problematic for the smaller contrasts in Table 1, such as UniSpeech addition (DER 5.55) versus concatenation (DER 5.58), and it weakens the claim that addition is consistently better than concatenation. The authors should report mean and variance over at least three seeds for the main fusion comparisons, or otherwise justify why single runs are sufficient.
  4. [Section 4.2, Table 1] Averaging the DER over the six PTMs in Table 1 shows concatenation giving 5.285 versus addition at 5.572, so the statement that 'addition emerges as the most effective fusion strategy in four out of six models' is not a robust aggregate conclusion. The large wins of concatenation on wav2vec 2.0 and HuBERT outweigh the smaller wins of addition on other models. The paper should report the average or a paired comparison across models, and either temper the claim about addition being generally best or explain why the model-specific exceptions do not affect the practical recommendation.
minor comments (5)
  1. [Section 2.2] The cross-attention description should clarify the exact dimensions: whether the projected MFCC and PTM features are already 128-dimensional before attention, and whether layer normalization is applied to the residual stream only or to the attended output as well.
  2. [Section 4.2] The comparisons to rVAD and to MRCG features are reported only as percentages in prose, without a table or detailed experimental setup. Please provide the underlying numbers, the evaluation protocol, and the model configuration so these claims are verifiable.
  3. [Table 1] The caption states 'Bold represents the best result,' but the table as rendered does not show any bold entries. Please ensure the formatting is visible in the final version.
  4. [Abstract and Table 2] The 'absolute average improvement of 2.04%' is computed from the per-dataset DER differences in Table 2 (3.82, 1.40, and 0.92 percentage points on AMI, Callhome, and VoxConverse, respectively); stating this computation in the text would help readers reproduce the claim.
  5. [General] Several references contain line breaks in URLs (e.g., references 1, 2, 3, 4); these should be cleaned or placed on a single line for the camera-ready version.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper is an empirical benchmark with externally evaluated baselines and no fitted-parameter predictions.

full rationale

This paper reports measurements, not a derivation. The three fusion techniques (concatenation, addition, cross-attention) are defined independently of the outcome, and the claim that addition outperforms cross-attention is a direct reading of Table 1 rather than a quantity fitted from that table. The comparison to Pyannote uses an external baseline trained with the same datasets and protocol, so it is not circular. The only self-citation, reference [19] on Whisper prompt-tuning, is used for background on Whisper's multilingual capability and is not load-bearing for the VAD fusion claims. The paper explicitly labels its 'VAD is a simpler task' explanation as a hypothesis, not an input or derived result. The weakness that the cross-attention configuration is untuned (two heads, fixed 128-dimensional hidden space) is a fairness and generalizability concern, not a circularity: it does not make the reported DER values equivalent to the model definitions or to any fitted parameter. No step in the paper reduces by construction to its own inputs, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new mathematical axioms or invented entities. The main assumptions are experimental design choices: frozen PTM encoders, a fixed cross-attention configuration, and comparable dataset splits. No free parameters are fitted to test data.

assumptions (3)
  • domain assumption Pre-trained model encoders are kept frozen during training and their features are used as fixed representations.
    The authors state this in Section 2.1 to isolate feature quality rather than fine-tuning behavior. This assumption shapes all reported results and may not hold if fine-tuning changes the comparison.
  • domain assumption The cross-attention fusion implementation with two attention heads and a 128-dimensional hidden space is a representative attention-based fusion method.
    Section 2.2 defines this configuration without tuning. The paper's central conclusion depends on this being a fair comparison, which is not verified by ablating attention hyperparameters.
  • domain assumption The dataset splits and training protocols are comparable across models and with the Pyannote baseline.
    The authors follow the split methodology of [30] and train Pyannote with the same datasets (Section 3.1, 3.2). Any inconsistency in split or preprocessing would affect the headline comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion." pith.science (2026). https://pith.science/paper/GQLVZV55

@misc{pith2026250601365,
  author       = {Pith},
  title        = {Pith review of: Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GQLVZV55}},
  note         = {Machine review of arXiv:2506.01365}
}
read the original abstract

Voice Activity Detection (VAD) plays a key role in speech processing, often utilizing hand-crafted or neural features. This study examines the effectiveness of Mel-Frequency Cepstral Coefficients (MFCCs) and pre-trained model (PTM) features, including wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper. We propose FusionVAD, a unified framework that combines both feature types using three fusion strategies: concatenation, addition, and cross-attention (CA). Experimental results reveal that simple fusion techniques, particularly addition, outperform CA in both accuracy and efficiency. Fusion-based models consistently surpass single-feature models, highlighting the complementary nature of MFCCs and PTM features. Notably, our best-performing fusion model exceeds the state-of-the-art Pyannote across multiple datasets, achieving an absolute average improvement of 2.04%. These results confirm that simple feature fusion enhances VAD robustness while maintaining computational efficiency.

Figures

Figures reproduced from arXiv: 2506.01365 by the authors.

Figure 1
Figure 1. Overview of the FusionVAD Framework with Different Feature Fusion Strategies. non-speech classification. 2. Introduces a simple yet effective feature fusion method that combines MFCC and PTM representations. 3. Conducts a comprehensive analysis of state-of-the-art PTMs to evaluate their effectiveness for VAD. 4. Demonstrates that addition-based feature fusion enhances both accuracy and computational efficiency. 2. M… view at source ↗
Figure 3
Figure 3. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Analysis of training time for all the FusionVAD mod￾els using different fusion techniques (M: minutes and S: sec￾onds). niques outperform the more complex CA method for VAD. We hypothesize that VAD is inherently a simpler task that does not require extensive contextual information, unlike more complex tasks such as ASR. Additionally, the cross-attention method in￾troduces a higher number of trainable parameters, as … view at source ↗
Figures from the paper (1 more)
Figure 2
Figure 2. Figure 2: Feature fusion outputs (Green: Addition, Red: Con￾catenation, and Purple: Cross-Attention) along with the orig￾inal reference (Yellow) for all FusionVAD models on a single audio segment from the AMI file ”EN2004a”. This highlights the advantage of combining features th…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 34 canonical work pages

  1. [1]

    Introduction V oice Activity Detection (V AD) is the task of detecting speech segments within an audio signal [1]. It serves as a fundamental pre-processing step for various speech-related applications, in- cluding Automatic Speech Recognition (ASR), Speaker Recog- nition, Speaker Verification, and Speaker Diarization [2, 3, 4]. By accurately identifying ...

    work page

  2. [2]

    Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion

    Examines the role of attention mechanism in feature fusion for V oice Activity Detection (V AD) and shows that attention- based fusion is not always necessary for effective speech and arXiv:2506.01365v1 [cs.SD] 2 Jun 2025 Figure 1:Overview of the FusionVAD Framework with Different Feature Fusion Strategies. non-speech classification

    work page Pith review arXiv 2025

  3. [3]

    Introduces a simple yet effective feature fusion method that combines MFCC and PTM representations

    work page

  4. [4]

    Conducts a comprehensive analysis of state-of-the-art PTMs to evaluate their effectiveness for V AD

    work page

  5. [5]

    Demonstrates that addition-based feature fusion enhances both accuracy and computational efficiency

    work page

  6. [6]

    MFCC vs PTM Features Pre-trained model based features have proven effective for various speech tasks, including V AD

    Methodology 2.1. MFCC vs PTM Features Pre-trained model based features have proven effective for various speech tasks, including V AD. These models leverage self-attention to capture long-range dependencies and generate contextual representations, which are particularly beneficial in noisy environments. In contrast, traditional hand-crafted fea- tures lik...

    work page

  7. [7]

    Dataset and Evaluation Metrics We conducted all our experiments on three publicly available datasets, i.e., AMI, Callhome, and V oxConverse, to ensure do- main diversity

    Experiments 3.1. Dataset and Evaluation Metrics We conducted all our experiments on three publicly available datasets, i.e., AMI, Callhome, and V oxConverse, to ensure do- main diversity. We followed the dataset split methodology from [30]. Since V oxConverse lacks an official training set, we parti- tioned its development set into 144 training files and ...

    work page

  8. [8]

    A survey of convo- lutional neural networks: analysis, applications, and prospects,

    Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convo- lutional neural networks: analysis, applications, and prospects,” IEEE transactions on neural networks and learning systems, vol. 33, no. 12, pp. 6999–7019, 2021

    work page 2021

Show all 43 references
  1. [9]

    MFCC vs PTM Features First 3 columns in Table 1 shows the performance of V AD with individual features in terms of DER, FAR and MR

    Results and Analysis 4.1. MFCC vs PTM Features First 3 columns in Table 1 shows the performance of V AD with individual features in terms of DER, FAR and MR. Whisper shows better performance than all other models. It is observed that MFCCs show high FAR than all other PTM feat...

  2. [10]

    Our experiments show that simple fusion methods like addition and concatenation consistently outperform the more complex cross-attention mechanism

    Conclusion This study investigates the impact of different feature fusion techniques for V oice Activity Detection (V AD) by combining hand-crafted MFCC features with pre-trained model (PTM) fea- tures. Our experiments show that simple fusion methods like addition and concaten...

  3. [11]

    Temporal modeling using di- lated convolution and gating for voice-activity-detection,

    S.-Y . Chang, B. Li, G. Simko, T. N. Sainath, A. Tripathi, A. van den Oord, and O. Vinyals, “Temporal modeling using di- lated convolution and gating for voice-activity-detection,” in2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, ...

    2018

  4. [12]

    Wavoice: An mmwave-assisted noise-resistant speech recogni- tion system,

    T. Liu, C. Wang, Z. Li, M.-C. Huang, W. Xu, and F. Lin, “Wavoice: An mmwave-assisted noise-resistant speech recogni- tion system,”ACM Transactions on Sensor Networks, vol. 20, no. 4, pp. 1–29, 2024

    2024

  5. [13]

    Profile-error-tolerant target-speaker voice activity detection,

    D. Wang, X. Xiao, N. Kanda, M. Yousefi, T. Yoshioka, and J. Wu, “Profile-error-tolerant target-speaker voice activity detection,” in ICASSP 2024-2024 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 11 906–11 910

    2024

  6. [14]

    Speaker embeddings with weakly supervised voice activity detection for efficient speaker diarization,

    J. Thienpondt and K. Demuynck, “Speaker embeddings with weakly supervised voice activity detection for efficient speaker diarization,”arXiv preprint arXiv:2405.09142, 2024

    2024 arXiv

  7. [15]

    Unveiling the state-of-the-art: A com- prehensive survey on voice activity detection techniques,

    R. M. Patil and C. Patil, “Unveiling the state-of-the-art: A com- prehensive survey on voice activity detection techniques,” in2024 Asia Pacific Conference on Innovation in Technology (APCIT). IEEE, 2024, pp. 1–5

    2024

  8. [16]

    V oice activity detection: Fusion of time and frequency domain features with a svm classifier,

    S. Alimi and O. Awodele, “V oice activity detection: Fusion of time and frequency domain features with a svm classifier,”Com- put. Eng. Intell. Syst, vol. 13, no. 3, pp. 20–29, 2022

    2022

  9. [17]

    Analy- sis of derivative of instantaneous frequency and its application to voice activity detection,

    B. T. Nguyen, Y . Wakabayashi, K. Iwai, and T. Nishiura, “Analy- sis of derivative of instantaneous frequency and its application to voice activity detection,”Applied Acoustics, vol. 181, p. 108116, 2021

    2021

  10. [18]

    Robust speech recognition via large-scale weak supervision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

    2023

  11. [19]

    A review of recurrent neural networks: Lstm cells and network architectures,

    Y . Yu, X. Si, C. Hu, and J. Zhang, “A review of recurrent neural networks: Lstm cells and network architectures,”Neural compu- tation, vol. 31, no. 7, pp. 1235–1270, 2019

    2019

  12. [20]

    A hybrid cnn-bilstm voice activity detector,

    N. Wilkinson and T. Niesler, “A hybrid cnn-bilstm voice activity detector,” inICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 6803–6807

    2021

  13. [21]

    Feature learn- ing with raw-waveform cldnns for voice activity detection

    R. Zazo, T. N. Sainath, G. Simko, and C. Parada, “Feature learn- ing with raw-waveform cldnns for voice activity detection.” inIn- terspeech, 2016, pp. 3668–3672

    2016

  14. [22]

    wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,”Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020

    2020

  15. [23]

    Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdi- nov, and A. Mohamed, “Hubert: Self-supervised speech represen- tation learning by masked prediction of hidden units,”IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021

    2021

  16. [24]

    Wavlm: Large-scale self- supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiaoet al., “Wavlm: Large-scale self- supervised pre-training for full stack speech processing,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022

    2022

  17. [25]

    A closer look at wav2vec2 embeddings for on-device single-channel speech en- hancement,

    R. Shankar, K. Tan, B. Xu, and A. Kumar, “A closer look at wav2vec2 embeddings for on-device single-channel speech en- hancement,” inICASSP 2024-2024 IEEE International Confer- ence on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 751–755

    2024

  18. [26]

    Unispeech-sat: Universal speech rep- resentation learning with speaker aware pre-training,

    S. Chen, Y . Wu, C. Wang, Z. Chen, Z. Chen, S. Liu, J. Wu, Y . Qian, F. Wei, J. Liet al., “Unispeech-sat: Universal speech rep- resentation learning with speaker aware pre-training,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (I...

    2022

  19. [27]

    Scaling speech technology to 1,000+ languages,

    V . Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandiet al., “Scaling speech technology to 1,000+ languages,”Journal of Machine Learning Research, vol. 25, no. 97, pp. 1–52, 2024

    2024

  20. [28]

    Spot the conversation: speaker diarisation in the wild,

    J. S. Chung, J. Huh, A. Nagrani, T. Afouras, and A. Zisserman, “Spot the conversation: speaker diarisation in the wild,”arXiv preprint arXiv:2007.01216, 2020

    2007 arXiv

  21. [29]

    Base version checkpoints are considered for wav2vec 2.01, Hu- 1https://huggingface.co/facebook/ wav2vec2-base BERT2, WavLM3, UniSpeech 4and Whisper5

    is used for training and testing the models and PTM speech encoders model checkpoints are obtained from huggingface. Base version checkpoints are considered for wav2vec 2.01, Hu- 1https://huggingface.co/facebook/ wav2vec2-base BERT2, WavLM3, UniSpeech 4and Whisper5. One billio...

  22. [30]

    Enhancing whisper’s accu- racy and speed for indian languages through prompt-tuning and tokenization,

    K. Tripathi, R. Gothi, and P. Wasnik, “Enhancing whisper’s accu- racy and speed for indian languages through prompt-tuning and tokenization,” inICASSP 2025-2025 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    2025

  23. [31]

    Self-supervised speech representation learning: A review,

    A. Mohamed, H.-y. Lee, L. Borgholt, J. D. Havtorn, J. Edin, C. Igel, K. Kirchhoff, S.-W. Li, K. Livescu, L. Maaløeet al., “Self-supervised speech representation learning: A review,”IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1179–1210, 2022

    2022

  24. [32]

    Investigating prosodic signatures via speech pre- trained models for audio deepfake source attribution,

    O. C. Phukan, D. Singh, S. R. Behera, A. B. Buduru, and R. Sharma, “Investigating prosodic signatures via speech pre- trained models for audio deepfake source attribution,”arXiv preprint arXiv:2412.17796, 2024

    2024 arXiv

  25. [33]

    Multi-view multi-task modeling with speech foundation models for speech forensic tasks,

    O. C. Phukan, D. Koshal, S. R. Behera, A. B. Buduru, and R. Sharma, “Multi-view multi-task modeling with speech foundation models for speech forensic tasks,”arXiv preprint arXiv:2410.12947, 2024

    2024 arXiv

  26. [34]

    A transformer-based voice activity detector,

    B. Karan, J. J. van V ¨uren, F. de Wet, and T. Niesler, “A transformer-based voice activity detector,” inProc. Interspeech 2024, 2024, pp. 3819–3823

    2024

  27. [35]

    Multitask detection of speaker changes, overlapping speech and voice activity using wav2vec 2.0,

    M. Kune ˇsov´a and Z. Zaj ´ıc, “Multitask detection of speaker changes, overlapping speech and voice activity using wav2vec 2.0,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  28. [36]

    Feature extrac- tion using mfcc,

    S. Gupta, J. Jaafar, W. W. Ahmad, and A. Bansal, “Feature extrac- tion using mfcc,”Signal & Image Processing: An International Journal, vol. 4, no. 4, pp. 101–108, 2013

    2013

  29. [37]

    Unleashing the killer corpus: experiences in creating the multi-everything ami meeting corpus,

    J. Carletta, “Unleashing the killer corpus: experiences in creating the multi-everything ami meeting corpus,”Language Resources and Evaluation, vol. 41, pp. 181–190, 2007

    2007

  30. [38]

    2000 nist speaker recognition evaluation,

    “2000 nist speaker recognition evaluation,”” https://catalog.ldc.upenn.edu/LDC2001S97, 2000

    2000

  31. [39]

    Pyannote. audio: neural building blocks for speaker diarization,

    H. Bredin, R. Yin, J. M. Coria, G. Gelly, P. Korshunov, M. Lavechin, D. Fustes, H. Titeux, W. Bouaziz, and M.-P. Gill, “Pyannote. audio: neural building blocks for speaker diarization,” inICASSP 2020-2020 IEEE International Conference on Acous- tics, Speech and Signal Processi...

    2020

  32. [40]

    End-to-end speaker segmen- tation for overlap-aware resegmentation,

    H. Bredin and A. Laurent, “End-to-end speaker segmen- tation for overlap-aware resegmentation,”arXiv preprint arXiv:2104.04045, 2021

    2021 arXiv

  33. [41]

    Speaker recognition from raw wave- form with sincnet,

    M. Ravanelli and Y . Bengio, “Speaker recognition from raw wave- form with sincnet,” in2018 IEEE spoken language technology workshop (SLT). IEEE, 2018, pp. 1021–1028

    2018

  34. [42]

    rvad: An unsupervised segment- based robust voice activity detection method,

    Z.-H. Tan, N. Dehaket al., “rvad: An unsupervised segment- based robust voice activity detection method,”Computer speech & language, vol. 59, pp. 1–21, 2020

    2020

  35. [43]

    Boosted deep neural networks and multi-resolution cochleagram features for voice activity detec- tion

    X.-L. Zhang and D. Wang, “Boosted deep neural networks and multi-resolution cochleagram features for voice activity detec- tion.” inINTERSPEECH, 2014, pp. 1534–1538

    2014

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.