ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
read the original abstract
Current speaker verification techniques rely on a neural network to extract speaker representations. The successful x-vector architecture is a Time Delay Neural Network (TDNN) that applies statistics pooling to project variable-length utterances into fixed-length speaker characterizing embeddings. In this paper, we propose multiple enhancements to this architecture based on recent trends in the related fields of face verification and computer vision. Firstly, the initial frame layers can be restructured into 1-dimensional Res2Net modules with impactful skip connections. Similarly to SE-ResNet, we introduce Squeeze-and-Excitation blocks in these modules to explicitly model channel interdependencies. The SE block expands the temporal context of the frame layer by rescaling the channels according to global properties of the recording. Secondly, neural networks are known to learn hierarchical features, with each layer operating on a different level of complexity. To leverage this complementary information, we aggregate and propagate features of different hierarchical levels. Finally, we improve the statistics pooling module with channel-dependent frame attention. This enables the network to focus on different subsets of frames during each of the channel's statistics estimation. The proposed ECAPA-TDNN architecture significantly outperforms state-of-the-art TDNN based systems on the VoxCeleb test sets and the 2019 VoxCeleb Speaker Recognition Challenge.
This paper has not been read by Pith yet.
Forward citations
Cited by 15 Pith papers
-
AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes
Introduces AVTrack dataset for audio-visual tracking in challenging human-centric scenes, demonstrating performance drops in existing methods.
-
DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition
DECKER improves cross-keyboard and cross-user keystroke identification from audio using domain-invariant embeddings on the new diverse HEAR dataset, with additional gains from language model sequence correction.
-
An Exploration of Mamba for Speech Self-Supervised Models
Mamba-based HuBERT models match or exceed Transformer versions on speech tasks while using far less compute for long sequences and streaming ASR.
-
TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech
TLDR groups codec tokens into patches for patch-level autoregressive modeling in pretrained TTS systems, yielding 1.8x speedup and 75% KV-cache reduction at patch size 4.
-
Fuzzy PyTorch: Rapid Numerical Variability Evaluation for Deep Learning Models
Fuzzy PyTorch integrates stochastic arithmetic into PyTorch through Probabilistic Rounding with Instruction Set Management, delivering 5-60x speedups over Verrou while scaling to 341 million parameter models.
-
DECKER: Domain-invariant Embedding for Cross-Keyboard Extraction and Recognition
DECKER is a domain-invariant four-stage framework (keyboard normalization, adversarial disentanglement, cross-keyboard contrastive alignment, acoustic style randomization) plus LLM post-processing that improves keystr...
-
MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation
MMControl adds multi-modal controls for identity, timbre, pose, and layout to unified audio-video diffusion models via dual-stream injection and adjustable guidance scaling.
-
MMControl: Unified Multi-Modal Control for Joint Audio-Video Generation
Translation function vectors extracted from a single English→X direction transfer across unseen target languages in three multilingual LLMs, extending language-agnosticity findings to task-level representations.
-
AI-Driven Analytics of Team-Teaching Talk: Acoustic Patterns across Experience, Cohorts and the Learning Design
Automated acoustic analysis of 36 team-teaching sessions reveals systematic loudness variation differences across teacher experience, student cohorts, and learning task design.
-
ProSDD: Learning Prosodic Representations for Speech Deepfake Detection against Expressive and Emotional Attacks
ProSDD learns speaker-conditioned prosodic variation from real speech via supervised masked prediction and jointly optimizes it with spoof detection, cutting EER substantially on ASVspoof 2024 and emotional datasets.
-
Time-Domain Voice Identity Morphing (TD-VIM): A Signal-Level Approach to Morphing Attacks on Speaker Verification Systems
TD-VIM creates signal-level morphed voice samples that achieve G-MAP attack success rates up to 99.74% against deep-learning and commercial speaker verification systems.
-
Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers
A framework combining stochastic zeroth-order optimization and dynamic low-rank surrogate modeling with an implicit projector-splitting integrator enables end-to-end training of hybrid neural networks containing black...
-
LISE : Listenable Interpretable Speaker Embeddings
LISE decomposes pretrained speaker embeddings into components that preserve ASV performance with negligible EER degradation and enable listeners to distinguish speakers at 83.9% accuracy.
-
DRL-CLBA: A Clean Label Backdoor Attack for Speech Classification via DDPG Reinforcement Learning
DRL-CLBA applies DDPG reinforcement learning and deep audio steganography to create sample-specific clean-label backdoor attacks on speech DNNs that resist fine-tuning, pruning, and spectral signature defenses.
-
RIVET: Robust Idempotent Voice Attribute Editing
RIVET enforces an idempotency objective during training of voice attribute editing models to improve robustness to noisy labels, outperforming standard training on controlled noise and the GLOBE dataset.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.