VoxCeleb2: Deep Speaker Recognition
read the original abstract
The objective of this paper is speaker recognition under noisy and unconstrained conditions. We make two key contributions. First, we introduce a very large-scale audio-visual speaker recognition dataset collected from open-source media. Using a fully automated pipeline, we curate VoxCeleb2 which contains over a million utterances from over 6,000 speakers. This is several times larger than any publicly available speaker recognition dataset. Second, we develop and compare Convolutional Neural Network (CNN) models and training strategies that can effectively recognise identities from voice under various conditions. The models trained on the VoxCeleb2 dataset surpass the performance of previous works on a benchmark dataset by a significant margin.
This paper has not been read by Pith yet.
Forward citations
Cited by 21 Pith papers
-
HoliDubber: Holistic Video Dubbing for Complex Acoustic Scenes via Text-Guided Audio Synthesis
HoliDubber introduces a patch-based autoregressive diffusion transformer for joint text-guided synthesis of speech and ambient audio in video dubbing, with a new benchmark showing outperformance over prior speech-only...
-
Hierarchical Codec Diffusion for Video-to-Speech Generation
HiCoDiT generates speech from video by conditioning low-level RVQ tokens on speaker identity and high-level tokens on facial expressions via a dual-scale normalized diffusion transformer.
-
Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models
Audio Flamingo 3 introduces an open large audio-language model achieving new state-of-the-art results on over 20 audio understanding and reasoning benchmarks using a unified encoder and curriculum training on open data.
-
The DeepSpeak Dataset
DeepSpeak provides over 100 hours of consented, identity-matched real and modern deepfake audiovisual content focused on talking heads, with evaluations showing existing detectors fail to generalize without retraining.
-
Real-Time Generation of Streamable Talking Portrait Video with Reference-Guided Deep Compression VAEs
A causal VAE with variable reference guidance and a Rectified Flow Transformer enables real-time streamable high-quality talking portrait video generation from audio and images.
-
Temporally-Aligned Evaluation for Audio-Driven Talking Head Generation
Reformulates evaluation of audio-driven talking head generation as a sequence alignment problem using Soft DTW, showing improved robustness and consistency across 20 methods and seven datasets.
-
IP-Adapter Is All You Need: Towards Fine-Tuning-Free Diffusion-Based Talking Face Generation
A fine-tuning-free framework combines pretrained Stable Diffusion with IP-Adapter plus three parameter-free modules to achieve improved lip synchronization and visual quality in talking face generation.
-
Head Similarity: Modeling Structured Whole-Head Appearance Beyond Face Recognition
Head Similarity extends identity recognition to structured whole-head similarity by capturing intra-identity appearance variations via hierarchical supervision on a weakly-labeled video benchmark.
-
Are DeepFakes Realistic Enough? Exploring Semantic Mismatch as a Novel Challenge
The paper introduces semantic mismatch between authentic audio and video as a new DeepFake detection challenge via the RARV-SMM class and demonstrates that a semantic reinforcement strategy with ImageBind embeddings i...
-
Explainable AI in Speaker Recognition -- Making Latent Representations Understandable
Speaker recognition networks form hierarchical clusters in latent space that can be matched to semantic classes using new HCCM algorithm and quantified by Liebig's score.
-
OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation
A large-scale, hierarchically-annotated human-centric video dataset and a three-level benchmark (OHBench) are presented, showing that fine-tuning on a subset of the data improves audio-video generation model performance.
-
OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation
OmniHuman is a new large-scale multi-scene dataset with video-, frame-, and individual-level annotations for human-centric video generation, accompanied by the OHBench benchmark that adds metrics aligned with human pe...
-
DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration
DVFace uses a spatio-temporal dual-codebook and asymmetric fusion in a one-step diffusion model to deliver better video face restoration quality, temporal consistency, and identity preservation than recent methods.
-
Speaker Recognition with Random Digit Strings Using Uncertainty Normalized HMM-based i-vectors
Digit-specific HMM i-vectors with uncertainty normalization reach 1.52% male and 1.77% female EER on RSR2015 part III using only that corpus and simple cosine scoring.
-
TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration
TIGER is a tri-prior fusion method for face video restoration using identity, geometry, and generative priors with progressive training to achieve SOTA identity fidelity and temporal stability on a new large-scale dataset.
-
Explainable AI in Speaker Recognition -- Attention Map Visualisation and Evaluation
The paper introduces Modified RISE-eval to evaluate GradCAM and LayerCAM attention maps on speaker recognition networks and reports distinct advantages for each method under different conditions.
-
Head-Pose-Aware Visual Speech Recognition with FiLM Modulation
HP-VSR-ResFiLM adds a single residual FiLM modulation block conditioned on head pose to a CNN visual encoder, yielding WER of 25.0% on LRS2 and 33.2% on LRS3 under standard training conditions.
-
TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration
TIGER fuses identity embeddings, disentangled 3D geometry parameters, and one-step generative priors with three-stage training to achieve claimed state-of-the-art identity fidelity and temporal stability in face video...
-
On Low-Bit Quantization Errors in Speaker Verification: Diagnostic and Mitigation
Diagnostic analysis of quantization errors in speaker verification models reveals a 2-bit knee point and supports a calibrated cascade that maintains FP32-level scores at reduced compute cost.
-
Explainable AI in Speaker Recognition -- Making Latent Representations Understandable
Hierarchical clustering of speaker recognition embeddings reveals that the network organizes representations by gender at upper levels and by gender-nationality conjunctions at lower levels, interpreted via a proposed...
-
Assessing the Energy and Carbon Emissions of Neural Speaker Verification Model in Training and Inference
Empirical study finds diminishing accuracy returns against steep energy growth for deeper and wider ResNet speaker verification models on VoxCeleb2.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.