pith. sign in

arxiv: 2111.02735 · v3 · pith:WMU7A6J5new · submitted 2021-11-04 · 💻 cs.CL · cs.NE· cs.SD· eess.AS

A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding

Yingzhi Wang , Abdelmoumene Boumadane , Abdelwahab Heba This is my paper
classification 💻 cs.CL cs.NEcs.SDeess.AS
keywords speechhubertrecognitionwav2vecaccuracyemotionspeakerverification
0
0 comments X
read the original abstract

Speech self-supervised models such as wav2vec 2.0 and HuBERT are making revolutionary progress in Automatic Speech Recognition (ASR). However, they have not been totally proven to produce better performance on tasks other than ASR. In this work, we explored partial fine-tuning and entire fine-tuning on wav2vec 2.0 and HuBERT pre-trained models for three non-ASR speech tasks: Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding. With simple proposed downstream frameworks, the best scores reached 79.58% weighted accuracy on speaker-dependent setting and 73.01% weighted accuracy on speaker-independent setting for Speech Emotion Recognition on IEMOCAP, 2.36% equal error rate for Speaker Verification on VoxCeleb1, 89.38% accuracy for Intent Classification and 78.92% F1 for Slot Filling on SLURP, showing the strength of fine-tuned wav2vec 2.0 and HuBERT on learning prosodic, voice-print and semantic representations.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Acoustic scattering AI for non-invasive object classifications: A case study on hair assessment

    cs.SD 2025-06 unverdicted novelty 6.0

    Acoustic scattering signals fed into fine-tuned self-supervised deep learning models classify hair type and moisture at nearly 90% accuracy as a non-invasive alternative to visual methods.

  2. SIGMA: Saliency-Guided Sparse Mask Attacks for Speech Emotion Recognition

    cs.SD 2026-06 unverdicted novelty 5.0

    SIGMA applies post-hoc XAI saliency maps to define reusable sparse masks for magnitude-bounded perturbations on self-supervised speech features, evaluated on IEMOCAP and TESS for competitive attack success with explan...

  3. Selective Capability Unlearning in End-to-End Spoken Language Understanding

    cs.CL 2026-06 unverdicted novelty 5.0

    Binding Subspace (BSU) attenuates intent-conditioned directions in autoregressive SLU models to reduce forced-prefix recoverability of unlearned intents.

  4. Joint Learning using Mixture-of-Expert-Based Representation for Speech Enhancement and Robust Emotion Recognition

    eess.AS 2025-09 unverdicted novelty 5.0

    Sparse MERIT uses frame-wise sparse mixture-of-experts with task-specific gating on self-supervised speech features to jointly optimize enhancement and emotion recognition, reporting gains over baselines on MSP-Podcas...

  5. OLIVE: View-Augmented Latent Prediction with Waveform Reconstruction for Speech SSL

    cs.CL 2026-06 unverdicted novelty 4.0

    OLIVE is a new self-supervised speech representation framework that unifies view-augmented masked latent prediction with waveform reconstruction under one objective.

  6. EMO-BOOST: Emotion-Augmented Audio-Visual Features for Improved Generalization in Deepfake Detection

    cs.AI 2026-05 unverdicted novelty 4.0

    Emo-Boost augments low-level deepfake detectors with intra- and inter-modal emotion consistency checks to raise cross-manipulation generalization AUC by 2.1% on FakeAVCeleb.