AffectCodec applies block-diagonal projections in residual FSQ to explicitly allocate bits to emotion and acoustic subspaces, combined with emotion conditioning, yielding better emotion preservation at low bitrates with competitive acoustic quality.
The msp-podcast corpus
11 Pith papers cite this work. Polarity classification is still indexing.
11
Pith papers citing it
abstract
The availability of large, high-quality emotional speech databases is essential for advancing speech emotion recognition (SER) in real-world scenarios. However, many existing databases face limitations in size, emotional balance, and speaker diversity. This study describes the MSP-Podcast corpus, summarizing our ten-year effort. The corpus consists of over 400 hours of diverse audio samples from various audio-sharing websites, all of which have Common Licenses that permit the distribution of the corpus. We annotate the corpus with rich emotional labels, including primary (single dominant emotion) and secondary (multiple emotions perceived in the audio) emotional categories, as well as emotional attributes for valence, arousal, and dominance. At least five raters annotate these emotional labels. The corpus also has speaker identification for most samples, and human transcriptions of the lexical content of the sentences for the entire corpus. The data collection protocol includes a machine learning-driven pipeline for selecting emotionally diverse recordings, ensuring a balanced and varied representation of emotions across speakers and environments. The resulting database provides a comprehensive, high-quality resource, better suited for advancing SER systems in practical, real-world scenarios.
years
2026 11representative citing papers
AffectCodec is an emotion-guided neural speech codec that preserves emotional cues during quantization while maintaining semantic fidelity and prosodic naturalness.
A reasoning-guided ordinal SER framework conditions LALMs on paired speech, trains on semantic and GeMAPS-derived reasoning traces, and applies direct preference optimization to improve comparative emotion prediction with only 5% of conventional training data.
SHALA-LLM is a new RL framework for LLM alignment that learns directly from annotator label distributions and prioritizes ambiguous samples, reducing Jensen-Shannon Distance by up to 62.1% and raising F1 by up to 16.7% on NLI and ER benchmarks.
Emotion-embedding cosine similarity (EMO-SIM) fails to track emotion under speaker or wording changes and misaligns with human perception, so it is unreliable for evaluating expressive speech generation.
Generated multilingual ASR transcripts fused by cascaded cross-modal transformers raise audio sentiment accuracy, and the multimodal knowledge distills into a stronger audio-only WavLM student with no inference cost.
Sympatheia introduces a continuous affect-conditioned speech dialogue model and the Sympatheia-18k synthetic dataset, showing improved emotional appropriateness over baselines when speech cues are limited.
AgentSteerTTS proposes a multi-agent framework with adversarial disentanglement, dual-stream anchoring via acoustic prototypes, and fast-slow feedback to achieve intent-faithful expressive TTS for composite instructions.
Sticky factorial HDP-HMMs applied to multimodal valence-arousal trajectories identify interpretable persistent emotional regimes in conversations, outperforming Gaussian HMM baselines in consistency metrics and enabling context-augmented LLM responses.
Distribution-based supervision for 9-class SER improves alignment with human annotator vote distributions over hard-label training.
Speech from self-control tasks in remote learning shows perceptible emotional variations along valence, arousal, and dominance that can be automatically predicted.
citing papers explorer
-
AffectCodec: Emotion-Preserving Neural Speech Codec with Block-Diagonal Residual FSQ
AffectCodec applies block-diagonal projections in residual FSQ to explicitly allocate bits to emotion and acoustic subspaces, combined with emotion conditioning, yielding better emotion preservation at low bitrates with competitive acoustic quality.
-
AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
AffectCodec is an emotion-guided neural speech codec that preserves emotional cues during quantization while maintaining semantic fidelity and prosodic naturalness.
-
Comparative Reasoning: Making an Audio Language Model Better at Comparing Emotions
A reasoning-guided ordinal SER framework conditions LALMs on paired speech, trains on semantic and GeMAPS-derived reasoning traces, and applies direct preference optimization to improve comparative emotion prediction with only 5% of conventional training data.
-
SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs
SHALA-LLM is a new RL framework for LLM alignment that learns directly from annotator label distributions and prioritizes ambiguous samples, reducing Jensen-Shannon Distance by up to 62.1% and raising F1 by up to 16.7% on NLI and ER benchmarks.
-
The False Resonance: A Critical Examination of Emotion Embedding Similarity for Speech Generation Evaluation
Emotion-embedding cosine similarity (EMO-SIM) fails to track emotion under speaker or wording changes and misaligns with human perception, so it is unreliable for evaluating expressive speech generation.
-
Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcripts
Generated multilingual ASR transcripts fused by cascaded cross-modal transformers raise audio sentiment accuracy, and the multimodal knowledge distills into a stronger audio-only WavLM student with no inference cost.
-
Sympatheia: Emotionally Adaptive Voice Assistant with Continuous Affect Conditioning
Sympatheia introduces a continuous affect-conditioned speech dialogue model and the Sympatheia-18k synthetic dataset, showing improved emotional appropriateness over baselines when speech cues are limited.
-
AgentSteerTTS: A Multi-Agent Closed-Loop Framework for Composite-Instruction Text-to-Speech
AgentSteerTTS proposes a multi-agent framework with adversarial disentanglement, dual-stream anchoring via acoustic prototypes, and fast-slow feedback to achieve intent-faithful expressive TTS for composite instructions.
-
Multimodal Hidden Markov Models for Persistent Emotional State Tracking
Sticky factorial HDP-HMMs applied to multimodal valence-arousal trajectories identify interpretable persistent emotional regimes in conversations, outperforming Gaussian HMM baselines in consistency metrics and enabling context-augmented LLM responses.
-
Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition
Distribution-based supervision for 9-class SER improves alignment with human annotator vote distributions over hard-label training.
-
Toward using Speech to Sense Student Emotion in Remote Learning Environments
Speech from self-control tasks in remote learning shows perceptible emotional variations along valence, arousal, and dominance that can be automatically predicted.