Pith. sign in

REVIEW 5 cited by

E-chat: Emotion-sensitive Spoken Dialogue System with Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.00475 v3 pith:3WQEUW5A submitted 2023-12-31 cs.SD eess.AS

Hongfei Xue , Yuhao Liang , Bingshen Mu , Shiliang Zhang , Mengzhe Chen , Qian Chen , Lei Xie This is my paper · ORCID
classification cs.SDeess.AS
keywords dialogueemotionalspeechspokenaudioe-chatemotion-sensitivemodel
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This study focuses on emotion-sensitive spoken dialogue in human-machine speech interaction. With the advancement of Large Language Models (LLMs), dialogue systems can handle multimodal data, including audio. Recent models have enhanced the understanding of complex audio signals through the integration of various audio events. However, they are unable to generate appropriate responses based on emotional speech. To address this, we introduce the Emotional chat Model (E-chat), a novel spoken dialogue system capable of comprehending and responding to emotions conveyed from speech. This model leverages an emotion embedding extracted by a speech encoder, combined with LLMs, enabling it to respond according to different emotional contexts. Additionally, we introduce the E-chat200 dataset, designed explicitly for emotion-sensitive spoken dialogue. In various evaluation metrics, E-chat consistently outperforms baseline model, demonstrating its potential in emotional comprehension and human-machine interaction.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dual Information Speech Language Models for Emotional Conversations

    cs.CL 2025-08 conditional novelty 7.0 of 10

    A dual-adapter design with equivalence replacement regularization lets frozen LLMs perceive both paralinguistic and linguistic information from speech for emotional conversation.

  2. OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios

    cs.CL 2025-01 conditional novelty 6.0 of 10

    ShareChatX and OmniChat show that large-scale synthetic spoken dialogue data improves multi-turn response quality and emotion prediction, setting a new state of the art on DailyTalk.

  3. X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

    cs.LG 2026-07 conditional novelty 5.0 of 10

    X3-OPD improves audio-grounded reasoning by training the audio student on its own rollouts with token-level teacher feedback, using a three-tier paired text-audio corpus.

  4. Leveraging Chain of Thought towards Empathetic Spoken Dialogue without Corresponding Question-Answering Data

    cs.CL 2025-01 conditional novelty 5.0 of 10

    Listen, Perceive, Express (LPE) uses two-stage ASR/SER training plus chain-of-thought prompting to let a frozen LLM generate empathetic responses from speech without question-answer fine-tuning data.

  5. MinMo: A Multimodal Large Language Model for Seamless Voice Interaction

    cs.CL 2025-01 conditional novelty 5.0 of 10

    A single aligned multimodal model, MinMo, achieves strong or state-of-the-art results on speech recognition, translation, emotion recognition, voice generation, and full-duplex dialogue while preserving the underlying...

Pith tools