EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models

Jianwu Dang; Jian Zong; Jingyu Zhang; Junyu Wang; Longbiao Wang; Meng Ge; Peiyuan Jiang; Shuqing Xie; Siyuan Zhang; Tianrui Wang

REVIEW 4 major objections 6 minor 45 references

EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A specialist judge trained on theory-grounded graded dialogues scores 83.8% on spoken emotional intelligence, near the 86% human baseline and far beyond GPT-4o-Audio (52.6%) and Gemini 2.5 Pro (54.0%).

desk verdict Theory-grounded EI benchmark and evaluator for spoken models, but the headline 83.8% number is only as credible as DeepSeek-R1's synthesized labels, so treat the near-human claim as contingent until an independent label audit lands. read the letter →

arxiv 2608.09189 v1 pith:YTF5HOXN submitted 2026-08-10 cs.CL

Junyu Wang , Siyuan Zhang , Peiyuan Jiang , Jian Zong , Jingyu Zhang , Tianrui Wang , Yuqin Lin , Zhenghui Chen

show 6 more authors

Shuqing Xie Ziyang Ma Meng Ge Xiaobao Wang Longbiao Wang Jianwu Dang

This is my paper · ORCID

classification cs.CL

keywords emotionalintelligencespokenlanguagemodelsevaluationbenchmarkrewardmodelfour-branchGroupRelativePolicyOptimizationparalinguisticperceptiondialoguealignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that emotional intelligence in spoken language models is a distinct, measurable, and trainable capability, not just acoustic perception, and that it can be anchored in the four-branch psychological model of emotional intelligence. To do this, the authors build EmoSBench, a ten-sub-task benchmark covering Perceiving, Understanding, Using, and Managing Emotion, and EmoDialogue, a bilingual dataset of over 70,000 dialogue instances in which candidate responses carry four-point emotional-intelligence grades. On that benchmark, their specialized evaluator EmoS reaches 83.8% average accuracy, against 86.0% for human raters, 52.6% for GPT-4o-Audio, and 54.0% for Gemini 2.5 Pro, and it keeps a 19.6-point lead over the best baseline on unconstrained YouTube conversations. A sympathetic reader would care because the result implies that frontier general-purpose speech models are far more emotionally deficient than earlier paralinguistic benchmarks made visible, and that a specialist judge, rather than a proprietary generalist, is the viable route to measuring and eventually aligning spoken emotional intelligence.

What carries the argument

The load-bearing structure is the four-branch emotional-intelligence hierarchy, operationalized into ten sub-task rubrics with explicit score-1-to-4 criteria; EmoDialogue instantiates the hierarchy as 72,588 bilingual dialogue instances whose user inputs, acoustic descriptions, and four graded candidate responses are generated by DeepSeek-R1 and rendered into speech with controllable emotion, rate, and volume. Training runs in two stages: supervised fine-tuning on the graded pairs, then GRPO with two shaped rewards. The Steep Exponential Accuracy Reward, $R_a(\Delta)=\alpha e^{-\lambda\Delta}$ with $\alpha=\lambda=2$ and $R_a=0$ when $\Delta\geq 3$, collapses reward sharply as the absolute score error $\Delta=|\hat{y}-y_{gt}|$ grows, while a Rationale Fidelity Reward is weighted by $\omega(\Delta)=1.0$ for exact hits and $0.1$ otherwise, using Qwen3-8B as an automated critic of the chain-of-thought rationale. Together these rewards enforce exact ordinal scoring and keep the reasoning honest, and the resulting model is the object whose behavior the benchmark reports.

What would settle it

Collect a new set of human-authored spoken dialogues with independent expert ratings on the same four-point scale and run EmoS on them; if its accuracy on this externally labeled data collapses toward the 52-54% level of GPT-4o-Audio and Gemini 2.5 Pro instead of the 83.8% seen on EmoSBench, the near-human performance is agreement with the synthetic labeler, not measurement of emotional intelligence. A cheaper probe is to feed EmoS the transcripts only: if its scores barely change when the audio is removed, the benchmark is being solved largely through text, contradicting the acoustic-perception claims.

Watch

Extended reading notes

Core claim

EmoSBench is the first evaluation of spoken language models organized around the full four-branch hierarchy of emotional intelligence, implemented as ten graded sub-tasks spanning basic acoustic perception, implicit attitude analysis, emotional state tracking, emotional causation analysis, emotion-cognition matching, emotion-driven plan adjustment, social strategy execution, proactive mitigation and buffering, conflict resolution and de-escalation, and value-aligned safety response. Measured by exact four-point match, human raters score 86.0%, while the best existing models fall far short: Qwen3-Omni at 50.1%, GPT-4o-Audio at 52.6%, and Gemini 2.5 Pro at 54.0%. EmoS, a Qwen2.5-Omni-7B model fine-tuned on EmoDialogue's graded response pairs with supervised fine-tuning followed by Group Relative Policy Optimization under a steep exponential accuracy reward and a rationale fidelity reward, reaches 83.8% on the benchmark and 62.9% on a real-world YouTube evaluation set, outperforming the strongest baseline there by 19.6 points. The paper takes these numbers to show that the prevailing practice of using proprietary general-purpose models as emotional-intelligence judges sets an artificially low ceiling, and that theory-grounded synthetic supervision can produce a near-human evaluator.

Load-bearing premise

The ground-truth grades in EmoDialogue, the labels EmoS is trained and tested against, are produced by DeepSeek-R1 following the authors' rubrics, and the results stand or fall on the assumption that those synthesized ordinal rankings measure the psychological construct of emotional intelligence rather than the generator's stylistic preferences.

Editorial extensions

If this is right

EmoS can be deployed as a specialist judge, replacing proprietary general-purpose models whose low scores currently set an artificially low and biased performance ceiling.
Because EmoS scores with explicit rationales and four-point ordinal precision, it can serve as a reward model for aligning spoken language models toward emotionally intelligent dialogue generation.
EmoSBench provides a reusable, theory-grounded protocol, four branches, ten sub-tasks, exact-match scoring, against which future spoken dialogue systems can be measured.
The roughly 26-to-30-point gap between frontier SLMs and the human baseline shows that current models lack most of the cognitive machinery of emotional intelligence, not merely acoustic perception.
On unconstrained YouTube dialogue, EmoS keeps a 19.6-point lead over the best baseline, indicating that the specialized training transfers beyond synthesized speech.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the training signal and much of the test signal flow from the same synthesis pipeline, the headline 83.8% is best read as agreement with the generator's grading criteria; the human-curated test set and real-world set narrow but do not close that gap.
A direct test of this would be to have EmoS and GPT-4o-Audio grade an independent corpus of human-authored dialogues carrying expert emotional-intelligence ratings; if EmoS's advantage shrinks substantially, the benchmark advantage partly reflects stylistic similarity to the synthetic labeler rather than the psychological construct.
EmoS's branch-level scores could be reused as diagnostics for speech synthesizers: a system whose outputs score poorly on the Perceiving and Managing branches is likely compressing acoustic affect, something current text-and-intelligibility metrics do not directly measure.
The SEAR reward's hard cutoff at $\Delta \geq 3$ makes EmoS a sharp exact-match judge, but it also means the model never learns graded credit for badly wrong answers, a deliberate fit for evaluation yet a point to revisit if EmoS is reused for fine-grained reward shaping in alignment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful core here is real: EmoSBench is the first spoken-language benchmark to operationalize all four branches of the Mayer-Salovey EI model, the ten sub-tasks are thoughtfully designed, and the ablations are internally consistent. The paper also ships a large bilingual training set, a two-stage SFT+GRPO recipe with a sensible steep exponential accuracy reward, and a real-world YouTube test set for generalization. For a community that has been measuring acoustic emotion classification and calling it EI, that is a genuine step forward, and the gap it reports between generalist SLMs and the specialized evaluator is consistent with the hypothesis that generalists cap out on this kind of fine-grained ordinal judgment.

Now the soft spots, in proportion. The load-bearing issue is label validity. Section 3.2 and Appendix C make clear that DeepSeek-R1 generates the four candidate responses and their hierarchical scores for both EmoDialogue and EmoSBench; the five expert annotators select samples on discriminability, diversity, and agreement, but nothing says they independently rescored against the rubrics. The stress-test note has it right: the 83.8% accuracy, which is the paper's headline and central claim, may substantially measure agreement with DeepSeek-R1's stylistic preferences rather than an external construct of EI. This is not fatal to the benchmark's usefulness, but it is a real circularity burden, and it should be flagged loudly in any review.

Two other gaps, both fixable and both probably minor-to-moderate. First, the real-world set has human labels and a 3-of-4 consensus rule, but no human baseline is reported on it, so the approaching-human-level claim rests only on the synthetic benchmark. Second, there are no error bars, no significance tests, and no IAA statistics anywhere, even though the rubrics are subtle enough (Score 3 vs. 4 often hinges on mechanical tone) that human agreement is not a given. Artifacts also are not released, which makes independent verification impossible right now.

Who this is for: anyone building spoken dialogue evaluation or reward models will want to read it. The citation pattern is fine; the four-branch theory is properly credited, and the comparison table against prior spoken benchmarks is honest. I would cite this paper, but I would not yet cite the 83.8% number as a fact. It deserves a serious referee: the framework is timely, the artifacts are potentially reusable, and the flaws are concrete and addressable.

Referee Report

4 major / 6 minor

Summary. This paper introduces EmoSBench, a benchmark for evaluating emotional intelligence (EI) in spoken language models based on the Mayer-Salovey four-branch model, with ten sub-tasks across Perceiving, Understanding, Using, and Managing emotion. The authors also construct EmoDialogue, a bilingual corpus of about 72k user-response quadruples with ordinal EI scores synthesized by DeepSeek-R1, and train EmoS, a Qwen2.5-Omni-based evaluator using SFT followed by GRPO with a steep exponential accuracy reward and a rationale fidelity reward. On EmoSBench, EmoS reaches 83.8% exact-match accuracy versus 86.0% for human raters and 52.6%/54.0% for GPT-4o-Audio/Gemini 2.5 Pro; on a 531-sample YouTube set, it reaches 62.9%. The paper's central claim is that EmoS provides a valid, human-approaching evaluation of spoken EI and generalizes to real-world interactions.

Significance. If the label-validity concern is resolved, the paper would be a meaningful contribution. It is one of the first benchmarks to operationalize the four-branch EI model for spoken dialogue, with unusually detailed rubrics (Appendix B) and generation prompts (Appendix C) that support replication. The ablations in Table 5 are internally consistent and show a plausible contribution from each reward component, and the real-world test set, while small, is a valuable stress test. Reporting exact-match accuracy with per-sub-task breakdowns makes the results easy to inspect. However, the current evidence does not establish that the benchmark measures EI rather than agreement with the label generator, so the significance is conditional on external validation of the synthesized labels.

major comments (4)

[Section 3.2, Appendix C, Table 4] Section 3.2 (Data Synthesis and Annotation) states that DeepSeek-R1 generates four hierarchically graded candidate responses for every user input, and the same pipeline produces EmoDialogue training pairs and EmoSBench test items. The subsequent 'Benchmark Test Set Construction' paragraph says five expert annotators select samples based on score discriminability, topic diversity, and high inter-annotator agreement, but it does not say that these annotators independently assigned or verified the ordinal scores. The human baseline in Table 4 is reported only as an average over six graduate students, with no protocol, no per-item agreement, and no clarification of whether the students scored the same DeepSeek-labeled items. Because EmoS is trained and evaluated on labels produced by the same generator family, the 83.8% accuracy may measure agreement with DeepSeek-R1's output distribution rather than emotional intelligence. The authors should add an independent human rescoring of a representative sample (ideally the full EmoSBench) using the Appendix B rubrics, report inter-annotator agreement (e.g., quadratic-weighted kappa or Krippendorff's alpha), and compare model accuracy against human majority labels.
[Tables 4, 5, 6] Tables 4-6 report no confidence intervals or significance tests. With roughly 4,000 test items (about 400 per sub-task) and 531 real-world items, the claimed gaps (83.8 vs. 86.0 human; 62.9 vs. 43.3 Gemini on the real-world set) may be within sampling error, especially after multiple comparisons across ten sub-tasks. The authors should provide bootstrap confidence intervals or standard errors for all headline averages and for the per-sub-task cells that support specific claims (e.g., EmoS BAP 90.8 vs. human 86.0; EmoS EPA 87.3 vs. human 82.0).
[Section 3.2, Table 6] Real-world evaluation set: no human baseline is reported for the 531 YouTube samples. The conclusion that EmoS 'sustains robust real-world generalization' is therefore only a relative claim against other SLMs; the paper cannot support the stronger interpretation that 62.9% is near human-level or that the drop from 83.8% to 62.9% is the expected cost of naturalistic speech. The authors should add a human baseline on the same 531 items, report the inter-annotator agreement of the four annotators (beyond the 3-of-4 consensus filter), and state the per-sub-task sample sizes.
[Section 3.3, Eq. (2)-(3)] The Rationale Fidelity Reward relies on Qwen3-8B as an automated critic of whether EmoS's rationale is faithful to psychological reasoning, but the paper provides no validation of this critic against human judgments and no analysis of whether RFR actually improves reasoning quality rather than simply shaping text style. Because 'valid reasoning' is one of the two stated purposes of the reward design, the authors should report agreement between Qwen3-8B and human raters on a sample of rationales, or at least show that RFR leads to rationales that human annotators judge as more faithful.

minor comments (6)

[Table 6] The Qwen3-Omni row is printed as '35.363.639.4' and the EmoS row as '64.757.662.9'; these should be '35.3 63.6 39.4' and '64.7 57.6 62.9', respectively.
[Appendix A] The large English/Chinese performance gap for EmoS (80.6 vs. 86.9 average) is not discussed; the authors should comment on whether this reflects language-specific dataset imbalance, prompt translation, or acoustic TTS differences.
[General] The paper does not include a data/code availability statement; a release statement would help reproducibility, particularly since benchmark validity depends on the exact synthesis prompts.
[Section 3.2] The phrase 'we predominantly employ a synthesis strategy' is vague because all EmoDialogue samples appear to be synthesized by DeepSeek-R1 and Doubao TTS; please clarify whether any non-synthetic samples are included.
[Eq. (1)] The condition \Delta<3 in a 1-to-4 scale is equivalent to \Delta \le 2; stating it as \Delta \le 2 would be clearer.
[Section 4.1] Exact-match accuracy is the sole metric; given the ordinal nature of the labels, adding a secondary metric such as macro-averaged rank correlation or adjacent accuracy would provide a more complete picture of near-miss behavior.

Circularity Check

1 steps flagged · score 6.0 of 10

EmoSBench and EmoDialogue share DeepSeek-R1-generated labels, so the 83.8% score largely measures agreement with the generator's grading rather than an independent EI ground truth.

self definitional [Section 3.2 (EmoDialogue: Data Synthesis and Annotation; Benchmark Test Set Construction) and Section 4.1 (Datasets and Metrics)]
"We leverage the reasoning capabilities of the DeepSeek-R1 [18] LLM to simulate realistic user-model interactions. For every user input, the system generates four distinct candidate responses, hierarchically graded on a four-point ordinal scale to reflect varying levels of EI. ... while reserving 4,000 samples for the independent test set."

The ground-truth ordinal scores in EmoSBench are the four-point rankings that DeepSeek-R1 itself generated while synthesizing EmoDialogue. Section 4.1 confirms that the 4,000-sample test set is a held-out split of the same EmoDialogue dataset on which EmoS is trained via SFT and GRPO. Therefore EmoS is both fitted and evaluated on labels produced by the same generator pipeline; the reported 83.8% accuracy is agreement with DeepSeek-R1's grading distribution, not with an independently established psychological measure of EI. The word 'independent' refers only to disjointness from the training split, not to an independent label source.

full rationale

The central evaluation loop is synthetic-to-synthetic. EmoDialogue is built by having DeepSeek-R1 generate user inputs, four candidate responses, and the ordinal EI score for each response. EmoSBench is curated from EmoDialogue by expert selection, and Section 4.1 explicitly reserves 4,000 EmoDialogue samples as the test set while using the remaining ~70,000 samples for SFT and GRPO. Thus EmoS is trained and tested on labels produced by the same generator. The 83.8% EmoSBench accuracy therefore measures how well EmoS reproduces DeepSeek-R1's grading distribution, not an externally anchored EI construct. The human baseline is a potential external check, but the paper does not document that human raters independently assigned scores or audited the synthetic labels, and no inter-annotator agreement statistics are given. On the real-world YouTube set, where labels are human-consensus, no human baseline is reported, so the 'approaching human-level performance' claim is supported only on the synthetic benchmark. This is partial circularity rather than a fully forced result: the four-branch theory, the detailed rubrics, and the human-consensus real-world set provide some independent grounding, but the headline benchmark result is substantially self-referential. Score 6 reflects that the central prediction is partly reduction-to-construction while real external checks remain incomplete.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the validity of LLM-generated EI labels and on a psychological theory. The paper contributes substantial empirical artifacts (benchmark, dataset, model) but no new theoretical entity. The main burden is that the training and evaluation labels come from the same generative pipeline, so the empirical result is not yet independently grounded outside the authors' pipeline.

free parameters (5)

alpha (SEAR scaling factor) = 2.0
Equation (1); chosen by hand to scale the exponential accuracy reward. No sensitivity analysis is reported.
lambda (SEAR distinctiveness coefficient) = 2.0
Equation (1); chosen by hand to set how fast the reward decays with score error.
omega residual (RFR weight for incorrect predictions) = 0.1
Equation (2); chosen by hand to retain 10% of the rationale reward when the predicted score is wrong.
GRPO group size G = 8
Section 4.1; number of sampled outputs per query used to estimate the group-relative baseline in Equation (4).
KL penalty beta = 0.02
Section 4.1; coefficient in the GRPO objective, Equation (5).

assumptions (5)

domain assumption Mayer-Salovey four-branch EI model is a valid operationalization of EI and maps cleanly onto the ten sub-tasks.
Sections 1 and 3.1 define EmoSBench around this theory; the benchmark's construct validity depends on this mapping.
ad hoc to paper DeepSeek-R1-generated four-response rankings are valid ground-truth labels for EI quality.
Section 3.2 synthesizes EmoDialogue and EmoSBench labels with DeepSeek-R1; no independent human rescoring of the full set is reported.
domain assumption Doubao TTS audio faithfully conveys the specified emotion and paralinguistic cues.
Section 3.2 uses emotional TTS plus injected non-verbal cues; quality control is WER/CER filtering and manual review, not perceptual validation of emotion.
ad hoc to paper Qwen3-8B can judge whether an EmoS rationale is faithful to the psychological reasoning.
Section 3.3 uses Qwen3-8B as the automated critic for RFR; no evidence or calibration is provided for this judgment.
domain assumption Six graduate students provide a fair human baseline.
Section 4.1 reports human results averaged from six graduate students; no inter-annotator agreement or annotation protocol is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models." pith.science (2026). https://pith.science/paper/YTF5HOXN

@misc{pith2026260809189,
  author       = {Pith},
  title        = {Pith review of: EmoS: A Theory-Grounded Framework for Evaluating and Aligning Emotional Intelligence in Spoken Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YTF5HOXN}},
  note         = {Machine review of arXiv:2608.09189}
}

read the original abstract

Despite significant advances in instruction-following and auditory comprehension, the evaluation of Emotional Intelligence (EI) in Spoken Language Models (SLMs) remains confined to rudimentary paralinguistic perception, lacking a systematic, theory-driven cognitive framework. We introduce EmoSBench, the first comprehensive EI evaluation benchmark for SLMs constructed upon the four-branch theoretical model, covering Perceiving, Understanding, Using, and Managing Emotion across ten sub-tasks. Preliminary assessments on EmoSBench reveal a substantial gap: even leading proprietary models like GPT-4o-Audio achieve only 52.6%, significantly trailing human baselines. To bridge this gap, we develop EmoS, a specialized evaluator model optimized via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO). To facilitate its effective training, we curate EmoDialogue, a bilingual dataset providing necessary fine-grained supervision through response pairs with rigorously defined EI gradations. Concurrently, we introduce a reward mechanism integrating a Steep Exponential Accuracy Reward (SEAR) and a Rationale Fidelity Reward (RFR) to enforce precise ordinal scoring and valid reasoning. Experiments demonstrate that EmoS reaches 83.8% accuracy, approaching human-level performance. Furthermore, evaluations on authentic, unconstrained spoken interactions validate its robust real-world generalization, establishing a foundational framework for advancing emotionally intelligent dialogue systems.

Figures

Figures reproduced from arXiv: 2608.09189 by the authors.

**Figure 2.** Illustration of the EmoS RL training pipeline via Group Relative Policy Optimization (GRPO), incorporating the Steep [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 19 canonical work pages

[1]

Junyi Ao, Yuancheng Wang, Xiaohai Tian, Dekun Chen, Jun Zhang, Lu Lu, Yuxuan Wang, Haizhou Li, and Zhizheng Wu. 2024. Sd-eval: A benchmark dataset for spoken dialogue understanding beyond words.Advances in Neural Information Processing Systems37 (2024), 56898–56918

work page 2024
[2]

Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468(2025)

arXiv 2025
[3]

Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, et al . 2025. Slam-omni: Timbre- controllable voice interaction system with single-stage training. InFindings of the Association for Computational Linguistics: ACL 2025. 2262–2282

work page 2025
[4]

Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T Tan, and Haizhou Li. 2024. Voicebench: Benchmarking llm-based voice assistants.arXiv preprint arXiv:2410.17196(2024)

arXiv 2024
[5]

Xize Cheng, Ruofan Hu, Xiaoda Yang, Jingyu Lu, Dongjie Fu, Zehan Wang, Shengpeng Ji, Rongjie Huang, Boyang Zhang, Tao Jin, et al. 2025. Voxdialogue: Can spoken dialogue systems understand information beyond words?. InThe Thirteenth International Conference on Learning Representations

work page 2025
[6]

Cary Cherniss. 2010. Emotional intelligence: Toward clarification of a concept. Industrial and organizational psychology3, 2 (2010), 110–126

work page 2010
[7]

Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al. 2024. Qwen2-audio technical report.arXiv preprint arXiv:2407.10759(2024)

arXiv 2024
[8]

Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

arXiv 2025

Show all 45 references

[9]

Stéphane Côté. 2014. Emotional intelligence in organizations.Annu. Rev. Organ. Psychol. Organ. Behav.1, 1 (2014), 459–488

2014
[10]

Wenqian Cui, Xiaoqi Jiao, Ziqiao Meng, and Irwin King. 2025. Voxeval: Bench- marking the knowledge understanding capabilities of end-to-end spoken lan- guage models.arXiv preprint arXiv:2501.04962(2025)

2025 arXiv
[11]

Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. Moshi: a speech-text foundation model for real-time dialogue.arXiv preprint arXiv:2410.00037(2024)

2024 arXiv
[12]

Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, et al . 2025. Kimi-audio technical report. arXiv preprint arXiv:2504.18425(2025)

2025 arXiv
[13]

Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. 2025. LLaMA-Omni: Seamless Speech Interaction with Large Language Models. InThe Thirteenth International Conference on Learning Representations

2025
[14]

Qingkai Fang, Yan Zhou, Shoutao Guo, Shaolei Zhang, and Yang Feng. 2025. Llama-omni2: Llm-based real-time spoken chatbot with autoregressive streaming speech synthesis.arXiv preprint arXiv:2505.02625(2025)

2025 arXiv
[15]

Kuofeng Gao, Shu-Tao Xia, Ke Xu, Philip Torr, and Jindong Gu. 2025. Benchmark- ing open-ended audio dialogue understanding for large audio-language models. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 4763–4784

2025
[16]

Xuelong Geng, Qijie Shao, Hongfei Xue, Shuiyuan Wang, Hanke Xie, Zhao Guo, Yi Zhao, Guojian Li, Wenjie Tian, Chengyou Wang, et al . 2025. Osum-echat: Enhancing end-to-end empathetic spoken chatbot via understanding-driven spoken dialogue.arXiv preprint arXiv:2508.09600(2025)

2025 arXiv
[17]

Sreyan Ghosh, Arushi Goel, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang-gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, et al. 2025. Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models. InThe Thirty-ninth An...

2025
[18]

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. 2025. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948(2025)

2025 arXiv
[19]

He Hu, Yucheng Zhou, Lianzhong You, Hongbo Xu, Qianning Wang, Zheng Lian, Fei Richard Yu, Fei Ma, and Laizhong Cui. 2025. Emobench-m: Benchmarking emotional intelligence for multimodal large language models.arXiv preprint arXiv:2502.04424(2025)

2025 arXiv
[20]

Chien-yu Huang, Ke-Han Lu, Shih-Heng Wang, Chi-Yuan Hsiao, Chun-Yi Kuan, Haibin Wu, Siddhant Arora, Kai-Wei Chang, Jiatong Shi, Yifan Peng, et al
[21]

Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, et al . 2025. Wavre- ward: Spoken dialogue models with generalist reward evaluators.arXiv preprint arXiv:2505.09558(2025)

2025
[22]

Yadong Li, Jun Liu, Tao Zhang, Song Chen, Tianpeng Li, Zehuan Li, Lijun Liu, Lingfeng Ming, Guosheng Dong, Da Pan, et al. 2025. Baichuan-omni-1.5 technical report.arXiv preprint arXiv:2501.15368(2025)

2025
[23]

Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, et al . 2025. MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix. InThe Thirty-ninth Annual Conference on Neural Informa...

2025
[24]

John D Mayer, Richard D Roberts, and Sigal G Barsade. 2008. Human abilities: Emotional intelligence.Annu. Rev. Psychol.59, 1 (2008), 507–536

2008
[25]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust speech recognition via large-scale weak supervision. InInternational conference on machine learning. PMLR, 28492–28518

2023
[26]

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems36 (2023), 53728–53741

2023
[27]

Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, and Minlie Huang. 2024. Emobench: Evalu- ating the emotional intelligence of large language models. InProceedings of the 62nd Annual Meeting of the Association for Com...

2024
[28]

S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. 2025. MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark. MM ’26, November 10–14, 2026, Rio de Janeiro, Brazil Juny...

2025
[29]

Peter Salovey and Daisy Grewal. 2005. The science of emotional intelligence. Current directions in psychological science14, 6 (2005), 281–285

2005
[30]

Peter Salovey and John D Mayer. 1990. Emotional intelligence.Imagination, cognition and personality9, 3 (1990), 185–211

1990
[31]

Bin Wang, Xunlong Zou, Geyu Lin, Shuo Sun, Zhuohan Liu, Wenyu Zhang, Zhengyuan Liu, AiTi Aw, and Nancy Chen. 2025. Audiobench: A universal bench- mark for audio large language models. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Associatio...

2025
[32]

Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025. MMSU: A Massive Multi-task Spoken Language Understanding and Reasoning Benchmark.arXiv preprint arXiv:2506.04779(2025)

2025 arXiv
[33]

Xiong Wang, Yangze Li, Chaoyou Fu, Yunhang Shen, Lei Xie, Ke Li, Xing Sun, and Long Ma. 2024. Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen llm.arXiv preprint arXiv:2411.00774(2024)

2024 arXiv
[34]

Lin, Andy T

Shu wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mo...

2021
[35]

Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al. 2025. Step-audio 2 technical report. arXiv preprint arXiv:2507.16632(2025)

2025 arXiv
[36]

Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chun- yan Miao. 2025. Audio-reasoner: Improving reasoning capability in large audio language models.arXiv preprint arXiv:2503.02318(2025)

2025
[37]

Zhifei Xie and Changqiao Wu. 2024. Mini-omni: Language models can hear, talk while thinking in streaming.arXiv preprint arXiv:2408.16725(2024)

2024 arXiv
[38]

Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, et al. 2025. Qwen2.5-omni technical report. arXiv preprint arXiv:2503.20215(2025)

2025 arXiv
[39]

Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. 2025. Qwen3-Omni Technical Report. arXiv preprint arXiv:2509.17765(2025)

2025 arXiv
[40]

Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, and Xie Chen. 2025. Uro-bench: A comprehensive benchmark for end-to-end spoken dialogue models.arXiv preprint arXiv:2502.17810(2025)

2025 arXiv
[41]

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. 2025. Qwen3 technical report.arXiv preprint arXiv:2505.09388(2025)

2025 arXiv
[42]

Qian Yang, Jin Xu, Wenrui Liu, Yunfei Chu, Ziyue Jiang, Xiaohuan Zhou, Yichong Leng, Yuanjun Lv, Zhou Zhao, Chang Zhou, et al. 2024. Air-bench: Benchmarking large audio-language models via generative comprehension. InProceedings of the 62nd Annual Meeting of the Association fo...

2024
[43]

Shu-wen Yang, Ming Tu, Andy T Liu, Xinghua Qu, Hung-yi Lee, Lu Lu, Yuxuan Wang, and Yonghui Wu. 2025. ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interaction.arXiv preprint arXiv:2511.08723(2025)

2025
[44]

modality misalignment,

Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang, Shengmin Jiang, Lei Zhao, Yuxiao Dong, and Jie Tang. 2024. Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot.arXiv preprint arXiv:2412.02612(2024). EmoS: A Theory-Grounded Framework for Evaluating and ...

2024 arXiv
[2024]

InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Dynamic-superb: Towards a dynamic, collaborative, and comprehensive instruction-tuning benchmark for speech. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 12136– 12140

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Junyi Ao, Yuancheng Wang, Xiaohai Tian, Dekun Chen, Jun Zhang, Lu Lu, Yuxuan Wang, Haizhou Li, and Zhizheng Wu. 2024. Sd-eval: A benchmark dataset for spoken dialogue understanding beyond words.Advances in Neural Information Processing Systems37 (2024), 56898–56918

work page 2024

[2] [2]

Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468(2025)

arXiv 2025

[3] [3]

Wenxi Chen, Ziyang Ma, Ruiqi Yan, Yuzhe Liang, Xiquan Li, Ruiyang Xu, Zhikang Niu, Yanqiao Zhu, Yifan Yang, Zhanxun Liu, et al . 2025. Slam-omni: Timbre- controllable voice interaction system with single-stage training. InFindings of the Association for Computational Linguistics: ACL 2025. 2262–2282

work page 2025

[4] [4]

Yiming Chen, Xianghu Yue, Chen Zhang, Xiaoxue Gao, Robby T Tan, and Haizhou Li. 2024. Voicebench: Benchmarking llm-based voice assistants.arXiv preprint arXiv:2410.17196(2024)

arXiv 2024

[5] [5]

Xize Cheng, Ruofan Hu, Xiaoda Yang, Jingyu Lu, Dongjie Fu, Zehan Wang, Shengpeng Ji, Rongjie Huang, Boyang Zhang, Tao Jin, et al. 2025. Voxdialogue: Can spoken dialogue systems understand information beyond words?. InThe Thirteenth International Conference on Learning Representations

work page 2025

[6] [6]

Cary Cherniss. 2010. Emotional intelligence: Toward clarification of a concept. Industrial and organizational psychology3, 2 (2010), 110–126

work page 2010

[7] [7]

Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al. 2024. Qwen2-audio technical report.arXiv preprint arXiv:2407.10759(2024)

arXiv 2024

[8] [8]

Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

arXiv 2025