Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition

Akifumi Yoshimoto; Cheng-Hung Hu; Tomoki Toda; Yusuke Yasuda

REVIEW 4 major objections 5 minor 36 references

Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training speech-quality and emotion models on pairwise comparisons between same-listener utterances outperforms averaging listener scores into a mean listener.

desk verdict A clean, systematic ablation of comparison-based training for listener scoring scales, with a real but fixable identifiability problem in the absolute-score claims. read the letter →

arxiv 2507.13626 v2 pith:KIBRMK2P submitted 2025-07-18 eess.AS cs.SD

Cheng-Hung Hu , Yusuke Yasuda , Akifumi Yoshimoto , Tomoki Toda This is my paper

classification eess.AScs.SD

keywords speechqualityassessmentcomparisonlearningunifiedlistenerscoringscalecontinuousemotionrecognitionmeanapproachembeddingarousalpredictionvalence

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Listener ratings for speech quality and emotion are biased by each listener's personal scoring scale, and prior systems either averaged scores into a 'mean listener' or modeled every listener separately. This paper proposes training on comparison scores—whether utterance A scored higher than B for the same listener—so that the model learns relationships on a unified listener scoring scale without needing listener embeddings. The authors show that this comparison-learning framework improves both tasks: on speech quality assessment it beats the UTMOS/UTP baselines, and on continuous emotion recognition it beats the direct-score baseline on arousal and valence. The paper also finds that whether the mean-listener trick helps depends on how much the same listeners reappear across the dataset, which explains why it helps on IEMOCAP but hurts on BVCC.

What carries the argument

The machinery is the comparison training objective used inside the UTP framework (the authors' earlier preference-based training framework). For each pair of utterances assessed by the same listener, the model predicts attribute scores $sc_1, sc_2$ and maps them through $\alpha(sc_1,sc_2)=2\cdot\sigma(sc_1-sc_2)-1$, where $\sigma$ is the logistic sigmoid; the target is the sign of the ground-truth difference, $\mathrm{sgn}(sc^{gt}_1-sc^{gt}_2)$, and the loss is the mean squared error between the two. Because the target encodes only the ordering, no absolute scale is imposed, which is what lets same-listener pairs from different listeners be pooled. Pair generation is the other load-bearing piece: it uses listener identity to form pairs even when listener embeddings are absent from the network, and the mean-listener variant treats the averaged ratings as an extra listener.

What would settle it

On the IEMOCAP test folds, regress the detached comparison-trained model's raw arousal and valence predictions against the human-annotated values and compute the slope and the mean absolute error; a slope far from 1 or a large mean absolute error together with a high SRCC would show that only the ranking, not the unified absolute scale, was learned.

Watch

Extended reading notes

Core claim

The central claim is that comparison scores, because they encode only ordinal relationships, can be pooled across listeners to form a unified listener scoring scale, so a model trained on same-listener utterance pairs with no listener embedding learns to predict better than models that average scores or embed listeners. Concretely, on the BVCC speech quality benchmark the UTP model without mean listener and without listener embedding reaches system-level SRCC 0.9420 and LCC 0.9376, the best among all UTMOS and UTP configurations. On IEMOCAP continuous emotion recognition, the CL model with text and mean listener reaches arousal SRCC 0.7703 and valence SRCC 0.8004, again the best configuration, and every CL configuration beats its DAS counterpart. The paper also shows the mean-listener trick is dataset-dependent: it helps in IEMOCAP, whose 29 listener sets overlap substantially, and degrades BVCC, whose 36 listener sets are disjoint, which the authors attribute to the differing comparability of the resulting comparison pairs.

Load-bearing premise

The argument depends on a model trained purely on pairwise comparison scores—which by construction ignore the absolute scale—being detachable and usable to predict absolute arousal and valence values; if the network has not actually recovered the human scoring scale, the emotion-recognition results would show only rank agreement, not accurate absolute prediction.

Editorial extensions

If this is right

Removing listener embeddings simplifies deployment: the same trained network can score any utterance without knowing which listener would hear it.
Ordinal comparison targets let systems be ranked without ever averaging Likert-scale ratings, sidestepping the distortion the paper attributes to treating ordinal data as numeric.
The framework transfers across tasks: the same comparison loss and pair-generation logic works for speech quality and for both arousal and valence, giving a single recipe for two domains.
Whether to add mean-listener scores becomes predictable from data structure: it helps on IEMOCAP, where listener sets overlap heavily, and hurts on BVCC, where the 36 listener sets are disjoint.
Text input matters mainly for valence: both DAS and CL models improve in valence with Whisper/BERT features, while arousal is unaffected or slightly worse with text in the DAS model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the comparison loss is invariant to any monotone transformation of the scores, the strongest version of the paper's claim—that a single unified absolute scale is recovered—is not guaranteed by the training objective; a calibration or distribution-matching check on the detached DAS head would settle it.
If comparison scores really are dataset-agnostic, the authors' proposed future work of training one model across multiple corpora becomes directly testable: pairwise accuracy on held-out language or domain pairs would predict whether a global unified scale exists.
A practical diagnostic emerges from the paper's analysis: the overlap statistics of listener sets can be used a priori to decide whether to add mean-listener scores before training any model.
In the SQA comparison-score evaluation mode, the threshold-selection and aggregation steps are themselves design choices; an ablation that replaces Differential Count with a direct DAS readout of the same network would reveal whether the gains come from the unified-scale training or from the aggregation scheme.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper extends the authors' own UTP comparison-learning framework to continuous speech emotion recognition with text input, and systematically ablates mean-listener augmentation and listener embeddings under an explicit unified-scale framing. That ablation is the actual contribution, and it's done thoroughly on public BVCC and IEMOCAP datasets. The finding that comparison training without listener embeddings helps SQA, and that the recipe transfers to CSER, is plausible and worth taking seriously. The authors also give a thoughtful, data-driven explanation for why mean-listener augmentation helps in IEMOCAP but not BVCC (listener-set overlap), which is more than most papers bother to do.

The soft spots are real. The comparison loss is MSE(2*sigmoid(sc1-sc2)-1, sign(sc_gt1-sc_gt2)). The target is a sign, so the training signal is essentially ordinal. Any strictly monotonic reparameterization of the predicted scores preserves the sign pattern, so the absolute scale used at inference is not determined by the loss. The paper reports LCC values for CL models in DAS evaluation mode (e.g., valence LCC 0.8023) and claims the model learns a unified listener scoring scale, but there is no analysis showing the network's output scale is aligned with the target DAS scale. SRCC is fine; LCC is not. This is a load-bearing gap because the "unified listener scoring scale" language implies absolute-score prediction, not just rank alignment.

The second soft spot is the SQA headline. The best UTP configuration (no mean listener, no listener embedding) beats the mean-listener UTP by 0.001 SRCC (0.9420 vs 0.9410) with equal LCC, and though they ran 20 trials, no variance or significance test is reported. That difference could easily be noise. The CSER comparison results are more substantial, but they also lack error bars.

None of this is fatal. The core recipe—train on within-listener pairs, drop listener embeddings—is sensible and the experiments are broad enough to deserve a serious referee. The authors should be asked to add an identifiability analysis (e.g., show the output scale is calibrated to DAS, or explicitly claim only rank-based prediction), report variance or significance tests for the key comparisons, and be more careful about the word "best" when the margin is 0.001. I'd send it to review, and if the authors fix the identifiability issue, I'd probably cite it. As is, it's a useful incremental extension but the absolute-score claims need support.

Referee Report

4 major / 5 minor

Summary. This paper proposes to train speech quality assessment (SQA) and continuous speech emotion recognition (CSER) models using within-listener pairwise comparison scores rather than direct assessment scores (DAS), with the goal of learning a single unified listener scoring scale. For SQA, the comparison-based UTP model is trained without mean-listener augmentation and without listener embeddings, and evaluated by aggregating pairwise comparisons into system-level scores. For CSER, the comparison-learning framework is extended to predict arousal and valence, with optional text input, mean-listener augmentation, and listener embeddings. Experiments on BVCC and IEMOCAP report that comparison-based training outperforms DAS-based training in all configurations, with the best SQA result for UTP without mean listener or listener embedding and the best CSER result for the CL model with mean listener and text.

Significance. If the empirical claims hold, the paper makes a useful contribution by showing that within-listener pairwise comparisons can serve as a training signal for both MOS prediction and emotion attribute prediction, avoiding the averaging of ordinal listener scores. The method is simple, builds on the authors' prior UTP framework, and is evaluated against strong public baselines (UTMOS, UTP, and the Wagner et al. DAS model) on standard datasets. The comparison-loss approach is not circular: the central results are measured against external baselines, and the loss does not reduce to a fitted quantity. The paper also provides a thoughtful analysis of why mean-listener augmentation helps on IEMOCAP but not on BVCC. However, the lack of statistical significance testing and the unaddressed identifiability of the absolute output scale currently weaken the support for the central claims.

major comments (4)

[Section 4.3, Table 1] The headline SQA claim that UTP without mean listener and listener embedding achieves the best performance rests on a 0.001 SRCC difference over UTP with mean listener (0.9420 vs. 0.9410) and identical LCC (0.9376). Although the experiment was run 20 times, only averages are reported; no standard deviation, confidence interval, or significance test is given. This difference is well within the run-to-run noise expected for system-level MOS prediction. Please report per-run statistics and a paired significance test across the 20 runs (e.g., Wilcoxon signed-rank) before claiming 'best among all models.'
[Section 3.2, comparison loss] The comparison loss is L_X = MSE(2*sigmoid(sc1 - sc2) - 1, sign(sc_gt1 - sc_gt2)), which targets only the sign of the score difference. This objective is invariant to adding a constant to all predicted scores, so the absolute scale of the network output is not identified by the training signal. The CSER results are obtained in DAS evaluation mode and reported as SRCC/LCC; these correlation metrics are invariant to monotonic/affine transformations, so the numerical results are not invalidated. However, the paper repeatedly claims that the model performs 'DAS prediction' and learns 'a unified listener scoring scale.' If those absolute-scale claims are intended, provide a calibration analysis (e.g., compare predicted and target score distributions, report RMSE or mean bias, or describe an architectural/regularization mechanism that anchors the scale); otherwise limit the conclusions to relative ordering.
[Section 4.4, Table 2 and Conclusion] The conclusion states that 'employing comparison scores without listener embedding' yields the best performance and a unified listener scoring scale. In Table 2, the best CL configuration also uses mean-listener augmentation and text input; the isolation of the listener-embedding effect is the comparison between the mean+text row with and without listener embedding (arousal SRCC 0.7586 vs. 0.7703, valence SRCC 0.7956 vs. 0.8004). Please state explicitly which row corresponds to each ablation and attribute the improvement to the appropriate factor; as written, the conclusion conflates the removal of listener embedding with the addition of text and mean-listener scoring.
[Section 4.4, dataset explanation] The explanation for the opposite effect of the mean-listener approach in BVCC and IEMOCAP is post-hoc and is based on a single dataset per task. Because the abstract claims robustness, this explanation should be framed as a hypothesis or supported with quantitative evidence (e.g., a measure of listener-set overlap or comparison-sign reliability) rather than presented as a conclusion.

minor comments (5)

[Section 4.3, text after Table 1] The sentence 'The model performed the worst when listener embeddings were used, and its performance further degraded when the mean listener approach was not applied' contradicts Table 1, where the no-mean/no-embedding UTP row is the best; please rephrase to describe the ordering accurately.
[Tables 1 and 2] The checkmark columns are not clearly aligned with their headers in the manuscript PDF, making it difficult to map each row to a configuration; use explicit configuration labels (e.g., 'none', 'mean', 'mean+text') or ensure column alignment.
[Section 1 and Section 3.1.2] There are minor typos: 'learining' should be 'learning' in the Introduction, and 'an BLSTM' should be 'a BLSTM' in Section 3.1.2.
[Section 3.2] The loss equation is unnumbered; adding an equation number would ease referencing in the text.
[Index Terms] The index terms include 'speech recognition,' which is not a focus of this paper; consider replacing it with 'speech emotion recognition' or 'listener modeling.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: comparison learning is tested against external baselines on public datasets; the 'unified listener scoring scale' label is definitional but not load-bearing.

full rationale

The paper's central empirical claim—that comparison-based training without listener embeddings improves SQA and CSER performance—is supported by experiments against external baselines (UTMOS [12], Wagner et al. [17]) on public datasets (BVCC, IEMOCAP). The comparison loss in Sec. 3.2 (Comp_dv = 2·sigmoid(sc1−sc2)−1, Comp_gt = sgn(sc_gt1−sc_gt2)) is fully specified and does not reduce the predicted absolute scores to a fitted quantity; the DAS evaluation mode for CSER uses the detached network's output as a genuine prediction, so the reported SRCC/LCC values are empirical results rather than constructed equivalences. The sign-only loss leaves the absolute scale underdetermined, which is a validity and identifiability concern but not circularity: the network could fail to align with the target scale, and the paper reports high LCC as an empirical outcome. The label 'unified listener scoring scale' is defined in Sec. 3.1.3 as the setting without listener embeddings, so the conclusion's statement that omitting listener embeddings yields a unified scale is partly definitional; however, this definition does no mathematical work in deriving the performance improvements, which rest on the independent benchmark comparisons. The only prominent self-citation is UTP [16], the authors' prior framework; its components are restated with equations in the text and its behavior is re-measured in this paper, so the self-citation is not load-bearing. No equation or fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' own prior work. The paper therefore contains no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No explicit fitted free parameters are introduced; hyperparameters are inherited from prior models. The main load-bearing premises are the existence of a shared latent ordering and the identifiability of absolute scores from comparison-only training.

assumptions (3)

domain assumption All listeners share a common latent ordering of utterances, so within-listener comparisons can be pooled into one unified scoring scale.
Invoked in Sections 1 and 3.3; without it, sign comparisons from different listeners conflict and no unified scale exists. The paper does not measure inter-listener agreement.
domain assumption Comparison-only training is sufficient to recover absolute scores at inference.
In Section 3.2, the DAS evaluation mode detaches the trained model and predicts raw arousal and valence values, but the comparison loss does not identify absolute scale.
domain assumption Same-listener pair generation is valid without listener embeddings when many listeners overlap, and the mean virtual listener provides additional reliable pairs.
Sections 3.3 and 4.4; the explanation for why the mean listener helps in IEMOCAP but not BVCC is post hoc and dataset-specific.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition." pith.science (2026). https://pith.science/paper/KIBRMK2P

@misc{pith2026250713626,
  author       = {Pith},
  title        = {Pith review of: Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KIBRMK2P}},
  note         = {Machine review of arXiv:2507.13626}
}

read the original abstract

Speech Quality Assessment (SQA) and Continuous Speech Emotion Recognition (CSER) are two key tasks in speech technology, both relying on listener ratings. However, these ratings are inherently biased due to individual listener factors. Previous approaches have introduced a mean listener scoring scale and modeled all listener scoring scales in the training set. However, the mean listener approach is prone to distortion from averaging ordinal data, leading to potential biases. Moreover, learning multiple listener scoring scales while inferring based only on the mean listener scale limits effectiveness. In contrast, our method focuses on modeling a unified listener scoring scale, using comparison scores to correctly capture the scoring relationships between utterances. Experimental results show that our method effectively improves prediction performance in both SQA and CSER tasks, proving its effectiveness and robustness.

Figures

Figures reproduced from arXiv: 2507.13626 by the authors.

**Figure 1.** DAS prediction (UTMOS) model for SQA task. Dense * 2 SSL feature SSL model Text Sequence BERT BLSTM Listener ID Arousal Valence [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 3.** The UTP framework: Line styles indicate training conditions, shapes indicate testing conditions for each module. dicted attribute scores of the two input utterances, respectively. The ground-truth comparison score is defined as Compgt = sgn(scgt1 − scgt2), where scgt1 and scgt2 are the ground-truth scores for the two input utterances. sgn is the sign function. The training loss is computed using the mean squared er… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 29 canonical work pages

[1]

Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition

Introduction With the increasing integration of speech technologies into daily life, Speech Quality Assessment (SQA) and Continuous Speech Emotion Recognition (CSER) have become critical re- search areas. SQA plays an essential role in various applica- tions such as hearing aids [1], speech synthesis systems [2, 3], and speech coding systems [4, 5], etc. ...

work page Pith review arXiv 2025
[2]

all listener

Related Work 2.1. Speech Emotion Recognition Speech Emotion Recognition (SER) can generally be divided into two categories: discrete SER (DSER) and continuous SER (CSER). In DSER, the task is typically focused on classifying emotions into predefined categories, such as happy, sad, angry, etc. [18] introduced the concept that each listener has a differ- en...

work page
[3]

prefer- ence

Method 3.1. DAS Prediction Model 3.1.1. Speech Quality Assessment As shown in Fig. 1, UTMOS [12] incorporates five inputs: the SSL feature, the data-domain ID, the phoneme sequence, the reference sequence, and the listener ID. The SSL feature is ex- tracted from a pretrained wav2vec2 model [21]. The phoneme sequence is recognized using a pretrained ASR mo...

work page
[4]

Datasets 4.1.1

Experiment 4.1. Datasets 4.1.1. BVCC We followed the setting of UTP and used BVCC [27] for the ex- periment. In the training set, there are 4,973 unique utterances, each evaluated 8 times, yielding a total of 39,784 utterance- score pairs. The set comprises 175 systems, and a total of 288 listeners participated in the evaluation. The development set conta...

work page arXiv
[5]

Conclusion In this paper, we explored listener scoring scale modeling for the SQA and CSER tasks. Specifically, we proposed a method that enables the CL model to learn on a unified listener scor- ing scale through comparison scores and extend the use of CL model from SQA task to CSER task. Our experimental results demonstrated that the performance of the ...

work page
[6]

Acknowledgement This work was partly supported by JST AIP Acceleration Re- search JPMJCR25U5, Japan

work page
[7]

Ef- fect of attentive listening robot on pleasure and arousal change in psychiatric daycare,

K. Ochi, K. Inoue, D. Lala, T. Kawahara, and H. Kumazaki, “Ef- fect of attentive listening robot on pleasure and arousal change in psychiatric daycare,”Advanced Robotics, vol. 37, no. 21, pp. 1382–1391, 2023

work page 2023
[8]

Speech intelligibility prediction using a neurogram similarity index measure,

A. Hines and N. Harte, “Speech intelligibility prediction using a neurogram similarity index measure,”Speech Communication, vol. 54, no. 2, pp. 306–320, 2012

2012

Show all 36 references

[9]

Glow-tts: A generative flow for text-to-speech via monotonic alignment search,

J. Kim, S. Kim, J. Kong, and S. Yoon, “Glow-tts: A generative flow for text-to-speech via monotonic alignment search,” inAd- vances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2...

2020
[10]

Grad-tts: A diffusion probabilistic model for text-to-speech,

V . Popov, I. V ovk, V . Gogoryan, T. Sadekova, and M. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in International Conference on Machine Learning. PMLR, 2021, pp. 8599–8608

2021
[11]

Low bit-rate speech coding with vq-vae and a wavenet decoder,

C. G ˆarbacea, A. van den Oord, Y . Li, F. S. Lim, A. Luebs, O. Vinyals, and T. C. Walters, “Low bit-rate speech coding with vq-vae and a wavenet decoder,” inICASSP 2019-2019 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 735–739

2019
[12]

Lpcnet: Improving neural speech synthesis through linear prediction,

J.-M. Valin and J. Skoglund, “Lpcnet: Improving neural speech synthesis through linear prediction,” inICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP). IEEE, 2019, pp. 5891–5895

2019
[13]

Developing conversational virtual humans for social emotion elicitation based on large language models,

J. Llanes-Jurado, L. G ´omez-Zaragoz´a, M. E. Minissi, M. Alca˜niz, and J. Mar´ın-Morales, “Developing conversational virtual humans for social emotion elicitation based on large language models,” Expert Systems with Applications, vol. 246, p. 123261, 2024

2024
[14]

Moslight: A lightweight data-efficient system for non-intrusive speech quality assessment,

Z. Li and W. Li, “Moslight: A lightweight data-efficient system for non-intrusive speech quality assessment,” inProc. Interspeech 2023, 2023, pp. 5386–5390

2023
[15]

An evaluation of speech-based recognition of emotional and physiological markers of stress,

A. Baird, A. Triantafyllopoulos, S. Z ¨ankert, S. Ottl, L. Christ, L. Stappen, J. Konzok, S. Sturmbauer, E.-M. Meßner, B. M. Kudielkaet al., “An evaluation of speech-based recognition of emotional and physiological markers of stress,”Frontiers in Com- puter Science, vol. 3, p....

2021
[16]

Mental health monitoring from speech and language,

I. Zubiaga, I. Menchaca, M. de Velasco, and R. Justo, “Mental health monitoring from speech and language,”Depression, vol. 3, p. 276, 2022

2022
[17]

Mining valence, arousal, and dominance: possibilities for de- tecting burnout and productivity?

M. M ¨antyl¨a, B. Adams, G. Destefanis, D. Graziotin, and M. Ortu, “Mining valence, arousal, and dominance: possibilities for de- tecting burnout and productivity?” inProceedings of the 13th in- ternational conference on mining software repositories, 2016, pp. 247–258

2016
[18]

Ldnet: Uni- fied listener dependent modeling in mos prediction for synthetic speech,

W.-C. Huang, E. Cooper, J. Yamagishi, and T. Toda, “Ldnet: Uni- fied listener dependent modeling in mos prediction for synthetic speech,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 896–900

2022
[19]

UTMOS: UTokyo-SaruLab System for V oice- MOS Challenge 2022,

T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: UTokyo-SaruLab System for V oice- MOS Challenge 2022,” inProc. Interspeech 2022, 2022, pp. 4521–4525

2022
[20]

Sqat- ld: Speech quality assessment transformer utilizing listener de- pendent modeling for zero-shot out-of-domain mos prediction,

K. Shen, D. Yan, L. Dong, Y . Ren, X. Wu, and J. Hu, “Sqat- ld: Speech quality assessment transformer utilizing listener de- pendent modeling for zero-shot out-of-domain mos prediction,” in2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–6

2023
[21]

wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,

A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representa- tions,” inAdvances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associat...

2020
[22]

Meaningless statistics,

H. M. Marcus-Roberts and F. S. Roberts, “Meaningless statistics,” Journal of Educational Statistics, vol. 12, no. 4, pp. 383–394, 1987

1987
[23]

Preference-based training framework for automatic speech quality assessment using deep neural network,

C.-H. Hu, Y . Yasuda, and T. Toda, “Preference-based training framework for automatic speech quality assessment using deep neural network,” inINTERSPEECH 2023, 2023, pp. 546–550

2023
[24]

Dawn of the trans- former era in speech emotion recognition: closing the valence gap,

J. Wagner, A. Triantafyllopoulos, H. Wierstorf, M. Schmitt, F. Burkhardt, F. Eyben, and B. W. Schuller, “Dawn of the trans- former era in speech emotion recognition: closing the valence gap,”IEEE Transactions on Pattern Analysis and Machine Intel- ligence, vol. 45, no. 9, pp. ...

2023
[25]

Speech emotion recognition based on listener-dependent emotion perception mod- els,

A. Ando, T. Mori, S. Kobashikawa, and T. Toda, “Speech emotion recognition based on listener-dependent emotion perception mod- els,”APSIPA Transactions on Signal and Information Processing, vol. 10, p. e6, 2021

2021
[26]

Multi-task conformer with multi-feature com- bination for speech emotion recognition,

J. Seo and B. Lee, “Multi-task conformer with multi-feature com- bination for speech emotion recognition,”Symmetry, vol. 14, no. 7, p. 1428, 2022

2022
[27]

Mbnet: Mos prediction for synthesized speech with mean-bias network,

Y . Leng, X. Tan, S. Zhao, F. Soong, X.-Y . Li, and T. Qin, “Mbnet: Mos prediction for synthesized speech with mean-bias network,” inICASSP 2021-2021 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 391–395

2021
[28]

Iemocap: Interactive emotional dyadic motion capture database,

C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “Iemocap: Interactive emotional dyadic motion capture database,”Language resources and evaluation, vol. 42, pp. 335–359, 2008

2008
[29]

Simple and effective zero-shot cross-lingual phoneme recognition,

Q. Xu, A. Baevski, and M. Auli, “Simple and effective zero-shot cross-lingual phoneme recognition,”arXiv preprint arXiv:2109.11680, 2021

2021 arXiv
[30]

A density-based algorithm for discovering clusters in large spatial databases with noise

M. Ester, H.-P. Kriegel, J. Sander, X. Xuet al., “A density-based algorithm for discovering clusters in large spatial databases with noise.” inkdd, vol. 96, no. 34, 1996, pp. 226–231

1996
[31]

Robust wav2vec 2.0: Analyzing domain shift in self- supervised pre-training,

W.-N. Hsu, A. Sriram, A. Baevski, T. Likhomanenko, Q. Xu, V . Pratap, J. Kahn, A. Lee, R. Collobert, G. Synnaeve, and M. Auli, “Robust wav2vec 2.0: Analyzing domain shift in self- supervised pre-training,” inInterspeech 2021, 2021, pp. 721–725

2021
[32]

Robust speech recognition via large-scale weak supervision,

A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” inInternational conference on machine learning. PMLR, 2023, pp. 28 492–28 518

2023
[33]

Bert: Pre-training of deep bidirectional transformers for language understanding,

J. Devlin, “Bert: Pre-training of deep bidirectional transformers for language understanding,”arXiv preprint arXiv:1810.04805, 2018

2018 arXiv
[34]

The V oiceMOS Challenge 2022,

W. C. Huang, E. Cooper, Y . Tsao, H.-M. Wang, T. Toda, and J. Ya- magishi, “The V oiceMOS Challenge 2022,” inProc. Interspeech 2022, 2022, pp. 4536–4540

2022
[36]

Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,

L.-W. Chen and A. Rudnicky, “Exploring wav2vec 2.0 fine tuning for improved speech emotion recognition,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

2023
[2022]

Due to its robust performance and relevance, the UTP model serves as the baseline for our proposed approach

Building on this foundation, the UTP model leverages a comparison learning framework to model listener ratings effec- tively. Due to its robust performance and relevance, the UTP model serves as the baseline for our proposed approach

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Unifying Listener Scoring Scales: Comparison Learning Framework for Speech Quality Assessment and Continuous Speech Emotion Recognition

Introduction With the increasing integration of speech technologies into daily life, Speech Quality Assessment (SQA) and Continuous Speech Emotion Recognition (CSER) have become critical re- search areas. SQA plays an essential role in various applica- tions such as hearing aids [1], speech synthesis systems [2, 3], and speech coding systems [4, 5], etc. ...

work page Pith review arXiv 2025

[2] [2]

all listener

Related Work 2.1. Speech Emotion Recognition Speech Emotion Recognition (SER) can generally be divided into two categories: discrete SER (DSER) and continuous SER (CSER). In DSER, the task is typically focused on classifying emotions into predefined categories, such as happy, sad, angry, etc. [18] introduced the concept that each listener has a differ- en...

work page

[3] [3]

prefer- ence

Method 3.1. DAS Prediction Model 3.1.1. Speech Quality Assessment As shown in Fig. 1, UTMOS [12] incorporates five inputs: the SSL feature, the data-domain ID, the phoneme sequence, the reference sequence, and the listener ID. The SSL feature is ex- tracted from a pretrained wav2vec2 model [21]. The phoneme sequence is recognized using a pretrained ASR mo...

work page

[4] [4]

Datasets 4.1.1

Experiment 4.1. Datasets 4.1.1. BVCC We followed the setting of UTP and used BVCC [27] for the ex- periment. In the training set, there are 4,973 unique utterances, each evaluated 8 times, yielding a total of 39,784 utterance- score pairs. The set comprises 175 systems, and a total of 288 listeners participated in the evaluation. The development set conta...

work page arXiv

[5] [5]

Conclusion In this paper, we explored listener scoring scale modeling for the SQA and CSER tasks. Specifically, we proposed a method that enables the CL model to learn on a unified listener scor- ing scale through comparison scores and extend the use of CL model from SQA task to CSER task. Our experimental results demonstrated that the performance of the ...

work page

[6] [6]

Acknowledgement This work was partly supported by JST AIP Acceleration Re- search JPMJCR25U5, Japan

work page

[7] [7]

Ef- fect of attentive listening robot on pleasure and arousal change in psychiatric daycare,

K. Ochi, K. Inoue, D. Lala, T. Kawahara, and H. Kumazaki, “Ef- fect of attentive listening robot on pleasure and arousal change in psychiatric daycare,”Advanced Robotics, vol. 37, no. 21, pp. 1382–1391, 2023

work page 2023

[8] [8]

Speech intelligibility prediction using a neurogram similarity index measure,

A. Hines and N. Harte, “Speech intelligibility prediction using a neurogram similarity index measure,”Speech Communication, vol. 54, no. 2, pp. 306–320, 2012

2012