Pith. sign in

REVIEW 1 major objections 6 minor 78 references

THAI Speech Emotion Recognition (THAI-SER) corpus

T0 review · 1 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces THAI-SER, the first sizeable Thai speech emotion recognition corpus, and shows that after filtering by a 0.71 agreement threshold its crowdsourced labels reach a Krippendorff's alpha of 0.692 and a human recognition…

desk verdict THAI-SER is the first sizeable Thai SER corpus and worth having, but the headline alpha is partly a same-data threshold artifact; send to review with revisions. read the letter →

arxiv 2507.09618 v1 pith:4N2KTKC4 submitted 2025-07-13 cs.SD eess.AS

Jilamika Wongpithayadisai , Chompakorn Chaksangchaichot , Soravitt Sangnark , Patawee Prakrankamanant , Krit Gangwanpongpun , Siwa Boonpunmongkol , Premmarin Milindasuta , Dangkamon Na-Pombejra
show 2 more authors
Sarana Nutanong Ekapol Chuangsuwanich
This is my paper
classification cs.SDeess.AS
keywords speechemotionrecognitionThailanguageactedcorpuselicitedcrowdsourcedannotationinter-annotatorreliabilityKrippendorff'salphatonal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

THAI-SER is the first sizeable Thai speech emotion recognition corpus, built from 41 hours 36 minutes of audio divided into 27,854 utterances by 200 professional actors, covering neutral, angry, happy, sad, and frustrated speech. The recordings mix scripted and improvised sessions and studio and Zoom environments, so the corpus can support both controlled acted-emotion research and out-of-domain testing for online meetings. The paper's central claim is that a crowdsourced annotation pipeline, with a pretest, gold and consistency questions, and an agreement-based filter at 0.71, yields labels reliable enough for research: the filtered corpus reaches a Krippendorff's alpha of 0.692, above the 0.667 recommendation, and a human recognition accuracy of 0.772. The paper also provides speaker-independent k-fold baselines and cross-corpus comparisons with IEMOCAP, Emo-DB, and EMOVO. A reader should care because Thai is a tonal language whose prosodic emotion cues may differ from the Western languages that dominate existing SER corpora.

What carries the argument

The load-bearing machinery is the annotation-quality pipeline rather than a single algorithm. Each utterance receives 3 to 8 crowdsourced labels; annotator trustworthiness is gated by gold utterances and duplicated consistency utterances in addition to a pretest with a trick question; per-utterance majority agreement is computed as the proportion of annotators selecting the majority emotion; then a cutoff of 0.71 is chosen by scanning agreement thresholds until the corpus-level Krippendorff's alpha, computed with the MASI distance for set-valued labels, reaches at least 0.667. The corpus design carries part of the argument as well: fixed emotion-neutral sentences for scripted takes strip out lexical context, improvised dyadic scenarios elicit more natural speech, and the two recording environments, studio and Zoom, create an in-domain and out-of-domain split used throughout the experiments.

What would settle it

Take a fresh held-out batch of THAI-SER utterances, or re-annotate a random subset with the same pretest, gold, and consistency checks, then apply the fixed 0.71 filter and report alpha and human recognition accuracy on that batch alone; if alpha falls below 0.667, or accuracy falls well below 0.772, the reliability numbers are not portable to new data.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a large Thai acted-and-elicited corpus can be annotated to research-grade reliability through a carefully gated crowdsourcing pipeline: the raw corpus has low agreement (alpha 0.413), but removing the 13,672 utterances with majority agreement below 0.71 leaves 14,182 utterances with alpha 0.692 and human recognition accuracy 0.772. The paper further shows that reliability is not uniform across the corpus: scripted high-intensity performances are the easiest for humans to recognize (0.883 human recognition accuracy), low-intensity scripted performances are the hardest (0.690), improvised sessions reach higher inter-annotator reliability than scripted sessions overall, and frustrated is the emotion most often confused with angry and sad. The authors present these differences as evidence that emotional intensity and acting style are design variables rather than noise, and that the filtered THAI-SER can serve as a benchmark and a cross-corpus testbed for speech emotion recognition.

Load-bearing premise

The 0.71 cutoff was picked by testing different cutoffs on the same corpus until a standard reliability score rose above 0.667, and the headline reliability figures depend on that cutoff continuing to work when new utterances are annotated.

Editorial extensions

If this is right

  • Filtered THAI-SER gives Thai speech emotion recognition its first speaker-independent benchmark: with the four basic emotions, the provided CNN+LSTM baseline reaches 67.34% weighted and 62.61% unweighted accuracy, and adding frustration lowers both to 59.80% and 57.81%.
  • Scripted-only training beats improvised-only and mixed training on THAI-SER (73.99% weighted accuracy for four emotions), the opposite of the usual IEMOCAP result, so acting style should be reported and tuned per corpus.
  • Studio-trained models drop sharply on Zoom recordings, to roughly 46 to 56 percent weighted accuracy, making the Zoom split a ready-made robustness benchmark for future speech emotion recognition work.
  • High-intensity scripted emotions are much easier to recognize than low-intensity ones (0.883 versus 0.690 human recognition accuracy), so corpus builders can treat intensity as a design lever rather than an incidental property of acted speech.
  • Cross-corpus results position THAI-SER as a useful source language: THAI-SER-trained models beat IEMOCAP-trained models on Emo-DB and EMOVO even after both corpora are pruned to matched hours and speakers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the 0.71 threshold generalizes beyond this corpus, the same pretest-plus-gold-plus-consistency annotation pipeline becomes a portable recipe for other under-resourced tonal languages, and its transferability could be tested by applying it to a small pilot corpus in a related language without re-tuning the cutoff.
  • Because THAI-SER carries up to eight annotations per utterance, it offers soft-label distributions richer than IEMOCAP's fixed three-annotator design; a test the paper does not run is whether training on those soft labels, or on ambiguous samples in curriculum order, closes part of the gap with human recognition accuracy.
  • The confusion between frustrated, angry, and sad suggests a hierarchical label scheme, for example coarse arousal and valence before fine emotion, might outperform flat five-way classification; this is directly testable on the released corpus.
  • The Zoom sessions could also be used to separate the causes of the out-of-domain performance drop by taking clean studio audio, passing it through the same Zoom encoding path, and comparing model accuracy before and after that controlled degradation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. The paper introduces THAI-SER, a new Thai speech emotion recognition corpus containing 27,854 utterances (41.61 hours) recorded from 200 professional actors in studio and Zoom environments, with both scripted and improvised sessions across five emotion categories. Annotations were collected via crowdsourcing with a multi-stage quality-control pipeline including pretests, gold utterances, consistency checks, and agreement-based filtering. The paper reports inter-annotator reliability (Krippendorff's alpha) and human recognition accuracy before and after filtering, analyzes reliability across demographics and recording conditions, and provides baseline and cross-corpus experiments. The corpus and experimental code are publicly released.

Significance. If the reliability claims hold, THAI-SER fills a clear gap as the first large-scale Thai (and a rare tonal-language) SER corpus, offering a valuable resource for speech emotion recognition in Southeast Asian languages. The paper's strengths include the size and diversity of the corpus, the inclusion of both controlled studio and realistic Zoom conditions, the unusually detailed documentation of the crowdsourcing quality-control pipeline, transparent reporting of raw unfiltered reliability metrics alongside filtered ones, and the provision of reproducible baseline code and cross-corpus benchmarks with public data release. The cross-corpus evaluation, including a size-matched pruning experiment with IEMOCAP, is a commendable effort at fair comparison.

major comments (1)
  1. [Section 4.2.1, Figure 8, Table 9] The agreement threshold of 0.71 is selected by scanning thresholds on the full 27,854-utterance corpus until Krippendorff's alpha reaches the target of 0.667 (raw alpha 0.413). The headline reliability values (alpha = 0.692, HRA = 0.772 on the 14,182 retained utterances) are then computed on the same utterances that determined the cutoff. Because the filter preferentially removes low-consensus items, selecting the cutoff to achieve the target alpha on the same data makes the reported reliability an in-sample, post hoc estimate. The abstract's claim that the corpus 'achieved an alpha score of 0.692, higher than a recommendation of 0.667' is therefore not supported as a statement about the generalizable reliability of the filtering procedure: a user applying the 0.71 threshold to newly collected utterances, or to a held-out portion of this corpus, may obtain alpha below 0.667. Please validate the threshold on held-out data (e.g., select the threshold on a development split of sessions and report alpha on a test split), or provide bootstrap/confidence intervals for the filtered alpha, and temper the abstract and conclusion claims accordingly.
minor comments (6)
  1. [Eq. (1)] The majority agreement formula is typeset ambiguously; the fraction appears to place the sum over annotators in the numerator and the sum over labels in the denominator, which is not mathematically well-defined. Please insert parentheses to make the intended expression clear, e.g., agreement(x_i) = max_k (1/N) * sum_{n=1}^N [ y_{nk} / (sum_{k'=1}^K y_{nk'}) ].
  2. [Eq. (6)] The HRA definition is not written as a statistic over utterances: the left-hand side is per-utterance while the sum runs over an index i up to N, without specifying whether N is the number of annotators or the number of utterances. Please rewrite as a corpus-level metric, e.g., HRA(D) = (1/|D|) sum_{x in D} 1(maj(y_x) = assigned(x)), and clarify the relationship to the values in Table 9.
  3. [Table 1] THAI-SER is dated 2021 in Table 1, but the manuscript is from 2025; please correct the year or explain if 2021 refers to the collection period.
  4. [Section 4.2.1] The phrase 'optimal threshold' is used for 0.71; since the threshold is chosen to satisfy a target alpha on the current corpus, please qualify it as, e.g., 'the smallest threshold that achieves the target alpha on the present corpus,' to avoid implying a general optimum.
  5. [Table 7] The mapping of 'other' emotions to the five categories is subjective; in particular, mapping 'surprise' to 'happy' and 'calm' to 'neutral' may not be universally accepted. Please provide a brief justification or at least acknowledge the potential bias introduced by this mapping.
  6. [Section 4.2.2, Tables 9-10] The differences in alpha and HRA across gender, age, and recording conditions are reported without uncertainty estimates. Adding confidence intervals or significance tests would help readers gauge the strength of these descriptive comparisons.

Circularity Check

1 steps flagged · score 6.0 of 10

Post-filter alpha is selected into existence: the 0.71 threshold is chosen to make alpha exceed 0.667 on the same corpus.

  1. fitted input called prediction [Section 4.2.1, Figure 8, Table 9; abstract]
    "The optimal threshold that yields an alpha score of ≥ 0.667 is 0.71. Thus, we strongly recommend that any researcher using the THAI-SER corpus remove utterances with an agreement score lower than 0.71 to ensure higher corpus reliability. ... our corpus, after filtering, achieved an alpha score of 0.692, higher than a recommendation of 0.667."

    The cutoff 0.71 is chosen by scanning agreement thresholds on the full 27,854-utterance corpus (raw alpha 0.413) and selecting the first cutoff at which Krippendorff's alpha reaches the 0.667 target, as shown in Figure 8. The alpha=0.692 and HRA=0.772 in Table 9 are then computed on the same 14,182 retained utterances that defined the cutoff. Consequently, the statement that the filtered corpus 'achieved' alpha above 0.667 is guaranteed by the selection rule rather than established independently; it is an in-sample, post-selection statistic. Applying the same 0.71 filter to new or held-out utterances could yield alpha below 0.667, so the headline reliability claim is not validated as a generalizable property.

full rationale

THAI-SER's corpus construction, annotation pipeline, and benchmarking are not circular: the dataset is a new resource, and the cross-corpus evaluations against IEMOCAP, Emo-DB, and EMOVO provide independent external grounding. The only load-bearing circularity is the reliability headline. Section 4.2.1 sets the 0.71 filter by scanning agreement cutoffs until Krippendorff's alpha on the full corpus crosses 0.667; Table 9 then reports alpha=0.692 on the same retained data. Thus the abstract's 'achieved ... higher than a recommendation of 0.667' is a direct consequence of the threshold selection, not an independently estimated property. This fits pattern 2 (fitted input called prediction) and warrants score 6 under the rubric (a headline result reduces by construction), while the remainder of the paper retains independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-set quality thresholds and on the assumption that director-assigned emotions are valid ground truth. No new theoretical entities are introduced; the corpus itself is a data resource rather than a postulated entity.

free parameters (3)
  • Agreement filter threshold = 0.71
    Selected in Section 4.2.1 by scanning possible thresholds and choosing the minimum value that pushes Krippendorff's alpha to >=0.667 on the filtered corpus; this cutoff determines the reported alpha=0.692 and HRA=0.772.
  • Annotator trustworthiness threshold = >50%
    Annotators are considered trustworthy only if they score above 50% on both gold confidence and consistency checks (Section 3.4); this hand-set criterion affects which annotations are retained.
  • Pretest passing criteria = 5/10 standard + 1 trick question
    Annotators must pass at least 5 of 10 standard pretest questions plus the trick question (Section 3.2); this hand-set threshold shapes the annotator pool.
assumptions (4)
  • domain assumption The emotion assigned to the actor by the director during recording is the correct ground-truth label for each utterance.
    Used in Eq. (6) for human recognition accuracy and in model training; if actors failed to convey the intended emotion, the reported accuracies would not measure perception of the intended emotion.
  • domain assumption Krippendorff's alpha with the MASI distance is an appropriate reliability measure for set-valued emotion annotations.
    Section 4.1.2; the reference threshold of 0.667 was originally proposed for nominal disagreement, and the MASI distance gives partial credit, which can raise alpha relative to a nominal distance.
  • ad hoc to paper The 0.71 agreement threshold chosen from the data is the optimal trade-off between reliability and corpus size.
    Section 4.2.1; the threshold is chosen because it yields alpha>=0.667, with no external validation of optimality.
  • domain assumption Audio-only information is sufficient for reliable emotion perception in Thai.
    The annotation task deliberately uses audio only (Section 3), although humans typically use facial expressions and semantics; this may lower annotator agreement but is a stated design choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of THAI Speech Emotion Recognition (THAI-SER) corpus." pith.science (2026). https://pith.science/paper/4N2KTKC4

@misc{pith2026250709618,
  author       = {Pith},
  title        = {Pith review of: THAI Speech Emotion Recognition (THAI-SER) corpus},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4N2KTKC4}},
  note         = {Machine review of arXiv:2507.09618}
}
read the original abstract

We present the first sizeable corpus of Thai speech emotion recognition, THAI-SER, containing 41 hours and 36 minutes (27,854 utterances) from 100 recordings made in different recording environments: Zoom and two studio setups. The recordings contain both scripted and improvised sessions, acted by 200 professional actors (112 females and 88 males, aged 18 to 55) and were directed by professional directors. There are five primary emotions: neutral, angry, happy, sad, and frustrated, assigned to the actors when recording utterances. The utterances are annotated with an emotional category using crowdsourcing. To control the annotation process's quality, we also design an extensive filtering and quality control scheme to ensure that the majority agreement score remains above 0.71. We evaluate our annotated corpus using two metrics: inter-annotator reliability and human recognition accuracy. Inter-annotator reliability score was calculated using Krippendorff's alpha, where our corpus, after filtering, achieved an alpha score of 0.692, higher than a recommendation of 0.667. For human recognition accuracy, our corpus scored up to 0.772 post-filtering. We also provide the results of the model trained on the corpus evaluated on both in-corpus and cross-corpus setups. The corpus is publicly available under a Creative Commons BY-SA 4.0, as well as our codes for the experiments.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 56 canonical work pages

  1. [1]

    , Devillers, L

    Abrilan_4 APACrefauthors Abrilian, S. , Devillers, L. , Buisine, S. Martin, J C. APACrefauthors \ 2005 . EmoTV1 : Annotation of real-life emotions for the specification of multimodal affective interfaces EmoTV1 : Annotation of real-life emotions for the specification of multimodal affective interfaces . HCI International Hci international \ ( 401, 407-408)

    work page 2005

  2. [2]

    \ Oğuz, K

    Mehmet_2 APACrefauthors Akçay, M.B. \ Oğuz, K. APACrefauthors \ 2020 . Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers . Speech Communication 116 56-76

    work page 2020

  3. [3]

    , Kobashikawa, S

    soft-label1 APACrefauthors Ando, A. , Kobashikawa, S. , Kamiyama, H. , Masumura, R. , Ijima, Y. Aono, Y. APACrefauthors \ 2018 . Soft-Target Training with Ambiguous Emotional Utterances for DNN-Based Speech Emotion Classification Soft-target training with ambiguous emotional utterances for dnn-based speech emotion classification . 2018 IEEE International ...

    arXiv 2018

  4. [4]

    , Masumura, R

    soft-label3 APACrefauthors Ando, A. , Masumura, R. , Kamiyama, H. , Kobashikawa, S. Aono, Y. APACrefauthors \ 2019 09 . Speech Emotion Recognition Based on Multi-Label Emotion Existence Model Speech emotion recognition based on multi-label emotion existence model . ( 2818-2822). APACrefDOI doi:10.21437/Interspeech.2019-2524 APACrefDOI

    work page doi:10.21437/interspeech.2019-2524 2019

  5. [5]

    , Wang, L

    doi:10.1177/0022022108321178 APACrefauthors Anolli, L. , Wang, L. , Mantovani, F. Toni, A.D. APACrefauthors \ 2008 . The Voice of Emotion in Chinese and Italian Young Adults The voice of emotion in chinese and italian young adults . Journal of Cross-Cultural Psychology 39 5 565-598 . APACrefURL https://doi.org/10.1177/0022022108321178 APACrefURL https://a...

    work page doi:10.1177/0022022108321178 2008

  6. [6]

    , Kiros, J.R

    ba2016layer APACrefauthors Ba, J.L. , Kiros, J.R. Hinton, G.E. APACrefauthors \ 2016 . Layer Normalization. Layer normalization

    work page 2016

  7. [7]

    \ Latorre, J

    cs_gold_1 APACrefauthors Buchholz, S. \ Latorre, J. APACrefauthors \ 2011 01 . Crowdsourcing Preference Tests, and How to Detect Cheating. Crowdsourcing preference tests, and how to detect cheating. ( 3053-3056)

    work page 2011

  8. [8]

    , Paeschke, A

    Emo-DB APACrefauthors Burkhardt, F. , Paeschke, A. , Rolfes, M. , Sendlmeier, W.F. Weiss, B. APACrefauthors \ 2005 . A database of German emotional speech A database of german emotional speech . Ninth European Conference on Speech Communication and Technology. Ninth european conference on speech communication and technology

    work page 2005

Show all 78 references
  1. [9]

    , Bulut, M

    IEMOCAP APACrefauthors Busso, C. , Bulut, M. , Lee, C C. , Kazemzadeh, A. , Mower, E. , Kim, S. Narayanan, S.S. APACrefauthors \ 2008 . IEMOCAP : Interactive emotional dyadic motion capture database IEMOCAP : Interactive emotional dyadic motion capture database . Language reso...

    2008

  2. [10]

    \ Narayanan, S

    CarlosBusso APACrefauthors Busso, C. \ Narayanan, S. APACrefauthors \ 2008 . Scripted dialogs versus improvisation: Lessons learned about emotional elicitation techniques from the IEMOCAP database Scripted dialogs versus improvisation: Lessons learned about emotional elicitati...

    2008

  3. [11]

    , Parthasarathy , S

    MSP-IMPROV APACrefauthors Busso , C. , Parthasarathy , S. , Burmania , A. , AbdelWahab , M. , Sadoughi , N. Provost , E.M. APACrefauthors \ 2017 . MSP-IMPROV : An Acted Corpus of Dyadic Interactions to Study Emotion Perception MSP-IMPROV : An acted corpus of dyadic interaction...

    2017

  4. [12]

    , Cooper , D.G

    CREMA-D APACrefauthors Cao , H. , Cooper , D.G. , Keutmann , M.K. , Gur , R.C. , Nenkova , A. Verma , R. APACrefauthors \ 2014 . CREMA-D : Crowd-Sourced Emotional Multimodal Actors Dataset CREMA-D : Crowd-sourced emotional multimodal actors dataset . IEEE Transactions on Affec...

    2014

  5. [13]

    , Lease, M

    cs_gold_2 APACrefauthors Carvalho, V. , Lease, M. Yilmaz, E. APACrefauthors \ 2011 05 . Crowdsourcing for search and data mining Crowdsourcing for search and data mining . ( 45, 5-6). APACrefDOI doi:10.1145/1988852.1988856 APACrefDOI

    2011

  6. [14]

    onlinelearning APACrefauthors Cen, L. , Wu, F. , Yu, Z.L. Hu, F. APACrefauthors \ 2016 . Chapter 2 - A Real-Time Speech Emotion Recognition System and its Application in Online Learning Chapter 2 - a real-time speech emotion recognition system and its application in online lea...

    2016

  7. [15]

    APACrefauthors \ 2020

    hopewebsite APACrefauthors Chantarapratin, N. APACrefauthors \ 2020 . Hope: Data Annotations. Hope: Data annotations. https://www.hopedata.org/ . Online; accessed between 15 June 2020 to 15 July 2020

    2020

  8. [16]

    , Wu, C C

    cs_intrinsic APACrefauthors Chen, K T. , Wu, C C. , Chang, Y C. Lei, C L. APACrefauthors \ 2009 . A Crowdsourceable QoE Evaluation Framework for Multimedia Content A crowdsourceable qoe evaluation framework for multimedia content . New York, NY, USA Association for Computing M...

    2009

  9. [17]

    , Kim, J

    chong2015exploring APACrefauthors Chong, C.S. , Kim, J. Davis, C. APACrefauthors \ 2015 . Exploring acoustic differences between Cantonese (tonal) and English (non-tonal) spoken expressions of emotions Exploring acoustic differences between cantonese (tonal) and english (non-t...

    2015

  10. [19]

    \ Lee, C C

    soft-label4 APACrefauthors Chou, H C. \ Lee, C C. APACrefauthors \ 2019 . Every Rating Matters: Joint Learning of Subjective Labels and Individual Annotators for Speech Emotion Classification Every rating matters: Joint learning of subjective labels and individual annotators f...

    2019

  11. [20]

    chu2023qwenaudioadvancinguniversalaudio APACrefauthors Chu, Y. , Xu, J. , Zhou, X. , Yang, Q. , Zhang, S. , Yan, Z. Zhou, J. APACrefauthors \ 2023 . Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models. Qwen-audio: Advancing univers...

    2023 arXiv

  12. [21]

    APACrefauthors \ 1960

    kappascore APACrefauthors Cohen, J. APACrefauthors \ 1960 . A Coefficient of Agreement for Nominal Scales A coefficient of agreement for nominal scales . Educational and Psychological Measurement 20 1 37-46 . APACrefURL https://doi.org/10.1177/001316446002000104 APACrefURL htt...

    1960 doi

  13. [22]

    , Iaderola, I

    EMOVO APACrefauthors Costantini, G. , Iaderola, I. , Paoloni, A. Todisco, M. APACrefauthors \ 2014 . EMOVO corpus: an Italian emotional speech database EMOVO corpus: an italian emotional speech database . International Conference on Language Resources and Evaluation (LREC 2014...

    2014

  14. [23]

    APACrefauthors \ 2012

    amazonmechanicalturk APACrefauthors Crowston, K. APACrefauthors \ 2012 . Amazon Mechanical Turk: A Research Tool for Organizations and Information Systems Scholars Amazon mechanical turk: A research tool for organizations and information systems scholars . A. Bhattacherjee\ B....

    2012

  15. [24]

    , Fidanza, G

    ser_kfold:journals/corr/abs-1802-05630 APACrefauthors Etienne, C. , Fidanza, G. , Petrovskii, A. , Devillers, L. Schmauch, B. APACrefauthors \ 2018 . Speech Emotion Recognition with Data Augmentation and Layer-wise Learning Rate Adjustment Speech emotion recognition with data ...

    2018 arXiv

  16. [25]

    lssed APACrefauthors Fan, W. , Xu, X. , Xing, X. , Chen, W. Huang, D. APACrefauthors \ 2021 . LSSED: A Large-Scale Dataset and Benchmark for Speech Emotion Recognition Lssed: A large-scale dataset and benchmark for speech emotion recognition . ICASSP 2021 - 2021 IEEE Internati...

    2021

  17. [26]

    APACrefauthors \ 2020

    wangwebsite APACrefauthors Gangwanpongpun, K. APACrefauthors \ 2020 . Wang: Data Market. Wang: Data market. https://www.wang.in.th/ . Online; accessed between 1 August 2020 to 15 March 2021

    2020

  18. [27]

    \ Beigi, H

    crosscorpus2 APACrefauthors Goel, S. \ Beigi, H. APACrefauthors \ 2020 . Cross Lingual Cross Corpus Speech Emotion Recognition. Cross lingual cross corpus speech emotion recognition

    2020

  19. [28]

    , Kroschel , K

    Grimm_3 APACrefauthors Grimm , M. , Kroschel , K. Narayanan , S. APACrefauthors \ 2008 . The Vera am Mittag German audio-visual emotional speech database The Vera am Mittag german audio-visual emotional speech database . 2008 IEEE International Conference on Multimedia and Exp...

    2008

  20. [29]

    , Hoßfeld, T

    cs_majority_1 APACrefauthors Hirth, M. , Hoßfeld, T. Tran-Gia, P. APACrefauthors \ 2011 . Cost-Optimal Validation Mechanisms and Cheat-Detection for Crowdsourcing Platforms Cost-optimal validation mechanisms and cheat-detection for crowdsourcing platforms . 2011 Fifth Internat...

    2011

  21. [30]

    , Scheuring, S

    cs_behavior_2 APACrefauthors Hirth, M. , Scheuring, S. , Hossfeld, T. , Schwartz, C. Tran-Gia, P. APACrefauthors \ 2014 07 . Predicting Result Quality in Crowdsourcing Using Application Layer Monitoring Predicting result quality in crowdsourcing using application layer monitor...

    2014

  22. [31]

    \ Schmidhuber, J

    lstm APACrefauthors Hochreiter, S. \ Schmidhuber, J. APACrefauthors \ 1997 12 . Long Short-term Memory Long short-term memory . Neural computation 9 1735-80 . APACrefDOI doi:10.1162/neco.1997.9.8.1735 APACrefDOI

    1997 doi

  23. [32]

    , Jue, G

    robot APACrefauthors Huahu, X. , Jue, G. Jian, Y. APACrefauthors \ 2010 . Application of Speech Emotion Recognition in Intelligent Household Robot Application of speech emotion recognition in intelligent household robot . 2010 International Conference on Artificial Intelligenc...

    2010 doi

  24. [33]

    , Provost, F

    cs_gold_5 APACrefauthors Ipeirotis, P.G. , Provost, F. Wang, J. APACrefauthors \ 2010 . Quality Management on Amazon Mechanical Turk Quality management on amazon mechanical turk . ( 64–67). New York, NY, USA Association for Computing Machinery . APACrefURL https://doi.org/10.1...

    2010

  25. [34]

    \ Hinton, E

    Jaitly2013VocalTL APACrefauthors Jaitly, N. \ Hinton, E. APACrefauthors \ 2013 . Vocal Tract Length Perturbation (VTLP) improves speech recognition Vocal tract length perturbation (vtlp) improves speech recognition

    2013

  26. [35]

    , Sornlertlamvanich, V

    Kasuriya_1 APACrefauthors Kasuriya, S. , Sornlertlamvanich, V. , Cotsomrong, P. , Kanokphara, S. Thatphithakkul, N. APACrefauthors \ 2004 . Thai Speech Recognition Corpora Thai speech recognition corpora . Journal of Chinese Language and Computing 14 4

    2004

  27. [36]

    \ Zitouni, I

    cs_behavior_4 APACrefauthors Kazai, G. \ Zitouni, I. APACrefauthors \ 2016 . Quality Management in Crowdsourcing Using Gold Judges Behavior Quality management in crowdsourcing using gold judges behavior . ( 267–276). New York, NY, USA Association for Computing Machinery . APAC...

    2016

  28. [37]

    kingma2017adam APACrefauthors Kingma, D.P. \ Ba, J. APACrefauthors \ 2017 . Adam: A Method for Stochastic Optimization. Adam: A method for stochastic optimization

    2017

  29. [38]

    , Walecki , R

    SEWA APACrefauthors Kossaifi , J. , Walecki , R. , Panagakis , Y. , Shen , J. , Schmitt , M. , Ringeval , F. Pantic , M. APACrefauthors \ 2021 . SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild Sewa db: A rich database for audio-visual emoti...

    2021

  30. [39]

    APACrefauthors \ 2004

    Krippendorff-2004 APACrefauthors Krippendorff, K. APACrefauthors \ 2004 . Content Analysis: An Introduction to Its Methodology (second edition) Content analysis: An introduction to its methodology (second edition) . Sage Publications

    2004

  31. [40]

    APACrefauthors \ 2011

    Krippendorff-2011 APACrefauthors Krippendorff, K. APACrefauthors \ 2011 . Computing Krippendorff's alpha-reliability Computing Krippendorff's alpha-reliability

    2011

  32. [41]

    , Zamani, M.A

    envrobust APACrefauthors Lakomkin, E. , Zamani, M.A. , Weber, C. , Magg, S. Wermter, S. APACrefauthors \ 2018 . On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks On the robustness of speech emotion recognition for human-robot...

    2018

  33. [42]

    APACrefauthors \ 2021

    crosscorpus1 APACrefauthors Lee, S w. APACrefauthors \ 2021 . Domain Generalization with Triplet Network for Cross-Corpus Speech Emotion Recognition Domain generalization with triplet network for cross-corpus speech emotion recognition . 2021 IEEE Spoken Language Technology Wo...

    2021

  34. [43]

    , Tao, J

    Li_5 APACrefauthors Li, Y. , Tao, J. , Chao, L. , Bao, W. Liu, Y. APACrefauthors \ 2017 . CHEAVD : a Chinese natural emotional audio-visual database CHEAVD : a chinese natural emotional audio-visual database . Journal of Ambient Intelligence and Humanized Computing 8 6 913-924

    2017

  35. [44]

    , Chilton, L.B

    cs_iterative APACrefauthors Little, G. , Chilton, L.B. , Goldman, M. Miller, R. APACrefauthors \ 2010 . Exploring iterative and parallel human computation processes Exploring iterative and parallel human computation processes . HCOMP '10. Hcomp '10

    2010

  36. [45]

    \ Russo, F.A

    RAVDESS APACrefauthors Livingstone, S.R. \ Russo, F.A. APACrefauthors \ 2018 . The Ryerson Audio-Visual Database of Emotional Speech and Song ( RAVDESS ): A dynamic, multimodal set of facial and vocal expressions in North American English The Ryerson audio-visual database of e...

    2018

  37. [46]

    \ Busso, C

    soft-label2 APACrefauthors Lotfian, R. \ Busso, C. APACrefauthors \ 2017 . Formulating emotion perception as a probabilistic model with application to categorical emotion classification Formulating emotion perception as a probabilistic model with application to categorical emo...

    2017

  38. [47]

    \ Busso, C

    MSP-PODCAST APACrefauthors Lotfian, R. \ Busso, C. APACrefauthors \ 2019 1 October-December . Building Naturalistic Emotionally Balanced Speech Corpus by Retrieving Emotional Speech From Existing Podcast Recordings Building naturalistic emotionally balanced speech corpus by re...

    2019

  39. [48]

    \ Busso, C

    curriculum APACrefauthors Lotfian, R. \ Busso, C. APACrefauthors \ 2019 2 . Curriculum Learning for Speech Emotion Recognition From Crowdsourced Labels Curriculum learning for speech emotion recognition from crowdsourced labels . IEEE/ACM Transactions on Audio, Speech, and Lan...

    2019

  40. [49]

    , Chang, R.K.C

    cs_behavior_3 APACrefauthors Mok, R.K.P. , Chang, R.K.C. Li, W. APACrefauthors \ 2017 . Detecting Low-Quality Workers in QoE Crowdtesting: A Worker Behavior-Based Approach Detecting low-quality workers in qoe crowdtesting: A worker behavior-based approach . IEEE Transactions o...

    2017

  41. [50]

    \ Vu, N.T

    neumann2017attentive APACrefauthors Neumann, M. \ Vu, N.T. APACrefauthors \ 2017 . Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech. Attentive convolutional neural network based sp...

    2017

  42. [51]

    , Sorokin, A

    cs_gold_4 APACrefauthors Oleson, D. , Sorokin, A. , Laughlin, G. , Hester, V. , Le, J. Biewald, L. APACrefauthors \ 2011 . Programmatic Gold: Targeted and Scalable Quality Assurance in Crowdsourcing Programmatic gold: Targeted and scalable quality assurance in crowdsourcing . ...

    2011

  43. [52]

    , Costantini, G

    DEMoS APACrefauthors Parada-Cabaleiro, E. , Costantini, G. , Batliner, A. , Schmitt, M. Schuller, B.W. APACrefauthors \ 2019 . DEMoS : An Italian emotional speech corpus DEMoS : An italian emotional speech corpus . Language Resources and Evaluation 1-43

    2019

  44. [53]

    , Palaz, D

    crosscorpus4 APACrefauthors Parry, J. , Palaz, D. , Clarke, G. , Lecomte, P. , Mead, R. , Berger, M. Hofer, G. APACrefauthors \ 2019 09 . Analysis of Deep Learning Architectures for Cross-Corpus Speech Emotion Recognition Analysis of deep learning architectures for cross-corpu...

    2019 doi

  45. [54]

    APACrefauthors \ 2006

    MASI APACrefauthors Passonneau, R. APACrefauthors \ 2006 . Measuring Agreement on Set-valued Items ( MASI ) for Semantic and Pragmatic Annotation Measuring agreement on set-valued items ( MASI ) for semantic and pragmatic annotation . Proceedings of the Fifth International Con...

    2006

  46. [55]

    , Monetta, L

    langdiif APACrefauthors Pell, M. , Monetta, L. , Paulmann, S. Kotz, S. APACrefauthors \ 2009 06 . Recognizing Emotions in a Foreign Language Recognizing emotions in a foreign language . Journal of Nonverbal Behavior 33 107-120 . APACrefDOI doi:10.1007/s10919-008-0065-7 APACrefDOI

    2009 doi

  47. [56]

    APACrefauthors \ 2000 01

    callcenter APACrefauthors Petrushin, V. APACrefauthors \ 2000 01 . Emotion in Speech: Recognition and Application to Call Centers Emotion in speech: Recognition and application to call centers . Proceedings of Artificial Neural Networks in Engineering

    2000

  48. [57]

    \ Dupuis, K

    TESS APACrefauthors Pichora-Fuller, M.K. \ Dupuis, K. APACrefauthors \ 2020 . Toronto emotional speech set (TESS) . Toronto emotional speech set (TESS) . Borealis . APACrefURL https://doi.org/10.5683/SP2/E8H2MF APACrefURL APACrefDOI doi:10.5683/SP2/E8H2MF APACrefDOI

    2020 doi

  49. [58]

    , Ghoshal, A

    Povey11thekaldi APACrefauthors Povey, D. , Ghoshal, A. , Boulianne, G. , Goel, N. , Hannemann, M. , Qian, Y. Stemmer, G. APACrefauthors \ 2011 . The kaldi speech recognition toolkit The kaldi speech recognition toolkit . In IEEE 2011 workshop. In ieee 2011 workshop

    2011

  50. [59]

    , Florêncio, D

    cs_behavior_1 APACrefauthors Ribeiro, F. , Florêncio, D. , Zhang, C. Seltzer, M. APACrefauthors \ 2011 . CROWDMOS: An approach for crowdsourcing mean opinion score studies Crowdmos: An approach for crowdsourcing mean opinion score studies . 2011 IEEE International Conference o...

    2011

  51. [60]

    , Asawaroengchai, C

    rubenstein2023audiopalmlargelanguagemodel APACrefauthors Rubenstein, P.K. , Asawaroengchai, C. , Nguyen, D.D. , Bapna, A. , Borsos, Z. , de Chaumont Quitry, F. Frank, C. APACrefauthors \ 2023 . AudioPaLM: A Large Language Model That Can Speak and Listen. Audiopalm: A large lan...

    2023 arXiv

  52. [61]

    , Rozenberg, S

    satt17_interspeech APACrefauthors Satt, A. , Rozenberg, S. Hoory, R. APACrefauthors \ 2017 . Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms . Proc. Interspeech 2017 Pro...

    2017 doi

  53. [62]

    , Provost, F

    cs_majority_2 APACrefauthors Sheng, V. , Provost, F. Ipeirotis, P. APACrefauthors \ 2008 08 . Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labelers Get another label? improving data quality and data mining using multiple, noisy labelers . ( 6...

    2008

  54. [63]

    \ Garcia-Molina, H

    cs_majority_4 APACrefauthors Venetis, P. \ Garcia-Molina, H. APACrefauthors \ 2012 August . Quality Control for Comparison Microtasks Quality control for comparison microtasks . CrowdKDD 2012. Crowdkdd 2012. Stanford InfoLab . APACrefURL http://ilpubs.stanford.edu:8090/1044/ A...

    2012

  55. [64]

    , Chen, S

    wang2023neuralcodeclanguagemodels APACrefauthors Wang, C. , Chen, S. , Wu, Y. , Zhang, Z. , Zhou, L. , Liu, S. Wei, F. APACrefauthors \ 2023 . Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers. Neural codec language models are zero-shot text to speech synt...

    2023 arXiv

  56. [65]

    , Cho, Y H

    mobileservice APACrefauthors Yoon, W J. , Cho, Y H. Park, K S. APACrefauthors \ 2007 . A Study of Speech Emotion Recognition and Its Application to Mobile Services A study of speech emotion recognition and its application to mobile services . Proceedings of the 4th Internation...

    2007

  57. [66]

    \ Callison-Burch, C

    cs_majority_3 APACrefauthors Zaidan, O. \ Callison-Burch, C. APACrefauthors \ 2011 01 . Crowdsourcing Translation: Professional Quality from Non-Professionals Crowdsourcing translation: Professional quality from non-professionals . ( 1220-1229)

    2011

  58. [67]

    , Zhang, Y

    zhou2025emotionaldimensioncontrollanguage APACrefauthors Zhou, K. , Zhang, Y. , Zhao, S. , Wang, H. , Pan, Z. , Ng, D. Ma, B. APACrefauthors \ 2025 . Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions. Emotional dime...

    2025

  59. [68]

    \ Carterette, B

    cs_gold_3 APACrefauthors Zhu, D. \ Carterette, B. APACrefauthors \ 2010 07 . An Analysis of Assessor Behavior in Crowdsourced Preference Judgments An analysis of assessor behavior in crowdsourced preference judgments

    2010

  60. [69]

    write newline

    " write newline " cite write " FUNCTION editor.postfix editor num.names #1 > "( )" "( )" if FUNCTION editor.trans.postfix editor num.names #1 > "( )" "( )" if FUNCTION trans.postfix translator num.names #1 > "( )" "( )" if FUNCTION authors.editors.reflist.apa5 'field := 'dot :...

  61. [70]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year eprint archive archivePrefix primaryClass adsurl adsnote version label INTEGERS o...

  62. [71]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or...

  63. [72]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year archivePrefix primaryClass adsurl adsnote version label extra.labe...

  64. [73]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap dupl...

  65. [74]

    Krit Gangwanpongpun

    @inproceedings kour2014real, title= Real-time segmentation of on-line handwritten arabic script , author= Kour, George and Saabne, Raid , booktitle= Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on , pages= 417--422 , year= 2014 , organizatio...

    2014 arXiv

  66. [75]

    write newline

    " write newline "" before.all 'output.state := FUNCTION output.doi doi empty skip "doi:" doi * "" * output if FUNCTION format.archive archivePrefix empty "" archivePrefix ":" * if FUNCTION format.primaryClass primaryClass empty "" " [" primaryClass * "] " * if FUNCTION format....

  67. [76]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter edition editor eprint howpublished institution journal key keywords month note number organization pages publisher school series title type url doi volume year archivePrefix primaryClass eid adsurl adsnote version label INTEGERS o...

  68. [77]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  69. [78]

    Available from:

    ENTRY address assignee author booktitle chapter cartographer day edition editor howpublished institution inventor journal key keywords month note number organization pages part publisher school series title type volume word year eprint doi url lastchecked updated archive archi...

  70. [79]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.