Pith. sign in

REVIEW 4 major objections 6 minor 33 references

SpEmoC: A Balanced Speaker-Segment Multimodal Emotion Benchmark

T0 review · 4 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SpEmoC: a balanced 30k-clip multimodal emotion benchmark with movie-level splits makes Fear and Disgust learnable and stabilizes cross-dataset recognition.

desk verdict New balanced MER dataset worth knowing about, but the paper's central transfer claim is contradicted by its own cross-dataset tables. read the letter →

arxiv 2607.18109 v1 pith:NUITYTVZ submitted 2026-07-20 cs.CV

Sania Bano , Shahzad Ahmad , Santosh Kumar Vipparthi , Sukalpa Chanda , Subrahmanyam Murala This is my paper
classification cs.CV
keywords SpEmoCmultimodalemotionrecognitionclass-balanceddatasetmovie-levelsplitcross-datasetgeneralizationhumanvalidationminorityemotionsaudio-textfusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SpEmoC, a multimodal emotion-recognition benchmark of 30,000 human-validated speaking clips cut from 3,100 movies and TV series, balanced across seven emotions. Its central claim is that two dataset-design choices—class balance and strict movie/series-level splits that keep any movie entirely within train, validation, or test—make trained models more stable across emotion categories and better at recognizing minority emotions like Fear and Disgust than existing benchmarks such as MELD and CAER. The authors show this through in-domain and cross-dataset experiments: models trained on SpEmoC retain nontrivial performance on Fear and Disgust when tested on other datasets, shrink the gap between weighted and macro F1, and provide better starting points for low-data fine-tuning. If correct, the work demonstrates that dataset construction decisions, not just model architecture, determine whether multimodal emotion recognition can generalize.

What carries the argument

The central object is the dataset itself, built by a scalable pipeline: Whisper-based dialogue segmentation, synchronized visual/audio/text extraction with a 0.1-second duration check, logit-based multimodal fusion of text and audio with a KL penalty, threshold filtering on neutral scores (θt=θa=0.05) plus YOLOv8 face-presence (≥90% frames), and human validation. The movie-level split is the key design mechanism for preventing content leakage. What carries the argument is the contrast between this balanced, split-controlled benchmark and the imbalanced, character-overlapping MELD/CAER datasets.

What would settle it

Annotate a random sample of the 256,544 clips discarded by the neutral/face filters and compare their human labels to the filter's predictions; if many discarded clips are genuinely emotional (or the kept set is skewed toward strong, frontal expressions), the benchmark's balance is an artifact of the filter rather than of natural emotion distribution.

Watch

Extended reading notes

Core claim

SpEmoC consists of 30,000 refined clips with synchronized video, audio, and text, labeled by a hybrid pipeline: DistilRoBERTa and Wav2Vec 2.0 produce logits that are fused via logit-based fusion with a KL-divergence penalty; clips then pass neutral-score thresholds and face-presence rules before being human-validated by multiple annotators with majority voting (Fleiss κ=0.62). The dataset is split at the movie/series level to prevent content leakage. In-domain and cross-dataset experiments with five multimodal models show consistently higher Macro-F1 and near-zero imbalance gaps on SpEmoC, and SpEmoC-trained models retain minority-emotion F1 under transfer to MELD and CAER.

Load-bearing premise

The labels are treated as unbiased ground truth for natural emotion, but human annotators only saw clips that had already passed automated neutral-score thresholds and a face-presence rule, so any bias in those filters is baked into the 30,000 labels before a human sees them.

Editorial extensions

If this is right

  • Models trained on SpEmoC generalize to MELD and CAER with nonzero F1 on Fear and Disgust, whereas models trained on those benchmarks often collapse to near-zero on those classes.
  • SpEmoC pretraining improves downstream fine-tuning on MELD and CAER, with the largest gains in the extreme low-data (10%) regime.
  • The imbalance gap between weighted and macro F1 drops to ≤0.65 on SpEmoC across five architectures, indicating much more uniform per-class performance.
  • Cross-dataset transfer from SpEmoC degrades more gracefully than transfers between MELD and CAER, avoiding brittle failures on rare emotions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The neutral-score pre-filter and face-presence rule mean the 30k clips are a curated subset of expressive, frontal visible speech; a model trained on SpEmoC may not learn neutral or profile/no-face expressions, so claims about 'emotion recognition' should be scoped to expressive frontal speech.
  • The movie-level split is a transferable design fix: any video-derived dataset with repeated characters or scenes could adopt it, and the evidence here suggests leakage inflates standard benchmarks' apparent generalization.
  • A testable extension is to use SpEmoC as pretraining for a spontaneous, non-acted conversation corpus to see if balanced supervision helps beyond scripted media.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SpEmoC, a 30,000-clip, near-class-balanced multimodal emotion benchmark curated from 306,544 speaking segments extracted from 3,100 movies and TV series. Each clip includes synchronized video, audio, and text, with labels obtained by a hybrid pipeline that fuses DistilRoBERTa and Wav2Vec 2.0 logits and then applies human validation. The authors enforce movie/series-level splits to prevent content leakage and benchmark five multimodal models in in-domain, cross-dataset, low-data, and modality-transfer settings. The central claim is that balanced supervision and careful splitting yield more stable class-wise emotion recognition and better transfer, particularly for minority emotions such as Fear and Disgust.

Significance. If validated, SpEmoC would be a valuable community resource: it is substantially larger than MELD/CAER in raw clip count, offers three synchronized modalities, uses strict content-based splits, and attempts to correct the chronic underrepresentation of Fear and Disgust. The authors should be credited for the scale of the collection effort and for making split hygiene an explicit design goal. However, the paper's causal claims about balanced data producing transferable stability are not established by the current evidence; the imbalance-gap metric is partly an artifact of balanced test priors, and the cross-dataset tables are inconsistent. The annotation pipeline also introduces a model-selection bias that threatens the claim that the human-validated labels are unbiased ground truth. With revised analysis and more cautious claims, the dataset could still be a useful benchmark, but the current manuscript needs substantive work.

major comments (4)
  1. [§4.1, Table 4; metric definition in §4] The near-zero imbalance gap on SpEmoC is largely a mathematical artifact of balanced test priors, not evidence of balanced learned representations. With W-F1 = Σ_c (n_c/N) F1_c and Macro-F1 = (1/7) Σ_c F1_c, any test set with exactly equal class priors gives W-F1 = Macro-F1 for any predictor, so Δ≈0 is guaranteed. Figure 2 shows near-equal SpEmoC test sizes; thus Δ≤0.65 in Table 4 does not demonstrate balanced class-wise learning. The text's statement that Δ 'quantifies bias toward majority categories' is invalid under balanced priors.
  2. [§4.2, Tables 5–8] The cross-dataset results do not support the abstract's claim that balanced training improves transfer stability. For TCL-MAP, Δ on MELD is 17.90 when trained on SpEmoC vs 17.22 when trained on MELD (Table 5); for MISA, 8.09 vs 6.48 (Table 6); for EMOE, 17.77 vs 20.85 (Table 8). Only MulT shows a large reduction (4.17 vs 20.21). Minority F1 is often not improved: TCL-MAP Fear is 3.96 after SpEmoC training vs 17.07 after MELD training; EMOE Fear is 6.06 in both. The conclusion of 'consistently more balanced class-wise performance' is an overstatement.
  3. [§3.3, Figure 5] The human validation stage is applied only to a candidate pool preselected by the same automated models whose neutral scores define the filtering thresholds: clips are retained only if θt=θa=0.05 on DistilRoBERTa/Wav2Vec neutral logits and if a face is present in ≥90% of frames. Any clip the automated models judge as neutral is removed before human review; only ~15% neutral is later kept. Consequently, the final 'neutral' class is not a natural sample of conversational neutrality, and the labels inherit model-family bias. Comparisons with MELD/CAER therefore conflate labeling-pipeline differences with emotion-understanding differences. This is not an internal inconsistency, but it weakens the benchmark's validity as unbiased ground truth and the causal interpretation of transfer results.
  4. [§4.3, Table 9] The low-data transfer claims lack the essential control. Comparing 'Baseline' with '+SpEmoC Pretraining' varies both the amount of source data and its class balance. To attribute gains to balanced supervision, the control should pretrain on an equally large imbalanced dataset (e.g., the 306k unfiltered pool or MELD) under the same protocol. Because Tables 5–8 show MELD training sometimes transfers as well or better, the current design cannot separate dataset balance from dataset size or source domain.
minor comments (6)
  1. [§3.2, Eq. (7)] The KL penalty weight λ=0.5 is fixed with no sensitivity analysis. Since this fusion is the core of the labeling pipeline, a short robustness check over λ would strengthen the method.
  2. [§3.1, Eq. (1)] The notation m in Eq. (1) is not indexed by k, although it represents the number of valid clips per video. Please write m_k to avoid ambiguity.
  3. [Table 4] The caption says 'Bold: Best Methods', but no bold formatting is visible. Also, per-class F1 values of 0.0 for some CAER entries with a positive W-F1 are plausible but deserve a one-sentence explanation.
  4. [Table 9] Table 9 is difficult to parse: entries like '57.13 / 31.40' do not clearly indicate which number belongs to MELD versus CAER, or whether the columns report W-F1, Macro-F1, or Δ. Please restructure the table or expand the caption.
  5. [§3.2] The 'fine-tuned DistilRoBERTa model' is not specified: no training data or checkpoint is given. If this model was fine-tuned on MELD or another emotion dataset, the pseudo-labeling process may have circularity with the downstream evaluation targets; this should be clarified.
  6. [Project Page] The project page entry says 'available at the following link arXiv:2607.18109v1', which is not a URL. Please provide the actual dataset release link or a data availability statement.

Circularity Check

1 steps flagged · score 4.0 of 10

In-domain imbalance-gap evidence is partly circular by construction; cross-dataset evidence remains independent but mixed.

  1. self definitional [Sec. 4.2, 'Imbalance-gap interpretation' (Tables 5-8); also Sec. 4.1 in-domain discussion]
    "In contrast, SpEmoC training yields much smaller gaps across architectures-TCL-MAP (Δ= 0.33, Table 5), EMOE (Δ= 0.34, Table 8), MulT (Δ= 0.65, Table 7), and MISA (Δ= 3.28, Table 6) reflecting more balanced class-wise recognition."

    The quoted Δ values are the SpEmoC→SpEmoC (in-domain) rows, not cross-dataset rows. The SpEmoC test set was intentionally constructed to be near-balanced (Sec. 3.3: 'resulting 30,000 refined samples that maintain balance across all seven emotions'; Fig. 2(a) class range ≈7.8-21%). When test priors are near-uniform, Weighted-F1 and Macro-F1 are nearly equal even for a random/constant predictor, so Δ≈0 is forced by the test-label distribution, not by learned balanced representations. The section presents these in-domain numbers as evidence for cross-dataset 'more balanced class-wise recognition,' but the same tables show SpEmoC→MELD Δ=17.90 (TCL-MAP), 8.09 (MISA), 17.77 (EMOE) vs. MELD→MELD 17.22, 6.48, 20.85. Thus a load-bearing piece of evidence reduces by construction to the dataset's own

full rationale

The clearest circular element is the use of the in-domain imbalance gap as evidence for the balanced-supervision claim. SpEmoC's test split is near-balanced by design, and on a near-uniform test prior W-F1 and Macro-F1 coincide for almost any predictor, so a small Δ does not indicate balanced learned representations or transferable stability. The paper quotes these in-domain Δs in the cross-dataset section while the actual SpEmoC→MELD Δs in the same tables are as large as or larger than MELD→MELD for TCL-MAP, MISA, and EMOE. That specific inference is circular by construction. However, the cross-dataset transfer experiments (Tables 5-8), low-data fine-tuning (Table 9), and pretraining/fine-tuning (Table 10) are external evidence not built into the dataset definition; the central claim is therefore not wholly reduced. The annotation-pipeline concern (DistilRoBERTa/Wav2Vec pre-filtering before human validation) is a validity threat rather than a derivation-cycle issue, since final labels are human-assigned. There is no self-citation chain or imported uniqueness theorem; all method citations are external. Score 4 reflects partial, not global, circularity.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim relies on a chain of domain assumptions about emotions, acted video, ASR segmentation, and model-based filtering. The only hand-tuned numeric inputs are the fusion weight, filtering thresholds, neutral retention proportion, and sync tolerance. No new physical or formal entity is postulated.

free parameters (5)
  • λ (KL penalty weight) = 0.5
    Eq. 7 in Sec 3.2; controls logit fusion of text/audio and shapes pseudo-labels; chosen without reported sensitivity analysis.
  • θf (face-presence threshold) = 0.9
    Sec 3.3; clips kept only if a face appears in ≥90% of frames; threshold set by manual experimentation.
  • θt = θa (neutral logit thresholds) = 0.05
    Sec 3.3; clips with neutral probability below 0.05 are kept; determined through manual experimentation and performance analysis, directly shaping class balance.
  • neutral retention proportion = ~15%
    Sec 3.3; a small portion of neutral clips is retained for balance; exact target distribution is hand-chosen.
  • ϵ (sync tolerance) = 0.1 s
    Eq. 2; allowable audio-visual duration deviation; arbitrary tolerance affecting which clips survive.
assumptions (6)
  • domain assumption Ekman's seven basic emotions are the correct discrete label space.
    Sec 3.2 taxonomy; all labels and cross-dataset comparisons depend on this vocabulary.
  • domain assumption Each speaking segment has a single dominant emotion.
    Sec 3.1; no speaker diarization is performed; annotation asks for the dominant emotional expression.
  • domain assumption Whisper word-level timestamps correctly identify utterance boundaries.
    Sec 3.1a; segmentation quality rests on ASR timestamps and terminal punctuation.
  • domain assumption Logit-based fusion with a uniform prior and softmax likelihoods produces valid pseudo-labels.
    Sec 3.2 Eqs. 5-7; a modeling choice, not derived from data.
  • domain assumption Movie/TV acted performances are representative of natural conversational emotion.
    Sec 1 and Sec 3; the dataset claims 'natural speaking segments' but the source is scripted content.
  • domain assumption Class balance improves generalization.
    Sec 4; this is the hypothesis tested by the experiments, not an independent fact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SpEmoC: A Balanced Speaker-Segment Multimodal Emotion Benchmark." pith.science (2026). https://pith.science/paper/NUITYTVZ

@misc{pith2026260718109,
  author       = {Pith},
  title        = {Pith review of: SpEmoC: A Balanced Speaker-Segment Multimodal Emotion Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NUITYTVZ}},
  note         = {Machine review of arXiv:2607.18109}
}
read the original abstract

Understanding human emotions in spoken conversations is a key challenge in affective computing, with applications in empathetic AI, human computer interaction, and mental health monitoring. However, existing datasets vary in scale, emotion distribution, modality alignment, and data partitioning strategies, which can influence reliable cross-dataset generalization and minority-emotion modeling. We introduce SpEmoC a Speaking segment Emotion for Conversations comprising 306,544 raw clips from 3,100 English language movies and TV series. From these, 30,000 high quality, class balanced clips are curated, featuring synchronized visual, audio, and textual modalities annotated for seven emotions through a hybrid pipeline that integrates pretrained models with human validation. SpEmoC uses strict movie- and series-level splits to prevent content overlap between split sets, allowing more reliable evaluation of model generalization. The dataset also maintains a near-balanced distribution across seven emotions, including minority classes such as Fear and Disgust, which supports more balanced learning across categories. Extensive experiments, including in-domain benchmarking, cross-dataset transfer, low-data training, class-imbalance analysis, and modality transfer show that balanced data and careful splitting lead to more stable performance across emotions when models are evaluated on other datasets. These results highlight the importance of dataset design for robust and transferable multimodal emotion recognition.

Figures

Figures reproduced from arXiv: 2607.18109 by the authors.

Figure 1
Figure 1. Overview of SpEmoC construction and evaluation. Multimodal filtering and expert validation refine 306K speaking clips into 30K balanced clips across seven emo￾tions for diverse benchmarking. Abstract. Understanding human emotions in spoken conversations is a key challenge in affective computing, with applications in empathetic AI, human computer interaction, and mental health monitoring. How￾ever, existing datasets … view at source ↗
Figure 2
Figure 2. (a) Emotion class distribution across train, validation, and test splits for SpE￾moC, MELD [20], and CAER [13]. SpEmoC exhibits a more balanced distribution across all seven emotions, while MELD and CAER are dominated by the Neutral class. (b) Class-wise baseline performance on SpEmoC, showing more consistent F1-scores due to the balanced emotion distribution. cross-dataset evaluations on MELD and CAER. These experi… view at source ↗
Figure 3
Figure 3. Examples from SpEmoC showing variation in genre, lighting, color, and ex￾pression. Each row displays 8 sampled frames from a distinct clip. 3.1 Data Collection and Processing Pipeline We develop a scalable multi-stage pipeline that converts long-form videos into synchronized video, audio, and text segments ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: SpEmoC construction and multimodal labeling pipeline. Movies are segmented into short dialogue-based clips (3-6 s), from which synchronized text, audio, and visual streams are extracted. Text and audio logits are fused to obtain clip-level emotion labels, while YOLOv8 …
Figure 5
Figure 5. Figure 5: Emotion class distribution before (blue) and after filtering (green). The initial 306K clips were dominated by the neutral class, while fear was underrepresented. A two-step filtering process was applied: threshold-based filtering, followed by human validation to remov…
Figure 6
Figure 6. Figure 6: Demographic distribution of the SpEmoC dataset across age groups, ethnicity, and gender, illustrating its diversity and balanced population coverage. (20%) sets. This ensures no overlap in scenes, actors, or dialogues between splits, enabling robust real-world evaluati…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 2 linked inside Pith

  1. [1]

    In: Proceedings of the 26th ACM interna- tional conference on Multimedia

    Albanie, S., Nagrani, A., Vedaldi, A., Zisserman, A.: Emotion recognition in speech using cross-modal transfer in the wild. In: Proceedings of the 26th ACM interna- tional conference on Multimedia. pp. 292–301 (2018)

    2018

  2. [2]

    Advances in neural information processing systems33, 12449–12460 (2020)

    Baevski, A., Zhou, Y., Mohamed, A., Auli, M.: wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems33, 12449–12460 (2020)

    2020

  3. [3]

    Language resources and evaluation42, 335–359 (2008)

    Busso, C., Bulut, M., Lee, C.C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J.N., Lee, S., Narayanan, S.S.: Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation42, 335–359 (2008)

    2008

  4. [4]

    Advances in Neural Information Processing Systems37, 110805– 110853 (2024)

    Cheng, Z., Cheng, Z.Q., He, J.Y., Wang, K., Lin, Y., Lian, Z., Peng, X., Haupt- mann, A.: Emotion-llama: Multimodal emotion recognition and reasoning with in- struction tuning. Advances in Neural Information Processing Systems37, 110805– 110853 (2024)

    2024

  5. [5]

    Cognition & emotion6(3-4), 169–200 (1992)

    Ekman, P.: An argument for basic emotions. Cognition & emotion6(3-4), 169–200 (1992)

    1992

  6. [6]

    Pattern recognition44(3), 572– 587 (2011)

    El Ayadi, M., Kamel, M.S., Karray, F.: Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern recognition44(3), 572– 587 (2011)

    2011

  7. [7]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference

    Fang, Y., Huang, W., Wan, G., Su, K., Ye, M.: Emoe: Modality-specific enhanced dynamic emotion experts. In: Proceedings of the Computer Vision and Pattern Recognition Conference. pp. 14314–14324 (2025)

    2025

  8. [8]

    In: Proceedings of the Thirteenth Language Resources and Evaluation Conference

    Feng, S., Lubis, N., Geishauser, C., Lin, H.c., Heck, M., van Niekerk, C., Gasic, M.: Emowoz: A large-scale corpus and labelling scheme for emotion recognition in task- oriented dialogue systems. In: Proceedings of the Thirteenth Language Resources and Evaluation Conference. pp. 4096–4113 (2022)

    2022

Show all 33 references
  1. [9]

    FFmpeg Developers: Ffmpeg.https://ffmpeg.org(2025), accessed: 29 June 2026

    2025

  2. [10]

    arXiv preprint arXiv:2403.02167 (2024)

    Gómez-Zaragozá, L., del Amor, R., Castro-Bleda, M.J., Naranjo, V., Raya, M.A., Marín-Morales, J.: Emovome: A dataset for emotion recognition in spontaneous real-life speech. arXiv preprint arXiv:2403.02167 (2024)

    2024 arXiv

  3. [11]

    In: Proceedings of the 28th ACM international conference on multimedia

    Hazarika, D., Zimmermann, R., Poria, S.: Misa: Modality-invariant and-specific representations for multimodal sentiment analysis. In: Proceedings of the 28th ACM international conference on multimedia. pp. 1122–1131 (2020)

    2020

  4. [12]

    biometrics pp

    Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. biometrics pp. 159–174 (1977)

    1977

  5. [13]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Lee, J., Kim, S., Kim, S., Park, J., Sohn, K.: Context-aware emotion recognition networks. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 10143–10152 (2019)

    2019

  6. [14]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Li, S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learn- ing for expression recognition in the wild. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2852–2861 (2017)

    2017

  7. [15]

    PLOS ONE13(5), e0196391 (2018)

    Livingstone, S.R., Russo, F.A.: The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expres- sions in north american english. PLOS ONE13(5), e0196391 (2018)

    2018

  8. [16]

    In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG)

    Lucey, P., Cohn, J.F., Prkachin, K.M., Solomon, P.E., Matthews, I.: Painful data: The unbc-mcmaster shoulder pain expression archive database. In: 2011 IEEE International Conference on Automatic Face & Gesture Recognition (FG). pp. 57–64 (2011) 16 S. Bano et al

    2011

  9. [17]

    Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI) pp

    Nojavanasghari, B., Hughes, T., Morency, L.P.: Emoreact: A multimodal approach and dataset for recognizing emotional responses in children. Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI) pp. 137– 144 (2016)

    2016

  10. [18]

    MIT Press, Cambridge, MA (1997),https: //direct.mit.edu/books/monograph/4296/Affective-Computing, accessed: 29 June 2026

    Picard, R.W.: Affective Computing. MIT Press, Cambridge, MA (1997),https: //direct.mit.edu/books/monograph/4296/Affective-Computing, accessed: 29 June 2026

    1997

  11. [19]

    Information Fusion37, 98–125 (2017)

    Poria, S., Cambria, E., Bajpai, R., Hussain, A.: A review of affective comput- ing: From unimodal analysis to multimodal fusion. Information Fusion37, 98–125 (2017)

    2017

  12. [20]

    In: Proceedings of the 57th annual meeting of the association for computational lin- guistics

    Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: Meld: A multimodal multi-party dataset for emotion recognition in conversations. In: Proceedings of the 57th annual meeting of the association for computational lin- guistics. pp. 527–536 (2019)

    2019

  13. [21]

    In: International conference on machine learning

    Radford,A.,Kim,J.W.,Xu,T.,Brockman,G.,McLeavey,C.,Sutskever,I.:Robust speech recognition via large-scale weak supervision. In: International conference on machine learning. pp. 28492–28518. PMLR (2023)

    2023

  14. [22]

    arXiv preprint arXiv:1910.01108 (2019)

    Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    1910 arXiv

  15. [23]

    Speech communication53(9-10), 1062–1087 (2011)

    Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge. Speech communication53(9-10), 1062–1087 (2011)

    2011

  16. [24]

    In: Findings of the Association for Computational Linguistics: ACL

    Sun, H., Zhao, J., Wang, X., Zhao, S., Zhou, J., Wang, H., Yang, X., Wang, Y., Lin, Y.: Emotiontalk: An interactive chinese multimodal emotion dataset with rich annotations. In: Findings of the Association for Computational Linguistics: ACL

  17. [25]

    In: Pro- ceedingsofthe57thannualmeetingoftheassociationforcomputationallinguistics

    Tsai, Y.H.H., Bai, S., Liang, P.P., Kolter, J.Z., Morency, L.P., Salakhutdinov, R.: Multimodal transformer for unaligned multimodal language sequences. In: Pro- ceedingsofthe57thannualmeetingoftheassociationforcomputationallinguistics. pp. 6558–6569 (2019)

    2019

  18. [26]

    In: 2024 International Conference on Ad- vances in Data Engineering and Intelligent Computing Systems (ADICS)

    Varghese, R., Sambath, M.: Yolov8: A novel object detection algorithm with en- hanced performance and robustness. In: 2024 International Conference on Ad- vances in Data Engineering and Intelligent Computing Systems (ADICS). pp. 1–6. IEEE (2024)

    2024

  19. [27]

    IEEE Intelligent Systems28(3), 46–53 (2013)

    Wöllmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., Morency, L.P.: Youtube movie reviews: Sentiment analysis in an audio-visual context. IEEE Intelligent Systems28(3), 46–53 (2013)

    2013

  20. [28]

    In: Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers)

    Zadeh, A.B., Liang, P.P., Poria, S., Cambria, E., Morency, L.P.: Multimodal lan- guage analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. In: Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long ...

    2018

  21. [29]

    In: Proceedings of the European conference on computer vision (ECCV)

    Zeng, J., Shan, S., Chen, X.: Facial expression recognition with inconsistently an- notated datasets. In: Proceedings of the European conference on computer vision (ECCV). pp. 222–237 (2018)

    2018

  22. [30]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Zhang, S., Pan, Y., Wang, J.Z.: Learning emotion representations from verbal and nonverbal communication. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18993–19004 (2023)

    2023

  23. [31]

    In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

    Zhao, J., Zhang, T., Hu, J., Liu, Y., Jin, Q., Wang, X., Li, H.: M3ed: Multi-modal multi-scene multi-label emotional dialogue database. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 5699–5710 (2022) SpE...

    2022

  24. [32]

    In: Proceedings of the AAAI conference on artificial intelligence

    Zhou, Q., Xu, H., Li, H., Zhang, H., Zhang, X., Wang, Y., Gao, K.: Token-level contrastive learning with modality-aware prompting for multimodal intent recog- nition. In: Proceedings of the AAAI conference on artificial intelligence. vol. 38, pp. 17114–17122 (2024)

    2024

  25. [2026]

    9054–9071 (2026)

    pp. 9054–9071 (2026)

    2026

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.