Pith. sign in

REVIEW 5 major objections 7 minor 87 references

MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper introduces a broad open benchmark that scores language and vision models on sentiment and emotion tasks across 35 languages and three modalities, then uses it to compare 20 models.

desk verdict A genuinely useful new benchmark resource, but the evaluative claims are fragile: single-run, single-prompt, no baselines, and some conclusions confounded by model family. read the letter →

arxiv 2505.24423 v1 pith:ZSLK3GGR submitted 2025-05-30 cs.CL

Zhiwei Liu , Lingfei Qian , Qianqian Xie , Jimin Huang , Kailai Yang , Sophia Ananiadou This is my paper
classification cs.CL
keywords affectiveanalysissentimentemotiondetectionmultilingualbenchmarkmultimodalevaluationlargelanguagemodelsvision-languageinstructiontuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to give the field a single reusable yardstick for how well large language models and vision-language models handle affective analysis: sentiment polarity, sentiment intensity, emotion classification, and emotion intensity. It assembles MMAFFBen from existing open datasets, reformats them into instruction-style prompts covering text, image, and video in 35 languages, and evaluates 20 models on the result. Beyond the benchmark, it builds an instruction-tuning dataset and two fine-tuned models, then reports that the fine-tuned models reach the best overall multimodal scores while a general-purpose commercial model leads among untuned systems. A sympathetic reader would take the contribution as a measurement instrument plus a first systematic map of where current models stand, not as a claim about what any single model fundamentally can or cannot do.

What carries the argument

The load-bearing object is MMAFFBen itself: 14 public datasets reformatted through a single instruction template, namely 'Task: [task description]' followed by 'Content: [raw text or transcription]', with intensity scores normalized to $[-1,1]$ for sentiment and $[0,1]$ for emotion, and label sets aligned to a small number of emotion taxonomies. The template does the work of turning heterogeneous datasets into a zero-shot instruction-following evaluation, and the normalized real-valued intensity tasks let the benchmark measure graded affective judgment rather than only discrete labels.

What would settle it

Take any MMAFFBen subset, paraphrase the task instruction into two or three alternative wordings with the same labels, rerun the same models, and check whether the model ordering holds; if orderings shift materially, single-template scores cannot be read as affective capability.

Watch

Extended reading notes

Core claim

The central discovery claimed is that a multilingual, multimodal affective-analysis testbed is feasible by reusing and normalizing existing datasets, and that on such a testbed task-specific instruction tuning still improves affective performance substantially, whereas among zero-shot untuned models a commercial generalist model outperforms all open alternatives. The paper also reports that multimodal models mostly preserve text-only affective ability, that adding video experience appears to help on static image tasks, and that almost all models score higher in English than in other languages.

Load-bearing premise

The rankings rest on the premise that one fixed prompt template per dataset is a fair and stable way to measure affective ability, so score differences reflect affect understanding rather than phrasing or formatting.

Editorial extensions

If this is right

  • Any future language or vision model can be scored on the same four affective tasks and 35-language grid, making cross-model, cross-language, and cross-modality comparisons directly possible.
  • The benchmark's results imply that instruction tuning on affective data is still a reliable way to lift multimodal models, since the two tuned models beat their untuned bases on most text, image, and video subsets.
  • General-purpose closed models are the strongest zero-shot affect systems tested, which sets the bar open-weight models would need to meet.
  • Because most models do better in English, the results point to multilingual affective data as a concrete bottleneck for current language and vision models.
  • Multimodal training does not appear to come at the cost of text affective skill, and video-aware models even gain on image emotion tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If prompt sensitivity is as large as it often is for instruction-tuned models, the single template per dataset could make the rankings an artifact of phrasing; a paraphrase-swap test would show whether MMAFFBen measures affect understanding or template tolerance.
  • The video datasets enter as text transcription plus video frames, not as audio, so 'video' here tests visual-plus-text affect and leaves audio prosody untested despite the source datasets being audio-rich.
  • The reported overall averages mix Pearson correlations and macro F1 scores across tasks, so aggregate rankings could differ if one metric were replaced by another; per-task tables remain the safer comparison.
  • The tuned models' edge on the Chinese facial dataset suggests their backbone may carry an Eastern-face advantage, a testable bias hypothesis for broader facial affect benchmarks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper introduces MMAFFBen, a proposed open-source benchmark for multilingual and multimodal affective analysis, covering text, image, and video modalities across 35 languages and four tasks: sentiment polarity, sentiment intensity, emotion classification, and emotion intensity. The authors also construct an instruction-tuning dataset MMAFFIn, fine-tune Qwen2.5-VL-based MMAFFLM-3B and MMAFFLM-7B on it, and report evaluations of 20 models including GPT-4o-mini, EmoLlama, Llama, LLaVA, Qwen2.5-VL, and InternVL2.5 variants. Results are presented in Tables 4 and 5 with per-dataset and aggregate metrics.

Significance. If made robust, the benchmark is a potentially valuable community resource: it addresses a real gap in multilingual, multimodal affective evaluation, provides a broad task coverage (regression and classification, text/image/video), reports per-language breakdowns, and ships open data and model links. The paper also names concrete strengths such as a public GitHub and HuggingFace release, and explicitly documents prompts and metrics in Table 9. However, the evaluative conclusions as currently presented are undermined by a train/test overlap for the proposed MMAFFLM models, a single-prompt single-run protocol with no baselines, and cross-family comparisons that do not support the causal claims made in Sections 5.2 and 5.3.

major comments (5)
  1. [Section 2.3 and Section 3 (Table 3); Section 5] The MMAFFLM models are trained on MMAFFIn, which is built from the remaining (training/validation) portions of the same datasets whose test portions form MMAFFBen. For datasets with an existing split, Section 2.3 says 'we adopt the default test set'; for unsplit datasets, 20% is randomly sampled for MMAFFBen. Section 3 then says MMAFFIn uses 'the remaining data of each dataset.' Consequently, MMAFFLM-7B's headline result (Table 5, 'All' column: 49.0 vs GPT-4o-mini 48.7, and Section 5.4's 'achieves the best overall performance across modalities') is an in-distribution, supervised result, not a zero-shot one, and is not comparable to the other models in the systematic comparison claimed in Section 4 ('zero-shot performance of 20 representative...'). The authors should either exclude MMAFFLM from the zero-shot tables, clearly label it as fine-tuned on the benchmark's own training splits, or provide an analysis showing that leakage does not explain the gap.
  2. [Section 2.3 and Table 9; Section 5] Every task uses exactly one prompt template, one label ordering, and one output format, and every reported number is a single run with no seeds, no error bars, and no significance tests. For affective tasks, model outputs can be sensitive to label wording, label order, and output-format compliance. The claims that GPT-4o-mini 'consistently achieves the highest' (Section 5.1) and 'significantly outperforms' all open-source models (Section 5.4) are therefore not statistically supported, and the benchmark's stability as a measurement instrument remains untested. Please add multiple prompt variants, repeated runs with different temperatures/seeds, and random/majority baselines, and report variance or confidence intervals.
  3. [Section 5.2 and Section 5.3] The statements that 'LM-TI models underperform compared to LM-TIV models' and 'LM-TV models generally underperform compared to LM-TIV models' compare different model families, base architectures, parameter counts, and training corpora. The later inference that 'the integration of video data enhances sentiment analysis performance even on static image tasks' is a causal claim not supported by these uncontrolled comparisons. Please rephrase these as descriptive observations about the specific evaluated model sets, or add controlled comparisons (e.g., the same backbone with and without video training).
  4. [Section 2.4 and Tables 4-5] The evaluation protocol does not describe how raw model outputs are converted to reported metrics: how is the 'number. emotion' format parsed, what happens with invalid or out-of-range outputs, what decoding parameters (temperature, max tokens) are used, and how are regression outputs normalized for pcc? Without this information the numbers in Tables 4 and 5 are not reproducible. Please specify the complete inference pipeline and ideally release the evaluation code.
  5. [Section 5.1, Table 4] EmoLlama-chat-7B is fine-tuned on SemEval training data and evaluated on SemEval test data, as the paper itself notes ('likely due to being fine-tuned on its corresponding training data'). Presenting this result in the same zero-shot comparison table without a clear marker is misleading. Please separate supervised and zero-shot results or explicitly flag them in the table.
minor comments (7)
  1. [Table 2 vs. Section 2.3] The video dataset is spelled 'SANSEMO' in Table 2 but 'SAMSEMO' in the text and reference [5]; please unify the spelling.
  2. [Section 5.4] 'SAMSEMO, MMS, and XAD datasets' should be 'SAMSEMO, MMS, and XED datasets'.
  3. [Section 2.3] 'polity labels' should be 'polarity labels'.
  4. [Section 5.3] 'whthin' is a typo for 'within'.
  5. [Table 9, CFAPS-EI] The prompt template fixes 'Emotion E: surprise,' but CFAPS includes intensity ratings for multiple emotion categories; please clarify whether the emotion is provided per sample or is indeed fixed, and how the model knows which emotion to rate.
  6. [Table 5, 'All' column] The 'All' column is ambiguous for models that do not support all modalities; please state explicitly which tasks and modalities are averaged for each model, since the text-only models show dashes while the LM-TIV models receive an aggregate.
  7. [Section 2.3] For SemEval-2018, the footnote in Section 5.1 notes license limitations on the test set; please state in the benchmark description itself that the SemEval test data is not redistributed and that results are reported separately due to this restriction.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark is an external-data aggregation and the model comparisons use held-out test splits.

full rationale

MMAFFBen is constructed by taking existing annotated datasets and reformatting them into instruction-response pairs (Section 2.1 and Table 9); no benchmark-derived quantity is fed back into the construction or into the claimed predictions. The central contribution is the aggregated evaluation suite, and its validity rests on external ground-truth datasets and on the behavior of externally trained models, not on equations fitted to the benchmark's own outputs. The only apparent overlap is the inclusion of EmoLlama-chat-7b, a model from the authors' prior work [33]; however, the paper explicitly flags that its strong SemEval_en result is "likely due to being fine-tuned on its corresponding training data" (Section 5.1), so this is an admitted caveat rather than an imported premise used to prove the benchmark's value. Likewise, MMAFFIn is built from the non-test portions of the same source datasets, and MMAFFLM models are evaluated on the held-out test splits, which is a standard supervised evaluation protocol, not a circular prediction. No uniqueness theorem or ansatz is imported from prior work, and no known result is merely renamed. The single-prompt, single-run evaluation design is a legitimate robustness limitation, but it does not make any claimed derivation equivalent to its inputs, so it does not constitute circularity under the specified criteria.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no fitted parameters or new theoretical entities. It relies on the validity of existing dataset annotations, the representativeness of the sampled test splits, and the fairness of the prompt template as a measure of affective capability.

assumptions (4)
  • domain assumption The labels and intensity scores in the source datasets are valid ground truth for affective analysis.
    Section 2.3 treats SemEval 2018, XED, MMS, EMOTIC, FER2013, CFAPS, SAMSEMO, CHSIMS, and MELD as reliable annotation sources for constructing test items.
  • domain assumption The single prompt template per dataset is a fair and stable probe of affective capability.
    Section 2.3 and Table 9 define the instruction format; all LMs are evaluated under exactly these prompts, but no prompt-sensitivity or formatting checks are reported.
  • domain assumption For datasets without a pre-defined split, a 20% random sample is representative of the full dataset.
    Section 2.3 states '20% of the data is randomly sampled to serve as the evaluation set', without reporting the seed or stratification, so representativeness is assumed.
  • domain assumption Cross-dataset score averaging (pcc and macro F1) is a meaningful way to rank models.
    Tables 4 and 5 average Pearson correlations and F1 scores across heterogeneous tasks, implicitly assuming these metrics are comparable; no normalization is discussed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs." pith.science (2026). https://pith.science/paper/ZSLK3GGR

@misc{pith2026250524423,
  author       = {Pith},
  title        = {Pith review of: MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZSLK3GGR}},
  note         = {Machine review of arXiv:2505.24423}
}
read the original abstract

Large language models and vision-language models (which we jointly call LMs) have transformed NLP and CV, demonstrating remarkable potential across various fields. However, their capabilities in affective analysis (i.e. sentiment analysis and emotion detection) remain underexplored. This gap is largely due to the absence of comprehensive evaluation benchmarks, and the inherent complexity of affective analysis tasks. In this paper, we introduce MMAFFBen, the first extensive open-source benchmark for multilingual multimodal affective analysis. MMAFFBen encompasses text, image, and video modalities across 35 languages, covering four key affective analysis tasks: sentiment polarity, sentiment intensity, emotion classification, and emotion intensity. Moreover, we construct the MMAFFIn dataset for fine-tuning LMs on affective analysis tasks, and further develop MMAFFLM-3b and MMAFFLM-7b based on it. We evaluate various representative LMs, including GPT-4o-mini, providing a systematic comparison of their affective understanding capabilities. This project is available at https://github.com/lzw108/MMAFFBen.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

87 extracted references · 57 canonical work pages

  1. [1]

    Francisca Adoma Acheampong, Chen Wenyu, and Henry Nunoo-Mensah. 2020. Text-based emotion detection: Advances, challenges, and opportunities.Engi- neering Reports2, 7 (2020), e12189. MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

    2020

  2. [2]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  3. [3]

    Marvin M Agüero-Torales, José I Abreu Salas, and Antonio G López-Herrera

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. 2025. Qwen2. 5-VL Technical Report.arXiv preprint arXiv:2502.13923(2025)

    arXiv 2025

  5. [5]

    Paweł Bujnowski, Bartłomiej Kuzma, Bartłomiej Paziewski, Jacek Rutkowski, Joanna Marhula, Zuzanna Bordzicka, and Piotr Andruszkiewicz. 2024. SAM- SEMO: New dataset for multilingual and multimodal emotion recognition. In Proc. Interspeech 2024. 2925–2929

    2024

  6. [6]

    Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database.Language resources and evaluation42 (2008), 335–359

    2008

  7. [7]

    Ganesh Chandrasekaran, Tu N Nguyen, and Jude Hemanth D. 2021. Multimodal sentimental analysis for social media applications: A comprehensive review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery11, 5 (2021), e1415

    2021

  8. [8]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling.arXiv preprint arXiv:2412.05271(2024)

    arXiv 2024

Show all 87 references
  1. [9]

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. 2024. How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open- Source Suites.arXiv preprint arXiv:2404.16821(2024)

    2024 arXiv

  2. [10]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al . 2024. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Compu...

    2024

  3. [11]

    Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and Alexander Hauptmann. 2024. Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning.arXiv preprint arXiv:2406.11161(2024)

    2024 arXiv

  4. [12]

    M DªDevika, CªSunitha, and Amal Ganesh. 2016. Sentiment analysis: a com- parative study on different approaches.Procedia Computer Science87 (2016), 44–49

    2016

  5. [13]

    Katherine L Dixon-Gordon, Amelia Aldao, and Andres De Los Reyes. 2015. Emo- tion regulation in context: Examining the spontaneous use of strategies across emotional intensity and type of emotion.Personality and Individual Differences 86 (2015), 271–276

    2015

  6. [14]

    Paul Ekman. 1992. An argument for basic emotions.Cognition & emotion6, 3-4 (1992), 169–200

    1992

  7. [15]

    Bharat Gaind, Varun Syal, and Sneha Padgalwar. 2019. Emotion detection and analysis on social media.arXiv preprint arXiv:1901.08458(2019)

    2019 arXiv

  8. [16]

    Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, et al. 2024. Mini-internvl: A flexible- transfer pocket multimodal model with 5% parameters and 90% performance. arXiv preprint arXiv:2410.16261(2024)

    2024 arXiv

  9. [17]

    Xu Gong, Yu-Xia Huang, Yan Wang, and Yue-jia Luo. 2011. Revision of the Chinese facial affective picture system.Chinese mental health journal(2011)

    2011

  10. [18]

    Chao-Chun Hsu, Sheng-Yeh Chen, Chuan-Chun Kuo, Ting-Hao Huang, and Lun- Wei Ku. 2018. EmotionLines: An Emotion Corpus of Multi-Party Conversations. InProceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

    2018

  11. [19]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card.arXiv preprint arXiv:2410.21276(2024)

    2024 arXiv

  12. [20]

    Clayton Hutto and Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. InProceedings of the international AAAI conference on web and social media, Vol. 8. 216–225

    2014

  13. [21]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al . 2023. Mistral 7B.arXiv preprint arXiv:2310.06825(2023)

    2023 arXiv

  14. [22]

    Kaggle. 2013. FER2013 Dataset. https://www.kaggle.com/datasets/msambare/ fer2013

    2013

  15. [23]

    Nikitas Karanikolas, Eirini Manga, Nikoletta Samaridi, Eleni Tousidou, and Michael Vassilakopoulos. 2023. Large language models versus natural language understanding and generation. InProceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics. 278–290

    2023

  16. [24]

    Ronak Kosti, Jose M Alvarez, Adria Recasens, and Agata Lapedriza. 2020. Context based emotion recognition using emotic dataset.arXiv preprint arXiv:2003.13401 (2020)

    2020 arXiv

  17. [25]

    Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-NeXT: Stronger LLMs Super- charge Multimodal Capabilities in the Wild. https://llava-vl.github.io/blog/2024- 05-10-llava-next-stronger-llms/

    2024

  18. [26]

    Yifan Li, Anh Dao, Wentao Bao, Zhen Tan, Tianlong Chen, Huan Liu, and Yu Kong. 2024. Facial Affective Behavior Analysis with Instruction Tuning.European Conference on Computer Vision (ECCV) 2024(2024)

    2024

  19. [27]

    Guibiao Liao, Jiankun Li, and Xiaoqing Ye. 2024. VLM2Scene: Self-supervised image-text-LiDAR learning with foundation models for autonomous driving scene understanding. InProceedings of the AAAI Conference on Artificial Intelli- gence, Vol. 38. 3351–3359

    2024

  20. [28]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Procee...

    2014

  21. [29]

    Github link. [n. d.]. online_shopping_10_cats. https://github.com/SophonPlus/ ChineseNlpCorpus/tree/master/datasets/online_shopping_10_cats

  22. [30]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning

    2023

  23. [31]

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024. LLaVA-NeXT: Improved reasoning, OCR, and world knowl- edge. https://llava-vl.github.io/blog/2024-01-30-llava-next/

    2024

  24. [32]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruc- tion Tuning

    2023

  25. [33]

    Zhiwei Liu, Kailai Yang, Qianqian Xie, Tianlin Zhang, and Sophia Ananiadou

  26. [34]

    Laurent Mertens, Elahe Yargholi, Hans Op de Beeck, Jan Van den Stock, and Joost Vennekens. 2024. Findingemo: An image dataset for emotion recognition in the wild.Advances in Neural Information Processing Systems37 (2024), 4956–4996

    2024

  27. [35]

    AI Meta. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, cus- tomizable models.Meta AI Blog. Retrieved December20 (2024), 2024

    2024

  28. [36]

    Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko

    Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. SemEval-2018 Task 1: Affect in Tweets. InProceedings of International Workshop on Semantic Evaluation (SemEval-2018). New Orleans, LA, USA

    2018

  29. [37]

    Emily Öhman, Marc Pàmies, Kaisla Kajava, and Jörg Tiedemann. 2020. XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection. InProceed- ings of the 28th International Conference on Computational Linguistics. 6542–6552

    2020

  30. [38]

    Robert Plutchik. 1980. A general psychoevolutionary theory of emotion.Emotion: Theory, research, and experience1 (1980)

    1980

  31. [39]

    Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 527–536

    2019

  32. [40]

    Shraman Pramanick, Rama Chellappa, and Subhashini Venugopalan. 2024. Spiqa: A dataset for multimodal question answering on scientific papers.arXiv preprint arXiv:2407.09413(2024)

    2024 arXiv

  33. [41]

    Rainer Reisenzein and Martin Junge. 2024. Measuring the intensity of emotions. Frontiers in Psychology15 (2024), 1437843

    2024

  34. [42]

    Phillip Shaver, Judith Schwartz, Donald Kirson, and Cary O’connor. 1987. Emotion knowledge: further exploration of a prototype approach.Journal of personality and social psychology52, 6 (1987), 1061

    1987

  35. [43]

    Xincheng Shuai, Henghui Ding, Xingjun Ma, Rongcheng Tu, Yu-Gang Jiang, and Dacheng Tao. 2024. A survey of multimodal-guided image editing with text-to-image diffusion models.arXiv preprint arXiv:2406.14555(2024)

    2024 arXiv

  36. [44]

    SMP2020-EWECT. 2020. The Evaluation of Weibo Emotion Classification Tech- nology (SMP2020-EWECT). https://smp2020ewect.github.io/

    2020

  37. [45]

    Leimin Tian, Catherine Lai, and Johanna D Moore. 2018. Polarity and Intensity: the Two Aspects of Sentiment Analysis. InProceedings of Grand Challenge and Workshop on Human Multimodal Language (Challenge-HML). 40–47

    2018

  38. [46]

    Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, and Jifeng Dai. 2024. Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization.arXiv preprint arXiv:2411.10442(2024)

    2024 arXiv

  39. [47]

    Dingkang Yang, Shuai Huang, Shunli Wang, Yang Liu, Peng Zhai, Liuzhen Su, Mingcheng Li, and Lihua Zhang. 2022. Emotion recognition for multiple context awareness. InEuropean conference on computer vision. Springer, 144–162

    2022

  40. [48]

    Qize Yang, Detao Bai, Yi-Xing Peng, and Xihan Wei. 2025. Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis.arXiv preprint arXiv:2501.09502(2025)

    2025 arXiv

  41. [49]

    Qu Yang, Mang Ye, and Bo Du. 2024. Emollm: Multimodal emotional understand- ing meets large language models.arXiv preprint arXiv:2406.16442(2024)

    2024 arXiv

  42. [50]

    Wenmeng Yu, Hua Xu, Fanyang Meng, Yilin Zhu, Yixiao Ma, Jiele Wu, Jiyun Zou, and Kaicheng Yang. 2020. Ch-sims: A chinese multimodal sentiment analysis Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Zhiwei Liu et al. dataset with fine-grained annotation of modality. In...

    2020

  43. [51]

    AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis- Philippe Morency. 2018. Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. InProceedings of the 56th Annual Meeting of the Association for Computation...

    2018

  44. [52]

    Yuanhan Zhang, Bo Li, haotian Liu, Yong jae Lee, Liangke Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-NeXT: A Strong Zero-shot Video Understanding Model. https://llava-vl.github.io/blog/2024-04-30-llava-next- video/

    2024

  45. [53]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: Sy...

    2024 arXiv

  46. [54]

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2019. Semantic understanding of scenes through the ade20k dataset.International Journal of Computer Vision127 (2019), 302–321

    2019

  47. [55]

    Łukasz Augustyniak, Szymon Woźniak, Marcin Gruza, Piotr Gramacki, Krzysztof Rajda, Mikołaj Morzy, and Tomasz Kajdanowicz. 2023. Massively Multilingual Corpus of Sentiment Datasets and Multi-faceted Sentiment Classification Bench- mark. arXiv:2306.07902 [cs.CL] A Performance Ac...

    2023 arXiv

  48. [58]

    anticipation, 8. love, 9. optimism, 10. pessimism, and 11. trust. You have two options: (a) Select one or more of the 11 emotions that represent the emotional state depicted in the text. (b) Choose ‘0. neutral’ if the image does not express any emotion. Please provide your sel...

  49. [59]

    You have two options: (a) Select one emotion that best represents the emotional state conveyed in the text

    surprise. You have two options: (a) Select one emotion that best represents the emotional state conveyed in the text. (b) Choose ‘0. neutral’ if the text does not express any emotion. Please provide your selection in the following format: number. emotion. Text: raw text EWECT-...

  50. [60]

    You have two options: (a) Select one emotion that best represents the emotional state conveyed in the text

    surprise. You have two options: (a) Select one emotion that best represents the emotional state conveyed in the text. (b) Choose ‘0. neutral’ if the text does not express any emotion. Please provide your selection in the following format: number. emotion. Text: raw text Online...

  51. [61]

    anticipation, 8

    disgust, 7. anticipation, 8. trust. You have two options: (a) Select one or more of the 8 emotions that represent the emotional state depicted in the text. (b) Choose ‘0. neutral’ if the text does not express any emotion. Please provide your selection in the following format: ...

  52. [62]

    Happiness: feeling delighted; feeling enjoyment or amusement

  53. [63]

    Sadness: feeling unhappy, sorrow, disappointed, or discouraged

  54. [64]

    Anger: intense displeasure or rage; furious; resentful

  55. [65]

    Fear: feeling suspicious or afraid of danger, threat, evil or pain; horror

  56. [66]

    Surprise: sudden discovery of something unexpected

  57. [67]

    Aversion: feeling disgust, dislike, repulsion; feeling hate

  58. [68]

    Excitement: feeling enthusiasm; stimulated; energetic

  59. [69]

    Peace: well being and relaxed; no worry; having positive thoughts or sensations; satisfied

  60. [70]

    Affection: fond feelings; love; tenderness

  61. [71]

    Annoyance: bothered by something or someone; irritated; impatient; frustrated

  62. [72]

    Anticipation: state of looking forward; hoping on or getting prepared for possible future events

  63. [73]

    Confidence: feeling of being certain; conviction that an outcome will be favorable; encouraged; proud

  64. [74]

    Disapproval: feeling that something is wrong or reprehensible; contempt; hostile

  65. [75]

    Disconnection: feeling not interested in the main event of the surrounding; indifferent; bored; distracted

  66. [76]

    Disquietment: nervous; worried; upset; anxious; tense; pressured; alarmed

  67. [77]

    Doubt/Confusion: difficulty to understand or decide; thinking about different options

  68. [78]

    Embarrassment: feeling ashamed or guilty

  69. [79]

    Engagement: paying attention to something; absorbed into something; curious; interested

  70. [80]

    Esteem: feelings of favourable opinion or judgement; respect; admiration; gratefulness

  71. [81]

    Fatigue: weariness; tiredness; sleepy

  72. [82]

    Pain: physical suffering

  73. [83]

    Pleasure: feeling of delight in the senses

  74. [84]

    Sensitivity: feeling of being physically or emotionally wounded; feeling delicate or vulnerable

  75. [85]

    Suffering: psychological or emotional pain; distressed; anguished

  76. [86]

    Sympathy: state of sharing others emotions, goals or troubles; supportive; compassionate

  77. [87]

    You can select one or more emotions from the list to represent the image’s emotional expression

    Yearning: strong desire to have something; jealous; envious; lust. You can select one or more emotions from the list to represent the image’s emotional expression. MMAFFBen: A Multilingual and Multimodal Affective Analysis Benchmark for Evaluating LLMs and VLMs Conference acro...

    2018

  78. [2021]

    Deep learning and multilingual sentiment analysis on social media data: An overview.Applied Soft Computing107 (2021), 107373

    2021

  79. [2024]

    InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Emollms: A series of emotional large language models and annotation tools for comprehensive affective analysis. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5487–5496

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.