Pith. sign in

REVIEW 3 major objections 8 minor 40 references

Foundation models learn to predict audience emotions from video

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 23:41 UTC pith:4XOP7E4Q

load-bearing objection New dataset and benchmark for video-to-audience-reaction prediction; LLM-annotated ground truth is the main soft spot but the stress-test concern overstates the problem. the 3 major comments →

arxiv 2607.06875 v1 pith:4XOP7E4Q submitted 2026-07-08 cs.CV cs.LG

Video2Reaction: Mapping Video to Audience Reaction Distribution in the Wild

Trang Nguyen , Sidong Zhang , Shiv Shankar , Gauri Jagatap , Deepak Chandran , Andrea Fanelli , Madalina Fiterau This is my paper
classification cs.CV cs.LG
keywords reactionaudiencepredictionvideocontentvideo2reactionwildbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper introduces Video2Reaction, a dataset of over 10,000 movie clips paired with distributions of audience emotional reactions derived from YouTube comments. The authors frame audience reaction prediction as label distribution learning, where each video maps to a probability distribution over 21 emotion categories rather than a single label. They build a two-stage LLM-based annotation pipeline that rephrases comments to surface implicit reactions, then extracts emotion labels via majority voting across three open-source models, achieving 86% correctness under blind human verification. The central empirical finding is that pretrained vision-language models (Gemini 2.5 Flash, LLaVA-Next-Video, Qwen2.5-VL) fail at zero-shot prediction of audience reaction distributions from video alone, but lightweight low-rank finetuning on Video2Reaction transforms them into state-of-the-art predictors, achieving 77% Top-3 F1 for dominant reaction identification and roughly doubling cosine similarity on full distribution matching compared to temperature-scaled baselines. Classical label distribution learning methods remain competitive on distribution-shape metrics but underperform on dominant reaction prediction. The paper also shows that text modality (clip descriptions) provides substantial gains for both classical and foundation models, and presents preliminary evidence of cross-dataset transfer to a separate video emotion benchmark.

Core claim

The paper establishes that predicting the distribution of audience emotional reactions from video content alone is a learnable task, but one that requires task-specific finetuning: foundation vision-language models cannot perform it zero-shot despite extensive video pretraining, yet become capable predictors after lightweight adaptation. The gap between zero-shot failure and finetuned competence reveals that modeling collective, distributional audience emotion is a distinct capability not acquired through generic multimodal pretraining.

What carries the argument

The central object is the induced emotion distribution, a probability vector over 21 fine-grained reaction categories derived from social media comments and mapped to each video clip. The pipeline connecting video to this distribution has three load-bearing components: a two-stage LLM annotation pipeline (rephrase-then-label with three-agent majority voting), a label distribution learning formulation that treats each clip as having a soft target over emotions rather than a single label, and a benchmark evaluation split into full distribution matching (Chebyshev, Clark, KL, Cosine, Intersection, CAD) and dominant reaction prediction (MRR, Top-1 Probability Error, weighted Top-k F1).

Load-bearing premise

The distribution of emotions extracted from YouTube comments is treated as a faithful proxy for the true distribution of emotions experienced by the broader audience, but the paper verifies whether LLM labels match comment intent, not whether commenting viewers represent the full audience. Commenters are self-selected, and several emotion categories show 0% annotation correctness in human evaluation.

What would settle it

If the comment-derived reaction distributions systematically diverge from emotions experienced by a representative sample of viewers (e.g., measured in a controlled lab setting with the same clips), the benchmark's ground truth labels would not measure what they claim, and model performance gains would reflect comment-distribution fitting rather than audience emotion prediction.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Content creators and recommendation systems could use finetuned video models to anticipate audience emotional responses before release, without needing to collect comments post-hoc.
  • The two-stage rephrase-then-label annotation pipeline offers a reusable template for constructing distributional emotion labels from noisy social media at scale, adaptable to other platforms and evolving cultural contexts.
  • The finding that text modality (clip descriptions) substantially boosts performance for both classical and foundation models suggests that current video encoders miss information about narrative context that text readily provides, pointing to a limitation in visual understanding.
  • The cross-dataset transfer result, where finetuning on Video2Reaction plus 1% of a target domain outperforms training on 10% of target data alone, suggests the dataset captures generalizable signals about video-to-emotion mapping.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper introduces Video2Reaction, a dataset of over 10,000 movie clips paired with audience reaction distributions derived from YouTube comments. A two-stage multi-agent LLM pipeline annotates comments into 21 fine-grained emotion categories, achieving 86% correctness under dual-blind human verification. The authors establish a benchmark for video-to-reaction-distribution prediction, evaluating classical label distribution learning (LDL) methods, adapted multimodal models, and foundation vision-language models (VLMs). Key findings: pretrained VLMs fail in zero-shot settings, but LoRA finetuning yields state-of-the-art performance (77% Top-3 F1 for dominant reaction prediction). The paper also provides longitudinal analysis showing audience reactions are non-stationary, and preliminary cross-dataset transfer results to the VCE benchmark.

Significance. The paper addresses an important gap in affective computing: most existing video emotion datasets capture perceived (character) emotions rather than induced (audience) emotions, and those that do capture induced emotions rely on controlled lab settings. Video2Reaction is the first large-scale dataset mapping video to distributional induced emotions from social media, which is ecologically valid and scalable. The two-stage annotation pipeline with open-source LLMs is a practical contribution enabling continuous dataset updates. The comprehensive benchmark spanning four algorithm families and two evaluation axes (full distribution and dominant reaction) is well-designed. The release of dataset, code, and pretrained features supports reproducibility. The finding that finetuned VLMs substantially outperform classical LDL methods on dominant reaction prediction while remaining competitive on distribution metrics is valuable for the community. However, the significance of the benchmark results is partially tempered by concerns about the validity of LLM-generated ground-truth labels for certain emotion categories, as detailed below.

major comments (3)
  1. Table 13 (Appendix B.2) shows that three emotion categories—nervousness (0% correct, 100% incorrect), embarrassment (0% correct, 66.67% incorrect), and anger (0% correct, 75% not sure)—have effectively zero human-verified annotation correctness. Yet Table 19 (Appendix D.4) shows these same categories achieving near-perfect per-class Top-3 F1 scores for finetuned LLaVA-Next: nervousness 0.9981, embarrassment 0.9985, anger 0.9959. This disconnect is load-bearing for the central claim that finetuning 'transforms VLMs into state-of-the-art predictors of audience reaction distributions.' If the ground-truth labels for these categories are essentially noise (0% human correctness), near-perfect F1 indicates the model has learned to replicate systematic LLM annotation patterns—including errors—rather than predict genuine audience emotions. The paper should explicitly address this: (1) report the
  2. The 86% overall annotation correctness (Table 11) is measured at the comment level, but the benchmark trains and evaluates on clip-level distributions aggregated from these comments (§3.2). No validation is provided that aggregated clip-level distributions from LLM-annotated comments match distributions obtainable from a representative sample of actual viewers. The self-selected commenter population (YouTube users who choose to comment) may systematically differ from the broader viewing audience in emotional response. The paper acknowledges this as a limitation (§6) but does not quantify its potential impact on benchmark validity. A targeted analysis—for example, comparing LLM-derived distributions to human-annotated distributions on a subset of clips—would substantially strengthen the claim that the benchmark measures genuine audience reaction prediction rather than commenter reaction
  3. The cross-dataset transfer experiment (§5.2, Appendix D.2, Table 18) uses only 1% of VCE training data and reports Top-3 accuracy of 0.46 (Qwen2.5-VL) and 0.44 (LLaVA-Next), compared to 0.35/0.37 with 10% of VCE data alone. While the authors frame this as 'meaningful cross-dataset transfer,' the absolute performance remains far below fully supervised performance (VideoMAE at 0.68 with 100% EmoDiversity data). The claim that Video2Reaction 'provides complementary supervision signals that can partially substitute expensive human-annotated data' is overstated given that 1% of target data is a very low baseline. The authors should temper this claim or provide additional evidence (e.g., performance with 5% or 10% of target data combined with Video2Reaction pretraining).
minor comments (8)
  1. §3.2, Figure 2 caption: 'AI Agent 1' and 'AI Agent 2' both show the same rephrased text, which appears to be a copy-paste error in the figure. This makes the multi-agent design unclear.
  2. Table 2: The 21 reaction categories are grouped by sentiment (Positive, Negative, Ambiguous), but the grouping of 'realization' and 'curiosity' as Ambiguous could benefit from brief justification, as these are not standard sentiment categories.
  3. §4.2: The CAD metric maps 21 categories onto a 'valence-arousal-based emotional space' but the mapping itself is not provided in the main text or appendix. This hinders reproducibility of this metric.
  4. Table 5: KL divergence for finetuned LLaVA-Next (3.1765) is worse than SA-BFGS (0.5976) and several other baselines, yet the text states finetuned models achieve 'best and comparable performances.' The framing should clarify that finetuned VLMs excel on Chebyshev, Intersection, and Cosine but underperform on KL divergence.
  5. §3.4: The longitudinal analysis shows reactions shift over time (e.g., 'There Will Be Blood' shifted from positive to disapproval), but the dataset is released as a single aggregate distribution per clip. Consider clarifying whether time-stratified distributions are available for future longitudinal research.
  6. Appendix B.2, Table 13: The 'anger' category shows 0% correct and 75% 'Not Sure,' which differs from the pattern for nervousness (100% incorrect). These represent different failure modes (ambiguity vs. systematic mislabeling) and should be discussed separately.
  7. §5.4, Table 7: The ablation shows text modality provides significant gains, but the 'Text' input for LLaVA-Next is the clip description from the YouTube channel, not the comments. This should be clarified to avoid confusion about whether the model has access to audience-generated text at inference time.
  8. Reference [6] cites 'Gemini 2.5' with a 2025 arXiv ID (2507.06261), which appears to be a future-dated preprint. Please verify this reference.

Circularity Check

0 steps flagged

No significant circularity; label-quality concerns are validity issues, not derivation circularity

full rationale

The paper's derivation chain is: (1) YouTube comments are collected for movie clips, (2) a set of open-source text LLMs (LLaMA-3.1-8B, Qwen2.5-14B, DeepSeek-R1-Distill-Qwen-7B) annotate comments with emotion labels via a two-stage pipeline, (3) these annotations are aggregated into clip-level reaction distributions serving as ground truth, (4) separate vision-language models (LLaVA-Next-Video-7B, Qwen2.5-VL, Gemini) are finetuned on video clips paired with these distributions, and (5) the finetuned VLMs are evaluated on held-out clips against the same labeling process. Each step has independent content: the annotation LLMs differ from the evaluated VLMs, the VLMs could in principle fail to learn the mapping (and do fail in zero-shot settings with Top-1 F1 below 0.30), and the evaluation uses held-out data. The paper also provides external validation of label quality via dual-blind human verification (86% correctness, Table 11) and a human-LLM correlation study (0.40 vs 0.43 inter-rater, Section 3.2). The skeptic's observation that 0%-correctness categories (nervousness, embarrassment, anger in Table 13) show near-perfect per-class F1 (Table 19) is a serious validity concern—it suggests the model may be replicating systematic LLM annotation artifacts rather than learning genuine emotion prediction—but this is a data quality and benchmark validity issue, not a circularity in the derivation chain. The prediction is not forced by construction: the VLM receives only video input and must independently learn to map visual content to the LLM-derived label distributions. No equation or definition in the paper reduces a claimed prediction to its own inputs. The minor score of 2 reflects that the ground-truth generation and model evaluation share the same labeling pipeline, creating a weak coupling, but this coupling is explicitly acknowledged and partially validated by independent human evaluation.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 2 invented entities

See above.

free parameters (6)
  • LoRA rank r = 8
    Chosen for VLM finetuning; not tuned against the target metric.
  • LoRA alpha = 16 (paper states 16 in §5, 6 in Appendix C.3)
    Scaling factor for LoRA; inconsistency between main text and appendix.
  • Temperature scaling parameter = tuned on validation data
    Applied to VLM zero-shot predictions for probability calibration.
  • Scene detection threshold = 3.0
    HSL color space threshold for PySceneDetect; chosen by hand.
  • Minimum views filter = 10000
    Data collection filter for meaningful audience engagement.
  • Minimum comments filter = 10
    Data collection filter for distribution construction.
axioms (4)
  • domain assumption YouTube comments reflect genuine audience emotional reactions to video content
    Invoked in §3.1-3.2 where comments are collected and aggregated into reaction distributions. The paper acknowledges this is a limitation but the entire dataset construction depends on it.
  • domain assumption LLM-assigned emotion labels are valid proxies for human emotion labels
    Invoked in §3.2 where LLM annotations form the ground truth. Supported by 86% human verification and correlation study, but not fully independent.
  • domain assumption GoEmotions taxonomy (adapted to 21 categories) covers the relevant space of audience reactions to movie content
    Invoked in §3.2 where the reaction taxonomy is defined. 7 categories dropped due to underrepresentation.
  • standard math Label distribution learning framework is appropriate for modeling audience reaction diversity
    Invoked in §4.1; LDL is an established framework from Geng (2016).
invented entities (2)
  • Video2Reaction dataset independent evidence
    purpose: Benchmark and training resource for audience reaction prediction
    The dataset is publicly released with YouTube IDs and preprocessed features; its properties (imbalance, entropy, longitudinal variation) are empirically measurable.
  • Two-stage multi-agent annotation pipeline independent evidence
    purpose: Scalable, updatable annotation of audience reactions from comments
    The pipeline is described in full detail with prompts and evaluated against human annotations; it is reproducible and falsifiable.

pith-pipeline@v1.1.0-glm · 27660 in / 3664 out tokens · 168312 ms · 2026-07-09T23:41:48.126808+00:00 · methodology

0 comments
read the original abstract

Understanding and forecasting audience reactions to video content are crucial for improving content creation, recommendation systems, and media analysis. To enable audience reaction prediction and other content engagement applications, we introduce $\textbf{Video2Reaction}$, a multimodal dataset that maps short movie segments to a distribution of $\textit{induced emotions}$ of viewers in the wild, as expressed through social media. $\textbf{Video2Reaction}$ spans more than 10,000 videos and serves as a reliable benchmark as well as a training resource for audience reaction prediction. To enable cost-effective continuous annotations as reactions may change over time, we develop a two-stage multi-agent pipeline using only open-source LLMs, achieving 86% correctness under blind human verification despite the inherently noisy and subjective nature of the task. We establish the first benchmark for video-to-reaction-distribution prediction in the wild and show that pretrained foundation video models fail in zero-shot settings, while finetuning transforms them into state-of-the-art predictors capable of modeling both full reaction distributions and dominant responses from video alone. However, the task remains challenging: even the strongest methods achieve only 77% Top-3 F1 in dominant reaction prediction (LLaVA-Next), highlighting a substantial gap in modeling collective audience reaction. \modification{Dataset and code are available at our project page: https://information-fusion-lab-umass.github.io/video2reaction-bench.github.io

Figures

Figures reproduced from arXiv: 2607.06875 by Andrea Fanelli, Deepak Chandran, Gauri Jagatap, Madalina Fiterau, Shiv Shankar, Sidong Zhang, Trang Nguyen.

Figure 1
Figure 1. Figure 1: Video2Reaction is the first benchmark that uses video data to directly learn induced emotion distribution in the wild. characters in the scene or filmmaker’s emotion intent), audience reactions can vary greatly depending on personal, cultural, or temporal context. For example, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of Video2Reaction Two-Stage LLM-based Data Annota [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Key Statistics on Reaction Outcome in the Video2Reaction dataset. 1 2 3 4 5 6 7 8 9 Num. Unique Dominant Emotions 0 20 40 60 80 Count of Clips Mean: 3.46 | 1 emotion: 20 >2 emotions: 195 (a) Fine-grained Emotion Diversity 0 2 4 6 8 10 12 Num. Sentiment Transitions 0 20 40 60 Count of Clips Mean: 3.43 | 0 transitions: 58 >3 transitions: 126 (b) Sentiment Transition Frequency [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of fine-grained emotion and sentiment transitions across movie clips. Sentiment transitions are computed at the monthly level. 3.4 Longitudinal Analysis of Audience Reactions Further analysis on how audience reactions to the same clip evolve over time shows that audience reactions are non-stationary and can shift meaningfully over time, motivating our scalable annotation pipeline. As shown in … view at source ↗
Figure 5
Figure 5. Figure 5: illustrates how the four leading algorithms in our benchmark (SA￾BFGS and CTEN) model groundtruth label distribution across different entropy levels. Lower entropy represents more unimodal distribution and higher entropy represents more uniform distribution [PITH_FULL_IMAGE:figures/full_fig_p033_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Video-Comment Human Annotaiton Interface Dual Blind Human Verification . To assess the quality of automated reaction annotation, we randomly sample 100 movie clips with balanced representation across all movie genres. From each clip, 10 comments are randomly selected, yielding a total of 1,000 comments for human evaluation. Due to the subjec￾tive nature of fine-grained audience reactions, each comment is i… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages · 12 internal anchors

  1. [1]

    Bain, M., Nagrani, A., Brown, A., Zisserman, A.: Condensed movies: Story based retrieval with contextual embeddings (2020)

    work page 2020

  2. [2]

    IEEE Transactions on Affective Computing6(1), 43– 55 (2015)

    Baveye, Y., Dellandrea, E., Chamaret, C., Chen, L.: Liris-accede: A video database for affective content analysis. IEEE Transactions on Affective Computing6(1), 43– 55 (2015)

    work page 2015

  3. [3]

    Language resources and evaluation42, 335–359 (2008)

    Busso, C., Bulut, M., Lee, C.C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J.N., Lee, S., Narayanan, S.S.: Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation42, 335–359 (2008)

    work page 2008

  4. [4]

    Advances in Neural Information Processing Systems37, 45767–45790 (2024)

    Chen, L., Davis, J.Q., Hanin, B., Bailis, P., Stoica, I., Zaharia, M.A., Zou, J.Y.: Are more llm calls all you need? towards the scaling properties of compound ai systems. Advances in Neural Information Processing Systems37, 45767–45790 (2024)

    work page 2024

  5. [5]

    In: EMNLP (2024)

    Choi, J., Yun, J., Jin, K., Kim, Y.: Multi-news+: Cost-efficient dataset cleansing via llm-based data annotation. In: EMNLP (2024)

    work page 2024

  6. [6]

    Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities

    Comanici, G., Bieber, E., Schaekermann, M., et al.: Gemini 2.5: Pushing the fron- tier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 (2025),https://arxiv.org/ abs/2507.06261

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    DeepSeek-AI: Deepseek-r1: Incentivizing reasoning capability in llms via reinforce- ment learning (2025),https://arxiv.org/abs/2501.12948

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  8. [8]

    GoEmotions: A Dataset of Fine-Grained Emotions

    Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., Ravi, S.: Goemotions: A dataset of fine-grained emotions. arXiv preprint arXiv:2005.00547 (2020)

    work page internal anchor Pith review Pith/arXiv arXiv 2005

  9. [9]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

    Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirec- tional transformers for language understanding. CoRRabs/1810.04805(2018), http://arxiv.org/abs/1810.04805

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  10. [10]

    Geng, X.: Label Distribution Learning (Apr 2016).https://doi.org/10.48550/ arXiv.1408.6027,http://arxiv.org/abs/1408.6027

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  11. [11]

    In: IJCAI

    Geng, X., Hou, P.: Pre-release prediction of crowd opinion on movies by label distribution learning. In: IJCAI. pp. 3511–3517. Association for the Advancement of Artificial Intelligence (2015)

    work page 2015

  12. [12]

    The Llama 3 Herd of Models

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Let- man, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gregerson, A., Spataru, A., Roziere, B., Biron, B., Tang, B., Chern, B., Cauchete...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  13. [13]

    In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

    Han, W., Chen, H., Poria, S.: Improving multimodal fusion with hierarchical mu- tual information maximization for multimodal sentiment analysis. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. pp. 4946–4958 (2021)

    work page 2021

  14. [14]

    Hsu, W.N., Bolte, B., Tsai, Y.H.H., Lakhotia, K., Salakhutdinov, R., Mohamed, A.: Hubert: Self-supervised speech representation learning by masked prediction of hidden units (2021),https://arxiv.org/abs/2106.07447

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  15. [15]

    In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9

    work page 2022

  16. [16]

    Aligning Language Models to User Opinions

    Hwang, E., Majumder, B.P., Tandon, N.: Aligning language models to user opin- ions. arXiv preprint arXiv:2305.14929 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  17. [17]

    IEEE Transactions on Knowledge and Data Engineering33(4), 1619–1631 (2019)

    Jia, X., Li, Z., Zheng, X., Li, W., Huang, S.J.: Label distribution learning with label correlations on local samples. IEEE Transactions on Knowledge and Data Engineering33(4), 1619–1631 (2019)

    work page 2019

  18. [18]

    CommunityLM: Probing Partisan Worldviews from Language Models

    Jiang, H., Beeferman, D., Roy, B., Roy, D.: Communitylm: Probing partisan world- views from language models. arXiv preprint arXiv:2209.07065 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  19. [19]

    IEEE transactions on affective computing3(1), 18–31 (2011)

    Koelstra, S., Muhl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: Deap: A database for emotion analysis; using physiolog- ical signals. IEEE transactions on affective computing3(1), 18–31 (2011)

    work page 2011

  20. [20]

    In: Proceedings of the AAAI Conference on Artificial Intel- ligence

    Kou, Z., Wang, J., Jia, Y., Geng, X.: Exploiting multi-label correlation in label distribution learning. In: Proceedings of the AAAI Conference on Artificial Intel- ligence. Association for the Advancement of Artificial Intelligence (2024)

    work page 2024

  21. [21]

    Advances in Neural Information Processing Systems35, 18571– 18585 (2022)

    Mazeika, M., Tang, E., Zou, A., Basart, S., Chan, J.S., Song, D., Forsyth, D., Stein- hardt, J., Hendrycks, D.: How would the viewer feel? estimating wellbeing from video scenarios. Advances in Neural Information Processing Systems35, 18571– 18585 (2022)

    work page 2022

  22. [22]

    Meister, N., Guestrin, C., Hashimoto, T.: Benchmarking distributional alignment of large language models (2024),https://arxiv.org/abs/2411.05403

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  23. [23]

    IEEE Transactions on Affective Computing12(1), 36–52 (2019)

    Muszynski, M., Tian, L., Lai, C., Moore, J.D., Kostoulas, T., Lombardo, P., Pun, T., Chanel, G.: Recognizing induced emotions of movie audiences from multimodal information. IEEE Transactions on Affective Computing12(1), 36–52 (2019)

    work page 2019

  24. [24]

    In: Proceedings of the 36th annual acm symposium on user interface software and technology

    Park, J.S., O’Brien, J., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S.: Gener- ative agents: Interactive simulacra of human behavior. In: Proceedings of the 36th annual acm symposium on user interface software and technology. pp. 1–22 (2023) Video2Reaction 19

    work page 2023

  25. [25]

    MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations

    Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: Meld: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  26. [26]

    In: Proceedings of the 30th ACM international conference on multimedia

    Sun, H., Wang, H., Liu, J., Chen, Y.W., Lin, L.: Cubemlp: An mlp-based model for multimodal sentiment analysis and depression estimation. In: Proceedings of the 30th ACM international conference on multimedia. pp. 3722–3729 (2022)

    work page 2022

  27. [27]

    Team, Q.: Qwen2.5: A party of foundation models (September 2024),https:// qwenlm.github.io/blog/qwen2.5/, accessed: March 1, 2026

    work page 2024

  28. [28]

    5-vl/, accessed: March 1, 2026

    Team, Q.: Qwen2.5-vl (January 2025),https://qwenlm.github.io/blog/qwen2. 5-vl/, accessed: March 1, 2026

    work page 2025

  29. [29]

    Tian, L., Muszynski, M., Lai, C., Moore, J.D., Kostoulas, T., Lombardo, P., Pun, T., Chanel, G.: Recognizing induced emotions of movie audiences: Are induced and perceived emotions the same? In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). pp. 28–35. IEEE (2017)

    work page 2017

  30. [30]

    In: International Conference on Intelligent Systems and Pattern Recognition

    Trad, F., Chehab, A.: To ensemble or not: Assessing majority voting strategies for phishing detection with large language models. In: International Conference on Intelligent Systems and Pattern Recognition. pp. 158–173. Springer (2024)

    work page 2024

  31. [31]

    In: The Eleventh International Conference on Learning Representations (2023)

    Wang, X., Wei, J., Schuurmans, D., Le, Q.V., Chi, E.H., Narang, S., Chowdhery, A., Zhou, D.: Self-consistency improves chain of thought reasoning in language models. In: The Eleventh International Conference on Learning Representations (2023)

    work page 2023

  32. [32]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Wen, C., Zhang, X., Yao, X., Yang, J.: Ordinal label distribution learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 23481–23491 (2023)

    work page 2023

  33. [33]

    Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Yan, Z., Tomizuka, M., Gonzalez, J., Keutzer, K., Vajda, P.: Visual transformers: Token-based image representation and processing for computer vision (2020)

    work page 2020

  34. [34]

    Wu, Y., Chen, K., Zhang, T., Hui, Y., Berg-Kirkpatrick, T., Dubnov, S.: Large- scale contrastive language-audio pretraining with feature fusion and keyword-to- caption augmentation (2022).https://doi.org/10.48550/ARXIV.2211.06687, https://arxiv.org/abs/2211.06687

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2211.06687 2022

  35. [35]

    Advances in Neural Information Processing Systems37, 103737–103750 (2024)

    Xu, C., Liu, L., Jin, L., Du, G., Guo, Z., Zhao, Y., Huang, X., Li, R., et al.: Infer induced sentiment of comment response to video: A new task, dataset and baseline. Advances in Neural Information Processing Systems37, 103737–103750 (2024)

    work page 2024

  36. [36]

    Proceedings of the AAAI Conference on Artificial Intelligence31(1) (Feb 2017).https://doi.org/10.1609/aaai.v31i1

    Yang,J.,Sun,M.,Sun,X.:LearningVisualSentimentDistributionsviaAugmented Conditional Probability Neural Network. Proceedings of the AAAI Conference on Artificial Intelligence31(1) (Feb 2017).https://doi.org/10.1609/aaai.v31i1. 10485,https://ojs.aaai.org/index.php/AAAI/article/view/10485

    work page doi:10.1609/aaai.v31i1 2017

  37. [37]

    In: Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers)

    Zadeh, A.B., Liang, P.P., Poria, S., Cambria, E., Morency, L.P.: Multimodal lan- guage analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph. In: Proceedings of the 56th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers). pp. 2236–2246 (2018)

    work page 2018

  38. [38]

    Zhang, Y., Li, B., Liu, h., Lee, Y.j., Gui, L., Fu, D., Feng, J., Liu, Z., Li, C.: Llava- next: A strong zero-shot video understanding model (April 2024),https://llava- vl.github.io/blog/2024-04-30-llava-next-video/, accessed: March 1, 2026

    work page 2024

  39. [39]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Zhang, Z., Wang, L., Yang, J.: Weakly supervised video emotion detection and prediction via cross-modal temporal erasing network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 18888– 18897 (2023) 20 T. Nguyen et al

    work page 2023

  40. [40]

    joy", "sadness

    Zlatintsi, A., Koutras, P., Evangelopoulos, G., Malandrakis, N., Efthymiou, N., Pastra, K., Potamianos, A., Maragos, P.: Cognimuse: A multimodal video database annotated with saliency, events, semantics and emotion with application to sum- marization. EURASIP Journal on Image and Video Processing2017, 1–24 (2017) Video2Reaction 21 Video2Reaction: Mapping ...

    work page arXiv 2017