Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Multimodal Video Emotion Recognition with Reliable Reasoning Priors

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gemini reasoning priors lift video emotion accuracy to 84.68% on MER2024-SEMI, beating an 77.53% attention baseline by 7.15 points through distillation into a lightweight fusion network.

desk verdict The 7-point gain on MER2024 is not yet credible: the prompt in Figure 2 appears to feed the ground-truth label into Gemini, and without clarification or trace-level validation the claimed 'reliable priors' remain label-conditioned rationalizations. read the letter →

arxiv 2508.03722 v1 pith:K36XAHKC submitted 2025-07-29 cs.CV cs.AI

Zhepeng Wang , Yingjian Zhu , Guanghao Dong , Hongzhu Yi , Feng Chen , Xinming Wang , Jun Xie This is my paper
classification cs.CVcs.AI
keywords videoemotionrecognitionmultimodalfusionlargelanguagemodelsreasoningpriorschain-of-thoughtdistillationbalancedcontrastivelearningsemi-supervisedMER2024
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the reasoning traces of a large multimodal model can be distilled into a small, fast emotion-recognition network and measurably improve it. The authors use Gemini-2.0-exp to write per-modality explanations including facial Action Units, prosody, dialogue semantics, and a modality contribution ratio, then inject those descriptions as training priors during the fusion stage. On the MER2024-SEMI benchmark, the prior-enhanced model reaches 84.68% average accuracy, against 77.53% for the attention baseline and 79.72% for the contrastive-only variant. If the result holds, expensive chain-of-thought reasoning can be moved offline into a lightweight student at test time, while the proposed balanced dual-contrastive objective handles the skewed label distribution.

What carries the argument

The central object is the reliable reasoning prior $\mathcal{P} := \{c^* \mid \mathcal{I}_V, \mathcal{I}_A, \mathcal{I}_T, R\}$, a compact package of modality-separable descriptions and a modality contribution ratio generated by Gemini and injected at the fusion stage. The second load-bearing mechanism is Balanced Dual-Contrastive Learning, whose class-balanced denominator $\mathcal{D}_{m,i} = \frac{1}{C}\sum_c \frac{1}{|\mathcal{B}_c|}\sum_{k \in \mathcal{B}_c} \exp(z_{m,i}^\top z_k / \tau)$ replaces uniform negative sampling with a two-step sample-a-class-then-sample rule, so every class contributes exactly $1/C$ of the repulsive probability mass.

What would settle it

Regenerate the priors with the emotion label removed from the prompt, since the prompt in Figure 2 currently includes the word Angry before the analysis request, keep everything else fixed, and compare test accuracy; if accuracy collapses toward the 79.72% contrastive level, the priors were label-conditioned rationalizations rather than independent cues.

Watch

Extended reading notes

Core claim

The paper claims that multimodal video emotion recognition can be improved by injecting reliable reasoning priors produced by a multimodal large language model into a lightweight fusion network, while a balanced dual-contrastive loss handles the skewed label distribution. Formally, the prior is written as $\mathcal{P} := \{c^* \mid \mathcal{I}_V, \mathcal{I}_A, \mathcal{I}_T, R\}$, where $\mathcal{I}_V$, $\mathcal{I}_A$, and $\mathcal{I}_T$ are Gemini-generated Action-Unit, prosodic, and semantic descriptions and $R$ is the modality contribution ratio. These priors are fused with the learned embeddings of each modality as $\mathrm{Transformer}(\mathrm{MLP}(e_i + z_i))$, and the contribution ratio is used as the fusion weight. On MER2024-SEMI, the complete method achieves 84.68% average accuracy versus 77.53% for the baseline and 79.72% for the contrastive-only variant, with the largest gains on the rare classes Surprise and Worried.

Load-bearing premise

The load-bearing premise is that the Gemini-generated Action-Unit, prosody, and dialogue descriptions are honest evidence rather than rationalizations produced after seeing the true emotion label, and the paper offers no independent check of that.

Editorial extensions

If this is right

  • A lightweight fusion model can inherit chain-of-thought-level reasoning from an MLLM without paying the inference cost of generating that reasoning at test time.
  • Balanced dual-contrastive learning raises tail-class accuracy, most visibly Surprise (70.83% versus 43.24% for the contrastive variant) and Worried (76.22% versus 70.08%).
  • Audio is the most discriminative single modality at 74.16% accuracy, versus 66.30% for vision and 51.89% for text, so the prior-derived audio weight deserves particular attention.
  • Adding 4K balanced labeled samples lifts accuracy to 79.22%, confirming that label balancing, not just data volume, drives gains in the semi-supervised setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test the paper leaves implicit is to regenerate the priors with the emotion label removed from the prompt, since the prompt shown in Figure 2 contains the word Angry before the reasoning request; if accuracy then falls back to the contrastive-only level, the priors are label-conditioned rationalizations rather than independent evidence.
  • The class-balanced denominator is a general recipe: any long-tailed contrastive learner could replace uniform negative sampling with sample-a-class-first, and BDCL's gains on rare emotion classes suggest this is where such balancing pays off.
  • The Gemini-produced modality contribution ratios could be compared against learned or oracle fusion weights across other multimodal datasets; if the ratios are stable, they would act as a cheap regularizer for fusion architectures beyond emotion recognition.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-stage multimodal emotion recognition framework for the MER2024 benchmark. In Stage 1, a balanced dual-contrastive loss (BDCL) is used for semi-supervised representation learning. In Stage 2, reasoning traces generated by Gemini-2.0-exp are injected as priors into a lightweight fusion network, with modality contribution ratios used as fusion weights. The authors report an average accuracy of 84.68 on MER2024-SEMI, compared with 79.72 for the contrastive variant and 77.53 for the attention baseline, and attribute the gain to the reliability of the MLLM-derived priors.

Significance. If the central mechanism were validated, the paper would make a useful contribution: it would show that reasoning priors from large multimodal models can be transferred to a lightweight, task-specific fusion architecture, and the BDCL formulation targets a real problem in emotion recognition. The use of a public benchmark, the attempt to separate modality contributions, and the comparison against several LLM-based and fusion-based baselines are strengths. However, the evidence for the central claim is incomplete: the prompt shown in Figure 2 appears to include the ground-truth label before Gemini is asked to reason, the isolated contribution of the priors is never measured, and no statistical confidence estimates are provided. These issues must be resolved before the claimed gains can be attributed to 'reliable reasoning priors.'

major comments (4)
  1. [Section 3.1, Figure 2, Eq. (2)] The prompt displayed in Figure 2 reads 'Please use the acoustic, visual and lexical information to recognize the emotion in the video. Angry. Please analyze the character action units...' This places the ground-truth label before Gemini is asked to produce the reasoning traces. As printed, the prior set P := {c* | IV, IA, IT, R} in Eq. (2) is generated with c* already supplied to the model, making the 'reasoning' a label-conditioned rationalization rather than independent multimodal evidence. The paper contains no human or automated validation of the AU descriptions, prosodic interpretations, or contribution ratios against the actual inputs. I ask the authors to (i) confirm whether the actual prompt template included the emotion label; (ii) if so, generate the priors without the label and rerun the stage-2 experiments; and (iii) provide a sample-level validation or consistency check showing that the traces correspond to the input content rather than to the label. This is load-bearing because Eq. (2) is the foundation of the proposed mechanism.
  2. [Section 4.2, Table 1 and Section 4.3, Table 2] The central claim that reasoning priors are responsible for the performance gain is not supported by the ablations as reported. 'Ours' differs from 'Contrastive' by both the injection of the Gemini priors and any additional supervision from the prior-guided tuning stage, while 'Contrastive' differs from 'Attention' by the contrastive loss. Table 2's 'AVT+Prior' row (84.68) is compared against 'AVT' (77.53), but the latter is the Attention baseline without BDCL, so the comparison conflates the effect of the priors with the effect of BDCL. An ablation that adds the priors to the Contrastive model, and separately to the Attention model, is needed to isolate the prior's contribution. Without such an ablation, the 4.96-point gap over 'Contrastive' cannot be attributed specifically to the reasoning priors.
  3. [Section 4.2, Table 1; Section 4.3, Tables 2 and 3] No error bars, repeated runs, or significance tests are reported for any result. The average accuracy gains are accompanied by inconsistent per-class changes: for example, Angry drops from 82.74 (Contrastive) to 78.65 (Ours), while Happy rises from 87.73 to 91.89. Given that the test partition has limited per-class samples, the reported differences may be within run-to-run or sample-to-sample noise. I request multiple training runs (at least three seeds) with mean and standard deviation for the main table and the key ablations, or a statistical test where feasible. This is particularly important because the headline gain over 'Contrastive' is around 5 accuracy points.
  4. [Section 3.2, Eqs. (6)-(9)] The class-balanced denominator is defined inconsistently across the two loss terms. In Eq. (6), the intra-modality denominator uses B_c^{(m)}, the set of embeddings of modality m with label c. In Eq. (7), the inter-modality denominator uses B_c without the modality restriction, which means the negatives for a visual anchor include audio and text embeddings as well. The claim in Eq. (9) that this yields a balanced negative distribution is not derived in the text, and the proportionality sign obscures the normalization. Since BDCL is one of the two main contributions, the authors should give a precise definition, a derivation of Eq. (9), and clarify which embeddings the denominator includes.
minor comments (5)
  1. [Keywords] The keyword 'Relaible Reasoning' contains a typo; it should be 'Reliable Reasoning'.
  2. [Figure 3] The figure caption says 'Semi-Surprised Training'; this appears to be a typo for 'Semi-Supervised Training'.
  3. [Section 3.2, paragraph 4] The sentence 'However, since the labels of the original expression samples are not balanced, we introduce' is incomplete and trails off before 'To that end'. Please revise.
  4. [Section 4.3, Table 3] Table 3 reports results with different data sampling schemes, but the setup is not clearly connected to the full pipeline: it is unclear whether these runs use BDCL, the priors, or neither. Please specify the configuration used for each row.
  5. [Section 4.3, Table 2] The rows 'A', 'V', and 'T' report unimodal results, but it is not described whether these use the prior-enhanced training or only the base fusion model. This should be stated explicitly so the reader can interpret the modality comparison.

Circularity Check

1 steps flagged · score 6.0 of 10

Figure 2's Gemini prompt supplies the emotion label ('Angry.') before the reasoning request, so Eq. (2)'s prior P contains the target c* by construction; the stage-2 'prior-guided' gain may reflect label-conditioned rationalizations rather than independent multimodal reasoning.

  1. self definitional [Section 3.1, Eq. (2), and Fig. 2 (Reliable Reasoning Process)]
    "P := {c∗ | IV , IA, IT , R} (2) ... Please use the acoustic, visual and lexical information to recognize the emotion in the video.Angry. Please analyze the character action units in the video, combine the video, audio and dialogs to comprehensively judge the character's emotions, and give the contribution of the features. Then give the final emotion prediction."

    As displayed in the paper, the Gemini prompt contains the emotion label 'Angry.' before the reasoning request. Eq. (2) then defines the reasoning prior P as including c*, the final emotion, and Eq. (3) presents c* = arg max S(Im) as if it were inferred from the modality evidence. Under the shown template, c* is supplied as input rather than derived, so the 'prior' is a post-hoc rationalization of the target label. Stage 2 uses exactly these priors ('We use reliable emotion priors data generated by Gemini in 3.1 as our stage 2 tuning dataset') to supervise the lightweight fusion network, meaning the auxiliary supervision contains the answer by construction.

full rationale

The final comparison in Table 1 is against a held-out test partition of MER2024, so the benchmark protocol itself is not circular. However, the paper's own Figure 2 shows the emotion label ('Angry.') embedded in the Gemini prompt before the reasoning request, and Eq. (2) packages the result as P := {c* | IV, IA, IT, R}, including the target c*. With that displayed template, the 'reliable reasoning priors' used for Stage 2 tuning are label-conditioned rationalizations: the supposed output of the reasoning process is an input to the generator, so the distillation step trains the lightweight network on targets baked into the teacher's prompt. The reported gain over the Contrastive baseline (84.68 vs 79.72) is therefore not attributable, on the evidence provided, to independent multimodal reasoning. The Balanced Dual Contrastive Learning loss is a standard balanced InfoNCE variant that appropriately cites prior work [25] and is not circular. Self-citations are not load-bearing in this paper. Because the test evaluation is held out and the final prediction is still produced by the fusion network, the circularity is partial, affecting the claimed mechanism and interpretation of the gain rather than the entire benchmark comparison, hence score 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on one major domain assumption, reliability of Gemini's reasoning traces, and one dataset assumption, non-overlap between the MER2023 test partition used for training and the MER2024 test set. The contrastive loss introduces three hyperparameters whose values are chosen without sensitivity analysis. No new physical or conceptual entities are postulated.

free parameters (3)
  • lambda_inter = 0.2
    Weight for the inter-modality contrastive term in Eq. 5; chosen by the authors with no sensitivity analysis reported.
  • lambda_intra = 0.2
    Weight for the intra-modality contrastive term in Eq. 5; chosen by the authors with no sensitivity analysis reported.
  • tau = 0.1
    Temperature in the InfoNCE losses, stated as a conventional setting but central to the contrastive objective; no sweep is reported.
assumptions (4)
  • domain assumption Gemini's generated AU combinations, prosodic cues, semantic content, and modality contribution ratios are reliable and correspond to the true emotion.
    Central premise of stage-2 prior guided tuning. Section 3.1 says Gemini analyzes video samples along three dimensions, but no validation of the traces is provided.
  • domain assumption The MER2023 test partition can be used as additional training data without leaking information relevant to the MER2024 test set.
    Section 4.1 uses 'partially MER2023 test dataset provided by Emotion-Llama' as supervised training data; no overlap or leakage analysis is reported.
  • ad hoc to paper The class-balanced denominator D in Eq. 8 causes every class to contribute exactly 1/C of the negative probability mass.
    The text asserts this property but only sketches the gradient expectation in Eq. 9; no full proof or empirical verification of the balancing claim is given.
  • standard math InfoNCE-based dual contrastive losses improve emotion class separability in multimodal feature space.
    Standard contrastive learning assumption, supported by cited prior work [24, 25].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multimodal Video Emotion Recognition with Reliable Reasoning Priors." pith.science (2026). https://pith.science/paper/K36XAHKC

@misc{pith2026250803722,
  author       = {Pith},
  title        = {Pith review of: Multimodal Video Emotion Recognition with Reliable Reasoning Priors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K36XAHKC}},
  note         = {Machine review of arXiv:2508.03722}
}
read the original abstract

This study investigates the integration of trustworthy prior reasoning knowledge from MLLMs into multimodal emotion recognition. We employ Gemini to generate fine-grained, modality-separable reasoning traces, which are injected as priors during the fusion stage to enrich cross-modal interactions. To mitigate the pronounced class-imbalance in multimodal emotion recognition, we introduce Balanced Dual-Contrastive Learning, a loss formulation that jointly balances inter-class and intra-class distributions. Applied to the MER2024 benchmark, our prior-enhanced framework yields substantial performance gains, demonstrating that the reliability of MLLM-derived reasoning can be synergistically combined with the domain adaptability of lightweight fusion networks for robust, scalable emotion recognition.

Figures

Figures reproduced from arXiv: 2508.03722 by the authors.

Figure 1
Figure 1. Mapping between Action Units and emotions [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Reliable Reasoning Process where R denotes the contribution of each modality. For each modality prior, it may imply different emotion labels c with a latent mapping S(Im) :→ c, MLLMs will synthesize all priors to recognize a certain emotion. c ∗ = arg max S(Im) (3) In subsequent training stages, these trustworthy reasoning traces can be in￾corporated as priors guided dataset, guiding the multimodal fusion network an… view at source ↗
Figure 3
Figure 3. The whole framework of the multimodal fusion paradigm. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Latent feature space comparison. 5 Conclusion In this study, we explored the integration of reliable reasoning priors from MLLMs into lightweight multimodal video emotion recognition frameworks. Reliable priors not only enhance multimodal integration but also serve as …

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. WINELL: Wikipedia Never-Ending Updating with LLM Agents

    cs.CL 2025-07 unverdicted novelty 5.0 of 10

    WiNELL is a multi-agent LLM framework that selects new facts about an entity and generates human-reviewable Wikipedia edit suggestions, with editor models trained on Wikipedia edit history.

  2. Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitable

    cs.CV 2026-05 unverdicted novelty 4.0 of 10

    Life-logging video streams create an inevitable privacy-utility trade-off that is a foundational challenge for always-on AI systems.

Reference graph

Works this paper leans on

33 extracted references · 19 canonical work pages · cited by 2 Pith papers

  1. [25]

    Balanced contrastive learning for long-tailed visual recognition

    Jianggang Zhu, Zheng Wang, Jingjing Chen, Yi-Ping Phoebe Chen, and Yu-Gang Jiang. Balanced contrastive learning for long-tailed visual recognition. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6908–6917, 2022

    work page 2022

  2. [1]

    Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  3. [2]

    Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D

    Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivi- son, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishir...

    arXiv 2024

  4. [3]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  5. [4]

    Train- ing language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Train- ing language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022

    work page 2022

  6. [5]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jin- guo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  7. [6]

    R., Shravan Venkatraman, Vigya Sharma, Santhosh Malarvannan, and Modigari Narendra

    Joe Dhanith P. R., Shravan Venkatraman, Vigya Sharma, Santhosh Malarvannan, and Modigari Narendra. Multimodal emotion recognition using audio–video trans- former fusion with cross attention.arXiv preprint arXiv:2407.18552, 2025

    work page arXiv 2025

  8. [7]

    Tacfn: Transformer-based adaptive cross-modal fusion network for multimodal emotion recognition.Artificial Intelligence Research, 2, 2023

    Feng Liu, Ziwang Fu, Yunlong Wang, and Qijian Zheng. Tacfn: Transformer-based adaptive cross-modal fusion network for multimodal emotion recognition.Artificial Intelligence Research, 2, 2023

    work page 2023

Show all 33 references
  1. [8]

    Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph

    AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis- Philippe Morency. Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph. InProceedings of the 56th Annual Meet- ing of the Association for Computational L...

    2018

  2. [9]

    Livingstone and Frank A

    Steven R. Livingstone and Frank A. Russo. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english.PLOS ONE, 13(5):e0196391, 2018

    2018

  3. [10]

    Multimodal emotion recognition with vision–language prompting and modality dropout

    Anbin Qi, Zhongliang Liu, Xinyong Zhou, Jinba Xiao, Fengrun Zhang, Qi Gan, Ming Tao, Gaozheng Zhang, and Lu Zhang. Multimodal emotion recognition with vision–language prompting and modality dropout. InProceedings of the 2nd Inter- national Workshop on Multimodal and Responsibl...

    2024

  4. [11]

    Visual and textual prompts for enhancing emotion recognition in video

    Zhifeng Wang, Qixuan Zhang, Peter Zhang, Wenjia Niu, Kaihao Zhang, Ramesh Sankaranarayana, Sabrina Caldwell, and Tom Gedeon. Visual and textual prompts for enhancing emotion recognition in video. InProceedings of the 2024 IEEE In- ternational Conference on Image Processing (IC...

    2024

  5. [12]

    Ov-mer: Towards open-vocabulary multi- modal emotion recognition

    Zheng Lian, Haiyang Sun, Licai Sun, Haoyu Chen, Lan Chen, Hao Gu, Zhuofan Wen, Shun Chen, Siyuan Zhang, Hailiang Yao, Bin Liu, Rui Liu, Shan Liang, Ya Li, Jiangyan Yi, and Jianhua Tao. Ov-mer: Towards open-vocabulary multi- modal emotion recognition. In Proceedings of the 42nd...

    2025

  6. [13]

    Af- fectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models

    Zheng Lian, Haoyu Chen, Lan Chen, Haiyang Sun, Licai Sun, Yong Ren, Zebang Cheng, Bin Liu, Rui Liu, Xiaojiang Peng, Jiangyan Yi, and Jianhua Tao. Af- fectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models. InProceedings of ...

    2025

  7. [14]

    From sys- tem 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yux- uan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From sys- tem 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025

    2025 arXiv

  8. [15]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  9. [16]

    Visual programming: Compositional visual reasoning without training.arXiv preprint arXiv:2211.11559, 2022

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training.arXiv preprint arXiv:2211.11559, 2022

    2022 arXiv

  10. [17]

    Visual chain-of-thought prompting for knowledge- based visual reasoning

    Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Zhiqing Sun, Dan Gutfreund, and Chuang Gan. Visual chain-of-thought prompting for knowledge- based visual reasoning. InProceedings of the 38th AAAI Conference on Artificial Intelligence, 2024. AAAI 2024

    2024

  11. [18]

    Improving visual com- monsense in language models via multiple image generation

    Guy Yariv, Idan Schwartz, Yossi Adi, and Sagie Benaim. Improving visual com- monsense in language models via multiple image generation. arXiv preprint arXiv:2406.13621, 2024

    2024 arXiv

  12. [19]

    An explainable vision question answer model via diffusion chain-of-thought

    Chunhao Lu, Qiang Lu, and Jake Luo. An explainable vision question answer model via diffusion chain-of-thought. InProceedings of the European Conference on Computer Vision (ECCV), 2024. ECCV 2024

    2024

  13. [20]

    The relation between valence and arousal in subjective experience.Psychological bulletin, 139(4):917, 2013

    Peter Kuppens, Francis Tuerlinckx, James A Russell, and Lisa Feldman Barrett. The relation between valence and arousal in subjective experience.Psychological bulletin, 139(4):917, 2013

    2013

  14. [21]

    Arousal, valence, and mem- ory for detail.Memory, 12(2):237–247, 2004

    Terry Libkuman, Charles Stabler, and Hajime Otani. Arousal, valence, and mem- ory for detail.Memory, 12(2):237–247, 2004

    2004

  15. [22]

    Facial action coding system.Environmental Psychology & Nonverbal Behavior, 1978

    Paul Ekman and Wallace V Friesen. Facial action coding system.Environmental Psychology & Nonverbal Behavior, 1978

    1978

  16. [23]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  17. [24]

    Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748, 2018. 12 Authors Suppressed Due to Excessive Length

    2018 arXiv

  18. [26]

    Mer 2024: Semi-supervised learning, noise robustness, and open-vocabulary multimodal emotion recognition

    Zheng Lian, Haiyang Sun, Licai Sun, Zhuofan Wen, Siyuan Zhang, Shun Chen, Hao Gu, Jinming Zhao, Ziyang Ma, Xie Chen, et al. Mer 2024: Semi-supervised learning, noise robustness, and open-vocabulary multimodal emotion recognition. In Proceedings of the 2nd International Worksho...

    2024

  19. [27]

    Mer 2023: Multi-label learning, modality robustness, and semi-supervised learning

    Zheng Lian, Haiyang Sun, Licai Sun, Kang Chen, Mngyu Xu, Kexin Wang, Ke Xu, Yu He, Ying Li, Jinming Zhao, et al. Mer 2023: Multi-label learning, modality robustness, and semi-supervised learning. InProceedings of the 31st ACM inter- national conference on multimedia, pages 961...

    2023

  20. [28]

    Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning.Advances in Neural Information Processing Systems, 37:110805–110853, 2024

    Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and Alexander Hauptmann. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning.Advances in Neural Information Processing Systems, 37:110805–110853, 2024

    2024

  21. [29]

    Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models

    YunfeiChu,JinXu,XiaohuanZhou,QianYang,ShiliangZhang,ZhijieYan,Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919, 2023

    2023 arXiv

  22. [30]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024

    2024 arXiv

  23. [31]

    Minigpt-v2:Largelanguagemodelasaunifiedinterfaceforvision-language multi-task learning

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed El- hoseiny. Minigpt-v2:Largelanguagemodelasaunifiedinterfaceforvision-language multi-task learning. arXiv preprint arXiv:2310.09478, 2023

    arXiv

  24. [32]

    Video- llava: Learning united visual representation by alignment before projection.arXiv preprint arXiv:2311.10122, 2023

    Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video- llava: Learning united visual representation by alignment before projection.arXiv preprint arXiv:2311.10122, 2023

    2023 arXiv

  25. [33]

    Video-llama: An instruction-tuned audio- visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023

    Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio- visual language model for video understanding.arXiv preprint arXiv:2306.02858, 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.