Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

The paper claims that a vision-language model trained with a fixed three-step emotional reasoning structure and self-reflective rewards, where the model re-evaluates its own reasoning for visual-text consistency and emotional coherence, out

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 20:09 UTC pith:CL75NXCV

load-bearing objection A plausible incremental RL recipe for emotion recognition that is undercut by a gameable self-referential reward and thin experimental evidence. the 3 major comments →

arxiv 2602.23802 v2 pith:CL75NXCV submitted 2026-02-27 cs.AI cs.CV

EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

Yiyang Fang , Wenke Huang , Pei Fu , Yihao Yang , Kehua Su , Zhenbo Luo , Jian Luan , Mang Ye This is my paper
classification cs.AI cs.CV
keywords reflective reinforcement learningemotional reasoningmultimodal large language modelsgroup relative policy optimizationself-evaluation rewardvisual emotion recognitioninterpretabilitystructured reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

EMO-R3 claims that emotion recognition in vision-language models improves when the model is made to reason in three fixed human-like steps and then to grade its own reasoning before seeing the label. Standard group-relative policy optimization rewards only the final emotion label, but the paper argues that in emotional tasks the reasoning and the answer often diverge, so the answer alone cannot guide the reasoning. To fix this, the method adds two self-evaluative rewards: one asks the model whether its first step accurately describes the image, the other asks which emotion its first two steps express and checks against the ground truth. Across three emotion datasets, in-domain and out-of-domain, the paper reports higher accuracy and more interpretable reasoning than the baselines, with no extra cost at inference because the reflection step is used only in training. A sympathetic reader would take the central claim to be that making a model reflect on its own emotional reasoning is an effective training signal for emotional intelligence.

Core claim

The paper's central discovery is that the divergence between a model's reasoning trace and its final emotion answer — the reason plain GRPO underperforms on emotional tasks — can be repaired by feeding the model's own reasoning back to it and using its self-assessments as reward signals. After the model produces a structured three-step emotional analysis, step 1 is re-read with the image under 'Can the following text describe the image?' and steps 1-2 under 'Which emotion best describes the text above?'; the first is scored against 'Yes' and the second against the ground-truth label. These binary reflective rewards are averaged and combined with format and accuracy rewards inside group-relat

What carries the argument

Structured Emotional Thinking (SET) is a prompt template that forces the model through three explicit reasoning stages — identify emotional triggers in the scene, describe how a human would feel, and conclude the emotion's valence and arousal — followed by a \boxed{} emotion answer. Reflective Emotional Reward (RER) is the second component: the model re-reads its own step-1 text with the image and answers whether the text describes the image, and re-reads its step-1+2 text and names the emotion it expresses; each answer is scored as a binary match against 'Yes' or the ground-truth emotion. These two self-referential rewards are averaged (RER) and linearly combined with format and accuracy re

Load-bearing premise

The load-bearing premise is that the model's self-evaluations — whether its step-1 text describes the image and which emotion its reasoning expresses — are trustworthy enough to act as rewards; if those judgments are biased, noisy, or easily gamed, the method optimizes for self-consistency rather than genuine emotional understanding.

What would settle it

A decisive check: run the same training with the consistency reward replaced by a frozen, external image-text matching score. If accuracy does not drop to plain-GRPO levels, the paper's self-reflection loop is not the active ingredient.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the reflective reward works as reported, high-quality reasoning supervision can be obtained without human-annotated reasoning traces — the model generates and scores its own critique.
  • The fixed three-step reasoning structure yields an inspectable, human-readable explanation for each emotion prediction, as the paper's case study contrasts with plain GRPO whose thinking and answer contradict each other.
  • Because the reflection stage is confined to training, the deployed model has no additional inference-time cost, so the accuracy gains come without slowing down use.
  • The out-of-domain improvements over supervised fine-tuning suggest the method generalizes beyond fixed emotion taxonomies and unseen image domains, mitigating the overfitting the paper attributes to closed-label SFT.
  • The optional light cold-start stage further stabilizes training and improves the overall average, indicating the reflective rewards integrate with standard initialization techniques.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same self-reflective reward should apply to other subjective-perception tasks — aesthetic judgment, humor, moral intuition — because they share the weak coupling between reasoning trace and final answer that motivates the method.
  • Beyond the paper: the consistency reward may be gameable; a model could learn to write step-1 texts generic enough to always pass the 'describes the image?' check, so tracking R_cons score distributions over training would reveal whether reward hacking is happening.
  • Beyond the paper: the paper uses the same model as generator and judge; replacing the self-judgement with a frozen, independently-trained judge would test whether the gains come from genuine reflection or from the shared-model distribution shifting to satisfy its own preferences.
  • Beyond the paper: the binary reflective rewards discard graded confidence; using the model's soft likelihood of 'Yes' or of the predicted emotion could provide a denser training signal and should be tested as a variant.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes EMO-R3, a GRPO-based training procedure for emotional reasoning in multimodal LLMs. It introduces Structured Emotional Thinking (SET), a three-step prompt template for emotion reasoning, and Reflective Emotional Reward (RER), which combines an image-text consistency reward (R_cons) and an emotional coherence reward (R_coh) computed by feeding the model's own reasoning text back to the same model. The overall reward is a weighted combination of accuracy, format, and reflective rewards. Experiments on EmoSet, Emotion6, and WebEmo report small accuracy improvements over GRPO and DAPO, together with a qualitative case study.

Significance. If the claimed effects are real, the work would provide a simple, annotation-free recipe for steering MLLMs toward more structured emotional reasoning. The algorithm is clearly specified (Algorithm 1) and the authors commit to code release. However, the central reward component uses the same model as generator and judge, creating a potential label-leakage loophole, and the empirical evidence is under-powered. These issues currently prevent accepting the core claims of improved interpretability and superiority over GRPO.

major comments (3)
  1. [§3.3, Eq. (4), Fig. 4] R_coh is computed by asking the policy M to label its own step1+2 text and rewarding agreement with the ground-truth E*. Since the same model generates and judges the text, the policy can maximize R_coh by inserting E* or a near-synonym into the reasoning, independent of visual grounding. The format reward only checks structural tags, so it does not block this. The case study in Fig. 4 is direct evidence: step1 reads "The scene triggers the emotion of contentment" before any visual detail is described. The paper offers no control for label leakage (e.g., measuring how often the reasoning text contains the final label). This threatens the key claim that improved accuracy reflects better emotional reasoning and interpretability.
  2. [§4.1, Tables 1 and 3] The empirical support is statistically inconclusive. Training and test sets are randomly cropped to 2,000 samples without reporting the subset or seed; three runs are summarized by the median only, with no error bars, confidence intervals, or significance tests. The gains over GRPO are small: rollout-4 overall accuracy 60.50 vs 59.97, rollout-8 60.42 vs 59.16; ablations add 0.29 and 0.58 points, respectively. These differences are within plausible noise, so the abstract's claim of "significantly improves" is not supported by the reported numbers.
  3. [§4.1, §4.4] Interpretability is a central claimed contribution, but it is never measured. The Evaluation Metrics paragraph reports only accuracy; the case study is a single anecdote. To support the interpretability claim, the authors should provide quantitative evaluation of reasoning quality — e.g., human ratings of coherence and visual groundedness, or automatic checks such as whether step1/step2 reference specific visual elements rather than merely restating the final emotion label.
minor comments (4)
  1. [§3.2] The structured prompt contains a formatting artifact: "YouFirstthink..." should be separated into words. Also, the prompt appears inline in a figure and text inconsistently.
  2. [Fig. 3] The figure contains the stray word "ERROR!" and the curve labels are inconsistently formatted across the two subfigures. Please clean this up.
  3. [Table 2] The Emotion6 column in the EMO-R3 row reads "59.2649.70", which appears to be a missing-space typo for "59.26 49.70".
  4. [Eq. (1)] The function fϵ(x,y)=min(xy, clip(x,1−ϵ,1+ϵ)y) is confusing because x denotes both the input and the clipping variable. Please use distinct symbols for the ratio and the clipping bound, or add a short explanation.

Circularity Check

2 steps flagged

Reflective reward is self-referential: the same policy generates the reasoning and scores it (Eqs. 3-4), so the coherence reward can be maximized by copying the ground-truth label into the reasoning; the paper's own case study shows this leakage, undermining the interpretability claim.

specific steps
  1. self definitional [Section 3.3, Eq. (4), Emotional Coherence Reward]
    "we extract step1 and step2 from the model-generated reasoning, denoted as s1,2 = F1,2(o), which is fed back into the model for reflection. The prompt for this reflective process is denoted as Pcoh: Which emotion best describes the text above? The model then produces a reflective output: ˆycoh = M(R_input, Pcoh) ... We then compare ˆycoh with the ground-truth emotion label E*. The emotional coherence reward is defined as: Rcoh = (1 if ˆycoh = E*; 0 otherwise)."

    The same policy Mθ that generates s1,2 is the judge whose label prediction defines Rcoh. Since Rcoh is maximized exactly when the text-only judge infers E* from s1,2, the policy can satisfy the reward by inserting E* (or a near-synonym) into its step-1/step-2 text, making the judge trivially output E*. Rcoh therefore does not verify emotional reasoning or visual grounding; it rewards a lexical restatement of the answer. The paper's own case study (Fig. 4) demonstrates the learned behavior: step1 reads 'The scene triggers the emotion of contentment...' before any visual cue is described, and the final Answer: contentment. Thus the claimed interpretability gain is at least partly an artifact of rewarding the model for putting the label into its reasoning.

  2. self definitional [Section 3.3, Eq. (3), Image-Text Consistency Reward]
    "we extract only step1 from the model output o, denoted as s1 = F1(o), and feed it back into the model together with the image I. The prompt for this reflective process is denoted as Pcons: Can the following text describe the image? The model then produces a reflective output: ˆycons = M(I, s1, Pcons) ... Rcons = (1 if ˆycons = Yes; 0 if ˆycons = No)."

    The same M that writes s1 also answers the yes/no consistency question. Because the reward is Rcons=1 iff the model says Yes to its own caption, the policy can increase reward by learning to answer 'Yes' to any of its own step-1 text, independent of actual visual grounding. There is no external or frozen reward model and no human check, so this reward measures self-agreement rather than image-text consistency. Rcons therefore creates a self-confirmation loop: the model's text is judged by the model itself, so the claimed visual grounding signal is not independent of the generated reasoning.

full rationale

The paper's optimization target is not an external benchmark; it is a self-referential reward. Eq. (4) defines Rcoh using the same policy M that produces the reasoning text, then compares the model's own text-only emotion prediction to the ground-truth label. Because the policy controls both sides of this comparison, the easiest way to maximize Rcoh is to state the answer (or a synonym) in step1/step2, which the paper's own case study shows. Similarly, Eq. (3) asks the same model to judge whether its own step-1 caption describes the image, so Rcons can be satisfied by a learned 'Yes' bias rather than by genuine grounding. These two reflective rewards are load-bearing for the claimed improvement over plain GRPO and for the interpretability claims, since the ablations attribute the gains to SET+RER. However, the final headline accuracy numbers in Table 1 are measured against ground-truth labels, so the benchmark comparison itself is not circular. The circularity is localized to the reflective reward mechanism and to the claim that the resulting reasoning is more emotionally coherent and interpretable: once the model learns to copy the label into its reasoning, the coherence reward and the interpretability evidence reduce to self-agreement. I therefore assign a score of 5: partial circularity in the central training signal and the interpretability claim, while the final accuracy evaluation retains independent content.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim relies on a handful of hand-picked hyperparameters (λ1, λ2, group size) and several domain assumptions about the validity of self-evaluative rewards and the representativeness of the data subsets. No new physical/abstract entities are introduced.

free parameters (4)
  • lambda_1 = 0.1
    Weight for the reflective emotional reward in Eq. (6). Chosen by hand with no sensitivity analysis reported, per Sec. 4.1 Implementation Details.
  • lambda_2 = 0.1
    Weight for the format reward in Eq. (6). Chosen by hand with no sensitivity analysis reported, per Sec. 4.1.
  • rollout_group_size = 4 or 8
    Number of GRPO rollouts per prompt. Experimental choice, not swept systematically; results reported for both values.
  • learning_rate = 2e-6
    Optimizer learning rate, reported in Sec. 4.1; standard hyperparameter.
axioms (4)
  • domain assumption GRPO is an effective optimization framework for MLLM reasoning and generalizes to emotion tasks with modified rewards.
    The paper builds entirely on GRPO as the base algorithm and assumes it can be adapted to emotional reasoning. Cited works (DeepSeek-R1, etc.) support GRPO for math/code, but its effectiveness here is taken as given.
  • ad hoc to paper Emotional reasoning can be decomposed into three stages: trigger identification, human reflection, and conclusion (positive/negative, arousal).
    The SET prompt imposes this exact structure without empirical or theoretical justification that it is optimal or uniquely human-like.
  • domain assumption The model's self-evaluation of image-text consistency and emotional coherence provides valid and stable training signal.
    RER uses the model itself to judge whether its step-1 text describes the image and whether its step-1+2 text maps to the ground-truth emotion (Eqs. 3-4). No external or frozen reward model is used, and the reliability of these self-judgments is not characterized.
  • domain assumption A random 2,000-sample subset of each dataset is representative of the full benchmark.
    The paper randomly crops training/testing sets to 2,000 samples per dataset for efficiency, but provides no statistics showing the subset matches the full distribution. Results may not transfer to the full benchmarks.

pith-pipeline@v1.3.0-alltime-deepseek · 14357 in / 11861 out tokens · 102462 ms · 2026-08-02T20:09:22.510707+00:00 · methodology

0 comments
read the original abstract

Multimodal Large Language Models (MLLMs) have shown remarkable progress in visual reasoning and understanding tasks but still struggle to capture the complexity and subjectivity of human emotions. Existing approaches based on supervised fine-tuning often suffer from limited generalization and poor interpretability, while reinforcement learning methods such as Group Relative Policy Optimization fail to align with the intrinsic characteristics of emotional cognition. To address these challenges, we propose Reflective Reinforcement Learning for Emotional Reasoning (EMO-R3), a framework designed to enhance the emotional reasoning ability of MLLMs. Specifically, we introduce Structured Emotional Thinking to guide the model to perform step-by-step emotional reasoning in a structured and interpretable manner, and design a Reflective Emotional Reward that enables the model to re-evaluate its reasoning based on visual-text consistency and emotional coherence. Extensive experiments demonstrate that EMO-R3 significantly improves both the interpretability and emotional intelligence of MLLMs, achieving superior performance across multiple visual emotional understanding benchmarks.

Figures

Figures reproduced from arXiv: 2602.23802 by Jian Luan, Kehua Su, Mang Ye, Pei Fu, Wenke Huang, Yihao Yang, Yiyang Fang, Zhenbo Luo.

Figure 1
Figure 1. Figure 1: Illustration of the motivation. (a) SFT relies on hu￾man annotations but is constrained by fixed labels and limited cat￾egories, resulting in poor generalization and interpretability. It per￾forms well on in-domain pairs like “landscape–awe” but struggles with out-of-domain or unseen cases (e.g., “movement-surprise”). (b) Although GRPO improves generalization, its think process is not emotion-oriented and … view at source ↗
Figure 2
Figure 2. Figure 2: Architecture illustration of EMO-R3. The upper part presents the Structured Emotional Thinking prompt, which consists of three consecutive thinking steps followed by a final answer. The lower part illustrates the Reflective Emotional Reward mechanism, where multiple rollout samples are evaluated based on image–text consistency and emotional coherence, and are jointly optimized with the original Format and … view at source ↗
Figure 3
Figure 3. Figure 3: Training and testing accuracy during the training pro [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Case study between GRPO and EMO-R3 on the EmoSet dataset. Please see Sec. 4.4 for details. 0 100 200 300 400 500 600 700 0 10 20 30 40 50 60 70 80 Ours GRPO 126% Training Time Per Step (s) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Efficiency analysis on the training process. See Sec. 4.5. tations. When the Reflective Emotional Reward (RER) is further introduced, the model achieves additional improve￾ments, suggesting that reflective self-assessment encour￾ages the model to better align its emotional reasoning with the underlying multimodal evidence. Taken together, these findings demonstrate that the combined use of SET and RER not … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs

    cs.CV 2026-04 unverdicted novelty 7.0

    EmoTrans is a new video benchmark with four progressive tasks that measures how well current multimodal LLMs handle dynamic emotion transitions rather than static recognition.

  2. ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detection

    cs.AI 2026-06 unverdicted novelty 6.0

    ThinkDeception introduces MLLMs, a multimodal CoT dataset, and VAC-GRPO progressive RL to convert deception detection into interpretable reasoning and claims new SOTA accuracy plus rationale quality.

Reference graph

Works this paper leans on

70 extracted references · 37 linked inside Pith · cited by 2 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 6

    Pith/arXiv arXiv 2025

  2. [2]

    Chat-based person retrieval via dialogue-refined cross- modal alignment

    Yang Bai, Yucheng Ji, Min Cao, Jinqiao Wang, and Mang Ye. Chat-based person retrieval via dialogue-refined cross- modal alignment. InCVPR, 2025. 3

    2025

  3. [3]

    Visual instruction tuning with 500x fewer parameters through modality linear representation-steering.arXiv preprint arXiv:2412.12359,

    Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, V olker Tresp, and Yunpu Ma. Visual instruction tuning with 500x fewer parameters through modality linear representation-steering.arXiv preprint arXiv:2412.12359,

    Pith/arXiv arXiv

  4. [4]

    Seed-grpo: Semantic entropy enhanced grpo for uncertainty-aware policy optimization.arXiv preprint arXiv:2505.12346, 2025

    Minghan Chen, Guikun Chen, Wenguan Wang, and Yi Yang. Seed-grpo: Semantic entropy enhanced grpo for uncertainty-aware policy optimization.arXiv preprint arXiv:2505.12346, 2025. 2

    Pith/arXiv arXiv 2025

  5. [5]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InCVPR, pages 24185–24198, 2024. 1, 2

    2024

  6. [6]

    Emotion-llama: Multimodal emo- tion recognition and reasoning with instruction tuning

    Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and Alexander Hauptmann. Emotion-llama: Multimodal emo- tion recognition and reasoning with instruction tuning. In NeurIPS, 2024. 1, 3

    2024

  7. [7]

    Emoe: Modality-specific enhanced dynamic emotion experts

    Yiyang Fang, Wenke Huang, Guancheng Wan, Kehua Su, and Mang Ye. Emoe: Modality-specific enhanced dynamic emotion experts. InCVPR, 2025. 1

    2025

  8. [8]

    Catch your emotion: Sharpening emotion perception in multimodal large language models

    Yiyang Fang, Jian Liang, Wenke Huang, He Li, Kehua Su, and Mang Ye. Catch your emotion: Sharpening emotion perception in multimodal large language models. InICML,

  9. [9]

    Video-r1: Reinforcing video reasoning in mllms.arXiv preprint arXiv:2503.21776,

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms.arXiv preprint arXiv:2503.21776,

    Pith/arXiv arXiv

  10. [10]

    On de- signing effective rl reward at training time for llm reasoning

    Jiaxuan Gao, Shusheng Xu, Wenjie Ye, Weilin Liu, Chuyi He, Wei Fu, Zhiyu Mei, Guangju Wang, and Yi Wu. On de- signing effective rl reward at training time for llm reasoning. arXiv preprint arXiv:2410.15115, 2024. 3

    Pith/arXiv arXiv 2024

  11. [11]

    Making the V in VQA matter: Ele- vating the role of image understanding in Visual Question Answering

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Ba- tra, and Devi Parikh. Making the V in VQA matter: Ele- vating the role of image understanding in Visual Question Answering. InCVPR, 2017. 3

    2017

  12. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 2, 3

    Pith/arXiv arXiv 2025

  13. [13]

    Onellm: One framework to align all modalities with language

    Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, and Xi- angyu Yue. Onellm: One framework to align all modalities with language. InCVPR, pages 26584–26595, 2024. 3

    2024

  14. [14]

    Boosting mllm reasoning with text-debiased hint- grpo.arXiv preprint arXiv:2503.23905, 2025

    Qihan Huang, Weilong Dai, Jinlong Liu, Wanggui He, Hao Jiang, Mingli Song, Jingyuan Chen, Chang Yao, and Jie Song. Boosting mllm reasoning with text-debiased hint- grpo.arXiv preprint arXiv:2503.23905, 2025. 3

    Pith/arXiv arXiv 2025

  15. [15]

    Keeping yourself is important in downstream tuning multimodal large language model.arXiv preprint arXiv:2503.04543, 2025

    Wenke Huang, Jian Liang, Xianda Guo, Yiyang Fang, Guancheng Wan, Xuankun Rong, Chi Wen, Zekun Shi, Qingyun Li, Didi Zhu, et al. Keeping yourself is important in downstream tuning multimodal large language model.arXiv preprint arXiv:2503.04543, 2025. 1, 2

    Pith/arXiv arXiv 2025

  16. [16]

    Learn from downstream and be yourself in multimodal large language model fine-tuning

    Wenke Huang, Jian Liang, Zekun Shi, Didi Zhu, Guancheng Wan, He Li, Bo Du, Dacheng Tao, and Mang Ye. Learn from downstream and be yourself in multimodal large language model fine-tuning. InICML, 2025. 1, 2

    2025

  17. [17]

    Be confident: Uncovering overfitting in mllm multi-task tuning

    Wenke Huang, Jian Liang, Guancheng Wan, Didi Zhu, He Li, Jiawei Shao, Mang Ye, Bo Du, and Dacheng Tao. Be confident: Uncovering overfitting in mllm multi-task tuning. InICML, 2025. 3

    2025

  18. [18]

    Mapo: Mixed advantage policy optimization.arXiv preprint arXiv:2509.18849, 2025

    Wenke Huang, Quan Zhang, Yiyang Fang, Jian Liang, Xu- ankun Rong, Huanjin Yao, Guancheng Wan, Ke Liang, Wen- wen He, Mingjun Li, et al. Mapo: Mixed advantage policy optimization.arXiv preprint arXiv:2509.18849, 2025. 2

    arXiv 2025

  19. [19]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. InCVPR, pages 6700–6709, 2019. 3

    2019

  20. [20]

    Learning from teaching reg- ularization: Generalizable correlations should be easy to im- itate.NeurIPS, 37:966–994, 2024

    Can Jin, Tong Che, Hongwu Peng, Yiyuan Li, Dimitris Metaxas, and Marco Pavone. Learning from teaching reg- ularization: Generalizable correlations should be easy to im- itate.NeurIPS, 37:966–994, 2024. 2

    2024

  21. [21]

    Two heads are better than one: Test- time scaling of multi-agent collaborative reasoning.arXiv preprint arXiv:2504.09772, 2025

    Can Jin, Hongwu Peng, Qixin Zhang, Yujin Tang, Dimitris N Metaxas, and Tong Che. Two heads are better than one: Test- time scaling of multi-agent collaborative reasoning.arXiv preprint arXiv:2504.09772, 2025. 2

    Pith/arXiv arXiv 2025

  22. [22]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024. 1, 2

    Pith/arXiv arXiv 2024

  23. [23]

    Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning.arXiv preprint arXiv:2504.06958, 2025

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning.arXiv preprint arXiv:2504.06958, 2025. 3

    Pith/arXiv arXiv 2025

  24. [24]

    Mon- key: Image resolution and text label are important things for large multi-modal models

    Zhang Li, Biao Yang, Qiang Liu, Zhiyin Ma, Shuo Zhang, Jingxu Yang, Yabo Sun, Yuliang Liu, and Xiang Bai. Mon- key: Image resolution and text label are important things for large multi-modal models. InCVPR, pages 26763–26773,

  25. [25]

    Explainable multimodal emotion recognition.arXiv preprint arXiv:2306.15401, 2023

    Zheng Lian, Haiyang Sun, Licai Sun, Hao Gu, Zhuofan Wen, Siyuan Zhang, Shun Chen, Mingyu Xu, Ke Xu, Kang Chen, et al. Explainable multimodal emotion recognition.arXiv preprint arXiv:2306.15401, 2023. 3

    Pith/arXiv arXiv 2023

  26. [26]

    Ex- plainable multimodal emotion reasoning.CoRR, 2023

    Zheng Lian, Licai Sun, Mingyu Xu, Haiyang Sun, Ke Xu, Zhuofan Wen, Shun Chen, Bin Liu, and Jianhua Tao. Ex- plainable multimodal emotion reasoning.CoRR, 2023. 3

    2023

  27. [27]

    Affectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models.arXiv preprint arXiv:2501.16566, 2025

    Zheng Lian, Haoyu Chen, Lan Chen, Haiyang Sun, Licai Sun, Yong Ren, Zebang Cheng, Bin Liu, Rui Liu, Xiaojiang Peng, et al. Affectgpt: A new dataset, model, and benchmark for emotion understanding with multimodal large language models.arXiv preprint arXiv:2501.16566, 2025. 1, 2, 3

    Pith/arXiv arXiv 2025

  28. [28]

    Lorasculpt: Sculpting lora for harmonizing gen- eral and specialized knowledge in multimodal large language models

    Jian Liang, Wenke Huang, Guancheng Wan, Qu Yang, and Mang Ye. Lorasculpt: Sculpting lora for harmonizing gen- eral and specialized knowledge in multimodal large language models. InCVPR, 2025. 1, 2

    2025

  29. [29]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755, 2014. 3

    2014

  30. [30]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InCVPR,

  31. [31]

    Reinforcement learning meets large language models: A survey of advancements and applications across the llm lifecycle.arXiv preprint arXiv:2509.16679, 2025

    Keliang Liu, Dingkang Yang, Ziyun Qian, Weijie Yin, Yuchi Wang, Hongsheng Li, Jun Liu, Peng Zhai, Yang Liu, and Lihua Zhang. Reinforcement learning meets large language models: A survey of advancements and applications across the llm lifecycle.arXiv preprint arXiv:2509.16679, 2025. 2, 3

    Pith/arXiv arXiv 2025

  32. [32]

    Dora: Weight-decomposed low-rank adaptation.arXiv preprint arXiv:2402.09353, 2024

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. Dora: Weight-decomposed low-rank adaptation.arXiv preprint arXiv:2402.09353, 2024. 3

    Pith/arXiv arXiv 2024

  33. [33]

    Guardreasoner-vl: Safeguarding vlms via reinforced reasoning.arXiv preprint arXiv:2505.11049,

    Yue Liu, Shengfang Zhai, Mingzhe Du, Yulin Chen, Tri Cao, Hongcheng Gao, Cheng Wang, Xinfeng Li, Kun Wang, Junfeng Fang, et al. Guardreasoner-vl: Safeguarding vlms via reinforced reasoning.arXiv preprint arXiv:2505.11049,

    Pith/arXiv arXiv

  34. [34]

    Visual- rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual- rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025. 3

    Pith/arXiv arXiv 2025

  35. [35]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. InNeurIPS,

  36. [36]

    ChartQA: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Do Long, Jia Qing Tan, Shafiq Joty, and Ena- mul Hoque. ChartQA: A benchmark for question answering about charts with visual and logical reasoning. InACL, 2022. 3

    2022

  37. [37]

    Con- templating visual emotions: Understanding and overcoming dataset bias

    Rameswar Panda, Jianming Zhang, Haoxiang Li, Joon- Young Lee, Xin Lu, and Amit K Roy-Chowdhury. Con- templating visual emotions: Understanding and overcoming dataset bias. InECCV, pages 579–595, 2018. 6

    2018

  38. [38]

    A mixed bag of emotions: Model, predict, and transfer emotion distributions

    Kuan-Chuan Peng, Tsuhan Chen, Amir Sadovnik, and An- drew C Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. InCVPR, pages 860–868,

  39. [39]

    Direct preference optimization: Your language model is secretly a reward model.NeurIPS, 36:53728–53741, 2023

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model.NeurIPS, 36:53728–53741, 2023. 2

    2023

  40. [40]

    Scalpel vs

    Neel Rajani, Aryo Pradipta Gema, Seraphina Goldfarb- Tarrant, and Ivan Titov. Scalpel vs. hammer: Grpo ampli- fies existing capabilities, sft replaces them.arXiv preprint arXiv:2507.10616, 2025. 1

    Pith/arXiv arXiv 2025

  41. [41]

    Group robust preference optimization in reward- free rlhf

    Shyam Sundhar Ramesh, Yifan Hu, Iason Chaimalas, Viraj Mehta, Pier Giuseppe Sessa, Haitham Bou Ammar, and Ilija Bogunovic. Group robust preference optimization in reward- free rlhf. InNeurIPS, pages 37100–37137, 2024. 2, 3

    2024

  42. [42]

    Improving llm-generated code quality with grpo.arXiv preprint arXiv:2506.02211, 2025

    Maxime Robeyns and Laurence Aitchison. Improving llm-generated code quality with grpo.arXiv preprint arXiv:2506.02211, 2025. 2

    Pith/arXiv arXiv 2025

  43. [43]

    Backdoor clean- ing without external guidance in mllm fine-tuning.arXiv preprint arXiv:2505.16916, 2025

    Xuankun Rong, Wenke Huang, Jian Liang, Jinhe Bi, Xun Xiao, Yiming Li, Bo Du, and Mang Ye. Backdoor clean- ing without external guidance in mllm fine-tuning.arXiv preprint arXiv:2505.16916, 2025. 1

    Pith/arXiv arXiv 2025

  44. [44]

    Safegrpo: Self-rewarded mul- timodal safety alignment via rule-governed policy optimiza- tion.arXiv preprint arXiv:2511.12982, 2025

    Xuankun Rong, Wenke Huang, Tingfeng Wang, Daiguo Zhou, Bo Du, and Mang Ye. Safegrpo: Self-rewarded mul- timodal safety alignment via rule-governed policy optimiza- tion.arXiv preprint arXiv:2511.12982, 2025. 2

    arXiv 2025

  45. [45]

    Proximal policy optimization algo- rithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms.arXiv preprint arXiv:1707.06347, 2017. 2, 3

    Pith/arXiv arXiv 2017

  46. [46]

    Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of math- ematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 2, 3, 6

    Pith/arXiv arXiv 2024

  47. [47]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. InCVPR, pages 8317–8326, 2019. 3

    2019

  48. [48]

    Delving into rl for image generation with cot: A study on dpo vs

    Chengzhuo Tong, Ziyu Guo, Renrui Zhang, Wenyu Shan, Xinyu Wei, Zhenghao Xing, Hongsheng Li, and Pheng-Ann Heng. Delving into rl for image generation with cot: A study on dpo vs. grpo.arXiv preprint arXiv:2505.17017, 2025. 2

    Pith/arXiv arXiv 2025

  49. [49]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3

    Pith/arXiv arXiv 2023

  50. [50]

    Safety in large reasoning models: A survey.arXiv preprint arXiv:2504.17704, 2025

    Cheng Wang, Yue Liu, Baolong Li, Duzhen Zhang, Zhongzhi Li, and Junfeng Fang. Safety in large reasoning models: A survey.arXiv preprint arXiv:2504.17704, 2025. 3

    Pith/arXiv arXiv 2025

  51. [51]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 1, 2

    Pith/arXiv arXiv 2024

  52. [52]

    Chain-of-thought prompting elicits reasoning in large lan- guage models.NeurIPS, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models.NeurIPS, 35:24824–24837, 2022. 3

    2022

  53. [53]

    Emovit: Revolutionizing emotion insights with vi- sual instruction tuning

    Hongxia Xie, Chu-Jun Peng, Yu-Wen Tseng, Hung-Jen Chen, Chan-Feng Hsu, Hong-Han Shuai, and Wen-Huang Cheng. Emovit: Revolutionizing emotion insights with vi- sual instruction tuning. InCVPR, pages 26596–26605, 2024. 1, 3

    2024

  54. [54]

    Emo-llama: Enhancing facial emo- tion understanding with instruction tuning.arXiv preprint arXiv:2408.11424, 2024

    Bohao Xing, Zitong Yu, Xin Liu, Kaishen Yuan, Qi- lang Ye, Weicheng Xie, Huanjing Yue, Jingyu Yang, and Heikki K ¨alvi¨ainen. Emo-llama: Enhancing facial emo- tion understanding with instruction tuning.arXiv preprint arXiv:2408.11424, 2024. 1, 3

    Pith/arXiv arXiv 2024

  55. [55]

    Is dpo superior to ppo for llm alignment? a comprehensive study

    Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. Is dpo superior to ppo for llm alignment? a comprehensive study. arXiv preprint arXiv:2404.10719, 2024. 2

    Pith/arXiv arXiv 2024

  56. [56]

    Context de-confounded emo- tion recognition

    Dingkang Yang, Zhaoyu Chen, Yuzheng Wang, Shunli Wang, Mingcheng Li, Siao Liu, Xiao Zhao, Shuai Huang, Zhiyan Dong, Peng Zhai, et al. Context de-confounded emo- tion recognition. InCVPR, pages 19005–19015, 2023. 1

    2023

  57. [57]

    Emoset: A large-scale visual emotion dataset with rich attributes

    Jingyuan Yang, Qirui Huang, Tingting Ding, Dani Lischin- ski, Danny Cohen-Or, and Hui Huang. Emoset: A large-scale visual emotion dataset with rich attributes. InICCV, pages 20383–20394, 2023. 6

    2023

  58. [58]

    Emollm: Multimodal emo- tional understanding meets large language models.arXiv preprint arXiv:2406.16442, 2024

    Qu Yang, Mang Ye, and Bo Du. Emollm: Multimodal emo- tional understanding meets large language models.arXiv preprint arXiv:2406.16442, 2024. 1, 3

    Pith/arXiv arXiv 2024

  59. [59]

    Treerpo: Tree relative policy optimization.arXiv preprint arXiv:2506.05183, 2025

    Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. Treerpo: Tree relative policy optimization.arXiv preprint arXiv:2506.05183, 2025. 2

    arXiv 2025

  60. [60]

    R1-sharevl: Incentivizing reasoning capability of multimodal large language models via share-grpo.arXiv preprint arXiv:2505.16673, 2025

    Huanjin Yao, Qixiang Yin, Jingyi Zhang, Min Yang, Yibo Wang, Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, et al. R1-sharevl: Incentivizing reasoning capability of multimodal large language models via share-grpo.arXiv preprint arXiv:2505.16673, 2025. 2

    Pith/arXiv arXiv 2025

  61. [61]

    A survey of safety on large vision- language models: Attacks, defenses and evaluations.arXiv preprint arXiv:2502.14881, 2025

    Mang Ye, Xuankun Rong, Wenke Huang, Bo Du, Nenghai Yu, and Dacheng Tao. A survey of safety on large vision- language models: Attacks, defenses and evaluations.arXiv preprint arXiv:2502.14881, 2025. 1

    Pith/arXiv arXiv 2025

  62. [62]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions.TACL, 2:67–78, 2014

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hocken- maier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions.TACL, 2:67–78, 2014. 3

    2014

  63. [63]

    Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xi- aochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gao- hong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025. 2, 6

    Pith/arXiv arXiv 2025

  64. [64]

    R1-vl: Learn- ing to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learn- ing to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025. 2, 3

    Pith/arXiv arXiv 2025

  65. [65]

    Microemo: Time-sensitive multimodal emotion recognition with subtle clue dynamics in video dialogues

    Liyun Zhang, Zhaojie Luo, Shuqiong Wu, and Yuta Nakashima. Microemo: Time-sensitive multimodal emotion recognition with subtle clue dynamics in video dialogues. In ACM MM Workshop, pages 110–115, 2024. 1, 3

    2024

  66. [66]

    How can llm guide rl? a value-based approach

    Shenao Zhang, Sirui Zheng, Shuqi Ke, Zhihan Liu, Wanxin Jin, Jianbo Yuan, Yingxiang Yang, Hongxia Yang, and Zhao- ran Wang. How can llm guide rl? a value-based approach. arXiv preprint arXiv:2402.16181, 2024. 3

    Pith/arXiv arXiv 2024

  67. [67]

    Facephi: Lightweight multimodal large language model for facial landmark emotion recogni- tion

    Hongjin Zhao, Zheyuan Liu, Yang Liu, Zhenyue Qin, Jiaxu Liu, and Tom Gedeon. Facephi: Lightweight multimodal large language model for facial landmark emotion recogni- tion. InICLR Workshop, 2024. 1, 3

    2024

  68. [68]

    Galore: Memory- efficient llm training by gradient low-rank projection.arXiv preprint arXiv:2403.03507, 2024

    Jiawei Zhao, Zhenyu Zhang, Beidi Chen, Zhangyang Wang, Anima Anandkumar, and Yuandong Tian. Galore: Memory- efficient llm training by gradient low-rank projection.arXiv preprint arXiv:2403.03507, 2024. 3

    Pith/arXiv arXiv 2024

  69. [69]

    R1-omni: Ex- plainable omni-multimodal emotion recognition with rein- forcement learning.arXiv preprint arXiv:2503.05379, 2025

    Jiaxing Zhao, Xihan Wei, and Liefeng Bo. R1-omni: Ex- plainable omni-multimodal emotion recognition with rein- forcement learning.arXiv preprint arXiv:2503.05379, 2025. 2, 3

    Pith/arXiv arXiv 2025

  70. [70]

    Reinforced mllm: A survey on rl-based reasoning in multimodal large language models

    Guanghao Zhou, Panjia Qiu, Cen Chen, Jie Wang, Zheming Yang, Jian Xu, and Minghui Qiu. Reinforced mllm: A survey on rl-based reasoning in multimodal large language models. arXiv preprint arXiv:2504.21277, 2025. 2, 3

    Pith/arXiv arXiv 2025