Pith. sign in

REVIEW 3 major objections 5 minor 4 cited by

GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding a group-relative consistency bonus to outcome-supervised RL fixes the reasoning-answer mismatch in multimodal video models.

desk verdict GRPO-CARE is a solid incremental method for RL post-training of video MLLMs with a genuinely useful new benchmark, but the headline consistency gain rests on an unvalidated likelihood proxy that may measure self-predictability rather than logical coherence. read the letter →

arxiv 2506.16141 v1 pith:ZUB5MBFD submitted 2025-06-19 cs.CV cs.AIcs.CLcs.LG

Yi Chen , Yuying Ge , Rui Wang , Yixiao Ge , Junhao Cheng , Ying Shan , Xihui Liu This is my paper
classification cs.CVcs.AIcs.CLcs.LG
keywords multimodallargelanguagemodelsvideoreasoningreinforcementlearningGRPOchain-of-thoughtconsistencyrewarddesigngeneralizationbenchmarkegocentricunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that outcome-supervised reinforcement learning for multimodal large language models can be improved by rewarding not just whether the final answer is correct but whether the reasoning trace actually supports that answer. Using a new three-level video benchmark built from egocentric task videos, it shows that standard GRPO reaches only a 57.9 percent reasoning-to-answer consistency rate even while improving accuracy. Its proposed GRPO-CARE replaces the KL penalty with an adaptive, group-relative consistency bonus computed with a slowly evolving reference model, yielding a 6.7 percent gain on the hardest out-of-distribution level and a 24.5 percent consistency improvement. The same training recipe transfers to six general video understanding benchmarks, which is why the paper argues consistency-aware rewards are a general post-training tool rather than a benchmark-specific fix.

What carries the argument

The central machinery is reference-likelihood calibration within a two-stage filtering pipeline. A reference model, initialized from the same weights as the online policy and updated by exponential moving average, is used in Phase 3 of the algorithm to score high-accuracy trajectories: it computes the average token likelihood of the answer conditioned on the multimodal question and the generated reasoning trace, clips the value at a maximum threshold, then compares each trajectory's clipped likelihood with the group mean minus a small margin. Trajectories that pass both the accuracy baseline and this relative consistency baseline receive a sparse consistency bonus, and the final advantage is computed from the total reward with no KL term. This makes the consistency signal adaptive and group-relative, which the paper argues avoids the over-constraint of static KL penalties and the over-fitting of dense likelihood rewards.

What would settle it

Generate correct-answer trajectories from a trained model, then corrupt each reasoning trace in two ways—replace it with unrelated fluent text and reverse the order of its steps—and measure the reference-model answer likelihood. If either corruption fails to lower the likelihood substantially, the consistency bonus is not measuring logical support, and the reported coherence gains would need a different explanation.

Watch

Extended reading notes

Core claim

The paper's central claim is that the logical consistency between a reasoning chain and its final answer is a usable training signal, and that a stable reference model can supply that signal without process annotations. GRPO-CARE computes, for each sampled trajectory with a correct answer, the average per-token likelihood the reference model assigns to the answer given the question and the reasoning trace. That likelihood is clipped, compared against the group mean minus a margin, and trajectories above the baseline receive a sparse bonus weighted by accuracy; the KL penalty is removed from the GRPO objective. On SEED-Bench-R1 the resulting policy beats standard GRPO on all three difficulty levels, and the consistency rate rises from 57.9 percent to 82.4 percent, so the paper concludes that group-relative consistency rewards provide a more reliable learning signal than KL regularization or dense likelihood weighting.

Load-bearing premise

The load-bearing premise is that the reference model's token-level likelihood of the answer given the reasoning trace tracks real logical coherence; if that proxy rewards fluent-but-wrong reasoning, the consistency bonus could reinforce the shortcuts it is meant to remove.

Editorial extensions

If this is right

  • The paper's results imply that outcome-supervised RL for video MLLMs can be made more interpretable without explicit reasoning annotations, by rewarding reasoning that supports the final answer.
  • Removing the KL penalty in favor of a group-relative consistency bonus should permit broader exploration of reasoning paths, which may matter for tasks where multiple valid reasoning styles exist.
  • SEED-Bench-R1's three-level validation split gives a reusable protocol for measuring whether post-training methods generalize beyond their training environment and task distribution.
  • Transfer results on six general video benchmarks suggest the consistency bonus is not tied to the planning task it was trained on and could be a default reward component for MLLM post-training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same reference-likelihood proxy should transfer to text-only LLM reasoning, where no visual grounding is needed; if it does, it offers a cheap alternative to stepwise process reward models.
  • My inference: the method is vulnerable to a form of reward hacking in which the policy learns reasoning templates that are fluent and answer-consistent in likelihood space but still shortcut genuine understanding; checking whether consistency gains survive when the reference model is re-initialized from an independent checkpoint would test this.
  • My inference: the benchmark's three-level generalization structure could be stress-tested further by adding unseen task domains beyond hobbies, recreation, and work, such as navigation or manipulation, where perception-reasoning balance is even more demanding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper targets the degradation of reasoning coherence observed when outcome-supervised GRPO is applied to multimodal LLMs on video-based planning tasks. It introduces SEED-Bench-R1, a video understanding benchmark built on egocentric videos, with a 50,269-sample training set and a three-level validation hierarchy for in-distribution, cross-environment, and cross-environment-task generalization. The main method, GRPO-CARE, replaces the standard KL penalty with an adaptive consistency bonus: for each trajectory, an EMA-updated reference model computes the token-level likelihood p_g of the answer given the question and reasoning trace, and a sparse group-relative bonus is given to correct trajectories whose p_g exceeds a baseline. Experiments with Qwen2.5-VL-7B report consistent gains over standard GRPO on SEED-Bench-R1 (e.g., L3 accuracy 46.7 to 53.4, consistency rate 57.9% to 82.4%) and transfer gains on six general video benchmarks. The paper claims that GRPO-CARE improves both answer accuracy and logical reasoning coherence without explicit process supervision.

Significance. If the claims hold, the paper makes two useful contributions: a systematically designed benchmark with clearly separated generalization tiers, and a simple, annotation-free consistency-aware RL method that improves both accuracy and reasoning quality. The benchmark construction is a solid practical contribution, and the transfer results in Table 4 indicate that training on SEED-Bench-R1 or Video-R1 data with CARE yields competitive results. The main comparisons are plausible and the method is clean. However, the central interpretability claim is not yet fully supported: the training reward is an unvalidated likelihood proxy, the consistency metric is a single GPT-4.1 pass with no human agreement study, and no error bars or hyperparameter sensitivity are reported. The stress-test concern about p_g as a self-predictability proxy lands: because the reference model is an EMA of the online policy and the KL penalty is removed, the reward could reward shallow cues rather than logical coherence. The paper is therefore a defensible but not fully established central claim, and the missing validation is fixable within the scope of a revision.

major comments (3)
  1. [§4, Algorithm 1 (Phase 3, line 13) and Table 3] The consistency bonus is computed from p_g, the EMA reference model's average answer-token likelihood given the question and reasoning trace, but the paper never validates that this proxy tracks the GPT-4.1 consistency judgments used to report the headline 24.5% consistency gain. There is no correlation analysis between p_g and GPT-4.1 (or human) consistency ratings, no perturbation test showing that p_g is sensitive to the logical content of the reasoning, and no human-agreement study for the GPT-4.1 judge. Because the reference model is an EMA of the online policy and the KL penalty is removed in Phase 5, p_g can be inflated by shallow cues such as restating the answer inside the reasoning or producing short templated reasoning that makes the answer predictable, without the reasoning being grounded in the video; the group-relative baseline does not remove this bias if all selected trajectories share the same cue. Consequently, the claim that GRPO-CARE improves logical coherence rather than self-predictability is not yet supported. I recommend adding (i) a correlation analysis of p_g with GPT-4.1 consistency on held-out trajectories, (ii) a human consistency-judgment agreement study on a sample, and (iii) a qualitative or quantitative audit showing that p_g changes when the reasoning is made inconsistent with the answer.
  2. [§4.1, Tables 2 and 3] All comparisons are single runs with no error bars or significance tests. For example, Table 3 reports GRPO L1=52.3 vs CARE L1=57.0 and L3=46.7 vs 53.4, but without variance over seeds or bootstrap intervals these differences, and the consistency-rate differences, cannot be distinguished from noise. I request multiple seeds or bootstrap confidence intervals for the main SEED-Bench-R1 comparisons, and the same for the consistency rate, which is currently computed by a single GPT-4.1 evaluation pass.
  3. [§4.1 and Algorithm 1] The method introduces at least seven hyperparameters (lambda_cons, gamma_acc, gamma_p, epsilon_p, EMA alpha, EMA update frequency k, and group size G) and removes the standard KL regularizer, but no sensitivity analysis is reported. The reader cannot tell whether the reported gains require fine-tuned constants or are stable over a reasonable range. This is particularly important because DenseCons and RefGen, which share the same likelihood-calibration idea, underperform on L3 in Table 3. I ask for an ablation over the central constants (e.g., lambda_cons, gamma_p, epsilon_p, and k) on at least one OOD level.
minor comments (5)
  1. [Abstract and Table 3] The '6.7% performance gain' and '24.5% improvement in consistency' are percentage-point differences (46.7 to 53.4 and 57.9% to 82.4%, respectively); please label them as points to avoid confusion with relative percentages.
  2. [§3.2] The phrase 'resolution 128×28×28' appears to be a typo; please state the actual input frame resolution and number of frames.
  3. [§3.2 and Table 2] The base model is called Qwen2.5-VL-Instruct-7B in the text but Qwen2.5-VL-7B in Table 2; please unify the name.
  4. [§4 and Algorithm 1] The text says the training process is 'detailed in Algorithm 4', but the listing is labeled Algorithm 1; please correct the cross-reference.
  5. [Table 4] Values for external methods are taken from the Video-R1 paper; please state the number of frames and evaluation settings used for every row, since several differences are within typical benchmark noise.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular step found: the training reward and the external GPT-4.1 consistency metric are distinct, and the reported gains are not forced by the paper's own definitions.

full rationale

I find no circular step that reduces GRPO-CARE's reported gains to its own inputs. The consistency bonus in Algorithm 1 (lines 13-20) is a token-level reference-model likelihood p_g = (1/|a_g|) sum_i π_phi(a_{g,i} | x, τ_g, a_{g,<i}), used only as a training reward. The reported consistency rate is independently measured: the paper states that 'consistency is evaluated by GPT-4.1 to assess whether the reasoning sufficiently supports the answer.' These are distinct quantities, so the 24.5% consistency improvement is not the training reward by construction. The benchmark SEED-Bench-R1 reuses the authors' own EgoPlan-Bench and EgoPlan-Bench2 as video/question sources, but this is dataset provenance rather than a load-bearing self-citation: the training and validation splits are newly constructed and human-verified. The EMA-updated reference model does create a self-referential reward loop (the reference is a smoothed copy of the online policy), but this is a standard training mechanism, not a derivation that assumes the conclusion. The main weakness is that the paper never validates that p_g tracks human or GPT-4.1 consistency judgments, so the consistency gain could partly reflect answer predictability rather than logical coherence. That is a correctness/validity concern, not circularity under the definitions in the reviewing rules.

Assumptions & free parameters 7 free parameters · 3 assumptions · 0 invented entities

The consistency bonus relies on a reference likelihood proxy for coherence, which is the main unvalidated assumption. Hyperparameters are manually chosen and not sensitivity-tested.

free parameters (7)
  • lambda_cons = 0.5
    Consistency reward coefficient, chosen by hand (Algorithm 1).
  • gamma_acc = 0.1
    Minimum accuracy threshold for relative baseline in trajectory selection (Algorithm 1).
  • gamma_p = 0.95
    Maximum likelihood cap to prevent over-optimization (Algorithm 1).
  • epsilon_p = 0.01
    Consistency margin subtracted from group mean before sparse bonus (Algorithm 1).
  • ema_alpha = 0.995
    EMA decay for reference model updates (Algorithm 1).
  • ema_update_freq = k=10 steps
    Reference model updated every k steps (Algorithm 1).
  • group_size_G = not reported
    Number of sampled trajectories per question, not specified in paper, affects advantage normalization and relative baseline.
assumptions (3)
  • domain assumption Reference-model likelihood of the answer conditioned on reasoning is a valid proxy for logical coherence.
    Core of consistency bonus; no validation that likelihood tracks human-perceived coherence.
  • domain assumption GPT-4.1 consistency evaluation accurately measures logical coherence.
    Consistency rates in Tables 3 are from GPT-4.1, no human agreement reported.
  • domain assumption Outcome-supervised GRPO on MLLMs causes reasoning-answer inconsistency because of sparse reward and KL penalty.
    Motivation for method, based on case studies, not a controlled test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning." pith.science (2026). https://pith.science/paper/ZUB5MBFD

@misc{pith2026250616141,
  author       = {Pith},
  title        = {Pith review of: GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZUB5MBFD}},
  note         = {Machine review of arXiv:2506.16141}
}
read the original abstract

Recent reinforcement learning approaches, such as outcome-supervised GRPO, have advanced Chain-of-Thought reasoning in large language models (LLMs), yet their adaptation to multimodal LLMs (MLLMs) is unexplored. To address the lack of rigorous evaluation for MLLM post-training methods, we introduce SEED-Bench-R1, a benchmark with complex real-world videos requiring balanced perception and reasoning. It offers a large training set and evaluates generalization across three escalating challenges: in-distribution, cross-environment, and cross-environment-task scenarios. Using SEED-Bench-R1, we find that standard GRPO, while improving answer accuracy, often reduces logical coherence between reasoning steps and answers, with only a 57.9% consistency rate. This stems from reward signals focusing solely on final answers, encouraging shortcuts, and strict KL penalties limiting exploration.To address this, we propose GRPO-CARE, a consistency-aware RL framework optimizing both answer correctness and reasoning coherence without explicit supervision. GRPO-CARE introduces a two-tiered reward: (1) a base reward for answer correctness, and (2) an adaptive consistency bonus, computed by comparing the model's reasoning-to-answer likelihood (via a slowly-evolving reference model) against group peers.This dual mechanism amplifies rewards for reasoning paths that are both correct and logically consistent. Replacing KL penalties with this adaptive bonus, GRPO-CARE outperforms standard GRPO on SEED-Bench-R1, achieving a 6.7% performance gain on the hardest evaluation level and a 24.5% improvement in consistency. It also shows strong transferability, improving model performance across diverse video understanding benchmarks. Our work contributes a systematically designed benchmark and a generalizable post-training framework, advancing the development of more interpretable and robust MLLMs.

Figures

Figures reproduced from arXiv: 2506.16141 by the authors.

Figure 1
Figure 1. (a) SEED-Bench-R1 (SB-R1) provides a systematic, three-level evaluation of post-training methods for MLLMs in video understanding, encompassing tasks that require both perception and reasoning to tackle complex real-world scenarios. (b) Our analysis identifies a key limitation of standard outcome-supervised GRPO: while it improves answer accuracy, it often compromises logical consistency between reasoning and answer… view at source ↗
Figure 2
Figure 2. Case study of an L3 question from SEED-Bench-R1, showing a video of task progress, [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. GRPO-CARE uses a two-tier reward system: a base reward for answer correctness ( [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    REVA-PO stabilizes GRPO-style RL for CXR report generation via response-level adaptive KL weights and validation-anchored policy resets, reporting new SOTA BLEU and clinical F1 scores.

  2. From Passive Observer to Active Critic: Reinforcement Learning Elicits Process Reasoning for Robotic Manipulation

    cs.RO 2026-03 conditional novelty 6.0 of 10

    Outcome-based GRPO plus initial/current-state image anchors turns a 7B video MLLM into a progress critic that halves MAE versus specialized baselines and reaches 67% on RoboFail zero-shot.

  3. VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    VL-Cogito, trained with progressive curriculum RL, online difficulty weighting, and dynamic length rewards, matches or beats prior reasoning MLLMs on ten multimodal benchmarks.

  4. Towards Hallucination-Free Music: A Reinforcement Learning Preference Optimization Framework for Reliable Song Generation

    cs.SD 2025-08 conditional novelty 5.0 of 10

    PER-based preference optimization (DPO, PPO, GRPO) reduces lyric-to-song hallucination in an audio language model, with the largest gains from DPO plus reject sampling.

Reference graph

Works this paper leans on

50 extracted references · 12 canonical work pages · cited by 4 Pith papers

  1. [1]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  2. [2]

    Learning to reason with LLMs, 2024

    work page 2024

  3. [3]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  4. [4]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    arXiv 2024

  5. [5]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    2022

  6. [6]

    Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms.arXiv preprint arXiv:2502.03373, 2025

    arXiv 2025

  7. [7]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    arXiv 2025

  8. [8]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    arXiv 2025

Show all 50 references
  1. [9]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  2. [10]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  3. [11]

    Video-r1: Reinforcing video reasoning in mllms, 2025

    Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms, 2025

    2025

  4. [12]

    Egoplan-bench: Benchmarking multimodal large language models for human-level planning.arXiv preprint arXiv:2312.06722, 2023

    Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, and Xihui Liu. Egoplan-bench: Benchmarking multimodal large language models for human-level planning.arXiv preprint arXiv:2312.06722, 2023

    2023 arXiv

  5. [13]

    Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios.arXiv preprint arXiv:2412.04447, 2024

    Lu Qiu, Yuying Ge, Yi Chen, Yixiao Ge, Ying Shan, and Xihui Liu. Egoplan-bench2: A benchmark for multimodal large language model planning in real-world scenarios.arXiv preprint arXiv:2412.04447, 2024

    2024 arXiv

  6. [14]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Jian Ma, Evan- gelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100. Internati...

    2022

  7. [15]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF conference on computer vision a...

    2022

  8. [16]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. 10

    2017

  9. [17]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wa...

    2025

  10. [18]

    Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective.arXiv preprint arXiv:2503.20783, 2025

    2025 arXiv

  11. [19]

    Let’s verify step by step, 2023

    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023

    2023

  12. [20]

    Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275, 2022

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback.arXiv preprint arXiv:2211.14275, 2022

    2022 arXiv

  13. [21]

    Alphamath almost zero: Process supervision without process

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: Process supervision without process. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems

  14. [22]

    Improve mathematical reasoning in language models by automated process supervision.arXiv preprint arXiv:2406.06592, 2024

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision.arXiv preprint arXiv:2406.06592, 2024

    2024 arXiv

  15. [23]

    Math-shepherd: A label-free step-by-step verifier for llms in mathematical reasoning.arXiv preprint arXiv:2312.08935, 2023

    Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. Math-shepherd: A label-free step-by-step verifier for llms in mathematical reasoning.arXiv preprint arXiv:2312.08935, 2023

    2023 arXiv

  16. [24]

    Llm critics help catch bugs in mathematics: Towards a better mathematical verifier with natural language feedback.arXiv preprint arXiv:2406.14024, 2024

    Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, et al. Llm critics help catch bugs in mathematics: Towards a better mathematical verifier with natural language feedback.arXiv preprint arXiv:2406.14024, 2024

    2024 arXiv

  17. [25]

    Evaluating mathematical reasoning beyond accuracy

    Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. Evaluating mathematical reasoning beyond accuracy. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 27723–27730, 2025

    2025

  18. [26]

    Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240, 2024

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction.arXiv preprint arXiv:2408.15240, 2024

    2024 arXiv

  19. [27]

    Warp: On the benefits of weight averaged rewarded policies, 2024

    Alexandre Ramé, Johan Ferret, Nino Vieillard, Robert Dadashi, Léonard Hussenot, Pierre-Louis Cedoz, Pier Giuseppe Sessa, Sertan Girgin, Arthur Douillard, and Olivier Bachem. Warp: On the benefits of weight averaged rewarded policies, 2024

    2024

  20. [28]

    Gtr: Guided thought reinforcement prevents thought collapse in rl-based vlm agent training, 2025

    Tong Wei, Yijun Yang, Junliang Xing, Yuanchun Shi, Zongqing Lu, and Deheng Ye. Gtr: Guided thought reinforcement prevents thought collapse in rl-based vlm agent training, 2025

    2025

  21. [29]

    Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification

    Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. Mm-math: Advancing multimodal math evaluation with process evaluation and fine-grained classification. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 1358–1375, 2024

    2024

  22. [30]

    Open-r1-video

    Xiaodong Wang and Peixi Peng. Open-r1-video. https://github.com/ Wang-Xiaodong1899/Open-R1-Video, 2025

    2025

  23. [31]

    R1-omni: Explainable omni-multimodal emotion recognition with reinforcing learning.arXiv preprint arXiv:2503.05379, 2025

    Jiaxing Zhao, Xihan Wei, and Liefeng Bo. R1-omni: Explainable omni-multimodal emotion recognition with reinforcing learning.arXiv preprint arXiv:2503.05379, 2025

    2025 arXiv

  24. [32]

    Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild

    Yuanyuan Liu, Wei Dai, Chuanxu Feng, Wenbin Wang, Guanghao Yin, Jiabei Zeng, and Shiguang Shan. Mafw: A large-scale, multi-modal, compound affective database for dynamic facial expression recognition in the wild. InProceedings of the 30th ACM international conference on multim...

    2022

  25. [33]

    Dfew: A large-scale database for recognizing dynamic facial expressions in the wild

    Xingxun Jiang, Yuan Zong, Wenming Zheng, Chuangao Tang, Wanchuang Xia, Cheng Lu, and Jiateng Liu. Dfew: A large-scale database for recognizing dynamic facial expressions in the wild. InProceedings of the 28th ACM international conference on multimedia, pages 2881–2889, 2020

    2020

  26. [34]

    Longvideobench: A benchmark for long- context interleaved video-language understanding.Advances in Neural Information Processing Systems, 37:28828–28857, 2024

    Haoning Wu, Dongxu Li, Bei Chen, and Junnan Li. Longvideobench: A benchmark for long- context interleaved video-language understanding.Advances in Neural Information Processing Systems, 37:28828–28857, 2024

    2024

  27. [35]

    Mvbench: A comprehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22195–22206, 2024

    2024

  28. [36]

    Tempcompass: Do video llms really understand videos? InFindings of the Association for Computational Linguistics ACL 2024, pages 8731–8772, 2024

    Yuanxin Liu, Shicheng Li, Yi Liu, Yuxiang Wang, Shuhuai Ren, Lei Li, Sishuo Chen, Xu Sun, and Lu Hou. Tempcompass: Do video llms really understand videos? InFindings of the Association for Computational Linguistics ACL 2024, pages 8731–8772, 2024

    2024

  29. [37]

    Mmbench-video: A long-form multi-shot benchmark for holistic video understanding

    Xinyu Fang, Kangrui Mao, Haodong Duan, Xiangyu Zhao, Yining Li, Dahua Lin, and Kai Chen. Mmbench-video: A long-form multi-shot benchmark for holistic video understanding. Advances in Neural Information Processing Systems, 37:89098–89124, 2024

    2024

  30. [38]

    Qwen2.5-vl technical report

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025 arXiv

  31. [39]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  32. [40]

    Llama-vid: An image is worth 2 tokens in large language models

    Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. InEuropean Conference on Computer Vision, pages 323–340. Springer, 2024

    2024

  33. [41]

    Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms.arXiv preprint arXiv:2406.07476, 2024

    2024 arXiv

  34. [42]

    Long context transfer from language to vision

    Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406.16852, 2024

    2024 arXiv

  35. [43]

    Vila: On pre-training for visual language models

    Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, and Song Han. Vila: On pre-training for visual language models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26689–26699, 2024

    2024

  36. [44]

    Unhackable temporal rewarding for scalable video mllms.arXiv preprint arXiv:2502.12081, 2025

    En Yu, Kangheng Lin, Liang Zhao, Yana Wei, Zining Zhu, Haoran Wei, Jianjian Sun, Zheng Ge, Xiangyu Zhang, Jingyu Wang, et al. Unhackable temporal rewarding for scalable video mllms.arXiv preprint arXiv:2502.12081, 2025

    2025 arXiv

  37. [45]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  38. [46]

    Kangaroo: A powerful video-language model supporting long-context video input.arXiv preprint arXiv:2408.15542, 2024

    Jiajun Liu, Yibing Wang, Hanghang Ma, Xiaoping Wu, Xiaoqi Ma, Xiaoming Wei, Jianbin Jiao, Enhua Wu, and Jie Hu. Kangaroo: A powerful video-language model supporting long-context video input.arXiv preprint arXiv:2408.15542, 2024

    2024 arXiv

  39. [47]

    Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024

    Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces.arXiv preprint arXiv:2412.14171, 2024. 12

    2024 arXiv

  40. [48]

    Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos

    Kairui Hu, Penghao Wu, Fanyi Pu, Wang Xiao, Yuanhan Zhang, Xiang Yue, Bo Li, and Ziwei Liu. Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos. arXiv preprint arXiv:2501.13826, 2025

    2025 arXiv

  41. [49]

    Mmvu: Measuring expert-level multi-discipline video understanding.arXiv preprint arXiv:2501.12380, 2025

    Yilun Zhao, Lujing Xie, Haowei Zhang, Guo Gan, Yitao Long, Zhiyuan Hu, Tongyan Hu, Weiyuan Chen, Chuhan Li, Junyang Song, et al. Mmvu: Measuring expert-level multi-discipline video understanding.arXiv preprint arXiv:2501.12380, 2025

    2025 arXiv

  42. [50]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024. 13

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.