Pith. sign in

REVIEW 4 major objections 4 minor 11 cited by

R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Share-GRPO fixes the two main failure modes of GRPO on multimodal reasoning by sharing responses and rewards across question variants.

desk verdict Useful empirical recipe for MLLM RL — expand question space and share trajectories/advantages — but Eq. 8's importance ratio is not a valid estimator, so the paper's central mechanism lacks a sound derivation. read the letter →

arxiv 2505.16673 v1 pith:JP7CKGEP submitted 2025-05-22 cs.CV cs.AIcs.CL

Huanjin Yao , Qixiang Yin , Jingyi Zhang , Min Yang , Yibo Wang , Wenhao Wu , Fei Su , Li Shen
show 3 more authors
Minghui Qiu Dacheng Tao Jiaxing Huang
This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords multimodallargelanguagemodelsreinforcementlearningGRPOreasoningsemanticallyconsistenttransformationsparserewardadvantagevanishingquestionspaceexpansion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Share-GRPO, a reinforcement-learning algorithm for multimodal large language models that treats each training question not as a single prompt but as a family of semantically equivalent variants. By rewriting the text with GPT-4o and applying safe image transformations such as rotation and noise, it creates an expanded question space, then lets the model generate responses for every variant and shares both the responses and their rewards across the whole family during policy updates. The paper's claim is that this sharing fixes two failures of plain GRPO on multimodal models: rewards are too sparse early in training, and advantages collapse toward zero when all responses to a question become homogeneous. With the shared objective and a hierarchical advantage estimate, R1-ShareVL-7B reaches 75.4 on MathVista, 67.0 on MMStar, and 58.1 on MMMU without any cold-start supervised fine-tuning, outperforming the GRPO baseline and prior RL-trained reasoning MLLMs. A sympathetic reader would care because the method offers a way to get more usable reward signal out of the same questions, rather than more data or a separate reward model.

What carries the argument

The load-bearing object is the expanded question family produced by semantically consistent transformation (SCT). An SCT is a text or image rewrite that is supposed to preserve the question's meaning and correct answer while changing its surface form: GPT-4o paraphrases of the text, and visual operations such as rotation or noise injection with an appended textual hint. The second mechanism is hierarchical shared advantage estimation, which computes a global normalized advantage across all variants' responses and a local normalized advantage within each variant, combining the two only for responses generated from the same variant. The third is the shared policy objective in the paper's Eq. (8), which evaluates each response's probability under every variant of the same seed question, so trajectories discovered while solving one version of the question can improve the model on all versions. Together these make the reward signal denser and the advantages non-degenerate.

What would settle it

Compute the answer-consistency rate of the transformations: take a few hundred training questions, apply the same GPT-4o paraphrases and image rotations or noise used in training, and check with a strong judge or human whether the variant's correct answer matches the original. If a non-negligible fraction of variants flip the answer, the shared reward labels in Eq. (8) are corrupted; a direct test would compare MathVista scores with and without consistency-filtered variants.

Watch

Extended reading notes

Core claim

Share-GRPO's central claim is that the two known failures of GRPO when applied to multimodal models, sparse reward and advantage vanishing, are information problems rather than data or architecture problems. For a given image-text question, it generates $m$ semantically consistent variants: GPT-4o paraphrases offline, and randomly chosen visual changes with a matching textual hint online. The policy samples $n$ responses per variant, and then the responses are shared: every response is scored under every variant of the same seed question in the policy objective, and its advantage is computed from a global normalization across all m variant responses plus a local normalization within its own variant. The claimed effect is that the reward signal becomes denser, the valid-advantage ratio stays higher during training, and the trained model generalizes to both mathematical and general reasoning benchmarks. The reported numbers are 75.4 on MathVista for the 7B model against 72.8 for GRPO, 67.0 on MMStar, 58.1 on MMMU, and a 32B model that reaches 77.6, 70.2, 70.1, 59.0, 40.3, and 86.2 on the six benchmarks, all without cold-start supervised fine-tuning.

Load-bearing premise

The load-bearing premise is that every semantically consistent transformation, every GPT-4o paraphrase and every rotation or noise injection, leaves the question's meaning and its correct answer unchanged; the paper reports no check of this, and if any variant's answer differs, the shared reward and the policy update in Eq. (8) are computed against a wrong label.

Editorial extensions

If this is right

  • If Share-GRPO is correct, RL for MLLM reasoning does not require a cold-start SFT stage: starting from Qwen2.5-VL-7B, the 75.4 MathVista score is reached by online RL that shares responses and rewards across variants.
  • Sharing makes sampling more efficient: R1-ShareVL-7B with six responses per question beats GRPO with twenty-four responses, so the method converts extra sampling into extra signal rather than extra cost.
  • Generalization to non-mathematical reasoning follows: gains on MMStar and MMMU indicate that shared RL training does not sacrifice multi-discipline accuracy the way some reasoning-only RL runs do.
  • Combining Share-GRPO with dynamic sampling gives a further boost, reaching 75.9 on MathVista and 59.0 on MMMU, so the two mechanisms are complementary rather than redundant.
  • Scaling holds: the 32B model improves on average by +2.7 over its base model and +3.4 over MM-Eureka-32B.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the paper never checks answer consistency of the transformed variants, a testable extension is to filter variants by comparing their predicted answers with the original before reward assignment; if consistency filtering further improves training, it would confirm that semantic preservation, not mere diversity, is the active ingredient.
  • Editorial inference: the same sharing principle should transfer to text-only LLM reasoning, where semantic paraphrases are cheaper and visual augmentations are absent; if it transfers, question-space expansion could replace part of the data scaling that current RL-for-LLM pipelines rely on.
  • Editorial inference: the hierarchical advantage can also be read as a variance-reduction device; comparing Share-GRPO against GRPO with matched total samples and matched compute would separate the benefit of sharing from the benefit of simply seeing more responses.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Share-GRPO, an online reinforcement learning method for multimodal large language models (MLLMs) that aims to mitigate sparse reward and advantage vanishing in GRPO-style training. Each input question is expanded into m semantically consistent variants via offline GPT-4o text rewriting and online image transformations (rotation, noise), and the policy generates n responses per variant. The method then optimizes a shared policy objective (Eq. 8) that evaluates responses from every variant against every variant, using a hierarchical advantage that combines global (across all variants) and local (within one variant) reward normalization. Training with Qwen2.5-VL-7B and 32B on 52K samples from MM-Eureka, without cold-start SFT, reportedly improves MathVista, MMStar, MMMU, MathVerse, MathVision, and AI2D over the base model, GRPO, and several prior RL-based reasoning MLLMs. Ablations attribute the gains to shared policy optimization and shared advantage estimation.

Significance. If the method is sound, the paper would make a useful empirical contribution: the sparse-reward and advantage-vanishing problems are real, the proposed recipe is simple and does not require a reward model, and the evaluation spans six benchmarks. The ablations are incremental and clearly show that each added component contributes to the final MathVista score. The main weaknesses are that the core shared-policy objective is not derived as an unbiased estimator, the semantic-consistency assumption underlying the whole pipeline is not validated, and all headline numbers are single-run results with hyperparameters selected on a test benchmark. These issues currently prevent the paper from establishing Share-GRPO as a principled method; however, they appear addressable with additional derivations, diagnostics, and experiments.

major comments (4)
  1. [§3.2.3, Eq. (8)] The shared policy objective in Eq. (8) is not a valid surrogate for the GRPO objective when k≠j. In those terms the response o_i^{Q_j} was sampled from π_old(·|Q_j), but the ratio is written as π_θ(o_i^{Q_j}|Q_k)/π_old(o_i^{Q_j}|Q_k). An unbiased importance-sampling surrogate for target question Q_k with behavior policy π_old(·|Q_j) would require π_θ(o|Q_k)/π_old(o|Q_j), or equivalently an additional density-ratio correction π_old(o|Q_k)/π_old(o|Q_j); the form in Eq. (8) is biased by the discrepancy between π_old(·|Q_k) and π_old(·|Q_j). This bias is independent of whether the SCT variants are semantically consistent, because different prompts generally induce different response distributions, and it can change during training. The paper provides no derivation, no off-policy correction, and no diagnostic such as the KL divergence between variant-conditional old-policy distributions. Consequently, the central mechanism of Share-GRPO is not established as a principled policy-gradient update. Please re-derive the objective with the correct importance weights, bound the bias, or explicitly reposition Eq. (8) as a heuristic and supply diagnostics showing that the bias is small in practice.
  2. [§3.2.1] The assumption that SCT preserves the correct answer is asserted but never validated. The offline textual rewrites in Eq. (3) are generated by GPT-4o with no consistency check, and the online visual transformations in Eq. (4) (rotation, noise injection) are described only qualitatively, with the accompanying textual transformation τ not specified concretely. If a variant has a different answer than the seed question, the rule-based reward computed in §3.2.2 is wrong, and Eqs. (5)–(8) train the model toward incorrect solutions while the hierarchical advantage can amplify that error. Please report consistency checks (e.g., answer audits on a sample, automatic verification of variant answers on rule-checkable tasks, or human evaluation) and exclude or correct inconsistent variants during training.
  3. [§4.2–§4.4, Tables 1–5] The empirical comparison is based on single runs with no standard errors or significance tests, and the hyperparameters m, n, p plus the design choices in Tables 2, 4, and 5 are tuned on MathVista, which is also one of the headline benchmarks. This makes it hard to separate the method's intrinsic effect from selection effects and run-to-run variance. Please provide multiple seeds or error bars for the main comparisons and ablations, and state clearly which hyperparameters are fixed before evaluation and which are selected on test benchmarks.
  4. [§3.2.2, Eq. (7)] The hierarchical advantage in Eq. (7) combines global and local advantages only for j=k and uses the global advantage alone for j≠k. The paper gives no justification for excluding the local term for cross-variant evaluations, and no sensitivity analysis for the relative weighting of global and local terms. Please clarify the intended estimator and test alternative weightings or inclusion rules, since this design choice directly affects the training signal.
minor comments (4)
  1. [§3.2.2, Eq. (5)] The notation in Eq. (5) uses RQj_i in the numerator but rQj_i inside the mean and std expressions; please unify the reward notation across the equations.
  2. [§3.2.3, Eq. (8)] The summation structure in Eq. (8) is ambiguous: it averages over i=1..n, while responses are indexed as o_i^{Q_j} for each variant j, and the total number of responses is m·n. Please clarify what i indexes and why the normalization is 1/n rather than 1/(m·n).
  3. [Figure 2] Figure 2 contains informal Chinese annotations and very small text that is hard to read; please replace them with clean English labels and enlarge the diagram.
  4. [§4.4, Table 3] The dynamic sampling baseline is introduced only by reference to [58]; please describe briefly how it is applied and whether it uses the same reward functions and rollout settings as Share-GRPO.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central result is an empirical RL comparison, and no equation reduces to its inputs or to a load-bearing self-citation.

full rationale

The paper's strongest claim is empirical: Share-GRPO improves benchmark scores relative to GRPO and prior RL-trained MLLMs (Table 1). The method's components are stated explicitly: SCT variants are generated by GPT-4o rewrites and image transformations (Eqs. 3-4), rewards are rule-based, advantages are computed hierarchically from those rewards (Eqs. 5-7), and the policy objective in Eq. 8 is a proposed surrogate objective rather than a quantity fitted to the reported results. No parameter is fitted from MathVista, MMStar, or MMMU and then renamed a prediction; the ablations in Table 2 attribute gains to ablating the stated components on an external benchmark. The self-citations (R1-VL [4], Mulberry [25]) are contextual literature references and are not used to justify the method's validity or to forbid alternatives. The unvalidated semantic-consistency assumption and the possible importance-ratio mismatch in Eq. 8 are correctness risks, not circularity, because they do not make the benchmark outcomes equivalent to the method's inputs by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

Central claim relies mainly on the unverified consistency of generated question variants; the method's hyperparameters are tuned on the headline benchmark; no new physical entities are introduced.

free parameters (4)
  • m (number of question variants per original question) = 2 (main), 4 studied
    Selected in Section 4.4 Table 4 based on MathVista accuracy; larger m gives higher score but more compute.
  • n (number of sampled responses per variant) = 6 (main), 3+3 / 6+6 studied
    Section 4.4 Table 5 shows Share-GRPO (6+6) best; chosen to beat GRPO with fewer total samples.
  • p (probability of applying online multimodal SCT) = 0.3
    Reported in Section 4.1 with no sensitivity study or rationale.
  • local/global advantage combination weight = 1 (equal sum)
    Eq. 7 adds local and global advantages with equal weight; no ablation of alternative weights.
assumptions (3)
  • domain assumption Semantically consistent transformations preserve the original question's intent and correct answer.
    Stated in Section 3.2.1; no verification that GPT-4o paraphrases or image rotations/noise leave the answer unchanged.
  • ad hoc to paper A correct reasoning path for one question variant is a valid correct path for every other variant of the same seed question.
    Required by Eq. 8's shared policy objective, which uses responses sampled under Qj in the ratio for Qk and assigns them advantages computed for Qj.
  • domain assumption Rule-based accuracy and format rewards correctly identify correct answers for all transformed variants.
    Used throughout Section 3.2.2; if a variant's answer key is wrong after transformation, rewards are noisy.

how reviews work

0 comments
Cite this review

Pith. "Pith review of R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO." pith.science (2026). https://pith.science/paper/JP7CKGEP

@misc{pith2026250516673,
  author       = {Pith},
  title        = {Pith review of: R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JP7CKGEP}},
  note         = {Machine review of arXiv:2505.16673}
}
read the original abstract

In this work, we aim to incentivize the reasoning ability of Multimodal Large Language Models (MLLMs) via reinforcement learning (RL) and develop an effective approach that mitigates the sparse reward and advantage vanishing issues during RL. To this end, we propose Share-GRPO, a novel RL approach that tackle these issues by exploring and sharing diverse reasoning trajectories over expanded question space. Specifically, Share-GRPO first expands the question space for a given question via data transformation techniques, and then encourages MLLM to effectively explore diverse reasoning trajectories over the expanded question space and shares the discovered reasoning trajectories across the expanded questions during RL. In addition, Share-GRPO also shares reward information during advantage computation, which estimates solution advantages hierarchically across and within question variants, allowing more accurate estimation of relative advantages and improving the stability of policy training. Extensive evaluations over six widely-used reasoning benchmarks showcase the superior performance of our method. Code will be available at https://github.com/HJYao00/R1-ShareVL.

Figures

Figures reproduced from arXiv: 2505.16673 by the authors.

Figure 1
Figure 1. (a) Share-GRPO expands the question space via semantically consistent transformations, and then explores diverse reasoning trajectories from different question variants and shares the discovered trajectories among them. (b) Share-GRPO provides denser rewards and higher valid advantage ratios compared to GRPO, demonstrating its effectiveness in mitigating sparse reward and advantage vanishing issues. (c) Share-GRPO o… view at source ↗
Figure 3
Figure 3. Qualitative Results of reasoning capability of R1-ShareVL on mathematical problems [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Perception Before Supervision: Self-Contained Visual Distillation from Counterfactual Blind Spots

    cs.CV 2026-08 conditional novelty 7.0 of 10

    CVPD finds visual blind spots by comparing a model's own crop, full-image, and ghosted responses, then uses the crop as a positive teacher and the ghost as a negative teacher for dense self-distillation.

  2. Multi-Branch Policy Optimization for Multimodal Large Language Models

    cs.CV 2026-08 conditional novelty 6.0 of 10

    MBPO uses branch-level, sibling-relative advantages in tree-structured rollouts to improve credit assignment when training multimodal large language models with reinforcement learning.

  3. SIVA-RL: Sensitivity-Invariance Visual Alignment for Multimodal Reinforcement Learning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    SIVA-RL uses the observed reward drop between clean and locally edited images to route training toward sensitivity or invariance, improving GRPO/DAPO-based multimodal RL across nine benchmarks.

  4. H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Token-level confidence arbitration between VL and text teachers during on-policy distillation improves multimodal student reasoning over single-teacher OPD and GRPO.

  5. Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A structured five-step reasoning template plus diverse-trajectory cold start and diversity-preserving two-stage RL lifts a 7B multimodal model to state-of-the-art multi-image reasoning on several benchmarks.

  6. LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Model

    cs.CV 2025-08 conditional novelty 6.0 of 10

    RL training on preference-labeled critic data transforms a 7B vision-language model into both a stronger critic and a stronger generative policy, improving average benchmark accuracy by 5.7% and enabling self-critique...

  7. VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoning

    cs.CV 2025-07 conditional novelty 6.0 of 10

    VL-Cogito, trained with progressive curriculum RL, online difficulty weighting, and dynamic length rewards, matches or beats prior reasoning MLLMs on ten multimodal benchmarks.

  8. MMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI

    cs.AI 2025-06 conditional novelty 6.0 of 10

    MMReason is an open-ended multimodal reasoning benchmark that filters out guessable and memorized questions and scores model answers both by final answer and by intermediate steps.

  9. Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A GRPO framework that treats thinking as a tool call and uses dual-level regulation so multimodal models learn when to reason versus answer directly.

  10. EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

    cs.AI 2026-02 conditional novelty 5.0 of 10

    EMO-R3, which combines a three-step emotional reasoning prompt with a reward for the model agreeing with its own image–emotion judgments, raises visual emotion-recognition accuracy by about one point over plain GRPO.

  11. How to Train a Leader: Hierarchical Reasoning in Multi-Agent LLMs

    cs.MA 2025-07 conditional novelty 5.0 of 10

    A leader LLM trained with a GRPO variant that conditions on frozen agent responses improves both collaborative and zero-shot accuracy on BBH, MATH, and MMLU.

Reference graph

Works this paper leans on

63 extracted references · 3 canonical work pages · cited by 11 Pith papers

  1. [1]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    arXiv 2025

  2. [2]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  3. [3]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    arXiv 2024

  4. [4]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937, 2025

    arXiv 2025

  5. [5]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    arXiv 2025

  6. [6]

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    arXiv 2025

  7. [7]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  8. [8]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

Show all 63 references
  1. [9]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  2. [10]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processi...

    2024

  3. [11]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vis...

    2024

  4. [12]

    Internvl- x: Advancing and accelerating internvl series with efficient visual token compression

    Dongchen Lu, Yuyao Sun, Zilu Zhang, Leping Huang, Jianliang Zeng, Mao Shu, and Huo Cao. Internvl- x: Advancing and accelerating internvl series with efficient visual token compression. arXiv preprint arXiv:2503.21307, 2025

    2025 arXiv

  5. [13]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  6. [14]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  7. [15]

    Dense connector for mllms

    Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhiheng Li, Wanli Ouyang, and Jingdong Wang. Dense connector for mllms. Advances in Neural Information Processing Systems, 37:33108–33140, 2024

    2024

  8. [16]

    Valley2: Exploring multimodal models with scalable vision-language design

    Ziheng Wu, Zhenghao Chen, Ruipu Luo, Can Zhang, Yuan Gao, Zhentao He, Xian Wang, Haoran Lin, and Minghui Qiu. Valley2: Exploring multimodal models with scalable vision-language design. arXiv preprint arXiv:2501.05901, 2025

    2025 arXiv

  9. [17]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  10. [18]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716...

    2022

  11. [19]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024. 10

    2024

  12. [20]

    Gem: Empowering mllm for grounded ecg understanding with time series and images

    Xiang Lan, Feng Wu, Kai He, Qinghao Zhao, Shenda Hong, and Mengling Feng. Gem: Empowering mllm for grounded ecg understanding with time series and images. arXiv preprint arXiv:2503.06073, 2025

    2025

  13. [21]

    Llavanext: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llavanext: Improved reasoning, ocr, and world knowledge, 2024

    2024

  14. [22]

    The dawn of lmms: Preliminary explorations with gpt-4v (ision)

    Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The dawn of lmms: Preliminary explorations with gpt-4v (ision). arXiv preprint arXiv:2309.17421, 9(1):1, 2023

    2023 arXiv

  15. [23]

    Multimodal chain-of-thought reasoning in language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923, 2023

    2023 arXiv

  16. [24]

    Llava-cot: Let vision language models reason step-by-step, 2024

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2024. URL https://arxiv. org/abs/2411.10440

    2024 arXiv

  17. [25]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319, 2024

    2024 arXiv

  18. [26]

    O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning

    Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570, 2025

    2025 arXiv

  19. [27]

    Rlaif: Scaling reinforcement learning from human feedback with ai feedback

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Ren Lu, Thomas Mesnard, Johan Ferret, Colton Bishop, Ethan Hall, Victor Carbune, and Abhinav Rastogi. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. 2023

    2023

  20. [28]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Co...

    2024

  21. [29]

    Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness. arXiv preprint arXiv:2405.17220, 2024

    2024

  22. [30]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  23. [31]

    R1-v: Reinforcing super generalization ability in vision-language models with less than $3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. https://github.com/Deep-Agent/R1-V , 2025. Accessed: 2025-02-02

    2025

  24. [32]

    Reason-rft: Reinforcement fine-tuning for visual reasoning

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752, 2025

    2025

  25. [33]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  26. [34]

    Sft or rl? an early investigation into training r1-like reasoning large vision-language models

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models. arXiv preprint arXiv:2504.11468, 2025

    2025 arXiv

  27. [35]

    Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503.17352, 2025

    2025 arXiv

  28. [36]

    Skywork r1v: pioneering multimodal reasoning with chain-of-thought

    Yi Peng, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, et al. Skywork r1v: pioneering multimodal reasoning with chain-of-thought. arXiv preprint arXiv:2504.05599, 2025

    2025 arXiv

  29. [37]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning. arXiv preprint arXiv:2503.18013, 2025

    2025 arXiv

  30. [38]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019

    2019

  31. [39]

    Lxmert: Learning cross-modality encoder representations from transformers

    Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490, 2019. 11

    1908 arXiv

  32. [40]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PmLR, 2020

    2020

  33. [41]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020

    2020

  34. [42]

    Distral: Robust multitask reinforcement learning

    Yee Teh, Victor Bapst, Wojciech M Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu. Distral: Robust multitask reinforcement learning. Advances in neural information processing systems, 30, 2017

    2017

  35. [43]

    Multi-task deep reinforcement learning with popart

    Matteo Hessel, Hubert Soyer, Lasse Espeholt, Wojciech Czarnecki, Simon Schmitt, and Hado Van Hasselt. Multi-task deep reinforcement learning with popart. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 3796–3803, 2019

    2019

  36. [44]

    Learning to communicate with deep multi-agent reinforcement learning

    Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement learning. Advances in neural information processing systems, 29, 2016

    2016

  37. [45]

    Multi-agent actor-critic for mixed cooperative-competitive environments

    Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems, 30, 2017

    2017

  38. [46]

    Sharing knowledge in multi-task deep reinforcement learning

    Carlo D’Eramo, Davide Tateo, Andrea Bonarini, Marcello Restelli, and Jan Peters. Sharing knowledge in multi-task deep reinforcement learning. arXiv preprint arXiv:2401.09561, 2024

    2024 arXiv

  39. [47]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  40. [48]

    Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning

    Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, Xuchen Song, et al. Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning. arXiv preprint arXiv:2504.16656, 2025

    2025 arXiv

  41. [50]

    Easyr1: An efficient, scalable, multi-modality rl training framework

    Zheng Yaowei, Lu Junting, Wang Shenzhi, Feng Zhangchi, Kuang Dongdong, and Xiong Yuwen. Easyr1: An efficient, scalable, multi-modality rl training framework. https://github.com/hiyouga/EasyR1, 2025

    2025

  42. [51]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia...

    2024

  43. [52]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  44. [53]

    Claude 3.5 sonnet, 2024

    Anthropic. Claude 3.5 sonnet, 2024

    2024

  45. [54]

    Improve vision language model chain-of-thought reasoning

    Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprint arXiv:2410.16198, 2024

    2024 arXiv

  46. [55]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  47. [56]

    X-reasoner: Towards generalizable reasoning across modalities and domains

    Qianchu Liu, Sheng Zhang, Guanghui Qin, Timothy Ossowski, Yu Gu, Ying Jin, Sid Kiblawi, Sam Preston, Mu Wei, Paul V ozila, et al. X-reasoner: Towards generalizable reasoning across modalities and domains. arXiv preprint arXiv:2505.03981, 2025

    2025 arXiv

  48. [57]

    Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934, 2025

    2025 arXiv

  49. [58]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025

    2025 arXiv

  50. [59]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023

    arXiv

  51. [60]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024. 12

    2024 arXiv

  52. [61]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. arXiv preprint arXiv:2311.16502, 2023

    2023 arXiv

  53. [62]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186. Sprin...

    2024

  54. [63]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hong- sheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  55. [64]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pages 235–251. S...

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.