Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Skywork-R1V3 Technical Report

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a reinforcement-learning post-training recipe can transfer text-only reasoning to a vision-language model, giving a 38B open model 76.0% on MMMU, at entry-level human expert level.

desk verdict Genuinely useful VLM RL post-training report with a novel checkpoint-selection metric, but the headline MMMU number rests on a modified, unreleased eval protocol and should be treated as provisional. read the letter →

arxiv 2507.06167 v3 pith:HMY3XSZX submitted 2025-07-08 cs.CL cs.CV

Wei Shen , Jiangbo Pei , Yi Peng , Xuchen Song , Yang Liu , Jian Peng , Haofeng Sun , Yunzhuo Hao
show 3 more authors
Peiyu Wang Jianhao Zhang Yahui Zhou
This is my paper
classification cs.CLcs.CV
keywords vision-languagemodelmultimodalreasoningreinforcementlearningGRPOchain-of-thoughtcross-modalalignmentMMMUbenchmarktransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a purely post-training reinforcement-learning (RL) pipeline can give a vision-language model (VLM) the kind of step-by-step reasoning that text-only large language models have, without any continued pre-training on visual data. The authors report that their 38B model, Skywork-R1V3, reaches 76.0% on the MMMU benchmark, a score they describe as matching entry-level human experts and surpassing much larger open and closed models. The core insight is that RL does not just teach a reasoning style; it activates a generalizable reasoning ability, while supervised fine-tuning on the same data only imitates the style. The paper also identifies the vision-language connector as the essential component that must stay trainable during RL, and introduces a cheap diagnostic, the entropy of critical reasoning tokens, for choosing checkpoints that have genuinely learned to reason.

What carries the argument

The connector module — the learned projection between the visual encoder and the language model — is the central object that the paper argues must remain trainable for cross-modal alignment during RL. The second machinery is a monitoring metric: the entropy of the model's output distribution at critical reasoning-initiation tokens such as 'Wait' or 'Alternatively'; the authors find high entropy at these positions correlates with genuine reasoning, while low entropy marks style-mimicking checkpoints. The RL objective itself is GRPO with group-normalized advantages plus a format reward, trained on a small math dataset.

What would settle it

Run Skywork-R1V3-38B on the official, unmodified MMMU evaluation with the standard answer-checking rule and a single generation per item, and compare the score to published baselines; if it drops materially below 76.0% or below the comparison models, the headline claim fails. A second check: freeze the connector during RL and see whether training collapses as the paper predicts it will.

Watch

Extended reading notes

Core claim

The central discovery is that three small post-training stages transfer and amplify reasoning in a VLM: a cold-start SFT on about 12K distilled reasoning traces from a reasoning model, GRPO reinforcement learning on 15K multimodal math questions, and a final connector-only tuning on 10K multi-domain samples. The authors report that this lifts MMMU from 64.3% to 76.0% and generalizes from math to physics, logic, and other disciplines. They argue that the connector module, not the visual encoder, is the load-bearing bridge: freezing or removing it during RL leads to reward collapse, repetitive outputs, and loss of reasoning, while freezing the visual encoder still allows training to proceed. They also report that RL-trained models generalize out-of-domain while SFT-trained models do not, and that a critical-token entropy metric predicts which checkpoints truly reason.

Load-bearing premise

The central performance claim rests on the assumption that the paper's modified MMMU scoring — an added rule-based answer check and averaging over five random generation runs — yields numbers comparable to the official benchmark; if that adjustment inflates their score, the headline result is not established.

Editorial extensions

If this is right

  • A text-only LLM's reasoning skills can be transferred to a vision-language model with a small cold-start dataset and RL, so large-scale multimodal pretraining may not be required for strong visual reasoning.
  • An open 38B model can rival or beat closed 72B+ and proprietary models on MMMU, MathVista, and related benchmarks, narrowing the open-closed gap in multimodal reasoning.
  • Connector-only tuning after RL can rebalance domain knowledge without eroding reasoning, offering a low-cost way to broaden a reasoning model's expertise.
  • Critical-token entropy gives a training-time signal for when RL has produced genuine reasoning, potentially replacing expensive validation-based checkpoint selection.
  • Increasing the thinking token budget improves MMMU accuracy steadily up to a saturation point, so deployment can trade latency against accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The critical-token entropy metric is a transferable idea: the same signal could be used to detect the emergence of reasoning in text-only LLM RL runs, or to decide when to stop training.
  • The paper's ablations imply that the connector is a bottleneck for aligning vision with language throughout training, not just at initialization; if so, connector-focused interventions (architecture, initialization, or dedicated data) could improve other VLMs more cheaply than scaling the backbone.
  • The MMMU evaluation used a modified protocol (rule-based check, five-run average), so the 76.0% is not guaranteed to be leaderboard-comparable; a rerun on the official protocol would settle whether the ranking claims transfer.
  • If connector-only tuning preserves RL-learned reasoning while injecting new knowledge, it offers a modular route toward unified understanding-and-generation VLMs, where a frozen reasoning core is paired with a retrained connector.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents Skywork-R1V3-38B, an open-source vision-language model whose post-training pipeline consists of cold-start supervised fine-tuning on reasoning traces generated by Skywork-R1V2, GRPO reinforcement learning on 15K multimodal math problems, and a connector-only tuning stage on multi-domain data. The authors report 76.0% on MMMU, claiming state-of-the-art performance among open-source models and parity with entry-level human experts, and introduce an entropy-of-critical-tokens metric for checkpoint selection during RL. Additional sections analyze curriculum learning, learning-rate stability, clip-higher and dynamic sampling, slow versus fast thinking, thinking budgets, and chain-of-thought hallucination.

Significance. If the reported results are reproducible, the paper would make a meaningful contribution: it would show that an RL-based post-training recipe can transfer text-domain reasoning to a VLM without continued pre-training, that the vision-language connector is a load-bearing component during RL, and that a 38B model can approach or match much larger proprietary models on visual reasoning benchmarks. The report is unusually detailed in its training recipes and includes useful ablations (module activation, connector-only tuning, curriculum and learning-rate studies). The release of model weights and the stated intention to release the evaluation framework are concrete reproducibility assets. However, the central performance claim rests on a modified evaluation protocol whose comparability to published leaderboards is not yet established, and several secondary claims (entropy correlation, connector necessity) are supported only qualitatively or on single benchmarks. These factors make the current version informative but not yet a citable reference for the headline numbers.

major comments (4)
  1. [§4 and Table 1] The headline MMMU result of 76.0% is obtained under a modified protocol: VLMEvalKit was used 'with targeted modifications (i.e. add rule-based check for MMMU)', decoding temperature was 1.0, and results are averaged over five runs with no variance reported. Because comparison scores in Table 1 may or may not use the same protocol, and because the asterisk convention does not mark the MMMU row even though Section 4 states that MMMU was modified, the 76.0% versus 70.3% (QVQ-72B), 72.2% (InternVL3-78B), and 75.0% (Claude 3.7 Sonnet) margins are not yet comparable to official leaderboard numbers. Please re-run the final model on the official MMMU harness, report per-run scores with mean and standard deviation, and state explicitly for every row of Table 1 which evaluation protocol was used. The paper's own statement that the evaluation framework will be released acknowledges that this artifact is currently missing.
  2. [§5.1 and Figure 5] The claim that entropy of critical reasoning tokens 'shows a strong correlation' with validation reasoning performance is supported only by a qualitative figure with no correlation coefficient, no definition of the prompt or token set used to detect critical tokens, and no threshold other than the entropy value 6.8 mentioned later in §6.5. Because the same metric is used for checkpoint selection, a quantitative demonstration on held-out checkpoints is needed to rule out selection bias. Please report the correlation statistic, the exact token set and detection rule, and the validation protocol.
  3. [§3.2 and §1] The central claim of transferring reasoning skills from text-only LLMs to visual tasks is complicated by the provenance of the cold-start data: Skywork-R1V2 is itself a VLM created by stitching InternViT-6B-448px-V2.5 with QwQ-32B, and its traces were filtered by the authors' Skywork-VL-Reward model. The paper should clarify what specifically is transferred from the text-only QwQ-32B, whether R1V2 underwent any multimodal training before generating traces, and whether the 12K selected instances overlap with evaluation benchmarks. A contamination check on MMMU, GAOKAO, and the math benchmarks would strengthen the generalization claims.
  4. [§5.2, §5.6, and Table 2] The assertion that the connector is an 'absolute prerequisite' for stable RL learning and that connector-only tuning 're-balances' knowledge is based on a small set of ablations, with each condition reported as a single accuracy number and no confidence intervals. Given that Table 2 reports 74.5, 76.0, 73.2, and 69.7 on MMMU, differences of 1.5 points may be within run-to-run noise under the T=1.0 protocol. Please provide multiple seeds or per-run breakdowns for the key ablations, and temper the causal language accordingly.
minor comments (6)
  1. [§3.1, Eq. (2)] The notation 'Rθ(â, a) likely represents an accuracy score' is informal; please give the exact definition of the verifier model, the rule-based checker, and how their outputs are combined.
  2. [Table 1 note] The asterisk note 'results from our evaluation framework' is ambiguous because some rows without asterisks (e.g., MMMU) are also described in Section 4 as using the modified framework; please make the table self-explanatory.
  3. [§6.4 and Table 5] The hallucination analysis is based on 47 samples with 'I can't see the image' versus 853 without; the 51.1% versus 77.3% difference would benefit from a significance test and from clarification of how the subset was identified.
  4. [§5.3 and §5.4] The curriculum and learning-rate analyses report training curves but no quantitative validation numbers for the comparison; please add the final evaluation scores for each condition.
  5. [§6.1 and Table 3] The SFT baseline is trained on reasoning-style data distilled from the final Skywork-R1V3-38B, which makes the comparison to RL less clean; please state that this is a deliberate choice and discuss the possible effect of distillation quality.
  6. [Throughout] There are several typographical errors ('Skyword-R1V3' in §5.2, 'costumed GRPO' in §3.3, 'assistant' spacing in §6.2 templates) and the conclusion's claim of being the 'first predominantly RL-enhanced open-source VLM' should be softened in light of prior work such as MM-Eureka and other RL-trained open VLMs.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: headline benchmark scores are external measurements; self-generated training data and the entropy heuristic are not predictions derived from those scores.

full rationale

I find no step that reduces a claimed result to its own inputs. The headline 76.0% MMMU and all benchmark comparisons are external measurements; no equation in the paper defines those scores in terms of a fitted parameter or a self-citation. The cold-start dataset generated by the authors' own Skywork-R1V2 and filtered by Skywork-VL-Reward is a training-data construction step, not a prediction: the downstream model is independently evaluated on public benchmarks, so the self-use does not force the outcome. The critical-token entropy is presented as an empirical checkpoint-selection heuristic with a threshold observed during training (6.8); it is correlated with, not used to compute, the reported MMMU accuracy. Section 4's modified VLMEvalKit protocol (rule-based check for MMMU, temperature 1.0, five-run averaging) is a comparability and reproducibility concern that the authors acknowledge by planning to release the framework, but it is not circular because the measured score is not algebraically implied by the evaluation modification. Thus the central claims rest on external benchmark measurements rather than on self-referential derivation.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The model builds on InternVL-38B and R1V2-distilled cold-start data; the main free parameters are the reward weight epsilon, the entropy threshold, and the hand-picked critical token set. The axioms are domain assumptions about the base model, the quality of self-generated training data, the stability of no-KL GRPO, and the validity of the chosen benchmarks.

free parameters (3)
  • Reward weight epsilon = 0.8
    In Eq. 3 the final reward is a weighted sum of accuracy and format rewards; the paper states 'Empirically, we set epsilon to 0.8' without a tuning curve.
  • High-entropy threshold = 6.8
    Section 6.5 defines high-entropy tokens as those with entropy greater than 6.8; this threshold is chosen from the authors' observations, not derived from theory.
  • Critical token set = Wait, Alternatively, Hmm
    Section 5.1 defines the critical reasoning initiation points by examples like 'Wait...' and 'Alternatively...'; the exact set of tokens is chosen by hand and not exhaustively specified.
assumptions (4)
  • domain assumption InternVL-38B base model already has adequate pretrained visual and language knowledge, so post-training alone can unlock reasoning.
    Section 3.2 builds Skywork-R1V3 on InternVL-38B and claims no continue pre-training is needed.
  • domain assumption R1V2's reasoning traces, filtered by Skywork-VL-Reward and GPT-4o, are valid and sufficient for cold-start transfer.
    Section 3.2 uses R1V2 to generate about 12k cold-start samples; the quality filter relies on the authors' own reward model.
  • domain assumption GRPO with no KL penalty and increasing context length is stable for this VLM.
    Section 3.3 applies GRPO with no KL during long CoT training and reports it works, but provides no convergence or stability argument.
  • domain assumption MMMU and the other selected benchmarks measure generalizable visual reasoning.
    Section 4 evaluates on MMMU, MathVista, and related benchmarks, treating them as representative of reasoning ability.
invented entities (1)
  • Critical reasoning token entropy, measured at tokens such as 'Wait' and 'Alternatively'
    purpose: Checkpoint selection during RL to distinguish genuine reasoning from style mimicry
    Section 5.1 introduces this metric and shows a correlation with MMMU on the authors' own runs; it has not been validated externally or reproduced by other groups.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Skywork-R1V3 Technical Report." pith.science (2026). https://pith.science/paper/HMY3XSZX

@misc{pith2026250706167,
  author       = {Pith},
  title        = {Pith review of: Skywork-R1V3 Technical Report},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HMY3XSZX}},
  note         = {Machine review of arXiv:2507.06167}
}
read the original abstract

We introduce Skywork-R1V3, an advanced, open-source vision-language model (VLM) that pioneers a new approach to visual reasoning. Its key innovation lies in effectively transferring reasoning skills from text-only Large Language Models (LLMs) to visual tasks. The strong performance of Skywork-R1V3 primarily stems from our elaborate post-training RL framework, which effectively activates and enhances the model's reasoning ability, without the need for additional continue pre-training. Through this framework, we further uncover the fundamental role of the connector module in achieving robust cross-modal alignment for multimodal reasoning models. In addition, we introduce a unique indicator of reasoning capability, the entropy of critical reasoning tokens, which has proven highly effective for checkpoint selection during RL training. Skywork-R1V3 achieves state-of-the-art results on MMMU, significantly improving from 64.3% to 76.0%. This performance matches entry-level human capabilities. Remarkably, our RL-powered post-training approach enables even the 38B parameter model to rival top closed-source VLMs. The implementation successfully transfers mathematical reasoning to other subject-related reasoning tasks. We also include an analysis of curriculum learning and reinforcement finetuning strategies, along with a broader discussion on multimodal reasoning. Skywork-R1V3 represents a significant leap in multimodal reasoning, showcasing RL as a powerful engine for advancing open-source VLM capabilities.

Figures

Figures reproduced from arXiv: 2507.06167 by the authors.

Figure 1
Figure 1. Benchmark performance of Skywork-R1V3. arXiv:2507.06167v3 [cs.CL] 10 Jul 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Data distribution across the three training stages. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The Performance of Skywork-R1V3-38B on PhyX-MC-Text-Minimal Evaluation Settings For evaluation, we use a generation token budget of 16,384 and a decoding temperature of 1.0. Results are averaged over five runs. We adopt VLMEvalKit (Duan et al., 2024) as the primary evaluation framework, with tar￾geted modifications (i.e. add rule-based check for MMMU (Yue et al., 2024)) to better support long chain-of-thought reason… view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Model Rankings on 2025 GAOKAO Math results indicate that our Skywork-R1V3-38B can effectively handle complex physics problems, outper￾forming much larger open-source models and even approaching the advanced proprietary model’s performance. 5 Empirical Analysis on Reinf…
Figure 5
Figure 5. Figure 5: The entropy of critical token vs. MMMU accuracy [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Ablation Studies of Module Activation Impact on MathVista Performance [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Comparison of Curriculum Learning Aspects. This figure analyzes key aspects of curriculum learning. (a) illustrates the significant drop in validation accuracy during Stage 2. (b) shows that a higher learning rate can lead to faster learning efficiency but also instabi…
Figure 8
Figure 8. Figure 8: Impact of Learning Rates on RL Performance in Stage 2 (Hard Problems). This figure shows key metrics for different learning rate strategies on Hard Problems (a) A higher learning rate indicates the policy’s instability on exploration when tackling Hard Problems. (b) Th…
Figure 9
Figure 9. Figure 9: Model performance with different thinking budgets. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Entropy distribution at different GRPO steps [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: Response length during training: A comparison of two base models [PITH_FULL_IMAGE:figures/full_fig_p014_11.png]
Figure 12
Figure 12. Figure 12: example 1 Prompt 请解释这个 MEME 的含义 Model Response <think> 好的，我现在需要分析用户提供的这个 meme。首先看一下图片和文字内容。图片上有 两个人物：前面是一位女士坐在床边，低头看着手机，表情有些困惑或不悦。后面是一个男人站 在洗手间门口，似乎正在换衣服或者整理衣物。文字部分写的是：“当她在你的个人笔记本电脑 里看到一个名为“models”的文件夹，里面包含一些以 .pkl、.pt 或 .pth 结尾的文件。” 首先，我需要理解各个元素的含义。文件夹名 “models” 在编程中…
Figure 13
Figure 13. Figure 13: example 2 Prompt 我买一千瓶这个饮料，中奖的钱的期望是多少？ Model Response <think>...</think> 根据瓶身标签信息，需要计算各奖项的中奖金额期望值。具 体如下： 各奖项及其中奖概率： • 666 元红包：0.00008%（即 0.0000008） • 66 元红包：0.005%（即 0.00005） • 2 元红包：0.5%（即 0.005） • 1 元红包：1.0%（即 0.01） • 0.5 元红包：16.5%（即 0.165） 各奖项期望值： • 666 元：0.0005328…
Figure 14
Figure 14. Figure 14: example 3 Prompt The circuit in <image 1> consists of a general inverter con￾nected to a load. Suppose the range of logical 1 is defined to be 4 to 5 V. Determine the minimum RL in order that the output be 1 when the input voltage v1 is zero. Model Response <think>...…
Figure 15
Figure 15. Figure 15: example 4 Prompt You are participating in a geo-location challenge. Based on the provided image: 1. Carefully analyze the clues in the image (including architectural style, signage, vegetation, terrain, etc.); 2. Reason step by step to determine which continent, count…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Kalman Meets Curriculum: Efficient Dynamic Prompt Selection for Adaptive RL Finetuning

    cs.LG 2026-07 conditional novelty 6.0 of 10

    A Kalman filter with policy-update-coupled process noise tracks non-stationary prompt difficulty and selects intermediate-difficulty batches for RL finetuning without extra rollouts.

  2. MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    A new benchmark of 2,000 K-12 math questions photographed in real scenes finds that multimodal LLMs perform significantly worse on authentic handheld images than on clean benchmarks.

Reference graph

Works this paper leans on

54 extracted references · 7 canonical work pages · cited by 2 Pith papers

  1. [1]

    Claude-3.7, 2024

    Anthropic. Claude-3.7, 2024. URL https://www.anthropic.com/news/claude-3-7-sonnet

    work page 2024

  2. [2]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    arXiv 2024

  3. [3]

    The entropy mechanism of reinforcement learning for reasoning language models, 2025

    Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, Zhiyuan Liu, Hao Peng, Lei Bai, Wanli Ouyang, Yu Cheng, Bowen Zhou, and Ning Ding. The entropy mechanism of reinforcement learning for reasoning language models, 2025. URL https://arxiv.org/abs/2505.22617

    arXiv 2025

  4. [4]

    Gemini 2.5: Our most intelligent ai modeld

    GOOGLE DEEPMIND. Gemini 2.5: Our most intelligent ai modeld. 2025. URL https://blog.google/technology/google-deepmind/gemini-model-thinking-updates-march-2025/#gemini-2-5-thinking

    work page 2025

  5. [5]

    Deepseek-v3 technical report, 2024

    DeepSeek-AI. Deepseek-v3 technical report, 2024. URL https://arxiv.org/abs/2412.19437

    arXiv 2024

  6. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    arXiv 2025

  7. [7]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia, pp.\ 11198--11201, 2024

    work page 2024

  8. [8]

    Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models, 2024

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models, 2024. URL https://arxiv.org/abs/2310.14566

    arXiv 2024

Show all 54 references
  1. [9]

    Seed1.5-vl technical report, 2025

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, et al. Seed1.5-vl technical report, 2025. URL https://arxiv.org/abs/2505.07062

    2025 arXiv

  2. [10]

    Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark

    Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can mllms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501.05444, 2025

    2025 arXiv

  3. [11]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model, 2025. URL https://arxiv.org/abs/2503.24290

    2025 arXiv

  4. [12]

    The platonic representation hypothesis, 2024

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis, 2024. URL https://arxiv.org/abs/2405.07987

    2024 arXiv

  5. [13]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  6. [14]

    Csvqa: A chinese multimodal benchmark for evaluating stem reasoning capabilities of vlms, 2025

    Ai Jian, Weijie Qiu, Xiaokun Wang, Peiyu Wang, Yunzhuo Hao, Jiangbo Pei, Yichen Wei, Yi Peng, and Xuchen Song. Csvqa: A chinese multimodal benchmark for evaluating stem reasoning capabilities of vlms, 2025. URL https://arxiv.org/abs/2505.24120

    2025 arXiv

  7. [15]

    Visual instruction tuning, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023. URL https://arxiv.org/abs/2304.08485

    2023 arXiv

  8. [16]

    Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp.\ 216--233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pp.\ 216--233. Springer, 2024

    2024

  9. [17]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023

    arXiv

  10. [18]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  11. [19]

    Reinforcement learning with verifiable rewards: Grpo's effective loss, dynamics, and success amplification, 2025

    Youssef Mroueh. Reinforcement learning with verifiable rewards: Grpo's effective loss, dynamics, and success amplification, 2025. URL https://arxiv.org/abs/2503.06639

    2025

  12. [20]

    Gui agents: A survey, 2024

    Dang Nguyen, Jian Chen, Yu Wang, et al. Gui agents: A survey, 2024. URL https://arxiv.org/abs/2412.13501

    2024

  13. [21]

    Gpt-4o system card, 2024

    OpenAI. Gpt-4o system card, 2024. URL https://openai.com/index/hello-gpt-4o/

    2024

  14. [22]

    Skywork r1v: Pioneering multimodal reasoning with chain-of-thought, 2025

    Yi Peng, Peiyu Wang, Xiaokun Wang, Yichen Wei, Jiangbo Pei, Weijie Qiu, Ai Jian, Yunzhuo Hao, Jiachun Pan, Tianyidan Xie, Li Ge, Rongxian Zhuang, Xuchen Song, Yang Liu, and Yahui Zhou. Skywork r1v: Pioneering multimodal reasoning with chain-of-thought, 2025. URL https://arxiv....

    2025 arXiv

  15. [23]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning?, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achie...

    2024 arXiv

  16. [24]

    Proximal policy optimization algorithms, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347

    2017 arXiv

  17. [25]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402.03300

    2024 arXiv

  18. [26]

    Phyx: Does your model have the "wits" for physical reasoning?, 2025

    Hui Shen, Taiqiang Wu, Qi Han, Yunta Hsieh, Jizhou Wang, Yuyue Zhang, Yuxin Cheng, Zijian Hao, Yuansheng Ni, Xin Wang, Zhongwei Wan, Kai Zhang, Wendong Xu, Jing Xiong, Ping Luo, Wenhu Chen, Chaofan Tao, Zhuoqing Mao, and Ngai Wong. Phyx: Does your model have the "wits" for phy...

    2025 arXiv

  19. [27]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  20. [28]

    Mimo-vl technical report, 2025

    Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, et al. Mimo-vl technical report, 2025. URL https://arxiv.org/abs/2506.03569

    2025 arXiv

  21. [29]

    Qvq: To see the world with wisdom

    Qwen Team. Qvq: To see the world with wisdom. https://qwenlm.github.io/blog/qvq-72b-preview/, 2024

    2024

  22. [30]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  23. [31]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language...

    2023 arXiv

  24. [32]

    Vlm see, robot do: Human demo video to robot action plan via vision language model, 2024 a

    Beichen Wang, Juexiao Zhang, Shuwen Dong, Irving Fang, and Chen Feng. Vlm see, robot do: Human demo video to robot action plan via vision language model, 2024 a . URL https://arxiv.org/abs/2410.08792

    2024

  25. [33]

    Wait, we don't need to "wait"! removing thinking tokens improves reasoning efficiency, 2025 a

    Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou. Wait, we don't need to "wait"! removing thinking tokens improves reasoning efficiency, 2025 a . URL https://arxiv.org/abs/2506.08343

    2025 arXiv

  26. [34]

    Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning, 2025 b

    Peiyu Wang, Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, Xuchen Song, Yang Liu, and Yahui Zhou. Skywork r1v2: Multimodal hybrid reinforcement learning for reasoning, 2025 b . URL https://arxiv.org/abs/2504.16656

    2025 arXiv

  27. [35]

    Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model's pe...

    2024 arXiv

  28. [36]

    Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning, 2025 c

    Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effe...

    2025 arXiv

  29. [37]

    Skywork-vl reward: An effective reward model for multimodal understanding and reasoning, 2025 d

    Xiaokun Wang, Peiyu Wang, Jiangbo Pei, Wei Shen, Yi Peng, Yunzhuo Hao, Weijie Qiu, Ai Jian, Tianyidan Xie, Xuchen Song, Yang Liu, and Yahui Zhou. Skywork-vl reward: An effective reward model for multimodal understanding and reasoning, 2025 d . URL https://arxiv.org/abs/2505.07263

    2025 arXiv

  30. [38]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  31. [39]

    Seephys: Does seeing help thinking? -- benchmarking vision-based physics reasoning, 2025

    Kun Xiang, Heng Li, Terry Jingchen Zhang, Yinya Huang, Zirong Liu, Peixin Qu, Jixi He, Jiaqi Chen, Yu-Jie Yuan, Jianhua Han, Hang Xu, Hanhui Li, Mrinmaya Sachan, and Xiaodan Liang. Seephys: Does seeing help thinking? -- benchmarking vision-based physics reasoning, 2025. URL ht...

    2025

  32. [40]

    Logicvista: Multimodal llm logical reasoning benchmark in visual contexts, 2024

    Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts, 2024. URL https://arxiv.org/abs/2407.04973

    2024 arXiv

  33. [41]

    Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models, 2025

    Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, Wenhai Wang, Jifeng Dai, and Jinguo Zhu. Visulogic: A benchmark for evaluating visual reasoning in multi-modal large language models, 2025. URL https://arxi...

    2025 arXiv

  34. [42]

    Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M

    Yi Xu, Yuxin Hu, Zaiwei Zhang, Gregory P. Meyer, Siva Karthik Mustikovela, Siddhartha Srinivasa, Eric M. Wolff, and Xin Huang. Vlm-ad: End-to-end autonomous driving through vision-language model supervision, 2024. URL https://arxiv.org/abs/2412.14446

    2024 arXiv

  35. [43]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen...

    2025 arXiv

  36. [44]

    Mme-reasoning: A comprehensive benchmark for logical reasoning in mllms, 2025

    Jiakang Yuan, Tianshuo Peng, Yilei Jiang, Yiting Lu, Renrui Zhang, Kaituo Feng, Chaoyou Fu, Tao Chen, Lei Bai, Bo Zhang, and Xiangyu Yue. Mme-reasoning: A comprehensive benchmark for logical reasoning in mllms, 2025. URL https://arxiv.org/abs/2505.21327

    2025 arXiv

  37. [45]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on C...

    2024

  38. [46]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark, 2025

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark, 2025. URL https://arxiv.org/abs/2409.02813

    2025 arXiv

  39. [47]

    Are unified vision-language models necessary: Generalization across understanding and generation, 2025

    Jihai Zhang, Tianle Li, Linjie Li, Zhengyuan Yang, and Yu Cheng. Are unified vision-language models necessary: Generalization across understanding and generation, 2025. URL https://arxiv.org/abs/2505.23043

    2025 arXiv

  40. [48]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?, 2024

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and Hongsheng Li. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?, 2024. URL https://arxiv.org/abs/2403.14624

    2024 arXiv

  41. [49]

    Fireedit: Fine-grained instruction-based image editing via region-aware vision language model, 2025

    Jun Zhou, Jiahao Li, Zunnan Xu, Hanhui Li, Yiji Cheng, Fa-Ting Hong, Qin Lin, Qinglin Lu, and Xiaodan Liang. Fireedit: Fine-grained instruction-based image editing via region-aware vision language model, 2025. URL https://arxiv.org/abs/2503.19839

    2025 arXiv

  42. [50]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li,...

    2025 arXiv

  43. [51]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  44. [52]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  45. [53]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  46. [54]

    当她在你的个人笔记本电脑里看到一个名为“models

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.