Pith. sign in

REVIEW 5 major objections 5 minor 41 references

Multi-Branch Policy Optimization for Multimodal Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that multimodal reinforcement learning should assign credit at vision-language decision points, where a model commits to one visual interpretation, rather than across entire response trajectories.

desk verdict A plausible and compute-controlled method for tree-structured RL in multimodal LLMs, but the visual-hypothesis mechanism is asserted rather than measured and the nearest tree-based baselines are missing. read the letter →

arxiv 2608.07581 v1 pith:GKF5GQG7 submitted 2026-08-05 cs.CV cs.AI

Shuai Lyu , Yuning Gong , Ruiling Gao , Xiaoran Shang , Zhonghong Ou , Ping Zong , Yifan Zhu , Yuan Sun
show 2 more authors
Yang Qin Peng Hu
This is my paper · ORCID
classification cs.CVcs.AI
keywords Multi-BranchPolicyOptimizationcreditassignmentmultimodallargelanguagemodelsreinforcementlearningsibling-relativeadvantagereasoningtreetemporalreplaybuffervisual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that reinforcement learning for multimodal large language models fails when it credits every token in a response with the same advantage, because visual reasoning contains decision points where the model commits to one interpretation of an image over another. It proposes Multi-Branch Policy Optimization (MBPO), which grows a tree of rollouts at these vision-language boundaries, so that sibling branches share the same prefix and differ in the visual hypothesis they pursue. A sibling-relative advantage, formed by normalizing each branch's propagated value against its siblings under the same parent, then assigns segment-level credit to the branch rather than the whole trajectory. The authors report that this keeps the fraction of non-zero advantages high throughout training, where GRPO's decays toward zero, and yields consistent gains over GRPO, DAPO, GSPO, Shuffle-R1, and MM-Eureka on geometry and multimodal reasoning benchmarks.

What carries the argument

The load-bearing object is the reasoning tree with sibling-relative branch advantage. A node's value is the mean reward of its children for internal nodes; Eq. (10) defines $A_{\text{sib}}(n)$ as the normalized difference between a node's value and the mean of its siblings, divided by the sibling standard deviation, which isolates how much better one continuation is than alternatives generated from the same prefix. Branching is triggered by a vision-language boundary marker `<look>` plus fixed-length segmentation, and the temporal replay buffer (Eq. 12) restricts reuse to recent segments and samples balanced by question. The hybrid `<look>`-plus-fixed segmentation is what aligns branches with visual interpretation changes rather than arbitrary token positions.

What would settle it

Measure the visual diversity of sibling branches: for a sample of questions, take a parent node's $K$ sampled continuations and compare their image-attention maps or the image regions they refer to. If sibling continuations mostly share the same groundings and differ only in wording or arithmetic, or if the reward spread across siblings disappears when the image is replaced by a blank placeholder, then the sibling-relative advantage is not isolating visual interpretation and the claimed mechanism would fail.

Watch

Extended reading notes

Core claim

The central claim is that representing each candidate answer as a multimodal reasoning tree, with branching at points where the model re-examines the image, converts credit assignment from a trajectory-level average into a comparison among sibling hypotheses. The tree is built by breadth-first parallel expansion with branch factor $K$ per depth; after leaves are scored, rewards propagate bottom-up as means over children (Eq. 6), and each node's advantage is its value minus the mean of its siblings, divided by their standard deviation (Eq. 10). This sibling-relative advantage is then used in a PPO update with a temporal replay buffer that keeps only segments younger than $T_{\max}$ and balances questions in each mini-batch. The paper's evidence is that MBPO keeps the valid advantage ratio near 1.0 while GRPO declines, produces a denser non-zero advantage distribution, and on MMRL18K MBPO-Qwen-VL-7B reaches 52.6 on MathVerse and 30.6 on MathVision, beating MM-Eureka-Qwen-7B by 1.0 and 2.5 points.

Load-bearing premise

The method assumes sibling branches generated from the same prefix are genuinely distinct visual hypotheses, so that the normalized value difference between a branch and its siblings measures the causal effect of choosing one visual interpretation; if siblings differ mostly for non-visual reasons or are near-duplicates, the branch advantages inherit that noise.

Editorial extensions

If this is right

  • If sibling-relative advantages work as claimed, then RL for multimodal LLMs should be designed around decision boundaries rather than whole responses, making segment-level credit the default learning signal.
  • The relative advantage degeneration that GRPO suffers is avoidable: MBPO maintains a near-1.0 valid advantage ratio and a denser non-zero advantage distribution, so training stays informative longer.
  • Compute-matched comparisons on Geo3K show gains of +2.0 to +8.7 accuracy points at the same cumulative TFLOPs, suggesting the gains come from credit quality rather than a larger rollout budget.
  • The observed self-correction behavior rises with MBPO training (self-correction rate from 13.3% to 60.9%, successful corrections from 4.7% to 29.8%), implying branch-level credit encourages the model to revisit images and revise its reasoning.
  • Larger tree branch factors (8-8-8) increase replay-buffer question diversity and late-training accuracy, while 6-6-6 offers a better efficiency-accuracy trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to measure, on a held-out set, whether sibling branches generated under the same prefix actually differ in visual interpretation (for example, in image-attention maps) rather than only in wording or arithmetic; if visual diversity is low, branch-relative advantages may reduce to noisy token-level comparisons.
  • The mechanism suggests an early-termination rule: if the local parent-child value difference in Eq. (7) is below a threshold, stop expanding that branch. The paper lists this as future work rather than testing it.
  • The same tree-relative credit idea could transfer to text-only reasoning if branching is defined at uncertainty boundaries instead of `<look>` markers, but the paper only demonstrates the multimodal setting.
  • The temporal replay buffer with question-balanced sampling could be ablated independently to see whether its gains persist without tree-structured rollouts, isolating the two contributions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces Multi-Branch Policy Optimization (MBPO), a tree-structured reinforcement learning framework for multimodal large language models that constructs reasoning trees via breadth-first parallel expansion, computes segment-level sibling-relative advantages as in Eq. (10), and reuses past segments through a temporal replay buffer with question-balanced sampling. The authors report consistent gains over GRPO, DAPO, GSPO, Shuffle-R1, and MM-Eureka on Geometry3K, K12, and several out-of-domain benchmarks, and argue that branching at vision-language decision boundaries improves credit assignment and mitigates relative advantage degeneration.

Significance. If the central claims hold, MBPO would be a useful contribution to RL training of multimodal LLMs, offering a concrete alternative to trajectory-level advantage assignment and providing a released open-source implementation. The reported gains on MathVerse, MathVision, ChartQA, HallusionBench, and WeMath are encouraging. However, the significance is substantially weakened by (i) the absence of comparisons to tree-based RL baselines (TreePO, TreeRPO, TreeRL) that are most relevant to the method, (ii) a lack of direct evidence that sibling branches correspond to distinct visual hypotheses, and (iii) a compute-controlled analysis whose control conditions are not clearly specified. The paper's own ablation (Table 7) suggests that fixed-length branching, not vision-language-boundary branching, drives most of the improvement, which undercuts the proposed mechanism.

major comments (5)
  1. [Section 4.4, Table 7] The central mechanism claim that MBPO branches at vision-language decision boundaries to explore distinct visual hypotheses is not supported by the ablation. Table 7 shows that <look>-only branching achieves 45.17, worse than fixed-M branching at 48.35, and the hybrid's gain over fixed-M is only 1.56 points (49.91 vs 48.35). If visual boundary branching were the source of the improvement, one would expect it to outperform or at least match fixed-length segmentation. The paper does not measure whether sibling branches commit to different visual interpretations; the diversity statistics in Figure 7 are about distinct questions, not visual hypotheses, and the self-correction analysis in Tables 4-5 relies on textual cue words. This leaves the causal premise of Eq. (3) and Eq. (10) unverified.
  2. [Section 3.1, Eq. (1), Figure 2] The 'relative advantage degeneration' motivation is partly circular. The valid advantage ratio in Eq. (1) counts advantages with |A_i| > 0. For GRPO, advantages are group-relative and can be exactly zero when a response equals the group mean or when all rewards in the group are identical. For MBPO, Eq. (10) normalizes sibling-relative differences by std + epsilon, so advantages are non-zero by construction whenever sibling values differ, and the epsilon term ensures non-zero values even when they do not. Thus Figure 2's near-1.0 VAR for MBPO may be an artifact of the advantage definition rather than evidence of a better learning signal. The paper should report VAR for MBPO computed using the same trajectory-level advantage formula as GRPO on the same rollouts to make the comparison meaningful.
  3. [Section 2.2 and Section 4.2] No tree-based RL baselines are evaluated. The related work explicitly positions MBPO against TreePO, TreeRPO, and TreeRL, which are the closest methods in using tree-structured rollouts for policy optimization. Without including these or equivalent tree-based methods in Tables 1-3, the observed gains cannot be attributed to branch-level sibling-relative advantages over trajectory-level advantages; they could simply stem from the tree-structured sampling strategy or from a larger number of effective samples per question. Adding at least one tree-based baseline on the same benchmarks is essential for the paper's central comparative claim.
  4. [Section 4.3, Table 6] The compute-controlled analysis is not convincingly controlled. The paper states that per-epoch GPU cost is nearly identical (19.48 vs 19.32 GPU-hours on 4 GPUs) despite MBPO generating K^D children per question with branching factors such as [6,6,6]. For a fixed token budget per segment, this implies MBPO generates far more tokens per question than GRPO, so the claim that the gains are not due to increased rollout expenditure requires an explicit specification of the total token budget or rollout count per question per method. Table 6 aligns methods by cumulative TFLOPs but does not state how many generated tokens each method consumed at each aligned checkpoint. Without this information, the 'compute-matched' comparison does not rule out the possibility that MBPO simply uses more rollouts.
  5. [Tables 1-3] All reported results are single-run point estimates without standard deviations, confidence intervals, or significance tests. The claimed in-domain gains over the best baselines are often small (e.g., 1.0 and 2.5 points on MathVerse and MathVision in Table 1; 1.63 points on Geo3K Math Avg in Table 2), and the K12 comparison in Table 3 shows MBPO-Qwen-7B is within 0.03 points of DAPO on K12 accuracy (67.68 vs 67.65). Given the known variance of RL fine-tuning runs, these differences may not be statistically robust. The paper should report multiple seeds or at least a significance analysis for the central comparisons.
minor comments (5)
  1. [Algorithm 1, line 26] Line 26 says 'compute normalized sibling-relative advantages by Eq. (7)', but Eq. (7) is the parent-child difference A(n) = R(n) - R(parent(n)); the normalized sibling-relative advantage is defined in Eq. (10). The algorithm should refer to Eq. (10).
  2. [Section 3.3, Eq. (8)-(10)] The text states that sibling nodes share the same sequence length, but the <look>-based branching can produce segments of varying lengths, as evidenced by the average segment-length values in Table 7 (e.g., 287.4 tokens for <look>-only). Please clarify how Eq. (10) ensures a fair comparison when sibling subsequences have different lengths.
  3. [Figure 7 and Section 4.3] Figure 7 measures 'Distinct Questions' in the replay buffer, which is a question-coverage statistic, not a measure of visual hypothesis diversity among sibling branches. The caption and the surrounding text should be careful not to imply that this measures perceptual diversity.
  4. [Table 1 footnote] The footnote says all RL methods are grouped and retrained on MMRL18K, but the table includes cold-start models marked with † and models from different base architectures. Please specify the exact training protocol, data, and base model for each baseline, and clarify which values are taken from original papers versus re-implemented.
  5. [General notation] The dataset is referred to as both 'MMK12' and 'K12' in different places (e.g., Section 4.1 vs Tables 2-3). Please unify the terminology.

Circularity Check

1 steps flagged · score 3.0 of 10

Mechanism evidence is partly definitional: MBPO's near-1 valid advantage ratio is guaranteed by the normalized sibling-relative definition in Eq. (10), while the headline benchmark gains remain independent.

  1. self definitional [Section 3.3, Eq. (10); Section 4.3 'Advantage Density Analysis'; Figure 6]
    "The branch-level advantage of node n is denoted as A(n) and is defined through relative comparison with its sibling nodes. ... A_sib(n)= [Vhat(n)-mean_{n' in Sib(n)} Vhat(n')]/[std_{n' in Sib(n)} Vhat(n')+epsilon]. ... In contrast, MBPO shows a different profile, with little mass at zero and density concentrated on non-zero advantages."

    Eq. (10) defines every branch advantage as a normalized deviation from the sibling mean, so any non-identical sibling rewards produce a nonzero advantage regardless of whether the branch corresponds to a distinct visual hypothesis. The paper's evidence against 'relative advantage degeneration' (VAR near 1.0 in Fig. 2 and 'little mass at zero' in Fig. 6) is therefore guaranteed by the definition of the advantage, not an independent confirmation that credit assignment is better. The degeneration phenomenon is itself measured by VAR (Eq. 1), so MBPO's high VAR is structurally expected.

full rationale

The headline performance comparisons are independent and not circular: MBPO is evaluated against GRPO, DAPO, GSPO, Shuffle-R1, and MM-Eureka on held-out benchmarks, and the compute-matched analysis addresses the alternative explanation of extra rollout budget. No load-bearing self-citation chain was found; the cited prior work on relative advantage degeneration is external, and the paper's own contributions do not reduce to a self-citation. The circular component is confined to the mechanism evidence: the valid advantage ratio (VAR) is defined as the proportion of nonzero advantages, and Eq. (10) defines branch advantages as normalized sibling deviations, so nonzero advantages are nearly guaranteed by construction whenever siblings differ. Thus the observed 'stable non-zero advantage proportion' and 'little mass at zero' are definitional consequences rather than empirical confirmation of improved credit assignment. The missing evidence that sibling branches represent distinct visual hypotheses is a genuine mechanistic gap, but it is a correctness/evidence concern, not circularity. Overall, the central empirical claim stands independently, while one piece of supporting evidence reduces by definition, warranting a score of 3.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a heuristic value model (mean child reward), on the assumption that sampled sibling branches isolate visual interpretation differences, and on several hand-chosen hyperparameters (branch factors, segment length, buffer window). No new physical entities are introduced. The main unresolved burden is the lack of a formal or empirical check that sibling branches correspond to distinct visual hypotheses.

free parameters (5)
  • Branch factor configuration [K1..KD] = 6-6-6 selected via ablation
    Tree width and depth are chosen by ablating on Geo3K (Section 4.4, Figure 8); other tested values are 4-4-4 and 8-8-8.
  • Segment length M = 100 tokens
    Chosen via ablation on Geo3K (Table 7); controls branching density and semantic alignment.
  • Replay time window Tmax = 8 training iterations
    Hyperparameter in Eq. (12) controlling policy staleness; no sensitivity analysis is reported.
  • Max subsequences per question Smax = 32
    Hyperparameter in Eq. (13) controlling question balance in mini-batch sampling; no sensitivity analysis is reported.
  • Numerical stabilizer epsilon in Eq. (10) = unspecified
    Used in the sibling-relative advantage normalization; the exact value is not reported.
assumptions (4)
  • domain assumption Sibling branches sampled from the same policy under a shared prefix represent distinct, comparable visual hypotheses, so sibling-relative advantage differences isolate the value of committing to one visual interpretation.
    Invoked in Eq. (3) and Eq. (10); if siblings are just random token-level samples, branch-relative advantages do not measure visual grounding differences. This is the weakest load-bearing premise.
  • ad hoc to paper Mean reward propagation from leaf nodes to internal nodes (Eq. 6 and Eq. 9) gives a valid estimate of the value of an internal reasoning state.
    No theoretical justification is provided; it is a Monte Carlo mean over sampled children, biased by finite samples and by how leaf rewards are scored.
  • domain assumption The <look> marker, when present, marks causally meaningful vision-language decision boundaries, and fixed-length splits at M tokens preserve these boundaries when the marker is absent.
    Used in Algorithm 1 lines 14-18 and Section 3.2; no analysis of the marker's reliability or semantic alignment is provided.
  • domain assumption The rule-based reward r(s_m) correctly measures reasoning correctness on all benchmarks.
    Eq. (2) and Eq. (5) assume a scalar reward faithfully reflects whether the answer matches ground truth, but the exact reward function is not specified in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Branch Policy Optimization for Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/GKF5GQG7

@misc{pith2026260807581,
  author       = {Pith},
  title        = {Pith review of: Multi-Branch Policy Optimization for Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GKF5GQG7}},
  note         = {Machine review of arXiv:2608.07581}
}
read the original abstract

Group-based reinforcement learning methods for multimodal large language models typically rely on trajectory-level credit assignment that applies a single advantage to all tokens in a response. However, multimodal reasoning involves substantially higher perceptual uncertainty than text-only settings, where the model must repeatedly re-examine visual information to verify intermediate interpretations, and different visual groundings can lead to divergent reasoning paths, making such uniform credit assignment particularly inadequate and causing relative advantages to progressively degenerate toward zero. To address these challenges, we propose Multi-Branch Policy Optimization (MBPO), a tree-based framework that constructs reasoning trees at vision-language decision boundaries, enabling sibling branches to explore diverse visual hypotheses and assigning segment-level credit through branch-relative advantages. We further introduce a temporal replay buffer to reuse informative segments while controlling policy staleness. Experiments on several multimodal reasoning benchmarks show that MBPO outperforms representative baselines, improving both learning signal quality and optimization efficiency. The code is publicly available at https://github.com/ShuaiLyu0110/MBPO.

Figures

Figures reproduced from arXiv: 2608.07581 by the authors.

Figure 2
Figure 2. Relative advantage degeneration during training on [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 1
Figure 1. The difference between MBPO and GRPO in terms [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. The illustration of MBPO. The figure illustrates how MBPO uses tree-structured rollouts to share reasoning branches [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Time-window replay buffer with question-balanced mini-batch sampling, designed to limit stale trajectories and [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Test accuracy during the training process of MBPO [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Estimated advantage distribution across training [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Replay buffer diversity under different MBPO tree [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: (a) Test accuracy over training time (hours) for MBPO, GRPO, and DAPO on Geo3K. (b) Test accuracy over training [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 10 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Ibar Federico Anderson. 2025. Comparative Analysis Between Industrial Design Methodologies Versus the Scientific Method: AI: Claude 3.7 Sonnet.Preprints.org (2025)

    work page 2025

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al . 2025. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923(2025)

    arXiv 2025

  4. [4]

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. 2025. Sft or rl? an early investigation into training r1-like reasoning large vision-language models.arXiv preprint arXiv:2504.11468(2025)

    arXiv 2025

  5. [5]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al . 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271(2024)

    arXiv 2024

  6. [6]

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261(2025)

    arXiv 2025

  7. [7]

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang

  8. [8]

    Yong Deng, Guoqing Wang, Zhenzhe Ying, Xiaofeng Wu, Jinzhen Lin, Wen- wen Xiong, Yuqin Dai, Shuo Yang, Zhanwei Zhang, Qiwen Wang, et al . 2025. Atom-searcher: Enhancing agentic deep research via fine-grained atomic thought reward.arXiv preprint arXiv:2508.12800(2025)

    arXiv 2025

Show all 41 references
  1. [9]

    Lihuang Fang, Yuchen Zou, Jin kebing, and Jinghui Qin. 2026. EmoAgent- R1:Towards Multimodal Emotion Understanding with Reinforcement Learning- based Dynamic Agent Specialization. InProceedings of the 34th ACM International Conference on Multimedia

    2026

  2. [10]

    Yanglin Feng, Yang Qin, Dezhong Peng, Rui Wang, Xiaomin Song, and Peng Hu. [n. d.]. Multimodal Nested Learning for Decoupled and Coordinated Optimization. InForty-third International Conference on Machine Learning

  3. [11]

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. 2024. HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large ...

    2024 arXiv

  4. [12]

    Zhenyu Hou, Ziniu Hu, Yujiang Li, Rui Lu, Jie Tang, and Yuxiao Dong. 2025. Treerl: Llm reinforcement learning with on-policy tree search. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 12355–12369

    2025

  5. [13]

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749(2025)

    2025 arXiv

  6. [14]

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. 2024. Openai o1 system card.arXiv preprint arXiv:2412.16720(2024)

    2024 arXiv

  7. [15]

    Sicong Leng, Jing Wang, Jiaxi Li, Hao Zhang, Zhiqiang Hu, Boqiang Zhang, Hang Zhang, Yuming Jiang, Xin Li, Deli Zhao, et al. 2025. Mmr1: Advancing the frontiers of multimodal reasoning

    2025

  8. [16]

    Yizhi Li, Qingshui Gu, Zhoufutu Wen, Ziniu Li, Tianshun Xing, Shuyue Guo, Tianyu Zheng, Xin Zhou, Xingwei Qu, Wangchunshu Zhou, et al. 2025. Treepo: Bridging the gap of policy optimization and efficacy and inference efficiency with heuristic tree-based modeling.arXiv preprint ...

    2025 arXiv

  9. [17]

    Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. 2025. Noisyrollout: Reinforcing visual reasoning with data augmentation, 2025a.URL https://arxiv. org/abs/2504.13055(2025)

    2025

  10. [18]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts. arXiv:2310.02255 [cs.CV] https://arxiv.org/ab...

    2024 arXiv

  11. [19]

    Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. 2021. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning.arXiv preprint arXiv:2105.04165(2021)

    2021 arXiv

  12. [20]

    Shuai Lyu, Zijing Tian, Zhonghong Ou, Yifan Zhu, Xiao Zhang, Qiankun Ha, Haoran Luo, and Meina Song. 2025. TSVC: Tripartite Learning with Semantic Variation Consistency for Robust Image-Text Retrieval.Proceedings of the AAAI Conference on Artificial Intelligence39, 18 (April 2...

    2025

  13. [21]

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. arXiv:2203.10244 [cs.CL] https://arxiv.org/abs/2203.10244

    2022 arXiv

  14. [22]

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. 2025. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365(2025)

    2025 arXiv

  15. [23]

    Siwei Meng, Yawei Luo, Shu Zhang, and Ping Liu. 2026. When Physical Pref- erences Meet Semantic Constraints: Physical and Semantic Direct Preference Optimization for Text-to-Video Generation.arXiv preprint arXiv:2607.16947 (2026)

    2026 arXiv

  16. [24]

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. 2024. We-Math: Does Your Large Multimodal Model...

    2024 arXiv

  17. [25]

    Yang Qin, Chao Chen, Zhihang Fu, Dezhong Peng, Xi Peng, and Peng Hu. 2025. Human-centered interactive learning via mllms for text-to-image person re- identification. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR). IEEE, 14390–14399

    2025

  18. [26]

    Yang Qin, Yuan Sun, Xi Peng, Dezhong Peng, Joey Tianyi Zhou, Xiaomin Song, and Peng Hu. 2026. Robust semi-paired multimodal learning for cross-modal retrieval. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 24964–24972

    2026

  19. [27]

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning.arXiv preprint arXiv:2504.08837(2025)

    2025 arXiv

  20. [28]

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024. Measuring multimodal mathematical reasoning with math-vision dataset.Advances in Neural Information Processing Systems37 (2024), 95095–95169

    2024

  21. [29]

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. 2025. Sota with less: Mcts- guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934(2025)

    2025 arXiv

  22. [30]

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. 2025. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615(2025)

    2025 arXiv

  23. [31]

    Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. 2025. Treerpo: Tree relative policy optimization.arXiv preprint arXiv:2506.05183(2025)

    2025

  24. [32]

    Huanjin Yao, Qixiang Yin, Jingyi Zhang, Min Yang, Yibo Wang, Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, et al. 2025. R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO. arXiv preprint arXiv:2505.16673(2025)

    2025 arXiv

  25. [33]

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. Dapo: An open- source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476 (2025)

    2025 arXiv

  26. [34]

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. 2025. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937(2025)

    2025 arXiv

  27. [35]

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. 2024. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems?. InEuropean MM ’26, November 10–14, 2026, Rio de Janeiro,...

    2024

  28. [36]

    Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin

  29. [37]

    Zhihong Shao, Peiyi Wang

    Qihao Zhu. Zhihong Shao, Peiyi Wang. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. https://arxiv.org/abs/ 2402.03300

    2024 arXiv

  30. [38]

    arXiv:2507.18071 [cs.LG] https: //arxiv.org/abs/2507.18071

    Group Sequence Policy Optimization. arXiv:2507.18071 [cs.LG] https: //arxiv.org/abs/2507.18071

    arXiv

  31. [39]

    Linghao Zhu, Yiran Guan, Dingkang Liang, Jianzhong Ju, Zhenbo Luo, Bin Qin, Jian Luan, Yuliang Liu, and Xiang Bai. 2025. Shuffle-r1: Efficient rl framework for multimodal large language models via data-centric dynamic shuffle.arXiv preprint arXiv:2508.05612(2025)

    2025

  32. [40]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. 2025. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479(2025)

    2025 arXiv

  33. [2025]

    Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement.arXiv preprint arXiv:2503.17352(2025)

    2025 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.