Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PeRL shows that permuting image order during RL training makes a 7B VLM reason reliably across multiple images and beat prior baselines.

desk verdict PeRL is a useful, believable RL recipe for multi-image VLMs, but the authors must pin down the permutation answer-modification rule and validate GPT-4o labels before I'd trust the headliner gains. read the letter →

arxiv 2506.14907 v1 pith:56IQOZI4 submitted 2025-06-17 cs.CV cs.AI

Yizhen Zhang , Yang Ding , Shuoshuo Zhang , Xinchen Zhang , Haoling Li , Zhong-zhi Li , Peijie Wang , Jie Wu
show 4 more authors
Lei Ji Yelong Shen Yujiu Yang Yeyun Gong
This is my paper
classification cs.CVcs.AI
keywords reinforcementlearningvision-languagemodelsmulti-imagereasoningpositionalbiasGRPOpermutationaugmentationrolloutfilteringinterleavedmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to make vision-language models reason reliably across multiple interleaved images, not just within a single image. It claims that a reinforcement-learning recipe called PeRL, which randomly permutes the order of input images, rewrites the question when the image order changes the meaning, and filters rollouts to concentrate on informative samples, mitigates the positional bias that makes multi-image models flip answers when image order changes. If this is right, a model trained this way scores 76.39 on Mantis-Eval, 58.53 on BLINK, and 54.23 on MMIU, beating existing 7B instruction-tuned and R1-style baselines, while staying competitive on single-image math benchmarks. The central message is that multi-image reasoning can be improved with a small RL-only dataset plus an order-diversifying exploration mechanism.

What carries the argument

The load-bearing object is the permutation-augmented GRPO group: for each training sample, the pipeline creates $n_s = 1$ randomly swapped image order, updates the answer to match via $\Lambda(y,\sigma)$, and pools rollouts from all orders into one baseline so the policy receives gradient signal from the difference between orders. The second mechanism is rollout filtering: questions are scored by average accuracy over 10 policy rollouts, and only low-scoring questions are kept, shifting the difficulty mean from 0.78 to 0.38. The third is the semantic-variation prompt that produces the indicator $S$ and the answer-transformation function $\Lambda$, introduced in Equation (1) to prevent augmented samples from teaching stale answers after a permutation changes the meaning.

What would settle it

Take 200 questions from the 22K training set, relabel semantic equivalence with human annotators, and compare the GPT-4o labels for both the change decision and the transformed answer; if disagreement is more than a small fraction, the advantage estimates are computed against wrong answers. Alternatively, run the final model on every permutation of a held-out set of 100 multi-image questions and check whether accuracy and the predicted option stay invariant.

Watch

Extended reading notes

Core claim

PeRL establishes that permutation-enhanced GRPO is an effective way to train interleaved vision-language models to reason across images. Starting from a Qwen2.5-VL-7B-Instruct policy and only 22K multi-image examples plus 36K single-image examples, the method computes semantic-variation labels with GPT-4o, then in each training step generates one randomly permuted copy of each sample and merges all rollouts from all orders into a single advantage baseline. The shared baseline makes permuted and unpermuted trajectories compete, which spreads the advantage distribution and prevents the near-zero advantages that naive GRPO produces on multi-image inputs. The paper reports consistent gains across five multi-image benchmarks and no degradation on three single-image math benchmarks.

Load-bearing premise

Everything rests on GPT-4o's semantic-equivalence judgments being right: if it cannot reliably decide whether permuting the image order changes the intended answer, or cannot rewrite the answer correctly, then the permutation-augmented rewards teach the policy to be confidently wrong.

Editorial extensions

If this is right

  • Multi-image positional reasoning can be improved with RL alone on a small curated subset of roughly 22K examples, without large-scale supervised multi-image instruction tuning.
  • The permutation trick generalizes across benchmark families: gains appear on natural-scene tasks (Mantis-Eval, BLINK, MMIU) and on out-of-domain multi-image math benchmarks (Remi, MV-MATH).
  • Permuting more than once per sample ($n_s = 2$) helps spatially oriented tasks like BLINK and Mantis-Eval but slightly hurts reasoning-centric benchmarks, so permutation intensity is a tuning knob rather than a free lunch.
  • Because the method is a plug-in to GRPO, the same recipe could be transferred to other VLM backbones and other interleaved multimodal settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the GPT-4o semantic-equivalence labels are noisy on a nontrivial fraction of the 22K samples, the advantage estimator normalizes against wrong answers; auditing a few hundred of these labels against human judgment is the cheapest check of the whole pipeline.
  • The permutation operation enforces invariance under the symmetric group on image positions, making PeRL a group-invariance data augmentation for order-sensitive inputs; the same idea should transfer to video frame order or multi-view reasoning.
  • The observed advantage-spread effect suggests a testable hypothesis: any augmentation that creates input diversity within a GRPO group may improve learning efficiency, so image corruptions, crops, or question rephrasing could substitute when order is semantically fixed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes PeRL, a GRPO-based reinforcement learning method for interleaved multi-image vision-language reasoning. The method augments each training sample by permuting the order of input images and correspondingly modifying the expected answer, then pools rollouts from the original and permuted variants to compute advantages. A preprocessing pipeline filters Mantis-Instruct by question type, rephrases open-ended questions with GPT-4o, performs rollout-based difficulty filtering using Qwen2.5-VL-7B, and labels semantic variation under image permutation with GPT-4o. The resulting model, initialized from Qwen2.5-VL-7B-Instruct and trained on 22K multi-image and 36K single-image examples, is evaluated on five multi-image and three single-image benchmarks. The paper reports state-of-the-art results on Mantis-Eval, BLINK, and MMIU among compared 7B models, competitive single-image math performance, and ablations on permutation count, data mix, and rollout diversity.

Significance. If the empirical results are reproducible, PeRL is a simple and data-efficient enhancement: it extends R1-style rule-based RL to multi-image positional reasoning without supervised fine-tuning, and the out-of-domain gains on Remi and MV-MATH are encouraging. The paper's strengths include broad evaluation across eight benchmarks, a disentanglement ablation separating single-image and multi-image data (Table 3), a rollout-diversity analysis, and a public code link. However, the central data-generation mechanism -- the semantic-equivalence labeling and the answer transformation Lambda -- is under-specified and unvalidated, and several training hyperparameters needed to reproduce the pipeline are missing. The correctness-risk is concentrated in this mechanism; if the labels or transformed answers are wrong for a meaningful fraction of the 22K multi-image examples, the GRPO rewards in Eqs. (4)-(6) are computed against corrupted targets, which would undermine the reported gains.

major comments (3)
  1. [Section 3.3, Eq. (1); Algorithm 1] The answer transformation Lambda(y, sigma) is never concretely defined. Eq. (1) defines yhat = S(x, xhat)*y + (1 - S(x, xhat))*Lambda(y, sigma), but the text only says that Lambda adapts y 'such as reindexing visual choices,' and Algorithm 1's online generation step says only 'Generate randomly swapped images and modify answers correspondingly.' This is load-bearing because every rule-based reward in Eqs. (4)-(6) is computed against yhat: if GPT-4o's semantic-equivalence judgment S is wrong, or if Lambda produces an incorrect reindexed or rewritten answer for open-ended or free-form questions, then the policy is trained against corrupted targets for a meaningful fraction of the 22K multi-image examples. The paper provides no human validation of S, no consistency check between the offline GPT-4o semantic-variation labels and the online permutation implementation, and no per-question-type specification of Lambda. Please specify Lambda exactly for each question type in Mantis-Instruct, report its coverage and failure modes, and provide an agreement study (human or rule-based) for both S and Lambda.
  2. [Section 3.3, Fig. 3b; Section 5] The difficulty-filtering pipeline is not reproducible. The text reports that questions are filtered based on average rollout accuracy, with the mean score falling from 0.78 to 0.38, but it never gives the filtering threshold, the resampling target, or the number of examples retained after each preprocessing stage. In addition, the initial permutation probability alpha_0 in Algorithm 1 is never reported, even though Section 4.2 states that the swap probability decays linearly over training. Without these values, the 22K multi-image training subset cannot be reconstructed, and the paper's data-efficiency claim cannot be checked. Please report alpha_0, the decay schedule, the exact difficulty threshold, the final dataset sizes after each stage, and the number of examples removed by rule-based filtering, format rephrasing, rollout filtering, and semantic-variation checking.
  3. [Table 2 and Fig. 5] The permutation-intensity ablation and the training-curve comparisons appear to be single runs, and several differences are small relative to expected evaluation noise. For example, ns=1 versus ns=2 changes Mantis-Eval from 76.39 to 77.78, BLINK from 58.53 to 59.50, MathVista from 73.0 to 72.2, and Remi from 41.65 to 39.04; Mantis-Eval has only 217 test samples (Table 4). The claim that 'increasing ns from 0 to 2 improves performance on most benchmarks' and the associated trade-off conclusion need repeated seeds with standard deviations or significance tests. Without these, the recommendation of ns=2 and the interpretation of the permutation gain over naive GRPO are not statistically supported.
minor comments (4)
  1. [Section 3 and Section 3.3] The heading 'Formuation' in Section 3 should be 'Formulation', and the heading 'Data Prepocessing' in Section 3.3 should be 'Data Preprocessing'.
  2. [Section 5 vs. Appendix A.1] The main text states a batch size of 256 in Section 5, while Appendix A.1 states a batch size of 128; please reconcile this inconsistency.
  3. [Algorithm 1] The role of alpha_t is ambiguous: if alpha_t is the probability that a swap is applied, then some of the ns+1 samples may be identical to the original input; please define P((x1, y1), alpha_t) explicitly and state what happens when no swap is sampled for a given i.
  4. [References] The reference list contains malformed entries: [6] includes 'less than $3' in the title and an incomplete URL, and [1] lacks a complete author list and standard bibliographic information.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: PeRL's headline gains are measured on external benchmarks; the self-referential rollout-filtering step is a data-construction choice, not a fitted-input-as-prediction.

full rationale

PeRL's claimed derivation chain is: (1) curate Mantis-Instruct data with rule-based filtering, GPT-4o reformatting, Qwen2.5-VL rollout filtering, and GPT-4o semantic-variation labeling; (2) train Qwen2.5-VL-7B-Instruct with GRPO using permuted image orders and modified answers (Eqs. 1-6, Algorithm 1); and (3) evaluate on the external benchmarks Mantis-Eval, BLINK, MMIU, MathVista, MathVerse, MathVision, Remi, and MV-MATH. None of the evaluation quantities is defined in terms of the trained model's outputs or the fitted preprocessing choices. The rollout-filtering step uses the base Qwen2.5-VL model to score difficulty and select training data, but this is a data-set construction step, not a 'prediction' later validated against those same scores; all headline numbers come from held-out external benchmarks. The semantic-variation label S(x,xhat) from GPT-4o and the answer-transformation Lambda(y,sigma) in Eq. (1) are under-specified and unvalidated, which is a correctness and robustness risk, but not circularity: the training target yhat is an input to the pipeline, not the output of the derivation being verified. The paper cites its own MV-MATH benchmark [40] and survey [21] with overlapping authorship, but those citations are not used to justify the method's effectiveness; MV-MATH is an external test set whose labels were fixed before this training run. Therefore no claim in the paper reduces by construction to its own inputs, and no self-citation chain is load-bearing.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method introduces no new theoretical entities. It relies on GRPO, a proprietary label oracle (GPT-4o), and a set of unverified data-selection assumptions. The key unstated free parameters are alpha_0 and the difficulty-filter threshold.

free parameters (5)
  • alpha_0 (initial image-swap probability) = not reported
    Linearly decaying factor alpha_t controls how often images are permuted during RL training; the initial value is never given in the paper, yet it directly controls the exploration schedule.
  • difficulty filter threshold / resampling target = not reported (mean difficulty shifted from 0.78 to 0.38)
    Section 3.3 filters questions based on Qwen2.5-VL rollout accuracy, but the exact inclusion rule is not specified, and the post-filter mean score (0.38) suggests a strong preference for hard samples that shapes the training distribution.
  • ns (number of permuted samples per input) = 1
    Set to 1 in main experiments; ablation shows ns=2 is better on some benchmarks and worse on others, so this choice affects reported results.
  • n (rollouts per order) = 6
    Specified in Section 5; group size is 12 total rollouts with ns=1.
  • beta (KL coefficient) = 0.01
    Standard GRPO hyperparameter, chosen without sensitivity analysis.
assumptions (4)
  • standard math GRPO advantage and policy loss equations (Eqs. 2-3) are valid and correctly implemented
    Taken from DeepSeekMath [30]; not rederived.
  • domain assumption GPT-4o's semantic equivalence labels S and answer transformations Lambda are accurate for the 22K multi-image samples
    Invoked in Section 3.3 Eq. (1) and used to construct permuted training labels; no validation is provided.
  • domain assumption Rule-based reward functions correctly judge correctness after GPT-4o reformats questions into multiple-choice form
    Section 4: reward is computed by accuracy and format; the paper does not analyze reward noise.
  • domain assumption Qwen2.5-VL rollout accuracy is a useful difficulty measure for filtering training data
    Section 3.2-3.3: samples are filtered by the same model family that is subsequently trained; this is a self-bootstrapping assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning." pith.science (2026). https://pith.science/paper/56IQOZI4

@misc{pith2026250614907,
  author       = {Pith},
  title        = {Pith review of: PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56IQOZI4}},
  note         = {Machine review of arXiv:2506.14907}
}
read the original abstract

Inspired by the impressive reasoning capabilities demonstrated by reinforcement learning approaches like DeepSeek-R1, recent emerging research has begun exploring the use of reinforcement learning (RL) to enhance vision-language models (VLMs) for multimodal reasoning tasks. However, most existing multimodal reinforcement learning approaches remain limited to spatial reasoning within single-image contexts, yet still struggle to generalize to more complex and real-world scenarios involving multi-image positional reasoning, where understanding the relationships across images is crucial. To address this challenge, we propose a general reinforcement learning approach PeRL tailored for interleaved multimodal tasks, and a multi-stage strategy designed to enhance the exploration-exploitation trade-off, thereby improving learning efficiency and task performance. Specifically, we introduce permutation of image sequences to simulate varied positional relationships to explore more spatial and positional diversity. Furthermore, we design a rollout filtering mechanism for resampling to focus on trajectories that contribute most to learning optimal behaviors to exploit learned policies effectively. We evaluate our model on 5 widely-used multi-image benchmarks and 3 single-image benchmarks. Our experiments confirm that PeRL trained model consistently surpasses R1-related and interleaved VLM baselines by a large margin, achieving state-of-the-art performance on multi-image benchmarks, while preserving comparable performance on single-image tasks.

Figures

Figures reproduced from arXiv: 2506.14907 by the authors.

Figure 1
Figure 1. Compare PeRL against baseline approaches, including instruction-tuned or R1-like rein [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A showcase highlighting the challenges faced by existing VLMs: 1) Fail to recognize or [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of Data Preprocessing and Difficulty Distribution. (a) The multi-stage pipeline [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: An illustration of PeRL framework. (x1, y1) denote original input sample, while {(xi , yi)} ns+1 i=2 represent samples generated through permutation operation, ns is the number of permutations per sample and n is the number of rollouts per order. oi,k,ri,k and Ai,k rep…
Figure 5
Figure 5. Figure 5: Permutation GRPO achieves more consistent training behavior. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: A showcase for motivation If image 3 is the reference image, which image of the other two is more similar to the reference image? Image 2 image 3 Here is a statement describing these 2 images: The train in one of the images has just come around a bend. Is it true or fa…
Figure 7
Figure 7. Figure 7: Training Data preprocessing pipeline explanation and several cases [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Case study demonstrating positional bias. The model correctly identifies the similar image [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Difference on advantage between Naive GRPO and Permuation GRPO during training [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoning

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    RAPO uses an information-theoretic lower bound on visual gain to select high-entropy reflection anchors and optimizes a chain-masked KL surrogate, delivering gains over baselines on reasoning benchmarks across LVLM backbones.

Reference graph

Works this paper leans on

54 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4v(ision) system card. 2023

    work page 2023

  2. [2]

    Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    arXiv 2024

  3. [3]

    Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.Advances in neural information processing systems, 35:23716–23736, 2022

    2022

  4. [4]

    Openflamingo: An open- source framework for training large autoregressive vision-language models.arXiv preprint arXiv:2308.01390, 2023

    Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open- source framework for training large autoregressive vision-language models.arXiv preprint arXiv:2308.01390, 2023

    arXiv 2023

  5. [5]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  6. [6]

    R1-v: Reinforcing super generaliza- tion ability in vision-language models with less than $3.https://github.com/Deep-Agent/ R1-V, 2025

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generaliza- tion ability in vision-language models with less than $3.https://github.com/Deep-Agent/ R1-V, 2025. Accessed: 2025-02-02

    2025

  7. [7]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  8. [8]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.Science China Information Sciences, 67(12):220101, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites.Science China Information Sciences, 67(12):220101, 2024

    2024

Show all 54 references
  1. [9]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer visi...

    2024

  2. [10]

    Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: An early exploration to complex vision-language reasoning via iterative self- improvement.arXiv preprint arXiv:2503.17352, 2025

    2025 arXiv

  3. [11]

    Vlmevalkit: An open-source toolkit for evaluating large multi-modality models

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. InProceedings of the 32nd ACM International Conference on Multimedia,...

    2024

  4. [12]

    Blink: Multimodal large language models can see but not perceive

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. InEuropean Conference on Computer Vision, pages 148–166. Springer, 2024

    2024

  5. [13]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 10

    2025 arXiv

  6. [14]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  7. [15]

    Mantis: Interleaved multi-image instruction tuning.arXiv preprint arXiv:2405.01483, 2024

    Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning.arXiv preprint arXiv:2405.01483, 2024

    2024 arXiv

  8. [16]

    Remi: A dataset for reasoning with multiple images.Advances in Neural Information Processing Systems, 37:60088– 60109, 2024

    Mehran Kazemi, Nishanth Dikkala, Ankit Anand, Petar Devic, Ishita Dasgupta, Fangyu Liu, Bahare Fatemi, Pranjal Awasthi, Sreenivas Gollapudi, Dee Guo, et al. Remi: A dataset for reasoning with multiple images.Advances in Neural Information Processing Systems, 37:60088– 60109, 2024

    2024

  9. [17]

    Mimic-it: Multi-modal in-context instruction tuning.arXiv preprint arXiv:2306.05425, 2023

    Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Mimic-it: Multi-modal in-context instruction tuning.arXiv preprint arXiv:2306.05425, 2023

    2023 arXiv

  10. [18]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  11. [19]

    Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models

    Feng Li, Renrui Zhang, Hao Zhang, Yuanhan Zhang, Bo Li, Wei Li, Zejun Ma, and Chunyuan Li. Llava-next-interleave: Tackling multi-image, video, and 3d in large multimodal models. arXiv preprint arXiv:2407.07895, 2024

    2024 arXiv

  12. [20]

    Migician: Revealing the magic of free-form multi-image grounding in multimodal large language models.arXiv preprint arXiv:2501.05767, 2025

    You Li, Heyu Huang, Chi Chen, Kaiyu Huang, Chao Huang, Zonghao Guo, Zhiyuan Liu, Jinan Xu, Yuhua Li, Ruixuan Li, et al. Migician: Revealing the magic of free-form multi-image grounding in multimodal large language models.arXiv preprint arXiv:2501.05767, 2025

    2025 arXiv

  13. [21]

    From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025

    Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. From system 1 to system 2: A survey of reasoning large language models.arXiv preprint arXiv:2502.17419, 2025

    2025 arXiv

  14. [22]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  15. [23]

    Noisyrollout: Reinforcing visual reasoning with data augmentation

    Xiangyan Liu, Jinjie Ni, Zijian Wu, Chao Du, Longxu Dou, Haonan Wang, Tianyu Pang, and Michael Qizhe Shieh. Noisyrollout: Reinforcing visual reasoning with data augmentation. arXiv preprint arXiv:2504.13055, 2025

    2025

  16. [24]

    Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning.arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  17. [25]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    arXiv

  18. [26]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  19. [27]

    Mmiu: Multimodal multi-image understanding for evaluating large vision-language models.arXiv preprint arXiv:2408.02718, 2024

    Fanqing Meng, Jin Wang, Chuanhao Li, Quanfeng Lu, Hao Tian, Jiaqi Liao, Xizhou Zhu, Jifeng Dai, Yu Qiao, Ping Luo, et al. Mmiu: Multimodal multi-image understanding for evaluating large vision-language models.arXiv preprint arXiv:2408.02718, 2024

    2024 arXiv

  20. [28]

    Compositional chain-of- thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024

    2024

  21. [29]

    Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl.arXiv preprint arXiv:2503.07536, 2025. 11

    2025 arXiv

  22. [30]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  23. [31]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  24. [32]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  25. [33]

    Generative multimodal models are in-context learners

    Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages ...

    2024

  26. [34]

    Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning.arXiv preprint arXiv:2503.20752, 2025

    2025

  27. [35]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  28. [36]

    Identifying and mitigating position bias of multi-image vision-language models.arXiv preprint arXiv:2503.13792, 2025

    Xinyu Tian, Shu Zou, Zhaoyuan Yang, and Jing Zhang. Identifying and mitigating position bias of multi-image vision-language models.arXiv preprint arXiv:2503.13792, 2025

    2025 arXiv

  29. [37]

    Muirbench: A comprehensive benchmark for robust multi-image understanding.arXiv preprint arXiv:2406.09411, 2024

    Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding.arXiv preprint arXiv:2406.09411, 2024

    2024 arXiv

  30. [38]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    2025 arXiv

  31. [39]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  32. [40]

    Mv- math: Evaluating multimodal math reasoning in multi-visual contexts.arXiv preprint arXiv:2502.20808, 2025

    Peijie Wang, Zhong-Zhi Li, Fei Yin, Xin Yang, Dekang Ran, and Cheng-Lin Liu. Mv- math: Evaluating multimodal math reasoning in multi-visual contexts.arXiv preprint arXiv:2502.20808, 2025

    2025 arXiv

  33. [41]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  34. [42]

    Deepseek-vl2: Mixture-of-experts vision- language models for advanced multimodal understanding.arXiv preprint arXiv:2412.10302, 2024

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of-experts vision- language models for advanced multimodal understanding.arXiv preprint arXiv:2412.10302, 2024

    2024 arXiv

  35. [43]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  36. [44]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024. 12

    2024 arXiv

  37. [45]

    Perception-r1: Pioneering perception policy with reinforcement learning.arXiv preprint arXiv:2504.07954, 2025

    En Yu, Kangheng Lin, Liang Zhao, Jisheng Yin, Yana Wei, Yuang Peng, Haoran Wei, Jian- jian Sun, Chunrui Han, Zheng Ge, et al. Perception-r1: Pioneering perception policy with reinforcement learning.arXiv preprint arXiv:2504.07954, 2025

    2025 arXiv

  38. [46]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    2025 arXiv

  39. [47]

    Weaving context across images: Improving vision-language models through focus-centric visual chains.arXiv preprint arXiv:2504.20199, 2025

    Juntian Zhang, Yuhan Liu, Wei Liu, Jian Luan, Rui Yan, et al. Weaving context across images: Improving vision-language models through focus-centric visual chains.arXiv preprint arXiv:2504.20199, 2025

    2025 arXiv

  40. [48]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186. Springer, 2024

    2024

  41. [49]

    aha moment

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s" aha moment" in visual reasoning on a 2b non-sft model.arXiv preprint arXiv:2503.05132, 2025. 13 A Technical Appendices and Supplementary Material A.1 Training and Evaluation Detail...

    2025 arXiv

  42. [50]

    <image> <image> <image>

    A question is considered aReference-Image Comparisonif it satisfies all of the following conditions: • The question presents three or more images (e.g., "<image> <image> <image>"). • One image is clearly identified as thereference image(e.g., "image 1 is the reference"). • The...

  43. [51]

    A", "B",

    Convert the original QA pair into a multiple-choice question: • Rephrase the assistant’s response into an answer option (e.g., "A", "B", "C"). • Use placeholder tokens (<image>) in both the question and the options. • Include only the images being compared (exclude the referen...

  44. [52]

    question

    Determine the type of the question. Output Format Return a JSON object with the following structure: { "question": "<multiple-choice question in specified format>", "answer": "<correct option letter>", "question_type": "<ReferenceComparison or Other>" } Example Input:"Question...

  45. [53]

    If only the order of the images (e.g., <image_1>, <image_2>, etc.) is changed, would the answer to the question need to change?

  46. [54]

    should_change

    Is the question structured such that a single main image appears in the question body, and other images are referenced in the choices? Respond in the following JSON format: { "should_change": true or false, "is_multichoice_images": true or false } Note: This applies to both mu...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.