Pith. sign in

REVIEW 3 major objections 6 minor 113 references

Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Vision-G1 claims a 7B visual-reasoning model can beat much larger systems through data curation and multi-round RL.

desk verdict A useful RLVR dataset and pipeline wrapped in benchmark claims that are partly contaminated and one internally contradicted; the evaluation needs a careful redo, but the method is worth refereeing. read the letter →

arxiv 2508.12680 v1 pith:7IYRJTX2 submitted 2025-08-18 cs.CV cs.CL

Yuheng Zha , Kun Zhou , Yujia Wu , Yushu Wang , Jie Feng , Zhi Xu , Shibo Hao , Zhengzhong Liu
show 2 more authors
Eric P. Xing Zhiting Hu
This is my paper · ORCID
classification cs.CVcs.CL
keywords visualreasoningvision-languagemodelreinforcementlearningdatacurationinfluencefunctioncurriculumGRPOmulti-domaindataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a vision-language model can be taught to reason across many visual domains, not just math, by curating a large verifiable-answer dataset and training with reinforcement learning. It assembles 46 datasets into 8 domains, filters them with an influence-function score and a difficulty band, then runs multi-round GRPO with a data curriculum. The resulting 7B model, Vision-G1, is reported to reach state-of-the-art scores on 17 visual-reasoning benchmarks, beating similar-sized RL-trained models and, on several benchmarks, bigger or proprietary systems such as GPT-4o and Gemini-1.5 Flash. If the result holds, it would show that general visual reasoning can be elicited at small scale through data curation rather than model size alone.

What carries the argument

The load-bearing mechanism is a three-stage curation-and-training loop. First, influence-function filtering scores each training instance by gradient similarity to other instances inside and outside its domain (after LoRA fine-tuning on reject-sampled reasoning chains) and removes negative-influence samples. Second, difficulty filtering estimates each instance's rollout accuracy with k=16 samples and keeps only instances with average accuracy between 0.2 and 0.8. Third, multi-round GRPO RL rebuilds the dataset from the current checkpoint each round, re-selects moderately difficult questions, and trains on about 40k curated questions per round. The paper argues that the interaction of these three components is what lets a 7B model generalize across infographic, math, spatial, cross-image, GUI, medical, science, and common-sense reasoning.

What would settle it

Run the released training set against the evaluation benchmarks with near-duplicate detection, then retrain on a strictly deduplicated subset; if the reported gains shrink or disappear, the central claim of generalization would be falsified.

Watch

Extended reading notes

Core claim

The central claim is that a general visual-reasoning VLM can be trained by assembling 46 verifiable-answer datasets across 8 domains, filtering out unhelpful samples with an influence-function gradient similarity score, and then running multi-round GRPO reinforcement learning with difficulty-based data selection. The paper reports that the resulting 7B model, Vision-G1, achieves the best average performance among 7B baselines across the benchmarks and, on several of them, matches or exceeds much larger open models and proprietary systems. It also reports that removing either the data selection or the multi-round curriculum lowers performance, and that mixing all domains beats training on any single domain.

Load-bearing premise

The reported benchmark gains rest on the assumption that evaluation questions were not seen, in identical or near-identical form, during RL training; the paper does not report deduplication or holdout of evaluation splits.

Editorial extensions

If this is right

  • A 7B-scale VLM trained with this recipe can beat much larger open models and some proprietary systems on several visual-reasoning benchmarks, so parameter count is not the only route to strong multimodal reasoning.
  • Curating data across many domains protects against overfitting to a single domain; the paper shows that single-domain training, especially medical-only, underperforms mixed training even on that same domain.
  • Influence-function filtering and difficulty-based selection are each necessary: removing either one lowers scores on the reported ablation benchmarks.
  • Reasoning skills transfer across domains, since the model improves on logical reasoning benchmarks without being explicitly trained on logical-reasoning datasets.
  • The public release of the model, code, and training dataset enables independent reproduction and extension of the pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the data-curation recipe, not RL alone, drives most of the reported gains: the no-data-selection ablation loses roughly 4-6 points on MathVista, MathVision, and LogicVista, a larger drop than removing multi-round training.
  • We infer that strict contamination checks on the released data, comparing known training sources like ChartQA, VQA-RAD, SLAKE, and Path-VQA against the evaluation sets, would clarify how much of the SOTA gap reflects genuine generalization rather than near-duplicate exposure.
  • We infer that the pipeline is likely transferable to other backbones and modalities such as video or 3D perception, but the paper only tests one 7B backbone, so that transfer remains untested.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. Vision-G1 (arXiv:2508.12680) proposes a pipeline for training a 7B vision-language model for general visual reasoning. The authors assemble 46 datasets across 8 domains, convert them to a verifiable rule-reward format, filter them with an influence-function score (Eq. 2) and a difficulty filter that keeps instances with average rollout accuracy in [0.2, 0.8], and train Qwen2.5-VL-7B-Instruct with multi-round GRPO and a data curriculum. Results are reported on general, mathematical, and domain-specific benchmarks (Tables 1-3), with claims of state-of-the-art performance among similarly sized models and superiority over proprietary models such as GPT-4o. The ablations (Table 4) indicate that the proposed components contribute to the reported in-paper gains.

Significance. If the evaluation were clean, this would be a useful contribution: it addresses a real bottleneck—the scarcity of verifiable reward data outside mathematical reasoning—with a plausible, fairly general recipe, and the ablations suggest that the data-selection and curriculum components matter. The open release of model, code, and dataset is also a concrete strength. However, as submitted, the headline generalization claims are not supported because several evaluation benchmarks also appear as training sources, no deduplication or holdout procedure is reported, and one of the paper's own summary claims is contradicted by its own table. The contribution should be judged only after a re-run on properly held-out benchmarks.

major comments (3)
  1. [§3.1/Table 5; §4.1/Tables 1, 3] Table 5 lists ChartQA, VQA-RAD, SLAKE, and Path-VQA as training data sources, and Section 4.1 lists the same four datasets among the evaluation benchmarks reported in Tables 1 and 3. The manuscript nowhere states which official splits were used, whether the training and evaluation items were deduplicated, or whether any evaluation split was held out of the final 40k-question training set. Given this overlap, the reported gains on these benchmarks (e.g., ChartQA 90.8 vs. 88.3 for Qwen2.5-VL-7B, and the SLAKE and Path-VQA improvements in Table 3) are consistent with the model having encountered the same or near-duplicate questions during RL, rather than with generalization. In addition, Figure 5 and Section 4.1 report that MathVista-mini is used as a validation signal during training, so the MathVista row in Table 1 is not a clean held-out measurement. These issues directly undercut the abstract's state-of-the-art claim and the Section 4.2 statement that the method achieves the best performance on most benchmarks.
  2. [§4.2, Table 2] The paragraph on math-related visual reasoning states that 'our Vision-G1 achieves the best performance' and later asserts that Vision-G1 achieves higher average accuracy than Vision-R1. Table 2's Avg. column shows Vision-R1-7B at 42.5 and Vision-G1 at 42.1, so both claims are contradicted by the paper's own numbers. The text needs to be corrected or the comparison and averaging procedure need to be recomputed.
  3. [Abstract; §4.2] The abstract's claim that the model 'outperforms ... proprietary models like GPT-4o' is not supported by the reported tables as a general statement. In Table 1, GPT-4o is ahead of Vision-G1 on MMMU-Val (69.1 vs. 53.4) and MMMU-Pro (51.9 vs. 41.2), and in Table 2 GPT-4o's average is 45.7 vs. 42.1. Vision-G1 does beat GPT-4o on several specific benchmarks (e.g., ChartQA 90.8 vs. 85.7 and MathVista 76.1 vs. 63.8), so the claim should be restricted to those benchmarks rather than stated as general superiority.
minor comments (6)
  1. [§1] There is a typo in the introduction: 'there hav been some initial attempts' should be 'there have been some initial attempts'.
  2. [Appendix A.2] The text says 'we also collect four comprehensive visual reasoning datasets' and then lists five datasets: MM-R1, VerMulti, ThinkLite, ViRL39K, and MMK12. The count should be corrected.
  3. [Abstract, §4.2, §5] The number of evaluation benchmarks is inconsistent: the abstract and introduction say 17, Section 4.2 says 18, and the conclusion says 17.
  4. [References] References [15] and [68] are the same MM-Eureka paper, and references [18] and [19] both carry the title 'Vision-R1' with different author lists; one of the two citations is likely to be incorrect.
  5. [Eq. (1)] The symbol Sim in Eq. (1) is not defined; the authors should state explicitly that it denotes cosine similarity.
  6. [§4.2] The text spells 'InternVL2.5-78B' as 'InvernVL2.5-78B'; this should be corrected.

Circularity Check

2 steps flagged · score 6.0 of 10

Evaluation benchmarks overlap RL training sources (ChartQA, VQA-RAD, SLAKE, Path-VQA) and MathVista-mini is used as a training-time validation signal, making part of the SOTA evidence fitted inputs rather than held-out predictions.

  1. fitted input called prediction [Table 5 (training sources) vs Section 4.1 Evaluation Benchmarks and Tables 1/3]
    "Table 5: "Training Data ... ChartQA[52] ... VQA-RAD[60], SLAKE[62], Path-VQA[61]"; Section 4.1: "we evaluate on ... ChartQA [52] ... and medical visual reasoning with VQA-RAD [60], PathVQA [61], and SLAKE [62]"."

    ChartQA, VQA-RAD, SLAKE, and Path-VQA are listed both as RL training sources (Table 5) and as evaluation benchmarks (Section 4.1; Tables 1 and 3). The RL reward is exact answer accuracy on the training items (Section 4.1: "we use the answer accuracy as the reward for each response"), so a model trained on those items has already been optimized for the very quantity reported as benchmark accuracy. The paper reports no deduplication, split holdout, or version separation, so the gains (e.g., ChartQA 90.8 vs Qwen2.5-VL-7B 88.3) are at least partly forced by the training input rather than measuring generalization.

  2. fitted input called prediction [Section 4.3 Training Process Analysis / Figure 5 vs Table 1 MathVista]
    "Section 4.3: "we log the mean reward scores and the MathVista-Mini test scores at 10-step intervals throughout all rounds’ training process"; Figure 5: "The accuracy score on MathVista-mini over steps.""

    MathVista-mini is used as a validation signal to monitor and stop RL training (Section 4.1: "stop training once the reward score and validation set results converge"), and Figure 5 reports its accuracy over training steps. Table 1 then presents MathVista as an external benchmark result (Vision-G1: 76.1). If the reported MathVista number is the same mini split that guided validation decisions, the result is a selected-on validation score, not an independent holdout prediction; the paper does not state which split is reported, so the MathVista SOTA claim is not cleanly external as written.

full rationale

The core pipeline—influence-function filtering, difficulty-based selection, and GRPO multi-round RL—does not reduce by construction; the ablations compare against random selection and single-round training, so the method has independent content. However, the headline SOTA evidence is partially circular because four benchmarks used for evaluation are also training sources and MathVista-mini is used as a training-time validation signal. The paper reports no deduplication or holdout of evaluation splits, so the affected benchmark numbers are partly fitted inputs renamed as predictions. Self-citation is not load-bearing here; the issue is benchmark overlap and validation leakage in the empirical claims, while the remaining benchmarks provide some external check.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on three unvalidated assumptions: the rule-based verifier is a faithful reward in all eight domains, the LoRA-gradient influence function estimates true RL data value, and the evaluation benchmarks are unseen during training. The last assumption is contradicted by the paper's own data source list. The hand-set thresholds and mixing proportions are free parameters that also shape the results.

free parameters (4)
  • difficulty filter interval = [0.2, 0.8]
    Instances with average rollout accuracy outside [0.2, 0.8] are discarded each round (Section 3.2, Difficulty-based Data Filtering). The interval is hand-chosen and directly controls the training set composition.
  • domain mixing proportions = math 50%, other domains uniform
    Section 4.1: 'Math-related problems constitute half of the training dataset; the remaining domains are uniformly represented.' This hand-set mixture is not optimized or justified by experiment.
  • difficulty rollout count k = 16
    Section 4.1 sets k=16 'to balance efficiency and performance'; the choice affects difficulty estimates.
  • final training set size = 40k
    After influence filtering the final training set contains 40k questions (Section 4.1); the threshold determining this size is not specified.
assumptions (4)
  • domain assumption The rule-based verifier (normalized exact match + math-verify) provides a valid reward for all eight domains.
    Section 3.1 retains only instances with verifiable ground truth, and Section 4.1 uses exact match and math-verify. For medical, spatial, and common-sense items, exact string matching may be an incomplete or noisy correctness signal; the paper does not validate reward quality per domain.
  • domain assumption Influence functions computed on LoRA gradients of a reject-sampled subset approximate the data influence for the full RL training.
    Section 3.2: 'we fine-tune a LoRA module on high-quality reasoning chains, obtained via reject sampling from the base VLM on a subset... For estimating the instance influence... we sample the rollouts from the base VLM and compute the gradients on LoRA parameters.' No theoretical or empirical justification is given that this gradient similarity predicts RL value.
  • domain assumption Evaluation benchmarks are disjoint from training data.
    Needed for the SOTA claims. Table 5 lists ChartQA, VQA-RAD, SLAKE, and Path-VQA as training sources, while Section 4.1 lists them as evaluation benchmarks. No deduplication step is reported.
  • standard math GRPO and the standard KL penalty are applicable to VLM post-training as in prior work.
    Equation (3) follows DeepSeekMath [35]; the paper uses it without modification, relying on prior verification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation." pith.science (2026). https://pith.science/paper/7IYRJTX2

@misc{pith2026250812680,
  author       = {Pith},
  title        = {Pith review of: Vision-G1: Towards General Vision Language Reasoning with Multi-Domain Data Curation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7IYRJTX2}},
  note         = {Machine review of arXiv:2508.12680}
}
read the original abstract

Despite their success, current training pipelines for reasoning VLMs focus on a limited range of tasks, such as mathematical and logical reasoning. As a result, these models face difficulties in generalizing their reasoning capabilities to a wide range of domains, primarily due to the scarcity of readily available and verifiable reward data beyond these narrowly defined areas. Moreover, integrating data from multiple domains is challenging, as the compatibility between domain-specific datasets remains uncertain. To address these limitations, we build a comprehensive RL-ready visual reasoning dataset from 46 data sources across 8 dimensions, covering a wide range of tasks such as infographic, mathematical, spatial, cross-image, graphic user interface, medical, common sense and general science. We propose an influence function based data selection and difficulty based filtering strategy to identify high-quality training samples from this dataset. Subsequently, we train the VLM, referred to as Vision-G1, using multi-round RL with a data curriculum to iteratively improve its visual reasoning capabilities. Our model achieves state-of-the-art performance across various visual reasoning benchmarks, outperforming similar-sized VLMs and even proprietary models like GPT-4o and Gemini-1.5 Flash. The model, code and dataset are publicly available at https://github.com/yuh-zha/Vision-G1.

Figures

Figures reproduced from arXiv: 2508.12680 by the authors.

Figure 1
Figure 1. Radar chart showing the performance of our pro [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of our approach, consisting of collecting and preprocessing a mixture of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Source dataset distribution of our Vision [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Heatmap illustrating the contribution of each data domain. Darker [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The progress in the multi-round RL training. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

113 extracted references · 17 canonical work pages

  1. [1]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  2. [2]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    arXiv

  3. [3]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021

    2021

  4. [4]

    SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024

    2024

  5. [5]

    Dapo: An open-source llm reinforcement learning system at scale, 2025

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, W...

    2025

  6. [6]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model

    Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model. arXiv preprint arXiv:2503.24290, 2025

    arXiv 2025

  7. [7]

    Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks

    Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arXiv:2504.05118, 2025

    arXiv 2025

  8. [8]

    Revisiting reinforcement learning for llm reasoning from a cross-domain perspective

    Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nilabjo Dey, Yuheng Zha, et al. Revisiting reinforcement learning for llm reasoning from a cross-domain perspective. arXiv preprint arXiv:2506.14965, 2025

    arXiv 2025

Show all 113 references
  1. [9]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025

    2025 arXiv

  2. [10]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  3. [11]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  4. [12]

    Visual-rft: Visual reinforcement fine-tuning

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  5. [13]

    Vlm-r1: A stable and generalizable r1-style large vision-language model, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model, 2025

    2025

  6. [14]

    Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl, 2025

    Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-r1: Empowering 3b lmms with strong reasoning abilities through two-stage rule-based rl, 2025

    2025

  7. [16]

    Virgo: A preliminary exploration on reproducing o1-like mllm

    Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o1-like mllm. arXiv preprint arXiv:2501.01904, 2025

    2025 arXiv

  8. [17]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  9. [18]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision- guided reinforcement learning

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision- guided reinforcement learning. arXiv preprint arXiv:2503.18013, 2025

    2025 arXiv

  10. [19]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  11. [20]

    Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934, 2025

    2025 arXiv

  12. [21]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    2025 arXiv

  13. [22]

    Kimi-vl technical report

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504.07491, 2025

    2025 arXiv

  14. [23]

    Improve vision language model chain-of-thought reasoning, 2024

    Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning, 2024

    2024

  15. [24]

    Llava-cot: Let vision language models reason step-by-step, 2025

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025

    2025

  16. [25]

    Spatialvlm: Endowing vision-language models with spatial reasoning capabilities

    Boyuan Chen, Zhuo Xu, Sean Kirmani, Brain Ichter, Dorsa Sadigh, Leonidas Guibas, and Fei Xia. Spatialvlm: Endowing vision-language models with spatial reasoning capabilities. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14455–14465, 2024

    2024

  17. [26]

    Mixed-r1: Unified reward perspective for reasoning capability in multimodal large language models

    Shilin Xu, Yanwei Li, Rui Yang, Tao Zhang, Yueyi Sun, Wei Chow, Linfeng Li, Hang Song, Qi Xu, Yunhai Tong, et al. Mixed-r1: Unified reward perspective for reasoning capability in multimodal large language models. arXiv preprint arXiv:2505.24164, 2025

    2025 arXiv

  18. [27]

    Modomodo: Multi-domain data mixtures for multimodal llm reinforcement learning

    Yiqing Liang, Jielin Qiu, Wenhao Ding, Zuxin Liu, James Tompkin, Mengdi Xu, Mengzhou Xia, Zhengzhong Tu, Laixi Shi, and Jiacheng Zhu. Modomodo: Multi-domain data mixtures for multimodal llm reinforcement learning. arXiv preprint arXiv:2505.24871, 2025

    2025 arXiv

  19. [28]

    Less: Selecting influential data for targeted instruction tuning

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Selecting influential data for targeted instruction tuning. arXiv preprint arXiv:2402.04333, 2024

    2024 arXiv

  20. [29]

    Estimating training data influence by tracking gradient descent

    Garima Pruthi, Frederick Liu, Mukund Sundararajan, and Satyen Kale. Estimating training data influence by tracking gradient descent. ArXiv, abs/2002.08484, 2020

    2002 arXiv

  21. [30]

    GPT-4o System Card

    OpenAI. GPT-4o System Card. https://openai.com/index/gpt-4o-system-card/ , 2024

    2024

  22. [31]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  23. [32]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479, 2025. 12

    2025 arXiv

  24. [33]

    Qvq: To see the world with wisdom, December 2024

    Qwen Team. Qvq: To see the world with wisdom, December 2024

    2024

  25. [34]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  26. [35]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  27. [36]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025

  28. [37]

    R1-v: Reinforcing super gen- eralization ability in vision-language models with less than $3

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super gen- eralization ability in vision-language models with less than $3. https://github.com/ Deep-Agent/R1-V, 2025. Accessed: 2025-02-02

    2025

  29. [38]

    Sft or rl? an early investigation into training r1-like reasoning large vision- language models

    Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision- language models. arXiv preprint arXiv:2504.11468, 2025

    2025 arXiv

  30. [39]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023

    2023

  31. [40]

    Lima: Less is more for alignment

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024

    2024

  32. [41]

    Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning

    Hao Chen, Yiming Zhang, Qi Zhang, Hantao Yang, Xiaomeng Hu, Xuetao Ma, Yifan Yang- gong, and Junbo Zhao. Maybe only 0.5% data is needed: A preliminary exploration of low training data instruction tuning. arXiv preprint arXiv:2305.09246, 2023

    2023 arXiv

  33. [42]

    Llm-assisted code cleaning for training accurate code generators

    Naman Jain, Tianjun Zhang, Wei-Lin Chiang, Joseph E Gonzalez, Koushik Sen, and Ion Stoica. Llm-assisted code cleaning for training accurate code generators. arXiv preprint arXiv:2311.14904, 2023. 13

    2023 arXiv

  34. [43]

    What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning

    Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312.15685, 2023

    2023 arXiv

  35. [44]

    Astraios: Parameter-efficient instruction tuning code large language models

    Terry Yue Zhuo, Armel Zebaze, Nitchakarn Suppattarachai, Leandro von Werra, Harm de Vries, Qian Liu, and Niklas Muennighoff. Astraios: Parameter-efficient instruction tuning code large language models. arXiv preprint arXiv:2401.00788, 2024

    2024 arXiv

  36. [45]

    Octopack: Instruc- tion tuning code large language models

    Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro V on Werra, and Shayne Longpre. Octopack: Instruc- tion tuning code large language models. arXiv preprint arXiv:2308.07124, 2023

    2023 arXiv

  37. [46]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  38. [47]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representat...

    2024

  39. [48]

    Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multi- modal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on ...

    2024

  40. [49]

    Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark

    Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang, Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang, Huan Sun, et al. Mmmu-pro: A more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813, 2024

    2024 arXiv

  41. [50]

    Are we on the right way for evaluating large vision- language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision- language models? arXiv preprint arXiv:2403.20330, 2024

    2024 arXiv

  42. [51]

    Logicvista: Multimodal llm logical reasoning benchmark in visual contexts

    Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logicvista: Multimodal llm logical reasoning benchmark in visual contexts. arXiv preprint arXiv:2407.04973, 2024

    2024 arXiv

  43. [52]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2263–2279, 2022

    2022

  44. [53]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

    2024

  45. [54]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Computer Vision, pages 169–186. Sprin...

    2024

  46. [55]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv:2402....

    2024 arXiv

  47. [56]

    We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284, 2024

    2024 arXiv

  48. [57]

    Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models

    Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. arXiv preprint arXiv:2411.00836, 2024. 14

    2024 arXiv

  49. [58]

    Charxiv: Charting gaps in realistic chart understanding in multimodal llms

    Zirui Wang, Mengzhou Xia, Luxi He, Howard Chen, Yitao Liu, Richard Zhu, Kaiqu Liang, Xindi Wu, Haotian Liu, Sadhika Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. Advances in Neural Information Processing Systems , 37:113569–113697, 2024

    2024

  50. [59]

    Chartqapro: A more diverse and challenging benchmark for chart question answering

    Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmo- hammadi, et al. Chartqapro: A more diverse and challenging benchmark for chart question answering. arXiv prepri...

    2025 arXiv

  51. [60]

    A dataset of clinically generated visual questions and answers about radiology images

    Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images. Scientific data, 5(1):1–10, 2018

    2018

  52. [61]

    Pathvqa: 30000+ questions for medical visual question answering

    Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286, 2020

    2003 arXiv

  53. [62]

    Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering

    Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650–1654. IEEE, 2021

    2021

  54. [63]

    Muirbench: A comprehensive benchmark for robust multi-image understanding

    Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. Muirbench: A comprehensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.09411, 2024

    2024 arXiv

  55. [64]

    Ovis: Structural embedding alignment for multimodal large language model

    Shiyin Lu, Yang Li, Qing-Guo Chen, Zhao Xu, Weihua Luo, Kaifu Zhang, and Han-Jia Ye. Ovis: Structural embedding alignment for multimodal large language model. arXiv preprint arXiv:2405.20797, 2024

    2024 arXiv

  56. [65]

    Minicpm-v: A gpt-4v level mllm on your phone

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  57. [66]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024

    2024 arXiv

  58. [67]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    2024

  59. [68]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  60. [69]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  61. [70]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  62. [71]

    Figureqa: An annotated figure dataset for visual reasoning

    Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning. arXiv preprint arXiv:1710.07300, 2017

    2017 arXiv

  63. [72]

    Dvqa: Understanding data visualizations via question answering

    Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visualizations via question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5648–5656, 2018

    2018

  64. [73]

    Plotqa: Reasoning over scientific plots

    Nitesh Methani, Pritha Ganguly, Mitesh M Khapra, and Pratyush Kumar. Plotqa: Reasoning over scientific plots. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1527–1536, 2020. 15

    2020

  65. [74]

    Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning

    Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. arXiv preprint arXiv:2209.14610, 2022

    2022 arXiv

  66. [75]

    Mapqa: A dataset for question answering on choropleth maps

    Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. Mapqa: A dataset for question answering on choropleth maps. arXiv preprint arXiv:2211.08545, 2022

    2022 arXiv

  67. [76]

    Chartbench: A benchmark for complex visual reasoning in charts

    Zhengzhuo Xu, Sinan Du, Yiyan Qi, Chengjin Xu, Chun Yuan, and Jian Guo. Chartbench: A benchmark for complex visual reasoning in charts. arXiv preprint arXiv:2312.15915, 2023

    2023 arXiv

  68. [77]

    Unichart: A universal vision-language pretrained model for chart comprehension and reasoning

    Ahmed Masry, Parsa Kavehzadeh, Xuan Long Do, Enamul Hoque, and Shafiq Joty. Unichart: A universal vision-language pretrained model for chart comprehension and reasoning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14662–14684, 2023

    2023

  69. [78]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021

    2021

  70. [79]

    Harnessing webpage uis for text-rich visual understanding

    Junpeng Liu, Tianyue Ou, Yifan Song, Yuxiao Qu, Wai Lam, Chenyan Xiong, Wenhu Chen, Graham Neubig, and Xiang Yue. Harnessing webpage uis for text-rich visual understanding. arXiv preprint arXiv:2410.13824, 2024

    2024 arXiv

  71. [80]

    Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning

    Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2105.04165, 2021

    2021 arXiv

  72. [81]

    An augmented benchmark dataset for geometric question answering through dual parallel text encoding

    Jie Cao and Jing Xiao. An augmented benchmark dataset for geometric question answering through dual parallel text encoding. In Proceedings of the 29th international conference on computational linguistics, pages 1511–1520, 2022

    2022

  73. [82]

    Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression

    Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression. arXiv preprint arXiv:2212.02746, 2022

    2022 arXiv

  74. [83]

    Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. arXiv preprint arXiv:2105.14517, 2021

    2021 arXiv

  75. [84]

    Solving geometry problems: Combining text and diagram interpretation

    Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geometry problems: Combining text and diagram interpretation. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1466–1476, 2015

    2015

  76. [85]

    Clevr-math: A dataset for com- positional language, visual and mathematical reasoning

    Adam Dahlgren Lindström and Savitha Sam Abraham. Clevr-math: A dataset for com- positional language, visual and mathematical reasoning. arXiv preprint arXiv:2208.05358, 2022

    2022 arXiv

  77. [86]

    Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning

    Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. arXiv preprint arXiv:2110.13214, 2021

    arXiv

  78. [87]

    A corpus for reasoning about natural language grounded in photographs

    Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018

    2018 arXiv

  79. [88]

    Image retrieval from contextual descriptions

    Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, and Siva Reddy. Image retrieval from contextual descriptions. arXiv preprint arXiv:2203.15867, 2022

    2022 arXiv

  80. [89]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. VQA: Visual Question Answering. In International Conference on Computer Vision (ICCV), 2015. 16

    2015

  81. [90]

    Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning

    Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan L Yuille. Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogn...

    2023

  82. [91]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 , pages 235–251....

    2016

  83. [92]

    Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension

    Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern rec...

    2017

  84. [93]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507–2521, 2022

    2022

  85. [94]

    Vizwiz grand challenge: Answering visual questions from blind people

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3608–3...

    2018

  86. [95]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326, 2019

    2019

  87. [96]

    A-okvqa: A benchmark for visual question answering using world knowledge

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision, pages 146–162. Springer, 2022

    2022

  88. [97]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195–3204, 2019

    2019

  89. [98]

    Pmc-vqa: Visual instruction tuning for medical visual question answering

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415, 2023

    2023 arXiv

  90. [99]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s pe...

    2024 arXiv

  91. [100]

    Chart-r1: Chain-of-thought supervision and reinforcement for advanced chart reasoner

    Lei Chen, Xuanle Zhao, Zhixiong Zeng, Jing Huang, Yufeng Zhong, and Lin Ma. Chart-r1: Chain-of-thought supervision and reinforcement for advanced chart reasoner. arXiv preprint arXiv:2507.15509, 2025

    2025

  92. [101]

    You are a QUESTION-TYPE classifier (do **NOT** answer the question itself)

    Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang, Jiaqi Wang, et al. Vlmevalkit: An open-source toolkit for evaluating large multi-modality models. In Proceedings of the 32nd ACM International Conference on Multimedia...

    2024

  93. [102]

    Identify the values for each year: - 2002: 19.4% - 2003: 23.2% - 2004: 27.8%

    2002

  94. [103]

    Add these values together: 19.4 + 23.2 + 27.8

  95. [104]

    First, add 19.4 and 23.2: 19.4 + 23.2 = 42.6

  96. [105]

    D Limitations In this paper, we train a general reasoning VLM, namely Vision-G1, and test its effectiveness on a variety of benchmarks

    Next, add the result to 27.8: 42.6 + 27.8 = 70.4 So, the sum of 2002, 2003, and 2004 is 70.4 . D Limitations In this paper, we train a general reasoning VLM, namely Vision-G1, and test its effectiveness on a variety of benchmarks. Despite it, here are few limitations about thi...

    2002

  97. [106]

    Count the baseballs in one bucket: - There are 6 baseballs in one bucket

  98. [107]

    Count the number of buckets: - There are 4 buckets in total

  99. [108]

    24 Vision-G1 Let’s count the baseballs in each bucket:

    Multiply the number of baseballs in one bucket by the number of buckets: - 6 baseballs/bucket * 4 buckets = 24 baseballs Therefore, the total number of baseballs is 24. 24 Vision-G1 Let’s count the baseballs in each bucket:

  100. [109]

    The first bucket has 5 baseballs. 2. The second bucket has 5 baseballs. 3. The third bucket has 5 baseballs. 4. The fourth bucket has 5 baseballs. Now, let’s add them up: 5 + 5 + 5 + 5 = 20 Let’s count: - Bucket 1: 5 - Bucket 2: 5 - Bucket 3: 5 - Bucket 4: 5 20 general, it sho...

  101. [110]

    - We need to form two two-digit numbers

    **Identify the digits and their possible placements:** - We have the digits 2, 3, 4, and 5. - We need to form two two-digit numbers. - The digits must be used exactly once

  102. [111]

    - The largest digit is 5, so we place 5 in the tens place of the first number

    **Maximize the first number:** - To maximize the first number, we should place the largest digits in the tens place. - The largest digit is 5, so we place 5 in the tens place of the first number. - The next largest digit is 4, so we place 4 in the tens place of the second numb...

  103. [112]

    - The next largest digit is 3, so we place 3 in the tens place of the second number

    **Maximize the second number:** - To maximize the second number, we should place the next largest digits in the tens place. - The next largest digit is 3, so we place 3 in the tens place of the second number. - The remaining digit is 2, so we place 2 in the tens place of the f...

  104. [113]

    - The second number is 34

    **Form the numbers:** - The first number is 52. - The second number is 34

  105. [114]

    Therefore, the biggest number that they could make is 86

    **Calculate the sum:** - Sum = 52 + 34 = 86. Therefore, the biggest number that they could make is 86 . Vision-G1 To determine the largest possible sum, we need to place the digits 2, 3, 4, and 5 in the squares such that the two numbers are maximized. Let’s denote the two numb...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.