Pith. sign in

REVIEW 5 major objections 5 minor 1 cited by

Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A single reinforcement-learning reward design, matched to each answer format, improves Qwen2.5-VL and InternVL models on four visual-reasoning benchmarks by 2-5 percentage points.

desk verdict Genuinely useful mixed-reward RL post-training paper, but the missing deduplication check against evaluation suites keeps me from fully trusting the headline gains. read the letter →

arxiv 2505.24164 v1 pith:VF4BZCXH submitted 2025-05-30 cs.CL cs.CV

Shilin Xu , Yanwei Li , Rui Yang , Tao Zhang , Yueyi Sun , Wei Chow , Linfeng Li , Hang Song
show 4 more authors
Qi Xu Yunhai Tong Xiangtai Li Hao Fei
This is my paper
classification cs.CLcs.CV
keywords multimodallargelanguagemodelsreinforcementlearningGRPOrewarddesignvisualreasoningBMASopen-endedpost-training
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Mixed-R1 claims that a single reinforcement-learning post-training recipe can improve a multimodal large language model across several different visual reasoning tasks at once, rather than requiring one training run for math, another for charts, another for grounding, and another for captions. The authors assemble a 45K-sample training set from existing open datasets and filter it with a simple rule: generate eight responses per question with Qwen2.5-VL-7B, and drop any question whose eight responses all receive the same reward, on the grounds that such questions are too easy or too hard for GRPO to learn from. They then train with four reward functions matched to the answer format, including a newly proposed open-ended reward called BMAS that uses the model's own tokenizer embeddings instead of an external judge model. The reported result is a 2-5 percentage point average improvement on MathVista, MathVision, MMMU, and MMStar for Qwen2.5-VL-3B, Qwen2.5-VL-7B, and InternVL2.5-4B.

What carries the argument

The load-bearing object is the mixed reward design running under Group Relative Policy Optimization (GRPO), a rule-based RL method that computes advantages from rewards within a group of sampled responses: four content rewards selected by data type, plus a format reward that enforces <think> and <answer> tags. For open-ended answers, BMAS is the central new mechanism: it computes pairwise cosine similarity between the policy model's input-token embeddings of the generated text and the ground-truth text, then takes the average of the row-wise maxima and column-wise maxima of that similarity matrix. This gives a reward that can tell long-form captions apart without an external judge LLM, and the maximum-based aggregation is what the paper credits for stable training.

What would settle it

Train the same GRPO and Mixed-Reward setup on a random 45K sample drawn from the same unfiltered source pools; if the random sample matches Mixed-45K's gains on MathVista, MathVision, MMMU, and MMStar, then the all-same-reward filtering rule is not doing the work the paper assigns to it.

Watch

Extended reading notes

Core claim

The paper's central claim is that the bottleneck for R1-style reinforcement learning in multimodal models is reward design, not the RL algorithm: each task family needs a reward matched to its output format, and open-ended text needs a stable dense reward that does not require a second model. Mixed-R1 sorts data into yes/no, multiple-choice, chart, grounding, and open-ended categories; assigns matching, chart, IoU, and BMAS rewards; and combines them with a format reward inside GRPO. The distinctive piece is BMAS, which embeds the generated response and the ground truth with the policy model's own tokenizer embeddings, computes pairwise cosine similarities, and scores the response by averaging the maximum similarity over each row and each column of the similarity matrix. In the paper's experiments this design improves Qwen2.5-VL-3B, Qwen2.5-VL-7B, and InternVL2.5-4B on MathVista, MathVision, MMMU, and MMStar, with average gains of 2-5 percentage points over the base models.

Load-bearing premise

The filter's rule that a question is worth keeping only if its eight sampled responses earn different rewards is the load-bearing premise; if that heuristic selects examples for training stability rather than for generalization, the Mixed-45K dataset may not be the cause of the reported gains.

Editorial extensions

If this is right

  • One GRPO run with mixed rewards can lift several distinct capabilities at once, so task-specific post-training pipelines are not required for the five data types covered.
  • Open-ended caption and VQA data can be rewarded from the model's own tokenizer embeddings, removing the extra model cost of LLM-as-judge rewards.
  • The ablation that adds rewards in easy-to-hard order shows each added data family contributes, with open-ended data delivering the final and largest jump.
  • Dataset scale has a sweet spot: 20K underperforms, 90K improves MathVision and MMStar but hurts MathVista, so 45K is the reported best balance of gain and cost.
  • Format rewards for <think>/<answer> can be combined with content rewards at weight 0.5 without collapsing accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the reward-format taxonomy is probably portable to video and multi-image inputs, since the four reward types depend on output form (choice, number, box, text) rather than on the visual modality; the paper only tests still images.
  • Inference: BMAS may owe its stability to using the policy model's own embedding space, so a testable prediction is that freezing the reward embeddings from an earlier checkpoint or swapping in a different tokenizer will reduce its advantage.
  • Inference: the all-same-reward filter could be selecting for examples where the GRPO advantage is nonzero rather than for examples that generalize; a direct check is comparing Mixed-45K against a random 45K sample from the same sources.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Mixed-R1, an RL post-training framework for multimodal LLMs that combines four task-specific reward functions (matching, chart, IoU, and a new tokenizer-based open-ended reward BMAS) with a filtered mixed dataset Mixed-45K. The authors apply GRPO to Qwen2.5-VL-3B/7B and InternVL2.5-4B and report average improvements of 2–5% on MathVista, MathVision, MMMU, MMStar, MMBench, and AI2D. The paper also ablates reward components, data scale, and open-ended reward designs, and releases code, data, and model weights.

Significance. If the reported gains are robust and reflect generalization rather than benchmark overlap, the paper would make a useful contribution: it is among the first to combine heterogeneous MLLM tasks in a single GRPO run, and its BMAS reward avoids the cost of an external judge model. The dataset and code release would facilitate reproducibility. However, the current evidence is weakened by the absence of statistical error bars, a possible training/evaluation overlap, and an inverted format-reward equation, so the significance hinges on these being resolved.

major comments (5)
  1. [§3.1, Table 1] Mixed-45K is built from MapQA, CLEVR, FigureQA, GeoQA, UniGeo, ScienceQA, ChartQA, DocVQA, InfographicVQA, RefCOCO, and ALLaVA, but the paper never reports a deduplication or overlap analysis against the evaluation suites (MathVista, MathVision, MMMU, MMStar). Because MathVista in particular was constructed in part from existing chart/geometry/QA datasets (including ChartQA and FigureQA), and the §3.1 filter deliberately keeps examples on which the base model has partial success, even a small overlap could let GRPO memorize benchmark items and inflate the 2–5% average gains. The authors should quantify exact-duplicate and image-overlap rates between Mixed-45K and each evaluation benchmark, and either exclude overlapping items or report results on a cleaned subset.
  2. [Tables 1–4] All results are single runs with no error bars, multiple seeds, or significance tests. The headline improvements are 2–3% on the average, which is within the typical run-to-run variability of GRPO on small datasets; e.g., Table 3(a) shows a 3.4-point swing on MathVista between the 45K and 90K conditions. The authors should provide at least three seeds for the main comparisons (or otherwise quantify variance) before the claimed improvements can be taken as reliable.
  3. [Equation (5)] Equation (5) defines r_format = 0 if output matches format and 1 if it doesn't, which is the opposite of the surrounding prose ('assigning a reward value of 1 if the response conforms to the specified format and 0 otherwise'). This is a direct contradiction in a load-bearing definition, since the format reward is applied to every training example. The authors must state the correct definition and clarify which variant was used in the experiments.
  4. [§3.2, BMAS Reward] The BMAS reward is computed using 'the input embedding space of the policy model,' but the paper does not state whether those embeddings are detached or frozen during GRPO. Since the policy's embedding weights are updated during training, the reward signal would change as training progresses, making it non-stationary and potentially explaining why alternative embedding-based rewards (ModernBert, Sentence-BERT) underperform. Please specify how the embeddings are obtained (frozen vs. trainable, detached vs. attached) and justify the stability of the reward.
  5. [§3.1] The data filter drops an example when all g=8 model samples receive identical rewards under the exact same reward functions that later define the GRPO training signal. This conflates 'useful for training' with 'easy to score under the chosen rewards,' and the filter's preference for partial-success examples may systematically select for base-model-friendly benchmark-like items. The paper should ablate the filter (e.g., train on unfiltered 45K or a random subset) and report whether the gains in Table 1 persist.
minor comments (5)
  1. [§3.1] The text contains a typo: 'GPRO' should be 'GRPO' (it appears twice, once near the description of the data filter and once in §3.2).
  2. [Equation (2)] The standard deviation is written as std({ri, r2, ...}); the first index should be r1 (or the notation should be made consistent) to avoid confusion with the group index i.
  3. [Reference [8]] Reference [8] is incomplete: the title ends with 'with less than $3' and the intended model size is missing.
  4. [Figure 4] Figure 4's caption does not define what the plotted curves represent, and the x-axis label 'Steps' is ambiguous; the comparison would be easier to interpret if the training reward and completion length were shown on shared axes with clear legends.
  5. [Table 4] The training setup for Table 4 is not described; in particular, it is unclear whether the comparison trains only on the open-ended portion of Mixed-45K, which would explain the lower absolute numbers relative to Table 2. Please state the training data and hyperparameters used for this ablation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in Mixed-R1: benchmark gains are externally anchored, with only mild self-referential data curation that does not force any reported result.

full rationale

The paper's central claim—that Mixed-R1's mixed reward design and Mixed-45K post-training improve Qwen2.5-VL/InternVL on MathVista, MathVision, MMMU, and MMStar—is supported by evaluations on benchmarks that are external to the training objective. Each reward function (matching, chart, IoU, BMAS) is computed against ground-truth labels, numeric values, bounding boxes, or reference text, and GRPO optimizes those ground-truth-based rewards rather than the benchmark scores themselves. The Sec. 3.1 data filter is self-referential in a weak sense because it uses the same reward functions to drop examples where all sampled rewards are identical, but this is a data-curation heuristic for avoiding vanishing advantages, not a fitted parameter renamed as a prediction; it does not by construction determine any particular benchmark outcome. BMAS also uses the policy model's own tokenizer embeddings, but the ground-truth response provides an external anchor, and the ablation in Table 4 shows that BMAS performs only comparably to alternative open-ended rewards, indicating it is not a vacuous or defining signal. The authors' self-citations appear only in related-work and methodological framing and are not load-bearing for the empirical result. No equation in the paper reduces to its own input by construction. The largest caveat is the absence of a reported deduplication analysis between Mixed-45K source datasets and the evaluation suites, which is a contamination risk rather than a logical circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central result is empirical, resting on GRPO and on hand-chosen reward design decisions (lambda, chart tolerance, data filter g). No new physical or mathematical entities are introduced.

free parameters (3)
  • lambda (format reward weight) = 0.5
    Hand-chosen hyperparameter that balances the format reward against content rewards in the final reward; not ablated.
  • chart reward tolerance = 1e-2
    Hand-chosen threshold for treating a numeric chart answer as correct; affects the chart reward signal.
  • number of sampled responses g in data filter = 8
    Hand-chosen; the data filter's decision to drop or keep an example depends on the rewards of these 8 samples.
assumptions (5)
  • domain assumption GRPO objective and advantage formula from DeepSeekMath/DeepSeek-R1 are valid for policy optimization.
    Sec 3.2 builds on the GRPO framework without re-deriving or validating its assumptions.
  • domain assumption The rule-based reward functions (matching, chart, IoU, BMAS) provide correct learning signals for their respective tasks.
    Sec 3.2 assumes these rewards accurately reflect answer quality; no calibration or analysis of reward noise is provided.
  • domain assumption The evaluation benchmarks (MathVista, MathVision, etc.) measure general reasoning capability in multimodal LLMs.
    Sec 4 uses lmms-eval scores as the success metric without questioning benchmark validity.
  • domain assumption The policy model's token embeddings are a semantically meaningful space for computing similarity.
    Sec 3.2 BMAS uses the model's own input embeddings; no independent validation that cosine similarity in this space reflects semantic or factual correctness.
  • domain assumption All collected data conforms to the expected training format before filtering.
    Sec 3.1 states 'we assume that all data conforms to the training format', which is a precondition for the data engine.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/VF4BZCXH

@misc{pith2026250524164,
  author       = {Pith},
  title        = {Pith review of: Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VF4BZCXH}},
  note         = {Machine review of arXiv:2505.24164}
}
read the original abstract

Recent works on large language models (LLMs) have successfully demonstrated the emergence of reasoning capabilities via reinforcement learning (RL). Although recent efforts leverage group relative policy optimization (GRPO) for MLLMs post-training, they constantly explore one specific aspect, such as grounding tasks, math problems, or chart analysis. There are no works that can leverage multi-source MLLM tasks for stable reinforcement learning. In this work, we present a unified perspective to solve this problem. We present Mixed-R1, a unified yet straightforward framework that contains a mixed reward function design (Mixed-Reward) and a mixed post-training dataset (Mixed-45K). We first design a data engine to select high-quality examples to build the Mixed-45K post-training dataset. Then, we present a Mixed-Reward design, which contains various reward functions for various MLLM tasks. In particular, it has four different reward functions: matching reward for binary answer or multiple-choice problems, chart reward for chart-aware datasets, IoU reward for grounding problems, and open-ended reward for long-form text responses such as caption datasets. To handle the various long-form text content, we propose a new open-ended reward named Bidirectional Max-Average Similarity (BMAS) by leveraging tokenizer embedding matching between the generated response and the ground truth. Extensive experiments show the effectiveness of our proposed method on various MLLMs, including Qwen2.5-VL and Intern-VL on various sizes. Our dataset and model are available at https://github.com/xushilin1/mixed-r1.

Figures

Figures reproduced from arXiv: 2505.24164 by the authors.

Figure 1
Figure 1. Results of Our Mixed-R1. Our post-training method improves Qwen2.5-VL on four different challenging benchmarks by large margins, on both the 3B model and the 7B model. aim to solve the visual grounding [44] tasks by introducing an explicit IoU reward or mAP (mean average precision) reward, where the distance serves as a reward signal to optimize the model’s localization ability. However, these works mainly focus on … view at source ↗
Figure 2
Figure 2. The distribution of the training dataset (a) and dataset filter pipeline (b). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Mixed-45K data types and reward examples used in our Mixed-45K. JGRP O(θ) = E[q ∼ P(Q), {oi} G i=1 ∼ πθold (O|q)] 1 G X G i=1  min  πθ(oi |q) πθold (oi |q) Ai , clip  πθ(oi |q) πθold (oi |q) , 1 − ε, 1 + ε  Ai  − βDKL (πθ||πref )  , (1) where ε and β are hyper-parameters, and Ai represents the advantage of the candidate response oi relative to other candidate responses. And it is computed using a group of rewa… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of different Open-Ended reward designs on Qwen2.5-VL-3B. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison examples of our Mixed-R1 compared with Qwen2.5-VL-7B-Instruct. We [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding

    cs.CV 2026-01 conditional novelty 6.0 of 10

    A structured five-step reasoning template plus diverse-trajectory cold start and diversity-preserving two-stage RL lifts a 7B multimodal model to state-of-the-art multi-image reasoning on several benchmarks.

Reference graph

Works this paper leans on

79 extracted references · 28 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 3

    arXiv 2023

  2. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenhang Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, K. Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, ...

    arXiv 2023

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025. 1, 2, 3, 4, 7, 9

    arXiv 2025

  4. [4]

    Mapqa: A dataset for question answering on choropleth maps.arXiv preprint arXiv:2211.08545, 2022

    Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. Mapqa: A dataset for question answering on choropleth maps.arXiv preprint arXiv:2211.08545, 2022. 4

    arXiv 2022

  5. [5]

    Allava: Harnessing gpt4v-synthesized data for lite vision-language models.arXiv preprint arXiv:2402.11684, 2024

    Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen, Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing gpt4v-synthesized data for lite vision-language models.arXiv preprint arXiv:2402.11684, 2024. 4

    arXiv 2024

  6. [6]

    Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression.arXiv preprint arXiv:2212.02746,

    Jiaqi Chen, Tong Li, Jinghui Qin, Pan Lu, Liang Lin, Chongyu Chen, and Xiaodan Liang. Unigeo: Unifying geometry logical reasoning via reformulating mathematical expression.arXiv preprint arXiv:2212.02746,

    arXiv

  7. [7]

    Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning.arXiv preprint arXiv:2105.14517, 2021

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning.arXiv preprint arXiv:2105.14517, 2021. 4

    arXiv 2021

  8. [8]

    R1-v: Reinforcing super generalization ability in vision-language models with less than $3.https://github.com/Deep-Agent/R1-V, 2025

    Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3.https://github.com/Deep-Agent/R1-V, 2025. 1, 3

    work page 2025

Show all 79 references
  1. [9]

    Are we on the right way for evaluating large vision-language models?arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models?arXiv preprint arXiv:2403.20330, 2024. 1, 7, 9

    2024 arXiv

  2. [10]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024. 1, 2, 3

    2024 arXiv

  3. [11]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, 2023. 3

    2023

  4. [12]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. 7

    2025

  5. [13]

    Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution

    Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim M Alabdulmohsin, et al. Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution. InNeurIPS, 2023. 3

    2023

  6. [14]

    Bert: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirec- tional transformers for language understanding. InNAACL, 2019. 6, 8

    2019

  7. [15]

    On path to multimodal generalist: General-level and general-bench

    Hao Fei, Yuan Zhou, Juncheng Li, Xiangtai Li, Qingshan Xu, Bobo Li, Shengqiong Wu, Yaoting Wang, Junbao Zhou, Jiahao Meng, et al. On path to multimodal generalist: General-level and general-bench. arXiv preprint arXiv:2505.04620, 2025. 1

    2025 arXiv

  8. [16]

    Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis.arXiv preprint arXiv:2405.21075, 2024. 1

    2024 arXiv

  9. [17]

    Vita-1.5: Towards gpt-4o level real-time vision and speech interaction

    Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction. arXiv preprint arXiv:2501.01957, 2025. 1 10

    2025 arXiv

  10. [18]

    Cantor: Inspiring multimodal chain-of-thought of mllm.ACM MM, 2024

    Timin Gao, Peixian Chen, Mengdan Zhang, Chaoyou Fu, Yunhang Shen, Yan Zhang, Shengchuan Zhang, Xiawu Zheng, Xing Sun, Liujuan Cao, and Rongrong Ji. Cantor: Inspiring multimodal chain-of-thought of mllm.ACM MM, 2024. 1

    2024

  11. [19]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. InCVPR, 2024. 1

    2024

  12. [20]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 1, 2, 5

    2025 arXiv

  13. [21]

    Free video-llm: Prompt-guided visual perception for efficient training-free video llms.arXiv preprint arXiv:2410.10441, 2024

    Kai Han, Jianyuan Guo, Yehui Tang, Wei He, Enhua Wu, and Yunhe Wang. Free video-llm: Prompt-guided visual perception for efficient training-free video llms.arXiv preprint arXiv:2410.10441, 2024. 1

    2024 arXiv

  14. [22]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    arXiv

  15. [23]

    Memory-space visual prompting for efficient vision-language fine-tuning.arXiv preprint arXiv:2405.05615, 2025

    Shibo Jie, Yehui Tang, Ning Ding, Zhi-Hong Deng, Kai Han, and Yunhe Wang. Memory-space visual prompting for efficient vision-language fine-tuning.arXiv preprint arXiv:2405.05615, 2025. 1

    2025 arXiv

  16. [24]

    Clevr: A diagnostic dataset for compositional language and elementary visual reasoning

    Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, 2017. 4

    2017

  17. [25]

    Figureqa: An annotated figure dataset for visual reasoning.arXiv preprint arXiv:1710.07300,

    Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Ákos Kádár, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning.arXiv preprint arXiv:1710.07300,

    arXiv

  18. [26]

    A diagram is worth a dozen images

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. InECCV, 2016. 1, 4, 7

    2016

  19. [27]

    Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer.arXiv preprint arXiv:2408.03326,

    arXiv

  20. [28]

    Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023

    Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension.arXiv preprint arXiv:2307.16125, 2023. 1

    2023 arXiv

  21. [29]

    Mvbench: A comprehensive multi-modal video understanding benchmark

    Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. InCVPR, 2024. 1

    2024

  22. [30]

    Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning

    Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:2504.06958, 2025. 1, 3, 6

    2025 arXiv

  23. [31]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437,

    arXiv

  24. [32]

    Visual spatial reasoning.TACL, 2023

    Fangyu Liu, Guy Emerson, and Nigel Collier. Visual spatial reasoning.TACL, 2023. 4

    2023

  25. [33]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InCVPR, 2024. 1, 3

    2024

  26. [34]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InNeurIPS, 2023. 3

    2023

  27. [35]

    Mmbench: Is your multi-modal model an all-around player? In ECCV, 2024

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In ECCV, 2024. 1, 7

    2024

  28. [36]

    Ocrbench: on the hidden mystery of ocr in large multimodal models

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: on the hidden mystery of ocr in large multimodal models. SCIS, 2024. 1

    2024

  29. [37]

    Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, et al. Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024. 1, 3 11

    2024 arXiv

  30. [38]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. InICLR, 2024. 3, 7, 9

    2024

  31. [39]

    Learn to explain: Multimodal reasoning via thought chains for science question answering

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. InNeurIPS, 2022. 4

    2022

  32. [40]

    Cheap and quick: Efficient vision-language instruction tuning for large language models.NeurIPS, 2023

    Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, and Rongrong Ji. Cheap and quick: Efficient vision-language instruction tuning for large language models.NeurIPS, 2023. 3

    2023

  33. [41]

    Feast your eyes: Mixture-of-resolution adaptation for multimodal large language models.arXiv preprint arXiv:2403.03003,

    Gen Luo, Yiyi Zhou, Yuxin Zhang, Xiawu Zheng, Xiaoshuai Sun, and Rongrong Ji. Feast your eyes: Mixture-of-resolution adaptation for multimodal large language models.arXiv preprint arXiv:2403.03003,

    arXiv

  34. [42]

    Video-rag: Visually-aligned retrieval-augmented long video comprehension.arXiv preprint arXiv:2411.13093, 2024

    Yongdong Luo, Xiawu Zheng, Xiao Yang, Guilin Li, Haojia Lin, Jinfa Huang, Jiayi Ji, Fei Chao, Jiebo Luo, and Rongrong Ji. Video-rag: Visually-aligned retrieval-augmented long video comprehension.arXiv preprint arXiv:2411.13093, 2024. 1

    2024

  35. [43]

    Mllm-selector: Necessity and diversity-driven high-value data selection for enhanced visual instruction tuning.arXiv preprint arXiv:2503.20502, 2025

    Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Jiayi Ji, Jie Lou, Debing Zhang, and Rongrong Ji. Mllm-selector: Necessity and diversity-driven high-value data selection for enhanced visual instruction tuning.arXiv preprint arXiv:2503.20502, 2025. 1

    2025 arXiv

  36. [44]

    Generation and comprehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. InCVPR, 2016. 2, 3, 4

    2016

  37. [45]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning.arXiv preprint arXiv:2203.10244, 2022

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning.arXiv preprint arXiv:2203.10244, 2022. 4

    2022 arXiv

  38. [46]

    Infographicvqa

    Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographicvqa. InWACV, 2022. 4

    2022

  39. [47]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InWACV, 2021. 4

    2021

  40. [48]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning.arXiv preprint arXiv:2503.07365, 2025. 1, 3

    2025 arXiv

  41. [49]

    Training language models to follow instructions with human feedback.NeurIPS, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.NeurIPS, 2022. 3

    2022

  42. [50]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InICML, 2021. 3

    2021

  43. [51]

    Direct preference optimization: Your language model is secretly a reward model.NeurIPS, 2023

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model.NeurIPS, 2023. 3

    2023

  44. [52]

    Eve: Efficient multimodal vision language models with elastic visual experts.arXiv preprint arXiv:2501.04322, 2025

    Miao Rang, Zhenni Bi, Chuanjian Liu, Yehui Tang, Kai Han, and Yunhe Wang. Eve: Efficient multimodal vision language models with elastic visual experts.arXiv preprint arXiv:2501.04322, 2025. 1

    2025 arXiv

  45. [53]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019. 8

    1908 arXiv

  46. [54]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017. 3, 4

    2017 arXiv

  47. [55]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024. 3, 4, 5, 7

    2024 arXiv

  48. [56]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025. 1, 3 12

    2025 arXiv

  49. [57]

    Long-vita: Scaling large multi-modal models to 1 million tokens with leading short-context accuray.arXiv preprint arXiv:2502.05177, 2025

    Yunhang Shen, Chaoyou Fu, Shaoqi Dong, Xiong Wang, Peixian Chen, Mengdan Zhang, Haoyu Cao, Ke Li, Xiawu Zheng, Yan Zhang, et al. Long-vita: Scaling large multi-modal models to 1 million tokens with leading short-context accuray.arXiv preprint arXiv:2502.05177, 2025. 1

    2025

  50. [58]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023. 3

    2023 arXiv

  51. [59]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023. 3

    2023 arXiv

  52. [60]

    Measuring multimodal mathematical reasoning with math-vision dataset.arXiv preprint arXiv:2402.14804, 2024

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hong- sheng Li. Measuring multimodal mathematical reasoning with math-vision dataset.arXiv preprint arXiv:2402.14804, 2024. 3, 6, 7, 9

    2024 arXiv

  53. [61]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 3

    2024 arXiv

  54. [62]

    Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference.arXiv preprint arXiv:2412.13663, 2024

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context fin...

    2024 arXiv

  55. [63]

    Qwen2.5-1m technical report.arXiv preprint arXiv:2501.15383,

    An Yang, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoyan Huang, Jiandong Jiang, Jianhong Tu, Jianwei Zhang, Jingren Zhou, Junyang Lin, Kai Dang, Kexin Yang, Le Yu, Mei Li, Minmin Sun, Qin Zhu, Rui Men, Tao He, Weijia Xu, Wenbiao Yin, Wenyuan Yu, Xiafei Qiu, Xingzhang R...

    arXiv

  56. [64]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization.arXiv preprint arXiv:2503.10615, 2025. 1, 3

    2025 arXiv

  57. [65]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800,

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800,

    arXiv

  58. [66]

    Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490, 2023

    Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities.arXiv preprint arXiv:2308.02490, 2023. 1

    2023 arXiv

  59. [67]

    Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv preprint arXiv:2501.04001, 2025

    Haobo Yuan, Xiangtai Li, Tao Zhang, Zilong Huang, Shilin Xu, Shunping Ji, Yunhai Tong, Lu Qi, Jiashi Feng, and Ming-Hsuan Yang. Sa2va: Marrying sam2 with llava for dense grounded understanding of images and videos.arXiv preprint arXiv:2501.04001, 2025. 1

    2025 arXiv

  60. [68]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A m...

    2024

  61. [69]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InICCV, 2023. 3

    2023

  62. [70]

    Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning.arXiv preprint arXiv:2503.18013, 2025

    Yufei Zhan, Yousong Zhu, Shurong Zheng, Hongyin Zhao, Fan Yang, Ming Tang, and Jinqiao Wang. Vision-r1: Evolving human-free alignment in large vision-language models via vision-guided reinforcement learning.arXiv preprint arXiv:2503.18013, 2025. 1, 3

    2025 arXiv

  63. [71]

    Lmms-eval: Reality check on the evaluation of large multimodal models.arXiv preprint arXiv:2407.12772, 2024

    Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Reality check on the evaluation of large multimodal models.arXiv preprint arXiv:2407.12772, 2024. 6

    2024 arXiv

  64. [72]

    Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding

    Tao Zhang, Xiangtai Li, Hao Fei, Haobo Yuan, Shengqiong Wu, Shunping Ji, Chen Change Loy, and Shuicheng Yan. Omg-llava: Bridging image-level, object-level, pixel-level reasoning and understanding. InNeurIPS, 2024. 1 13

    2024

  65. [73]

    Pixel-sail: Single transformer for pixel-grounded understanding.arXiv preprint arXiv:2504.10465, 2025

    Tao Zhang, Xiangtai Li, Zilong Huang, Yanwei Li, Weixian Lei, Xueqing Deng, Shihao Chen, Shunping Ji, and Jiashi Feng. Pixel-sail: Single transformer for pixel-grounded understanding.arXiv preprint arXiv:2504.10465, 2025. 1

    2025 arXiv

  66. [74]

    Enhancing multimodal large language models complex reason via similarity computation.AAAI, 2024

    Xiaofeng Zhang, Fanshuo Zeng, Yihao Quan, Zheng Hui, and Jiawei Yao. Enhancing multimodal large language models complex reason via similarity computation.AAAI, 2024. 1

    2024

  67. [75]

    MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data

    Yilun Zhao, Yunxiang Li, Chenying Li, and Rui Zhang. MultiHiertt: Numerical reasoning over multi hierarchical tabular and textual data. InACL, 2022. 4

    2022

  68. [76]

    Mlvu: A comprehensive benchmark for multi-task long video understanding.arXiv preprint arXiv:2406.04264, 2024

    Junjie Zhou, Yan Shu, Bo Zhao, Boya Wu, Shitao Xiao, Xi Yang, Yongping Xiong, Bo Zhang, Tiejun Huang, and Zheng Liu. Mlvu: A comprehensive benchmark for multi-task long video understanding.arXiv preprint arXiv:2406.04264, 2024. 1

    2024 arXiv

  69. [77]

    Are they the same? exploring visual correspondence shortcomings of multimodal llms.arXiv preprint arXiv:2501.04670, 2025

    Yikang Zhou, Tao Zhang, Shilin Xu, Shihao Chen, Qianyu Zhou, Yunhai Tong, Shunping Ji, Jiangning Zhang, Xiangtai Li, and Lu Qi. Are they the same? exploring visual correspondence shortcomings of multimodal llms.arXiv preprint arXiv:2501.04670, 2025. 1

    2025 arXiv

  70. [78]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Yuchen Duan, Hao Tian, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 1, 3

    2025 arXiv

  71. [79]

    Genimage: A million-scale benchmark for detecting ai-generated image.NeurIPS,

    Mingjian Zhu, Hanting Chen, Qiangyu Yan, Xudong Huang, Guanyu Lin, Wei Li, Zhijun Tu, Hailin Hu, Jie Hu, and Yunhe Wang. Genimage: A million-scale benchmark for detecting ai-generated image.NeurIPS,

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.