Pith. sign in

REVIEW 4 major objections 6 minor 12 cited by

More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multimodal reasoning models hallucinate more than their non-reasoning base models on perception tasks, and longer reasoning chains make it worse by shifting attention away from the image.

desk verdict The paper's core finding—reasoning models hallucinate more on perception—is solid; the length-causality claim is undermined by a steering method fit on the same test benchmark. read the letter →

arxiv 2505.21523 v3 pith:42LJFWK4 submitted 2025-05-23 cs.CL cs.AIcs.CV

Chengzhi Liu , Zhongxing Xu , Qingyue Wei , Juncheng Wu , James Zou , Xin Eric Wang , Yuyin Zhou , Sheng Liu This is my paper · ORCID
classification cs.CLcs.AIcs.CV
keywords multimodalreasoningvisualhallucinationlengthattentionanalysisRH-AUCRH-Benchperceptionbenchmarksreinforcementlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that test-time compute, the extra thinking that multimodal language models are trained to do, systematically weakens their visual grounding. Comparing eight reasoning-augmented models with their non-reasoning base models on five perception benchmarks, the paper finds that the reasoning versions hallucinate more even when the underlying backbone and scale are the same. Attention analysis attributes the effect to a shift away from image tokens toward instruction tokens, a shift that grows as reasoning chains get longer. To make the trade-off measurable, the paper introduces RH-AUC, a metric that summarizes how perception accuracy changes as reasoning length varies, and RH-Bench, a 1,000-sample benchmark spanning reasoning and perception tasks. The central message is that reasoning length has a non-monotonic, task-dependent effect on performance, so evaluations of multimodal reasoning must measure perceptual fidelity and reasoning quality jointly rather than accuracy alone.

What carries the argument

The mechanism that carries the length experiments is latent state steering. The paper collects long and short reasoning traces on the test benchmark, extracts the average hidden-state vector after the attention layer from the reasoning span at each layer, and defines a direction vector $d_\ell = S^{\ell}_{\text{long}} - S^{\ell}_{\text{short}}$; shifting hidden states by $\alpha d_\ell$ with $\alpha \in [-0.15, 0.15]$ stretches or compresses the reasoning chain while keeping the query and model fixed. The measurement object is RH-AUC, the area under the curve formed by points $(R_T, H_T)$, where $R_T$ is reasoning accuracy and $H_T$ is hallucination performance at reasoning length $T$, min-max normalized and integrated with the trapezoidal rule; higher values mean a better balance of reasoning and perception. RH-Bench provides the 1,000 samples for this measurement, with 500 reasoning questions from four math and science benchmarks and 500 perception questions from four hallucination benchmarks, each offered in multiple-choice and open-ended form.

What would settle it

Force one reasoning model to produce longer chains with token-budget forcing or test-time scaling only, never latent state steering, and measure per-token attention to image tokens and hallucination rates on MMVP and MMHalu; if longer chains do not reduce visual attention or do not increase hallucination, the claimed length-attention-hallucination chain is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the reasoning paradigm itself, not model scale, degrades visual grounding. Reasoning-augmented models consistently allocate less attention to visual tokens and more attention to instruction tokens than their non-reasoning counterparts, and extending the reasoning chain further reduces visual attention and increases reliance on language priors, which raises hallucination rates on perception tasks. The relationship between reasoning length and performance is non-monotonic: moderate lengths are best, reasoning-heavy tasks such as MathVista benefit from longer chains, and perception-heavy tasks such as MMHalu peak at shorter lengths. The paper captures this balance with RH-AUC, computed from min-max normalized pairs of reasoning accuracy and hallucination performance at different lengths via the trapezoidal rule, and with RH-Bench, a 1,000-sample diagnostic benchmark with multiple-choice and open-ended reasoning and perception questions. On that benchmark, larger models, RL-only training rather than SFT-plus-RL, and domain-matched training data all improve the reasoning-hallucination balance, while the sheer volume of training data does not.

Load-bearing premise

The paper assumes that adjusting the steering-strength parameter changes only how long the model thinks, not what it attends to or how it reasons, so the rise-and-fall accuracy curves are caused by reasoning length itself rather than by the steering direction.

Editorial extensions

If this is right

  • Evaluation of multimodal reasoning models should report perception accuracy and reasoning accuracy jointly; single-point metrics can rank models misleadingly because the two peaks occur at different reasoning lengths.
  • The optimal reasoning length is task-dependent, so a fixed inference budget or a universal think-longer policy will hurt perception-heavy tasks even as it helps math-style tasks.
  • Training data should be balanced by domain and type rather than by volume alone; adding more visual data does not guarantee better visual grounding.
  • RL-only training appears to produce more concise and adaptive reasoning, so supervised fine-tuning that forces imitation of long reasoning formats may be a cause of hallucination amplification rather than just a correlate.
  • Scaling model size improves the reasoning-hallucination balance, so the trade-off is not fixed by the reasoning paradigm itself.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the causal story holds, inference-time systems could treat reasoning length as a tunable hyperparameter and stop a chain once visual evidence stops changing the answer, reducing hallucination while preserving reasoning gains; the paper does not implement such a controller.
  • The attention result suggests an untested intervention: explicitly re-anchoring attention to image tokens at intervals during long chains should reduce hallucination if the mechanism is causal.
  • The RH-AUC formulation is not tied to vision and language; it could be applied to any model family where output length trades off against a fidelity signal, such as factual consistency in long-form text generation.
  • Because the training-data findings are observational, a controlled study that trains one backbone on matched data with different domain mixes is the natural next step; the paper itself notes this limitation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper studies hallucination in multimodal reasoning models that generate extended reasoning chains. It reports that reasoning models hallucinate more on visual perception tasks than their non-reasoning base models, attributes this to reduced attention on visual tokens, and investigates how reasoning-chain length affects the reasoning-hallucination balance. The authors introduce RH-AUC, a metric that summarizes the trade-off between reasoning accuracy and hallucination across reasoning lengths, and RH-Bench, a 1,000-sample diagnostic benchmark. Using latent state steering to control reasoning length, they find a non-monotonic relationship between length and performance, with task-dependent optimal lengths, and they report that larger models and RL-only training generally yield better reasoning-perception balance.

Significance. If the causal claims hold, the paper provides a valuable warning that test-time compute in multimodal LLMs may systematically trade perceptual accuracy for language-prior fluency, with direct implications for how multimodal reasoning models should be evaluated. The observational core—that reasoning models hallucinate more than their base models across multiple benchmarks—is supported by consistent results on five established hallucination datasets and is a useful empirical contribution. The release of RH-Bench and RH-AUC constitutes a practical evaluation resource, and the attention analyses offer a plausible mechanism. However, the reasoning-length and attention-decline claims rest on a steering method whose validation is incomplete, so the stronger causal conclusions are not yet established.

major comments (4)
  1. [Section 4.1, Eq. (2)] The steering direction d_l is computed from long and short reasoning traces drawn from the same test benchmark on which accuracy is subsequently measured. The paper does not show that d_l encodes reasoning length rather than content differences (e.g., answer correctness, topic, or hallucination patterns) between the long and short trace pools. Adding α·d_l to hidden states could change what the model generates in ways unrelated to length, making the non-monotonic accuracy curves in Figure 7 and the attention decline in Figure 6 potential artifacts of the steering direction. Please validate the steering on held-out items, report monotonic control of length on a separate set, and compare with the budget-forcing and test-time-scaling controls introduced in Section 4.1.
  2. [Section 3.2, Figure 6] The claim that longer reasoning chains reduce visual attention is based on comparing 'normal thinking' with 'overthinking' produced by the same latent state steering. Without evidence that steering isolates length, the observed shift in attention from visual to instruction tokens may be caused by the content of the steering direction rather than by chain length per se. Please provide attention analyses under budget forcing or test-time scaling as complementary controls, or demonstrate that d_l is orthogonal to answer content and other semantic features of the traces.
  3. [Section 4.2, Figure 7 and Section 5.1, Eq. (3)] The central conclusion that reasoning length has a non-monotonic effect on performance, and that the optimum is task-dependent, is supported only by curves generated with latent state steering. The other two control strategies from Section 4.1 are not shown in the main text. If the steering intervention is confounded with content, the claimed non-monotonicity and task-specific optimal intervals may not reflect natural reasoning-length effects. Please include results for budget forcing and test-time scaling, or explicitly restrict the claims to the steering intervention and acknowledge the confound.
  4. [Section 5.2, Training Dataset] The conclusions about the influence of training data type, domain, and volume are drawn from comparisons among existing models with different training pipelines, data sizes, and data sources, without controlled retraining experiments. The Limitation section appropriately acknowledges this observational nature, but the abstract and Section 5.2 state these as findings. Please soften the causal language in the abstract and key claims, or add a clear caveat that these are correlational observations based on public technical reports.
minor comments (6)
  1. [Section 4.1 heading] The heading contains a typo: 'Straregies' should be 'Strategies'.
  2. [Section 3.2, Figure 6 caption] In the text describing Figure 6, 'the model correctly identifies the gray well' should read 'gray wall'.
  3. [Section 5.2] There are several typos: 'traing data size' should be 'training data size', and 'multimodel reasoning models' should be 'multimodal reasoning models'.
  4. [Section 4.2] The 'Zerothink' condition is mentioned without a definition in the main text; please clarify how it is implemented and how it differs from a simple length reduction.
  5. [References] References [8] and [9] both cite the same Insight-V paper, and references [49] and [50] both cite the same Seg-Zero paper; please deduplicate.
  6. [Section 5.1, RH-AUC] The RH-AUC computation uses min-max normalization and a specific set of reasoning lengths; please discuss how sensitive the metric is to the chosen length set and whether the normalization preserves comparability across models with different accuracy ranges.

Circularity Check

2 steps flagged · score 6.0 of 10

Latent-state steering directions are computed from the same test benchmark used for evaluation, so the non-monotonic length-accuracy curves and attention decline are not cleanly identified as effects of reasoning length; external hallucination comparisons remain independent.

  1. fitted input called prediction [Section 4.1 (Eq. 1-2) and Section 4.2 (Figure 7)]
    "we collect responses from the test benchmark and categorize them into long reasoning traces Rlong and short reasoning traces Rshort based on token length. ... As α increases, the length of the reasoning chain extends, as shown below: dℓ = Sℓ_long − Sℓ_short, Sℓ_steering = Sℓ + αdℓ. (2) ... we employ the Latent State Steering method adjusts the thinking step for reasoning and perception tasks. It is evident that as the thinking length increases, the model’s performance across tasks generally follows a rising-then-falling trajectory."

    The steering direction d_l is estimated from long and short traces drawn from the same test benchmark on which accuracy is subsequently measured, and the main-text non-monotonic curves (Figure 7) are produced by applying this in-sample direction. The paper does not validate that d_l encodes only reasoning length rather than content differences (topic, answer correctness, or hallucination content) in those traces; if long and short traces differ in content, S^ℓ_steering = S^ℓ + αd_l changes what the model generates, not merely how long it reasons. The claimed causal relationship between length and performance is therefore not identified independently of the steering construction.

  2. fitted input called prediction [Section 3.2, Figure 6 caption]
    "In normal thinking, the model generates outputs as typically expected, while in overthinking, the reasoning length is adjusted using Latent State Steering (Section 4.1). Longer reasoning chains further exacerbate the degradation of attention to visual information and focus toward linguistic priors."

    The 'overthinking' condition is generated with the same d_l fit to long/short traces from the test benchmark, so the attention reduction in Figure 6 is measured under a steering direction that may carry content/accuracy information. The conclusion that longer chains cause reduced visual attention rests on the unvalidated assumption that αd_l increases length without altering attention-relevant content; otherwise the observed attention shift is an artifact of the steering direction, not a length effect.

full rationale

The external hallucination comparisons in Section 2 (reasoning models vs. base models on MMVP, MMEval-Pro, VMCBench, Bingo, MMHAL) and the descriptive RH-AUC definition (Eq. 3) are not circular: they use independent benchmarks and summarize measured curves. The score is elevated because the central length-effect claims in Sections 3.2 and 4.2 rely on Latent State Steering (Eq. 1-2), whose direction d_l is computed from long and short traces collected from the same test benchmark used for evaluation. Without evidence that d_l isolates token length from trace content, the non-monotonic accuracy curves and the visual-attention decline are partly induced by the in-sample steering construction rather than by reasoning length alone. The paper's Limitation section acknowledges the observational nature of the training-data conclusions but does not address this steering in-sample issue. This is partial circularity, not full circularity: accuracy is not directly fitted, and the base-vs-reasoning hallucination finding stands independently.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rely on three classes of input: evaluation thresholds fixed by the authors, a steering method derived from the test benchmark, and an observational comparison of training reports. The alpha range and long/short split are hand-chosen parameters. No new physical or conceptual entities are postulated.

free parameters (3)
  • alpha (latent steering scale) = range [-0.15, 0.15]
    Hand-chosen range to expand or shorten reasoning traces; the value is a control knob rather than a fitted optimum, but it determines the set of reasoning lengths used to compute RH-AUC.
  • long/short trace split threshold for steering = not specified in paper
    Reasoning traces are split into R_long and R_short by token length; the cut point is not reported, and it affects the steering direction d_l.
  • hallucination score threshold = 3 out of 6
    GPT-4o scores open-ended perception responses from 0 to 6; responses below 3 are labeled hallucinations. This threshold defines H_T but is a design choice rather than a fitted constant.
assumptions (4)
  • domain assumption Attention allocation to visual tokens is a valid and causal proxy for visual grounding.
    Section 3 interprets reduced attention as the cause of hallucination without quantitative mediation evidence.
  • domain assumption Accuracy on the perception portion of RH-Bench measures hallucination (via GPT-4o scoring).
    Section 5.1 defines perception accuracy from a 0-6 score with threshold 3; this assumes GPT-4o judgments align with human perception.
  • ad hoc to paper The steering direction d_l generalizes to control reasoning length without changing content.
    Section 4.1 computes d_l from test-benchmark traces and applies it to the same benchmark; the assumption that it is a clean length control is unverified.
  • standard math Trapezoidal integration and min-max normalization are appropriate for comparing balance curves across models.
    Equation 3 uses the trapezoidal rule; the comparability of RH-AUC depends on whether normalization constants are shared across models, which is not specified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models." pith.science (2026). https://pith.science/paper/42LJFWK4

@misc{pith2026250521523,
  author       = {Pith},
  title        = {Pith review of: More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/42LJFWK4}},
  note         = {Machine review of arXiv:2505.21523}
}
read the original abstract

Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance on tasks such as multimodal math reasoning. However, this improved reasoning ability often comes with increased hallucination: as generations become longer, models tend to drift away from image-grounded content and rely more heavily on language priors. Attention analysis shows that longer reasoning chains lead to reduced focus on visual inputs, which contributes to hallucination. To systematically study this phenomenon, we introduce RH-AUC, a metric that quantifies how a model's perception accuracy changes with reasoning length, allowing us to evaluate whether the model preserves visual grounding during reasoning. We also release RH-Bench, a diagnostic benchmark that spans a variety of multimodal tasks, designed to assess the trade-off between reasoning ability and hallucination. Our analysis reveals that (i) larger models typically achieve a better balance between reasoning and perception, and (ii) this balance is influenced more by the types and domains of training data than by its overall volume. These findings underscore the importance of evaluation frameworks that jointly consider both reasoning quality and perceptual fidelity.

Figures

Figures reproduced from arXiv: 2505.21523 by the authors.

Figure 1
Figure 1. (a) Example of outputs from a reasoning model and a non-reasoning model on a perception task. Red highlights indicate visual hallucination. Multimodal reasoning models are generally more prone to amplifying hallucinations during the reasoning process compared to their non-reasoning counterparts. (b) Performance of different models on reasoning and perception tasks in the RH-Bench dataset. Better performing models ar… view at source ↗
Figure 2
Figure 2. Comparison of reasoning and non-reasoning models on five perception benchmarks. Results [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Performance across four perception benchmarks comparing Base, RL, and SFT+RL. [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Two common types of hallucination patterns observed in multimodal reasoning models. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Attention allocation and visual grounding between reasoning and non reasoning models. [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Attention shift in the reasoning model under different reasoning length. In normal thinking, [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Reasoning-Hallucination balance of multimodal reasoning models under varying reasoning [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: (a) Accuracy trends on the RH-Bench reasoning task across different reasoning lengths for 3B and 7B models. Larger models typically exhibit more stable performance across varying reasoning lengths. (b) Comparison of SFT+RL and RL-only training paradigms in terms of RH-…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs

    cs.CV 2025-11 unverdicted novelty 8.0 of 10

    MVI-Bench supplies the first taxonomy and dataset focused on misleading visual inputs to measure LVLM robustness, with tests on 18 models revealing clear weaknesses.

  2. When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Thinking-mode VLMs collapse answer-token entropy, but thinking-chain entropy and length serve as robust, zero-cost hallucination predictors.

  3. StreamFlow: Dynamic Memory Flows for Streaming Video Understanding

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A streaming-video memory system that filters redundant frames before encoding, stores older video as latent tokens, and re-injects them when visual attention drops reaches 67.73% on StreamingBench.

  4. H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Token-level confidence arbitration between VL and text teachers during on-policy distillation improves multimodal student reasoning over single-teacher OPD and GRPO.

  5. Starve to Perceive: Taming Lazy Perception in VLMs with Constrained Visual Bandwidth

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Constraining visual token budgets during SFT and RL forces VLMs to learn functional active perception, yielding ~5% relative gains and strong transfer to unconstrained evaluation.

  6. SAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplification

    cs.CV 2026-02 conditional novelty 6.0 of 10

    Adaptively scaling visual attention boosting per token, guided by a combined entropy-and-visual-grounding risk score, reduces hallucinations in LVLMs more than fixed boosting.

  7. SafeWork-R1: Coevolving Safety and Intelligence under the AI-45$^{\circ}$ Law

    cs.AI 2025-07 conditional novelty 6.0 of 10

    SafeWork-R1 shows that a staged RL pipeline with safety, value, and knowledge verifiers can improve both safety and general reasoning scores over a base multimodal model.

  8. MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Ten leading VLMs mostly fail to report removed essential object parts as missing, and simulated detector evidence, image tools, longer reasoning, and an easier fine-tune barely improve accuracy.

  9. VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation

    cs.CL 2025-10 conditional novelty 5.0 of 10

    EVisRAG, an evidence-guided observe-record-reason-answer pipeline trained with reward-scoped GRPO, improves multi-image visual QA accuracy by about 19% over its backbone VLM across five benchmarks.

  10. A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models

    cs.CL 2025-09 conditional novelty 4.0 of 10

    A structured literature survey concluding that reasoning capabilities do not automatically make LLMs more trustworthy and can introduce new vulnerabilities in safety, robustness, and privacy.

  11. Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A survey organizes multimodal reasoning research into a staged roadmap and proposes native large multimodal reasoning models that unify perception, generation, and agentic planning.

  12. Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle

    cs.CL 2025-09 conditional novelty 3.0 of 10

    A survey that maps reinforcement learning methods, datasets, benchmarks, and open-source tools across the full training lifecycle of large language models, focusing on verifiable-reward reasoning.

Reference graph

Works this paper leans on

54 extracted references · 12 canonical work pages · cited by 12 Pith papers

  1. [1]

    Nothing in excess: Mitigating the exaggerated safety for llms via safety-conscious activation steering

    Zouying Cao, Yifei Yang, and Hai Zhao. Nothing in excess: Mitigating the exaggerated safety for llms via safety-conscious activation steering. arXiv preprint arXiv:2408.11491, 2024

    arXiv 2024

  2. [2]

    Chang, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue

    Edward Y . Chang, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in llms. CoRR, abs/2502.03373, 2025

    arXiv 2025

  3. [3]

    Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.20330, 2024

    arXiv 2024

  4. [4]

    Physbench: Benchmarking and enhancing vision-language models for physical world understanding

    Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision-language models for physical world understanding. arXiv preprint arXiv:2501.16411, 2025

    arXiv 2025

  5. [5]

    Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges

    Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287, 2023

    arXiv 2023

  6. [6]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv 2025

  7. [7]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huaj...

    arXiv 2024

  8. [9]

    Insight-v: Exploring long-chain visual reasoning with multimodal large language models

    Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint arXiv:2411.14432, 2024. 10

    arXiv 2024

Show all 54 references
  1. [10]

    Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models

    Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proce...

    2024

  2. [11]

    Mmevalpro: Calibrating multimodal benchmarks towards trustworthy and efficient evaluation

    Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, et al. Mmevalpro: Calibrating multimodal benchmarks towards trustworthy and efficient evaluation. arXiv preprint arXiv:2407.00468, 2024

    2024 arXiv

  3. [12]

    Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation

    Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In Proceedings of the IEEE/CVF Conferenc...

    2024

  4. [13]

    Self-introspective decoding: Alleviating hallucinations for large vision-language models

    Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. Self-introspective decoding: Alleviating hallucinations for large vision-language models. In The Thirteenth International Conference on Learning Representations, 2025

    2025

  5. [14]

    Q-insight: Understanding image quality via visual reinforcement learning

    Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang. Q-insight: Understanding image quality via visual reinforcement learning. arXiv preprint arXiv:2503.22679, 2025

    2025 arXiv

  6. [15]

    Evaluating object hallucination in large vision-language models

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processi...

    2023

  7. [16]

    Ocean-r1: An open and generalizable large vision-language model enhanced by reinforcement learning

    Ming Lingfeng, Li Yadong, Chen Song, Xu Jianhua, Zhou Zenan, and Chen Weipeng. Ocean-r1: An open and generalizable large vision-language model enhanced by reinforcement learning. https://github.com/VLM-RL/Ocean-R1 , 2025. Accessed: 2025-04-03

    2025

  8. [17]

    In-context vectors: Making in context learning more effective and controllable through latent space steering

    Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv:2311.06668, 2023

    2023 arXiv

  9. [18]

    Reducing hallucinations in large vision-language models via latent space steering

    Sheng Liu, Haotian Ye, and James Zou. Reducing hallucinations in large vision-language models via latent space steering. In The Thirteenth International Conference on Learning Representations

  10. [19]

    Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216–233. Springer, 2024

    2024

  11. [20]

    Visual-rft: Visual reinforcement fine-tuning

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint arXiv:2503.01785, 2025

    2025 arXiv

  12. [21]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representat...

    2024

  13. [22]

    Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022

    Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022

    2022

  14. [23]

    Pace: Parsimonious concept engineering for large language models

    Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Callison-Burch, and René Vidal. Pace: Parsimonious concept engineering for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. 11

    2024

  15. [24]

    Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  16. [25]

    s1: Simple test-time scaling, 2025

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling, 2025

    2025

  17. [26]

    Learning to reason with LLMs

    OpenAI. Learning to reason with LLMs. 2024

    2024

  18. [27]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning? CoRR, abs/2407.01284, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma Gongque, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achie...

    2024 arXiv

  19. [28]

    Object hallucination in image captioning

    Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045, 2018

    2018

  20. [29]

    Vlm-r1: A stable and generalizable r1-style large vision-language model

    Haozhan Shen, Zilun Zhang, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-r1: A stable and generalizable r1-style large vision-language model. 2025. Accessed: 2025-02-15

    2025

  21. [30]

    MM-MATH: Advancing multimodal math evaluation with process evaluation and fine-grained classification

    Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. MM-MATH: Advancing multimodal math evaluation with process evaluation and fine-grained classification. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Findings of the Association for Computational Linguistics: EM...

    2024

  22. [31]

    Content-rich aigc video qual- ity assessment via intricate text alignment and motion-aware consistency

    Shangkun Sun, Xiaoyu Liang, Bowen Qu, and Wei Gao. Content-rich aigc video qual- ity assessment via intricate text alignment and motion-aware consistency. arXiv preprint arXiv:2502.04076, 2025

    2025 arXiv

  23. [32]

    Aligning large multimodal models with factually augmented rlhf

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525, 2023

    2023 arXiv

  24. [33]

    Reason-rft: Reinforcement fine-tuning for visual reasoning

    Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rft: Reinforcement fine-tuning for visual reasoning. arXiv preprint arXiv:2503.20752, 2025

    2025

  25. [34]

    Llamav-o1: Rethinking step-by-step visual reasoning in llms

    Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-o1: Rethinking step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.06186, 2025

    2025 arXiv

  26. [35]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9568–9578, 2024

    2024

  27. [36]

    Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504.08837, 2025

    2025 arXiv

  28. [37]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024

    2024

  29. [38]

    Visualprm: An effective process reward model for multimodal reasoning

    Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291, 2025

    2025 arXiv

  30. [39]

    Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficient visual reasoning self-improvement. arXiv preprint arXiv:2504.07934, 2025. 12

    2025 arXiv

  31. [40]

    Fast-slow thinking for large vision-language model reasoning

    Wenyi Xiao, Leilei Gan, Weilong Dai, Wanggui He, Ziwei Huang, Haoyuan Li, Fangxun Shu, Zhelun Yu, Peng Zhang, Hao Jiang, et al. Fast-slow thinking for large vision-language model reasoning. arXiv preprint arXiv:2504.18458, 2025

    2025

  32. [41]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  33. [42]

    Llava-cot: Let vision language models reason step-by-step, 2024

    Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2024

    2024

  34. [43]

    Redstar: Does scaling long-cot data unlock better slow-reasoning systems? arXiv preprint arXiv:2501.11284, 2025

    Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, et al. Redstar: Does scaling long-cot data unlock better slow-reasoning systems? arXiv preprint arXiv:2501.11284, 2025

    2025 arXiv

  35. [44]

    R1-onevision: Ad- vancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. R1-onevision: Ad- vancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  36. [45]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    2024 arXiv

  37. [46]

    Clearsight: Visual signal enhancement for object hallucination mitigation in multimodal large language models.arXiv preprint arXiv:2503.13107, 2025

    Hao Yin, Guangzong Si, and Zilei Wang. Clearsight: Visual signal enhancement for object hallucination mitigation in multimodal large language models.arXiv preprint arXiv:2503.13107, 2025

    2025 arXiv

  38. [47]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Co...

    2024

  39. [48]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on C...

    2024

  40. [49]

    Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement

    Liu Yuqi, Peng Bohao, Zhong Zhisheng, Yue Zihao, Lu Fanbin, Yu Bei, and Jia Jiaya. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement. 2025

    2025

  41. [50]

    Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025

    Liu Yuqi, Peng Bohao, Zhong Zhisheng, Yue Zihao, Lu Fanbin, Yu Bei, and Jia Jiaya. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025

    2025

  42. [51]

    Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning

    Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, et al. Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884, 2024

    arXiv

  43. [52]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503.12937, 2025

    2025 arXiv

  44. [53]

    Improve vision language model chain-of-thought reasoning

    Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprint arXiv:2410.16198, 2024

    2024 arXiv

  45. [54]

    Automated generation of chal- lenging multiple-choice questions for vision language model evaluation

    Yuhui Zhang, Yuchang Su, Yiming Liu, Xiaohan Wang, James Burgess, Elaine Sui, Chenyu Wang, Josiah Aklilu, Alejandro Lozano, Anjiang Wei, et al. Automated generation of chal- lenging multiple-choice questions for vision language model evaluation. arXiv preprint arXiv:2501.03225, 2025

    2025 arXiv

  46. [55]

    Marco-o1: Towards open reasoning models for open-ended solutions

    Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-o1: Towards open reasoning models for open-ended solutions. arXiv preprint arXiv:2411.14405, 2024. 13

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.