Pith. sign in

REVIEW 3 major objections 43 references

Self-drawn visual steps often hurt math reasoning; early attention can abort them before they finish.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 01:05 UTC pith:KLHZU3CQ

load-bearing objection Solid training-free gate for harmful UMM visual CoT steps; the utility-gap diagnosis and dual early-attention signals are the real contribution, with the usual hyperparameter and model-family caveats. the 3 major comments →

arxiv 2607.10004 v1 pith:KLHZU3CQ submitted 2026-07-10 cs.CV

Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning

Wenxi Gao , Guanxi Lu , Didi Zhu , Hao Mark Chen , Quan Deng , Zhican Wang , Jiankang Deng , Hongxiang Fan This is my paper
classification cs.CV
keywords unified multimodal modelsinterleaved multimodal reasoningvisual chain-of-thoughtadaptive visual gatingearly diffusion attentiongeneration utility gapmathematical reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Unified multimodal models that interleave text with self-generated diagrams for visual math problems frequently produce images that mislead later reasoning or waste compute. Roughly a quarter of samples can be harmed by those images, and image generation dominates latency and memory. The authors show that two attention statistics available after only a few early diffusion steps—Generation Intent (whether the model is attending to a concrete textual drawing plan) and Visual Fidelity (whether generation stays grounded in the input diagram)—predict whether a visual step will help. Their training-free gate, AdaViG, aborts the generation when both signals are weak, discarding the partial image and continuing in text. On two Bagel-based CoT systems and five math benchmarks, this raises accuracy by up to 5.7 percent while cutting image-generation FLOPs by 25–91 percent and wall-clock latency by 15–46 percent.

Core claim

Intrinsic visual steps in unified multimodal chain-of-thought are not automatically useful: helpful and harmful generations largely cancel, creating a generation utility gap. Early diffusion attention yields two predictive signals—Generation Intent I and Visual Fidelity F—and aborting only when both are below fixed thresholds improves final answer accuracy while removing most of the cost of the aborted generations.

What carries the argument

AdaViG: a training-free adaptive visual gate that, at a fixed early diffusion step t⋆, computes Generation Intent I (relative attention mass of VAE queries on preceding text versus input-image tokens) and Visual Fidelity F (attention mass on input ViT tokens), then aborts the image and resets to text if both fall below thresholds.

Load-bearing premise

Two hand-tuned early-attention numbers, read at one fixed diffusion step on a small calibration set, are assumed to stay predictive of final generation utility across models and benchmarks.

What would settle it

On a held-out math-vision suite, measure whether the same (t⋆, τI, τF) pair still yields accuracy gains and FLOP reductions, or whether re-tuning is required for each new model or domain; if gains disappear without re-tuning, the signals are not general.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Interleaved visual CoT systems can raise accuracy by selectively suppressing harmful self-drawn images rather than generating every triggered visual step.
  • Most of the diffusion cost of low-utility images can be avoided after only a few early steps, cutting latency and KV-cache memory.
  • The same two signals can filter training traces or serve as reward terms so future UMM training keeps helpful visual steps and discards harmful ones.
  • Models that trigger many images gain mainly efficiency; models that already generate more carefully gain mainly accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any multimodal generator whose early denoising attention already separates plan tokens from input tokens could reuse the same two-signal abort rule without retraining.
  • The observed attention drift after a bad image is inserted suggests that post-hoc reweighting of generated versus input image tokens might recover accuracy even when abort is too late.
  • If generation quality itself is weak, gating alone cannot supply missing visual detail; the method mainly protects against noise rather than creating better drawings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper studies unified multimodal models (UMMs) that interleave text and self-generated visual steps for visual mathematical reasoning. It documents a generation utility gap: intrinsic visual steps are often harmful (~26–30% of cases in counterfactual Cgen vs Ctxt comparisons) as well as expensive (~50% of latency, ~80% of extra KV-cache). From early diffusion attention it defines two signals—Generation Intent I (Eq. 4: relative VAE attention to preceding text vs input image) and Visual Fidelity F (Eq. 5: VAE attention to input ViT tokens)—and proposes AdaViG, a training-free gate that aborts a triggered image generation at step t⋆=5 when both signals fall below fixed thresholds (Eq. 6). On Bagel-Zebra-CoT and Bagel-Canvas across DynaMath, WeMath, MathVerse, MathVista, and OlympiadBench, AdaViG reports accuracy gains up to +5.7% with 25–91% fewer image-generation FLOPs and 15.4–45.6% lower wall-clock latency (Tables 1–2), supported by ablations of single-signal and OR gates, random/text-only/DAP-ICoT abort baselines, case studies, and attention analyses.

Significance. If the results hold, the work is a clear and practical contribution to interleaved multimodal CoT. It reframes visual generation from a fixed next-token decision into a utility-aware runtime choice, and does so without training. Strengths that should be credited include: (i) a well-motivated empirical diagnosis of the utility gap and of Intent Vacancy / Perception Drift; (ii) a simple, introspective gate built only from attention masses already produced in early diffusion; (iii) multi-benchmark accuracy and efficiency tables with FLOPs and latency; (iv) ablations showing that the conjunction of I and F outperforms I-only, F-only, I∨F, random abort, text-only, and an adapted DAP-ICoT baseline; and (v) qualitative attention and case studies that make the failure modes concrete. The training-free, plug-and-play framing is attractive for UMM inference and for possible use as a filter or reward signal in future CoT training.

major comments (3)
  1. The central mechanistic claim is that early-diffusion (I, F) at fixed t⋆=5 predicts generation utility well enough that a single gate yields the reported accuracy gains. Fig. 3 and §5.2 establish the low–low concentration of harmful traces on MathVision, and thresholds are chosen on MathVision testmini (§6.1). Main results (Table 1) are on five other benchmarks, but the manuscript does not report (I, F) vs utility scatterplots, abort-vs-utility correlations, or per-benchmark ROC-style checks on those sets. Appendix A.8 only varies thresholds on MathVista. Without this transfer analysis, it remains possible that part of the +2.2–5.7% gains is due to a gate that happens to drop enough generations rather than to the claimed predictive structure of I and F. Please add at least one non-MathVision joint (I, F) utility plot (or abort precision/recall by utility label) and a short sensitivity ta
  2. Thresholds are model-specific—(0.5, 0.04) for Bagel-Zebra-CoT vs (0.6, 0.01) for Bagel-Canvas (§6.1)—while the abstract and introduction present AdaViG as a training-free, plug-and-play method. The large gap in τF (4×) and the different generation frequencies of the two models (Fig. 8: ~4.8 vs ~1.3 images/sample) suggest that the operating point is not architecture- or SFT-invariant. This does not invalidate the empirical gains, but it weakens the universality claim. Please either (a) provide a single shared (τI, τF) setting and re-report Table 1, or (b) explicitly reframe AdaViG as requiring a small held-out calibration set per model and quantify how many samples are needed for stable thresholds.
  3. All UMM-with-CoT experiments use Bagel-7B variants (Bagel-Zebra-CoT and Bagel-Canvas) with a shared MoT diffusion interface (§3, §6.1). The definitions of I and F (Eqs. 4–5) and the abort algorithm (Appendix A.3) are tied to VAE-query attention over Stext / Sin in that interface. The paper claims a method for “unified multimodal reasoning,” yet there is no experiment on another UMM family (e.g., Janus-Pro, Emu3, or a non-Bagel MathCanvas-style model) or on a non-math interleaved setting. A load-bearing scope statement is needed: either demonstrate the gate on one additional architecture, or clearly limit the claim to Bagel-style MoT diffusion CoT and discuss what would have to change for other UMM generators.

Circularity Check

0 steps flagged

No significant circularity: AdaViG is an empirical, training-free gate whose accuracy/efficiency claims are measured on held-out benchmarks, not forced by definition or by a self-citation chain.

full rationale

The paper's load-bearing chain is observational and experimental, not a first-principles derivation that collapses into its inputs. Generation Intent I and Visual Fidelity F are defined from already-computed early diffusion attention masses (Eqs. 4–5: relative text vs. input attention and input-ViT attention mass at fixed t⋆). The abort rule (Eq. 6) is a free conjunction of two weak-signal thresholds; those thresholds are ordinary hyperparameters tuned on MathVision testmini (304 samples) and then applied unchanged to DynaMath, WeMath, MathVerse, MathVista, and OlympiadBench. The reported +2.2–5.7% accuracy and FLOPs/latency reductions (Tables 1–2) are empirical outcomes of running the gate versus baselines, not quantities algebraically equal to the fit or renamed from the tuning set. There is no uniqueness theorem, no self-citation that alone forces the method, and no claim that I or F 'predict' a quantity that was already fitted into them. Concerns about threshold transfer across datasets are generalization/robustness issues, not circularity. Score 0 is therefore appropriate: the work is self-contained against external benchmarks and does not reduce its central claims by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 3 invented entities

The central claim rests on empirical definitions of two attention-derived scores, a fixed early decision step, and two thresholds tuned on a small held-out set. No new physical entities are postulated; the load-bearing modeling choices are the attention-mass definitions and the conservative dual-threshold abort rule.

free parameters (3)
  • t⋆ (early abort diffusion step) = 5
    Fixed at 5 of T=50; chosen as accuracy–cost trade-off (Figure 9). Central claim depends on deciding before most diffusion cost is spent.
  • τI (Generation Intent threshold) = 0.5 (Zebra) / 0.6 (Canvas)
    Hand-tuned per model on MathVision testmini; abort requires I < τI. Different values for Bagel-Zebra-CoT (0.5) and Bagel-Canvas (0.6).
  • τF (Visual Fidelity threshold) = 0.04 (Zebra) / 0.01 (Canvas)
    Hand-tuned per model on MathVision testmini; abort requires F < τF. Values 0.04 (Zebra) and 0.01 (Canvas).
axioms (4)
  • domain assumption Early diffusion VAE-query attention mass to text vs input image segments is a reliable proxy for whether the model has a concrete textual drawing plan (Generation Intent).
    Section 5.2 defines I from mt⋆(Qvae, Stext) / (text + input); motivated by case analysis but not proved to be necessary or sufficient outside the studied models.
  • domain assumption Early diffusion attention mass to input-image ViT tokens (Visual Fidelity F) indicates whether generation remains grounded in the original diagram.
    Section 5.2 equation (5); used jointly with I for the abort rule.
  • ad hoc to paper Aborting only when both I and F are weak preserves helpful generations better than single-signal or OR rules.
    Section 5.3 conservative rule g(I,F); supported by ablation Table 3 but is a design choice of the paper.
  • domain assumption Image-generation FLOPs scale linearly with executed diffusion steps, so abort rate r yields ΔFLOPs ≈ r(1−t⋆/T).
    Appendix A.4; standard for diffusion cost accounting but ignores fixed overheads.
invented entities (3)
  • Generation Intent (I) no independent evidence
    purpose: Scalar early-diffusion statistic used as one abort signal for visual steps.
    Defined in Eq. (4) as relative attention mass of VAE queries to text vs input image; not a previously standard named quantity in the cited UMM CoT literature.
  • Visual Fidelity (F) no independent evidence
    purpose: Scalar early-diffusion statistic measuring grounding on input ViT tokens; second abort signal.
    Defined in Eq. (5); paper-specific operationalization of perception drift.
  • AdaViG gate no independent evidence
    purpose: Training-free runtime policy that aborts or continues each triggered image generation.
    Algorithm 1; the method being proposed.

pith-pipeline@v1.1.0-grok45 · 24936 in / 3451 out tokens · 31896 ms · 2026-07-14T01:05:51.298111+00:00 · methodology

0 comments
read the original abstract

Unified multimodal models (UMMs) with interleaved reasoning, which generate both textual and visual steps as part of intermediate reasoning traces, have demonstrated great potential for visual mathematical reasoning tasks. However, we identify a key insight in this paradigm: generating intermediate visual reasoning steps is not always beneficial and can even be harmful, as self-generated visual steps may introduce erroneous visual evidence that misleads subsequent reasoning. Moreover, frequently triggering visual steps during reasoning incurs substantial computational and memory overhead, degrading inference efficiency. To address these accuracy and efficiency challenges, we observe that the model's internal signals can indicate whether a visual step will benefit reasoning before the entire visual generation is completed. Specifically, this work identifies two internal signals: 1) Generation Intent, which reflects whether the model has a concrete textual plan for what to draw, and 2) Visual Fidelity, which measures whether the visual generation remains grounded in the original input image. Leveraging these internal signals, we propose AdaViG, a training-free adaptive visual gating method for unified multimodal reasoning. AdaViG dynamically evaluates each triggered visual step at an early visual generation stage and aborts it when both signals are weak, thereby preventing misleading visual evidence from entering the reasoning trace while avoiding unnecessary computation. Comprehensive experiments demonstrate that AdaViG improves accuracy by up to 5.7% while reducing visual generation FLOPs by 25.0%-91.0% and wall-clock latency by 15.4%-45.6%.

Figures

Figures reproduced from arXiv: 2607.10004 by Didi Zhu, Guanxi Lu, Hao Mark Chen, Hongxiang Fan, Jiankang Deng, Quan Deng, Wenxi Gao, Zhican Wang.

Figure 1
Figure 1. Figure 1: A representative harmful generation case from MathVision under the fixed MathCanvas [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Accuracy and efficiency im￾provements over MathCanvas. However, the decision of whether to emit a textual or vi￾sual step in UMMs with CoT is primarily determined by the next action token. This paradigm lacks an explicit mech￾anism for assessing whether a visual step would actually improve reasoning quality. We identify two systemic issues with this design: (1) Harmful Generation Error. Self￾generated visu… view at source ↗
Figure 3
Figure 3. Figure 3: AdaViG uses early diffusion signals to predict generation utility. Left: A harmful trace has low I (e.g., no clear drawing plan in the text) and low F (e.g., the generated diagram drifts from the input image), placing the case in the abort region; the generated image distorts the input 3 × 3 lattice into a 2 × 2 pattern with extraneous polygons. Middle: The joint distribution of (I, F) at diffusion step t … view at source ↗
Figure 6
Figure 6. Figure 6: Generated images can mislead later reasoning. MathCanvas commits a perception drift, generating an irregular non-closed polygon that distorts the input. This induces an attention drift: post-image text attends more to the generated image (Orange) than to the input (Blue) in 92% of tokens, with the largest gap in the early span. Heatmaps show attention concentrating on irrelevant edges, propagating erroneou… view at source ↗
Figure 7
Figure 7. Figure 7: Relative latency and KV-cache memory breakdown across benchmarks. ! ,#  ,#$+, /'  ,# -"   [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Early aborting step trade-off [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Generated images attract more attention than input images during later text reasoning [PITH_FULL_IMAGE:figures/full_fig_p013_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Helpful case attends more to text reasoning and input image ViT than harmful case. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Case study between our method and MathCanvas. [PITH_FULL_IMAGE:figures/full_fig_p014_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Case study between our method and MathCanvas. [PITH_FULL_IMAGE:figures/full_fig_p015_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Case study between our method and MathCanvas. [PITH_FULL_IMAGE:figures/full_fig_p016_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 20 linked inside Pith

  1. [1]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025

    2025

  2. [2]

    Reasoning in the dark: Interleaved vision-text reasoning in latent space.arXiv preprint arXiv:2510.12603, 2025

    Chao Chen, Zhixin Ma, Yongqi Li, Yupeng Hu, Yinwei Wei, Wenjie Li, and Liqiang Nie. Reasoning in the dark: Interleaved vision-text reasoning in latent space.arXiv preprint arXiv:2510.12603, 2025

    arXiv 2025

  3. [3]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling.arXiv preprint arXiv:2501.17811, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning.arXiv preprint arXiv:2506.05331, 2025

    Xinyan Chen, Renrui Zhang, Dongzhi Jiang, Aojun Zhou, Shilin Yan, Weifeng Lin, and Hongsheng Li. Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning.arXiv preprint arXiv:2506.05331, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Codeplot-cot: Mathematical visual reasoning by thinking with code-driven images.arXiv preprint arXiv:2510.11718, 2025

    Chengqi Duan, Kaiyue Sun, Rongyao Fang, Manyuan Zhang, Yan Feng, Ying Luo, Yufang Liu, Ke Wang, Peng Pei, Xunliang Cai, et al. Codeplot-cot: Mathematical visual reasoning by thinking with code-driven images.arXiv preprint arXiv:2510.11718, 2025

    arXiv 2025

  8. [8]

    Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing.arXiv preprint arXiv:2503.10639, 2025

    Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing.arXiv preprint arXiv:2503.10639, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Interleaved-modal chain-of-thought

    Jun Gao, Yongqi Li, Ziqiang Cao, and Wenjie Li. Interleaved-modal chain-of-thought. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 19520–19529, 2025

    2025

  10. [10]

    Can we generate images with cot? let’s verify and reinforce image generation step by step.arXiv preprint arXiv:2501.13926, 2025

    Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Rui Huang, Haoquan Zhang, Manyuan Zhang, Jiaming Liu, Shanghang Zhang, Peng Gao, et al. Can we generate images with cot? let’s verify and reinforce image generation step by step.arXiv preprint arXiv:2501.13926, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Visual programming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14953–14962, 2023

    2023

  12. [12]

    Unicorn: Towards self-improving unified multimodal models through self-generated supervision.arXiv preprint arXiv:2601.03193, 2026

    Ruiyan Han, Zhen Fang, XinYu Sun, Yuchen Ma, Ziheng Wang, Yu Zeng, Zehui Chen, Lin Chen, Wenxuan Huang, Wei-Jie Xu, et al. Unicorn: Towards self-improving unified multimodal models through self-generated supervision.arXiv preprint arXiv:2601.03193, 2026

    arXiv 2026

  13. [13]

    Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper...

    2024

  14. [14]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models.Advances in Neural Information Processing Systems, 37:139348–139379, 2024

    Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models.Advances in Neural Information Processing Systems, 37:139348–139379, 2024. 10

    2024

  15. [15]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  16. [16]

    Zebra-cot: A dataset for interleaved vision language reasoning.arXiv preprint arXiv:2507.16746, 2025

    Ang Li, Charles Wang, Deqing Fu, Kaiyu Yue, Zikui Cai, Wang Bill Zhu, Ollie Liu, Peng Guo, Willie Neiswanger, Furong Huang, et al. Zebra-cot: A dataset for interleaved vision language reasoning.arXiv preprint arXiv:2507.16746, 2025

    arXiv 2025

  17. [17]

    Imagine while reasoning in space: Multimodal visualization-of-thought.arXiv preprint arXiv:2501.07542, 2025

    Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vuli ´c, and Furu Wei. Imagine while reasoning in space: Multimodal visualization-of-thought.arXiv preprint arXiv:2501.07542, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Aimcot: Active information-driven multimodal chain-of-thought for vision-language reasoning.arXiv preprint arXiv:2509.25699, 2025

    Xiping Li and Jianghong Ma. Aimcot: Active information-driven multimodal chain-of-thought for vision-language reasoning.arXiv preprint arXiv:2509.25699, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Rea- soning within the mind: Dynamic multimodal interleaving in latent space.arXiv preprint arXiv:2512.12623, 2025

    Chengzhi Liu, Yuzhe Yang, Yue Fan, Qingyue Wei, Sheng Liu, and Xin Eric Wang. Rea- soning within the mind: Dynamic multimodal interleaving in latent space.arXiv preprint arXiv:2512.12623, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Let’s think with images efficiently! an interleaved-modal chain-of-thought reasoning framework with dynamic and precise visual thoughts

    Xu Liu, Yongheng Zhang, Qiguang Chen, Yao Li, Sheng Wang, and Libo Qin. Let’s think with images efficiently! an interleaved-modal chain-of-thought reasoning framework with dynamic and precise visual thoughts. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 32213–32221, 2026

    2026

  21. [21]

    Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts.arXiv preprint arXiv:2310.02255, 2023

    Pith/arXiv arXiv 2023

  22. [22]

    Unirl: Self-improving unified multimodal models via supervised and reinforcement learning.arXiv preprint arXiv:2505.23380, 2025

    Weijia Mao, Zhenheng Yang, and Mike Zheng Shou. Unirl: Self-improving unified multimodal models via supervised and reinforcement learning.arXiv preprint arXiv:2505.23380, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning?, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achieve human-like mathematical reasoning?, 2024

    2024

  24. [24]

    Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration

    Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. Zoomeye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 6613–6629, 2025

    2025

  25. [25]

    Mathcanvas: Intrinsic visual chain-of-thought for multimodal mathematical reasoning.arXiv preprint arXiv:2510.14958, 2025

    Weikang Shi, Aldrich Yu, Rongyao Fang, Houxing Ren, Ke Wang, Aojun Zhou, Changyao Tian, Xinyu Fu, Yuxuan Hu, Zimu Lu, et al. Mathcanvas: Intrinsic visual chain-of-thought for multimodal mathematical reasoning.arXiv preprint arXiv:2510.14958, 2025

    arXiv 2025

  26. [26]

    Generation enhances understanding in unified multimodal models via multi- representation generation.arXiv preprint arXiv:2601.21406, 2026

    Zihan Su, Hongyang Wei, Kangrui Cen, Yong Wang, Guanhua Chen, Chun Yuan, and Xi- angxiang Chu. Generation enhances understanding in unified multimodal models via multi- representation generation.arXiv preprint arXiv:2601.21406, 2026

    Pith/arXiv arXiv 2026

  27. [27]

    Llamav-o1: Rethinking step-by-step visual reasoning in llms

    Omkar Thawakar, Dinura Dissanayake, Ketan Pravin More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Ilmuz Zaman Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-o1: Rethinking step-by-step visual reasoning in llms. InFindings of the Association for Computational Linguistics: ACL 2025, pages 24290–24315, 2025

    2025

  28. [28]

    Unirl-zero: Reinforcement learning on unified models with joint language model and diffusion model experts.arXiv preprint arXiv:2510.17937, 2025

    Fu-Yun Wang, Han Zhang, Michael Gharbi, Hongsheng Li, and Taesung Park. Unirl-zero: Reinforcement learning on unified models with joint language model and diffusion model experts.arXiv preprint arXiv:2510.17937, 2025

    arXiv 2025

  29. [29]

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning.arXiv preprint arXiv:2505.15966, 2025

    Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning.arXiv preprint arXiv:2505.15966, 2025. 11

    Pith/arXiv arXiv 2025

  30. [30]

    Measuring multimodal mathematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 37:95095–95169, 2024

    2024

  31. [31]

    Mathcoder-vl: Bridging vision and code for en- hanced multimodal mathematical reasoning

    Ke Wang, Junting Pan, Linda Wei, Aojun Zhou, Weikang Shi, Zimu Lu, Han Xiao, Yunqiao Yang, Houxing Ren, Mingjie Zhan, et al. Mathcoder-vl: Bridging vision and code for en- hanced multimodal mathematical reasoning. InFindings of the Association for Computational Linguistics: ACL 2025, pages 2505–2534, 2025

    2025

  32. [32]

    Emu3: Next-token prediction is all you need

    Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Visuothink: Empowering lvlm reasoning with multimodal tree search

    Yikun Wang, Siyin Wang, Qinyuan Cheng, Zhaoye Fei, Liang Ding, Qipeng Guo, Dacheng Tao, and Xipeng Qiu. Visuothink: Empowering lvlm reasoning with multimodal tree search. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 21707–21719, 2025

    2025

  34. [34]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  35. [35]

    Muse-vl: Modeling unified vlm through semantic discrete encoding

    Rongchang Xie, Chen Du, Ping Song, and Chang Liu. Muse-vl: Modeling unified vlm through semantic discrete encoding. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 24135–24146, 2025

    2025

  36. [36]

    Llava-cot: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2087–2098, 2025

    2087

  37. [37]

    Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search.arXiv preprint arXiv:2412.18319, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Provision: Programmatically scaling vision-centric instruction data for multimodal language models.arXiv preprint arXiv:2412.07012, 2024

    Jieyu Zhang, Le Xue, Linxin Song, Jun Wang, Weikai Huang, Manli Shu, An Yan, Zixian Ma, Juan Carlos Niebles, Silvio Savarese, et al. Provision: Programmatically scaling vision-centric instruction data for multimodal language models.arXiv preprint arXiv:2412.07012, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal llm truly see the diagrams in visual math problems? InEuropean Conference on Computer Vision, pages 169–186. Springer, 2024

    2024

  40. [40]

    Mul- timodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923, 2023

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923, 2023

    Pith/arXiv arXiv 2023

  41. [41]

    Unified multimodal understanding and generation models: Advances, challenges, and opportunities.arXiv preprint arXiv:2505.02567, 2025

    Shanshan Zhao, Xinjie Zhang, Jintao Guo, Jiakui Hu, Lunhao Duan, Minghao Fu, Yong Xien Chng, Guo-Hua Wang, Qing-Guo Chen, Zhao Xu, et al. Unified multimodal understanding and generation models: Advances, challenges, and opportunities.arXiv preprint arXiv:2505.02567, 2025

    arXiv 2025

  42. [42]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    ×1×1−!"×(2−1)

    Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models.arXiv preprint arXiv:2411.00836, 2024. 12 A Appendix A.1 Detailed Analysis on How Generated Images Mislead Reasoning Section 5.1 shows a single case where a generated image...

    Pith/arXiv arXiv 2024