Pith. sign in

REVIEW 3 major objections 109 references

Multimodal models can internalize visual tools and self-regulate when to use them, matching external-tool methods on fine-grained perception while cutting inference latency by up to ~86%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 07:00 UTC pith:ASW5BTKQ

load-bearing objection Practical Pareto win: discrete implicit tool tokens + NTG/MC-Reward cut TwI latency ~86% at 8K while beating larger models on high-res perception; self-reg proxy is hand-tuned but numbers hold. the 3 major comments →

arxiv 2607.11106 v1 pith:ASW5BTKQ submitted 2026-07-13 cs.CV

Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools

Xiuwei Chen , Quanlin Chen , Wentao Hu , Zisheng Chen , Kun Xiang , Zehua Ma , Mingyang Zhang , Jianhua Han
show 3 more authors
Hanhui Li Hang Xu Xiaodan Liang
This is my paper
classification cs.CV
keywords multimodal large language modelsThinking with Imagesimplicit visual toolsself-regulated tool usechain-of-thoughtreinforcement learninginference efficiencyfine-grained visual perception
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Thinking with Images has improved fine-grained visual perception by letting models zoom, crop, and otherwise manipulate images, but each external tool call and re-encoding costs latency and compute. This paper argues that the same behaviors can be folded into the model as implicit tool tokens, and that the model can learn a cognitive boundary so it only invokes those tools when internal knowledge is not enough. Training proceeds in two stages: supervised fine-tuning on formalized chains of thought that mix tool and no-tool trajectories, then a self-regulated reward that penalizes redundant tool use. The result is a model that routes between parametric knowledge and internalized visual operations, reporting state-of-the-art perception accuracy with single-pass inference speed. A sympathetic reader cares because it offers a practical path to strong visual reasoning without the I/O tax of agentic tool stacks.

Core claim

BEE shows that visual tool invocation can be trained as an implicit, self-regulated behavior inside an MLLM: formalized CoT supervision activates tool slots and adaptive switching, and a self-regulated reward (guided by Net Tool Gain) teaches the model to invoke those slots only when group rollout accuracy indicates the task exceeds its current capability, yielding high fine-grained perception performance with far lower inference latency than external-tool Thinking-with-Images systems.

What carries the argument

Self-regulated implicit visual tools (IvT) plus MC-Reward: discrete tool-slot tokens stand in for operations such as zoom or crop; Net Tool Gain measures effective versus redundant tool use; MC-Reward uses online group-average accuracy p against a threshold τ to encourage tools on hard queries and penalize them on easy ones.

Load-bearing premise

That the average accuracy of a small group of sampled rollouts is a trustworthy live signal of whether a problem is inside or outside the model’s capability, so a fixed threshold cleanly tells the model when tools help and when they waste effort.

What would settle it

On a held-out high-resolution perception set, disable the self-regulated reward (or fix tool use always-on / always-off) and check whether Net Tool Gain and end-to-end accuracy-plus-latency still improve together relative to the SFT-only model and to an external-tool baseline; if accuracy rises only when tools fire on problems the base model already solves, the boundary-calibration claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Fine-grained perception at high resolution can be served in a single forward pass without sandbox APIs or repeated image re-encoding.
  • Redundant tool calls can be measured (Net Tool Gain) and trained down, lowering cost per sample while preserving or improving accuracy.
  • Implicit tool slots of modest length (even a handful of discrete tokens) can substitute for explicit zoom/crop/rotation pipelines on the evaluated benchmarks.
  • The same self-regulated routing can keep general reasoning competitive rather than trading it away for perception gains.
  • Deployment simplifies to a standard instruct MLLM stack, with reported throughput and cost advantages over multi-turn agentic tool systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If group-rollout accuracy is a good boundary signal, similar self-regulated gating could apply to other latent or tool-augmented modalities (search, code execution) without hard-coding budgets.
  • The largest reported gains on monitoring and driving-style subtasks suggest the paradigm is most valuable where base models under-perceive dense or distant objects rather than where OCR is already strong.
  • Failure modes on multi-object scenes and sparse logical CoT imply that data coverage, not only the reward shape, still bounds when self-regulation helps versus when the model skips needed tools or steps.
  • A natural next test is whether NTG-style rewards remain stable as base models get stronger and the easy/hard mix of a fixed benchmark drifts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The paper proposes BEE, a two-stage training paradigm that internalizes visual tool operations (zoom, crop, rotate, etc.) as discrete implicit tool-slot tokens rather than external API calls with image re-encoding. Stage 1 uses formalized CoT SFT on a 320K mixture of tool and no-tool trajectories to activate tool-slot generation and basic adaptive switching. Stage 2 applies DAPO with a composite reward that includes MC-Reward, which shapes tool propensity from the online group-average accuracy p relative to a fixed threshold τ (default 2/3), so tools are encouraged only when internal knowledge appears insufficient. The authors introduce Net Tool Gain (NTG) to diagnose redundant tool use after SFT+simple RL, then show that MC-Reward raises NTG from 6.2% to 66% while cutting easy-problem tool rates to ~0.2%. Empirically, BEE-4B/7B/8B improve fine-grained perception (V*, HRBench, MME-RealWorld) over strong open-source and several closed-source baselines, stay competitive on general/OOD reasoning, and report large inference-speed gains (up to ~86.2% vs DeepEyes at 8K) with lower deployment cost.

Significance. If the results hold under broader scrutiny, BEE is a practically important step for multimodal reasoning: it shows that much of the accuracy benefit of “Thinking with Images” can be retained without the I/O and re-encoding tax of explicit tool agents, while adding an explicit self-regulation objective rather than a fixed latent budget. The combination of formalized tool-slot CoT, the NTG diagnostic, system-level latency/throughput/cost tables, multi-backbone scaling (4B/7B/8B), and controlled ablations on slot format, token length, and MC-Reward is stronger engineering evidence than many concurrent latent-reasoning papers. The work is therefore significant for efficient MLLM deployment and for research on capability-aware tool routing, even if the metacognitive interpretation of p is only partially validated.

major comments (3)
  1. Section 3.3.1 and the definition of p: the central “self-regulated knowledge routing” claim rests on treating group-average accuracy over G rollouts as an online capability boundary, then gating tool advantage via fixed τ=2/3 and the hand-specified continuous reward in Eq. (10). The paper shows that easy-problem tool rates fall and NTG rises (Fig. 8, Table 7), but does not independently test whether p is a reliable metacognitive signal versus a noisy co-evolving statistic that mainly induces a global tool-frequency prior. A load-bearing addition would be (i) held-out calibration of p against true difficulty (e.g., frozen-base accuracy or human difficulty bins), (ii) an ablation that replaces p with a non-capability signal (random or length-based) while keeping the same reward shape, and/or (iii) reporting tool-use conditional accuracy stratified by p at evaluation time. Without this, the
  2. Eqs. (9)–(10) and Appendix A.1: MC-Reward coefficients (α1,β1,α2,β2,γ,μ) and the cosine buffer are hand-chosen after a theoretical admissible-region argument. Table 17 varies τ but does not report a comparable sensitivity study for the reward coefficients or for G. Because Stage-2 policy updates are driven by this shaped advantage, the manuscript should either (a) show that nearby coefficient choices yield the same qualitative routing and NTG gains, or (b) describe a selection protocol (grid search on a validation split, automatic calibration) so the result is not tied to one opaque reward surface. This is especially important given the claim that continuous shaping is necessary to avoid collapse of tool use.
  3. Tables 1–2 and the efficiency protocol (Sec. 4.1, Table 4, Fig. 5): the Pareto claim versus agentic TwI methods is central, yet several comparisons mix different codebases, tool sandboxes, and (for Thyme) a non-vLLM stack. The paper already standardizes hardware and BF16 batch-1 settings, which is good, but it should also report (i) matched maximum generation length / decoding hyperparameters for all agentic baselines, (ii) tool-call count and re-encode count per sample, and (iii) at least mean±std over multiple seeds or evaluation shuffles for the main BEE numbers. Without variance and fully matched agentic budgets, the ~86.2% acceleration figure and some accuracy margins could be partly protocol-dependent.

Circularity Check

0 steps flagged

No load-bearing circularity: NTG is a post-hoc diagnostic, MC-Reward is a hand-designed online shaping signal, and final claims rest on external public-benchmark measurements rather than self-fitted quantities.

full rationale

BEE is an empirical two-stage training method (formalized CoT SFT on constructed trajectories + DAPO RL with composite rewards). The central performance and efficiency claims (Tables 1–2, Fig. 1, 86.2 % acceleration) are measured on held-out public benchmarks (V*, HRBench, MME-RealWorld, LogicVista, POPE) against external baselines, using rule-based or independent judge models (Qwen3-VL). NTG (Eq. 1) is defined from a base-vs-tool comparison on a test set solely to diagnose redundant tool use after Stage 1 (NTG = 6.2 % for BEE-Static); it is never fitted as a target and is only re-measured post-hoc after MC-Reward (NTG = 66 %). MC-Reward itself (Sec. 3.3, Eqs. 4–10) uses the online group-average accuracy p over G rollouts as a dynamic capability proxy against a fixed threshold τ = 2/3, with continuous linear+cosine coefficients chosen analytically in the appendix to satisfy the desired sign conditions of Eq. 5; the coefficients are not optimized against the final reported benchmark numbers, and the paper notes that any similar-trend parameters work. Tool-call-rate trajectories (Fig. 8) and ablations (Tables 5–10, 17–18) are likewise observational. Residual citations of the authors’ own prior latent-reasoning work (EVA) appear only as related-work baselines, not as uniqueness theorems or premises that force the present results. Consequently the derivation chain does not reduce by construction to its inputs; the only residual is ordinary non-load-bearing self-citation of prior latent methods, warranting a score of 1 rather than 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central claim rests on a small set of hand-chosen scalars (τ, reward coefficients) and on the modeling assumption that discrete semantic tokens can stand in for real visual operations. No new physical entities are postulated; the free parameters are ordinary RL hyper-parameters whose values are stated and ablated.

free parameters (4)
  • capability threshold τ = 2/3
    Fixed at 2/3; decides when tool use is encouraged versus penalized. Ablated in Table 17 but still a free design choice.
  • MC-Reward coefficients (α1,β1,α2,β2,γ,μ) = α1=0.3, β1=-0.2, α2=-0.5, β2=0.3, γ=0.3, μ=0.15
    Hand-selected continuous functions that shape tool advantage (Eq. 10). Alternative values are said to work if the qualitative trend is preserved, but the exact numbers are free parameters.
  • number of implicit tokens K = typically 4–16
    Length of the tool-slot sequence; ablated in Fig. 6 (4–64 tokens).
  • rollout group size G = 8
    Number of trajectories used to estimate p; set to 8 in the RL hyper-parameter table.
axioms (4)
  • domain assumption Group-average accuracy p over G rollouts is a faithful online estimate of the model’s current capability boundary.
    Section 3.3.1; without this proxy the MC-Reward cannot decide when to encourage or penalize tools.
  • domain assumption Discrete semantic tokens produced by a VQ-style tokenizer adequately simulate the information gained from real visual tool operations (crop, zoom, rotate, etc.).
    Section 3.1 and Appendix A.3; the entire latency claim rests on never re-encoding real images.
  • domain assumption Mixed formalized CoT trajectories (with and without tool slots) are sufficient to activate both tool representations and adaptive switching.
    Stage-1 design (Section 3.2.1); if the mixture is poorly balanced the later RL stage cannot recover a clean boundary.
  • standard math Standard DAPO / GRPO-style policy gradient with the composite reward converges to a useful self-regulated policy.
    Optimization objective Eq. 3; ordinary RL assumption.
invented entities (3)
  • Implicit visual tool (IvT) / tool slot tokens independent evidence
    purpose: Replace external image re-encoding with autoregressive discrete tokens that stand for tool outputs.
    Core technical invention; independent evidence is the ablation showing that replacing them with <unk> or original-image tokens hurts accuracy (Table 5).
  • Net Tool Gain (NTG) metric independent evidence
    purpose: Quantify the marginal benefit of tool use after subtracting base-model competence and tool-induced errors.
    Diagnostic introduced in Section 3.2.2; used to motivate MC-Reward. Falsifiable on any held-out set.
  • MC-Reward (self-regulated confidence calibration) independent evidence
    purpose: Continuous, difficulty-aware reward that encourages tools only when p < τ.
    Section 3.3; the shaping functions are new but ordinary RL constructs.

pith-pipeline@v1.1.0-grok45 · 45400 in / 3157 out tokens · 41167 ms · 2026-07-14T07:00:48.322153+00:00 · methodology

0 comments
read the original abstract

Recent multimodal large language models (MLLMs) have made remarkable progress on fine-grained perception tasks under the "Thinking with Images" (TwI) paradigm by iteratively performing various visual tool operations. However, this paradigm relies heavily on frequent external tool calls and repeated image re-encoding, which leads to substantial computational overhead and inference latency. To address these issues, we propose Beyond the Eye (BEE), a novel implicit visual tool paradigm centered on self-regulated capability. BEE directly incorporates visual tool invocation behaviors into the training objective and encourages the model to develop a self-regulated invocation mechanism. This design enables the model to adaptively balance internal knowledge and implicit tools, avoiding redundant tool usage while substantially reducing inference latency. Specifically, BEE involves a two-stage training process: (1) Formalized Chain-of-Thought (CoT) Supervised Fine-tuning (SFT). We construct CoT trajectories with structured tool slots and mixed invocation states. This stage activates the model's implicit tool representations and adaptive switching capability. (2) Self-regulated Reward-Driven Alignment. To address redundant tool usage caused by ambiguous cognitive boundaries, we first introduce the Net Tool Gain (NTG) metric to quantify this phenomenon. Based on this observation, we further propose a self-regulated reward mechanism. This mechanism penalizes ineffective tool dependency and encourages the model to perform knowledge routing, ensuring that implicit tools are invoked only when the model's internal knowledge is insufficient. BEE achieves state-of-the-art performance in fine-grained visual perception while remaining competitive in general reasoning tasks and achieving substantial gains in inference efficiency.

Figures

Figures reproduced from arXiv: 2607.11106 by Hang Xu, Hanhui Li, Jianhua Han, Kun Xiang, Mingyang Zhang, Quanlin Chen, Wentao Hu, Xiaodan Liang, Xiuwei Chen, Zehua Ma, Zisheng Chen.

Figure 1
Figure 1. Figure 1: (Left) Average scores across multimodal perception benchmarks, including V* Bench, HRBench, MME-RealWorld, MME-RealWorld-Lite. BEE demonstrates superior performance compared with much larger open-source models (e.g., Qwen3-VL-235B) and closed-source models (e.g., Gemini-2.5-Pro). (Right) Inference time comparison (s/sample) across different resolutions. BEE exhibits higher inference efficiency than TwI and… view at source ↗
Figure 4
Figure 4. Figure 4: To fundamentally overcome this limitation, we introduce the Second training stage, namely self-regulated reward-driven alignment. This stage aims to mitigate the model’s overfitting to pre-constructed CoT labels and further improve its self￾regulated capabilities, thereby enhancing complex reasoning and generalization. Based on this observation, we propose a self-regulated reward mechanism. By penalizing i… view at source ↗
Figure 2
Figure 2. Figure 2: (a) Overview of BEE. BEE adaptively selects between implicit visual tool behavior and direct reasoning based on the model’s current capability. (b) Two-Stage Training Pipeline. In Stage 1 Formalized CoT Supervised Fine-Tuning, the model is trained using pre-constructed formalized CoT labels to activate the model’s implicit tool representations, adaptive switching capability, and basic reasoning ability. In… view at source ↗
Figure 3
Figure 3. Figure 3: Statistics of SFT training data. Top: The distribution of tasks across three main categories. Bottom: The data length statistic. are activated only when the model’s internal knowledge is insufficient, leading to more efficient and adaptive reasoning. BEE supports self-regulated optimization of its reasoning process. Compared with conventional explicit tool invocation methods based on external APIs, this im… view at source ↗
Figure 4
Figure 4. Figure 4: Data-driven comparison of performance and tool usage rate. BEE-7B-Static still makes extensive use of tools even within the range of its own capabilities (e.g., problems solvable by the base model), leading to unnecessary resource consumption and indicating a lack of capability boundary awareness. 3.2.2 Net Tool Gain (NTG) Metric To understand how the model uses external tools, we first examine the distrib… view at source ↗
Figure 5
Figure 5. Figure 5: Benchmark accuracy versus inference speed. Our models achieve higher accuracy than the base models and agentic baselines while maintaining the efficiency of single￾pass inference. evaluate object hallucination in MLLMs. We set the threshold τ to 2/3 in our experiments. Baselines. To ensure a comprehensive evaluation, we bench￾mark our BEE against five distinct categories of baselines. (i) closed-source fro… view at source ↗
Figure 6
Figure 6. Figure 6: Effect of token length. TABLE 6: Performance comparison on the MME-Real-Lite. BEE shows larger improvements on more challenging perception and reasoning tasks, as well as monitoring and autonomous driving tasks, where the baseline performs relatively poorly. Perception Model Monitoring Autonomous Driving OCR Diagram and Table Remote Sensing Overall Qwen2.5-VL-7B [1] 29.1 31.7 90.4 85.0 44.0 49.7 BEE-7B 41.… view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of the MC-Reward curve. MC-Reward is defined over two continuous regions. In the easy-problem region, using tools always yields a lower reward than direct reasoning. In the difficult-problem region, using tools consistently yields a higher reward than direct reasoning. 100% 0.4% 0.2% 100% 25% 41% [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of BEE tool calling rate across tasks during training. During the early stage of SFT training, the model makes many redundant tool calls, with the call rate approaching 100% on both easy and difficult problems. As training progresses, the model quickly adapts its tool-use behavior and gradually reduces unnecessary calls on easy problems, eventually reaching a call rate of around 0.2%. On difficu… view at source ↗
Figure 9
Figure 9. Figure 9: Performance comparison of different components across tasks. The introduction of MC-Reward enables the model to use tools more appropriately and improves its self￾regulated ability. 5.2 Analysis of Implicit Tool Usage [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: illustrates the effectiveness of implicit tokens using two ablation strategies. Original Masked Replaced Attended Masked i j i j i i [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Training framework of the semantic tokenizer. Eθ(·): F = Eθ(B). The features are projected via an MLP gϕ(·) into a quantization space: H = gϕ(F). We then apply vector quantization with a learnable codebook Z = {zk} K k=1, where feature is mapped to its nearest code: \mathbf {Z_q}, {I_q} = \mathop {\arg \min }\limits _{k \in \{1,\dots ,K\}} \| {H} - \mathbf {Z[k]} \|_2^2 (26) Where K is the codebook size, … view at source ↗
Figure 12
Figure 12. Figure 12: Variant forms of tool slot in the training phase [PITH_FULL_IMAGE:figures/full_fig_p024_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Data format of the final SFT training set [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Visualization of perception and reasoning tokens. (a): T-SNE visualization without text-vision decoupling. (b) T-SNE visualization with text-vision decoupling [PITH_FULL_IMAGE:figures/full_fig_p030_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: The training reward of Qwen2.5-VL-7B [PITH_FULL_IMAGE:figures/full_fig_p031_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Tool usage rate and accuracy over time [PITH_FULL_IMAGE:figures/full_fig_p031_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Number of problems of different difficulties during training [PITH_FULL_IMAGE:figures/full_fig_p032_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Mathematical reasoning case [PITH_FULL_IMAGE:figures/full_fig_p035_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: General reasoning case [PITH_FULL_IMAGE:figures/full_fig_p036_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Real-world understanding case [PITH_FULL_IMAGE:figures/full_fig_p037_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Real-world understanding case [PITH_FULL_IMAGE:figures/full_fig_p038_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Real-world understanding case [PITH_FULL_IMAGE:figures/full_fig_p039_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Real-world understanding case [PITH_FULL_IMAGE:figures/full_fig_p040_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Medical case [PITH_FULL_IMAGE:figures/full_fig_p041_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Medical case [PITH_FULL_IMAGE:figures/full_fig_p042_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: Autonomous driving case [PITH_FULL_IMAGE:figures/full_fig_p043_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Logical reasoning case [PITH_FULL_IMAGE:figures/full_fig_p044_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: Logical reasoning case [PITH_FULL_IMAGE:figures/full_fig_p045_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Autonomous driving case [PITH_FULL_IMAGE:figures/full_fig_p046_29.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

109 extracted references · 16 linked inside Pith

  1. [1]

    Qwen2.5-vl technical report,

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P . Wang, S. Wang, J. Tanget al., “Qwen2.5-vl technical report,”arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Qwen3-vl technical report,

    S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Geet al., “Qwen3-vl technical report,”arXiv preprint arXiv:2511.21631, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” inProceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  4. [4]

    Gpt-4o system card,

    A. Hurst, A. Lerer, A. P . Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radfordet al., “Gpt-4o system card,”arXiv preprint arXiv:2410.21276, 2024

    Pith/arXiv arXiv 2024

  5. [5]

    Visual instruc- tion tuning,

    H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruc- tion tuning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  6. [6]

    Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. C. H. Hoi, “Blip- 2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” inProceedings of the International Conference on Learning Representations (ICLR), 2023

    2023

  7. [7]

    Instructblip: Towards general-purpose vision-language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. A. Li, P . Fung, and S. C. H. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  8. [8]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

    G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosenet al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,”arXiv preprint arXiv:2507.06261, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    From system 1 to system 2: A survey of reasoning large language models,

    Z.-Z. Li, D. Zhang, M.-L. Zhang, J. Zhang, Z. Liu, Y. Yao, H. Xu, J. Zheng, P .-J. Wang, X. Chen, Y. Zhang, F. Yin, J. Dong, Z. Guo, L. Song, and C.-L. Liu, “From system 1 to system 2: A survey of reasoning large language models,”IEEE Transactions on Pattern Analysis and Machine Intelligence (TP AMI), vol. 48, pp. 3335–3354, 2025

    2025

  10. [10]

    Multimodal chain-of-thought rea- soning: A comprehensive survey,

    Y. Wang, S. Wu, Y. Zhang, S. Yan, Z. Liu, J. Luo, and H. Fei, “Multimodal chain-of-thought rea- soning: A comprehensive survey,”arXiv preprint arXiv:2503.12605, 2025

    Pith/arXiv arXiv 2025

  11. [11]

    Improve vision language model chain-of-thought reasoning,

    R. Zhang, B. Zhang, Y. Li, H. Zhang, Z. Sun, Z. Gan, Y. Yang, R. Pang, and Y. Yang, “Improve vision language model chain-of-thought reasoning,” inPro- ceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  12. [12]

    Insight-v: Exploring long-chain visual reasoning with multimodal large language models,

    Y. Dong, Z. Liu, H.-L. Sun, J. Yang, W. Hu, Y. Rao, and Z. Liu, “Insight-v: Exploring long-chain visual reasoning with multimodal large language models,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  13. [13]

    Multimodal chain-of-thought reasoning in language models,

    Z. Zhang, A. Zhang, M. Li, H. Zhao, G. Karypis, and A. J. Smola, “Multimodal chain-of-thought reasoning in language models,”Transactions on Machine Learning Research (TMLR), 2023

    2023

  14. [14]

    R1-zero’s

    H. Zhou, X. Li, R. Wang, M. Cheng, T. Zhou, and C.-J. Hsieh, “R1-zero’s ”aha moment” in visual reasoning on a 2b non-sft model,”arXiv preprint arXiv:2503.05132, 2025

    Pith/arXiv arXiv 2025

  15. [15]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization,

    J. Zhang, J. Huang, H. Yao, S. Liu, X. Zhang, S. Lu, and D. Tao, “R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization,” inProceedings of the IEEE Interna- tional Conference on Computer Vision (ICCV), 2025

    2025

  16. [16]

    R1- onevision: Advancing generalized multimodal reason- ing through cross-modal formalization,

    Y. Yang, X. He, H. Pan, X. Jiang, Y. Deng, X. Yang, H. Lu, D. Yin, F. Rao, M. Zhu, B. Zhang, and W. Chen, “R1- onevision: Advancing generalized multimodal reason- ing through cross-modal formalization,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2025

    2025

  17. [17]

    Grit: Teaching mllms to think with images,

    Y. Fan, X. He, D. Yang, K. Zheng, C.-C. Kuo, Y. Zheng, S. J. Narayanaraju, X. Guan, and X. E. Wang, “Grit: Teaching mllms to think with images,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  18. [18]

    Retool: Reinforcement learning for strategic tool use in llms,

    J. Feng, S. Huang, X. Qu, G. Zhang, Y. Qin, B. Zhong, C. Jiang, J. Chi, and W. Zhong, “Retool: Reinforcement learning for strategic tool use in llms,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  19. [19]

    Thyme: Think beyond images,

    Y.-F. Zhang, X. Lu, S. Yin, C. Fu, W. Chen, X. Hu, B. Wen, K. Jiang, C. Liu, T. Zhanget al., “Thyme: Think beyond images,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  20. [20]

    Mini-o3: Scaling up reasoning patterns and interaction turns for visual search,

    X. Lai, J. Li, W. Li, T. Liu, T. Li, and H. Zhao, “Mini-o3: Scaling up reasoning patterns and interaction turns for visual search,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  21. [21]

    Deepeyes: Incentivizing

    Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, C. Shen, and X. Yu, “Deepeyes: Incentivizing” think- ing with images” via reinforcement learning,” in Proceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  22. [22]

    Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl,

    X. Zhang, Z. Gao, B. Zhang, P . Li, X. Zhang, Y. Liu, T. Yuan, Y. Wu, Y. Jia, S.-C. Zhuet al., “Chain-of-focus: Adaptive visual search and zooming for multimodal reasoning via rl,” inProceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  23. [23]

    Visual sketchpad: Sketching as a visual chain of thought for multimodal language models,

    Y. Hu, W. Shi, X. Fu, D. Roth, M. Ostendorf, L. Zettle- moyer, N. A. Smith, and R. Krishna, “Visual sketchpad: Sketching as a visual chain of thought for multimodal language models,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  24. [24]

    Reason- ing beyond language: A comprehensive survey on latent chain-of-thought reasoning,

    X. Chen, A. Zhao, H. Xia, X. Lu, H. Wang, Y. Chen, W. Zhang, J. Wang, W. Li, and X. Shen, “Reason- ing beyond language: A comprehensive survey on latent chain-of-thought reasoning,”arXiv preprint 14 arXiv:2505.16782, 2025

    arXiv 2025

  25. [25]

    The latent space: Founda- tion, evolution, mechanism, ability, and outlook,

    X. Yu, Z. Chen, Y. Heet al., “The latent space: Founda- tion, evolution, mechanism, ability, and outlook,”arXiv preprint arXiv:2604.02029, 2026

    Pith/arXiv arXiv 2026

  26. [26]

    Monet: Reasoning in latent vi- sual space beyond image and language,

    Q. Wang, Y. Shi, Y. Wang, Y. Zhang, P . Wan, K. Gai, X. Ying, and Y. Wang, “Monet: Reasoning in latent vi- sual space beyond image and language,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  27. [27]

    Latent visual states for efficient multimodal reasoning,

    X. Chen, W. Hu, Y. Wang, Z. Chen, L. Zhang, K. Xiang, J. Han, H.-L. Zhen, J. Zou, H. Xuet al., “Latent visual states for efficient multimodal reasoning,”arXiv preprint arXiv:2606.24233, 2026

    Pith/arXiv arXiv 2026

  28. [28]

    Dapo: An open-source llm reinforcement learning system at scale,

    Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, T. Fan, G. Liuet al., “Dapo: An open-source llm reinforcement learning system at scale,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  29. [29]

    Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models,

    W. Wang, L. Ding, M. Zeng, X. Zhou, L. Shen, Y. Luo, W. Yu, and D. Tao, “Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models,” in Proceedings of the AAAI Conference on Artificial Intelli- gence (AAAI), 2025

    2025

  30. [30]

    Mme- realworld: Could your multimodal llm challenge high- resolution real-world scenarios that are difficult for humans?

    Y.-F. Zhang, H. Zhang, H. Tian, C. Fu, S. Zhang, J. Wu, F. Li, K. Wang, Q. Wen, Z. Zhanget al., “Mme- realworld: Could your multimodal llm challenge high- resolution real-world scenarios that are difficult for humans?” inProceedings of the International Conference on Learning Representations (ICLR), 2025

    2025

  31. [31]

    Kimi k2. 5: Visual agentic intelligence,

    K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chenet al., “Kimi k2. 5: Visual agentic intelligence,”arXiv preprint arXiv:2602.02276, 2026

    Pith/arXiv arXiv 2026

  32. [32]

    Openai o1 system card,

    A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El- Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carneyet al., “Openai o1 system card,”arXiv preprint arXiv:2412.16720, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Intern vl: Scaling up vision foundation models and aligning for generic visual- linguistic tasks,

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, Z. Muyan, Q. Zhang, X. Zhu, L. Lu, B. Li, P . Luo, T. Lu, Y. Qiao, and J. Dai, “Intern vl: Scaling up vision foundation models and aligning for generic visual- linguistic tasks,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  34. [34]

    Llava-onevision: Easy visual task transfer,

    B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, Y. Li, Z. Liu, and C. Li, “Llava-onevision: Easy visual task transfer,”Transactions on Machine Learning Research (TMLR), 2025

    2025

  35. [35]

    Mm-llms: Recent advances in multimodal large language models,

    D. Zhang, Y. Yu, J. Dong, C. Li, D. Su, C. Chu, and D. Yu, “Mm-llms: Recent advances in multimodal large language models,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    2024

  36. [36]

    Moe-llava: Mixture of experts for large vision-language models,

    B. Lin, Z. Tang, Y. Ye, J. Huang, J. Zhang, Y. Pang, P . Jin, M. Ning, J. Luo, and L. Yuan, “Moe-llava: Mixture of experts for large vision-language models,”IEEE Transactions on Multimedia (TMM), vol. 28, pp. 4408– 4419, 2026

    2026

  37. [37]

    Efficient multimodal large language models: a survey,

    Y. Jin, J. Li, Y. Liu, T. Gu, K. Wu, Z. Jiang, M. He, B. Zhao, X. Tan, Z. Gan, Y. Wang, C. Wang, and L. Ma, “Efficient multimodal large language models: a survey,” Visual Intelligence, vol. 3, 2024

    2024

  38. [38]

    Llava-cot: Let vision language models reason step-by- step,

    G. Xu, P . Jin, Z. Wu, H. Li, Y. Song, L. Sun, and L. Yuan, “Llava-cot: Let vision language models reason step-by- step,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2025

    2025

  39. [39]

    Atomthink: Multimodal slow thinking with atomic step reasoning,

    K. Xiang, Z. Liu, T. J. Zhang, Y. Huang, Y. Nie, K. Cai, Y. Yin, R. Huang, H. Li, Y. Zeng, Y.-J. Yuan, J. Han, L. Hong, H. Xu, and X. Liang, “Atomthink: Multimodal slow thinking with atomic step reasoning,”IEEE Transactions on Pattern Analysis and Machine Intelligence (TP AMI), vol. 48, no. 05, pp. 5725–5741, 2026

    2026

  40. [40]

    Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning,

    D. Zhang, J. Wu, J. Lei, T. Che, J. Li, T. Xie, X. Huang, S. Zhang, M. Pavone, Y. Li, W. Ouyang, and D. Zhou, “Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning,” in Proceedings of the Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), 2025

    2025

  41. [41]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning?

    R. Qiao, Q. Tan, G. Dong, M. MinhuiWu, C. Sun, X. Song, J. Wang, Z. GongQue, S. Lei, Y. Zhang, Z. Wei, M. Zhang, R. Qiao, X. Zong, Y. Xu, P . Yang, Z. Bao, M. Diao, C. Li, and H. Zhang, “We-math: Does your large multimodal model achieve human-like mathematical reasoning?” inProceedings of the Annual Meeting of the Association for Computational Linguistics...

    2025

  42. [42]

    Lmm-r1: Empowering 3b lmms with strong reasoning abili- ties through two-stage rule-based rl,

    Y. Peng, G. Zhang, M. Zhang, Z. You, J. Liu, Q. Zhu, K. Yang, X. Xu, X. Geng, and X. Yang, “Lmm-r1: Empowering 3b lmms with strong reasoning abili- ties through two-stage rule-based rl,”arXiv preprint arXiv:2503.07536, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Deepseek-r1 incentivizes reasoning in llms through reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, P . Wang, Q. Zhu, R. Xu, R. Zhanget al., “Deepseek-r1 incentivizes reasoning in llms through reinforcement learning,” Nature, vol. 645, pp. 633–638, 2025

    2025

  44. [44]

    Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?

    Y. Yue, Z. Chen, R. Lu, A. Zhao, Z. Wang, S. Song, and G. Huang, “Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  45. [45]

    Visual-rft: Visual reinforcement fine-tuning,

    Z. Liu, Z. Sun, Y. Zang, X. wen Dong, Y. Cao, H. Duan, D. Lin, and J. Wang, “Visual-rft: Visual reinforcement fine-tuning,” inProceedings of the IEEE International Conference on Computer Vision (ICCV), 2025

    2025

  46. [46]

    Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model,

    J. Hu, Y. Zhang, Q. Han, D. Jiang, X. Zhang, and H. yeung Shum, “Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  47. [47]

    Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning,

    H. Wang, C. Qu, Z. Huang, W. Chu, F. Lin, and W. Chen, “Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  48. [48]

    Group-in-group policy optimization for llm agent training,

    L. Feng, Z. Xue, T. Liu, and B. An, “Group-in-group policy optimization for llm agent training,” inProceed- ings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  49. [49]

    Ttrl: Test-time reinforcement learning,

    Y. Zuo, K. Zhang, S. Qu, L. Sheng, X. Zhu, Y. Zhang, B. Qi, Y. Sun, G. Cui, N. Ding, and B. Zhou, “Ttrl: Test-time reinforcement learning,” inProceedings of the Advances in Neural Information Processing Systems 15 (NeurIPS), 2025

    2025

  50. [50]

    Mmr1: Enhancing multi- modal reasoning with variance-aware sampling and open resources,

    S. Leng, J. Wang, J. Li, H. Zhang, Z. Hu, B. Zhang, Y. Jiang, H. Zhang, X. Li, L. Bing, D. Zhao, W. Lu, Y. Rong, A. Sun, and S. Lu, “Mmr1: Enhancing multi- modal reasoning with variance-aware sampling and open resources,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR) Findings, 2026

    2026

  51. [51]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models,

    W. Huang, B. Jia, Z. Zhai, S. Cao, Z. Ye, F. Zhao, Z. Xu, Y. Hu, and S. Lin, “Vision-r1: Incentivizing reasoning capability in multimodal large language models,” in Proceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  52. [52]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based rein- forcement learning,

    F. Meng, L. Du, Z. Liu, Z. Zhou, Q. Lu, D. Fu, T. Han, B. Shi, W. Wang, J. Heet al., “Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based rein- forcement learning,”Transactions on Machine Learning Research (TMLR), 2026

    2026

  53. [53]

    Mimo-vl technical report,

    X. L.-C. T. Z. Yue, Z. Lin, Y.-H. Song, W. Wang, S.- Q. Ren, S. Gu, S. Liet al., “Mimo-vl technical report,” arXiv preprint arXiv:2506.03569, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Kimi k2: Open agentic intelligence,

    K. Team, Y. Bai, Y. Bao, G. Chen, J. Chen, N. Chen, R. Chen, Y. Chen, Y. Chen, Y. Chenet al., “Kimi k2: Open agentic intelligence,”arXiv preprint arXiv:2507.20534, 2025

    Pith/arXiv arXiv 2025

  55. [55]

    Webwatcher: Breaking new frontier of vision-language deep research agent,

    X. Geng, P . Xia, Z. Zhang, X. Wang, Q. Wang, R. Ding, C. Wang, J. Wu, Y. Zhao, K. Liet al., “Webwatcher: Breaking new frontier of vision-language deep research agent,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  56. [56]

    Agent rl scaling law: Agent rl with spontaneous code execution for mathematical problem solving,

    X. Mai, H. Xu, Z.-Z. Li, W. Wang, J. Hu, Y. Zhang, W. Zhanget al., “Agent rl scaling law: Agent rl with spontaneous code execution for mathematical problem solving,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  57. [57]

    Webshaper: Agentically data synthesizing via information-seeking formalization,

    Z. Tao, J. Wu, W. Yin, J. Zhang, B. Li, H. Shen, K. Li, L. Zhang, X. Wang, Y. Jianget al., “Webshaper: Agentically data synthesizing via information-seeking formalization,” inProceedings of the International Confer- ence on Learning Representations (ICLR), 2026

    2026

  58. [58]

    Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning,

    Z. Xue, L. Zheng, Q. Liu, Y. Li, X. Zheng, Z. Ma, and B. An, “Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning,” inProceed- ings of the International Conference on Learning Represen- tations (ICLR), 2026

    2026

  59. [59]

    Codev: Code with images for faithful visual reasoning via tool-aware policy optimization,

    X. Hou, S. Xu, M. Biyani, M. Li, J. Liu, T. C. Hollon, and B. Wang, “Codev: Code with images for faithful visual reasoning via tool-aware policy optimization,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  60. [60]

    Reinforced visual perception with tools,

    Z. Zhou, D. Chen, Z. Ma, Z. Hu, M. Fu, S. Wang, Y. Wan, Z. Zhao, and R. Krishna, “Reinforced visual perception with tools,”arXiv preprint arXiv:2509.01656, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    Pyvision: Agentic vision with dynamic tool- ing,

    S. Zhao, H. Zhang, S. Lin, M. Li, Q. Wu, K. Zhang, and C. Wei, “Pyvision: Agentic vision with dynamic tool- ing,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS) Workshop, 2025

    2025

  62. [62]

    Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use,

    M. Wu, J. Yang, J. Jiang, M. Li, K. Yan, H. Yu, M. Zhang, C. Zhai, and K. Nahrstedt, “Vtool-r1: Vlms learn to think with images via reinforcement learning on multimodal tool use,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  63. [63]

    Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning,

    Y. X. Chng, T. Hu, W. Tong, X. Li, J. Chen, H. Yu, J. Lu, H. Guo, H. Deng, X. Chen, G. Huang, D. Lin, and L. Lu, “Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026

    2026

  64. [64]

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning,

    A. Su, H. Wang, W. Ren, F. Lin, and W. Chen, “Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  65. [65]

    Active-o3: Empowering multimodal large language models with active perception via grpo,

    M. Zhu, H. Zhong, C. Zhao, Z. Du, Z. Huang, M. Liu, H. Chen, C. Zou, J. Chen, M. Yanget al., “Active-o3: Empowering multimodal large language models with active perception via grpo,” inInternational Conference on Machine Learning (ICML), 2026

    2026

  66. [66]

    High-resolution visual reasoning via multi- turn grounding-based reinforcement learning,

    X. Huang, Y. Dong, W. Tian, B. Li, R. Feng, and Z. Liu, “High-resolution visual reasoning via multi- turn grounding-based reinforcement learning,” inPro- ceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) Findings, 2026

    2026

  67. [67]

    Syncloop: A multimodal dual-loop frame- work for self-improving mathematical reasoning,

    X. Chen, W. Hu, H. Li, J. Zhou, Z. Chen, M. Cao, Y. Zeng, K. Zhang, Y.-J. Yuan, J. Han, H. Xu, and X. Liang, “Syncloop: A multimodal dual-loop frame- work for self-improving mathematical reasoning,” in Proceedings of the European Conference on Computer Vision (ECCV), 2026

    2026

  68. [68]

    Deepeyesv2: Toward agentic multimodal model,

    J. Hong, C. Zhao, C. Zhu, W. Lu, G. Xu, and X. Yu, “Deepeyesv2: Toward agentic multimodal model,” in Proceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  69. [69]

    Skywork-r1v4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch,

    Y. Zhang, L. Hu, H. Sun, P . Wang, Y. Wei, S. Yin, J. Pei, W. Shen, P . Xia, Y. Penget al., “Skywork-r1v4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch,”arXiv preprint arXiv:2512.02395, 2025

    arXiv 2025

  70. [70]

    Training large language models to reason in a continuous latent space,

    S. Hao, S. Sukhbaatar, D. Su, X. Li, Z. Hu, J. Weston, and Y. Tian, “Training large language models to reason in a continuous latent space,” inProceedings of the Conference on Language Modeling (COLM), 2025

    2025

  71. [71]

    Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space,

    H. Li, C. Li, T. Wu, X. Zhu, Y. Wang, Z. Yu, E. H. Jiang, S.-C. Zhu, Z. Jia, Y. N. Wuet al., “Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space,”arXiv preprint arXiv:2505.13308, 2025

    arXiv 2025

  72. [72]

    Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning,

    X. Chen, R. Zhang, D. Jiang, A. Zhou, S. Yan, W. Lin, and H. Li, “Mint-cot: Enabling interleaved visual tokens in mathematical chain-of-thought reasoning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  73. [73]

    Latent visual reasoning,

    B. Li, X. Sun, J. Liu, Z. Wang, J. Wu, X. Yu, H. Chen, E. Barsoum, M. Chen, and Z. Liu, “Latent visual reasoning,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  74. [74]

    Perception tokens enhance visual reasoning in multimodal language models,

    M. Bigverdi, Z. Luo, C.-Y. Hsieh, E. Shen, D. Chen, L. G. Shapiro, and R. Krishna, “Perception tokens enhance visual reasoning in multimodal language models,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

    2025

  75. [75]

    Chain-of-visual-thought: Teaching vlms to see and think better with continuous visual tokens,

    Y. Qin, B. Wei, J. Ge, K. Kallidromitis, S. Fu, T. Darrell, and X. Wang, “Chain-of-visual-thought: Teaching vlms to see and think better with continuous visual tokens,” 16 arXiv preprint arXiv:2511.19418, 2025

    arXiv 2025

  76. [76]

    Latent sketchpad: Sketching visual thoughts to elicit multimodal reasoning in mllms,

    H. Zhang, W. Wu, C. Li, N. Shang, Y. Xia, Y. Huang, Y. Zhang, L. Dong, Z. Zhang, L. Wang, T.-P . Tan, and F. Wei, “Latent sketchpad: Sketching visual thoughts to elicit multimodal reasoning in mllms,”arXiv preprint arXiv:2510.24514, 2025

    arXiv 2025

  77. [77]

    Open vision reasoner: Transferring linguistic cognitive behavior for visual reasoning,

    Y. Wei, L. Zhao, J. Sun, K. Lin, J. Yin, J. Hu, Y. Zhang, E. Yu, H. Lv, Z. Weng, J. Wang, C. Han, Y. Peng, Q. Han, Z. Ge, X. Zhang, D. Jiang, and V . M. Patel, “Open vision reasoner: Transferring linguistic cognitive behavior for visual reasoning,” inProceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2025

    2025

  78. [78]

    Thinkmorph: Emergent properties in multimodal interleaved chain-of-thought reasoning,

    J. Gu, Y. Hao, H. W. Wang, L. Li, M. Shieh, Y. Choi, R. Krishna, and Y. Cheng, “Thinkmorph: Emergent properties in multimodal interleaved chain-of-thought reasoning,” inProceedings of the International Conference on Learning Representations (ICLR), 2026

    2026

  79. [79]

    Reasoning within the mind: Dynamic multimodal interleaving in latent space,

    C. Liu, Y. Yang, Y. Yang, Y. Fan, Q. Wei, S. Liu, and X. E. Wang, “Reasoning within the mind: Dynamic multimodal interleaving in latent space,” inIEEE Conference on Computer Vision and Pattern Recognition (CVPR) Findings, 2026

    2026

  80. [80]

    Interleaved latent visual reasoning with selective perceptual modeling,

    S. Dong, S. Wang, X. Liu, C. Li, H. Hou, and Z. Wei, “Interleaved latent visual reasoning with selective perceptual modeling,” inProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2026

    2026

Showing first 80 references.