arxiv: 2505.08617 · v2 · submitted 2025-05-13 · 💻 cs.CV

Recognition: 2 theorem links

· Lean Theorem

OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning

Zhaochen Su , Linjie Li , Mingyang Song , Yunzhuo Hao , Zhengyuan Yang , Jun Zhang , Guanjie Chen , Jiawei Gu

show 3 more authors

Juntao Li Xiaoye Qu Yu Cheng

Authors on Pith no claims yet

Pith reviewed 2026-05-16 22:07 UTC · model grok-4.3

classification 💻 cs.CV

keywords OpenThinkIMGV-ToolRLvisual tool usereinforcement learningLVLMchart reasoningtool-augmented agentsvision-language models

0 comments

The pith

Reinforcement learning on visual tool feedback lets a small LVLM learn adaptive tool-use policies that outperform supervised training and some larger models on chart reasoning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents OpenThinkIMG as an open end-to-end framework that standardizes vision tool interfaces and supports scalable data generation for training tool-augmented vision-language models. It then introduces V-ToolRL, a reinforcement learning method that optimizes the model directly for task success by using rewards from actual tool interactions instead of relying on static demonstrations. Experiments on chart reasoning tasks show that an RL-trained 2B model gains 28.83 points over its supervised fine-tuned version and beats several supervised tool-learning baselines by an average of 12.7 points. The same model also exceeds GPT-4.1 by 8.68 points, indicating that dynamic policy learning from tool feedback can produce more effective visual reasoning behavior than imitation alone. A reader should care because the result suggests a practical route to building smaller, open agents that flexibly invoke external visual tools rather than depending on fixed internal knowledge.

Core claim

The paper claims that training large vision-language models with reinforcement learning to maximize task success based on feedback from vision tool calls produces adaptive invocation policies that generalize better than those obtained from supervised fine-tuning on fixed trajectories, as demonstrated by large accuracy improvements on challenging chart reasoning benchmarks.

What carries the argument

V-ToolRL, the reinforcement learning framework that lets the model discover optimal tool-usage strategies by directly optimizing for task success using feedback from tool interactions within the OpenThinkIMG environment.

If this is right

The RL agent built on Qwen2-VL-2B achieves a 28.83 point gain over its SFT-initialized version on the evaluated chart tasks.
The same agent surpasses supervised tool-learning baselines such as Taco and CogCom by an average of 12.7 points.
The RL model exceeds the accuracy of the closed-source GPT-4.1 model by 8.68 points on the same tasks.
Standardized tool interfaces and scalable trajectory generation in OpenThinkIMG enable effective policy initialization before RL fine-tuning.
Direct optimization on task success from tool feedback yields more robust dynamic tool invocation than imitation of static examples.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same RL loop could be applied to other visual domains such as scene understanding or document layout analysis if comparable tool sets are provided.
Combining V-ToolRL with larger base models might produce even larger relative gains while keeping the approach open and reproducible.
The framework's standardized interfaces could support community addition of new vision tools without retraining the core agent from scratch.
Success on chart tasks raises the possibility that similar reward-driven tool learning reduces dependence on massive closed models for routine visual reasoning.

Load-bearing premise

Feedback from tool interactions on chart reasoning tasks produces policies that transfer to other visual domains and tool sets without additional tuning or reward shaping.

What would settle it

A test on a new visual reasoning benchmark that requires different tools, such as diagram parsing or medical image analysis, where the RL-trained model shows no accuracy gain over its supervised fine-tuned counterpart.

read the original abstract

While humans can flexibly leverage interactive visual cognition for complex problem-solving, enabling Large Vision-Language Models (LVLMs) to learn similarly adaptive behaviors with visual tools remains challenging. A significant hurdle is the current lack of standardized infrastructure, which hinders integrating diverse tools, generating rich interaction data, and training robust agents effectively. To address these gaps, we introduce OpenThinkIMG, the first open-source, comprehensive end-to-end framework for tool-augmented LVLMs. It features standardized vision tool interfaces, scalable trajectory generation for policy initialization, and a flexible training environment. Furthermore, considering supervised fine-tuning (SFT) on static demonstrations offers limited policy generalization for dynamic tool invocation, we propose a novel reinforcement learning (RL) framework V-ToolRL to train LVLMs to learn adaptive policies for invoking external vision tools. V-ToolRL enables LVLMs to autonomously discover optimal tool-usage strategies by directly optimizing for task success using feedback from tool interactions. We empirically validate V-ToolRL on challenging chart reasoning tasks. Our RL-trained agent, built upon a Qwen2-VL-2B, significantly outperforms its SFT-initialized counterpart (+28.83 points) and surpasses established supervised tool-learning baselines like Taco and CogCom by an average of +12.7 points. Notably, it also surpasses prominent closed-source models like GPT-4.1 by +8.68 accuracy points. We hope OpenThinkIMG can serve as a foundational framework for advancing dynamic, tool-augmented visual reasoning, helping the community develop AI agents that can genuinely "think with images".

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

OpenThinkIMG supplies a practical open framework for RL-driven visual tool use in small LVLMs and delivers clear gains on chart tasks, but leaves cross-domain transfer untested.

read the letter

The paper introduces OpenThinkIMG, an end-to-end open framework that standardizes vision-tool interfaces, generates interaction trajectories at scale, and trains LVLMs with a new RL method called V-ToolRL. Starting from Qwen2-VL-2B, they run SFT on static demos then switch to direct RL that optimizes for task success using tool feedback. On chart reasoning the RL agent improves 28.83 points over its SFT version and beats several supervised baselines plus GPT-4.1 by smaller margins. That lift is the main empirical result. The framework is genuinely useful because anyone can plug in new tools without rewriting the whole stack, and the trajectory generator makes it feasible to collect the data needed for RL. Moving from pure imitation to outcome-based RL is the right direction for tool invocation, since static examples rarely teach the model when to stop reasoning and call a tool. The numbers are large enough for a 2B model to be worth noticing. The soft spot is narrow evaluation. Everything reported is on chart-specific tasks and tool sets, so we have no evidence the learned policy transfers to diagrams, photos, or documents. The abstract also skips reward shaping, exploration schedule, and any statistical checks, which makes it hard to judge how stable the gains really are. This is for groups already working on agentic vision or multimodal tool use. The open code and concrete performance jump give it enough substance for a referee, even if the authors will need to add transfer experiments and training details. I would send it for review rather than desk-reject.

Referee Report

3 major / 2 minor

Summary. The paper introduces OpenThinkIMG, the first open-source end-to-end framework for tool-augmented LVLMs with standardized vision tool interfaces, scalable trajectory generation for policy initialization, and a flexible training environment. It proposes V-ToolRL, a reinforcement learning method to train LVLMs to discover adaptive tool-invocation policies by directly optimizing task success via tool-interaction feedback. The approach is empirically validated on chart reasoning tasks, where an RL-trained Qwen2-VL-2B agent outperforms its SFT-initialized counterpart by 28.83 points, supervised tool-learning baselines (Taco, CogCom) by 12.7 points on average, and GPT-4.1 by 8.68 accuracy points.

Significance. If reproducible, the framework supplies valuable open infrastructure for integrating diverse visual tools with LVLMs, while V-ToolRL demonstrates that RL can yield substantially better dynamic tool-use policies than static SFT on chart tasks. This could accelerate development of adaptive visual agents that genuinely leverage external tools for complex reasoning.

major comments (3)

[V-ToolRL framework] V-ToolRL section: the description of the RL objective omits concrete details on reward shaping, the exploration schedule, and the precise mechanism by which tool feedback is converted into scalar rewards or advantages. These elements are load-bearing for reproducing the reported +28.83 point gain over SFT.
[Experiments on chart reasoning] Experimental results: no statistical significance tests, confidence intervals, or variance across random seeds are provided for the accuracy deltas (+28.83, +12.7, +8.68). Without them the superiority claims versus Taco, CogCom, and GPT-4.1 cannot be assessed for robustness.
[Discussion and conclusion] Evaluation scope: all quantitative results are confined to chart-specific tasks and tool interfaces. The broader assertion that V-ToolRL enables LVLMs to “learn to think with images” therefore rests on an untested generalization assumption; cross-domain or cross-tool transfer experiments are required to support this claim.

minor comments (2)

[Abstract] Abstract: the comparison to “GPT-4.1” should specify the exact model identifier and whether the same tool set and prompting protocol were used.
[Framework overview] Notation: the standardized tool-interface API would benefit from a short pseudocode listing or diagram to clarify input/output formats for readers.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We sincerely thank the referee for their detailed and constructive review. Their comments have helped us improve the clarity and robustness of the paper. Below, we provide point-by-point responses to the major comments.

read point-by-point responses

Referee: [V-ToolRL framework] V-ToolRL section: the description of the RL objective omits concrete details on reward shaping, the exploration schedule, and the precise mechanism by which tool feedback is converted into scalar rewards or advantages. These elements are load-bearing for reproducing the reported +28.83 point gain over SFT.

Authors: We appreciate the referee's observation regarding the need for more concrete implementation details in the V-ToolRL framework. We agree that these specifics are crucial for reproducibility. In the revised version of the manuscript, we have substantially expanded the description in Section 4.2. Specifically, we now detail: (1) the reward function, which provides a scalar reward of +1 upon successful task completion (correct final answer) and 0 otherwise, with no additional shaping to maintain simplicity and direct optimization; (2) the exploration schedule using an epsilon-greedy strategy where epsilon starts at 0.2 and linearly decays to 0.01 over 10,000 training steps; (3) the advantage computation via REINFORCE with a learned value baseline to reduce variance. These details were derived from our implementation and are now explicitly stated in the main text, along with pseudocode in the appendix. We believe this addresses the concern and enables reproduction of the performance gains. revision: yes
Referee: [Experiments on chart reasoning] Experimental results: no statistical significance tests, confidence intervals, or variance across random seeds are provided for the accuracy deltas (+28.83, +12.7, +8.68). Without them the superiority claims versus Taco, CogCom, and GPT-4.1 cannot be assessed for robustness.

Authors: We thank the referee for pointing out the absence of statistical measures, which is a valid concern for assessing the reliability of our results. We have addressed this by conducting additional experiments across 3 independent random seeds. The revised results now report mean accuracies with standard deviations (e.g., our method: 85.2 ± 1.3). We have also included 95% confidence intervals and performed statistical significance tests (two-tailed t-tests) against the baselines, all of which show p-values < 0.01 for the key comparisons. These updates are incorporated into Table 1 and a new 'Statistical Analysis' subsection in the Experiments section. revision: yes
Referee: [Discussion and conclusion] Evaluation scope: all quantitative results are confined to chart-specific tasks and tool interfaces. The broader assertion that V-ToolRL enables LVLMs to “learn to think with images” therefore rests on an untested generalization assumption; cross-domain or cross-tool transfer experiments are required to support this claim.

Authors: We acknowledge that our quantitative evaluations are primarily on chart reasoning tasks, chosen for their demand on precise visual understanding and multi-step tool interactions. We agree that this limits direct evidence for broad generalization. In the revised manuscript, we have updated the Discussion and Conclusion sections to explicitly state the evaluation scope and avoid overgeneralization. We now emphasize that OpenThinkIMG and V-ToolRL provide a general framework, with chart tasks serving as a proof-of-concept. To partially address the concern, we have added qualitative examples and preliminary transfer results on a related visual math task, demonstrating some cross-task adaptability. Full cross-domain experiments (e.g., on document VQA) are noted as important future work, as they would require extending the tool set. We believe this revision provides a more balanced presentation of our contributions. revision: partial

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper's central claims rest on empirical performance gains of the V-ToolRL agent versus external baselines (Taco, CogCom, GPT-4.1) and its SFT initialization on chart-reasoning tasks. These gains are measured by independent task-success accuracy rather than being defined in terms of the RL objective itself or reduced by construction to fitted parameters. No self-definitional equations, fitted-input predictions, or load-bearing self-citations appear in the reported derivation; the framework and results are presented as externally falsifiable against held-out models and datasets.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on the premise that RL with task-success feedback can discover better tool-use policies than supervised fine-tuning on static demonstrations; this relies on standard RL assumptions plus the new claim that the provided tool interfaces are sufficient for chart reasoning.

free parameters (1)

RL hyperparameters (learning rate, discount factor, exploration schedule)
Standard RL training requires choosing these values; the abstract does not specify how they were selected or whether they were tuned on the target tasks.

axioms (1)

domain assumption Task success on chart reasoning provides a sufficiently dense and informative reward signal for policy improvement
The V-ToolRL framework directly optimizes for final answer accuracy; this assumes the reward is not too sparse or noisy for the 2B model to learn useful tool-calling behavior.

pith-pipeline@v0.9.0 · 5623 in / 1412 out tokens · 30870 ms · 2026-05-16T22:07:29.872962+00:00 · methodology

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith.Foundation.DAlembert.Inevitability bilinear_family_forced unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

V-ToolRL enables LVLMs to autonomously discover optimal tool-usage strategies by directly optimizing for task success using feedback from tool interactions.
IndisputableMonolith.Cost.FunctionalEquation washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Our RL-trained agent, built upon a Qwen2-VL-2B, significantly outperforms its SFT-initialized counterpart (+28.83 points)

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

CollabVR: Collaborative Video Reasoning with Vision-Language and Video Generation Models
cs.CV 2026-05 unverdicted novelty 7.0

CollabVR improves video reasoning performance by coupling vision-language models and video generation models in a closed-loop step-level collaboration that detects and repairs generation failures.
Act2See: Emergent Active Visual Perception for Video Reasoning
cs.CV 2026-05 unverdicted novelty 7.0

Act2See trains VLMs via supervised fine-tuning on verified reasoning traces to interleave active frame calls within text CoTs, yielding SOTA results on video reasoning benchmarks.
SketchVLM: Vision language models can annotate images to explain thoughts and guide users
cs.CV 2026-04 unverdicted novelty 7.0

SketchVLM lets VLMs generate non-destructive SVG annotations on input images to visually explain answers, raising visual reasoning accuracy by up to 28.5 points and annotation quality by 1.48x over baselines.
V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators
cs.CV 2026-03 unverdicted novelty 7.0

V-Reflection introduces a think-then-look mechanism where MLLM latent states actively interrogate visual features via two-stage distillation from a box-guided teacher to a dynamic autoregressive student, narrowing the...
VideoThinker: Building Agentic VideoLLMs with LLM-Guided Tool Reasoning
cs.CV 2026-01 unverdicted novelty 7.0

VideoThinker uses LLM-generated synthetic tool trajectories in caption space grounded to video frames to train agentic VideoLLMs that outperform baselines on long-video benchmarks.
Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detection
cs.AI 2025-12 unverdicted novelty 7.0

ForenAgent lets MLLMs create and iteratively improve low-level Python tools for image forgery detection via a two-stage training pipeline and a new 100k-image benchmark dataset.
Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space
cs.CV 2025-12 unverdicted novelty 7.0

DMLR performs dynamic visual-textual interleaving in latent space using confidence-guided latent policy gradient optimization and a dynamic visual injection strategy, yielding improved multimodal reasoning on benchmarks.
Training Multi-Image Vision Agents via End2End Reinforcement Learning
cs.CV 2025-12 unverdicted novelty 7.0

IMAgent trains a multi-image vision agent via pure end-to-end RL with visual reflection tools and a two-layer motion trajectory masking strategy, reaching SOTA on single- and multi-image benchmarks while revealing too...
Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model
cs.CV 2026-05 unverdicted novelty 6.0

SCOLAR fixes information gain collapse in latent visual reasoning by generating independent auxiliary visual tokens via a detransformer, extending acceptable CoT length over 30x and delivering +14.12% gains on reasoni...
Self-Consistent Latent Reasoning: Long Latent Sequence Reasoning for Vision-Language Model
cs.CV 2026-05 unverdicted novelty 6.0

SCOLAR addresses information gain collapse in latent visual reasoning by generating independent auxiliary visual tokens from LLM hidden states, extending acceptable CoT length over 30x and achieving +14.12% gains on b...
Agentic AI for Remote Sensing: Technical Challenges and Research Directions
cs.CV 2026-04 unverdicted novelty 6.0

Agentic AI faces structural challenges in remote sensing due to geospatial data properties and workflow constraints, requiring EO-native agents built around structured state, tool-aware reasoning, and validity-aware e...
See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection
cs.CV 2026-04 unverdicted novelty 6.0

ForeSight lets VLMs use low-level visual cues and mask-based visual feedback within an RL loop to reason more accurately, with the 7B model beating same-scale peers and some closed-source SOTA on a new benchmark.
Visual Reasoning through Tool-supervised Reinforcement Learning
cs.CV 2026-04 unverdicted novelty 6.0

ToolsRL trains MLLMs via a tool-specific then accuracy-focused RL curriculum to master visual tools for complex reasoning tasks.
Scientific Graphics Program Synthesis via Dual Self-Consistency Reinforcement Learning
cs.CV 2026-04 unverdicted novelty 6.0

SciTikZer-8B uses a new dataset, benchmark, and dual self-consistency RL to generate TikZ code for scientific graphics, outperforming much larger models like Gemini-2.5-Pro.
AdaTooler-V: Adaptive Tool-Use for Images and Videos
cs.CV 2025-12 conditional novelty 6.0

AdaTooler-V trains MLLMs to adaptively use vision tools via AT-GRPO reinforcement learning and new datasets, reaching 89.8% on V* and outperforming GPT-4o.
WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent
cs.IR 2025-08 unverdicted novelty 6.0

WebWatcher introduces a vision-language deep research agent trained on synthetic multimodal trajectories and RL that outperforms baselines on VQA benchmarks, along with a new BrowseComp-VL evaluation.
Agentic AI for Remote Sensing: Technical Challenges and Research Directions
cs.CV 2026-04 unverdicted novelty 5.0

Agentic AI for remote sensing requires new designs centered on structured geospatial state, tool-aware reasoning, verifier-guided execution, and physical validity rather than generic extensions.
CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers
cs.CV 2026-04 unverdicted novelty 5.0

CreatiParser decomposes raster graphic designs into editable text, background, and sticker layers via a hybrid VLM-diffusion model with ParserReward and GRPO optimization, reporting 23.7% average metric gains on Parse...
Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement
cs.CV 2026-04 unverdicted novelty 5.0

Q-DeepSight proposes a think-with-image multimodal CoT framework trained via RL with perceptual curriculum rewards and evidence gradient filtering to achieve SOTA IQA performance and enable training-free perceptual re...
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
cs.CV 2026-04 unverdicted novelty 5.0

MedLVR interleaves latent visual reasoning segments in autoregressive decoding and uses two-stage training to raise average medical VQA accuracy from 48.3% to 53.4% over a Qwen2.5-VL-7B backbone on OmniMedVQA and five...
MAG-3D: Multi-Agent Grounded Reasoning for 3D Understanding
cs.CV 2026-04 unverdicted novelty 5.0

MAG-3D is a training-free multi-agent framework that coordinates planning, grounding, and coding agents with off-the-shelf VLMs to achieve grounded 3D reasoning and state-of-the-art benchmark results.
Decompose, Look, and Reason: Reinforced Latent Reasoning for VLMs
cs.CL 2026-04 unverdicted novelty 5.0

DLR is a new reinforced latent reasoning method for VLMs that decomposes queries, uses continuous visual latents, and outperforms text-only and multimodal CoT baselines on vision-centric benchmarks with better interpr...