Pith. sign in

REVIEW 4 major objections 42 references

CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration

T0 review · 4 major / 0 minor · reviewed 2026-07-11 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Complex image creation needs multi-tool orchestration, and CanvasAgent learns it from large executable trajectories via SFT then hybrid-reward RL.

desk verdict Solid systems paper: large executable multi-tool creation dataset plus SFT+GRPO agent; main risk is judge-coupled train/eval, not a broken method. read the letter →

arxiv 2607.05465 v1 pith:GDLE6ROS submitted 2026-07-06 cs.CV cs.AI

Hairui Zhu , Yiying Yang , Tengjin Weng , Ziyu Lu , Xiao Yao , Xiaoyang Ye , Lin Ma , Wenhao Jiang This is my paper · ORCID
classification cs.CVcs.AI
keywords multimodalagentsvisualtooluseimageeditinggenerationreinforcementlearningGRPOorchestrationCanvasCraft
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Many real image requests are not one-shot generation or editing: they need generating assets, localizing and segmenting regions, editing only those regions, compositing, reading text, transforming geometry, and enhancing the result. Existing multimodal agents are built mainly for perception, search, or narrow retouching and lack large-scale supervision for full executable creation trajectories. This paper introduces CanvasCraft—140K fully annotated multi-step tool trajectories plus 10K RL task specifications—and CanvasAgent, which first imitates those trajectories with supervised fine-tuning, then improves long-horizon planning with GRPO under a hybrid reward that scores both the final image and the process. During execution the agent inspects intermediate outputs, tracks named visual assets, and adapts tool choices to the evolving state. On a 250-task evaluation split, the full SFT+RL agent markedly outperforms base tool-equipped MLLMs and SFT-only training on alignment, trajectory quality, rule validity, and overall hybrid reward, while also beating strong image-only models on instruction alignment.

What carries the argument

The hybrid trajectory reward R(τ) = 0.3 R_align + 0.1 R_aes + 0.2 R_traj + 0.4 R_rule, combining LLM-as-judge scores for image-prompt alignment, aesthetics, and process reasonableness with deterministic format/action validity and efficiency penalties, optimized by GRPO after SFT bootstrapping so the agent both finishes the visual request and executes stateful, executable tool chains.

What would settle it

A larger blinded human preference study on the full 250-sample evaluation set that ranks CanvasAgent (SFT+RL) no higher than SFT-only or strong single-model editors on task and key-detail alignment would show that the hybrid-reward pipeline is not producing better real complex edits.

Watch

Extended reading notes

Core claim

CanvasAgent, initialized by supervised fine-tuning on CanvasCraft-SFT’s executable reasoning–action trajectories and then optimized with GRPO on CanvasCraft-RL under a hybrid outcome-plus-process reward, learns to orchestrate heterogeneous visual tools for complex multi-step image creation and editing, producing substantially higher final-image alignment and more valid multi-tool trajectories than base multimodal models or SFT alone.

Load-bearing premise

The work treats LLM judges for alignment, aesthetics, and trajectory quality as trustworthy enough proxies for real success that training and reporting against them establishes genuine gains in multi-tool image creation.

Editorial extensions

If this is right

  • Open-ended image requests that exceed one model call become solvable by multi-turn orchestration of generation, grounding, segmentation, edit, composite, OCR, and enhancement tools.
  • Large executable trajectory supervision plus weaker task-level RL specs is enough to teach tool ordering, intermediate-asset tracking, and recovery.
  • Outcome and process rewards are complementary: dropping either collapses either image quality or trajectory validity.
  • SFT alone underuses tools; post-SFT RL raises average tool-call depth and improves both images and process scores.
  • Success metrics for manipulation-centered agents must score trajectories, not only final images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same SFT-then-hybrid-reward pattern could transfer to video agents that must maintain temporal asset state across frames.
  • Because judges and policy share a model family, measured gains may partly reflect judge-matching; independent judges or larger human studies would separate the two.
  • Explicit asset-state bookkeeping inside the action reward is a reusable interface pattern for any multimodal agent that produces intermediate files.
  • Once fixed-toolkit orchestration is reliable, dynamic tool discovery becomes the natural next bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper introduces CanvasCraft, a large-scale multimodal tool-use resource (≈140K fully annotated SFT trajectories plus 10K RL task specifications and a 250-sample eval split), and CanvasAgent, a Qwen3-VL-8B-based agent that orchestrates 11 heterogeneous visual tools (generation, edit, grounding, SAM, extract, overlay, crop, OCR, rotate, flip, SR) for complex multi-step image creation and editing. Training is two-stage: SFT on executable reasoning–action trajectories, then GRPO on CanvasCraft-RL with a hybrid reward (Eq. 1) that mixes LLM-as-judge outcome scores (alignment, aesthetics) and process scores (trajectory judge plus rule-based format/action/efficiency terms). On the in-house 250-sample eval split, CanvasAgent (SFT+RL) reports large gains over the base MLLM and SFT-only (overall hybrid reward 0.821 vs 0.426/0.557; alignment 0.869; trajectory 0.849; Table 4), with ablations supporting both stages and both reward components (Table 5) and a qualitative multi-image case (Fig. 5).

Significance. The work targets a genuine and under-served setting: manipulation-centered, stateful, multi-tool visual creation rather than perception/search or single-model editing. The scale of CanvasCraft, the explicit asset-state tracking protocol, the hybrid process+outcome reward (including deterministic executability checks), and the public code/dataset release are concrete contributions that the community can build on. If the reported gains hold under independent evaluation, CanvasAgent would be a useful reference system for long-horizon visual tool orchestration. Strengths that should be credited include the fully executable SFT trajectories, the R/L/D difficulty design for RL tasks, the rule-based action validity decomposition (Eqs. 2–6), and the training-strategy and reward ablations in Table 5.

major comments (4)
  1. §4.1–4.2, Table 4, Eq. (1), Appendix A: The central quantitative claim (SFT+RL overall 0.821, alignment 0.869, trajectory 0.849) is measured primarily by the same family of LLM-as-judge scores (R_align, R_aes, R_traj) that dominate the GRPO reward (weights 0.3+0.1+0.2). The policy base is also Qwen3-VL-8B and the judge is Qwen3.5-Plus. Rule-based R_rule (weight 0.4) and the w/o-outcome ablation (Table 5) partially decouple pure circularity, but they do not establish that the magnitude of improvement on complex creation is independent of judge optimization. A substantially larger human study and/or held-out judges/metrics are needed before the Table 4 headline numbers can be treated as reliable evidence of task success.
  2. §4.1, Table 6: External human validation is limited to 12 samples scored 1–5 on three axes. That is too small to corroborate a 250-sample automatic leaderboard or to support the claim that judge scores align with human preference. Expand human evaluation (more samples, inter-annotator agreement, preferably blind comparison against strong image-only and agent baselines) and report it as a primary rather than secondary check.
  3. §4.1–4.2: All automatic results are on the CanvasCraft-RL evaluation split (250 in-distribution tasks with expected tool sets used in reward design). There is no external or out-of-distribution benchmark (e.g., public multi-step editing suites, user-written free-form requests, or held-out tool-chain templates). Without that, it remains unclear whether CanvasAgent generalizes beyond the reverse-engineered CanvasCraft distribution (Figs. 2–3, Tables 2–3).
  4. §4.1, Table 4: Comparisons to image-only models (Qwen-Image-2.0, Wan2.7-Image, GPT-Image-2) report only alignment/aesthetics and omit trajectories by design; general MLLMs given the tool set but not trained on CanvasCraft are weak process baselines. The paper should either (i) equip stronger agentic baselines with the same 11-tool interface and comparable rollout budget, or (ii) more carefully frame Table 4 as an in-framework ablation rather than a broad SOTA claim for complex multi-tool creation.

Circularity Check

0 steps flagged · score 0.0 of 10

No derivation circularity: CanvasAgent’s claims are empirical SFT+GRPO results on a constructed dataset and hybrid reward, not first-principles predictions forced by their inputs.

full rationale

This is an empirical systems paper. The load-bearing claim is that SFT on CanvasCraft-SFT then GRPO on CanvasCraft-RL with hybrid reward R(τ)=0.3 R_align+0.1 R_aes+0.2 R_traj+0.4 R_rule (Eq. 1) improves multi-tool image creation (Table 4: overall 0.821 vs SFT 0.557 / base 0.426). That chain is experimental training and measurement, not a mathematical derivation that reduces to its premises by construction. CanvasCraft-SFT trajectories are reverse-engineered from tool-chain templates and real tool execution (Fig. 2); SFT then imitates those trajectories—standard supervised bootstrapping, not self-definition of a predicted quantity. RL uses expected tool sets only as weak supervision and explores rollouts under GRPO; ablations (Table 5) show outcome and process terms can diverge (w/o outcome: trajectory 0.907 but alignment 0.320), so reported gains are not forced by a single fitted identity. LLM-as-judge train/eval coupling and a small 12-sample human study (Table 6) are validity/reward-hacking risks, not Pith circularity (no fitted parameter renamed as an independent prediction; no uniqueness theorem imported from overlapping authors; no ansatz smuggled via self-citation; no renaming of a known law). No self-citation is load-bearing for the central result. Honest finding: score 0, empty steps.

Assumptions & free parameters 4 free parameters · 5 assumptions · 3 invented entities

This is an empirical systems paper. Load-bearing structure is not a mathematical derivation but design choices: fixed 11-tool environment, reverse-engineered SFT trajectories, GRPO with hand-weighted hybrid reward, and LLM judges as both training signal and primary metric. Free parameters are the reward weights and difficulty bins; axioms are standard RL/MLLM practice plus domain assumptions about judge reliability and tool sufficiency; invented entities are the named dataset/agent/reward package rather than new physical objects.

free parameters (4)
  • Hybrid reward weights (0.3 R_align, 0.1 R_aes, 0.2 R_traj, 0.4 R_rule)
    Hand-chosen coefficients in Eq. 1 that define the optimization target; no principled derivation or sweep reported for the central claim.
  • Rule reward mix (0.4 R_format, 0.6 R_action) and action sub-weights (0.25/0.25/0.20/0.20/0.10)
    Eqs. 2–4 fix how executability is scored; values are design choices that shape process optimization.
  • Efficiency penalty coefficient λ_eff and penalty terms
    λ_eff and the composition of P_eff (Eq. 7) are not numerically specified beyond form; they control length/cost tradeoffs during RL.
  • R/L/D difficulty thresholds (Table 2)
    Easy/medium/hard cutoffs for reasoning, trajectory length, and tool diversity define the RL task distribution and reported difficulty stats.
assumptions (5)
  • domain assumption GRPO relative ranking of multi-rollout trajectories is a valid policy improvement method for multi-turn visual tool use.
    Invoked in §3.2.2; standard in recent LLM RL literature but not re-proven here.
  • domain assumption LLM-as-judge scores for image–prompt alignment, aesthetics, and trajectory quality track human notions of success closely enough to train and evaluate agents.
    Central to §3.3 and §4 metrics; Appendix A prompts operationalize this; only weakly checked by 12-sample human study.
  • ad hoc to paper A fixed set of 11 tools (Table 1) with structured JSON schemas is sufficient to represent complex image creation/editing workflows of interest.
    Scope assumption of CanvasCraft/CanvasAgent; limitations section notes fixed tool set.
  • ad hoc to paper Reverse-engineered tool-chain templates executed on PICO-Banana-400K images yield valid expert trajectories for SFT.
    §3.1.1 construction pipeline; quality filters are described but not independently validated against human editors.
  • domain assumption Next-token SFT on trajectories plus hybrid-reward GRPO yields policies that generalize within the CanvasCraft task distribution.
    Standard two-stage agent training assumption underlying all experimental claims.
invented entities (3)
  • CanvasCraft (SFT + RL + eval splits) independent evidence
    purpose: Provide large-scale executable multi-tool image-creation supervision and RL task specs.
    New dataset artifact; independent evidence is the public HF release and construction description, not external prior measurement.
  • CanvasAgent (SFT+GRPO policy over 11 visual tools)
    purpose: Orchestrate heterogeneous visual tools with asset tracking and intermediate inspection.
    Named trained system; evidence is internal benchmarks and linked code, not independent third-party replication yet.
  • Task-specific hybrid reward R(τ) combining outcome and process scores
    purpose: Supply dense training signal for both final images and executable trajectories under GRPO.
    Design construct of the paper (Eq. 1); success is measured largely by the same family of signals.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration." pith.science (2026). https://pith.science/paper/GDLE6ROS

@misc{pith2026260705465,
  author       = {Pith},
  title        = {Pith review of: CanvasAgent: Enabling Complex Image Creation and Editing via Visual Tool Orchestration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDLE6ROS}},
  note         = {Machine review of arXiv:2607.05465}
}
read the original abstract

Complex image creation and editing often require more than a single generation or editing model. A user request may involve synthesizing images, localizing objects, segmenting regions, editing selected content, compositing intermediate assets, reading text, and enhancing the final result. Such tasks shift multimodal agents from perception-augmented reasoning to manipulation-centered visual creation, where tools must actively transform visual states rather than merely inspect them. However, existing multimodal tool-use agents are mostly optimized for perception, search, or domain-specific editing, and lack large-scale supervision for executable image-creation trajectories. In this paper, we introduce CanvasCraft, a large-scale multimodal tool-use dataset for complex image creation and editing, and \textbf{CanvasAgent}, a tool-augmented multimodal agent that learns to orchestrate heterogeneous visual tools through multi-turn interaction. CanvasCraft contains 140K fully annotated executable trajectories and 10K RL task specifications. CanvasAgent is first trained with SFT to learn executable reasoning-action trajectories, and is then optimized with GRPO using a hybrid reward that combines outcome- and process-level signals. During rollout, CanvasAgent inspects intermediate results, tracks visual assets, and adapts tool decisions to the evolving visual state. Experiments evaluate both final image quality and trajectory behavior, demonstrating the effectiveness of CanvasAgent and the proposed dataset for complex multi-tool image creation workflows.

Figures

Figures reproduced from arXiv: 2607.05465 by the authors.

Figure 1
Figure 1. Overview of CanvasCraft and CanvasAgent. CanvasCraft provides supervised tool-use [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. CanvasCraft-SFT data construction pipeline. The pipeline constructs executable tool-use [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. CanvasCraft-RL data construction pipeline. The pipeline generates difficulty-aware tasks [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: CanvasCraft data example. CanvasCraft-SFT provides the complete execution chain, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Qualitative case study of CanvasAgent on a complex multi-image editing task. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 20 linked inside Pith

  1. [1]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. Instructpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18392–18402, 2023

    2023

  2. [2]

    Sensenova-MARS: Empowering multimodal agentic reasoning and search via reinforcement learning.arXiv preprint arXiv:2512.24330, 2025

    Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, et al. Sensenova-MARS: Empowering multimodal agentic reasoning and search via reinforcement learning.arXiv preprint arXiv:2512.24330, 2025

    arXiv 2025

  3. [3]

    ToolScope: An agentic framework for vision-guided and long-horizon tool use.arXiv preprint arXiv:2510.27363, 2025

    Mengjie Deng, Guanting Dong, and Zhicheng Dou. ToolScope: An agentic framework for vision-guided and long-horizon tool use.arXiv preprint arXiv:2510.27363, 2025

    arXiv 2025

  4. [4]

    Guid- ing instruction-based image editing via multimodal large language models

    Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan. Guid- ing instruction-based image editing via multimodal large language models. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  5. [5]

    Beyond seeing: Evaluating multimodal LLMs on tool-enabled image perception, transformation, and reasoning.arXiv preprint arXiv:2510.12712, 2025

    Xingang Guo, Utkarsh Tyagi, Advait Gosai, Paula Vergara, Jayeon Park, Ernesto Gabriel Hernandez Montoya, Chen Bo Calvin Zhang, Bin Hu, Yunzhong He, Bing Liu, and Rak- shith Sharma Srinivasa. Beyond seeing: Evaluating multimodal LLMs on tool-enabled image perception, transformation, and reasoning.arXiv preprint arXiv:2510.12712, 2025

    arXiv 2025

  6. [6]

    Visual programming: Compositional visual reasoning without training.arXiv preprint arXiv:2211.11559, 2023

    Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training.arXiv preprint arXiv:2211.11559, 2023. 12

    arXiv 2023

  7. [7]

    Deepeyesv2: Toward agentic multimodal model.arXiv preprint arXiv:2511.05271, 2025

    Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, and Xing Yu. Deepeyesv2: Toward agentic multimodal model.arXiv preprint arXiv:2511.05271, 2025

    arXiv 2025

  8. [8]

    Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training LLMs to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    arXiv 2025

Show all 42 references
  1. [9]

    Jarvisart: Liberating human artistic creativity via an intelligent photo retouching agent.arXiv preprint arXiv:2506.17612, 2025

    Yunlong Lin, Zixu Lin, Kunjie Lin, Jinbin Bai, Panwang Pan, Chenxin Li, Haoyu Chen, Zhongdao Wang, Xinghao Ding, Wenbo Li, and Shuicheng Yan. Jarvisart: Liberating human artistic creativity via an intelligent photo retouching agent.arXiv preprint arXiv:2506.17612, 2025

    2025 arXiv

  2. [10]

    Jarvisevo: Towards a self-evolving photo editing agent with synergistic editor-evaluator optimization.arXiv preprint arXiv:2511.23002, 2025

    Yunlong Lin, Linqing Wang, Kunjie Lin, Zixu Lin, Kaixiong Gong, Wenbo Li, Bin Lin, Zhenxi Li, Shiyi Zhang, Yuyang Peng, Wenxun Dai, Xinghao Ding, Chunyu Wang, and Qinglin Lu. Jarvisevo: Towards a self-evolving photo editing agent with synergistic editor-evaluator optimization....

    2025

  3. [11]

    Chameleon: Plug-and-play compositional reasoning with large language models

    Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  4. [12]

    Pico-banana-400k: A large-scale dataset for text-guided image editing, 2025

    Yusu Qian, Eli Bocek-Rivele, Liangchen Song, Jialing Tong, Yinfei Yang, Jiasen Lu, Wenze Hu, and Zhe Gan. Pico-banana-400k: A large-scale dataset for text-guided image editing, 2025

    2025

  5. [13]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022

    2022

  6. [14]

    Fleet, and Mohammad Norouzi

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(4):4713–4726, 2022

    2022

  7. [15]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  8. [16]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y . K. Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  9. [17]

    ZoomEye: Enhancing multimodal LLMs with human-like zooming capabilities through tree-based image exploration

    Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jianwei Yin. ZoomEye: Enhancing multimodal LLMs with human-like zooming capabilities through tree-based image exploration. InProceedings of the 2025 Conference on Empirical Methods in Natural...

    2025

  10. [18]

    HuggingGPT: Solving AI tasks with ChatGPT and its friends in Hugging Face.arXiv preprint arXiv:2303.17580, 2023

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving AI tasks with ChatGPT and its friends in Hugging Face.arXiv preprint arXiv:2303.17580, 2023

    2023 arXiv

  11. [19]

    Emu Edit: Precise image editing via recognition and generation tasks.arXiv preprint arXiv:2311.10089, 2023

    Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu Edit: Precise image editing via recognition and generation tasks.arXiv preprint arXiv:2311.10089, 2023

    2023 arXiv

  12. [20]

    Codedance: A dynamic tool-integrated MLLM for executable visual reasoning

    Qi Song, Honglin Li, Yingchen Yu, Haoyi Zhou, Lin Yang, Song Bai, Qi She, Zilong Huang, and Yunqing Zhao. Codedance: A dynamic tool-integrated MLLM for executable visual reasoning. arXiv preprint arXiv:2512.17312, 2025

    2025

  13. [21]

    OpenThinkIMG: Learning to think with images via visual tool reinforcement learning.arXiv preprint arXiv:2505.08617, 2025

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and Yu Cheng. OpenThinkIMG: Learning to think with images via visual tool reinforcement learning.arXiv preprint arXiv:2505.08617, 2025

    2025 arXiv

  14. [22]

    ViperGPT: Visual inference via python execution for reasoning.arXiv preprint arXiv:2303.08128, 2023

    Didac Suris, Sachit Menon, and Carl V ondrick. ViperGPT: Visual inference via python execution for reasoning.arXiv preprint arXiv:2303.08128, 2023. 13

    2023 arXiv

  15. [23]

    Adatooler-v: Adaptive tool-use for images and videos.arXiv preprint arXiv:2512.16918, 2025

    Chaoyang Wang, Kaituo Feng, Dongyang Chen, Zhongyu Wang, Zhixun Li, Sicheng Gao, Meng Meng, Xu Zhou, Manyuan Zhang, Yuzhang Shang, et al. Adatooler-v: Adaptive tool-use for images and videos.arXiv preprint arXiv:2512.16918, 2025

    2025 arXiv

  16. [24]

    Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning.arXiv preprint arXiv:2505.15966, 2025

    Haozhe Wang, Alex Su, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning.arXiv preprint arXiv:2505.15966, 2025

    2025 arXiv

  17. [25]

    Qwen-Image technical report.arXiv preprint arXiv:2508.02324, 2025

    Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-Image technical report.arXiv preprint arXiv:2508.02324, 2025

    2025 arXiv

  18. [26]

    Visual ChatGPT: Talking, drawing and editing with visual foundation models.arXiv preprint arXiv:2303.04671, 2023

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual ChatGPT: Talking, drawing and editing with visual foundation models.arXiv preprint arXiv:2303.04671, 2023

    2023 arXiv

  19. [27]

    MMSearch-R1: Incentivizing LMMs to search.arXiv preprint arXiv:2506.20670, 2025

    Jinming Wu, Zihao Deng, Wei Li, Yiding Liu, Bo You, Bo Li, Zejun Ma, and Ziwei Liu. MMSearch-R1: Incentivizing LMMs to search.arXiv preprint arXiv:2506.20670, 2025

    2025 arXiv

  20. [28]

    Act wisely: Cultivating meta-cognitive tool use in agentic multimodal models.arXiv preprint arXiv:2604.08545, 2026

    Shilin Yan, Jintao Tong, Hongwei Xue, Xiaojun Tang, Yangyang Wang, Kunyu Shi, Guannan Zhang, Ruixuan Li, and Yixiong Zou. Act wisely: Cultivating meta-cognitive tool use in agentic multimodal models.arXiv preprint arXiv:2604.08545, 2026

    2026 arXiv

  21. [29]

    MM-REACT: Prompting ChatGPT for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. MM-REACT: Prompting ChatGPT for multimodal reasoning and action.arXiv preprint arXiv:2303.11381, 2023

    2023 arXiv

  22. [30]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  23. [31]

    MagicBrush: A manually annotated dataset for instruction-guided image editing

    Kai Zhang, Lingbo Mo, Wenhu Chen, Huan Sun, and Yu Su. MagicBrush: A manually annotated dataset for instruction-guided image editing. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  24. [32]

    Tool-R1: Sample-efficient reinforcement learning for agentic tool use.arXiv preprint arXiv:2509.12867, 2025

    Yabo Zhang, Yihan Zeng, Qingyun Li, Zhen Hu, Kavin Han, and Wangmeng Zuo. Tool-R1: Sample-efficient reinforcement learning for agentic tool use.arXiv preprint arXiv:2509.12867, 2025

    2025

  25. [33]

    Thyme: Think beyond images.arXiv preprint arXiv:2508.11630, 2025

    Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images.arXiv preprint arXiv:2508.11630, 2025

    2025 arXiv

  26. [34]

    Skywork-R1V4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch.arXiv preprint arXiv:2512.02395, 2025

    Yifan Zhang, Liang Hu, Haofeng Sun, Peiyu Wang, Yichen Wei, Shukang Yin, Jiangbo Pei, Wei Shen, Peng Xia, Yi Peng, et al. Skywork-R1V4: Toward agentic multimodal intelligence through interleaved thinking with images and deepresearch.arXiv preprint arXiv:2512.02395, 2025

    2025

  27. [35]

    Pyvision: Agentic vision with dynamic tooling.arXiv preprint arXiv:2507.07998, 2025

    Shitian Zhao, Haoquan Zhang, Shaoheng Lin, Ming Li, Qilong Wu, Kaipeng Zhang, and Chen Wei. Pyvision: Agentic vision with dynamic tooling.arXiv preprint arXiv:2507.07998, 2025

    2025 arXiv

  28. [36]

    thinking with images

    Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing “thinking with images” via reinforcement learning.arXiv preprint arXiv:2505.14362, 2025. 14 A LLM-as-Judge Prompts Details A.1 Prompt for Alignment Score IM...

    2025 arXiv

  29. [38]

    The input image, if one was provided

  30. [39]

    The final output image, if one was produced

  31. [40]

    Focus on semantic correctness, requested objects/actions, positions, colors, text, preservation of the input image when editing, and overall visual fidelity

    The final text response/trajectory only as auxiliary context Score ONLY whether the final output satisfies the user’s visual request. Focus on semantic correctness, requested objects/actions, positions, colors, text, preservation of the input image when editing, and overall vi...

  32. [41]

    The user’s task prompt

  33. [42]

    The user’s input image, if one was provided

  34. [43]

    score": 0.0} 17 Table 7: Distribution of tool-chain types in CanvasCraft-SFT. The “Multi-tool Hard

    The final output image, if one was produced Your job is to score ONLY the aesthetic quality of the final output image. Do NOT score whether the image semantically satisfies the user’s request. Do NOT score whether the tool-use trajectory was reasonable. Those are evaluated sep...

Pith tools

Reviewed July 11, 2026 · model on record in the stance chip above.