Pith. sign in

REVIEW 4 major objections 5 minor 12 cited by

Thinking with Generated Images

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A single unified model can improve image generation by spontaneously generating intermediate visual thoughts, either as subgoals or as self-critiqued hypotheses.

desk verdict A genuinely novel mechanism for letting LMMs think in images, but the reported gains aren't yet attributable to the visual thoughts because a direct same-data SFT control is missing. read the letter →

arxiv 2505.22525 v1 pith:PKWXCOSB submitted 2025-05-28 cs.CV cs.AIcs.CL

Ethan Chern , Zhulin Hu , Steffi Chern , Siqi Kou , Jiadi Su , Yan Ma , Zhijie Deng , Pengfei Liu This is my paper · ORCID
classification cs.CVcs.AIcs.CL
keywords thinkingwithgeneratedimageslargemultimodalmodelsvisualchain-of-thoughttest-timescalingautoregressiveimagegenerationself-critiqueintermediatesubgoalsreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that large multimodal models can reason across text and vision by generating their own intermediate images, not just by processing user-provided ones. This 'thinking with generated images' is implemented as a native long-multimodal thought process on a unified autoregressive LMM (Anole-7b), trained with curated synthetic data. On vision generation benchmarks, the approach improves multi-object accuracy (GenEval 'Two Obj' from 0.38 to 0.57) and overall image-prompt alignment (DPG-Bench from 62.83 to 67.14 after self-critique). If true, it opens a new axis of test-time scaling where models explore visual hypotheses as part of reasoning. The central move is to treat the generated image itself as a thinking step, not as a final output.

What carries the argument

The central object is the native long-multimodal thought process: a sequence of interleaved tokens from a unified vocabulary that includes both text tokens and image tokens (1024 VQ-VAE codebook tokens per image, delimited by special markers). It is instantiated on Anole-7b, a unified autoregressive next-token-prediction LMM, which is fine-tuned on synthetic data built with external generators (Flux for images, QVQ for critique reasoning). Training adds a visual reconstruction loss (MSE between projected hidden states and ground-truth codebook features) to the standard cross-entropy loss, and inference uses tuned classifier-free guidance with separate conditioning scales for the original prompt, the intermediate subgoal images, and the critique text.

What would settle it

Take the same synthetic prompt set used for TwGI-Anole-7b-Obj. and TwGI-Anole-7b-Crit., and fine-tune Anole-7b on the same data but with all intermediate visual thought tokens removed (i.e., direct prompt-to-final-image). If GenEval Two Obj and DPG-Bench scores match or exceed those of the thought models, the claimed benefit of generated visual thoughts is not causal. Conversely, if the models without visual thoughts score lower by the same margins, the claim gains support.

Watch

Extended reading notes

Core claim

The paper establishes that a fine-tuned unified autoregressive LMM can be trained to interleave text and image tokens in a single autoregressive pass, so that it spontaneously generates intermediate visual thoughts while solving a vision generation task. Two complementary thought processes are demonstrated: vision generation with intermediate visual subgoals, where the model renders objects separately and then combines them, and vision generation with self-critique, where the model generates an initial visual hypothesis, reflects on it in text, and refines it. The resulting models, TwGI-Anole-7b-Obj. and TwGI-Anole-7b-Crit., outperform the base Anole-7b on GenEval and DPG-Bench, with the subgoal model raising Two Obj accuracy from 0.38 to 0.57 and the self-critique model raising DPG-Bench overall from 62.83 to 67.14. The paper argues that these gains come from the visual thinking process itself, which enables test-time scaling across modalities.

Load-bearing premise

The observed improvements are caused by the intermediate visual thoughts themselves, rather than by the extra supervised fine-tuning data, the reconstruction loss, or the tuned guidance schedules — yet no baseline is trained on the same prompts with direct image-to-image generation, so the causal role of visual thinking is not established.

Editorial extensions

If this is right

  • If the mechanism works as claimed, future unified LMMs could be trained to decompose complex visual tasks into subgoals, improving compositional image generation without needing external tools or multi-agent pipelines.
  • The self-critique loop implies that a single model can act as its own critic and refiner, so visual hypotheses can be iteratively improved at inference time, effectively enabling test-time scaling across both text and image tokens.
  • The same native long-multimodal thought process should extend to other modalities (audio, video, 3D), since the vocabulary is defined as a union of per-modality token spaces.
  • The approach suggests that stronger base unified LMMs will likely amplify the benefit, since the observed failure cases (e.g., failing to combine subgoals, failing to execute its own critique) are attributed to base-model limitations.
  • The reconstruction loss and CFG tuning are load-bearing components; the paper reports that these contribute roughly 3 points on GenEval during the JourneyDB stage, so they are part of the claimed improvement mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate the causal effect of the visual thoughts themselves: the supervised fine-tuning data, the reconstruction loss, and the custom CFG schedules could account for part or all of the gains. A control baseline trained on the same prompts with direct image-to-image generation (no intermediate visual tokens) would settle this, but no such baseline is reported.
  • The self-critique gains (DPG-Bench 62.83 to 67.14) are measured on the same model's own initial hypothesis, so they might partly reflect test-time compute rather than a learned reasoning ability; a fair comparison would pair the critique model's final output against a direct generation at similar token budget.
  • The choice of benchmarks (GenEval, DPG-Bench) is limited to standard image generation; the paper argues for real-world tasks like design or 3D modeling, but does not evaluate those, so the general 'visual imagination' claim remains a hypothesis.
  • The data pipeline relies on strong external generators (Flux, QVQ) to produce the thought chains, which raises a question of whether the fine-tuned model learns to think or merely imitates the annotations; a test without such supervision would clarify the emergence of spontaneous visual thinking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes 'Thinking with Generated Images', a paradigm in which a unified autoregressive large multimodal model (Anole-7b) is fine-tuned to generate intermediate visual thoughts as part of its reasoning process. Two mechanisms are instantiated: (1) vision generation with intermediate visual subgoals, where the model decomposes a prompt into sub-images and then composes a final image, and (2) vision generation with self-critique, where the model generates an initial visual hypothesis, critiques it in text, and produces a refined image. The authors construct synthetic SFT data using external models (QVQ, Flux, etc.), train on two stages, and report gains on GenEval and DPG-Bench, including a 0.38-to-0.57 improvement on GenEval Two Obj for the subgoal model and a 62.83-to-67.14 improvement on DPG-Bench Overall for the self-critique final output. The paper claims that this enables spontaneous, iterative multimodal reasoning and test-time scaling.

Significance. If the causal claim is established, this is a useful step toward multimodal cognition and test-time scaling for unified LMMs. The paper is clearly written, releases an open-source suite, and provides a detailed synthetic data pipeline. The core scientific value, however, depends on isolating whether the gains come from the intermediate visual thoughts themselves or from confounded factors such as additional SFT data, different CFG settings, and hyperparameters tuned on the evaluation benchmark. The paper does not currently provide that isolation, so the significance is conditional on additional controlled experiments.

major comments (4)
  1. [§3.6, Tables 1 and 2] The central causal claim—that intermediate visual thoughts drive the improvements—is not tested. The baseline Anole-7b is only continued-pretrained on JourneyDB; no model is fine-tuned on the same curated synthetic prompts with only direct image-to-image generation as the target. Consequently, the observed gains (e.g., GenEval Two Obj 0.38→0.57, DPG-Bench Overall 58.32→68.44 for the subgoal model) could be due to the additional SFT data or the specific prompt distribution rather than to the thinking process. A same-prompt, same-loss, same-CFG direct-generation SFT control is required before the improvement can be attributed to visual thoughts.
  2. [§3.6 and Appendix A.3 (Table 5)] The self-critique comparison between TwGI-Anole-7b-Crit. (visual hypo.) and TwGI-Anole-7b-Crit. (final) changes both the CFG scales and the conditioning. The final pass uses different guidance weights (full conditions 2.0→1.5, image conditions 1.2→0.8) and additionally conditions on the real first-round image plus the critique text. The DPG-Bench gain from 62.83 to 67.14 and the GenEval gain from 0.45 to 0.48 could therefore stem from having a second chance conditioned on a good prior image and from the retuned CFG, not from the learned critique reasoning. An ablation that holds CFG and conditioning fixed, or that removes the critique text while keeping the same prior-image condition, is needed.
  3. [Appendix A.1, A.2, A.3] Several hyperparameters were selected on the same benchmarks used for the main results. The reconstruction loss weight λ was tuned on GenEval (Table 3) and then used in the reported GenEval results, and the CFG scales in Table 5 were presumably chosen on the same benchmarks. This introduces selection bias. In addition, the two mechanisms use very different stage-2 dataset sizes (5k vs. 40k samples), and no error bars or multiple-seed runs are reported. At minimum, a held-out validation set for hyperparameter selection and repeated evaluation runs are necessary to support the quantitative claims.
  4. [§B.3, Figure 9] The paper's own failure case for self-critique shows that the model can produce a correct critique and still fail to generate the corresponding corrected image (the TV remote example). This directly qualifies the central claim that models can 'critique their own visual hypotheses and refine them as integral components of their reasoning process.' The mechanism is not sufficient for consistent improvement; the paper should either temper the claim or provide a more detailed analysis of when the critique leads to correction and when it does not.
minor comments (5)
  1. [§3.6, Table 1] The row for TwGI-Anole-7b-Obj. omits GenEval Overall and several categories (Single Obj, Counting, Colors), even though the text says it 'consistently outperforms.' Please report the full table or explicitly justify the omission, as the missing entries make it difficult to verify the claim.
  2. [§3.6, Table 2] The DPG-Bench 'Other' category declines from 43.60 (Anole-7b) to 42.00 (TwGI-Anole-7b-Obj.). The text highlights only the improvements in Entity, Attribute, and Relation; the decline in a major category should be acknowledged and discussed.
  3. [§3.7] The text interchangeably uses 'Anole-7b-Obj' and 'TwGI-Anole-7b-Obj'; please standardize the model naming to avoid confusion.
  4. [Appendix A.1] The reconstruction loss uses f_vq to obtain codebook features, but the manuscript does not state whether this VQ-VAE encoder is frozen from Anole or fine-tuned. Please clarify the training status of the visual encoder.
  5. [Throughout] There are minor typographical issues, including inconsistent capitalization in 'Thinking with Generated images' in Section 2.2 and equation references that appear as 'equation 1' rather than a numbered citation. A final proofread would be beneficial.

Circularity Check

2 steps flagged · score 4.0 of 10

Benchmark-fitted loss and CFG choices partially underwrite the headline gains; the core visual-thought mechanism is not definitionally circular but the 'prediction' status of the benchmark deltas is weakened.

  1. fitted input called prediction [Appendix A.1, Table 3; Section 3.4]
    "Results in Tab. 3 demonstrate that λ= 1 yields optimal performance, improving GenEval scores by approximately 3 points. Neither Ltd alone nor the combination Lmm +λL rec +µL td surpassed our proposed objective. Thus, we adopt Ltotal =L mm +L rec withλ= 1for all experiments in this paper."

    The reconstruction-loss weight λ is an input chosen by maximizing GenEval on a 50k-image JourneyDB fine-tune, and the same weight is then fixed 'for all experiments' in the models whose Table 1 GenEval scores are presented as evidence. The claim that λ=1 is optimal is just the argmax of Table 3; reporting the resulting benchmark numbers as an independent demonstration of the method is a fitted input presented as a prediction. This does not force the TwGI-vs-Anole delta by itself, but it means the absolute GenEval numbers carry a selection effect from the benchmark used to choose the loss.

  2. fitted input called prediction [Appendix A.3, Table 5; Section 3.7]
    "We apply the following CFG settings to ensure balanced conditioning, enabling the model to concurrently leverage text-based thoughts, previous visual hypothesis, visual subgoals, and original prompts. Specific configurations are shown in Tab. 5."

    The self-critique evidence compares a 'visual hypo.' pass with a 'final' pass that is run under a different CFG recipe: the final pass adds prompt conditions at scale 5.0 and changes full/image scales from 2.0/1.2 to 1.5/0.8 (Table 5). Because these inference-time conditioning and scale values are tuned on the same GenEval/DPG-Bench targets, the reported improvement (0.45→0.48 GenEval; 62.83→67.14 DPG-Bench) is not attributable solely to the learned critique reasoning; part of the gain is constructed by benchmark-fitted inference settings. A same-CFG, critique-ablated control would be needed to treat the improvement as a prediction of the self-critique mechanism.

full rationale

The paper contains no self-definitional derivation, no imported uniqueness theorem, and no ansatz smuggled through self-citation. Anole is the authors' own prior model, but it is used as an open-source artifact rather than as an unverifiable premise; citing it is not load-bearing in a circular sense. The genuine circularity burden is narrower: the loss weight (Appendix A.1) and the inference CFG recipe (Appendix A.3) were selected on the same GenEval/DPG-Bench metrics that later appear as headline evidence, so part of the reported delta is a benchmark-fitted input rather than an out-of-sample prediction. The self-critique comparison changes CFG and conditioning between the 'visual hypo' and 'final' passes, so the improvement cannot be uniquely credited to the learned critique reasoning. Additionally, no baseline is SFT-trained on the same synthetic prompts with direct image-to-image targets, so the causal role of intermediate visual thoughts is underdetermined; that is a confounding/correctness concern rather than definitional circularity. These issues justify a moderate score, not a high one: the central capability (interleaved visual thought generation) is demonstrated qualitatively, and the improvements are not entirely forced by definition.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central method depends on a chain of modeling assumptions: Anole can natively interleave images and text, the external models used to synthesize training data provide correct critiques and refinements, and the standard benchmarks measure what the method claims to improve. The reconstruction loss weight and CFG scales are fit parameters selected with the evaluation benchmarks.

free parameters (3)
  • Reconstruction loss weight lambda = 1.0
    Chosen by a GenEval sweep in Appendix A.1 (Table 3) and then used in all final experiments, so the benchmark used for selection is the same one used for reporting.
  • CFG scale weights for intermediate visual subgoals = full 5.0, image 0.0, negative 3.0, prompt 0.0
    Reported in Appendix A.3 (Table 5). No validation protocol is stated; these are likely selected on the evaluation benchmarks.
  • CFG scale weights for self-critique final output = full 2.0, image 1.2, negative 3.0, prompt 5.0
    Reported in Appendix A.3 (Table 5). These balance text thoughts and image conditions and are not shown to be fixed before evaluation.
assumptions (4)
  • domain assumption Anole-7b is a valid unified autoregressive LMM with native interleaved image-text generation, and its 1024-token per image representation is sufficient for the studied tasks.
    The entire method builds on Anole (Chern et al. 2024). If Anole cannot faithfully generate or condition on intermediate images, the thought process would not work as described. This enters in Section 2.3.2.
  • domain assumption GenEval and DPG-Bench automated metrics are valid measures of text-to-image alignment and compositional correctness.
    The paper draws all quantitative conclusions from these benchmarks without human evaluation or error analysis beyond a few cases. This enters in Section 3.6.
  • domain assumption The synthetic data pipeline, using QVQ for critiques and Flux/Flux-Redux for images, produces correct and useful supervision for learning visual thinking.
    The model is trained to imitate these external chains. If the critiques are wrong or the refined images are not better, the trained model would learn faulty behavior. This enters in Section 3.3.
  • domain assumption Classifier-free guidance with negative prompts and original prompt conditions does not mask or inflate the effect of the visual thought process.
    The CFG balances several condition terms, and the paper does not ablate whether the gains come from the thoughts or from the guidance recipe. This enters in Section 3.5 and Appendix A.3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Thinking with Generated Images." pith.science (2026). https://pith.science/paper/PKWXCOSB

@misc{pith2026250522525,
  author       = {Pith},
  title        = {Pith review of: Thinking with Generated Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKWXCOSB}},
  note         = {Machine review of arXiv:2505.22525}
}
read the original abstract

We present Thinking with Generated Images, a novel paradigm that fundamentally transforms how large multimodal models (LMMs) engage with visual reasoning by enabling them to natively think across text and vision modalities through spontaneous generation of intermediate visual thinking steps. Current visual reasoning with LMMs is constrained to either processing fixed user-provided images or reasoning solely through text-based chain-of-thought (CoT). Thinking with Generated Images unlocks a new dimension of cognitive capability where models can actively construct intermediate visual thoughts, critique their own visual hypotheses, and refine them as integral components of their reasoning process. We demonstrate the effectiveness of our approach through two complementary mechanisms: (1) vision generation with intermediate visual subgoals, where models decompose complex visual tasks into manageable components that are generated and integrated progressively, and (2) vision generation with self-critique, where models generate an initial visual hypothesis, analyze its shortcomings through textual reasoning, and produce refined outputs based on their own critiques. Our experiments on vision generation benchmarks show substantial improvements over baseline approaches, with our models achieving up to 50% (from 38% to 57%) relative improvement in handling complex multi-object scenarios. From biochemists exploring novel protein structures, and architects iterating on spatial designs, to forensic analysts reconstructing crime scenes, and basketball players envisioning strategic plays, our approach enables AI models to engage in the kind of visual imagination and iterative refinement that characterizes human creative, analytical, and strategic thinking. We release our open-source suite at https://github.com/GAIR-NLP/thinking-with-generated-images.

Figures

Figures reproduced from arXiv: 2505.22525 by the authors.

Figure 1
Figure 1. Real-world tasks that require Thinking with Generated Images. These tasks often require visual foresight and imagination, which text-based thought alone cannot fully accomplish. *Equal Contribution. † Partial work done at Bytedance Seed. ‡Corresponding author. 1 arXiv:2505.22525v1 [cs.CV] 28 May 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Examples of “seeing with images” vs. “thinking with images” vs. “thinking with generated images.” [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison between related works on multimodal cognition. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Demonstration of our long-multimodal thought process on GenEval. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Demonstration of our long-multimodal thought process on DPG-Bench. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Our data collection pipeline for Thinking with Generated Images. 3.3.1 High-Quality Image-Generation Prompts We curate diverse, complex prompts that go beyond simple object depiction to include variations in color, orientation, and shape with multiple elements. We dedu…
Figure 7
Figure 7. Figure 7: shows cases of vision generation with intermediate subgoals. The model demonstrates its ability to decompose complex visual generation tasks into manageable sub-components. When generating broccoli and vase, the model first generates a high-quality image of a broccoli,…
Figure 8
Figure 8. Figure 8: shows cases of vision generation with self-critique. The model demonstrates its ability to analyze initial visual hypothesis and iteratively improve them through reflection. For generating a yin-yang symbol with a tiger head, the model initially generates a imperfect y…
Figure 9
Figure 9. Figure 9: Example failure cases of using thinking with generated images. Either the intermediate images generated are correct but fails to combine them together in the end, or the critique is correct but didn’t generate the correspond￾ing correct image. 16 [PITH_FULL_IMAGE:figu…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process

    cs.AI 2026-07 conditional novelty 7.0 of 10

    BRAID jointly optimizes text and image generation in interleaved multi-modal reasoning by casting the full trajectory as a unified MDP with shared advantages and a VLM process reward.

  2. Demystifying Video Reasoning

    cs.CV 2026-03 conditional novelty 7.0 of 10

    Video diffusion models reason along the denoising trajectory (Chain-of-Steps), not primarily across frames, and this mechanism can be nudged by ensembling latent trajectories.

  3. MIRA: Medical Image Reflection for Agentic Diagnosis

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A two-stage pipeline, MCTS-based supervised fine-tuning plus GRPO reinforcement learning with a validation-gated reflection memory, makes an 8B medical vision-language model both more accurate and more selective in tool use.

  4. VC-Tooler: Learning Compositional and Adaptive Visual Tool Use

    cs.CV 2026-08 conditional novelty 6.0 of 10

    VC-Tooler trains Qwen3-VL-8B on hierarchically synthesized tool trajectories via SFT then GRPO with a judge-based tool reward, reaching open-source SOTA on V* (95.8) and VTC-Bench (35.3) and transferring to 11 unseen ...

  5. FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Verification

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A self-judging multi-agent VLM injects process-image helpfulness verdicts into tool observations and scales tool rewards by the helpful-call ratio, improving accuracy and tool faithfulness.

  6. OPLD: On-Policy Latent Distillation for Multimodal Reasoning

    cs.CV 2026-07 conditional novelty 6.0 of 10

    On-policy distillation of a CoT-privileged teacher’s token preferences and latent trajectories yields a CoT-free student that beats prior visual-latent methods on several multimodal benchmarks.

  7. BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

    cs.CV 2026-07 conditional novelty 6.0 of 10

    BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.

  8. Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    Supervising text–image handoffs with Reflective SFT and Flow-GRPO (MoTiF) reduces modal isolation and raises accuracy on four visual puzzle benchmarks versus end-task-only training.

  9. MMRad-22K: A Structured Multimodal Evidence Dataset for Chest X-ray Report Generation

    cs.CV 2026-02 conditional novelty 6.0 of 10

    A 21,994-case chest X-ray dataset with interleaved regional text and image crops helps LVLMs generate more clinically accurate reports than text-only chain-of-thought.

  10. MentisOculi: Revealing the Limits of Reasoning with Mental Imagery

    cs.AI 2026-02 conditional novelty 6.0 of 10

    Visual thoughts — latent tokens, interleaved images, or video rollouts — do not currently improve multi-step reasoning over text-only baselines in frontier models.

  11. IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation

    cs.CV 2025-12 conditional novelty 6.0 of 10

    Interleaving motion generation with text-motion assessment and refinement improves alignment between generated human motion and goal text.

  12. UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving

    cs.CV 2026-01 conditional novelty 5.0 of 10

    A unified VLM for autonomous driving that couples trajectory planning with future-frame image generation improves open- and closed-loop planning metrics on Bench2Drive and nuScenes.

Reference graph

Works this paper leans on

25 extracted references · 17 canonical work pages · cited by 12 Pith papers

  1. [1]

    Extract the corresponding hidden states:h (j) img ∈R 1024×D

    work page

  2. [2]

    Project to codebook space: p(j) =f proj(h(j) img;θ proj), where fproj is an additional trainable linear projection layer that maps from hidden dimensionDto codebook dimensionD ′, i.e.,f proj :R D →R D′

    work page

  3. [3]

    Blind Ambition,

    Retrieve original codebook features: c(j) =f vq(imagej;θ vq), where c(j) ∈R 1024×D′ represents the quantized codebook features for thej-th image The visual reconstruction loss is computed as the mean squared error (MSE) between the projected hidden states and the original codebook features: Lrec = 1 Nimg NimgX j=1 1 1024·D ′ ∥p(j) −c (j)∥2 2 where Nimg is...

    work page 2025

  4. [19]

    Chengzu Li, Wenshan Wu, Huanyu Zhang, Yan Xia, Shaoguang Mao, Li Dong, Ivan Vuli´c, and Furu Wei. 2025. Imagine while reasoning in space: Multimodal visualization-of-thought.arXiv preprint arXiv:2501.07542

    arXiv 2025

  5. [20]

    Allen Newell. 2014. Reasoning, problem solving, and decision processes: The problem space as a fundamental category. InAttention and performance VIII, pages 693–718. Psychology Press

    work page 2014

  6. [21]

    Fei Ni, Jianye Hao, Shiguang Wu, Longxin Kou, Jiashun Liu, Yan Zheng, Bin Wang, and Yuzheng Zhuang

    work page

  7. [22]

    OpenAI. 2025. Thinking with images

    work page 2025

  8. [23]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. 2022. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695

    work page 2022

Show all 25 references
  1. [24]

    Herbert A Simon and Allen Newell. 1971. Human problem solving: The state of the theory in 1970.American psychologist, 26(2):145

    1971

  2. [25]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters.arXiv preprint arXiv:2408.03314

    2024 arXiv

  3. [26]

    Zhaochen Su, Linjie Li, Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, and Yu Cheng. 2025. Openthinkimg: Learning to think with images via visual tool reinforcement learning

    2025

  4. [27]

    Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, et al. 2023. Journeydb: A benchmark for generative image understanding.Advances in neural information processing systems, 36:49659–49678

    2023

  5. [28]

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. 2024. Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525

    2024 arXiv

  6. [29]

    D´ıdac Sur´ıs, Sachit Menon, and Carl V ondrick. 2023. Vipergpt: Visual inference via python execution for reasoning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 11888–11898

    2023

  7. [30]

    Chameleon Team. 2024a. Chameleon: Mixed-modal early-fusion foundation models.arXiv preprint arXiv:2405.09818

    arXiv

  8. [31]

    Qwen Team. 2024b. Qvq: To see the world with wisdom

  9. [32]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al

  10. [33]

    Shijie Xia, Yiwei Qin, Xuefeng Li, Yan Ma, Run-Ze Fan, Steffi Chern, Haoyang Zou, Fan Zhou, Xiangkun Hu, Jiahe Jin, Yanheng He, Yixin Ye, Yixiu Liu, and Pengfei Liu. 2025. Generative ai act ii: Test time scaling drives cognition engineering

    2025

  11. [34]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  12. [35]

    Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. 2023. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381

    2023 arXiv

  13. [36]

    Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang-Chieh Chen. 2025. An image is worth 32 tokens for reconstruction and generation.Advances in Neural Information Processing Systems, 37:128940–128966

    2025

  14. [37]

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. 2023. Multimodal chain-of-thought reasoning in language models.arXiv preprint arXiv:2302.00923

    2023 arXiv

  15. [38]

    Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. 2024. Transfusion: Predict the next token and diffuse images with one multi-modal model.arXiv preprint arXiv:2408.11039. 11 A. Experim...

    2024 arXiv

  16. [2022]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837

  17. [2024]

    InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13991–14000

    Generate subgoal images before act: Unlocking the chain-of-thought reasoning in diffusion model for robot manipulation with multimodal prompts. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13991–14000

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.