Pith. sign in

REVIEW 4 major objections 8 minor 56 references

Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generation

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that diffusion models can learn chain-of-thought-style image reasoning through SRRL, a self-reflective reinforcement learning algorithm that chains denoising trajectories with a condition-guided forward process and uses…

desk verdict Interesting RL-for-diffusion recipe, but the inversion math is wrong and the evaluation is circular. read the letter →

arxiv 2505.22407 v1 pith:QVYETUF7 submitted 2025-05-28 cs.CV

Jiadong Pan , Zhiyuan Ma , Kaiyan Zhang , Ning Ding , Bowen Zhou This is my paper
classification cs.CV
keywords diffusionmodelsreinforcementlearningchain-of-thoughtself-reflectionimagereasoningphysicallawscounterintuitivephenomenaprocessreward
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that diffusion-based image generators can be given a genuine reasoning ability—not just better prompt following—by training them to reflect on their own outputs across repeated denoising rounds. The proposed algorithm, SRRL, treats each complete denoising trajectory as one step in a chain of thought, scores the resulting image with a reward model, and then re-noises that image in a condition-guided way to start the next reasoning round. The aim is to make models generate images that obey physical laws, such as plants growing taller in sunlight, and also images that deliberately defy physics, such as water flowing upward in a tilted glass. The central claim is that this reflective loop introduces image reasoning by chain-of-thought into diffusion-based generation for the first time.

What carries the argument

The load-bearing mechanism is the pair of processes described in Sections 3.2 and 3.3. Multi-round reflective denoising treats each full trajectory from $x_T^k$ to $x_0^k$ as one chain-of-thought step, evaluates $x_0^k$ with a reward model, and uses the normalized advantage of the best and worst samples to update the policy via PPO. Condition guided forward process then maps $x_0^k$ to $x_T^{k+1}$ through Equation 7, a DDIM-inversion-style recursion whose classifier-free guidance scale is set below the denoising scale, creating the guidance gap that carries text condition into the next round. This gap, formalized in Equation 13 as proportional to $(\lambda_{\text{forward}} - \lambda)$ times the conditional noise, is what the argument says lets reflective iteration preserve semantic content while improving prompt adherence.

What would settle it

Run SRRL against a matched control in which the condition-guided forward process is replaced by ordinary noise of the same scale while reward and policy updates stay identical; if VQAScore still improves round over round, the guidance gap is not the operative component. A second check is to measure semantic drift by comparing CLIP similarity between $x_0^k$ and $x_0^{k+1}$: large drops would show that the re-noising loop does not preserve content.

Watch

Extended reading notes

Core claim

SRRL's central claim is that chaining multiple denoising trajectories through a condition-guided re-noising step turns a diffusion model into a self-reflective reasoner. The paper defines the whole reflective process as a long Markov decision process whose reward comes from intermediate images, not only the final image, and optimizes it with PPO-style updates, reward normalization, and contrastive sampling of the best and worst samples. The condition-guided forward process re-noises an intermediate image using classifier-free guidance at a lower scale than the denoising process, creating a guidance gap that injects the text condition into the next round's starting latent. The paper reports that round over round this yields images that increasingly match physical-law and counterintuitive prompts, and qualitative comparisons show results it says rival those of GPT-4o.

Load-bearing premise

The whole loop depends on the condition-guided forward process approximately preserving the intermediate image's semantic content; the derivation assumes the model's noise prediction is nearly identical at adjacent denoising states and that only the conditional-noise part matters, so if those approximations fail, the re-noised image drifts and any apparent improvement could come from the reward model rather than from learned reflection.

Editorial extensions

If this is right

  • Physical-law prompts improve across reflection rounds, such as mirrors increasingly reflecting light with varying intensity consistent with optical laws.
  • Counterintuitive prompts eventually produce images that violate common sense, such as ice cubes floating in hot coffee without melting.
  • The same training improves standard text-image alignment on the "a(n) [animal] [activity]" template, beating the compared baselines on ImageReward and VQAScore.
  • Process rewards on intermediate images replace outcome-only rewards, so the model can self-correct before the final image is complete.
  • Case-study outputs on physical and counterintuitive prompts are claimed to be comparable to or better than those of GPT-4o.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: the same loop could be applied with any reward model, so physical-law prompts are likely one instance of a general mechanism for constraint-driven image generation; the paper only demonstrates physics and animal-activity cases.
  • Editorial: because the training loop uses the reward model to select best and worst samples, part of the round-over-round gain may come from reward-model selection rather than a learned reasoning skill; an ablation with unconditioned re-noising would separate the two.
  • Editorial: the chain-of-thought analogy is approximate—the steps are trajectories, not tokens or intermediate text—so the method is better described as iterative refinement with process rewards than as chain-of-thought in the large-language-model sense.
  • Editorial: a testable extension is to run the same condition-guided re-noising loop at inference time on an untrained model; if it already improves physical-law prompts, the training signal may be amplifying an existing self-correction property rather than creating it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes SRRL, a self-reflective reinforcement learning algorithm for text-to-image diffusion models. The method treats an entire DDIM denoising trajectory as a single Chain-of-Thought step, then re-noises the produced image via a condition-guided forward process and repeats this for K rounds. The authors claim that this introduces image reasoning through CoT into diffusion models for the first time, enabling generation that adheres to physical laws and deliberately counterintuitive phenomena, and they report qualitative comparisons against GPT-4o. Quantitative experiments on a simple 'a(n) [animal] [activity]' template show improved CLIP Score, ImageReward, and VQAScore relative to DDPO and B2-DiffuRL.

Significance. If the central mechanism were sound, SRRL would be a notable contribution: it is clearly presented, uses open-source backbones (SD v1.4 and SD XL), provides pseudocode, prompt lists, and a public project page, and the Limitations section honestly notes that the reward models are not reasoning-specific. However, the core novelty is the condition-guided forward process (Eq. 7-9 and Appendix A.2), and that derivation contains a load-bearing mathematical error. Moreover, the only quantitative evidence is obtained by optimizing and evaluating on the same reward models, so the reported gains are partly by construction. The qualitative physical-law results, while suggestive, are not backed by a quantitative benchmark or inter-rater evaluation. Thus, as it stands, the paper does not establish a learned image-reasoning capability.

major comments (4)
  1. [§3.3, Eq. (8), and Appendix A.2] The inversion used for the condition-guided forward process is not the inverse of the DDIM denoising step. The standard deterministic DDIM update is x_{t-1} = sqrt(alpha_bar_{t-1}/alpha_bar_t) x_t + (sqrt(1-alpha_bar_{t-1}) - sqrt(alpha_bar_{t-1}(1-alpha_bar_t)/alpha_bar_t)) epsilon_theta(x_t,t), with sqrt(alpha_bar_{t-1}/alpha_bar_t) = 1/sqrt(alpha_t). Equation (8) instead writes sqrt(alpha_bar_{t-1})/sqrt(alpha_t) as the coefficient of x_t, and Eq. (9) inherits this error. Because the forward process does not properly invert the denoising process, the repeated renoising loop is not guaranteed to preserve the semantic content of the intermediate image, and the claim that the trajectory is a valid chain of thought is unsupported.
  2. [Appendix A.2, Eq. (13)] The derivation of condition injection through the guidance gap discards the unconditional-noise term. The difference between CFG-weighted noise predictions at guidance scales lambda_forward and lambda is (lambda_forward - lambda)(epsilon_theta(x,c,t) - epsilon_theta(x,phi,t)), not (lambda_forward - lambda) epsilon_theta(x,c,t) as written in Eq. (13). The omitted unconditional term is exactly the term that classifier-free guidance is designed to amplify, and no argument is given that it is negligible. Without this term, the conclusion that a guidance gap injects the text condition is not established, and the method's core mechanism remains unproven.
  3. [§4.6, Table 1, and Fig. 8] The quantitative evaluation uses CLIP Score, VQAScore, and ImageReward, which are the same functions used as reward models during RL training. Reported improvements over DDPO and B2-DiffuRL therefore partly reflect direct optimization of the evaluation metrics, not a generalizable reasoning capability. This is consistent with the Limitations section, which acknowledges that these reward models are designed for text-image alignment or human preference rather than physical-law reasoning. The paper provides no quantitative evaluation on the physical-law or counterintuitive prompts, no held-out reward models, and no error bars; the central claim of image reasoning is therefore supported only by selected qualitative examples.
  4. [§4.5, Fig. 6] The comparison with GPT-4o is based on a handful of selected images with no evaluation protocol, no scoring criteria, and no inter-rater reliability measure. The visual styles differ substantially (SRRL produces photorealistic images, GPT-4o produces cartoon-like images), so direct visual comparison is not a controlled test of reasoning quality. The abstract's claim that SRRL exhibits 'superior performance even compared with GPT-4o' is not substantiated by the evidence presented.
minor comments (8)
  1. [§3.2] The term 'process reward model' is misleading: the reward r(x^k_0,c) is computed on the denoised image at the end of each round, not on intermediate noisy states. If the authors intend 'process' to mean across reflection rounds, this should be defined explicitly and distinguished from the standard meaning in RL.
  2. [Algorithm 1 and Algorithm 2] The pseudocode says 'Noise injection x_i^0 to x_{i+1}^T with p_theta' without specifying the condition-guided forward parameters; Algorithm 2 mentions DDIM inversion scheduler, but the training algorithm does not. The relationship between the training-time forward guidance scale (0.5) and inference-time scales should be stated in the pseudocode.
  3. [Eq. (7) and Eq. (10)] The notation is ambiguous: gamma_t is defined as sqrt(alpha_t/alpha_bar_{t-1}) in Eq. (10), but the product in Eq. (11) runs over gamma_i for i=0 to T, where gamma_0 is never defined; also the subscripts of eta_t and gamma_t are inconsistent. This makes it difficult to reproduce the claimed telescoping form.
  4. [Fig. 8] The labels 'Partial Alignment', 'Fully Alignment', and 'Reflection Refinement' are not defined in the text or the caption; they appear to be qualitative phase labels, but the reader is left to guess their meaning.
  5. [Table 1] No error bars, standard deviations, or number of seeds are reported. Given the small differences among methods (e.g., CLIP Score 0.3624 vs. 0.3662), statistical significance is unclear.
  6. [§4.6] The statement that CLIP Score 'tends to degrade when the number of training epochs is too high' is not supported by any experiment or figure in the paper; either add the evidence or remove the claim.
  7. [Related Work] The paper does not compare against Zigzag diffusion sampling [1], which also uses CFG-based self-reflection in the sampling process. A direct comparison would help clarify the difference between a learned reflection capability and an inference-time heuristic.
  8. [Figures and text] Minor language and typographical issues: 'a ant' in Fig. 10, 'cold coffer' in Fig. 4 caption, and 'condition injection reflection forward process' in Appendix A.2 are awkward. A careful proofread is needed.

Circularity Check

2 steps flagged · score 6.0 of 10

Reported gains are partly circular: SRRL is trained with the same CLIP/ImageReward/VQAScore functions it reports as evidence, and the 'process reward' is defined as the round-end outcome reward, so the PRM claim is a relabeling of ORM evaluation.

  1. self definitional [Sec. 3.2, Eq. (6) and reward definition below it]
    "After each round of the denoising process, SRRL evaluates intermediate images using reward models, which provide process rewards for the entire multi-round process. ... The reward includes process rewards of intermediate samples: R(sk_t, ak_t) = {r(xk_0, c), if t=0; 0, otherwise."

    The stated motivation for PRMs is that 'intermediate samples in the denoising process carry noise, making accurate reward evaluation difficult.' But the process reward is defined to be r(xk_0,c) at t=0 only: the clean image at the end of each round, scored by the same outcome reward model used for ordinary RL. No noisy intermediate xt is ever rewarded, so the 'process reward' is, by construction, the outcome reward of that round's endpoint. The claimed contribution of 'allowing process reward models (PRMs) to address the issue' is therefore a relabeling of repeated outcome-reward evaluation, not a new mechanism for scoring noisy intermediates.

  2. fitted input called prediction [Sec. 4.1 'Reward models and metrics'; Appendix B configuration table; Table 1]
    "We use CLIP Score [9], ImageReward [53], and VQAScore [26] to evaluate the text-image alignment and image reasoning abilities of models. ... reward function r CLIP Score [9], ImageReward [53], VQAScore [26] ... CLIP Score↑ 0.3624 ... ImageReward↑ 0.2823 ... VQAScore↑ 0.6045"

    The same three functions serve as the RL reward in Eq. (6) and as the reported evaluation metrics in Table 1. Policy-gradient training with advantages computed from r(x0,c) directly maximizes these functions on the training prompts, so a post-training increase in CLIP Score, ImageReward, and VQAScore is partly forced by construction. Reporting these gains as evidence of 'image reasoning ability' is thus an evaluation on the training objective; it cannot independently establish a learned reasoning capability beyond optimizing the specified reward models.

full rationale

No load-bearing self-citation chain is present: the authors' earlier papers are cited only as related work, and the central mechanism does not depend on a uniqueness theorem or an imported ansatz. The two circular steps above are definitional and optimizational rather than external. The Appendix A.2 derivation of the condition guided forward process is also unsound (Eq. 8 misstates the DDIM coefficient and Eq. 13 drops the unconditional CFG term), but I treat that as a correctness risk rather than circularity because the failure is an algebraic error, not a self-referential reduction. Since the quantitative evaluation is performed on the precise reward functions being optimized and the 'process reward' is the round-end outcome reward, the reported improvements and the PRM framing are partially circular; qualitative case studies retain some independent content, so the paper is not fully reducible to its inputs. Score 6 reflects this partial, construction-level circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on hand-chosen guidance scales, a fixed number of rounds, and an approximate inversion step. No new physical entities are introduced. The evaluation metrics are the training rewards, which is the main circularity burden.

free parameters (4)
  • forward guidance scale = 0.5 (App B); 1.0 (Sec 3.3)
    Manual choice that the condition injection claim depends on; the guidance gap between forward and denoising is central to the method.
  • denoising guidance scale = 3.0 (App B); 4.5 (Sec 3.3)
    Manual choice; the gap is used to justify text injection in the forward process.
  • self-reflection rounds K = 10
    Number of reflective rounds; the claimed reasoning improvement grows with K (Fig. 8), and K is chosen without ablation.
  • LoRA rank and alpha = 4 and 4
    Capacity of the fine-tuned adapter; chosen without ablation.
assumptions (4)
  • domain assumption epsilon_theta(~x_{t-1}, c, t, lambda) is approximately epsilon_theta(~x_t, c, t, lambda)
    Used in Appendix A.2 to derive the condition guided forward process (Eq. 9); standard DDIM inversion approximation but not exact, and the paper does not quantify the error.
  • ad hoc to paper A CFG guidance scale gap between forward and denoising injects text condition and yields progressively better samples
    Stated in Sec 3.3 and Appendix A.2 (Eq. 13); the derivation assumes linearity in lambda and drops the unconditional noise term, so the claim is not established.
  • domain assumption Reward models (CLIP Score, ImageReward, VQAScore) are valid proxies for image reasoning ability
    Used as both training reward and evaluation metric; their alignment with 'reasoning' is assumed, not validated.
  • standard math The diffusion denoising process is a Markov decision process
    Standard DDPO formulation; accepted in prior work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generation." pith.science (2026). https://pith.science/paper/QVYETUF7

@misc{pith2026250522407,
  author       = {Pith},
  title        = {Pith review of: Self-Reflective Reinforcement Learning for Diffusion-based Image Reasoning Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QVYETUF7}},
  note         = {Machine review of arXiv:2505.22407}
}
read the original abstract

Diffusion models have recently demonstrated exceptional performance in image generation task. However, existing image generation methods still significantly suffer from the dilemma of image reasoning, especially in logic-centered image generation tasks. Inspired by the success of Chain of Thought (CoT) and Reinforcement Learning (RL) in LLMs, we propose SRRL, a self-reflective RL algorithm for diffusion models to achieve reasoning generation of logical images by performing reflection and iteration across generation trajectories. The intermediate samples in the denoising process carry noise, making accurate reward evaluation difficult. To address this challenge, SRRL treats the entire denoising trajectory as a CoT step with multi-round reflective denoising process and introduces condition guided forward process, which allows for reflective iteration between CoT steps. Through SRRL-based iterative diffusion training, we introduce image reasoning through CoT into generation tasks adhering to physical laws and unconventional physical phenomena for the first time. Notably, experimental results of case study exhibit that the superior performance of our SRRL algorithm even compared with GPT-4o. The project page is https://jadenpan0.github.io/srrl.github.io/.

Figures

Figures reproduced from arXiv: 2505.22407 by the authors.

Figure 1
Figure 1. Illustration of self-reflective reasoning step. Through self-reflective processes of repeated [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of SRRL. SRRL includes two processes: multi-round reflective denoising process [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Reasoning generation of images related to physical laws. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Reasoning generation of images related to unconventional physical phenomena. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Reasoning generation process of the prompt related to a balance. Initially, the model [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Example of generated images of physical phenomenon related prompts and unconventional [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Examples of prompt template "a(n) [animal] [activity]" by baselines [ [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Performance of multi-round self-reflection of SRRL. The left is results of physical phe [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Reasoning generation process of prompt related to cars. Common sense dictates that cars [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Prompts of the template "a(n) [animal] [activity]". [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: Physical phenomenon related prompts. E.3 Unconventional Physical Phenomena Prompts "A ball rolling uphill against gravity, surprising onlookers.", "Dominoes falling in reverse, standing themselves up.", "Detective finds a footprint leading to a floating shoe.", "A mag…
Figure 12
Figure 12. Figure 12: Unconventional physical phenomena prompts. [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 15 canonical work pages

  1. [1]

    Zigzag diffusion sampling: Diffusion models can self-improve via self-reflection

    Lichen Bai, Shitong Shao, Zikai Zhou, Zipeng Qi, Zhiqiang Xu, Haoyi Xiong, and Zeke Xie. Zigzag diffusion sampling: Diffusion models can self-improve via self-reflection. In The Thirteenth International Conference on Learning Representations, volume 2, 2024

    work page 2024

  2. [2]

    Training diffusion models with reinforcement learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305.13301, 2023

    arXiv 2023

  3. [3]

    Classifier-free guidance is a predictor-corrector

    Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is a predictor-corrector. arXiv preprint arXiv:2408.09000, 2024

    arXiv 2024

  4. [4]

    Cfg++: Manifold-constrained classifier free guidance for diffusion models

    Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold-constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2406.08070, 2024

    arXiv 2024

  5. [5]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in NeurIPS, 34:8780–8794, 2021

    work page 2021

  6. [6]

    Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858–79885, 2023

    2023

  7. [7]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    arXiv 2025

  8. [8]

    Can we generate images with cot? let’s verify and reinforce image generation step by step

    Ziyu Guo, Renrui Zhang, Chengzhuo Tong, Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let’s verify and reinforce image generation step by step. arXiv preprint arXiv:2501.13926, 2025

    arXiv 2025

Show all 56 references
  1. [9]

    Clipscore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. In EMNLP, pages 7514–7528, 2021

    2021

  2. [10]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in NeurIPS, 33:6840–6851, 2020

    2020

  3. [11]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  4. [12]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 10

    2022

  5. [13]

    Towards better alignment: Training diffusion models with reinforcement learning against sparse rewards

    Zijing Hu, Fengda Zhang, Long Chen, Kun Kuang, Jiahui Li, Kaifeng Gao, Jun Xiao, Xin Wang, and Wenwu Zhu. Towards better alignment: Training diffusion models with reinforcement learning against sparse rewards. arXiv preprint arXiv:2503.11240, 2025

    2025 arXiv

  6. [14]

    T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723–78747, 2023

    2023

  7. [15]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  8. [16]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jia- jun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024

    2024 arXiv

  9. [17]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  10. [18]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024

    2024 arXiv

  11. [19]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    2024 arXiv

  12. [20]

    T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot

    Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo, Shilin Yan, Pheng- Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and token-level cot. arXiv preprint arXiv:2505.00703, 2025

    2025 arXiv

  13. [21]

    Comat: Aligning text-to-image diffusion model with image-to-text concept matching

    Dongzhi Jiang, Guanglu Song, Xiaoshi Wu, Renrui Zhang, Dazhong Shen, Zhuofan Zong, Yu Liu, and Hongsheng Li. Comat: Aligning text-to-image diffusion model with image-to-text concept matching. Advances in Neural Information Processing Systems , 37:76177–76209, 2024

    2024

  14. [22]

    Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency

    Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanwei Li, Yu Qi, Xinyan Chen, Liuhui Wang, Jianhan Jin, Claire Guo, Shen Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency. arXiv preprint arXiv:2502.09621, 2025

    2025 arXiv

  15. [23]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  16. [24]

    Self-guidance: Boosting flow and diffusion generation on their own

    Tiancheng Li, Weijian Luo, Zhiyang Chen, Liyuan Ma, and Guo-Jun Qi. Self-guidance: Boosting flow and diffusion generation on their own. arXiv preprint arXiv:2412.05827, 2024

    2024

  17. [25]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023

    2023

  18. [26]

    Evaluating text-to-visual generation with image-to-text generation

    Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pages 366–384. Springer, 2024

    2024

  19. [27]

    Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models. CoRR, 2023

    2023

  20. [28]

    Safe-sd: Safe and traceable stable diffusion with text prompt trigger for invisible generative watermarking

    Zhiyuan Ma, Guoli Jia, Biqing Qi, and Bowen Zhou. Safe-sd: Safe and traceable stable diffusion with text prompt trigger for invisible generative watermarking. In ACM Multimedia 2024. 11

    2024

  21. [29]

    Adapedit: Spatio-temporal guided adaptive edit- ing algorithm for text-based continuity-sensitive image editing

    Zhiyuan Ma, Guoli Jia, and Bowen Zhou. Adapedit: Spatio-temporal guided adaptive edit- ing algorithm for text-based continuity-sensitive image editing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4154–4161, 2024

    2024

  22. [30]

    Efficient diffusion models: A comprehensive survey from principles to practices

    Zhiyuan Ma, Yuzhu Zhang, Guoli Jia, Liangliang Zhao, Yichao Ma, Mingjie Ma, Gaofeng Liu, Kaiyan Zhang, Ning Ding, Jianjun Li, et al. Efficient diffusion models: A comprehensive survey from principles to practices. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  23. [31]

    Neural residual diffusion models for deep scalable vision generation

    Zhiyuan Ma, Liangliang Zhao, Biqing Qi, and Bowen Zhou. Neural residual diffusion models for deep scalable vision generation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    2024

  24. [32]

    Tango 2: Aligning diffusion-based text-to-audio generations through direct pref- erence optimization

    Navonil Majumder, Chia-Yu Hung, Deepanway Ghosal, Wei-Ning Hsu, Rada Mihalcea, and Soujanya Poria. Tango 2: Aligning diffusion-based text-to-audio generations through direct pref- erence optimization. In Proceedings of the 32nd ACM International Conference on Multimedia, pages...

    2024

  25. [33]

    Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, et al. Mm-eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365, 2025

    2025 arXiv

  26. [34]

    Distributed repre- sentations of words and phrases and their compositionality

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed repre- sentations of words and phrases and their compositionality. Advances in neural information processing systems, 26, 2013

    2013

  27. [35]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 42...

    2024

  28. [36]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, pages 8162–8171, 2021

    2021

  29. [37]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  30. [38]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  31. [39]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, pages 8748–8763, 2021

    2021

  32. [40]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  33. [41]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, pages 8821–8831, 2021

    2021

  34. [42]

    Diffusion policy policy optimization

    Allen Z Ren, Justin Lidard, Lars L Ankile, Anthony Simeonov, Pulkit Agrawal, Anirudha Majumdar, Benjamin Burchfiel, Hongkai Dai, and Max Simchowitz. Diffusion policy policy optimization. arXiv preprint arXiv:2409.00588, 2024

    2024 arXiv

  35. [43]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 12

    2022

  36. [44]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22500...

    2023

  37. [45]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information...

    2022

  38. [46]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  39. [47]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020

    2010 arXiv

  41. [49]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438–12448, 2020

    2020

  42. [50]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2020

    2020

  43. [51]

    Diffusion model alignment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and...

    2024

  44. [52]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  45. [53]

    Imagereward: Learning and evaluating human preferences for text-to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903–15935, 2023

    2023

  46. [54]

    R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization

    Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503.10615, 2025

    2025 arXiv

  47. [55]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023

    2023

  48. [56]

    a(n) [animal] [activity]

    Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Ziyu Guo, Shicheng Li, Yichi Zhang, Chengzhuo Tong, Jiaming Liu, Aojun Zhou, Bin Wei, et al. Mavis: Mathematical visual instruction tuning with an automatic data engine. arXiv preprint arXiv:2407.08739, 2024. 13 A Derivations A.1 Equatio...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.