Pith. sign in

REVIEW 4 major objections 5 minor 56 references

Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rhet2Pix outperforms GPT-4o and Grok-3 at generating images from rhetorical language.

desk verdict Rhet2Pix is a plausible engineering recipe for steering diffusion models away from literal metaphor vehicles, but its headline numbers are circular: Table 1's metrics appear to be the training rewards, so the SOTA claim over GPT-4o and Grok-3 is unsupported. read the letter →

arxiv 2505.22792 v2 pith:SIGZTV5X submitted 2025-05-28 cs.CV

Yuxi Zhang , Yueting Li , Xinyu Du , Sibo Wang This is my paper
classification cs.CV
keywords rhetoricaltext-to-imagegenerationmetaphorvisualizationdiffusionmodelsreinforcementlearningMarkovdecisionprocessrewardsparsityPPOpromptdecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that rhetorical text-to-image generation fails because models align images to literal word embeddings of the metaphorical vehicle, and proposes Rhet2Pix, a framework that recasts the task as a two-layer Markov decision process. The outer layer decomposes a rhetorical prompt into progressively enriched sub-prompts, while the inner layer treats diffusion denoising as a policy optimization problem with discounts that spread a final reward across all denoising steps. On a filtered FLUTE metaphor and simile benchmark, Rhet2Pix reports higher semantic and elemental alignment scores than GPT-4o, Grok-3, and academic baselines. The claim matters because it suggests that a comparatively small fine-tuned Stable Diffusion model can outperform much larger multimodal models on figurative language.

What carries the argument

The central object is a two-layer Markov decision process. The outer MDP models the staged progression of prompts: the state is the image generated at a stage and the action is the semantic increment added to the prompt; advantages are computed with Generalized Advantage Estimation. The inner MDP models diffusion denoising: the state is the latent $x_t$ with prompt and timestep, the action is the next denoised latent $x_{t-1}$, and the terminal reward is propagated to each step via $\hat{A}^{(j)}_t = \gamma_{\text{denoise}}^t \hat{A}_j$. This discounting is what the paper claims solves reward sparsity, because every adjacent action pair gets a nonzero advantage. Policy updates use a PPO clipped surrogate over shuffled action pairs.

What would settle it

Run a human study where annotators choose which image best captures the figurative meaning of a set of rhetorical prompts, comparing Rhet2Pix against GPT-4o and Grok-3. If annotators do not prefer Rhet2Pix images at a clear rate, or if the reported alignment scores do not predict those choices, the central claim would be refuted.

Watch

Extended reading notes

Core claim

Rhet2Pix establishes that the literal-visual bias in rhetorical image generation can be corrected by separating semantic planning from pixel synthesis. In the outer layer, an LLM extracts key factors (subject, vehicle, theme, emotion) and builds a sequence of staged prompts, each adding context while preserving the subject. In the inner layer, the denoising trajectory of a Stable Diffusion model is optimized with PPO, where each outer-layer advantage is discounted backward across denoising steps so that early actions receive credit for the final rhetorical alignment. A combined reward penalizes the presence of the metaphorical vehicle and rewards staged semantic alignment and aesthetic quality. The paper reports that on its benchmark Rhet2Pix achieves a Sum-alignment of 0.6671 versus 0.4192 for Grok-3, and qualitatively produces images that depict the intended subject rather than the literal vehicle.

Load-bearing premise

The evaluation assumes that the CLIP-similarity scores used to measure alignment are valid measures of rhetorical fidelity, but these scores appear to be the same terms used to train the model, and no human or external benchmark verifies that they correspond to correct metaphorical images.

Editorial extensions

If this is right

  • If correct, small diffusion models fine-tuned on rhetorical data can outperform general MLLMs on figurative language without architectural changes.
  • The two-layer MDP with discounted rewards could be applied to other long-horizon generation tasks where feedback is only available at the end, such as text-to-video or layout generation.
  • The vehicle-penalty reward offers a reusable recipe for preventing literal over-alignment in any task with metaphorical or abstract semantics.
  • Because only the final prompt is used at inference, the method adds no extra generation cost, only a fine-tuning cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own limitation note admits that a fundamentally new backbone beyond text-image embedding similarity may be needed; this suggests the gains may be bounded by the CLIP embedding space used for both reward and evaluation.
  • A natural extension would be to test Rhet2Pix on other figures of speech such as hyperbole or irony, which may not map to concrete visual subjects as directly as metaphor and simile.
  • The reported elemental-alignment advantage for Rhet2Pix (0.4017 vs 0.1741) may partly reflect that the vehicle penalty is hand-tuned on the same distribution, so a distribution shift in rhetorical prompts could erode the gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Rhet2Pix, a framework for rhetorical text-to-image generation that formulates the task as a two-layer Markov Decision Process and fine-tunes a Stable Diffusion backbone with a PPO-style policy optimization. In the outer layer, an LLM decomposes a rhetorical prompt into staged sub-prompts whose images are evaluated by a rhetorical reward; in the inner layer, the final reward is discounted backward along the denoising trajectory to address reward sparsity. The paper reports quantitative results in Table 1 claiming that Rhet2Pix outperforms GPT-4o, Grok-3, and several academic baselines on 'semantic-alignment' and 'elemental-alignment' metrics, and provides qualitative examples and ablations.

Significance. If the claims were supported, the paper would make a useful contribution to an underexplored task: generating images that preserve figurative meaning rather than literal vehicle depiction. The paper is also commendable for planning to release code and data. However, the headline quantitative result is not currently supported because the evaluation metrics in Table 1 are not defined independently of the training reward in Section 4.5. The paper's own limitation statement in Appendix C further weakens the validity of CLIP-based alignment as the sole measure of rhetorical fidelity. As a result, the empirical contribution is unverified.

major comments (4)
  1. [Section 5.2, Table 1] The evaluation metrics are never defined in Section 5.2, and their names ('Semantic-alignment', 'Elemental-alignment') exactly match the reward terms 'staged semantic alignment' and 'elemental alignment' in Section 4.5. The formulas there are CLIP cosine similarities, including the manually tuned vehicle penalty threshold tau. Because Rhet2Pix is trained via PPO to maximize these same quantities, its higher scores on this table are a direct consequence of optimizing the evaluation metric rather than evidence of better rhetorical understanding. The comparison to GPT-4o and Grok-3 is therefore circular. The authors must either define the evaluation metrics as external, independent measures, or replace this table with human evaluation or a standard benchmark (e.g., MetaCLUE or T2I-CompBench) applied to all models under identical conditions.
  2. [Appendix C] The paper explicitly concedes that fully addressing rhetorical text-to-image generation 'may necessitate a fundamentally new backbone—one that surpasses conventional text-image embedding similarity.' This statement directly contradicts the validity of using CLIP-based embedding similarities as the sole evaluation metric for rhetorical fidelity in Table 1. If the authors believe this limitation statement, then the quantitative evaluation cannot be used to substantiate the claim of outperforming SOTA MLLMs. The paper should either justify how the CLIP metrics capture rhetorical meaning despite this acknowledged limitation, or adopt a more appropriate evaluation protocol.
  3. [Section 5.3, Figure 5] The ablation study is presented only qualitatively, with a few image examples. The claims that 'scene modeling' and 'reinforcement learning' are crucial for performance are not supported by any quantitative measurement. Without quantitative ablations using the same (independent) evaluation protocol as the main comparison, it is impossible to assess the contribution of each component or to rule out that the observed qualitative differences are due to cherry-picked examples.
  4. [Section 4.5] The reward function contains several manually set free parameters that are not analyzed: the monotonic weight vector w(j) for staged alignment, the vehicle penalty threshold tau, and the discount factors gamma, lambda, and gamma_denoise. In particular, tau is described as 'manually determined by empirical analysis of CLIP similarity scores'—the same score distribution used in the evaluation table. This creates a risk that the reported improvements reflect tuning of the evaluation-aligned reward rather than a general advance. A sensitivity analysis of these parameters is needed to establish robustness.
minor comments (5)
  1. [Section 4.3/4.4] The inner-layer MDP defines the action as a_t = x_{t-1}, but Section 4.4 repeatedly refers to 'adjacent action pairs (s_t, s_{t+1})'. This is inconsistent: the transition pair should be (s_t, a_t) or (s_t, s_{t-1}), depending on the time indexing. Please clarify the notation.
  2. [Algorithm 1] Line 49 defines the critic loss as E[(V_phi(s_t) - Ahat)^2], which is not the standard value loss used in PPO (typically (V_phi(s_t) - (Ahat + V_old(s_t)))^2 or a similar target). This may lead to biased value estimation and could affect the reported training behavior.
  3. [Section 4.5] The staged semantic alignment reward assumes each prompt P^j_i is decomposed into sub-sentences S^k_i, but the paper does not specify how this decomposition is obtained. A reproducible description (e.g., an LLM prompt or parsing rule) is needed.
  4. [Table 3] The hyperparameter table describes a ViT-based critic and its architectural components, but the main text does not describe the critic architecture. Consider moving a brief description into the main text or at least referencing the table when the critic is introduced in Section 4.4.
  5. [Throughout] There are several minor grammatical issues and awkward phrasings (e.g., 'environmental steps' in Figure 8, inconsistent capitalization of 'state-of-the-art'). A careful proofreading pass would improve readability.

Circularity Check

1 steps flagged · score 8.0 of 10

Table 1's Semantic-alignment and Elemental-alignment scores are the paper's own Sec. 4.5 training rewards, so the SOTA claim reduces by construction to reward maximization.

  1. fitted input called prediction [Section 5.2, Table 1 (evaluation) and Section 4.5 'Tailored Reward Mechanism' (training reward)]
    "Quantitative evaluation. As reported in Table 1, Rhet2Pix performs consistently well across all alignment evaluation metrics, achieving a clear lead in overall performance."

    Table 1's 'Semantic-alignment' and 'Elemental-alignment' columns are not defined in Sec. 5.2, but their names reproduce the reward modules defined in Sec. 4.5: 'staged semantic alignment' (r_stage = Σ_j w_j ⟨v, u_j⟩) and 'elemental alignment' (r_subject = Σ_k ⟨v, f_text(k)/||f_text(k)||⟩, with the manually fit vehicle-penalty threshold τ). Rhet2Pix is trained via PPO (Eq. 3) to maximize exactly these CLIP-cosine rewards, so its Sum-alignment 0.6671 versus Grok-3's 0.4192 is the expected consequence of optimizing the evaluation metric, not evidence of better rhetorical understanding.

full rationale

The paper's RL fine-tuning procedure is a genuine training method, and the qualitative samples provide some visual evidence. However, the central quantitative claim of outperforming GPT-4o and Grok-3 rests on Table 1, whose Semantic-alignment and Elemental-alignment columns coincide, by name and formula, with the staged-semantic and elemental reward terms in Sec. 4.5 that Rhet2Pix was trained to maximize. Because the evaluation metric is the training objective, the reported lead is forced by construction; no independent metric definition, human rating, or external benchmark (e.g., MetaCLUE or T2I-CompBench) is provided. Appendix C warns that conventional text-image embedding similarity may be insufficient for true rhetorical generation, which is precisely the similarity used in both training and evaluation. There is no load-bearing self-citation or imported uniqueness theorem; the circularity is in the evaluation design, not in the training algorithm.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The framework does not introduce new physical or conceptual entities, but it relies on several tuned parameters and unstated assumptions about CLIP validity, LLM reliability, denoising-step importance, and reward decomposition. The missing hyperparameters alone prevent reproduction.

free parameters (6)
  • vehicle penalty threshold tau
    Manually determined from CLIP similarity statistics (Sec. 4.5); triggers a -1.0 penalty when the literal vehicle appears. Value not reported.
  • staged semantic-alignment weights w(j)
    Monotonic decreasing weights over sub-sentences in Sec. 4.5; values not reported.
  • GAE discount factors gamma and lambda
    Used in Eq. (1) to compute outer-layer advantages; not listed in Table 3.
  • inner-layer discount gamma_denoise
    Used in Sec. 4.4 and Algorithm 1 to weight denoising timesteps; not listed in Table 3.
  • number of stages C
    Outer-layer horizon for staged prompts; examples use 3, but the value for experiments is unspecified.
  • dataset filtering criteria
    Sec. 5.1: 'additional filtering step to select high-quality metaphor and simile samples'; criteria and counts not reported.
assumptions (4)
  • domain assumption CLIP cosine similarity is a valid proxy for semantic and elemental alignment in reward and evaluation.
    The reward and apparent metrics rely on CLIP embeddings (Sec. 4.5). No external validation ties CLIP similarity to correct rhetorical interpretation.
  • domain assumption GPT-4o generate-verify-retry produces valid key factors and staged prompts.
    The outer-layer prompts are generated by GPT-4o with a self-verification loop (Sec. 4.2). No human audit of the extracted factors or staged prompts is reported.
  • ad hoc to paper Later denoising steps matter more than early steps for semantic detail, justifying gamma_denoise weighting.
    Sec. 4.4 and Appendix B assert this based on a citation to Hu et al. 2025, but the specific exponential weighting is asserted without derivation or ablation.
  • ad hoc to paper The reward decomposition into semantic, elemental, and aesthetic terms is complete and correctly balanced.
    The three reward modules in Sec. 4.5 are combined without reported weights scaling and without ablations isolating each component.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization." pith.science (2026). https://pith.science/paper/SIGZTV5X

@misc{pith2026250522792,
  author       = {Pith},
  title        = {Pith review of: Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SIGZTV5X}},
  note         = {Machine review of arXiv:2505.22792}
}
read the original abstract

Generating images from rhetorical languages remains a critical challenge for text-to-image models. Even state-of-the-art (SOTA) multimodal large language models (MLLM) fail to generate images based on the hidden meaning inherent in rhetorical language--despite such content being readily mappable to visual representations by humans. A key limitation is that current models emphasize object-level word embedding alignment, causing metaphorical expressions to steer image generation towards their literal visuals and overlook the intended semantic meaning. To address this, we propose Rhet2Pix, a framework that formulates rhetorical text-to-image generation as a multi-step policy optimization problem, incorporating a two-layer MDP diffusion module. In the outer layer, Rhet2Pix converts the input prompt into incrementally elaborated sub-sentences and executes corresponding image-generation actions, constructing semantically richer visuals. In the inner layer, Rhet2Pix mitigates reward sparsity during image generation by discounting the final reward and optimizing every adjacent action pair along the diffusion denoising trajectory. Extensive experiments demonstrate the effectiveness of Rhet2Pix in rhetorical text-to-image generation. Our model outperforms SOTA MLLMs such as GPT-4o, Grok-3 and leading academic baselines across both qualitative and quantitative evaluations. The code and dataset used in this work are publicly available.

Figures

Figures reproduced from arXiv: 2505.22792 by the authors.

Figure 1
Figure 1. Rhet2Pix framework. Our approach formulates rhetorical image generation as a two-layer Markov Decision Process. model to generate its literal visual counterpart. This cre￾ates a counter-effect — stronger object-level correspon￾dence results in images that deviate further from the in￾tended semantic meaning. This issue highlights a fun￾damental limitation in current MLLMs, which predom￾inantly depend on surface-level… view at source ↗
Figure 2
Figure 2. Sampling and optimization over adjacent action pairs. We collect denoising trajectories across N rhetorical inputs, each with C staged prompts and T diffusion steps, yielding N × C × T adjacent action pairs. These pairs are shuffled across input, stage, and timestep dimensions to re￾move correlation. Sampled pairs are used to compute action likelihoods under current and past policies, and the policy is updated via P… view at source ↗
Figure 3
Figure 3. Effectiveness of our reward mechanism in cap￾turing rhetorical semantics. From left to right, the reward increases with images that better align with the underlying rhetorical meaning, receiving higher scores. All examples are generated from the prompt “My bedroom is a pig sty.” 5 Experiments 5.1 Experiment settings Dataset. In this work, we focus on metaphor (including personification) and simile, as they often des… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison across models on rhetorical prompts. Rhet2Pix generates semantically faithful and visu￾ally coherent images by correctly distinguishing metaphorical vehicles from intended subjects. Baseline models often fail to capture rhetorical meaning, produc…
Figure 5
Figure 5. Figure 5: Ablation study results. Scene modeling enhances semantic alignment by refining the interpretation of rhetorical text, while our method further integrates reinforcement learning to improve artistic quality and image detail. results reflect its ability to balance high-le…
Figure 6
Figure 6. Figure 6: Illustrative example of the inference process of [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Illustrative example of the multi-stage image generation pipeline in Rhet2Pix. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Reward curve of our method and DDPO when fine [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: More samples generated by our method compared with other baselines [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: More samples generated by our method compared with other baselines [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

56 extracted references · 18 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    R.; Driscoll, B.; Narayana, P.; Changpinyo, S.; Jia, Z.; Damle, S.; Pruthi, G.; Basu, S.; Guibas, L.; Freeman, W

    Akula, A. R.; Driscoll, B.; Narayana, P.; Changpinyo, S.; Jia, Z.; Damle, S.; Pruthi, G.; Basu, S.; Guibas, L.; Freeman, W. T.; et al. 2023. Metaclue: Towards comprehensive visual metaphors research. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 23201--23211

    work page 2023

  4. [4]

    T.; Yang, Z.; Qi, Z.; Han, J.; Sahoo, S

    Arriola, M.; Gokaslan, A.; Chiu, J. T.; Yang, Z.; Qi, Z.; Han, J.; Sahoo, S. S.; and Kuleshov, V. 2025. Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models. arXiv preprint arXiv:2503.09573

    arXiv 2025

  5. [5]

    Batzolis, G.; Stanczuk, J.; Sch \"o nlieb, C.-B.; and Etmann, C. 2021. Conditional image generation with score-based diffusion models. arXiv preprint arXiv:2111.13606

    arXiv 2021

  6. [6]

    Betker, J.; Goh, G.; Jing, L.; Brooks, T.; Wang, J.; Li, L.; Ouyang, L.; Zhuang, J.; Lee, J.; Guo, Y.; et al. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3): 8

    2023

  7. [7]

    Black, K.; Janner, M.; Du, Y.; Kostrikov, I.; and Levine, S. 2024. Training Diffusion Models with Reinforcement Learning. arXiv:2305.13301

    arXiv 2024

  8. [8]

    Brade, S.; Wang, B.; Sousa, M.; Oore, S.; and Grossman, T. 2023. Promptify: Text-to-image generation through interactive prompt exploration with large language models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1--14

    work page 2023

Show all 56 references
  1. [10]

    Chakrabarty, T.; Saakyan, A.; Ghosh, D.; and Muresan, S. 2022 b . FLUTE: Figurative Language Understanding through Textual Explanations. arXiv:2205.12404

    2022 arXiv

  2. [11]

    Chakrabarty, T.; Saakyan, A.; Winn, O.; Panagopoulou, A.; Yang, Y.; Apidianaki, M.; and Muresan, S. 2023. I spy a metaphor: Large language models and diffusion models co-create visual metaphors. arXiv preprint arXiv:2305.14724

    2023 arXiv

  3. [12]

    Chakrabarty, T.; Zhang, X.; Muresan, S.; and Peng, N. 2021. MERMAID: Metaphor generation with symbolism and discriminative decoding. arXiv preprint arXiv:2103.06779

    2021 arXiv

  4. [13]

    Dai, D.; Dong, L.; Hao, Y.; Sui, Z.; Chang, B.; and Wei, F. 2021. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696

    2021 arXiv

  5. [14]

    Dai, Z.; Asgharivaskasi, A.; Duong, T.; Lin, S.; Tzes, M.-E.; Pappas, G.; and Atanasov, N. 2024. Optimal Scene Graph Planning with Large Language Model Guidance. arXiv:2309.09182

    2024 arXiv

  6. [15]

    Dong, H.; Xiong, W.; Goyal, D.; Zhang, Y.; Chow, W.; Pan, R.; Diao, S.; Zhang, J.; Shum, K.; and Zhang, T. 2023. Raft: Reward ranked finetuning for generative foundation model alignment. arXiv preprint arXiv:2304.06767

    2023 arXiv

  7. [16]

    Fan, Y.; Watkins, O.; Du, Y.; Liu, H.; Ryu, M.; Boutilier, C.; Abbeel, P.; Ghavamzadeh, M.; Lee, K.; and Lee, K. 2023. DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Models. arXiv:2305.16381

    2023 arXiv

  8. [17]

    K.; Wang, S.; Lu, Y.; Zhu, M.; Wang, B.; and Chen, W

    Feng, Y.; Wang, X.; Wong, K. K.; Wang, S.; Lu, Y.; Zhu, M.; Wang, B.; and Chen, W. 2023. Promptmagician: Interactive prompt engineering for text-to-image creation. IEEE Transactions on Visualization and Computer Graphics, 30(1): 295--305

    2023

  9. [18]

    Gao, J.; Hu, K.; Xu, G.; and Xu, H. 2023. Can pre-trained text-to-image models generate visual goals for reinforcement learning? Advances in Neural Information Processing Systems, 36: 38297--38310

    2023

  10. [19]

    Gong, H.; Gupta, K.; Jain, A.; and Bhat, S. 2020. IlliniMet: Illinois system for metaphor detection with contextual and linguistic information. In Proceedings of the second workshop on figurative language processing, 146--153

    2020

  11. [20]

    Hao, Y.; Chi, Z.; Dong, L.; and Wei, F. 2023. Optimizing prompts for text-to-image generation. Advances in Neural Information Processing Systems, 36: 66923--66939

    2023

  12. [21]

    Ho, J.; Jain, A.; and Abbeel, P. 2020. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 6840--6851

    2020

  13. [22]

    J.; Norouzi, M.; and Salimans, T

    Ho, J.; Saharia, C.; Chan, W.; Fleet, D. J.; Norouzi, M.; and Salimans, T. 2022. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47): 1--33

    2022

  14. [23]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022. Lora: Low-rank adaptation of large language models. ICLR, 1(2): 3

    2022

  15. [24]

    Hu, Z.; Zhang, F.; Chen, L.; Kuang, K.; Li, J.; Gao, K.; Xiao, J.; Wang, X.; and Zhu, W. 2025. Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewards. arXiv preprint arXiv:2503.11240

    2025 arXiv

  16. [25]

    Huang, K.; Sun, K.; Xie, E.; Li, Z.; and Liu, X. 2023. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36: 78723--78747

    2023

  17. [26]

    Jiang, D.; Guo, Z.; Zhang, R.; Zong, Z.; Li, H.; Zhuo, L.; Yan, S.; Heng, P.-A.; and Li, H. 2025. T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT. arXiv:2505.00703

    2025 arXiv

  18. [27]

    P.; Littman, M

    Kaelbling, L. P.; Littman, M. L.; and Moore, A. W. 1996. Reinforcement Learning: A Survey. arXiv:cs/9605103

    1996 arXiv

  19. [29]

    Lee, K.; Liu, H.; Ryu, M.; Watkins, O.; Du, Y.; Boutilier, C.; Abbeel, P.; Ghavamzadeh, M.; and Gu, S. S. 2023 b . Aligning text-to-image models using human feedback. arXiv preprint arXiv:2302.12192

    2023 arXiv

  20. [30]

    W.; Klebanov, B

    Leong, C. W.; Klebanov, B. B.; and Shutova, E. 2018. A report on the 2018 VUA metaphor detection shared task. In Proceedings of the workshop on figurative language processing, 56--66

    2018

  21. [31]

    Liu, V.; and Chilton, L. B. 2022. Design guidelines for prompt engineering text-to-image generative models. In Proceedings of the 2022 CHI conference on human factors in computing systems, 1--23

    2022

  22. [32]

    Mo, W.; Zhang, T.; Bai, Y.; Su, B.; Wen, J.-R.; and Yang, Q. 2024. Dynamic prompt optimizing for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 26627--26636

    2024

  23. [33]

    P.; Perelman, A.; and et al

    OpenAI; Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; and et al. 2024. GPT-4o System Card. arXiv:2410.21276

    2024 arXiv

  24. [34]

    H.; and Riedel, S

    Petroni, F.; Rockt \"a schel, T.; Lewis, P.; Bakhtin, A.; Wu, Y.; Miller, A. H.; and Riedel, S. 2019. Language models as knowledge bases? arXiv preprint arXiv:1909.01066

    2019 arXiv

  25. [35]

    Prabhudesai, M.; Goyal, A.; Pathak, D.; and Fragkiadaki, K. 2024. Aligning Text-to-Image Diffusion Models with Reward Backpropagation. arXiv:2310.03739

    2024 arXiv

  26. [36]

    Prasad, A.; Koller, A.; Hartmann, M.; Clark, P.; Sabharwal, A.; Bansal, M.; and Khot, T. 2024. ADaPT: As-Needed Decomposition and Planning with Language Models. arXiv:2311.05772

    2024 arXiv

  27. [37]

    Ramesh, A.; Pavlov, M.; Goh, G.; Gray, S.; Voss, C.; Radford, A.; Chen, M.; and Sutskever, I. 2021. Zero-shot text-to-image generation. In International conference on machine learning, 8821--8831. Pmlr

    2021

  28. [38]

    Rana, K.; Haviland, J.; Garg, S.; Abou-Chakra, J.; Reid, I.; and Suenderhauf, N. 2023. SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning. arXiv:2307.06135

    2023 arXiv

  29. [39]

    Z.; Lidard, J.; Ankile, L

    Ren, A. Z.; Lidard, J.; Ankile, L. L.; Simeonov, A.; Agrawal, P.; Majumdar, A.; Burchfiel, B.; Dai, H.; and Simchowitz, M. 2024. Diffusion policy policy optimization. arXiv preprint arXiv:2409.00588

    2024 arXiv

  30. [40]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022 a . High-Resolution Image Synthesis with Latent Diffusion Models. arXiv:2112.10752

    2022 arXiv

  31. [41]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022 b . High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695

    2022

  32. [42]

    Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597

    2015 arXiv

  33. [43]

    Saharia, C.; Chan, W.; Saxena, S.; Li, L.; Whang, J.; Denton, E.; Ghasemipour, S. K. S.; Ayan, B. K.; Mahdavi, S. S.; Lopes, R. G.; Salimans, T.; Ho, J.; Fleet, D. J.; and Norouzi, M. 2022. Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. arXiv:2...

    2022 arXiv

  34. [44]

    Schuhmann, C.; Beaumont, R.; Vencu, R.; Gordon, C.; Wightman, R.; Cherti, M.; Coombes, T.; Katta, A.; Mullis, C.; Wortsman, M.; Schramowski, P.; Kundurthy, S.; Crowson, K.; Schmidt, L.; Kaczmarczyk, R.; and Jitsev, J. 2022. LAION-5B: An open large-scale dataset for training ne...

    2022 arXiv

  35. [45]

    Sohl-Dickstein, J.; Weiss, E.; Maheswaranathan, N.; and Ganguli, S. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2256--2265. pmlr

    2015

  36. [46]

    Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502

    2020 arXiv

  37. [47]

    Sun, P.; Jiang, Y.; Chen, S.; Zhang, S.; Peng, B.; Luo, P.; and Yuan, Z. 2024. Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation. arXiv:2406.06525

    2024 arXiv

  38. [48]

    Tian, K.; Jiang, Y.; Yuan, Z.; Peng, B.; and Wang, L. 2024. Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction. arXiv:2404.02905

    2024 arXiv

  39. [49]

    Tsvetkov, Y.; Boytsov, L.; Gershman, A.; Nyberg, E.; and Dyer, C. 2014. Metaphor detection with cross-lingual model transfer. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 248--258

    2014

  40. [50]

    Veale, T. 2016. Round up the usual suspects: Knowledge-based metaphor generation. In Proceedings of the Fourth Workshop on Metaphor in NLP, 34--41

    2016

  41. [51]

    Wallace, B.; Dang, M.; Rafailov, R.; Zhou, L.; Lou, A.; Purushwalkam, S.; Ermon, S.; Xiong, C.; Joty, S.; and Naik, N. 2023. Diffusion Model Alignment Using Direct Preference Optimization. arXiv:2311.12908

    2023 arXiv

  42. [52]

    Wang, Z.; Jiang, Y.; Zheng, H.; Wang, P.; He, P.; Wang, Z.; Chen, W.; and Zhou, M. 2023. Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models. arXiv:2304.12526

    2023 arXiv

  43. [53]

    Xie, Y.; Pan, Z.; Ma, J.; Jie, L.; and Mei, Q. 2023. A prompt log analysis of text-to-image generation systems. In Proceedings of the ACM Web Conference 2023, 3892--3902

    2023

  44. [54]

    Xu, J.; Liu, X.; Wu, Y.; Tong, Y.; Li, Q.; Ding, M.; Tang, J.; and Dong, Y. 2023. ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. arXiv:2304.05977

    2023 arXiv

  45. [55]

    Yang, L.; Tian, Y.; Li, B.; Zhang, X.; Shen, K.; Tong, Y.; and Wang, M. 2025. MMaDA: Multimodal Large Diffusion Language Models. arXiv:2505.15809

    2025 arXiv

  46. [56]

    Yang, L.; Yu, Z.; Meng, C.; Xu, M.; Ermon, S.; and Cui, B. 2024. Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs. arXiv:2401.11708

    2024 arXiv

  47. [57]

    Zhang, C.; Zhang, C.; Zhang, M.; and Kweon, I. S. 2023. Text-to-image diffusion models in generative ai: A survey. arXiv preprint arXiv:2303.07909

    2023 arXiv

  48. [58]

    Zhang, S.; Yang, X.; Feng, Y.; Qin, C.; Chen, C.-C.; Yu, N.; Chen, Z.; Wang, H.; Savarese, S.; Ermon, S.; et al. 2024. Hive: Harnessing human feedback for instructional visual editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9026--9036

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.