Pith. sign in

REVIEW 4 major objections 7 minor 59 references

ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ShortFT aligns diffusion models to human preferences by backpropagating reward gradients through a short distilled denoising chain.

desk verdict A clever shortcut for reward backprop in diffusion fine-tuning, but the paper overstates its completeness and never validates the shortcut's fidelity. read the letter →

arxiv 2507.22604 v1 pith:MOPPOJ53 submitted 2025-07-30 cs.CV

Xiefan Guo , Miaomiao Cui , Liefeng Bo , Di Huang This is my paper
classification cs.CV
keywords diffusionmodelalignmentrewardbackpropagationdenoisingshortcutfew-steptrajectory-preservingdistillationLoRAfine-tuninghumanpreference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ShortFT claims that the length of the denoising chain, not the architecture, is the main obstacle to end-to-end reward backpropagation for diffusion model alignment. By inserting a trajectory-preserving few-step model (4-step Hyper-SD) as a denoising shortcut, the chain shrinks from 50 steps to 4, making full reward-gradient backpropagation through the sampling process both feasible and stable. Fine-tuning the foundational SD 1.5 model with this shortcut and timestep-aware LoRA adapters improves human-preference alignment (HPS v2 33.88, PickScore 24.16) over truncated and partially unrolled baselines (DRaFT-LV 33.13/23.35, DRTune 32.79/23.22) under the same six-hour, two-GPU budget, and also improves symmetry. If correct, ShortFT makes direct reward supervision of all denoising stages practical for text-to-image models of both UNet and Transformer design.

What carries the argument

The denoising shortcut: a 4-step Hyper-SD model that retains the original SD 1.5 denoising trajectory while skipping most timesteps; this shortens the backpropagation chain so reward gradients reach every stage of sampling. Assisting machinery: timestep-aware LoRA adapters (a distinct low-rank branch for each denoising segment, capturing how prompt-dependence shifts from text to visual features over the chain) and a progressive training strategy (gradually replacing shortcut steps with original denoising steps so the final model is optimized for inference-time full-chain generation).

What would settle it

Run ShortFT with the same setup but replace Hyper-SD with a trajectory-reformulating few-step model (e.g., an adversarial distillation that intentionally changes the denoising path); if alignment gains vanish or degrade, the shortcut's trajectory preservation is the operative cause. Alternatively, measure per-timestep output differences between the 4-step Hyper-SD path and the 50-step DDIM path on SD 1.5 at the same noise inputs and show whether the reward-gradient directions agree.

Watch

Extended reading notes

Core claim

At its core, ShortFT claims that a trajectory-preserving few-step diffusion model distilled from the base model provides a valid denoising shortcut—a shorter path through the sampling process that stays on the same ODE trajectory—and that optimizing on this shortcut, rather than truncating the chain or dropping gradient terms, is what makes end-to-end backpropagation of the reward gradient effective. The paper constructs a shortcut-based denoising chain of four denoising segments, fine-tunes the original SD 1.5 weights with LoRA adapters specialized per timestep segment, and trains progressively so that later stages replace the shortcut with the original chain, closing the train-inference gap. The result is better alignment with diverse rewards (human preference, symmetry, compressibility, combined) and generalization to wild prompts, with fine-tuning the foundational model outperforming fine-tuning the distilled few-step model itself (HPS v2 35.97 vs 32.92 after 10k steps).

Load-bearing premise

The shortcut model Hyper-SD stays close enough to SD 1.5's original 50-step denoising trajectory that reward gradients computed through the 4-step chain are valid for fine-tuning the full model, and the progressive training fully repairs the residual error.

Editorial extensions

If this is right

  • End-to-end reward backpropagation becomes a practical fine-tuning recipe for text-to-image models: 4-step shortcuts remove the need for gradient checkpointing and make full-chain supervision feasible on two GPUs in hours.
  • The same shortcut recipe transfers to different reward functions—HPS v2, PickScore, symmetry, compressibility, and weighted combinations—without changing the fine-tuning procedure.
  • Fine-tuning the foundational SD 1.5 model through the shortcut yields higher alignment (HPS v2 35.97) than fine-tuning the distilled Hyper-SD model directly (32.92), because the full model retains generation capacity the distillation lost.
  • The method generalizes beyond UNet architectures to Transformer-based SD 3, indicating the shortcut principle is architecture-agnostic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The load-bearing assumption that Hyper-SD's trajectory is close to SD 1.5's suggests ShortFT's gains will track the fidelity of the distilled shortcut: better trajectory-preserving distillers should yield further alignment gains, while trajectory-reformulating distillers (which intentionally change the path) should break the effect—an experiment the paper does not run.
  • Timestep-aware LoRA may be a general principle for reward alignment: since early denoising steps encode prompt semantics and later steps refine visual detail, per-segment adapters allow the reward to adjust different stages differently, and this could be combined with other alignment objectives (e.g., DPO-style losses) as well.
  • Because the shortcut is only used during training and removed during inference, ShortFT effectively turns a distillation model into a proxy for computing full-chain gradients; one could test whether the same proxy idea works for other long unrolls, such as video diffusion or multi-step LLM samplers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes Shortcut-based Fine-Tuning (ShortFT), a method for aligning text-to-image diffusion models with differentiable reward functions. ShortFT uses a trajectory-preserving few-step diffusion model (4-step Hyper-SD distilled from SD 1.5) as a 'denoising shortcut' that shortens the denoising/backpropagation chain, combined with a timestep-aware LoRA parameterization and a progressive training strategy intended to close the training-inference gap. Experiments on HPDv2 with HPS v2, PickScore, and Symmetry rewards report improved scores over DRaFT-LV and DRTune under the same computational budget, along with qualitative comparisons, a small user study, and ablations on the LoRA and progressive-training components.

Significance. The core idea of exploiting a distilled few-step model as a differentiable proxy to shorten backpropagation for reward fine-tuning is novel and potentially useful for reducing the cost and instability of end-to-end reward optimization. The paper provides a reasonable breadth of experiments: multiple reward functions, an architecture generalization (SD 3), and component ablations. However, the central mechanism is currently overstated and internally inconsistent, and the empirical validation is thin: the shortcut fidelity is not quantitatively established, the main results lack error bars, and the claim of 'complete gradient backpropagation' is contradicted by the paper's own description of truncated backpropagation. If the authors can clarify the gradient-truncation scheme, add quantitative validation of the shortcut's trajectory/Jacobian fidelity under LoRA updates, and report variance across seeds, the method would be a solid contribution. As it stands, the paper's central claims are not yet fully supported.

major comments (4)
  1. [Sec. 3.3, Abstract, Fig. 2(d)] The paper repeatedly advertises 'complete gradient backpropagation' and 'full gradient backpropagation throughout the denoising chain' (Abstract; Sec. 3.3; Fig. 2(d) caption), but the last paragraph of Sec. 3.3 states: 'in line with [6], we also employ the truncated backpropagation technique.' This is a direct contradiction. If truncation is used, the method is closer to DRaFT-LV/DRTune-style gradient approximation than advertised, and the claimed novelty of 'complete' backprop is not supported. Please specify exactly which gradient paths are truncated, where in the chain truncation occurs, how the resulting approximation differs from DRaFT-LV's truncation, and revise the claims accordingly.
  2. [Sec. 3.2, Fig. 3, Sec. 3.3] The load-bearing premise that the Hyper-SD shortcut remains a faithful differentiable proxy for the full 50-step SD1.5 denoising chain is not quantitatively validated. Fig. 3 reports HPS v2 deviations on a few handpicked prompts using the original SD1.5, not a LoRA-perturbed model, and it measures neither trajectory distance nor gradient alignment. Since training updates LoRA weights, the input distribution seen by the fixed shortcut changes, and the shortcut's output Jacobian can diverge from the true chain's Jacobian. The progressive training strategy is a heuristic mitigation; no number is given for shortcut error or for when gradient transfer fails. Please provide quantitative evidence, e.g., per-step output discrepancy and gradient cosine similarity between the shortcut and full chain before and after LoRA updates, or an ablation with varying shortcut lengths/steps, to support the central premise.
  3. [Table 1, Sec. 4.1, Sec. 4.3] All main results in Table 1 are from single runs with no error bars or significance tests, and the HPS v2 margin over DRaFT-LV is small (33.88 vs 33.13, a difference of 0.75). Because HPS v2 is also the training reward (and is used as a joint regularization term alongside PickScore), the evaluation is partly circular, as the authors acknowledge is common to baselines. Please report mean±std over at least three seeds for all methods, state the number of training steps/iterations each method uses under the fixed computational budget, and clarify whether the 33.88 score is obtained with the combined-reward training objective or with HPS v2 alone. The user study (11 volunteers, 10 questions per pair) and the GPT-4V evaluation (20 queries per pair) are too small to support the strong preference percentages in Fig. 9; please report exact counts and confidence intervals.
  4. [Sec. 4.4, Sec. 4.1, Fig. 1 caption] The claim that 'the obtained HPS v2 score of 35.97 surpasses the reported score for DRaFT-LV in [6]' is incomplete: the exact DRaFT-LV number under the same 10k-step protocol is not provided, and no comparison is given for DRTune or for ShortFT's own 6-hour budget. Additionally, the 'Combined reward' weights (PickScore=10, HPS v2=2, Aesthetic=0.05) appear only in the Fig. 1 caption and are not fully specified in Sec. 4.1, where the text says only that 'HPS v2 and PickScore in a ratio of 1:10' are amalgamated as a joint regularization term. Please specify the exact combined-reward objective used for each experiment and clarify which reward is optimized in Table 1.
minor comments (7)
  1. [Throughout] The method name is inconsistent: 'SHORT FT', 'ShortFT', and 'SHORTFT' are all used; please adopt a single spelling (e.g., ShortFT).
  2. [Fig. 2(d) caption] The caption states 'complete gradient backpropagation through the entire denoising chain,' which is at odds with the truncated-backpropagation statement in Sec. 3.3; please align the figure description with the actual algorithm.
  3. [Sec. 3.2, Fig. 3] The sentence 'These observations collectively indicate the reliability and validity of the denoising shortcut' overstates what three handpicked prompts and HPS v2 deviations can establish; please rephrase to reflect the limited scope of the evidence.
  4. [Sec. 4.2] The text says 'Fig. 7 and 8 present the quantitative comparison,' but the figures show qualitative examples; please correct 'quantitative' to 'qualitative'.
  5. [Sec. 4.3, Fig. 9] The user study and GPT-4V evaluation lack details on the number of unique prompts, inter-annotator agreement, and confidence intervals; please report these to support the preference percentages.
  6. [Sec. 4.4, Table 2] The comparison between tuning SD 1.5 and tuning Hyper-SD is based on a single HPS v2 score per strategy; please include additional metrics (e.g., PickScore, Fréchet Inception Distance) and multiple seeds to substantiate the claim of superiority.
  7. [Sec. 4.1] The paper does not state the random seed(s) used for training and evaluation; please provide seed values for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ShortFT is an empirical fine-tuning method whose claims are benchmarked against external HPDv2 data and pre-trained Hyper-SD shortcuts, with no derivation that reduces to its own inputs.

full rationale

The paper's core contribution is an empirical training strategy: use the pre-trained 4-step Hyper-SD as a denoising shortcut to backpropagate reward gradients into a LoRA fine-tuned SD 1.5. The shortcut model is an external, fixed artifact, not derived from ShortFT's objective, and the reward gradients are evaluated against external HPS v2, PickScore, and Symmetry benchmarks. The fact that HPS v2 is used both as the training reward and as an evaluation metric is a shared design choice with the baselines (DRaFT-LV, DRTune), and it constitutes potential reward overfitting, not a circular derivation: the reported HPS v2 score is not equal to the training objective by construction. The paper explicitly acknowledges that the shortcut 'inherently introduces errors' (Sec. 3.3), which is a stated limitation rather than a hidden equivalence. The statement 'in line with [6], we also employ the truncated backpropagation technique' (Sec. 3.3) conflicts with the abstract's 'complete gradient backpropagation' claim, but this is a methodological inconsistency, not a circularity. No load-bearing step is justified solely by a self-citation: the two self-citations ([15], [16]) appear only in a general applications list. The central derivation chain—construct a shorter chain with an externally distilled model, backpropagate a differentiable reward, and evaluate on held-out benchmarks—contains no equation that is equivalent to its own input by construction. Therefore the circularity score is 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim depends on empirical assumptions about distillation fidelity, gradient reliability, and LoRA capacity. No free parameter is fit to the benchmark in a formal sense, but several hyperparameters are chosen by hand and not ablated. No new physical or ontological entities are introduced; timestep-aware LoRA is a parameterization scheme, not an entity.

free parameters (6)
  • Number of shortcut segments k = 4
    Determines how many LoRA branches and shortcut positions are used; set in Sec. 4.1 without a reported sweep.
  • LoRA anchor timesteps = {761, 501, 261, 1}
    Timesteps where LoRA branches are applied; chosen by hand, no ablation on alternative sets (Sec. 4.1).
  • LoRA rank = 128
    Adapter capacity; set following [6] in Sec. 4.1.
  • Learning rate = 5e-5
    Constant learning rate used with AdamW; no sensitivity analysis (Sec. 4.1).
  • Reward regularization weight ratio = HPS v2 : PickScore = 1:10
    Used as a joint regularization term; the ratio is stated without ablation or justification (Sec. 4.1).
  • Shortcut distilled model = 4-step Hyper-SD
    Chosen among trajectory-preserving few-step models; no comparison to other distilled models (Sec. 4.1).
assumptions (4)
  • domain assumption The reward gradient backpropagated through the denoising chain is a usable optimization signal
    Sec. 3.1, Eq. (1), shared with DRaFT, DRTune, and AlignProp, treats the sampling process as differentiable and the reward gradient as meaningful.
  • domain assumption Trajectory-preserving few-step models preserve the SD1.5 trajectory closely enough for shortcut gradients to transfer
    Sec. 3.2 and Fig. 3 rely on this; Sec. 3.3 acknowledges the shortcut 'inherently introduces errors' in fine details.
  • domain assumption LoRA adapters of rank 128 have enough capacity to capture the alignment shift
    Sec. 3.3 introduces LoRA without capacity analysis or comparison to full fine-tuning.
  • ad hoc to paper The progressive training strategy closes the training-inference gap caused by the shortcut
    Sec. 3.3 and Fig. 6 are designed specifically to address this gap; validation is only qualitative via Fig. 13.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning." pith.science (2026). https://pith.science/paper/MOPPOJ53

@misc{pith2026250722604,
  author       = {Pith},
  title        = {Pith review of: ShortFT: Diffusion Model Alignment via Shortcut-based Fine-Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MOPPOJ53}},
  note         = {Machine review of arXiv:2507.22604}
}
read the original abstract

Backpropagation-based approaches aim to align diffusion models with reward functions through end-to-end backpropagation of the reward gradient within the denoising chain, offering a promising perspective. However, due to the computational costs and the risk of gradient explosion associated with the lengthy denoising chain, existing approaches struggle to achieve complete gradient backpropagation, leading to suboptimal results. In this paper, we introduce Shortcut-based Fine-Tuning (ShortFT), an efficient fine-tuning strategy that utilizes the shorter denoising chain. More specifically, we employ the recently researched trajectory-preserving few-step diffusion model, which enables a shortcut over the original denoising chain, and construct a shortcut-based denoising chain of shorter length. The optimization on this chain notably enhances the efficiency and effectiveness of fine-tuning the foundational model. Our method has been rigorously tested and can be effectively applied to various reward functions, significantly improving alignment performance and surpassing state-of-the-art alternatives.

Figures

Figures reproduced from arXiv: 2507.22604 by the authors.

Figure 1
Figure 1. Example results synthesized by SHORTFT. SHORTFT endeavors to achieve the alignment of diffusion models with reward functions by facilitating the end-to-end backpropagation of the targeted reward gradient throughout the denoising chain. Our method has exhibited remarkable efficacy, particularly evident in the realms of text-image alignment and the overall enhancement of image quality (Top). Moreover, its versatility … view at source ↗
Figure 2
Figure 2. Comparison of fine-tuning strategies. (a) The vanilla backpropagation-based fine-tuning strategy, which suffers from lengthy backpropagation chains. (b) DRaFT-K [6] and ReFL [52] truncate the backpropagation chain, focusing on the latter half of the denoising chain, they ignore the direct supervision at the early stage, resulting in suboptimal alignment with text prompts. (c) AlignProp [34] and DRTune [51] truncate … view at source ↗
Figure 3
Figure 3. Denoising shortcut. The trajectory-preserving few-step diffusion model naturally introduces denoising shortcut, allowing for flexible skipping within the denoising chain while still ensuring high-quality and consistent image synthesis. The 4-step Hyper-SD distilled from SD 1.5 is used in our experiments. SHORTCUT(i) denotes completing the denoising process from timestep i to 0 using the few-step diffusion model. In … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Illustration of SHORTFT. The core of the proposed method is the Shortcut-based Fine-Tuning (SHORTFT), which leverages the trajectory-preserving few-step diffusion model as the shortcut (identified as blue arrow) to achieve direct end-to-end backpropagation through the …
Figure 5
Figure 5. Figure 5: Illustration of timestep-aware LoRA. (a) Vanilla SD denoising chain; (b) Shortcut-based denoising chain. In accor￾dance with the interesting time dynamics in the text-to-image dif￾fusion model denoising process revealed by [1], different from existing methods that shar…
Figure 6
Figure 6. Figure 6: Illustration of progressive training strategy. Corre￾sponding to the time-step aware LoRA design, we develop the pro￾gressive training strategy, which eliminates the training-inference gap introduced by the shortcut-based fine-tuning. a vanilla SD denoising chain compr…
Figure 7
Figure 7. Figure 7: Qualitative comparison on PickScore and HPS v2. Each image is generated with the same text prompt and random seed for all methods. Our method outperforms existing methods in both text-image alignment and image quality. Reward functions. The proposed method is evaluated…
Figure 8
Figure 8. Figure 8: Qualitative comparison on HPS v2 and Symmetry. Each image is generated with the same text prompt and random seed for all methods. Our method outperforms existing methods in both text-image alignment and image quality. Human AI (GPT) 6.36% 93.64% 5% 95% 35.45% 64.55% 20…
Figure 9
Figure 9. Figure 9: Human and AI preference evaluation against current methods. SHORTFT performs over other counterparts. 4.3. Quantitative comparison Objective evaluation [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison on tuning SD 1.5 (SHORTFT) and Hyper-SD. Fine-tuning SD 1.5 significantly outperforms fine￾tuning Hyper-SD, with the former enjoying more exquisite details. [26–28] explore fine-tuning few-step diffusion models and have achieved certain successe…
Figure 13
Figure 13. Figure 13: Ablation study on progressive training strategy. The red circle marks the incoherent local details, i.e., unsmooth hair. ciency in managing Compressibility and Combined reward. Generalization to wild text prompts. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_13.png]
Figure 11
Figure 11. Figure 11: Example results synthesized by SHORTFT on SD 3. Fine-tuning SD 3. SHORTFT is an architecture-agnostic fine-tuning strategy, applicable to both UNet-based (SD 1.5) and Transformer-based (SD 3) architectures. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Generalization to wild text prompts from Sora. Our method is capable of effectively handling the wild text prompts. Other reward functions. SHORTFT exhibits remarkable versatility, demonstrating efficacy across a spectrum of re￾ward functions, significantly improving …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 34 canonical work pages

  1. [6]

    Directly fine-tuning diffusion models on differentiable re- wards

    Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable re- wards. In ICLR, 2024. 2, 3, 4, 5, 6, 7

    work page 2024

  2. [1]

    ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, et al. ediffi: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022. 2, 3, 4, 5

    arXiv 2022

  3. [2]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 3

    work page 2023

  4. [3]

    Training diffusion models with reinforce- ment learning

    Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforce- ment learning. arXiv preprint arXiv:2305.13301, 2023. 2, 3

    arXiv 2023

  5. [4]

    Enhancing diffusion models with text-encoder reinforcement learning

    Chaofeng Chen, Annan Wang, Haoning Wu, Liang Liao, Wenxiu Sun, Qiong Yan, and Weisi Lin. Enhancing diffusion models with text-encoder reinforcement learning. In ECCV,

    work page

  6. [5]

    Deep reinforcement learn- ing from human preferences

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learn- ing from human preferences. In NeurIPS, 2017. 2

    work page 2017

  7. [7]

    Emu: Enhanc- ing image generation models using photogenic needles in a haystack

    Xiaoliang Dai, Ji Hou, Chih-Yao Ma, Sam Tsai, Jialiang Wang, Rui Wang, Peizhao Zhang, Simon Vandenhende, Xi- aofang Wang, Abhimanyu Dubey, et al. Emu: Enhanc- ing image generation models using photogenic needles in a haystack. arXiv preprint arXiv:2309.15807, 2023. 3

    arXiv 2023

  8. [8]

    Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models

    Fei Deng, Qifei Wang, Wei Wei, Tingbo Hou, and Matthias Grundmann. Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models. In CVPR,

    work page

Show all 59 references
  1. [9]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021. 2, 3

    2021

  2. [10]

    Raft: Reward ranked finetuning for generative foundation model alignment

    Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, Kashun Shum, and Tong Zhang. Raft: Reward ranked finetuning for generative foundation model alignment. arXiv preprint arXiv:2304.06767, 2023. 2, 3

    2023 arXiv

  3. [11]

    Re- inforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Re- inforcement learning for fine-tuning text-to-image diffusion models. In NeurIPS, 2023. 3

    2023

  4. [12]

    Re- inforcement learning for fine-tuning text-to-image diffusion models

    Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Moham- mad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Re- inforcement learning for fine-tuning text-to-image diffusion models. In NeurIPS, 2023. 2

    2023

  5. [13]

    Policy shaping: Integrating human feedback with reinforcement learning

    Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L Isbell, and Andrea L Thomaz. Policy shaping: Integrating human feedback with reinforcement learning. In NeurIPS, 2013. 2

    2013

  6. [14]

    Matryoshka diffusion models.arXiv preprint arXiv:2310.15111, 2023

    Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Josh Susskind, and Navdeep Jaitly. Matryoshka diffusion models.arXiv preprint arXiv:2310.15111, 2023. 3

    2023 arXiv

  7. [15]

    I4vgen: Image as free stepping stone for text-to- video generation

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, Liefeng Bo, and Di Huang. I4vgen: Image as free stepping stone for text-to- video generation. arXiv preprint arXiv:2406.02230, 2024

    2024 arXiv

  8. [16]

    Initno: Boosting text-to-image diffu- sion models via initial noise optimization

    Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffu- sion models via initial noise optimization. In CVPR, 2024

    2024

  9. [17]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 2, 3

    2022 arXiv

  10. [18]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 2, 3

    2020

  11. [19]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In ICLR, 2022. 5

    2022

  12. [20]

    T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation

    Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xi- hui Liu. T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation. In NeurIPS, 2023. 3

    2023

  13. [21]

    Distilling diffusion models into condi- tional gans

    Minguk Kang, Richard Zhang, Connelly Barnes, Sylvain Paris, Suha Kwak, Jaesik Park, Eli Shechtman, Jun-Yan Zhu, and Taesung Park. Distilling diffusion models into condi- tional gans. In ECCV, 2024. 3, 4

    2024

  14. [22]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In NeurIPS, 2022. 2, 3

    2022

  15. [23]

    Consistency trajectory mod- els: Learning probability flow ode trajectory of diffusion

    Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Mu- rata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory mod- els: Learning probability flow ode trajectory of diffusion. arXiv preprint arXiv:2310.02279, 2023. 2, 3, 4

    arXiv

  16. [24]

    Pick-a-pic: An open dataset of user preferences for text-to-image generation

    Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Ma- tiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In NeurIPS, 2023. 3, 6

    2023

  17. [25]

    Aligning text- to-image models using human feedback

    Kimin Lee, Hao Liu, Moonkyung Ryu, Olivia Watkins, Yuqing Du, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, and Shixiang Shane Gu. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192, 2023. 2, 3

    2023 arXiv

  18. [26]

    Reward guided latent consistency distillation

    Jiachen Li, Weixi Feng, Wenhu Chen, and William Yang Wang. Reward guided latent consistency distillation. arXiv preprint arXiv:2403.11027, 2024. 3, 8

    2024 arXiv

  19. [27]

    T2v- turbo: Breaking the quality bottleneck of video consis- tency model with mixed reward feedback

    Jiachen Li, Weixi Feng, Tsu-Jui Fu, Xinyi Wang, Sug- ato Basu, Wenhu Chen, and William Yang Wang. T2v- turbo: Breaking the quality bottleneck of video consis- tency model with mixed reward feedback. arXiv preprint arXiv:2405.18750, 2024

    2024 arXiv

  20. [28]

    T2v- turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design.arXiv preprint arXiv:2410.05677, 2024

    Jiachen Li, Qian Long, Jian Zheng, Xiaofeng Gao, Robinson Piramuthu, Wenhu Chen, and William Yang Wang. T2v- turbo-v2: Enhancing video generation model post-training through data, reward, and conditional guidance design.arXiv preprint arXiv:2410.05677, 2024. 3, 8

  21. [29]

    Aligning diffusion mod- els by optimizing human utility

    Shufan Li, Konstantinos Kallidromitis, Akash Gokul, Yusuke Kato, and Kazuki Kozuka. Aligning diffusion mod- els by optimizing human utility. In NeurIPS, 2024. 3 9

    2024

  22. [30]

    Step-aware prefer- ence optimization: Aligning preference with denoising per- formance at each step

    Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang, Ji Li, and Liang Zheng. Step-aware prefer- ence optimization: Aligning preference with denoising per- formance at each step. arXiv preprint arXiv:2406.04314 ,

    arXiv

  23. [31]

    Sdxl- lightning: Progressive adversarial diffusion distillation

    Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl- lightning: Progressive adversarial diffusion distillation. arXiv preprint arXiv:2402.13929, 2024. 3, 4

    2024 arXiv

  24. [32]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICML, 2021. 2, 3

    2021

  25. [33]

    Training lan- guage models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training lan- guage models to follow instructions with human feedback. In NeurIPS, 2022. 2

    2022

  26. [34]

    Aligning text-to-image diffusion models with reward backpropagation

    Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to-image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2023. 2, 3, 4, 5

    2023 arXiv

  27. [35]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, 2021. 6

    2021

  28. [36]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS, 2023. 3

    2023

  29. [37]

    Hyper-sd: Trajectory segmented consistency model for efficient image synthesis

    Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. arXiv preprint arXiv:2404.13686, 2024. 2, 3, 4, 5

    2024 arXiv

  30. [38]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 2, 3, 6

    2022

  31. [39]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In ICLR, 2022. 2, 3, 4

    2022

  32. [40]

    Fast high- resolution image synthesis with latent adversarial diffusion distillation

    Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high- resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015, 2024. 3, 4

    2024 arXiv

  33. [41]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In ECCV, 2024. 3, 4

    2024

  34. [42]

    Laoin aes- thetic predictor

    Christoph Schuhmann and Romain Beaumont. Laoin aes- thetic predictor. https://laion.ai/blog/laion- aesthetics/, 2022. 2, 3

    2022

  35. [43]

    A picture is worth a thousand words: Principled recaptioning improves image generation

    Eyal Segalis, Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. A picture is worth a thousand words: Principled recaptioning improves image generation. arXiv preprint arXiv:2310.16656, 2023. 3

    2023 arXiv

  36. [44]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 2, 3, 5

    2021

  37. [45]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 2, 3

    2021

  38. [46]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023. 2, 3, 4

    2023

  39. [47]

    Learning to summarize with human feed- back

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea V oss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feed- back. In NeurIPS, 2020. 2

    2020

  40. [48]

    Diffusion model align- ment using direct preference optimization

    Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model align- ment using direct preference optimization. In CVPR, 2024. 3

    2024

  41. [49]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  42. [50]

    Human preference score: Better aligning text-to- image models with human preference

    Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hong- sheng Li. Human preference score: Better aligning text-to- image models with human preference. In ICCV, 2023. 2, 3

    2023

  43. [51]

    Deep reward supervisions for tuning text-to-image diffusion models

    Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. In ECCV, 2024. 2, 3, 4, 5, 6, 7

    2024

  44. [52]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. In NeurIPS, 2023. 2, 3, 5

    2023

  45. [53]

    Raphael: Text-to-image generation via large mixture of diffusion paths

    Zeyue Xue, Guanglu Song, Qiushan Guo, Boxiao Liu, Zhuo- fan Zong, Yu Liu, and Ping Luo. Raphael: Text-to-image generation via large mixture of diffusion paths. In NeurIPS,

  46. [54]

    Using human feedback to fine-tune diffusion models without any reward model

    Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In CVPR, 2024. 3

    2024

  47. [55]

    A dense reward view on aligning text-to-image diffusion with prefer- ence

    Shentao Yang, Tianqi Chen, and Mingyuan Zhou. A dense reward view on aligning text-to-image diffusion with prefer- ence. arXiv preprint arXiv:2402.08265, 2024. 3

    2024 arXiv

  48. [56]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Micha ¨el Gharbi, Richard Zhang, Eli Shecht- man, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024. 3, 4

    2024

  49. [57]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 3

    2023

  50. [58]

    Large-scale reinforcement learning for diffusion models

    Yinan Zhang, Eric Tzeng, Yilun Du, and Dmitry Kislyuk. Large-scale reinforcement learning for diffusion models. In ECCV, 2024. 3

    2024

  51. [59]

    Fine-tuning language models from human preferences

    Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. 2 10

    1909 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.