Pith. sign in

REVIEW 3 major objections 7 minor 97 references

You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read One denoising step's gradient is enough to steer diffusion outputs

desk verdict A practical and well-evaluated one-step backprop shortcut for diffusion sampling, with a theory that does not quite cover the parameter-update implementation. read the letter →

arxiv 2505.07477 v1 pith:372NYZOT submitted 2025-05-12 cs.LG cs.CV

classification cs.LGcs.CV
keywords DiffusionmodelsBackpropagationthroughtimeGradientshortcutsPicarditerationParalleldenoisingControllablegenerationRewardalignmentFixed-pointoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that optimizing a differentiable downstream metric through a diffusion model does not require backpropagating through all denoising steps. Retaining the computational graph of a single denoising step—the first step when optimizing the initial latent, and one uniformly sampled step when optimizing network parameters—is enough to deliver effective gradients. The resulting method, Shortcut Diffusion Optimization (SDO), is presented as a generic and lightweight alternative to full backpropagation, cutting computation by roughly 90 percent and memory by about 35 percent on the tested tasks. A sympathetic reader would care because this turns expensive per-image or per-model customization of diffusion models into something feasible on a single GPU.

What carries the argument

The load-bearing object is the Picard iteration map $F_\theta$ (Eq. 8), which updates an entire denoising trajectory from a previous guess and introduces skip dependencies between $x_N$ and every earlier point in the sequence. SDO keeps only one application of this map in the computational graph, replacing the costly inverse $(I-\partial F_\theta/\partial x_{0:N})^{-1}$ of implicit differentiation with no inversion at all. Proposition 4.1 makes the shortcut principled by showing that the fixed point of $F_\theta$ is exactly the DDIM trajectory, and the contraction assumption $\lambda\in[0,1)$ controls the error, with bounds $\lambda^2\rho/(1-\lambda)$ for latents and $\lambda\rho L_F/(1-\lambda)$ for parameters.

What would settle it

Compute the operator norm of $\partial F_\theta/\partial x_{0:N}$ along the actual DDIM or DPM-solver trajectory used in the experiments; if the largest singular value is not below 1, the contraction condition in Theorem 4.1 fails and the theoretical justification for the one-step shortcut does not apply to those trajectories.

Watch

Extended reading notes

Core claim

Shortcut Diffusion Optimization treats the sampling chain as a Picard fixed-point system $x^*_{0:N}=F_\theta(x^*_{0:N})$ whose fixed point coincides with the DDIM discretization of the same initial noise (Proposition 4.1). Differentiating the fixed-point constraint and keeping only the last Picard iteration gives a one-step gradient $\widehat{\nabla}_{(\cdot)}J = \partial J/\partial x_{0:N}\,\partial F_\theta/\partial(\cdot)$, which for the latent reduces to $\partial J/\partial x_0\,\partial x_{N-1}/\partial x_N$ and for the parameters to one uniformly sampled timestep $i'\sim\mathrm{Uniform}(\{1,\ldots,N\})$. Theorem 4.1 bounds the distance between this shortcut gradient and the true gradient by $\lambda^2\rho/(1-\lambda)$ for the latent and $\lambda\rho L_F/(1-\lambda)$ for the parameters, assuming $F_\theta$ is an $L_F$-Lipschitz contraction with constant $\lambda<1$. The central discovery is that the approximate gradient not only costs about ten percent of full backpropagation but often reaches better values of the downstream metric, which the authors attribute to avoiding gradient explosion over the long chain.

Load-bearing premise

The denoising update map $F_\theta$ is a contraction with $\lambda<1$ along the actual DDIM and DPM-solver trajectories used in the experiments, a condition the paper infers from empirical parallel-denoising convergence rather than measuring directly.

Editorial extensions

If this is right

  • SDO lowers the per-gradient cost of diffusion customization to about one tenth of full backpropagation and reduces memory to roughly two-thirds, making per-image tasks such as style transfer and aesthetic enhancement practical on a single GPU.
  • Because the parameter gradient uses only one uniformly sampled timestep, end-to-end reward fine-tuning avoids the gradient explosion that full-chain backpropagation exhibits at 30–50 sampling steps.
  • The same backward pass works for transformer-based diffusion backbones, multistep consistency models, and a second-order solver, since the forward sampler is unchanged and only the retained graph is different.
  • SDO covers every differentiable variable in the chain—initial latent, intermediate latent, network parameters, and prompt embeddings—so one routine spans controlled generation, adversarial perturbation, and reward alignment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the same fixed-point argument should transfer to any iterative generative sampler with a contractive update map, such as flow matching or few-step consistency distillation, because only the map $F_\theta$ and its contraction constant matter.
  • Measuring the contraction constant directly rather than inferring it would turn the theory into a per-step schedule: retain the graph only at steps where local contraction holds, potentially reducing the cost below one retained step per variable.
  • The large gap in adversarial success rate (about 96 percent versus roughly 52 percent for full backpropagation) hints that one-step gradients escape poor local optima in adversarial loss landscapes; comparing gradient alignment or loss curvature between the two methods would test this interpretation.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes Shortcut Diffusion Optimization (SDO), a method for backpropagating through diffusion sampling while retaining the computational graph of only one denoising step. The authors reformulate sequential diffusion sampling as a Picard fixed-point iteration, show that the fixed point of that iteration coincides with the Euler-style DDIM trajectory (Proposition 4.1), and derive a one-step gradient approximation for both latent and parameter optimization (Eqs. 14-17). They provide a contraction-based error bound (Theorem 4.1) and support the method with experiments on text-guided manipulation, style-guided generation, aesthetic enhancement, adversarial sample generation, and reward fine-tuning on Stable Diffusion, PixArt-alpha, LCM, and DPM-solver. The paper claims roughly 90% compute savings over full backpropagation while matching or exceeding its performance.

Significance. If the empirical findings hold, SDO is a practically valuable contribution: it offers a simple, architecture-agnostic way to reduce the cost of differentiable downstream objectives in diffusion models, with broad experimental coverage across latent optimization, parameter fine-tuning, transformer backbones, consistency models, and higher-order solvers. The paper also has genuine strengths: Proposition 4.1 is a clean equivalence for Euler-style DDIM, the method is not circular in the sense that no fitted constants enter the derivation, and the code is publicly released. The main weakness is that the theoretical justification is narrower than the implementation and experiments: Theorem 4.1 relies on an unverified contraction assumption, and the proved bound covers a full-sum estimator that is not the estimator used in the parameter fine-tuning experiments. These gaps weaken the 'principled rather than heuristic' claim but do not, in my view, undermine the empirical contribution.

major comments (3)
  1. [§4.4, Theorem 4.1] The theoretical justification rests on the assumption that F_theta in Eq. (8) is a contraction with constant lambda < 1, but lambda is never measured for the DDIM or DPM-solver trajectories used in the experiments. Section 4.4 infers lambda is small from the empirical parallel-denoising convergence reported in [28]-[30], yet those results concern parallel Picard solvers, not the sequential DDIM/DPM-solver trajectories that SDO actually differentiates. Because Proposition 4.1 only establishes trajectory equivalence for the Euler-style DDIM update (Eq. 5), the contraction assumption is not transferable to the higher-order DPM-solver experiments in Section 5.4. If F_theta is not a contraction along these trajectories, Theorem 4.1 provides no guarantee and SDO reduces to a truncation heuristic; the paper should either measure lambda (or a proxy) on the actual trajectories, or explicitly restrict the theoretical claims to the Euler-DDIM setting.
  2. [§4.2.3 and §4.3, Eq. (17) vs. Fig. 5] Theorem 4.1 bounds the error of the full-sum one-step gradient in Eq. (16), namely -1/N sum_i (∂J/∂x0)(∂u_theta(x_i)/∂θ), but the parameter fine-tuning implementation retains the computational graph for only one uniformly sampled timestep (Fig. 5) and optimizes with the single-term estimator of Eq. (20). The theorem does not bound the bias or variance of this single-sample estimator relative to the true gradient; unbiasedness of the estimator is not a bounded approximation in the sense of Theorem 4.1, and the bound in the theorem is derived for the sum over all timesteps, not for the sampled term. The paper should either provide a variance or concentration bound for the single-step estimator, or revise the claim that the parameter experiments are justified by Theorem 4.1.
  3. [§5.4 and §4.4] The experiments extend SDO to the DPM-solver, PixArt-α, and LCM, but the theoretical analysis covers only the Euler-style DDIM update via Proposition 4.1. For higher-order solvers and consistency-model sampling, the fixed-point map F_theta in Eq. (8) is not the iteration being differentiated, so neither Proposition 4.1 nor Theorem 4.1 applies. These experiments are valuable as empirical evidence, but the paper should state this limitation explicitly and avoid implying that the theoretical justification covers all supported solvers.
minor comments (7)
  1. [Section 5, opening sentence] The sentence 'In this section, we apply AdjointDPM to perform several tasks' appears to be a copy-paste error; it should read 'we apply SDO'.
  2. [Proposition 4.1] The proposition statement contains the typo 'if the initial noise x∗_N = x∗_N'; it should read x∗_N = x_N.
  3. [Eq. (17) and Fig. 5] Equation (17) samples i ~ Uniform({1,...,N}), while Fig. 5 calls random.randint(0, len(scheduler.timesteps)), which samples indices 0,...,N-1; please clarify the indexing consistency.
  4. [Section 5.1.1 and Table 1] The FlowGrad citation appears as [38] in Table 1 and the experimental setup text but as [57] in Section 5.1.1; the citation should be made consistent.
  5. [Abstract and Section 5.2] The abstract's '~90%' cost reduction is measured for latent-variable gradient computation in Table 3, whereas the reward fine-tuning experiments report about 50% runtime compared with AlignProp-uniform; the abstract should qualify the scope of the 90% claim.
  6. [Tables 1, 2, 4, and 5] The quantitative tables report point estimates without standard errors or repeated-seed intervals; adding these would strengthen the comparisons, especially where the reported gaps between methods are small.
  7. [Figure 15] The prompt in the figure contains the typo 'playfulpand', and the caption labels 'PixArt-α' and 'LCM' do not clearly indicate which rows correspond to which model; please clean up the figure and caption.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the SDO gradient shortcut is derived from the Picard fixed-point equation and evaluated against external baselines; the only self-citation is non-load-bearing.

full rationale

The paper's central claim is that one-step gradient shortcuts suffice for optimizing differentiable metrics through diffusion sampling. The derivation is not circular: the one-step gradient in Eq. 14 is defined from the fixed-point update F_theta, and Proposition 4.1 proves that the fixed-point sequence coincides with the DDIM trajectory when the initial noises match. The latent-variable gradient (Eq. 15) and the parameter gradient (Eq. 16) follow from differentiating that fixed-point update, with no fitted constants. The parameter-gradient step in Eq. 17 is explicitly presented as an unbiased expectation estimator of Eq. 16, and the implementation in Eq. 20 is said to be proportional to Eq. 17; this is an approximation, not a quantity fitted to the target metrics. The contraction assumption in Theorem 4.1 is a stated sufficient condition, and the inference that lambda is small from the cited parallel-denoising convergence results [28]-[30] is external empirical evidence, not an input that already contains the paper's conclusion. The skeptic's concern about the single-sample estimator not being covered by the Theorem 4.1 bound is a rigor gap between theory and implementation, but it is not a circularity: the paper does not define the predicted quantity as the bounded one. The only self-citation is [72] in Section 4.5, used to contrast SDO with previous truncation approaches; that citation does not carry the derivation or the experimental validation. Because the method is validated against external baselines (AdjointDPM, DOODL, AlignProp variants, FreeDoM, UGD) and external rewards (aesthetic score, HPSv2), the central result is self-contained rather than circular. Score 1 reflects the minor non-load-bearing self-citation, not any circular derivation step.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central derivation introduces no fitted constants and no invented entities. The hand-chosen numbers in the experiments, such as the latent start timestep and CLIP trade-off lambda, are task-specific and do not determine the efficiency claim. The theoretical result rests on contraction and Lipschitz assumptions that are plausible but not verified for the tested models; the paper's own limitation section notes the need for differentiable rewards and the presence of reward hacking.

free parameters (2)
  • Latent start timestep for style and aesthetic tasks = t = 600 of 1000
    Hand-selected in Sec. 5.1.2 and 5.1.3 to preserve semantic content; task-specific experimental choice, not part of the core SDO algorithm.
  • CLIP-fidelity trade-off lambda in Eq. 23 = 0.5 for attributes, 0.7 for other prompts
    Task-specific balancing coefficient chosen in Sec. 5.1.1 to weight alignment against identity preservation.
assumptions (5)
  • domain assumption The Picard update F_theta is a contraction with constant lambda in [0,1) on the actual diffusion sampling trajectory.
    Required by Theorem 4.1; the paper does not measure lambda and instead cites empirical convergence of parallel denoising from [28]-[30].
  • domain assumption The DDIM trajectory equals the Picard fixed point for the same initial noise (Proposition 4.1).
    Proved for an Euler-style DDIM discretization, but the proof assumes the numerical solver matches the Picard integral approximation exactly; higher-order solvers like DPM-Solver introduce additional terms.
  • domain assumption The objective J has rho-bounded gradients and u_theta is L_F-Lipschitz with respect to theta.
    Used in Theorem 4.1; not verified for CLIP, aesthetic, HPSv2, or classifier-based losses in the experiments.
  • ad hoc to paper Diffusion Picard iteration converges within a few iterations, so lambda is small enough for the one-step gradient to be accurate.
    Invoked in Sec. 4.4 to argue SDO is a good approximation; relies on empirical observations rather than a formal bound for the tested models.
  • standard math The implicit function theorem and contraction mapping theorem apply to the fixed-point constraint in Eq. 12.
    Standard results used to derive Eq. 13 and the bounds in Theorem 4.1, assuming differentiability and invertibility of I minus the Jacobian.

how reviews work

0 comments
Cite this review

Pith. "Pith review of You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts." pith.science (2026). https://pith.science/paper/372NYZOT

@misc{pith2026250507477,
  author       = {Pith},
  title        = {Pith review of: You Only Look One Step: Accelerating Backpropagation in Diffusion Sampling with Gradient Shortcuts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/372NYZOT}},
  note         = {Machine review of arXiv:2505.07477}
}
abstract

Diffusion models (DMs) have recently demonstrated remarkable success in modeling large-scale data distributions. However, many downstream tasks require guiding the generated content based on specific differentiable metrics, typically necessitating backpropagation during the generation process. This approach is computationally expensive, as generating with DMs often demands tens to hundreds of recursive network calls, resulting in high memory usage and significant time consumption. In this paper, we propose a more efficient alternative that approaches the problem from the perspective of parallel denoising. We show that full backpropagation throughout the entire generation process is unnecessary. The downstream metrics can be optimized by retaining the computational graph of only one step during generation, thus providing a shortcut for gradient propagation. The resulting method, which we call Shortcut Diffusion Optimization (SDO), is generic, high-performance, and computationally lightweight, capable of optimizing all parameter types in diffusion sampling. We demonstrate the effectiveness of SDO on several real-world tasks, including controlling generation by optimizing latent and aligning the DMs by fine-tuning network parameters. Compared to full backpropagation, our approach reduces computational costs by $\sim 90\%$ while maintaining superior performance. Code is available at https://github.com/deng-ai-lab/SDO.

Figures

Figures reproduced from arXiv: 2505.07477 by the authors.

Figure 1
Figure 1. Shortcut Diffusion Optimization (SDO) enables diverse applications through backpropagation in diffusion sampling. Parts (a)-(d) illustrate controlled generation by optimizing latent variables: (a) modifying an image based on text instructions using CLIP [1]; (b) controlling texture according to a given style reference while preserving content; (c) enhancing image aesthetics guided by an aesthetic predictor; and (d) … view at source ↗
Figure 2
Figure 2. Graphical models of DMs illustrating (a) sequential [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Plain backpropagation inefficiently differentiates the objective function through the entire forward sampling process. In contrast, SDO backpropagation requires differentiation only at the specific timestep associated with the variable being optimized (e.g., xN at step N or θ at step i ′ ), providing an efficient shortcut for the backward pass of the gradient. The primary source of computational complexity lies in t… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: PyTorch implementation of SDO for optimizing xN . The function set_grad_enabled ensures that only the one-step computation graph required for backpropagation is retained. scheduler stands for solver scheduling, e.g. DDIM [24]. params = {’params’: model.parameters(), ’l…
Figure 5
Figure 5. Figure 5: PyTorch implementation of SDO for optimizing model parameters. The program samples the timesteps and retains the computational graph only for the sampled timesteps, ensuring lightweight backpropagation while en￾abling optimization across all timesteps. 4.4 Theoretical …
Figure 6
Figure 6. Figure 6: Qualitative results of text-guided image manipulation [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Comparison of style-guided generation between [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Qualitative results of aesthetic enhancement applied [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Adversarial examples targeting the ResNet50 [83] [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: (a) Gradient norms of parameters in plain backpropagation explode as steps increase, potentially causing optimization [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Comparison of generated images from fine-tuned models using the same seeds. Each row, from top to bottom, displays results for: no fine-tuning, fine-tuning on Aesthetic score, fine-tuning on HPSv2, and a mixture of both LoRA weights. The model fine-tuned on a simple p…
Figure 13
Figure 13. Figure 13: Generated images from SDO fine-tuned models [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 15
Figure 15. Figure 15: Style-guided generation using latent consistency [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Visualization of text-guided face manipulation using [PITH_FULL_IMAGE:figures/full_fig_p014_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

97 extracted references · 30 canonical work pages

  1. [28]

    Parallel sampling of diffusion models,

    A. Shih, S. Belkhale, S. Ermon, D. Sadigh, and N. Anari, “Parallel sampling of diffusion models,” Advances in Neural Information Processing Systems, vol. 36, 2024

  2. [30]

    Accelerating parallel sampling of diffusion models,

    Z. Tang, J. Tang, H. Luo, F. Wang, and T.-H. Chang, “Accelerating parallel sampling of diffusion models,” in Forty-first International Conference on Machine Learning, 2024

  3. [1]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning. PMLR, 2021, pp. 8748–8763

  4. [2]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P . Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020

  5. [3]

    Score-based generative modeling through stochastic differential equations,

    Y. Song, J. Sohl-Dickstein, D. P . Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456, 2020

  6. [4]

    High-resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P . Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695

  7. [5]

    Scaling rectified flow transformers for high-resolution image synthesis,

    P . Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ¨uller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel et al., “Scaling rectified flow transformers for high-resolution image synthesis,” in Forty-first International Conference on Machine Learning, 2024

  8. [6]

    Hierar- chical text-conditional image generation with clip latents,

    A. Ramesh, P . Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierar- chical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

Show all 97 references
  1. [7]

    Diffusion models in vision: A survey,

    F.-A. Croitoru, V . Hondru, R. T. Ionescu, and M. Shah, “Diffusion models in vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 10 850–10 869, 2023

  2. [8]

    Renaissance: A survey into ai text-to-image generation in the era of large model,

    F. Bie, Y. Yang, Z. Zhou, A. Ghanem, M. Zhang, Z. Yao, X. Wu, C. Holmes, P . Golnari, D. A. Cliftonet al., “Renaissance: A survey into ai text-to-image generation in the era of large model,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

  3. [9]

    Video diffusion models,

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet, “Video diffusion models,” Advances in Neural Information Processing Systems, vol. 35, pp. 8633–8646, 2022

  4. [10]

    Sora: A review on background, technology, limitations, and opportunities of large vision models,

    Y. Liu, K. Zhang, Y. Li, Z. Yan, C. Gao, R. Chen, Z. Yuan, Y. Huang, H. Sun, J. Gao et al., “Sora: A review on background, technology, limitations, and opportunities of large vision models,”arXiv preprint arXiv:2402.17177, 2024

  5. [11]

    Magictime: Time-lapse video generation models as metamorphic simulators,

    S. Yuan, J. Huang, Y. Shi, Y. Xu, R. Zhu, B. Lin, X. Cheng, L. Yuan, and J. Luo, “Magictime: Time-lapse video generation models as metamorphic simulators,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

  6. [12]

    Geodiff: A geometric diffusion model for molecular conformation generation,

    M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang, “Geodiff: A geometric diffusion model for molecular conformation generation,” arXiv preprint arXiv:2203.02923, 2022

  7. [13]

    Equiv- ariant diffusion for molecule generation in 3d,

    E. Hoogeboom, V . G. Satorras, C. Vignac, and M. Welling, “Equiv- ariant diffusion for molecule generation in 3d,” in International conference on machine learning. PMLR, 2022, pp. 8867–8887

  8. [14]

    Fast graph generation via spectral diffusion,

    T. Luo, Z. Mo, and S. J. Pan, “Fast graph generation via spectral diffusion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 3496–3508, 2023

  9. [15]

    Planning with diffusion for flexible behavior synthesis,

    M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine, “Planning with diffusion for flexible behavior synthesis,” arXiv preprint arXiv:2205.09991, 2022

  10. [16]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” The International Journal of Robotics Research , p. 02783649241273668, 2023

  11. [17]

    Idql: Implicit q-learning as an actor-critic method with diffusion policies,

    P . Hansen-Estruch, I. Kostrikov, M. Janner, J. G. Kuba, and S. Levine, “Idql: Implicit q-learning as an actor-critic method with diffusion policies,” arXiv preprint arXiv:2304.10573, 2023

  12. [18]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis,

    X. Wu, Y. Hao, K. Sun, Y. Chen, F. Zhu, R. Zhao, and H. Li, “Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis,” arXiv preprint arXiv:2306.09341, 2023. 15

  13. [19]

    Styledrop: Text-to-image generation in any style,

    K. Sohn, N. Ruiz, K. Lee, D. C. Chin, I. Blok, H. Chang, J. Barber, L. Jiang, G. Entis, Y. Li et al., “Styledrop: Text-to-image generation in any style,” arXiv preprint arXiv:2306.00983, 2023

  14. [20]

    Tagmol: Target-aware gradient-guided molecule generation,

    V . Dorna, D. Subhalingam, K. Kolluru, S. Tuli, M. Singh, S. Singal, N. Krishnan, and S. Ranu, “Tagmol: Target-aware gradient-guided molecule generation,” arXiv preprint arXiv:2406.01650, 2024

  15. [21]

    Diffusion posterior sampling for general noisy inverse problems,

    H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye, “Diffusion posterior sampling for general noisy inverse problems,” arXiv preprint arXiv:2209.14687, 2022

  16. [22]

    Freedom: Training-free energy-guided conditional diffusion model,

    J. Yu, Y. Wang, C. Zhao, B. Ghanem, and J. Zhang, “Freedom: Training-free energy-guided conditional diffusion model,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 23 174–23 184

  17. [23]

    Align- ing text-to-image diffusion models with reward backpropagation,

    M. Prabhudesai, A. Goyal, D. Pathak, and K. Fragkiadaki, “Align- ing text-to-image diffusion models with reward backpropagation,” arXiv preprint arXiv:2310.03739, 2023

  18. [24]

    Denoising diffusion implicit models,

    J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502, 2020

  19. [25]

    Training deep nets with sublinear memory cost,

    T. Chen, B. Xu, C. Zhang, and C. Guestrin, “Training deep nets with sublinear memory cost,” arXiv preprint arXiv:1604.06174, 2016

  20. [26]

    Adjointdpm: Adjoint sensitivity method for gradient backpropagation of diffusion probabilistic models,

    J. Pan, J. H. Liew, V . Tan, J. Feng, and H. Yan, “Adjointdpm: Adjoint sensitivity method for gradient backpropagation of diffusion probabilistic models,” in The Twelfth International Conference on Learning Representations, 2024

  21. [27]

    End-to-end diffusion latent optimization improves classifier guidance,

    B. Wallace, A. Gokul, S. Ermon, and N. Naik, “End-to-end diffusion latent optimization improves classifier guidance,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 7280–7290

  22. [29]

    Parasolver: A hierarchical parallel integral solver for diffusion models,

    J. Lu, Z. Zhu, and J. Hou, “Parasolver: A hierarchical parallel integral solver for diffusion models,” in The Thirteenth International Conference on Learning Representations

  23. [31]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3836–3847

  24. [32]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,

    C. Mou, X. Wang, L. Xie, Y. Wu, J. Zhang, Z. Qi, and Y. Shan, “T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 5, 2024, pp. 4296– 4304

  25. [33]

    A diffusion model translator for efficient image-to-image translation,

    M. Xia, Y. Zhou, R. Yi, Y.-J. Liu, and W. Wang, “A diffusion model translator for efficient image-to-image translation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

  26. [34]

    Diffusion models in low-level vision: A survey,

    C. He, Y. Shen, C. Fang, F. Xiao, L. Tang, Y. Zhang, W. Zuo, Z. Guo, and X. Li, “Diffusion models in low-level vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

  27. [35]

    Diffusion models beat gans on image synthesis,

    P . Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

  28. [36]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models,

    A. Nichol, P . Dhariwal, A. Ramesh, P . Shyam, P . Mishkin, B. Mc- Grew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741, 2021

  29. [37]

    Guided diffusion for inverse molecular design,

    T. Weiss, E. Mayo Yanes, S. Chakraborty, L. Cosmo, A. M. Bronstein, and R. Gershoni-Poranne, “Guided diffusion for inverse molecular design,” Nature Computational Science, vol. 3, no. 10, pp. 873–882, 2023

  30. [38]

    Flowgrad: Controlling the output of generative odes with gradients,

    X. Liu, L. Wu, S. Zhang, C. Gong, W. Ping, and Q. Liu, “Flowgrad: Controlling the output of generative odes with gradients,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 24 335–24 344

  31. [39]

    Ditto: Diffusion inference-time t-optimization for music generation,

    Z. Novack, J. McAuley, T. Berg-Kirkpatrick, and N. J. Bryan, “Ditto: Diffusion inference-time t-optimization for music generation,” arXiv preprint arXiv:2401.12179, 2024

  32. [40]

    Universal guidance for diffusion models,

    A. Bansal, H.-M. Chu, A. Schwarzschild, S. Sengupta, M. Goldblum, J. Geiping, and T. Goldstein, “Universal guidance for diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 843–852

  33. [41]

    Nice: Non-linear independent components estimation,

    L. Dinh, D. Krueger, and Y. Bengio, “Nice: Non-linear independent components estimation,” arXiv preprint arXiv:1410.8516, 2014

  34. [42]

    Density estimation using real nvp,

    L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” arXiv preprint arXiv:1605.08803, 2016

  35. [43]

    L. S. Pontryagin, Mathematical theory of optimal processes. Routledge, 2018

  36. [44]

    Neu- ral ordinary differential equations,

    R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neu- ral ordinary differential equations,” Advances in neural information processing systems, vol. 31, 2018

  37. [45]

    Adjointdeis: Efficient gradients for diffusion models,

    Z. W. Blasingame and C. Liu, “Adjointdeis: Efficient gradients for diffusion models,” arXiv preprint arXiv:2405.15020, 2024

  38. [46]

    Learning to summarize with human feedback,

    N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Rad- ford, D. Amodei, and P . F. Christiano, “Learning to summarize with human feedback,” Advances in Neural Information Processing Systems, vol. 33, pp. 3008–3021, 2020

  39. [47]

    Fine-tuning language models from human preferences,

    D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P . Christiano, and G. Irving, “Fine-tuning language models from human preferences,” arXiv preprint arXiv:1909.08593, 2019

  40. [48]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P . Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

  41. [49]

    Aligning text-to-image models using human feedback,

    K. Lee, H. Liu, M. Ryu, O. Watkins, Y. Du, C. Boutilier, P . Abbeel, M. Ghavamzadeh, and S. S. Gu, “Aligning text-to-image models using human feedback,” arXiv preprint arXiv:2302.12192, 2023

  42. [50]

    Human preference score: Better aligning text-to-image models with human preference,

    X. Wu, K. Sun, F. Zhu, R. Zhao, and H. Li, “Human preference score: Better aligning text-to-image models with human preference,”

  43. [51]

    Reinforcement learning for fine-tuning text-to-image diffusion models,

    Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P . Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee, “Reinforcement learning for fine-tuning text-to-image diffusion models,” Advances in Neural Information Processing Systems, vol. 36, 2024

  44. [52]

    Training diffusion models with reinforcement learning,

    K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine, “Training diffusion models with reinforcement learning,” arXiv preprint arXiv:2305.13301, 2023

  45. [53]

    Directly fine- tuning diffusion models on differentiable rewards,

    K. Clark, P . Vicol, K. Swersky, and D. J. Fleet, “Directly fine- tuning diffusion models on differentiable rewards,” arXiv preprint arXiv:2309.17400, 2023

  46. [54]

    Deep reward supervisions for tuning text-to-image diffusion models,

    X. Wu, Y. Hao, M. Zhang, K. Sun, Z. Huang, G. Song, Y. Liu, and H. Li, “Deep reward supervisions for tuning text-to-image diffusion models,” in European Conference on Computer Vision. Springer, 2024, pp. 108–124

  47. [55]

    Generative modeling by estimating gradients of the data distribution,

    Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” Advances in neural information processing systems, vol. 32, 2019

  48. [56]

    Flow matching for generative modeling,

    Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” arXiv preprint arXiv:2210.02747, 2022

  49. [57]

    Flow straight and fast: Learning to generate and transfer data with rectified flow,

    X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,” arXiv preprint arXiv:2209.03003, 2022

  50. [58]

    Recurrent neural networks,

    L. R. Medsker, L. Jain et al., “Recurrent neural networks,” Design and Applications, vol. 5, no. 64-67, p. 2, 2001

  51. [59]

    Long short-term memory,

    S. Hochreiter, “Long short-term memory,” Neural Computation MIT- Press, 1997

  52. [60]

    D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986,” Biometrika, vol. 71, no. 599-607, p. 6, 1986

  53. [61]

    Backpropagation through time: what it does and how to do it,

    P . J. Werbos, “Backpropagation through time: what it does and how to do it,” Proceedings of the IEEE, vol. 78, no. 10, pp. 1550–1560, 1990

  54. [62]

    Optimizing millions of hyperparameters by implicit differentiation,

    J. Lorraine, P . Vicol, and D. Duvenaud, “Optimizing millions of hyperparameters by implicit differentiation,” in International conference on artificial intelligence and statistics . PMLR, 2020, pp. 1540–1552

  55. [63]

    Efficient and modular implicit differentiation,

    M. Blondel, Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares- L´opez, F. Pedregosa, and J.-P . Vert, “Efficient and modular implicit differentiation,” Advances in neural information processing systems , vol. 35, pp. 5230–5242, 2022

  56. [64]

    Griewank and A

    A. Griewank and A. Walther, Evaluating derivatives: principles and techniques of algorithmic differentiation. SIAM, 2008

  57. [65]

    S. G. Krantz and H. R. Parks, The implicit function theorem: history, theory, and applications. Springer Science & Business Media, 2002

  58. [66]

    Is attention better than matrix decomposition?

    Z. Geng, M.-H. Guo, H. Chen, X. Li, K. Wei, and Z. Lin, “Is attention better than matrix decomposition?” 2021. [Online]. Available: https://arxiv.org/abs/2109.04553

  59. [67]

    Jfb: Jacobian-free backpropagation for implicit networks,

    S. W. Fung, H. Heaton, Q. Li, D. McKenzie, S. Osher, and W. Yin, “Jfb: Jacobian-free backpropagation for implicit networks,” in 16 Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 6, 2022, pp. 6648–6656

  60. [68]

    One-step differentiation of iterative algorithms,

    J. Bolte, E. Pauwels, and S. Vaiter, “One-step differentiation of iterative algorithms,” Advances in Neural Information Processing Systems, vol. 36, 2024

  61. [69]

    Learning iterative reasoning through energy minimization,

    Y. Du, S. Li, J. Tenenbaum, and I. Mordatch, “Learning iterative reasoning through energy minimization,” in International Conference on Machine Learning. PMLR, 2022, pp. 5570–5582

  62. [70]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “Pytorch: An imperative style, high-performance deep learning library,”Advances in neural information processing systems, vol. 32, 2019

  63. [71]

    Consistency model is an effective posterior sample approximation for diffusion inverse solvers,

    T. Xu, Z. Zhu, J. Li, D. He, Y. Wang, M. Sun, L. Li, H. Qin, Y. Wang, J. Liu et al., “Consistency model is an effective posterior sample approximation for diffusion inverse solvers,” arXiv preprint arXiv:2403.12063, 2024

  64. [72]

    Physics- aligned schr\

    Z. Li, H. Dou, S. Fang, W. Han, Y. Deng, and L. Yang, “Physics- aligned schr\”{o} dinger bridge,” arXiv preprint arXiv:2409.17825, 2024

  65. [73]

    Fusedream: Training-free text-to-image generation with improved clip+ gan space optimization,

    X. Liu, C. Gong, L. Wu, S. Zhang, H. Su, and Q. Liu, “Fusedream: Training-free text-to-image generation with improved clip+ gan space optimization,” arXiv preprint arXiv:2112.01573, 2021

  66. [74]

    Progressive growing of gans for improved quality, stability, and variation,

    T. Karras, “Progressive growing of gans for improved quality, stability, and variation,” arXiv preprint arXiv:1710.10196, 2017

  67. [75]

    A style-based generator archi- tecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator archi- tecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 4401–4410

  68. [76]

    Adam: A method for stochastic optimization,

    D. P . Kingma, “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014

  69. [77]

    Diffusionclip: Text-guided diffusion models for robust image manipulation,

    G. Kim, T. Kwon, and J. C. Ye, “Diffusionclip: Text-guided diffusion models for robust image manipulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 2426–2435

  70. [78]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P . Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

  71. [79]

    Arcface: Additive angular margin loss for deep face recognition,

    J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 4690–4699

  72. [80]

    Perceptual losses for real-time style transfer and super-resolution,

    J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. Springer, 2016, pp. 694–711

  73. [81]

    Large-scale classification of fine-art paintings: Learning the right metric on the right feature,

    B. Saleh and A. Elgammal, “Large-scale classification of fine-art paintings: Learning the right metric on the right feature,” arXiv preprint arXiv:1505.00855, 2015

  74. [82]

    Simulacra aesthetic captions,

    J. D. Pressman, K. Crowson, and S. C. Contributors, “Simulacra aesthetic captions,” Stability AI, Technical Report, 2022. [Online]. Available: https://github.com/JD-P/simulacra-aesthetic-captions

  75. [83]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

  76. [84]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

  77. [85]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochre- iter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems, vol. 30, 2017

  78. [86]

    Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,

    Y. Qu, X. Shen, X. He, M. Backes, S. Zannettou, and Y. Zhang, “Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,” in Proceedings of the 2023 ACM SIGSAC conference on computer and communications security, 2023, pp. 3403–3417

  79. [87]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

  80. [88]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

  81. [89]

    Scalable diffusion models with trans- formers,

    W. Peebles and S. Xie, “Scalable diffusion models with trans- formers,” in Proceedings of the IEEE/CVF international conference on computer vision, 2023, pp. 4195–4205

  82. [90]

    Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers,

    N. Ma, M. Goldstein, M. S. Albergo, N. M. Boffi, E. Vanden- Eijnden, and S. Xie, “Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers,” in European Conference on Computer Vision. Springer, 2024, pp. 23–40

  83. [91]

    Pixart-alpha : Fast training of diffusion trans- former for photorealistic text-to-image synthesis,

    J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y. Wu, Z. Wang, J. Kwok, P . Luo, H. Luet al., “Pixart-alpha : Fast training of diffusion trans- former for photorealistic text-to-image synthesis,” arXiv preprint arXiv:2310.00426, 2023

  84. [92]

    Consistency models,

    Y. Song, P . Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” in International Conference on Machine Learning. PMLR, 2023, pp. 32 211–32 252

  85. [93]

    Latent consistency models: Synthesizing high-resolution images with few-step infer- ence,

    S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao, “Latent consistency models: Synthesizing high-resolution images with few-step infer- ence,” arXiv preprint arXiv:2310.04378, 2023

  86. [94]

    Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,

    C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu, “Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps,” Advances in Neural Information Processing Systems , vol. 35, pp. 5775–5787, 2022

  87. [95]

    Diffusers: State-of-the-art diffusion models,

    P . von Platen, S. Patil, A. Lozhkov, P . Cuenca, N. Lambert, K. Rasul, M. Davaadorj, D. Nair, S. Paul, W. Berman, Y. Xu, S. Liu, and T. Wolf, “Diffusers: State-of-the-art diffusion models,” https://github.com/ huggingface/diffusers, 2022

  88. [96]

    Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models,

    F. Deng, Q. Wang, W. Wei, T. Hou, and M. Grundmann, “Prdp: Proximal reward difference prediction for large-scale reward finetuning of diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7423–7433

  89. [2023]

    Available: https://arxiv.org/abs/2303.14420

    [Online]. Available: https://arxiv.org/abs/2303.14420

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.