Pith. sign in

REVIEW 3 major objections 5 minor 43 references

Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Noise Consistency Training lets a pre-trained one-step generator take on new controls — edges, depth, reference images — by enforcing agreement across diffused noise levels, and proves this agreement makes the generator sample the…

desk verdict A one-step control adapter with real practical appeal, but the theory section doesn't cover the actual loss being optimized. read the letter →

arxiv 2506.19741 v1 pith:H5B2PFYO submitted 2025-06-24 cs.LG cs.CVstat.ML

Yihong Luo , Shuchen Xue , Tianyang Hu , Jing Tang This is my paper · ORCID
classification cs.LGcs.CVstat.ML
keywords NoiseConsistencyTrainingone-stepgeneratorcontrollablegenerationadapterdistributionmatchingmaximummeandiscrepancydiffusiondistillationconditional
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces Noise Consistency Training (NCT), a way to give a pretrained one-step image generator a new conditioning signal — edges, depth, a reference image — without retraining the base generator or running diffusion distillation. The key move is to diffuse the generator's input noise: a noise vector $z$ is interpolated toward pure noise by $z_t = \sqrt{1-\sigma_t^2}\,z + \sigma_t\,\epsilon$, so that pairs $(z_t, c)$ slowly decouple from the condition $c$. The training loss asks the adapted generator to produce the same image from a less-coupled noisy input as from a more-coupled one, with a stop-gradient, plus a boundary loss that pins the fully coupled input to the original generator's output. The paper proves that if both the boundary loss and the summed MMD between adjacent pushed-forward noise levels are exactly zero, the adapted generator maps an independent noise-condition pair to the joint distribution $p_\theta(x)p(c|x)$. Experiments report that with one function evaluation NCT matches or beats a 50-step ControlNet baseline on edge, depth, and super-resolution controls, and matches a 100-step image-prompt adapter at 1/100th the compute.

What carries the argument

The mechanism is the noise-space consistency loss plus boundary loss on a variance-preserved diffusion of the latent $z$. For a shared noise sample $z$ and a condition $c$ drawn from $p(c|f_\theta(z))$, the loss compares $f_{\theta,\phi}(z_{t_{k+1}}, c)$ with the stop-gradient of $f_{\theta,\phi}(z_{t_k}, c)$, where $z_t = \sqrt{1-\sigma_t^2}\,z + \sigma_t\,\epsilon$. As $t$ increases, the pair $(z_t, c)$ becomes less coupled; aligning outputs across adjacent $t$ levels teaches the adapter to rely on $c$ without destroying the one-step generator's mapping. The boundary loss $d(f_{\theta,\phi}(z,c), f_\theta(z))$ anchors the boundary at $t=0$. Lemma 1 interprets the diffusion as an interpolation of joint latent-condition distributions; Lemma 2 identifies the summed single-particle consistency loss as a practical estimate of the summed MMD; Theorem 1 chains zero MMD over adjacent levels to conclude $(T_\phi)_\#\rho = \eta$ with $T_\phi(z,c) = (f_{\theta,\phi}(z,c), c)$. The dual variable $\lambda$ in the primal-dual update balances the consistency and boundary terms.

What would settle it

Train NCT on a benchmark where $p(x|c)$ is deliberately multimodal, then compute the MMD with a characteristic kernel between the adapted generator's joint output distribution and $p_\theta(x)p(c|x)$; if that characteristic-kernel MMD stays bounded away from zero while the NCT loss goes to zero, the surrogate is not enforcing the consistency condition the theorem requires.

Watch

Extended reading notes

Core claim

The central claim is that a consistency condition imposed in the noise space, not the image space, is sufficient to inject a new control into a frozen one-step generator. For coupled pairs $(z,c)$ sampled from $p(z)p(c|f_\theta(z))$, diffusing $z$ along a variance-preserving schedule creates an interpolation from fully coupled noise-condition pairs to independent pairs; the consistency loss forces the adapter's outputs at adjacent noise levels to agree. Together with the boundary loss, these two conditions imply, by Theorem 1, that $f_{\theta,\phi}(z,c)$ with $z$ and $c$ sampled independently yields samples distributed as $p_\theta(x)p(c|x)$. The theorem itself requires zero population MMD with a characteristic kernel and zero boundary loss; in practice the method minimizes a single-particle surrogate with kernels such as $-\|x-y\|^2$ or pseudo-Huber, which the paper explicitly notes are not proper characteristic kernels. The reported result is controllable generation in one forward pass, with FID and condition-consistency better than ControlNet at 50 NFE, DI+ControlNet at 1 NFE, and JDM at 1 NFE on edge, depth, and 8x super-resolution tasks.

Load-bearing premise

The result depends on the assumption that the single-particle noise-consistency surrogate, using kernels such as $-\|x-y\|^2$ that are not strictly characteristic, drives the population MMD and the boundary loss all the way to zero for some reachable parameter set — a gap between Theorem 1 and Algorithm 1 that the paper leaves open.

Editorial extensions

If this is right

  • With one function evaluation, a generator fitted with an NCT adapter can follow structural controls such as edge maps and depth maps at quality comparable to 50- or 100-step diffusion baselines.
  • New controls can be added to an already distilled one-step generator without retraining the base model, without diffusion distillation, and without access to the original training images.
  • Different adapters trained for different controls can be combined at inference time, so an image-prompt adapter and a structure adapter can be stacked to obtain joint text, image, and structure control.
  • The training objective is a distributional-matching objective in the generator's latent space, so the same machinery transfers from ControlNet-style adapters to IP-Adapter-style image-prompt adapters.
  • The primal-dual formulation gives an automatic early-stopping signal when the dual variable vanishes, avoiding expensive hyperparameter tuning for the boundary-loss weight.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the boundary-plus-consistency mechanism works as proven, NCT is a generic recipe: any condition expressible as a likelihood $p(c|x)$ can be injected into any one-step generator by training only a small adapter, and the same argument should extend beyond images to other latent-native generators.
  • A natural stress test is whether the single-particle, non-characteristic kernel surrogate is sufficient when the conditional distribution $p(x|c)$ is multimodal; a kernel of the form $-\|x-y\|^2$ cannot separate two distributions with the same mean, so the theoretical guarantee may need a characteristic kernel or a larger particle number to hold strictly.
  • The test-time composition result suggests that NCT adapters could be composed multiplicatively by merging condition likelihoods, provided each adapter's boundary condition is preserved; this looks testable without any further training.
  • The tight coupling between the boundary condition and the frozen generator's image prior suggests that NCT's quality ceiling is set by the original one-step generator; if the base generator drifts or is fine-tuned, adapters may need to be re-anchored.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Noise Consistency Training (NCT), a method for adding conditioning signals to a pre-trained one-step generator without retraining the base diffusion model. An adapter network is trained using a noise-consistency loss that compares the adapted generator's outputs on two noisy versions of the same latent noise, together with a boundary loss that anchors the adapter to the frozen one-step generator on fully coupled (z,c) pairs. The authors prove a theorem (Theorem 1 / Theorem A.1) stating that if the boundary loss is zero and the summed MMD between adjacent pushed-forward distributions is zero, then the adapted generator maps independent p(z)p(c) to the target joint distribution p_theta(x)p(c|x). They further claim in Lemma 2 that the practical noise-consistency loss is a single-particle estimator of this summed MMD. Experiments on Canny, HED, depth, super-resolution, and image-prompted generation report lower FID and consistency scores than ControlNet, DI+ControlNet, and JDM, using one network evaluation per image.

Significance. If the theoretical result applied to the implemented method, NCT would be a valuable lightweight approach to controllable one-step generation, avoiding additional distillation and preserving the base generator. The paper is also commendable for tackling a relevant problem and for providing a modular adapter formulation with experiments across several control modalities. However, the theoretical guarantee is the central justification of the method, and the connection between the theorem and the practical loss is not established: Lemma 2 is algebraically incorrect, and the kernels used in the implementation are not characteristic. The experimental evaluation is weakened by an unconventional FID protocol, an internal dataset, and the absence of error bars. The idea is interesting, but the paper does not currently provide a sound theoretical foundation or a fully convincing empirical validation for its central claims.

major comments (3)
  1. [§3.2, Lemma 2 and Appendix A.3, Remark 1] The claim that the noise-consistency loss is a practical estimation of the summed MMD is algebraically incorrect. For the implemented kernel k(x,y)=-||x-y||^2, one has MMD^2(P,Q)=2||μ_P-μ_Q||^2, whereas the single-particle paired expectation E||X-Y||^2 with independent X~P and Y~Q equals ||μ_P-μ_Q||^2 + tr Σ_P + tr Σ_Q. The variance terms remain positive even when P=Q, so the surrogate can be minimized by collapsing variance rather than by matching the interpolated distributions. Moreover, Eq. (6) and Algorithm 1 use the same z and ϵ in both z_{t_n} and z_{t_{n-1}}, so the two evaluations are not independent draws from p_{θ,φ,t_n} and p_{θ,φ,t_{n-1}}; the one-particle MMD estimator interpretation therefore does not apply. The appendix provides no proof of Lemma 2; Remark 1 only restates the implementation. Consequently, the chain of equalities (T_φ)#ν_{t_N}=...=(T_φ)#ν_{t_0} in Theorem A.1 is not connected to the loss that Algorithm 1 actually minimizes.
  2. [§3.2, Theorem 1 and Remark 1] Even if the surrogate loss were an MMD estimator, the theorem's consistency condition requires a characteristic kernel, while the practical loss uses kernels that are not characteristic. The paper itself notes that k(x,y)=-||x-y||^2 is not a proper positive-definite kernel; for this kernel, MMD^2(P,Q)=2||μ_P-μ_Q||^2, which characterizes only first moments rather than full distributions. The pseudo-Huber kernel mentioned in Remark 1 is conditionally positive definite and induces an energy distance, but the paper does not establish that it is characteristic on the relevant image space, nor that the single-particle paired loss estimates the corresponding MMD. Thus the conclusion that f_{θ,φ} maps independent p(z)p(c) to p_θ(x)p(c|x) is not established for the objective actually minimized.
  3. [§4.1, Evaluation Metric] The FID protocol is nonstandard: the FID score is computed by comparing images generated by the unconditional base model against images generated with the conditional inputs, rather than by comparing generated images to a real-image reference distribution. This measures the marginal distance from the base generator, not image quality in the usual FID sense. In addition, the paper reports no error bars or statistical significance tests, and the dataset is internal. The abstract's claim of 'state-of-the-art controllable generation' is therefore not independently supported by the reported numbers.
minor comments (5)
  1. [§3.2, Eq. (6)] The notation E_{z,c|z,ϵ} in the simplified expression is ill-formed; the expectation should be over z, c, and ϵ separately.
  2. [§3.2 and Appendix A.3] The symbol N is used both for the number of time steps in Eq. (7) and for the particle number in Remark 1; please disambiguate these two uses.
  3. [§4.1] Please specify how the consistency measure h(x) is defined for the super-resolution task, where the condition is a low-resolution image rather than an edge or depth map.
  4. [§4.3 and Appendix B] There are several typos: 'find find a shortcut solution', 'Contorlnet', and 'Geneartion' should be corrected.
  5. [§4.2, Table 2] The IP-Adapter result is taken from the official report; please clarify whether the base model, prompt set, and evaluation protocol are identical to those used for NCT so that the comparison is meaningful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 derives conditional matching from explicit boundary and consistency conditions, and the target distribution is defined by the frozen generator and external condition model, not by the adapter.

full rationale

The derivation chain is self-contained. In Appendix A.3, Theorem A.1 assumes (1) zero boundary loss, which by Lemma A.2 gives (T_phi)#nu_t0 = eta, and (2) zero summed MMD between adjacent pushed-forward distributions, which with a characteristic kernel forces (T_phi)#nu_tk to be constant along the interpolation. Lemma A.3 identifies nu_t0 with the coupled measure gamma(z)p(c|f_theta(z)) and nu_tN with the independent coupling rho = gamma(z)p(c); combining the chain with the boundary condition yields (T_phi)#rho = eta. The target eta = p_theta(x)p(c|x) is built from the frozen f_theta and the external p(c|x) model, so the conclusion is not assumed among the inputs. The practical noise-consistency loss (Eq. 6) is presented as a single-particle surrogate of the population MMD; Remark 1 itself flags that -||x-y||^2 is not a proper positive-definite kernel and that the choice works only in practice. Whether the one-particle paired loss actually drives the population MMD to zero is an idealization/correctness gap between Algorithm 1 and the theorem, not a circular reduction. No parameter is fitted to a subset of data and then reported as a prediction; the boundary and consistency conditions are theoretical sufficient conditions, not fitted quantities. Self-citations such as Diff-Instruct [3] and JDM [9] appear as construction of the base one-step generator and as comparison baselines; they do not carry the central derivation. The empirical SOTA claims rest on the reported benchmarks rather than on the theorem, and are not circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard measure-theoretic tools, domain assumptions about the frozen generator and the condition model, and idealized conditions (existence and reachability of a parameter set with exactly zero losses) that the practical algorithm does not guarantee. No new physical entities are introduced.

free parameters (3)
  • Margin ξ for boundary constraint = not reported
    Eq. (9) introduces a fixed margin ξ in the constrained optimization; the primal-dual update (Eq. 12) depends on ξ to set the constraint, but the paper never specifies its value.
  • Kernel choice for MMD surrogate = k(x,y) = -||x-y||² or pseudo-Huber
    Remark 1 introduces a non-characteristic kernel to map the theoretical MMD to the practical l2/pseudo-Huber loss; this choice is hand-picked to make the training work and is not derived.
  • Noise schedule αt, σt = unspecified VP schedule
    Eq. (5) uses variance-preserving diffusion with coefficients αt, σt, but the exact schedule is not given; results may depend on this choice.
assumptions (4)
  • standard math MMD with a characteristic kernel is a metric, and standard measure-theoretic tools (Fubini, push-forward change of variables) apply.
    Used throughout Appendix A; standard mathematical background.
  • domain assumption The pre-trained one-step generator fθ has output distribution pθ(x) ≈ p_d(x) and remains frozen; its latent space is standard Gaussian so diffused z_t remains valid input for all t.
    Section 3.2, Eq. (5), and Section 2; the whole approach assumes fθ can map any standard Gaussian latent to a high-quality image, and that z_t for all t is standard Gaussian under VP diffusion.
  • domain assumption The condition distribution p(c|x) is known and samplable; for structural controls it is the deterministic extractor (Canny, HED, depth), for image prompts the paper does not specify how c is sampled for training.
    Section 3.2 and Algorithm 1 line 5 'Sample condition c corresponding to x by p(c|x)'; the theory and algorithm depend on this coupling, but the image-prompt experiments (Sec. 4.2) do not explain the pairing procedure.
  • ad hoc to paper There exists a parameter set φ satisfying the boundary condition and the consistency condition exactly (zero population MMD), and the practical surrogate loss drives the training toward this set.
    Theorem 1 assumes these conditions; the paper does not prove the loss landscape contains such a φ or that gradient descent on the single-particle surrogate converges to it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls." pith.science (2026). https://pith.science/paper/H5B2PFYO

@misc{pith2026250619741,
  author       = {Pith},
  title        = {Pith review of: Noise Consistency Training: A Native Approach for One-Step Generator in Learning Additional Controls},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H5B2PFYO}},
  note         = {Machine review of arXiv:2506.19741}
}
read the original abstract

The pursuit of efficient and controllable high-quality content generation remains a central challenge in artificial intelligence-generated content (AIGC). While one-step generators, enabled by diffusion distillation techniques, offer excellent generation quality and computational efficiency, adapting them to new control conditions--such as structural constraints, semantic guidelines, or external inputs--poses a significant challenge. Conventional approaches often necessitate computationally expensive modifications to the base model and subsequent diffusion distillation. This paper introduces Noise Consistency Training (NCT), a novel and lightweight approach to directly integrate new control signals into pre-trained one-step generators without requiring access to original training images or retraining the base diffusion model. NCT operates by introducing an adapter module and employs a noise consistency loss in the noise space of the generator. This loss aligns the adapted model's generation behavior across noises that are conditionally dependent to varying degrees, implicitly guiding it to adhere to the new control. Theoretically, this training objective can be understood as minimizing the distributional distance between the adapted generator and the conditional distribution induced by the new conditions. NCT is modular, data-efficient, and easily deployable, relying only on the pre-trained one-step generator and a control signal model. Extensive experiments demonstrate that NCT achieves state-of-the-art controllable generation in a single forward pass, surpassing existing multi-step and distillation-based methods in both generation quality and computational efficiency. Code is available at https://github.com/Luo-Yihong/NCT

Figures

Figures reproduced from arXiv: 2506.19741 by the authors.

Figure 1
Figure 1. Framework description of our method. model’s ODE trajectories on an instance-by-instance basis. These methods can encounter difficulties with precise instance-level matching. 2) Distribution matching, often realized via score distillation [5, 3, 29, 4], which aims to align the output distributions of the student and teacher models using divergence metrics. Our work utilizes a pre-trained one-step generator, which it… view at source ↗
Figure 2
Figure 2. Qualitative comparisons on controllable generation across different control signals against [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Visual samples of image-reference generations. The samples are generated by our NCT [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Both boundary loss and noise consistency [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 16 canonical work pages

  1. [1]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  2. [2]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. PMLR, 2015

    2015

  3. [3]

    Diff- instruct: A universal approach for transferring knowledge from pre-trained diffusion models

    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff- instruct: A universal approach for transferring knowledge from pre-trained diffusion models. Advances in Neural Information Processing Systems, 36, 2023

    work page 2023

  4. [4]

    You only sample once: Taming one-step text-to-image synthesis by self-cooperative diffusion gans, 2024

    Yihong Luo, Xiaolong Chen, Xinghua Qu, Tianyang Hu, and Jing Tang. You only sample once: Taming one-step text-to-image synthesis by self-cooperative diffusion gans, 2024

    work page 2024

  5. [5]

    One-step diffusion with distribution matching distillation.arXiv preprint arXiv:2311.18828, 2023

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation.arXiv preprint arXiv:2311.18828, 2023

    arXiv 2023

  6. [6]

    Consistency models, 2023

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, 2023

    2023

  7. [7]

    Inductive moment matching.arXiv preprint arXiv:2503.07565, 2025

    Linqi Zhou, Stefano Ermon, and Jiaming Song. Inductive moment matching.arXiv preprint arXiv:2503.07565, 2025

    arXiv 2025

  8. [8]

    Sdxs: Real-time one-step latent diffusion models with image conditions.arxiv, 2024

    Xuanwu Yin Yuda Song, Zehao Sun. Sdxs: Real-time one-step latent diffusion models with image conditions.arxiv, 2024

    work page 2024

Show all 43 references
  1. [9]

    Adding additional control to one-step diffusion with joint distribution matching.arXiv preprint arXiv:2503.06652, 2025

    Yihong Luo, Tianyang Hu, Yifan Song, Jiacheng Sun, Zhenguo Li, and Jing Tang. Adding additional control to one-step diffusion with joint distribution matching.arXiv preprint arXiv:2503.06652, 2025

    2025 arXiv

  2. [10]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    2023 arXiv

  3. [11]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  4. [12]

    Universal guidance for diffusion models

    Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Roni Sengupta, Micah Goldblum, Jonas Geip- ing, and Tom Goldstein. Universal guidance for diffusion models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  5. [13]

    Elucidating the design space of classifier-guided diffusion generation.arXiv preprint arXiv:2310.11311, 2023

    Jiajun Ma, Tianyang Hu, Wenjia Wang, and Jiacheng Sun. Elucidating the design space of classifier-guided diffusion generation.arXiv preprint arXiv:2310.11311, 2023

    2023 arXiv

  6. [14]

    Reward- instruct: A reward-centric approach to fast photo-realistic image generation.arXiv preprint arXiv:2503.13070, 2025

    Yihong Luo, Tianyang Hu, Weijian Luo, Kenji Kawaguchi, and Jing Tang. Reward- instruct: A reward-centric approach to fast photo-realistic image generation.arXiv preprint arXiv:2503.13070, 2025

    2025 arXiv

  7. [15]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 10

    2023

  8. [16]

    A kernel two-sample test.The Journal of Machine Learning Research, 13(1):723–773, 2012

    Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test.The Journal of Machine Learning Research, 13(1):723–773, 2012

    2012

  9. [17]

    Integral probability metrics and their generating classes of functions.Advances in applied probability, 29(2):429–443, 1997

    Alfred Müller. Integral probability metrics and their generating classes of functions.Advances in applied probability, 29(2):429–443, 1997

    1997

  10. [18]

    Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models.arXiv preprint arXiv:2211.01095, 2022

    Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models.arXiv preprint arXiv:2211.01095, 2022

    2022 arXiv

  11. [19]

    Unipc: A unified predictor- corrector framework for fast sampling of diffusion models.Advances in Neural Information Processing Systems, 36:49842–49869, 2023

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor- corrector framework for fast sampling of diffusion models.Advances in Neural Information Processing Systems, 36:49842–49869, 2023

    2023

  12. [20]

    Accelerating diffusion sampling with optimized time steps

    Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li. Accelerating diffusion sampling with optimized time steps. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8292–8301, 2024

    2024

  13. [21]

    Freeu: Free lunch in diffusion u-net

    Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733–4743, 2024

    2024

  14. [22]

    The surprising effectiveness of skip-tuning in diffusion sampling

    Jiajun Ma, Shuchen Xue, Tianyang Hu, Wenjia Wang, Zhaoqiang Liu, Zhenguo Li, Zhi-Ming Ma, and Kenji Kawaguchi. The surprising effectiveness of skip-tuning in diffusion sampling. arXiv preprint arXiv:2402.15170, 2024

    2024 arXiv

  15. [23]

    Knowledge distillation in iterative generative models for improved sampling speed.arXiv preprint arXiv:2101.02388, 2021

    Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed.arXiv preprint arXiv:2101.02388, 2021

    2021 arXiv

  16. [24]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14297–14306, June 2023

    2023

  17. [25]

    Progressive distillation for fast sampling of diffusion models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. InInternational Conference on Learning Representations, 2022

    2022

  18. [26]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023

    2023

  19. [27]

    Improved techniques for training consistency models

    Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  20. [28]

    Perflow: Piecewise rectified flow as universal plug-and-play accelerator.arXiv preprint arXiv:2405.07510, 2024

    Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator.arXiv preprint arXiv:2405.07510, 2024

    2024 arXiv

  21. [29]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation

    Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. InInternational Conference on Machine Learning, 2024

    2024

  22. [30]

    CCM: Real-time controllable visual content creation using text-to-image consistency models

    Jie Xiao, Kai Zhu, Han Zhang, Zhiheng Liu, Yujun Shen, Zhantao Yang, Ruili Feng, Yu Liu, Xueyang Fu, and Zheng-Jun Zha. CCM: Real-time controllable visual content creation using text-to-image consistency models. InForty-first International Conference on Machine Learning, 2024

    2024

  23. [31]

    Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis, 2024

    2024

  24. [32]

    Learning few-step diffusion models by trajectory distribution matching, 2025

    Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching, 2025

    2025

  25. [33]

    Instaflow: One step is enough for high-quality diffusion-based text-to-image generation.arXiv preprint arXiv:2309.06380, 2023

    Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, and Qiang Liu. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation.arXiv preprint arXiv:2309.06380, 2023. 11

    2023 arXiv

  26. [34]

    Con- strained learning with non-convex losses.arXiv:2103.05134, 2021

    Luiz FO Chamon, Santiago Paternain, Miguel Calvo-Fullana, and Alejandro Ribeiro. Con- strained learning with non-convex losses.arXiv:2103.05134, 2021

    2021 arXiv

  27. [35]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    2022

  28. [36]

    A computational approach to edge detection.PAMI, 1986

    John Canny. A computational approach to edge detection.PAMI, 1986

    1986

  29. [37]

    Holistically-nested edge detection

    Saining Xie and Zhuowen Tu. Holistically-nested edge detection. InICCV, 2015

    2015

  30. [38]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    2017

  31. [39]

    Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. 2023

    2023

  32. [40]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings,...

    2014

  33. [41]

    Att= 0, the joint density isp 0(z,c) =γ(z)p(c|f θ(z))

  34. [42]

    Att= 1, the joint density isp 1(z,c) =γ(z)p(c). Proof. The joint density at timet is given bypt(z,c) =p t(z|c)p(c). Substituting the definition of pt(z|c): pt(z,c) =p(c) Z Rm N(z;α tz0,σ 2 tIm)pdata(z0|c)dz0 Now, substitute the definition ofp data(z0|c) = γ(z0)p(c|fθ(z0)) p(c)...

  35. [128]

    We use a probability of 0.05 to drop text during training. 18

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.