Elucidating the Preconditioning in Consistency Distillation

Fan Bao; Guande He; Jianfei Chen; Jun Zhu; Kaiwen Zheng

REVIEW 3 major objections 6 minor 24 references

Elucidating the Preconditioning in Consistency Distillation

T0 review · 3 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the preconditioning coefficients in consistency distillation can be derived rather than hand-crafted, and that the resulting Analytic-Precond accelerates consistency-trajectory-model training by 2x to 3x in…

desk verdict A genuinely new preconditioning family and credible 2-3x multi-step acceleration; the 'analytic optimality' label is not actually derived. read the letter →

arxiv 2502.02922 v3 pith:U35HL7HW submitted 2025-02-05 cs.LG cs.CV

Kaiwen Zheng , Guande He , Jianfei Chen , Fan Bao , Jun Zhu This is my paper · ORCID

classification cs.LGcs.CV

keywords consistencydistillationdiffusionmodelspreconditioningprobabilityflowODEtrajectorytrainingaccelerationgap

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Consistency distillation trains a student network to jump backward along a teacher diffusion model's probability-flow ODE; the stability of that jump depends on preconditioning coefficients that blend the input data with the network output. The paper argues that these coefficients should be understood as a discretization of the teacher ODE, which turns them from hand-crafted choices into tunable quantities. It proposes Analytic-Precond, which sets the coefficients from the teacher denoiser through two closed-form expressions, one controlling the stiffness of the ODE and one minimizing a bound on the consistency gap between teacher and optimal student denoiser. In consistency trajectory models the new coefficients nearly coincide with the old ones for long jumps but differ for short intermediate jumps, and there they give 2x to 3x training acceleration at matched FID while keeping student trajectories closer to the teacher's.

What carries the argument

The load-bearing object is a generalized ODE obtained by rescaling the teacher PF-ODE with functions $L_t$, $S_t$ built from parameter curves $l_t$, $s_t$; discretizing it with the Euler method gives the coefficient family $f(t,s)=[L_t S_t+(l_t-1)(\eta_s-\eta_t)]/(L_s S_s)$, $g(t,s)=(\eta_s-\eta_t)/(L_s S_s)$, from which the consistency function is $f_\theta(x,t,s)=f(t,s)x+g(t,s)D_\theta(x,t,s)$. Proposition 3.1 bounds the consistency gap by a maximum over the jump interval of $\|dg_\phi(x_\tau,\tau)/d\lambda_\tau - s_\tau g_\phi(x_\tau,\tau)\|$, and Eqs. (15) and (17) choose $l_t$, $s_t$ by closed-form pointwise minimization of the local versions of these terms, computed from teacher Jacobian-vector products. A backward-Euler rewriting of the coefficients is used to stabilize long jumps.

What would settle it

Train a CTM with $s_t$ values deliberately far from Eq. (17), for instance random constants, and compare two-step FID training curves with Analytic-Precond; if the curves overlap, the speedup does not come from minimizing the consistency gap. A more direct check is to evaluate the supremum in Proposition 3.1 for optimized versus non-optimized $s_t$ and see whether pointwise minimization actually lowers the bound.

Watch

Extended reading notes

Core claim

The paper's central claim is that the right preconditioning for consistency distillation follows from discretizing a generalized version of the teacher PF-ODE, $d(L_t x_t)/d\eta_t = g_\phi(x_t,t)/S_t$, which is exactly equivalent to the original ODE but carries two free parameter curves $\{l_t,s_t\}$. Euler discretization of this ODE produces a family of consistency functions $f_\theta(x,t,s)=f(t,s)x+g(t,s)D_\theta(x,t,s)$; the paper chooses $l_t$ by minimizing the Frobenius norm of the vector field's Jacobian and $s_t$ by minimizing the pointwise expectation of $\|dg_\phi(x_t,t)/d\lambda_t - s_t g_\phi(x_t,t)\|^2$, the local version of the term controlled by a proved bound on the consistency gap $\|D_{\theta^*}(x_t,t,s)-D_\phi(x_t,t)\|$. The resulting Analytic-Precond preserves single-step and CM performance, accelerates two-step CTM training by 2x to 3x on CIFAR-10, FFHQ 64, and ImageNet 64, and improves trajectory alignment in a 1-D Gaussian-mixture illustration.

Load-bearing premise

The derivation assumes that minimizing the pointwise expected error at the jump's starting time also shrinks the worst-case error over the entire jump interval; the paper does not prove that monotonicity, so the optimality claim rests on that step.

Editorial extensions

If this is right

For two-step CTM generation, Analytic-Precond reaches the reference FID in roughly a third to half the training iterations of the original preconditioning.
Multi-step generation with 3 to 10 function evaluations keeps small but consistent FID improvements, indicating better alignment between consistency function and denoiser.
In single-step or long-jump settings the optimized coefficients are nearly identical to existing ones, so the original preconditioning is already close to optimal for those cases.
Analytic-Precond remains compatible with GAN-loss training, with no performance degradation, even though that regime no longer follows the teacher ODE faithfully.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper's benchmarks, the same teacher-only coefficient computation could be applied to larger latent diffusion models at a cost below 1% of training time, so the acceleration should transfer if the trajectory-geometry assumption holds.
The consistency-gap criterion suggests a general design rule for distilling any ODE trajectory: choose the student parameterization to minimize the mismatch between the teacher vector field and the student's one-step map, not just the endpoint error.
The backward-Euler rewriting hints that the direction of discretization matters as much as the coefficients; testing midpoint or exponential-integrator discretizations of the same generalized ODE could yield further gains.
If the proof bridge between the supremum bound and the pointwise minimization is closed, Analytic-Precond becomes a genuine optimality result; until then the 2x to 3x speedup remains an empirical property.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The stress-test note has it right: the step from Prop. 3.1 to Eq. (17) is the paper's weak joint. The bound in (16) is a supremum over the interval [s,t] of a trajectory-dependent quantity; Eq. (17) minimizes a pointwise expectation at time t. No monotonicity or uniform-convergence argument bridges them, so calling the resulting coefficients 'analytically optimal' is not supported by the theory. That said, the empirical acceleration is real: across CIFAR-10 (un/conditional), FFHQ-64, and ImageNet-64, the two-step CTM training curves show a consistent 2-3x speedup to matched FID, and the final multi-step FIDs are slightly better. These are external metrics, not the fitted consistency gap, so the 2-3x claim is not circular.

What is genuinely new is the generalized ODE family (Eqs. 10-14), which recovers CTM preconditioning as a special case and gives a principled way to generate alternatives. The formulas for l_t and s_t (Eqs. 15 and 17) are new, and the paper correctly treats BCM as concurrent rather than claiming priority. The toy example and trajectory-alignment plots provide useful intuition, and the limitation section is honest about single-step parity and modest final FID gains.

Minor issues: the backward-Euler rewrite in Sec. 3.3 is under-specified—it defines the backward coefficients implicitly without giving explicit formulas or a derivation, so a reader cannot reproduce them without guesswork. Table 2's header says ImageNet 256x256 while the text says 64x64; presumably a typo. No code or error bars are provided, which matters for a paper whose main claim is training-time acceleration.

The central idea is solid and the empirical contribution is useful, but the 'Analytic' label should be softened to 'heuristic with empirical support' unless the authors can close the supremum-to-pointwise gap or show numerically that the minimizer of the pointwise objective also reduces the sup-norm bound in the relevant regime. As is, the paper deserves peer review, but a referee should push on that derivation and on code release.

For a reader working on diffusion distillation, this is a useful and mostly credible paper. I'd bring it to a reading group if it comes up, and I'd cite it for the generalized preconditioning family. I would not desk-reject it; a serious referee could turn it into a strong paper with moderate revisions.

Referee Report

3 major / 6 minor

Summary. This paper studies the preconditioning coefficients used in consistency distillation and connects them to the discretization of a generalized probability-flow ODE with free parameters L_t and S_t. The authors propose a family of preconditionings and derive a schedule, Analytic-Precond, whose coefficients l_t and s_t are computed from the teacher denoiser by minimizing a bound on the consistency gap. They report that the method matches the baseline in single-step generation and accelerates CTM training by 2x to 3x in two-step generation, with consistent FID improvements across several datasets.

Significance. If fully rigorous, the paper would provide the first principled design of preconditioning in consistency distillation, replacing hand-crafted coefficients with a teacher-derived schedule. The generalized-ODE construction is elegant, the coefficient computation is cheap (estimated at less than 1% of training time), and the reported FID improvements are consistent across datasets. The main weakness is that the step from the supremum bound in Proposition 3.1 to the pointwise least-squares objective in Eq. (17) is not justified, so the claimed analytic optimality is not established; the observed acceleration remains an empirical finding. The paper also lacks error bars on its training curves, which weakens the quantitative strength of the acceleration claim.

major comments (3)

[Section 3.3, Proposition 3.1 to Eq. (17)] The bound in Eq. (16) involves a supremum over tau in [s,t] of a per-trajectory residual, but Eq. (17) minimizes the pointwise squared expectation at time t. The text's argument that the prefactor (t/s)^{3C}-1/(3C) is approximately 1 for small log(t/s) does not explain why the minimizer of the pointwise objective should reduce the supremum over the interval, nor why an expectation over x_t is a valid proxy for the sup over tau along each trajectory. No monotonicity or uniform-convergence argument is supplied. This is load-bearing because the paper's central novelty is an analytically optimal preconditioning; without this step, the coefficients are a heuristic minimization of a proxy, and the 2x to 3x acceleration is an empirical finding rather than a consequence of the theory. Please add a proof or an empirical study showing that the pointwise proxy tracks the supremum bound, and adjust the claims accordingly.
[Section 3.3, backward Euler rewriting] The transformation from Eq. (14) to the backward coefficients is under-specified. The forward discretization contains D_phi(x_t,t), but the rewritten equation 'x_t = f_hat(s,t)x_s + g_hat(s,t)D_phi' does not state at which argument D_phi is evaluated (x_s, x_t, or an intermediate point), nor how f_hat and g_hat are computed from the forward coefficients. Because the backward Euler form is used in the experiments, this ambiguity directly affects reproducibility. Please derive the coefficients explicitly and specify the arguments of D_phi.
[Section 5.1, Figures 2 and 3] The training curves are single runs with no error bars or multiple seeds. Given that the central quantitative claim is a 2x to 3x training acceleration, the variability of these curves should be quantified. At minimum, provide a second independent run for the main CIFAR-10 CTM comparison to confirm that the observed acceleration is consistent, and clarify whether the FID values in Table 2 come from a single run or an average over seeds.

minor comments (6)

[Section 3.3, Eq. (17)] The notation dg_phi(x_tau,tau)/dlambda_tau in Eq. (16) and dg_phi(x_t,t)/dlambda_t in Eq. (17) would be clearer with explicit parentheses, e.g., d(g_phi(x_tau,tau))/dlambda_tau, and with the dependence on x_tau stated where the expectation is taken.
[Table 1] The Analytic-Precond entry for f(t,s) appears inconsistent with Eq. (14): Eq. (14) gives f(t,s) = [L_t S_t + (l_t - 1)(eta_s - eta_t)]/(L_s S_t), while Table 1 lists 'L_t S_s/(L_s S_s) + (1-l_s)(eta_s - eta_t)'. Please check the formula and the subscripts, and ensure the table matches the main derivation.
[Section 3.2, Eqs. (10)-(14)] The definitions of L_t and S_t are introduced verbally; writing the explicit integral forms L_t = exp(integral_{lambda_T}^{lambda_t} l_{t_lambda} dlambda) and S_t = exp(integral_{lambda_T}^{lambda_t} s_{t_lambda} dlambda) would help readers follow the change of variables and the Euler discretization.
[Section 5.3, Figure 7] The toy example is not described in sufficient detail; please clarify how the trajectories are computed, how 'the optimal denoiser' is obtained, and which preconditioning coefficients are used for the comparison.
[Limitations section] The limitation that Analytic-Precond 'fails to differ from previous preconditionings on long jumps' should be explicitly reconciled with the local derivation of Eq. (17); a short discussion of how the optimized coefficients behave for large t/s would be helpful.
[Appendix A.1, Eq. (22)] In the mean value theorem step, the interval [t, t_lambda_eta] should be written as [min{t, t_lambda_eta}, max{t, t_lambda_eta}] to avoid ambiguity when lambda is not monotone in eta.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Analytic-Precond's coefficients are explicit analytic minimizers, and the reported 2-3x acceleration is an external FID measurement rather than a fitted quantity.

full rationale

The derivation chain is self-contained and does not reduce to its inputs by construction. The generalized ODE in Eq. (12) is algebraically equivalent to the original teacher PF-ODE, and the induced preconditioning coefficients in Eq. (14) follow from Euler discretization of that equivalent ODE. The optimized parameters are explicit analytic minimizers: Eq. (15) minimizes an expected Frobenius norm of the ODE vector-field gradient and gives a closed-form solution for lt, while Eq. (17) is the closed-form least-squares coefficient for regressing dg_phi/dlambda on g_phi. Neither step fits a parameter to the FID numbers reported later. The 2-3x training acceleration claims in Section 5.1 are measured training curves on standard benchmarks (CIFAR-10, FFHQ, ImageNet), and Table 2 reports external FID sample quality rather than the consistency-gap objective that was minimized. The author-overlapping citation to DPM-Solver-v3 (Zheng et al., 2023a) motivates the lt criterion but is not load-bearing, because the paper itself states the explicit argmin and derives its closed-form solution; the center of the method does not depend on an unverified self-citation. The step from the Proposition 3.1 sup-norm bound to the pointwise expectation in Eq. (17) is a genuine mathematical gap and a correctness risk, but it is not circularity: a pointwise minimizer is not shown to minimize the supremum, so the 'analytically optimal' label is heuristic rather than forced by the bound. The optimality claim is made relative to an explicitly stated design criterion, not relative to the measured FID, so it is a transparent design choice rather than a disguised prediction.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central derivation introduces two free functional parameters, lt and st, that define a generalized ODE family. These are computed from the teacher denoiser, not from student training outcomes, which keeps the empirical acceleration externally benchmarked. The main unstated load-bearing assumptions are the proxy-to-supremum step in Eq. (17) and the boundedness of lt and st assumed in Proposition 3.1. No new physical entities are introduced.

free parameters (2)

lt(t) = lt = 1 - E_q(xt)[tr(grad_xt Dphi(xt,t))] / d (Eq. 15)
Introduced in the generalized ODE (Eq. 11); selected to minimize the Frobenius norm of the ODE right-hand side gradient. Computed from the teacher denoiser.
st(t) = st = E_q(xt)[gphi^T dgphi/dlambda_t] / E_q(xt)[||gphi||^2] (Eq. 17)
Introduced in the generalized time variable eta_t; selected by OLS to minimize the pointwise squared consistency-gap integrand, not the actual supremum in Proposition 3.1.

assumptions (6)

domain assumption The teacher denoiser Dphi correctly represents the PF-ODE trajectory, so the distillation target in Eq. (5) is the ground truth.
The method and experiments rely on pretrained EDM teachers; if the teacher is inaccurate, the student trajectory cannot align with the true ODE.
ad hoc to paper The optimal student denoiser Dtheta* satisfies Eq. (19), which is the Euler discretization of Eq. (13) with Dphi replaced by Dtheta.
Proposition 3.1 defines theta* in this way; no training dynamics or convergence to this object is analyzed.
ad hoc to paper The free functions lt and st are bounded by a constant C (|lt|, |st| <= C) so that the exponential factor in Eq. (16) is finite.
The paper assumes this in Proposition 3.1 but does not verify it for the values returned by Eqs. (15) and (17).
ad hoc to paper Minimizing the pointwise squared norm E[||dgphi/dlambda - st gphi||^2] at each t is a valid proxy for minimizing the supremum over tau in the consistency-gap bound.
Eq. (17) uses OLS at t, while Eq. (16) bounds a max over trajectories; no monotonicity or uniform-control argument is supplied.
domain assumption The expectations in Eqs. (15) and (17) are well approximated by Monte Carlo samples and Hutchinson's trace estimator.
Appendix B.1 uses 1024-4096 samples; no confidence intervals or convergence checks are reported.
standard math Dphi is twice differentiable enough for the chain rule and mean value theorem used in the proof of Proposition 3.1.
The proof in Appendix A.1 relies on differentiating hphi along the trajectory and applying the mean value theorem; neural-network denoisers are only piecewise smooth in practice.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Elucidating the Preconditioning in Consistency Distillation." pith.science (2026). https://pith.science/paper/U35HL7HW

@misc{pith2026250202922,
  author       = {Pith},
  title        = {Pith review of: Elucidating the Preconditioning in Consistency Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U35HL7HW}},
  note         = {Machine review of arXiv:2502.02922}
}

abstract

Consistency distillation is a prevalent way for accelerating diffusion models adopted in consistency (trajectory) models, in which a student model is trained to traverse backward on the probability flow (PF) ordinary differential equation (ODE) trajectory determined by the teacher model. Preconditioning is a vital technique for stabilizing consistency distillation, by linear combining the input data and the network output with pre-defined coefficients as the consistency function. It imposes the boundary condition of consistency functions without restricting the form and expressiveness of the neural network. However, previous preconditionings are hand-crafted and may be suboptimal choices. In this work, we offer the first theoretical insights into the preconditioning in consistency distillation, by elucidating its design criteria and the connection to the teacher ODE trajectory. Based on these analyses, we further propose a principled way dubbed \textit{Analytic-Precond} to analytically optimize the preconditioning according to the consistency gap (defined as the gap between the teacher denoiser and the optimal student denoiser) on a generalized teacher ODE. We demonstrate that Analytic-Precond can facilitate the learning of trajectory jumpers, enhance the alignment of the student trajectory with the teacher's, and achieve $2\times$ to $3\times$ training acceleration of consistency trajectory models in multi-step generation across various datasets.

Figures

Figures reproduced from arXiv: 2502.02922 by the authors.

**Figure 1.** Consistency distillation with preconditioning coefficients α, β. Recent advancements in few-step or even single-step generation of diffusion models are concentrated on distillation methods (Luhman & Luhman, 2021; Salimans & Ho, 2022; Meng et al., 2022a; Song et al., 2023; Kim et al., 2023; Sauer et al., 2023). Particularly, consistency models (CMs) (Song et al., 2023) have emerged as a prominent method for diffusio… view at source ↗

**Figure 2.** Training curves for single-step generation, and visualization of preconditionings for single [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Training curves for two-step generation. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Visualizations of the preconditioning coefficient [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Effects of BCM’s preconditioning on CTMs. Visualizations To intuitively understand the distinctions between Analytic-Precond and the original preconditioning in CTMs, we investigate the variations in coefficients f(t, s), g(t, s). We find that Analytic-Precond yields …

**Figure 6.** Figure 6: Effects of AnalyticPrecond with GAN loss. Compatibility with GAN loss CTMs introduce GAN loss to further enhance the one-step generation quality, employing a discriminator and adopting an alternative optimization approach akin to GANs. As shown in [PITH_FULL_IMAGE:…

**Figure 7.** Figure 7: Visualizations of the trajectory alignment, comparing [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Random samples produced by CTM and CTM + Analytic-Precond (Ours) with NFE=2. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages

[1]

Vidu: a highly consistent, dynamic and skilled text-to- video generator with diffusion models

Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to- video generator with diffusion models. arXiv preprint arXiv:2405.04233,

arXiv
[3]

For CIFAR-10 (unconditional), we train the model with a batch size of 256 for 200K iterations, which takes 5 days on 4 GPU cards

We run the experiments on a cluster of NVIDIA A800 GPU cards. For CIFAR-10 (unconditional), we train the model with a batch size of 256 for 200K iterations, which takes 5 days on 4 GPU cards. For CIFAR-10 (conditional), we train the model with a batch size of 512 for 150K iterations, which takes 4 days on 8 GPU cards. For FFHQ 64 ×64 (unconditional), we t...

work page 2009
[4]

C A DDITIONAL SAMPLES 16 Published as a conference paper at ICLR 2025 (a) CTM (CIFAR-10, Uncond) (b) CTM + Ours (CIFAR-10, Uncond) (c) CTM (CIFAR-10, Cond) (d) CTM + Ours (CIFAR-10, Cond) (e) CTM (FFHQ 64 × 64, Uncond) (f) CTM + Ours (FFHQ 64 × 64, Uncond) (g) CTM (ImageNet 64 × 64, Cond) (h) CTM + Ours (ImageNet 64 × 64, Cond) Figure 8: Random samples pr...

work page 2025
[6]

SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models

Martin Gonzalez, Nelson Fernandez, Thuy Tran, Elies Gherbi, Hatem Hajri, and Nader Masmoudi. Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. arXiv preprint arXiv:2305.14267,

work page Pith review arXiv
[7]

For CIFAR-10 and FFHQ 64 ×64, we select N = 18and the maximum number of sampling steps as 17, i.e., not restricting the range of jumping from t to s

The number of sampling steps, determined by the gap between t and s, is restricted to avoid excessive training time. For CIFAR-10 and FFHQ 64 ×64, we select N = 18and the maximum number of sampling steps as 17, i.e., not restricting the range of jumping from t to s. For ImageNet 64 ×64, we set N = 40and the maximum number of sampling steps to 20, so that ...

work page 2025
[9]

Learning multiple layers of features from tiny images

11 Published as a conference paper at ICLR 2025 Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report,

work page 2025
[11]

Maximum likelihood training for score-based diffusion odes by high order denoising score matching

Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In International Conference on Machine Learning, pp. 14429–14460. PMLR, 2022a. Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for...

arXiv
[12]

Latent consistency models: Synthe- sizing high-resolution images with few-step inference

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378,

arXiv

Show all 24 references

[13]

On distillation of guided diffusion models

Chenlin Meng, Ruiqi Gao, Diederik P Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In NeurIPS 2022 Workshop on Score-Based Methods, 2022a. Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdi...

2022
[14]

Hierarchical text- conditional image generation with CLIP latents

Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125,

arXiv
[15]

Adversarial diffusion dis- tillation

Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. arXiv preprint arXiv:2311.17042,

arXiv
[16]

Fast high-resolution image synthesis with latent adversarial diffusion distillation

Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rom- bach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015,

arXiv
[19]

Sageatten- tion2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization

Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. Sageatten- tion2: Efficient attention with thorough outlier smoothing and per-thread int4 quantization. arXiv preprint arXiv:2411.10958,
[20]

Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration

Jintao Zhang, Jia Wei, Pengle Zhang, Jun Zhu, and Jianfei Chen. Sageattention: Accurate 8-bit attention for plug-and-play inference acceleration. In International Conference on Learning Rep- resentations (ICLR), 2025a. Jintao Zhang, Chendong Xiang, Haofeng Huang, Jia Wei, Haoc...
[2009]

Bidirectional consistency models

Liangchen Li and Jiajun He. Bidirectional consistency models. arXiv preprint arXiv:2403.18035,

arXiv
[2011]

Videolcm: Video latent consistency model

Xiang Wang, Shiwei Zhang, Han Zhang, Yu Liu, Yingya Zhang, Changxin Gao, and Nong Sang. Videolcm: Video latent consistency model. arXiv preprint arXiv:2312.09109,

arXiv
[2014]

Photorealistic video generation with diffusion models

Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models. arXiv preprint arXiv:2312.06662,

arXiv
[2015]

Denoising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a. Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score- based diffusion models. In Advan...

2025
[2018]

Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics

Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm-solver-v3: Improved diffusion ode solver with empirical model statistics. In Thirty-seventh Conference on Neural Information Pro- cessing Systems, 2023a. Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Improved techniqu...

2025 arXiv
[2020]

Imagen video: High definition video generation with diffusion models

Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303,

arXiv
[2021]

Scaling rectified flow transformers for high-resolution image synthesis

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206,

arXiv
[2022]

ImageNet: A large-scale hier- archical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE,

2009
[2023]

Weiss, Mohammad Norouzi, and William Chan

10 Published as a conference paper at ICLR 2025 Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Wave- grad: Estimating gradients for waveform generation. In International Conference on Learning Representations,

2025
[2024]

Stable video diffusion: Scaling latent video diffusion models to large datasets

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127,

arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Vidu: a highly consistent, dynamic and skilled text-to- video generator with diffusion models

Fan Bao, Chendong Xiang, Gang Yue, Guande He, Hongzhou Zhu, Kaiwen Zheng, Min Zhao, Shilong Liu, Yaole Wang, and Jun Zhu. Vidu: a highly consistent, dynamic and skilled text-to- video generator with diffusion models. arXiv preprint arXiv:2405.04233,

arXiv

[2] [3]

For CIFAR-10 (unconditional), we train the model with a batch size of 256 for 200K iterations, which takes 5 days on 4 GPU cards

We run the experiments on a cluster of NVIDIA A800 GPU cards. For CIFAR-10 (unconditional), we train the model with a batch size of 256 for 200K iterations, which takes 5 days on 4 GPU cards. For CIFAR-10 (conditional), we train the model with a batch size of 512 for 150K iterations, which takes 4 days on 8 GPU cards. For FFHQ 64 ×64 (unconditional), we t...

work page 2009

[3] [4]

C A DDITIONAL SAMPLES 16 Published as a conference paper at ICLR 2025 (a) CTM (CIFAR-10, Uncond) (b) CTM + Ours (CIFAR-10, Uncond) (c) CTM (CIFAR-10, Cond) (d) CTM + Ours (CIFAR-10, Cond) (e) CTM (FFHQ 64 × 64, Uncond) (f) CTM + Ours (FFHQ 64 × 64, Uncond) (g) CTM (ImageNet 64 × 64, Cond) (h) CTM + Ours (ImageNet 64 × 64, Cond) Figure 8: Random samples pr...

work page 2025

[4] [6]

SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models

Martin Gonzalez, Nelson Fernandez, Thuy Tran, Elies Gherbi, Hatem Hajri, and Nader Masmoudi. Seeds: Exponential sde solvers for fast high-quality sampling from diffusion models. arXiv preprint arXiv:2305.14267,

work page Pith review arXiv

[5] [7]

For CIFAR-10 and FFHQ 64 ×64, we select N = 18and the maximum number of sampling steps as 17, i.e., not restricting the range of jumping from t to s

The number of sampling steps, determined by the gap between t and s, is restricted to avoid excessive training time. For CIFAR-10 and FFHQ 64 ×64, we select N = 18and the maximum number of sampling steps as 17, i.e., not restricting the range of jumping from t to s. For ImageNet 64 ×64, we set N = 40and the maximum number of sampling steps to 20, so that ...

work page 2025

[6] [9]

Learning multiple layers of features from tiny images

11 Published as a conference paper at ICLR 2025 Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report,

work page 2025

[7] [11]

Maximum likelihood training for score-based diffusion odes by high order denoising score matching

Cheng Lu, Kaiwen Zheng, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In International Conference on Machine Learning, pp. 14429–14460. PMLR, 2022a. Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for...

arXiv

[8] [12]

Latent consistency models: Synthe- sizing high-resolution images with few-step inference

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378,

arXiv