REVIEW 13 cited by
Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Diffusion models, which convert noise into new data instances by learning to reverse a Markov diffusion process, have become a cornerstone in contemporary generative modeling. While their practical power has now been widely recognized, the theoretical underpinnings remain far from mature. In this work, we develop a suite of non-asymptotic theory towards understanding the data generation process of diffusion models in discrete time, assuming access to $\ell_2$-accurate estimates of the (Stein) score functions. For a popular deterministic sampler (based on the probability flow ODE), we establish a convergence rate proportional to $1/T$ (with $T$ the total number of steps), improving upon past results; for another mainstream stochastic sampler (i.e., a type of the denoising diffusion probabilistic model), we derive a convergence rate proportional to $1/\sqrt{T}$, matching the state-of-the-art theory. Imposing only minimal assumptions on the target data distribution (e.g., no smoothness assumption is imposed), our results characterize how $\ell_2$ score estimation errors affect the quality of the data generation processes. In contrast to prior works, our theory is developed based on an elementary yet versatile non-asymptotic approach without resorting to toolboxes for SDEs and ODEs. Further, we design two accelerated variants, improving the convergence to $1/T^2$ for the ODE-based sampler and $1/T$ for the DDPM-type sampler, which might be of independent theoretical and empirical interest.
Forward citations
Cited by 13 Pith papers
-
Denoising growth complexity: Data geometry and certified schedules for diffusion sampling
A new measure, the denoising growth complexity, provides local KL error bounds for Euler diffusion samplers and yields certified, geometry-adaptive schedules.
-
Low-dimensional adaptation of diffusion models: Convergence in total variation
Under exact score functions and a covering-number notion of intrinsic dimension, DDIM and DDPM reach TV error epsilon in O-tilde(k/epsilon) iterations.
-
Fast Score-Based Sampling via Log-Concave Reductions
Score-based sampling reduces to a short sequence of strongly log-concave sampling problems, giving √d polylog(1/ε) complexity bounds and logarithmic dependence on the condition number for log-concave targets.
-
Faster Diffusion Models via Higher-Order Approximation
A new higher-order ODE sampler for diffusion models is proven to reach ε total-variation accuracy with eO(d^{1+2/K}/ε^{1/K}) iterations under mild assumptions.
-
Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions
Consistency models with approximate self-consistency converge to the data distribution in Wasserstein distance under mild data assumptions, with a provable improvement from a second sampling step.
-
HYVINT: Intensity-Driven Hypergraph Generation with Variational Embeddings
HYVINT introduces an intensity-driven incidence mechanism and tractable variational estimator for hypergraph generation, with error bounds and empirical gains in fidelity, novelty, and diversity.
-
Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attention
Sparse attention with chunk-aware sparsity growth and hierarchical frame/block selection accelerates autoregressive video diffusion at ~1.3x with VBench quality on par with dense attention.
-
Provable Diffusion Posterior Sampling for Bayesian Inversion
A diffusion posterior sampler using Monte Carlo Langevin score estimation and warm start is proven to converge in Wasserstein-2 distance under semi-log-concavity and sub-Gaussian assumptions, and outperforms DPS/TV on...
-
Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models
A TV convergence bound O(d^{7/4} ε^{1/2} + d(dH)^p) is proved for p-th order (exponential) Runge-Kutta samplers of probability-flow ODEs under C² smoothness of the learned score.
-
Learning Single Index Models with Diffusion Priors
A method called SIM-DMIS recovers signals from single index model measurements in about 150 neural function evaluations by starting diffusion model inversion at an intermediate time matched to the measurement noise level.
-
On the Feature Learning in Diffusion Models
Diffusion models learn signal and noise features in proportion to n times the SNR squared, while classifiers sharply switch between the two.
-
A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate
Diffusion model generation error is decomposed through an entropy-production-rate identity that claims O(h²) Euler–Maruyama KL bounds and unifies score SDE, PF-ODE, flow matching, and stochastic interpolant analyses.
-
Non-asymptotic convergence bound of conditional diffusion models
CARD's generated conditional distribution is shown to converge in Wasserstein distance to the true conditional distribution, with a separate score-estimation error bound controlled by network resolution and distributi...
Discussion (0). Continue with ORCID to comment.