Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis

Arnaud Doucet; George Deligiannidis; Patrick Rebeschini; Tyler Farghly

Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis

T0 review · 1 major / 0 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Diffusion models generalise because their training and sampling procedures are implicitly regularised, and one quantity — score stability — turns that algorithmic insensitivity into explicit generalisation-gap bounds.

desk verdict A genuinely new algorithm-dependent stability framework for diffusion models, with a real but restrictive assumption problem in the flagship ERM result. read the letter →

arxiv 2507.03756 v1 pith:SFCUMWGV submitted 2025-07-04 stat.ML cs.LGmath.STstat.TH

Tyler Farghly , Patrick Rebeschini , George Deligiannidis , Arnaud Doucet This is my paper

classification stat.MLcs.LGmath.STstat.TH MSC 68T0568T0760J60

keywords diffusionmodelsscorestabilityalgorithmicgeneralisationgapimplicitregularisationdenoisingmatchingmanifoldhypothesisstochasticgradientdescent

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

It has been shown that when training and sampling are performed perfectly, diffusion models memorise their training data, yet in practice they generate novel samples; this paper argues the explanation is algorithmic rather than architectural. It introduces score stability, which measures how much a score-matching algorithm's output changes when one training point is replaced by an independent draw, and proves that a small stability constant directly bounds the expected generalisation gap between population and empirical denoising score-matching losses. Applying the framework to three standard settings, the paper finds that the denoising objective is implicitly regularised on its own, that coarse sampler discretisation acts as a second regulariser, and that the high-variance gradient noise of SGD acts as a third, with stability bounds that do not grow with the number of training iterations. If the framework is right, diffusion-model generalisation can be engineered by tuning algorithmic sensitivity rather than by carefully constrained network classes.

What carries the argument

Score stability (Definition 2): a score-matching algorithm $A_{\mathrm{sm}}$ is $\epsilon_{\mathrm{stab}}$-score stable if, for every single training point $x_i$ replaced by an independent copy $\tilde x$, the expected $L^2$ distance between the two output score functions, integrated over the noising process started at $\tilde x$ and minimised over couplings of the algorithm's randomness, is at most $\epsilon_{\mathrm{stab}}^2$. Three mechanisms carry the argument. Lemma 1 shows the empirical denoising objective has a unique minimiser, the empirical score $\nabla\log\hat p_t$, which makes the objective strongly convex in function space, so ERM outputs a stable function. A Harnack inequality for the Ornstein–Uhlenbeck semigroup converts that strong-convexity control, which lives in $L^2(\hat p_t)$, into the $L^2(p_{t|0}(\cdot|\tilde x))$ control that score stability demands, and pulls in the manifold dimension $d_*$. For the optimisation results, a Wasserstein-contraction coupling of two SGD trajectories — built from reflection coupling — turns the denoising estimator's high-variance gradient noise into a contractive force, so the stability bound stops growing with the iteration count.

What would settle it

Run the leave-one-out experiment the definition describes: train the ERM score on a dataset and on each single-point replacement, compute the averaged $L^2$ difference of the two networks over the noising process, and check whether it decays with $N$ at the rate $\epsilon_{\mathrm{stab}}^2 \lesssim C\,(C\,C_{\mathrm{sm}} N^{-2}+\mathbb{E}[\hat{\ell}_{\mathrm{sm}}(\hat s)])^c$ that Proposition 6 predicts. On the same family of trained networks, evaluate $\sup_{s,s'\in H}\|s(\cdot,t)-s'(\cdot,t)\|_\infty\,\sigma_t^2$ across noise scales: if it is unbounded as $t\to0$, Assumption 5 fails and the automatic-stability conclusion does not hold for realistic score networks.

Watch

Extended reading notes

Core claim

The central claim is that score stability is the right quantity governing diffusion generalisation: an algorithm whose output changes little under single-point dataset replacement necessarily has a small expected generalisation gap. Theorem 3 makes this precise, bounding the gap between square roots of expected population and expected empirical denoising score-matching losses by $\epsilon_{\mathrm{stab}}$, and the score-matching gap by $2\epsilon_{\mathrm{stab}}\,\mathbb{E}[\hat{\ell}_{\mathrm{dsm}}(\hat s)]^{1/2}+\epsilon_{\mathrm{stab}}^2$. Proposition 6 then shows the workhorse algorithm — ERM over any hypothesis class on the denoising objective — is automatically score stable under a manifold hypothesis: $\epsilon_{\mathrm{stab}}^2 \lesssim C\,(C\,C_{\mathrm{sm}} N^{-2}+\mathbb{E}[\hat{\ell}_{\mathrm{sm}}(\hat s)])^c$ for any $c<1$, with the constant depending on the manifold dimension $d_*$, not the ambient dimension, and degrading as the early-stopping noise scale $\epsilon\to0$. The same framework identifies coarse sampler discretisation and the intrinsic gradient noise of SGD as additional implicit regularisers, so the paper concludes that diffusion models generalise because of how they are trained and sampled, not because of the model class they use.

Load-bearing premise

The load-bearing premise is that every pair of score functions in the hypothesis class obeys a uniform bound scaling with noise, $\|s(\cdot,t)-s'(\cdot,t)\|_\infty \le D_H/\sigma_t^2$ for every $t$ (Assumption 5); that is far stronger than the local smoothness real neural score networks are known to satisfy, and if a trained network family violates it, the automatic-regularisation bound for ERM — the paper's headline result — no longer applies.

Editorial extensions

If this is right

Generalisation bounds for diffusion models can be stated almost without assumptions on the architecture: the bounds are controlled by the algorithm's sensitivity to data, so they apply to any score-function class meeting the structural conditions.
Early stopping is itself a regulariser: the stability constant grows like $\epsilon^{-d_*/4}$ as the smallest noise scale $\epsilon\to0$, so stopping the forward process sooner tightens the generalisation gap at the price of matching the target only at scale $\epsilon$.
Sampler coarseness buys generalisation: increasing the discretisation step shrinks the stability contribution to the KL bound while inflating the discretisation error, and Corollary 8 optimises the trade-off into rates of order $B_\kappa^{1/(2(d_*+1))}$ for the expected KL divergence.
SGD noise is a stabiliser: with weight decay, clipping, and decaying step sizes, the stability bound scales like $K^{\bar\eta\upsilon/(\bar\eta\upsilon+1)} N^{-1}$, and when the gradient noise is strong enough the bound becomes independent of the number of iterations $K$ entirely (Proposition 14).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One untested consequence is a memorisation diagnostic: an empirical leave-one-out score-stability constant computed on real training runs would predict when a diffusion model begins copying training samples — a quantity the paper defines but never measures.
The sampler-regularisation trade-off predicts a concrete curve the paper does not plot: for fixed data and early-stopping level, the generalisation gap should shrink monotonically as discretisation coarsens while sample quality worsens, locating the optimal step near the $\kappa$ of Corollary 8.
The analysis suggests that gradient noise and early stopping are substitutable regularisers: tuning mini-batch size and the number of noise resamples per gradient step, which set the estimator's variance, should shift the memorisation threshold in the same direction as changing the stopping time — a knob the paper leaves unexplored.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The short version: this is a real contribution. The score-stability framework is new, and Theorem 3 is a clean, correct-looking bridge from stability to generalization for both the denoising and score-matching losses. The applications to early stopping, coarse discretization, and SGD all follow a coherent proof strategy, and the appendix gives the derivations. This is exactly the kind of algorithm-dependent analysis the diffusion theory literature has been missing; it complements the uniform-convergence results of Oko et al., Azangulov et al., and Tang and Yang.

The main soft spot is Proposition 6, the flagship 'denoising regularisation' result. The stress-test note is right: Assumption 5 requires uniform L-infinity control over the whole hypothesis class, with the 1/sigma_t^2 scaling. Standard ReLU/SiLU score networks are unbounded on R^d, so the assumption is false for them. The Harnack-inequality step in the proof genuinely needs this bound to close the high-moment term, and without it the quadratic bootstrap for the stability constant does not go through. So 'irrespective of the hypothesis class' is an overstatement; the result holds for uniformly bounded classes, and enforcing boundedness is itself a nontrivial architectural imposition. This is an addressable problem—the framework survives, but the headline claim should be toned down.

Two smaller issues. First, Proposition 14 uses a Gaussian approximation of the gradient noise, so it describes a modified SGD algorithm, not exact SGD. That should be stated more prominently, though as an illustrative analysis it's fine. Second, the paper says these regularisation sources are 'unique to diffusion models' and 'overlooked.' Early stopping, discretization coarseness, and gradient noise have all been discussed in the empirical literature, and the paper doesn't engage with that. The novelty here is the formal stability treatment, not the discovery of the phenomena.

The citation pattern looks solid. The reliance on Potaptchik et al. for the discretization lemma is fine, but the authors should note it as an imported result in the main text.

Who is this for: theorists working on diffusion model generalization. It deserves a serious referee. My recommendation: engage, and ask for revision—make the assumptions honest, either by proving an extended version that relaxes Assumption 5 or by clearly framing the result as applying to bounded classes.

Referee Report

1 major / 0 minor

Summary. The paper develops an algorithm-dependent generalisation theory for diffusion models. It introduces a notion of score stability, which measures how a score-matching algorithm's output changes when one training example is replaced, and proves that a score-stable algorithm has bounded expected generalisation gap for both the denoising score-matching and the score-matching losses (Theorem 3). The main applications are: (i) empirical risk minimisation of the denoising score-matching objective under the manifold hypothesis, claimed to be automatically stable and hence implicitly regularised (Proposition 6); (ii) coarse-discretisation sampling, where the discretisation step is traded against generalisation (Proposition 7 and Corollary 8); and (iii) SGD with clipping and weight decay, with and without an added Gaussian approximation of gradient noise (Propositions 11 and 14). The proofs are given in substantial appendices and rely on strong convexity in function space, Wang's Harnack inequality, and Wasserstein contraction couplings.

Significance. The score-stability framework is a meaningful contribution: it adapts the classical algorithmic-stability toolbox to score matching in a way that yields clean generalisation bounds independent of model complexity, and it connects three distinct sources of regularisation (early stopping, discretisation, and optimisation noise) in one language. Theorem 3 is simple and useful, and the proof strategy is carefully developed. However, the headline claim that the denoising objective is automatically stable irrespective of the hypothesis class is not supported: the key ERM result, Proposition 6, depends on Assumption 5, which is a strong class-level uniform L-infinity bound with a specific time decay. Standard neural-network score models, which are typically unbounded on R^d and do not collapse to a common function as sigma_t grows, violate this assumption. The sampler result also inherits an unstated diameter condition. With these limitations stated honestly, the paper remains a valuable contribution as a conditional analysis of constrained hypothesis classes and as a template for algorithm-dependent bounds in diffusion models.

major comments (1)

[Lemma 22] Lemma 22 is imported from Potaptchik et al. and stated without proof. Since it is used directly in Proposition 7, a precise pointer to the corresponding theorem in the cited paper, or a short proof sketch in the appendix, would help the reader verify the dependence of the constants, particularly in relation to the diameter condition noted above.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the framework transfers a proved stability estimate into a generalisation gap, and the ERM stability proof uses a legitimate quadratic bootstrap rather than assuming its conclusion.

full rationale

Walking the derivation chain, Theorem 3 is a transference result rather than a circular restatement. Score stability is defined as a bound on expected coupled L2 score differences under a held-out initial point, and the proof bounds the generalisation gap by that same expectation via the triangle inequality and Jensen; this is direct but not definitionally identical to the gap. The substantive work lies in bounding the stability constant. In Proposition 6, Lemma 19 makes the stability constant appear on both sides of an inequality, but this is a standard self-bounding argument: the proof defines epsilon as the actual stability constant, derives a quadratic inequality, and solves it, rather than assuming the desired bound. The most restrictive element is Assumption 5, which requires uniform L-infinity control of the whole score class with 1/sigma_t^2 scaling; this limits applicability to typical unbounded ReLU/SiLU networks, but that is a conditionality or correctness concern, not circularity. The paper also cites prior work with overlapping authors, notably Lemma 22 from Potaptchik et al. 2024 and the reflection-coupling method of Farghly and Rebeschini 2021, but those are used as external theorems or as reproduced proof techniques, not as the conclusion being derived. No claimed prediction reduces to its own input by construction.

Assumptions & free parameters 3 free parameters · 9 assumptions · 1 invented entities

The central claim rests on several domain assumptions (manifold hypothesis, uniform L∞ class control, noise covariance lower bound) that are typical in the literature but restrict applicability. No data fitting occurs; the hyperparameters ε, κ are user-chosen design variables. Score stability is a new analytic construct introduced by the paper. The external lemmas cited from overlapping authors (Potaptchik et al., Benton et al., Farghly and Rebeschini) carry independent proofs and are not circular.

free parameters (3)

early stopping time epsilon = not fitted (user-chosen)
Controls the denoising regularization; bounds in Prop 6 scale like epsilon^{-d*} and the 'regularization' claim requires epsilon > 0.
discretization step kappa = not fitted (user-chosen)
In Prop 7, kappa trades off stability (decreasing) against discretization error (increasing); the optimal kappa is defined implicitly in Corollary 8.
exponent c in Prop 6 = arbitrary in (0,1)
Appears in all ERM stability bounds; the result holds for any c<1 at the cost of larger constants, so it is a proof artifact rather than a fitted constant.

assumptions (9)

domain assumption νdata is supported on a smooth submanifold of dimension d* with reach τ_reach and density bounded below by c_ν > 0 (Section 4, Assumption 4).
Used for Lemma 18 and Lemma 21, which control the empirical measure on small balls and are essential for the ERM stability proof.
domain assumption For all s,s' in H, ||s(·,t)-s'(·,t)||∞ ≤ D_H/σ_t^2 on supp(τ) (Section 4, Assumption 5).
Uniform L∞ control of the whole hypothesis class is required to bound high-order moments in the Harnack-inequality step of Proposition 6.
domain assumption Score network s_θ is Lipschitz and smooth in θ with L(x,t), M(x,t) having bounded average squares (Section 6.1, Assumption 9).
Standard structural assumption for optimization stability; relaxations of global smoothness allow ReLU-type networks.
domain assumption For all θ, x, t: ℓ̂dsm(s_θ;{x},δ_t) ≤ B_ℓ^2/σ_t^4 (Section 6.1, Assumption 10).
Ensures the per-point loss is bounded, which controls gradient estimator deviations in the SGD stability proof.
domain assumption Gradient-noise covariance of clipped gradients is uniformly ≥ Σ with spectral gap λ_gap > 0 (Section 6.2, Assumption 12).
Needed for the contractive behavior in Proposition 14; can fail in overparameterized networks with flat directions.
domain assumption Lipschitz and smoothness in θ with respect to the seminorm ||·||Σ+, with fourth-moment bounds (Section 6.2, Assumption 13).
Adapts the Wasserstein contraction machinery to anisotropic, state-dependent noise in the SGD analysis.
standard math EM discretization error bound D(p_ϵ || A_em(s)) ≲ ℓ_sm(s;τ̂) + D(p_T||p_∞) + Δ_{κ,K} (Appendix D, Lemma 22 from Potaptchik et al. 2024).
External convergence result used to convert score-matching error into KL divergence between true and generated distributions.
standard math Harnack inequality: E[φ(X_t)|X_0=x] ≤ E[φ(X_t)^p|X_0=y]^{1/p} exp(...) for p>1 (Appendix C, Lemma 20 from Wang 1997).
Transfers L2(ˆpt) stability bounds to the L2(pt|0) metric required by the score stability definition.
standard math Wasserstein contraction results for discrete diffusions with anisotropic noise (Appendix F, based on Eberle 2016 and Majka et al. 2020).
Provides the exponential contraction used to make Proposition 14's stability bound independent of the number of iterations.

invented entities (1)

score stability constant ε_stab
purpose: Quantify sensitivity of score-matching algorithms to single-point dataset perturbations; used to bound generalization gaps.
Defined in Definition 2 and used throughout. It is a proof tool, not a falsifiable observable, but it is a new theoretical construct introduced by this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis." pith.science (2026). https://pith.science/paper/SFCUMWGV

@misc{pith2026250703756,
  author       = {Pith},
  title        = {Pith review of: Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SFCUMWGV}},
  note         = {Machine review of arXiv:2507.03756}
}

read the original abstract

The success of denoising diffusion models raises important questions regarding their generalisation behaviour, particularly in high-dimensional settings. Notably, it has been shown that when training and sampling are performed perfectly, these models memorise training data -- implying that some form of regularisation is essential for generalisation. Existing theoretical analyses primarily rely on algorithm-independent techniques such as uniform convergence, heavily utilising model structure to obtain generalisation bounds. In this work, we instead leverage the algorithmic aspects that promote generalisation in diffusion models, developing a general theory of algorithm-dependent generalisation for this setting. Borrowing from the framework of algorithmic stability, we introduce the notion of score stability, which quantifies the sensitivity of score-matching algorithms to dataset perturbations. We derive generalisation bounds in terms of score stability, and apply our framework to several fundamental learning settings, identifying sources of regularisation. In particular, we consider denoising score matching with early stopping (denoising regularisation), sampler-wide coarse discretisation (sampler regularisation) and optimising with SGD (optimisation regularisation). By grounding our analysis in algorithmic properties rather than model structure, we identify multiple sources of implicit regularisation unique to diffusion models that have so far been overlooked in the literature.

Figures

Figures reproduced from arXiv: 2507.03756 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

63 extracted references · 58 canonical work pages

[1]

E. Aamari. Convergence Rates for Geometric Inference. PhD thesis, Université Paris-Saclay, Sept. 2017

work page 2017
[2]

Aamari, J

E. Aamari, J. Kim, F. Chazal, B. Michel, A. Rinaldo, and L. Wasserman. Estimating the reach of a manifold. Electronic Journal of Statistics, 13 0 (1): 0 1359--1399, 2019

work page 2019
[3]

Attia and T

A. Attia and T. Koren. Uniform stability for first-order empirical risk minimization. In P.-L. Loh and M. Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pages 3313--3332. PMLR, 2022

work page 2022
[4]

Azangulov, G

I. Azangulov, G. Deligiannidis, and J. Rousseau. Convergence of diffusion models under the manifold hypothesis in high-dimensions. arXiv preprint arXiv:2409.18804, 2024

arXiv 2024
[5]

Bakry, I

D. Bakry, I. Gentil, and M. Ledoux. Analysis and Geometry of Markov Diffusion Operators. Springer International Publishing, 2014

work page 2014
[6]

P. L. Bartlett, A. Montanari, and A. Rakhlin. Deep learning: a statistical viewpoint. Acta Numerica, 30: 0 87–201, 2021

work page 2021
[7]

Benton, V

J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Nearly d -linear convergence bounds for diffusion models via stochastic localization. In International Conference on Learning Representations, 2024

work page 2024
[8]

Beyler and F

E. Beyler and F. Bach. Optimal denoising in score-based generative models: The role of data regularity. arXiv [cs.LG], Mar. 2025

work page 2025

Show all 63 references

[9]

C. M. Bishop. Training with noise is equivalent to tikhonov regularization. Neural Computation, 7 0 (1): 0 108--116, 1995

1995
[10]

Bousquet and A

O. Bousquet and A. Elisseeff. Stability and generalization. The Journal of Machine Learning Research, 2: 0 499--526, 2002

2002
[11]

Charles and D

Z. Charles and D. Papailiopoulos. Stability and generalization of learning algorithms that converge to global optima. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, p...

2018
[12]

S. Chen, S. Chewi, J. Li, Y. Li, A. Salim, and A. R. Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In International Conference on Learning Representations, 2023

2023
[13]

S. Chen, V. Kontonis, and K. Shah. Learning general G aussian mixtures with efficient score matching. arXiv preprint arXiv:2404.18893, 2024

2024 arXiv
[14]

Clerico, T

E. Clerico, T. Farghly, G. Deligiannidis, B. Guedj, and A. Doucet. Generalisation under gradient descent via deterministic PAC -bayes. arXiv [stat.ML], Sept. 2022

2022
[15]

De Bortoli

V. De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Transactions on Machine Learning Research, 2022

2022
[16]

De Bortoli, J

V. De Bortoli, J. Thornton, J. Heng, and A. Doucet. Diffusion S chr \"o dinger bridge with applications to score-based generative modeling. In Advances in Neural Information Processing Systems, 2021

2021
[17]

Devroye and T

L. Devroye and T. Wagner. Distribution-free performance bounds for potential function rules. IEEE Transactions on Information Theory, 25 0 (5): 0 601--604, Sept. 1979

1979
[18]

Dupuis, M

B. Dupuis, M. Haddouche, G. Deligiannidis, and U. Simsekli. Understanding the generalization error of markov algorithms through poissonization. arXiv [stat.ML], Feb. 2025 a

2025
[19]

Dupuis, D

B. Dupuis, D. Shariatian, M. Haddouche, A. Durmus, and U. Simsekli. Algorithm- and data-dependent generalization bounds for score-based generative models. arXiv [stat.ML], June 2025 b

2025
[20]

A. Eberle. Reflection couplings and contraction rates for diffusions. Probability Theory and Related Fields, 166 0 (3): 0 851--886, Dec. 2016

2016
[21]

Eberle and M

A. Eberle and M. B. Majka. Quantitative contraction rates for M arkov chains on general state spaces. Electronic Journal of Probability, 24: 0 1--36, Jan. 2019

2019
[22]

Esser, S

P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M \"u ller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024

2024
[23]

Farghly and P

T. Farghly and P. Rebeschini. Time-independent generalization bounds for SGLD in non-convex settings. In Advances in Neural Information Processing Systems, 2021

2021
[24]

Gatmiry, J

K. Gatmiry, J. Kelner, and H. Lee. Learning mixtures of gaussians using diffusion models. arXiv [cs.LG], Apr. 2024

2024
[25]

Hardt, B

M. Hardt, B. Recht, and Y. Singer. Train faster, generalize better: Stability of stochastic gradient descent. In International Conference on Machine Learning, pages 1225--1234, 2016

2016
[26]

U. G. Haussmann and E. Pardoux. Time reversal of diffusions. The Annals of Probability, 14 0 (4): 0 1188--1205, 1986

1986
[27]

J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, 2020

2020
[28]

Huang, Y

Z. Huang, Y. Wei, and Y. Chen. Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality. arXiv [cs.LG], Oct. 2024

2024
[29]

Hyv \"a rinen

A. Hyv \"a rinen. Estimation of Non-Normalized statistical models by score matching. Journal of Machine Learning Research, 6 0 (24): 0 695--709, 2005

2005
[30]

Karras, M

T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. In S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh , editor, Advances in Neural Information Processing Systems, 2022

2022
[31]

Karras, M

T. Karras, M. Aittala, J. Lehtinen, J. Hellsten, T. Aila, and S. Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24174--24184, June 2024

2024
[32]

Kearns and D

M. Kearns and D. Ron. Algorithmic stability and sanity-check bounds for leave-one-out cross-validation. Neural Computation, 11 0 (6): 0 1427--1453, 1999

1999
[33]

H. Lee, J. Lu, and Y. Tan. Convergence for score-based generative modeling with polynomial complexity. Advances in Neural Information Processing Systems, 35: 0 22870--22882, 2022

2022
[34]

P. Li, Z. Li, H. Zhang, and J. Bian. On the generalization properties of diffusion models. In Advances in Neural Information Processing Systems, 2023

2023
[35]

Loaiza-Ganem, B

G. Loaiza-Ganem, B. L. Ross, R. Hosseinzadeh, A. L. Caterini, and J. C. Cresswell. Deep generative models through the lens of the manifold hypothesis: A survey and new connections. arXiv [cs.LG], Apr. 2024

2024
[36]

M. B. Majka, A. Mijatovic, and L. Szpruch. Nonasymptotic bounds for sampling algorithms without log-concavity. Ann. Appl. Probab., 30 0 (4): 0 1534--1581, Aug. 2020

2020
[37]

W. Mou, L. Wang, X. Zhai, and K. Zheng. Generalization bounds of sgld for non-convex learning: Two theoretical viewpoints. In Bubeck, Sébastien and Perchet, Vianney and Rigollet, Philippe , editor, Proceedings of the 31st Conference On Learning Theory, volume 75 of Proceedings...

2018
[38]

Nesterov

Y. Nesterov. Lectures on convex optimization. Springer optimization and its applications. Springer International Publishing, Cham, Switzerland, 2 edition, Dec. 2018

2018
[39]

G. Neu, G. K. Dziugaite, M. Haghifam, and D. M. Roy. Information-theoretic generalization bounds for stochastic gradient descent. In Proceedings of Thirty Fourth Conference on Learning Theory, volume 134 of Proceedings of Machine Learning Research, pages 3526--3545. PMLR, 2021

2021
[40]

K. Oko, S. Akiyama, and T. Suzuki. Diffusion models are minimax optimal distribution estimators. In International Conference on Machine Learning, 2023

2023
[41]

Pensia, V

A. Pensia, V. Jog, and P.-L. Loh. Generalization error bounds for noisy, iterative algorithms. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 546--550, June 2018

2018
[42]

Pidstrigach

J. Pidstrigach. Score-based generative models detect manifolds. In Advances in Neural Information Processing Systems, 2022

2022
[43]

Potaptchik, I

P. Potaptchik, I. Azangulov, and G. Deligiannidis. Linear convergence of diffusion models under the manifold hypothesis. arXiv preprint arXiv:2410.09046, 2024

2024 arXiv
[44]

Ramesh, P

A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

2022 arXiv
[45]

Rombach, A

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) . IEEE, June 2022

2022
[46]

Saharia, W

C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, J. Ho, D. J. Fleet, and M. Norouzi. Photorealistic Text-to-Image diffusion models with deep language understanding. In Advances in Neural Information ...

2022
[47]

K. Shah, S. Chen, and A. Klivans. Learning mixtures of G aussians using the DDPM objective. In Advances in Neural Information Processing Systems, 2023

2023
[48]

Sietsma and R

J. Sietsma and R. J. F. Dow. Creating artificial neural networks that generalize. Neural Networks, 4 0 (1): 0 67--79, 1991

1991
[49]

Simsekli, O

U. Simsekli, O. Sener, G. Deligiannidis, and M. A. Erdogdu. Hausdorff dimension, heavy tails, and generalization in neural networks. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 51...

2020
[50]

Sohl-Dickstein, E

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, 2015

2015
[51]

Song and D

Y. Song and D. P. Kingma. How to train your energy-based models. arXiv preprint arXiv:2101. 03288, 2021

2021
[52]

Y. Song, C. Durkan, I. Murray, and S. Ermon. Maximum likelihood training of Score-Based diffusion models. In Advances in Neural Information Processing Systems, 2021 a

2021
[53]

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-Based generative modeling through stochastic differential equations. In 9International Conference on Learning Representations, 2021 b

2021
[54]

Srivastava, G

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15 0 (56): 0 1929--1958, 2014

1929
[55]

Tang and Y

R. Tang and Y. Yang. Adaptivity of diffusion models to manifold structures. In International Conference on Artificial Intelligence and Statistics, 2024

2024
[56]

S. Vary, D. Martínez-Rubio, and P. Rebeschini. Black-box uniform stability for non-euclidean empirical risk minimization. arXiv [cs.LG], Dec. 2024

2024
[57]

M. J. Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, 2019

2019
[58]

F.-Y. Wang. Logarithmic sobolev inequalities on noncompact riemannian manifolds. Probability Theory and Related Fields, 109 0 (3): 0 417--424, Nov. 1997

1997
[59]

P. Wang, H. Zhang, Z. Zhang, S. Chen, Y. Ma, and Q. Qu. Diffusion models learn low-dimensional distributions via subspace clustering. arXiv [cs.LG], Sept. 2024

2024
[60]

J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, R. J. Ragotte, L. F. Milles, et al. De novo design of protein structure and function with RFdiffusion . Nature, 620 0 (7976): 0 1089--1100, 2023

2023
[61]

Williams, A

C. Williams, A. Campbell, A. Doucet, and S. Syed. Score-optimal diffusion schedules. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

2024
[62]

H. Yang. A mathematical framework for learning probability distributions. arXiv preprint arXiv:2212.11481, 2022

2022 arXiv
[63]

Zhang, S

C. Zhang, S. Bengio, M. Hardt, G. Brain, B. Recht, and O. Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64 0 (3): 0 107--115, 2021

2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

E. Aamari. Convergence Rates for Geometric Inference. PhD thesis, Université Paris-Saclay, Sept. 2017

work page 2017

[2] [2]

Aamari, J

E. Aamari, J. Kim, F. Chazal, B. Michel, A. Rinaldo, and L. Wasserman. Estimating the reach of a manifold. Electronic Journal of Statistics, 13 0 (1): 0 1359--1399, 2019

work page 2019

[3] [3]

Attia and T

A. Attia and T. Koren. Uniform stability for first-order empirical risk minimization. In P.-L. Loh and M. Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory, volume 178 of Proceedings of Machine Learning Research, pages 3313--3332. PMLR, 2022

work page 2022

[4] [4]

Azangulov, G

I. Azangulov, G. Deligiannidis, and J. Rousseau. Convergence of diffusion models under the manifold hypothesis in high-dimensions. arXiv preprint arXiv:2409.18804, 2024

arXiv 2024

[5] [5]

Bakry, I

D. Bakry, I. Gentil, and M. Ledoux. Analysis and Geometry of Markov Diffusion Operators. Springer International Publishing, 2014

work page 2014

[6] [6]

P. L. Bartlett, A. Montanari, and A. Rakhlin. Deep learning: a statistical viewpoint. Acta Numerica, 30: 0 87–201, 2021

work page 2021

[7] [7]

Benton, V

J. Benton, V. De Bortoli, A. Doucet, and G. Deligiannidis. Nearly d -linear convergence bounds for diffusion models via stochastic localization. In International Conference on Learning Representations, 2024

work page 2024

[8] [8]

Beyler and F

E. Beyler and F. Bach. Optimal denoising in score-based generative models: The role of data regularity. arXiv [cs.LG], Mar. 2025

work page 2025