Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Generalization through variance: how noise shapes inductive biases in diffusion models

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Noise in the training target, not memorized scores, is what lets diffusion models generalize, a path-integral theory argues.

desk verdict A genuinely new mechanism for diffusion-model generalization via the covariance of the DSM proxy score, with clean asymptotic results for linear and NTK models, but the general claim leans on an unverified second-order cumulant truncation. read the letter →

arxiv 2504.12532 v1 pith:LGCXGK3H submitted 2025-04-16 cs.LG cond-mat.dis-nncs.AI

John J. Vastola This is my paper · ORCID
classification cs.LGcond-mat.dis-nncs.AI
keywords diffusionmodelsdenoisingscorematchinggeneralizationinductivebiasestimatorvariancepathintegralV-kernelneuraltangentkernel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that diffusion models generalize, filling gaps in the training set rather than merely reproducing it, because the denoising score matching objective trains against a noisy 'proxy score' that only equals the true score in expectation. Averaging over training-realization randomness turns the probability-flow ODE into an effective stochastic differential equation whose noise has covariance given by a 'V-kernel', built from the covariance of the score estimator; with an unbiased estimator and finitely many training examples, generalization occurs if and only if this V-kernel is nonzero. The paper derives explicit V-kernels for a naive memorizing estimator, for linear feature models, and for lazy infinite-width networks, and shows the kernel concentrates in boundary regions between training examples, which is why learned distributions resemble the training data with gaps filled in. This matters because it converts a puzzling empirical fact into a mechanism that can be computed from the DSM objective, the forward process, the model's features, and the training-set geometry.

What carries the argument

The load-bearing object is the V-kernel, the covariance kernel of the learned score estimator's fluctuations, $V(x,t;x',t') = D_t \operatorname{Cov}_\theta[\hat{s}(x,t),\hat{s}(x',t')] D_{t'}$, defined for any score estimator and computed in closed form for three model classes. The paper's other central device is a time-reversed Martin-Siggia-Rose path integral representation of the probability-flow ODE, which turns the ensemble average of the learned distribution into a Gaussian average over score fluctuations; keeping only the mean and second cumulant yields the effective SDE of Proposition 3.1. The V-kernel carries the argument because it is the only place where finite-sample randomness of the DSM proxy score enters the typical dynamics, and its proxy-score covariance factor $C(x,t) = S_t^{-1} + \nabla^2 \log p(x|t)$ is what localizes generalization to boundary regions.

What would settle it

Measure the skewness (third cumulant) of the DSM proxy-score target at a boundary point for finite $P$, and compare a model trained on that target against the effective-SDE prediction: if the third cumulant is non-negligible, the distribution predicted from Eq. (8) will deviate from the observed one, falsifying the Gaussian truncation. A simpler version is to scan $F/P$ and $\epsilon$ in the one-dimensional three-point linear model of Fig. 2 and check whether the measured average learned distribution matches the semiclassical formula (18); systematic mismatch that grows with the third cumulant would show that the V-kernel alone does not carry the inductive bias.

Watch

Extended reading notes

Core claim

The central discovery is a mechanism, not just a phenomenon: the noisy target in denoising score matching has a state-dependent covariance $C(x,t)$, and this covariance, propagated through the diffusion tensor and the model's feature kernel, becomes the noise term of an effective reverse SDE that describes the typical distribution a diffusion model learns. The paper shows that for an unbiased score estimator with finitely many training samples, the only thing separating true probability-flow dynamics from the model's typical sampling dynamics is this V-kernel noise. Concretely, Proposition 5.1 and Proposition 5.2 give V-kernels of the form $\lim_{P\to\infty} (F/P) D_t \mathbb{E}_{z''}[\lambda_{t''}^2/\mathbb{E}_t[\lambda_t]^2\, k(z;z'') C(z'') k(z'';z')] D_{t'}$ for linear models and lazy neural networks, while the naive estimator gives the local form $V = \kappa D_t C(x,t) D_t \delta(t-t')$. Because $C$ is large in boundary regions between nearby training points and small near isolated points or far from data, the typical learned distribution interpolates dense regions of the training set, preserves data dimensionality, and leaves outliers memorized.

Load-bearing premise

The paper assumes that the randomness in the learned score is nearly Gaussian, so that only its mean and covariance matter; if the score estimator's fluctuations are significantly non-Gaussian, the V-kernel picture of generalization would be incomplete.

Editorial extensions

If this is right

  • Even a model that memorizes its training samples at every reverse step produces a nontrivial V-kernel and hence generalization, as Proposition 4.1 shows for the naive interleaved estimator.
  • Generalization disappears as $P\to\infty$ unless the number of features $F$ grows with $P$; the ratio $F/P$ controls the strength of the V-kernel, predicting a sharp memorization-to-generalization transition as capacity outstrips sample count.
  • Asymmetries in the forward process's noise matrix $G_t$ enter $V$ through the diffusion tensor, so the forward process is not just a sampler detail but an inductive-bias dial.
  • Feature-related spectral biases multiply the proxy-score covariance in the V-kernel, which explains why the same data distribution is generalized differently, for example square versus cross gap-filling, for Gaussian versus Fourier features and different data orientations.
  • Outliers, isolated points, and regions far from training data have near-zero proxy-score covariance, so generalization through variance is a boundary-smearing mechanism rather than an extrapolation-to-novelty mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the V-kernel is the mechanism, then the time weighting $\lambda_t$ of the DSM objective could be engineered to sculpt which gaps are filled, for instance by down-weighting times where $C$ is large in undesired regions to suppress unwanted mode blending.
  • Editorial inference: the theory yields a falsifiable quantitative prediction for simple linear models: the measured average learned distribution should match the effective-SDE prediction over a range of $F/P$ and $\epsilon$, with discrepancies appearing exactly when the score estimator's third cumulant becomes non-negligible.
  • Editorial inference: since $C$ is the Fisher information of the posterior over $x_0$, the same V-kernel formalism could be adapted to other training objectives such as sliced score matching, denoiser prediction, or noise prediction by swapping the target covariance, giving a unified treatment of generalization across training losses.
  • Editorial inference: the temporal autocorrelation structure of $V$ in the NTK case means the effective dynamics are non-Markovian in time, which may be a mechanism by which sampling spends extra time in boundary regions, an effect that could show up in measured trajectory statistics.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes that diffusion models generalize because the denoising score matching (DSM) objective trains against a noisy "proxy score" whose expectation is the true score but whose state-dependent covariance—the "V-kernel"—perturbs the reverse-time dynamics. Using a Martin-Siggia-Rose path integral, the paper derives an effective SDE for the ensemble-averaged learned distribution (Prop. 3.1), then computes the V-kernel for three tractable cases: an interleaved naive estimator that uses proxy-score samples at each Euler step (Prop. 4.1), expressive linear feature models (Prop. 5.1), and lazy infinite-width neural networks in the NTK regime (Prop. 5.2). The paper argues that this covariance-driven noise fills gaps in the training distribution, explains capacity-dependent memorization/generalization transitions, and interacts with feature-related inductive biases. The theory is illustrated with 1D and 2D simulations of linear models.

Significance. If the proposed mechanism is correct, it offers an original and potentially influential explanation of a genuinely puzzling phenomenon: even though the DSM optimum equals the true score, finite-sample diffusion models learn smoothed distributions that interpolate the training data. The paper's main strengths are the self-contained path-integral formalism (Appendices D-G), explicit V-kernel formulas showing dependence on the capacity ratio F/P and on feature kernels, and reproducible toy experiments (code is provided). The qualitative predictions—capacity-driven memorization-to-generalization transitions, boundary smearing, and feature-dependent gap filling—are falsifiable. The main caveat is that the central object is an annealed average of the learned distribution, and the key approximations are not quantitatively controlled; the paper is more convincing as a mechanism proposal than as a theorem.

major comments (3)
  1. [Section 3, Eq. (7), Prop. 3.1] The paper defines [q(x0|xT)] as the arithmetic mean E_theta[q(x0|xT;theta)] and calls it the "typical" learned distribution. Since q is a nonlinear functional of the score estimator, the annealed average E_theta[q] differs from the quenched/typical value exp(E_theta log q); the V-kernel noise in Prop. 3.1 describes the smoothed mixture of models, not necessarily the distribution of any single trained model. This distinction is load-bearing because the statement "generalization occurs iff V != 0" refers to this averaged object. The paper should either justify that the annealed average is the appropriate observable (e.g., by showing self-averaging or by restricting to a regime where annealed and quenched averages agree to leading order) or explicitly reframe the central claim as being about the average learned distribution rather than the typical one.
  2. [Section 3, Eq. (52), Appendix D.3, Prop. 4.1] The effective SDE in Eq. (8) is derived by dropping all cumulants above second order in the characteristic function of the score estimator. The paper flags this assumption but does not bound the neglected terms. For the naive estimator of Prop. 4.1, the per-time-step estimator is a discrete mixture over the M training examples and is not Gaussian. A direct estimate under the 1/sqrt(Delta t) scaling in Eq. (55) suggests that the third and higher cumulants of the integrated action are O(Delta t^{3/2}) and vanish in the continuum limit, so the approximation is plausibly valid; however, the paper does not present this or any equivalent smallness condition, and for finite Delta t or large kappa the higher cumulants need not be negligible. Since the headline claim rests on this truncation, the manuscript should provide the missing calculation or state the validity conditions explicitly.
  3. [Appendix F.4-F.5, Eq. (78), Prop. 5.1] The linear-model V-kernel is computed as the covariance over x0 draws conditional on fixed x,t samples, and the empirical moments mu-hat and Sigma-hat in the kernel Q are treated as deterministic. The general V-kernel in Sec. 3 averages over all training-sample realizations. Because Q in Eq. (74) depends on mu-hat and Sigma-hat, fluctuations of the x,t samples also contribute to the score-estimator covariance. The paper should show that those contributions are subleading relative to the F/P term (or equivalently redefine the ensemble in Prop. 5.1 so that it matches Sec. 3). Without this, Prop. 5.1 and Prop. 5.2 compute a slightly different object from the one used in the central argument.
minor comments (5)
  1. [Section 5, Eqs. (12)-(15), Appendix G] The symbol k is reused for the feature kernel in Prop. 5.1 and for the NTK feature kernel in Prop. 5.2, and Appendix G also uses K and k for the Gram matrix and an input-dependent vector. The footnote acknowledges the overload, but the notation is still hard to follow; consider renaming one of these kernels.
  2. [Eq. (17), Appendix G.3] The infinite-training-time V-kernel contains a prefactor kappa(Delta z) = F Delta z / P that depends on an unspecified discretization volume Delta z; the paper should define Delta z and state explicitly how kappa(Delta z) is held fixed in the F,P -> infinity limit.
  3. [Fig. 2 caption] The caption states "Average learned distribution (N = 100)" without defining N in the main text; please state that N is the number of models in the ensemble average.
  4. [Sec. 2 and Appendix A] The early text says that optimizing J0 or J1 "reproduces training examples"; this is only true in the limit epsilon -> 0, since for finite epsilon the PF-ODE reproduces p(x0|epsilon). The paper later acknowledges this, but the early phrasing could mislead.
  5. [Sec. 6, Eq. (18), Appendix K] The semiclassical approximation is described as a "small noise" approximation, but the actual small parameter is kappa (proportional to F/P); please make this explicit where Eq. (18) is introduced.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the V-kernel is derived from the DSM objective and estimator covariance, not assumed or fit.

full rationale

I found no circular step. The central object V(z;z') is defined as D_t Cov_theta[s-hat(z), s-hat(z')] D_t' (Sec. 3, Prop. 3.1) and then computed from first principles for the naive, linear, and NTK estimators (Props. 4.1, 5.1, 5.2). The claim that 'generalization occurs iff V != 0' is a direct consequence of the cumulant expansion truncated at second order; the paper explicitly states this approximation ('Assuming higher-order terms can be neglected'), so the claim is conditional on an assumption, not equivalent to its input by construction. The linear and NTK V-kernels are obtained by averaging the DSM-optimal estimator over sample realizations, with the F/P scaling arising from the central limit theorem rather than from fitting. Numerical figures compare the derived formula to independently simulated averages, so they are not fitted inputs renamed as predictions. The only self-citations (Wang & Vastola 2023, 2024) occur in Appendix K in an aside about when PF-ODE dynamics are exactly solvable and are not load-bearing for the main derivation. The second-order truncation and Gaussian assumption are an unverified approximation for finite P; that is a correctness or robustness risk, not circularity, and does not raise the circularity score.

Assumptions & free parameters 2 free parameters · 8 assumptions · 0 invented entities

The paper's contribution is a derivation, but it rests on several explicit modeling choices: Gaussian approximation of the estimator ensemble, PF-ODE sampling, delta-mixture data, linear features or NTK limits, and a small-noise semiclassical step. No free parameters are fit to data; the kappa and F/P quantities scale the predicted V-kernel but are inputs, not fitted values. No new physical entities are introduced.

free parameters (2)
  • kappa (variance prefactor in naive estimator, Prop. 4.1)
    Introduced by hand in Eq. 55 to control the strength of the noisy proxy-score correction; the predicted V-kernel is linear in it. No value is fit to data, so this is a modeling knob, not a fitted constant.
  • kappa = F/P (capacity ratio in linear and NTK limits, Props. 5.1-5.2)
    The asymptotic V-kernel is proportional to this ratio, which is taken to remain finite in the P to infinity limit. It is an input architecture and training-set ratio, not fitted from data. The 'provided the limit exists' clause hides the value.
assumptions (8)
  • ad hoc to paper The score estimator's characteristic function can be represented by a second-order cumulant expansion; estimator fluctuations are approximately Gaussian.
    Invoked in Sec. 3 before Prop. 3.1 and in Appendix D.3 to reduce the averaged path integral to an effective SDE with mean and V-kernel only. It is the main tractability assumption and is explicitly acknowledged.
  • domain assumption The reverse process is the probability flow ODE (PF-ODE) with a fixed discretization, rather than the full stochastic reverse SDE or alternative samplers.
    Introduced in Sec. 2 and used throughout. The theory does not cover stochastic samplers, higher-order solvers, or diffusion with auxiliary variables, as acknowledged in Sec. 7.
  • domain assumption Training data are modeled as a discrete delta mixture p_data(x0) = sum_m delta(x0 - mu_m) / M.
    Used in Sec. 2 and Appendix C to define boundary regions and compute explicit proxy score covariances. The theory does not address continuous or latent-space data distributions.
  • domain assumption The forward process has linear drift and additive Gaussian noise (VP-SDE or EDM form), with a diagonal diffusion tensor in the concrete examples.
    Sec. 2 Eq. 1 and Table 1. The covariance computations rely on Gaussian transition kernels.
  • domain assumption For linear models, features are independent, square-integrable, and expressive enough that the true score lies in the feature span; the feature covariance is invertible.
    Sec. 5.1 and Appendix F. Unbiasedness of the estimator and the V-kernel formula depend on these conditions.
  • domain assumption For neural networks, the infinite-width NTK and lazy regime applies, training is full-batch gradient descent, the NTK has finite rank F, and F and P go to infinity with kappa fixed.
    Sec. 5.2 and Appendix G. The closed-form learned score and the asymptotic V-kernel rely on these limits.
  • ad hoc to paper The small-noise semiclassical or saddlepoint approximation is valid for small kappa.
    Appendix K uses a heuristic semiclassical treatment to relate the V-kernel to the learned density. Functional-determinant manipulations are formal rather than rigorously controlled.
  • standard math Standard results used without proof: DSM optima equal the true score (Vincent 2011), the NTK gradient-flow solution, the MSR path integral representation, and the Mercer decomposition of kernels.
    Used in Secs. 2, 3, and 5 and in the appendices as background from the cited literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Generalization through variance: how noise shapes inductive biases in diffusion models." pith.science (2026). https://pith.science/paper/LGCXGK3H

@misc{pith2026250412532,
  author       = {Pith},
  title        = {Pith review of: Generalization through variance: how noise shapes inductive biases in diffusion models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LGCXGK3H}},
  note         = {Machine review of arXiv:2504.12532}
}
read the original abstract

How diffusion models generalize beyond their training set is not known, and is somewhat mysterious given two facts: the optimum of the denoising score matching (DSM) objective usually used to train diffusion models is the score function of the training distribution; and the networks usually used to learn the score function are expressive enough to learn this score to high accuracy. We claim that a certain feature of the DSM objective -- the fact that its target is not the training distribution's score, but a noisy quantity only equal to it in expectation -- strongly impacts whether and to what extent diffusion models generalize. In this paper, we develop a mathematical theory that partly explains this 'generalization through variance' phenomenon. Our theoretical analysis exploits a physics-inspired path integral approach to compute the distributions typically learned by a few paradigmatic under- and overparameterized diffusion models. We find that the distributions diffusion models effectively learn to sample from resemble their training distributions, but with 'gaps' filled in, and that this inductive bias is due to the covariance structure of the noisy target used during training. We also characterize how this inductive bias interacts with feature-related inductive biases.

Figures

Figures reproduced from arXiv: 2504.12532 by the authors.

Figure 1
Figure 1. Visualization of proxy score variance (tr [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Average learned distribution (N = 100) for a linear model with Gaussian features trained on different sample draws from a 1D data distribution {−1, 0, 1}. Red: average learned distribution; black: true distribution; gray: PF-ODE approximation of true distribution. Different values of the time cutoff ϵ and ratio F/P are shown. Note that there is more generalization as both become larger. See Appendix G for the full d… view at source ↗
Figure 3
Figure 3. Generalization of a 2D data distribu￾tion depends on features used and data orientation. Heatmaps of samples from N = 100 linear mod￾els are shown in different conditions, with train￾ing data (red dots) overlaid. Notice that which gaps are ‘filled in’, e.g., whether a square shape or cross shape is made, depends on both factors. Gap-filling inductive bias. Given that the V￾kernel is especially sensitive to the ‘gaps… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unified Audio Intelligence Without Regressing on Text Intelligence

    cs.CL 2026-07 conditional novelty 6.0 of 10

    A unified 30B MoE audio-text LLM achieves state-of-the-art audio understanding, generation, and speech tasks while preserving text reasoning comparable to its text-only backbone.

Reference graph

Works this paper leans on

61 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    On exact computation with an infinitely wide neural net

    Sanjeev Arora, Simon S Du, Wei Hu, Zhiyuan Li, Russ R Salakhutdinov, and Ruosong Wang. On exact computation with an infinitely wide neural net. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings...

    work page 2019

  3. [3]

    On the inductive bias of neural tangent kernels

    Alberto Bietti and Julien Mairal. On the inductive bias of neural tangent kernels. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/c4ef9c39b300931b69a36...

    work page 2019

  4. [4]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 22563--22575, June 2023

    work page 2023

  5. [5]

    Self-consistent dynamical field theory of kernel evolution in wide neural networks

    Blake Bordelon and Cengiz Pehlevan. Self-consistent dynamical field theory of kernel evolution in wide neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp.\ 32240--32256. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/p...

    work page 2022

  6. [6]

    Self-consistent dynamical field theory of kernel evolution in wide neural networks*

    Blake Bordelon and Cengiz Pehlevan. Self-consistent dynamical field theory of kernel evolution in wide neural networks*. Journal of Statistical Mechanics: Theory and Experiment, 2023 0 (11): 0 114009, nov 2023. doi:10.1088/1742-5468/ad01b0. URL https://dx.doi.org/10.1088/1742-5468/ad01b0

    doi:10.1088/1742-5468/ad01b0 2023

  7. [7]

    Spectrum dependent learning curves in kernel regression and wide neural networks

    Blake Bordelon, Abdulkadir Canatar, and Cengiz Pehlevan. Spectrum dependent learning curves in kernel regression and wide neural networks. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp.\ 1024--1034. PMLR, 13--18 Jul 2020. URL https:/...

    work page 2020

  8. [8]

    Convergence of denoising diffusion models under the manifold hypothesis

    Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=MhK5aXo3gB. Expert Certification

    work page 2022

Show all 61 references
  1. [9]

    Diffusion schr\"odinger bridge with applications to score-based generative modeling

    Valentin De Bortoli, James Thornton, Jeremy Heng, and Arnaud Doucet. Diffusion schr\"odinger bridge with applications to score-based generative modeling. In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, ...

    2021

  2. [10]

    Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks

    Abdulkadir Canatar, Blake Bordelon, and Cengiz Pehlevan. Spectral bias and task-model alignment explain generalization in kernel regression and infinitely wide neural networks. Nature Communications, 12 0 (1): 0 2914, May 2021. ISSN 2041-1723. doi:10.1038/s41467-021-23103-1. U...

    2021 doi

  3. [11]

    Extracting training data from diffusion models

    Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tram \`e r, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusion models. In 32nd USENIX Security Symposium (USENIX Security 23), pp.\ 5253--5270, Anaheim, CA,...

    2023

  4. [12]

    Denoising likelihood score matching for conditional score-based data generation

    Chen-Hao Chao, Wei-Fang Sun, Bo-Wun Cheng, Yi-Chen Lo, Chia-Che Chang, Yu-Lun Liu, Yu-Lin Chang, Chia-Ping Chen, and Chun-Yi Lee. Denoising likelihood score matching for conditional score-based data generation. In International Conference on Learning Representations, 2022. URL...

    2022

  5. [13]

    Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data

    Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proc...

    2023

  6. [14]

    Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran ...

    2018

  7. [15]

    The probability flow ode is provably fast

    Sitan Chen, Sinho Chewi, Holden Lee, Yuanzhi Li, Jianfeng Lu, and Adil Salim. The probability flow ode is provably fast. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 68552--685...

    2023

  8. [16]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations, 2023 c . URL https://openrevie...

    2023

  9. [17]

    On lazy training in differentiable programming

    L\' e na\" c Chizat, Edouard Oyallon, and Francis Bach. On lazy training in differentiable programming. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associa...

    2019

  10. [18]

    Group equivariant convolutional networks

    Taco Cohen and Max Welling. Group equivariant convolutional networks. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp.\ 2990--2999, New York...

    2016

  11. [19]

    Crisanti and H

    A. Crisanti and H. Sompolinsky. Path integral approach to random neural networks. Phys. Rev. E, 98: 0 062120, Dec 2018. doi:10.1103/PhysRevE.98.062120. URL https://link.aps.org/doi/10.1103/PhysRevE.98.062120

    2018 doi

  12. [20]

    Schoenberg, and Sandy Engelhardt

    Salman Ul Hassan Dar, Arman Ghanaat, Jannik Kahmann, Isabelle Ayx, Theano Papavassiliu, Stefan O. Schoenberg, and Sandy Engelhardt. Investigating data memorization in 3d latent diffusion models for medical image synthesis. In Deep Generative Models: Third MICCAI Workshop, DGM4...

    2023 doi

  13. [21]

    GENIE : Higher-order denoising diffusion solvers

    Tim Dockhorn, Arash Vahdat, and Karsten Kreis. GENIE : Higher-order denoising diffusion solvers. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022 a . URL https://openreview.net/forum?id=LKEYuYNOqx

    2022

  14. [22]

    Score-based generative modeling with critically-damped langevin diffusion

    Tim Dockhorn, Arash Vahdat, and Karsten Kreis. Score-based generative modeling with critically-damped langevin diffusion. In International Conference on Learning Representations, 2022 b . URL https://openreview.net/forum?id=CzceR82CYc

    2022

  15. [23]

    Disentangling feature and lazy training in deep neural networks

    Mario Geiger, Stefano Spigler, Arthur Jacot, and Matthieu Wyart. Disentangling feature and lazy training in deep neural networks. Journal of Statistical Mechanics: Theory and Experiment, 2020 0 (11): 0 113301, nov 2020. doi:10.1088/1742-5468/abc4de. URL https://dx.doi.org/10.1...

    2020 doi

  16. [24]

    Neural network-based score estimation in diffusion models: Optimization and generalization

    Yinbin Han, Meisam Razaviyayn, and Renyuan Xu. Neural network-based score estimation in diffusion models: Optimization and generalization. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=h8GeqOxtd4

    2024

  17. [25]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33: 0 6840--6851, 2020

    2020

  18. [26]

    Neural tangent kernel: Convergence and generalization in neural networks

    Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 3...

    2018

  19. [27]

    Generalization in diffusion models arises from geometry-adaptive harmonic representations

    Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and St \'e phane Mallat. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?...

    2024

  20. [28]

    An analytic theory of creativity in convolutional diffusion models

    Mason Kamb and Surya Ganguli . An analytic theory of creativity in convolutional diffusion models . arXiv e-prints, art. arXiv:2412.20292, December 2024. doi:10.48550/arXiv.2412.20292

    2024 doi arXiv

  21. [29]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https://openrevi...

    2022

  22. [30]

    Analyzing and improving the training dynamics of diffusion models, 2024

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models, 2024

    2024

  23. [31]

    Path integrals in quantum mechanics, statistics, polymer physics, and financial markets

    Hagen Kleinert. Path integrals in quantum mechanics, statistics, polymer physics, and financial markets. World Scientific Publishing Company, 2006

    2006

  24. [32]

    Pseudo numerical methods for diffusion models on manifolds

    Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. arXiv preprint arXiv:2202.09778, 2022

    2022 arXiv

  25. [33]

    Boomerang: Local sampling on image manifolds using diffusion models

    Lorenzo Luzi, Paul M Mayer, Josue Casco-Rodriguez, Ali Siahkoohi, and Richard Baraniuk. Boomerang: Local sampling on image manifolds using diffusion models. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=NYdThkjNW1

    2024

  26. [34]

    P. C. Martin, E. D. Siggia, and H. A. Rose. Statistical dynamics of classical systems. Phys. Rev. A, 8: 0 423--437, Jul 1973. doi:10.1103/PhysRevA.8.423. URL https://link.aps.org/doi/10.1103/PhysRevA.8.423

    1973 doi

  27. [35]

    Dynamical mean-field theory for stochastic gradient descent in gaussian mixture classification

    Francesca Mignacco, Florent Krzakala, Pierfrancesco Urbani, and Lenka Zdeborov\' a . Dynamical mean-field theory for stochastic gradient descent in gaussian mixture classification. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Inf...

    2020

  28. [36]

    Towards a mechanistic explanation of diffusion model generalization

    Matthew Niedoba, Berend Zwartsenberg, Kevin Murphy, and Frank Wood. Towards a mechanistic explanation of diffusion model generalization. arXiv e-prints, art. arXiv:2411.19339, February 2025. URL https://arxiv.org/abs/2411.19339

    2025 arXiv

  29. [37]

    On the spectral bias of neural networks

    Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville. On the spectral bias of neural networks. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machi...

  30. [38]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 10684--10695, 2022

    2022

  31. [39]

    A theory of neural tangent kernel alignment and its influence on training, 2022

    Haozhe Shan and Blake Bordelon. A theory of neural tangent kernel alignment and its influence on training, 2022. URL https://arxiv.org/abs/2105.14301

    2022 arXiv

  32. [40]

    On the generalization benefit of noise in stochastic gradient descent

    Samuel Smith, Erich Elsen, and Soham De. On the generalization benefit of noise in stochastic gradient descent. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research...

    2020

  33. [41]

    Smith and Quoc V

    Samuel L. Smith and Quoc V. Le. A bayesian perspective on generalization and stochastic gradient descent. In International Conference on Learning Representations, 2018. URL https://openreview.net/forum?id=BJij4yg0Z

    2018

  34. [42]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis Bach and David Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of ...

    2015

  35. [43]

    Diffusion art or digital forgery? investigating data replication in diffusion models

    Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Diffusion art or digital forgery? investigating data replication in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 6048--6058...

    2023

  36. [44]

    Understanding and mitigating copying in diffusion models

    Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Understanding and mitigating copying in diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023 b . URL https://openreview.net/forum?id=HtMXRGbUMt

    2023

  37. [45]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, ...

    2019

  38. [46]

    Sliced score matching: A scalable approach to density and score estimation

    Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Ryan P. Adams and Vibhav Gogate (eds.), Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, volume 115 of Proceedings of ...

    2020

  39. [47]

    Score-based generative modeling through stochastic differential equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=PxTIG12RRHS

    2021

  40. [48]

    Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L

    George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Villecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. ...

    2023

  41. [49]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 23 0 (7): 0 1661--1674, 2011. doi:10.1162/NECO_a_00142

    2011 doi

  42. [50]

    Kakade, and Boaz Barak

    Nikhil Vyas, Sham M. Kakade, and Boaz Barak. On provable copyright protection for generative models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29...

    2023

  43. [51]

    The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications

    Binxu Wang and John Vastola. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=I0uknSHM2j

    2024

  44. [52]

    Binxu Wang and John J. Vastola. Diffusion models generate images like painters: an analytical theory of outline first, details later. arXiv e-prints, art. arxiv:2303.02490, March 2023. URL https://arxiv.org/abs/2303.02490

    2023 arXiv

  45. [53]

    Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, and Nathan Srebro

    Blake Woodworth, Suriya Gunasekar, Jason D. Lee, Edward Moroshko, Pedro Savarese, Itay Golan, Daniel Soudry, and Nathan Srebro. Kernel and rich regimes in overparametrized models. In Jacob Abernethy and Shivani Agarwal (eds.), Proceedings of Thirty Third Conference on Learning...

    2020

  46. [54]

    Jaakkola

    Yilun Xu, Shangyuan Tong, and Tommi S. Jaakkola. Stable target field for reduced variance score estimation in diffusion models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=WmIwYTd0YTF

    2023

  47. [55]

    Diffusion models: A comprehensive survey of methods and applications

    Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Comput. Surv., 56 0 (4), nov 2023. ISSN 0360-0300. doi:10.1145/3626235. URL https://do...

    2023 doi

  48. [56]

    TaeHo Yoon, Joo Young Choi, Sehyun Kwon, and Ernest K. Ryu. Diffusion probabilistic models generalize when they fail to memorize. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling , 2023. URL https://openreview.net/forum?id=shciCbSk9h

    2023

  49. [57]

    Understanding deep learning requires rethinking generalization

    Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=Sy8gdB9xx

    2017

  50. [58]

    The emergence of reproducibility and consistency in diffusion models

    Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu. The emergence of reproducibility and consistency in diffusion models. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=HsliOqZkc0

    2024

  51. [59]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  52. [60]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  53. [61]

    " id="W5M0MpCehiHzreSzNTczkc9d

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.