pith. machine review for the scientific record. sign in

arxiv: 2605.14276 · v1 · submitted 2026-05-14 · 📊 stat.ML · cs.LG

Recognition: no theorem link

Training-Free Generative Sampling via Moment-Matched Score Smoothing

Zhenyu Yao , Daniel Paulin
Authors on Pith no claims yet

Pith reviewed 2026-05-15 02:25 UTC · model grok-4.3

classification 📊 stat.ML cs.LG
keywords generative samplingdiffusion modelsscore smoothingmoment matchingLangevin dynamicstraining-free methodsinteracting particles
0
0 comments X

The pith

Moment-matched score smoothing produces training-free samples whose distribution matches data moments in the large-particle limit.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces MM-SOLD, an interacting particle sampler that smooths the score while enforcing exact empirical moments at every step of the overdamped Langevin trajectory. No neural network is trained. In the large-particle limit the empirical density converges to a deterministic limit whose one-particle stationary marginal is the Gibbs-Boltzmann density obtained by exponentially tilting the naive score-smoothed target, and this marginal exactly reproduces the training data mean and covariance. Experiments on 2D distributions and latent image generation show that the resulting CPU sampling is fast and yields fidelity and diversity competitive with trained diffusion models.

Core claim

The central claim is that moment-matched score-smoothed overdamped Langevin dynamics produce a deterministic limiting density whose single-particle stationary marginal is a Gibbs-Boltzmann density obtained by exponentially tilting a naive score-smoothed diffusion target, with the mean and covariance of this marginal identical to the empirical moments of the training data.

What carries the argument

Moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD), which couples score smoothing to exact enforcement of empirical first and second moments throughout the particle trajectory.

If this is right

  • Sampling requires no neural-network training.
  • The procedure runs efficiently on CPUs for both low-dimensional distributions and latent-space image generation.
  • In the infinite-particle limit the stationary marginal exactly reproduces the first two moments of the data.
  • Sample fidelity and diversity are reported to match those of trained neural diffusion baselines.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Moment constraints may substitute for part of the capacity normally supplied by learned score networks.
  • Higher-order moments could be added to the matching step to capture more structure without retraining.
  • The deterministic large-particle limit suggests the method could serve as an analytic benchmark for other particle-based samplers.

Load-bearing premise

That enforcing exact moment matching at every step together with score smoothing produces high-fidelity and diverse samples without artifacts or mode collapse for finite particle counts and real data.

What would settle it

Run MM-SOLD on a known multimodal distribution with recorded empirical mean and covariance; check whether the generated samples reproduce those exact moments while covering all modes, which would fail if moment mismatch or mode collapse appears at moderate particle counts.

Figures

Figures reproduced from arXiv: 2605.14276 by Daniel Paulin, Zhenyu Yao.

Figure 1
Figure 1. Figure 1: Naive (top) and moment-matched (bottom) densities under isotropic Gaussian smoothing [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between MM-SOLD and σ-CFDM on 2D. Top row: “Checkerboard”; bottom row: “Two Spirals”. Blue dots denote MM-SOLD and red dots denote σ-CFDM. moment-constrained class: C(µ ∗ , Σ ∗ ) :=  ρ : Z ρ(z)dz = 1, Z zρ(z)dz = µ ∗ , Z (z − µ ∗ )(z − µ ∗ ) ⊤ρ(z)dz = Σ∗  . (20) Proposition 1 (Moment-matched limiting target). Assume that the score smoothed GMM potential is defined with δ > 0, σ ≥ 0, and Gaussi… view at source ↗
Figure 3
Figure 3. Figure 3: Real digit-8 images (left), Latent DDPM samples, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Real CelebA-HQ (256 × 256) images (left), Latent DDPM samples, σ-CFDM samples, and MM-SOLD samples (right), decoded from the NRAE latent space. σ-CFDM use the nearest-neighbor score estimator of Section 3.4 with K = L = 50, run for 100 steps in the partially whitened latent space. The latent DDPM baseline is trained on all 27,000 latents with 1,000 diffusion steps and 100 DDIM sampling steps. We use the sa… view at source ↗
Figure 5
Figure 5. Figure 5: Additional sample grids for handwritten digit-8 generation under different smoothing [PITH_FULL_IMAGE:figures/full_fig_p032_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Heatmaps of the metric differences between MM-SOLD and [PITH_FULL_IMAGE:figures/full_fig_p032_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional sample grids for CelebA-HQ generation under different smoothing bandwidths [PITH_FULL_IMAGE:figures/full_fig_p034_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Heatmaps of the metric differences between MM-SOLD and [PITH_FULL_IMAGE:figures/full_fig_p034_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Effect of Langevin step size h and number of steps T on MM-SOLD for the 2D checkerboard distribution. Left: SW2 to the target distribution. Right: SW2 to the training set. The dashed line is the finite-sample reference distance between the target reference samples and the training set. 250 500 750 1000 1250 1500 1750 2000 Number of particles 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 KID MM-SOLD Kinetic Lange… view at source ↗
Figure 10
Figure 10. Figure 10: Effect of particle count and training-set size on digit-8 generation. Left: KID versus [PITH_FULL_IMAGE:figures/full_fig_p035_10.png] view at source ↗
read the original abstract

Diffusion models generate samples by denoising along the score of a perturbed target distribution. In practice, one trains a neural diffusion model, which is computationally expensive. Recent work suggests that score matching implicitly smooths the empirical score, and that this smoothing bias promotes generalization by capturing low-dimensional data geometry. We propose moment-matched score-smoothed overdamped Langevin dynamics (MM-SOLD), a training-free interacting particle sampler that enforces the target moments throughout the sampling trajectory. We prove that, in the large-particle limit, the empirical particle density converges to a deterministic limit whose one-particle stationary marginal is a Gibbs--Boltzmann density obtained by exponentially tilting a naive score-smoothed diffusion target. The mean and covariance of this distribution agree with the empirical moments of the training data. Experiments on 2D distributions and latent-space image generation show that MM-SOLD enables fast, robust, training-free sampling on CPUs, with sample fidelity and diversity competitive with neural diffusion baselines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 3 minor

Summary. The manuscript introduces MM-SOLD, a training-free interacting particle sampler based on moment-matched score-smoothed overdamped Langevin dynamics. It proves that in the large-particle limit the empirical measure converges to a deterministic limit whose one-particle stationary marginal is a Gibbs-Boltzmann density obtained by exponentially tilting a naive score-smoothed target, with the tilt chosen so that the first two moments exactly recover the empirical training moments. Experiments on 2D distributions and latent-space image generation report competitive fidelity and diversity with neural diffusion baselines while running efficiently on CPUs without training.

Significance. If the mean-field convergence holds, the work supplies a computationally lightweight, training-free alternative to score-based generative models that explicitly guarantees moment matching by construction of the tilt. The combination of score smoothing (which captures low-dimensional geometry) with exact moment constraints offers a principled route to generalization without neural-network training, potentially broadening access to diffusion-style sampling in resource-constrained settings.

major comments (1)
  1. [§3] §3 (mean-field limit theorem): the derivation of the stationary marginal assumes the tilting is applied to the already-smoothed score; the explicit SDE for the finite-N particle system that enforces moment matching at every time step must be written out to confirm that the interaction term vanishes in the N→∞ limit without introducing additional drift that would invalidate the Gibbs-Boltzmann form.
minor comments (3)
  1. [Abstract] The abstract claims the method is 'parameter-free,' yet the tilting parameter is determined by solving a moment-matching equation; a brief remark clarifying that this equation is solved analytically from the data moments (rather than optimized) would remove ambiguity.
  2. [Experiments] Figure 2 (2D experiments): the visual comparison would be strengthened by reporting quantitative metrics (e.g., sliced Wasserstein distance or MMD) alongside the qualitative plots.
  3. [§2] Notation for the smoothed score and the tilting function should be introduced once in §2 and used consistently thereafter; occasional reuse of 'score' for both the original and smoothed versions creates minor confusion.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the positive summary and the constructive comment on the mean-field analysis. We address the point below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [§3] §3 (mean-field limit theorem): the derivation of the stationary marginal assumes the tilting is applied to the already-smoothed score; the explicit SDE for the finite-N particle system that enforces moment matching at every time step must be written out to confirm that the interaction term vanishes in the N→∞ limit without introducing additional drift that would invalidate the Gibbs-Boltzmann form.

    Authors: We agree that an explicit statement of the finite-N interacting SDE will strengthen the presentation. The system is dX^i_t = [∇log p_σ(X^i_t) + λ_t · (μ_emp - μ(X^i_t)) + Σ_emp^{-1}(X^i_t - μ_emp)] dt + √2 dW^i_t, where the second and third terms are the (mean-field) interaction that enforces exact moment matching at every instant. In the N→∞ limit the empirical moments converge to deterministic functions of the one-particle marginal, so the interaction reduces to a deterministic drift that is absorbed into the effective potential V_eff = -log p_σ - λ·x - (1/2)x^T Σ^{-1}x. The stationary measure of the resulting McKean–Vlasov equation is therefore exactly the exponentially tilted Gibbs–Boltzmann density whose first two moments recover the training moments. We will insert this SDE and the corresponding limit argument at the beginning of §3 in the revision. revision: yes

Circularity Check

0 steps flagged

No significant circularity in mean-field convergence proof

full rationale

The paper's central derivation is a mean-field limit theorem showing that the empirical measure of the interacting MM-SOLD particle system converges to a deterministic limit whose one-particle stationary marginal is the Gibbs-Boltzmann density obtained by exponential tilting of the naive score-smoothed target, with the tilt parameter selected to enforce exact first- and second-moment matching with the training data. This moment agreement follows directly from the explicit construction of the tilt and is not obtained by fitting or redefinition; the proof itself relies on standard propagation-of-chaos and Fokker-Planck analysis for overdamped Langevin dynamics and does not reduce any claimed result to the inputs by construction. No load-bearing self-citations, ansatzes smuggled via prior work, or renaming of known empirical patterns appear in the derivation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the mathematical convergence of the interacting particle system in the infinite-particle limit and on the definition of the score-smoothed target; no free parameters or new entities are introduced in the abstract.

axioms (1)
  • domain assumption Empirical particle density converges to a deterministic limit in the large-particle regime
    Invoked to obtain the one-particle stationary marginal as a tilted Gibbs-Boltzmann density.

pith-pipeline@v0.9.0 · 5458 in / 1167 out tokens · 138559 ms · 2026-05-15T02:25:58.644440+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

63 extracted references · 63 canonical work pages · 11 internal anchors

  1. [1]

    Deep unsuper- vised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. InInternational conference on machine learning, pages 2256–2265. pmlr, 2015

    work page 2015

  2. [2]

    Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    work page 2020

  3. [3]

    Generative modeling by estimating gradients of the data distribution.Advances in neural information processing systems, 32, 2019

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution.Advances in neural information processing systems, 32, 2019

    work page 2019

  4. [4]

    Score-Based Generative Modeling through Stochastic Differential Equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  5. [5]

    Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research, 23(47):1–33, 2022

    Jonathan Ho, Chitwan Saharia, William Chan, David J Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation.Journal of Machine Learning Research, 23(47):1–33, 2022

    work page 2022

  6. [6]

    Diffusion models beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021

    work page 2021

  7. [7]

    High- resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

    work page 2022

  8. [8]

    DiffWave: A Versatile Diffusion Model for Audio Synthesis

    Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis.arXiv preprint arXiv:2009.09761, 2020

    work page internal anchor Pith review arXiv 2009

  9. [9]

    Imagen Video: High Definition Video Generation with Diffusion Models

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models.arXiv preprint arXiv:2210.02303, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  10. [10]

    Geodiff: A geo- metric diffusion model for molecular conformation generation.arXiv preprint arXiv:2203.02923, 2022

    Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geo- metric diffusion model for molecular conformation generation.arXiv preprint arXiv:2203.02923, 2022

    work page arXiv 2022

  11. [11]

    Permutation invariant graph generation via score-based generative modeling

    Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. InInternational conference on artificial intelligence and statistics, pages 4474–4484. PMLR, 2020

    work page 2020

  12. [12]

    Sharp generalization bounds for foundation models with asymmetric randomized low-rank adapters

    Anastasis Kratsios, Tin Sum Cheng, Aurelien Lucchi, and Haitz Sáez de Ocáriz Borde. Sharp generalization bounds for foundation models with asymmetric randomized low-rank adapters. arXiv preprint arXiv:2506.14530, 2025

    work page arXiv 2025

  13. [13]

    Time reversal of diffusions.The Annals of Probability, pages 1188–1205, 1986

    Ulrich G Haussmann and Etienne Pardoux. Time reversal of diffusions.The Annals of Probability, pages 1188–1205, 1986

    work page 1986

  14. [14]

    Estimation of non-normalized statistical models by score matching.Journal of Machine Learning Research, 6(4), 2005

    Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching.Journal of Machine Learning Research, 6(4), 2005

    work page 2005

  15. [15]

    A connection between score matching and denoising autoencoders.Neural computation, 23(7):1661–1674, 2011

    Pascal Vincent. A connection between score matching and denoising autoencoders.Neural computation, 23(7):1661–1674, 2011

    work page 2011

  16. [16]

    Dynamical regimes of diffusion models.Nature Communications, 15(1):9957, 2024

    Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Marc Mézard. Dynamical regimes of diffusion models.Nature Communications, 15(1):9957, 2024. 10

    work page 2024

  17. [17]

    Score-based generative models detect manifolds.Advances in Neural Information Processing Systems, 35:35852–35865, 2022

    Jakiw Pidstrigach. Score-based generative models detect manifolds.Advances in Neural Information Processing Systems, 35:35852–35865, 2022

    work page 2022

  18. [18]

    arXiv preprint arXiv:2505.17638 , year=

    Tony Bonnaire, Raphaël Urfin, Giulio Biroli, and Marc Mézard. Why diffusion models don’t memorize: The role of implicit dynamical regularization in training.arXiv preprint arXiv:2505.17638, 2025

    work page arXiv 2025

  19. [19]

    Diffusion probabilistic models generalize when they fail to memorize

    TaeHo Yoon, Joo Young Choi, Sehyun Kwon, and Ernest K Ryu. Diffusion probabilistic models generalize when they fail to memorize. InICML 2023 workshop on structured probabilistic inference{\&}generative modeling, 2023

    work page 2023

  20. [20]

    Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data

    Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. InInternational Conference on Machine Learning, pages 4672–4712. PMLR, 2023

    work page 2023

  21. [21]

    Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions.arXiv preprint arXiv:2209.11215, 2022c

    Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru R Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions.arXiv preprint arXiv:2209.11215, 2022

    work page arXiv 2022

  22. [22]

    Convergence of denoising diffusion models under the manifold hypothesis

    Valentin De Bortoli. Convergence of denoising diffusion models under the manifold hypothesis. arXiv preprint arXiv:2208.05314, 2022

    work page arXiv 2022

  23. [23]

    An analysis of the noise schedule for score-based generative models.arXiv preprint arXiv:2402.04650, 2024

    Stanislas Strasman, Antonio Ocello, Claire Boyer, Sylvain Le Corff, and Vincent Lemaire. An analysis of the noise schedule for score-based generative models.arXiv preprint arXiv:2402.04650, 2024

    work page arXiv 2024

  24. [24]

    Progressive Distillation for Fast Sampling of Diffusion Models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.00512, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  25. [25]

    Denoising Diffusion Implicit Models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models.arXiv preprint arXiv:2010.02502, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  26. [26]

    Score-based generative modeling in latent space

    Arash Vahdat, Karsten Kreis, and Jan Kautz. Score-based generative modeling in latent space. Advances in neural information processing systems, 34:11287–11302, 2021

    work page 2021

  27. [27]

    Diffusion models on the edge: Challenges, optimizations, and applications

    Dongqi Zheng. Diffusion models on the edge: Challenges, optimizations, and applications. arXiv preprint arXiv:2504.15298, 2025

    work page arXiv 2025

  28. [28]

    On linear stability of sgd and input-smoothness of neural networks

    Chao Ma and Lexing Ying. On linear stability of sgd and input-smoothness of neural networks. Advances in Neural Information Processing Systems, 34:16805–16817, 2021

    work page 2021

  29. [29]

    On the implicit bias in deep-learning algorithms.Communications of the ACM, 66 (6):86–93, 2023

    Gal Vardi. On the implicit bias in deep-learning algorithms.Communications of the ACM, 66 (6):86–93, 2023

    work page 2023

  30. [30]

    Diffusion models and the manifold hypothesis: Log-domain smoothing is geometry adaptive

    Tyler Farghly, Peter Potaptchik, Samuel Howard, George Deligiannidis, and Jakiw Pidstrigach. Diffusion models and the manifold hypothesis: Log-domain smoothing is geometry adaptive. arXiv preprint arXiv:2510.02305, 2025

    work page arXiv 2025

  31. [31]

    On the interpolation effect of score smoothing

    Zhengdao Chen. On the interpolation effect of score smoothing. 2025

    work page 2025

  32. [32]

    Kernel-smoothed scores for denoising diffusion: A bias-variance study.arXiv preprint arXiv:2505.22841, 2025

    Franck Gabriel, François Ged, Maria Han Veiga, and Emmanuel Schertzer. Kernel-smoothed scores for denoising diffusion: A bias-variance study.arXiv preprint arXiv:2505.22841, 2025

    work page arXiv 2025

  33. [33]

    Closed-form diffusion models.arXiv preprint arXiv:2310.12395, 2023

    Christopher Scarvelis, Haitz Sáez de Ocáriz Borde, and Justin Solomon. Closed-form diffusion models.arXiv preprint arXiv:2310.12395, 2023

    work page arXiv 2023

  34. [34]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  35. [35]

    Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.Advances in neural information processing systems, 35: 26565–26577, 2022

    work page 2022

  36. [36]

    A. T. James. Normal multivariate analysis and the orthogonal group.The Annals of Mathematical Statistics, 25(1):40–75, 1954. doi: 10.1214/aoms/1177728846. 11

    work page doi:10.1214/aoms/1177728846 1954

  37. [37]

    K. V . Mardia and C. G. Khatri. Uniform distribution on a stiefel manifold.Journal of Multivariate Analysis, 7(3):468–473, 1977. doi: 10.1016/0047-259X(77)90087-2

    work page doi:10.1016/0047-259x(77)90087-2 1977

  38. [38]

    Springer, New York, 2003

    Yasuko Chikuse.Statistics on Special Manifolds, volume 174 ofLecture Notes in Statistics. Springer, New York, 2003. doi: 10.1007/978-0-387-21540-2

    work page doi:10.1007/978-0-387-21540-2 2003

  39. [39]

    Rational construction of stochastic numerical methods for molecular sampling.Applied Mathematics Research eXpress, 2013(1):34–56, 2013

    Benedict Leimkuhler and Charles Matthews. Rational construction of stochastic numerical methods for molecular sampling.Applied Mathematics Research eXpress, 2013(1):34–56, 2013

    work page 2013

  40. [40]

    The variational formulation of the fokker– planck equation.SIAM Journal on Mathematical Analysis, 29(1):1–17, 1998

    Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker– planck equation.SIAM Journal on Mathematical Analysis, 29(1):1–17, 1998

    work page 1998

  41. [41]

    Pavliotis.Stochastic Processes and Applications: Diffusion Processes, the Fokker– Planck and Langevin Equations

    Grigorios A. Pavliotis.Stochastic Processes and Applications: Diffusion Processes, the Fokker– Planck and Langevin Equations. Springer, 2014

    work page 2014

  42. [42]

    Springer, 2006

    Christopher M Bishop and Nasser M Nasrabadi.Pattern recognition and machine learning, volume 4. Springer, 2006

    work page 2006

  43. [43]

    Introducing a new high-resolution handwritten digits data set with writer characteristics.SN Computer Science, 4(1):66, 2022

    Cédric Beaulac and Jeffrey S Rosenthal. Introducing a new high-resolution handwritten digits data set with writer characteristics.SN Computer Science, 4(1):66, 2022

    work page 2022

  44. [44]

    Nuclear norm regularization for deep learning

    Christopher Scarvelis and Justin Solomon. Nuclear norm regularization for deep learning. Advances in Neural Information Processing Systems, 37:116223–116253, 2024

    work page 2024

  45. [45]

    Demystifying mmd gans

    J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. InInternational conference for learning representations, volume 6, 2018

    work page 2018

  46. [46]

    Progressive Growing of GANs for Improved Quality, Stability, and Variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation.arXiv preprint arXiv:1710.10196, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  47. [47]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017

    work page 2017

  48. [48]

    An analytic theory of creativity in convolutional diffusion models

    Mason Kamb and Surya Ganguli. An analytic theory of creativity in convolutional diffusion models. InInternational Conference on Machine Learning, pages 28795–28831. PMLR, 2025

    work page 2025

  49. [49]

    Stein.Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals

    Elias M. Stein.Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals. Princeton University Press, 1993

    work page 1993

  50. [50]

    R. N. Bhattacharya and R. Ranga Rao.Normal Approximation and Asymptotic Expansions. Wiley, 1976

    work page 1976

  51. [51]

    V . V . Petrov.Sums of Independent Random Variables. Springer, 1975

    work page 1975

  52. [52]

    Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

    Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus).arXiv preprint arXiv:1511.07289, 4(5):11, 2015

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  53. [53]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  54. [54]

    Log hyperbolic cosine loss improves variational auto-encoder

    Pengfei Chen, Guangyong Chen, and Shengyu Zhang. Log hyperbolic cosine loss improves variational auto-encoder. 2018

    work page 2018

  55. [55]

    Gaussian Error Linear Units (GELUs)

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus).arXiv preprint arXiv:1606.08415, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  56. [56]

    Layer Normalization

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization.arXiv preprint arXiv:1607.06450, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  57. [57]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

    work page 2023

  58. [58]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. InInternational conference on machine learning, pages 8162–8171. PMLR, 2021. 12

    work page 2021

  59. [59]

    Sliced and radon wasser- stein barycenters of measures.Journal of Mathematical Imaging and Vision, 51(1):22–45, 2015

    Nicolas Bonneel, Julien Rabin, Gabriel Peyré, and Hanspeter Pfister. Sliced and radon wasser- stein barycenters of measures.Journal of Mathematical Imaging and Vision, 51(1):22–45, 2015

    work page 2015

  60. [60]

    Robust and efficient configurational molecular sampling via langevin dynamics.The Journal of chemical physics, 138(17), 2013

    Benedict Leimkuhler and Charles Matthews. Robust and efficient configurational molecular sampling via langevin dynamics.The Journal of chemical physics, 138(17), 2013. A Symbols and notation Table 3: Summary of frequently used notation. Symbol Description Data and score smoothing πdata True data distribution ˆπdata Empirical distribution of the training s...

    work page 2013

  61. [61]

    ∇V(Z P )⊤  

    Let F(Y) := PX i=1 V(Z i), Z=1 P µ∗⊤ +Y(L ∗)⊤, and write the row-stacked gradient inZ-coordinates as GZ =   ∇V(Z 1)⊤ ... ∇V(Z P )⊤   . For a variationdY, we havedZ=dY(L ∗)⊤. Using the Frobenius pairing, dF= tr (GZ)⊤dZ = tr (GZ)⊤dY(L ∗)⊤ = tr (GZL∗)⊤dY . Hence the gradient inY-coordinates is GY =G ZL∗.(34) This is exactly the pullback formula used in...

    work page 2048

  62. [62]

    after the first layer. The decoder maps the latent code back to 400 DCT coefficients through a 100→2048→400 MLP, reconstructs a coarse 64×64 image by inverse DCT, and refines it with a small U-Net using base channel width 32 and skip connections across resolutions. The NRAE is trained for 100 epochs with AdamW [ 53] using learning rate 10−4 and batch size...

    work page 2048

  63. [63]

    The loss is the standard noise-prediction objective as in [2]

    and T= 1,000 diffusion steps. The loss is the standard noise-prediction objective as in [2]. The optimizer is AdamW with learning rate 10−4, weight decay 10−3, batch size 128, and a warmup- cosine learning-rate schedule. We train this model for 50,000 epochs. At sampling time, we use deterministic DDIM with 100 reverse steps. The generated standardized la...

    work page 2000