Transformers converge globally to the optimal DDPM denoiser for multi-token GMMs via self-attention mean denoising, with explicit token and iteration requirements.
hub
& Mézard, M.Why Diffusion Models Don’t Memorize: The Role of Implicit Dynamical Regularization in TrainingarXiv:2505.17638 [cs]
16 Pith papers cite this work, alongside 2 external citations. Polarity classification is still indexing.
16
Pith papers citing it
2
external citations · external index
hub tools
citation-role summary
background 1
method 1
citation-polarity summary
years
2026 16representative citing papers
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
Discrete diffusion models learn data support before frequencies because the exact reverse process decomposes edits into a dominant validity scale and a finer probability coefficient.
Diffusion models show grokking on modular addition by composing periodic operand representations in simple data regimes or by separating arithmetic computation from visual denoising across timesteps in varied regimes.
Defines diffusion processes on implicit data manifolds via proximity-graph approximations to the infinitesimal generator and carré-du-champ operator, proves convergence in law to the continuous manifold process, and provides an Euler-Maruyama integrator validated on synthetic and MNIST manifolds.
AutoSpeed learns annotation-free, stage-adaptive robot motion speeds by optimizing policies toward the minimum-cost DCT-retimed multi-speed demonstration target.
Year-long rollouts of nine AI weather models reveal three instability regimes whose occurrence depends on how models treat small spatio-temporal scales, with stable models functioning as denoisers.
Authors link memorization to internal instability in diffusion models via latent norms, propose step-wise detection and mitigation achieving AUC >0.999 and 0% memorization rate on Stable Diffusion 1.4.
SiLD is a score-matching framework that learns both manifold projection and intrinsic density from a single objective, with proven sample complexity depending only on intrinsic dimension.
DiffICL breaks the quality-privacy tradeoff in small-data tabular synthesis by using in-context learning on pretrained structural priors to generate data that is both higher quality and less memorizing of training samples.
Using the mosaic controlled dataset framework, experiments show scene complexity dominates over concept imbalance in diffusion model failures for multi-object generation, with counting especially hard in low-data regimes and compositional generalization collapsing under held-out combinations.
Denoising diffusion learns pair-wise input statistics at linear sample complexity and fourth-order cumulants only at cubic complexity, unless latent variables are correlated.
Diffusion models overfit denoising loss at intermediate noise but generalize in inference as model error smooths the flow field and sampling paths avoid memorized noisy training data.
Diffusion models recover known ENSO variability structure from synthetic LIM data when given enough samples, but require pre-training on CMIP6 plus fine-tuning to match observations with the ~700 samples available in ERSSTv5.
In the oracle continuous-time setting, stochastic interpolation models recover training samples exactly, with deviations controlled by discretization and estimation errors, leading to theoretical definitions of overfitting and underfitting.
diffGHOST is a conditional diffusion model that segments learned latent space to identify and mitigate memorization of critical trajectory samples, aiming to deliver privacy guarantees alongside data utility.
citing papers explorer
-
Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMs
Transformers converge globally to the optimal DDPM denoiser for multi-token GMMs via self-attention mean denoising, with explicit token and iteration requirements.
-
Training-Free Generative Sampling via Moment-Matched Score Smoothing
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
-
Support Before Frequency in Discrete Diffusion
Discrete diffusion models learn data support before frequencies because the exact reverse process decomposes edits into a dominant validity scale and a finer probability coefficient.
-
Grokking of Diffusion Models: Case Study on Modular Addition
Diffusion models show grokking on modular addition by composing periodic operand representations in simple data regimes or by separating arithmetic computation from visual denoising across timesteps in varied regimes.
-
Diffusion Processes on Implicit Manifolds
Defines diffusion processes on implicit data manifolds via proximity-graph approximations to the infinitesimal generator and carré-du-champ operator, proves convergence in law to the continuous manifold process, and provides an Euler-Maruyama integrator validated on synthetic and MNIST manifolds.
-
AutoSpeed: Annotation-Free Stage-Adaptive Motion Speed Learning for Robot Manipulation
AutoSpeed learns annotation-free, stage-adaptive robot motion speeds by optimizing policies toward the minimum-cost DCT-retimed multi-speed demonstration target.
-
Can AI Weather Models Predict Beyond Two Weeks? A Quantitative Benchmark and Analysis of Long Rollouts
Year-long rollouts of nine AI weather models reveal three instability regimes whose occurrence depends on how models treat small spatio-temporal scales, with stable models functioning as denoisers.
-
Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations
Authors link memorization to internal instability in diffusion models via latent norms, propose step-wise detection and mitigation achieving AUC >0.999 and 0% memorization rate on Stable Diffusion 1.4.
-
Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine
SiLD is a score-matching framework that learns both manifold projection and intrinsic density from a single objective, with proven sample complexity depending only on intrinsic dimension.
-
Breaking the Quality-Privacy Tradeoff in Tabular Data Generation via In-Context Learning
DiffICL breaks the quality-privacy tradeoff in small-data tabular synthesis by using in-context learning on pretrained structural priors to generate data that is both higher quality and less memorizing of training samples.
-
When Do Diffusion Models learn to Generate Multiple Objects?
Using the mosaic controlled dataset framework, experiments show scene complexity dominates over concept imbalance in diffusion model failures for multi-object generation, with counting especially hard in low-data regimes and compositional generalization collapsing under held-out combinations.
-
A theory of learning data statistics in diffusion models, from easy to hard
Denoising diffusion learns pair-wise input statistics at linear sample complexity and fourth-order cumulants only at cubic complexity, unless latent variables are correlated.
-
Diffusion Models Memorize in Training -- and Generalize in Inference
Diffusion models overfit denoising loss at intermediate noise but generalize in inference as model error smooths the flow field and sampling paths avoid memorized noisy training data.
-
Learning Climate Variability from Scarce Data with Diffusion Models: A Test Case for ENSO
Diffusion models recover known ENSO variability structure from synthetic LIM data when given enough samples, but require pre-training on CMIP6 plus fine-tuning to match observations with the ~700 samples available in ERSSTv5.
-
A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models
In the oracle continuous-time setting, stochastic interpolation models recover training samples exactly, with deviations controlled by discretization and estimation errors, leading to theoretical definitions of overfitting and underfitting.
-
diffGHOST: Diffusion based Generative Hedged Oblivious Synthetic Trajectories
diffGHOST is a conditional diffusion model that segments learned latent space to identify and mitigate memorization of critical trajectory samples, aiming to deliver privacy guarantees alongside data utility.