REVIEW 20 cited by
Generalization in diffusion models arises from geometry-adaptive harmonic representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Deep neural networks (DNNs) trained for image denoising are able to generate high-quality samples with score-based reverse diffusion algorithms. These impressive capabilities seem to imply an escape from the curse of dimensionality, but recent reports of memorization of the training set raise the question of whether these networks are learning the "true" continuous density of the data. Here, we show that two DNNs trained on non-overlapping subsets of a dataset learn nearly the same score function, and thus the same density, when the number of training images is large enough. In this regime of strong generalization, diffusion-generated images are distinct from the training set, and are of high visual quality, suggesting that the inductive biases of the DNNs are well-aligned with the data density. We analyze the learned denoising functions and show that the inductive biases give rise to a shrinkage operation in a basis adapted to the underlying image. Examination of these bases reveals oscillating harmonic structures along contours and in homogeneous regions. We demonstrate that trained denoisers are inductively biased towards these geometry-adaptive harmonic bases since they arise not only when the network is trained on photographic images, but also when it is trained on image classes supported on low-dimensional manifolds for which the harmonic basis is suboptimal. Finally, we show that when trained on regular image classes for which the optimal basis is known to be geometry-adaptive and harmonic, the denoising performance of the networks is near-optimal.
Forward citations
Cited by 20 Pith papers
-
An exact information theory of generalization phase transitions in Bayesian diffusion models
Bayesian diffusion models memorize training data when mutual information between restricted observations and training data exceeds log dataset size, and generalize otherwise.
-
An analytic theory of creativity in convolutional diffusion models
Convolutional diffusion models generate novel images by assembling locally consistent patch mosaics of training patches, and this mechanism is captured by an analytic score machine that predicts individual model outputs.
-
Toward a mechanistic understanding of inference in visual cortex and diffusion models
A sparse-coding circuit with learned pairwise interactions, trained by score matching, reproduces diffusion-model-like contour completion and claims to expose the mechanism behind it.
-
Generalization and Memorization in Rectified Flow
Rectified Flow models peak in membership-inference vulnerability at the flow midpoint under uniform training; U-shaped timestep sampling suppresses memorization without harming FID.
-
When and how can inexact generative models still sample from the data manifold?
Inexact generative models stay on the data manifold because infinitesimal learning errors perturb the density only along the manifold, when top Lyapunov vectors align with the support boundary.
-
ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features
ConceptAttention shows that linear projections in the output space of DiT attention layers yield sharper concept-localizing saliency maps than cross-attention maps, reaching state-of-the-art zero-shot segmentation.
-
Trajectory inference via Acceleration Matching
Acceleration Matching is a simulation-free, flow-matching-style algorithm for multi-marginal trajectory inference that regresses onto an explicit kinetic Brownian bridge acceleration field in phase space.
-
Fitting Image Diffusion Models on Video Datasets
A shared-noise temporal consistency regularizer for image diffusion training accelerates convergence and lowers FID on the HandCo video dataset.
-
Diffusion models under low-noise regime
Diffusion models trained on disjoint data converge at high noise but diverge near the data manifold, and they fail to denoise very small perturbations accurately.
-
Navigating the Latent Space Dynamics of Neural Models
Autoencoders implicitly define a latent vector field whose attractors encode the model's memorized and generalized knowledge, enabling data-free probing and out-of-distribution detection.
-
Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models
In overparameterized diffusion models, generalization happens first and memorization starts later, with the memorization time growing linearly with dataset size.
-
The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications
Learned diffusion score fields behave like Gaussian score fields at high noise, enabling an analytical 'teleportation' that skips early sampling steps without hurting FID.
-
Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation
Training on the best of K generated candidates improves image, video, and language generative models, with the reported gains growing with scale and enabling single-pass end-to-end generation.
-
On the Separability of Information in Diffusion Models
Diffusion models devote most of their information budget to class-agnostic texture, and the small class-relevant slice is what classifier-free guidance amplifies.
-
Generation Properties of Stochastic Interpolation under Finite Training Set
For finite training sets, stochastic interpolation models reproduce training samples under deterministic generation and output noise-perturbed copies under stochastic generation.
-
Deterministic diffusion models for Lagrangian turbulence: robustness and encoding of extreme events
Using DDIM, diffusion models generate accurate Lagrangian turbulence statistics with as few as 25 steps, and extreme acceleration events coincide with localized bumps in the initial latent noise.
-
GMem: A Modular Approach for Ultra-Efficient Generative Models
GMem conditions diffusion models on a fixed bank of DINOv2 features and reports much lower FID at far fewer epochs than SiT and REPA baselines on ImageNet.
-
A solvable generative model with a linear, one-step denoiser
The paper derives a closed-form KL divergence for a one-step linear diffusion model on Gaussian data, reports a sample-size threshold at n=d, and gives a heuristic argument that more diffusion steps improve quality.
-
On the Complexity-Faithfulness Trade-off of Gradient-Based Explanations
The paper introduces EF and ΔEF as spectral metrics, but ΔEF is derived from EF, making the complexity-faithfulness trade-off partly tautological.
-
Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Control
Frequency-filtered noise in the diffusion forward process steers what the denoiser learns, yielding modest FID gains on some datasets and partial recovery after known-band corruption.
Discussion (0). Continue with ORCID to comment.