REVIEW 16 cited by
An analytic theory of creativity in convolutional diffusion models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
An analytic theory of creativity in convolutional diffusion models
read the original abstract
We obtain an analytic, interpretable and predictive theory of creativity in convolutional diffusion models. Indeed, score-matching diffusion models can generate highly original images that lie far from their training data. However, optimal score-matching theory suggests that these models should only be able to produce memorized training examples. To reconcile this theory-experiment gap, we identify two simple inductive biases, locality and equivariance, that: (1) induce a form of combinatorial creativity by preventing optimal score-matching; (2) result in fully analytic, completely mechanistically interpretable, local score (LS) and equivariant local score (ELS) machines that, (3) after calibrating a single time-dependent hyperparameter can quantitatively predict the outputs of trained convolution only diffusion models (like ResNets and UNets) with high accuracy (median $r^2$ of $0.95, 0.94, 0.94, 0.96$ for our top model on CIFAR10, FashionMNIST, MNIST, and CelebA). Our model reveals a locally consistent patch mosaic mechanism of creativity, in which diffusion models create exponentially many novel images by mixing and matching different local training set patches at different scales and image locations. Our theory also partially predicts the outputs of pre-trained self-attention enabled UNets (median $r^2 \sim 0.77$ on CIFAR10), revealing an intriguing role for attention in carving out semantic coherence from local patch mosaics.
Forward citations
Cited by 16 Pith papers
-
When Do Diffusion Models learn to Generate Multiple Objects?
Diffusion models' multi-object generation is limited primarily by scene complexity and held-out combinations rather than imbalance, with counting difficult in low data and compositional generalization collapsing as mo...
-
Training-Free Imitation Learning with Closed-Form Diffusion Policies
Closed-Form Diffusion Policies enable training-free imitation learning by using closed-form scores derived from demonstration data, achieving competitive benchmark performance with millisecond inference and composable...
-
Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models
Symmetry breaking and nonlocality phase transitions occur nearly simultaneously during diffusion model generation in modern transformers.
-
A Minimal Model of Representation Collapse: Frustration, Stop-Gradient, and Dynamics
A minimal embedding model shows representation collapse arises from frustrated samples through slow dynamics and is prevented by stop-gradient.
-
Emergence of Nonequilibrium Latent Cycles in Unsupervised Generative Modeling
A nonequilibrium latent-variable Markov model spontaneously develops cycles during likelihood training that enhance generative performance over equilibrium approaches.
-
Score-based Membership Inference on Diffusion Models
Presents SimA, a score-based single-query membership inference attack for diffusion models and LDMs that uses denoiser output norm to reveal training set proximity and outperforms multi-query baselines on eight datasets.
-
An Analytical Theory of Spectral Bias in the Learning Dynamics of Diffusion Models
Analytic solution of full-batch gradient flow for linear and convolutional denoisers in diffusion models yields a universal inverse-variance spectral law for learning times of eigenmodes.
-
Unsupervised Causal Abstractions Discovery
Low-rank graphs induce latents that form causal abstractions, with identifiability results and a practical objective enabling unsupervised learning of high-level SCMs from low-level measurements.
-
Mechanisms of Misgeneralization in Physical Sequence Modeling
Generative sequence models for physical tasks exhibit physical misgeneralization where local prediction errors propagate through physical measurements to distort aggregate distributions over quantities like distance o...
-
Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Field
Flow-Direct constructs a reusable non-parametric guidance field from the log-density ratio of base and target distributions using all accumulated reward samples for feedback-efficient guidance in flow models.
-
When Do Diffusion Models learn to Generate Multiple Objects?
Using the mosaic controlled dataset framework, experiments show scene complexity dominates over concept imbalance in diffusion model failures for multi-object generation, with counting especially hard in low-data regi...
-
Language Diffusion Models are Associative Memories Capable of Retrieving Unseen Data
Uniform-based discrete diffusion models behave as associative memories that retrieve unseen data, with a dataset-size-driven memorization-to-generalization transition detectable via conditional entropy of token predictions.
-
Generalization in LLM Problem Solving: The Case of the Shortest Path
LLMs show strong spatial generalization to unseen maps in shortest-path tasks but fail length scaling due to recursive instability, with data coverage setting hard limits.
-
Diffusion Models Memorize in Training -- and Generalize in Inference
Diffusion models overfit denoising loss at intermediate noise but generalize in inference as model error smooths the flow field and sampling paths avoid memorized noisy training data.
-
Local Diffusion Models and Phases of Data Distributions
The paper introduces a phase framework for data distributions connected by local denoisers and demonstrates that reverse diffusion consists of trivial and data phases separated by a transition where local score functi...
-
Principles and Practice of Deep Representation Learning: or a Mathematical Theory of Memory
The book presents principles from optimization and information theory to explain deep network architectures and enable new interpretable models.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.