In overparameterized diffusion models, generalization happens first and memorization starts later, with the memorization time growing linearly with dataset size.
Generative diffusion in very large dimensions
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Generative models based on diffusion have become the state of the art in the last few years, notably for image generation. Here, we analyse them in the high-dimensional limit, where data are formed by a very large number of variables. We use methods from statistical physics and focus on two well-controlled high-dimensional cases: a Gaussian model and the Curie-Weiss model of ferromagnetism. In the latter case, we highlight the mechanism of symmetry breaking in the inverse diffusion, and point out that, in order to reconstruct the relative asymmetry of the two low-temperature states, and thus to obtain the correct probability weights, one needs a database with a number of points much larger than the dimension of each data point. We characterize the scaling laws in the number of data and in the number of dimensions for an efficient generation.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models
In overparameterized diffusion models, generalization happens first and memorization starts later, with the memorization time growing linearly with dataset size.