Pith. sign in

REVIEW 6 major objections 6 minor 1 cited by

Memorization and Generalization in Generative Diffusion under the Manifold Hypothesis

T0 review · 6 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For data on a hidden D-dimensional manifold, diffusion models memorize at a time set by the latent dimension D rather than the ambient dimension N, and the best generalization point lies inside the memorization phase.

desk verdict New scaling laws for diffusion memorization on manifolds, built on an imported collapse criterion that deserves a derivation or a simulation before the phase diagram is taken as established. read the letter →

arxiv 2502.09578 v2 pith:LMD4KKLM submitted 2025-02-13 cond-mat.dis-nn

Beatrice Achilli , Luca Ambrogioni , Carlo Lucibello , Marc Mézard , Enrico Ventura This is my paper · ORCID
classification cond-mat.dis-nn MSC 82B4482C31
keywords diffusionmodelsmemorizationgeneralizationmanifoldhypothesisRandomEnergyModelcollapsetransitionempiricalscoreHidden
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Diffusion models generate samples by reversing a noising process, and when the score is estimated from a finite training set the reverse dynamics eventually collapses onto individual training points: this is memorization. This paper studies that collapse when the $P$ training points are not spread through all $N$ dimensions but lie on a $D$-dimensional manifold, with $P = e^{\alpha N}$ and $D = \alpha_D N$. It claims that the collapse time is set by the intrinsic dimension $D$, not the ambient dimension $N$: in the homogeneous Gaussian case $t_c = \frac{\sigma^2 N/D}{e^{2 \log P / D} - 1}$, so highly structured data avoid the usual curse of dimensionality. It further claims that collapse is exactly the condensation transition of an associated random energy model, and that the KL-optimal stopping time for sampling, $t_g$, satisfies $0 < t_g < t_c$ for every $\alpha$ and $\alpha_D$. A sympathetic reader should care because this gives a closed-form, testable prediction of when diffusion models memorize, and it locates the best generalization inside the memorization phase.

What carries the argument

The load-bearing object is the mapping of the empirical log-density, $\log p^{\mathrm{emp}}_t(x) = \log \sum_{\mu} e^{-\|x-\xi_\mu\|^2/(2t)}$, to a Random Energy Model (REM): the $P$ training points act as energy levels $\epsilon_\mu = -\|x-\xi_\mu\|^2/(2t)$, and the cumulant generating function $\zeta_t(\lambda)$ with its Legendre transform controls the free energy. The collapse criterion, $\alpha + \zeta_{t_c}(1) = -1/2$, compares the starting training point's exponential rate $-1/2$ with the combined rate of all other points and is solved to give $t_c$. A posterior-mean identity for typical trajectories supplies $\zeta'_{t_c}(1) = -1/2$, making collapse coincide with REM condensation; for nonlinear manifolds $\zeta_t$ is computed through a replica-symmetric saddle point, while the Jacobian spectrum of the score supplies a separate random-matrix criterion for generalization before collapse.

What would settle it

Simulate the reverse diffusion driven by the empirical score for Hidden Manifold Model data with known $N$, $D$, $P$ and $\sigma^2$, and record the first time at which typical trajectories entering a training point's basin stay trapped; compare with Eq. (19). If the measured trapping time does not scale as $(\sigma^2 N/D)/(e^{2\log P/D}-1)$, or if $\zeta'_{t_c}(1)$ computed from the same ensemble is not $-1/2$, so collapse and condensation occur at different times, the central claim fails. A second check is to measure the KL divergence between target and empirical distributions and test whether its minimum occurs at $t_g < t_c$ for all $\alpha$ and $\alpha_D$.

Watch

Extended reading notes

Core claim

The paper's central discovery is that memorization in empirical-score diffusion is a phase transition of an effective Random Energy Model whose energy levels are the squared distances from the current point to the training points. For Hidden Manifold Model data, the collapse time at which a typical trajectory is trapped in the basin of a training point is $t_c = \frac{\sigma^2 N/D}{e^{2 \log P / D} - 1}$. Because this time depends on $N$ and $P$ only through $D$ and $\log P$, memory capacity and memorization timing are controlled by the latent manifold dimension rather than the ambient dimension. The same REM calculation shows collapse occurs at the condensation threshold, established through a posterior-mean identity that gives $\zeta'_{t_c}(1) = -1/2$; above $t_c$ the empirical score tracks the true score, while below it trajectories are trapped. The KL divergence between the exact and empirical distributions has its minimum at $t_g < t_c$, so the best generalization is attained while the trajectory is already inside a training point's basin, and as $\alpha_D \to 0$ the empirical model converges to the exact one with $t_g$ shrinking faster than $t_c$.

Load-bearing premise

The load-bearing premise is the imported collapse criterion that a typical trajectory is trapped when the exponential rate of the starting training point, $-1/2$, equals the rate $\alpha + \zeta_{t_c}(1)$ of the sum over all other training points, a signal-to-noise comparison assumed rather than derived from the diffusion equation or a learned score.

Editorial extensions

If this is right

  • For structured data, the number of training points needed to generalize before collapse scales as $P_{\min} = \epsilon^{-D/2}$, exponential in the latent dimension rather than the ambient dimension.
  • Collapse is the same event as REM condensation, so the memory capacity of the empirical score is exponential in the latent dimension and is fixed by the collapse criterion.
  • An ordering $0 < t_g < t_c < t_o$ holds for all $\alpha$ and $\alpha_D$, and sampling at $t_g$ gives the smallest KL divergence to the true distribution even though it lies inside the memorization phase.
  • As $\alpha_D \to 0$, $t_g$ and $t_c$ both go to zero but $t_g$ goes faster, and at $t_c$ the empirical distribution converges to the exact one, so structure improves generalization.
  • The empirical and true scores coincide for $t > t_c$, so memorization only affects the final stage of denoising.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not train neural-network scores; an extension would test whether a learned score inherits the same phase boundary, with $t_c$ still governed by $D$ rather than $N$ up to corrections from score error.
  • Because the best sampling time lies in the condensed phase, early stopping in practice should be viewed as choosing the noise level at which each trajectory is close to the manifold but not yet pinned; on real image data one could check whether sample-quality metrics peak before collapse as the estimated intrinsic dimension decreases.
  • The formula suggests a practical data-scaling law: to avoid memorization, the number of training examples must grow exponentially in the intrinsic dimension, not the pixel dimension; subsampling a large image dataset and measuring collapse times for different estimated intrinsic dimensions would test this directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper studies memorization and generalization in empirical-score diffusion models when the training data lie on a D-dimensional manifold embedded in N dimensions, in the proportional limit D = α_D N and with P = e^{αN} training points. The authors analyze the reverse diffusion driven by the empirical score through a mapping to a Random Energy Model, deriving (i) an onset time t_o for the appearance of basins of attraction, (ii) a collapse time t_c at which typical reverse trajectories become trapped by training points, (iii) an equivalence between collapse and the REM condensation transition via a Nishimori condition, and (iv) a generalization time t_g minimizing a renormalized KL divergence, which is claimed to satisfy 0 < t_g < t_c for all α, α_D. The central quantitative result is the formula t_c = σ² N/D / (e^{2 log P/D} - 1) for homogeneous Gaussian manifold data, together with the conclusion that the effective dimensionality controlling memorization is the latent dimension D rather than the ambient N. The paper also gives a random-matrix criterion for generalization before collapse, leading to P_min ∼ ε^{-D/2}.

Significance. If the central claims hold, this is a timely and useful contribution to the statistical-physics theory of diffusion models. The paper provides explicit, parameter-free predictions — most notably that the collapse time depends on the latent dimension D rather than the ambient dimension N, and that the optimal stopping time lies inside the memorization phase. The Nishimori-based proof in Appendix C that the collapse criterion coincides with the REM condensation threshold is a clean and nontrivial formal step, and the replica computations for linear and nonlinear manifolds are carried out in considerable detail. The paper also explicitly identifies the divergent entropy in the KL divergence and works with the finite time-dependent part. The main weakness is that the load-bearing collapse criterion itself is imported as a heuristic from prior work and is not derived from the reverse SDE, nor is it checked by reverse-process simulations; several secondary claims (the KL expression in Eq. (34) and the universality of t_g < t_c) are also not fully supported as written.

major comments (6)
  1. [§5.1, Eq. (14)] The collapse criterion is imported from Ref. [7] and is a signal-to-noise comparison of log-densities evaluated on a forward bridge trajectory x_t = ξ_1 + ω√t, not a derivation from the reverse SDE (2) or from the score landscape of the empirical mixture. Because the reverse process is initialized from N(0, t_f I) and its typical trajectories are not conditioned on a training point, it does not follow without further argument that the reverse dynamics become trapped at t_c. This criterion feeds directly into Eq. (19), into the claimed D-dependence, into the collapse-condensation equivalence, and into the downstream t_g < t_c conclusion. I ask the authors to either derive the criterion from the reverse SDE or to validate it by direct simulation of the reverse process driven by the empirical score, including finite-N extrapolation and a discussion of non-typical trajectories. Appendix C proves consistency between the collapse criterion and the REM condensation threshold under the same typicality assumption, but it does not prove that reverse trajectories collapse at that time.
  2. [§5.1, Eq. (18) vs Appendix A, Eq. (46)] The main-text expression for ζ_t(λ) is inconsistent with the corresponding Appendix A expression. As printed, Eq. (18) lacks the α_D factors that appear inside the logarithm and in the denominator of the second term in Eq. (46). Using Eq. (18) in the collapse condition (14) gives t_c = σ²/(e^{2 log P/D} − 1), whereas Eq. (19) reports t_c = σ² N/D /(e^{2 log P/D} − 1). The Appendix A computation leading to Eq. (49) supports the claimed formula, but the main text must be corrected so that the displayed generating function and the displayed collapse time are consistent.
  3. [§6.1, Eq. (34)] The expression for the KL divergence between the true and empirical distributions in the condensed phase contains an undefined term H_t and an unproved step: the paper states that the annealed approximation holds in the uncondensed phase and then jumps to Eq. (34). This equation is used to support the claim that the empirical and true scores coincide for t > t_c and to interpret Fig. 7. The definition of H_t and the justification of the annealed step, together with the sense in which Eq. (34) is an equality or an asymptotic equivalence, need to be supplied or the claim should be weakened accordingly.
  4. [§6.2, Eq. (37)] The statement that 0 < t_g < t_c holds for all α and α_D is presented as a general result, but the support appears to be numerical observation of the curves in Figs. 7 and 8. No analytic argument or systematic finite-size study is given, and the theoretical curves themselves depend on the unresolved items above. Since this inequality is one of the paper's headline conclusions, the authors should either prove it from the REM formulas or provide a detailed numerical study, including error bars and the extrapolation used to establish the claimed ordering for all α, α_D. In addition, because the minimization is performed on the time-dependent part of the renormalized KL divergence (Appendix E), the main text should state explicitly that s_0 is time-independent and that the minimizer of the full KL coincides with the minimizer of the renormalized quantity.
  5. [§5.1.1, Appendices B.2 and D.2] The nonlinear-manifold results rest on a replica symmetric ansatz for the saddle-point computation of ζ_t. The paper does not provide a stability check of the RS saddle point, nor does it discuss where replica symmetry might be expected to break down. Since Figs. 3, 4, 7, and 8 all use the RS approximation for nonlinear activations, the absence of a stability analysis leaves the quantitative nonlinear predictions less secure than the linear-manifold results. Please add an RS stability analysis, or justify why the integrals in the energetic term are convex enough to make RS exact or reliable.
  6. [§6.3, Eqs. (42)–(45)] The 'generalization condition' tc ≤ t_RMT_g depends on an arbitrary threshold δ ≈ 1, which is later fixed to δ = 1 − ε. The resulting P_min = ε^{-D/2} is therefore a scaling statement with respect to an unspecified small parameter ε, not a principled minimal sample size. The paper should clarify that no canonical δ is defined, state how the choice of δ affects the constant in P_min, and avoid presenting Eq. (45) as a precise data requirement without discussing the arbitrariness of the threshold.
minor comments (6)
  1. [§1, Introduction] There are numerous typographical errors, including 'Closet to our work' instead of 'Closest to our work', 'as showed by' instead of 'as shown by', and 'on-set time' for 'onset time' in Section 5.2. The manuscript would benefit from a careful proofreading pass.
  2. [§5.1, Eq. (19)] The sentence 'The so-called "curse of dimensionality", i.e. the need for a number of training data points that scales exponentially in the visible dimension ... has been mitigated' is imprecise: Eq. (19) gives t_c as a function of P and D, but the claim that this means the curse of dimensionality is avoided should be phrased as a statement about the scaling of t_c with α/α_D, as done later in the paper.
  3. [§6.3, Eq. (44)] The symbol Δ appears in Eq. (44) where δ was used in Eq. (42), and the relationship between Δ and ε is not stated in the displayed line. This should be made consistent.
  4. [§6.1] The sentence 'We have proved in Section 6.1 that the true score coincides with the empirical one for t > t_c' in Section 6.3 overstates what Eq. (34) establishes, especially given that H_t is undefined and the derivation is incomplete. The wording should be softened or the proof completed.
  5. [§4, Eq. (7)] The definition of ζ(λ) in Eq. (7) uses Eω log E_{ε|ω}, while the text says that the cumulant generating function has a well-defined limit with probability one over ω; the relationship between the quenched and annealed averages should be stated more explicitly, since it is important for the later use of the annealed approximation.
  6. [Figures 2–5, 7–8] The figures show theoretical curves only, with no markers from finite-N simulations. Given that the paper makes quantitative predictions, adding even a small number of finite-N data points for the linear and Gaussian cases would make the figures substantially more persuasive.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the paper's t_c, t_g, and the collapse-condensation equivalence are calculations from the imported collapse criterion and REM formalism, not reductions to fitted inputs; self-citations from overlapping authors are present but do not force the conclusions.

full rationale

The derivation is self-contained conditional on the collapse criterion imported from Ref. [7] and the REM formalism of Ref. [25]. Eq. (14) is a signal-to-noise comparison of log-densities; the paper uses it as a premise rather than deriving it from the reverse SDE. That is a correctness or robustness concern, not a circularity: the paper does not fit constants to obtain Eq. (19), and t_c follows by explicit evaluation of the cumulant generating function zeta_t (Eqs. 17-19). The collapse-condensation equivalence is not definitional: Eq. (25) is the REM condensation condition, and the proof requires the nontrivial Nishimori identity zeta'_tc(1) = -1/2 (Appendix C); only after that identity is proved do Eq. (14) and Eq. (25) coincide. The generalization time t_g is obtained by minimizing a separately computed KL free-energy expression (Eq. 36) and is not constructed to satisfy 0 < t_g < t_c; that inequality is a derived and numerical outcome. The Jacobian result Eq. (40) is imported from Ref. [36] with overlapping authors, but it is used as an external RMT statement rather than being tuned to the paper's conclusions. Therefore no load-bearing step reduces by construction to its inputs. The reliance on Refs. [7,25,36], all involving overlapping authorship, is the main caveat; if those premises were unsound the conclusions would shift, but that is a support and correctness issue rather than circular reasoning.

Assumptions & free parameters 1 free parameters · 8 assumptions · 0 invented entities

The central derivations rest on the standard REM formalism, the imported collapse criterion of [7], the Nishimori identity, and an unverified replica symmetric ansatz for nonlinear manifolds. There are no fitted parameters in the main tc/tg results; the only hand-chosen threshold is the spectral gap delta in Section 6.3. The empirical score is used as a proxy for the learned score, which is a modeling choice, not an invented entity.

free parameters (1)
  • delta = delta = 1 - epsilon (small epsilon > 0) in the RMT generalization criterion
    In Section 6.3, the spectral gap threshold delta is chosen by hand (Eq. 42) to define the time t_g^RMT; the final P_min = epsilon^{-D/2} depends on this arbitrary epsilon.
assumptions (8)
  • domain assumption Data are generated by the Hidden Manifold Model: xi_mu = g(F z_mu / sqrt(D)) with z_mu ~ N(0, I_D), F i.i.d. Gaussian, and P = e^{alpha N} in the N -> infinity limit.
    This generative process is introduced in Section 2 and underlies every derivation; the paper does not test other data geometries.
  • standard math The REM free energy concentrates and the energy levels behave as i.i.d. with a well-defined cumulant generating function (Section 4, Eqs. 6-10).
    The analysis of the empirical score mixture relies on standard large-deviation and REM concentration results, invoked throughout Sections 5 and 6.
  • domain assumption Collapse criterion: a typical trajectory collapses when the exponential rate of the target Gaussian term equals that of the sum over all other training points, alpha + zeta_tc(1) = -1/2 (Eq. 14).
    Imported from Ref. [7] and used as the definition of memorization; not derived from the diffusion SDE in this paper.
  • domain assumption Nishimori condition at lambda = 1 applies to the average over typical trajectories, yielding zeta'_t(1) = -1/2 (Appendix C).
    This identity is the crux of the proof that collapse and condensation coincide; it requires the data-generating and likelihood models to match at inverse temperature beta = 1.
  • ad hoc to paper Replica symmetric ansatz for the nonlinear manifold computation (Appendix B.2, D.2).
    The saddle-point equations are solved under replica symmetry with no stability check; the correctness of RS for the nonlinear manifold is not established.
  • ad hoc to paper The KL divergence is renormalized by subtracting the formally divergent entropy s0 of the manifold-supported target distribution (Appendix E).
    The paper computes only the time-dependent part of DKL[p0 | pemp_t] because s0 diverges; the assumption that the argmin of the renormalized quantity equals the argmin of the true KL is not fully justified.
  • domain assumption For t > t_c the true score and empirical score coincide (Section 6.1, Eq. 34), based on the annealed approximation and an undefined H_t term.
    This is used to justify the RMT generalization criterion in Section 6.3, but the derivation is incomplete as written.
  • domain assumption The score Jacobian spectrum for linear manifolds is given by Eq. (40) from Ref. [36], and the nonlinear extension is argued via small-time curvature.
    Section 6.3 builds on the spectrum of J_t from the authors' prior work [36]; the nonlinear extension is heuristic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memorization and Generalization in Generative Diffusion under the Manifold Hypothesis." pith.science (2026). https://pith.science/paper/LMD4KKLM

@misc{pith2026250209578,
  author       = {Pith},
  title        = {Pith review of: Memorization and Generalization in Generative Diffusion under the Manifold Hypothesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LMD4KKLM}},
  note         = {Machine review of arXiv:2502.09578}
}
abstract

We study the memorization and generalization capabilities of Diffusion Models (DMs) when data lies on a structured latent manifold. Specifically, we consider a set of $P$ data points in $N$ dimensions confined to a latent subspace of dimension $D = \alpha_D N$, following the Hidden Manifold Model (HMM). We analyze the reverse diffusion process using the empirical score function as a proxy, and characterize it in the high-dimensional limit $P = \exp(\alpha N)$, $N \gg 1$, by exploiting a connection with the Random Energy Model (REM). We show that a characteristic time $t_o$ marks the emergence of traps in the time-dependent potential, which however do not affect typical trajectories. The size of their basins of attraction is computed at all times. We derive the collapse time $t_c < t_o$, at which trajectories fall into the basin of a training point, signaling memorization. An explicit formula for $t_c$ as a function of $P$ and $\alpha_D$ shows that the curse of dimensionality is avoided for structured data ($\alpha_D \ll 1$), even with nonlinear manifolds. We also prove that collapse corresponds to the condensation transition in the REM. Generalization is quantified via the Kullback-Leibler divergence between the exact distribution and the reverse one at time $t$. We find a distinct time $t_g < t_c < t_o$ minimizing this divergence. Surprisingly, the best generalization occurs inside the memorization phase. We conclude that generalization in DMs improves with data structure, as $t_g \to 0$ faster than $t_c$ when $\alpha_D \to 0$.

Figures

Figures reproduced from arXiv: 2502.09578 by the authors.

Figure 1
Figure 1. Pictorial representation of the phases identified in the reverse process (from large times to t = 0) driven by the empirical score. The evolution of a typical trajectory is represented by a dotted blue line. For t < to, data points form a basin around them, but the typical trajectory is not affected by this. For t < tc, the basins of attraction of the data points cover the whole space, trajectories cannot escape the… view at source ↗
Figure 2
Figure 2. Semi-logarithmic plots of tc in the linear manifold case (solid) compared to the homogeneous Gaussian case (dashed) for different values of αD (Left) and α (Right). This function can be computed using the replica method, as shown in Appendix B.2. We find an expression for ζt in the RS approximation, i.e. ζt(λ; qd, q0, m, qˆd, qˆ0, mˆ ) = −αDmmˆ − αD 2 (qdqˆd − q0qˆ0) + αDGS(ˆqd, qˆ0, mˆ ) + GE(λ, t; qd, q0, m), (22)… view at source ↗
Figure 3
Figure 3. Semi-logarithmic plots of tc in the hidden manifold case (solid) with tanh activation compared to the linear manifold case (dashed) for different values of αD (Left) and α (Right). In order to establish that the condensation and collapse phenomena happen at the same time, tc = tcond, we would therefore need to prove that ζ ′ tc (1) = − 1 2 . (26) This is indeed what we find for a typical trajectory as a consequence … view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: (Left) Onset time to as a function of α/αD in semi-log scale in the hidden manifold case (solid) with tanh activation compared to the linear manifold case (dashed); (Center) to as a function of αD for fixed α in semi-log scale for tanh activation; (Right) comparison of…
Figure 5
Figure 5. Figure 5: The violet line gives the radius R of the basin of attraction around one data point as a function of time. All particles at a distance smaller than R(t) collapse to the same data point, while the ones at larger distances do not. The basin of attraction appears at t = t…
Figure 6
Figure 6. Figure 6: In these Figures, at two given times, the heat map indicates the empirical sampling distribution of a DM, red arrows represent the empirical score while green ones represent the exact score. The black dots denote individual data points. Panel A depicts such quantities …
Figure 7
Figure 7. Figure 7: Time-dependent component of the KL divergence between target distribution and empirical distribution as a function of time t and for different values of α. The data are generated from a HMM with tanh activation and aspect ratio αD = 0.5. We report with colored dashed l…
Figure 8
Figure 8. Figure 8: (Left) Generalization time tg as a function of α/αD in semi-log scale for tanh (solid) and linear (dashed) activation; (Center) tg as a function of αD for fixed α in semi-log scale for tanh (solid) and linear (dashed) activation; (Right) comparison of the generalizatio…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models

    cs.LG 2025-05 conditional novelty 6.0 of 10

    In overparameterized diffusion models, generalization happens first and memorization starts later, with the memorization time growing linearly with dataset size.

Reference graph

Works this paper leans on

39 extracted references · 27 canonical work pages · cited by 1 Pith paper

  1. [14]

    Analysis of diffusion models for manifold data

    Anand Jerry George, Rodrigo Veiga, and Nicolas Macris. Analysis of diffusion models for manifold data. arXiv:2502.04339, 2025

    arXiv 2025

  2. [7]

    Dynamical regimes of diffusion models

    Giulio Biroli, Tony Bonnaire, Valentin de Bortoli, and Marc Mézard. Dynamical regimes of diffusion models. Nature Communications, 15(1):9957, 2024

    work page 2024

  3. [1]

    Losing dimensions: Geometric memorization in generative diffusion

    Beatrice Achilli, Enrico Ventura, Gianluigi Silvestri, Bao Pham, Gabriel Raya, Dmitry Krotov, Carlo Lucibello, and Luca Ambrogioni. Losing dimensions: Geometric memorization in generative diffusion. arXiv:2410.08727, 2024

    arXiv 2024

  4. [2]

    In search of dispersed memories: Generative diffusion models are associative memory networks

    Luca Ambrogioni. In search of dispersed memories: Generative diffusion models are associative memory networks. Entropy, 5(26):381, 2024

    work page 2024

  5. [3]

    The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking, and critical instability

    Luca Ambrogioni. The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking, and critical instability. Entropy, 27(3), 2025

    work page 2025

  6. [4]

    Representation learning: A review and new perspectives

    Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828, 2013

    2013

  7. [5]

    Kernel density estimators in large dimensions

    Giulio Biroli and Marc Mézard. Kernel density estimators in large dimensions. arXiv:2408.05807, 2024

    arXiv 2024

  8. [6]

    Generative diffusion in very large dimensions

    Giulio Biroli and Marc Mézard. Generative diffusion in very large dimensions. Journal of Statistical Mechanics: Theory and Experiment, 2023(9):093402, 2023

    work page 2023

Show all 39 references
  1. [8]

    Shallow diffusion networks provably learn hidden low-dimensional structure

    Nicholas Boffi, Arthur Jacot, Stephen Tu, and Ingvar Ziemann. Shallow diffusion networks provably learn hidden low-dimensional structure. International Conference on Learning Representations, 2025

    2025

  2. [9]

    Score approximation, estimation and distribu- tion recovery of diffusion models on low-dimensional data

    Minshuo Chen, Kaixuan Huang, Tuo Zhao, and Mengdi Wang. Score approximation, estimation and distribu- tion recovery of diffusion models on low-dimensional data. In International Conference on Machine Learning, pages 4672–4712. PMLR, 2023

    2023

  3. [10]

    Weiss, Mohammad Norouzi, and William Chan

    Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation, 2020. arXiv:2009.00713

    2020 arXiv

  4. [11]

    Approximation by superpositions of a sigmoidal function

    George Cybenko. Approximation by superpositions of a sigmoidal function. Math. Control Signals Systems, 2: 303–314, 1989

    1989

  5. [12]

    On a model of associative memory with huge storage capacity

    Mete Demircigil, Judith Heusel, Matthias Löwe, Sven Upgang, and Franck Vermet. On a model of associative memory with huge storage capacity. Journal of Statistical Physics, 168:288–299, 2017

    2017

  6. [13]

    Random-energy model: An exactly solvable model of disordered systems

    Bernard Derrida. Random-energy model: An exactly solvable model of disordered systems. Physical Review B, 24(5):2613–2626, 1981. ISSN 0163-1829. doi: 10.1103/PhysRevB.24.2613. 17

    1981 doi

  7. [15]

    Generalisation error in learning with random features and the hidden manifold model

    Federica Gerace, Bruno Loureiro, Florent Krzakala, Marc Mézard, and Lenka Zdeborová. Generalisation error in learning with random features and the hidden manifold model. In International Conference on Machine Learning, pages 3452–3462. PMLR, 2020

    2020

  8. [16]

    Modeling the influence of data structure on learning in neural networks: The hidden manifold model

    Sebastian Goldt, Marc Mézard, Florent Krzakala, and Lenka Zdeborová. Modeling the influence of data structure on learning in neural networks: The hidden manifold model. Physical Review X, 10(4):041044, 2020

    2020

  9. [17]

    The gaussian equivalence of generative models for learning with shallow neural networks

    Sebastian Goldt, Bruno Loureiro, Galen Reeves, Florent Krzakala, Marc Mézard, and Lenka Zdeborová. The gaussian equivalence of generative models for learning with shallow neural networks. In Mathematical and Scientific Machine Learning, pages 426–471. PMLR, 2022

    2022

  10. [18]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Neural Information Processing Systems. Conference on Neural Information Processing Systems, 2020

    2020

  11. [19]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022. arXiv:2204.03458

    2022 arXiv

  12. [20]

    Memory in Plain Sight: A Survey of the Uncanny Resemblances between Diffusion Models and Associative Memories, 2023

    Benjamin Hoover, Hendrik Strobelt, Dmitry Krotov, Judy Hoffman, Zsolt Kira, and Duen Horng Chau. Memory in Plain Sight: A Survey of the Uncanny Resemblances between Diffusion Models and Associative Memories, 2023. arXiv:2309.16750

    2023 arXiv

  13. [21]

    A new frontier for hopfield networks

    Dmitry Krotov. A new frontier for hopfield networks. Nature Reviews Physics, pages 1–2, 2023

    2023

  14. [22]

    Dense associative memory for pattern recognition

    Dmitry Krotov and John Hopfield. Dense associative memory for pattern recognition. Advances in Neural Information Processing Systems, 2016

    2016

  15. [23]

    Cresswell, and Gabriel Loaiza-Ganem

    Brendan Leigh Ross, Hamidreza Kamkari, Tongzi Wu, Rasa Hosseinzadeh, Zhaoyan Liu, George Stein, Jesse C. Cresswell, and Gabriel Loaiza-Ganem. A geometric framework for understanding memorization in generative models. International Conference on Learning Representations, 2025

    2025

  16. [24]

    On the generalization properties of diffusion models

    Puheng Li, Zhong Li, Huishuai Zhang, and Jiang Bian. On the generalization properties of diffusion models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 2097–2127. Curran Ass...

    2023

  17. [25]

    The Exponential Capacity of Dense Associative Memories

    Carlo Lucibello and Marc Mézard. The Exponential Capacity of Dense Associative Memories. Physical Review Letters, 132:077301, 2024

    2024

  18. [26]

    Sampling, Diffusions, and Stochastic Localization

    Andrea Montanari. Sampling, Diffusions, and Stochastic Localization. 2023. doi: 10.48550/arXiv.2305.10690. arXiv:2305.10690

    2023 doi arXiv

  19. [27]

    Posterior Sampling from the Spiked Models via Diffusion Processes

    Andrea Montanari and Yuchen Wu. Posterior Sampling from the Spiked Models via Diffusion Processes. 2023. arXiv.2304.11449

    2023 arXiv

  20. [28]

    Exact results and critical properties of the ising model with competing interactions

    Hidetoshi Nishimori. Exact results and critical properties of the ising model with competing interactions. J. Phys. C: Solid State Phys., 13:4071–4076, 1980. 18

    1980

  21. [29]

    Score-Based Generative Models Detect Manifolds.Conference on Neural Information Processing Systems, 2022

    Jakiw Pidstrigach. Score-Based Generative Models Detect Manifolds.Conference on Neural Information Processing Systems, 2022

    2022

  22. [30]

    Hopfield networks is all you need

    Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlovi´ c, Geir Kjetil Sandve, Victor Greiff, David Kreil, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hop...

    2021

  23. [31]

    Spontaneous symmetry breaking in generative diffusion models

    Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion models. In Neural Information Processing Systems. Conference on Neural Information Processing Systems, 2023

    2023

  24. [32]

    A phase transition in diffusion models reveals the hierarchical nature of data

    Antonio Sclocchi, Alessandro Favero, and Mathieu Wyart. A phase transition in diffusion models reveals the hierarchical nature of data. Proceedings of the National Academy of Sciences, 122(1):e2408799121, 2025

    2025

  25. [33]

    Weiss, Niru Maheswaranathan, and Surya Ganguli

    Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning. ICML, 2015

    2015

  26. [34]

    Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole

    Yang Song, Jascha Sohl-Dickstein, Diederik P . Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations. ICLR, 2020

    2020

  27. [35]

    Diffusion models encode the intrinsic dimension of data manifolds

    Jan Pawel Stanczuk, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. Diffusion models encode the intrinsic dimension of data manifolds. In Forty-first International Conference on Machine Learning, 2024

    2024

  28. [36]

    Manifolds, random matrices and spectral gaps: The geometric phases of generative diffusion

    Enrico Ventura, Beatrice Achilli, Gianluigi Silvestri, Carlo Lucibello, and Luca Ambrogioni. Manifolds, random matrices and spectral gaps: The geometric phases of generative diffusion. International Conference on Learning Representations, 2025

    2025

  29. [37]

    Generalization error of gan from the discriminator’s perspective

    Hongkang Yang and Weinan E. Generalization error of gan from the discriminator’s perspective. Research in the Mathematical Sciences, 9, 2021

    2021

  30. [38]

    Generalization and memorization: The bias potential model

    Hongkang Yang and Weinan E. Generalization and memorization: The bias potential model. In Joan Bruna, Jan Hesthaven, and Lenka Zdeborova, editors,Proceedings of the 2nd Mathematical and Scientific Machine Learning Conference, volume 145 of Proceedings of Machine Learning Resea...

    2022

  31. [39]

    Error bounds for approximations with deep relu networks

    Dmitry Yarotsky. Error bounds for approximations with deep relu networks. Math. Control Signals Systems, 94: 103–114, 2017. 19 A Collapse Time for Homogeneous Gaussian Data When the data points live a linear manifold we can consider the basis in which the manifold has diagonal...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.