{"id":"1299f1b8-a651-4b01-8015-8b5da4182ed5","arxiv_id":"2411.17807","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper derives a closed-form KL divergence for a one-step linear diffusion model on Gaussian data, reports a sample-size threshold at n=d, and gives a heuristic argument that more diffusion steps improve quality.","lead":"Researchers build a toy diffusion model with a single linear denoising step trained on Gaussian data, and derive an explicit formula for how far the generated distribution sits from the true one. The formula shows the gap shrinks once the training set has more points than the data dimension, and offers a toy-level explanation for why more diffusion steps improve image quality.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix C solves a different regression than Section 2 defines; the true residual variance is e^{2T}ΔT, not ΔT, changing the O(λhat) term in (2.7) by a factor e^{2T}.","rationale":"After reading in good faith, I find the central theorem is not established for the model it claims to analyze. The paper's core idea — using deterministic equivalence to get a closed-form KL for a linear one-step denoiser — is reasonable, and the machinery in Appendix B is standard. But the proof in Appendix C changes the data-generating process: it assumes an independent additive noise with variance ΔT around a linear map with slope e^T, whereas (2.2) defines X as a noisy version of Y, making the joint distribution Gaussian with conditional slope a* and residual variance e^{2T}ΔT to leading order. This is not a matter of external consensus; it is an internal mismatch. The mismatch modifies the leading O(λhat) term of (2.7) by a factor e^{2T}, as shown by computing the fluctuation contribution. The numerical figures mainly probe α<<1, where the O(λhat) term is suppressed, so they do not detect the error. Lemma 2 remains plausible for the corrected model, but Lemma 3's scaling argument inherits the same issue. I therefore agree with the reader's REJECT verdict: the explicit formulas are not derived for the model in Section 2. The paper still has value as a solvable analysis of the Appendix C regression, but the stated central claim is not supported.","tokens_in":20237,"tokens_out":38119,"duration_ms":288701,"concrete_test":"Re-derive the O(λhat) coefficient of ⟨KLvar⟩ for the model (2.2) using the exact conditional regression Y|X with slope a* = e^{-T}σ²/(e^{-2T}σ²+ΔT) and residual covariance Σ_ε = σ²ΔT/(e^{-2T}σ²+ΔT)I. For α<1, compute E[Tr(δ^Tδ)] = Tr(Σ_ε) E[Tr((x^T x)^{-1})] with E[Tr((x^T x)^{-1})] = α/(σ_X²(1−α)), and compare the resulting KLvar_linear = (1/2)e^{-2T}E[Tr(δ^Tδ)] with the first term of (2.7). If KLvar_linear is e^{2T} times the formula's first term, the Appendix C model is not equivalent to Section 2.","verdict_should_be":"REJECT","load_bearing_attack":"Theorem 1 is proved in Appendix C, but the regression problem solved there is not the one defined in Section 2. From (2.2), with Y_k ~ N(μ, σ²I) and Z_k ~ N(0, I), the pair (X, Y) is jointly Gaussian. The conditional regression slope is a* = e^{-T}σ²/(e^{-2T}σ²+ΔT) and the residual covariance is Σ_ε = σ²ΔT/(e^{-2T}σ²+ΔT)I, with the residual independent of X. Appendix C instead solves Y_k = e^T X_k + Z_k with Z_k ~ N(0, ΔT I) independent of X, i.e. slope e^T and residual variance ΔT. Footnote 8 asserts leading-order equivalence, but this is false at the order retained: at λhat << 1, a* − e^T = O(λhat) and Σ_ε ≈ e^{2T}ΔT I, a relative error of order 1 in the fluctuation variance. The linear-in-λhat term of (2.7) is proportional to E[Tr((θ1 − Eθ1)(θ1 − Eθ1)^T)] = Tr(Σ_ε) E[Tr((x^T x)^{-1})]. Using the true Σ_ε replaces the paper's coefficient d α λhat e^{-2T}(1−e^{-2T})/(2(1−α)) with d α λhat (1−e^{-2T})/(2(1−α)) — a factor e^{2T} larger. Thus (2.7) and (2.8) describe the Appendix C model, not the generative model of Section 2. The error is internal: the defined data-generating process and the analyzed regression disagree.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a single-step, linear-denoiser generative model in which noisy samples are produced by an Ornstein-Uhlenbeck-style forward process, a linear regression denoiser is trained on those samples, and the generated distribution is obtained by applying the denoiser to a new noisy sample. The main claimed result is a closed-form, high-dimensional asymptotic expression for the variance part of the KL divergence between the generated and the underlying isotropic Gaussian distributions, together with a phase-transition statement at α = d/n = 1 and an explanation, based on that formula, of why more diffusion steps improve output quality in practical DDPMs. The proof uses deterministic equivalence / random matrix theory in Appendix C, and numerical experiments are reported in Figures 1 and 2.","tokens_in":20570,"tokens_out":10111,"duration_ms":95158,"significance":"If the central derivation were correct, the paper would provide a rare exactly solvable toy model in which the generation gap of a diffusion-like model can be predicted in the n,d → ∞ limit with fixed α = d/n. The claimed explicit formulas (2.7) and (2.8), the monotonicity result for α < 1, and the qualitative explanation of step-count effects in Section 3 would be useful and testable. The paper also connects to a well-established random-matrix toolkit rather than fitting free constants, which is a strength. However, the main theorem is derived for a regression problem that does not match the generative model defined in Section 2, so the headline formulas are not established for the stated model; this makes the significance of the numerical comparison in Figure 1 unclear.","major_comments":[{"comment":"The regression solved in Appendix C is not the regression defined in Section 2. In the generative model, X_k = e^{-T}Y_k + sqrt(ΔT)Z_k with Y_k and Z_k independent, so the pair (X,Y) is jointly Gaussian and the residual Y_k - e^T X_k = -e^T sqrt(ΔT)Z_k is correlated with X_k. The true conditional slope is a_* = e^{-T}σ²/(e^{-2T}σ²+ΔT), with residual covariance σ²ΔT/(e^{-2T}σ²+ΔT)I. Appendix C instead solves Y_k = e^T X_k + Z_k with X_k and Z_k independent, i.e., slope e^T and residual variance ΔT. Footnote 8's moment comparison does not establish equivalence because the least-squares estimator is controlled by the conditional distribution, not by the first two marginal moments. At λhat ≪ 1, a_* - e^T = O(λhat) and the residual covariance is approximately e^{2T}ΔT I, a factor e^{2T} larger than ΔT. Consequently the O(λhat) term of (2.7) is off by a factor e^{2T}: the correct coefficient is d α λhat(1-e^{-2T})/(2(1-α)), not d α λhat e^{-4T}(e^{2T}-1)/(2(1-α)). Thus (2.7), (2.8), Lemma 3, and the comparison in Figure 1 describe a different model than the one defined in Section 2.","section":"Appendix C, Eq. (C.1) and Section 2, Eq. (2.2)"},{"comment":"The one-line proof of Lemma 2, 'Derivative of RHS in (2.7) with respect to α is positive', is not a complete verification: it is not stated whether the derivative is taken at fixed d or fixed n, and the derivative is not displayed. Since monotonicity of KLvar in n/d is one of the paper's central qualitative claims, this step needs a full derivation or a clear reference to a verified calculation.","section":"Theorem 1 / Lemma 2"},{"comment":"The abstract states that the paper presents 'an explicit formula for the Kullback-Leibler divergence', but Theorem 1 actually computes the expectation of KLvar, which is only a lower bound on KL because KLmean is dropped. The smallness of KLmean is supported only by a numerical regime in Figure 1(a), not by an analytic bound. This is a discrepancy between the stated contribution and the proved statement, and it should be corrected in the presentation.","section":"Appendix C, Eq. (C.10) / Theorem 1"}],"minor_comments":[{"comment":"The application of Lemma 3 to DDPM steps is heuristic: the identification β_t = 1 - e^{-2βt/s} is used to map each step to the linear model, but the argument that 'each step improves' and the implicit neglect of correlations between steps are not justified. The claims would be stronger if the predicted 1/s² scaling were tested quantitatively against the experiments in Figure 2.","section":"Section 3, Eq. (3.2)"},{"comment":"The displayed formulas for C_{a,b} and B_a contain branch-dependent expressions with sgn functions, and the final ridgeless results use particular branches without explaining which regime each branch corresponds to. A reader cannot easily verify the final simplification without additional explanation.","section":"Appendix C, Eqs. (C.24)-(C.28)"},{"comment":"There are numerous typographical and notation issues, including inconsistent use of hat symbols (e.g., λhat vs λ), missing parentheses in displayed formulas, and undefined quantities such as 'Thse are in agreement with the plot in figure 1'. These should be cleaned up in any revision.","section":"Throughout"},{"comment":"The figure legend states that 'we can clearly see that image quality improves as diffusion steps increase', but no quantitative metric or error bars are reported for the image-quality claim. Please provide a precise evaluation protocol.","section":"Figure 2"}],"recommendation":"reject","confidential_remarks":"The central problem is the errors-in-variables mismatch between the generative model (2.2) and the regression analyzed in Appendix C. This is not a minor technical gap: it changes the O(λhat) coefficient in the main formula by a factor e^{2T}. The authors could potentially repair the paper by redoing Appendix C under the correct conditional regression, but the current manuscript's main theorem is not valid for the model it defines, and the numerical comparison in Figure 1 cannot be used to validate the stated formula. I therefore recommend rejection rather than major revision, while noting that a corrected derivation of the same style could form the basis of a future submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper has a clear and checkable central claim: a closed-form KL divergence for a one-step linear denoiser trained on n isotropic Gaussian samples, with an explicit n = d threshold for the onset of monotonic decrease. That combination is new relative to the cited literature, and the high-dimensional calculation is a serious attempt. So the paper deserves a careful look, not a dismissive one.\n\nUnfortunately, the proof does not analyze the model defined in Section 2. Equation (2.2) sets X_k = e^{-T}Y_k + sqrt(DeltaT)Z_k, so regressing Y on X is errors-in-variables: the noise entering X also appears in the residual. For Gaussian data, the conditional slope is a* = e^{-T} sigma^2/(e^{-2T}sigma^2 + DeltaT) and the residual covariance is sigma^2 DeltaT/(e^{-2T}sigma^2 + DeltaT) I. At small lambda-hat these differ from e^T and DeltaT at first order. Appendix C instead solves Y = e^T X + Z with X and Z independent, and footnote 8's claim of leading-order equivalence is false at the order retained. Redoing the fluctuation part alone changes the linear-in-lambda-hat term of (2.7) by a factor e^{2T}, and the slope bias contributes at the same order. So (2.7)-(2.8), as proved, describe a different model than the one defined in Section 2.\n\nThe reader's separate concern about Lemma 3 does not hold up: taking alpha -> 0 in (2.7) gives KLvar/d proportional to lambda^2 e^{4T}(1-e^{-2T})^2, which is exactly Lemma 3's statement.\n\nWhat the paper does well: the setup is clean, the high-dimensional limit is the right one, and the qualitative alpha = 1 threshold is interesting enough that a corrected calculation could be a useful benchmark. Section 3's story about diffusion steps is heuristic and does not follow from the theorem.\n\nAs it stands, the central theorem lacks a valid derivation. That is a load-bearing flaw, but it is specific and in principle fixable: redo the calculation with the correct conditional regression, or change the model definition to match Appendix C. I would send it to a referee who knows random matrix theory and errors-in-variables, because the error is subtle and the corrected version might be valuable. I would not cite it in its current form.","headline":"The paper's central theorem is proved for a different regression model than the one defined in Section 2, leaving the main formula unsupported; still, the setup is a good idea and the flaw is fixable.","tokens_in":21124,"tokens_out":10513,"would_cite":false,"duration_ms":77496,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60B20","62J07","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A solvable diffusion model yields an exact formula for the generation gap.","keywords":["diffusion models","linear denoiser","Kullback-Leibler divergence","high-dimensional asymptotics","deterministic equivalence","random matrix theory","generation gap","sample complexity"],"falsifier":"Simulate the Section 2 model at $d=2000$, $n=4000$ ($\\alpha=0.5$), $T=2$, and $\\hat\\lambda=10^{-3}$, and compare the seed-averaged empirical $\\mathrm{KL}_{\\mathrm{var}}$ with the $\\alpha<1$ formula (2.7); agreement to first order in $\\hat\\lambda$ would confirm the independence assumption, while a mismatch linear in $\\hat\\lambda$ would show the errors-in-variables correlation matters.","tokens_in":19950,"feed_emoji":"📉","tokens_out":9484,"duration_ms":78077,"temperature":0.7,"pith_summary":"This paper constructs a deliberately simple generative model—a single linear denoising step trained on finitely many samples—and asks exactly how far its output distribution is from the true distribution it was trained on. The main result is an explicit formula for the expected Kullback-Leibler divergence between generated and sampling distributions for isotropic Gaussian data, valid when the dimension $d$ and training size $n$ grow together and the noise scale is small. The formula shows that the divergence starts falling monotonically with more data once $n$ exceeds $d$, a much friendlier threshold than the exponential scale suggested by the curse of dimensionality. The same calculation is then used to explain a practical observation: in realistic diffusion models, increasing the number of diffusion steps improves output quality.","feed_headline":"Exact formula predicts a toy diffusion model's generation gap","feed_subtitle":"A closed-form KL error shows when more training data starts to help and why extra steps matter.","key_machinery":"The central object is a linear denoiser $\\hat Y = \\hat\\theta_0 + \\hat\\theta_1 X$ trained by least squares, paired with the KL divergence decomposed into mean and variance parts, $\\mathrm{KL} = \\mathrm{KL}_{\\mathrm{mean}} + \\mathrm{KL}_{\\mathrm{var}}$. The variance part is computed in the $n,d\\to\\infty$ limit using the principle of deterministic equivalence from free random-matrix theory: the empirical covariance of the design matrix is treated as a free product of the population covariance with a white Wishart matrix, giving explicit traces $\\mathrm{Tr}\\,\\hat\\Sigma_{\\theta_1}$ and $\\mathrm{Tr}\\,\\hat\\Sigma_{\\theta_1}^2$ via Stieltjes-transform identities. This carries the argument because it turns the random least-squares estimator into a deterministic function of $\\alpha=d/n$, the diffusion time $T$, and the noise parameter $\\hat\\lambda$.","core_discovery":"In the high-dimensional limit $n,d\\to\\infty$ with $\\alpha=d/n$ fixed and noise scale $\\lambda=\\hat\\lambda\\sigma^2 e^{-2T}$ small, the expected variance contribution to the KL divergence between the generated Gaussian and the true isotropic Gaussian sampling distribution is given by closed-form expressions. For $\\alpha<1$ the paper derives\n$$\\langle \\mathrm{KL}_{\\mathrm{var}}\\rangle = \\frac{d\\$\\alpha$\\hat\\$\\lambda$ $e^{{-4T}}$($e^{{2T}}$-1)}{2(1-\\$\\alpha$)} + \\frac{d\\hat\\$lambda^{2}$ $e^{{-8T}}$($e^{{2T}}$-1)^2\\big(\\$alpha^{2}$+(1-\\$\\alpha$)^3 $e^{{4T}}$+4\\$\\alpha$(1-\\$\\alpha$)^2 $e^{{2T}}$\\big)}{4(1-\\$\\alpha$)^3} + O(\\hat\\$lambda^{3}$),$$\nwith a parallel expression in (2.8) for $\\alpha>1$ that begins with $d(\\alpha-1)/(4\\alpha)$. The formula implies that once $n>d$, the KL divergence is monotonically decreasing in $n/d$, and that in the $n/d\\to\\infty$ limit the divergence per dimension scales like $\\lambda^2 e^{4T}(1-e^{-2T})^2$, so exact recovery of the underlying distribution requires $\\lambda e^{2T}(1-e^{-2T})\\ll 1$. For a Gaussian-mixture training set with the convolutional architecture used in practical diffusion models, the same single-step analysis predicts that the final denoising step improves by a factor of $s^2$ relative to a one-step model, explaining empirically observed quality gains from more diffusion steps.","pith_inferences":["Beyond the paper: the mismatch between the regression model in (C.1) and the generative noise model in (2.2) is testable; recomputing the first-order coefficient with the errors-in-variables regression would show whether the displayed formula is the exact generation gap of the defined model or of a nearby model.","Beyond the paper: composing $s$ copies of the linear denoiser should give a closed-form KL gap in $s$ with the same free-product calculus, providing an analytic check of the $s^2$ scaling that is only argued heuristically.","Beyond the paper: for non-Gaussian sampling densities with known covariance spectrum, the same deterministic traces would replace $d$ with spectral sums, extending the formula to a wider class of data while keeping the calculation explicit."],"forward_implications":["The generation gap per dimension becomes a function of the ratio $\\alpha=d/n$ alone, so in the proportional regime adding data beyond $n>d$ monotonically shrinks the KL divergence, with a singular divergence at $n=d$.","Exact recovery of the sampling distribution requires $\\lambda e^{2T}(1-e^{-2T})\\ll 1$; finite diffusion time, not just noise scale, sets the achievable fidelity.","In a multi-step diffusion model with Gaussian-mixture data, the last denoising step's error scales as $\\lambda^2\\beta^2(t/s)^2$, giving a factor $s^2$ improvement in the final step as the number of steps grows.","The practical criterion $\\lambda\\beta\\ll 1$ emerges for good generation quality, connecting the toy model's solvability to hyperparameter choices in realistic diffusion schedules."],"supporting_citations":[{"why":"Supplies the Ornstein-Uhlenbeck diffusion kernel used to motivate the noisy-sample construction in (2.2).","marker":"Sohl-Dickstein et al. (2015)"},{"why":"Supplies the discrete diffusion recursion (3.1) that the linear single-step analysis is mapped onto.","marker":"Ho et al. (2020)"},{"why":"Provides the high-dimensional ridge-regression asymptotics used to evaluate the deterministic traces.","marker":"Dobriban and Wager (2018)"},{"why":"Gives the ridgeless least-squares limit used to obtain the explicit small-$\\hat\\lambda$ expressions.","marker":"Hastie et al. (2022)"},{"why":"The paper points here for a recent treatment of the principle of deterministic equivalence used in Appendix C.","marker":"Atanasov et al. (2024)"}],"fun_headline_variants":["Exact KL formula for one-step linear denoiser","Single-step diffusion: closed-form generation error","When more data beats noise: exact toy-model answer","More steps improve quality: theory explains why","Linear denoiser: exact KL gap, no approximations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The explicit formulas assume the noise in the denoising regression is independent of the noise added to the training inputs, although the generative model defines them through the same random draw; the paper asserts, without proof, that this mismatch affects only higher-order terms.","fun_headline_variants_meta":{"raw":{"variants":["Exact KL formula for one-step linear denoiser","Single-step diffusion: closed-form generation error","When more data beats noise: exact toy-model answer","More steps improve quality: theory explains why","Linear denoiser: exact KL gap, no approximations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000206,"raw_usage":{"total_tokens":1410,"prompt_tokens":970,"completion_tokens":440,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":366}},"tokens_in":586,"tokens_out":440,"duration_ms":38857,"temperature":1.0,"reasoning_tokens":366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:50:57.052996+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate the Section 2 model at $d=2000$, $n=4000$ ($\\alpha=0.5$), $T=2$, and $\\hat\\lambda=10^{-3}$, and compare the seed-averaged empirical $\\mathrm{KL}_{\\mathrm{var}}$ with the $\\alpha<1$ formula (2.7); agreement to first order in $\\hat\\lambda$ would confirm the independence assumption, while a mismatch linear in $\\hat\\lambda$ would show the errors-in-variables correlation matters.","supporting_citations":[],"review_version":1}