{"id":"339b296f-b76d-41f1-a8cb-250c311293ad","arxiv_id":"2412.07935","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Replacing Gaussian step noise in diffusion models with Laplace or uniform noise yields comparable CIFAR-10 image quality, but the claimed invariance is a classical convergence result, not a new discovery.","lead":"Diffusion models usually add Gaussian noise step by step when learning to generate images. This paper says the noise distribution can be changed to Laplace or uniform and the model still converges to the same process as step size goes to zero, then derives new losses and shows slightly different image styles on CIFAR-10.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (5) is not a valid ELBO for non-Gaussian increments: it omits conditioning on x_0 and the appendix derivations condition on x_k rather than x_{k+1}; all Section 4 losses inherit this gap.","rationale":"The paper's practical contribution is the expanded class of training losses, and every one of them is derived from Eq. (5). The invariance principle in Theorem 3.2, even if fully proved, only concerns weak convergence of forward interpolated paths; it does not by itself justify a training objective for non-Gaussian reverse transitions. The reader's weakest assumption correctly identifies Eq. (5) as the load-bearing step. Checking the appendix confirms the problem is concrete: Lemmas A.1-A.4 condition on x_k, while Eq. (5) requires conditioning on x_{k+1}, and the derivation of a variational bound requires a posterior conditioned on x_0. For non-Gaussian increments the unconditional reverse kernel is genuinely different and is not determined by the first two moments, so the simplification to expressions like (9), (10), (13), and (14) is unsupported. The tightness gap in Lemma B.1 is real but secondary: the bound in Eq. (68) has the wrong order in Delta t, and Case 2 of the proof does not yield the claimed O(|t-s|^2) bound. However, that theorem is plausibly true by classical martingale/prohorov arguments, whereas Eq. (5) is essential and appears invalid as stated. The experiments are illustrative and cannot substitute for a valid objective. I therefore find no reason to change the reader's REJECT verdict.","tokens_in":20046,"tokens_out":18165,"duration_ms":183264,"concrete_test":"Re-derive the variational bound for a two-step scalar chain with Laplace increments, using the exact posterior q(x_1|x_2,x_0), and compare it with the RHS of Eq. (5) for an arbitrary fixed theta. A single numerical or symbolic instance where the RHS exceeds log p_theta(x_0) — or exceeds the true ELBO — falsifies Eq. (5) and settles the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. (5) is asserted as the likelihood bound used to derive every loss in Section 4, but it is neither derived nor correct as written. A variational lower bound requires a variational family q(x_{1:T}|x_0); the standard DDPM/VDM choice is q(x_{k-1}|x_k,x_0), which depends on x_0. Eq. (5) instead uses unconditional reverse kernels q(Delta x_k|x_{k+1}). For a structured random walk with non-Gaussian z_k, q(Delta x_k|x_{k+1}) is not equal to q(Delta x_k|x_{k+1},x_0), and it depends on the marginal density of x_k, not just on f, g, and the moments of z_k. Moreover, the appendix (Lemmas A.1-A.4) defines both p_theta and q as conditionals on x_k, e.g., p_theta(Delta x_k|x_k)=N(...), which are forward transition densities, not the generative reverse transition p_theta(x_k|x_{k+1}) appearing in Eq. (4). The KL integrals in Table 1 therefore do not evaluate the expectation in Eq. (5). This is not repaired by Theorem 3.2: weak convergence of forward paths does not imply that the one-step reverse conditionals of the non-Gaussian chain are Gaussian in the limit, nor that the proposed Laplace/Uniform q's are the correct variational posterior. Without Eq. (5), none of the losses (9), (10), (13), (14) are justified, so the paper's central claim that non-Gaussian increments yield a generalized training framework is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to generalize diffusion models by allowing the step increments Δx_k to be drawn from non-Gaussian distributions. Its main theoretical contribution is Theorem 3.2, a 'structured invariance principle' stating that a random walk with increments Δx_k = f(x_k,t_k)Δt_k + g(t_k)√Δt_k z_k, where z_k has zero mean and unit variance, converges in distribution to the Itô diffusion (1) as the step size goes to zero. On this basis the paper derives several training losses (Eqs. 9, 10, 13, 14) corresponding to Laplace and Uniform choices for q and p_θ, and reports CIFAR-10 experiments in Table 2 claiming competitive likelihood and sample quality.","tokens_in":20379,"tokens_out":6963,"duration_ms":68798,"significance":"If Theorem 3.2 were correctly proved and the likelihood bound (5) were valid, the paper would identify a genuinely useful design degree of freedom: the finite-step increment distribution in diffusion models. The proposed losses are explicit and are stated in a way that is directly testable, and the KL computations in Appendix A.1–A.3 are checkable. However, the central derivation is not sound. The ELBO in Eq. (5) is asserted without proof and is not a valid variational bound for non-Gaussian increments, and the tightness argument for Theorem 3.2 contains a concrete mathematical error. Because the training losses in Section 4 all depend on Eq. (5), the central claim of a generalized training framework is unsupported as it stands. The contribution is therefore conditional on a substantive reworking of the theory rather than on local corrections.","major_comments":[{"comment":"Equation (5) is the likelihood bound from which every loss in Section 4 is derived, but no derivation is given and, as written, it is not a valid variational lower bound for the reverse model in Eq. (4). A valid ELBO must involve a variational family over the latent trajectory conditioned on x0; in standard diffusion models this is q(x_{k-1} | x_k, x_0), whose dependence on x0 is essential. The paper instead writes KL(q(Δx_k | x_{k+1}) || p_θ(Δx_k | x_{k+1})) with no conditioning on x0 and no explicit variational family. For non-Gaussian structured increments, q(Δx_k | x_{k+1}) is not determined by the first two moments and is generally intractable. Moreover, the appendix derivations in Lemmas A.1–A.4 do not evaluate the expression in Eq. (5): they compute KL divergences between forward transition densities conditioned on x_k, e.g., p_θ(Δx_k | x_k) = N(Δx_k; f̂_θ(x_k,t_k)Δt_k, g(t_k)^2 Δt_k), rather than the reverse kernels q(Δx_k | x_{k+1}) and p_θ(Δx_k | x_{k+1}) appearing in Eq. (4). Therefore Eqs. (9), (10), (13), and (14) are not justified as ELBO-based losses. This gap is load-bearing: without Eq. (5), the paper's central claim of a generalized training framework is unsupported, and Theorem 3.2, even if repaired, concerns only the forward process and does not repair this issue.","section":"Section 2, Eq. (5); Appendix A.5"},{"comment":"The tightness argument in Lemma B.1 does not establish the Kolmogorov criterion it invokes. Equation (68) gives an upper bound E||Δx_k||^4 ≤ C√Δt (the displayed constant is O(√Δt)), but the preceding line actually suggests a bound of order Δt for the drift term and √Δt for the diffusion term, so the claimed order is already not justified. More importantly, in Case 1 the paper derives E||x_n(s) − x_n(t)||^4 ≤ (n|t−s|) E||Δx_k||^4 ≤ C_1 √n |t−s|, and then, using |t−s| ≤ n^{−1}, asserts a |t−s|^2 bound. This is not a passage from a first moment to a fourth power; it is already the fourth moment, and the bound is O(n^{−1/2}), not O(|t−s|^2). Case 2 has the same defect. Case 3 relies on Lemma B.3, which is stated for sums of independent mean-zero terms and is not connected to the original dependent increments in a way that yields the required |t−s|^2 bound. Thus the tightness half of Theorem 3.2 is not proved.","section":"Appendix B.3, Lemma B.1, Eq. (68)"},{"comment":"Even if Theorem 3.2 were proved, it would not establish the abstract's claim that the reversed process is invariant to the distribution of the increments. The theorem is a forward-time weak convergence statement for the càdlàg interpolation of the structured random walk. The reverse-time generative model in Eqs. (2)–(4) requires convergence of the one-step reverse conditionals q(x_k | x_{k+1}) (or q(Δx_k | x_{k+1})) to the reverse transitions of the limiting SDE. Weak convergence of forward paths does not imply convergence of these reverse conditionals for non-Gaussian increments; the reverse conditionals depend on the full marginal density of x_{k+1}, not just on f, g, and the first two moments of z_k. No lemma in Appendix B addresses reverse-time conditioning. Consequently the paper's main conceptual claim—that the reversed process is invariant to the increment distribution—is formally unsupported.","section":"Theorem 3.2 and Section 2"},{"comment":"The asymptotic claim in Eq. (11) is mathematically incorrect. Let a = v_k ||ε − ε_θ(x_k,t_k)||_1 and d(a) = exp(−a) − 1 + a. As t_k → 0, for fixed ε−ε_θ, v_k → 0 and hence a → 0. In this limit d(a) ~ a^2/2, so a/d(a) ~ 2/a → ∞, not 1. Equation (11) therefore does not show that the Laplace loss behaves like a weighted L1 norm when t is small. Equation (12) is correct as ||r_k||_1 → ∞, but it does not rescue Eq. (11). This is not merely cosmetic: the interpretation of the non-Gaussian losses as L1-type objectives is part of the paper's motivation and is used to explain the Laplace models' behavior.","section":"Section 4.2, Eq. (11)"}],"minor_comments":[{"comment":"The sign convention for Δx_k is inconsistent: the abstract and Eq. (4) define Δx_k := x_k − x_{k+1}, while Definition 1 and the appendices use Δx_k := x_{k+1} − x_k. This creates confusion in the derivation of the reverse losses and should be fixed throughout.","section":"Abstract, Eq. (4), Definition 1"},{"comment":"The constant C is reported inconsistently. Equation (13) states C = 1/2(1 + log(π/6)) ≈ 0.34, but 1/2(1 + log(π/6)) ≈ 0.18, while Table 1 writes 1/2(1 + log√(π/6)) ≈ 0.34. The numerical value and the expression should be reconciled.","section":"Section 4.3, Eq. (13), Table 1"},{"comment":"The 'Linear Sub-Gaussian' KL derivation is incomplete and is not connected to any loss in Section 4 or Table 1; the section ends mid-derivation, and the notation s_1, s_2 is never defined in the main text. Either complete the derivation or remove the section.","section":"Appendix A.4"},{"comment":"The experimental support is difficult to assess. Table 2 reports BPD and FID for four configurations, but the footnote says these metrics are computed with different architectures, no standard DDPM/VDM baselines are given, no error bars or training details are provided, and the sentence in Section 5 mentions down-sampled ImageNet but no ImageNet results are shown.","section":"Section 5, Table 2"},{"comment":"The paper states that 'we verify that we can recover Donsker's theorem when we let f = 0 and g = 1,' but no derivation is shown. Since Donsker's theorem requires independent increments, while the increments in Definition 1 are not independent when f ≠ 0, a brief verification or reference would be helpful.","section":"Section 3.2"}],"recommendation":"reject","confidential_remarks":"The paper would need a fundamentally different derivation of its training losses: either a correct ELBO with a variational posterior that conditions on x0 and is tractable for the chosen increment distributions, or a direct derivation from the reverse-time SDE. The tightness proof also needs a genuine fix, not just local correction. These are load-bearing issues that go beyond a revision of presentation. A secondary concern is novelty: forward-time invariance principles for triangular arrays of dependent increments are classical, and the paper's contribution would need to be framed carefully against that literature even after the technical gaps are repaired."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper's core intuition—that the continuous-time limit cares only about the first two moments of the increment distribution—is correct and classical. The specific non-Gaussian losses, however, are not justified by the argument given. Eq. (5) is asserted without derivation and is not the standard diffusion ELBO: a variational bound conditions the forward posterior on x_0, and the paper's q(Δx_k|x_{k+1}) is not that object. Worse, the appendix derives the losses using p_θ(Δx_k|x_k) and q(Δx_k|x_k), swapping x_k for x_{k+1}; so Eq. (4), Eq. (5), and the Table 1 losses do not line up. All four losses inherit this gap.\n\nWhat is genuinely nice: the Laplace and uniform KL computations in Appendix A are correct as far as I checked, and the uniform-Gaussian constant matches. The paper honestly cites Donsker and acknowledges that its theorem is inspired by it. The \"surprisingly few assumptions\" intuition is right—moment matching really is enough for the weak convergence.\n\nThe soft spots, in proportion: the novelty claim is overstated. Ethier-Kurtz and Stroock–Varadhan martingale problem results cover time-inhomogeneous Markov chains of exactly this kind; Theorem 3.2 is a special case, not a new theorem. The tightness proof in Lemma B.1 also does not close: E||Δx_k||^4 ≤ C√Δt cannot be turned into the |s−t|^2 bound needed for Kolmogorov's criterion. That gap is real but probably fixable, since the convergence is true. The ELBO problem is the load-bearing flaw, because every proposed loss is derived from it.\n\nThe experiments are illustrative at best: one BPD and one FID per configuration, different architectures for the two metrics, no error bars, no code. \"Competitive\" is not substantiated.\n\nWho gets value from this paper: someone thinking about non-Gaussian forward processes in diffusion models will find the intuition useful, but should not build on the losses as derived. The paper deserves a serious referee, mainly to document exactly where the ELBO fails and to check whether any of the losses can be rescued. I would not accept it in current form. A fixed version—valid ELBO or score-matching objectives, repaired tightness, better experiments—could be a narrow but real contribution.","headline":"The invariance idea is right but not new, and the loss derivations rest on an invalid ELBO; the paper needs major revision before it contributes.","tokens_in":20903,"tokens_out":6543,"would_cite":false,"duration_ms":66161,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60F17","60H10","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that diffusion models keep the same continuous-time limit no matter the distribution of their noise steps, provided each step has the matching mean and variance.","keywords":["diffusion models","non-Gaussian increments","invariance principle","structured random walks","score-based generative models","likelihood bound","generative modeling"],"falsifier":"Take a simple Ornstein-Uhlenbeck SDE and simulate structured random walks with $z_k$ drawn from Gaussian, Laplace, uniform, and a two-point distribution with matching mean and variance; if the finite-dimensional distributions at a fixed time $t$ do not all converge to the same OU Gaussian law as $\\Delta t_k \\to 0$, Theorem 3.2 is false. Separately, one can check Eq. (5) directly: for a Laplace-increment model with a fixed score network, compute the left- and right-hand sides by Monte Carlo on a small dataset and see whether the claimed lower bound actually holds.","tokens_in":19805,"feed_emoji":"🎲","tokens_out":6785,"duration_ms":80981,"temperature":0.7,"pith_summary":"Diffusion models normally inject Gaussian noise at every step, and the Gaussian assumption is treated as part of the model. This paper tries to establish that the Gaussian is not needed: as the step size goes to zero, any increment distribution with the correct conditional mean and variance drives the same continuous-time diffusion. If the claim holds, the noise distribution becomes a free design parameter, and diffusion models gain new families of training losses and new visual behavior. The paper proves an invariance principle for structured random walks, derives four concrete non-Gaussian diffusion models, and tests them on image data.","feed_headline":"No Gaussian required: diffusion limits only need mean and variance","feed_subtitle":"A new invariance principle lets diffusion models use Laplace, uniform, or other noise, opening fresh loss functions.","key_machinery":"The central object is the structured random walk, whose increments decompose as $$\\$\\Delta$ x_k = f(x_k,t_k)\\,\\$\\Delta$ t_k + g(t_k)\\sqrt{\\$\\Delta$ t_k}\\,z_k$$ with $\\mathbb{E}[z_k]=0$ and $\\operatorname{Var}(z_k)=1$, plus a finite-fourth-moment condition on $z_k$. Theorem 3.2 is the load-bearing result: it supplies the invariance that lets any such zero-mean, unit-variance noise define a valid diffusion model in the small-step limit. Closed-form KL divergences among Gaussian, Laplace, and uniform increment families then translate that invariance into concrete loss functions.","core_discovery":"The paper's central claim is Theorem 3.2, the Structured Invariance Principle. It states that, under mild regularity conditions (Lipschitz drift and diffusion, linear growth, and finite fourth moments of the noise variables), the càdlàg continuous-time interpolation of any structured random walk converges in distribution to the solution of the corresponding Itô SDE as the time steps go to zero. Because a structured random walk only pins down the first two conditional moments of each increment, the theorem says any zero-mean, unit-variance noise source works in the limit. The paper uses this to argue that the reversed diffusion process, and therefore the likelihood bound and training losses, is invariant to the distribution of the increment $\\Delta x_k$ when the step size is small, so Gaussian increments are a choice, not a requirement.","pith_inferences":["Beyond the paper: the same invariance suggests that noise distributions with finite variance but non-trivial higher moments could inject prior structure into generation, for example sparse noise for sparse data, although the paper only demonstrates Laplace and uniform cases.","Beyond the paper: because the proof relies on finite fourth moments, heavy-tailed increment distributions with infinite variance would be expected to break convergence; testing a Cauchy-increment random walk would directly delineate the boundary of the theorem.","Beyond the paper: the framework implies that the standard score-matching loss is only one point in a continuum of objectives, so a practical next step is to benchmark whether the $\\ell^1$-type losses improve robustness to outlier pixels or adversarial perturbations in generated images.","The paper leaves the statistical guarantees of these alternative objectives open; verifying consistency and asymptotic normality for the non-Gaussian losses would determine whether they are sound estimators beyond their empirical performance."],"forward_implications":["In the small-step limit, the choice of increment family $\\{z_k\\}$ --- Gaussian, Laplace, uniform, or other --- does not change the continuous-time process being sampled.","New training objectives become available: a Laplace/Laplace model yields a loss that behaves like a weighted $\\ell^1$ error on the score residual, while uniform/Laplace pairs produce a piecewise $\\ell^2$/ $\\ell^1$ loss.","Diffusion models can be designed by picking the increment distribution separately from the score-network parameterization, so the noise family is a tunable stylistic and inductive-bias knob.","The experimental results indicate that non-Gaussian increments retain competitive likelihood and sample quality while producing visually different samples, such as more saturated colors for Laplace increments."],"supporting_citations":[{"why":"Supplies the original diffusion-model likelihood formulation that Eq. (5) extends to non-Gaussian increments.","marker":"Sohl-Dickstein et al., 2015"},{"why":"Defines the Gaussian forward process and the epsilon-parameterized loss that Section 4.1 recovers as a special case.","marker":"Ho et al., 2020"},{"why":"Gives the reverse-time SDE, probability-flow ODE, and score-based framework used throughout the paper.","marker":"Song et al., 2020b"},{"why":"Provides the invariance-principle background and Theorem 13.1 used in the proof of Theorem 3.2.","marker":"Billingsley, 2013"},{"why":"Supplies the SDE-solver strong-convergence results applied in Lemma B.5.","marker":"Kloeden & Platen, 1992"},{"why":"Provides the variational diffusion parameterization and training protocol used for likelihood experiments, and is recovered as a Gaussian special case in Corollary 3.1.","marker":"Kingma et al., 2021"},{"why":"Supplies the architecture and training procedure used for FID evaluation.","marker":"Karras et al., 2022"},{"why":"Gives the reverse-time diffusion equation that underlies the generative process.","marker":"Anderson, 1982"}],"fun_headline_variants":["Diffusion needs only mean and variance, not Gaussian","Any noise works: SDE limit depends on moments only","Gaussian not required for diffusion: invariance principle","Mean and variance suffice: non-Gaussian diffusion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All the proposed non-Gaussian loss functions inherit their validity from the likelihood bound in Eq. (5), which is asserted without derivation; if that bound is not a genuine evidence lower bound for arbitrary increment distributions, the new training objectives have no theoretical justification.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion needs only mean and variance, not Gaussian","Any noise works: SDE limit depends on moments only","Gaussian not required for diffusion: invariance principle","Mean and variance suffice: non-Gaussian diffusion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1260,"prompt_tokens":836,"completion_tokens":424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":452,"completion_tokens_details":{"reasoning_tokens":362}},"tokens_in":452,"tokens_out":424,"duration_ms":3774,"temperature":1.0,"reasoning_tokens":362,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:24:24.078972+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a simple Ornstein-Uhlenbeck SDE and simulate structured random walks with $z_k$ drawn from Gaussian, Laplace, uniform, and a two-point distribution with matching mean and variance; if the finite-dimensional distributions at a fixed time $t$ do not all converge to the same OU Gaussian law as $\\Delta t_k \\to 0$, Theorem 3.2 is false. Separately, one can check Eq. (5) directly: for a Laplace-increment model with a fixed score network, compute the left- and right-hand sides by Monte Carlo on a small dataset and see whether the claimed lower bound actually holds.","supporting_citations":[{"cited_title":"Convergence of probability measures","cited_arxiv_id":null,"evidence_quote":"Provides the invariance-principle background and Theorem 13.1 used in the proof of Theorem 3.2."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SDE-solver strong-convergence results applied in Lemma B.5."}],"review_version":1}