{"id":"6b7c1956-08f8-4704-9274-d35a56d1c703","arxiv_id":"2509.00336","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Diffusion models learn a velocity field that matches the Wasserstein gradient flow of the OU process, not a true score, and errors in the Stein null space leave density transport exact.","lead":"DNNs that denoise images output vector fields that fail the mathematical tests every true score function must pass, yet the models still generate sharp samples. The paper argues diffusion training is really flow matching to a Wasserstein gradient flow, a view that makes the 'probability flow' natural and explains why broken vector fields can still transport density.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fig. 3's null-kernel claim relies on the true marginal score, which is intractable for image data; if the paper used the conditional score instead, eq. (20) no longer supports the central explanation.","rationale":"The reader's weakest-assumption analysis identifies the same load-bearing concern: the empirical support for the null-kernel claim depends on a true marginal score that is not available. My reading confirms this is the most critical weak point. The paper's direct evidence that the learned vector field is non-conservative (integral and differential constraint violations) does not depend on the marginal score and is credible. However, the novel theoretical contribution—that non-conservative errors can be harmless because they lie in the null kernel of the Stein operator—is supported only by Fig. 3, which requires e relative to the true marginal score. Since that score is intractable for the evaluated datasets, and since the paper does not describe how it was obtained, the null-kernel claim is currently unsupported. The paper's own footnote concedes the pointwise condition fails per-sample, further weakening the empirical basis. The correct verdict remains conditional: the theoretical framing is reasonable, but the empirical cornerstone needs to be re-run with a tractable marginal score or a clearly described estimator. No verdict adjustment is needed because the reader already assigned CONDITIONAL.","tokens_in":9389,"tokens_out":5533,"duration_ms":69929,"concrete_test":"Run a synthetic Gaussian-mixture experiment (e.g., 10 components in 64 dimensions) where the marginal score is exactly computable. Train a small DDPM. On the same forward samples, compute e_true = s_marginal - vθ and e_cond = s_conditional - vθ. Evaluate |∇·e + s_marginal^T e| for both. If e_cond appears to satisfy the null-kernel condition while e_true does not, Fig. 3's confinement result is an artifact of using conditional scores. Additionally, on MNIST, estimate s_marginal by Monte Carlo averaging over many clean samples in the Gaussian-mixture expression and repeat the Fig. 3 analysis pointwise on a dense grid, not averaged over 256 samples.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central explanation for why non-conservative errors are harmless rests on Fig. 3 and eq. (20). Eq. (20) is derived for e = s_marginal - vθ, where s_marginal = ∇ log ρ(x,t) is the score of the true forward marginal density. Section 3, item 2 states that s(x,t) is 'analytically exact' because the OU process is used to generate samples. For a single initial sample x0, the OU transition kernel yields the conditional score ∇ log p_t(x|x0), not the marginal score ∇ log ρ(x,t). For MNIST, CIFAR-10, or a VAE latent space, the marginal score is a mixture over the data distribution and is intractable. If the e used in Fig. 3 was computed against the conditional score—the actual DDPM/flow-matching regression target—then the operator in (19)-(20) is evaluated with the wrong s, and the null-kernel condition no longer controls marginal density transport. Footnote 5 further concedes that the pointwise condition (20) fails per-sample and is only met on average over 256 samples, but condition (20) is required pointwise to keep the transport exact. Without a valid estimate of e relative to the true marginal score, Fig. 3 does not establish that trained networks are confined to the Stein null kernel, undermining the key theoretical explanation of the paradox.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper challenges the standard interpretation of diffusion models as learning the score function. It presents numerical evidence (MNIST, latent MNIST, and CIFAR-10) that trained diffusion networks violate both integral and differential constraints that true score functions must satisfy, and argues that the learned vector fields are non-conservative. To explain why such networks still generate good samples, the paper advocates viewing diffusion training as flow matching to the velocity field of a Wasserstein Gradient Flow (WGF), rather than as learning the reverse-time SDE. The authors derive a sufficient condition—Equation (20)—under which a non-conservative error field e(x,t) does not affect the marginal density transport, and report in Figure 3 that the error field of a trained latent diffusion model is approximately confined to the null kernel of the associated Stein operator. The central conceptual claim is that diffusion models are better understood as deterministic flow matching on the WGF, with the reverse-time SDE being unnecessary at the level of marginal densities.","tokens_in":9655,"tokens_out":5701,"duration_ms":73748,"significance":"If the central claim holds, the paper offers a principled explanation for a puzzling observation: neural networks that fail to be conservative can still generate high-quality samples. The WGF/flow-matching perspective is elegant, connects diffusion models to a well-developed mathematical framework, and shifts attention from path-wise reverse-time SDE theory to marginal density transport. The derivation of the error-transport condition is straightforward and correct, and the numerical tests for non-conservativity are a valuable falsifiable check on common assumptions. However, the empirical support for the key null-kernel claim currently depends on a quantity—the true marginal score—that is intractable for real data and is not clearly defined in the manuscript. The paper's contribution is therefore promising but not yet fully established.","major_comments":[{"comment":"The paper states that the OU process is used to 'compute analytically exact s(x,t)' for matching the neural velocity field. This is only true for the conditional score ∇ log p_t(x|x0). For a data distribution that is an empirical mixture (MNIST, CIFAR-10, VAE latent), the marginal score s(x,t) = ∇ log ∫ p_t(x|x0) dµ0(x0) is intractable. If the error e in Figure 3 and Eq. (20) is computed against the conditional score—the actual DDPM regression target—then Eq. (20) no longer governs marginal density transport. The manuscript must specify exactly which score was used and how it was computed. If only the conditional score is available, the null-kernel conclusion is not supported.","section":"Section 3, item 2; Figure 3; Eq. (20)"},{"comment":"There is a mismatch between the stated WGF velocity and the flow-matching objective. Eq. (9) defines v_WGF = -x - s(x,t), but Eq. (15) minimizes E ||v_θ - s||^2. If v_θ is the full velocity field, the target should be -x - s; if v_θ is the score network, then the connection to WGF velocity is incomplete. This inconsistency affects the central formulation and needs to be corrected or clarified.","section":"Section 3, Eq. (15)"},{"comment":"Eq. (20) is a pointwise condition that must hold for each x,t to guarantee exact density transport. Footnote 5 concedes that for each sample the condition 'does not seem to be met' and only the average over 256 samples agrees. The claim that trained networks are 'statistically confined in the null kernel' is therefore not established. The authors should report the distribution of the Stein-operator value, the fraction of samples that satisfy the condition within a tolerance, and how violations affect the transported density.","section":"Footnote 5, Eq. (20)"},{"comment":"The empirical evidence for the null-kernel claim lacks baselines and error bars. A comparison with a random non-conservative field, or with a field constructed to satisfy Eq. (20), would show whether the observed average behavior is meaningful. As written, the figure only demonstrates that the average Stein value is small relative to the error magnitude; it does not show that the trained network is special in this respect.","section":"Figure 3; Section 3"}],"minor_comments":[{"comment":"In Eq. (17), '∇ x log(x(τ), −τ)' is missing the density argument; it should be ∇ log ρ(x(τ), −τ) or similar.","section":"Eq. (17)"},{"comment":"Typo: 'wether' should be 'whether'.","section":"Section 2, introduction"},{"comment":"After Eq. (8), '∇x log(·, t)' should read '∇x log ρ(·, t)' for clarity.","section":"Eq. (8)-Eq. (9)"},{"comment":"Error bars or confidence intervals should be reported; currently the figures show only point statistics, which makes it difficult to judge the significance of the reported violations.","section":"Figure 2 and Figure 3"},{"comment":"Reference [17] is a Hugging Face Spaces link, not a stable archival source. Please provide an archival reference or versioned code release for reproducibility.","section":"Appendix; reference [17]"},{"comment":"The statement that DDPM's uniform time weighting 'effectively solves a flow-matching problem' is plausible but needs a more careful citation or derivation; the connection between time weighting and flow-matching loss is asserted rather than shown.","section":"Section 3, paragraph after Eq. (15)"}],"recommendation":"major_revision","confidential_remarks":"The paper has an interesting central idea and the mathematical skeleton is largely sound. The main concern is that the key empirical claim—that trained networks' errors are confined to the null kernel of the Stein operator—rests on the use of the true marginal score, which is not available for the tested datasets. If the authors can clarify how s was obtained, or re-cast the analysis so that it does not require intractable marginal scores, the contribution could be suitable. As written, the load-bearing empirical support is not yet convincing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know: the paper's math is sound but mostly re-derives known results. What's actually new is the empirical claim that trained diffusion networks violate the conservative-field constraints of a true score, and the proposed explanation via the null kernel of the Stein operator. The null-kernel evidence, however, does not hold up as presented.\n\nThe derivation is clean: the WGF velocity v = -x - ∇log ρ is exactly the probability flow ODE from Song et al., and the paper says so. Condition (20) is a one-line identity from the continuity equation, not a fitted result. The authors are also honest in Section 3 that Anderson's reverse-time SDE is still needed for path measures; only marginal densities are claimed. That restraint is appreciated.\n\nThe empirical diagnostics in Fig. 2 are a real contribution: measuring line integrals and Jacobian asymmetries of trained networks at scale, and showing they deviate from zero. But they lack baselines or error bars, so magnitude is hard to interpret.\n\nThe load-bearing problem is Fig. 3. The error field is defined relative to the true marginal score s(x,t) = ∇log ρ(x,t). For MNIST or CIFAR, that marginal score is a mixture over the data distribution and intractable. The paper says s is 'analytically exact' because the OU process is used, but that gives the conditional score ∇log p_t(x|x0), not the marginal. If the e plotted in Fig. 3 was computed against the conditional score—which is the actual regression target—then eq. (20) no longer controls marginal density transport, and the null-kernel explanation collapses. Footnote 5 adds that the condition only holds on average over 256 samples, not per-sample, but eq. (20) is required pointwise. So the central empirical support for 'errors live in the null kernel' is not established.\n\nThe novelty is modest: the WGF framing is a useful re-packaging, and the paper explicitly acknowledges prior work. I would not cite it for new math, but I would cite the constraint-violation measurements if I were writing about score approximation.\n\nWho is this for? Someone thinking about why diffusion sampling works despite imperfect score estimation. The theoretical framework is a helpful lens, but the key experiment needs rework. I'd send it to review—the question is important and the authors think carefully—but I'd expect major revision on Fig. 3 and a proper description of how s was computed.","headline":"Useful empirical observation and a clean re-derivation of known equivalence, but the load-bearing null-kernel claim rests on an intractable marginal score and is not established.","tokens_in":10214,"tokens_out":2419,"would_cite":true,"duration_ms":29214,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that diffusion training actually performs flow matching to the Wasserstein gradient flow velocity, not score learning.","keywords":["diffusion models","score function","Wasserstein gradient flow","flow matching","conservative vector fields","Stein operator","probability flow","normalizing flows"],"falsifier":"Take a trained model, compute the true marginal score s(x,t) by a high-accuracy method for representative forward samples, and evaluate the Stein expression div(s-NN) + s^T(s-NN) pointwise. If it is non-zero well beyond Monte Carlo error while sampling quality remains high, the null-kernel explanation fails; if the closed-loop integral (11) is zero to numerical precision, the empirical premise that networks are non-conservative collapses.","tokens_in":9203,"feed_emoji":"🌊","tokens_out":7276,"duration_ms":79038,"temperature":0.7,"pith_summary":"The paper asks whether the vector field learned by a diffusion model is actually the score function, the gradient of the log-density of noisy data. It reports numerical evidence from MNIST, a latent MNIST variant, and CIFAR-10 that trained networks violate both the line-integral and the cross-derivative constraints that every true gradient field must obey. Despite that, these networks generate samples well. The paper's resolution is that diffusion training should be understood as flow matching to the velocity field of a Wasserstein gradient flow (WGF), not as learning the reverse-time stochastic differential equation of [2]. On that view, sampling is just integrating a deterministic ODE backward in time, and approximation errors that lie in the null space of a Stein-like operator do not corrupt the transported density, which reconciles the non-conservative fields with successful generation.","feed_headline":"Trained diffusion nets fail score checks yet generate well","feed_subtitle":"The paper recasts them as flow matching to a Wasserstein gradient flow; non-conservative errors can still transport density exactly.","key_machinery":"The load-bearing object is the WGF velocity v_WGF(x,t) = -x - grad log rho(x,t), the steepest-descent direction for an energy combining quadratic potential and negative entropy; it is the target of the flow-matching regression. The second mechanism is the Stein-type operator acting on the approximation error e(x,t) = s(x,t) - NN(x,t): the condition 0 = div e + s^T e defines the null kernel in which a non-conservative error leaves the marginal density unchanged.","core_discovery":"The central claim is that \"existing diffusion modeling is better understood as modeling a normalizing flow, through performing flow matching to the WGF velocity (9), rather than learning the reverse stochastic differential equation established by [2].\" Concretely, the training objective is an L2 regression of the neural field v_theta(t,x) against v_WGF(x,t) = -x - grad log rho(x,t), and sampling integrates dx/dtau = x + NN(x,-tau) backward in time. Because this is deterministic transport of the marginal density, Anderson's reverse-time SDE [2] is only needed for path-level equivalence, not for the marginals that generation cares about. The paper also derives a sufficient condition for exact","pith_inferences":["A testable corollary: explicitly enforcing the null-kernel condition as a regularizer should not hurt generation if the condition is what makes non-conservative transport work; if it hurts, the residual is being absorbed elsewhere in the dynamics.","The same non-conservative-but-null-kernel argument likely applies to other interpolant-based generative models, so similar violations of conservative constraints may be expected there too.","If training is really flow matching with uniform time weights, then reweighting the loss by likelihood-based importance or by noise level may be unnecessary for sample quality; this could simplify schedule design."],"forward_implications":["Sampling from a diffusion model can be a deterministic ODE; the reverse-time SDE of [2] is not required for marginal-level generation.","A neural field that fails the conservative constraints of a true score can still be a perfect generative model, provided its error stays in the Stein-null kernel.","The uniform time weighting used in DDPM-style training [10] is natural: it is the flow-matching objective, not a variation of a likelihood bound.","The 'probability flow' of [20] is intrinsic to WGF theory; identifying diffusion training as WGF flow matching gives a variational basis for it."],"supporting_citations":[{"why":"Provides the reverse-time SDE construction that the paper argues is unnecessary at the marginal level; the baseline being reinterpreted.","marker":"[2]"},{"why":"Introduces the variational (JKO) formulation that grounds the Wasserstein gradient flow interpretation.","marker":"[11]"},{"why":"Defines the Riemannian/Wasserstein gradient flow geometry used to derive the velocity field.","marker":"[15]"},{"why":"Supplies the flow-matching training framework that the paper claims diffusion training actually implements.","marker":"[12]"},{"why":"Popularized score-based SDE diffusion models and the 'probability flow' ODE that the paper reframes via WGF.","marker":"[20]"},{"why":"Provides the DDPM training loss, which the paper rereads as a uniform-weighted flow-matching objective.","marker":"[10]"},{"why":"Introduces the Stein operator whose null kernel the paper uses to characterize harmless approximation errors.","marker":"[13]"}],"fun_headline_variants":["Diffusion models don't learn scores but still work","Diffusion models aren't learning scores but flow matching","Score learning is wrong: diffusion models learn WGF velocity","Non-conservative errors don't break diffusion generation","Diffusion models are actually flow matching, not score learning"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The analysis assumes the true marginal score of the noisy data is known for each forward sample and that averaging the Stein condition over 256 samples tests the pointwise condition; if the score used is only the conditional score, the null-kernel error bound no longer controls marginal density transport.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion models don't learn scores but still work","Diffusion models aren't learning scores but flow matching","Score learning is wrong: diffusion models learn WGF velocity","Non-conservative errors don't break diffusion generation","Diffusion models are actually flow matching, not score learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000737,"raw_usage":{"total_tokens":3132,"prompt_tokens":750,"completion_tokens":2382,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":2318}},"tokens_in":494,"tokens_out":2382,"duration_ms":19097,"temperature":1.0,"reasoning_tokens":2318,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:42:31.365522+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained model, compute the true marginal score s(x,t) by a high-accuracy method for representative forward samples, and evaluate the Stein expression div(s-NN) + s^T(s-NN) pointwise. If it is non-zero well beyond Monte Carlo error while sampling quality remains high, the null-kernel explanation fails; if the closed-loop integral (11) is zero to numerical precision, the empirical premise that networks are non-conservative collapses.","supporting_citations":[{"cited_title":"Reverse-time diffusion equation models","cited_arxiv_id":null,"evidence_quote":"Provides the reverse-time SDE construction that the paper argues is unnecessary at the marginal level; the baseline being reinterpreted."},{"cited_title":"Flow Matching for Generative Modeling","cited_arxiv_id":null,"evidence_quote":"Supplies the flow-matching training framework that the paper claims diffusion training actually implements."},{"cited_title":"Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm","cited_arxiv_id":null,"evidence_quote":"Introduces the Stein operator whose null kernel the paper uses to characterize harmless approximation errors."}],"review_version":1}