{"id":"ed21720e-f60f-4e3e-bf14-1a746321827a","arxiv_id":"2412.07904","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A change of variables formula for score functions enables training diffusion models in one space and sampling in another, and a new nonlinear generalization of sliced score matching.","lead":"This paper derives a formula for how the score function (the gradient of log-density) transforms under a smooth, invertible change of variables, and uses it to extend diffusion models and score matching to transformed spaces. It then introduces a generalized sliced score matching method using random nonlinear projections instead of linear ones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GSSM's un-normalized loss is identifiable only if random gradients span R^n; Assumption 2 (only ∇v≠0) does not guarantee the minimizer equals the true score.","rationale":"I read the paper in good faith and attempted to verify the central mathematical results. Theorem 1 is a direct application of the change-of-variables formula and Jacobi's identity; the scalar corollary and the dimension-reduction Corollary 2 check out on simple test cases. The reverse-time Itô lemma also appears internally consistent: the transform-then-reverse and reverse-then-transform derivations agree, and a concrete example with a time-dependent scaling of Brownian motion matches Anderson's formula directly. The main remaining substantive weakness is in the GSSM application. The paper does not claim to estimate the transformed 1D score directly; it proposes a loss over the original score network. For that loss to identify the true score, the random projection directions ∇v(x) must span R^n at each data point, i.e., E_v[∇v(x)∇v(x)^T] must be positive definite. Assumption 2, which only states ∇v(x) ≠ 0 and integrability, is insufficient; a distribution over v with a fixed projection direction satisfies it and yields a loss that is blind to the orthogonal component of the score. This is not merely a normalization issue: it changes the set of minimizers. The reader's verdict of CONDITIONAL is appropriate, because the gap is fixable by adding an explicit full-rank condition on E[∇v∇v^T] (and, for the quadratic family, verifying it), and the rest of the paper appears sound. I agree with the reader's weakest-assumption identification, and my concrete test directly settles whether the gap is real.","tokens_in":22201,"tokens_out":18815,"duration_ms":167616,"concrete_test":"Fix n=2, take p = N(0, I), and let the random family be v(x) = u^T x + c with u = (1,0) fixed and c ~ N(0,1). Then ∇v(x) = u for every x, so M(x) = uu^T is rank 1. For sθ_w(x) = ∇log p(x) + (0,w), the GSSM objective (74) is independent of w, because both the quadratic term and the divergence term involve only the first component of sθ. Numerically evaluate (74) for w = −2, 0, 2; identical values demonstrate that the minimizer is non-unique and the true score has no privileged status, falsifying the claim that Assumption 2 suffices. Re-running the same check with a family whose gradients span R^2 (e.g., u drawn from a full-rank distribution) should show a unique minimizer at w = 0, confirming that the missing spanning condition is exactly what is needed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the identification condition for Generalized Sliced Score Matching. The derivation in Appendix C starts from the exact transformed-space score matching loss (63), which contains a 1/||∇v(X)||^4 weight. In C.2 the paper replaces this with the un-normalized loss (65), and the final objective (74) is obtained by integration by parts on (65). This replacement is harmless only if one reinterprets (65) as a weighted projected score matching objective: it equals E_{p_d}[ (sθ − ∇log p)^T M(X) (sθ − ∇log p) ], where M(x) = E_v[∇v(x)∇v(x)^T]. For the minimizer to equal the true score, M(x) must be positive definite for p_d-almost every x. Assumption 2 only requires ∇v(x) ≠ 0 almost surely and E||∇v||^2 < ∞; this does not imply positive definiteness. If v is drawn from a family whose gradients all lie in a proper subspace, the loss is constant along that subspace and the score is not identifiable. Thus the stated conditions do not guarantee the central GSSM claim. The main-text equation (14) also has an apparent missing '+' between the third and fourth terms and a malformed second term, but this is presentation; the identification gap is substantive.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper derives a change-of-variables formula for score functions (Theorem 1), uses it to obtain a reverse-time Itô lemma for score-based diffusion models (Corollary 3) and a Generalized Sliced Score Matching method (GSSM). It demonstrates the diffusion result on a chess-position simplex example and GSSM on UCI density estimation tasks. The appendices contain the proofs and additional results.","tokens_in":22459,"tokens_out":11566,"duration_ms":98624,"significance":"If the GSSM identification issue is resolved, the paper offers a useful extension of score matching to nonlinear projections and a theoretically justified method for training in one space and sampling in another. Theorem 1 is a clean identity that could serve as a reference. The paper is honest about limitations and follows established experimental protocols. However, the main new method (GSSM) currently lacks a stated sufficient condition for identifiability, and several derivations in the appendices are not fully rigorous, so the significance is conditional.","major_comments":[{"comment":"The un-normalized loss (65) is a valid score matching objective only if the matrix M(x) = E_v[∇v(x)∇v(x)^T] is positive definite for p_d-almost every x. Assumption 2 only asserts ∇v(x) ≠ 0 almost surely, which does not imply positive definiteness; for example, if ∇v(x) lies in a fixed proper subspace, M(x) is singular and the loss is constant along the orthogonal complement. The quadratic family in Section 4.2 likely satisfies the stronger condition because b has full-rank covariance, but the paper neither states this condition nor proves it for the quadratic family. Without it, the central GSSM claim that minimizing (65) yields the true score is unsupported.","section":"§C.2, Assumption 2, Eq. (65)"},{"comment":"Equation (14) contains a missing '+' between the third and fourth terms, and the second term is written as (∇xv)^T ∇_x(sθ ∇xv), which is ambiguous and not what the appendix derives. The correct expression in Eq. (74) is ∇v^T ∇_x sθ ∇v plus a separate Hessian term sθ^T Hv ∇v. As printed, Eq. (14) is not a well-formed objective and must be corrected for reproducibility.","section":"§3.2.3, Eq. (14)"},{"comment":"The proof of Corollary 2 claims that Eq. (32) is 'trivially true where ∂v/∂x_i = 0', but in that case the right-hand side equals −∂²v/∂x_i², which is generally nonzero. Summing Eq. (32) over all i therefore introduces spurious second-derivative terms for coordinates with zero partial derivative. The proof does not justify the stated formula for v with ∂v/∂x_i = 0 and ∂²v/∂x_i² ≠ 0 at some x. Since Corollary 2 underlies the GSSM derivation, this gap needs to be addressed.","section":"Appendix A.2, Eq. (32)"},{"comment":"Theorem 2 is not a valid statement about score functions: if v: R → R^n, then Y = v(X) is supported on a one-dimensional curve in R^n and has no density with respect to Lebesgue measure, so ∇_y log q(y) is not defined in the usual sense. The component-wise treatment ignores the deterministic dependence among the entries of Y. This result should be removed or substantially revised.","section":"Appendix A.3, Theorem 2"}],"minor_comments":[{"comment":"The identification of the linear case would be clearer if the notation explicitly stated that ∇xv is independent of x when v is linear, so that E_v[v^T∇_x(v^T sθ)] reduces to E_v[v^T(∇sθ)v] as in Eq. (13).","section":"§3.2.3"},{"comment":"The claim that increasing w increases the number of pieces is supported only by the qualitative Figure 5; please report a quantitative measure, such as average piece counts, to substantiate this claim.","section":"§4.1.4"},{"comment":"Test log-likelihoods are reported without error bars or significance tests; given the small UCI datasets, the claim that GSSM-VR outperforms SSM-VR would be stronger with confidence intervals.","section":"Appendix D.4"},{"comment":"There is a typo: 'Guassian' should be 'Gaussian'.","section":"Appendix E.1"},{"comment":"The limitations list acknowledges smoothness and invertibility restrictions, but it could also note that Corollary 2 requires conditional score terms that are not available in practice; the GSSM derivation avoids them only through cancellation, so this should be stated explicitly.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's main theoretical identity (Theorem 1) and its application to reverse-time diffusion (Corollary 3) appear sound, and the experiments are a reasonable first validation. The GSSM identification gap is fixable by stating the positive-definiteness condition on E_v[∇v∇v^T] and verifying it for the quadratic family. The Corollary 2 proof and Theorem 2 issues require careful revision; in particular, Theorem 2 should be removed unless it is reformulated for distributions supported on submanifolds. Overall, a major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real contribution here is the generalized sliced score matching (GSSM) objective, not the score change-of-variables theorem. Theorem 1 is a known identity in the normalizing-flow literature, and presenting it as the main result oversells the paper. That said, the GSSM extension, from linear projections to gradients of smooth functions, is a legitimate and reasonably useful generalization, and the variance-reduced quadratic version is a nice practical addition.\n\nThe GSSM loss in Eq. 74 is correct once you fix the missing '+' in the main-text version of Eq. 14. The derivation is sound, if a bit terse. Experiments on the three UCI datasets with DKEFs show GSSM-VR competitive with SSM-VR, though there are no error bars, and the chess demonstration is qualitative.\n\nThe soft spot is the one the stress-test flags, and it is real. The exact transformed-space score matching loss (63) carries a 1/||∇v||^4 weight. The paper replaces it with the un-normalized loss (65) in Appendix C.2, which corresponds to a weighted projection objective with matrix M(x) = E_v[∇v(x)∇v(x)^T]. For the minimizer to equal the true score, M(x) must be positive definite for pd-almost every x. Assumption 2 only asks for ∇v ≠ 0 and finite second moment; that does not imply full rank. So the identifiability condition is unstated. In the quadratic example it happens to hold — the Rademacher b gives E[bb^T] = I, so M(x) is positive definite whenever x ≠ 0 — so this is a fixable gap, not a fatal flaw. Add the condition, say it holds for the chosen family, and the central claim stands.\n\nOther issues are minor: Remark 2's identity is mis-stated, Eq. 14 has a typo, and the paper doesn't acknowledge that Theorem 1 is standard.\n\nWho is this for? Someone working on score-based density estimation or diffusion in constrained spaces will get a useful tool and a clear target for improvement. It deserves a serious referee, but only after the author adds the missing identifiability assumption and cleans up the presentation. I'd send it back for major revision rather than desk reject.","headline":"GSSM is a genuine extension worth fixing up, but the headline formula is a known identity and the identifiability condition is unstated.","tokens_in":22997,"tokens_out":3178,"would_cite":true,"duration_ms":46513,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A change-of-variables identity for score functions lets diffusion models train in one space and sample in another.","keywords":["score functions","change of variables","score-based diffusion models","reverse-time Itô lemma","sliced score matching","generalized sliced score matching","probability simplex","density estimation"],"falsifier":"Train a score network with the un-normalized generalized sliced score matching loss on a two-dimensional mixture of Gaussians, whose true score is known analytically, using the quadratic transformation $v(x) = \\tfrac{1}{2} x^\\top A x + b^\\top x$ with random symmetric $A$. If the recovered vector field differs from the true score at points where $\\{Ax + b\\}$ does not span $\\mathbb{R}^2$, the un-normalized objective is not a faithful score matching loss; if it matches everywhere, the spanning condition is not the binding issue.","tokens_in":21960,"feed_emoji":"📐","tokens_out":9723,"duration_ms":83623,"temperature":0.7,"pith_summary":"This paper derives a change of variables formula for score functions, the gradients of log-densities that score-based diffusion models estimate. For a smooth invertible map $\\phi$, it claims the score in the transformed space equals the pullback of the original score through the inverse Jacobian plus a divergence correction that accounts for the map's curvature. From this identity it proves a reverse-time Itô lemma, so a diffusion process can be trained in one space and reversed in another without relearning the score in the target space, and it builds a generalized sliced score matching objective that replaces random linear projections with gradients of arbitrary smooth functions. A reader should care because this gives a principled way to apply score-based generative models to constrained geometries, such as sampling on a probability simplex, while keeping training in an unconstrained Euclidean space.","feed_headline":"Diffusion models can train in one space and sample in another","feed_subtitle":"A new identity rewrites score functions under smooth maps, enabling simplex sampling and broader sliced score matching.","key_machinery":"The load-bearing object is the score change-of-variables identity in Theorem 1, an exact formula for how log-density gradients behave under a smooth bijection. It decomposes the transformed score into two pieces: the inverse-Jacobian pullback of the original score, and a divergence term $\\nabla_x \\cdot \\left(J_{\\phi^{-1}}(\\phi(x))^\\top\\right)$ evaluated at $x = \\phi^{-1}(y)$, which is the score-space analogue of the log-determinant correction in the density change-of-variables formula. The dimension-reduction corollary for $y = v(x)$ with $v: \\mathbb{R}^n \\to \\mathbb{R}$ expresses the scalar score through $\\nabla v(x)$, the Laplacian $\\Delta v$, and conditional-score terms, and this corollary is what generates the generalized sliced score matching objective. The reverse-time Itô lemma is the same identity transplanted into the standard time-reversal formula for stochastic differential equations.","core_discovery":"The central claim is that score functions transform by an exact identity: for $y = \\phi(x)$, $\\nabla_y \\log q(y) = J_{\\phi^{-1}}(y)^\\top \\nabla_x \\log p(x) + \\nabla_x \\cdot \\left(J_{\\phi^{-1}}(\\phi(x))^\\top\\right)\\big|_{x=\\phi^{-1}(y)}$. In one dimension this reduces to $(\\phi^{-1})'(y)\\,\\nabla_x \\log p(\\phi^{-1}(y)) + (\\phi^{-1})''(y)/(\\phi^{-1})'(y)$. The paper uses this identity to show that reversing the transformed forward SDE yields the same law as transforming the reversed process, so training a score network in $x$-space and sampling in $y$-space is pathwise equivalent to training directly in $y$-space. It also derives a dimension-reduction version for scalar functions $v$, and claims the resulting generalized sliced score matching loss—with extra Hessian and Laplacian terms—is a valid score matching objective that reduces to standard sliced score matching when $v$ is linear.","pith_inferences":["The identity would transfer score-based samplers other than diffusion—for example Langevin dynamics or annealed importance sampling—to constrained spaces whenever the map is smooth and invertible and the Jacobian-divergence correction is computable.","Because the paper substitutes an un-normalized generalized sliced score matching loss for the exact $1/\\|\\nabla v\\|^4$-weighted score-matching loss, a natural empirical check is to compare the recovered score against an analytically known score in low dimensions; the paper's experiments compare loss values rather than recovered score fields.","The scalar factor in the chess experiment acts as a temperature-like control over piece density, so it could be reinterpreted as annealing the target distribution rather than merely rescaling the drift.","Learned smooth maps (normalizing flows) could replace hand-derived transformations in the reverse-time lemma, trading analytic Jacobians for automatic differentiation, with Hessian-vector products as the main cost."],"forward_implications":["A diffusion model trained in one coordinate system can be sampled in another: the transformed reverse drift is written entirely in terms of the original score, the Jacobian, and the Hessian tensor of $\\phi$, with no separate network in the target space.","The reverse-time Itô lemma gives a pathwise equivalence: reversing the transformed forward process yields the same law as transforming the reversed process.","Generalized sliced score matching strictly generalizes sliced score matching: when $v(x) = v^\\top x$ the Hessian and Laplacian terms vanish and the loss reduces to the standard sliced score matching objective.","With quadratic random functions, the variance-reduced version has a closed-form quadratic loss for kernel exponential family models, and the reported experiments show lower score-matching loss than variance-reduced sliced score matching on two of three benchmark datasets.","On the probability simplex, the additive logistic transformation lets a Gaussian variance-preserving SDE trained in $\\mathbb{R}^{12}$ generate valid chess-position representations, with a scalar factor controlling the number of pieces."],"supporting_citations":[{"why":"Supplies the identity $\\nabla \\cdot (\\operatorname{Cof}(J_f)) = 0$ used in the proof of Theorem 1.","marker":"[12]"},{"why":"Supplies the classical change-of-variables formula for densities that the score version extends.","marker":"[15]"},{"why":"Supplies the score matching loss that generalized sliced score matching generalizes.","marker":"[18]"},{"why":"Supplies sliced score matching, the linear-projection method and baseline that generalized sliced score matching extends.","marker":"[29]"},{"why":"Supplies denoising score matching used to train the diffusion score network.","marker":"[32]"},{"why":"Supplies the score-based diffusion SDE framework and the variance-preserving SDE used in experiments.","marker":"[31]"},{"why":"Supplies the reverse-time SDE formula on which the reverse-time Itô lemma is built.","marker":"[1]"},{"why":"Supplies the Itô lemma for transformed processes used to compute transformed drift and diffusion.","marker":"[20]"},{"why":"Supplies the prior probability-simplex diffusion approach and baseline for the chess example.","marker":"[14]"},{"why":"Supplies the additive logistic transformation that maps $\\mathbb{R}^{12}$ to the probability simplex.","marker":"[3]"}],"fun_headline_variants":["Train in X, sample in Y: score identity unlocks diffusion","Score change of variables enables cross-space sampling","Generalized sliced score matching via smooth mappings","Reverse-time Itô lemma from score transformation","Diffusion decoupled: one score for two spaces"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's generalized sliced score matching objective swaps an exactly weighted loss for an un-normalized one; the two have the same minimizer only if the random projection directions span the whole space at every data point, a condition the paper assumes without proof.","fun_headline_variants_meta":{"raw":{"variants":["Train in X, sample in Y: score identity unlocks diffusion","Score change of variables enables cross-space sampling","Generalized sliced score matching via smooth mappings","Reverse-time Itô lemma from score transformation","Diffusion decoupled: one score for two spaces"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1571,"prompt_tokens":959,"completion_tokens":612,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":540}},"tokens_in":575,"tokens_out":612,"duration_ms":6420,"temperature":1.0,"reasoning_tokens":540,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:27:39.511112+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a score network with the un-normalized generalized sliced score matching loss on a two-dimensional mixture of Gaussians, whose true score is known analytically, using the quadratic transformation $v(x) = \\tfrac{1}{2} x^\\top A x + b^\\top x$ with random symmetric $A$. If the recovered vector field differs from the true score at points where $\\{Ax + b\\}$ does not span $\\mathbb{R}^2$, the un-normalized objective is not a faithful score matching loss; if it matches everywhere, the spanning condition is not the binding issue.","supporting_citations":[{"cited_title":"Partial differential equations, volume 19","cited_arxiv_id":null,"evidence_quote":"Supplies the identity $\\nabla \\cdot (\\operatorname{Cof}(J_f)) = 0$ used in the proof of Theorem 1."},{"cited_title":"Sliced score matching: A scalable approach to density and score estimation, 2019","cited_arxiv_id":null,"evidence_quote":"Supplies sliced score matching, the linear-projection method and baseline that generalized sliced score matching extends."},{"cited_title":"Brownian motion and stochastic calculus, volume 113","cited_arxiv_id":null,"evidence_quote":"Supplies the Itô lemma for transformed processes used to compute transformed drift and diffusion."},{"cited_title":"Diffusion on the probability simplex, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the prior probability-simplex diffusion approach and baseline for the chess example."},{"cited_title":"Logistic-normal distributions: Some properties and uses","cited_arxiv_id":null,"evidence_quote":"Supplies the additive logistic transformation that maps $\\mathbb{R}^{12}$ to the probability simplex."}],"review_version":1}