{"id":"b249f24c-e600-4303-bc98-25f2307a1046","arxiv_id":"2502.02300","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Conditional Time Score Matching estimates density ratios by regressing closed-form conditional time scores, giving faster learning and theoretical error bounds.","lead":"This paper presents a faster estimator for density ratios, built by regressing a neural network on closed-form 'conditional time scores' along an interpolating path between two distributions. The method, called Conditional Time Score Matching, reduces per-step cost by several times and reaches competitive accuracy on synthetic and image benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's TSM=CTSM equality needs regularity conditions: for the VP path the conditional time score has infinite variance at t→1, so with uniform weighting the CTSM loss is infinite and the claimed additive constant is not finite.","rationale":"The central claim of the paper is that minimizing CTSM regresses the marginal time score, and that this is equivalent to TSM up to an additive constant. This rests entirely on the mixture identity Eq. 7, whose validity requires differentiating under the integral in Appendix D.1 and finiteness of the resulting losses. The reader identified the lack of regularity conditions as the weakest assumption; my concern sharpens this into a concrete failure mode: for the VP path used throughout, the conditional time score has infinite second moment at the endpoint, making the CTSM loss infinite under uniform weighting. The identical-Gaussians example is a stark counterexample: the marginal time score is zero (TSM loss finite), yet the conditional regression targets have divergent variance (CTSM loss infinite), so Theorem 1 cannot hold as stated for arbitrary positive λ. This is a genuine correctness gap in the theoretical contribution, not merely a missing footnote. However, the paper's actual method uses time-score normalization and ε-clipping, which tame the singularity, so the practical estimator is not obviously invalidated. The core conditional-expectation identity is standard and correctly applied; the code is provided; and the empirical speedups are plausible. Thus the appropriate outcome is to require the authors to state the needed regularity conditions and to formulate Theorem 1, Theorem 4, and Proposition 5 on a compact time interval [ε,1-ε] with a weighting that guarantees finite losses. This is exactly a conditional acceptance, matching the reader's verdict, so no change to the verdict is needed.","tokens_in":30784,"tokens_out":13374,"duration_ms":132483,"concrete_test":"Analytically verify for the VP path α_t=t, p0=p1=N(0,I), that the conditional time score variance from Eq. 65 (with c=1) is Var[f_t] ≈ D/(2(1-t)^2) near t=1, and hence ∫_0^1 Var[f_t]dt diverges. Since the true marginal time score is zero, this shows L_CTSM(θ) with λ=1 is infinite for all θ while L_TSM is finite, contradicting the literal statement of Theorem 1. Then confirm that with time-score normalization λ(t)∝1/Var[f_t] and t restricted to [ε,1-ε], the CTSM loss is finite and its minimizer is the zero score, demonstrating the fix.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eq. 7, used to prove Theorem 1 (and Theorem 3), requires differentiating under the integral in Eq. 78 without stated conditions. For the paper's own VP path (α_t=t), the conditional time score f_t(x,z)=∂t log pt(x|z) in Eq. 15/62 has variance (Eq. 65) of order D/(2(1-t)^2) as t→1. Therefore ∫_0^1 E_{p(z),pt(x|z)}[f_t^2]dt is infinite, so the CTSM loss Eq. 8 with any positive λ that does not vanish sufficiently fast (e.g., λ=1) is infinite for every θ. In the simple case p0=p1=N(0,I), the marginal time score is identically 0, so the TSM loss Eq. 4 is finite while the CTSM loss is infinite; the 'additive constant' in Theorem 1 would have to be infinite, and the gradient proof in Appendix D.2 is not valid when the loss is not finite. The paper's practical safeguards (ε-clipping, time-score normalization Eq. 20-21) avoid the singularity, but Theorem 1, Theorem 4, and Proposition 5 are stated without these restrictions, leaving the central theoretical claim incomplete.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes new objectives, Conditional Time Score Matching (CTSM) and its vectorized variant (CTSM-v), for estimating the time score ∂t log pt(x) along a probability path connecting two densities, and uses the integrated time score to estimate the density ratio log p1(x)/p0(x). The objectives are obtained by conditioning on a latent variable z so that the conditional time score is available in closed form; the paper proves that the TSM and CTSM losses have the same minimizer up to an additive constant (Theorems 1--3), gives a general error bound and an asymptotic M-estimation bound (Theorem 4 and Proposition 5), and reports experiments on Gaussians, Gaussian mixtures, mutual information estimation, and MNIST energy-based modeling, claiming substantial speedups and competitive or better accuracy relative to TSM.","tokens_in":31002,"tokens_out":10690,"duration_ms":105928,"significance":"The core idea---regressing conditional time scores rather than the marginal time score---is a natural and useful extension of denoising score matching and flow matching to density ratio estimation, and Theorem 3 generalizes the known conditional-score identity to arbitrary functions. If the theoretical statements are made rigorous with appropriate regularity conditions, and if the empirical comparison is properly controlled, the contribution would be valuable: it offers a tractable objective that avoids the double backpropagation required by TSM, provides closed-form weighting functions for VP and Schrödinger bridge paths, and supplies formal error guarantees that prior work in this line (Rhodes et al., 2020; Choi et al., 2022) lacked. The release of code and the detailed appendices are strengths. However, as stated, the theoretical results are incomplete and partly not applicable to the paper's own VP path, and the main experimental comparison does not isolate the proposed objective from the concurrent change of weighting function.","major_comments":[{"comment":"The equality between the TSM loss (Eq. 4) and the CTSM loss (Eq. 8) requires the conditional time score to have finite second moments, but for the VP path with αt = t used throughout the paper this fails. Equation (65) gives Var[∂t log pt(x|z)] = O((1−t)^{-2}) as t → 1, so with uniform t and λ(t)=1 the CTSM loss is infinite for every θ. As a concrete counterexample, for p0 = p1 = N(0, I) the marginal time score is identically zero and the TSM loss is finite, while the CTSM loss is infinite, so the claimed finite additive constant in Theorem 1 cannot exist; the gradient proof in D.2 is not valid for infinite losses. The subsequent theorems that rely on the same identity (Theorems 2 and 3, and through them Proposition 5) inherit this issue. The paper's practical safeguards (ε-clipping, time score normalization Eq. 20--21, importance sampling with t1 = 0.9) avoid the singularity, but the theorems are stated without these restrictions. The statements should be revised to hold on a truncated interval t ∈ [0, 1−ε], or under a weighting λ(t) that vanishes sufficiently fast near t=1, or under explicit finite-moment assumptions, and the proofs should justify differentiating under the integral in Eq. (78).","section":"§3.1, Theorem 1 / Appendix D.1--D.2"},{"comment":"Proposition 5 is an M-estimation result, but its validity requires the population loss to be finite and the gradient covariance G(θ*) in Eq. (114) to be finite. As noted above, for the VP path the residual w(x,z,t) = ∂t log pt(x|z) − ∂t log pt(x) has infinite second moment near t=1 under uniform weighting, so G(θ*) is infinite and the asymptotic expansion √N(θ̂ − θ*) ⇒ N(0, H^{-1} G H^{-1}) is not justified. Even if one restricts to a truncated time interval, the proposition should explicitly state the regularity conditions (finite second moments of the conditional time score, interchange of differentiation and integration, well-specified model, and standard M-estimation conditions) rather than leaving them implicit in the proof. Without these, the claimed o(1/N) rate for the final density-ratio error is not established for the paper's own running example.","section":"§5, Proposition 5 / Eq. (114)"},{"comment":"The theorem is stated as a bound on KL(p1, p̂1), but Eq. (23) defines log p̂1(x) as an integral of an estimated time score plus log p0(x), which does not in general produce a normalized density. For unnormalized p̂1, the quantity E_{p1}[log p1 − log p̂1] is not the KL divergence; the proof in D.3 actually bounds the squared pointwise error of the log-density ratio, E_{p1}[(log p1(x) − log p̂1(x))^2]. The theorem should either be restated for this squared log-ratio error, or a normalization-constant term should be carried through. Separately, the proof of the discretization term uses the left rectangular rule with K equal subintervals, but Eq. (23) does not specify that the ti form a uniform grid; if the ti are arbitrary or sampled (e.g., from an importance distribution as in §4), the bound L(x)/(2K) in Eq. (24) does not follow. Both issues affect Proposition 5, which invokes Theorem 4.","section":"§5, Theorem 4 / Eq. (24) and Appendix D.3"},{"comment":"The empirical comparison is confounded by a concurrent change of weighting function. Throughout Section 6, TSM is trained with Stein score normalization (Eq. 19, λ(t) ∝ kt) while CTSM and CTSM-v are trained with time score normalization (Eq. 20--21), and in the EBM experiments the CTSM-v architecture additionally differs from TSM's (Fourier feature embeddings vs. linear embeddings, and a different U-Net in the ambient-space experiments). Since the authors declare time score normalization to be an integral part of CTSM, the observed accuracy differences cannot be attributed solely to the proposed objective. Appendix Table 4 provides a partial TSM-with-time-normalization comparison only for the Gaussian toy problem, which helps but is not the main text comparison. The main claims would be supported by reporting TSM+Time normalization in the main figures, or by adding an ablation in which CTSM uses Stein normalization, and by separating the effect of the vectorized objective from the effect of the weighting and architecture changes.","section":"§6, Tables 2--3 and Figure 2"}],"minor_comments":[{"comment":"The variance formula in Eq. (65) is missing a factor of D that follows from Lemma 6: the variance of the scalar time score is proportional to D. Since D does not depend on t, the proportional weighting in Eq. (21) is unaffected, but the displayed formula is incorrect as a standalone identity.","section":"Appendix B.1, Eq. (65)"},{"comment":"The notation xi and x<i in the definition of the vectorized conditional time score is introduced informally; for the VP path the components do not depend on x<i, but this should be stated explicitly to avoid confusion about the ordering of the autoregressive decomposition.","section":"§3.2, Eq. (9)"},{"comment":"Proposition 5 should state explicitly that the model is assumed to be well-specified (existence of θ* with s_{θ*}(x,t) = ∂t log pt(x)) and that standard M-estimation regularity conditions hold; these are currently only mentioned inside the proof.","section":"§5, Proposition 5"},{"comment":"The statement that 'in practice, CTSM could not be used for this task' is reported without supporting results; since the paper advocates the vectorized variant, a brief explanation or a reference to preliminary experiments would help the reader assess the limitation of the non-vectorized objective.","section":"§6.4"},{"comment":"The definition of the time score normalization for CTSM-v is said to assume each dimension has equal scale; the manuscript would benefit from stating what is done when this assumption is violated and how the scalar weighting λ̂(t) is applied to the vectorized objective.","section":"§4, Eq. (21)"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is promising and the empirical speedups are striking, but the theoretical claims as currently stated are not valid for the paper's own VP path without additional assumptions, and the empirical comparison mixes the proposed objective with a change of weighting function. These issues are fixable within the scope of a revision: the theorems can be restated on a truncated time interval or with finite-moment conditions, the KL statement can be corrected to a squared log-ratio error bound, and the experiments can be supplemented with controlled ablations. I would not reject the paper, but the current version should not be accepted as the theoretical guarantees and the attribution of empirical gains stand."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core move here is simple and useful: use the conditional-expectation identity to regress on the conditional time score instead of the intractable marginal time score, then integrate over time to get the log density ratio. Applied to time scores this is new as far as I know, and the method delivers credible speedups over TSM. The per-step cost is genuinely lower, the mutual information results converge much faster, and the code is public. The authors also deserve credit for stating the BPD caveat for unnormalized models instead of hiding it.\n\nThe general framework in Theorem 3 is a restatement of the standard denoising/flow-matching regression identity, and the authors acknowledge that lineage; their novelty claim is only about the time-score application, which is fair. The proofs in Appendix D are coherent, and Theorem 4 and Proposition 5 are standard M-estimation bounds with assumptions stated.\n\nThe soft spot is real and load-bearing. Equation (7) requires differentiating under the integral, and no regularity conditions are given. For the paper's own VP path with alpha_t = t, the conditional time score has variance proportional to 1/(1-t)^2 near t=1, so the CTSM loss with uniform weighting is infinite for every theta. The stress-test example works: if p0 = p1 = N(0,I), the marginal time score is identically zero, so the TSM loss is finite, while the CTSM loss with lambda=1 is infinite. The claimed additive constant in Theorem 1 is then not finite. The paper's practical safeguards--epsilon clipping and the time-score normalization--sidestep the singularity, but the theorems are stated without these restrictions. This is fixable by requiring lambda(t)(1-t)^2 to be integrable near t=1, or by restricting t to [epsilon, 1-epsilon], but it has to be fixed before the theoretical guarantees are taken at face value.\n\nThe empirical comparison is somewhat confounded: CTSM-v differs from TSM in weighting, architecture details, importance sampling, and vectorization, so the exact source of the gain is not isolated. The appendix ablations help, and the speed advantage is robust because it comes from avoiding second derivatives in t, so this is a moderate concern, not a fatal one.\n\nWho is this for: researchers doing density ratio estimation, mutual information estimation, or EBM training. It deserves a serious referee: the empirical contribution is solid, the code is available, and the theoretical gap is patchable. I would send it to review and ask for a revision that states the missing conditions and re-examines the theorem statements.","headline":"A useful DRE trick with an overreaching theory: the CTSM objective works in practice, but the stated equivalence theorem needs regularity conditions it does not have.","tokens_in":31589,"tokens_out":3215,"would_cite":true,"duration_ms":34920,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G07","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"Density ratio estimation can be made faster and more accurate by learning the time score through conditional probability paths; the paper proves the conditional objective is equivalent to the original up to a constant.","keywords":["density ratio estimation","time score matching","conditional probability paths","score matching","flow matching","mutual information estimation","energy-based modeling","theoretical error bounds"],"falsifier":"Construct $p_0$ and $p_1$ with a known ratio and choose a path that is not a mixture of Gaussian conditionals, such as $p_t(x) = (1-t)p_0(x) + t p_1(x)$; train the CTSM objective, integrate the resulting score, and compare with the true $\\log(p_1/p_0)$. A systematic discrepancy would show the mixture identity is load-bearing.","tokens_in":30548,"feed_emoji":"📊","tokens_out":7346,"duration_ms":67911,"temperature":0.7,"pith_summary":"Density ratios between two distributions can be written as the time integral of a time score, $\\partial_t \\log p_t(x)$, along a probability path connecting them. The paper establishes that this time score can be learned by regressing on conditional probability paths: the Conditional Time Score Matching loss equals the original Time Score Matching loss up to an additive constant, so the intractable marginal score is replaced by a closed-form conditional target. A vectorized variant learns the score componentwise and is several times faster per optimization step while matching or improving accuracy, including on MNIST in pixel space. The paper also proves a general error bound for the resulting density ratio estimator, decomposing error into discretization error and finite-sample estimation error.","feed_headline":"Conditional paths make density ratios several times faster to learn","feed_subtitle":"By regressing on closed-form conditional scores, the method matches TSM accuracy at a fraction of the cost.","key_machinery":"The load-bearing object is the conditioning variable $z$ with Gaussian conditional paths $p_t(x \\mid z) = \\mathcal{N}(x; \\mu_t(z), k_t I)$, for example the variance-preserving path $x = \\alpha_t x_1 + \\sqrt{1-\\alpha_t^2}\\,x_0$. The identity $\\partial_t \\log p_t(x) = \\mathbb{E}_{p_t(z \\mid x)}[\\partial_t \\log p_t(x \\mid z)]$ lets the intractable marginal score be replaced by a tractable conditional score inside a squared loss; expanding the loss shows the cross-term matches because of this expectation identity, leaving only a constant difference. A vectorized version sums per-dimension conditional scores, and a novel time-score normalization weighting $\\lambda(t)$ stabilizes training.","core_discovery":"The central discovery is that the marginal time score $\\partial_t \\log p_t(x)$ equals the posterior expectation of the conditional time score $\\partial_t \\log p_t(x \\mid z)$ when the marginal path is a mixture of conditional paths, and that square-loss regression against the two targets produces the same minimizer up to an additive constant (Theorems 1-3). This holds for scalar time scores and for the vectorized autoregressive decomposition, so a network can be trained with a closed-form Gaussian target and then the estimated score integrated over time, $\\log(p_1(x)/p_0(x)) = \\int_0^1 \\hat{s}(x,t)\\,dt$ approximately, to recover the density ratio. The paper further derives an explicit KL error bound (Theorem 4) and an asymptotic sample-size rate (Proposition 5) for integrating an estimated time score.","pith_inferences":["If the mixture identity fails, for a path not representable as $p_t(x) = \\int p_t(x\\mid z)\\,p(z)\\,dz$ with $z$ independent of $t$, the CTSM objective regresses a different target and the integrated ratio becomes biased; testing this on non-Gaussian or deterministic paths would quantify the method's robustness.","The same conditional-expectation trick could be applied to other intractable functionals, such as the Stein score or velocities, unifying conditional score matching and flow matching objectives under one theorem; the paper notes related versions but does not push this unification.","The vectorized objective suggests an autoregressive view of time scores; in very high dimensions, learning per-dimension components may interact with network inductive biases differently than learning the scalar score, which could be studied by ablating the vectorization.","Weighting by the inverse variance of the conditional time score is derived under an assumed constant $c$ set to 1; estimating $c$ from data is a natural tuning knob that the paper reports sometimes helps and sometimes hurts."],"forward_implications":["The TSM objective can be optimized without double automatic differentiation, because CTSM never evaluates $\\partial_t \\log p_t(x)$ or its parameter derivatives; this is why each iteration is several times cheaper.","The learned time score, integrated over time, yields the density ratio and hence an unnormalized model for $p_1$ when $p_0$ is known; the paper demonstrates this by modeling MNIST directly in pixel space.","The error bound shows that accuracy scales as $1/K^2$ in the number of integration steps and $1/N$ in sample size, so both discretization and optimization contribute in a controlled way.","Vectorized CTSM is essential in practice: the non-vectorized CTSM was essentially untrainable on MNIST, while CTSM-v achieved competitive or better BPD than TSM at a fraction of the per-step cost.","The general regression equivalence (Theorem 3) applies to any intractable function that is a conditional expectation of a tractable one, not only time scores."],"supporting_citations":[{"why":"Introduces the time score matching objective and the time-score identity that the paper extends and accelerates.","marker":"Choi et al. (2022)"},{"why":"Supplies the denoising score matching insight that conditional targets can replace intractable marginal targets in a regression loss.","marker":"Vincent (2011)"},{"why":"Provides the Gaussian conditional probability path framework and closed-form conditional targets used throughout the paper.","marker":"Lipman et al. (2023)"},{"why":"Establishes the telescoping density-ratio estimation approach and the experimental benchmarks the paper compares against.","marker":"Rhodes et al. (2020)"},{"why":"Defines the variance-preserving path and the diffusion-model weighting schemes that the paper adapts to time scores.","marker":"Song et al. (2021b)"},{"why":"Gives the noise-contrastive estimation baseline that time score methods are designed to improve upon.","marker":"Gutmann & Hyvärinen (2012)"},{"why":"Demonstrates vectorized flow matching objectives and conditional path decompositions that inspire the CTSM-v variant.","marker":"Tong et al. (2024a)"},{"why":"Provides the integration-by-parts argument that underlies the intractable TSM objective the paper replaces.","marker":"Hyvärinen (2005)"}],"fun_headline_variants":["Closed-form conditioning accelerates density ratio estimation","Faster density ratios via conditional probability paths","Integrating conditional scores speeds density ratio estimation","Conditional paths cut time-score learning cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the marginal probability path is exactly a mixture of the chosen conditional paths with the conditioning variable drawn independently of time; if that decomposition does not hold, the conditional loss is no longer equivalent to the time score loss.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form conditioning accelerates density ratio estimation","Faster density ratios via conditional probability paths","Integrating conditional scores speeds density ratio estimation","Conditional paths cut time-score learning cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000153,"raw_usage":{"total_tokens":1154,"prompt_tokens":842,"completion_tokens":312,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":258}},"tokens_in":458,"tokens_out":312,"duration_ms":3645,"temperature":1.0,"reasoning_tokens":258,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T12:37:41.888735+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct $p_0$ and $p_1$ with a known ratio and choose a path that is not a mixture of Gaussian conditionals, such as $p_t(x) = (1-t)p_0(x) + t p_1(x)$; train the CTSM objective, integrate the resulting score, and compare with the true $\\log(p_1/p_0)$. A systematic discrepancy would show the mixture identity is load-bearing.","supporting_citations":[{"cited_title":"Density ratio estimation via infinitesimal classification","cited_arxiv_id":null,"evidence_quote":"Introduces the time score matching objective and the time-score identity that the paper extends and accelerates."},{"cited_title":"A connection between score matching and denoising autoencoders","cited_arxiv_id":null,"evidence_quote":"Supplies the denoising score matching insight that conditional targets can replace intractable marginal targets in a regression loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Gaussian conditional probability path framework and closed-form conditional targets used throughout the paper."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the telescoping density-ratio estimation approach and the experimental benchmarks the paper compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the noise-contrastive estimation baseline that time score methods are designed to improve upon."}],"review_version":1}