REVIEW 3 major objections 5 minor 45 references
Distinguishing Cause from Effect with Causal Velocity Models
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper establishes a noise-agnostic score identity for bivariate causal direction: a joint distribution is generated by an SCM with causal velocity $v$ if and only if $s_x(x,y)=-\partial_y v(y,x)-v(y,x)s_y(x,y)+s_x(x)$, and comparing…
desk verdict The velocity–score identity is the real contribution; the direction-selection rule is a heuristic that needs either a separation proof or a humbler framing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the causal velocity $v(y,x)$, the derivative at the factual point of the counterfactual curve that maps an observed $y$ to its value had the cause $x$ been changed infinitesimally. It plays the role of the velocity field in an ODE whose time direction is the cause variable, generated by the flow $\varphi_{x,x'}(y)=f_{x'}(f_x^{-1}(y))$. The paper pairs this velocity with the continuity equation $\partial_x \log p(y\mid x) = -\partial_y v(y,x) - v(y,x)\partial_y \log p(y\mid x)$, converting it into the score identity of Theorem 4.1; this conversion is the mechanism that turns observational scores into a noise-agnostic regression target.
What would settle it
Compute the true joint and marginal scores for a generated SCM outside the ANM and LSNM classes and fit the velocity in the anti-causal direction using those exact scores; if the anti-causal fit reaches zero or matches the causal fit, the direction-selection claim fails. A concrete variant is to choose a velocity whose flow has closed-form counterfactuals, generate the exact joint density, and then check whether any alternative velocity in the same family satisfies the Theorem 4.1 identity in the reverse direction.
Extended reading notes
Core claim
The paper's central claim is an if-and-only-if identity. Under regularity conditions, a joint distribution with full support and differentiable log-densities can be represented by a structural causal model with velocity $v$ exactly when $$s_x(x,y) = -\partial_y v(y,x) - v(y,x)\,s_y(x,y) + s_x(x)$$ holds for all $(x,y)$, where $s_x(x,y)$ and $s_y(x,y)$ are partial derivatives of the joint log-density and $s_x(x)$ is the marginal score of the cause. The identity is the log form of the continuity equation applied to the SCM flow, and it makes the noise distribution disappear from the estimation problem. The paper fits $v$ by minimizing $L(v)=\mathbb{E}[(s_x(X)-\partial_y v(Y,X)-s_v(X,Y))^2]$, where $s_v(x,y)=s_x(x,y)+v(y,x)s_y(x,y)$ is the directional derivative of the log-density along the causal curve, and chooses the direction with the smaller fitted objective. Because nothing in the objective requires an additive or location-scale mechanism, the same procedure applies to a wider class of models and needs no assumption on the noise.
Load-bearing premise
The load-bearing premise is that the joint and marginal score functions of the observed distribution can be estimated accurately enough from finite data; the paper's own ablations show discovery quality degrades when score estimates are poor, and discrete or integer-valued benchmark instances cannot be used because their scores are not defined.
Editorial extensions
If this is right
- For any mechanism whose velocity can be represented, causal direction can be decided without fitting an additive-noise or location-scale model and without Gaussian assumptions on the noise.
- The same goodness-of-fit value can flag model misspecification or non-identifiability when the score is estimated well, because the identity holds only when the velocity matches the data-generating SCM.
- Integrating the estimated velocity recovers counterfactual curves, so velocity estimation yields a counterfactual model from observational data as a by-product.
- If the score estimators converge, the fitted objective converges at a rate no worse than the score estimators, so improvements in nonparametric score estimation directly improve the reliability of the causal discovery procedure.
Reading between the lines
- Applied to multivariate data, the pairwise score comparison could in principle be combined with graph-structure search, since the identity is local and does not depend on the noise distribution; the paper itself leaves that extension unexplored.
- The identifiability equation obtained by writing the continuity equation in both directions suggests that similar uniform non-identifiability criteria might be derivable for other mechanism classes beyond ANM and LSNM, not just case-by-case checks.
- Because the method needs a well-defined log-density, extending it to discrete or integer-valued variables would require smoothed densities or surrogate scores; the paper explicitly drops such benchmark instances rather than handling them.
- The velocity parametrization also gives practitioners a way to encode mechanistic knowledge directly as infinitesimal-intervention responses, so a domain-specified velocity could be tested against data before making any parametric noise assumption.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a parametrization of bivariate structural causal models (SCMs) through a “causal velocity” v(y,x), viewing the cause variable as time in a dynamical system. The central theoretical result (Theorem 4.1) is an identity relating the joint score, marginal score, and velocity via the continuity equation: sx(x,y) = -∂y v(y,x) - v(y,x) sy(x,y) + sx(x), holding if and only if the joint distribution is representable by an SCM with that velocity. The paper proposes a goodness-of-fit objective L(v) based on this identity, estimates scores nonparametrically, fits velocities in both candidate directions, and selects the direction with smaller objective. Theorem 5.1 states a consistency result for the empirical objective at a fixed v, and Proposition 5.2 derives a PDE condition for non-identifiability. Experiments on synthetic and benchmark data show the method succeeding on Velocity and Sigmoid benchmarks where ANM/LSNM methods struggle, with ablation studies examining the dependence on score estimation quality.
Significance. The velocity-based parametrization is a conceptually new and elegant way to connect SCMs, counterfactuals, and score functions, extending functional causal discovery beyond the ANM and LSNM classes without specifying noise distributions. The paper is honest about the central role of score estimation, provides code, and includes oracle-score experiments that cleanly isolate the behavior of the objective. If the method is reliable, it offers a practical tool for bivariate causal discovery in settings where existing methods are misspecified. However, the theoretical support for the direction-selection rule is incomplete, and the empirical results show non-negligible error rates even in well-specified settings, which tempers the practical significance until the decision rule is better understood.
major comments (3)
- [Section 5, Eq. (15)–(18)] The proposed causal discovery rule selects the direction with the smaller empirical objective, but the paper provides no population-level separation guarantee for the velocity classes actually used (B-LIN, B-QUAD, V-NN). Theorem 4.1 characterizes representability in a single direction, and Theorem 5.1 only bounds |That L_n(v) - L(v)| for a fixed v; neither controls the gap between the two directional infima. Since every smooth full-support bivariate density is generated by some bijective SCM in both directions, all identifiability must come from restricting the velocity class, and Section 5.2 explicitly stops short of a uniform identifiability analysis, stating that a general analysis “may require new techniques.” This is load-bearing for the central claim that the method distinguishes cause from effect.
- [Appendix B.3 (proof of Theorem 5.1)] The consistency proof bounds the empirical-process term by Rademacher averages over the class {f(s,·) : s ∈ B(H,M)}. In the expansion (36), product classes such as B(H1)⊗B(H2) and B(H1)⊗B(H3)⊗{h2} are treated as closed balls in tensor-product RKHSs. The set of products of two functions with bounded RKHS norm is not itself a ball in the tensor-product RKHS, and the assertion that “every term in (36) is the Rademacher average of a closed ball in an RKHS of bounded functions” is not justified. Thus Theorem 5.1, which is presented as establishing consistency of the estimation procedure, is not fully supported by the supplied proof.
- [Section 5.2, Proposition 5.2] Proposition 5.2 is stated as an “if and only if” characterization of non-identifiability, but the proof in Appendix C only equates the two mixed partial derivatives (50) and (51). Sufficiency—that equality of these derivatives implies existence of a joint density satisfying both directional continuity equations—is not shown. Moreover, the proposition does not yield a measure-zero or lower-bound argument for the specific basis classes (B-LIN, B-QUAD) used in the experiments, so the identifying power of those classes is only empirically demonstrated, not theoretically established.
minor comments (5)
- [Section 3, Definition 3.1] The sentence “It is easy to see that Equation (9) satisfies the axioms of a flow Theorem 2.1” is missing a word; it should read “of a flow in Theorem 2.1.”
- [Section 7.2] The phrase “which we suspect is due to the score being well-estimated” is informal; a more precise statement or a reference to the score-MSE results would strengthen the explanation.
- [Section 7.3, Table 8] For the well-specified LSNM experiment, the success rate is only 73% at n=5000 and 80% at n=10000 even when the true score is not used; the text should explicitly discuss why a well-specified model with improving score estimates still does not approach perfect accuracy.
- [Section 7.1 / Table 4] The Sigmoid benchmark is described verbally as “a variation on LSNMs with additional post-nonlinear and affine transformations,” but the mechanism in Table 4 is sufficiently complex that a brief interpretation or a reference to the table in the main text would improve reproducibility.
- [Figure 5 caption] The caption states “Upper and lower limits indicate 1st– 3rd quartile over 100 datasets,” but the figure appears to show shaded bands; specifying whether the limits are bands or error bars would help the reader.
Circularity Check
No significant circularity: the score–velocity identity is derived from the continuity equation, and the causal-direction rule is a goodness-of-fit comparison rather than a fitted restatement of the target.
full rationale
The central result, Theorem 4.1 (Eq. 14), is derived from the log-continuity equation (Eq. 13), which the paper obtains from standard flow/transport results (Theorem A.2, from Santambrogio and Arnold). The 'if and only if' claim is supported by the uniqueness of continuity-equation solutions for a fixed velocity and by the flow construction in Theorem 3.2. No parameter is fitted to the causal-direction label. The objective L(v) in Eq. 15 is the squared residual of the identity, and causal discovery compares this residual under a restricted velocity class in each direction. That is a goodness-of-fit procedure, not a re-derivation of the conclusion from assumptions that already contain it. The paper explicitly acknowledges the main limitation: Section 5.2 says a general identifiability analysis 'may require new techniques,' and Theorem 5.1 bounds |hat L_n(v) - L(v)| for fixed v rather than the argmin gap. These are real methodological gaps for the direction-selection rule, but they are not circularity. The only self-citation (Dance & Bloem-Reddy 2024, cocycles) is contextual and is not used to prove Theorem 3.2 or Theorem 4.1; those proofs rely on Arnold and Santambrogio. The paper is therefore self-contained for its main derivation, with the caveat that population-level separation of the two directional losses is not established.
Assumptions & free parameters
free parameters (2)
- Velocity model parameters (basis coefficients or MLP weights) =
Varies by dataset
- Score estimator bandwidth and regularization =
Median heuristic for Stein; Silverman rule for KDE; lambda=0.1 for Stein
assumptions (4)
- standard math Continuity equation solution uniqueness (Santambrogio 2015, Thm 4.4)
- domain assumption SCM is bijective with differentiability and full support (Section B.1)
- domain assumption Score estimators converge in RKHS norm at rate n^{-1/alpha} (Theorem 5.1)
- domain assumption Identifiability: the causal direction yields a strictly smaller limit objective than the anticausal direction for the chosen velocity model class
Cite this review
Pith. "Pith review of Distinguishing Cause from Effect with Causal Velocity Models." pith.science (2026). https://pith.science/paper/ZEYO4YCK
@misc{pith2026250205122,
author = {Pith},
title = {Pith review of: Distinguishing Cause from Effect with Causal Velocity Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZEYO4YCK}},
note = {Machine review of arXiv:2502.05122}
}
read the original abstract
Bivariate structural causal models (SCM) are often used to infer causal direction by examining their goodness-of-fit under restricted model classes. In this paper, we describe a parametrization of bivariate SCMs in terms of a causal velocity by viewing the cause variable as time in a dynamical system. The velocity implicitly defines counterfactual curves via the solution of initial value problems where the observation specifies the initial condition. Using tools from measure transport, we obtain a unique correspondence between SCMs and the score function of the generated distribution via its causal velocity. Based on this, we derive an objective function that directly regresses the velocity against the score function, the latter of which can be estimated non-parametrically from observational data. We use this to develop a method for bivariate causal discovery that extends beyond known model classes such as additive or location scale noise, and that requires no assumptions on the noise distributions. When the score is estimated well, the objective is also useful for detecting model non-identifiability and misspecification. We present positive results in simulation and benchmark experiments where many existing methods fail, and perform ablation studies to examine the method's sensitivity to accurate score estimation.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Albergo, M. and Vanden-Eijnden, E. Building normalizing flows with stochastic interpolants. In ICLR , 2023
work page 2023
-
[3]
Arnold, L. Random Dynamical Systems. Springer Berlin Heidelberg, 1998
work page 1998
-
[4]
Bartlett, P. L. and Mendelson, S. Rademacher and G aussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3 0 (11), 2002
work page 2002
-
[5]
o baum, P., Janzing, D., Washio, T., Shimizu, S., and Sch \
Bl \"o baum, P., Janzing, D., Washio, T., Shimizu, S., and Sch \"o lkopf, B. Cause-effect inference by comparing regression errors. In AISTATS , 2018
work page 2018
-
[6]
J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., Vander P las, J., Wanderman- M ilne, S., and Zhang, Q
Bradbury, J., Frostig, R., Hawkins, P., Johnson, M. J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., Vander P las, J., Wanderman- M ilne, S., and Zhang, Q. JAX : composable transformations of P ython+ N um P y programs, 2018. URL http://github.com/jax-ml/jax
2018
-
[7]
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. K. Neural ordinary differential equations. In NeurIPS , 2018
work page 2018
-
[8]
Dance, H. and Bloem-Reddy, B. Causal inference with cocycles. arXiv preprint arXiv:2405.13844, 2024
arXiv 2024
Show all 45 references
-
[9]
Inferring deterministic causal relations
Daniu s is, P., Janzing, D., Mooij, J., Zscheischler, J., Steudel, B., Zhang, K., and Sch \"o lkopf, B. Inferring deterministic causal relations. In UAI , 2010
2010
-
[10]
and Nguyen, T
Duong, B. and Nguyen, T. Bivariate causal discovery via conditional divergence. In CLeaR , 2022
2022
-
[11]
Fonollosa, J. A. Conditional distribution variability measures for causality detection. In Cause Effect Pairs in Machine Learning. Springer, 2019
2019
-
[12]
Learning Bivariate Functional Causal Models, pp.\ 101--153
Goudet, O., Kalainathan, D., Sebag, M., and Guyon, I. Learning Bivariate Functional Causal Models, pp.\ 101--153. Springer International Publishing, 2019
2019
-
[13]
Measuring statistical dependence with Hilbert--Schmidt norms
Gretton, A., Bousquet, O., Smola, A., and Sch \"o lkopf, B. Measuring statistical dependence with Hilbert--Schmidt norms. In ALT , 2005
2005
-
[14]
Hodgkinson, L., van der Heide, C., Roosta, F., and Mahoney, M. W. Stochastic continuous normalizing flows: training SDEs as ODEs . In UAI , 2021
2021
-
[15]
M., Peters, J., and Sch \"o lkopf, B
Hoyer, P., Janzing, D., Mooij, J. M., Peters, J., and Sch \"o lkopf, B. Nonlinear causal discovery with additive noise models. In NeurIPS , 2008
2008
-
[16]
o lkopf, B., B \
Immer, A., Schultheiss, C., Vogt, J. E., Sch \"o lkopf, B., B \"u hlmann, P., and Marx, A. On the identifiability and estimation of causal location-scale noise models. In ICML , 2023
2023
-
[17]
Causal normalizing flows: from theory to practice
Javaloy, A., S \'a nchez-Mart \' n, P., and Valera, I. Causal normalizing flows: from theory to practice. In NeurIPS , 2024
2024
-
[18]
Causal D iscovery T oolbox: uncovering causal relationships in P ython
Kalainathan, D., Goudet, O., and Dutta, R. Causal D iscovery T oolbox: uncovering causal relationships in P ython. Journal of Machine Learning Research, 21 0 (37), 2020
2020
-
[19]
Foundations of Modern Probability
Kallenberg, O. Foundations of Modern Probability. Springer International Publishing, 2021
2021
-
[20]
J., and Duvenaud, D
Kelly, J., Bettencourt, J., Johnson, M. J., and Duvenaud, D. K. Learning differential equations that are easy to solve. In NeurIPS , 2020
2020
-
[21]
Rank-based causal discovery for post-nonlinear models
Keropyan, G., Strieder, D., and Drton, M. Rank-based causal discovery for post-nonlinear models. In AISTATS , 2023
2023
-
[22]
Causal autoregressive flows
Khemakhem, I., Monti, R., Leech, R., and Hyvarinen, A. Causal autoregressive flows. In AISTATS , 2021
2021
-
[23]
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In ICLR , 2015
2015
-
[24]
and Turner, R
Li, Y. and Turner, R. E. Gradient estimators for implicit models. In ICLR , 2018
2018
-
[25]
Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In ICLR , 2023
2023
-
[26]
Causal discovery with score matching on additive models with arbitrary noise
Montagna, F., Noceti, N., Rosasco, L., Zhang, K., and Locatello, F. Causal discovery with score matching on additive models with arbitrary noise. In CLeaR , 2023 a
2023
-
[27]
Scalable causal discovery with score matching
Montagna, F., Noceti, N., Rosasco, L., Zhang, K., and Locatello, F. Scalable causal discovery with score matching. In CLeaR , 2023 b
2023
-
[28]
Regression by dependence minimization and its application to causal inference in additive noise models
Mooij, J., Janzing, D., Peters, J., and Sch \"o lkopf, B. Regression by dependence minimization and its application to causal inference in additive noise models. In ICML , 2009
2009
-
[29]
M., Peters, J., Janzing, D., Zscheischler, J., and Sch \"o lkopf, B
Mooij, J. M., Peters, J., Janzing, D., Zscheischler, J., and Sch \"o lkopf, B. Distinguishing cause from effect using observational data: methods and benchmarks. Journal of Machine Learning Research, 17 0 (32), 2016
2016
-
[30]
Counterfactual identifiability of bijective causal models
Nasr-Esfahany, A., Alizadeh, M., and Shah, D. Counterfactual identifiability of bijective causal models. In ICML , 2023
2023
-
[31]
Causality
Pearl, J. Causality. Cambridge University Press, 2009
2009
-
[32]
and B \"u hlmann, P
Peters, J. and B \"u hlmann, P. Identifiability of G aussian structural equation models with equal error variances. Biometrika, 101 0 (1), 2014
2014
-
[33]
M., Janzing, D., and Sch \"o lkopf, B
Peters, J., Mooij, J. M., Janzing, D., and Sch \"o lkopf, B. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 15 0 (58), 2014
2014
-
[34]
Score matching enables causal discovery of nonlinear additive noise models
Rolland, P., Cevher, V., Kleindessner, M., Russell, C., Janzing, D., Sch \"o lkopf, B., and Locatello, F. Score matching enables causal discovery of nonlinear additive noise models. In ICML , 2022
2022
-
[35]
Optimal Transport for Applied Mathematicians
Santambrogio, F. Optimal Transport for Applied Mathematicians. Birkh\" a user Cham, 2015
2015
-
[36]
and B \"u hlmann, P
Schultheiss, C. and B \"u hlmann, P. On the pitfalls of G aussian likelihood scoring for causal discovery. Journal of Causal Inference, 11 0 (1), 2023
2023
-
[37]
Silverman, B. W. Density estimation for statistics and data analysis. Routledge, 2018
2018
-
[38]
Support Vector Machines
Steinwart, I. Support Vector Machines. Springer, 2008
2008
-
[39]
Strobl, E. V. and Lasko, T. A. Identifying patient-specific root causes with the heteroscedastic noise model. Journal of Computational Science, 72, 2023
2023
-
[40]
Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery
Tagasovska, N., Chavez-Demoulin, V., and Vatter, T. Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery. In ICML , 2020
2020
-
[41]
Optimal transport for causal discovery
Tu, R., Zhang, K., Kjellstr \"o m, H., and Zhang, C. Optimal transport for causal discovery. In ICLR , 2022
2022
-
[42]
Wibisono, A., Wu, Y., and Yang, K. Y. Optimal score estimation via empirical Bayes smoothing. In COLT , 2024
2024
-
[43]
A., Marx, A., and Vreeken, J
Xu, S., Mian, O. A., Marx, A., and Vreeken, J. Inferring cause and effect in the presence of heteroscedastic noise. In ICML , 2022
2022
-
[44]
and Hyv \"a rinen, A
Zhang, K. and Hyv \"a rinen, A. On the identifiability of the post-nonlinear causal model. In UAI , 2009
2009
-
[45]
Nonparametric score estimators
Zhou, Y., Shi, J., and Zhu, J. Nonparametric score estimators. In ICML , 2020
2020
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.