REVIEW 3 major objections 5 minor 51 references
Identifying Causal Direction via Dense Functional Classes
T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read A minimum description length ratio over dense functional classes asymptotically identifies causal direction in low-noise bivariate additive noise models; the cubic-spline instantiation LCUBE reports high precision on benchmarks.
desk verdict Plausible method, strong empirical numbers, but the identifiability proof does not go through as written: the score is not an IRSF because λ depends on n and knot spacings, not just parameter count. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the score ratio L(Y|X, θ_X) / L(X|Y, θ_Y), where each code is a two-part MDL description: a parameter part log m + Σ log u_j + ((m+4)/2) log n and a goodness-of-fit part (n/2) log(RSS/n). The proof works by showing this score is an IRSF—a scoring function that adds a strictly increasing penalty for error to a strictly increasing penalty for parameter count—so the ratio inherits the regression-error ratio of Assumptions 1–3. The density property of the function class lets the approximation error be absorbed into the low-noise term, and Assumption 4 keeps the anti-causal model from being simpler than the causal one. For LCUBE, cubic splines supply the dense class and
What would settle it
Simulate X → Y with a nonlinear spline function f and low noise, then choose a function whose inverse is a spline with fewer knots; compute the LCUBE ratio for increasing n. If the ratio eventually exceeds 1, Assumption 4 is violated and Theorem 1's conclusion fails; a direct linearity check of f should likewise predict whether the method goes undecided.
Extended reading notes
Core claim
The paper proves that for a bivariate additive noise model Y = f(X) + (1/n)N, with compactly supported cause and low noise, the direction X → Y is identifiable from a ratio of two-part MDL code lengths. Theorem 1 states that if the regression functions come from any dense functional class and Assumptions 1–4 hold, then L(Y|X, θ_X) / L(X|Y, θ_Y) converges to at most 1 as n grows, with equality only when f is linear. The instantiation LCUBE uses cubic regression splines, which are dense in C([0,1]); its score is an IRSF, so the theorem applies. The paper also reports that LCUBE achieves AUDRC 87 on the Tuebingen pairs and perfect AUDRC on several low-noise synthetic benchmarks, with no Gaussia
Load-bearing premise
The load-bearing premise is Assumption 4: the best-fitting anti-causal function must need at least as many parameters as the best-fitting causal function; if a more compact inverse representation exists, the inequality behind the theorem can reverse.
Editorial extensions
If this is right
- Under the theorem, any dense functional class satisfying Assumptions 1–4 inherits the same asymptotic identifiability result, so the method is a template rather than a spline-only trick.
- The score remains valid for non-Gaussian low-noise errors, extending regression-error causal inference beyond Gaussian likelihood settings.
- For linear f, the score ratio converges to 1, so the algorithm should output 'undecided' rather than a direction; users should expect ties exactly in the linear case.
- LCUBE's one hyperparameter and low computational cost make it practical for large collections of bivariate pairs, where neural approaches need hours.
- Because the assumptions are testable, practitioners can check a given dataset for compact support and low noise before trusting the direction output.
Reading between the lines
- Editorial inference: Assumption 4 is checkable on each dataset by comparing fitted parameter counts in both directions; if the anti-causal spline is systematically more compact, the claimed limit can fail and the score should not be trusted.
- Editorial inference: because the theorem is stated for any dense class, the same proof strategy could be instantiated with other dense bases, but the paper only works out cubic splines; extending to another class would require a new practical MDL encoding and a new check of Assumption 4.
- Editorial inference: the strong empirical numbers come mostly from low-noise synthetic benchmarks and one real-world collection; a wider set of real cause-effect pairs with known ground truth would test how the asymptotic guarantee holds at finite sample sizes.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an MDL-based bivariate causal score for low-noise additive noise models, using function classes that are dense in C([0,1]). Theorem 1 claims that, under Assumptions 1-4, the ratio of conditional code lengths L(Yn|X,thetaX)/L(X|Yn,thetaY) converges to at most 1, with equality iff the regression function is linear. The method LCUBE instantiates the score with cubic regression splines. Corollary 1 asserts that the LCUBE score is identifiable by showing that the code length in Eq. (24) is an IRSF. The paper reports experiments on synthetic benchmarks and the Tuebingen cause-effect pairs dataset, with competitive or superior AUDRC/accuracy compared to several state-of-the-art methods.
Significance. If the identifiability theorem were correct, the paper would give a clean theoretical justification for a simple, fast, one-hyperparameter MDL score over dense function classes, going beyond the AIC/BIC instantiations of the IRSF framework. The empirical contribution is real: the authors provide public code, evaluate on a wide range of benchmarks, and report strong AUDRC on the Tuebingen pairs. However, the central proof connecting LCUBE to the IRSF framework is invalid as written: the score in Eq. (24) is not an IRSF in the sense of Definition 1, and Assumption 4 is asserted rather than derived for cubic splines. The theoretical claim is therefore not currently established, although it may be repairable with a substantial revision.
major comments (3)
- [Section V-H, Eq. (25)] The proof of Corollary 1 defines lambda(||beta_phi||_0) := log m + sum_j log u_j + ((m+4)/2) log n. This depends on the data-derived knot spacings u_j defined in Eq. (22) and on the sample size n, not solely on the parameter count ||beta_phi||_0 as Definition 1 requires. Two spline models with the same number of parameters and the same RSS can therefore receive different values of L(Y|X,theta). Consequently Eq. (24) is not an IRSF, and Lemma 1 cannot be invoked for LCUBE. This is load-bearing for both Theorem 1 and Corollary 1.
- [Section IV, proof of Theorem 1] The verification of the gamma condition is also not in IRSF form. The authors write gamma := (n/2) log(RSS(theta)/n) and say it is strictly increasing 'for n'. Definition 1 requires gamma: R_{>=0} -> R to be strictly increasing in the expected squared error only. The factor n and the use of the finite-sample RSS mean that the score is a function of (n, RSS, knot positions), not of (ELSE, parameter count). The asymptotic convergence RSS/n -> ELSE does not make the finite-sample score an IRSF.
- [Section V-H and Section VI-A, Assumption 4] Assumption 4 is load-bearing for Lemma 1 and Corollary 1. For cubic splines it is asserted rather than derived. The text says 'we can therefore assume' and cites Kilbertus et al. [29] for low-degree polynomials, but no argument is given for splines with variable or equidistant knots. The intuitive smoothness discussion in Section VI-A does not establish ||beta_phi||_0 <= ||beta_psi||_0. If the anti-causal spline can be represented with fewer effective parameters, the ratio inequality in Lemma 1 can fail.
minor comments (5)
- [Section IV, proof of Theorem 1] Typo: 'numerator and denominator' is written as 'nomonator an denominator'. Also, the sentence claiming that fixed parameterizations 'do not affect the limit' is unclear and should be expanded or removed.
- [Section III-A, Eq. (3)] The transition from Eq. (2) to Eq. (3) is inconsistent: the text mentions approximation error 1/(2n)N and then writes Y_n ≈ h(X) + (1/n)N. Please clarify the noise scaling.
- [Section V-H, Eq. (25)] The claim that the product m * prod_j u_j * n^{(m+4)/2} is a natural number for even m does not address the more serious issue that lambda must depend only on the parameter count. Also, if any u_j = 0, log u_j is undefined; the Schoenberg-Whitney conditions should be explicitly imposed in Algorithm 1 and checked in practice.
- [References] References [8] and [43] are the same paper (Galkin), and [2] and [25] are the same paper (Janzing and Schoelkopf). Please consolidate.
- [Table I] The column headings are confusing: '1. column', '2. column', and '2. column' are repeated. Clarify which columns correspond to the 10-dataset versus 13-dataset averages.
Circularity Check
No significant circularity: the derivation is conditional on external, independently stated assumptions, and the causal score is not fitted to the target direction.
full rationale
The paper's central claim is conditional on Assumptions 1-4 and is justified by importing external results: the IRSF framework and its identifiability theorem from Marx and Vreeken [30], and the RECI error-ratio inequality from Blobaum et al. [31]. These are not the authors' own prior claims, and they are not verified by fitting to the target. The MDL score L(Y|X,theta) is computed from the data by Algorithm 1, and the causal decision is the argmin of two independently computed scores; the true direction is not used to construct the score. There is no step where a fitted parameter is renamed as a prediction, and no load-bearing self-citation. The proof of Corollary 1 does contain a formal gap: Eq. (25) defines lambda to include knot spacings u_j and sample size n, so L in Eq. (24) is not literally an IRSF in the sense of Definition 1 (which requires lambda to depend only on the parameter count), and the sentence 'Function lambda is strictly increasing function for n' changes the argument from parameter count to n. This is a correctness risk in the identifiability proof, but it is not circularity: the theorem's conclusion does not reduce to its own assumption, and the cited IRSF/RECI theorems are independent external results. Minor self-citations ([14], [15]) appear only in auxiliary approximation-rate discussions and are not load-bearing.
Assumptions & free parameters
free parameters (1)
- mmax, the maximum number of knots =
not reported in the paper
assumptions (7)
- domain assumption Assumption 1: the effect can be written as Y = f(X) + alpha N, an additive noise model.
- domain assumption Assumption 2: the noise N is unbiased with unit variance.
- domain assumption Assumption 3: X and N have compact supports, and Y is rescaled to [0,1].
- domain assumption Assumption 4: the anti-causal regression psi has at least as many parameters as the causal regression phi, ||beta_phi||_0 <= ||beta_psi||_0.
- standard math The class of cubic splines with increasing knots is dense in C([0,1]).
- domain assumption The knots used in the code length are a subsequence of the observed x_i and satisfy the Schoenberg-Whitney conditions.
- domain assumption MRSS converges to ELSE for large n.
Cite this review
Pith. "Pith review of Identifying Causal Direction via Dense Functional Classes." pith.science (2026). https://pith.science/paper/3W6G7KYH
@misc{pith2026250900538,
author = {Pith},
title = {Pith review of: Identifying Causal Direction via Dense Functional Classes},
year = {2026},
howpublished = {\url{https://pith.science/paper/3W6G7KYH}},
note = {Machine review of arXiv:2509.00538}
}
read the original abstract
We address the problem of determining the causal direction between two univariate, continuous-valued variables, X and Y, under the assumption of no hidden confounders. In general, it is not possible to make definitive statements about causality without some assumptions on the underlying model. To distinguish between cause and effect, we propose a bivariate causal score based on the Minimum Description Length (MDL) principle, using functions that possess the density property on a compact real interval. We prove the identifiability of these causal scores under specific conditions. These conditions can be easily tested. Gaussianity of the noise in the causal model equations is not assumed, only that the noise is low. The well-studied class of cubic splines possesses the density property on a compact real interval. We propose LCUBE as an instantiation of the MDL-based causal score utilizing cubic regression splines. LCUBE is an identifiable method that is also interpretable, simple, and very fast. It has only one hyperparameter. Empirical evaluations compared to state-of-the-art methods demonstrate that LCUBE achieves superior precision in terms of AUDRC on the real-world Tuebingen cause-effect pairs dataset. It also shows superior average precision across common 10 benchmark datasets and achieves above average precision on 13 datasets.
Figures
Reference graph
Works this paper leans on
-
[30]
A. Marx, J. Vreeken. Identifiability of cause and effect using regularized regression. 25th ACM SIGKDD KDD. 2019
work page 2019
-
[29]
N. Kilbertus, G. Parascandolo, B. Sch ¨olkopf. Generalization in anti-causal learning. arXiv:1812.00524 (2018)
arXiv 2018
-
[1]
P. B ¨uhlmann, J. Peters, J. Ernest. CAM: Causal additive models, high-dimensional order search and penalized regression. Ann. Statist. 42 (6), pp. 2526 - 2556, 2014
work page 2014
-
[2]
D. Janzing, B. Sch ¨olkopf. Causal Inference Using the Algorithmic Markov Condition. IEEE Trans on Inform. Tech. 56/10, 5168–94, 2010
work page 2010
- [3]
-
[4]
M. Li, P. Vit ´anyi, An Introduction to Kolmogorov Complexity and its Applications. Springer, 2019, vol. 4
work page 2019
-
[5]
K. Budhathoki, J. Vreeken. MDL for causal inference on discrete data. IEEE ICDM 2017
work page 2017
-
[6]
C. De Boor. A Practical Guide to Splines, V olume 27. Springer-Verlag New York, 1978
work page 1978
Show all 51 references
-
[7]
A. Marx, J. Vreeken. Formally justifying MDL-based inference of cause and effect. In AAAI 2022
2022
-
[8]
P.V . Galkin. The uniqueness of the element of best mean approximation to a continuous function using splines with fixed nodes. Mathematical notes of the Academy of Sciences of the USSR, 15 (1):3–8, 1974
1974
-
[9]
Goudet, D
O. Goudet, D. Kalainathan, P. Caillou, I. Guyon, D. Lopez-Paz, M. Sebag. Learning functional causal models with generative neural networks. Explainable and Interpr. Models in Comp. Vis. and ML, pp. 39–80, 2018
2018
-
[10]
Rissanen
J. Rissanen. Modeling by shortest data description. Automatica 14.5 (1978): 465-471
1978
-
[11]
Gr ¨unwald
P. Gr ¨unwald. The minimum description length principle and reasoning under uncertainty. University of Amsterdam, 1998
1998
-
[12]
Guyon, A
I. Guyon, A. Statnikov, B.B. Batu. Cause Effect Pairs in Machine Learning. Springer, 2019
2019
-
[13]
Hall, W.W
C.A. Hall, W.W. Meyer. Optimal error bounds for cubic spline interpolation. Journal of Approximation Theory 16.2 (1976): 105-122
1976
-
[14]
Hlav ´aˇckov´a-Schindler, M
K. Hlav ´aˇckov´a-Schindler, M. Sanguineti. Bounds on the complexity of neural-network models and comparison with linear methods. Intern. Journal of Adapt. Control and Sign. Process. 17.2 (2003): 179-194
2003
-
[15]
Hlav ´aˇckov´a-Schindler
K. Hlav ´aˇckov´a-Schindler. Rates of approximation in a feedforward network depend on the types of computational unit. In book: Dealing with Complexity, Springer, pp 205–219, 1998
1998
-
[16]
Kovachki, S
N. Kovachki, S. Lanthaler, S. Mishra. On universal approximation and error bounds for Fourier neural operators. JMLR 22.290 (2021): 1-76
2021
-
[17]
G. Wahba. Interpolating spline methods for density estimation I. Equi-spaced knots. Annals of Stat. (1975): 30-48
1975
-
[18]
V . Vapnik. The nature of statistical learning theory. Springer Science and Business Media, 2013
2013
-
[19]
Hornik, M
K. Hornik, M. Stinchcombe, H. White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359-366, 1989
1989
-
[20]
Hoyer, D
P. Hoyer, D. Janzing, J.M. Mooij, J. Peters, B. Sch ¨olkopf. Nonlinear causal discovery with additive noise models. Advances in Neurips, 21, 2008
2008
-
[21]
Schultheiss, P
C. Schultheiss, P. B ¨uhlmann. On the pitfalls of Gaussian likelihood scoring for causal discovery. Journal of Causal Inference 11.1 (2023): 20220068
2023
-
[22]
Nowzohour, P
C. Nowzohour, P. B ¨uhlmann. Score-based causal learning in additive noise models. Statistics. 2016;50(3):471–85
2016
-
[23]
Immer, C
A. Immer, C. Schultheiss, J.E. V ogt, B. Sch ¨olkopf, P. B¨uhlmann, A. Marx. On the identifiability and estimation of causal location-scale noise models. ICDM, pp. 14316–14332. PMLR, 2023
2023
-
[24]
Kainen, V
P.C. Kainen, V . K ˚urkov´a. An integral upper bound for neural network approximation. Neural Computation 21.10 (2009): 2970-2989
2009
-
[25]
Janzing, B
D. Janzing, B. Sch ¨olkopf. Causal inference using the algorithmic Markov condition. IEEE Trans. on Information Theory, 56(10):5168–5194, 2010
2010
-
[26]
Janzing, J.M
D. Janzing, J.M. Mooij, K. Zhang, J. Lemeire, J. Zscheischler, P. Daniu ˇsis, B. Steudel, B. Sch ¨olkopf. Information-geometric approach to inferring causal directions. Artificial Intelligence, 182:1–31, 2012
2012
-
[27]
T.C.M. Lee. Regression spline smoothing using the minimum description length principle. Statistics and Probability Letters, 48(1):71–82, 2000
2000
-
[28]
A. Marx, J. Vreeken. Telling cause from effect using MDL-based local and global regression. 2017 IEEE ICDM, pp. 307–316. IEEE, 2017
2017
-
[31]
Bl ¨obaum, D
P. Bl ¨obaum, D. Janzing, T. Washio, S.Shimizu, B. Sch ¨olkopf. Cause-Effect Inference by Comparing Regression Errors. AISTATS 2018
2018
-
[32]
Mooij, J
J.M. Mooij, J. Peters, D. Janzing, J. Zscheischler, B. Sch ¨olkopf. Distinguishing cause from effect using observational data: methods and benchmarks. JMLR, 17(32):1–102, 2016
2016
-
[33]
J. Pearl. Causality. Cambridge University Press, 2009
2009
-
[34]
Peters, J.M
J. Peters, J.M. Mooij, D. Janzing, B. Sch ¨olkopf. Identifiability of causal graphs using functional models. arXiv preprint arXiv:1202.3757, 2012
2012 arXiv
-
[35]
Peters, J.M
J. Peters, J.M. Mooij, D. Janzing, B. Sch ¨olkopf. Causal discovery with continuous additive noise models. JMLR, 2014
2014
-
[36]
Zhang, A
K. Zhang, A. Hyv ¨arinen. 2009. On the Identifiability of the Post-nonlinear Causal Model. UAI 2009, 647–655
2009
-
[37]
Peters, D
J. Peters, D. Janzing, B. Sch ¨olkopf. Elements of causal inference: Foundations and learning algorithms. The MIT Press, 2017
2017
-
[38]
Rissanen
J. Rissanen. Stochastic complexity in statistical inquiry, V ol. 15. World Scientific, 1998
1998
-
[39]
Rissanen
J. Rissanen. Stochastic complexity. Information and Complexity in Statistical Modeling, pages 57–77, 2007
2007
-
[40]
M.H. Schultz. Spline analysis. Prentice Hall, 1973
1973
-
[41]
Shimizu, P.O
S. Shimizu, P.O. Hoyer, A. Hyv ¨arinen, A. Kerminen, M. Jordan. A linear non-Gaussian acyclic model for causal discovery. JMLR, 7(10), 2006
2006
-
[42]
Subbotin, N.I
Y .N. Subbotin, N.I. Chernykh. Order of the best spline approximations of some classes of functions. Math. Notes of the Academy of Sciences of the USSR, 7(1):20–26, 1970
1970
-
[43]
P.V . Galkin. The uniqueness of the element of best mean approximation to a continuous function using splines with fixed nodes. Math. Notes of the Academy of Sciences of the USSR 15.1 (1974): 3-8
1974
-
[44]
Tagasovska, V
N. Tagasovska, V . Chavez-Demoulin, T. Vatter. Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery. ICML, pp. 9311–9323. PMLR, 2020
2020
-
[45]
Q.-D. Tran, B. Duong, P. Nguyen, T. Nguyen. Robust estimation of causal heteroscedastic noise models. SIAM SDM, pp. 788–796, 2024
2024
-
[46]
S. Xu, O.A. Mian, A. Marx, J. Vreeken. Inferring cause and effect in the presence of heteroscedastic noise. ICML, pp. 24615-24630. 2022. APPENDIX A. Proof of Lemma 1: Under Assumptions 1-3 from Bl ¨obaum et al. in [31], it holds lim α→0 E[( ˜Yα − φ(X))2] E[(X − ψ( ˜Yα))2] ≤ 1 ...
2022
-
[47]
Thus based on this, L(m) ≈ log2 m
Code L(m): L(m) can be approximated by log m when m is reasonably large, see paper [38], Section 2.2.4. Thus based on this, L(m) ≈ log2 m
-
[48]
Since k = {k1,
Code L(k|m): First we will compute the code L(k|m). Since k = {k1, . . . , km} ⊂X, the sequence of indices k can be specified by the indices of those xi’s where a knot is placed. This set of sorted indices can be compactly specified by their successive differences. Define k0 =...
-
[49]
Given m, k, the values b, β can be computed from Eq
Code for L(b, β|m, k): Now we compute L(b, β|m, k). Given m, k, the values b, β can be computed from Eq. (16) and the result is the conditional maximum likelihood estimates of b and β (or the least square estimates if the assumption of normal errors is not given). Based on [38...
-
[50]
AUDRC: The area under the decision rate curve (AUDRC). [23] justifies the selection of AUDRC over AUROC, since it weights correctly identified X → Y pairs in the same way as correctly identified Y → X pairs and thus avoids an arbitrary selection of true positives and true nega...
-
[51]
← or → or ? = undecided) and wq is the weight of the pair, see [32]
Forced decision: Evaluation of accuracy ACC: Given (X, Y), the method has to decide either X → Y or Y → X; We evaluate the accuracy of these decisions as accuracy = PQ q=1 wqδ ˆdq,dq PQ q=1 wq , where dq is the true causal direction for the q’th T ¨ubingen pair (either ← or →)...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.