Pith. sign in

REVIEW 3 major objections 4 minor 80 references

When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that under an additive noise model the causal direction between two variables is recovered by an asymmetry in conditional denoising: the noise predicted for the effect is independent of the cause, while the noise…

desk verdict BiDD has a genuinely new decision rule and a broad empirical sweep, but its central consistency theorem relies on an invalid independence step, so the paper's key guarantee is currently unproven. read the letter →

arxiv 2506.23374 v1 pith:IA2O4MA3 submitted 2025-06-29 cs.LG

Dominik Meier , Sujai Hiremath , Promit Ghosal , Kyra Gan This is my paper
classification cs.LG
keywords causaldiscoverybivariatedirectionadditivenoisemodelsunobservedmediatorsdenoisingdiffusionconditionalindependencetestingmutualinformationestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Bivariate Denoising Diffusion (BiDD), a method for deciding whether X causes Y from a single pair of observed variables. Its decision rule trains two conditional diffusion models, one predicting the Gaussian noise added to Y while conditioning on X and one predicting the noise added to X while conditioning on Y, and compares how dependent each predicted noise is on its conditioning variable. Under an additive noise model, the paper proves that the predicted noise is independent of the conditioning variable in the causal direction and dependent on it in the anticausal direction, so the smaller mutual information identifies the cause. The same rule is conjectured, and empirically shown, to keep working when unobserved mediators break the additive noise assumption. This matters because most bivariate causal discovery methods fail precisely in that mediator-corrupted setting.

What carries the argument

The load-bearing object is the conditional noise predictor $\varepsilon_{A,\theta}(\tilde{A}_t, B, t)$ trained with the standard denoising diffusion objective to recover the Gaussian noise injected into $\tilde{A}_t$ while conditioning on $B$. For the causal direction the statistic $T = \tilde{Y} - f(X) = \varepsilon_Y + \varepsilon_1$ is sufficient, so $\hat{\varepsilon}_Y \perp\!\!\perp X$; in the anticausal direction no such sufficient statistic exists, and the paper argues via a case analysis on $h(\tilde{X}, Y) = E[\varepsilon_X \mid \tilde{X}, Y]$ that $\hat{\varepsilon}_X \not\perp\!\!\perp Y$. The decision rule compares $MI(\hat{\varepsilon}_Y, X)$ and $MI(\hat{\varepsilon}_X, Y)$ across diffusion timesteps and votes for the direction with the smaller dependence.

What would settle it

Simulate an additive noise model with non-Gaussian cause and noise, for instance $X \sim U(-1,1)$, $\varepsilon_1 \sim U(-1,1)$, and $Y = \tanh(X) + \varepsilon_1$; add Gaussian $\varepsilon_X$ and compute the true conditional expectation $E[\varepsilon_X \mid X + \varepsilon_X, Y]$ by numerical integration or a large-sample regression. If any admissible model of this form yields a prediction independent of Y, then Lemma 4.1 is false and the decision rule loses its anticausal guarantee.

Watch

Extended reading notes

Core claim

The central claim is an asymmetry in conditional denoising. For data generated by $Y = f(X) + \varepsilon_1$ with $X$ independent of $\varepsilon_1$ and $\varepsilon_1$ non-Gaussian, add independent Gaussian noise $\varepsilon_X$ and $\varepsilon_Y$ to both variables. The optimal prediction $E[\varepsilon_Y \mid Y + \varepsilon_Y, X]$ is a function only of $\varepsilon_Y + \varepsilon_1$, hence independent of $X$; the optimal prediction $E[\varepsilon_X \mid X + \varepsilon_X, Y]$ is, the paper argues, always dependent on $Y$. BiDD turns this into a decision rule: estimate both predictors with denoising diffusion models and use a mutual information test between each predicted noise and its conditioning variable; if $MI(\hat{\varepsilon}_Y, X) \leq MI(\hat{\varepsilon}_X, Y)$, declare $X \rightarrow Y$. Theorem 4.2 states that, with a consistent mutual information estimator and infinite data, this rule recovers the true direction whenever the ANM-UM reduces to a standard ANM, and the paper conjectures the rule remains correct under nonlinear hidden mediators.

Load-bearing premise

The consistency proof rests on the claim that in the anticausal direction the optimal denoising prediction is always dependent on the conditioning variable Y under an additive noise model; if some admissible model made that prediction independent of Y, the mutual-information comparison would no longer identify the cause.

Editorial extensions

If this is right

  • Causal orientation can be recovered from bivariate data by comparing a single dependence statistic rather than by testing residual independence or comparing prediction errors.
  • The method does not require the mechanism to be invertible or the noise to be Gaussian, so it applies to non-invertible and non-Gaussian additive noise settings.
  • When unobserved mediators are present, BiDD remains accurate on the tested synthetic data-generating processes while standard ANM methods and the previous cascade model can fail badly.
  • Because the decision is based only on the pair $(X, Y)$, BiDD can be used as a bivariate subroutine inside larger graph-learning procedures without knowing how many mediators lie between the two variables.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit is to replace the per-timestep majority vote with a single aggregated test statistic, weighting timesteps by their signal strength; this could improve finite-sample power without changing the core decision rule.
  • The same dependence asymmetry might persist for hidden confounders rather than mediators, since both corrupt the observed relationship through latent variables; testing this is a direct next step the paper lists as future work.
  • The paper's proof of anticausal dependence is case-based; a uniform proof covering all smooth mechanisms would also pin down which noise regimes give the largest asymmetry and could guide where the method is most reliable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ANM-UM, a bivariate additive-noise model with unobserved mediators, and analyzes when standard ANM-based methods fail under this model. It proposes BiDD, which trains conditional denoising diffusion models in both causal directions and selects the direction for which the predicted noise is less dependent on the conditioning variable. The main theoretical claim is Theorem 4.2, asserting asymptotic consistency of this decision rule under a standard ANM; consistency under hidden mediation is left as a conjecture. The empirical evaluation on synthetic data and the Tübingen cause-effect pairs shows that BiDD is competitive with or better than baselines, especially when mediators are present.

Significance. The BiDD decision rule is a novel and elegant criterion for bivariate causal orientation: it does not fit direction labels, it is not another residual-independence, score-matching, or MSE comparison, and the empirical study is broad, covering linear, invertible/non-invertible nonlinear mechanisms, several noise types, mediator depth, sample size, and a real-world benchmark. The causal-direction half of the proof (Appendix E.12.1) is sound. If Theorem 4.2 and the ANM-UM reducibility characterization were rigorously established, this would be a solid theoretical and methodological contribution. As submitted, however, the central consistency guarantee is not proven, and the paper's advertised 'rigorous characterization' of mediator-induced failures inherits the same style of informal functional-argument gaps. The current value of the paper is that of an empirical method with an unproven heuristic rationale.

major comments (3)
  1. [Appendix E.11, Lemma 4.1, Subcase C] The proof of Subcase C uses an invalid marginal-to-joint independence step. After assuming h(X̃,Y) = αX̃ + βY is independent of Y, the text writes X = −(β/α)Y + (h(X̃,Y) − αεX)/α and claims that (h(X̃,Y) − αεX)/α is independent of Y because h⊥Y and Y⊥εX. This does not follow: h(X̃,Y) and εX are dependent through X̃, so independence of each with Y does not imply independence of their difference with Y. In fact h(X̃,Y) − αεX = αX + βY, which depends on Y whenever β ≠ 0. Thus the contradiction is not established, and Lemma 4.1, the linear anticausal ingredient of Theorem 4.2, is unproven.
  2. [Appendix E.12.2] The nonlinear anticausal proof is not rigorous. The argument infers from d-separation that εX is not independent of Y given X̃ that the conditional mean E[εX|X̃,Y] varies with Y; conditional dependence can live in higher moments, so this inference requires an additional assumption that is neither stated nor proved. The subsequent case analysis is also under-specified: the decomposition h = A1(X̃) + A2(Y) + A3(X̃,Y) and the claim that additive terms 'cannot cancel' interaction terms are assertions rather than derived functional-equation statements; the four subcases for invertible f are listed without proof that they are exhaustive or that they follow from the ANM; the statement 'Case 1 cannot occur, as it violates Assumption 2.2' is not tied to the formal content of Assumption 2.2; and the non-invertible case is dismissed in a single sentence. Since the anticausal inequality MI(ε̂X,Y) > 0 is the load-bearing half of Decision Rule 1, Theorem 4.2 is not established.
  3. [Appendix D.4, Lemma 2.3] The proof of Lemma 2.3 misapplies Pexider's equation. From Y = h(F(X,ε1,...,ε_{i−1})+ε_i)+ε̃ = A1(X)+A2(ε1,...,ε_i)+A3(ε̃), the text 'implies' h(F(...)+ε_i)+ε̃ = A1(X)+A2(ε1,...,ε_i), silently dropping the A3(ε̃) term; the dropped term cannot in general be absorbed into A2 because it is a function of a different independent variable. Moreover, Pexider's equation applies to functions of sums of independent variables, whereas F(X,ε1,...,ε_{i−1}) is a function of several variables and is not independent of the variables appearing in A1 and A2. Because Lemmas 3.1–3.3 invoke Lemma 2.3, the paper's characterization of when ANM-UM is irreducible to ANM is not rigorously supported.
minor comments (4)
  1. [Section 5.2] The text says 'varying depth (Figure 2a), or fixed depth (one mediator) and varying sample size (Figure 2a)'; the second reference should be to Figure 2b.
  2. [Section 4.3] The claim that the rescaled forward process √ᾱ A + √(1−ᾱ)ε preserves the theoretical independence results is terse; the conditional expectation of the rescaled noise is not literally the same object as E[εA|Ã,B], so the transfer from theory to implementation needs a short derivation.
  3. [Appendix E.11, Step 1] The statement that 'Assumption 2.2 stipulates that ε1 is non-Gaussian' is inaccurate; Assumption 2.2 as defined is the absence of a backward ANM-UM, and the non-Gaussian condition follows only in the linear identifiable case.
  4. [Table 3] The Tübingen results are reported as point estimates without confidence intervals or standard errors; given the random subsampling of each pair to n = 3000, the variability across subsamples should be quantified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BiDD's statistic is derived from the ANM DGP, not fit to direction labels; self-citations are non-load-bearing.

full rationale

The central consistency claim (Theorem 4.2) is not equivalent to its inputs by construction. In the causal direction, the paper derives E[εY | eY, X] = E[εY | ε1 + εY] from the ANM DGP and the mutual independence of X, ε1, and εY (Eqs. 4.4–4.5), so the independence of the predicted noise from X follows rather than being assumed. The anticausal claim (Lemma 4.1) is attempted as a proof by contradiction from the same DGP and Assumption 2.2; even if the proof's independence step in Subcase C (Appendix E.11) is invalid because it treats marginal independences as joint independence, and even if the E.12.2 case analysis is incomplete, these are proof gaps, not circular reductions. Decision Rule 1 is parameter-free: it compares the mutual information between predicted noise and the conditioning variable and is never fitted to direction labels. Hidden-mediation performance is explicitly presented as a conjecture ('we conjecture that Decision Rule 1 remains consistent') and supported only by simulations from the ANM-UM family, which is limited external validation rather than circular reasoning. The self-citations [13,14] appear only in a list of methodological advances and are not load-bearing for the bivariate result; no uniqueness theorem is imported from the authors' own prior work. The main risks are the unproven Lemma 4.1 and the incomplete anticausal case analysis, which are correctness issues outside this circularity pass.

Assumptions & free parameters 6 free parameters · 7 assumptions · 3 invented entities

The central claims rest on the ANM-UM model and standard identifiability assumptions, plus regularity conditions in the proofs. No free parameters are fitted to direction labels, but several hyperparameters are chosen by hand. The hidden-mediation capability is a conjecture supported by synthetic experiments generated from the same model family, which limits external support.

free parameters (6)
  • Diffusion timesteps T = 256
    Chosen by hand in Appendix F.2.2; not fitted to labels and not varied in ablations.
  • Oversampling factor k = 10
    Chosen by hand for mutual information estimation in Appendix F.1.2; affects test-time noise sampling count.
  • HSIC kernel bandwidth = Median pairwise distance heuristic
    Selected via median heuristic in Appendix F.2.1; ablation varies the scaling factor 0.5 and 2 in Appendix H.
  • Voting threshold = Majority of T timesteps
    Decision rule in Subroutine 3; an alternative mean rule is evaluated in Appendix H.
  • Training epochs E = 4000
    Chosen by hand in Appendix F.2.2; no convergence criterion reported.
  • Diffusion noise schedule = Linear beta from 0.0001 to 0.02
    Standard heuristic from Ho et al., chosen by hand; not fitted to the causal direction.
assumptions (7)
  • domain assumption ANM-UM generative model (Eq. 2.1): X and all noises are mutually independent, a chain of T unobserved mediators, and Y = f_{T+1}(Z_T) + epsilon_{T+1}.
    Defines the problem class; all theoretical and synthetic results assume this model.
  • domain assumption Assumption 2.1: no observed confounders, acyclicity, no selection bias, and X not independent of Y.
    Standard identifiability hygiene; stated in Section 2 and used throughout.
  • domain assumption Assumption 2.2: no backward ANM-UM representation exists for the observed pair.
    Needed to identify the direction; used directly in Lemma 4.1 Subcase C to rule out a backward linear model.
  • standard math Pexider's equation result from Aczel: C(x+y) = D(x) + E(y) forces C, D, E to be linear functions.
    Used to prove Lemma 2.3 in Appendix D.4.
  • domain assumption Density regularity: X, epsilon1, epsilonX have everywhere-positive, absolutely continuous, differentiable densities; f is continuous and three-times differentiable.
    Invoked in the proofs of Lemma 4.1 and Theorem 4.2 (Appendices E.11 and E.12).
  • standard math d-separation in the graph epsilonX -> Xtilde <- X -> Y implies epsilonX is not independent of Y given Xtilde.
    Used in Appendix E.12.2 to argue the noise predictor is a nontrivial function of Y.
  • ad hoc to paper Case 1 in the nonlinear anticausal proof cannot occur because it would violate Assumption 2.2.
    Asserted without proof in Appendix E.12.2; this is part of the flawed case analysis.
invented entities (3)
  • Unobserved mediators Z_i (ANM-UM)
    purpose: Model the hidden chain between X and Y that breaks the ANM assumption.
    Latent variables are assumed, not predicted or measured; no falsifiable handle is offered.
  • Nonlinear mediator concept
    purpose: Characterize when ANM-UM is irreducible to ANM (Lemma 2.3).
    A definitional classification rather than a new physical entity.
  • BiDD independence test statistic MI(epsilon_hat_A, B) independent evidence
    purpose: Score for causal direction; choose the direction with lower dependence.
    The statistic is algorithmically defined and can be computed on any dataset; its predicted asymmetry for hidden mediation is a falsifiable empirical claim tested only on synthetic data.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery." pith.science (2026). https://pith.science/paper/IA2O4MA3

@misc{pith2026250623374,
  author       = {Pith},
  title        = {Pith review of: When Additive Noise Meets Unobserved Mediators: Bivariate Denoising Diffusion for Causal Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IA2O4MA3}},
  note         = {Machine review of arXiv:2506.23374}
}
read the original abstract

Distinguishing cause and effect from bivariate observational data is a foundational problem in many disciplines, but challenging without additional assumptions. Additive noise models (ANMs) are widely used to enable sample-efficient bivariate causal discovery. However, conventional ANM-based methods fail when unobserved mediators corrupt the causal relationship between variables. This paper makes three key contributions: first, we rigorously characterize why standard ANM approaches break down in the presence of unmeasured mediators. Second, we demonstrate that prior solutions for hidden mediation are brittle in finite sample settings, limiting their practical utility. To address these gaps, we propose Bivariate Denoising Diffusion (BiDD) for causal discovery, a method designed to handle latent noise introduced by unmeasured mediators. Unlike prior methods that infer directionality through mean squared error loss comparisons, our approach introduces a novel independence test statistic: during the noising and denoising processes for each variable, we condition on the other variable as input and evaluate the independence of the predicted noise relative to this input. We prove asymptotic consistency of BiDD under the ANM, and conjecture that it performs well under hidden mediation. Experiments on synthetic and real-world data demonstrate consistent performance, outperforming existing methods in mediator-corrupted settings while maintaining strong performance in mediator-free settings.

Figures

Figures reproduced from arXiv: 2506.23374 by the authors.

Figure 1
Figure 1. ANM-UM (Eq 2.1), where mediators Z1, . . . ZT and noises ε0, . . . εT +1 are all unobserved.    Z1 = f1(X) + ε1, Z2 = f2(Z1) + ε2, . . . ZT = fT (Pa(ZT )) + εT , Y = fT +1(Pa(Y )) + εT +1, (2.1) where X, {εi}i∈[T] are mutually independent. The functions {f1, . . . , fT +1} can be linear or nonlinear, and the ε can be arbitrary (Gaussian or non-Gaussian). Assumption 2.1 (ANM-UM Setting). Suppose X, Y f… view at source ↗
Figure 2
Figure 2. Tanh mechanism, uniform noise setting. accuracy 5/7 times. In contrast, all baselines except PNL and RESIT perform extremely poorly (≤ 50%) in at least two settings. PNL’s performance is significantly lower (∼ 10% − 20%) than BiDDTotal in almost every setting, while RESIT struggles in the neural network setting (≤ 70%). The other hidden mediator method, CANM, performs poorly (≤ 50%) for invertible mechanisms (linear… view at source ↗
Figure 3
Figure 3. Comparison between different training schedules for [PITH_FULL_IMAGE:figures/full_fig_p022_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Dependence in the Anticausal Direction: X˜ = X + εX and Y = f(X) + ε1. All root nodes are drawn from independent noise. Therefore, h(X, Y ˜ ) is always a function of a noise term dependent on Y . Suppose f is non-invertible. Then there exists no backwards model where X…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

80 extracted references · 66 canonical work pages

  1. [1]

    On applications and theory of functional equations

    Aczél, J. On applications and theory of functional equations . Academic Press, 2014

    work page 2014

  2. [2]

    M., Manibialoa, C., and McIsaac, F

    Addo, P. M., Manibialoa, C., and McIsaac, F. Exploring nonlinearity on the co2 emissions, economic production and energy use nexus: a causal discovery approach. Energy Reports, 7: 6196–6204, 2021

    work page 2021

  3. [3]

    DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization

    Bello, K., Aragam, B., and Ravikumar, P. DAGMA: Learning DAGs via M-matrices and a Log-Determinant Acyclicity Characterization. In Advances in Neural Information Processing Systems, 2022

    work page 2022

  4. [4]

    Cause-effect inference by comparing regression errors

    Bloebaum, P., Janzing, D., Washio, T., Shimizu, S., and Schoelkopf, B. Cause-effect inference by comparing regression errors. In Storkey, A. and Perez-Cruz, F. (eds.), Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics , volume 84 of Proceedings of Machine Learning Research , pp. 900–909. PMLR, 09–11 Apr 201...

    work page 2018

  5. [5]

    CAM: Causal additive models, high-dimensional order search and penalized regression

    Bühlmann, P., Peters, J., and Ernest, J. CAM: Causal additive models, high-dimensional order search and penalized regression. The Annals of Statistics, 42(6), December 2014. ISSN 0090-5364. doi: 10.1214/14-AOS1260. URL http://arxiv.org/abs/1310.1533. arXiv:1310.1533 [cs, stat]

    arXiv 2014

  6. [6]

    Causal discovery with cascade nonlinear additive noise models

    Cai, R., Qiao, J., Zhang, K., Zhang, Z., and Hao, Z. Causal discovery with cascade nonlinear additive noise models. arXiv preprint, arXiv:1905.09442, 2019

    arXiv 1905

  7. [7]

    Fom: Fourth-order moment based causal direction identification on the heteroscedastic data

    Cai, R., Ye, J., Qiao, J., Fu, H., and Hao, Z. Fom: Fourth-order moment based causal direction identification on the heteroscedastic data. Neural Networks, 124:193–201, 2020. ISSN 0893-6080. doi: https://doi.org/10.1016/j.neunet.2020.01.006. URL https://www.sciencedirect. com/science/article/pii/S0893608020300083

    work page doi:10.1016/j.neunet.2020.01.006 2020

  8. [8]

    Chickering, D. M. Learning Equivalence Classes of Bayesian Network Structures. Journal of Machine Learning Research, 2013

    work page 2013

Show all 80 references
  1. [9]

    Cyclical annealing schedule: A simple approach to mitigating kl vanishing

    Fu, H., Li, C., Liu, X., Gao, J., Celikyilmaz, A., and Carin, L. Cyclical annealing schedule: A simple approach to mitigating kl vanishing. In Burstein, J., Doran, C., and Solorio, T. (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association f...

    2019 doi

  2. [10]

    A kernel statistical test of independence

    Gretton, A., Fukumizu, K., Teo, C., Song, L., Schölkopf, B., and Smola, A. A kernel statistical test of independence. Advances in neural information processing systems , 20, 2007

    2007

  3. [11]

    Lagging inference networks and posterior collapse in variational autoencoders

    He, J., Spokoyny, D., Neubig, G., and Berg-Kirkpatrick, T. Lagging inference networks and posterior collapse in variational autoencoders. arXiv preprint arXiv:1901.05534, 2019

    1901 arXiv

  4. [12]

    Beta-vae: Learning basic visual concepts with a constrained variational framework

    Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. Beta-vae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations , 2017

    2017

  5. [13]

    Hybrid top-down global causal discovery with local search for linear and nonlinear additive noise models

    Hiremath, S., Maasch, J., Gao, M., Ghosal, P., and Gan, K. Hybrid top-down global causal discovery with local search for linear and nonlinear additive noise models. NeurIPS 2024, 2024. URL https://arxiv.org/abs/2405.14496. https://arxiv.org/abs/2405.14496

    2024 arXiv

  6. [14]

    Losam: Local search in additive noise models with mixed mechanisms and general noise for global causal discovery

    Hiremath, S., Ghosal, P., and Gan, K. Losam: Local search in additive noise models with mixed mechanisms and general noise for global causal discovery. UAI 2025, 2025. URL https://arxiv.org/abs/2410.11759. https://arxiv.org/abs/2410.11759

    2025 arXiv

  7. [15]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

    2020

  8. [16]

    O., Janzing, D., Mooij, J

    Hoyer, P. O., Janzing, D., Mooij, J. M., Peters, J., and Schölkopf, B. Nonlinear causal discovery with additive noise models. In Advances in Neural Information Processing Systems 21 (NIPS 2008), 2008

    2008

  9. [17]

    Generalized Score Functions for Causal Discovery

    Huang, B., Zhang, K., Lin, Y., Schölkopf, B., and Glymour, C. Generalized Score Functions for Causal Discovery. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pp. 1551–1560, London United Kingdom, July 2018. ACM. ISBN 978-...

    2018

  10. [18]

    E., Schölkopf, B., Bühlmann, P., and Marx, A

    Immer, A., Schultheiss, C., Vogt, J. E., Schölkopf, B., Bühlmann, P., and Marx, A. On the Identifiability and Estimation of Causal Location-Scale Noise Models, June 2023. URL http: //arxiv.org/abs/2210.09054. arXiv:2210.09054 [cs, stat]

    2023 arXiv

  11. [19]

    and Scholkopf, B

    Janzing, D. and Scholkopf, B. Causal Inference Using the Algorithmic Markov Condition. IEEE Transactions on Information Theory , 56(10):5168–5194, October 2010. ISSN 0018-9448, 1557-9654. doi: 10.1109/TIT.2010.2060095. URL http://ieeexplore.ieee.org/ document/5571886/

    2010

  12. [20]

    Information-geometric approach to inferring causal directions

    Janzing, D., Mooij, J., Zhang, K., Lemeire, J., Zscheischler, J., Daniušis, P., Steudel, B., and Schölkopf, B. Information-geometric approach to inferring causal directions. Artificial Intelli- gence, 182-183:1–31, May 2012. ISSN 00043702. doi: 10.1016/j.artint.2012.01.002. UR...

    2012 doi

  13. [21]

    Identifying confounders using additive noise models

    Janzing, D., Peters, J., Mooij, J., and Schölkopf, B. Identifying confounders using additive noise models. arXiv preprint arXiv:1205.2640, 2012

    2012 arXiv

  14. [22]

    Elements of causal inference: foundations and learning algorithms

    Jonas Peters, Dominik Janzing, and Bernhard Scholkopf. Elements of causal inference: foundations and learning algorithms . The MIT Press, Cambridge, Massachusetts, 2017. ISBN 978-0-262-03731-0. URL https://mitpress.mit.edu/9780262037310/ elements-of-causal-inference/

    2017

  15. [23]

    Learning to induce causal structure

    Ke, N., Chiappa, S., Wang, J., Bornschein, J., Goyal, A., Rey, M., Weber, T., Botvinick, M., Mozer, M., and Rezende, D. Learning to induce causal structure. International Conference on Learning Representations, 2023

    2023

  16. [24]

    The UCI Machine Learning Repository

    Kelly, M., Longjohn, R., and Nottingham, K. The UCI Machine Learning Repository. https: //archive.ics.uci.edu, 2025. Accessed 22 May 2025. 11

    2025

  17. [25]

    Kolmogorov, A. N. Three approaches to the quantitative definition ofinformation’. Problems of information transmission, 1(1):1–7, 1965

    1965

  18. [26]

    Estimating mutual information.Physical Review E, 69(6):066138, June 2004

    Kraskov, A., Stögbauer, H., and Grassberger, P. Estimating mutual information.Physical Review E, 69(6):066138, June 2004. ISSN 1539-3755, 1550-2376. doi: 10.1103/PhysRevE.69.066138. URL https://link.aps.org/doi/10.1103/PhysRevE.69.066138

    2004 doi

  19. [27]

    Greedy Relaxations of the Sparsest Permuta- tion Algorithm, 2022

    Lam, W.-Y., Andrews, B., and Ramsey, J. Greedy Relaxations of the Sparsest Permuta- tion Algorithm, 2022. URL https://proceedings.mlr.press/v180/lam22a/ lam22a.pdf

    2022

  20. [28]

    J., Srinivasan, R., Ong, C

    Lee, J. J., Srinivasan, R., Ong, C. S., Alejo, D., Schena, S., Shpitser, I., Sussman, M., Whit- man, G. J., and Malinsky, D. Causal determinants of postoperative length of stay in cardiac surgery using causal graphical learning. The Journal of Thoracic and Cardio- vascular Sur...

    2022

  21. [29]

    Efficient Neural Causal Discovery without Acyclic- ity Constraints, February 2022

    Lippe, P., Cohen, T., and Gavves, E. Efficient Neural Causal Discovery without Acyclic- ity Constraints, February 2022. URL http://arxiv.org/abs/2107.10483. arXiv:2107.10483 [cs, stat]

    2022 arXiv

  22. [30]

    Causal discovery via conditional independence testing with proxy variables

    Liu, M., Sun, X., Qiao, Y., and Wang, Y. Causal discovery via conditional independence testing with proxy variables. ICML 2024, 2024. URL https://arxiv.org/pdf/2305. 05281

    2024

  23. [31]

    Maeda, T. N. and Shimizu, S. Causal Additive Models with Unobserved Variables. InProceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence , pp. 10, 2021

    2021

  24. [32]

    and Vreeken, J

    Marx, A. and Vreeken, J. Identifiability of cause and effect using regularized regression. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 852–861, 2019

    2019

  25. [33]

    Miao, W., Geng, Z., and Tchetgen Tchetgen, E. J. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika, 105(4):987–993, 2018. ISSN 0006-3444

    2018

  26. [34]

    Identifying effects of multiple treat- ments in the presence of unmeasured confounding

    Miao, W., Hu, W., Ogburn, E., and Zhou, X.-H. Identifying effects of multiple treat- ments in the presence of unmeasured confounding. Journal of American Statistical As- sociation, 2022. URL https://www.tandfonline.com/doi/full/10.1080/ 01621459.2021.2023551#abstract

    2022

  27. [35]

    Causal Discovery with Score Matching on Additive Models with Arbitrary Noise

    Montagna, F., Noceti, N., Rosasco, L., Zhang, K., and Locatello, F. Causal Discovery with Score Matching on Additive Models with Arbitrary Noise. In Proceedings of the 2nd Conference on Causal Learning and Reasoning . arXiv, April 2023. URL http://arxiv.org/abs/ 2304.03265. ar...

    2023 arXiv

  28. [36]

    Scalable Causal Discovery with Score Matching

    Montagna, F., Noceti, N., Rosasco, L., Zhang, K., and Locatello, F. Scalable Causal Discovery with Score Matching. In Proceedings of the 2nd Conference on Causal Learning and Reasoning . arXiv, April 2023. URL http://arxiv.org/abs/2304.03382. arXiv:2304.03382 [cs, stat]

    2023 arXiv

  29. [37]

    M., Bloebaum, P., Kirschbaum, E., and Locatello, F

    Montagna, F., Faller, P. M., Bloebaum, P., Kirschbaum, E., and Locatello, F. Score matching through the roof: linear, nonlinear, and latent variables causal discovery. arXiv preprint arXiv:2407.18755, 2024

    2024 arXiv

  30. [38]

    and Janzing, D

    Mooij, J. and Janzing, D. Distinguishing between cause and effect. JMLR Workshop and Conference Proceedings, 6:147–156, 2010

    2010

  31. [39]

    Regression by dependence minimization and its application to causal inference in additive noise models

    Mooij, J., Janzing, D., Peters, J., and Schölkopf, B. Regression by dependence minimization and its application to causal inference in additive noise models. In Proceedings of the 26th annual international conference on machine learning , pp. 745–752, 2009. 12

    2009

  32. [41]

    M., Peters, J., Janzing, D., Zscheischler, J., and Scholkopf, B

    Mooij, J. M., Peters, J., Janzing, D., Zscheischler, J., and Scholkopf, B. Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks. Journal of Machine Learning Research, 17, 2016

    2016

  33. [42]

    M., Peters, J., Janzing, D., Zscheischler, J., and Schölkopf, B

    Mooij, J. M., Peters, J., Janzing, D., Zscheischler, J., and Schölkopf, B. Distinguishing cause from effect using observational data: methods and benchmarks. Journal of Machine Learning Research, 17(32):1–102, 2016

    2016

  34. [43]

    On the role of sparsity and dag constraints for learning linear dags

    Ng, I., Ghassami, A., and Zhang, K. On the role of sparsity and dag constraints for learning linear dags. Advances in Neural Information Processing Systems , 33:17943–17954, 2020

    2020

  35. [44]

    Identifiability of additive noise models using conditional variances.Journal of Machine Learning Research, 21(75):1–34, 2020

    Park, G. Identifiability of additive noise models using conditional variances.Journal of Machine Learning Research, 21(75):1–34, 2020

    2020

  36. [45]

    PyTorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Te- jani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. PyTorch: An impe...

    2019

  37. [46]

    Myth, confusion, and science in causal analysis

    Pearl, J. Myth, confusion, and science in causal analysis. Technical Report R-348, University of California, Los Angeles, Los Angeles, CA, 2009. URL http://ftp.cs.ucla.edu/ pub/stat_ser/r348.pdf

    2009

  38. [47]

    Causal Discovery with Continuous Additive Noise Models, April 2014

    Peters, J., Mooij, J., Janzing, D., and Schölkopf, B. Causal Discovery with Continuous Additive Noise Models, April 2014. URL http://arxiv.org/abs/1309.6779. arXiv:1309.6779 [stat]

    2014 arXiv

  39. [48]

    Elements of causal inference: foundations and learning algorithms

    Peters, J., Janzing, D., and Schölkopf, B. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017

    2017

  40. [49]

    N., and Shimizu, S

    Pham, T., Maeda, T. N., and Shimizu, S. Causal additive models with unobserved causal paths and backdoor paths. arXiv preprint arXiv:2502.07646, 2025

    2025 arXiv

  41. [50]

    Beware of the simulated dag! causal discovery benchmarks may be easy to game

    Reisach, A., Seiler, C., and Weichwald, S. Beware of the simulated dag! causal discovery benchmarks may be easy to game. Advances in Neural Information Processing Systems , 34: 27772–27784, 2021

    2021

  42. [51]

    G., Tami, M., Seiler, C., Chambaz, A., and Weichwald, S

    Reisach, A. G., Tami, M., Seiler, C., Chambaz, A., and Weichwald, S. A Scale-Invariant Sorting Criterion to Find a Causal Order in Additive Noise Models. In 37th Conference on Neural Information Processing Systems . arXiv, October 2023. URL http://arxiv.org/abs/ 2303.18211. ar...

    2023 arXiv

  43. [52]

    Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models

    Rolland, P., Cevher, V., Kleindessner, M., Russel, C., Scholkopf, B., Janzing, D., and Locatello, F. Score Matching Enables Causal Discovery of Nonlinear Additive Noise Models. In Proceedings of the 39 th International Conference on Machine Learning , 2022

    2022

  44. [53]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 10684–10695, 2022

    2022

  45. [54]

    D., Muñoz-Marí, J., van Nes, E

    Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M. D., Muñoz-Marí, J., van Nes, E. H., Peters, J., Quax, R., Re- ichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., Zhang, K., and Zsch...

    2019 doi

  46. [55]

    A., and Nolan, G

    Sachs, K., Perez, O., Pe’er, D., Lauffenburger, D. A., and Nolan, G. P. Causal protein-signaling networks derived from multiparameter single-cell data. Science, 308(5721):523–529, 2005

    2005

  47. [56]

    Q., and Tsaftaris, S

    Sanchez, P., Liu, X., O’Neil, A. Q., and Tsaftaris, S. A. Diffusion models for causal discovery via topological ordering. arXiv preprint arXiv:2210.06201, 2022

    2022 arXiv

  48. [57]

    On causal and anticausal learning

    Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K., and Mooij, J. On causal and anticausal learning. arXiv preprint arXiv:1206.6471, 2012

    2012 arXiv

  49. [58]

    O., Hyvarinen, A., and Kerminen, A

    Shimizu, S., Hoyer, P. O., Hyvarinen, A., and Kerminen, A. A Linear Non-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research , 7:2003–2030, 2006

    2003

  50. [59]

    O., Bollen, K., and Hoyer, P

    Shimizu, S., Inazumi, T., Sogawa, Y., Hyvarinen, A., Kawahara, Y., Washio, T., Hoyer, P. O., Bollen, K., and Hoyer, P. Directlingam: A direct method for learning a linear non-gaussian structural equation model. Journal of Machine Learning Research-JMLR , 12(Apr):1225–1248, 2011

    2011

  51. [60]

    An Anytime Algorithm for Causal Inference

    Spirtes, P. An Anytime Algorithm for Causal Inference. InProceedings of the Eighth International Workshop on Artificial Intelligence and Statistics , volume R3, pp. 278–285. PMLR, 2001

    2001

  52. [61]

    Causation, Prediction, and Search , volume 81 of Lecture Notes in Statistics

    Spirtes, P., Glymour, C., and Scheines, R. Causation, Prediction, and Search , volume 81 of Lecture Notes in Statistics . Springer New York, New York, NY, 2000. ISBN 978-1-4612-7650-0 978-1-4612-2748-9. doi: 10.1007/978-1-4612-2748-9. URL http://link.springer. com/10.1007/978-...

    doi

  53. [62]

    Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery

    Tagasovska, N., Chavez-Demoulin, V., and Vatter, T. Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery. In International Conference on Machine Learning, pp. 9311–9323. PMLR, 2020

    2020

  54. [63]

    Turing, A. M. et al. On computable numbers, with an application to the entscheidungsproblem. J. of Math, 58(345-363):5, 1936

    1936

  55. [64]

    A., Marx, A., and Vreeken, J

    Xu, S., Mian, O. A., Marx, A., and Vreeken, J. Inferring cause and effect in the presence of heteroscedastic noise. In International Conference on Machine Learning , pp. 24615–24630. PMLR, 2022

    2022

  56. [65]

    Ordering-based causal discovery for linear and nonlinear relations

    Xu, Z., Li, Y., Liu, C., and Gui, N. Ordering-based causal discovery for linear and nonlinear relations. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  57. [66]

    and Qu, A

    Yuan, Y. and Qu, A. De-confounding causal inference using latent multiple-mediator pathways. Journal of the American Statistical Association, 119(547):2051–2065, 2023. doi: 10.1080/01621459. 2023.2240461. URL https://doi.org/10.1080/01621459.2023.2240461

    2023

  58. [67]

    and Hyvarinen, A

    Zhang, K. and Hyvarinen, A. On the Identifiability of the Post-Nonlinear Causal Model. Uncertainty in Artificial Intelligence, 2009

    2009

  59. [68]

    K., and Xing, E

    Zheng, X., Aragam, B., Ravikumar, P. K., and Xing, E. P. Dags with no tears: Continuous optimization for structure learning. Advances in neural information processing systems , 31, 2018

    2018

  60. [69]

    Learning sparse nonparametric dags

    Zheng, X., Dan, C., Aragam, B., Ravikumar, P., and Xing, E. Learning sparse nonparametric dags. In International Conference on Artificial Intelligence and Statistics , pp. 3414–3425. Pmlr, 2020

    2020

  61. [70]

    Kolmogorov complexity

    Zhu, Z., Locatello, F., and Cevher, V. Sample Complexity Bounds for Score- Matching: Causal Discovery and Generative Modeling, 2023. URL https: //proceedings.neurips.cc/paper_files/paper/2023/file/ 0a3dc35a2391cabcb59a6b123544e3db-Paper-Conference.pdf. 14 Appendix A Notation P...

    2023

  62. [71]

    First, we will restate the DGP of X, Y, εX , ˜X, and all assumptions

  63. [72]

    This proof will proceed in the following steps

    We define the regression function h( ˜X, Y) := E[εX | ˜X, Y]. This proof will proceed in the following steps. (a) We will characterize h( ˜X, Y), splitting it up into univariate functions and functions with interaction terms. (b) We will outline all possible cases (and subcase...

  64. [73]

    We will characterize h( ˜X, Y), showing that it must be a non-trivial function of Y and ˜X

  65. [74]

    We will outline all possible cases (and subcases), in whichh( ˜X, Y) is a non-trivial function of Y

  66. [75]

    For each case (and associated subcases) we will show that h( ˜X, Y) is a function of a noise term dependent on Y

  67. [76]

    We conclude that, ash( ˜X, Y) is always a function of noise dependent onY , h( ˜X, Y) ̸ ⊥ ⊥Y . 25 Note that h( ˜X, Y) can be decomposed as h( ˜X, Y) = A1( ˜X) + A2(Y ) + A3( ˜X, Y), where A3 contains only (linear or nonlinear) interaction between ˜X and Y , while A1, A2 are un...

  68. [77]

    X = f1(Y ) + e1, Y⊥ ⊥e1

  69. [78]

    X = f2(Y, e2), Y⊥ ⊥e2

  70. [79]

    X = f3(Y, e3), Y̸ ⊥ ⊥e3

  71. [80]

    Note that Case 1 cannot occur, as it violates Assumption 2.2 by allowing for the existence of a backwards model X → Y with additive noise

    X = f4(Y ) + e4, Y̸ ⊥ ⊥e4 We note that, in Case 2 and 3 functionsf2 and f3 induce nonlinear interactions between their inputs Y, e2 and e3. Note that Case 1 cannot occur, as it violates Assumption 2.2 by allowing for the existence of a backwards model X → Y with additive noise...

    2000

  72. [81]

    bring modest, but consistent, improvements. H.2 Conditioning vs Non-Conditioning H.2.1 Setup To test how conditioning impacts the performance of our methods, we provide an ablation study where we train on an unconditional loss. That means that the diffusion model does not have...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.