REVIEW 5 major objections 5 minor 35 references
Hybrid Causal Identification and Causal Mechanism Clustering
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A mixture of additive noise models is identifiable from pooled observational data, so comparing variational likelihood bounds in the two directions finds the true causal direction, and clustering the residual reveals the generating…
desk verdict The algorithmic combination is interesting and the empirical results are strong, but the identifiability proof has a fatal algebraic gap, so the causal decision rule currently rests on sand. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Hybrid Additive Noise Model (HANM), a weighted sum of $K$ additive noise mechanisms, together with the mixture conditional variational auto-encoder built to fit it. The identifiability argument reduces the existence of a reverse HANM to a third-order ordinary differential equation (Eq. 7) on $\xi = \log p(X)$: generically no reverse model can satisfy it, so the likelihood gap between the ELBO bounds (Eqs. 25 and 26) is the decision rule. For clustering, the residual $w\epsilon$ of the chosen direction is the extracted causal feature space, and the clustering objective minimizes the squared distance from residuals to cluster centers in that space.
What would settle it
Generate data from a known forward HANM with two near-linear components and small noise, then run MCVCI many times; if the forward and reverse ELBO scores are statistically indistinguishable across seeds, the reverse-identifiability premise fails.
Extended reading notes
Core claim
The central discovery is a generic identifiability result for the Hybrid Additive Noise Model (HANM), written as $Y = \sum_{k=1}^K w_k(f_k(x_k) + \epsilon_k)$ with $\sum w_k = 1$ and $x_k \perp\!\!\perp \epsilon_k$. The paper proves that if a forward HANM holds, a reverse HANM $X = \sum_{k=1}^K w_k(g_k(y_k) + \tilde\epsilon_k)$ can exist only when the log-density of $X$ satisfies a specific third-order ordinary differential equation constraint; generically no such reverse model exists. Building on this, the mixture conditional variational auto-encoder maximizes an ELBO that lower-bounds $\log p(Y|X)$, and the same bound in the reverse direction lower-bounds $\log p(X|Y)$; comparing the two scores gives the causal direction. The residual $w\epsilon$ in the winning direction is then used as the feature space for causal mechanism clustering.
Load-bearing premise
The proof that a reverse hybrid additive noise model is almost impossible collapses the mixture's $K$ components into single functions in Eqs. (10)-(11) and asserts genericity without a formal measure argument, so if the flexible neural regressor can approximate the reverse direction anyway, the two likelihood scores become nearly equal and the direction decision loses its basis.
Editorial extensions
If this is right
- Bivariate causality on pooled heterogeneous data can be decided without knowing which environment each sample came from.
- Cluster assignments can be derived from the data-generating mechanism rather than from raw feature distance, separating shifts in function or noise.
- The likelihood gap defines a confidence score $\tau$, so users can set a threshold and abstain on low-confidence pairs instead of committing to a direction.
- The variational regressor does not require Gaussian noise within a component, so the method covers nonlinear functions and non-Gaussian additive noise.
- On the SIM, SIM-G, SIM-ln, and cause-effect-pair benchmarks the reported direction accuracy exceeds the comparison methods, and on the simulated and air-quality clustering tasks the reported ARI and NMI exceed the baselines.
Reading between the lines
- A direct stress test the paper does not run: generate near-linear, low-noise two-component mixtures where the reverse regression is almost as simple as the forward one, and check whether the two ELBO scores become statistically indistinguishable; if they do, the generic identifiability claim needs a measure-theoretic qualification.
- The residual-clustering idea suggests a practical extension: the same trained model could detect unknown environment switches or label shifts by inspecting whether residuals naturally split into more clusters than the assumed $K$.
- Using the confidence threshold $\tau$ as an abstention rule in downstream applications would convert the top-k decision-rate curve into a deployable policy, but the paper does not propose such a procedure.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MCVCI, a method for bivariate causal direction identification on heterogeneous observational data, and MCVCC, an extension for clustering causal mechanisms. The model is a mixture of additive noise models (HANM); the authors claim an identifiability result (Lemma 1/Theorem 1) stating that a reverse HANM is 'almost impossible', and use it to justify a likelihood-ratio decision rule computed via a mixture conditional variational auto-encoder (CVAE). MCVCC clusters the residuals of the selected direction. Experiments on SIM/SIM-G/SIM-ln, CEP, and BAFU air data report accuracy/ARI/NMI improvements over several baselines, including ANM-MM, CANM, and classical methods. The central theoretical claim, however, rests on a proof that contains elementary algebraic errors in the log-density and derivative calculations, and the 'almost impossible' assertion lacks any measure-theoretic argument. The decision rule therefore lacks a valid identifiability foundation, and the empirical results, while reported as favorable, are not connected to a sound theoretical justification.
Significance. If the HANM identifiability theorem and the ELBO-based decision rule were correct, the paper would offer a practically useful extension of additive-noise causal inference to heterogeneous data, and the clustering application would be a modest novel contribution. The experimental coverage is broad, with several external benchmarks (SIM, SIM-G, SIM-ln, CEP) and a comparison against many recent methods, including ANM-MM and CANM. These strengths are real and deserve credit. However, the theoretical pillar—the identifiability of the HANM—is not established: the proof of Lemma 1 replaces the logarithm of a mixture by a weighted sum of logarithms, collapses sums of component derivatives into single functions without justification, and asserts genericity without a measure. Since the likelihood-ratio criterion depends entirely on this theorem, the central claim is unsupported. The paper does not provide machine-checked proofs or reproducible code, and the reported performance cannot compensate for the absence of a valid theoretical guarantee.
major comments (5)
- [Section III-C, Eq. (8)] The proof begins by writing the log-joint in the reverse model as π(X,Y)=Σ_k w_k v_k(y_k−f_k(x_k)) + ξ(X). But p(X,Y) is a mixture density, so log p(X,Y)=ξ(X)+log Σ_k w_k p_{ε_k}(y_k−f_k(x_k)). The paper replaces the log of the sum with the weighted sum of logs, which is not an identity. Every subsequent derivative in Eqs. (9)–(11) is therefore taken from an expression that is not the actual log-density; the derived ODE (7) and the conclusion of Lemma 1 do not follow.
- [Section III-C, Eq. (10)] Equation (10) collapses Σ_k w_k ev''_k(x_k−g_k(y_k)) g'_k(y_k) into a single function ev''(X−g(Y)). This step is valid only if all component functions g_k are identical (or under other unstated conditions that force the sum to form a single function of (X,Y)). No such condition is given. The same issue affects Eq. (11), where Σ_k w_k ev''_k is replaced by a single ev''. The derivation of Eq. (12) and the subsequent cancellation steps depends on this unjustified collapse, so the proof of Lemma 1 is incomplete at a load-bearing point.
- [Section III-C, Theorem 1] The theorem concludes that 'it is almost impossible to exist a hybrid ANM satisfying the condition from Y→X' and states that the set of solutions log p_X is contained in a three-dimensional affine space. No measure-theoretic argument is provided: there is no definition of the space of distributions or functions over which a measure is being taken, no statement of the genericity assumption, and no proof that the constraint set has measure zero. The 'almost impossible' claim is asserted rather than demonstrated. Since the likelihood-ratio decision rule in Eqs. (25)–(26) is justified solely by this genericity, the rule lacks a theoretical guarantee.
- [Section III-B and III-D, Eqs. (3), (5), (22)] The generative model is internally inconsistent. Equation (3) defines Y as a weighted sum of component functions, Y=Σ_k w_k(f_k(x_k)+ε_k), which is a single functional relationship. In contrast, Eq. (5) and the subsequent proof treat p(Y|X) as a mixture density Σ_k w_k p_{ε_k}(...), and Eq. (22) again claims log p(Y|X)=Σ_k w_k log p_{ε_k}(...). A weighted sum of functions is not the same as a mixture of conditional densities, and the logarithm of a mixture is not the mixture of logarithms. This inconsistency affects both the identifiability proof and the ELBO derivation: the variational model in Fig. 1 and Eq. (23) uses a mixture decomposition, while the causal criterion in Eqs. (25)–(26) is written as a sum of log-likelihoods. The relationship between the fitted CVAE and the stated HANM is therefore unclear, and the clustering objective in Eq. (27), which uses the residual from the selected direction, inherits this ambiguity.
- [Section III-D, Eqs. (23)–(26)] The ELBO in Eq. (24) is a lower bound for log pθ(Y|X) under a specific conditional generative model, but the paper does not show that this lower bound is a faithful proxy for the HANM marginal log-likelihood when comparing the two directions. In particular, the KL term and the encoder distribution qφ are optimized separately for the forward and reverse models, and the comparison in Algorithm 1 (step 7) uses the sum log p(X)+ELBO vs. log p(Y)+ELBO. Without an analysis of the approximation gap, two nearly equal ELBOs can arise even when the true likelihoods differ; conversely, a flexible VAE might approximate the reverse direction well, making the scores close and the decision unreliable. The empirical confidence analysis in Section IV-A4 measures only the decision rate of the top k%, not the validity of the underlying likelihood comparison.
minor comments (5)
- [Section IV-A4] The confidence paragraph contains a duplicated sentence: 'The higher the value of τ, the more correct our decision will be. Furthermore, we can set a threshold t to require τ ≥ t.' appears twice in a row. This should be cleaned up.
- [Throughout] Several typos and nonstandard notations occur, e.g., 'casual' for 'causal' in the abstract and Algorithm 1 outputs, 'eϵ' vs. 'ε' inconsistency, and 'wϵc' in Section III-E. Please proofread carefully.
- [Table IV] The header of Table IV repeats 'DIFFERENT CLUSTER NUMBERS CONDITION' from Table III; the actual setting (BAFU air data) should be stated in the caption.
- [References] The reference list is incomplete or inconsistent in places: reference [32] is formatted as 'S., Hoyer, et al.' with an unusual author order, and several cited works (e.g., the ANM-MM paper, the PNL paper) would benefit from full bibliographic data.
- [Section IV-B2] The description of the constructed simulated datasets is terse: it does not specify how the component index k is assigned to samples, how many samples per component are drawn, or how the mixture weights w_k are chosen. Please provide the full data-generation protocol so that the clustering experiments can be reproduced.
Circularity Check
No circularity found: the derivation chain is self-contained and the empirical evaluation is against external benchmarks.
full rationale
The paper's central derivation is an identifiability theorem for the Hybrid Additive Noise Model, followed by an ELBO-based likelihood comparison and a clustering method on the residual of the chosen direction. No step reduces to its own inputs by construction. The identifiability argument (Lemma 1 and Theorem 1, Eqs. 7-21) is derived internally from differential equations and genericity reasoning, not from a fitted parameter or from the paper's own prior results. The likelihood decision rule (Eqs. 25-26) is applied to held-out test sets on public benchmarks (SIM, SIM-G, SIM-ln, CEP), and the clustering method (MCVCC) is evaluated against ground-truth labels on synthetic data and the BAFU dataset; cluster labels are not used during training, so the empirical claims are self-contained. The residual features used for clustering are model outputs, not fitted parameters, so there is no fitted-input-called-prediction circularity. While the proof of Theorem 1 contains algebraic steps that appear mathematically incorrect (e.g., Eq. (8) replaces the log of a mixture density with a weighted sum of component log-densities), that is a correctness risk, not circularity, and is outside the scope of this analysis. No self-citations are load-bearing. Score 0.
Assumptions & free parameters
free parameters (3)
- K (number of mixture components) =
Not reported; selected per dataset during training
- C (number of clusters in MCVCC) =
2 (f1-f5, BAFU), 3 and 4 in Table III experiments
- Confidence threshold t =
Not specified
assumptions (4)
- domain assumption Observed data are generated by a HANM with a finite, known number of components K, with each component an additive noise model.
- ad hoc to paper A reverse HANM is 'almost impossible': the set of joint distributions admitting both forward and reverse HANM has measure zero.
- ad hoc to paper The ELBO of the mixture conditional VAE is a faithful proxy for the true conditional log-likelihood when comparing X->Y vs Y->X.
- domain assumption The residual theta = X - eX or Y - eY from the selected causal direction is a sufficient feature for causal mechanism clustering.
Cite this review
Pith. "Pith review of Hybrid Causal Identification and Causal Mechanism Clustering." pith.science (2026). https://pith.science/paper/EPDE24HK
@misc{pith2026250721792,
author = {Pith},
title = {Pith review of: Hybrid Causal Identification and Causal Mechanism Clustering},
year = {2026},
howpublished = {\url{https://pith.science/paper/EPDE24HK}},
note = {Machine review of arXiv:2507.21792}
}
read the original abstract
Bivariate causal direction identification is a fundamental and vital problem in the causal inference field. Among binary causal methods, most methods based on additive noise only use one single causal mechanism to construct a causal model. In the real world, observations are always collected in different environments with heterogeneous causal relationships. Therefore, on observation data, this paper proposes a Mixture Conditional Variational Causal Inference model (MCVCI) to infer heterogeneous causality. Specifically, according to the identifiability of the Hybrid Additive Noise Model (HANM), MCVCI combines the superior fitting capabilities of the Gaussian mixture model and the neural network and elegantly uses the likelihoods obtained from the probabilistic bounds of the mixture conditional variational auto-encoder as causal decision criteria. Moreover, we model the casual heterogeneity into cluster numbers and propose the Mixture Conditional Variational Causal Clustering (MCVCC) method, which can reveal causal mechanism expression. Compared with state-of-the-art methods, the comprehensive best performance demonstrates the effectiveness of the methods proposed in this paper on several simulated and real data.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Domain adaptation under structural causal models,
Chen, Yuansi, and Peter B ¨uhlmann, “Domain adaptation under structural causal models,” J. Mach. Learn. Res. , vol. 22, no. 1, pp. 11856-11935, 2021. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 8 TABLE IV CLUSTERING RESULTS OF COMPARISON METHODS AND MCVCC ON DIFFERENT CLUSTER NUMBERS CONDITION. k-means SpeClu GMM CV AE-km ANM-MM MCVCC ARI -0...
work page 2021
-
[2]
Stable learning establishes some common ground between causal inference and ma-chine learning,
Cui, Peng, and Susan Athey, “Stable learning establishes some common ground between causal inference and ma-chine learning,” Nat. Mach. Intell., vol. 4, no. 2, pp. 110-115, 2022
work page 2022
-
[3]
Causal discovery on discrete data with extensions to mixture model,
Liu, Furui, and Laiwan Chan, “Causal discovery on discrete data with extensions to mixture model,” ACM Trans. Intell. Syst. Technol. , vol. 7, no. 2, pp. 1-19, 2015
work page 2015
-
[4]
Pearl, Judea, Causality, Cambridge university press, 2009
work page 2009
-
[5]
On the identifiability of the post- nonlinear causal model,
Zhang, Kun, and Aapo Hyvarinen, “On the identifiability of the post- nonlinear causal model,” arXiv:1205.2599, 2012
arXiv 2012
-
[6]
Nonlinear causal discovery with additive noise models,
Hoyer, Patrik, et al., “Nonlinear causal discovery with additive noise models,” Adv. Neural Inf. Process. Syst. , vol. 21, 2008
work page 2008
-
[7]
Information-geometric ap- proach to inferring causal directions,
Janzing, Dominik, and Bernhard Sch ¨olkopf, “Information-geometric ap- proach to inferring causal directions,” J. Artif. Intell. , vol. 182, pp. 1- 31,2012
work page 2012
-
[8]
Cause-effect inference by comparing regression errors,
Bl ¨obaum, Patrick, et al, “Cause-effect inference by comparing regression errors,” in Proc. Int. Artif. Intell. Statist. , pp. 900-909, 2018
work page 2018
Show all 35 references
-
[9]
Adversarial orthogonal regression: Two non-linear regressions for causal inference,
Heydari, M. Reza, Saber Salehkaleybar, and Kun Zhang, “Adversarial orthogonal regression: Two non-linear regressions for causal inference,” Neural Netw., vol. 143, pp. 66-73, 2021
2021
-
[10]
Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery,
Tagasovska, Natasa, Val ´erie Chavez-Demoulin, and Thibault Vatter, “Distinguishing cause from effect using quantiles: Bivariate quantile causal discovery,” in Proc. Int. Conf. Mach Learn. , PMLR, 2020
2020
-
[11]
Causal Inference with Heteroscedastic Noise Mod- els,
Xu, Sascha, et al., “Causal Inference with Heteroscedastic Noise Mod- els,” in Proc. of AAAI Workshop Inf. Theoretic Causal Inference and Discovery, 2022
2022
-
[12]
Causal inference and mechanism clustering of a mixture of additive noise models,
Hu, Shoubo, et al., “Causal inference and mechanism clustering of a mixture of additive noise models,” Adv. Neural Inf. Process. Syst. , vol. 31, 2018
2018
-
[13]
Pearl, Judea, Models, reasoning and inference , Cambridge, UK: Cam- bridge University Press, vol. 19, no. 2, pp. 3, 2000
2000
-
[14]
Multi-source causal feature selection,
Yu, Kui, et al., “Multi-source causal feature selection,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 42, no. 9, pp. 2240-2256, 2019
2019
-
[15]
Learning structured output representation using deep conditional generative models,
Sohn, Kihyuk, Honglak Lee, and Xinchen Yan, “Learning structured output representation using deep conditional generative models,” Adv. Neural Inf. Process. Syst. , vol. 28, 2015. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 9 (a) (b) (c) (d) (e) (f) (g) Fig. 5. Th...
2015
-
[16]
Identifying cause and effect on discrete data using additive noise models,
Peters, Jonas, Dominik Janzing, and Bernhard Sch ¨olkopf. “Identifying cause and effect on discrete data using additive noise models,” in Proc. 9th Int. Workshop Artif. Intell. Statist. , 2010
2010
-
[17]
Telling cause from effect using MDL-based local and global regression,
Marx, Alexander, and Jilles Vreeken. “Telling cause from effect using MDL-based local and global regression,” in Proc. IEEE Int.Conf. Data Mining, IEEE, pp. 307-316, 2017
2017
-
[18]
Identifiability of cause and effect using regularized regression,
Marx, Alexander, and Jilles Vreeken. “Identifiability of cause and effect using regularized regression,” in Proc. 25th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining , pp. 852-861, 2019
2019
-
[19]
Causal network learning with non- invertible functional relationship,
Wang, Bingling, and Qing Zhou, “Causal network learning with non- invertible functional relationship,” Comput. Stat. Data Anal., vol. 156, pp. 107141, 2021
2021
-
[20]
Causal discov- ery toolbox: Uncovering causal relationships in python,
Kalainathan, Diviyan, Olivier Goudet, and Ritik Dutta, “Causal discov- ery toolbox: Uncovering causal relationships in python,” J. Mach. Learn. Res., vol. 21, no. 1, pp.1406-1410, 2020
2020
-
[21]
Distance-based clustering of CGH data,
Liu, Jun, et al., “Distance-based clustering of CGH data,” Bioinformat- ics, vol. 22, no. 16, pp. 1971-1978, 2006
1971
-
[22]
Causal discovery with continuous additive noise models,
Peters, Jonas, et al., “Causal discovery with continuous additive noise models,” J. Mach. Learn. Res. , 2014
2014
-
[23]
Causal discovery with cascade nonlinear additive noise models,
Cai, Ruichu, et al., “Causal discovery with cascade nonlinear additive noise models,” arXiv:1905.09442, 2019
1905 arXiv
-
[24]
Distinguishing cause from effect using obser- vational data: methods and benchmarks,
Mooij, Joris M., et al., “Distinguishing cause from effect using obser- vational data: methods and benchmarks,” J. Mach. Learn. Res. , vol. 17, no. 1, pp. 1103-1204, 2016
2016
-
[25]
Inference of cause and effect with unsupervised inverse regression,
Sgouritsa, Eleni, et al., “Inference of cause and effect with unsupervised inverse regression,” Artif. Intell. Statist. , PMLR, 2015
2015
-
[26]
Jonas Peters
Peter B ¨uhlmann. Jonas Peters. Jan Ernest. ”CAM: Causal additive models, high-dimensional order search and penalized regression.” Ann. Statist., 42 (6) 2526 - 2556, December 2014. https://doi.org/10.1214/14- AOS1260
2014 doi
-
[27]
Some methods for classification and analysis of multivariate observations,
MacQueen, James, “Some methods for classification and analysis of multivariate observations,” in Proc. Fifth Berkeley Symp. Math. Statist. and Probab., vol. 1, no. 14, 1967
1967
-
[28]
On spectral clustering: Analysis and an algorithm,
Ng, Andrew, Michael Jordan, and Yair Weiss., “On spectral clustering: Analysis and an algorithm,” in Adv. Neural Inf. Process. Syst. , vol. 14, 2001
2001
-
[29]
The infinite Gaussian mixture model,
Rasmussen, Carl, “The infinite Gaussian mixture model,” in Adv. Neural Inf. Process. Syst. , vol. 12, 1999
1999
-
[30]
Normalized cuts and image segmenta- tion,
Shi, Jianbo, and Jitendra Malik, “Normalized cuts and image segmenta- tion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, 2000
2000
-
[31]
Comparing partitions,
Hubert, Lawrence, and Phipps Arabie, “Comparing partitions,” J. Clas- sif., vol. 2, pp. 193-218, 1985
1985
-
[32]
A linear non-Gaussian acyclic model for causal discovery,
S., Hoyer, et al.,“A linear non-Gaussian acyclic model for causal discovery,” J. Mach. Learn. Res. , vol. 7, no. 10, 2006
2006
-
[33]
Auto-encoding variational bayes,
Kingma, Diederik P., and Max Welling, “Auto-encoding variational bayes,” arXiv:1312.6114, 2013
2013 arXiv
-
[34]
Explanation in artificial intelligence: Insights from the social sciences,
Miller, Tim, “Explanation in artificial intelligence: Insights from the social sciences,” Artif. Intell., vol. 267, pp. 1-38, 2019. Saixiong Liu Saixiong Liu is currently pursuing the Ph.D.degree at the School of Computer Science and Information Engineering and working at the ...
2019
-
[2020]
She was a visiting scholar with the City University of Hong Kong, Hong Kong, China, in 2019
She is currently a teacher with the School of Information, Shanxi University of Finance and Economics. She was a visiting scholar with the City University of Hong Kong, Hong Kong, China, in 2019. Her research interests include associations mining, multimodal learning, data min...
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.