REVIEW 2 major objections 4 minor 47 references
Structure Learning on Clustered Data
T0 review · 2 major / 4 minor · reviewed 2026-07-10 · grok-4.5
Pith's one-line read DAG structure learning can recover a shared global graph while letting edge strengths vary by cluster through fixed and random effects.
desk verdict Solid continuous mixed-DAG method with a clean union-acyclicity constraint and the first structure-recovery consistency result for this setting; modeling assumptions are the main limit, not the math. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Differentiable graph coupling: the log-determinant acyclicity function applied to B⊙B+Γ, which vanishes if and only if the union of the fixed-effect graph and the random-effect graph is acyclic.
What would settle it
Generate clustered data from a mixed DAG with known unequal node noise variances or correlated random effects across parents; if the estimator still claims unique recovery of the true edge sets, the identifiability claim is false.
Extended reading notes
Core claim
Under a linear Gaussian mixed structural equation model, the population fixed-effect matrix and the random-effect variance matrix are jointly identifiable from clustered observations whenever their union graph is a DAG, and an ℓ1-penalized continuous estimator recovers both the parameters and the edge sets consistently as the number of clusters tends to infinity.
Load-bearing premise
The model assumes every node has the same unit noise variance and that random effects for different parents of a node are independent; if either fails, uniqueness of the fixed- and random-effect parameters can break.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes mixed DAGs for structure learning on clustered data: each cluster follows a linear Gaussian SEM X^(i)=X^(i)W^(i)+ε^(i) with W^(i)=B+U^(i), where B is a sparse fixed-effects adjacency and the columns of U^(i) are independent N(0,diag(γ_k)) random effects. The estimator minimizes a nodewise negative log-likelihood (Eq. 2) plus ℓ1 penalties on B and Γ subject to the union graph G(B)∪G(Γ) being acyclic. Acyclicity is enforced by the differentiable coupling h(B,Γ)=-log det(I-B⊙B-Γ) (Eq. 4, Prop. 1). A path-following proximal-gradient algorithm (Algs. 1–2) is shown to produce decreasing objective values with vanishing successive differences (Thm. 3). Identifiability of (B0,Γ0) from the clustered distribution is proved under the equal-noise, diagonal-RE model (Thm. 4); parameter and structure recovery consistency as m o∞ follow under standard ℓ1 and irrepresentability conditions (Thm. 5). Synthetic, semi-synthetic (ANDES) and a small proteomics example show gains over fixed-effect and masked baselines.
Significance. Clustered heterogeneity is common in biomedical and longitudinal settings yet largely unaddressed by continuous DAG methods. Extending the fixed/random-effects decomposition to weighted adjacency matrices, together with a lossless union-acyclicity constraint and a first-order algorithm with batched likelihood evaluations, is a natural and practically useful contribution. The paper supplies complete proofs of the continuous constraint, proximal convergence, identifiability and structure recovery (via primal-dual witness), multi-replicate experiments with standard errors, and an asymptotic guarantee that is new for mixed DAGs. These strengths make the work a solid addition to scalable causal discovery under known cluster structure, provided the modeling assumptions are clearly scoped.
major comments (2)
- The likelihood (Eq. 2), the KL uniqueness argument of Theorem 4 / Appendix D, and the Hessian/irrepresentability conditions of Assumption 2 all rest on equal unit noise variances (σ_k^{2}=1) and independent diagonal random-effect covariances. The manuscript notes that node-specific variances can be accommodated by a simple rescaling, but never states the corresponding identifiability result or the modified irrepresentability conditions. Because these assumptions are load-bearing for the central statistical claims, the paper should either (i) prove that Theorems 4–5 continue to hold under free σ_k^{2} (and, ideally, under a modest relaxation of the diagonal-RE restriction) or (ii) explicitly restrict the claims to the equal-noise, diagonal-RE model class and discuss the practical consequences of misspecification.
- Section 6 and Appendix F rely on a post-hoc hard threshold of 0.05 and a polishing re-optimization that removes both the ℓ1 penalties and the acyclicity constraint. Theorem 5 concerns the exact minimizer of the constrained program (3)/(8). The manuscript never shows that the thresholded/polished estimator inherits the same asymptotic guarantees, nor does it report sensitivity of SHD/F1 to the threshold value. A short argument or numerical check that the reported recovery rates remain valid after these post-processing steps is needed for the experimental claims to support the theory.
minor comments (4)
- Figure 2 reports wall-clock times only for the full path-following procedure; a brief breakdown of the cost of the batched Cholesky/Woodbury steps versus the log-det acyclicity term would help readers assess scalability claims for large m or unbalanced ni.
- The real-data example (p=6, Figure 5) is useful for visualization but too small to illustrate the method’s claimed ability to handle hundreds of nodes; a short remark on this limitation would set expectations.
- Notation for the spectral-radius domain D (Eq. 5) and the feasible set F (Eq. 8) is introduced late; a single early display of the constrained program would improve readability.
- Code and data-generation scripts are not mentioned; releasing them would strengthen reproducibility of the 30-replicate experiments.
Circularity Check
No significant circularity: identifiability and consistency are derived from the external SEM DGP under stated assumptions, not by construction from fitted quantities or self-citation chains.
full rationale
The paper's central claims (Proposition 1 on the union-acyclicity characterization, Theorems 4–5 on identifiability and asymptotic structure recovery) are proved from first principles relative to an external linear-Gaussian mixed SEM data-generating process with equal unit noise variances and diagonal random-effect covariances. The continuous constraint h(B,Γ)=−logdet(I−B⊙B−Γ) is an explicit, lossless extension of the external log-det characterization of Bello et al. (2022); its equivalence proof (Appendix A) does not presuppose the estimator. The negative log-likelihood (2), proximal path-following algorithm, and convergence guarantee (Theorem 3) are standard first-order constructions whose fixed points are not defined in terms of the recovered graphs. Consistency (Theorem 5) is classical penalized-likelihood asymptotics (uniform LLN, local strong convexity, primal-dual witness under irrepresentability) relative to the true (B0,Γ0) of the DGP, not a quantity fitted from the same data and then re-labeled a prediction. Self-citations (Thompson et al. on contextual/variational DAGs and high-dimensional mixed models) appear only in related-work and background sections and are not invoked as uniqueness theorems that force the present results. Synthetic experiments generate data from the same SEM and therefore test recovery under the model rather than circularly re-predicting fitted constants. The equal-noise and diagonal-RE assumptions are modeling restrictions (correctly flagged by the reader) but do not create definitional circularity inside the derivation chain. Score 1 reflects only the ordinary presence of author self-citations that are non-load-bearing.
Assumptions & free parameters
free parameters (4)
- λ1, λ2 (sparsity penalties)
- Hard-threshold 0.05 on |βjk| and γjk
- Continuation path μ^(t)
- Box bounds MB, MΓ on ||B||∞, ||Γ||∞
assumptions (5)
- domain assumption Linear Gaussian SEM with equal unit noise variances σ²_k=1 and independent diagonal random effects u_k^(i)~N(0,diag(γ_k)).
- standard math h(W)=-log det(I-W)=0 iff G(W) is a DAG on the spectral-radius domain (Bello et al. 2022).
- domain assumption Population Hessian H=∇²L(B0,Γ0) positive definite and irrepresentability inequalities with gap η∈(0,1) (Assumption 2).
- domain assumption Clusters X^(1),…,X^(m) iid with bounded common size n; λ1,λ2→0 with √m λj→∞ and λ2/λ1→κ∈(0,∞) (Assumption 1).
- ad hoc to paper Union graph G(B)∪G(Γ) is the correct acyclicity object for cluster-specific W(i)=B+U(i).
invented entities (2)
-
Mixed DAG (W(i)=B+U(i) with sparse fixed B and sparse variance Γ)
-
Differentiable graph coupling h(B,Γ)=-log det(I-B⊙B-Γ)
Cite this review
Pith. "Pith review of Structure Learning on Clustered Data." pith.science (2026). https://pith.science/paper/VJKHZAOC
@misc{pith2026260708238,
author = {Pith},
title = {Pith review of: Structure Learning on Clustered Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/VJKHZAOC}},
note = {Machine review of arXiv:2607.08238}
}
read the original abstract
Recent algorithmic advances have made directed acyclic graph (DAG) structure learning scalable for causal discovery. Yet, the currently available techniques assume a completely homogeneous population, precluding their application to clustered data where cluster-specific variations (e.g., patient-specific effects) are common. We address this issue by introducing a new approach that estimates a global structure while accounting for local cluster-level effects. The key idea is to extend the fixed- and random-effects framework of classical mixed models to the structure learning setting. Towards this end, we present a differentiable graph coupling mechanism that guarantees the union of the fixed- and random-effects graphs remains acyclic. Computationally, we provide a provably convergent first-order method and leverage efficient batched updates across clusters. Statistically, we establish identifiability of the model and show that our approach recovers the true structure asymptotically. In experiments on real and synthetic data, our proposal detects dependencies missed by alternative estimators, underscoring its value for structure learning in clustered settings.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Kevin Bello and Bryon Aragam and Pradeep Ravikumar , booktitle =
-
[2]
Xun Zheng and Bryon Aragam and Pradeep Ravikumar and Eric P Xing , booktitle =
-
[3]
Contextual directed acyclic graphs , volume =
Ryan Thompson and Edwin V Bonilla and Robert Kohn , booktitle =. Contextual directed acyclic graphs , volume =
-
[4]
Bayesian graphical regression , volume =
Yang Ni and Francesco C Stingo and Veerabhadran Baladandayuthapani , number =. Bayesian graphical regression , volume =. Journal of the American Statistical Association , pages =
-
[5]
Integrating random effects in deep neural networks , volume =
Giora Simchoni and Saharon Rosset , journal =. Integrating random effects in deep neural networks , volume =
-
[6]
A relaxation approach to feature selection for linear mixed effects models , volume =
Aleksei Sholokhov and James V Burke and Damian F Santomauro and Peng Zheng and Aleksandr Aravkin , number =. A relaxation approach to feature selection for linear mixed effects models , volume =. Journal of Computational and Graphical Statistics , pages =
-
[7]
Ryan Thompson and Matt P Wand and Joanna J J Wang , title =
-
[8]
Exact estimation of multiple directed acyclic graphs , volume =
Chris J Oates and Jim Q Smith and Sach Mukherjee and James Cussens , number =. Exact estimation of multiple directed acyclic graphs , volume =. Statistics and Computing , pages =
Show all 47 references
-
[9]
Global optimality in bivariate gradient-based
Chang Deng and Kevin Bello and Pradeep Ravikumar and Bryon Aragam , booktitle =. Global optimality in bivariate gradient-based
-
[10]
Learning subject-specific directed acyclic graphs with mixed effects structural equation models from observational data , volume =
Xiang Li and Shanghong Xie and Peter McColgan and Sarah J Tabrizi and Rachael I Scahill and Donglin Zeng and Yuanjia Wang , journal =. Learning subject-specific directed acyclic graphs with mixed effects structural equation models from observational data , volume =
-
[11]
Learning
Harold Bae and Stefano Monti and Monty Montano and Martin H Steinberg and Thomas T Perls and Paola Sebastiani , journal =. Learning
-
[12]
Flexible copula-based mixed models in deep learning: A scalable approach to arbitrary marginals , volume =
Giora Simchoni and Saharon Rosset , booktitle =. Flexible copula-based mixed models in deep learning: A scalable approach to arbitrary marginals , volume =
-
[13]
Using random effects to account for high-cardinality categorical features and repeated measures in deep neural networks , volume =
Giora Simchoni and Saharon Rosset , booktitle =. Using random effects to account for high-cardinality categorical features and repeated measures in deep neural networks , volume =
-
[14]
Giora Simchoni and Saharon Rosset , title =
-
[15]
Mixed effects neural networks (
Yunyang Xiong and Hyunwoo J Kim and Vikas Singh , booktitle =. Mixed effects neural networks (
-
[16]
Markov equivalence and consistency in differentiable structure learning , volume =
Chang Deng and Kevin Bello and Pradeep Ravikumar and Bryon Aragam , booktitle =. Markov equivalence and consistency in differentiable structure learning , volume =
-
[17]
Analytic
Zhen Zhang and Ignavier Ng and Dong Gong and Yuhang Liu and Mingming Gong and Biwei Huang and Kun Zhang and Anton van den Hengel and Javen Qinfeng Shi , booktitle =. Analytic
-
[18]
Learning sparse nonparametric
Xun Zheng and Chen Dan and Bryon Aragam and Pradeep Ravikumar and Eric P Xing , booktitle =. Learning sparse nonparametric
-
[19]
Optimizing
Chang Deng and Kevin Bello and Bryon Aragam and Pradeep Ravikumar , booktitle =. Optimizing
-
[20]
Truncated matrix power iteration for differentiable
Zhen Zhang and Ignavier Ng and Dong Gong and Yuhang Liu and Ehsan M Abbasnejad and Mingming Gong and Kun Zhang and Javen Qinfeng Shi , booktitle =. Truncated matrix power iteration for differentiable
-
[21]
Causation, Prediction, and Search , year =
Peter Spirtes and Clark Glymour and Richard Scheines , address =. Causation, Prediction, and Search , year =
-
[22]
Causality: Models, Reasoning, and Inference , year =
Judea Pearl , address =. Causality: Models, Reasoning, and Inference , year =
-
[23]
Local computations with probabilities on graphical structures and their application to expert systems , volume =
S L Lauritzen and D J Spiegelhalter , number =. Local computations with probabilities on graphical structures and their application to expert systems , volume =. Journal of the Royal Statistical Society: Series B (Methodological) , pages =
-
[24]
Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference , year =
Judea Pearl , address =. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference , year =
-
[25]
Causal inference and developmental psychology , volume =
E Michael Foster , number =. Causal inference and developmental psychology , volume =. Developmental Psychology , pages =
-
[26]
Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics , volume =
Guido W Imbens , number =. Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics , volume =. Journal of Economic Literature , pages =
-
[27]
Use of directed acyclic graphs (
Peter W G Tennant and Eleanor J Murray and Kellyn F Arnold and Laurie Berrie and Matthew P Fox and Sarah C Gadd and Wendy J Harrison and Claire Keeble and Lynsie R Ranker and Johannes Textor and Georgia D Tomova and Mark S Gilthorpe and George T H Ellison , number =. Use of di...
-
[28]
D'ya like
Matthew J Vowels and Necati Cihan Camgoz and Richard Bowden , number =. D'ya like. ACM Computing Surveys , pages =
-
[29]
A survey of
Neville Kenneth Kitson and Anthony C Constantinou and Zhigao Guo and Yang Liu and Kiattikun Chobtham , number =. A survey of. Artificial Intelligence Review , pages =
-
[30]
Linear and Generalized Linear Mixed Models and Their Applications , edition =
Jiming Jiang and Thuan Nguyen , address =. Linear and Generalized Linear Mixed Models and Their Applications , edition =
-
[31]
Causal structure discovery from distributions arising from mixtures of
Basil Saeed and Snigdha Panigrahi and Caroline Uhler , booktitle =. Causal structure discovery from distributions arising from mixtures of
-
[32]
Causal discovery from heterogeneous/nonstationary data , volume =
Biwei Huang and Kun Zhang and Jiji Zhang and Joseph Ramsey and Ruben Sanchez-Romero and Clark Glymour and Bernhard Schölkopf , journal =. Causal discovery from heterogeneous/nonstationary data , volume =
-
[33]
Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas Köpf and Edward Yang and Zach DeVito and Martin Raison and Alykhan Tejani...
-
[34]
Best subset, forward stepwise or lasso?
Trevor Hastie and Robert Tibshirani and Ryan Tibshirani , number =. Best subset, forward stepwise or lasso?. Statistical Science , pages =
-
[35]
Proximal algorithms in statistics and machine learning , volume =
Nicholas G Polson and James G Scott and Brandon T Willard , number =. Proximal algorithms in statistics and machine learning , volume =. Statistical Science , pages =
-
[36]
Matrix Analysis and Applied Linear Algebra , year =
Carl D Meyer , address =. Matrix Analysis and Applied Linear Algebra , year =
-
[37]
Large sample estimation and hypothesis testing , volume =
Whitney K Newey and Daniel McFadden , booktitle =. Large sample estimation and hypothesis testing , volume =
-
[38]
Sharp thresholds for high-dimensional and noisy sparsity recovery using _1 -constrained quadratic programming (lasso) , volume =
Martin J Wainwright , number =. Sharp thresholds for high-dimensional and noisy sparsity recovery using _1 -constrained quadratic programming (lasso) , volume =. IEEE Transactions on Information Theory , pages =
-
[39]
On model selection consistency of lasso , volume =
Peng Zhao and Bin Yu , journal =. On model selection consistency of lasso , volume =
-
[40]
Identifiability of
J Peters and P Bühlmann , number =. Identifiability of. Biometrika , pages =
-
[41]
Variable selection in linear mixed effects models , volume =
Yingying Fan and Runze Li , number =. Variable selection in linear mixed effects models , volume =. Annals of Statistics , pages =
-
[42]
Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs , volume =
Ali Shojaie and George Michailidis , number =. Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs , volume =. Biometrika , pages =
-
[43]
Ryan Thompson and Edwin V Bonilla and Robert Kohn , booktitle =
-
[44]
Matrix Algebra From a Statistician's Perspective , publisher =
David A Harville , address =. Matrix Algebra From a Statistician's Perspective , publisher =
-
[45]
Publicationes Mathematicae Debrecen , pages =
P Erdős and A Rényi , number =. Publicationes Mathematicae Debrecen , pages =
-
[46]
Science , pages =
Albert-László Barabási and Réka Albert , number =. Science , pages =
-
[47]
On-line student modeling for coached problem solving using Bayesian networks , year =
Cristina Conati and Abigail S Gertner and Kurt Vanlehn and Marek J Druzdzel , booktitle =. On-line student modeling for coached problem solving using Bayesian networks , year =
Reviewed July 10, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.