Pith. sign in

REVIEW 2 major objections 4 minor 47 references

Structure Learning on Clustered Data

T0 review · 2 major / 4 minor · reviewed 2026-07-10 · grok-4.5

Pith's one-line read DAG structure learning can recover a shared global graph while letting edge strengths vary by cluster through fixed and random effects.

desk verdict Solid continuous mixed-DAG method with a clean union-acyclicity constraint and the first structure-recovery consistency result for this setting; modeling assumptions are the main limit, not the math. read the letter →

arxiv 2607.08238 v1 pith:VJKHZAOC submitted 2026-07-09 cs.LG stat.MEstat.ML

classification cs.LGstat.MEstat.ML
keywords causaldiscoverycontinuousacyclicitydirectedacyclicgraphsheterogeneousdatamixedeffectsstructurelearningclustered
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most scalable methods for learning directed acyclic graphs assume every observation comes from the same causal model. Clustered data break that assumption: patients, sites, or batches share a common skeleton of dependencies but differ in the strength of those edges. This paper imports the classical fixed-and-random-effects idea into continuous DAG learning. A single sparse matrix of fixed effects captures the population graph; a second matrix of random-effect variances captures which edges are allowed to vary by cluster. A differentiable coupling constraint forces the union of the two graphs to stay acyclic, so every cluster-specific graph remains a valid DAG. The resulting estimator is identifiable, recovers the true structure as the number of clusters grows, and on synthetic and proteomics data finds both shared and patient-specific edges that homogeneous baselines miss. The practical payoff is that structure learning can now be applied to the clustered measurements that dominate medicine and biology without forcing every cluster into the same edge weights.

What carries the argument

Differentiable graph coupling: the log-determinant acyclicity function applied to B⊙B+Γ, which vanishes if and only if the union of the fixed-effect graph and the random-effect graph is acyclic.

What would settle it

Generate clustered data from a mixed DAG with known unequal node noise variances or correlated random effects across parents; if the estimator still claims unique recovery of the true edge sets, the identifiability claim is false.

Watch

Extended reading notes

Core claim

Under a linear Gaussian mixed structural equation model, the population fixed-effect matrix and the random-effect variance matrix are jointly identifiable from clustered observations whenever their union graph is a DAG, and an ℓ1-penalized continuous estimator recovers both the parameters and the edge sets consistently as the number of clusters tends to infinity.

Load-bearing premise

The model assumes every node has the same unit noise variance and that random effects for different parents of a node are independent; if either fails, uniqueness of the fixed- and random-effect parameters can break.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes mixed DAGs for structure learning on clustered data: each cluster follows a linear Gaussian SEM X^(i)=X^(i)W^(i)+ε^(i) with W^(i)=B+U^(i), where B is a sparse fixed-effects adjacency and the columns of U^(i) are independent N(0,diag(γ_k)) random effects. The estimator minimizes a nodewise negative log-likelihood (Eq. 2) plus ℓ1 penalties on B and Γ subject to the union graph G(B)∪G(Γ) being acyclic. Acyclicity is enforced by the differentiable coupling h(B,Γ)=-log det(I-B⊙B-Γ) (Eq. 4, Prop. 1). A path-following proximal-gradient algorithm (Algs. 1–2) is shown to produce decreasing objective values with vanishing successive differences (Thm. 3). Identifiability of (B0,Γ0) from the clustered distribution is proved under the equal-noise, diagonal-RE model (Thm. 4); parameter and structure recovery consistency as m o∞ follow under standard ℓ1 and irrepresentability conditions (Thm. 5). Synthetic, semi-synthetic (ANDES) and a small proteomics example show gains over fixed-effect and masked baselines.

Significance. Clustered heterogeneity is common in biomedical and longitudinal settings yet largely unaddressed by continuous DAG methods. Extending the fixed/random-effects decomposition to weighted adjacency matrices, together with a lossless union-acyclicity constraint and a first-order algorithm with batched likelihood evaluations, is a natural and practically useful contribution. The paper supplies complete proofs of the continuous constraint, proximal convergence, identifiability and structure recovery (via primal-dual witness), multi-replicate experiments with standard errors, and an asymptotic guarantee that is new for mixed DAGs. These strengths make the work a solid addition to scalable causal discovery under known cluster structure, provided the modeling assumptions are clearly scoped.

major comments (2)
  1. The likelihood (Eq. 2), the KL uniqueness argument of Theorem 4 / Appendix D, and the Hessian/irrepresentability conditions of Assumption 2 all rest on equal unit noise variances (σ_k^{2}=1) and independent diagonal random-effect covariances. The manuscript notes that node-specific variances can be accommodated by a simple rescaling, but never states the corresponding identifiability result or the modified irrepresentability conditions. Because these assumptions are load-bearing for the central statistical claims, the paper should either (i) prove that Theorems 4–5 continue to hold under free σ_k^{2} (and, ideally, under a modest relaxation of the diagonal-RE restriction) or (ii) explicitly restrict the claims to the equal-noise, diagonal-RE model class and discuss the practical consequences of misspecification.
  2. Section 6 and Appendix F rely on a post-hoc hard threshold of 0.05 and a polishing re-optimization that removes both the ℓ1 penalties and the acyclicity constraint. Theorem 5 concerns the exact minimizer of the constrained program (3)/(8). The manuscript never shows that the thresholded/polished estimator inherits the same asymptotic guarantees, nor does it report sensitivity of SHD/F1 to the threshold value. A short argument or numerical check that the reported recovery rates remain valid after these post-processing steps is needed for the experimental claims to support the theory.
minor comments (4)
  1. Figure 2 reports wall-clock times only for the full path-following procedure; a brief breakdown of the cost of the batched Cholesky/Woodbury steps versus the log-det acyclicity term would help readers assess scalability claims for large m or unbalanced ni.
  2. The real-data example (p=6, Figure 5) is useful for visualization but too small to illustrate the method’s claimed ability to handle hundreds of nodes; a short remark on this limitation would set expectations.
  3. Notation for the spectral-radius domain D (Eq. 5) and the feasible set F (Eq. 8) is introduced late; a single early display of the constrained program would improve readability.
  4. Code and data-generation scripts are not mentioned; releasing them would strengthen reproducibility of the 30-replicate experiments.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: identifiability and consistency are derived from the external SEM DGP under stated assumptions, not by construction from fitted quantities or self-citation chains.

full rationale

The paper's central claims (Proposition 1 on the union-acyclicity characterization, Theorems 4–5 on identifiability and asymptotic structure recovery) are proved from first principles relative to an external linear-Gaussian mixed SEM data-generating process with equal unit noise variances and diagonal random-effect covariances. The continuous constraint h(B,Γ)=−logdet(I−B⊙B−Γ) is an explicit, lossless extension of the external log-det characterization of Bello et al. (2022); its equivalence proof (Appendix A) does not presuppose the estimator. The negative log-likelihood (2), proximal path-following algorithm, and convergence guarantee (Theorem 3) are standard first-order constructions whose fixed points are not defined in terms of the recovered graphs. Consistency (Theorem 5) is classical penalized-likelihood asymptotics (uniform LLN, local strong convexity, primal-dual witness under irrepresentability) relative to the true (B0,Γ0) of the DGP, not a quantity fitted from the same data and then re-labeled a prediction. Self-citations (Thompson et al. on contextual/variational DAGs and high-dimensional mixed models) appear only in related-work and background sections and are not invoked as uniqueness theorems that force the present results. Synthetic experiments generate data from the same SEM and therefore test recovery under the model rather than circularly re-predicting fitted constants. The equal-noise and diagonal-RE assumptions are modeling restrictions (correctly flagged by the reader) but do not create definitional circularity inside the derivation chain. Score 1 reflects only the ordinary presence of author self-citations that are non-load-bearing.

Assumptions & free parameters 4 free parameters · 5 assumptions · 2 invented entities

The central claims rest on a linear Gaussian mixed SEM with equal noise variances and diagonal random-effect covariances, continuous log-det acyclicity applied to B⊙B+Γ, ℓ1 penalties with a specific asymptotic regime, and an irrepresentability condition on the population Hessian. Free parameters are the two sparsity weights, the continuation schedule, and the post-hoc hard threshold. The mixed-DAG decomposition and the coupling function are the main methodological constructs introduced here.

free parameters (4)
  • λ1, λ2 (sparsity penalties)
    Tuned on a validation set over a 30-point log grid in [10^{-3},1]; control which fixed and random edges are selected and enter the consistency regime of Assumption 1.
  • Hard-threshold 0.05 on |βjk| and γjk
    Applied after optimization to force exact zeros (Appendix F); not derived from the continuous constraint and affects reported SHD/F1.
  • Continuation path μ^(t)
    μ^(1)=1, μ^(t+1)=0.1 μ^(t) for T=5; controls how tightly h(B,Γ)=0 is enforced and thus the final acyclicity of the reported graphs.
  • Box bounds MB, MΓ on ||B||∞, ||Γ||∞
    Technical compactness constants in the feasible set F (Eq. 8); assumed larger than true parameters for the consistency proof.
assumptions (5)
  • domain assumption Linear Gaussian SEM with equal unit noise variances σ²_k=1 and independent diagonal random effects u_k^(i)~N(0,diag(γ_k)).
    Defines the likelihood (2) and is used for identifiability in Theorem 4 / Appendix D (variance=1 characterizes sources).
  • standard math h(W)=-log det(I-W)=0 iff G(W) is a DAG on the spectral-radius domain (Bello et al. 2022).
    Lifted to the mixed setting as h(B,Γ)=-log det(I-B⊙B-Γ) in Proposition 1.
  • domain assumption Population Hessian H=∇²L(B0,Γ0) positive definite and irrepresentability inequalities with gap η∈(0,1) (Assumption 2).
    Required for structure-recovery consistency in Theorem 5; analogous to Zhao–Yu / Wainwright conditions.
  • domain assumption Clusters X^(1),…,X^(m) iid with bounded common size n; λ1,λ2→0 with √m λj→∞ and λ2/λ1→κ∈(0,∞) (Assumption 1).
    Asymptotic regime for Theorem 5; equal sizes stated for simplicity, claimed to extend to bounded unequal sizes.
  • ad hoc to paper Union graph G(B)∪G(Γ) is the correct acyclicity object for cluster-specific W(i)=B+U(i).
    Modeling choice that defines the constraint set; if random effects could introduce cycles not visible in the union of supports, the constraint would be misspecified.
invented entities (2)
  • Mixed DAG (W(i)=B+U(i) with sparse fixed B and sparse variance Γ)
    purpose: Represent population-level edges plus cluster-specific deviations under one global acyclic skeleton.
    Core model object of the paper; not previously optimized continuously for structure learning.
  • Differentiable graph coupling h(B,Γ)=-log det(I-B⊙B-Γ)
    purpose: Enforce acyclicity of the union of fixed- and random-effect graphs with a single smooth constraint.
    Direct extension of Bello’s log-det to the mixed support; equivalence proved in Proposition 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structure Learning on Clustered Data." pith.science (2026). https://pith.science/paper/VJKHZAOC

@misc{pith2026260708238,
  author       = {Pith},
  title        = {Pith review of: Structure Learning on Clustered Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VJKHZAOC}},
  note         = {Machine review of arXiv:2607.08238}
}
read the original abstract

Recent algorithmic advances have made directed acyclic graph (DAG) structure learning scalable for causal discovery. Yet, the currently available techniques assume a completely homogeneous population, precluding their application to clustered data where cluster-specific variations (e.g., patient-specific effects) are common. We address this issue by introducing a new approach that estimates a global structure while accounting for local cluster-level effects. The key idea is to extend the fixed- and random-effects framework of classical mixed models to the structure learning setting. Towards this end, we present a differentiable graph coupling mechanism that guarantees the union of the fixed- and random-effects graphs remains acyclic. Computationally, we provide a provably convergent first-order method and leverage efficient batched updates across clusters. Statistically, we establish identifiability of the model and show that our approach recovers the true structure asymptotically. In experiments on real and synthetic data, our proposal detects dependencies missed by alternative estimators, underscoring its value for structure learning in clustered settings.

Figures

Figures reproduced from arXiv: 2607.08238 by the authors.

Figure 1
Figure 1. Schematic of mixed DAGs. The fixed-effects graph [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Run time in seconds as a function of the number of nodes [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Graphs learned on the VectraPolarisData. All edges in the graph produced by the mixed DAG contain both fixed and random effects. 7 Final Remarks Many fields, including medicine and biology, give rise to clustered data with heterogeneous effects, making them unsuitable …
Figure 6
Figure 6. Figure 6: Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p037_6.png]
Figure 7
Figure 7. Figure 7: Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p038_7.png]
Figure 8
Figure 8. Figure 8: Performance on synthetic data generated from scale-free (degree exponent 2) DAGs [PITH_FULL_IMAGE:figures/full_fig_p039_8.png]
Figure 9
Figure 9. Figure 9: Performance on synthetic data generated from scale-free (degree exponent 3) DAGs [PITH_FULL_IMAGE:figures/full_fig_p040_9.png]
Figure 10
Figure 10. Figure 10: Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p041_10.png]
Figure 11
Figure 11. Figure 11: Performance on synthetic data generated from Erdős–Rényi DAGs with [PITH_FULL_IMAGE:figures/full_fig_p042_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

47 extracted references · 47 canonical work pages

  1. [1]

    Kevin Bello and Bryon Aragam and Pradeep Ravikumar , booktitle =

  2. [2]

    Xun Zheng and Bryon Aragam and Pradeep Ravikumar and Eric P Xing , booktitle =

  3. [3]

    Contextual directed acyclic graphs , volume =

    Ryan Thompson and Edwin V Bonilla and Robert Kohn , booktitle =. Contextual directed acyclic graphs , volume =

  4. [4]

    Bayesian graphical regression , volume =

    Yang Ni and Francesco C Stingo and Veerabhadran Baladandayuthapani , number =. Bayesian graphical regression , volume =. Journal of the American Statistical Association , pages =

  5. [5]

    Integrating random effects in deep neural networks , volume =

    Giora Simchoni and Saharon Rosset , journal =. Integrating random effects in deep neural networks , volume =

  6. [6]

    A relaxation approach to feature selection for linear mixed effects models , volume =

    Aleksei Sholokhov and James V Burke and Damian F Santomauro and Peng Zheng and Aleksandr Aravkin , number =. A relaxation approach to feature selection for linear mixed effects models , volume =. Journal of Computational and Graphical Statistics , pages =

  7. [7]

    Ryan Thompson and Matt P Wand and Joanna J J Wang , title =

  8. [8]

    Exact estimation of multiple directed acyclic graphs , volume =

    Chris J Oates and Jim Q Smith and Sach Mukherjee and James Cussens , number =. Exact estimation of multiple directed acyclic graphs , volume =. Statistics and Computing , pages =

Show all 47 references
  1. [9]

    Global optimality in bivariate gradient-based

    Chang Deng and Kevin Bello and Pradeep Ravikumar and Bryon Aragam , booktitle =. Global optimality in bivariate gradient-based

  2. [10]

    Learning subject-specific directed acyclic graphs with mixed effects structural equation models from observational data , volume =

    Xiang Li and Shanghong Xie and Peter McColgan and Sarah J Tabrizi and Rachael I Scahill and Donglin Zeng and Yuanjia Wang , journal =. Learning subject-specific directed acyclic graphs with mixed effects structural equation models from observational data , volume =

  3. [11]

    Learning

    Harold Bae and Stefano Monti and Monty Montano and Martin H Steinberg and Thomas T Perls and Paola Sebastiani , journal =. Learning

  4. [12]

    Flexible copula-based mixed models in deep learning: A scalable approach to arbitrary marginals , volume =

    Giora Simchoni and Saharon Rosset , booktitle =. Flexible copula-based mixed models in deep learning: A scalable approach to arbitrary marginals , volume =

  5. [13]

    Using random effects to account for high-cardinality categorical features and repeated measures in deep neural networks , volume =

    Giora Simchoni and Saharon Rosset , booktitle =. Using random effects to account for high-cardinality categorical features and repeated measures in deep neural networks , volume =

  6. [14]

    Giora Simchoni and Saharon Rosset , title =

  7. [15]

    Mixed effects neural networks (

    Yunyang Xiong and Hyunwoo J Kim and Vikas Singh , booktitle =. Mixed effects neural networks (

  8. [16]

    Markov equivalence and consistency in differentiable structure learning , volume =

    Chang Deng and Kevin Bello and Pradeep Ravikumar and Bryon Aragam , booktitle =. Markov equivalence and consistency in differentiable structure learning , volume =

  9. [17]

    Analytic

    Zhen Zhang and Ignavier Ng and Dong Gong and Yuhang Liu and Mingming Gong and Biwei Huang and Kun Zhang and Anton van den Hengel and Javen Qinfeng Shi , booktitle =. Analytic

  10. [18]

    Learning sparse nonparametric

    Xun Zheng and Chen Dan and Bryon Aragam and Pradeep Ravikumar and Eric P Xing , booktitle =. Learning sparse nonparametric

  11. [19]

    Optimizing

    Chang Deng and Kevin Bello and Bryon Aragam and Pradeep Ravikumar , booktitle =. Optimizing

  12. [20]

    Truncated matrix power iteration for differentiable

    Zhen Zhang and Ignavier Ng and Dong Gong and Yuhang Liu and Ehsan M Abbasnejad and Mingming Gong and Kun Zhang and Javen Qinfeng Shi , booktitle =. Truncated matrix power iteration for differentiable

  13. [21]

    Causation, Prediction, and Search , year =

    Peter Spirtes and Clark Glymour and Richard Scheines , address =. Causation, Prediction, and Search , year =

  14. [22]

    Causality: Models, Reasoning, and Inference , year =

    Judea Pearl , address =. Causality: Models, Reasoning, and Inference , year =

  15. [23]

    Local computations with probabilities on graphical structures and their application to expert systems , volume =

    S L Lauritzen and D J Spiegelhalter , number =. Local computations with probabilities on graphical structures and their application to expert systems , volume =. Journal of the Royal Statistical Society: Series B (Methodological) , pages =

  16. [24]

    Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference , year =

    Judea Pearl , address =. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference , year =

  17. [25]

    Causal inference and developmental psychology , volume =

    E Michael Foster , number =. Causal inference and developmental psychology , volume =. Developmental Psychology , pages =

  18. [26]

    Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics , volume =

    Guido W Imbens , number =. Potential outcome and directed acyclic graph approaches to causality: Relevance for empirical practice in economics , volume =. Journal of Economic Literature , pages =

  19. [27]

    Use of directed acyclic graphs (

    Peter W G Tennant and Eleanor J Murray and Kellyn F Arnold and Laurie Berrie and Matthew P Fox and Sarah C Gadd and Wendy J Harrison and Claire Keeble and Lynsie R Ranker and Johannes Textor and Georgia D Tomova and Mark S Gilthorpe and George T H Ellison , number =. Use of di...

  20. [28]

    D'ya like

    Matthew J Vowels and Necati Cihan Camgoz and Richard Bowden , number =. D'ya like. ACM Computing Surveys , pages =

  21. [29]

    A survey of

    Neville Kenneth Kitson and Anthony C Constantinou and Zhigao Guo and Yang Liu and Kiattikun Chobtham , number =. A survey of. Artificial Intelligence Review , pages =

  22. [30]

    Linear and Generalized Linear Mixed Models and Their Applications , edition =

    Jiming Jiang and Thuan Nguyen , address =. Linear and Generalized Linear Mixed Models and Their Applications , edition =

  23. [31]

    Causal structure discovery from distributions arising from mixtures of

    Basil Saeed and Snigdha Panigrahi and Caroline Uhler , booktitle =. Causal structure discovery from distributions arising from mixtures of

  24. [32]

    Causal discovery from heterogeneous/nonstationary data , volume =

    Biwei Huang and Kun Zhang and Jiji Zhang and Joseph Ramsey and Ruben Sanchez-Romero and Clark Glymour and Bernhard Schölkopf , journal =. Causal discovery from heterogeneous/nonstationary data , volume =

  25. [33]

    Adam Paszke and Sam Gross and Francisco Massa and Adam Lerer and James Bradbury and Gregory Chanan and Trevor Killeen and Zeming Lin and Natalia Gimelshein and Luca Antiga and Alban Desmaison and Andreas Köpf and Edward Yang and Zach DeVito and Martin Raison and Alykhan Tejani...

  26. [34]

    Best subset, forward stepwise or lasso?

    Trevor Hastie and Robert Tibshirani and Ryan Tibshirani , number =. Best subset, forward stepwise or lasso?. Statistical Science , pages =

  27. [35]

    Proximal algorithms in statistics and machine learning , volume =

    Nicholas G Polson and James G Scott and Brandon T Willard , number =. Proximal algorithms in statistics and machine learning , volume =. Statistical Science , pages =

  28. [36]

    Matrix Analysis and Applied Linear Algebra , year =

    Carl D Meyer , address =. Matrix Analysis and Applied Linear Algebra , year =

  29. [37]

    Large sample estimation and hypothesis testing , volume =

    Whitney K Newey and Daniel McFadden , booktitle =. Large sample estimation and hypothesis testing , volume =

  30. [38]

    Sharp thresholds for high-dimensional and noisy sparsity recovery using _1 -constrained quadratic programming (lasso) , volume =

    Martin J Wainwright , number =. Sharp thresholds for high-dimensional and noisy sparsity recovery using _1 -constrained quadratic programming (lasso) , volume =. IEEE Transactions on Information Theory , pages =

  31. [39]

    On model selection consistency of lasso , volume =

    Peng Zhao and Bin Yu , journal =. On model selection consistency of lasso , volume =

  32. [40]

    Identifiability of

    J Peters and P Bühlmann , number =. Identifiability of. Biometrika , pages =

  33. [41]

    Variable selection in linear mixed effects models , volume =

    Yingying Fan and Runze Li , number =. Variable selection in linear mixed effects models , volume =. Annals of Statistics , pages =

  34. [42]

    Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs , volume =

    Ali Shojaie and George Michailidis , number =. Penalized likelihood methods for estimation of sparse high-dimensional directed acyclic graphs , volume =. Biometrika , pages =

  35. [43]

    Ryan Thompson and Edwin V Bonilla and Robert Kohn , booktitle =

  36. [44]

    Matrix Algebra From a Statistician's Perspective , publisher =

    David A Harville , address =. Matrix Algebra From a Statistician's Perspective , publisher =

  37. [45]

    Publicationes Mathematicae Debrecen , pages =

    P Erdős and A Rényi , number =. Publicationes Mathematicae Debrecen , pages =

  38. [46]

    Science , pages =

    Albert-László Barabási and Réka Albert , number =. Science , pages =

  39. [47]

    On-line student modeling for coached problem solving using Bayesian networks , year =

    Cristina Conati and Abigail S Gertner and Kurt Vanlehn and Marek J Druzdzel , booktitle =. On-line student modeling for coached problem solving using Bayesian networks , year =

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.