REVIEW 1 major objections 5 minor 114 references
Deep Gaussian processes on DAGs keep input distinctions alive with depth and recover explaining-away under partial observations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-13 01:29 UTC pith:44UZIWGX
load-bearing objection Solid first generalisation of DGPs to DAGs with real non-collapse theory and a usable structured VI scheme; known-graph assumption is the main scope limit, not a crack in the claims. the 1 major comments →
Deep Gaussian Processes on Directed Acyclic Graphs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A Deep Gaussian Process defined directly on a DAG, with node-wise fusion kernels and observations at arbitrary nodes, does not suffer prior collapse whenever sufficiently many v-star-separating nodes appear across progressive antichains; the same architecture admits a structured variational approximation that retains posterior dependence among co-parents and yields state-of-the-art multi-fidelity emulation while recovering low-fidelity contributions.
What carries the argument
v-star-separating nodes (fresh GP modules whose two-case contrast covariance is bounded below almost surely) together with progressive antichain decompositions; they convert local non-degeneracy into almost-sure lower bounds on the asymptotic frequency of non-collapsed depths (Theorem 1) and turn intermediate observations into stochastic skip connections (Theorem 2).
Load-bearing premise
The directed acyclic graph itself is known and correctly specified; every non-collapse guarantee and the moralised precision construction stand or fall with that fixed topology.
What would settle it
On a chain or layered DAG whose kernels satisfy the paper’s conditions, draw many independent prior realisations of two distinct inputs and check whether the empirical frequency of depths with maximum contrast above a fixed epsilon falls below the claimed almost-sure lower bound 1-(1-p_epsilon)^s.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Deep Gaussian Processes on Directed Acyclic Graphs (DAG-DGPs): independent GP priors on nodewise maps with fusion kernels for multi-parent nodes, and heterogeneous noisy observations at arbitrary nodes. It proves almost-sure lower bounds on the asymptotic frequency of depths at which two-case input distinctions are preserved under progressive antichain decompositions when antichains contain sufficiently many v*-separating nodes (Theorem 1), characterises indegree/outdegree effects under radial fusion, and shows that intermediate observations act as stochastic skip connections (Theorem 2). It also proves non-collapse for the input-connected chain of Dunlop et al. (2018). A structured variational family (DAG-SVI) is built from the moralised ancestral graph and chordal completion so that explaining-away and compositional uncertainty are retained; mean-field DAG-VI is recovered as a special case. Theory is checked by Monte Carlo, and the method is demonstrated on a latent collider, the Sachs protein network, and multi-fidelity heavy-ion emulation, where it improves on published baselines and recovers low-fidelity contributions via Shapley values.
Significance. If the results hold, the paper supplies a single probabilistic framework that unifies chain DGPs, multi-fidelity DGPs, and graphical multi-fidelity emulators, with the first non-collapse theory for DAG compositions and a proof of the Dunlop et al. input-connection observation. The structured variational construction is a genuine extension of existing chain DGP approximations and is shown to capture explaining-away, which mean-field cannot. Empirical gains on the heavy-ion task (Table 2) and recovery of low-fidelity contributions are practically useful for scientific multi-fidelity modelling. Strengths include detailed appendix proofs (progressive antichains, conditional non-degeneracy, Azuma–Hoeffding frequency bounds, route retention), explicit kernel classes that certify separation, and multi-seed/fold reporting with competitive baselines.
major comments (1)
- No load-bearing technical error was found in the central theorems or the variational construction under the paper’s stated premises (known DAG, stated kernel classes, progressive antichains). The known-DAG assumption is a genuine scope limitation for causal and structure-discovery settings, but it is already flagged in Sec. 8.2 and does not invalidate the non-collapse bounds or the reported empirical gains under that assumption.
minor comments (5)
- Sec. 8.2 and the Sachs experiment: the 2-D latent confounder layer is ad hoc; a short sensitivity check (e.g., dimension 1 vs 2, or no confounder) would strengthen the claim that DAG-SVI’s extrapolation gains are not driven by that modelling choice alone.
- Fig. 2 and App. E.5: the branching-tree scaling is clear, but a one-sentence statement of when the sparse backend’s RH term remains favourable for denser moral graphs would help practitioners choose between dense and sparse DAG-SVI.
- Notation: the two filtrations F_ℓ (module-generated) and H_ℓ (state-generated) are carefully defined in the appendix; a brief cross-reference in Sec. 4 would reduce the chance of confusion for readers who stay in the main text.
- Table 1 / App. G.4: reporting PICP and NLPD for the Sachs interpolation task (as done for extrapolation in Table 3) would make the interpolation–extrapolation comparison more complete.
- Typographical: arXiv line breaks occasionally split math (e.g., product fusion displays); ensure final camera-ready equations are unbroken.
Circularity Check
No significant circularity: non-collapse bounds and variational constructions are derived from the prior and graph structure without reducing to fitted targets or self-justifying definitions.
full rationale
The central theoretical claims (Theorem 1 on almost-sure lower bounds for contrast frequency under repeated v*-separating nodes; the recovery of Dunlop et al. input-connection non-collapse as Corollary 2; intermediate-observation refresh as Theorem 2) are obtained from the DAG-DGP prior recursion, progressive antichain decompositions, conditional Gaussian non-degeneracy of separating nodes, and standard martingale/Azuma–Hoeffding/Borel–Cantelli arguments. Separation is certified by explicit kernel classes (additive/ANOVA root retention, multi-fidelity discrepancy kernels) whose contrast covariances are computed directly from the kernel definition, not fitted to the target frequency. The structured variational family (DAG-SVI) is constructed from the moralised ancestral graph and chordal completion, recovering known chain and mean-field special cases by restriction; the ELBO and ancestral sampling follow from the usual variational lower bound without circular dependence on the predictive metrics. Empirical results (collider explaining-away visualisation, Sachs interpolation/extrapolation, heavy-ion RMSE/CRPS/Shapley) evaluate held-out predictive performance and post-hoc attributions under a known DAG; they do not feed back into the statements of the theorems. Self-citations are limited to standard DGP/multi-fidelity baselines and do not supply load-bearing uniqueness or ansatzes for the main derivations. The known-DAG modelling assumption is an explicit scope limitation, not an internal circular reduction.
Axiom & Free-Parameter Ledger
free parameters (4)
- kernel length-scales and variances (per node)
- number and locations of inducing points M_w
- observation-noise variances σ_u²
- variational mean m and precision Λ of q_H(U)
axioms (5)
- domain assumption Nodewise GP modules are mutually independent a priori and the joint prior factorises according to the DAG recursion (Eq. 2).
- domain assumption The DAG G=(V,E) is known, acyclic and correctly specified.
- ad hoc to paper A node is v*-separating if its two-point contrast covariance is bounded below by v* I almost surely (Definition in App. A.3).
- standard math Chordal completion of the moralised ancestral graph admits a clique-separator Gaussian representation (Lauritzen 1996).
- standard math Progressive antichain decompositions exist for every finite DAG (Prop. 1).
invented entities (3)
-
DAG-DGP prior and fusion kernels Φ_w
independent evidence
-
v*-separating node
no independent evidence
-
DAG-SVI structured variational family q_H
independent evidence
read the original abstract
Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour, and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \cite{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theoretical results and our methodology, and model a latent-collider DAG, a protein signalling network, and a multi-fidelity heavy-ion collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability of the simulator hierarchy.
Figures
Reference graph
Works this paper leans on
-
[1]
Bayesian Optimization using Partially Observable
Kiroriwal, Saksham and Pfrommer, Julius and Beyerer, J. Bayesian Optimization using Partially Observable. NeurIPS 2025 Workshop MLxOR: Mathematical Foundations and Operational Integration of Machine Learning for Uncertainty-Aware Decision-Making , year=
2025
-
[2]
Learning Bayesian Networks with the
Marco Scutari , journal =. Learning Bayesian Networks with the. 2010 , volume =
2010
-
[3]
doi:10.48550/arXiv.1805.03108 , url =
Ramsey, Joseph and Andrews, Bryan , year =. doi:10.48550/arXiv.1805.03108 , url =. 1805.03108 , archivePrefix =
-
[4]
and Magliacane, Sara and Claassen, Tom , title =
Mooij, Joris M. and Magliacane, Sara and Claassen, Tom , title =. Journal of Machine Learning Research , year =
-
[5]
Rasmussen, Carl Edward and Williams, Christopher K. I. , year =
-
[6]
Foundations and Trends in Machine Learning , year =
-
[7]
Advances in Neural Information Processing Systems , volume =
Salimbeni, Hugh and Deisenroth, Marc , title =. Advances in Neural Information Processing Systems , volume =. 2017 , publisher =
2017
-
[8]
Digraphs: Theory, Algorithms and Applications , edition =
Bang-Jensen, J. Digraphs: Theory, Algorithms and Applications , edition =. 2009 , publisher =
2009
-
[9]
1988 , publisher =
Pearl, Judea , title =. 1988 , publisher =
1988
-
[10]
Variational Learning of Inducing Variables in Sparse
Titsias, Michalis , booktitle =. Variational Learning of Inducing Variables in Sparse. 2009 , editor =
2009
-
[11]
Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence , pages =
Hensman, James and Fusi, Nicol. Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence , pages =. 2013 , publisher =
2013
-
[12]
Williams, Christopher K. I. and Rasmussen, Carl Edward , title =. Advances in Neural Information Processing Systems , volume =. 1995 , publisher =
1995
-
[13]
Advances in Neural Information Processing Systems , volume =
Snelson, Edward and Ghahramani, Zoubin , title =. Advances in Neural Information Processing Systems , volume =. 2005 , publisher =
2005
-
[14]
2006 , doi =
Uncertain Judgements: Eliciting Experts' Probabilities , author =. 2006 , doi =
2006
-
[15]
Qualitative Combination of
Del Sagrado, Jos. Qualitative Combination of. International Journal of Intelligent Systems , year =
-
[16]
A Discrete-Time
Boudali, Hichem and Bechta Dugan, Joanne , journal =. A Discrete-Time. 2005 , volume =
2005
-
[17]
, journal =
Heckerman, David and Geiger, Dan and Chickering, David M. , journal =. Learning. 1995 , volume =
1995
-
[18]
and Girolami, Mark A
Dunlop, Matthew M. and Girolami, Mark A. and Stuart, Andrew M. and Teckentrup, Aretha L. , journal =. How Deep Are Deep. 2018 , url =
2018
-
[19]
Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics , pages =
Avoiding Pathologies in Very Deep Networks , author =. Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics , pages =. 2014 , editor =
2014
-
[20]
, series =
Neal, Radford M. , series =. 1996 , doi =
1996
-
[21]
and Michiardi, Pietro and Filippone, Maurizio , booktitle =
Cutajar, Kurt and Bonilla, Edwin V. and Michiardi, Pietro and Filippone, Maurizio , booktitle =. Random Feature Expansions for Deep. 2017 , editor =
2017
-
[22]
Cutajar, Kurt and Pullin, Mark and Damianou, Andreas and Lawrence, Neil D. and Gonz. Deep. 2019 , journal =. doi:10.48550/arXiv.1903.07320 , url =. 1903.07320 , archivePrefix =
-
[23]
Variational Auto-encoded Deep Gaussian Processes
Dai, Zhenwen and Damianou, Andreas and Gonz. Variational Auto-encoded Deep. 2015 , journal =. doi:10.48550/arXiv.1511.06455 , url =. 1511.06455 , archivePrefix =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1511.06455 2015
-
[24]
Journal of Machine Learning Research , year =
Knoblauch, Jeremias and Jewson, Jack and Damoulas, Theodoros , title =. Journal of Machine Learning Research , year =
-
[25]
Robust Deep Gaussian Processes
Knoblauch, Jeremias , title =. arXiv preprint arXiv:1904.02303 , year =. doi:10.48550/arXiv.1904.02303 , url =. 1904.02303 , archivePrefix =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1904.02303 1904
-
[26]
and Walker, Stephen G
Bissiri, Pier Giovanni and Holmes, Chris C. and Walker, Stephen G. , title =. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , year =
-
[27]
Goldstein, Harvey , title =. 2011 , publisher =. doi:10.1002/9780470973394 , isbn =
-
[28]
Bollen, Kenneth A. , title =. 1989 , publisher =. doi:10.1002/9781118619179 , isbn =
-
[29]
, title =
Lauritzen, Steffen L. , title =. 1996 , publisher =
1996
-
[30]
and Ruppert, David and Stefanski, Leonard A
Carroll, Raymond J. and Ruppert, David and Stefanski, Leonard A. and Crainiceanu, Ciprian M. , title =. 2006 , edition =. doi:10.1201/9781420010138 , isbn =
-
[31]
Advances in Neural Information Processing Systems , volume =
Lindinger, Jakob and Reeb, David and Lippert, Christoph and Rakitsch, Barbara , title =. Advances in Neural Information Processing Systems , volume =. 2020 , publisher =
2020
-
[32]
and Dawid, A
Cowell, Robert G. and Dawid, A. Philip and Lauritzen, Steffen L. and Spiegelhalter, David J. , title =. 1999 , publisher =
1999
-
[33]
Introduction to Statistical Relational Learning , editor =
Koller, Daphne and Friedman, Nir and Getoor, Lise and Taskar, Ben , title =. Introduction to Statistical Relational Learning , editor =. 2007 , chapter =. doi:10.7551/mitpress/7432.003.0004 , isbn =
-
[34]
Blair, Jean R. S. and Peyton, Barry W. , title =. Graph Theory and Sparse Matrix Computation , editor =. 1993 , publisher =
1993
-
[35]
Rue, H. 2005 , publisher =. doi:10.1201/9780203492024 , isbn =
-
[36]
and Tarjan, Robert E
Rose, Donald J. and Tarjan, Robert E. and Lueker, George S. , title =. SIAM Journal on Computing , volume =. 1976 , doi =
1976
-
[37]
Compositional Uncertainty in Deep
Ustyuzhaninov, Ivan and Kazlauskaite, Ieva and Kaiser, Markus and Bodin, Erik and Campbell, Neill and Ek, Carl Henrik , booktitle =. Compositional Uncertainty in Deep. 2020 , editor =
2020
-
[38]
, title =
Bishop, Christopher M. , title =. 2006 , publisher =
2006
-
[39]
, title =
Mirsky, L. , title =. The American Mathematical Monthly , volume =. 1971 , doi =
1971
-
[40]
A Graphical Multi-Fidelity
Ji, Yi and Mak, Simon and Soeder, Derek and Paquet, Jean-Fran. A Graphical Multi-Fidelity. Technometrics , volume =. 2024 , doi =
2024
-
[41]
and Nickisch, Hannes and Rasmussen, Carl Edward , title =
Duvenaud, David K. and Nickisch, Hannes and Rasmussen, Carl Edward , title =. Advances in Neural Information Processing Systems , volume =. 2011 , publisher =
2011
-
[42]
Journal of Multivariate Analysis , volume =
Durrande, Nicolas and Ginsbourger, David and Roustant, Olivier and Carraro, Luca , title =. Journal of Multivariate Analysis , volume =. 2013 , doi =
2013
-
[43]
Tohoku Mathematical Journal , series =
Azuma, Kazuoki , title =. Tohoku Mathematical Journal , series =. 1967 , doi =
1967
-
[44]
Journal of the American Statistical Association , volume =
Hoeffding, Wassily , title =. Journal of the American Statistical Association , volume =. 1963 , doi =
1963
-
[45]
IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =
Liu, Yuhao and Ajirak, Marzieh and Djuri. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. 2024 , doi =
2024
-
[46]
and Jacobs, Robert A
Jordan, Michael I. and Jacobs, Robert A. , title =. Neural Computation , volume =. 1994 , doi =
1994
-
[47]
and Aitchison, Laurence , booktitle =
Ober, Sebastian W. and Aitchison, Laurence , booktitle =. Global Inducing Point Variational Posteriors for. 2021 , editor =
2021
-
[48]
Athreya, Krishna B. and Ney, Peter E. , title =. 1972 , publisher =. doi:10.1007/978-3-642-65371-1 , isbn =
-
[49]
Schoenberg, I. J. , title =. Annals of Mathematics , series =. 1938 , doi =
1938
-
[50]
Schoenberg, I. J. , title =. Transactions of the American Mathematical Society , volume =. 1938 , doi =
1938
-
[51]
1941 , publisher =
Widder, David Vernon , title =. 1941 , publisher =
1941
-
[52]
and O'Hagan, Anthony , title =
Kennedy, Marc C. and O'Hagan, Anthony , title =. Biometrika , volume =. 2000 , doi =
2000
-
[53]
and Karniadakis, George Em , title =
Perdikaris, Paris and Raissi, Maziar and Damianou, Andreas and Lawrence, Neil D. and Karniadakis, George Em , title =. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences , volume =. 2017 , doi =
2017
-
[54]
Sachs, Karen and Perez, Omar and Pe'er, Dana and Lauffenburger, Douglas A. and Nolan, Garry P. , title =. Science , year =. doi:10.1126/science.1105809 , pmid =
-
[55]
2009 , publisher =
Pearl, Judea , title =. 2009 , publisher =
2009
-
[56]
Review of Multi-Fidelity Models , journal =
Fern. Review of Multi-Fidelity Models , journal =. 2023 , volume =
2023
-
[57]
Friedman, Nir , title =. Science , year =. doi:10.1126/science.1094068 , pmid =
-
[58]
, title =
Greenland, Sander and Pearl, Judea and Robins, James M. , title =. Epidemiology , year =
-
[59]
Little, Roderick J. A. and Rubin, Donald B. , title =. 2019 , publisher =. doi:10.1002/9781119482260 , isbn =
-
[60]
and O'Hagan, Anthony , title =
Kennedy, Marc C. and O'Hagan, Anthony , title =. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , year =
-
[61]
and Matias, Catherine and Rhodes, John A
Allman, Elizabeth S. and Matias, Catherine and Rhodes, John A. , title =. The Annals of Statistics , year =
-
[62]
Structural and Practical Identifiability Analysis of Partially Observed Dynamical Models by Exploiting the Profile Likelihood , journal =
Raue, Andreas and Kreutz, Clemens and Maiwald, Thomas and Bachmann, Julie and Schilling, Marcel and Klingm. Structural and Practical Identifiability Analysis of Partially Observed Dynamical Models by Exploiting the Profile Likelihood , journal =. 2009 , volume =
2009
-
[63]
, title =
Neal, Radford M. , title =. 1995 , address =
1995
-
[64]
Philip , title =
Dawid, A. Philip , title =. Journal of the Royal Statistical Society: Series B (Methodological) , year =
-
[65]
Wainwright, Martin J. and Jordan, Michael I. , title =. Foundations and Trends in Machine Learning , year =. doi:10.1561/2200000001 , publisher =
-
[66]
, booktitle =
Damianou, Andreas and Lawrence, Neil D. , booktitle =. Deep. 2013 , editor =
2013
-
[67]
Lawrence, Neil D. and Moore, Andrew J. , booktitle =. Hierarchical. 2007 , publisher =. doi:10.1145/1273496.1273557 , url =
-
[68]
, booktitle =
Lawrence, Neil D. , booktitle =. 2003 , publisher =
2003
-
[69]
, journal =
Lawrence, Neil D. , journal =. Probabilistic Non-Linear Principal Component Analysis with. 2005 , url =
2005
-
[70]
and Hern
Bui, Thang D. and Hern. Deep. Proceedings of the 33rd International Conference on Machine Learning , pages =. 2016 , volume =
2016
-
[71]
Inference in Deep
Havasi, Marton and Hern. Inference in Deep. Advances in Neural Information Processing Systems , volume =. 2018 , publisher =
2018
-
[72]
and Higdon, David , journal =
Sauer, Annie and Gramacy, Robert B. and Higdon, David , journal =. Active Learning for Deep. 2023 , doi =
2023
-
[73]
Hern. Deep. Machine Learning and Knowledge Discovery in Databases , editor =. 2021 , publisher =. doi:10.1007/978-3-030-67664-3_29 , note =
-
[74]
Alpha-Divergence Minimization for Deep
Villacampa-Calvo, Carlos and Hern. Alpha-Divergence Minimization for Deep. International Journal of Approximate Reasoning , volume =. 2022 , doi =
2022
-
[75]
Salimbeni, Hugh and Dutordoir, Vincent and Hensman, James and Deisenroth, Marc , booktitle =. Deep. 2019 , editor =
2019
-
[76]
Advances in Neural Information Processing Systems , volume =
Yu, Haibin and Chen, Yizhou and Low, Bryan Kian Hsiang and Jaillet, Patrick and Dai, Zhongxiang , title =. Advances in Neural Information Processing Systems , volume =. 2019 , publisher =
2019
-
[77]
Dai, Zhenwen and Damianou, Andreas C. and Gonz. Variational Auto-encoded Deep. Proceedings of the 4th International Conference on Learning Representations , year =. 1511.06455 , archivePrefix =
-
[78]
Mattos, C. Recurrent. Proceedings of the International Conference on Learning Representations , year =. 1511.06644 , archivePrefix =
-
[79]
Matthews, Alexander G. de G. and Hensman, James and Turner, Richard and Ghahramani, Zoubin , booktitle =. On Sparse Variational Methods and the. 2016 , editor =
2016
-
[80]
Nested Variational Compression in Deep Gaussian Processes
Hensman, James and Lawrence, Neil D. , year =. Nested Variational Compression in Deep. doi:10.48550/arXiv.1412.1370 , url =. 1412.1370 , archivePrefix =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1412.1370
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.