pith. sign in

arxiv: 2512.20552 · v2 · submitted 2025-12-23 · 💻 cs.IT · math.IT· stat.ML

Information-theoretic signatures of causality in Bayesian networks and hypergraphs

Sung En Chiang , Zhaolu Liu , Robert L. Peach , Mauricio Barahona This is my paper

Pith reviewed 2026-05-16 20:03 UTC · model grok-4.3

classification 💻 cs.IT math.ITstat.ML
keywords partial information decompositioncausal discoveryBayesian networkshypergraphsunique informationsynergistic informationcollider detectioninformation theory
0
0 comments X

The pith

Unique information characterizes direct causal neighbors while synergy identifies colliders in Bayesian networks and hypergraphs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows that partial information decomposition provides direct signatures for causal relationships in multivariate systems. In standard Bayesian networks, the unique information a variable provides about another exactly marks whether one is a direct cause or effect of the other. Synergistic information, by contrast, flags the presence of collider structures where multiple causes converge on an effect. This local mapping allows the causal neighborhood of each variable to be read off from its information profile alone. The approach extends to hypergraphs, where the same components distinguish additional roles such as co-heads and co-tails and expose a collider effect specific to multi-parent or multi-child hyperedges.

Core claim

We establish the first theoretical correspondence between PID components and causal structure in both Bayesian networks and hypergraphs. Unique information precisely characterizes direct causal neighbors, while synergy identifies collider relationships. This establishes a localist causal discovery paradigm in which the structure surrounding each variable can be recovered from its immediate informational footprint, eliminating the need for global search over graph space. Extending these results to more expressive causal representation, we prove that PID signatures in Bayesian hypergraphs differentiate parents, children, co-heads, and co-tails, revealing a novel collider effect unique to multi

What carries the argument

Partial Information Decomposition (PID) of the mutual information between sources and a target into redundant, unique, and synergistic components, used to map these parts onto specific causal roles in graphs and hypergraphs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Algorithms for causal discovery could be redesigned to compute PID locally first and only then assemble global graphs, potentially scaling better to high-dimensional data.
  • This framework might generalize to settings with latent variables if the PID terms can still be estimated reliably from observations.
  • Connections to other higher-order measures such as integrated information or transfer entropy in dynamical systems could be explored to unify static and dynamic causality inference.

Load-bearing premise

The causal Markov condition and faithfulness assumption must hold, ensuring that all conditional independencies are reflected exactly in the probability distribution and therefore in the PID decomposition.

What would settle it

Find a joint distribution over variables that satisfies the causal Markov condition for a known graph but where the unique information between a pair does not match their direct neighbor status, or where synergy is nonzero without a collider.

Figures

Figures reproduced from arXiv: 2512.20552 by Mauricio Barahona, Robert L. Peach, Sung En Chiang, Zhaolu Liu.

Figure 1
Figure 1. Figure 1: Factorization of distributions according to Bayesian network and Bayesian hyper￾graphs. (a) A Bayesian network where X1, X2, X3 and X4 are the parents of X5. (b) A Bayesian hypergraph where X1 and X2 (similarly X3 and X4) are co-parents, X5 and X6 are co-heads. (c) A Bayesian hypergraph where conditioning on X5 only induces dependence between {X1, X2} and {X3, X4} whereas in (a) conditioning on X5 induces … view at source ↗
Figure 2
Figure 2. Figure 2: Example of maximal hyperedge construction. The Bayesian hypergraph with three hyperedges in (a) has the same conditional independence properties as the one in (b). Yet (b) provides a more parsimonious representation using the maximal hyperedge construction. By combining our results on Bayesian hypergraph unique information and co-tail identification (Theorems 4 and 5), we can pin down the PID signature of … view at source ↗
Figure 3
Figure 3. Figure 3: Hasse diagrams of the redundancy lattice for d = 2 (left) and d = 3 (right) source variables. Each node corresponds to an element of the redundancy lattice, and edges indicate the partial order defined by information containment. For d = 2, the lattice contains 4 elements, yielding the familiar bivariate PID structure. For d = 3, the lattice contains 18 elements, illustrating the rapid growth in lattice si… view at source ↗
read the original abstract

Analyzing causality in multivariate systems involves establishing how information is generated, distributed and combined. Traditional causal discovery frameworks are capable of multivariate reasoning but their intrinsic pairwise graph topology restricts them to do so only indirectly by integrating multivariate information across pairwise edges. Higher-order information theory provides direct tools that can explicitly model higher-order interactions. In particular, Partial Information Decomposition (PID) allows the decomposition of the information that a set of sources provides about a target into redundant, unique, and synergistic components. Yet the mathematical connection between such higher-order information-theoretic measures and causal structure remains undeveloped. Here we establish the first theoretical correspondence between PID components and causal structure in both Bayesian networks and hypergraphs. We first show that in Bayesian networks unique information precisely characterizes direct causal neighbors, while synergy identifies collider relationships. This establishes a localist causal discovery paradigm in which the structure surrounding each variable can be recovered from its immediate informational footprint, eliminating the need for global search over graph space. Extending these results to more expressive causal representation, we prove that PID signatures in Bayesian hypergraphs differentiate parents, children, co-heads, and co-tails, revealing a novel collider effect unique to multi-tail hyperedges. Our results position PID as a rigorous, model-agnostic foundation for inferring both pairwise and higher-order causal structure, and introduce a fundamentally local information-theoretic viewpoint on causal discovery.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper claims to establish the first theoretical correspondence between Partial Information Decomposition (PID) components and causal structure in Bayesian networks and hypergraphs. In Bayesian networks, unique information precisely characterizes direct causal neighbors while synergy identifies collider relationships, enabling a localist causal discovery paradigm based on each variable's immediate informational footprint. The results are extended to Bayesian hypergraphs, where PID signatures differentiate parents, children, co-heads, and co-tails and reveal a novel collider effect unique to multi-tail hyperedges. The derivations rely on the causal Markov condition and faithfulness to align PID atoms exactly with graph structure.

Significance. If the correspondences hold, this work supplies a model-agnostic, information-theoretic foundation for inferring both pairwise and higher-order causal structure. The localist recovery claim, if verified, would eliminate the need for global search over graph space and position PID as a rigorous tool for causal discovery in multivariate systems, including those naturally represented by hypergraphs.

major comments (2)
  1. [§3.2] §3.2 (Bayesian networks): the claim that unique information is nonzero exactly for direct neighbors and zero otherwise is load-bearing for the localist paradigm; the proof sketch should explicitly verify that this holds for both discrete and continuous variables under the chosen PID measure (e.g., I_min or I_broja) without additional restrictions on the joint distribution.
  2. [§4.1] §4.1 (hypergraph extension): the differentiation of co-heads versus co-tails via synergy terms relies on replacing pairwise edges with multi-tail relations; the manuscript must show that the resulting PID decomposition remains exhaustive and non-negative without introducing new axioms beyond the Markov condition and faithfulness.
minor comments (3)
  1. [Abstract / §1] The abstract and introduction should include a brief comparison with existing information-theoretic causal discovery methods (e.g., those based on transfer entropy or directed information) to clarify the precise novelty of the PID signatures.
  2. [§2] Notation for PID atoms (unique, redundant, synergistic) is introduced but not uniformly typeset; adopt a single consistent notation (e.g., Unq, Red, Syn) and define each atom at first appearance.
  3. [Figure 2] Figure 2 (illustrative BN and hypergraph) would benefit from explicit labeling of the PID values computed for each node to allow direct visual verification of the claimed signatures.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive assessment and recommendation for minor revision. The comments identify opportunities to strengthen the explicitness of the proofs, which we will address directly in the revised manuscript.

read point-by-point responses

  1. Referee: [§3.2] §3.2 (Bayesian networks): the claim that unique information is nonzero exactly for direct neighbors and zero otherwise is load-bearing for the localist paradigm; the proof sketch should explicitly verify that this holds for both discrete and continuous variables under the chosen PID measure (e.g., I_min or I_broja) without additional restrictions on the joint distribution.

    Authors: We agree that greater explicitness will improve clarity. In the revised version we will expand the proof in §3.2 to verify the claim in full detail: unique information is nonzero if and only if the source is a direct causal neighbor, and zero otherwise. The argument will be stated separately for discrete and continuous variables, using the PID measure already employed in the manuscript, and will rely exclusively on the causal Markov condition together with faithfulness. No further restrictions on the joint distribution will be introduced. This expanded derivation directly supports the localist recovery procedure. revision: yes

  2. Referee: [§4.1] §4.1 (hypergraph extension): the differentiation of co-heads versus co-tails via synergy terms relies on replacing pairwise edges with multi-tail relations; the manuscript must show that the resulting PID decomposition remains exhaustive and non-negative without introducing new axioms beyond the Markov condition and faithfulness.

    Authors: The PID atoms in the hypergraph setting are obtained from the standard definition of partial information decomposition; exhaustiveness and non-negativity are therefore inherited from the PID axioms themselves. In the revision we will insert a short paragraph (or appendix note) in §4.1 that explicitly records this fact: under the causal Markov condition and faithfulness alone, the decomposition of mutual information into the usual redundant, unique, and synergistic atoms remains exhaustive and non-negative for hyperedges of arbitrary arity. No additional axioms are required. The distinction between co-heads and co-tails then follows immediately from the multi-tail structure and the same two assumptions used in the pairwise case. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper derives correspondences between PID atoms (unique information for direct neighbors, synergy for colliders) and causal roles in Bayesian networks and hypergraphs directly from the definitions of partial information decomposition together with the causal Markov condition and faithfulness. These are standard external assumptions that do not reduce the target result to a fitted parameter, a self-referential equation, or a load-bearing self-citation chain. No step in the provided derivation chain is shown to be equivalent to its inputs by construction; the localist recovery claim follows from the structural properties of the graphs without internal circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on the standard causal Markov condition and faithfulness assumption from graphical causal models together with the usual axioms of Shannon information theory; no free parameters or new entities are introduced.

axioms (2)
  • domain assumption Causal Markov condition holds
    Links conditional independencies in the data to the absence of direct edges in the graph.
  • domain assumption Faithfulness assumption holds
    Ensures that all independencies implied by the graph are observable and not canceled by parameter choices.

pith-pipeline@v0.9.0 · 5548 in / 1124 out tokens · 29162 ms · 2026-05-16T20:03:32.078311+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 2 internal anchors

  1. [1]

    The co-information lattice

    Anthony J Bell. The co-information lattice. In Proceedings of the fifth international workshop on independent component analysis and blind signal separation: ICA, volume 2003, 2003

    work page 2003

  2. [2]

    Quantifying unique information

    Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay. Quantifying unique information. Entropy, 16 0 (4): 0 2161--2183, 2014. ISSN 1099-4300. doi:10.3390/e16042161

    work page doi:10.3390/e16042161 2014

  3. [3]

    Higher-order networks

    Ginestra Bianconi. Higher-order networks. Elements in Structure and Dynamics of Complex Networks, 2021

    work page 2021

  4. [4]

    What are higher-order networks? SIAM Review, 65 0 (3): 0 686--731, 2023

    Christian Bick, Elizabeth Gross, Heather A Harrington, and Michael T Schaub. What are higher-order networks? SIAM Review, 65 0 (3): 0 686--731, 2023

    work page 2023

  5. [5]

    Lattice theory, volume 25

    Garrett Birkhoff. Lattice theory, volume 25. American Mathematical Soc., 1940

    work page 1940

  6. [6]

    Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, 1 edition, 2007. ISBN 0387310738

    work page 2007

  7. [7]

    Inferring spatial and signaling relationships between cells from single cell transcriptomic data

    Zixuan Cang and Qing Nie. Inferring spatial and signaling relationships between cells from single cell transcriptomic data. Nature communications, 11 0 (1): 0 2084, 2020

    work page 2084

  8. [8]

    Quantifying higher-order epistasis: beware the chimera

    Uthsav Chitra, Brian J Arnold, and Benjamin J Raphael. Quantifying higher-order epistasis: beware the chimera. bioRxiv, 2024

    work page 2024

  9. [9]

    Social contagion models on hypergraphs

    Guilherme Ferraz de Arruda, Giovanni Petri, and Yamir Moreno. Social contagion models on hypergraphs. Physical Review Research, 2 0 (2): 0 023032, 2020

    work page 2020

  10. [10]

    Higher-order causal structure learning with additive models

    James Enouen, Yujia Zheng, Ignavier Ng, Yan Liu, and Kun Zhang. Higher-order causal structure learning with additive models. arXiv preprint arXiv:2511.03831, 2025

    work page arXiv 2025

  11. [11]

    Simplicial models of social contagion

    Iacopo Iacopini, Giovanni Petri, Alain Barrat, and Vito Latora. Simplicial models of social contagion. Nature communications, 10 0 (1): 0 1--9, 2019

    work page 2019

  12. [12]

    A computation of the ninth dedekind number

    Christian J \"a kel. A computation of the ninth dedekind number. Journal of Computational Algebra, 6: 0 100006, 2023

    work page 2023

  13. [13]

    On a hypergraph probabilistic graphical model

    Mohammad Ali Javidian, Zhiyu Wang, Linyuan Lu, and Marco Valtorta. On a hypergraph probabilistic graphical model. Annals of Mathematics and Artificial Intelligence, 88 0 (9): 0 1003--1033, 2020

    work page 2020

  14. [14]

    Estimating high-dimensional directed acyclic graphs with the pc-algorithm

    Markus Kalisch and Peter B \"u hlman. Estimating high-dimensional directed acyclic graphs with the pc-algorithm. Journal of Machine Learning Research, 8 0 (3), 2007

    work page 2007

  15. [15]

    Quantifying & modeling multimodal interactions: An information decomposition framework

    Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, et al. Quantifying & modeling multimodal interactions: An information decomposition framework. Advances in Neural Information Processing Systems, 36: 0 27351--27393, 2023

    work page 2023

  16. [16]

    Foundations & trends in multimodal machine learning: Principles, challenges, and open questions

    Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. Foundations & trends in multimodal machine learning: Principles, challenges, and open questions. ACM Computing Surveys, 56 0 (10): 0 1--42, 2024

    work page 2024

  17. [17]

    Information-theoretic measures on lattices for high-order interactions

    Zhaolu Liu, Mauricio Barahona, and Robert L Peach. Information-theoretic measures on lattices for high-order interactions. The International Conference on Artificial Intelligence and Statistics, 28, 2025

    work page 2025

  18. [18]

    Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work

    Joseph T Lizier, Nils Bertschinger, J \"u rgen Jost, and Michael Wibral. Information decomposition of target effects from multi-source interactions: Perspectives on previous, current and future work. Entropy, 20 0 (4): 0 307, 2018

    work page 2018

  19. [19]

    A synergistic core for human brain evolution and cognition

    Andrea I Luppi, Pedro AM Mediano, Fernando E Rosas, Negin Holland, Tim D Fryer, John T O’Brien, James B Rowe, David K Menon, Daniel Bor, and Emmanuel A Stamatakis. A synergistic core for human brain evolution and cognition. Nature Neuroscience, 25 0 (6): 0 771--782, 2022

    work page 2022

  20. [20]

    A synergistic workspace for human consciousness revealed by integrated information decomposition

    Andrea I Luppi, Pedro AM Mediano, Fernando E Rosas, Judith Allanson, John Pickard, Robin L Carhart-Harris, Guy B Williams, Michael M Craig, Paola Finoia, Adrian M Owen, et al. A synergistic workspace for human consciousness revealed by integrated information decomposition. Elife, 12: 0 RP88173, 2024 a

    work page 2024

  21. [21]

    Information decomposition and the informational architecture of the brain

    Andrea I Luppi, Fernando E Rosas, Pedro AM Mediano, David K Menon, and Emmanuel A Stamatakis. Information decomposition and the informational architecture of the brain. Trends in Cognitive Sciences, 2024 b

    work page 2024

  22. [22]

    Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures

    Aobo Lyu, Andrew Clark, and Netanel Raviv. Multivariate partial information decomposition: Constructions, inconsistencies, and alternative measures. arXiv preprint arXiv:2508.05530, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  23. [23]

    A general framework for interpretable neural learning based on local information-theoretic goal functions

    Abdullah Makkeh, Marcel Graetz, Andreas C Schneider, David A Ehrlich, Viola Priesemann, and Michael Wibral. A general framework for interpretable neural learning based on local information-theoretic goal functions. Proceedings of the National Academy of Sciences, 122 0 (10): 0 e2408125122, 2025

    work page 2025

  24. [24]

    Decomposing causality into its synergistic, unique, and redundant components

    \'A lvaro Mart \' nez-S \'a nchez, Gonzalo Arranz, and Adri \'a n Lozano-Dur \'a n. Decomposing causality into its synergistic, unique, and redundant components. Nature Communications, 15 0 (1): 0 9296, 2024

    work page 2024

  25. [25]

    Toward a unified taxonomy of information dynamics via integrated information decomposition

    Pedro AM Mediano, Fernando E Rosas, Andrea I Luppi, Robin L Carhart-Harris, Daniel Bor, Anil K Seth, and Adam B Barrett. Toward a unified taxonomy of information dynamics via integrated information decomposition. Proceedings of the National Academy of Sciences, 122 0 (39): 0 e2423297122, 2025

    work page 2025

  26. [26]

    Causality: Models, Reasoning and Inference

    Judea Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X

    work page 2009

  27. [27]

    Reconsidering unique information: Towards a multivariate information decomposition

    Johannes Rauh, Nils Bertschinger, Eckehard Olbrich, and J \"u rgen Jost. Reconsidering unique information: Towards a multivariate information decomposition. In 2014 IEEE International Symposium on Information Theory, pages 2232--2236. IEEE, 2014

    work page 2014

  28. [28]

    Quantifying high-order interdependencies via multivariate extensions of the mutual information

    Fernando E Rosas, Pedro AM Mediano, Michael Gastpar, and Henrik J Jensen. Quantifying high-order interdependencies via multivariate extensions of the mutual information. Physical Review E, 100 0 (3): 0 032305, 2019

    work page 2019

  29. [29]

    On the foundations of combinatorial theory: I

    Gian-Carlo Rota. On the foundations of combinatorial theory: I. theory of m \"o bius functions. In Classic Papers in Combinatorics, pages 332--360. Springer, 1964

    work page 1964

  30. [30]

    Higher-order organization of multivariate time series

    Andrea Santoro, Federico Battiston, Giovanni Petri, and Enrico Amico. Higher-order organization of multivariate time series. Nature Physics, 19 0 (2): 0 221--229, 2023

    work page 2023

  31. [31]

    Random walks on simplicial complexes and the normalized hodge 1-laplacian

    Michael T Schaub, Austin R Benson, Paul Horn, Gabor Lippner, and Ali Jadbabaie. Random walks on simplicial complexes and the normalized hodge 1-laplacian. SIAM Review, 62 0 (2): 0 353--391, 2020

    work page 2020

  32. [32]

    Synergy, redundancy, and independence in population codes

    Elad Schneidman, William Bialek, and Michael J Berry. Synergy, redundancy, and independence in population codes. Journal of Neuroscience, 23 0 (37): 0 11539--11553, 2003

    work page 2003

  33. [33]

    Causation, prediction, and search

    Peter Spirtes, Clark N Glymour, and Richard Scheines. Causation, prediction, and search. MIT press, 2000

    work page 2000

  34. [34]

    Prevalence and patterns of higher-order drug interactions in escherichia coli

    Elif Tekin, Cynthia White, Tina Manzhu Kang, Nina Singh, Mauricio Cruz-Loya, Robert Damoiseaux, Van M Savage, and Pamela J Yeh. Prevalence and patterns of higher-order drug interactions in escherichia coli. NPJ systems biology and applications, 4 0 (1): 0 31, 2018

    work page 2018

  35. [35]

    Varley and Patrick Kaminski

    Thomas F. Varley and Patrick Kaminski. Untangling synergistic effects of intersecting social identities with partial information decomposition. Entropy, 24 0 (10), 2022. ISSN 1099-4300

    work page 2022

  36. [36]

    Partial entropy decomposition reveals higher-order structures in human brain activity

    Thomas F Varley, Maria Pope, Maria Grazia Puxeddu, Joshua Faskowitz, and Olaf Sporns. Partial entropy decomposition reveals higher-order structures in human brain activity. arXiv preprint arXiv:2301.05307, 2023

    work page arXiv 2023

  37. [37]

    Nonnegative Decomposition of Multivariate Information

    Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information. arXiv preprint arXiv:1004.2515, 2010

    work page internal anchor Pith review Pith/arXiv arXiv 2010