Recognition: 2 theorem links
· Lean TheoremStructural Impossibility of Antichain-Lattice Partial Information Decomposition
Pith reviewed 2026-05-13 16:42 UTC · model grok-4.3
The pith
No universal rule recovers mutual information from antichain-indexed atoms in partial information decomposition.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For general multivariate PID, there is no universal rule that recovers the decomposed mutual information from the antichain-indexed information atoms. In particular, two systems can share identical atoms regardless of any axioms while having different mutual information. These results reveal the limits of antichain-lattice and motivate relation-based foundations for multivariate information measures.
What carries the argument
The antichain lattice, which indexes partial information atoms by the source groups that can recover each component of the target variable.
If this is right
- Purely axiomatic adjustments within an antichain-lattice structure cannot resolve incompatibilities for three or more sources.
- System Information Decomposition yields a self-consistent entropy breakdown for the target-free three-variable case with an operational redundancy definition.
- The sum of antichain atoms cannot be guaranteed to equal total mutual information in a unique and consistent manner.
- Relation-based representations are needed to overcome the representational obstruction in general multivariate information measures.
Where Pith is reading between the lines
- Alternative indexing structures based on explicit relations or graphs could avoid the non-uniqueness of mutual information recovery.
- Applications such as feature attribution in machine learning may encounter ambiguous results when using current antichain PID methods.
- Empirical checks on specific distributions where atoms match but mutual information differs could confirm the structural mismatch in practice.
Load-bearing premise
That the antichain lattice provides the correct indexing structure for partial information components in the multivariate setting.
What would settle it
A function that maps any set of antichain-indexed atoms back to the correct mutual information value for every possible joint distribution would disprove the claim.
Figures
read the original abstract
Partial Information Decomposition (PID) represents multivariate mutual information via antichain-lattice that aims to specify which source groups can recover which informational components of a target. For three or more sources, widely desired PID axioms become mutually incompatible. This is often treated as an axiomatic tuning issue. This paper argues that the obstruction is representational, rooted in the antichain indexing itself, so that purely axiomatic adjustments within an antichain-lattice structure cannot resolve it in general. We first introduce System Information Decomposition (SID) for the special target-free three-variable setting, obtaining a self-consistent entropy decomposition with an operational redundancy definition. More fundamentally, we then show that for general multivariate PID, there is no universal rule that recovers the decomposed mutual information from the antichain-indexed information atoms. In particular, two systems can share identical atoms regardless of any axioms while having different mutual information. These results reveal the limits of antichain-lattice and motivate relation-based foundations for multivariate information measures.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that the well-known incompatibilities among desired axioms for partial information decomposition (PID) in the multivariate setting are not merely an issue of axiom selection but stem from a deeper representational limitation: the use of antichain-lattice indexing for information atoms. For the special target-free three-variable case the authors introduce System Information Decomposition (SID), which yields a self-consistent entropy decomposition together with an operational definition of redundancy. The central result is a proof that no universal recovery rule exists that can reconstruct the decomposed mutual information from antichain-indexed atoms; explicitly, there exist distinct systems that possess identical atoms (independent of any particular choice of axioms) yet differ in their mutual information.
Significance. If the non-uniqueness result holds, the work is significant for information theory because it relocates the PID impasse from the level of axiom tuning to the choice of indexing structure itself. The explicit construction of SID supplies a concrete, axiom-independent base case, while the general counter-example demonstrates that identical atom assignments can correspond to different mutual-information values. This supplies a clear motivation for exploring relation-based rather than lattice-based foundations for multivariate information measures.
major comments (1)
- [Section presenting the general multivariate result] The general impossibility argument (that two systems can share identical antichain-indexed atoms yet differ in mutual information) is load-bearing for the central claim. The manuscript should explicitly exhibit the two systems, state the precise atom values that are identical, and show the differing mutual-information values, including the entropy calculations that establish the difference.
minor comments (2)
- [SID introduction] The operational definition of redundancy in the SID construction should be stated as a numbered definition or equation so that its independence from standard PID redundancy measures is immediately verifiable.
- [Notation and preliminaries] Notation for the antichain-indexed atoms should be introduced once and used consistently; currently the transition from the three-variable SID to the general case introduces new symbols without an explicit mapping table.
Simulated Author's Rebuttal
We thank the referee for the positive assessment and the recommendation of minor revision. The suggestion to make the general impossibility result fully explicit is constructive and will strengthen the presentation of the central claim.
read point-by-point responses
-
Referee: [Section presenting the general multivariate result] The general impossibility argument (that two systems can share identical antichain-indexed atoms yet differ in mutual information) is load-bearing for the central claim. The manuscript should explicitly exhibit the two systems, state the precise atom values that are identical, and show the differing mutual-information values, including the entropy calculations that establish the difference.
Authors: We agree that an explicit numerical counter-example will improve readability and verifiability. In the revised manuscript we will add a dedicated example subsection that constructs two concrete systems (binary random variables with explicitly defined joint distributions), lists the identical antichain-indexed atom values for each, and supplies the direct entropy calculations confirming that the mutual information differs. This addition leaves the existing general proof unchanged while addressing the load-bearing character of the result. revision: yes
Circularity Check
Derivation self-contained via explicit counter-example
full rationale
The paper introduces SID as an operational decomposition for the target-free three-variable case using standard entropy properties, then demonstrates the structural non-uniqueness result by exhibiting two systems that share identical antichain-indexed atoms yet differ in mutual information. This construction rests directly on the lattice representation and entropy axioms without parameter fitting, self-referential definitions, or load-bearing self-citations. The impossibility claim is shown by concrete counter-example rather than by reducing to prior author work or ansatz smuggling. No step in the provided derivation chain collapses to its own inputs by construction.
Axiom & Free-Parameter Ledger
axioms (1)
- standard math Standard properties of entropy and mutual information
invented entities (1)
-
System Information Decomposition (SID)
no independent evidence
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
there is no universal rule that recovers the decomposed mutual information from the antichain-indexed information atoms... two systems can share identical atoms... while having different mutual information (Theorem 1, Lemma 5)
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
antichain-indexed atoms are not informative enough to determine the decomposed quantity
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Forward citations
Cited by 1 Pith paper
-
Closed-Form Gaussian Estimators for Multi-Source Partial Information Decomposition
Closed-form log-determinant expressions provide the first covariance-based estimators for multi-source PID quantities including redundancy, unique information, and synergy in Gaussian variables.
Reference graph
Works this paper leans on
-
[1]
Nonnegative Decomposition of Multivariate Information
Paul L Williams and Randall D Beer. Nonnegative decomposition of multivariate information.arXiv preprint arXiv:1004.2515, 2010
work page Pith review arXiv 2010
-
[2]
Jason Crampton and George Loizou. The completion of a poset in a lattice of antichains.International Mathematical Journal, 1(3):223–238, 2001
work page 2001
-
[3]
Faisal Hamman and Sanghamitra Dutta. Demystifying local and global fairness trade-offs in federated learning using partial information decom- position.arXiv preprint arXiv:2307.11333, 2023
-
[4]
Thomas F Varley, Maria Pope, Maria Grazia, Joshua, and Olaf Sporns. Partial entropy decomposition reveals higher-order information structures in human brain activity.Proceedings of the National Academy of Sciences, 120(30):e2300888120, 2023
work page 2023
-
[5]
Ehren L Newman, Thomas F Varley, Vibin K Parakkattu, Samantha P Sherrill, and John M Beggs. Revealing the dynamics of neural informa- tion processing with multivariate information decomposition.Entropy, 24(7):930, 2022
work page 2022
-
[6]
Fernando E Rosas, Pedro AM Mediano, Henrik J Jensen, Anil K Seth, Adam B Barrett, Robin L Carhart-Harris, and Daniel Bor. Rec- onciling emergences: An information-theoretic approach to identify causal emergence in multivariate data.PLoS computational biology, 16(12):e1008289, 2020
work page 2020
-
[7]
Pedro AM Mediano, Fernando E Rosas, Andrea I Luppi, Henrik J Jensen, Anil K Seth, Adam B Barrett, Robin L Carhart-Harris, and Daniel Bor. Greater than the parts: a review of the information decomposition approach to causal emergence.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 380(2227), 2022
work page 2022
-
[8]
Bing Yuan, Jiang Zhang, Aobo Lyu, Jiayun Wu, Zhipeng Wang, Mingzhe Yang, Kaiwei Liu, Muyun Mou, and Peng Cui. Emergence and causality in complex systems: A survey of causal emergence and related quantita- tive studies.Entropy, 26(2):108, 2024
work page 2024
-
[9]
Paul Pu Liang, Yun Cheng, Xiang Fan, Chun Kai Ling, Suzanne Nie, Richard Chen, Zihao Deng, Nicholas Allen, Randy Auerbach, Faisal Mahmood, et al. Quantifying & modeling multimodal interactions: An information decomposition framework.Advances in Neural Information Processing Systems, 36:27351–27393, 2023
work page 2023
-
[10]
Intersection information based on common randomness.Entropy, 16(4):1985–2000, 2014
Virgil Griffith, Edwin KP Chong, Ryan G James, Christopher J Ellison, and James P Crutchfield. Intersection information based on common randomness.Entropy, 16(4):1985–2000, 2014
work page 1985
-
[11]
Robin AA Ince. Measuring multivariate redundant information with pointwise common change in surprisal.Entropy, 19(7):318, 2017
work page 2017
-
[12]
Bivariate measure of redundant information.Physical Review E, 87(1):012130, 2013
Malte Harder, Christoph Salge, and Daniel Polani. Bivariate measure of redundant information.Physical Review E, 87(1):012130, 2013
work page 2013
-
[13]
Quantifying unique information.Entropy, 16(4):2161–2183, 2014
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, Jürgen Jost, and Nihat Ay. Quantifying unique information.Entropy, 16(4):2161–2183, 2014
work page 2014
-
[14]
Explicit formula for partial information decomposition
Aobo Lyu, Andrew Clark, and Netanel Raviv. Explicit formula for partial information decomposition. In2024 IEEE International Symposium on Information Theory (ISIT), pages 2329–2334. IEEE, 2024
work page 2024
-
[15]
Shared information—new insights and problems in decomposing infor- mation in complex systems
Nils Bertschinger, Johannes Rauh, Eckehard Olbrich, and Jürgen Jost. Shared information—new insights and problems in decomposing infor- mation in complex systems. InProceedings of the European conference on complex systems 2012, pages 251–269. Springer, 2013
work page 2012
-
[16]
Reconsidering unique information: Towards a multivariate information decomposition
Johannes Rauh, Nils Bertschinger, Eckehard Olbrich, and Jürgen Jost. Reconsidering unique information: Towards a multivariate information decomposition. In2014 IEEE International Symposium on Information Theory, pages 2232–2236. IEEE, 2014
work page 2014
-
[17]
A novel approach to the partial information decom- position.Entropy, 24(3):403, 2022
Artemy Kolchinsky. A novel approach to the partial information decom- position.Entropy, 24(3):403, 2022
work page 2022
-
[18]
Philip Hendrik Matthias, Abdullah Makkeh, Michael Wibral, and Aaron J Gutknecht. Novel inconsistency results for partial information decompo- sition.arXiv preprint arXiv:2512.16662, 2025
-
[19]
Aobo Lyu, Andrew Clark, and Netanel Raviv. Multivariate partial in- formation decomposition: Constructions, inconsistencies, and alternative measures.Physical Review E, 113(3):034102, 2026
work page 2026
-
[20]
Conor Finn and Joseph T Lizier. Pointwise partial information decom- positionusing the specificity and ambiguity lattices.Entropy, 20(4):297, 2018
work page 2018
-
[21]
Robin AA Ince. The partial entropy decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal.arXiv preprint arXiv:1702.01591, 2017
-
[22]
Daniel Chicharro and Stefano Panzeri. Synergy and redundancy in dual decompositions of mutual information gain and information loss.Entropy, 19(2):71, 2017
work page 2017
-
[23]
Fernando E Rosas, Pedro AM Mediano, Borzoo Rassouli, and Adam B Barrett. An operational information decomposition via synergistic disclosure.Journal of Physics A: Mathematical and Theoretical, 53(48):485001, 2020
work page 2020
-
[24]
Towards a synergy-based approach to measuring information modification
Joseph T Lizier, Benjamin Flecker, and Paul L Williams. Towards a synergy-based approach to measuring information modification. In2013 IEEE Symposium on Artificial Life (ALIFE), pages 43–51. IEEE, 2013
work page 2013
-
[25]
System information decomposition.arXiv preprint arXiv:2306.08288, 2023
Aobo Lyu, Bing Yuan, Ou Deng, Mingzhe Yang, Andrew Clark, and Jiang Zhang. System information decomposition.arXiv preprint arXiv:2306.08288, 2023
-
[26]
Peter Gács, Janos Korner, et al. Common information is far less than mutual information.Problems of Control and Information Theory, 2:149– 162, 1973
work page 1973
-
[27]
Himanshu Tyagi, Prakash Narayan, and Piyush Gupta. When is a function securely computable?IEEE Transactions on Information Theory, 57(10):6337–6350, 2011. APPENDIX A. Comparison between SID and two sources PID Fig. 4. Comparison between SID and two-source PID. SID extends the scope of 2-source PID from mutual infor- mationI(S\S i;S i)to the joint entropyH...
work page 2011
-
[28]
Proof of Lemma 1: Proof.In( ¯S1, ¯S2, ¯S3, ¯T), let ¯S1 and ¯S2 be two indepen- dent Bernoulli(1/2)variables, let ¯S3 = ¯S1 ⊕ ¯S2, and let ¯T= ( ¯S1, ¯S2, ¯S3). Therefore, we have I( ¯T; ¯S1, ¯S2, ¯S3) = 2.(16) Our subsequent proof idea is to use Property 1 to obtain the values of all PI-atoms in any system with two sources and the target variable (i.e.( ...
-
[29]
Derivation of SID Axiom 1:In SID, the mutual infor- mation between any two variables and the third one can be decomposed similarly to two-source PID. That is, for any distincti, j, k∈ {1,2,3},I(S i, Sj;S k)splits into four SI- atoms (analogous to (1)): I(S i, Sj;S k) = Ψ S({{i}{j}{k}}) + ΨS({{i}{k}}) + ΨS({{j}{k}}) + ΨS({{ij}{k}}),(21) and the two-variabl...
-
[30]
Proof of Lemma 2: Proof.We consider the linear constraints relating to the follow- ing ten unknowns (the ten SI-atoms of a three-variable system). Define the following vector of atoms: X= h Ψ123({{1}{2}{3}}), Ψ123({{1}{2}}),Ψ 123({{1}{3}}),Ψ 123({{2}{3}}), Ψ123({{1}{23}}),Ψ 123({{2}{13}}),Ψ 123({{3}{12}}), Ψ123({{1}}),Ψ 123({{2}}),Ψ 123({{3}}) iT , and th...
-
[31]
Proof of Lemma 3: Proof.SID Axiom 2 (Commutativity) is clearly satisfied by Definition 5, since the condition is symmetric with respect to the input variables; SID Axiom 4 (Self-redundancy) is also satisfied by the definition. SID Axiom 3 (Monotonicity) follows from Definition 5 since adding a new variable imposes additional constraints on the maximizatio...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.