Pith. sign in

REVIEW 3 major objections 6 minor 26 references

The many faces of multivariate information

T0 review · 3 major / 6 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Three standard multivariate information measures—the dual total correlation, S-information, and negative O-information—are all special cases of a single parameterized function, Δ^k, whose parameter k selects the order of interaction it dete

desk verdict The unification of S, D, and −O into the Δ^k family is correct and genuinely useful, but the abstract overstates the sign-hierarchy claim: the interpretation is only proven for disjoint pure-k subsets, and the simplest non-disjoint pure 4th-order synergy (even parity) already gives Δ^2 > 0, contradicting the abstract's 'lower-order dominance' reading. read the letter →

arxiv 2601.08030 v2 pith:KMWHWVHG submitted 2026-01-12 cs.IT math.IT

Thomas F. Varley This is my paper · ORCID
classification cs.ITmath.IT MSC 94A17
keywords multivariateinformationhigher-orderinteractionssynergyredundancyO-informationS-informationdualtotalcorrelationentropicconjugation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Three measures of multivariate information that are usually treated as distinct—the dual total correlation, the S-information, and the negative O-information—are all special cases of a single function, Δ^k(X) = (N−k)T(X) − Σ_i T(X^{−i}), for k = 0, 1, and 2. The paper argues that the integer k tunes which order of interaction the measure is sensitive to: a negative value signals dominance by interactions of order greater than k, a positive value by lower-order interactions, and zero for pure kth-order dependencies. The same construction under entropic conjugation produces a conjugate family Γ^k arranged into a hierarchy of higher-order redundancies. Because the mechanism is combinatorial, the framework extends to any non-negative set function with the right marginalization properties, yielding topological analogues of the information measures. The unification matters because it replaces a zoo of ad hoc statistics with one tunable quantity and suggests a way to estimate the dominant order of interaction in a system.

What carries the argument

The central object is the function Δ^k(X) = (N−k)T(X) − Σ_i T(X^{−i}), a 'whole-minus-sum-of-parts' statistic comparing the total correlation of a system to the sum of its leave-one-out marginals. Its free integer parameter k determines how many times the whole-system term is counted; the combinatorial interpretation is that N−k is the number of single-element removals that leave a kth-order interaction intact. This counting rule is what turns the sign of Δ^k into an order selector. The conjugate function Γ^k(X) = S(X) − kD(X) plays the mirror role for redundancy hierarchies. Together they show that the three named measures are faces of one family.

What would settle it

Construct a mixed system with a known strong 4th-order synergistic interaction plus a weak pairwise correlation, and compute Δ^2; if Δ^2 ≥ 0 while the 4th-order term clearly dominates, the sign-to-order interpretation fails.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the dual total correlation D(X), the S-information S(X), and the negative O-information −O(X) are not separate quantities but evaluations of one function, Δ^k(X) = (N−k)T(X) − Σ_i T(X^{−i}), where T is the total correlation. Setting k=0, 1, 2 gives S, D, and −O respectively. The accompanying conjecture—that the parameter k selects the order of interaction the statistic responds to—is supported by counting arguments: (N−k) is the number of leave-one-out marginals that preserve a kth-order interaction, and for a system built purely of kth-order interactions Δ^k = 0. The paper proves this for pure kth-order synergies and redundancies and shows additivity ov

Load-bearing premise

The claim that the sign of Δ^k reveals the order of dominant interactions is proven only for systems composed purely of kth-order synergies or redundancies; for mixed systems—the common empirical case—it is a conjecture, and if any mixture breaks the sign pattern the central interpretation collapses.

Editorial extensions

If this is right

  • Researchers can use a single tunable statistic Δ^k instead of computing S-information, dual total correlation, and O-information separately.
  • The largest k for which Δ^k(X) > 0 provides an estimate of the highest-order synergy that still dominates the system's lower-order structure.
  • The conjugate hierarchy Γ^k offers the redundancy counterpart, with the largest k for Γ^k > 0 indicating how much of the system can be compressed into lower-order redundancies.
  • Since the construction is combinatorial, any function satisfying the three stated criteria yields a valid higher-order interaction measure, opening the door to non-entropic and structured (e.g., graph-theoretic) analogues.
  • The sign of Δ^k at successive k values gives a spectrum of interaction orders, refining the binary synergy/redundancy classification of O-information.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the sign interpretation is only proven for pure interactions, a practical user should treat the order-tuned reading of Δ^k on mixed data as a heuristic; a natural next step would be to test the hierarchy on synthetic mixtures with known interaction orders.
  • The combinatorial form suggests that similar 'whole vs leave-one-out parts' statistics could be built for other aggregate functions such as variance, mutual information with a target, or risk measures, potentially transferring the synergy/redundancy vocabulary to non-Shannon settings.
  • The optimal k for a real dataset might serve as a fingerprint of scale: systems with the same Δ^k spectrum could be said to have the same interaction-order profile, offering a new kind of system comparison.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a unifying function Δ^k(X) = (N−k)T(X) − ∑_i T(X^{-i}), where T is the total correlation, and shows that the S-information, dual total correlation, and negative O-information are respectively Δ^0, Δ^1, and Δ^2. It also introduces a conjugate function Γ^k via entropic conjugation and claims that Δ^k and Γ^k form hierarchies that diagnose whether a system is dominated by interactions of order higher or lower than k. The appendix proves the algebraic identities and additivity properties. The abstract further promises a generalization to arbitrary set functions and a graph-theoretic case study based on the cyclomatic number.

Significance. The algebraic unification is correct and genuinely clarifying: expressing S, D, and −O as members of a single parametric family is a useful observation, and the four appendix proofs are valid. If the order-tuning interpretation were established, the hierarchy would be a practical exploratory tool for higher-order interactions in complex systems. However, the interpretive claim that gives the unification its significance is currently conjectural: the body labels the key role of k as a conjecture, while the abstract presents it as an established result. The promised set-function generalization and graph case study are absent from the body. As written, the paper is a valuable formal observation wrapped in an overclaimed interpretive package.

major comments (3)
  1. [Abstract; §'The k parameter'; Proof 3] The central hierarchy claim is load-bearing and unsupported as stated. The abstract asserts that Δ^k(X)>0 means X is dominated by interactions of order greater than k and Δ^k(X)<0 means lower order, but the body introduces this as a conjecture ('We conjecture that the primary role of the k parameter is...'). Proof 3 only establishes Δ^k=0 for disjoint unions of independent k-element subsets with T(X^{-i})=0; it does not prove the sign claim for mixed or overlapping interaction orders. Moreover, 'dominated by interactions of order greater/less than k' is never formally defined, so the claim is not yet falsifiable. The Discussion's statement that 'for a system X with purely kth-order dependencies, Δ^k(X)=0' is stronger than what Proof 3 proves. The same overstatement applies to Γ^k: Proof 4 only covers k-element giant-bit copies, not general 'pure kth-order redundancy.' The abstract/body m
  2. [§'The k parameter'] The counting baseline is a heuristic, not a theorem. The paper states that if a system were composed purely of interactions of order k, the leave-one-out sum would equal (N−k)T(X), because each interaction is counted N−k times. This assumes additivity of T over the system's interactions, but T is not additive over overlapping interactions, and no formal definition of 'composed purely of interactions of order k' is given under which the identity holds. Since the sign interpretation of Δ^k depends on this baseline, the authors should either state and prove the counting identity for a precise class of distributions or explicitly label it as heuristic and explain what could falsify it.
  3. [Abstract; Discussion (final paragraphs)] The abstract promises that the framework generalizes to any set function f satisfying three criteria and that, using the graph cyclomatic number as a case study, topological analogues of dual total correlation, O-information, and S-information are derived. The supplied body contains none of this: the three criteria appear only as a list in the Discussion with no statement or proof that Δ^k defined with such an f retains the claimed order interpretation, and no graph cyclomatic example is present anywhere. If this material exists, it must be included; if not, the abstract and Discussion must be revised so the paper does not claim results it does not report.
minor comments (6)
  1. [Eq. (5)] Typo: 'todal correlation' should be 'total correlation'.
  2. [Introduction] Typo: 'interactionsquathem' should be 'interactions themselves' or similar.
  3. [§'The k parameter'] The XOR symbol is rendered as 'L' in 'X1 = X2 L X3'; it should be ⊕.
  4. [Equations (8), (16)] Equation numbering has glitches: some displays have equation numbers on blank lines. Please clean up the numbering.
  5. [§'A conjugate hierarchy of redundancies'] The entropic conjugation operator Conj(·) is used without a formal definition or citation beyond a reference. Please state explicitly how Conj acts on the random vector or on the function.
  6. [Eq. (15)] The allowed range of the 'free parameter' k is not specified. For the three named measures k∈{0,1,2}; please state the intended domain (e.g., 0≤k≤N) and the behavior outside it, if relevant.

Circularity Check

1 steps flagged · score 2.0 of 10

The Δ^k unification is algebraically self-contained; the only circular note is that the sign-hierarchy calibration (Δ^k=0 at pure order k) restates the definition of 'order-k interaction,' while the body itself labels the general role of k a conjecture.

  1. self definitional [Proof 3 (Appendix) and 'The k parameter' section]
    "By definition, a synergistic interaction of order k satisfies T(X −j αi ) = 0 for all j: removing any single element destroys the dependency entirely. Therefore: Δk(Xαi ) = 0"

    The paper uses Δ^k=0 for 'purely kth-order dependencies' as the calibration for the hierarchy claim, but this result is the defining property of order restated: an order-k synergistic interaction is defined by T(X^{-j})=0, and for a k-element block Δ^k(X)=−Σ_j T(X^{-j}), so the zero is built into the definition. The counting factor N−k is also drawn from the same definition ('N−k counts how many times an interaction of order k will survive'), so the sign interpretation is not an independent property of Δ^k. The body calls the general role of k a 'conjecture' while the abstract presents the hierarchy as established; overlapping/mixed systems are not proved.

full rationale

The central unification is not circular: Eq. (11) for D(X) is proven from the entropy definition in Proof 1, S(X) and −O(X) follow by linear combination, and Δ^k is introduced as the common algebraic form, so S=Δ^0, D=Δ^1, −O=Δ^2 are identities rather than fitted or predicted quantities. There are no data-driven parameters and no load-bearing self-citations: the one self-citation for Eq. (11) ([14]) is backed by an appendix proof. The only definitional issue is the interpretive hierarchy: the zero-at-pure-order-k calibration in Proof 3 restates the definition of order-k synergy, and the paper itself hedges the general k-role with 'We conjecture.' The abstract states the sign hierarchy as fact, which overstates the proven content, and the promised topological/graph-cyclomatic-number section is absent from the provided text. These are support/logic concerns more than circularity, so the score is low.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The paper introduces no physical entities. The new objects are mathematical functions (Δ^k, Γ^k) that are the paper's own contribution rather than unexplained postulates. The free parameter k is a user-selected integer, not a fitted constant. The main external inputs are standard entropy formalism, the entropic conjugation background, and the idealized definition of pure kth-order interactions, which is an assumption that may not hold in realistic data.

free parameters (1)
  • k
    The hierarchy index. It is not fitted to data; it is a free integer chosen by the user to select a member of the Δ^k family. The paper's core interpretation of Δ^k as tracking order-k interactions depends on this parameter, but it is the independent variable of the family, not an ad hoc fitted constant.
assumptions (5)
  • standard math Standard Shannon entropy and mutual information definitions and their basic properties (e.g., T(X)=ΣH(X_i)−H(X)).
    Invoked throughout, starting at Eq. (1)-(5).
  • standard math The identity D(X) = (N−1)T(X) − Σ T(X^{−i}) (Proof 1).
    Though cited to [14], it is fully proved in the appendix using only the definition of T, so it is a derived result, not an external assumption.
  • domain assumption The entropic conjugation framework of Rosas et al. [10] is accepted as background.
    Used in Section 'A conjugate hierarchy of redundancies' to define Γ^k = Conj(Δ^k). The paper does not prove or derive entropic conjugation itself.
  • domain assumption Definition of an 'order-k synergistic interaction' as one where T(X)>0 but T(X^{−i})=0 for all i.
    This idealization is used in Proof 3 and in the heuristic counting argument in 'The k parameter'. Real-world interactions rarely satisfy such a strict fragility condition.
  • ad hoc to paper Three criteria for generalizing Δ^k to any set function f: non-negativity, monotonicity under marginalization, and fragility of pure relationships.
    Introduced in the Discussion to justify the combinatorial nature of Δ^k. These are stipulated desiderata, not derived from information theory, and are used to argue that any f meeting them yields a valid synergy measure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The many faces of multivariate information." pith.science (2026). https://pith.science/paper/KMWHWVHG

@misc{pith2026260108030,
  author       = {Pith},
  title        = {Pith review of: The many faces of multivariate information},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KMWHWVHG}},
  note         = {Machine review of arXiv:2601.08030}
}
abstract

Extracting higher-order structures from multivariate data has become an area of intensive study in complex systems science, as these multipartite interactions can reveal insights into fundamental features of complex systems like emergent phenomena. Information theory provides a natural language for exploring these interactions, as it elegantly formalizes the problem of comparing "wholes" and "parts" using joint, conditional, and marginal entropies. A large number of distinct statistics have been developed over the years, all aiming to capture different aspects of "higher-order" information sharing. Here, we show that these functions are special cases of a more general function, $\Delta^{k}$ which is parameterized by a free parameter $k$. Generally, the $\Delta^{k}$ function is arranged into a hierarchy of increasingly high-order synergies; for a given value of $k$, if $\Delta^{k}(\mathbf{X})>0$, then $\mathbf{X}$ is dominated by interactions with order greater than $k$, while if $\Delta^{k}(\mathbf{X})<0$, then $\mathbf{X}$ is dominated by interactions with order lower than $k$. Using the entropic conjugation framework, we also find that the conjugate of $\Delta^{k}$, which we term $\Gamma^{k}$ is arranged into a similar hierarchy of increasingly high-order redundancies. Finally, we show that the interpretation of $\Delta^{k}$ as a measure of synergy is combinatorial, rather than specific to any particular information-theoretic measure, allowing us to generalize the whole framework and define measures of synergy on any set function that meets certain criteria. Using the graph cyclomatic number as a case study, we derive topological analogues of the dual total correlation, O-information, and S-information that describe the cyclic structure of simple graphs.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 3 linked inside Pith

  1. [1]

    Erik P. Hoel. When the Map Is Better Than the Territory.Entropy, 19(5):188, April 2017. Number: 5

    2017

  2. [2]

    Rosas, Pedro A

    Fernando E. Rosas, Pedro A. M. Mediano, Henrik J. Jensen, Anil K. Seth, Adam B. Barrett, Robin L. Carhart-Harris, and Daniel Bor. Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data.PLOS Computational Biology, 16(12):e1008289, December 2020. Number: 12 Publisher: Public Library of Science

    2020

  3. [3]

    Rosas, Bernhard C

    Fernando E. Rosas, Bernhard C. Geiger, Andrea I. Luppi, Anil K. Seth, Daniel Polani, Michael Gastpar, and Pedro A. M. Mediano. Software in the natural world: A computational approach to hierarchical emergence, February 2024. arXiv:2402.09090 [nlin] version: 1

    arXiv 2024

  4. [4]

    Varley and Erik Hoel

    Thomas F. Varley and Erik Hoel. Emergence as the conversion of information: a unifying theory.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 380(2227):20210150, July 2022. Publisher: Royal Society

    2022

  5. [5]

    Lionel Barnett and Anil K. Seth. Dynamical independence: discovering emergent macro- scopic processes in complex dynamical systems.arXiv:2106.06511 [nlin], June 2021. arXiv: 2106.06511

    arXiv 2021

  6. [6]

    Thomas F. Varley. Information theory for complex systems scientists: What, why, and how. Physics Reports, 1148:1–55, December 2025

    2025

  7. [7]

    Fernando Rosas, Pedro A. M. Mediano, Michael Gastpar, and Henrik J. Jensen. Quanti- fying High-order Interdependencies via Multivariate Extensions of the Mutual Information. Physical Review E, 100(3):032305, September 2019. Number: 3 arXiv: 1902.11239

    arXiv 2019

  8. [8]

    Watanabe

    S. Watanabe. Information Theoretical Analysis of Multivariate Correlation.IBM Journal of Research and Development, 4(1):66–82, January 1960. Number: 1

    1960

Show all 26 references
  1. [9]

    Tononi, O

    G. Tononi, O. Sporns, and G. M. Edelman. A measure for brain complexity: relating functional segregation and integration in the nervous system.Proceedings of the National Academy of Sciences, 91(11):5033–5037, May 1994. Number: 11

    1994

  2. [10]

    Rosas, Aaron J

    Fernando E. Rosas, Aaron J. Gutknecht, Pedro A. M. Mediano, and Michael Gastpar. Char- acterising high-order interdependence via entropic conjugation.Communications Physics, 8(1):1–11, August 2025. Publisher: Nature Publishing Group

    2025

  3. [11]

    Nonnegative entropy measures of multivariate symmetric correlations.Infor- mation and Control, 36(2):133–156, February 1978

    Te Sun Han. Nonnegative entropy measures of multivariate symmetric correlations.Infor- mation and Control, 36(2):133–156, February 1978

    1978

  4. [12]

    Abdallah and Mark D

    Samer A. Abdallah and Mark D. Plumbley. A measure of statistical complexity based on predictive information with application to finite spin systems.Physics Letters A, 376(4):275– 281, January 2012

    2012

  5. [13]

    James, Christopher J

    Ryan G. James, Christopher J. Ellison, and James P. Crutchfield. Anatomy of a bit: In- formation in a time series observation.Chaos: An Interdisciplinary Journal of Nonlinear Science, 21(3):037109, September 2011. Publisher: American Institute of Physics

    2011

  6. [14]

    Varley, Maria Pope, Joshua Faskowitz, and Olaf Sporns

    Thomas F. Varley, Maria Pope, Joshua Faskowitz, and Olaf Sporns. Multivariate informa- tion theory uncovers synergistic subsystems of the human cerebral cortex.Communications Biology, 6(1):1–12, April 2023. Number: 1 Publisher: Nature Publishing Group. 8

    2023

  7. [15]

    Mediano, Fernando E

    Marilyn Gatica, Rodrigo Cofr´ e, Pedro A.M. Mediano, Fernando E. Rosas, Patricio Orio, Ibai Diez, Stephan P. Swinnen, and Jesus M. Cortes. High-Order Interdependencies in the Aging Brain.Brain Connectivity, April 2021. Publisher: Mary Ann Liebert, Inc., publishers

    2021

  8. [16]

    Adarsh, Jitka Annen, Char- lotte Martial, Marie-Elisabeth Faymonville, Steven Laureys, Corine Sombrun, Ramakrish- nan Angarai Ganesan, Audrey Vanhaudenhuyse, and Olivia Gosseries

    Pradeep Kumar G., Rajanikant Panda, Kanishka Sharma, A. Adarsh, Jitka Annen, Char- lotte Martial, Marie-Elisabeth Faymonville, Steven Laureys, Corine Sombrun, Ramakrish- nan Angarai Ganesan, Audrey Vanhaudenhuyse, and Olivia Gosseries. Changes in high- order interaction measur...

    2024

  9. [17]

    Gorana Mijatovic, Laura Sparacino, Yuri Antonacci, Michal Javorka, Daniele Marinazzo, Sebastiano Stramaglia, and Luca Faes. Assessing High-Order Links in Cardiovascular and Respiratory Networks via Static and Dynamic Information Measures.IEEE Open Journal of Engineering in Med...

    2024

  10. [18]

    Gradients of O-information highlight synergy and redundancy in physiological ap- plications.Frontiers in Network Physiology, 3, January 2024

    Tomas Scagliarini, Laura Sparacino, Luca Faes, Daniele Marinazzo, and Sebastiano Stra- maglia. Gradients of O-information highlight synergy and redundancy in physiological ap- plications.Frontiers in Network Physiology, 3, January 2024. Publisher: Frontiers

    2024

  11. [19]

    Tomas Scagliarini, Daniele Marinazzo, Yike Guo, Sebastiano Stramaglia, and Fernando E. Rosas. Quantifying high-order interdependencies on individual patterns via the local O-information: Theory and applications to music analysis.Physical Review Research, 4(1):013184, March 202...

    2022

  12. [20]

    Integrated information in discrete dynamical systems: motivation and theoretical framework.PLoS computational biology, 4(6):e1000091, June 2008

    David Balduzzi and Giulio Tononi. Integrated information in discrete dynamical systems: motivation and theoretical framework.PLoS computational biology, 4(6):e1000091, June 2008

    2008

  13. [21]

    Thomas F. Varley. A Synergistic Perspective on Multivariate Computation and Causality in Complex Systems.Entropy, 26(10):883, October 2024. Number: 10 Publisher: Multidis- ciplinary Digital Publishing Institute

    2024

  14. [22]

    Rosas, Pedro A.M

    Fernando E. Rosas, Pedro A.M. Mediano, and Michael Gastpar. Characterising Directed and Undirected Metrics of High-Order Interdependence. In2024 IEEE International Symposium on Information Theory Workshops (ISIT-W), pages 1–6, July 2024

    2024

  15. [23]

    Thomas F. Varley. A scalable synergy-first backbone decomposition of higher-order struc- tures in complex systems.npj Complexity, 1(1):1–11, July 2024. Publisher: Nature Pub- lishing Group

    2024

  16. [24]

    Higham, and Naomichi Hatano

    Ernesto Estrada, Desmond J. Higham, and Naomichi Hatano. Communicability betweenness in complex networks.Physica A: Statistical Mechanics and its Applications, 388(5):764–774, March 2009. Number: 5

    2009

  17. [25]

    Varley, Pedro A

    Thomas F. Varley, Pedro A. M. Mediano, Alice Patania, and Josh Bongard. The topol- ogy of synergy: Linking topological and information-theoretic approaches to higher-order interactions in complex systems.PLOS Computational Biology, 21(11):e1013649, November

  18. [2025]

    giant bit

    Publisher: Public Library of Science. 9 Proof 1 Theorem.The dual total correlation can be expressed as a linear combination of joint and marginal total correlations: D(X) = (N−1)T(X)− NX i=1 T(X −i) (23) Proof.We expand the right-hand side using the definitionT(X) = PN i=1 H(X...

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.