REVIEW 36 cited by
Nonnegative Decomposition of Multivariate Information
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Nonnegative Decomposition of Multivariate Information
read the original abstract
Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general structure of the information that a set of sources provides about a given variable. We begin with a new definition of redundancy as the minimum information that any source provides about each possible outcome of the variable, averaged over all possible outcomes. We then show how this measure of redundancy induces a lattice over sets of sources that clarifies the general structure of multivariate information. Finally, we use this redundancy lattice to propose a definition of partial information atoms that exhaustively decompose the Shannon information in a multivariate system in terms of the redundancy between synergies of subsets of the sources. Unlike interaction information, the atoms of our partial information decomposition are never negative and always support a clear interpretation as informational quantities. Our analysis also demonstrates how the negativity of interaction information can be explained by its confounding of redundancy and synergy.
Forward citations
Cited by 36 Pith papers
-
DIPHINE: Diffusion-based $\Phi$-ID Neural Estimator
DIPHINE is the first diffusion-based neural estimator for the 16 ΦID atoms in continuous non-Gaussian dynamical systems, obtained by joint MI estimation followed by Möbius inversion.
-
Information-theoretic signatures of causality in Bayesian networks and hypergraphs
Partial information decomposition components map directly onto causal roles such as direct parents, children, and colliders in both pairwise Bayesian networks and higher-order hypergraphs.
-
Multivariate Partial Information Decomposition: Constructions, Inconsistencies, and Alternative Measures
The authors establish an impossibility theorem for lattice-based PID beyond three sources and introduce alternative measures of multivariate unique and synergistic information using dependency-eliminating random varia...
-
Explicit Formula for Partial Information Decomposition
Provides the first explicit formula for partial information decomposition atoms satisfying Williams and Beer's axioms via a novel do-operation.
-
From Statistical to Structural Synergy: A Predictability Framework to Quantify the Effects due to High-Order Mechanisms
Structural synergy, defined as the excess predictive power of a joint model over the best additive model, isolates non-additive interaction mechanisms from dependency-driven statistical synergy in complex systems.
-
The Representational Limit of Scalar Interactions: An Interventional Decomposition
Signed pairwise interaction scores conflate U/R/S; Stochastic Hi-Fi uses interventional masked inference to recover per-feature uniqueness, redundancy, and synergy profiles.
-
Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability
Introduces Synergistic Faithfulness metric based on Shapley Interaction Index to evaluate cross-modal synergy in VLM explainers, revealing over-reliance on visual salience in existing methods.
-
Closed-Form Gaussian Estimators for Multi-Source Partial Information Decomposition
Closed-form log-determinant expressions provide the first covariance-based estimators for multi-source PID quantities including redundancy, unique information, and synergy in Gaussian variables.
-
Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information
Channel importance splits into task relevance and local replaceability; local-axis metrics predict safe removal under pruning better than target-axis metrics across multiple CNNs and datasets.
-
Quantifying Spacetime Integration across a Partition with Synergy
Synergy-based measures of spacetime integration outperform current IIT practice when tested on simple deterministic networks.
-
Quantifying Spacetime Integration across a Partition with Synergy
Synergy-based measures from partial information decomposition are found more suitable than current practice for quantifying integration in simple deterministic networks for the Information Integration Theory of Consciousness.
-
Structural Impossibility of Antichain-Lattice Partial Information Decomposition
Antichain-lattice indexing in PID is structurally insufficient to recover mutual information from information atoms for multivariate cases.
-
Emergent Coordination in Multi-Agent Language Models
Multi-agent LLM systems can be steered via prompt design from mere aggregates to higher-order collectives with identity-linked differentiation and goal-directed complementarity, as measured by partial information deco...
-
Information-Theoretic Decomposition for Multimodal Interaction Learning
DMIL is a multimodal learning framework that decomposes sample-specific interactions into redundant, unique, and synergistic components via variational architecture and uses them for adaptive fine-tuning.
-
Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs
Adversarial fine-tuning evades activation-based steganography detection in five LLMs while preserving secret recovery, but a recontextualization dataset restores both ridge and MLP probe detectability.
-
Defining Unique, Redundant, and Synergistic Quantum Information
Extends PID to quantum domain with applications showing unique info must be zero in erasure-correctable QEC subsets and redundant info central to Zurek decoherence.
-
Before Fusion, Ask What to Keep: Contextual Calibration of Multimodal Signals
A pre-fusion calibration module modulates multimodal features using cross-modality support and conflict cues to improve performance on five benchmarks including sentiment analysis and audio-visual tasks.
-
Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty
The paper defines the Epistemic Trap from interacting state and dynamics uncertainties in RL and introduces an Adaptive Safety Architecture with a Compound Uncertainty Coefficient to enable active information-seeking ...
-
Emergence of information interference in stochastic systems with non-diagonal noise and switching environments
In stochastic systems with non-diagonal noise and switching environments, mutual information includes irreducible static and dynamic interference terms that prevent simple decomposition.
-
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
SynIB is an information-theoretic objective that adds a penalty for unimodal confidence to standard task loss, improving accuracy on synergy-dependent examples by up to 7.8% across synthetic XOR tasks and five real-wo...
-
Partial Effective Information Decomposition for Synergistic Causality
PEID decomposes the causal effect of multiple sources on a target under maximum-entropy interventions into unique and synergistic information, enabling hyperedge causal graphs and downward causation analysis.
-
Quantifying Spacetime Integration across a Partition with Synergy
Introduces four synergy-based measures of spacetime integration from partial information decomposition and finds them more suitable than current IIT practice for simple deterministic networks.
-
Quantifying Spacetime Integration across a Partition with Synergy
Four synergy-based integration measures from partial information decomposition are shown to outperform current IIT integration measures on simple deterministic networks.
-
Heterophily as a generative mechanism for self-organized synergistic interdependencies
Heterophily weakens pairwise couplings while inducing geometric constraints that create synergistic higher-order interdependencies in a co-evolving spin-glass model.
-
RAG-GNN: Integrating Retrieved Knowledge with Graph Neural Networks for Precision Medicine
RAG-GNN augments GNNs with retrieved literature knowledge via gated fusion to improve functional clustering of 379 proteins in cancer signaling networks, raising silhouette score by 0.093.
-
A scalable estimator of higher-order information in complex dynamical systems
Introduces M-information as a scalable measure of higher-order information integration in multivariate time series, computed via convex optimization and tested on neuronal and neuroimaging data.
-
Exo-Daisy World: Revisiting Gaia Theory through an Informational Architecture Perspective
The Exo-Daisy World model, built from stochastic differential equations, shows biosphere-environment correlations strengthening with stellar luminosity through distinct phases of information exchange quantified as rei...
-
BrainFIBRE: A Foundation Model via Information Decomposition for Brain Microstructure
BrainFIBRE presents a foundation model for brain microstructure that applies self-supervised partial information decomposition on NODDI maps to disentangle unique, synergistic, and redundant information and reports st...
-
Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
PID applied to MLLMs identifies task-specific modality interaction profiles that generalize across models, extend to tri-modal cases, and yield initial performance gains via reweighting.
-
Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty
The paper proposes an Adaptive Safety Architecture with a mutual-information-based Compound Uncertainty Coefficient, MaxInfoRL policies, and adaptive constraints to actively resolve compound epistemic uncertainty in R...
-
Rethinking Federated Unlearning via the Lens of Memorization
Introduces Grouped Memorization Evaluation and FedMemPrune to remove unique memorized information in federated unlearning while preserving overlapping knowledge.
-
Beyond Information Redundancy: Expanding Cross-Modal Knowledge Representation for Power Load Time Series Forecasting
KEMM-Net enriches power load time series representations with cross-modal text and visual knowledge via PID-guided contrastive learning to outperform baselines in few-shot forecasting scenarios.
-
More Is Different: Toward a Theory of Emergence in AI-Native Software Ecosystems
AI-native software ecosystems exhibit emergent behaviors best explained by complex adaptive systems theory, requiring new ecosystem-level monitoring and seven testable propositions that may extend or replace Lehman's laws.
-
ConceptTracer: Interactive Analysis of Concept Saliency and Selectivity in Neural Representations
ConceptTracer supplies an interactive interface and saliency/selectivity metrics to locate concept-responsive neurons in neural representations, shown on TabPFN.
-
DRIADA: A Python Toolkit for Cross-Scale Analysis of Single-Neuron Selectivity and Population Dynamics
DRIADA supplies a unified Python data model and workflow for single-neuron selectivity and population dynamics analysis, shown on synthetic data, 13-mouse hippocampal imaging, and a toroidal attractor simulation.
-
Beyond Information Redundancy: Expanding Cross-Modal Knowledge Representation for Power Load Time Series Forecasting
PrismNet combines text and image modalities with time series via a PID-guided contrastive learning module to boost few-shot power load forecasting accuracy and provide interpretability.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.