REVIEW 3 cited by
Formal Limitations on the Measurement of Mutual Information
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Formal Limitations on the Measurement of Mutual Information
read the original abstract
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).
Forward citations
Cited by 3 Pith papers
-
Dream to Control: Learning Behaviors by Latent Imagination
Dreamer learns to control from images by imagining and optimizing behaviors in a learned latent world model, outperforming prior methods on 20 visual tasks in data efficiency and final performance.
-
Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
Matching JEPA features to a rectified generalized Gaussian target produces sparse, non-negative representations with accuracy close to dense baselines.
-
NMINE: Normalized Mutual Information Neural Estimation
A fully neural estimator for normalized mutual information beats a KSG baseline on Gaussian data but fails to deliver its advertised scale-invariance property.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.