REVIEW 2 cited by
Formal Limitations on the Measurement of Mutual Information
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).
Forward citations
Cited by 2 Pith papers
-
Rectified LpJEPA: Joint-Embedding Predictive Architectures with Sparse and Maximum-Entropy Representations
Matching JEPA features to a rectified generalized Gaussian target produces sparse, non-negative representations with accuracy close to dense baselines.
-
NMINE: Normalized Mutual Information Neural Estimation
A fully neural estimator for normalized mutual information beats a KSG baseline on Gaussian data but fails to deliver its advertised scale-invariance property.
Discussion (0). Sign in to comment.