A benchmark of 18 biased entropy estimators on H, MI, and CMI concludes that Chao-Shen and Chao-Wang-Jost are fastest to converge and most accurate, though the CMI computation appears mis-specified.
Entropy and inference, revisited
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We study properties of popular near-uniform (Dirichlet) priors for learning undersampled probability distributions on discrete nonmetric spaces and show that they lead to disastrous results. However, an Occam-style phase space argument expands the priors into their infinite mixture and resolves most of the observed problems. This leads to a surprisingly good estimator of entropies of discrete distributions.
citation-role summary
background 1
citation-polarity summary
fields
cs.IT 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
To BEE or not to BEE: Estimating more than Entropy with Biased Entropy Estimators
A benchmark of 18 biased entropy estimators on H, MI, and CMI concludes that Chao-Shen and Chao-Wang-Jost are fastest to converge and most accurate, though the CMI computation appears mis-specified.