A benchmark of 18 biased entropy estimators on H, MI, and CMI concludes that Chao-Shen and Chao-Wang-Jost are fastest to converge and most accurate, though the CMI computation appears mis-specified.
On Generalized Sch\"urmann Entropy Estimators
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We present a new class of estimators of Shannon entropy for severely undersampled discrete distributions. It is based on a generalization of an estimator proposed by T. Schuermann, which itself is a generalization of an estimator proposed by myself in arXiv:physics/0307138. For a special set of parameters they are completely free of bias and have a finite variance, something with is widely believed to be impossible. We present also detailed numerical tests where we compare them with other recent estimators and with exact results, and point out a clash with Bayesian estimators for mutual information.
citation-role summary
background 1
citation-polarity summary
fields
cs.IT 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
To BEE or not to BEE: Estimating more than Entropy with Biased Entropy Estimators
A benchmark of 18 biased entropy estimators on H, MI, and CMI concludes that Chao-Shen and Chao-Wang-Jost are fastest to converge and most accurate, though the CMI computation appears mis-specified.