REVIEW 2 cited by
A New Lower Bound for Kullback-Leibler Divergence Based on Hammersley-Chapman-Robbins Bound
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this paper, we derive a useful lower bound for the Kullback-Leibler divergence (KL-divergence) based on the Hammersley-Chapman-Robbins bound (HCRB). The HCRB states that the variance of an estimator is bounded from below by the Chi-square divergence and the expectation value of the estimator. By using the relation between the KL-divergence and the Chi-square divergence, we show that the lower bound for the KL-divergence which only depends on the expectation value and the variance of a function we choose. This lower bound can also be derived from an information geometric approach. Furthermore, we show that the equality holds for the Bernoulli distributions and show that the inequality converges to the Cram\'{e}r-Rao bound when two distributions are very close. We also describe application examples and examples of numerical calculation.
Forward citations
Cited by 2 Pith papers
-
Causal Covariate Shift Correction using Fisher information penalty
A Fisher information penalty added to the loss during batchwise training is claimed to correct covariate shift and improve accuracy, but the supporting derivation and experiments are incomplete.
-
PIcsC: Partitioning-Induced Covariate Shift Correction
A Fisher-information regularizer is proposed to correct partition-induced covariate shift in cross-validation and federated learning, with reported gains of 3-5 points over FedAvg-class baselines.
Discussion (0). Continue with ORCID to comment.