Posterior uncertainty for kernel density estimates

Recent work in predictive Bayesian inference has enabled novel Bayesian interpretations of many well-known stochastic one-step-ahead predictive algorithms. In this paper, we study classic kernel density estimation in the predictive Bayesian framework. We prove that their predictive measures converge weakly almost surely $\unicode{x2013}$ meaning that their associated predictive resampling sequences are almost surely asymptotically exchangeable $\unicode{x2013}$ and we provide estimators for moments of the limiting random probability measure. We also show that the resampling sequences do not satisfy standard assumptions like being conditionally identically distributed (c.i.d.) or almost c.i.d. (a.c.i.d.), thus providing a non-trivial example of a predictive sequence which is not a.c.i.d. but nevertheless converges weakly almost surely. For Gaussian kernels, we show that the limiting directing measure is almost surely absolutely continuous with respect to the Lebesgue measure, meaning it emits a probability density. This enables us to derive credibility intervals for kernel density estimates, which we illustrate on two real datasets.