Gaussian and Bootstrap Approximations for Suprema of Empirical Processes

In this paper we develop non-asymptotic Gaussian approximation results for the sampling distribution of suprema of empirical processes when the indexing function class $\mathcal{F}_n$ varies with the sample size $n$ and may not be Donsker. Prior approximations of this type required upper bounds on the metric entropy of $\mathcal{F}_n$ and uniform lower bounds on the variance of $f \in \mathcal{F}_n$ which, both, limited their applicability to high-dimensional inference problems. In contrast, the results in this paper hold under simpler conditions on boundedness, continuity, and the strong variance of the approximating Gaussian process. The results are broadly applicable and yield a novel procedure for bootstrapping the distribution of empirical process suprema based on the truncated Karhunen-Lo{\`e}ve decomposition of the approximating Gaussian process. We demonstrate the flexibility of this new bootstrap procedure by applying it to three fundamental problems in high-dimensional statistics: simultaneous inference on parameter vectors, inference on the spectral norm of covariance matrices, and construction of simultaneous confidence bands for functions in reproducing kernel Hilbert spaces.