RECORD
Variable Selection Consistency of Gaussian Process Regression
Statistics Theory · Submitted 12 Dec 2019
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Bayesian nonparametric regression under a rescaled Gaussian process prior offers smoothness-adaptive function estimation with near minimax-optimal error rates. Hierarchical extensions of this approach, equipped with stochastic variable selection, are known to also adapt to the unknown intrinsic dimension of a sparse true regression function. But it remains unclear if such extensions offer variable selection consistency, i.e., if the true subset of important variables could be consistently learned from the data. It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors. Our result covers the high dimensional asymptotic setting where the predictor dimension is allowed to grow with the sample size. The proof utilizes Schwartz theory to establish that the posterior probability of wrong selection vanishes asymptotically. A necessary and challenging technical development involves providing sharp upper and lower bounds to small ball probabilities at all rescaling levels of the Gaussian process prior, a result that could be of independent interest.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Bayesian nonparametric regression under a rescaled Gaussian process prior offers smoothness-adaptive function estimation with near minimax-optimal error rates. | background 1.00 | 0.07 | 0.03 | asserted 1.00 |
| s1 | Hierarchical extensions of this approach, equipped with stochastic variable selection, are known to also adapt to the unknown intrinsic dimension of a sparse true regression function. | background 1.00 | 0.04 | 0.03 | asserted 0.99 |
| s2 | But it remains unclear if such extensions offer variable selection consistency, i.e., if the true subset of important variables could be consistently learned from the data. | gap 1.00 | 0.04 | 0.03 | unclear 0.62 |
| s3 | It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors. | result 0.95 | 0.92 | 0.91 | hedged possibility 0.90 |
| s4 | Our result covers the high dimensional asymptotic setting where the predictor dimension is allowed to grow with the sample size. | contribution 0.47 | 0.72 | 0.05 | asserted 0.98 |
| s5 | The proof utilizes Schwartz theory to establish that the posterior probability of wrong selection vanishes asymptotically. | method 0.98 | 0.88 | 0.05 | asserted 1.00 |
| s6 | A necessary and challenging technical development involves providing sharp upper and lower bounds to small ball probabilities at all rescaling levels of the Gaussian process prior, a result that could be of independent interest. | method 0.55 | 0.91 | 0.03 | asserted 0.79 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Bayesian nonparametric regression under a rescaled Gaussian process prior offers smoothness-adaptive function estimation with near minimax-optimal error rates. Hierarchical extensions of this approach, equipped with stochastic variable selection, are known to also adapt to the unknown intrinsic dimension of a sparse true regression function. But it remains unclear if such extensions offer variable selection consistency, i.e., if the true subset of important variables could be consistently learned from the data. It is shown here that variable consistency may indeed be achieved with such models at least when the true regression function has finite smoothness to induce a polynomially larger penalty on inclusion of false positive predictors. Our result covers the high dimensional asymptotic setting where the predictor dimension is allowed to grow with the sample size. The proof utilizes Schwartz theory to establish that the posterior probability of wrong selection vanishes asymptotically. A necessary and challenging technical development involves providing sharp upper and lower bounds to small ball probabilities at all rescaling levels of the Gaussian process prior, a result that could be of independent interest.
Human review
How the literature uses it
1 arXiv paper cite this in 2 sentences
Papers citing it · 1 sentence without a confident label
Cites it as background 1 paper
-
Theory for Bayesian estimation and variable selection in high-dimensional Gaussian nonparametric regression has been well-studied under basis expansions [ref] , tree and forest priors [ref] , and Gaussian process priors [ref] .
Bayesian Group Regularization in Generalized Linear Models with a Continuous Spike-and-Slab Prior · Generalized Additive Models · 0.97
What it cites
Cites as support 1 paper
-
Indeed, under such Bayesian approaches, the posterior distribution of the regression function is known to contract to the true function at a near minimax optimal rate, adapting to both unknown smoothness and sparsity [ref] .
Dimension adaptability of Gaussian process models with variable selection and projection · Introduction · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.