Optimal Inference After Model Selection
read the original abstract
To perform inference after model selection, we propose controlling the selective type I error; i.e., the error rate of a test given that it was performed. By doing so, we recover long-run frequency properties among selected hypotheses analogous to those that apply in the classical (non-adaptive) context. Our proposal is closely related to data splitting and has a similar intuitive justification, but is more powerful. Exploiting the classical theory of Lehmann and Scheff\'e (1955), we derive most powerful unbiased selective tests and confidence intervals for inference in exponential family models after arbitrary selection procedures. For linear regression, we derive new selective z-tests that generalize recent proposals for inference after model selection and improve on their power, and new selective t-tests that do not require knowledge of the error variance.
This paper has not been read by Pith yet.
Forward citations
Cited by 12 Pith papers
-
Post-ADC Inference: Valid Inference After Active Data Collection
Post-ADC inference supplies valid p-values and confidence intervals for data-dependent targets after active data collection by extending selective inference to correct for both adaptive sampling bias and post-hoc targ...
-
In-Sample Evaluation of Subgroups Identified by Generic Machine Learning
A conditional adaptive perturbation approach enables valid in-sample inference for machine learning-identified subgroups with nonregular boundaries via triple robustness.
-
Integrating Diagnostic Checks into Estimation
Residualizing estimators against diagnostic check statistics eliminates selective reporting distortions, reduces variance when the model is correct, and minimizes worst-case bias under local misspecification.
-
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking
SIREN corrects winner's curse bias in adaptive LLM benchmarking via selection-aware repeated splits and bootstrap for valid procedure-level confidence intervals.
-
A Leakage Bound for Confidence Sets after Black-Box Selection
Selected-target noncoverage after black-box selection is bounded by nominal fixed-target noncoverage plus average total variation distance between marginal and conditional laws of the inferential data.
-
Post-Screening Portfolio Selection
A Lasso-based screening step followed by low-dimensional mean-variance optimization on the selected assets improves high-dimensional portfolio construction, with a defactoring extension for strong factors.
-
$\phi$-Table: A Statistical Explanation for Global SHAP
The φ-table extends SHAP rankings into a statistical table by fitting standardized linear surrogates to the model response and reporting direction, uncertainty, fidelity, and stability.
-
Improving Power by Conditioning on Less in Post-selection Inference for Changepoints
Monte Carlo approximation of selective p-values in changepoint post-selection inference that conditions on less to improve power while remaining valid for any sample size.
-
Selective Inference via Marginal Screening for High Dimensional Classification
Derives asymptotic selective inference for high-dimensional logistic regression post marginal screening to enable valid hypothesis testing.
-
Weighted Holm Procedures: Theory, Properties, and Recommendations
The weighted Holm procedure (WHP) based on ordered weighted p-values is uniformly more powerful than the weighted alternative Holm procedure (WAP) based on ordered raw p-values, with stronger optimality properties und...
-
Statistical Test for Diffusion-Based Anomaly Localization via Selective Inference
A selective inference framework is proposed to provide p-values controlling false positive rates for diffusion-based anomaly localization in images.
-
Inference conditional on selection: a review
The review covers selective inference techniques that provide conditional guarantees for inference after data-dependent selection, demonstrated with examples from winner inference, regression trees, clustering, and si...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.