Post-ADC inference supplies valid p-values and confidence intervals for data-dependent targets after active data collection by extending selective inference to correct for both adaptive sampling bias and post-hoc target selection, relying only on noise assumptions.
hub
Optimal Inference After Model Selection
12 Pith papers cite this work. Polarity classification is still indexing.
12
Pith papers citing it
abstract
To perform inference after model selection, we propose controlling the selective type I error; i.e., the error rate of a test given that it was performed. By doing so, we recover long-run frequency properties among selected hypotheses analogous to those that apply in the classical (non-adaptive) context. Our proposal is closely related to data splitting and has a similar intuitive justification, but is more powerful. Exploiting the classical theory of Lehmann and Scheff\'e (1955), we derive most powerful unbiased selective tests and confidence intervals for inference in exponential family models after arbitrary selection procedures. For linear regression, we derive new selective z-tests that generalize recent proposals for inference after model selection and improve on their power, and new selective t-tests that do not require knowledge of the error variance.
hub tools
citation-role summary
background 1
other 1
citation-polarity summary
representative citing papers
A conditional adaptive perturbation approach enables valid in-sample inference for machine learning-identified subgroups with nonregular boundaries via triple robustness.
Residualizing estimators against diagnostic check statistics eliminates selective reporting distortions, reduces variance when the model is correct, and minimizes worst-case bias under local misspecification.
The φ-table extends SHAP rankings into a statistical table by fitting standardized linear surrogates to the model response and reporting direction, uncertainty, fidelity, and stability.
Monte Carlo approximation of selective p-values in changepoint post-selection inference that conditions on less to improve power while remaining valid for any sample size.
Derives asymptotic selective inference for high-dimensional logistic regression post marginal screening to enable valid hypothesis testing.
SIREN corrects winner's curse bias in adaptive LLM benchmarking via selection-aware repeated splits and bootstrap for valid procedure-level confidence intervals.
Selected-target noncoverage after black-box selection is bounded by nominal fixed-target noncoverage plus average total variation distance between marginal and conditional laws of the inferential data.
A Lasso-based screening step followed by low-dimensional mean-variance optimization on the selected assets improves high-dimensional portfolio construction, with a defactoring extension for strong factors.
A selective inference framework is proposed to provide p-values controlling false positive rates for diffusion-based anomaly localization in images.
The weighted Holm procedure (WHP) based on ordered weighted p-values is uniformly more powerful than the weighted alternative Holm procedure (WAP) based on ordered raw p-values, with stronger optimality properties under FWER control.
The review covers selective inference techniques that provide conditional guarantees for inference after data-dependent selection, demonstrated with examples from winner inference, regression trees, clustering, and single-cell RNA sequencing.
citing papers explorer
-
Post-ADC Inference: Valid Inference After Active Data Collection
Post-ADC inference supplies valid p-values and confidence intervals for data-dependent targets after active data collection by extending selective inference to correct for both adaptive sampling bias and post-hoc target selection, relying only on noise assumptions.
-
In-Sample Evaluation of Subgroups Identified by Generic Machine Learning
A conditional adaptive perturbation approach enables valid in-sample inference for machine learning-identified subgroups with nonregular boundaries via triple robustness.
-
Integrating Diagnostic Checks into Estimation
Residualizing estimators against diagnostic check statistics eliminates selective reporting distortions, reduces variance when the model is correct, and minimizes worst-case bias under local misspecification.
-
$\phi$-Table: A Statistical Explanation for Global SHAP
The φ-table extends SHAP rankings into a statistical table by fitting standardized linear surrogates to the model response and reporting direction, uncertainty, fidelity, and stability.
-
Improving Power by Conditioning on Less in Post-selection Inference for Changepoints
Monte Carlo approximation of selective p-values in changepoint post-selection inference that conditions on less to improve power while remaining valid for any sample size.
-
Selective Inference via Marginal Screening for High Dimensional Classification
Derives asymptotic selective inference for high-dimensional logistic regression post marginal screening to enable valid hypothesis testing.
-
Towards Reliable LLM Evaluation: Correcting the Winner's Curse in Adaptive Benchmarking
SIREN corrects winner's curse bias in adaptive LLM benchmarking via selection-aware repeated splits and bootstrap for valid procedure-level confidence intervals.
-
A Leakage Bound for Confidence Sets after Black-Box Selection
Selected-target noncoverage after black-box selection is bounded by nominal fixed-target noncoverage plus average total variation distance between marginal and conditional laws of the inferential data.
-
Post-Screening Portfolio Selection
A Lasso-based screening step followed by low-dimensional mean-variance optimization on the selected assets improves high-dimensional portfolio construction, with a defactoring extension for strong factors.
-
Statistical Test for Diffusion-Based Anomaly Localization via Selective Inference
A selective inference framework is proposed to provide p-values controlling false positive rates for diffusion-based anomaly localization in images.
-
Weighted Holm Procedures: Theory, Properties, and Recommendations
The weighted Holm procedure (WHP) based on ordered weighted p-values is uniformly more powerful than the weighted alternative Holm procedure (WAP) based on ordered raw p-values, with stronger optimality properties under FWER control.
-
Inference conditional on selection: a review
The review covers selective inference techniques that provide conditional guarantees for inference after data-dependent selection, demonstrated with examples from winner inference, regression trees, clustering, and single-cell RNA sequencing.