Stability of a Generalized Debiased Lasso with Applications to Resampling-Based Variable Selection
Pith reviewed 2026-05-24 01:02 UTC · model grok-4.3
The pith
A stability-based update to the generalized debiased Lasso approximates the estimator accurately for all but a vanishing fraction of coordinates under sub-Gaussian designs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A generalized debiased Lasso estimator based on a stability principle admits a simple update formula when a single column of the design matrix is perturbed. Under sub-Gaussian designs with well-conditioned covariance, in the proportional growth regime, the approximation is asymptotically accurate for all but a vanishing fraction of coordinates. The proof uses concentration and anti-concentration arguments to control error terms and sign changes, while comparable distributional limits remain open.
What carries the argument
The stability principle that supplies a simple update formula for the generalized debiased Lasso when one design column is perturbed.
If this is right
- The approximation significantly reduces the computational cost of resampling-based variable selection procedures.
- It applies to the conditional randomization test.
- It supports a local knockoff filter.
- The stability approximation holds in settings where full Gaussian distributional limits are still open.
Where Pith is reading between the lines
- Similar stability updates might be constructible for other high-dimensional penalized estimators beyond the Lasso.
- The cost reduction could extend to other resampling schemes such as bootstrap or cross-validation in high dimensions.
- Numerical checks in finite samples could test how quickly the vanishing fraction disappears as n grows.
- The gap between stable approximation and open distributional limits suggests stability may be provable under weaker conditions than full asymptotics.
Load-bearing premise
The design matrix satisfies sub-Gaussian tail bounds and has a well-conditioned covariance matrix, with analysis restricted to the proportional growth regime.
What would settle it
A sub-Gaussian design matrix with well-conditioned covariance for which the stability approximation error fails to vanish for a non-vanishing fraction of coordinates when p/n approaches a constant.
Figures
read the original abstract
We propose a generalized debiased Lasso estimator based on a stability principle. When a single column of the design matrix is perturbed, the estimator admits a simple update formula that can be computed from the original solution. Under sub-Gaussian designs with well-conditioned covariance, this approximation is asymptotically accurate for all but a vanishing fraction of coordinates in the proportional growth regime. The proof relies on concentration and anti-concentration arguments to control error terms and sign changes. In contrast, establishing comparable distributional limits (e.g., Gaussianity) under similar assumptions remains open. As an application, we show that the approximation significantly reduces the computational cost of resampling-based variable selection procedures, including the conditional randomization test and a local knockoff filter.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a generalized debiased Lasso estimator grounded in a stability principle. When a single column of the design matrix is perturbed, the estimator admits a simple update formula that can be computed from the original solution. Under sub-Gaussian designs with well-conditioned covariance, this approximation is asymptotically accurate for all but a vanishing fraction of coordinates in the proportional growth regime. The proof relies on concentration and anti-concentration arguments to control error terms and sign changes. As an application, the approximation significantly reduces the computational cost of resampling-based variable selection procedures, including the conditional randomization test and a local knockoff filter.
Significance. If the central asymptotic accuracy result holds, this work provides a computationally efficient method for approximating the debiased Lasso under column perturbations, which has direct implications for scalable resampling-based inference in high dimensions. The approach leverages standard concentration tools in a novel way for stability updates. The authors' note that stronger distributional limits remain open demonstrates appropriate caution. This contributes to the field by offering practical speedups without sacrificing the core statistical properties under the stated assumptions. The manuscript includes applications to established procedures like CRT and knockoffs, enhancing its relevance.
minor comments (1)
- [Abstract] Abstract: A brief parenthetical reference to the specific concentration inequalities employed in the proof would help readers quickly gauge the technical level.
Simulated Author's Rebuttal
We thank the referee for their positive summary, significance assessment, and recommendation to accept the manuscript. We appreciate the recognition of the stability-based update formula, its asymptotic accuracy under the stated assumptions, and the computational benefits for resampling procedures such as the CRT and local knockoffs.
Circularity Check
No significant circularity; derivation self-contained via concentration arguments
full rationale
The paper derives the stability-based update formula and its asymptotic accuracy from concentration and anti-concentration inequalities applied to sub-Gaussian designs in the proportional regime. No step reduces a claimed prediction or result to a fitted parameter, self-definition, or load-bearing self-citation. The update is presented as following directly from the perturbed Lasso solution, with error control shown via standard tail bounds rather than by construction or renaming. The abstract explicitly flags that stronger limits like Gaussianity remain open, confirming the argument does not smuggle in its own conclusion. This is the normal case of an independent derivation.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Design matrix entries are sub-Gaussian with well-conditioned covariance
- domain assumption Proportional growth regime (p/n → constant)
Reference graph
Works this paper leans on
-
[1]
Benjamin Aubin, Florent Krzakala, Yue Lu, and Lenka Zdeborov \'a . Generalization error in high-dimensional perceptrons: Approaching bayes error with convex optimization. Advances in Neural Information Processing Systems, 33: 0 12199--12210, 2020
work page 2020
-
[2]
A leave-one-out approach to approximate message passing
Zhigang Bao, Qiyang Han, and Xiaocong Xu. A leave-one-out approach to approximate message passing. arXiv preprint arXiv:2312.05911, 2023
- [3]
-
[4]
Second-order stein: Sure for sure and other applications in high-dimensional inference
Pierre C Bellec and Cun-Hui Zhang. Second-order stein: Sure for sure and other applications in high-dimensional inference. The Annals of Statistics, 49 0 (4): 0 1864--1903, 2021
work page 1903
-
[5]
De-biasing the lasso with degrees-of-freedom adjustment
Pierre C Bellec and Cun-Hui Zhang. De-biasing the lasso with degrees-of-freedom adjustment. Bernoulli, 28 0 (2): 0 713--743, 2022
work page 2022
-
[6]
Debiasing convex regularized estimators and interval estimation in linear models
Pierre C Bellec and Cun-Hui Zhang. Debiasing convex regularized estimators and interval estimation in linear models. The Annals of Statistics, 51 0 (2): 0 391--436, 2023
work page 2023
-
[7]
Controlling the false discovery rate: a practical and powerful approach to multiple testing
Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 57 0 (1): 0 289--300, 1995
work page 1995
-
[8]
Proximity of probability distributions in terms of fourier--stieltjes transforms
Sergei Germanovich Bobkov. Proximity of probability distributions in terms of fourier--stieltjes transforms. Russian Mathematical Surveys, 71 0 (6): 0 1021, 2016
work page 2016
-
[9]
Concentration of empirical distribution functions with applications to non-iid models
SG Bobkov and F G \"o tze. Concentration of empirical distribution functions with applications to non-iid models. Bernoulli, 16 0 (4): 0 1385--1414, 2010
work page 2010
-
[10]
Covering the sphere by equal spherical balls
K \'a roly B \"o r \"o czky and Gergely Wintsche. Covering the sphere by equal spherical balls. Discrete and Computational Geometry: The Goodman-Pollack Festschrift, pages 235--251, 2003
work page 2003
-
[11]
S Boucheron, G Lugosi, and P Massart. Inequalities. a nonasymptotic theory of independence, 2013
work page 2013
-
[12]
Algorithmic stability and generalization performance
Olivier Bousquet and Andr \'e Elisseeff. Algorithmic stability and generalization performance. Advances in neural information processing systems, 13, 2000
work page 2000
-
[13]
Statistical significance in high-dimensional linear models
Peter B \"u hlmann. Statistical significance in high-dimensional linear models. Bernoulli, pages 1212--1242, 2013
work page 2013
-
[14]
High-dimensional statistics with a view toward applications in biology
Peter B \"u hlmann, Markus Kalisch, and Lukas Meier. High-dimensional statistics with a view toward applications in biology. Annual Review of Statistics and Its Application, 1: 0 255--278, 2014
work page 2014
-
[15]
Panning for gold: model-x knockoffs for high dimensional controlled variable selection
Emmanuel Cand\'es, Yingying Fan, Lucas Janson, and Jinchi Lv. Panning for gold: model-x knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society Series B: Statistical Methodology, 80 0 (3): 0 551--577, 2018
work page 2018
-
[16]
The lasso with general gaussian designs with applications to hypothesis testing
Michael Celentano, Andrea Montanari, and Yuting Wei. The lasso with general gaussian designs with applications to hypothesis testing. The Annals of Statistics, 51 0 (5): 0 2194--2220, 2023
work page 2023
-
[17]
Yuxin Chen, Yuejie Chi, Jianqing Fan, Cong Ma, and Yuling Yan. Noisy matrix completion: Understanding statistical guarantees for convex relaxation via nonconvex optimization. SIAM journal on optimization, 30 0 (4): 0 3098--3121, 2020
work page 2020
-
[18]
Double/debiased/neyman machine learning of treatment effects
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, and Whitney Newey. Double/debiased/neyman machine learning of treatment effects. American Economic Review, 107 0 (5): 0 261--265, 2017
work page 2017
-
[19]
Double/debiased machine learning for treatment and structural parameters
Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21 0 (1): 0 C1--C68, 2018
work page 2018
-
[20]
High dimensional robust m-estimation: Asymptotic variance via approximate message passing
David Donoho and Andrea Montanari. High dimensional robust m-estimation: Asymptotic variance via approximate message passing. Probability Theory and Related Fields, 166: 0 935--969, 2016
work page 2016
-
[21]
Noureddine El Karoui. On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators. Probability Theory and Related Fields, 170: 0 95--175, 2018
work page 2018
-
[22]
On robust regression with high-dimensional predictors
Noureddine El Karoui, Daniel Bean, Peter J Bickel, Chuang Lim, and Bin Yu. On robust regression with high-dimensional predictors. Proceedings of the National Academy of Sciences, 110 0 (36): 0 14557--14562, 2013
work page 2013
-
[23]
Ipad: stable interpretable forecasting with knockoffs inference
Yingying Fan, Jinchi Lv, Mahrad Sharifvaghefi, and Yoshimasa Uematsu. Ipad: stable interpretable forecasting with knockoffs inference. Journal of the American Statistical Association, 115 0 (532): 0 1822--1834, 2020
work page 2020
-
[24]
Approximate message passing algorithms for rotationally invariant matrices
Zhou Fan. Approximate message passing algorithms for rotationally invariant matrices. The Annals of Statistics, 50 0 (1): 0 197--224, 2022
work page 2022
-
[25]
One-at-a-time knockoffs: controlled false discovery rate with higher power
Charlie K Guan, Zhimei Ren, and Daniel W Apley. One-at-a-time knockoffs: controlled false discovery rate with higher power. arXiv preprint arXiv:2502.18750, 2025
-
[26]
Universality of regularized regression estimators in high dimensions
Qiyang Han and Yandi Shen. Universality of regularized regression estimators in high dimensions. The Annals of Statistics, 51 0 (4): 0 1799--1823, 2023
work page 2023
-
[27]
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, 2 edition, 2009
work page 2009
-
[28]
Controlling the false discoveries in lasso
Hanwen Huang. Controlling the false discoveries in lasso. Biometrics, 73 0 (4): 0 1102--1110, 2017
work page 2017
-
[29]
A flexible framework for hypothesis testing in high dimensions
Adel Javanmard and Jason D Lee. A flexible framework for hypothesis testing in high dimensions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82 0 (3): 0 685--718, 2020
work page 2020
-
[30]
Confidence intervals and hypothesis testing for high-dimensional regression
Adel Javanmard and Andrea Montanari. Confidence intervals and hypothesis testing for high-dimensional regression. The Journal of Machine Learning Research, 15 0 (1): 0 2869--2909, 2014 a
work page 2014
-
[31]
Adel Javanmard and Andrea Montanari. Hypothesis testing in high-dimensional regression under the gaussian random design model: Asymptotic theory. IEEE Transactions on Information Theory, 60 0 (10): 0 6522--6554, 2014 b
work page 2014
-
[32]
Debiasing the lasso: Optimal sample size for gaussian designs
Adel Javanmard and Andrea Montanari. Debiasing the lasso: Optimal sample size for gaussian designs. The Annals of Statistics, 46 0 (6A): 0 2593--2622, 2018
work page 2018
-
[33]
Power of knockoff: The impact of ranking algorithm, augmented design, and symmetric statistic
Zheng Tracy Ke, Jun S Liu, and Yucong Ma. Power of knockoff: The impact of ranking algorithm, augmented design, and symmetric statistic. Journal of Machine Learning Research, 25 0 (3): 0 1--67, 2024
work page 2024
-
[34]
Black-box tests for algorithmic stability
Byol Kim and Rina Foygel Barber. Black-box tests for algorithmic stability. Information and Inference: A Journal of the IMA, 12 0 (4): 0 2690--2719, 2023
work page 2023
-
[35]
Bounding the smallest singular value of a random matrix without concentration
Vladimir Koltchinskii and Shahar Mendelson. Bounding the smallest singular value of a random matrix without concentration. International Mathematics Research Notices, 2015 0 (23): 0 12991--13008, 2015
work page 2015
-
[36]
Asymptotics for high dimensional regression m-estimates: fixed design results
Lihua Lei, Peter J Bickel, and Noureddine El Karoui. Asymptotics for high dimensional regression m-estimates: fixed design results. Probability Theory and Related Fields, 172: 0 983--1079, 2018
work page 2018
-
[37]
Gen Li and Yuting Wei. A non-asymptotic distributional theory of approximate message passing for sparse and robust regression. arXiv preprint arXiv:2401.03923, 2024
-
[38]
Causal and Selective Inference in Complex Statistical Models
Shuangning Li. Causal and Selective Inference in Complex Statistical Models. Department of Statistics, Stanford University, 2022
work page 2022
-
[39]
Yufan Li and Pragya Sur. Spectrum-aware adjustment: A new debiasing framework with applications to principal components regression. arXiv preprint arXiv:2309.07810, 2023
-
[40]
Random linear estimation with rotationally-invariant designs: Asymptotics at high temperature
Yufan Li, Zhou Fan, Subhabrata Sen, and Yihong Wu. Random linear estimation with rotationally-invariant designs: Asymptotics at high temperature. IEEE Transactions on Information Theory, 2023
work page 2023
-
[41]
From soft-minoration to information-constrained optimal transport and spiked tensor models
Jingbo Liu. From soft-minoration to information-constrained optimal transport and spiked tensor models. In 2023 IEEE International Symposium on Information Theory (ISIT), pages 666--671. IEEE, 2023
work page 2023
-
[42]
Power analysis of knockoff filters for correlated designs
Jingbo Liu and Philippe Rigollet. Power analysis of knockoff filters for correlated designs. Advances in Neural Information Processing Systems, 32, 2019
work page 2019
-
[43]
Second-order converses via reverse hypercontractivity
Jingbo Liu, Ramon Van Handel, and Sergio Verd \'u . Second-order converses via reverse hypercontractivity. Mathematical Statistics and Learning, 2 0 (2): 0 103--163, 2020
work page 2020
-
[44]
Fast and powerful conditional randomization testing via distillation
Molei Liu, Eugene Katsevich, Lucas Janson, and Aaditya Ramdas. Fast and powerful conditional randomization testing via distillation. Biometrika, 109 0 (2): 0 277--293, 2022
work page 2022
-
[45]
Cong Ma, Kaizheng Wang, Yuejie Chi, and Yuxin Chen. Implicit regularization in nonconvex statistical estimation: Gradient descent converges linearly for phase retrieval and matrix completion. In International Conference on Machine Learning, pages 3345--3354. PMLR, 2018
work page 2018
-
[46]
The distribution of the lasso: Uniform control over sparse balls and adaptive parameter tuning
L \'e o Miolane and Andrea Montanari. The distribution of the lasso: Uniform control over sparse balls and adaptive parameter tuning. The Annals of Statistics, 49 0 (4): 0 2313--2335, 2021
work page 2021
-
[47]
Universality of empirical risk minimization
Andrea Montanari and Basil N Saeed. Universality of empirical risk minimization. In Conference on Learning Theory, pages 4310--4312. PMLR, 2022
work page 2022
-
[48]
Vector approximate message passing for the generalized linear model
Philip Schniter, Sundeep Rangan, and Alyson K Fletcher. Vector approximate message passing for the generalized linear model. In 2016 50th Asilomar conference on signals, systems and computers, pages 1525--1529, 2016
work page 2016
-
[49]
The holdout randomization test for feature selection in black box models
Wesley Tansey, Victor Veitch, Haoran Zhang, Raul Rabadan, and David M Blei. The holdout randomization test for feature selection in black box models. Journal of Computational and Graphical Statistics, 31 0 (1): 0 151--162, 2022
work page 2022
-
[50]
Regularized linear regression: A precise analysis of the estimation error
Christos Thrampoulidis, Samet Oymak, and Babak Hassibi. Regularized linear regression: A precise analysis of the estimation error. In Conference on Learning Theory, pages 1683--1709. PMLR, 2015
work page 2015
-
[51]
On asymptotically optimal confidence regions and tests for high-dimensional models
Sara van de Geer, Peter B \"u hlmann, Yaacov Ritov, and Ruben Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42 0 (3): 0 1166--1202, 2014
work page 2014
-
[52]
Ramon Van Handel. Probability in high dimension. Lecture Notes (Princeton University), 2014
work page 2014
-
[53]
Estimation in rotationally invariant generalized linear models via approximate message passing
Ramji Venkataramanan, Kevin K \"o gler, and Marco Mondelli. Estimation in rotationally invariant generalized linear models via approximate message passing. In International Conference on Machine Learning, pages 22120--22144. PMLR, 2022
work page 2022
-
[54]
High-dimensional probability, 2009
Roman Vershynin. High-dimensional probability, 2009
work page 2009
-
[55]
High-dimensional probability: An introduction with applications in data science, volume 47
Roman Vershynin. High-dimensional probability: An introduction with applications in data science, volume 47. Cambridge university press, 2018
work page 2018
-
[56]
A Power and Prediction Analysis for Knockoffs with Lasso Statistics
Asaf Weinstein, Rina Barber, and Emmanuel Candes. A power and prediction analysis for knockoffs with lasso statistics. arXiv preprint arXiv:1712.06465, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[57]
Controlling false discovery rate using gaussian mirrors
Xin Xing, Zhigen Zhao, and Jun S Liu. Controlling false discovery rate using gaussian mirrors. Journal of the American Statistical Association, 118 0 (541): 0 222--241, 2023
work page 2023
-
[58]
Confidence intervals for low dimensional parameters in high dimensional linear models
Cun-Hui Zhang and Stephanie S Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76 0 (1): 0 217--242, 2014
work page 2014
-
[59]
Xinyi Zhong, Tianhao Wang, and Zhou Fan. Approximate message passing for orthogonally invariant ensembles: Multivariate non-linearities and spectral initialization. arXiv preprint arXiv:2110.02318, 2021
-
[60]
Post-selection inference via algorithmic stability
Tijana Zrnic and Michael I Jordan. Post-selection inference via algorithmic stability. The Annals of Statistics, 51 0 (4): 0 1666--1691, 2023
work page 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.