REVIEW 2 major objections 5 minor 37 references
Debiasing the Lasso under Weaker Tail Assumptions
T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper proves that standardizing the design matrix before debiasing the lasso gives asymptotically valid single-coordinate inference under finite moments, dependent and heteroskedastic errors, and, for one method, mild model…
desk verdict A serious theoretical extension of debiased lasso inference via standardization, but the advertised 'small price' is heavy and the t5 simulations lie outside the theorem's assumptions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the standardized design matrix $\tilde X$ with entries $(X_{ij}-\hat\mu_j)/\hat\sigma_j$, whose columns are exactly unit-norm in $\ell^2$; this turns the debiasing procedure into a self-normalized problem, so concentration inequalities that need only finite moments can control the relevant quadratic forms and linear scores. The argument runs through the decomposition $\sqrt{n}(\hat\beta^u_j - \hat\sigma_j \beta^0_j) = Z_j + \Delta_j$, where $Z_j$ is a score term that is shown to be asymptotically normal and $\Delta_j$ is a bias remainder shown to vanish in probability. The surrogate direction $\theta^{(j)}$ solving $\bar\Sigma\theta^{(j)} = e_j$ is the population target that the debiasing rows approximate, and a restricted eigenvalue condition on $\bar\Sigma$ supplies the lasso consistency that makes $\Delta_j$ small.
What would settle it
Take $n=10^4$, $p=2000$, independent standardized coordinates with a distribution that has finite $4.5$ moments but infinite fifth moments, choose $\theta^{(j)}=e_j$, and compute the empirical probability in Lemma 5 that $\max_k |(\sigma_k/\hat\nu_k)((1/n)\sum_i \bar X_{ik}\langle \bar X_i,\theta^{(j)}\rangle - \delta_{jk})| \ge h_0^{1/r}\sqrt{12a\log p}/n^{1/2-1/r}$. If the empirical frequency does not decay as fast as the lemma's bound as $n$ grows, the proof that $\theta^{(j)}$ is feasible for the debiasing problem collapses, and with it the claimed normal limit.
Extended reading notes
Core claim
The paper's central claim is Theorem 1: after standardization, for either debiasing method, $\sqrt{n}(\hat\beta^u_j/\hat\sigma_j - \beta^0_j)$ converges in distribution to $N(0, \mathbb{E}[\langle \bar X_1, \theta^{(j)}\rangle^2 \epsilon_1^2]/\sigma_j^2)$, where $\theta^{(j)}$ is the population surrogate column satisfying $\bar\Sigma \theta^{(j)} = e_j$ and $\bar\Sigma$ is the population correlation matrix of the standardized design. The convergence is obtained by showing that the bias remainder from the lasso penalty vanishes and that the score term obeys a central limit theorem, with the standardization letting self-normalized concentration inequalities supply the needed control under finite moments. For the second method, the same limit holds with an approximation error $\rho$ present, provided the sparsity of $\theta^{(j)}$ is controlled; the asymptotic variance is the variance of the best linear unbiased estimator of the coordinate, so the efficiency benchmark is retained.
Load-bearing premise
The argument depends on a moment bound, Assumption 4(iii), that controls how heavy the tail of the linear contrast $\langle \bar X_1, \theta^{(j)}\rangle$ can be: its $r'$-th absolute moment must be at most a constant times its variance raised to $r'/2$; finite moments of individual coordinates alone do not guarantee this, and the bound fails for designs as heavy-tailed as the $t_5$ distribution used in the simulations.
Editorial extensions
If this is right
- Confidence intervals and tests for a single coefficient of a high-dimensional linear model remain asymptotically valid after standardizing the design, with no subgaussianity and no independence between errors and covariates.
- Heteroskedastic errors whose conditional variance depends on the covariates are allowed, and for the second method a misspecified linear model with a small approximation error is also allowed.
- The required sparsity is $s = o(n^{1/2 - 1/q - 1/r}/\log p)$, which approaches the classical $o(\sqrt{n}/\log p)$ rate as the assumed moment orders $q,r$ grow.
- The plug-in estimator $\hat V_n = \frac{1}{n}\sum_i \langle \tilde X_i, m^{(j)}\rangle^2 (Y_i - \tilde X_i^\top \hat\beta - \hat\gamma)^2$ gives a consistent estimate of the asymptotic variance, so the confidence interval in Corollary 1 is directly usable.
Reading between the lines
- The standardization step may be doing double duty: beyond putting columns on a common scale, it replaces unknown population variances by data-dependent normalizers, which can shrink the effective tail weight of linear scores; this suggests the method may be more forgiving in practice than Assumption 4(iii) suggests, but the proof as written relies on that bound.
- The finite-moment sparsity trade-off implies a practical diagnostic: estimate the moment order of the standardized coordinates and the $\ell^1$-mass of the surrogate direction; where the implied sparsity rate cannot be met, the method should be expected to lose coverage.
- The same self-normalized concentration strategy could be exported to debiased estimation in other penalized $M$-estimators or to confidence intervals for linear functionals beyond single coordinates, as long as the analogous surrogate direction satisfies a hypercontractivity-type bound.
- One testable extension is to compare the empirical coverage of the interval in Corollary 1 for the $t_5$ design used in the paper's simulations against designs with only slightly more than four finite moments; if coverage holds broadly, the assumptions could likely be weakened further.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies high-dimensional linear regression with the lasso and two debiasing constructions. Working with the column-standardized design matrix \tilde X, it states (Theorem 1) that for a fixed coordinate j, under Assumptions 1\u20135, the debiased estimator satisfies sqrt(n)(\hat \beta^u_j / \hat \sigma_j \u2212 \beta^0_j) converging in distribution to N(0, E[\langle \bar X_1, \theta^{(j)} \rangle^2 \epsilon_1^2] / \sigma_j^2), for both a Javanmard\u2013Montanari-type method (Method 1) and a van de Geer-type method (Method 2). The proof decomposes the debiasing error into a lasso-bias term, approximation-error terms for Method 2, and a self-normalized linear score, using concentration results from the authors' earlier work. Corollary 1 gives a plug-in confidence interval based on an estimated variance.
Significance. If the result is correct, it is a useful step: it replaces sub-gaussian/bounded design assumptions with explicit finite-moment and self-normalization conditions, and it gives a concrete variance estimator. The asymptotic argument is detailed, and the central decomposition (8) is standard; I found no circularity in the asymptotic statement. The main weaknesses are quantitative: the sparsity rate (15) is very restrictive for any fixed moment order, and the heavy-tailed simulations in Section 6 do not satisfy the theorem's assumptions. These issues do not disprove the theorem, but they materially narrow the scope claimed in the abstract.
major comments (2)
- [Section 4, Eq. (15); Section 6] Assumption 3, as instantiated, is not compatible with the reported experiments. With q=r=4.9, the exponent in Eq. (15) is 1/2 \u2212 1/q \u2212 1/r = 0.0918, so the required rate is s = o(n^{0.0918}/log p); at n=600 this is about o(0.26), so even the Gaussian configuration with s=10 in Table 1 violates the assumption. The text's characterization of this sparsity requirement as 'slightly stronger' and 'close to' the usual sqrt(n)/log p rate is therefore not accurate for any fixed finite q and r; the price is a polynomial loss of n^{-1/q-1/r}. The authors should restate the sparsity claim quantitatively and either change the simulation parameters or present Table 1 as a heuristic robustness check outside the theorem's scope.
- [Section 4, Assumption 4(iv); Section 6, configurations B and C] Assumption 4(iv) requires max_k E|\bar X_{1k}|^{3r'} \le K_2 for some r'>2, which means every standardized covariate must have more than six finite moments. The t-Student(5) entries used in configurations B and C have tail index 5, so this moment is infinite for every admissible r'. Consequently, the heavy-tail rows of Table 1 do not instantiate Theorem 1. They should be explicitly labeled as informal robustness checks, or replaced by distributions satisfying Assumption 4(iv).
minor comments (5)
- [Section 4, paragraph following Eq. (16)] The sentence 'This slightly weaker requirement is ubiquitous in the literature' appears to refer to the authors' own stronger condition (15); reword to avoid confusion.
- [Section 4, discussion of Assumption 4] The discussion should state explicitly that Eq. (17) is a hypercontractivity-type structural condition and is not implied by the coordinate moment bounds in Assumption 2; otherwise readers may overestimate the class of heavy-tailed designs covered.
- [Section 6, opening paragraph] The statement that r=q=4.9 'can clearly be taken much higher in some of them' is true for Gaussian designs but not for the t-Student columns in configurations B and C; clarify which settings permit larger q,r.
- [Table 1 and text following it] Several active-set coverages are below 0.95 (for example, Method 1, configuration D, (600,1000,10,0.5), has bCS=0.922), so the claim that both methods 'approximately achieve or surpass' the nominal level should be qualified.
- [Throughout] There are minor typographical errors, including 'Notebly' in Section 1, 'Thesecondmethod' in the abstract, and 'vanished' used instead of 'vanishes' in the proof of Lemma 6; these should be cleaned up.
Circularity Check
No significant circularity: Theorem 1 is derived from explicit assumptions, and the self-cited technical tools are independent published results.
full rationale
The central asymptotic statement (Theorem 1) is not obtained by fitting a parameter to data and then predicting that same data; it is a limit theorem derived from Assumptions 1 through 5 via the decomposition in equation (8) and the five-step proof in Appendix A. The limit variance E[<Xbar_1, theta^(j)>^2 epsilon_1^2]/sigma_j^2 is computed from the population quantity theta^(j) defined by Sigma_bar theta^(j) = e_j, and the debiasing direction m^(j) is shown to approximate theta^(j) under the stated assumptions; no fitted value enters the asymptotic statement. The paper does rely on Theorem 5.2 of reference [31] in Corollary 3 and on Lemmas 8 and 9 of reference [37] in Lemmas 3 and 5, and one author is common to those works. However, these are cited as published results with their own explicit assumptions (restricted eigenvalue concentration for self-normalized designs and self-normalized concentration inequalities), and those assumptions do not include the debiased-lasso conclusion. Under the standard for this review, such citations are independent evidence and do not constitute circularity. The skeptical concern that the t5 simulations in Section 6 violate Assumption 4(iv), which requires moments of order 3r' > 6, and that Assumption 3's sparsity rate is weaker than the usual sqrt(n)/log p rate, is a correctness and scope issue, not a circularity issue; it does not show that the theorem reduces to its inputs. No step in the paper defines a target quantity in terms of itself, fits a parameter to a subset and calls the output a prediction, or imports a uniqueness theorem to force a choice. Hence the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Simulation tuning constants (c0, c', C, c2) =
c0=0.3, c'=2, C=c2=0.1 (Section 6)
assumptions (8)
- domain assumption Observations are i.i.d. with E[epsilon_i|X_i]=0, bounded conditional variance, and bounded q'-th moments (Assumption 1(i)-(iii)).
- domain assumption For Method 2, the approximation error rho satisfies E[rho_1]=0, E[rho_1 X_1k]=0, and E[rho_i^2] <= sigma_rho^2 s_theta/n (Assumption 1(iv)).
- domain assumption Finite 2g-th moments for standardized design coordinates and fourth-moment hypercontractivity E[(v^T bar X)^4] <= h v^T barSigma v for ||v||_0 <= n (Assumption 2, equations (12)-(13)).
- domain assumption Population restricted eigenvalue re(barSigma, S(beta0), w) >= k (Assumption 2, equation (14)).
- domain assumption Sparsity s = o(n^{1/2-1/q-1/r}/log p) (Assumption 3, equation (15)).
- domain assumption Existence of theta^(j) solving barSigma theta^(j)=e_j, with moment conditions including E[|<bar X_1,theta^(j)>|^{r'}] <= h0 (theta^(j)_j)^{r'/2} (Assumption 4, equation (17)).
- domain assumption theta^(j)_j bounded and l1/l0 sparsity of theta^(j) at the stated rates (Assumption 5, equations (19)-(21)).
- standard math Self-normalized concentration lemmas from Oliveira and Thompson [37] and Theorem 5.2 of Oliveira [31].
Cite this review
Pith. "Pith review of Debiasing the Lasso under Weaker Tail Assumptions." pith.science (2026). https://pith.science/paper/2ZTYLIX4
@misc{pith2026260804800,
author = {Pith},
title = {Pith review of: Debiasing the Lasso under Weaker Tail Assumptions},
year = {2026},
howpublished = {\url{https://pith.science/paper/2ZTYLIX4}},
note = {Machine review of arXiv:2608.04800}
}
read the original abstract
We consider the problem of high-dimensional inference with the lasso estimator. Different methods including 'double selection' techniques and multiple versions of the 'debiased lasso' have been proposed for this task with noticeable success. However, most guarantees assume strong hypotheses on the underlying data process and the errors in the linear regression model, such as subgaussian designs and independence between errors and the data itself. We show that 'standardizing' one's dataset -- a natural procedure in practical penalized regression -- leads to same results under much weaker hypotheses, paying only a small price for not assuming light tails. The key technical point allowed by this step is exploiting the concentration properties of self-normalized processes. Importantly, we prove our results for two different methods closely related to the 'debiased lasso'. The second method performs valid inference even for a misspecified linear model, under mild sparsity conditions similar to the 'double selection' literature.
Reference graph
Works this paper leans on
-
[1]
Jie Peng, Ji Zhu, Anna Bergamaschi, Wonshik Han, Dong-Young Noh, Jonathan R Pollack, and Pei Wang. Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer.Ann Appl Stat, 4(1):53–77, March 2010
work page 2010
-
[2]
High-dimensional statistics, with applications to genome-wide association studies.EMS Surv
Peter Bühlmann. High-dimensional statistics, with applications to genome-wide association studies.EMS Surv. Math. Sci., 4(1):45–75, 2017
work page 2017
-
[3]
KillianACMelsen,JonathanFKunst,JoséCrossa,MargaretRKrause,FredAvanEeuwijk,WillemKruijer,and Carel F W Peeters. Improving genomic prediction using High-Dimensional secondary phenotypes: The genetic latent factor approach.Biom J, 67(5):e70081, October 2025
work page 2025
-
[4]
Sparse discriminant analysis.Technomet- rics, 53(4):406–413, 2011
Line Clemmensen, Trevor Hastie, Daniela Witten, and Bjarne Ersbøll. Sparse discriminant analysis.Technomet- rics, 53(4):406–413, 2011
work page 2011
-
[5]
Variableselectionusingrandomforests.Pattern Recognition Letters, 31(14):2225–2236, 2010
RobinGenuer, Jean-MichelPoggi, andChristineTuleau-Malot. Variableselectionusingrandomforests.Pattern Recognition Letters, 31(14):2225–2236, 2010
work page 2010
-
[6]
Sixia Chen and Chao Xu. Handling high-dimensional data with missing values by modern machine learning techniques.J Appl Stat, 50(3):786–804, May 2022
work page 2022
-
[7]
Jianqing Fan, Kaizheng Wang, Yiqiao Zhong, and Ziwei Zhu. Robust high dimensional factor models with applications to statistical machine learning.Stat Sci, 36(2):303–327, April 2021
work page 2021
-
[8]
Alexandre Belloni, Victor Chernozhukov, and Christian Hansen. Inference on treatment effects after selection among high-dimensional controls†.The Review of Economic Studies, 81(2):608–650, 04 2014
work page 2014
Show all 37 references
-
[9]
Sparsemodelsandmethodsforoptimalinstrumentswith an application to eminent domain.Econometrica, 80(6):2369–2429, 2012
A.Belloni,D.Chen,V.Chernozhukov,andC.Hansen. Sparsemodelsandmethodsforoptimalinstrumentswith an application to eminent domain.Econometrica, 80(6):2369–2429, 2012
2012
-
[10]
High-dimensional methods and inference on structural and treatment effects.Journal of Economic Perspectives, 28(2):29–50, May 2014
Alexandre Belloni, Victor Chernozhukov, and Christian Hansen. High-dimensional methods and inference on structural and treatment effects.Journal of Economic Perspectives, 28(2):29–50, May 2014
2014
-
[11]
Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 12 2018
Robert Tibshirani. Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 12 2018
2018
-
[12]
Cun-HuiZhangandStephanieS.Zhang.Confidenceintervalsforlowdimensionalparametersinhighdimensional linearmodels.JournaloftheRoyalStatisticalSocietySeriesB:StatisticalMethodology,76(1):217–242,072013
-
[13]
On asymptotically optimal confidence regions and tests for high-dimensional models.The Annals of Statistics, 42(3):1166–1202, 2014
Sara van de Geer, Peter Bühlmann, Ya’acov Ritov, and Ruben Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models.The Annals of Statistics, 42(3):1166–1202, 2014
2014
-
[14]
Confidence intervals and hypothesis testing for high-dimensional regression.Journal of Machine Learning Research, 15(82):2869–2909, 2014
Adel Javanmard and Andrea Montanari. Confidence intervals and hypothesis testing for high-dimensional regression.Journal of Machine Learning Research, 15(82):2869–2909, 2014. 35 Debiasing the Lasso under Weaker Tail Assumptions
2014
-
[15]
Debiasing the debiased Lasso with bootstrap.Electronic Journal of Statistics, 14(1):2298 – 2337, 2020
Sai Li. Debiasing the debiased Lasso with bootstrap.Electronic Journal of Statistics, 14(1):2298 – 2337, 2020
2020
-
[16]
Bellec and Cun-Hui Zhang
Pierre C. Bellec and Cun-Hui Zhang. De-biasing the lasso with degrees-of-freedom adjustment.Bernoulli, 28(2):713 – 743, 2022
2022
-
[17]
Debiasing the lasso: Optimal sample size for Gaussian designs.The Annals of Statistics, 46(6A):2593 – 2622, 2018
Adel Javanmard and Andrea Montanari. Debiasing the lasso: Optimal sample size for Gaussian designs.The Annals of Statistics, 46(6A):2593 – 2622, 2018
2018
-
[18]
High-dimensionalinferenceforgeneralizedlinearmodelswith hidden confounding.J
JingOuyang,KeaneMingTan,andGongjunXu. High-dimensionalinferenceforgeneralizedlinearmodelswith hidden confounding.J. Mach. Learn. Res., 24(1), January 2023
2023
-
[19]
Debiasedlassoforgeneralizedlinearmodelswithadivergingnumberofcovariates
LuXia, BinNan, andYiLi. Debiasedlassoforgeneralizedlinearmodelswithadivergingnumberofcovariates. Biometrics, 79(1):344–357, 10 2021
2021
-
[20]
Confidence intervals for high-dimensional inverse covariance estimation
Jana Janková and Sara van de Geer. Confidence intervals for high-dimensional inverse covariance estimation. Electronic Journal of Statistics, 9(1):1205 – 1229, 2015
2015
-
[21]
Statistical inference on high-dimensional covariate-dependent gaussian graphical regressions.Biometrics, 81(4), October 2025
Xuran Meng, Jingfei Zhang, and Yi Li. Statistical inference on high-dimensional covariate-dependent gaussian graphical regressions.Biometrics, 81(4), October 2025
2025
-
[22]
Ageneraltheoryofhypothesistestsandconfidenceregionsforsparsehighdimensional models.The Annals of Statistics, 45(1):158 – 195, 2017
YangNingandHanLiu. Ageneraltheoryofhypothesistestsandconfidenceregionsforsparsehighdimensional models.The Annals of Statistics, 45(1):158 – 195, 2017
2017
-
[23]
Probability and Its Applications (New York)
Victor H Pena, T L Lai, and Qi-Man Shao.Self-normalized processes. Probability and Its Applications (New York). Springer, Berlin, Germany, December 2009
2009
-
[24]
Oxford University Press, 02 2013
Stéphane Boucheron, Gábor Lugosi, and Pascal Massart.Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 02 2013
2013
-
[25]
D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition.IEEE Trans. Inf. Theor., 47(7):2845–2862, September 2006
2006
-
[26]
Stable recovery of sparse signals and an oracle inequality.IEEE Trans
Tony Tony Cai, Lie Wang, and Guangwu Xu. Stable recovery of sparse signals and an oracle inequality.IEEE Trans. Inf. Theor., 56(7):3516–3522, July 2010
2010
-
[27]
E. J. Candes and T. Tao. Near-optimal signal recovery from random projections: Universal encoding strategies? IEEE Trans. Inf. Theor., 52(12):5406–5425, December 2006
2006
-
[28]
The Dantzig selector: Statistical estimation when p is much larger than n
Emmanuel Candes and Terence Tao. The Dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 35(6):2313 – 2351, 2007
2007
-
[29]
SimultaneousanalysisofLassoandDantzigselector
PeterJ.Bickel,Ya’acovRitov,andAlexandreB.Tsybakov. SimultaneousanalysisofLassoandDantzigselector. The Annals of Statistics, 37(4):1705 – 1732, 2009
2009
-
[30]
OntheconditionsusedtoproveoracleresultsfortheLasso.Electronic Journal of Statistics, 3(none):1360 – 1392, 2009
SaraA.vandeGeerandPeterBühlmann. OntheconditionsusedtoproveoracleresultsfortheLasso.Electronic Journal of Statistics, 3(none):1360 – 1392, 2009
2009
-
[31]
Thelowertailofrandomquadraticformswithapplicationstoordinaryleastsquares
RobertoImbuzeiroOliveira. Thelowertailofrandomquadraticformswithapplicationstoordinaryleastsquares. Probability Theory and Related Fields, 166(3):1175–1194, December 2016
2016
-
[32]
Tony Cai and Zijian Guo
T. Tony Cai and Zijian Guo. Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity.The Annals of Statistics, 45(2):615 – 646, 2017
2017
-
[33]
A remark on moment-dependent phase transitions in high- dimensional gaussian approximations.Statistics & Probability Letters, 211:110149, 2024
Anders Bredahl Kock and David Preinerstorfer. A remark on moment-dependent phase transitions in high- dimensional gaussian approximations.Statistics & Probability Letters, 211:110149, 2024
2024
-
[34]
Springer Science & Business Media, 2011
Peter Bühlmann and Sara van de Geer.Statistics for high-dimensional data: Methods, theory and applications. Springer Science & Business Media, 2011
2011
-
[35]
Reconstruction from anisotropic random measurements
Mark Rudelson and Shuheng Zhou. Reconstruction from anisotropic random measurements. In Shie Mannor, Nathan Srebro, and Robert C. Williamson, editors,Proceedings of the 25th Annual Conference on Learning Theory, volume 23 ofProceedings of Machine Learning Research, pages 10.1–...
-
[36]
Restricted eigenvalue conditions on subgaussian random matrices.arXiv: Statistics Theory, 2009
Shuheng Zhou. Restricted eigenvalue conditions on subgaussian random matrices.arXiv: Statistics Theory, 2009
2009
-
[37]
Sample average approximation with heavier tails II: localization in stochasticconvexoptimizationandpersistenceresultsforthelasso.Math.Program.,199(1-2):49–86,May2023
Roberto I Oliveira and Philip Thompson. Sample average approximation with heavier tails II: localization in stochasticconvexoptimizationandpersistenceresultsforthelasso.Math.Program.,199(1-2):49–86,May2023. 36
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.