On the efficiency of the de-biased Lasso

We consider the high-dimensional linear regression model $Y = X \beta^0 + \epsilon$ with Gaussian noise $\epsilon$ and Gaussian random design $X$. We assume that $\Sigma:= E X^T X / n$ is non-singular and write its inverse as $\Theta := \Sigma^{-1}$. The parameter of interest is the first component $\beta_1^0$ of $\beta^0$. We show that in the high-dimensional case the asymptotic variance of a debiased Lasso estimator can be smaller than $\Theta_{1,1}$. For some special such cases we establish asymptotic efficiency. The conditions include $\beta^0$ being sparse and the first column $\Theta_1$ of $\Theta$ being not sparse. These conditions depend on whether $\Sigma$ is known or not.