Mean and variance estimation in high-dimensional heteroscedastic models with non-convex penalties

Despite its prevalence in statistical datasets, heteroscedasticity (non-constant sample variances) has been largely ignored in the high-dimensional statistics literature. Recently, studies have shown that the Lasso can accommodate heteroscedastic errors, with minor algorithmic modifications (Belloni et al., 2012; Gautier and Tsybakov, 2013). In this work, we study heteroscedastic regression with linear mean model and log-linear variances model with sparse high-dimensional parameters. In this work, we propose estimating variances in a post-Lasso fashion, which is followed by weighted-least squares mean estimation. These steps employ non-convex penalties as in Fan and Li (2001), which allows us to prove oracle properties for both post-Lasso variance and mean parameter estimates. We reinforce our theoretical findings with experiments.