REVIEW 5 major objections 5 minor 18 references
Heteroscedastic Double Bayesian Elastic Net
T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read HDBEN jointly models the mean and log-variance with Bayesian elastic-net priors, claiming posterior concentration, exact support recovery, and asymptotic normality for both coefficient vectors under mild conditions.
desk verdict A promising model idea undermined by unsound theory: the paper's central variable-selection theorem contradicts its own continuous prior, and the contraction results are assumed rather than proved. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the hierarchical Bayesian elastic-net prior placed on both coefficient vectors: $\beta \mid \tau_\beta$ is normal with covariance $(D_\beta^{-1} + \lambda_{2,\beta} I)^{-1}$, and each $\tau_{\beta,j}$ follows an exponential distribution with rate $\lambda_{1,\beta}^2/2$, with the identical construction for $\gamma$. This scale-mixture-of-normals representation is the Bayesian analogue of the combined $\ell_1+\ell_2$ penalty, giving sparsity through the exponential mixing and stable grouped shrinkage through the ridge-type precision. In HDBEN it is applied twice, once for the mean and once for the log-variance, so that variance coefficients can be selected and shrunk in the same sense as mean coefficients. The posterior sampler exploits conditional conjugacy: $\beta$ has a normal full conditional, $\tau_\beta$ and $\tau_\gamma$ have inverse-Gaussian full conditionals, the shrinkage hyperparameters have Gamma full conditionals, and $\gamma$ is updated with a Metropolis-Hastings step.
What would settle it
Compute the posterior probability $P(\beta_j = 0 \mid y, X)$ in the stated model for any finite $n$ and $d$; it is zero for every $j$ because the posterior density of each coefficient is continuous, directly contradicting the claim that $P(S_\beta = S_{\beta_0}, S_\gamma = S_{\gamma_0} \mid y, X)$ tends to one. A simulation counterpart is to run the described Gibbs sampler on data with sparse true supports and record whether any MCMC draw of $\beta$ or $\gamma$ ever equals zero exactly.
Extended reading notes
Core claim
The paper's central claim is that applying the Bayesian elastic-net hierarchy to the log-variance coefficients $\gamma$, alongside the usual mean coefficients $\beta$, turns the heteroscedastic Gaussian model $y_i \sim N(X_i^T\beta, \exp(X_i^T\gamma))$ into a jointly regularized Bayesian procedure. HDBEN's posterior is claimed to concentrate around the true parameters at a rate of order $\sqrt{(s_\beta+s_\gamma)\log d / n}$, to recover the exact sparsity patterns of both $\beta$ and $\gamma$ with probability tending to one, and to be asymptotically normal on the active coefficients under mild regularity conditions. The same scale-mixture prior that gives the Bayesian elastic net its $\ell_1+\ell_2$ behavior is placed on both parameter blocks, so sparsity and grouping are induced in the variance model as well as the mean model. Simulations with $d$ up to 1000 report lower estimation error for $\beta$ than the compared baselines, particularly where heteroscedasticity is strong.
Load-bearing premise
The load-bearing premise is that a continuous Bayesian elastic-net prior can produce exact zeros in the posterior, yet under the paper's normal-exponential hierarchy every coefficient has a continuous posterior density, so the event of exact support recovery has posterior probability zero.
Editorial extensions
If this is right
- Users obtain one posterior that estimates both mean effects and variance effects, so prediction intervals can widen automatically where the predictors indicate higher volatility.
- Under the stated theory, the squared-error risk of the posterior means decays as $O((s_\beta+s_\gamma)\log d / n)$, the usual near-optimal high-dimensional rate.
- If exact support recovery holds, the fitted model tells the practitioner which covariates shift the response and which covariates change its variability, with no separate selection step.
- Asymptotic normality of the active coefficients would justify reporting posterior-based confidence intervals for the selected mean and variance parameters.
Reading between the lines
- Extension: because the normal-exponential hierarchy gives every coefficient a continuous posterior density, the exact-support consistency claim in Theorem 2 would need a spike-and-slab prior or a post-hoc thresholding rule to be literally true.
- Extension: the random-walk Metropolis update for the $d$-dimensional vector $\gamma$ is likely to mix more slowly as $d$ grows; a testable remedy is a reparameterized or adaptive sampler, and effective sample sizes should be compared across dimensions.
- Extension: the same dual elastic-net construction transfers to heteroscedastic GLMs where both location and dispersion depend on predictors, since the variance-coefficient prior does not rely on Gaussian responses.
- Extension: a finite-sample check of support recovery against thresholded HDBEN posteriors and a spike-and-slab competitor would show how much of the claimed selection consistency is visible in practice.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a Bayesian regression model that jointly models the conditional mean and log-variance via hierarchical elastic-net priors. The authors claim posterior concentration, variable selection consistency, and asymptotic normality of the posterior in high-dimensional settings, and they support these claims with simulation studies comparing HDBEN to OLS, Lasso, Elastic Net, Bayesian Lasso, and Bayesian Elastic Net. The manuscript contains four theorems: Theorem 1 asserts posterior concentration, Corollary 1 asserts a near-optimal contraction rate for posterior means, Theorem 2 asserts variable selection consistency, and Theorems 3-4 assert asymptotic normality and a higher-order expansion.
Significance. If the theoretical results were valid, the paper would offer a useful unified Bayesian treatment of heteroscedastic regression with simultaneous shrinkage of mean and variance coefficients. The model specification is natural and the simulation section suggests the method can perform well empirically. However, the theoretical core is not established: Theorem 2 is false as stated for the specified continuous prior, Corollary 1 assumes the rate it claims to prove, and Theorem 1's invocation of Schwartz's theorem omits the required testing and entropy conditions. Because the central claims of posterior concentration, selection consistency, and asymptotic normality are load-bearing for the paper's contribution, the significance of the work as presented is severely diminished.
major comments (5)
- [§3.1, Eqs. (3)-(6); §4, Theorem 2] Theorem 2 asserts P(Sβ = Sβ0, Sγ = Sγ0 | y, X) → 1, where Sβ = {j : βj ≠ 0}. Under the specified hierarchical prior, β and γ have continuous Normal-scale-mixture priors with no point mass at zero, and the likelihood is a continuous Gaussian density. Consequently, for every j, P(βj = 0 | y, X) = 0 and P(γj = 0 | y, X) = 0 for every finite n and every observed (y, X). For a sparse true support, the event {Sβ = Sβ0, Sγ = Sγ0} requires some coefficients to be exactly zero, so its posterior probability is identically zero, not converging to 1. The proof's "true negatives" step claims exact zeros arise from the L1 penalty, but the prior shrinks coefficients without producing exact zeros. This is not a technical gap; the theorem is internally inconsistent with the model specification.
- [§4, Corollary 1 and Theorem 2] The contraction rate is assumed rather than derived. Theorem 1 only states that there exists a sequence εn → 0 such that the posterior concentrates in an εn-neighborhood; it provides no rate. Corollary 1 then "sets" εn = sqrt((sβ+sγ) log d / n) and uses that to obtain the risk bound, and Theorem 2 repeats the same choice. This is circular. Moreover, posterior concentration in probability does not imply the unconditional expectation bound in Eq. (16); without control of the low-probability complement, the expected squared error can be large.
- [§4, Theorem 1 and Assumption 4.2] The proof of Theorem 1 invokes Schwartz's theorem after bounding the Kullback-Leibler divergence locally, but it never verifies the two essential hypotheses: the existence of exponentially consistent tests for the complement of the neighborhood, and prior mass conditions on K-L neighborhoods (the latter are only asserted via Assumption 4.4). In the high-dimensional regime d > n, Assumption 4.2 (full column rank of X as n → ∞) cannot hold, so the setting is internally inconsistent. Thus Theorem 1's conclusion is not established.
- [§4, Theorem 3] The Bernstein-von Mises result is not justified. The theorem assumes a LAN condition and smoothness but does not verify them for the heteroscedastic likelihood; more concretely, the displayed Fisher information matrix has zero off-diagonal blocks between the β and γ sub-vectors, but the log-likelihood's cross-derivatives with respect to β and γ are generally nonzero because the variance exp(Xᵢᵀγ) depends on γ. The limiting covariance in Theorem 3 is therefore incorrect for the stated model.
- [§3.2, Eqs. (10)-(11)] The full conditional for τβ is stated to be inverse Gaussian, but with Σβ = (Dβ^{-1} + λ2,β I)^{-1}, the conditional distribution of β_j given τβ has variance τβ,j/(1 + λ2,β τβ,j), not τβ,j. The joint prior is not the standard Bayesian-lasso scale-mixture representation, so the inverse Gaussian full conditional does not follow. This calls into question the correctness of the MCMC sampler used in the simulations.
minor comments (5)
- [§4, Theorem 1 proof] The proof contains a missing citation: "(see, e.g., ?)" should be replaced with a proper reference for the K-L divergence between Gaussian densities.
- [§4, Assumption 4.2] Assumption 4.2 states that the design matrix has full column rank as n → ∞, which contradicts the paper's high-dimensional setting where d can exceed n. If the intended assumption is d ≤ n for the asymptotic theory, this should be stated explicitly and reconciled with the simulation settings.
- [§4, Theorem 4] The theorem heading states "Assumptions 4.1–4.6 and 4.5," which duplicates Assumption 4.5; the intended list should be corrected.
- [§5, Figure 2 caption] The caption refers to "HDEN β1" while the method is called HDBEN throughout; the label should be updated.
- [§5.1, Table 1] Table 1 reports "Heteroscedasticity γ0,j [0.1, 1.5]" but the text later says "γ0 ∈ [10, 100]"; the values are inconsistent and the notation for the true variance coefficients should be clarified.
Circularity Check
Corollary 1 sets the contraction rate to the target rate and Theorem 2 builds on that assumed rate; the claimed support-recovery also relies on an exact-zero property the continuous elastic-net prior does not have.
-
self definitional
[Section 4, Corollary 1 proof, Eq. (16)]
"Given the sparsity levels sβ and sγ, and assuming a sub-Gaussian design matrix X, we set the contraction rate εn to be: εn = sqrt((sβ+sγ) ln d / n). ... Given the contraction rate εn, we have, E[||β̂ − β0||2^2 | X] ≤ C1εn^2, and E[||γ̂ − γ0||2^2 | X] ≤ C2εn^2."
Theorem 1 only asserts that some sequence εn → 0 exists; it does not identify the rate. The corollary's proof chooses εn := sqrt((sβ+sγ) ln d / n) and then asserts the moment bounds hold at that chosen εn. The advertised near-optimal rate in Eq. (16) is therefore an input selected by the authors, not a consequence of prior concentration, metric entropy, or any calculation involving the model. The 'prediction' reduces by construction to the chosen value of εn.
-
other
[Section 4, Theorem 2 proof, first paragraph and 'True Negatives' bullet]
"From Theorem 1, the posterior distribution concentrates around (β0, γ0) at rate εn = sqrt((sβ+sγ) ln d / n). ... True Negatives: For j /∈ Sβ0 ∪ Sγ0, the posterior distributions of βj and γj concentrate around zero due to the Laplace (L1) penalty, leading to exact zeros with high posterior probability."
The proof inherits the rate εn from Corollary 1, which was set rather than derived, so the support-recovery statement rests on an assumed rate. More fundamentally, the 'exact zeros' step is assumed rather than obtained from the model: Eqs. (3)-(6) specify a continuous Normal scale mixture with τβ,j | λ1,β ~ Exp(λ1,β^2/2) and no point mass at zero, so P(βj = 0 | y, X) = 0 and P(γj = 0 | y, X) = 0 for every finite n. For sparse truth, the event Sβ = Sβ0, Sγ = Sγ0 then has posterior probability zero, contradicting the claimed convergence to 1. The proof's conclusion is imported from Lasso intuition rather than derived from the paper's own prior specification.
full rationale
The paper contains no load-bearing self-citations; the theoretical chain is self-contained in that sense. However, the central rate claim is circular: Corollary 1 takes the target rate εn = sqrt((sβ+sγ) ln d / n) as an input, declares it the contraction rate, and then uses it to bound the posterior mean error, so Eq. (16) is equivalent to the chosen εn. Theorem 2 then builds its selection-consistency argument on that same assumed rate. In addition, Theorem 2's proof assumes that the continuous elastic-net prior produces exact zeros with high posterior probability, which is not a property of the hierarchical Normal-exponential prior in Eqs. (3)-(6); under that prior exact support recovery has posterior probability zero for sparse true parameters. This makes the variable-selection-consistency claim not only circular in its rate dependence but also internally inconsistent with the model specification. Because the abstract and Section 6 advertise posterior concentration and variable selection consistency as the main theoretical payoff, the central derivation chain is partially circular: a key 'prediction' (the rate) is assumed by construction, and the selection result inherits that assumption. Score 6 reflects partial circularity rather than a fully self-citation-driven or definitionally identical paper.
Assumptions & free parameters
free parameters (2)
- Prior hyperparameters (aβ1, bβ1, aγ1, bγ1, aβ2, bβ2, aγ2, bγ2) =
not specified
- MCMC proposal variance σγ² =
not specified
assumptions (9)
- domain assumption True data-generating process is linear with log-linear variance: y_i ~ N(x_i^T β0, exp(x_i^T γ0))
- domain assumption Design matrix X has full column rank as n → ∞
- domain assumption Bounded covariates and variance bounded away from zero: ||X_i|| ≤ M and exp(x_i^T γ) ≥ c
- domain assumption True parameters lie in prior support
- domain assumption Restricted eigenvalue condition of order s with constant κ
- domain assumption Hyperparameters aβ1, aγ1 > 1 and bβ1, bγ1 ≍ 1
- ad hoc to paper Schwartz's theorem conditions (testing and entropy bounds) hold in the high-dimensional regime
- ad hoc to paper Local asymptotic normality (LAN) and high-order smoothness of the likelihood
- ad hoc to paper Continuous elastic net prior can produce exact zero posterior values
Cite this review
Pith. "Pith review of Heteroscedastic Double Bayesian Elastic Net." pith.science (2026). https://pith.science/paper/6K6MYQLQ
@misc{pith2026250202032,
author = {Pith},
title = {Pith review of: Heteroscedastic Double Bayesian Elastic Net},
year = {2026},
howpublished = {\url{https://pith.science/paper/6K6MYQLQ}},
note = {Machine review of arXiv:2502.02032}
}
abstract
In many practical applications, regression models are employed to uncover relationships between predictors and a response variable, yet the common assumption of constant error variance is frequently violated. This issue is further compounded in high-dimensional settings where the number of predictors exceeds the sample size, necessitating regularization for effective estimation and variable selection. To address this problem, we propose the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a novel framework that jointly models the mean and log-variance using hierarchical Bayesian priors incorporating both $\ell_1$ and $\ell_2$ penalties. Our approach simultaneously induces sparsity and grouping in the regression coefficients and variance parameters, capturing complex variance structures in the data. Theoretical results demonstrate that proposed HDBEN achieves posterior concentration, variable selection consistency, and asymptotic normality under mild conditions which justifying its behavior. Simulation studies further illustrate that HDBEN outperforms existing methods, particularly in scenarios characterized by heteroscedasticity and high dimensionality.
Figures
Reference graph
Works this paper leans on
-
[1]
Arellano, M. (1987). Computing robust standard errors for w ithin-groups estimators. Oxford Bulletin of Economics & Statistics , 49(4)
work page 1987
-
[2]
Brooks, S. (1998). Markov chain monte carlo method and its ap plication. Journal of the royal statistical society: series D (the Statistician) , 47(1):69–100
work page 1998
-
[3]
Chib, S. and Greenberg, E. (1995). Understanding the metrop olis-hastings algorithm. The american statistician , 49(4):327–335
work page 1995
-
[4]
Croux, C., Dhaene, G., and Hoorelbeke, D. (2004). Robust sta ndard errors for robust estimators. CES-Discussion paper series (DPS) 03.16 , pages 1–20
work page 2004
-
[5]
Geyer, C. J. (1992). Practical markov chain monte carlo. Statistical science , pages 473–483
work page 1992
-
[6]
Geyer, C. J. (2011). Introduction to markov chain monte carl o. Handbook of markov chain monte carlo, 20116022(45):22
work page 2011
-
[7]
Hans, C. (2009). Bayesian lasso regression. Biometrika, 96(4):835–845
work page 2009
-
[8]
Koenker, R. (1981). A note on studentizing a test for heteros cedasticity. Journal of econometrics, 17(1):107–112
work page 1981
Show all 18 references
-
[9]
and Lin, N
Li, Q. and Lin, N. (2010). The bayesian elastic net. Bayesian Analysis, 5(1):151–170
2010
-
[10]
Long, J. S. and Ervin, L. H. (2000). Using heteroscedasticit y consistent standard errors in the linear regression model. The American Statistician , 54(3):217–224. 35
2000
-
[11]
and Yi, N
Mallick, H. and Yi, N. (2014). A new bayesian lasso. Statistics and its interface , 7(4):571. M¨ uller, H.-G. and Stadtm¨ uller, U. (1987). Estimation of heteroscedasticity in regression analysis. The annals of statistics , pages 610–625
2014
-
[12]
and Casella, G
Park, T. and Casella, G. (2008). The bayesian lasso. Journal of the american statistical association, 103(482):681–686
2008
-
[13]
and Cook, J
Ranstam, J. and Cook, J. A. (2018). Lasso regression. Journal of British Surgery , 105(10):1348–1348
2018
-
[14]
J., Schaffer, M
Rosopa, P . J., Schaffer, M. M., and Schroeder, A. N. (2013). M anaging heteroscedas- ticity in general linear models. Psychological methods, 18(3):335
2013
-
[15]
Schwartz, L. (1965). On bayes procedures. Zeitschrift f ¨ur Wahrscheinlichkeitstheorie und verwandte Gebiete, 4(1):10–26
1965
-
[16]
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodolo gy, 58(1):267–288
1996
-
[17]
White, H. (1980). A heteroskedasticity-consistent covari ance matrix estimator and a direct test for heteroskedasticity. Econometrica: journal of the Econometric Society , pages 817–838
1980
-
[18]
and Hastie, T
Zou, H. and Hastie, T. (2005). Regularization and variable s election via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301– 320. 36
2005
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.