Pith. sign in

REVIEW 5 major objections 5 minor 18 references

Heteroscedastic Double Bayesian Elastic Net

T0 review · 5 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Pith's one-line read HDBEN jointly models the mean and log-variance with Bayesian elastic-net priors, claiming posterior concentration, exact support recovery, and asymptotic normality for both coefficient vectors under mild conditions.

desk verdict A promising model idea undermined by unsound theory: the paper's central variable-selection theorem contradicts its own continuous prior, and the contraction results are assumed rather than proved. read the letter →

arxiv 2502.02032 v1 pith:6K6MYQLQ submitted 2025-02-04 stat.ME cs.AIstat.ML

classification stat.MEcs.AIstat.ML MSC 62F1562J0762J0562G20
keywords HeteroscedasticityBayesianElasticNetPosteriorConcentrationVariableSelectionHigh-DimensionalRegressionLog-variancemodelScalemixtureofnormalsMCMC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a Bayesian regression model that fits the conditional mean and the conditional log-variance together, with an elastic-net-style prior on both coefficient vectors. The central claim is that this dual regularization simultaneously selects predictors for the mean and for the variance even when the number of predictors exceeds the sample size. If the claim is right, a single fitted posterior yields both point estimates and uncertainty statements that adapt to predictor-dependent error variability. The authors state asymptotic guarantees: posterior concentration, exact support recovery, and asymptotic normality of active coefficients, supported by simulation comparisons against OLS, Lasso, elastic net, Bayesian Lasso, and Bayesian elastic net.

What carries the argument

The carrying mechanism is the hierarchical Bayesian elastic-net prior placed on both coefficient vectors: $\beta \mid \tau_\beta$ is normal with covariance $(D_\beta^{-1} + \lambda_{2,\beta} I)^{-1}$, and each $\tau_{\beta,j}$ follows an exponential distribution with rate $\lambda_{1,\beta}^2/2$, with the identical construction for $\gamma$. This scale-mixture-of-normals representation is the Bayesian analogue of the combined $\ell_1+\ell_2$ penalty, giving sparsity through the exponential mixing and stable grouped shrinkage through the ridge-type precision. In HDBEN it is applied twice, once for the mean and once for the log-variance, so that variance coefficients can be selected and shrunk in the same sense as mean coefficients. The posterior sampler exploits conditional conjugacy: $\beta$ has a normal full conditional, $\tau_\beta$ and $\tau_\gamma$ have inverse-Gaussian full conditionals, the shrinkage hyperparameters have Gamma full conditionals, and $\gamma$ is updated with a Metropolis-Hastings step.

What would settle it

Compute the posterior probability $P(\beta_j = 0 \mid y, X)$ in the stated model for any finite $n$ and $d$; it is zero for every $j$ because the posterior density of each coefficient is continuous, directly contradicting the claim that $P(S_\beta = S_{\beta_0}, S_\gamma = S_{\gamma_0} \mid y, X)$ tends to one. A simulation counterpart is to run the described Gibbs sampler on data with sparse true supports and record whether any MCMC draw of $\beta$ or $\gamma$ ever equals zero exactly.

Watch

Extended reading notes

Core claim

The paper's central claim is that applying the Bayesian elastic-net hierarchy to the log-variance coefficients $\gamma$, alongside the usual mean coefficients $\beta$, turns the heteroscedastic Gaussian model $y_i \sim N(X_i^T\beta, \exp(X_i^T\gamma))$ into a jointly regularized Bayesian procedure. HDBEN's posterior is claimed to concentrate around the true parameters at a rate of order $\sqrt{(s_\beta+s_\gamma)\log d / n}$, to recover the exact sparsity patterns of both $\beta$ and $\gamma$ with probability tending to one, and to be asymptotically normal on the active coefficients under mild regularity conditions. The same scale-mixture prior that gives the Bayesian elastic net its $\ell_1+\ell_2$ behavior is placed on both parameter blocks, so sparsity and grouping are induced in the variance model as well as the mean model. Simulations with $d$ up to 1000 report lower estimation error for $\beta$ than the compared baselines, particularly where heteroscedasticity is strong.

Load-bearing premise

The load-bearing premise is that a continuous Bayesian elastic-net prior can produce exact zeros in the posterior, yet under the paper's normal-exponential hierarchy every coefficient has a continuous posterior density, so the event of exact support recovery has posterior probability zero.

Editorial extensions

If this is right

  • Users obtain one posterior that estimates both mean effects and variance effects, so prediction intervals can widen automatically where the predictors indicate higher volatility.
  • Under the stated theory, the squared-error risk of the posterior means decays as $O((s_\beta+s_\gamma)\log d / n)$, the usual near-optimal high-dimensional rate.
  • If exact support recovery holds, the fitted model tells the practitioner which covariates shift the response and which covariates change its variability, with no separate selection step.
  • Asymptotic normality of the active coefficients would justify reporting posterior-based confidence intervals for the selected mean and variance parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: because the normal-exponential hierarchy gives every coefficient a continuous posterior density, the exact-support consistency claim in Theorem 2 would need a spike-and-slab prior or a post-hoc thresholding rule to be literally true.
  • Extension: the random-walk Metropolis update for the $d$-dimensional vector $\gamma$ is likely to mix more slowly as $d$ grows; a testable remedy is a reparameterized or adaptive sampler, and effective sample sizes should be compared across dimensions.
  • Extension: the same dual elastic-net construction transfers to heteroscedastic GLMs where both location and dispersion depend on predictors, since the variance-coefficient prior does not rely on Gaussian responses.
  • Extension: a finite-sample check of support recovery against thresholded HDBEN posteriors and a spike-and-slab competitor would show how much of the claimed selection consistency is visible in practice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a Bayesian regression model that jointly models the conditional mean and log-variance via hierarchical elastic-net priors. The authors claim posterior concentration, variable selection consistency, and asymptotic normality of the posterior in high-dimensional settings, and they support these claims with simulation studies comparing HDBEN to OLS, Lasso, Elastic Net, Bayesian Lasso, and Bayesian Elastic Net. The manuscript contains four theorems: Theorem 1 asserts posterior concentration, Corollary 1 asserts a near-optimal contraction rate for posterior means, Theorem 2 asserts variable selection consistency, and Theorems 3-4 assert asymptotic normality and a higher-order expansion.

Significance. If the theoretical results were valid, the paper would offer a useful unified Bayesian treatment of heteroscedastic regression with simultaneous shrinkage of mean and variance coefficients. The model specification is natural and the simulation section suggests the method can perform well empirically. However, the theoretical core is not established: Theorem 2 is false as stated for the specified continuous prior, Corollary 1 assumes the rate it claims to prove, and Theorem 1's invocation of Schwartz's theorem omits the required testing and entropy conditions. Because the central claims of posterior concentration, selection consistency, and asymptotic normality are load-bearing for the paper's contribution, the significance of the work as presented is severely diminished.

major comments (5)
  1. [§3.1, Eqs. (3)-(6); §4, Theorem 2] Theorem 2 asserts P(Sβ = Sβ0, Sγ = Sγ0 | y, X) → 1, where Sβ = {j : βj ≠ 0}. Under the specified hierarchical prior, β and γ have continuous Normal-scale-mixture priors with no point mass at zero, and the likelihood is a continuous Gaussian density. Consequently, for every j, P(βj = 0 | y, X) = 0 and P(γj = 0 | y, X) = 0 for every finite n and every observed (y, X). For a sparse true support, the event {Sβ = Sβ0, Sγ = Sγ0} requires some coefficients to be exactly zero, so its posterior probability is identically zero, not converging to 1. The proof's "true negatives" step claims exact zeros arise from the L1 penalty, but the prior shrinks coefficients without producing exact zeros. This is not a technical gap; the theorem is internally inconsistent with the model specification.
  2. [§4, Corollary 1 and Theorem 2] The contraction rate is assumed rather than derived. Theorem 1 only states that there exists a sequence εn → 0 such that the posterior concentrates in an εn-neighborhood; it provides no rate. Corollary 1 then "sets" εn = sqrt((sβ+sγ) log d / n) and uses that to obtain the risk bound, and Theorem 2 repeats the same choice. This is circular. Moreover, posterior concentration in probability does not imply the unconditional expectation bound in Eq. (16); without control of the low-probability complement, the expected squared error can be large.
  3. [§4, Theorem 1 and Assumption 4.2] The proof of Theorem 1 invokes Schwartz's theorem after bounding the Kullback-Leibler divergence locally, but it never verifies the two essential hypotheses: the existence of exponentially consistent tests for the complement of the neighborhood, and prior mass conditions on K-L neighborhoods (the latter are only asserted via Assumption 4.4). In the high-dimensional regime d > n, Assumption 4.2 (full column rank of X as n → ∞) cannot hold, so the setting is internally inconsistent. Thus Theorem 1's conclusion is not established.
  4. [§4, Theorem 3] The Bernstein-von Mises result is not justified. The theorem assumes a LAN condition and smoothness but does not verify them for the heteroscedastic likelihood; more concretely, the displayed Fisher information matrix has zero off-diagonal blocks between the β and γ sub-vectors, but the log-likelihood's cross-derivatives with respect to β and γ are generally nonzero because the variance exp(Xᵢᵀγ) depends on γ. The limiting covariance in Theorem 3 is therefore incorrect for the stated model.
  5. [§3.2, Eqs. (10)-(11)] The full conditional for τβ is stated to be inverse Gaussian, but with Σβ = (Dβ^{-1} + λ2,β I)^{-1}, the conditional distribution of β_j given τβ has variance τβ,j/(1 + λ2,β τβ,j), not τβ,j. The joint prior is not the standard Bayesian-lasso scale-mixture representation, so the inverse Gaussian full conditional does not follow. This calls into question the correctness of the MCMC sampler used in the simulations.
minor comments (5)
  1. [§4, Theorem 1 proof] The proof contains a missing citation: "(see, e.g., ?)" should be replaced with a proper reference for the K-L divergence between Gaussian densities.
  2. [§4, Assumption 4.2] Assumption 4.2 states that the design matrix has full column rank as n → ∞, which contradicts the paper's high-dimensional setting where d can exceed n. If the intended assumption is d ≤ n for the asymptotic theory, this should be stated explicitly and reconciled with the simulation settings.
  3. [§4, Theorem 4] The theorem heading states "Assumptions 4.1–4.6 and 4.5," which duplicates Assumption 4.5; the intended list should be corrected.
  4. [§5, Figure 2 caption] The caption refers to "HDEN β1" while the method is called HDBEN throughout; the label should be updated.
  5. [§5.1, Table 1] Table 1 reports "Heteroscedasticity γ0,j [0.1, 1.5]" but the text later says "γ0 ∈ [10, 100]"; the values are inconsistent and the notation for the true variance coefficients should be clarified.

Circularity Check

2 steps flagged · score 6.0 of 10

Corollary 1 sets the contraction rate to the target rate and Theorem 2 builds on that assumed rate; the claimed support-recovery also relies on an exact-zero property the continuous elastic-net prior does not have.

  1. self definitional [Section 4, Corollary 1 proof, Eq. (16)]
    "Given the sparsity levels sβ and sγ, and assuming a sub-Gaussian design matrix X, we set the contraction rate εn to be: εn = sqrt((sβ+sγ) ln d / n). ... Given the contraction rate εn, we have, E[||β̂ − β0||2^2 | X] ≤ C1εn^2, and E[||γ̂ − γ0||2^2 | X] ≤ C2εn^2."

    Theorem 1 only asserts that some sequence εn → 0 exists; it does not identify the rate. The corollary's proof chooses εn := sqrt((sβ+sγ) ln d / n) and then asserts the moment bounds hold at that chosen εn. The advertised near-optimal rate in Eq. (16) is therefore an input selected by the authors, not a consequence of prior concentration, metric entropy, or any calculation involving the model. The 'prediction' reduces by construction to the chosen value of εn.

  2. other [Section 4, Theorem 2 proof, first paragraph and 'True Negatives' bullet]
    "From Theorem 1, the posterior distribution concentrates around (β0, γ0) at rate εn = sqrt((sβ+sγ) ln d / n). ... True Negatives: For j /∈ Sβ0 ∪ Sγ0, the posterior distributions of βj and γj concentrate around zero due to the Laplace (L1) penalty, leading to exact zeros with high posterior probability."

    The proof inherits the rate εn from Corollary 1, which was set rather than derived, so the support-recovery statement rests on an assumed rate. More fundamentally, the 'exact zeros' step is assumed rather than obtained from the model: Eqs. (3)-(6) specify a continuous Normal scale mixture with τβ,j | λ1,β ~ Exp(λ1,β^2/2) and no point mass at zero, so P(βj = 0 | y, X) = 0 and P(γj = 0 | y, X) = 0 for every finite n. For sparse truth, the event Sβ = Sβ0, Sγ = Sγ0 then has posterior probability zero, contradicting the claimed convergence to 1. The proof's conclusion is imported from Lasso intuition rather than derived from the paper's own prior specification.

full rationale

The paper contains no load-bearing self-citations; the theoretical chain is self-contained in that sense. However, the central rate claim is circular: Corollary 1 takes the target rate εn = sqrt((sβ+sγ) ln d / n) as an input, declares it the contraction rate, and then uses it to bound the posterior mean error, so Eq. (16) is equivalent to the chosen εn. Theorem 2 then builds its selection-consistency argument on that same assumed rate. In addition, Theorem 2's proof assumes that the continuous elastic-net prior produces exact zeros with high posterior probability, which is not a property of the hierarchical Normal-exponential prior in Eqs. (3)-(6); under that prior exact support recovery has posterior probability zero for sparse true parameters. This makes the variable-selection-consistency claim not only circular in its rate dependence but also internally inconsistent with the model specification. Because the abstract and Section 6 advertise posterior concentration and variable selection consistency as the main theoretical payoff, the central derivation chain is partially circular: a key 'prediction' (the rate) is assumed by construction, and the selection result inherits that assumption. Score 6 reflects partial circularity rather than a fully self-citation-driven or definitionally identical paper.

Assumptions & free parameters 2 free parameters · 9 assumptions · 0 invented entities

The model introduces no new physical entity; it is a prior and likelihood construction. The free parameters are the unspecified hyperparameters and MCMC tuning. The axioms include the explicit assumptions and two unstated conditions (Schwartz conditions, exact-zero support) that the proofs silently require.

free parameters (2)
  • Prior hyperparameters (aβ1, bβ1, aγ1, bγ1, aβ2, bβ2, aγ2, bγ2) = not specified
    Gamma hyperprior shape and rate parameters appear in Section 3.1; the paper does not state their values for simulations, so results depend on unspecified user choices.
  • MCMC proposal variance σγ² = not specified
    Metropolis-Hastings step for γ in Section 3.2 requires a tuning parameter; its value is not stated, affecting acceptance rates and posterior accuracy.
assumptions (9)
  • domain assumption True data-generating process is linear with log-linear variance: y_i ~ N(x_i^T β0, exp(x_i^T γ0))
    Assumption 4.1, Section 4. The entire theory and simulations rely on this exact form.
  • domain assumption Design matrix X has full column rank as n → ∞
    Assumption 4.2. This cannot hold when d > n, contradicting the high-dimensional setting the paper advertises.
  • domain assumption Bounded covariates and variance bounded away from zero: ||X_i|| ≤ M and exp(x_i^T γ) ≥ c
    Assumption 4.3, used to bound KL divergence in Theorem 1.
  • domain assumption True parameters lie in prior support
    Assumption 4.4; necessary for Schwartz's theorem but not sufficient.
  • domain assumption Restricted eigenvalue condition of order s with constant κ
    Assumption 4.5; standard in high-dimensional lasso theory but unverified for the simulated designs.
  • domain assumption Hyperparameters aβ1, aγ1 > 1 and bβ1, bγ1 ≍ 1
    Assumption 4.6; stated but never used in the proofs.
  • ad hoc to paper Schwartz's theorem conditions (testing and entropy bounds) hold in the high-dimensional regime
    Invoked in Theorem 1 proof but never stated or checked; non-trivial when d grows with n.
  • ad hoc to paper Local asymptotic normality (LAN) and high-order smoothness of the likelihood
    Assumed in Theorems 3 and 4; no verification that the heteroscedastic Gaussian model with d growing satisfies LAN.
  • ad hoc to paper Continuous elastic net prior can produce exact zero posterior values
    Needed for Theorem 2's support equality; false under the given Normal-exponential hierarchy, which has no point mass at zero.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Heteroscedastic Double Bayesian Elastic Net." pith.science (2026). https://pith.science/paper/6K6MYQLQ

@misc{pith2026250202032,
  author       = {Pith},
  title        = {Pith review of: Heteroscedastic Double Bayesian Elastic Net},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6K6MYQLQ}},
  note         = {Machine review of arXiv:2502.02032}
}
abstract

In many practical applications, regression models are employed to uncover relationships between predictors and a response variable, yet the common assumption of constant error variance is frequently violated. This issue is further compounded in high-dimensional settings where the number of predictors exceeds the sample size, necessitating regularization for effective estimation and variable selection. To address this problem, we propose the Heteroscedastic Double Bayesian Elastic Net (HDBEN), a novel framework that jointly models the mean and log-variance using hierarchical Bayesian priors incorporating both $\ell_1$ and $\ell_2$ penalties. Our approach simultaneously induces sparsity and grouping in the regression coefficients and variance parameters, capturing complex variance structures in the data. Theoretical results demonstrate that proposed HDBEN achieves posterior concentration, variable selection consistency, and asymptotic normality under mild conditions which justifying its behavior. Simulation studies further illustrate that HDBEN outperforms existing methods, particularly in scenarios characterized by heteroscedasticity and high dimensionality.

Figures

Figures reproduced from arXiv: 2502.02032 by the authors.

Figure 1
Figure 1. Heteroscedasticity: ǫi ∼ exp(X⊤ i γ). 2 Preliminary In regression analysis, heteroscedasticity refers to the scenario where the variance of the error terms is not constant across observations. Consider the linear regression model: yi = X⊤ i β + ǫi , (1) for i = 1, . . . , n, where yi is the response variable, Xi ∈ R d are covariates, β are regression coefficients, and ǫi is the error term. Under homoscedasticity, th… view at source ↗
Figure 2
Figure 2. Comparison of posteriors. 5.2 Experimental Results As an illustrative example, we consider a three-dimensional case: β0 = (2.0, 1.0, 0.5)⊤ and γ0 = (1.5, 0.5, 0.0)⊤. That is, each dimension of X has a different level of het￾eroscedastic errors. In this setting, [PITH_FULL_IMAGE:figures/full_fig_p030_2.png] view at source ↗
Figure 3
Figure 3. Estimation of β and γ at different sparsity levels. sistently achieves substantially lower estimation errors compared to all competing meth￾ods. In particular, when sβ is low, HDBEN reduces the error by nearly a factor of two relative to EN and BLasso, and this advantage persists as the dimensionality increases [PITH_FULL_IMAGE:figures/full_fig_p031_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    Arellano, M. (1987). Computing robust standard errors for w ithin-groups estimators. Oxford Bulletin of Economics & Statistics , 49(4)

  2. [2]

    Brooks, S. (1998). Markov chain monte carlo method and its ap plication. Journal of the royal statistical society: series D (the Statistician) , 47(1):69–100

  3. [3]

    and Greenberg, E

    Chib, S. and Greenberg, E. (1995). Understanding the metrop olis-hastings algorithm. The american statistician , 49(4):327–335

  4. [4]

    Croux, C., Dhaene, G., and Hoorelbeke, D. (2004). Robust sta ndard errors for robust estimators. CES-Discussion paper series (DPS) 03.16 , pages 1–20

  5. [5]

    Geyer, C. J. (1992). Practical markov chain monte carlo. Statistical science , pages 473–483

  6. [6]

    Geyer, C. J. (2011). Introduction to markov chain monte carl o. Handbook of markov chain monte carlo, 20116022(45):22

  7. [7]

    Hans, C. (2009). Bayesian lasso regression. Biometrika, 96(4):835–845

  8. [8]

    Koenker, R. (1981). A note on studentizing a test for heteros cedasticity. Journal of econometrics, 17(1):107–112

Show all 18 references
  1. [9]

    and Lin, N

    Li, Q. and Lin, N. (2010). The bayesian elastic net. Bayesian Analysis, 5(1):151–170

  2. [10]

    Long, J. S. and Ervin, L. H. (2000). Using heteroscedasticit y consistent standard errors in the linear regression model. The American Statistician , 54(3):217–224. 35

  3. [11]

    and Yi, N

    Mallick, H. and Yi, N. (2014). A new bayesian lasso. Statistics and its interface , 7(4):571. M¨ uller, H.-G. and Stadtm¨ uller, U. (1987). Estimation of heteroscedasticity in regression analysis. The annals of statistics , pages 610–625

  4. [12]

    and Casella, G

    Park, T. and Casella, G. (2008). The bayesian lasso. Journal of the american statistical association, 103(482):681–686

  5. [13]

    and Cook, J

    Ranstam, J. and Cook, J. A. (2018). Lasso regression. Journal of British Surgery , 105(10):1348–1348

  6. [14]

    J., Schaffer, M

    Rosopa, P . J., Schaffer, M. M., and Schroeder, A. N. (2013). M anaging heteroscedas- ticity in general linear models. Psychological methods, 18(3):335

  7. [15]

    Schwartz, L. (1965). On bayes procedures. Zeitschrift f ¨ur Wahrscheinlichkeitstheorie und verwandte Gebiete, 4(1):10–26

  8. [16]

    Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodolo gy, 58(1):267–288

  9. [17]

    White, H. (1980). A heteroskedasticity-consistent covari ance matrix estimator and a direct test for heteroskedasticity. Econometrica: journal of the Econometric Society , pages 817–838

  10. [18]

    and Hastie, T

    Zou, H. and Hastie, T. (2005). Regularization and variable s election via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67(2):301– 320. 36

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.