REVIEW 3 major objections 4 minor 1 cited by
Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims to build the first Gibbs sampling algorithm for high-dimensional sparse Beta regression, using Polya-Gamma augmentation and a Horseshoe prior, together with the first posterior consistency and concentration-rate results…
desk verdict The paper's PG-augmented Gibbs sampler is built on a false identity, so it samples a different model, and the theory is an unproven sketch; the work should be rejected. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Polya-Gamma augmentation identity for a logistic kernel: $\frac{(e^{\eta})^{\varphi y}}{(1+e^{\eta})^{\varphi}} \propto e^{\kappa\eta}\int_0^\infty e^{-\omega\eta^2/2}p(\omega)\,d\omega$ with $\kappa = \varphi(y-\tfrac12)$. The paper applies this identity to the Beta regression likelihood to obtain a conditionally Gaussian form for each observation, which in turn yields a multivariate normal full conditional for $\beta$. The Horseshoe prior enters through its hierarchical inverse-gamma representation, giving conjugate updates for all shrinkage variances, and the fractional posterior $\pi_{n,\alpha}(\beta)\propto L_n(\beta)^\alpha\pi(\beta)$ supplies the theoretical framework in which the concentration proof is carried out.
What would settle it
Choose one observation with fixed $\varphi$, a grid of $\eta$ values, and a fixed $y\in(0,1)$; compute the true Beta likelihood $f(y;\mu(\eta),\varphi)$ and compare it with the marginal $\int \operatorname{PG}(\omega;\varphi,\eta)\exp(\kappa\eta-\omega\eta^2/2)\,d\omega$ over the grid. If the ratio of the two quantities changes with $\eta$, the augmentation does not reproduce the Beta likelihood and the sampler does not target the stated posterior.
Extended reading notes
Core claim
The paper's central claim is that a Beta regression likelihood, written with $\mu_i = (1+e^{-\eta_i})^{-1}$ and precision $\varphi$, can be transformed into a Gaussian form in $\eta_i$ through the Polya-Gamma identity: $\log p(y_i \mid \eta_i) \propto \kappa_i\eta_i - \omega_i\eta_i^2/2$ with $\kappa_i = \varphi(y_i - 1/2)$. This makes the full conditional of the regression vector $\beta$ multivariate normal, so inference can proceed by a closed-form Gibbs sampler alternating between Polya-Gamma draws, Gaussian draws for $\beta$, and inverse-gamma draws for the Horseshoe scale parameters. The paper further claims that, with the tempered likelihood $L_n(\beta)^\alpha$ and the Horseshoe prior, the fractional posterior concentrates around $\beta_0$ at rate $\varepsilon_n = K s^*\log(p/s^*)/n$, measured in $\alpha$-Rényi divergence, Hellinger distance, and total variation; the posterior mean estimator satisfies the same rate in prediction loss $\|X^\top(\hat\beta-\beta_0)\|_2^2$.
Load-bearing premise
The sampler is exact only if each Beta observation's likelihood equals, up to a constant independent of the linear predictor, the logistic form $(e^{\eta})^{\varphi y}/(1+e^{\eta})^{\varphi}$; the actual Beta density depends on $\eta$ through $\mu=(1+e^{-\eta})^{-1}$ in the shape parameters and through the $\beta$ normalizing constant, so this equivalence is the premise on which the whole Gibbs scheme rests.
Editorial extensions
If this is right
- Beta regression becomes usable in $p>n$ regimes: the method estimates coefficients and selects variables when the number of covariates exceeds the sample size, a setting where classical maximum-likelihood Beta regression cannot even be fitted.
- If the sampler is exact, it replaces Metropolis-Hastings updates for Beta regression with closed-form conditional draws, making posterior inference for bounded responses substantially faster.
- The claimed concentration bound says the fractional posterior learns the true coefficient vector at near-minimax rate $s^*\log(p/s^*)/n$ with probability at least $1-2/(n\varepsilon_n)$.
- The numerical comparisons claim the Horseshoe-Beta method reduces estimation error and improves variable-selection precision relative to transformed Lasso in both low- and high-dimensional simulations.
Reading between the lines
- If the Polya-Gamma rewrite is exact, the same augmentation strategy would plausibly extend to other unit-interval response models whose mean is linked through the logit function, not just the Beta family.
- The proof structure, which bounds divergence by squared linear-predictor distance, suggests the concentration result may carry over to neighboring bounded-response families with smooth densities, conditional on a similar prior-mass lemma.
- A natural testable extension is a zero-one inflated version of the sampler, where a point-mass mixture component is added while the Polya-Gamma augmentation handles the continuous part of the response.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a sparse Bayesian Beta regression model for high-dimensional bounded responses, using a fractional posterior with a Horseshoe prior. The authors claim two main contributions: a novel Gibbs sampling algorithm based on Polya-Gamma augmentation for posterior inference, and the first theoretical guarantees (posterior consistency and convergence rates) for Bayesian Beta regression. The manuscript includes simulation studies in low and high dimensions, a sensitivity analysis for the precision parameter, and a real-data application. The theoretical results are stated in Section 4 and rely on appendices for proofs.
Significance. The problem is practically relevant, and a correct sparse Bayesian Beta regression method with computable inference and theoretical guarantees would be a valuable addition. The paper, however, does not deliver these contributions as written. The central computational claim is based on an incorrect likelihood identity, and the theoretical results are not supported because the required lemmas are unproved and the appendix contains a placeholder instead of proofs. If the computational error were isolated, the method could be salvaged by using a different sampler, but that would change the paper's central novelty. The paper is therefore not publishable in its current form.
major comments (3)
- [Section 3] The Polya-Gamma identity used in Step 1, (e^{η})^{φ y}/(1+e^{η})^φ ∝ exp(κ_i η) ∫ exp(−ω η²/2)p(ω)dω with κ_i = φ(y_i − 1/2), is valid for a binomial logistic kernel but is not the Beta likelihood in Eq. (1). The Beta log-density contains the log-normalizer log Γ(φ) − log Γ(μ_i φ) − log Γ((1−μ_i)φ) and the terms (μ_i φ −1) log y_i + ((1−μ_i)φ−1) log(1−y_i), where μ_i depends on η_i through the logit link. The quadratic augmentation would require the log-density to be linear-quadratic in η_i, which it is not; for instance, the second derivative of the log-density with respect to η_i is not a constant independent of η_i. Consequently, the multivariate normal full conditional for β in Step 2 is not the full conditional under the stated posterior. The sampler targets a different, binomialized model, so the simulation results and trace plots cannot be interpreted as evidence about the Beta regression posterior the paper claims to fit.
- [Appendix A] The proofs of Lemmas 1, 2, and 3 are missing; the text reads 'Proofs of Lemma 1, 2, and 3 are given at the end of the proof section, ??', with a literal placeholder. These lemmas are the technical core needed to verify the KL conditions in Theorem 2 of [1] that underpin Theorem 1. Lemma 1 in particular must control the KL divergence between two Beta distributions with different means, which requires careful handling of the Beta normalizing constant; it is not proved, so the main concentration result is currently unsupported.
- [Section 4] Even granting Lemmas 1–3, the proof of Theorem 1 is not supplied. The paper cites Theorems 2 and 3 from Alquier and Ridgway [1] and Lemma 4 from the author's prior work [24], but it does not show that the required conditions hold for the Beta regression model. Specifically, it does not verify the existence of a distribution ρ_n satisfying ∫ KL(P_{β0}, P_β) ρ_n(dβ) ≤ ε_n and KL(ρ_n, π) ≤ n ε_n for the Beta likelihood with horseshoe prior, nor does it verify the second-moment condition needed for the high-probability bound in inequality (5). The unproved lemmas are not merely technical; they are exactly the steps that connect the prior mass bound of Lemma 4 to the KL divergence of the Beta model. The theoretical claims are therefore not established in the manuscript.
minor comments (4)
- [Throughout] The manuscript contains numerous typos and grammatical errors, including 'hierachical', 'conducte', 'P´olya' spacing, and inconsistent use of 'Cau+' for the half-Cauchy distribution. A careful copyedit is needed.
- [Table 1] In the n = 500, ρ_X = 0.5 block, the FDR for the Horseshoe method is reported as 0.99 (0.03), which appears to be an error; this value is inconsistent with the reported Precision and Specificity and should be corrected.
- [Section 5.1] The test set size is fixed at n_test = 30 regardless of training sample size; for n = 100 this is a sizable fraction, and for n = 1000 it is very small. Reporting results for a fixed test size makes comparisons across settings less informative.
- [Section 5.2] The claim that the Horseshoe method 'consistently' outperforms the Lasso is too strong for the s* = 20, n = 80 cases, where the Horseshoe recall is low (0.43 and 0.21) and the test prediction error is comparable to or worse than Lasso in some settings.
Circularity Check
No significant circularity: the convergence-rate theorem is a direct application of Alquier–Ridgway fed by an independent Horseshoe prior-mass lemma, and the sampler issue is a misapplied Pólya–Gamma identity, not a definitional reduction.
full rationale
The claimed derivation chain is not circular. The concentration theorem (Theorem 1) is obtained by instantiating the external fractional-posterior results of Alquier and Ridgway ([1], quoted as Theorems 2–3) with the prior-mass bound of Lemma 4. Lemma 4 is quoted from the author's own prior paper [24], a self-citation, but it is a parameter-free Horseshoe concentration statement whose stated assumptions (‖β0‖0 = s∗, ‖β0‖∞ ≤ C1) do not include the target Beta-regression posterior result, and the same class of result is externally available (e.g., [40]); under the standard here it is independent evidence, so it does not raise the circularity score. The Gibbs sampler's Gaussian full conditional rests on Step 2's assertion that 'we can rewrite the Beta log-likelihood ... as log p(yi|η) ∝ κiη − ωη²/2'. That assertion is false for the Beta likelihood displayed two lines earlier in Section 3 (the true log-density contains η-dependent gamma normalizing terms and η-dependent coefficients multiplying log yi and log(1−yi)); the equality does not hold, so the sampler targets a different, binomialized model. This is an error of validity, not circularity: the paper's own displayed Beta likelihood is not equal by construction to the binomial kernel, so the Gaussian conditional does not reduce to the stated posterior by definition—the identity is misapplied, not definitionally imposed. The theoretical derivation is nevertheless incomplete: Appendix A marks the proofs of Lemmas 1–3 with 'Proofs of Lemma 1, 2, and 3 are given at the end of the proof section, ??', so the KL conditions required by [1] are not actually verified in the manuscript. Incompleteness and mathematical error are distinct from circularity; no fitted parameter is renamed as a prediction and no rate is forced by a self-citation chain.
Assumptions & free parameters
free parameters (2)
- phi =
10 in simulations; not estimated in the real application
- alpha =
0.99
assumptions (4)
- ad hoc to paper There exists a distribution rho_n with integral KL(P_beta0, P_beta) rho_n(d beta) <= epsilon_n and KL(rho_n, pi) <= n epsilon_n for epsilon_n = K s* log(p/s*)/n.
- domain assumption pi_HS(norm(beta - beta0)_2 < delta) >= exp(-K s* log(p/s*)) for delta = sqrt(s* log(p/s*)/n).
- domain assumption Assumptions 1-4 are sufficient: p <= exp(n^b), norm(beta0)_inf <= C1, E norm(X)^2 <= Cx, and |X^T beta0| <= C2 a.s.
- domain assumption Lemmas 1-3: KL, Renyi, and log-likelihood Lipschitz bounds for Beta distributions with nearby means.
Cite this review
Pith. "Pith review of Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach." pith.science (2026). https://pith.science/paper/XSYP3RFS
@misc{pith2026250522211,
author = {Pith},
title = {Pith review of: Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach},
year = {2026},
howpublished = {\url{https://pith.science/paper/XSYP3RFS}},
note = {Machine review of arXiv:2505.22211}
}
read the original abstract
Bounded continuous responses -- such as proportions -- arise frequently in diverse scientific fields including climatology, biostatistics, and finance. Beta regression is a widely adopted framework for modeling such data, due to the flexibility of the Beta distribution over the unit interval. While Bayesian extensions of Beta regression have shown promise, existing methods are limited to low-dimensional settings and lack theoretical guarantees. In this work, we propose a novel Bayesian approach for high-dimensional sparse Beta regression framework that employs a tempered posterior. Our method incorporates the Horseshoe prior for effective shrinkage and variable selection. Most notable, we propose a novel Gibbs sampling algorithm using P\'olya-Gamma augmentation for efficient inference in Beta regression model. We also provide the first theoretical results establishing posterior consistency and convergence rates for Bayesian Beta regression. Through extensive simulation studies in both low- and high-dimensional scenarios, we demonstrate that our approach outperforms existing alternatives, offering improved estimation accuracy and model interpretability. Our method is implemented in the R package ``betaregbayes" available on Github.
Forward citations
Cited by 1 Pith paper
-
Bayesian Pliable Lasso with Horseshoe Prior for Interaction Effects in GLMs with Missing Responses
A shared-horseshoe Bayesian pliable lasso shrinks main and interaction effects jointly, enables uncertainty quantification, and handles missing responses via data augmentation.
Reference graph
Works this paper leans on
-
[1]
Alquier, P. and Ridgway, J. (2020). Concentration of tempered posteriors and of their variational approximations. The Annals of Statistics , 48(3):1475– 1497
work page 2020
-
[24]
Mai, T. T. (2024a). Concentration of a Sparse Bayesian Model With Horseshoe Prior in Estimating High-Dimensional Precision Matrix. Stat, 13(4):e70008
work page 2024
-
[2]
Barndorff-Nielsen, O. E. and Jørgensen, B. (1991). Some para metric models on the simplex. Journal of multivariate analysis , 39(1):106–116
work page 1991
-
[3]
Bayes, C. L., Baz´ an, J. L., and Garcıa, C. (2012). A new robust regression model for proportions. Bayesian Analysis , 7(4):841–866
work page 2012
-
[4]
C., Lecu´ e, G., and Tsybakov, A
Bellec, P. C., Lecu´ e, G., and Tsybakov, A. B. (2018). Slope meet s lasso: im- proved oracle bounds and optimality. The Annals of Statistics , 46(6B):3603– 3642
work page 2018
-
[5]
Bhadra, A., Datta, J., Polson, N. G., and Willard, B. (2017). The Ho rse- shoe+ Estimator of Ultra-Sparse Signals. Bayesian Analysis , 12(4):1105– 1131
work page 2017
-
[6]
Bhattacharya, A., Pati, D., and Yang, Y. (2019). Bayesian frac tional poste- riors. Annals of Statistics , 47(1):39–66
work page 2019
-
[7]
Bissiri, P. G., Holmes, C. C., and Walker, S. G. (2016). A general fr amework for updating belief distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology , 78(5):1103–1130
work page 2016
Show all 44 references
-
[8]
M., Polson, N
Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). The hors eshoe estimator for sparse signals. Biometrika, 97(2):465–480
2010
-
[9]
Castillo, I., Schmidt-Hieber, J., and van der Vaart, A. (2015). Ba yesian linear regression with sparse priors. The Annals of Statistics , 43(5):1986–2018
2015
-
[10]
and N´ u˜ nez-Ant´ on, V
Cepeda-Cuervo, E. and N´ u˜ nez-Ant´ on, V. (2013). Spatia l double gener- alized beta regression models: Extensions and application to study q uality of education in colombia. Journal of Educational and Behavioral Statistics , 38(6):604–628
2013
-
[11]
Chakraborty, A., Bhattacharya, A., and Mallick, B. K. (2020). Bayesian sparse multiple regression for simultaneous rank reduction and var iable selec- tion. Biometrika, 107(1):205–221
2020
-
[12]
and Zeileis, A
Cribari-Neto, F. and Zeileis, A. (2010). Beta regression in R. Journal of statistical software, 34:1–24
2010
-
[13]
and Cribari-Neto, F
Ferrari, S. and Cribari-Neto, F. (2004). Beta regression for modelling rates and proportions. Journal of applied statistics , 31(7):799–815
2004
-
[14]
I., Arellano-Valle, R
Figueroa-Z´ u˜ niga, J. I., Arellano-Valle, R. B., and Ferrari, S. L . (2013). Mixed beta regression: A bayesian perspective. Computational Statistics & Data Analysis , 61:137–147. 20
2013
-
[15]
I., Bayes, C
Figueroa-Z´ u˜ niga, J. I., Bayes, C. L., Leiva, V., and Liu, S. (20 22). Robust beta regression modeling with errors-in-variables: a bayesian appr oach and numerical applications. Statistical Papers, pages 1–24
-
[16]
A., and Calder ´ ın-Ojeda, E
G´ omez-D´ eniz, E., Sordo, M. A., and Calder ´ ın-Ojeda, E. (2014). The log– lindley distribution as an alternative to the beta regression model wit h appli- cations in insurance. Insurance: mathematics and Economics , 54:49–57
2014
-
[17]
and Ghosal, S
Jeong, S. and Ghosal, S. (2021). Posterior contraction in spa rse generalized linear models. Biometrika, 108(2):367–379
2021
-
[18]
Karlsson, P., M ˚ ansson, K., and Kibria, B. G. (2020). A liu estimat or for the beta regression model and its application to chemical data. Journal of Chemometrics, 34(10):e3300
2020
-
[19]
and McCullough, B
Kieschnick, R. and McCullough, B. D. (2003). Regression analys is of vari- ates observed on (0, 1): percentages, proportions and fractio ns. Statistical modelling, 3(3):193–213
2003
-
[20]
Knoblauch, J., Jewson, J., and Damoulas, T. (2022). An optimiza tion- centric view on bayes’ rule: Reviewing and generalizing variational inf erence. Journal of Machine Learning Research , 23(132):1–109
2022
-
[21]
Lemonte, A. J. and Baz´ an, J. L. (2016). New class of johnso n distributions and its associated regression model for rates and proportions. Biometrical Journal, 58(4):727–746
2016
-
[22]
and Eugenio, E
Liu, F. and Eugenio, E. C. (2018). A review and comparison of ba yesian and likelihood-based inferences in beta regression and zero-or-on e-inflated beta regression. Statistical methods in medical research , 27(4):1024–1044
2018
-
[23]
Liu, J. (2024). Beta regression for double-bounded respons e with correlated high-dimensional covariates. Stat, 13(2):e663
2024
-
[25]
Mai, T. T. (2024b). On high-dimensional classification by sparse general- ized bayesian logistic regression. arXiv preprint arXiv:2403.12832
2024 arXiv
-
[26]
Mai, T. T. (2024c). On properties of fractional posterior in ge neralized reduced-rank regression. arXiv preprint arXiv:2404.17850
2024 arXiv
-
[27]
Mai, T. T. (2025a). Concentration properties of fractional p osterior in 1-bit matrix completion. Machine Learning, 114(1):7
2025
-
[28]
Mai, T. T. (2025b). High-dimensional Bayesian Tobit regression for cen- sored response with Horseshoe prior. arXiv preprint arXiv:2505.08288 . 21
2025 arXiv
-
[29]
Mai, T. T. (2025c). Optimal sparse phase retrieval via a quasi- bayesian approach. arXiv preprint arXiv:2504.09509
2025 arXiv
-
[30]
and Schmidt, D
Makalic, E. and Schmidt, D. F. (2015). A simple sampler for the ho rseshoe estimator. IEEE Signal Processing Letters , 23(1):179–182
2015
-
[31]
Martin, R., Mess, R., and Walker, S. G. (2017). Empirical bayes p os- terior concentration in sparse high-dimensional linear models. Bernoulli, 23(3):1822–1847
2017
-
[32]
and Tang, Y
Martin, R. and Tang, Y. (2020). Empirical priors for prediction in sparse high-dimensional linear regression. Journal of Machine Learning Research , 21(144):1–30
2020
-
[33]
and Moineddin, R
Meaney, C. and Moineddin, R. (2014). A monte carlo simulation st udy comparing linear regression, beta regression, variable-dispersion beta regres- sion and fractional logit regression at recovering average differen ce measures in a two sample design. BMC medical research metho...
2014
-
[34]
M., and Ongaro, A
Migliorati, S., Di Brisco, A. M., and Ongaro, A. (2018). A new regr ession model for bounded responses. Bayesian Analysis , 13(3):845–872
2018
-
[35]
Mullen, R., Marshall, L., and McGlynn, B. (2013). A beta regressio n model for improved solar radiation predictions. Journal of applied meteorology and climatology, 52(8):1923–1938
2013
-
[36]
Paolino, P. (2001). Maximum likelihood estimation of models with bet a- distributed dependent variables. Political Analysis, 9(4):325–346
2001
-
[37]
and Vehtari, A
Piironen, J. and Vehtari, A. (2017). Sparsity information and r egularization in the horseshoe and other shrinkage priors. Electronic Journal of Statistics , 11(2):5018–5051
2017
-
[38]
G., Scott, J
Polson, N. G., Scott, J. G., and Windle, J. (2013). Bayesian infer ence for logistic models using p´ olya–gamma latent variables.Journal of the American statistical Association, 108(504):1339–1349
2013
-
[39]
and Shou, Y
Smithson, M. and Shou, Y. (2017). Cdf-quantile distributions f or modelling random variables on the unit interval. British Journal of Mathematical and Statistical Psychology, 70(3):412–438
2017
-
[40]
van der Pas, S., Szab´ o, B., and van der Vaart, A. (2017). Adaptive posterior contraction rates for the horseshoe. Electronic Journal of Statistics , 11:3196– 3225
2017
-
[41]
and Harremos, P
Van Erven, T. and Harremos, P. (2014). R´ enyi divergence a nd kullback- leibler divergence. IEEE Transactions on Information Theory , 60(7):3797– 3820. 22
2014
-
[42]
and Notaro, A
Vanni, L. and Notaro, A. (2025). The impact of study environm ent on student gpa ratios: A beta regression analysis. Research Gate, doi: 10.13140/RG.2.2.15437.14560
2025
-
[43]
and Qiu, Z
Zhang, P. and Qiu, Z. (2014). Regression analysis of proportio nal data using simplex distribution. Scientia Sinica Mathematica , 44:89
2014
-
[44]
and Huang, X
Zhou, H. and Huang, X. (2022). Bayesian beta regression for bounded responses with unknown supports. Computational Statistics & Data Analysis , 167:107345. 23
2022
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.