REVIEW 3 major objections 8 minor 29 references
Scalable Bayesian Quantile Regression via EM and INLA: The EM-INLA Algorithm
T0 review · 3 major / 8 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The paper proposes EM-INLA, an algorithm that makes Bayesian hierarchical quantile regression tractable for very large datasets by replacing MCMC with closed-form EM steps and INLA approximations.
desk verdict A practical EM+INLA hybrid for hierarchical quantile regression that scales to 400k observations; the EM theory is loose, but the closed-form updates and empirical validation make it worth refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is the normal-exponential mixture representation of the Asymmetric Laplace Distribution: Y = mu + theta V + kappa sqrt(sigma V) Z, with V exponential and Z standard normal, which makes Y conditionally Gaussian given V. The E-step then reduces to closed-form moments of a Generalized Inverse Gaussian distribution with index 1/2, and the M-step becomes a weighted Gaussian regression on pseudo-responses, solved by INLA; the scale and random-effect variances update analytically. The cluster-robust sandwich formula is the companion device that converts the working-likelihood posterior spread into calibrated frequentist intervals.
What would settle it
On a moderate simulated dataset where the exact M-step is computable, evaluate the penalized ALD log-likelihood after every EM-INLA iteration; any non-negligible decrease from one iteration to the next would falsify the implicit monotone-ascent premise of the algorithm.
Extended reading notes
Core claim
The central claim is that the Asymmetric Laplace working likelihood, via its normal-exponential mixture, turns each EM M-step into a weighted Gaussian regression that INLA can fit, so no MCMC and no smoothing of the check loss are needed. The E-step conditional distributions of latent variables are Generalized Inverse Gaussian with closed-form moments, the scale parameter has an exact closed-form update, and each random-effect variance is updated by posterior mean plus variance from INLA. A final INLA run conditional on the converged hyperparameters gives posterior marginals, and a one-pass cluster-robust sandwich correction restores near-nominal interval coverage that the naive empirical-Ba
Load-bearing premise
The M-step replaces the exact weighted-Gaussian maximizer with INLA's approximate fit, and the paper does not prove that the objective still ascends; if that approximation error accumulates across iterations, the algorithm could converge to a point far from the true marginal maximum.
Editorial extensions
If this is right
- EM-INLA removes the need for MCMC in hierarchical quantile regression, making quantile analyses of administrative-scale datasets feasible in minutes.
- Simulations indicate point estimates match Hamiltonian Monte Carlo across normal, heavy-tailed, and heteroscedastic error scenarios, with 15x to 50x speedups.
- The closed-form random-effect variance update stays strictly positive, avoiding the zero-variance boundary collapse seen in frequentist mixed-model quantile fits.
- Naive intervals from the final INLA call undercover; the analytic cluster-robust sandwich correction brings coverage to near nominal in one extra pass, matching a cluster bootstrap benchmark.
- On the 2023 national test data, quantile effects reveal that parental education and internet access matter most in the upper tail, while the private-school premium is largest at the bottom.
Reading between the lines
- Because the method inherits INLA's prior flexibility, the same EM-INLA loop should extend to spatial, temporal, or spatio-temporal random effects, giving scalable quantile versions of areal and longitudinal models.
- The absence of a formal convergence guarantee for the approximate M-step suggests a targeted diagnostic: monitoring the penalized ALD objective directly, not just sigma, would catch drift when INLA's approximation is crude.
- The empirical-Bayes conditioning on hyperparameters means practitioners should treat the sandwich or bootstrap intervals as the inferential output, not the raw posterior marginals; this is a design choice, not a bug.
- Fitting each quantile independently invites quantile crossing; combining EM-INLA with non-crossing or rearrangement methods would likely preserve its speed while restoring coherent quantile functions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes EM-INLA, an empirical-Bayes algorithm for hierarchical quantile regression based on the asymmetric Laplace (ALD) working likelihood. The ALD is written as a normal-exponential mixture, and the algorithm iterates between an E-step that computes closed-form Generalized Inverse Gaussian moments for the latent variables and an M-step that fits a weighted Gaussian regression with INLA. Closed-form updates are derived for the ALD scale and for the random-effect variances. The authors also propose an analytic cluster-robust sandwich correction for the fixed-effect intervals and compare it with a cluster bootstrap. The method is evaluated in four simulation scenarios against lqmm/qrLMM, QREM, and Hamiltonian Monte Carlo, and is applied to the 2023 Prueba Saber 11 data with 415,472 students nested in schools and municipalities. The central claims are: EM-INLA avoids MCMC, recovers parameters comparably to HMC, achieves 15x–50x speedups, produces calibrated intervals via the sandwich, and scales sublinearly with sample size.
Significance. If the central claims hold, this is a practically valuable contribution: it would bring empirical-Bayes hierarchical quantile regression to data sizes at which MCMC is infeasible, while retaining closed-form updates and a single-pass uncertainty correction. The manuscript has notable strengths: the derivations in Appendix A are explicit and the σ update reproduces the known ALD maximum-likelihood estimator in the baseline case; the simulations compare against independent external implementations (lqmm, qrLMM, QREM, HMC/Stan); the reproducibility statement indicates that code is available; and the interval-coverage problem is addressed directly, with the sandwich estimator checked against a cluster bootstrap. However, the load-bearing point that EM-INLA is a valid EM algorithm with a well-defined monotonically increasing objective is not established, and the authors themselves acknowledge that strict monotone ascent is not guaranteed. This gap must be resolved or the algorithm must be reframed as an approximate fixed-point procedure with appropriate convergence diagnostics.
major comments (3)
- [§3.1–3.4 and Algorithm 1, especially Eq. (10), Theorem 3.1 Eq. (14), and Prop. 3.2 Eq. (19)] The algorithm is presented as an EM algorithm, but no single objective is shown to be monotonically increased. The E-step in Eq. (10) conditions the latent variables v_i on point estimates of the location parameters, while the M-step for random-effect variances in Eq. (18)–(19) treats alpha as latent and uses posterior moments. Meanwhile, Theorem 3.1’s Q-function Eq. (14) omits the random-effect prior terms that the INLA fit in Algorithm 1, lines 4–6, actually maximizes. Thus there is an inconsistency between the Q-function used in the derivation and the criterion actually optimized by INLA. The Section 6 concession that 'strict monotone ascent ... is not mathematically guaranteed' is weaker than the actual problem: the paper does not identify a well-defined objective that EM-INLA increases. This is load-bearing for the 'EM' description and for the claim that the algorithm maximizes an e
- [§3.5, Eq. (20)] The stopping rule monitors only the relative change in σ. The paper states that once σ stabilizes, 'the location and variance updates reach a steady state,' but no argument is given for why flatness of σ implies convergence of β, α, and the random-effect variances. This matters because the objective-function gap above means that a flat σ trajectory could stop at a point that is not a fixed point of the full update vector. At minimum, the simulation section should report the full-vector changes over iterations, or the stopping rule should include the location parameters and variance components. Without this, the empirical convergence claims are incomplete.
- [§3.5, Eq. (21), and §4.5] The analytic sandwich estimator is derived from a first-order expansion of the penalized criterion, and its applicability to two-level nested random effects is asserted rather than proved. The authors note that the derivation assumes Gaussian random effects, exogeneity, and a constant residual density, and they relegate a formal asymptotic treatment to future work. The simulation coverage in Table 5 is encouraging, but it is based on one design with J2 = 10 top-level clusters and does not exercise the assumptions (e.g., non-Gaussian random effects, clustered covariates, or a larger number of top-level units). Since the application and the paper’s uncertainty-quantification claims rely on Eq. (21), the manuscript should either provide a more formal justification or clearly state the conditions under which the intervals are expected to be calibrated, with targeted simulations for those con
minor comments (8)
- [Section 1] The introduction describes the method as an 'exact EM algorithm,' but Section 6 states that the M-step is approximate because it uses an INLA fit. Please replace 'exact' with 'closed-form EM-type' or similar throughout, so the terminology is consistent.
- [Section 3.1, Eq. (10)] The conditioning notation in Eq. (10) is inconsistent with later expressions: Eq. (10) conditions on current parameter estimates, while Prop. 3.2 conditions on '˜y, β̂'. Please unify the notation and clarify that the posterior moments in Eq. (19) are INLA approximations.
- [Algorithm 1, step 6] Step 6 uses Var(α_j^(k) | ·) from INLA, but the variance update in Eq. (19) is derived as an EM conditional expectation. Since the INLA fit conditions on the current hyperparameters and uses a Laplace approximation, the variance used is not the exact EM conditional variance. This approximation should be stated explicitly next to Proposition 3.2.
- [§4.4, Table 3] The HMC results under M2 appear to be dominated by a small number of divergent fits (RMSE of σ2 between 16.8 and 22.3 against a true value of 4). Reporting the number of divergent transitions or other convergence diagnostics for the HMC runs would make the comparison more interpretable and would strengthen the claim that EM-INLA is 'comparable to HMC.'
- [§4.3, Table 2] The QREM variance collapse under M4 is mentioned qualitatively. Please report the frequency of boundary estimates (zero variance) across replications, so the reader can assess how typical the collapse is.
- [Reproducibility] The code availability statement says replication code is available at GitHub but no URL is given. Please provide a link or repository identifier for journal review.
- [Throughout] There are minor typographical issues, e.g., 'Wepropose' in the abstract and 'understating' where 'understating uncertainty' is intended. A careful copyedit is recommended.
- [References] Please verify the status and URLs of the references dated 2026, since some appear to be online-first or arXiv preprints; give the most citable version.
Circularity Check
No significant circularity: EM-INLA is validated against independent simulation truths and external benchmarks, and no derivation step reduces to its inputs by construction.
full rationale
The paper's central derivations are self-contained transformations of the ALD normal–exponential mixture: the E-step moments follow from the GIG kernel (Eq. 10–12), the M-step location update is an algebraic completion of the square yielding a weighted Gaussian criterion (Theorem 3.1, Eq. 13–14), and the scale and variance updates are obtained by differentiating the expected complete-data log-likelihood (Eq. 15–19). None of these steps presupposes the target estimates; the unknown scale and variance parameters are explicitly the arguments being optimized. The accuracy claim is tested against independent simulation truths and external implementations (HMC/Stan, QREM, lqmm, qrLMM), so the 'recovers the true parameters' statement is not a renamed fit. The only admitted gap—that replacing the exact M-step by an approximate INLA call means strict monotone ascent is not mathematically guaranteed—is a correctness/robustness limitation, not a circularity: the algorithm still targets a well-defined weighted Gaussian fit and the paper does not claim a monotonicity result derived from its own assumptions. No load-bearing self-citations are present, and no uniqueness theorem or ansatz is imported from the authors' prior work. The paper is accordingly a normal method-development contribution with external validation, warranting a circularity score of 0.
Assumptions & free parameters
free parameters (4)
- σ (ALD scale) =
estimated via Eq. (17) at each EM iteration
- σ_k² (random-effect variances) =
estimated via Eq. (19)
- Convergence tolerance tol =
unspecified
- Prior variance for fixed effects =
1000
assumptions (5)
- standard math ALD normal-exponential mixture representation (Kozumi & Kobayashi 2011)
- standard math E-step conditional moments of GIG(1/2, χ, ψ) (Jørgensen 1982)
- domain assumption ALD working likelihood provides valid posterior inference under misspecification (Sriram et al. 2013; Yang et al. 2016)
- domain assumption INLA's Laplace approximations are accurate enough for the latent Gaussian M-step
- domain assumption Sandwich assumptions: random effects exogenous and Gaussian, constant residual density f_i ≡ f
Cite this review
Pith. "Pith review of Scalable Bayesian Quantile Regression via EM and INLA: The EM-INLA Algorithm." pith.science (2026). https://pith.science/paper/SZIYH5O4
@misc{pith2026260710432,
author = {Pith},
title = {Pith review of: Scalable Bayesian Quantile Regression via EM and INLA: The EM-INLA Algorithm},
year = {2026},
howpublished = {\url{https://pith.science/paper/SZIYH5O4}},
note = {Machine review of arXiv:2607.10432}
}
read the original abstract
We propose EM-INLA, a scalable algorithm for empirical-Bayes hierarchical quantile regression that combines the Expectation-Maximization (EM) algorithm with Integrated Nested Laplace Approximations (INLA). The method exploits the normal-exponential mixture representation of the Asymmetric Laplace Distribution (ALD) to reformulate each M-step as a weighted Gaussian regression handled by INLA, with the ALD scale and random-effect variances updated in closed form at each iteration. The scale and variance hyperparameters are estimated by marginal maximum likelihood via the EM algorithm, and posterior marginals for the regression and random-effect parameters are obtained from a final INLA call conditional on the converged hyperparameter estimates. The result is an algorithm that avoids all MCMC sampling while scaling to large datasets and complex hierarchical structures that are intractable for standard fully Bayesian approaches. In a simulation study across four error scenarios, EM-INLA recovers the true parameters with accuracy comparable to Hamiltonian Monte Carlo (HMC) while achieving speedups ranging from 15x to over 50x depending on the scenario. The method is applied to the 2023 Prueba Saber 11 standardized test in Colombia to model the conditional quantiles of student scores as a function of socioeconomic covariates under a hierarchical structure of students nested within schools and municipalities, with over 400,000 observations.
Figures
Figures from the paper (7 more)
Reference graph
Works this paper leans on
-
[5]
Quantile and probability curves without crossing. Econometrica 78, 1093–1125. URL: http://www.jstor.org/stable/40664520. Chung,Y.,Rabe-Hesketh,S.,Dorie,V.,Gelman,A.,Liu,J.,2013. Anondegeneratepenalizedlikelihoodestimatorforvarianceparametersinmulti- levelmodels. Psychometrika78,685–709. URL:https://doi.org/10.1007/s11336-013-9328-2,doi:10.1007/s11336-013-...
arXiv 2013
-
[10]
Quantile regression for longitudinal data using the asymmetric Laplace distribution. Biostatis- tics 8, 140–154. URL:https://doi.org/10.1093/biostatistics/kxj039, doi:10.1093/biostatistics/kxj039, arXiv:https://academic.oup.com/biostatistics/article-pdf/8/1/140/700002/kxj039.pdf. Geraci, M., Bottai, M.,
-
[13]
Instituto Colombiano para la Evaluación de la Educación
Technical Report. Instituto Colombiano para la Evaluación de la Educación. Bogotá, Colombia. URL:https://www.icfes.gov.co/wp-content/uploads/2025/09/INFORME_NACIONAL_RESULTADOS_SABER_ 11_2024.pdf. T. Rodriguez & J. Cardona:Preprint submitted to ElsevierPage 28 of 29 Scalable Bayesian Quantile Regression via EM-INLA Jørgensen,B.,1982. StatisticalProperties...
-
[15]
Electronic Journal of Statistics 6, 354–381
The Bernstein-Von-Mises theorem under misspecification. Electronic Journal of Statistics 6, 354–381. URL:https://doi.org/10.1214/12-EJS675, doi:10.1214/12-EJS675. Koenker, R.,
-
[19]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73, 423–498
An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73, 423–498. URL: https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9868.2011.00777.x, doi:https://doi.org/10.1111/j. 1467-9868.2011.00777.x...
arXiv 2011
-
[24]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71, 319–392
Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 71, 319–392. URL: https://rss.onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9868.2008.00700.x, doi:https://doi.org/10.1111/ j.1467-9868.2008.00700.x,arXiv:https://rss.onl...
arXiv 2008
-
[26]
Posterior consistency of Bayesian quantile regression based on the misspecified asymmetric Laplace density. Bayesian Analysis 8, 479–504. doi:10.1214/13-BA817. Stan Development Team,
-
[28]
Statistics & Probability Letters 54, 437–447
Bayesian quantile regression. Statistics & Probability Letters 54, 437–447. URL:https://www.sciencedirect. com/science/article/pii/S0167715201001249, doi:https://doi.org/10.1016/S0167-7152(01)00124-9. Yue, Y.R., Rue, H.,
Show all 29 references
-
[29]
Computational Statistics & Data Analysis 55, 84–96
Bayesian inference for additive mixed quantile regression models. Computational Statistics & Data Analysis 55, 84–96. URL:https://www.sciencedirect.com/science/article/pii/S0167947310001933, doi:https://doi.org/10.1016/j. csda.2010.05.006. T. Rodriguez & J. Cardona:Preprint su...
2010 doi
-
[1977]
Journal of the Royal Statistical Society
Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39, 1–38. URL:http://www.jstor.org/stable/2984875. Eide, E., Showalter, M.H.,
-
[1978]
Econometrica 46, 33–50
Regression quantiles. Econometrica 46, 33–50. URL:http://www.jstor.org/stable/1913643. Koenker,R.,Ng,P.,Portnoy,S.,1994. Quantilesmoothingsplines. Biometrika81,673–680. URL:https://doi.org/10.1093/biomet/81.4. 673,doi:10.1093/biomet/81.4.673,arXiv:https://academic.oup.com/biom...
1994
-
[1982]
Journal of the Royal Statistical Society
Finding the observed information matrix when using the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 44, 226–233. URL:http://www.jstor.org/stable/2345828. Müller, U.K.,
-
[1983]
The Annals of Statistics 11, 95–103
On the convergence properties of the EM algorithm. The Annals of Statistics 11, 95–103. URL:http://www.jstor.org/ stable/2240463. Yang,Y.,Wang,H.J.,He,X.,2016. PosteriorinferenceinBayesianquantileregressionwithasymmetricLaplacelikelihood. InternationalStatistical Review 84, 32...
2016
-
[1991]
Annals of the Institute of Statistical Mathematics 43, 1–20
Bayesian image restoration, with two applications in spatial statistics. Annals of the Institute of Statistical Mathematics 43, 1–20. URL:https://doi.org/10.1007/BF00116466, doi:10.1007/BF00116466. Bofinger,E.,1975.Estimationofadensityfunctionusingorderstatistics.AustralianJou...
1975 doi
-
[1992]
Journal of the American Statistical Association 87, 58–68
Hierarchical spline models for conditional quantiles and the demand for electricity. Journal of the American Statistical Association 87, 58–68. URL:http://www.jstor.org/stable/2290452. Homan, M.D., Gelman, A.,
-
[2001]
Empirical Economics 26, 221–246
For whom the reductions count: A quantile regression analysis of class size and peer effects on scholastic achievement. Empirical Economics 26, 221–246. URL:https://doi.org/10.1007/s001810000054, doi:10.1007/s001810000054. Lindgren, F., Rue, H., Lindström, J.,
-
[2005]
volume 104 ofMonographs on Statistics and Applied Proba- bility
Gaussian Markov Random Fields: Theory and Applications. volume 104 ofMonographs on Statistics and Applied Proba- bility. 1ed.,ChapmanandHall/CRC,BocaRaton,FL. URL:https://doi.org/10.1201/9780203492024,doi:10.1201/9780203492024. Rue, H., Martino, S., Chopin, N.,
-
[2007]
Journal of the Royal Statistical Society
Bootstrapping clustered data. Journal of the Royal Statistical Society. Series B (Statistical Methodology) 69, 369–390. URL:http://www.jstor.org/stable/4623274. Galarza, C.E., Lachos, V.H.,
-
[2009]
The Annals of Statistics 37, 2011–2055
On the computational complexity of MCMC-based estimators in large samples. The Annals of Statistics 37, 2011–2055. URL:http://www.jstor.org/stable/30243694. Besag, J., York, J., Mollié, A.,
2011
-
[2010]
Biometrika 97, 825–838
Noncrossing quantile regression curve estimation. Biometrika 97, 825–838. URL:https://doi.org/10.1093/biomet/asq048, doi:10.1093/biomet/asq048, arXiv:https://academic.oup.com/biomet/article-pdf/97/4/825/665049/asq048.pdf. Carpenter, B., Gelman, A., Hoffman, M.D., Lee, D., Good...
-
[2011]
Journal of Statistical Computation and Simulation 81, 1565–1578
Gibbs sampling methods for Bayesian quantile regression. Journal of Statistical Computation and Simulation 81, 1565–1578. URL:https://doi.org/10.1080/00949655.2010.496117, doi:10.1080/00949655.2010.496117, arXiv:https://doi.org/10.1080/00949655.2010.496117. Levin, J.,
2010
-
[2012]
Journal of Econometrics170,76–91
Asymptotics for panel quantile regression models with individual effects. Journal of Econometrics170,76–91. URL:https://www.sciencedirect.com/science/article/pii/S0304407612000760,doi:https://doi. org/10.1016/j.jeconom.2012.02.007. Kleijn, B.J.K., van der Vaart, A.W.,
2012 doi
-
[2013]
Econometrica 81, 1805–1849
Risk of Bayesian inference in misspecified models, and the sandwich covariance matrix. Econometrica 81, 1805–1849. URL:https://onlinelibrary.wiley.com/doi/abs/10.3982/ECTA9097, doi:https://doi.org/10.3982/ECTA9097, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.3982/ECTA9097...
2026 arXiv
-
[2014]
Statistics and Computing 24, 461–479
Linear quantile mixed models. Statistics and Computing 24, 461–479. doi:10.1007/s11222-013-9381-9. Gordon, R.A.,
-
[2015]
Statistics & Probability Letters 107, 18–26
A sandwich likelihood correction for Bayesian quantile regression based on the misspecified asymmetric Laplace density. Statistics & Probability Letters 107, 18–26. URL:https://www.sciencedirect.com/science/article/pii/S016771521500276X, doi:https://doi.org/10.1016/j.spl.2015....
2015 doi
-
[2016]
Journal of Econometric Methods 5, 1–15
Quantile regression with clustered data. Journal of Econometric Methods 5, 1–15. URL:https://doi.org/ 10.1515/jem-2014-0011, doi:doi:10.1515/jem-2014-0011. Powell,J.L.,1986. Censoredregressionquantiles. JournalofEconometrics32,143–155. URL:https://www.sciencedirect.com/science...
2014
-
[2017]
Statistics and Its Interface 10, 471–482
Quantile regression in linear mixed models: a stochastic approximation EM approach. Statistics and Its Interface 10, 471–482. doi:10.4310/SII.2017.v10.n3.a10. Geraci,M.,2014. Linearquantilemixedmodels:ThelqmmpackageforLaplacequantileregression. JournalofStatisticalSoftware57,1...
2017 doi
-
[2021]
URL: https://arxiv.org/abs/1910.11479,arXiv:1910.11479
Quantile regression modelling via location and scale mixtures of normal distributions. URL: https://arxiv.org/abs/1910.11479,arXiv:1910.11479. Belloni, A., Chernozhukov, V.,
1910 arXiv
-
[2024]
URL:https://CRAN.R-project.org/ package=qrLMM, doi:10.32614/CRAN.package.qrLMM
qrLMM: Quantile Regression for Linear Mixed-Effects Models. URL:https://CRAN.R-project.org/ package=qrLMM, doi:10.32614/CRAN.package.qrLMM. r package version 2.3. Galarza, C.E., Lachos, V.H., Bandyopadhyay, D.,
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.