Adaptive Generalized Elliptical Slice Sampling
Pith reviewed 2026-05-22 08:04 UTC · model grok-4.3
The pith
An adaptive elliptical slice sampler improves efficiency when prior and target distributions mismatch.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We introduce an adaptive generalized elliptical slice sampler that offers compelling gains in sampling efficiency while preserving many of the appealing properties of the standard elliptical slice sampler. We demonstrate the utility of the adaptive algorithm on a broad collection of target distributions arising from realistic modeling scenarios, including generalized regression, deep Gaussian process surrogate modeling, and high-dimensional sparse regression. Collectively, these case studies demonstrate the efficiency and robustness of adaptive generalized elliptical slice sampling across target distributions that are non-elliptical, non-differentiable, multi-modal, and/or high-dimensional.
What carries the argument
The adaptive generalized elliptical slice sampler, which adds an adaptation step to the standard elliptical slice sampling procedure so that proposals better match the target distribution and reduce wasted steps.
If this is right
- Sampling becomes more efficient in generalized regression models without extra tuning.
- Exploration improves in deep Gaussian process surrogate models that are hard to sample from directly.
- The method scales to high-dimensional sparse regression problems while staying robust.
- It works reliably on targets that are non-elliptical, non-differentiable, multi-modal, or high-dimensional.
Where Pith is reading between the lines
- Similar adaptation steps could be added to other slice-sampling variants to gain efficiency without losing theoretical guarantees.
- The approach may lower the barrier to using Bayesian models in settings where manual tuning of MCMC is costly.
- Further tests on very large data sets could show whether the gains persist as dimension grows.
Load-bearing premise
The ergodicity result depends on the target and the adaptation rule satisfying fairly general regularity conditions whose exact requirements are not spelled out.
What would settle it
Run the adaptive sampler on a target distribution that violates the regularity conditions and check whether the chain fails to converge to the correct stationary distribution.
Figures
read the original abstract
Elliptical slice sampling is a widely used gradient-free Markov chain Monte Carlo algorithm that is tuning-free and capable of adapting to local characteristics of the target distribution. However, its primary limitation is that sampling efficiency can quickly degrade when there is a mismatch between the prior distribution and the target distribution. To directly address this limitation, we introduce an adaptive generalized elliptical slice sampler that offers compelling gains in sampling efficiency while preserving many of the appealing properties of the standard elliptical slice sampler. We demonstrate the utility of the adaptive algorithm on a broad collection of target distributions arising from realistic modeling scenarios; including generalized regression, deep Gaussian process surrogate modeling, and high-dimensional sparse regression. Collectively, these case studies demonstrate the efficiency and robustness of adaptive generalized elliptical slice sampling across target distributions that are non-elliptical, non-differentiable, multi-modal, and/or high-dimensional. Under fairly general regularity conditions, we establish that the proposed adaptive generalized elliptical slice sampling algorithm is ergodic.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces an adaptive generalized elliptical slice sampler (AGESS) to mitigate efficiency loss in standard elliptical slice sampling caused by prior-target mismatch. It claims compelling efficiency gains across non-elliptical, non-differentiable, multi-modal, and high-dimensional targets arising in generalized regression, deep Gaussian process surrogate modeling, and high-dimensional sparse regression, while establishing ergodicity under fairly general regularity conditions.
Significance. If the efficiency gains are reproducible and the ergodicity result holds for the specific adaptive rule, the method would offer a practical, largely tuning-free MCMC tool for Bayesian models where standard ESS degrades. The empirical case studies on realistic modeling scenarios provide a useful stress test of robustness.
major comments (2)
- [§4] §4 (Ergodicity theorem): The claim that the adaptive algorithm is ergodic rests on 'fairly general regularity conditions' for adaptive MCMC. However, the manuscript does not explicitly verify that the proposed adaptation mechanism for the generalized ellipse parameters satisfies the diminishing adaptation property (i.e., that adaptation parameters converge almost surely to a fixed value) or that the non-adaptive kernels satisfy a uniform minorization condition with respect to the target. This verification is load-bearing for the central claim that the sampler targets the correct distribution while incorporating adaptation.
- [§5] §5 (Numerical experiments): The reported efficiency gains are presented without quantitative controls such as effective sample size per CPU second, Gelman-Rubin statistics across multiple chains, or direct comparison tables against both standard ESS and other adaptive slice samplers. Without these, it is difficult to assess whether the observed improvements are robust or attributable to the adaptation rule itself.
minor comments (2)
- [§3] Notation for the generalized ellipse parameters and the adaptation update rule should be introduced with a single consolidated definition early in §3 to avoid repeated cross-references.
- [Figures in §5] Figure captions for the trace plots and autocorrelation functions would benefit from explicit mention of the number of post-burn-in samples and the thinning factor used.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed comments on our manuscript. We have addressed each major point below and incorporated revisions to strengthen the presentation of the ergodicity result and the numerical evidence.
read point-by-point responses
-
Referee: [§4] §4 (Ergodicity theorem): The claim that the adaptive algorithm is ergodic rests on 'fairly general regularity conditions' for adaptive MCMC. However, the manuscript does not explicitly verify that the proposed adaptation mechanism for the generalized ellipse parameters satisfies the diminishing adaptation property (i.e., that adaptation parameters converge almost surely to a fixed value) or that the non-adaptive kernels satisfy a uniform minorization condition with respect to the target. This verification is load-bearing for the central claim that the sampler targets the correct distribution while incorporating adaptation.
Authors: We agree that explicit verification of the diminishing adaptation property and uniform minorization is important for rigor. In the revised manuscript we have added a dedicated paragraph in §4 that proves the adaptation rule for the generalized ellipse parameters satisfies diminishing adaptation (the parameters converge almost surely to a fixed limiting value). We further verify that the family of non-adaptive kernels satisfies a uniform minorization condition with respect to the target by exploiting the known minorization properties of elliptical slice sampling under the regularity conditions already stated in the theorem. These additions make the application of the general adaptive MCMC ergodicity result fully explicit for our algorithm. revision: yes
-
Referee: [§5] §5 (Numerical experiments): The reported efficiency gains are presented without quantitative controls such as effective sample size per CPU second, Gelman-Rubin statistics across multiple chains, or direct comparison tables against both standard ESS and other adaptive slice samplers. Without these, it is difficult to assess whether the observed improvements are robust or attributable to the adaptation rule itself.
Authors: We acknowledge that additional quantitative diagnostics improve the assessment of robustness. The revised §5 now reports effective sample size per CPU second for every experiment, includes Gelman-Rubin statistics computed from multiple independent chains, and provides expanded comparison tables against both standard elliptical slice sampling and other adaptive slice samplers. These metrics confirm that the observed gains are reproducible and attributable to the adaptation rule rather than implementation artifacts. revision: yes
Circularity Check
No circularity: derivation relies on external regularity conditions and independent demonstrations
full rationale
The paper claims an adaptive generalized elliptical slice sampler with efficiency gains and ergodicity under fairly general regularity conditions. No equations, fitted parameters, or self-referential definitions appear in the abstract or described claims that would make any prediction equivalent to its inputs by construction. The ergodicity result is presented as following from standard regularity conditions for adaptive MCMC rather than from a self-citation chain or ansatz smuggled via prior work by the same authors. Case studies on non-elliptical targets are offered as empirical support rather than as the sole justification for the theoretical claim. This structure keeps the derivation self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption fairly general regularity conditions suffice for ergodicity of the adaptive algorithm
Reference graph
Works this paper leans on
-
[1]
Annual Review of Statistics and its application , volume=
Functional data analysis , author=. Annual Review of Statistics and its application , volume=. 2016 , publisher=
work page 2016
-
[2]
AStA Advances in Statistical Analysis , volume=
A survey of functional principal component analysis , author=. AStA Advances in Statistical Analysis , volume=. 2014 , publisher=
work page 2014
-
[3]
Journal of the American statistical association , volume=
Functional data analysis for sparse longitudinal data , author=. Journal of the American statistical association , volume=. 2005 , publisher=
work page 2005
-
[4]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Penalized spline models for functional principal component analysis , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2006 , publisher=
work page 2006
-
[5]
The Annals of Statistics , pages=
UNIFORM CONVERGENCE RATES FOR NONPARAMETRIC REGRESSION AND PRINCIPAL COMPONENT ANALYSIS IN FUNCTIONAL/LONGITUDINAL DATA , author=. The Annals of Statistics , pages=. 2010 , publisher=
work page 2010
-
[6]
Journal of Multivariate Analysis , volume=
Robust functional principal component analysis for non-Gaussian longitudinal data , author=. Journal of Multivariate Analysis , volume=. 2022 , publisher=
work page 2022
-
[7]
Statistics and Computing , volume=
Robust Bayesian functional principal component analysis , author=. Statistics and Computing , volume=. 2025 , publisher=
work page 2025
-
[8]
Sparse Bayesian infinite factor models , author=. Biometrika , volume=. 2011 , publisher=
work page 2011
-
[9]
Elliptical slice sampling , author=. Proceedings of the thirteenth international conference on artificial intelligence and statistics , pages=. 2010 , organization=
work page 2010
-
[10]
The Journal of Machine Learning Research , volume=
Parallel MCMC with generalized elliptical slice sampling , author=. The Journal of Machine Learning Research , volume=. 2014 , publisher=
work page 2014
-
[11]
Reversibility of elliptical slice sampling revisited , author=. Bernoulli , volume=. 2025 , publisher=
work page 2025
-
[12]
Journal of applied probability , volume=
Coupling and ergodicity of adaptive Markov chain Monte Carlo algorithms , author=. Journal of applied probability , volume=. 2007 , publisher=
work page 2007
-
[13]
Advances in Applied probability , volume=
Multivariate extremes, aggregation and dependence in elliptical distributions , author=. Advances in Applied probability , volume=. 2002 , publisher=
work page 2002
-
[14]
Generalized elliptical distributions: theory and applications , author=. 2004 , school=
work page 2004
-
[15]
Active learning for deep Gaussian process surrogates , author=. Technometrics , volume=. 2023 , publisher=
work page 2023
-
[16]
Non-stationary Gaussian process surrogates , author=
-
[17]
Nonstationary Gaussian Process Surrogates , author=. arXiv e-prints , pages=
-
[18]
SIAM/ASA Journal on Uncertainty Quantification , volume=
Computer emulation with nonstationary Gaussian processes , author=. SIAM/ASA Journal on Uncertainty Quantification , volume=. 2016 , publisher=
work page 2016
-
[19]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Bayesian inference for non-stationary spatial covariance structure via spatial deformations , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2003 , publisher=
work page 2003
-
[20]
Artificial intelligence and statistics , pages=
Deep gaussian processes , author=. Artificial intelligence and statistics , pages=. 2013 , organization=
work page 2013
-
[21]
Quantitative methods for current environmental issues , pages=
Space and space-time modeling using process convolutions , author=. Quantitative methods for current environmental issues , pages=. 2002 , publisher=
work page 2002
-
[22]
Proceedings of the twenty-first international conference on Machine learning , pages=
Parameter space exploration with Gaussian process trees , author=. Proceedings of the twenty-first international conference on Machine learning , pages=
-
[23]
Recursive pathways to marginal likelihood estimation with prior-sensitivity analysis , author=. Statistical Science , pages=. 2014 , publisher=
work page 2014
-
[24]
Robotics: Science and Systems VIII , volume=
The banana distribution is Gaussian: A localization study with exponential coordinates , author=. Robotics: Science and Systems VIII , volume=. 2013 , publisher=
work page 2013
-
[25]
Artificial intelligence and statistics , pages=
Handling sparsity via the horseshoe , author=. Artificial intelligence and statistics , pages=. 2009 , organization=
work page 2009
-
[26]
Electronic Journal of Statistics , volume=
Sparsity information and regularization in the horseshoe and other shrinkage priors , author=. Electronic Journal of Statistics , volume=. 2017 , publisher=
work page 2017
-
[27]
The horseshoe estimator for sparse signals , author=. Biometrika , pages=. 2010 , publisher=
work page 2010
-
[28]
Journal of Computational and Graphical Statistics , volume=
Structured shrinkage priors , author=. Journal of Computational and Graphical Statistics , volume=. 2024 , publisher=
work page 2024
-
[29]
Journal of Machine Learning Research , volume=
Scalable approximate MCMC algorithms for the horseshoe prior , author=. Journal of Machine Learning Research , volume=
-
[30]
Journal of the American Statistical Association , volume=
Dirichlet--Laplace priors for optimal shrinkage , author=. Journal of the American Statistical Association , volume=. 2015 , publisher=
work page 2015
-
[31]
Fast sampling with Gaussian scale mixture priors in high-dimensional regression , author=. Biometrika , year=
- [32]
-
[33]
Default Bayesian analysis with global-local shrinkage priors , author=. Biometrika , volume=. 2016 , publisher=
work page 2016
-
[34]
Advances in neural information processing systems , volume=
Generalized beta mixtures of Gaussians , author=. Advances in neural information processing systems , volume=
-
[35]
Generalized double Pareto shrinkage , author=. Statistica Sinica , volume=
-
[36]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Coupling-based convergence assessment of some Gibbs samplers for high-dimensional Bayesian regression with shrinkage priors , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2022 , publisher=
work page 2022
-
[37]
Surrogates: Gaussian process modeling, design, and optimization for the applied sciences , author=. 2020 , publisher=
work page 2020
-
[38]
International Conference on Machine Learning , pages=
Geometric convergence of elliptical slice sampling , author=. International Conference on Machine Learning , pages=. 2021 , organization=
work page 2021
-
[39]
Communications in Statistics-Simulation and Computation , volume=
Sampling by divergence minimization , author=. Communications in Statistics-Simulation and Computation , volume=. 2024 , publisher=
work page 2024
-
[40]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=
Fast sampling of Gaussian Markov random fields , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2001 , publisher=
work page 2001
-
[41]
Australian & New Zealand Journal of Statistics , volume=
Bayesian hyper-lassos with non-convex penalization , author=. Australian & New Zealand Journal of Statistics , volume=. 2011 , publisher=
work page 2011
-
[42]
Journal of the American Statistical Association , volume=
Bayesian Wavelet Regression on Curves With Application to a Spectroscopic Calibration Problem , author=. Journal of the American Statistical Association , volume=
-
[43]
Expectation propagation in linear regression models with spike-and-slab priors , author=. Machine Learning , volume=. 2015 , publisher=
work page 2015
-
[44]
Multivariate output analysis for Markov chain Monte Carlo , author=. Biometrika , volume=. 2019 , publisher=
work page 2019
-
[45]
Revisiting the gelman--rubin diagnostic , author=. Statistical Science , volume=. 2021 , publisher=
work page 2021
-
[46]
Journal of econometrics , volume=
Manipulation of the running variable in the regression discontinuity design: A density test , author=. Journal of econometrics , volume=. 2008 , publisher=
work page 2008
-
[47]
Regression Discontinuity Designs: Theory and Applications , pages=
Identification and estimation using a density discontinuity approach , author=. Regression Discontinuity Designs: Theory and Applications , pages=. 2017 , publisher=
work page 2017
- [48]
-
[49]
Journal of Econometrics , volume=
Randomized experiments from non-random selection in US House elections , author=. Journal of Econometrics , volume=. 2008 , publisher=
work page 2008
-
[50]
arXiv preprint arXiv:2507.05581 , year=
Density Discontinuity Regression , author=. arXiv preprint arXiv:2507.05581 , year=
-
[51]
A Conceptual Introduction to Hamiltonian Monte Carlo
A conceptual introduction to Hamiltonian Monte Carlo , author=. arXiv preprint arXiv:1701.02434 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[52]
Computational Statistics & Data Analysis , volume=
The Bayes factor for inequality and about equality constrained models , author=. Computational Statistics & Data Analysis , volume=. 2007 , publisher=
work page 2007
-
[53]
Regression discontinuity designs: Theory and applications , author=. 2017 , publisher=
work page 2017
-
[54]
Reliability Engineering & System Safety , volume=
Surrogate modeling of advanced computer simulations using deep Gaussian processes , author=. Reliability Engineering & System Safety , volume=. 2020 , publisher=
work page 2020
-
[55]
Journal of statistical software , volume=
Stan: A probabilistic programming language , author=. Journal of statistical software , volume=
-
[56]
Sparse high-dimensional models in economics , author=. Annu. Rev. Econ. , volume=. 2011 , publisher=
work page 2011
-
[57]
A simple and efficient algorithm for gene selection using sparse logistic regression , author=. Bioinformatics , volume=. 2003 , publisher=
work page 2003
-
[58]
Mitigating the multicollinearity problem and its machine learning approach: a review , author=. Mathematics , volume=. 2022 , publisher=
work page 2022
-
[59]
Handbook of Bayesian variable selection , pages=
Spike-and-slab meets LASSO: A review of the spike-and-slab LASSO , author=. Handbook of Bayesian variable selection , pages=. 2021 , publisher=
work page 2021
-
[60]
Lasso meets horseshoe , author=. Statistical Science , volume=. 2019 , publisher=
work page 2019
-
[61]
Inference with normal-gamma prior distributions in regression problems , author=. Bayesian Analysis , volume=
-
[62]
High-Dimensional Bayesian Regularised Regression with the BayesReg Package
High-dimensional Bayesian regularised regression with the BayesReg package , author=. arXiv preprint arXiv:1611.06649 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[63]
Annual review of statistics and its application , volume=
High-dimensional statistics with a view toward applications in biology , author=. Annual review of statistics and its application , volume=. 2014 , publisher=
work page 2014
-
[64]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Regression shrinkage and selection via the lasso , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 1996 , publisher=
work page 1996
-
[65]
Statistics and computing , volume=
Understanding predictive information criteria for Bayesian models , author=. Statistics and computing , volume=. 2014 , publisher=
work page 2014
-
[66]
The annals of statistics , volume=
Slice sampling , author=. The annals of statistics , volume=. 2003 , publisher=
work page 2003
-
[67]
Honest exploration of intractable probability distributions via Markov chain Monte Carlo , author=. Statistical Science , pages=. 2001 , publisher=
work page 2001
-
[68]
Symmetric multivariate and related distributions , author=. 2018 , publisher=
work page 2018
-
[69]
Journal of Multivariate Analysis , volume=
On the theory of elliptically contoured distributions , author=. Journal of Multivariate Analysis , volume=. 1981 , publisher=
work page 1981
-
[70]
A survey on continuous elliptical vector distributions , author=. Revista matem. 2003 , publisher=
work page 2003
-
[71]
SIAM Journal on Matrix Analysis and Applications , volume=
Refined perturbation bounds for eigenvalues of Hermitian and non-Hermitian matrices , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2009 , publisher=
work page 2009
-
[72]
arXiv preprint arXiv:2411.17084 , year=
Upper and lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo , author=. arXiv preprint arXiv:2411.17084 , year=
-
[73]
On the containment condition for adaptive Markov chain Monte Carlo algorithms , author=. 2009 , publisher=
work page 2009
-
[74]
horseshoe: Implementation of the Horseshoe Prior , author=. R package version 0.1. 0 , volume=
-
[75]
Journal of Computational and Graphical Statistics , volume=
Efficient sampling for Gaussian linear regression with arbitrary priors , author=. Journal of Computational and Graphical Statistics , volume=. 2019 , publisher=
work page 2019
-
[76]
Bayesian inference for logistic models using P
Polson, Nicholas G and Scott, James G and Windle, Jesse , journal=. Bayesian inference for logistic models using P. 2013 , publisher=
work page 2013
-
[77]
Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages=
Bayesian generalized horseshoe estimation of generalized linear models , author=. Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages=. 2019 , organization=
work page 2019
-
[78]
Air Markov chain Monte Carlo , author=. arXiv preprint arXiv:1801.09309 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[79]
Bayesian estimation of nonlinear Hawkes processes , author=. Bernoulli , volume=. 2024 , publisher=
work page 2024
-
[80]
Advances in Applied Probability , volume=
Renewal in Hawkes processes with self-excitation and inhibition , author=. Advances in Applied Probability , volume=. 2020 , publisher=
work page 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.