REVIEW 3 major objections 6 minor 23 references
Efficient Mirror-type Kernels for the Metropolis-Hastings Algorithm
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A mirror-reflected Metropolis proposal samples GLMM posteriors 2-20 times faster per second than HMC or NUTS.
desk verdict MirrorMALA is a legitimate new combination and the sparse whitening trick is the most useful bit, but the 2–20x speedup over NUTS/HMC is not supported by the paper's own tables. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the MirrorMALA proposal $$\$\theta$' = \mu^* + c(\mu^* - \$theta^{{(t)}}$) + \frac{\$epsilon^{2}$}{2}\Sigma^* \nabla \log \pi(\mu^* + c(\mu^* - \$theta^{{(t)}}$)) + \epsilon \sqrt{\Sigma^*}\, z,$$ with $c=1$ recommended; the mirror term supplies long jumps, the gradient term corrects the location of the mirrored point, and the preconditioning matrix $\Sigma^*$ is estimated from burn-in. The paper generalises this to Mirror-type kernels by inserting the mirror map into any proposal, and introduces a sparse whitening matrix $R$ that is the Cholesky factor of the estimated precision matrix with off-diagonal blocks among random-effects coordinates set to zero, so that block sampling remains cheap because conditional independence is preserved. The mirror map is what turns local moves into HMC-like long excursions without the cost of integration.
What would settle it
On a strongly skewed posterior — for example, a Gamma target with shape close to 0.1, or a GLMM with sparse data where the burn-in precision estimate is far from the true one — run MirrorMALA with the recommended $\epsilon = 0.5$ and $c = 1$ and compare effective sample size per second against NUTS; the paper predicts a large per-time win, so a measured drop to parity or worse would show that symmetry or the whitening premise is doing the work.
Extended reading notes
Core claim
The central discovery is that reflecting the current chain position $\theta^{(t)}$ about a point $\mu^*$ near the mode, $\mu^* + c(\mu^* - \theta^{(t)})$, and then following the MALA dynamics from that reflected point produces proposals that are both distant from the current state and concentrated in the target's high-density region. Because the chain jumps between the two sides of the distribution, odd-lag autocorrelations become negative, and the efficiency measure $E$ can exceed one, giving super-efficiency. The authors show analytically for an $N(0,1)$ target that MirrorMALA has the same acceptance probability as MALA, and numerically that MirrorMALA dominates random-walk, Mirror, and MALA kernels on univariate, Gaussian, and Bayesian logistic-regression targets. On GLMM posteriors, MirrorMALA with a sparse whitening matrix is the fastest sampler per unit time in the comparisons, with gains of 2-20 times over HMC or NUTS in per-second efficiency.
Load-bearing premise
The whole efficiency story rests on the target distribution being close enough to symmetric around the estimated mirror centre that a reflected proposal lands in a region of comparable density, and close enough to Gaussian that the precision-based whitening matrix decorrelates the coordinates; skewed or poorly estimated targets will reject the mirrored moves and the wall-clock advantage over NUTS shrinks or disappears.
Editorial extensions
If this is right
- A practitioner can replace a hand-tuned MALA proposal with the reflected version at nearly the same per-iteration cost and obtain lower autocorrelation and higher effective sample size on symmetric or mildly skewed targets.
- For GLMM posteriors, using the sparse whitening matrix before block sampling removes much of the cost of preconditioning, making MirrorMALA competitive with or faster than NUTS in wall-clock terms rather than only in effective-sample-size terms.
- Because Mirror-type kernels already include a mirror-embedded HMC proposal, the construction suggests a family of samplers in which reflection is combined with any proposal mechanism.
- The recommended tuning is simple: set $c=1$ and choose $\epsilon$ so the acceptance probability is high; for univariate targets $\epsilon = 0.5$ works, while high-dimensional targets need smaller scales.
Reading between the lines
- The same mirror embedding could be inserted into proposals beyond MALA and HMC, such as preconditioned Crank-Nicolson or manifold-based moves, and the efficiency gains would likely persist as long as the target is locally symmetric about the reflection centre.
- The sparse whitening idea is not specific to GLMMs: any hierarchical model whose random effects are conditionally independent given the fixed effects could use the same block-diagonal-plus-final-row precision structure, so the method extends to spatial or longitudinal latent-variable models.
- A testable extension would be a stochastic-gradient or mini-batch version of MirrorMALA; the mirror term may provide the global jumping that stochastic gradients lack, though the acceptance ratio would need a Metropolis-Hastings correction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MirrorMALA, an MH kernel that reflects the current state through an estimated center of the target and then applies a MALA-style Langevin drift, and it introduces the broader class of Mirror-type kernels. For the N(0,1) target it derives analytic average acceptance probabilities for RW, Mirror, MALA, and MirrorMALA (Eqs. 15-18) and verifies them numerically. It reports extensive 1-D experiments on five targets, multivariate Gaussian experiments, a Bayesian logistic regression example, and GLMM applications using a new sparse whitening transformation that preserves conditional independence for block sampling. The abstract claims per-time-unit efficiency 2-20 times higher than HMC/NUTS on GLMMs.
Significance. If the efficiency claims were established, the paper would offer a lightweight alternative to Hamiltonian methods for conditionally independent hierarchical models, combining a mechanism for negative autocorrelation with an easily implemented sparse preconditioner. The strengths of the paper are the clean analytic Pjump derivation for the normal case, the numerical verification of those formulas, the extensive 1-D and moderate-dimensional experiments, the clearly described sparse-whitening and block-sampling scheme, and the availability of code on GitHub. The central quantitative claim is not yet convincing because the comparison protocol conflates the mirror proposal with the new whitening preconditioner, and the reported speed-up factors are internally inconsistent.
major comments (3)
- [Abstract, §5.2.3, Tables 4-5] The headline per-time-efficiency comparison does not isolate the effect of the Mirror-type kernel. In Tables 4 and 5, all Mirror and MirrorMALA rows use the new sparse whitening (or the dense whitening) and block sampling, whereas HMC and NUTS use RStan's default diagonal mass matrix. Table 4 shows that RW (sparse), which uses no mirror move at all, already achieves 7.97e-5 ESS/sec, 6.5 times NUTS's 1.23e-5; Mirror (sparse) achieves 39.11e-5 and MirrorMALA (sparse) 23.08e-5. On the polypharmacy data, MirrorMALA (sparse) at 3.85e-5 is only 1.53 times NUTS (2.52e-5). Since the sparse whitening and blocking component alone produces a large part of the speed-up relative to the default-metric NUTS baseline, the abstract's attribution of the 2-20x gain to Mirror-type kernels is not established. A control run of NUTS or HMC with the same sparse precision metric, or a comparison of RW and MirrorMALA within the same whitened block sampler, is needed.
- [Abstract, §5.2.3, Tables 4-5] The numerical claims are inconsistent with the stated ranges. The abstract claims per-time efficiency '2-20 times higher than the HMC or NUTS algorithm', but Section 5.2.3 instead reports '2-30 times better' for the epilepsy data, and Table 5's MirrorMALA (sparse) on polypharmacy is only 1.53 times NUTS per second, below the claimed lower bound. No standard errors or confidence intervals are reported for the ESS/time ratios; several tables do not state the number of replicates, and the acceptance probabilities and ESS values are estimated from chains run at hand-tuned or fixed scales. The conclusions therefore rest on point estimates whose Monte Carlo uncertainty is unknown.
- [§5.1, §5.2.2] The robustness of the proposed gains is not quantified. Section 5.1 explicitly notes that the efficiency of Mirror-type kernels 'largely relies on the symmetry of the target distribution', and Section 5.2.2 presumes the GLMM posterior is close to Gaussian 'as long as the data size is not too small'. The paper does not provide a sensitivity analysis to the estimated mirror center, the estimated precision matrix, or the quality of the variational approximation used for initialization in Table 5. Such an analysis, for example shifting the mirror center or using a deliberately misspecified sparse precision matrix, would establish whether the GLMM gains are robust or depend on carefully estimated preconditioners.
minor comments (6)
- [§3.1] 'MirrrMALA' should be 'MirrorMALA'.
- [§5.2 and Discussion] 'In this session' should be 'In this section', and the Discussion refers to 'Section 5.4' although the paper has no Section 5.4.
- [Figure 7] The caption contains the ungrammatical phrase 'an averages over d variables'; it should also state explicitly that the plotted quantities are averages over the d variables and over the 5 replicates.
- [Tables 2-5] The units of the E/Time columns should be stated consistently in every table caption (seconds and the scaling factor 10^5) to avoid confusion.
- [Eq. (17)] The formula for Pmala_jump uses tan^{-1} and cot^{-1} without specifying the branch; the authors should state the principal branches or verify that the expression is continuous on the displayed range of epsilon.
- [§2.4] The description of the efficiency measure E should clarify that the CODA spectral-density estimate of ESS is itself random; reporting the number of replicates and the variability of the estimate would strengthen the comparisons.
Circularity Check
Only a minor self-citation in the Pjump derivation; the central MirrorMALA efficiency claims are empirical and not circular.
-
other
[Section 3.3, Eq. (16); reference [9]]
"[9] proved that P^{mirror}_{jump} = P^{rw}_{jump} = 2/π tan^{-1}(2/ε). (16)"
Reference [9] is a submitted manuscript by Jiao, Huang and Yang (2025), and the present paper's corresponding author is Xiyun Jiao, so the equality is imported from the authors' own unpublished work. However, the identity is elementary, is numerically verified in the same section, and is not needed for the central empirical 2-20x efficiency claim; the MirrorMALA Pjump equality (18) is derived from the independently derived MALA expression. The self-citation is therefore minor and not load-bearing.
full rationale
The paper's core claims are empirical: MirrorMALA is a defined composition of the mirror reflection (10) with the MALA proposal (13), and its performance is measured on 1-D targets, Gaussian targets, logistic regression, and GLMMs. The analytical Pjump result (18) follows from substituting θ(t) → -θ(t) in the MALA expression (17), not from assuming the conclusion. Section 5.2.2 explicitly states the Gaussian/sparse-precision presumption ('we presume it to be close as long as the data size is not too small'), and Section 5.1 notes the symmetry dependence of the plain Mirror kernel; these are limitation statements, not circular steps. The only potential circularity is the self-citation of reference [9] for Eq. (16), which is minor. The benchmark confound noted in the skeptic brief (NUTS/HMC use RStan's default diagonal metric while Mirror-type kernels use an estimated sparse precision matrix; RW (sparse) already gives 6.5x NUTS in Table 4) is a comparison-protocol issue rather than a definitional or fitted-input circularity, and no fitted constant is relabeled as a prediction.
Assumptions & free parameters
free parameters (4)
- Scale epsilon for MirrorMALA =
0.5 (or 1)
- Mirror constant c =
1
- Estimated target mean mu* =
burn-in estimate (or variational approximation for polypharmacy)
- Estimated covariance Sigma* =
burn-in estimate (or variational approximation)
assumptions (3)
- domain assumption The target distribution is approximately symmetric about the mirror center mu*, so that reflecting the current state through mu* lands in a region of comparable density.
- domain assumption The posterior distribution of the GLMM is close to Gaussian, so that the sparse precision structure (conditional independence of random effects given fixed effects) is a good approximation and the whitening transform based on the estimated precision matrix yields near-independent coordinates.
- standard math The acceptance ratio in Eq. 1 ensures pi-reversibility and hence correctness for any proposal kernel, including the new Mirror-type kernels.
Cite this review
Pith. "Pith review of Efficient Mirror-type Kernels for the Metropolis-Hastings Algorithm." pith.science (2026). https://pith.science/paper/FXPFDOJG
@misc{pith2026250606660,
author = {Pith},
title = {Pith review of: Efficient Mirror-type Kernels for the Metropolis-Hastings Algorithm},
year = {2026},
howpublished = {\url{https://pith.science/paper/FXPFDOJG}},
note = {Machine review of arXiv:2506.06660}
}
read the original abstract
We propose a new Metropolis-Hastings (MH) kernel by introducing the Mirror move into the Metropolis adjusted Langevin algorithm (MALA). This new kernel uses the strength of one kernel to overcome the shortcoming of the other, and generates proposals that are distant from the current position, but still within the high-density region of the target distribution. The resulting algorithm can be much more efficient than both Mirror and MALA, while stays comparable in terms of computational cost. We demonstrate the advantages of the MirrorMALA kernel using a variety of one-dimensional and multi-dimensional examples. The Mirror and MirrorMALA are both special cases of the Mirror-type kernels, a new suite of efficient MH proposals. We use the Mirror-type kernels, together with a novel method of doing the whitening transformation on high-dimensional random variables, which was inspired by Tan and Nott, to analyse the Bayesian generalized linear mixed models (GLMMs), and obtain the per-time-unit efficiency that is 2--20 times higher than the HMC or NUTS algorithm.
Reference graph
Works this paper leans on
-
[1]
Statistics and Computing 28, 259–275 (2018)
Tan, L.S., Nott, D.J.: Gaussian variational approximation with sparse precision matrices. Statistics and Computing 28, 259–275 (2018)
work page 2018
-
[2]
Metropolis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys.21, 1087–1092 (1953)
work page 1953
-
[3]
Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applica- tions. Biometrika 57, 97–109 (1970)
work page 1970
-
[4]
Springer Texts in Statistics, 274 (2004)
Robert, C.P., Casella, G.: Monte carlo statistical methods. Springer Texts in Statistics, 274 (2004)
work page 2004
-
[5]
Academic press, Cambridge, MA (2014)
Ross, S.M.: Introduction to Probability Models. Academic press, Cambridge, MA (2014)
work page 2014
-
[6]
The Annals of Applied Probability 7, 110–120 (1997)
Roberts, G.O., Gelman, A., Gilks, W.R.: Weak convergence and optimal scaling of random walk Metropolis algorithms. The Annals of Applied Probability 7, 110–120 (1997)
work page 1997
-
[7]
Bayesian statistics 5, 599–608 (1996)
Gelman, A., Roberts, G.O., Gilks, W.R.: Efficient metropolis jumping rules. Bayesian statistics 5, 599–608 (1996)
work page 1996
-
[8]
Yang, Z., Rodr ´ıguez, C.E.: Searching for efficient markov chain monte carlo proposal kernels. Proc. Natl. Acad. Sci. U.S.A. 110(48), 19307–19312 (2013)
work page 2013
Show all 23 references
-
[9]
Statistical Applications in Genetics and Molecular Biology submitted (2025)
Jiao, X., Huang, J., Yang, Z.: Super-efficient markov chain monte carlo algorithms for bayesian inference in population genomics. Statistical Applications in Genetics and Molecular Biology submitted (2025)
2025
-
[10]
Thawornwattana, Y ., Dalquen, D.A., Yang, Z.: Coalescent analysis of phylogenomic data confidently resolves the species relationships in the Anopheles gambiae species complex. Mol. Biol. Evol. 35(10), 2512–2527 (2018) 28
2018
-
[11]
Bernoulli, 341–363 (1996)
Roberts, G.O., Tweedie, R.L.: Exponential convergence of langevin distributions and their discrete approximations. Bernoulli, 341–363 (1996)
1996
-
[12]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) 60(1), 255–268 (1998)
Roberts, G.O., Rosenthal, J.S.: Optimal scaling of discrete approximations to langevin diffusions. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 60(1), 255–268 (1998)
1998
-
[13]
Physics letters B 195(2), 216–222 (1987)
Duane, S., Kennedy, A.D., Pendleton, B.J., Roweth, D.: Hybrid monte carlo. Physics letters B 195(2), 216–222 (1987)
1987
-
[14]
Handbook of markov chain monte carlo 2(11), 2 (2011)
Neal, R.M.: Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo 2(11), 2 (2011)
2011
-
[15]
Bernoulli 19(5A), 1501–1534 (2013)
Beskos, A., Pillai, N., Roberts, G., Sanz-Serna, J.-M., Stuart, A.: Optimal tuning of the hybrid Monte Carlo algorithm. Bernoulli 19(5A), 1501–1534 (2013)
2013
-
[16]
: The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo
Hoffman, M.D., Gelman, A., et al. : The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn. Res. 15(1), 1593–1623 (2014)
2014
-
[17]
Journal of the Royal Statistical Society Series B: Statistical Methodology 73(2), 123–214 (2011)
Girolami, M., Calderhead, B.: Riemann manifold langevin and hamiltonian monte carlo methods. Journal of the Royal Statistical Society Series B: Statistical Methodology 73(2), 123–214 (2011)
2011
-
[18]
Methodology and computing in applied probability 4, 337–357 (2002)
Roberts, G.O., Stramer, O.: Langevin diffusions and metropolis-hastings algorithms. Methodology and computing in applied probability 4, 337–357 (2002)
2002
-
[19]
Statistical Applications in Genetics and Molecular Biology (2010)
Frank, A., Asuncion, A.: Uci machine learning repository. Statistical Applications in Genetics and Molecular Biology (2010)
2010
-
[20]
Jiang, J., Nguyen, T.: Linear and Generalized Linear Mixed Models and Their Applica- tions vol. 1. Springer, New York, NY (2007)
2007
-
[21]
Biometrics, 657–671 (1990)
Thall, P.F., Vail, S.C.: Some covariance models for longitudinal count data with overdispersion. Biometrics, 657–671 (1990)
1990
-
[22]
Journal of the American statistical Association 88(421), 9–25 (1993)
Breslow, N.E., Clayton, D.G.: Approximate inference in generalized linear mixed models. Journal of the American statistical Association 88(421), 9–25 (1993)
1993
-
[23]
John Wiley & Sons, Hoboken, NJ (2013) 29
Hosmer Jr, D.W., Lemeshow, S., Sturdivant, R.X.: Applied Logistic Regression. John Wiley & Sons, Hoboken, NJ (2013) 29
2013
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.