REVIEW 3 major objections 5 minor 28 references
The occlusion process: improving sampler performance with parallel computation and variational approximation
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Wrapping an MCMC sampler with a parallel rejection layer that swaps autocorrelated chain states for independent region-restricted draws yields an unbiased estimator with lower variance and the chain's convergence guarantees.
desk verdict A genuinely new variance-reduction construction with careful inheritance theorems, but the paper's main applied claim overreaches: Algorithm 1 does not realize the kernel that the variance formulas analyze. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the occlusion kernel $K_{\mathrm{occ}}$ (equation 5), which extends the chain's kernel $K$ with a per-region success indicator $S_t$ of probability $\alpha(\rho(X_t))$ and an independent replacement sample $Y_t$ drawn from $P$ restricted to the region $\rho(X_t)$; the estimator (6) then averages $f$ over whichever variable, chain state or occluding draw, is active at each time. The variance analysis is carried by the resolution operator $\overrightarrow{P}f(x) := \sum_{i=1}^{R} \mu_i \mathbf{1}\{x \in X_i\}$, the piecewise-constant function of region means: it splits $\mathrm{Var}_P(f)$ into between-region variance $\mathrm{Var}_P(\overrightarrow{P}f)$ and within-region variance $\mathrm{Var}_P(\overleftarrow{P}f)$, and occlusion replaces the autocorrelated within-region variation with independent region-restricted draws. The region-restricted draws come from a rejection sampler with variational proposal $Q$, with the regions defined as level sets of the Radon-Nikodym derivative $\mathrm{d}\widetilde{P}/\mathrm{d}\widetilde{Q}$ and per-region acceptance probability $(Z_P/Z_Q)(1/C_i)P(X_i)$; spare threads run these samplers concurrently with the base chain, which is why the wall-clock cost does not rise.
What would settle it
Run the occlusion process on a target where $Q$ is deliberately chosen so that $(Z_P/Z_Q)(1/C_i)P(X_i)$ predicts near-certain acceptance on the regions the chain visits, with a functional $f$ that is piecewise constant on those regions, and measure the variance ratio $\mathrm{Var}(\hat{\mu}_{\mathrm{occ}})/\mathrm{Var}(\hat{\mu}_{\mathrm{chain}})$. Proposition 5 and equation (8) predict a ratio at or below 1; a measured ratio above 1 in this regime would falsify the variance-reduction claim.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that an MCMC estimator can be improved by a layer of occlusion: for a partition of the state space into regions, whenever a parallel rejection sampler built from a variational approximation $Q$ succeeds in drawing a sample from the target $P$ restricted to the region the chain has just entered, that independent sample replaces the chain's state in the Monte Carlo average. Proposition 7 states that the resulting estimator is unbiased and gives its exact variance as the stratified-proportional-allocation variance plus a covariance term for the residual autocorrelation of partially occluded states (equation 8). The occluded process does not merely keep its consistency for free: Theorems 10, 12, and 20 show it inherits a law of large numbers, convergence in normed function spaces, and geometric ergodicity from the base chain, and Corollary 21 adds a central limit theorem under reversibility. The paper also sets limits on the idea: Fact 2 shows that full occlusion can raise variance relative to the plain chain when the kernel has negative spectral part, and it leaves open whether that can happen with a partition into more than one region.
Load-bearing premise
The load-bearing premise is that a variational distribution $Q$ can be found whose rejection-sampler acceptance probability $(Z_P/Z_Q)(1/C_i)P(X_i)$ is non-negligible on the regions the chain actually visits; when $Q$ misses that mass the occlusion rate collapses and the estimator degenerates to the plain chain, as the authors' own low-temperature Ising run at $N=100$ shows.
Editorial extensions
If this is right
- Any existing $P$-invariant Markov chain can be wrapped in the occlusion layer, and the standard convergence toolkit (LLN, normed-space convergence, geometric ergodicity, CLT under reversibility) transfers unchanged, so users keep the guarantees they already paid for.
- On a threaded machine the occluded estimator runs in the same wall-clock time as the plain chain, so the variance reduction is essentially free whenever a reasonable variational approximation $Q$ is available.
- Variance reduction is guaranteed in two regimes: functionals that are piecewise constant on the regions, and functionals such as the Ising magnetisation (odd function, symmetric target, symmetric regions) averaged over a kernel with positive spectrum; the high-temperature Ising experiments instantiate the second.
- When the base chain already produces anticorrelated states, as Wolff's algorithm does at low temperature, occlusion's decorrelation can raise variance instead of lowering it, so the method should be aimed at positively autocorrelated samplers.
- The occluded estimator is not universally better than the plain chain: Fact 2 exhibits a one-region counterexample, and it remains open whether any multi-region partition can make the fully occluded estimator lose to the chain.
Reading between the lines
- My reading: the paper's variance formula (8) suggests a principled region-selection rule analogous to Neyman allocation, choosing the level-set constants $C_i$ to balance the acceptance probability $(Z_P/Z_Q)(1/C_i)P(X_i)$ against the within-region variances $\sigma_i^2$; the authors leave such tuning to future work.
- My reading: the open multi-region counterexample question could plausibly be settled spectrally; if $\mathrm{Var}(\hat{\mu}_{\mathrm{ideal}}) \leq \mathrm{Var}(\hat{\mu}_{\mathrm{chain}})$ holds whenever $K$ has nonnegative spectrum, then the positivity condition of Proposition 5 is not just sufficient but necessary for guaranteed improvement.
- My reading: the method's weakest practical link is the fixed, pre-trained $Q$; an online variant that re-fits $Q$ from accepted occluding samples as the chain runs would directly attack the near-zero occlusion proportions reported for the low-temperature Ising $N=100$ case.
- My reading: the chain states discarded by occlusion could be recycled rather than erased, for instance as control variates or Rao-Blackwellized combinations of $X_t$ and $Y_t$, converting the method from sample replacement into information reuse.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes the occlusion process, a wrapper around a P-invariant Markov chain that replaces some chain states with independent samples from P restricted to the region currently visited, obtained by rejection sampling with a variational proposal Q. The paper proves unbiasedness and a variance decomposition for the idealized estimator (Proposition 7), derives inheritance of a Law of Large Numbers, convergence in normed spaces, geometric ergodicity, and a CLT (Theorems 10, 12, 17, 20, Corollary 21), and presents a threaded implementation (Algorithm 1) claimed to run in the same wall-clock time as the underlying sampler. Two experiments, on a bimodal Gaussian mixture and on the Ising model over stochastic-block-model graphs, illustrate variance reduction, and the paper honestly reports cases where the method fails, such as the low-temperature N=100 Ising setting and the Wolff-algorithm comparison.
Significance. The theoretical core around the idealized kernel (5) is a genuine contribution: it connects MCMC variance reduction to stratified sampling and variational inference, and the inheritance theorems are cleanly proved through the operator identity K_occ^t g = K^t f_alpha. The paper is also honest about counterexamples and open questions, which strengthens credibility. However, the implementation in Algorithm 1 does not actually realize the kernel (5), so the variance and inheritance results do not currently cover the estimator that is simulated. This gap is load-bearing for the central claim that the method reduces variance while inheriting the underlying chain's properties.
major comments (3)
- [Section 4.2.1, Algorithm 1; Section 2.2.2 Eq. (5); Proposition 7; Section 7.1.4] The theory defines K_occ with independent Bernoulli indicators S_t conditional on the region sequence, and the proof of Proposition 7 explicitly uses the conditional independence {S_t} independent of {S_t'} given {rho(X_t)}. Algorithm 1 instead allocates, for each region i, exactly min{N_i, T_i} occlusions by sampling a uniform subset of the T_i visits. Conditional on T_i and N_i, the joint law of the indicators is hypergeometric, not independent Bernoulli: for example with T_i=2 and N_i=1, the event that both times are occluded has probability 0, whereas independent Bernoulli draws give probability 1/4. Moreover, alpha(i) := E[min{1,N_i/T_i}] is data-dependent, whereas (5) requires fixed alpha; the simulated process is therefore not a time-homogeneous Markov chain of the form K_occ. Consequently the variance formula (8), and all variance-reduction conclusions derived from it (Proposition 5 and Section 5.2.5), are not proven for the estimator actually simulated. The empirical variance reductions may survive, but the theoretical support for the implementation is missing.
- [Section 5.2.5] The claim that the high-temperature Ising experiment satisfies condition 2 of Proposition 5 is not adequately supported. First, the paper infers that the Markov kernel K is positive from positive lag-1 autocorrelations of the magnetisation functional; positivity of one autocovariance at lag 1 does not imply positivity of the spectrum of K on L^2(P), which is what Proposition 5 requires. Second, the argument that alpha is identically 1 is inferred from occlusion proportions in Figure 7, but proportions close to 1 do not imply alpha(i)=1 exactly for every region in every replication. The theoretical explanation of the variance reduction is therefore not established, even setting aside the mismatch between the theory and Algorithm 1.
- [Section 5.2.3 and Section 5.2.4] The empirical proof-of-concept is weakened by the use of oracle information in constructing the variational regions. The clusters V_i are set to the true SBM communities, while the text acknowledges that 'in practice we will not have access to the actual latent cluster structure' (Section 5.2.3). No experiment uses an estimated clustering, and the threshold constants C_1, C_2 and the parameters epsilon and beta_tilde are chosen with per-temperature values, so the claim in Section 5.2.4 that the parameters were 'not tuned' is hard to reconcile with the setup. The paper's own Section 5.2.4 also reports occlusion proportions near zero in the low-temperature N=100 case; this is an explicit limitation that should be discussed as restricting the method's domain rather than presented only as a tuning issue.
minor comments (5)
- [Section 1.1] The notation P(A) is used both for the probability of a set A under the target measure and, a few lines later, for the power set of A. Please use a different symbol, such as 2^A, for the power set.
- [Section 5.2.4 and Figure 7] The text refers to 'the top right graph' for both the lag-1 autocorrelation coefficients and the occlusion proportions, but the figure caption and surrounding discussion suggest these are different panels (top left and top right). Please align the textual references with the figure.
- [Section 7.2.6] The proof of Theorem 17 cites '[section 6.3.1]', which does not exist in the manuscript; this cross-reference should be replaced with the actual derivation or a specific equation number.
- [Acknowledgements] There is a typo: 'We would like to that Francois Perron' should read 'We would like to thank Francois Perron'.
- [Abstract and Section 4.2] The claim of 'no additional time-complexity' is only true in an asymptotic wall-clock sense with a sufficient number of threads and comparable per-step costs. As written, it could be misread as no additional computational work; please qualify the statement.
Circularity Check
The occlusion process paper's core theoretical results are derived from explicit definitions rather than assumed; the noted theory–implementation mismatch is a correctness gap, not circularity.
full rationale
The central claims—unbiasedness of the occlusion estimator (Proposition 7), the variance decomposition (8), and the inheritance theorems (LLN, normed convergence, geometric ergodicity)—are proved from the explicit kernel Kocc in (5) and the invariant measure Pocc in (7), with no fitted parameter renamed as a prediction and no assumption of the conclusion. The proofs invoke external results (Kac, Roberts–Rosenthal, Douc et al., Gallegos-Herrada et al.) whose assumptions do not include the occlusion-process claims. There are no load-bearing self-citations: the authors cite related work (Kick–Kac samplers, stratified sampling) but do not justify a central premise by their own prior results. The empirical section calibrates region boundaries C1, C2 from a pilot chain on the same target distribution, and Section 5.2.5 verifies variance-reduction conditions on the same runs used for the variance comparison; this is an in-sample evaluation limitation, but it is not a circular derivation because the empirical comparisons are not presented as first-principles predictions derived from the fitted values. The more serious issue—that Algorithm 1's post-hoc uniform allocation gives a hypergeometric joint law of occlusions rather than the independent Bernoulli structure assumed in the proof of Proposition 7—means the theoretical variance formula is not proven for the simulated estimator. This is a correctness gap between theory and implementation, not a circularity in the derivation chain. Under the stated standard, no circular step can be exhibited with a specific reduction of a result to its own input.
Assumptions & free parameters
free parameters (5)
- C1, C2 region thresholds (Ising) =
C1 = median, C2 = maximum of dP/dQ over a 20-second pilot Wolff chain
- epsilon in coarse-grained Ising variational family =
0.1 (beta=1, low temperature), 0.9 (beta=0.01, high temperature)
- beta_tilde scaling =
0.5 * beta
- Number of regions R (Ising) =
3 (R=2 in Gaussian example)
- True SBM community labels for variational clusters =
The true communities of the stochastic block model used to generate the graph
assumptions (5)
- domain assumption There exists a P-invariant Markov kernel K generating the chain {X_t}, and we can simulate it.
- domain assumption The variational distribution Q has support covering the visited regions and the Radon-Nikodym derivative dP/dQ is bounded on each region by the chosen constants C_i.
- domain assumption A threaded computer with at least 1 + C_rej cores is available so that rejection threads can run concurrently with the chain thread.
- domain assumption For the Ising variance-reduction claim, the Markov kernel K is positive (nonnegative spectrum) when applied to the magnetisation functional.
- domain assumption The target P is even and the magnetisation f is odd, and the regions satisfy X_i = -X_i.
Cite this review
Pith. "Pith review of The occlusion process: improving sampler performance with parallel computation and variational approximation." pith.science (2026). https://pith.science/paper/QSASU2YQ
@misc{pith2026241111983,
author = {Pith},
title = {Pith review of: The occlusion process: improving sampler performance with parallel computation and variational approximation},
year = {2026},
howpublished = {\url{https://pith.science/paper/QSASU2YQ}},
note = {Machine review of arXiv:2411.11983}
}
read the original abstract
Autocorrelations in MCMC chains increase the variance of the estimators they produce. We propose the occlusion process to mitigate this problem. It is a process that sits upon an existing MCMC sampler, and occasionally replaces its samples with ones that are decorrelated from the chain. We show that this process inherits many desirable properties from the underlying MCMC sampler, such as a Law of Large Numbers, convergence in a normed function space, and geometric ergodicity, to name a few. We show how to simulate the occlusion process at no additional time-complexity to the underlying MCMC chain. This requires a threaded computer, and a variational approximation to the target distribution. We demonstrate empirically the occlusion process' decorrelation and variance reduction capabilities on two target distributions. The first is a bimodal Gaussian mixture model in 1d and 100d. The second is the Ising model on an arbitrary graph, for which we propose a novel variational distribution.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
William G. Cochran. Sampling Techniques. John Wiley and Sons, 1977
work page 1977
-
[2]
Blei, Alp Kucukelbir, and Jon D
David M. Blei, Alp Kucukelbir, and Jon D. McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association, 112 0 (518): 0 859--877, 2017
2017
-
[3]
Matthew D. Parno and Youssef M. Marzouk. Transport map accelerated markov chain monte carlo. SIAM/ASA Journal on Uncertainty Quantification, 6 0 (2): 0 645--682, 2018. doi:10.1137/17M1134640
-
[4]
James Brofos, Marylou Gabri\'e, Marcus A. Brubaker, and Roy R. Lederman. Adaptation of the independent metropolis-hastings sampler with normalizing flow proposals. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, pages 5949--5986. PMLR, 2022
work page 2022
-
[5]
Pigeons.jl: Distributed sampling from intractable distributions, 2023
Nikola Surjanovic, Miguel Biron-Lattes, Paul Tiede, Saifuddin Syed, Trevor Campbell, and Alexandre Bouchard-C\^ot\'e'. Pigeons.jl: Distributed sampling from intractable distributions, 2023. URL https://arxiv.org/abs/2308.09769
arXiv 2023
-
[6]
An adaptive-mcmc scheme for setting trajectory lengths in hamiltonian monte carlo
Matthew Hoffman, Alexey Radul, and Pavel Sountsov. An adaptive-mcmc scheme for setting trajectory lengths in hamiltonian monte carlo. In Arindam Banerjee and Kenji Fukumizu, editors, Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 130 of Proceedings of Machine Learning Research, pages 3907--3915. PMLR, 13...
work page 2021
-
[7]
Randal Douc, Alain Durmus, Aurélien Enfroy, and Jimmy Olsson. Boost your favorite markov chain monte carlo sampler using kac's theorem: the kick-kac teleportation algorithm, 2023. URL https://arxiv.org/abs/2201.05002
work page Pith review arXiv 2023
-
[8]
On the notion of recurrence in discrete stochastic processes
Mark Kac. On the notion of recurrence in discrete stochastic processes . Bulletin of the American Mathematical Society, 53 0 (10): 0 1002 -- 1010, 1947
work page 1947
Show all 28 references
-
[9]
Art B. Owen. Monte Carlo theory, methods and examples. https://artowen.su.domains/mc/, 2013
2013
-
[10]
On the Two Different Aspects of the Representative Method : The Method of Stratified Sampling and the Method of Purposive Selection
Jerzy Neyman. On the Two Different Aspects of the Representative Method : The Method of Stratified Sampling and the Method of Purposive Selection . Journal of the Royal Statistical Society, 97 0 (4): 0 558--606, 12 1934. ISSN 0952-8385
1934
-
[11]
Neal and Jeffrey S
Radford M. Neal and Jeffrey S. Rosenthal. Efficiency of reversible mcmc methods: elementary derivations and applications to composite methods, 2024. URL https://arxiv.org/abs/2305.18268
2024 arXiv
-
[12]
Siran Liu, Petros Dellaportas, and Michalis K. Titsias. Can independent metropolis beat crude monte carlo?, 2024. URL https://arxiv.org/abs/2406.17699
2024 arXiv
-
[13]
Kung Sik Chan and Charles J. Geyer. Discussion: Markov chains for exploring posterior distributions. The Annals of Statistics, 22 0 (4): 0 1747--1758, 1994
1994
-
[14]
Geometric Ergodicity and Hybrid Markov Chains
Gareth Roberts and Jeffrey Rosenthal. Geometric Ergodicity and Hybrid Markov Chains . Electronic Communications in Probability, 2: 0 13 -- 25, 1997
1997
-
[15]
Markov chains for exploring posterior distributions
Luke Tierney. Markov chains for exploring posterior distributions. The Annals of Statistics, 22 0 (4): 0 1701--1728, 1994
1994
-
[16]
Estimating epidemic exponential growth rate and basic reproduction number
Junling Ma. Estimating epidemic exponential growth rate and basic reproduction number. Infectious Disease Modelling, 5: 0 129--141, 2020. ISSN 2468-0427
2020
-
[17]
Roberts and Jeffrey S
Gareth O. Roberts and Jeffrey S. Rosenthal. Optimal scaling for various Metropolis-Hastings algorithms . Statistical Science, 16 0 (4): 0 351 -- 367, 2001
2001
-
[18]
Algebraic and group structure for bipartite anisotropic ising model on a non-local basis
Francisco Delgado. Algebraic and group structure for bipartite anisotropic ising model on a non-local basis. International Journal of Quantum Information, 13 0 (07): 0 1550055, 2015
2015
-
[19]
Efficiently learning ising models on arbitrary graphs
Guy Bresler. Efficiently learning ising models on arbitrary graphs. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, page 771–782. Association for Computing Machinery, 2015
2015
-
[20]
Ising-like models on arbitrary graphs: The hadamard way
R\'emy Mosseri. Ising-like models on arbitrary graphs: The hadamard way. Phys. Rev. E, 91: 0 012142, Jan 2015
2015
-
[21]
Metropolis, Arianna W
N. Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, A. H. Teller, and Edward Teller. Equation of state calculations by fast computing machines. Journal of Chemical Physics, 21: 0 1087--1092, 1953
1953
-
[22]
Collective monte carlo updating for spin systems
Ulli Wolff. Collective monte carlo updating for spin systems. Phys. Rev. Lett., 62: 0 361--364, Jan 1989
1989
-
[23]
A review of stochastic block models and extensions for graph clustering
Clement Lee and Darren Wilkinson. A review of stochastic block models and extensions for graph clustering. Applied Network Science, 4, 12 2019
2019
-
[24]
Efficient exact sampling from the ising model using swendsen-wang
Mark Huber. Efficient exact sampling from the ising model using swendsen-wang. In Soda, pages 921--922, 1999
1999
-
[25]
S. C. Kou, Qing Zhou, and Wing Hung Wong. Equi-energy sampler with applications in statistical inference and statistical mechanics . The Annals of Statistics, 34 0 (4): 0 1581 -- 1619, 2006
2006
-
[26]
The squeeze method for generating gamma variates
George Marsaglia. The squeeze method for generating gamma variates. Computers & Mathematics with Applications, 3 0 (4): 0 321--325, January 1977. ISSN 0898-1221
1977
-
[27]
Gallegos-Herrada, David Ledvinka, and Jeffrey S
Marco A. Gallegos-Herrada, David Ledvinka, and Jeffrey S. Rosenthal. Equivalences of Geometric Ergodicity of Markov Chains . Journal of Theoretical Probability, 37 0 (2): 0 1230--1256, June 2024
2024
-
[28]
Riesz's lemma and orthogonality in normed spaces
Kazuo Hashimoto, Gen Nakamura, and Shinnosuke Oharu. Riesz's lemma and orthogonality in normed spaces . Hiroshima Mathematical Journal, 16 0 (2): 0 279 -- 304, 1986
1986
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.