REVIEW 3 major objections 5 minor 45 references
Particle Gibbs without the Gibbs bit
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper introduces m-PGibbs, a collapsed particle-MCMC kernel that marginalizes the parameter update in particle Gibbs, and reports that it explores parameter space more efficiently than PMMH for small particle counts.
desk verdict Neat idea, load-bearing flaw in Algorithm 1's index sampling; send to a careful referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is an auxiliary-variable construction that makes the parameter update collapse. The parameter proposal is split as $q(\theta'|\theta)=\int q(\theta'|u)q(u|\theta)\,du$; with $M$ parameter copies $\theta^{1:M}$ and a flag $l$, the paper forms the joint distribution $\pi(x_{0:T},\theta^{1:M},u,l)\propto\pi_T(x_{0:T}|\theta_l)q(u|\theta_l)\prod_{j\neq l}q(\theta_j|u)$. Conditional on $u$ and $\theta^{1:M}$, the trajectory target is the mixture $\pi_T(x_{0:T}|u,\theta^{1:M})=\sum_l\pi_T(x_{0:T},l|u,\theta^{1:M})$, and the running posterior over $l$ follows the one-step reweighting (12). A conditional sequential Monte Carlo kernel targeting this non-Markovian mixture is run, and $l$ is then sampled from its terminal posterior, with backward-sampling weights obtained by evaluating (12) backwards in time. The object doing the work is the categorical posterior over $l$, which makes the model computationally Markovian and keeps the per-iteration cost at $O(TNM)$.
What would settle it
Run m-PGibbs on a small linear-Gaussian state-space model where the exact posterior is available from a Kalman smoother, using a proposal $q(u|\theta),q(\theta'|u)$ for which the Barker acceptance probability can be computed in closed form; if the empirical stationary distribution differs from the exact posterior, or the acceptance rate does not approach the theoretical Barker probability as $N$ grows, the invariance claim is refuted.
Extended reading notes
Core claim
The central discovery is that the Gibbs sweep in particle Gibbs can be collapsed: instead of alternating a trajectory update and a parameter update, one can run a conditional sequential Monte Carlo kernel on a marginalized target and then sample the parameter index from a closed-form categorical distribution. The paper calls the resulting kernel m-PGibbs and claims it is a valid MCMC kernel for the joint target $\pi_T(x_{0:T},\theta)$, with the parameter effectively updated by a Barker acceptance step in the limit of infinitely many particles rather than by Metropolis-Hastings. Its empirical section shows a state-space example in which m-PGibbs sustains positive acceptance for small $N$ while PMMH does not, and only loses to PMMH for very large $N$, where PMMH approaches its ideal 27% acceptance and m-PGibbs approaches Barker's roughly 18%.
Load-bearing premise
The load-bearing premise is that the conditional sequential Monte Carlo step remains an exact, invariant update for the marginalized target that depends on the whole past trajectory, and that evaluating the one-step weight update (12) backwards gives the correct backward-sampling weights; the paper asserts this without proof, and standard theory does not cover such non-Markovian targets.
Editorial extensions
If this is right
- m-PGibbs offers a parameter-MCMC update whose acceptance is far less sensitive to the number of particles $N$; in the paper's example it keeps exploring at $N$ values where PMMH's acceptance is near zero.
- Because the limiting acceptance is Barker's rather than Metropolis-Hastings, m-PGibbs has a lower asymptotic acceptance ceiling than PMMH, so for very large $N$ a well-tuned PMMH can eventually become more statistically efficient.
- The algorithm inherits the backward-sampling machinery of conditional sequential Monte Carlo, so the stability advantages of CSMC over PIMH for fixed parameters carry over to joint parameter-trajectory inference.
- With the recommended $M=2$, the per-iteration cost is roughly twice that of PMMH, so the small-$N$ advantage is not an artifact of ignoring computational cost.
Reading between the lines
- The same auxiliary-index collapse could in principle be embedded in online or recursive particle parameter samplers, not just in offline MCMC, whenever the parameter proposal admits the $u$-decomposition; the paper does not explore this.
- Because the categorical posterior over $l$ is conjugate to any likelihood model, m-PGibbs extends parameter elimination beyond the conjugate settings where earlier work removed parameters exactly, at the price of carrying $M$ parameter copies.
- Replacing the Barker acceptance by a Metropolis-Hastings step—which the paper notes as possible but does not develop—would combine m-PGibbs's small-$N$ robustness with PMMH's higher asymptotic acceptance ceiling.
- For $M>2$ the method inherits the known limitations of multiproposal samplers in static regimes, so the paper's recommendation $M=2$ is conservative; better multi-try or gradient-informed choices for $q(u|\theta)$ might change the trade-off.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces 'marginalized particle Gibbs' (m-PGibbs), an MCMC method for joint parameter and trajectory inference in Feynman–Kac/state-space models. The construction augments the parameter with an auxiliary variable u and M parameter particles θ^{1:M}, one of which (index l) is tied to the trajectory in the target (6). The paper defines a non-Markovian target for the trajectory obtained by marginalizing over l (Section 3.2), proposes to sample it with a conditional SMC kernel (Algorithm 1), and then resamples l from its posterior (Algorithm 2). It claims m-PGibbs is a valid collapsed Gibbs sampler, that the parameter update is asymptotically a Barker acceptance step, and that the method mixes better than PMMH for small particle numbers. Section 4 gives an acceptance-rate comparison on a Gaussian state-space model with code provided.
Significance. If the invariance claims were established, m-PGibbs would be a useful addition to the particle MCMC toolbox: it avoids the trajectory/parameter Gibbs coupling of PGibbs, uses CSMC rather than PIMH as the underlying 'state-only' kernel, and has a transparent asymptotic interpretation in terms of Barker acceptance. The paper is clearly written, gives code, and honestly states the main technical caveats (no theoretical analysis, single empirical example). However, the correctness of the algorithm as written is compromised by an invalid final-index sampling step in Algorithm 1, and the non-Markovian invariance is asserted rather than proved; these issues currently block acceptance.
major comments (3)
- [§2, Algorithm 1, lines 13–14] Lines 13–14 do not sample the final trajectory index from the normalized weights W_T. For N=2 and W_T=(0.4,0.6), the printed procedure draws k'_T=2 with probability one and accepts with probability 0.4/0.6, giving P(K=1)=1/3 instead of 0.4; in general P(K=1)=1-Σ_{j=2}^N [W_T^j/(1-W_T^1)](W_T^1/W_T^j)=1-(N-1)W_T^1/(1-W_T^1), which is not a valid probability when W_T^1 ≥ 1/N. The required acceptance probability for the conditional proposal over {2,...,N} is 1-W_T^1 (or K should be drawn directly from Cat(W_T)). Because Algorithm 2 in §3.3 and the empirical results in §4 inherit their correctness from Algorithm 1, the validity claims are unsupported as printed.
- [§3.3 and §2] The central validity assertion—that Algorithm 2 defines a valid MCMC kernel—rests on the claim that Algorithm 1 is π-invariant for the non-Markovian marginalized target π_T(x_{0:T} | u, θ^{1:M}). No proof or explicit theorem is provided; the paper says in §3.3 that the incremental term (12) can be evaluated backwards in time, but (12) is a forward recursion for π_{t-1}(l | x_{0:t-1}, u, θ^{1:M}). The backward-sampling weights in Algorithm 1, lines 19–23, use γ_T(x_{0:t}^n, x'_{t+1:T})/γ_t(x_{0:t}^n); for this non-Markovian target the validity of that step requires an argument that the manuscript does not give. Please state and prove a lemma (or give a precise reference to a theorem covering exactly this non-Markovian CSMC construction) establishing invariance of the kernel in Algorithm 1 with the proposal and potential of Section 3.2, and clarify how π_T(l|x,u,θ^{1:M}) is obtained from the backward pass. This is load-bearing because the collapsed-Gibbs derivation of Algorithm 2 reduces to this invariance.
- [§4, Figure 1] The empirical claim that m-PGibbs explores the parameter space more efficiently than PMMH for smaller N is based on a single run per algorithm, raw acceptance rates, no error bars, and no comparison with the closest existing method (parameter-eliminated PGibbs of Wigren et al., 2019), even though Remark 2 connects m-PGibbs to that work. The text notes that m-PGibbs costs about twice as much as PMMH for the same N but the figure does not report acceptance rate per computation or effective sample size; the assertion that higher acceptance rate implies higher ESS (Tierney, 1998) is not generally valid when proposals and costs differ. Please provide repeated runs with Monte Carlo error, cost-adjusted measures (e.g., ESS per second or per iteration), and include the Wigren et al. benchmark or justify its omission.
minor comments (5)
- [Abstract and Introduction] There are typos in the abstract and introduction: 'algortihm', 'conversly', and '£rom' in Section 1.1.
- [§3.4] The sentence 'Both three choices are valid' should read 'All three choices are valid.'
- [Eq. (11) and Algorithm 2 line 5] The notation g_t(x_{0:t}; θ^{1:M}, u, l') is confusing because l' is the output index and the right-hand side of (11) does not depend on l'; please remove l' from the notation or explain its role.
- [References] The reference 'Pitt and and' is incomplete, and the entry 'Finke and Thiery (to appear, 2023)' should be formatted consistently.
- [§4] The calibration value is written 'τ = 0.152'; if this means 0.15^2, please clarify the notation.
Circularity Check
No circularity: m-PGibbs is derived from a declared augmented target via standard CSMC composition; the empirical acceptance rates are measurements, and the main risks are an unproved non-Markovian invariance and a possible indexing bug, not circular reductions.
full rationale
The derivation chain is self-contained. Section 3.1 defines the augmented target (6) from the model and the proposals; Section 3.2 derives the marginalized recursions (9)-(12) algebraically; Section 3.3 applies the standard CSMC kernel to the resulting non-Markovian Feynman-Kac target and adds a categorical index update. The target is not chosen to match the algorithm after the fact, and no fitted constant is relabelled as a prediction. Remark 2 explicitly attributes the parameter-elimination interpretation to Wigren et al. (2019), so the reuse is disclosed rather than smuggled. The self-citations to Corenflos and Finke (2024) and Corenflos and Särkkä (2025) point to localization and multi-proposal discussions and are not load-bearing. The empirical section measures acceptance rates rather than predicting fitted quantities; the proposal scale tau is taken from Chopin and Papaspiliopoulos (2020), not calibrated to the new method. The paper itself notes in Section 5 that a theoretical analysis is 'left for future work', and the invariance of Algorithm 1 for non-Markovian targets is asserted rather than proved; additionally, the final-index accept-reject in Algorithm 1 (lines 13-14) does not obviously implement K~Cat(W_T). These are correctness risks in the algorithm as printed, but they are not circular reductions of the claimed result to its inputs.
Assumptions & free parameters
free parameters (1)
- proposal step size tau =
0.15
assumptions (4)
- domain assumption The transition density p_t(x_t|x0:t-1,theta) and potential function g_t(x0:t;theta) can be evaluated in closed form.
- domain assumption The parameter proposal q(theta'|theta) admits an auxiliary-variable decomposition q(theta'|theta) = integral q(theta'|u) q(u|theta) du.
- standard math The CSMC kernel (Algorithm 1) is invariant for the marginalized target pi_T(x0:T|theta1:M,u) in the non-Markovian setting.
- ad hoc to paper The backward sampling weights for the marginalized CSMC can be computed by evaluating the incremental term (12) backwards.
Cite this review
Pith. "Pith review of Particle Gibbs without the Gibbs bit." pith.science (2026). https://pith.science/paper/Q4U6SLUD
@misc{pith2026250504611,
author = {Pith},
title = {Pith review of: Particle Gibbs without the Gibbs bit},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q4U6SLUD}},
note = {Machine review of arXiv:2505.04611}
}
read the original abstract
Exact parameter and trajectory inference in state-space models is typically achieved by one of two methods: particle marginal Metropolis-Hastings (PMMH) or particle Gibbs (PGibbs). PMMH is a pseudo-marginal algorithm which jointly proposes a new trajectory and parameter, and accepts or rejects both at once. PGibbs instead alternates between sampling from the trajectory, using an algorithm known as conditional sequential Monte Carlo (CSMC) and the parameter in a Hastings-within-Gibbs fashion. While particle independent Metropolis Hastings (PIMH), the parameter-free version of PMMH, is known to be statistically worse than CSMC, PGibbs can induce a slow mixing if the parameter and the state trajectory are very correlated. This has made PMMH the method of choice for many practitioners, despite theory and experiments favouring CSMC over PIMH for the parameter-free problem. In this article, we describe a formulation of PGibbs which bypasses the Gibbs step, essentially marginalizing over the trajectory distribution in a fashion similar to PMMH. This is achieved by considering the implementation of a CSMC algortihm for the state-space model integrated over the joint distribution of the current parameter and the parameter proposal. We illustrate the benefits of method on a simple example known to be challenging for PMMH.
Figures
Reference graph
Works this paper leans on
-
[1]
Agrawal, S., Vats, D., Łatuszyński, K., and Roberts, G. O. (2023). Optimal scaling of MCMC beyond Metropolis . Advances in Applied Probability , 55(2):492--509
work page 2023
-
[2]
Andrieu, C., Doucet, A., and Holenstein, R. (2010). Particle M arkov chain M onte C arlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 72(3):269--342. With discussion
work page 2010
-
[3]
Andrieu, C., Lee, A., and Vihola, M. (2018). Uniform ergodicity of the iterated conditional SMC and geometric ergodicity of particle Gibbs samplers. Bernoulli , 24(2):842--872
work page 2018
-
[4]
Andrieu, C. and Roberts, G. O. (2009). The pseudo-marginal approach for efficient Monte Carlo computations. The Annals of Statistics , 37(2):697 -- 725
work page 2009
-
[5]
Barker, A. A. (1965). M onte C arlo calculations of the radial distribution functions for a proton--electron plasma. Australian Journal of Physics , 18(2):119--134
work page 1965
-
[6]
Cappé, O., Moulines, E., and Rydén, T. (2005). Inference in hidden M arkov models . Springer Series in Statistics. Springer
work page 2005
-
[7]
Chopin, N., Jacob, P. E., and Papaspiliopoulos, O. (2013). SMC 2 : an efficient algorithm for sequential analysis of state space models. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 75(3):397--426
work page 2013
-
[8]
Chopin, N. and Papaspiliopoulos, O. (2020). An Introduction to Sequential M onte C arlo . Springer
work page 2020
Show all 45 references
-
[9]
and Singh, S
Chopin, N. and Singh, S. S. (2015). On particle G ibbs sampling. Bernoulli , 21(3):1855--1883
2015
-
[10]
and Finke, A
Corenflos, A. and Finke, A. (2024). Particle-MALA and Particle-mGRAD: Gradient-based MCMC methods for high-dimensional state-space models . arXiv preprint arXiv:2401.14868
2024 arXiv
-
[11]
and S \"a rkk \"a , S
Corenflos, A. and S \"a rkk \"a , S. (2025). Auxiliary MCMC samplers for parallelisable inference in high-dimensional latent dynamical systems . Electronic Journal of Statistics , 19(1):1370 -- 1424
2025
-
[12]
and M \'i guez, J
Crisan, D. and M \'i guez, J. (2018). Nested particle filters for online parameter estimation in discrete-time state-space Markov models . Bernoulli , 24(4A):3039 -- 3086
2018
-
[13]
Del Moral, P. (2004). F eynman- K ac Formulae: Genealogical and Interacting Particle Systems with Applications . Springer
2004
-
[14]
K., Deligiannidis, G., and Kohn, R
Doucet, A., Pitt, M. K., Deligiannidis, G., and Kohn, R. (2015). Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator . Biometrika , 102(2):295--313
2015
-
[15]
Finke, A. (2015). On Extended State-Space Constructions for M onte C arlo Methods . PhD thesis, Department of Statistics, University of Warwick, UK
2015
-
[16]
and Thiery, A
Finke, A. and Thiery, A. H. (to appear, 2023). Conditional sequential Monte Carlo in high dimensions. Annals of Statistics
2023
-
[17]
Gauraha, N. (2020). A Note on Particle Gibbs Method and its Extensions and Variants
2020
-
[18]
and Geman, D
Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on pattern analysis and machine intelligence , PAMI-6(6):721--741
1984
-
[19]
J., Salmond, D
Gordon, N. J., Salmond, D. J., and Smith, A. F. M. (1993). Novel approach to nonlinear/non- G aussian B ayesian state estimation. IEE Proceedings F, Radar and Signal Processing , 140(2):107--113
1993
-
[20]
Haario, H., Laine, M., Mira, A., and Saksman, E. (2006). DRAM : efficient adaptive MCMC . Statistics and computing , 16(4):339--354
2006
-
[21]
Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika , 57(1):97--109
1970
-
[22]
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME, Journal of Basic Engineering , 82(1):35--45
1960
-
[23]
S., and Vihola, M
Karjalainen, J., Lee, A., Singh, S. S., and Vihola, M. (2023). Mixing time of the conditional backward sampling particle filter. arXiv preprint arXiv:2312.17572
2023
-
[24]
S., and Vihola, M
Karppinen, S., Singh, S. S., and Vihola, M. (2023). Conditional particle filters with bridge backward sampling. Journal of Computational and Graphical Statistics , 0(0):1--15
2023
-
[25]
and Roberts, G
atuszy \'n ski, K. and Roberts, G. O. (2013). CLTs and asymptotic variance of time-sampled M arkov chains. Methodology and Computing in Applied Probability , 15:237--247
2013
-
[26]
S., and Vihola, M
Lee, A., Singh, S. S., and Vihola, M. (2020). Coupled conditional backward sampling particle filter. Annals of Statistics , 48(5):3066--3089
2020
-
[27]
and Schön, T
Lindsten, F. and Schön, T. B. (2013). Backward Simulation Methods for Monte Carlo Statistical Inference
2013
-
[28]
Liu, J. S. (1994). The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem . Journal of the American Statistical Association , 89(427):958--966
1994
-
[29]
Malory, S. (2021). Bayesian inference for stochastic processes . PhD thesis, Lancaster University
2021
-
[30]
Martino, L. (2018). A review of multiple try MCMC algorithms for signal processing. Digital Signal Processing , 75:134–152
2018
-
[31]
A., Lindsten, F., and Sch\"on, T
Naesseth, C. A., Lindsten, F., and Sch\"on, T. B. (2019). Elements of sequential Monte Carlo . arXiv preprint arXiv:1903.04797
2019 arXiv
-
[32]
and Ryden, T
Olsson, J. and Ryden, T. (2011). Rao-blackwellization of particle markov chain monte carlo methods using forward filtering backward sampling. IEEE Transactions on Signal Processing , 59(10):4606--4619
2011
-
[33]
O., and Sk \"o ld, M
Papaspiliopoulos, O., Roberts, G. O., and Sk \"o ld, M. (2007). A General Framework for the Parametrization of Hierarchical Models . Statistical Science , 22(1):59 -- 73
2007
-
[34]
Pitt, M. K. and and, N. S. (1999). Filtering via simulation: Auxiliary particle filters. Journal of the American Statistical Association , 94(446):590--599
1999
-
[35]
and Zanella, G
Pozza, F. and Zanella, G. (2024). On the fundamental limitations of multiproposal Markov chain Monte Carlo algorithms
2024
-
[36]
and Svensson, L
S \"a rkk \"a , S. and Svensson, L. (2023). Bayesian filtering and smoothing , volume 17. Cambridge university press
2023
-
[37]
Sherlock, C., Thiery, A., and Golightly, A. (2015). Efficiency of delayed-acceptance random walk M etropolis algorithms. arXiv preprint arXiv:1506.08155
2015 arXiv
-
[38]
S., Lindsten, F., and Moulines, E
Singh, S. S., Lindsten, F., and Moulines, E. (2017). Blocking strategies and stability of particle G ibbs samplers. Biometrika , 104(4):953--969
2017
-
[39]
Tierney, L. (1998). A note on M etropolis-- H astings kernels for general state spaces. Annals of applied probability , pages 1--9
1998
-
[40]
Titsias, M. K. and Papaspiliopoulos, O. (2018). Auxiliary gradient-based sampling algorithms. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 80(4):749--767
2018
-
[41]
Tjelmeland, H. (2004). Using all M etropolis-- H astings proposals to estimate mean values. preprint 4/2004, Norwegian University of Science and Technology, Trondheim, Norway
2004
-
[42]
B., Łatuszyński, K., and Roberts, G
Vats, D., Gonçalves, F. B., Łatuszyński, K., and Roberts, G. O. (2021). Efficient Bernoulli factory Markov chain Monte Carlo for intractable posteriors . Biometrika , 109(2):369--385
2021
-
[43]
Vihola, M. (2012). Robust adaptive Metropolis algorithm with coerced acceptance rate . Statistics and computing , 22:997--1008
2012
-
[44]
Whiteley, N. (2010). Discussion on particle M arkov chain M onte C arlo methods. Journal of the Royal Statistical Society: Series B , 72(3):306--307
2010
-
[45]
S., Murray, L., and Lindsten, F
Wigren, A., Risuleo, R. S., Murray, L., and Lindsten, F. (2019). Parameter elimination in particle Gibbs sampling. Advances in Neural Information Processing Systems , 32
2019
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.