Poisson Approximate Likelihood versus the block particle filter for a spatiotemporal measles model

Edward L. Ionides; Kunyang He; Yize Hao

REVIEW 3 major objections 6 minor 20 references

Poisson Approximate Likelihood versus the block particle filter for a spatiotemporal measles model

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper shows that the Poisson approximate likelihood filter used on a 40-town measles model reports log-likelihood estimates with positive Monte Carlo bias, so its reported advantage over a block particle filter and ARMA benchmarks…

desk verdict Solid, reproducible bias demonstration against the lookahead PAL filter on the 40-town measles model, but the abstract's comparative claim against the block particle filter overreaches because it compares different stochastic models. read the letter →

arxiv 2507.09121 v1 pith:QTGX2Y2A submitted 2025-07-12 stat.ME

Kunyang He , Yize Hao , Edward L. Ionides This is my paper · ORCID

classification stat.ME MSC 62M0565C05

keywords PoissonapproximatelikelihoodlookaheadfilterblockparticleMonteCarlobiaslog-likelihoodestimationspatiotemporalmeaslesmodelpartiallyobservedMarkovprocessproperscoringrule

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper revisits the Poisson approximate likelihood (PAL) filter applied to a 40-town measles metapopulation model and argues that the method's reported success is an artifact of positive Monte Carlo bias. The lookahead variant of PAL, called PALL, estimates log-likelihood values that fall as the number of particles grows: in the real-data comparison, increasing particles from 5,000 to 100,000 lowers the estimate by about 834 log units. On simulated data with known likelihood, PALL overshoots the true value while the vanilla version falls slightly below it, as an approximate filter should. The paper concludes that PAL does not outperform a standard block particle filter on this problem, and that the published comparison table does not represent PAL's true capabilities.

What carries the argument

The central machinery is the lookahead filter, a sequential Monte Carlo method whose one-step predictions may use data at or after the observation time. Two load-bearing ideas carry the argument. The first is the proper-scoring-rule property of log-likelihood: a genuine forecasting filter cannot beat the exact prediction distribution on average, so simulated data with a known likelihood provide a ground truth against which biased filters can be detected. The second is a two-point toy example of self-normalizing resampling in the lookahead mechanism, showing that with a fixed number of particles most resampled swarms overstate the likelihood while rare swarms carry enormous weights, so the likelihood estimate is positively biased. The paper then uses controlled experiments that vary the filter (PALL, vanilla PAL, block particle filter), particle count, number of towns, and spatial coupling, with the 5,000-versus-100,000 particle comparison isolating the bias in the published 40-town setting.

What would settle it

Use a modified lookahead filter that is unbiased for the likelihood (for instance, by replacing self-normalization with a correct weighting scheme) on the same 40-town measles data; if the log-likelihood estimate no longer decreases when particles are increased from 5,000 to 100,000 and still beats the block particle filter, the paper's central claim is wrong.

Watch

Extended reading notes

Core claim

The paper's central claim is that the lookahead implementation of PAL inflates its own log-likelihood estimates, so the strong showing reported in the original measles analysis is not trustworthy. The authors establish this by two routes: on datasets simulated from the model, PALL reports log-likelihoods above the known true value, and on both simulated and real data, increasing Monte Carlo effort moves the PALL estimate downward, a behavior inconsistent with unbiased estimation. The most direct evidence is the change from -64014.93 to -64848.76 (about 834 log units) when particles increase from 5,000 to 100,000 for the 40-town data. They further show the bias grows roughly linearly with the number of towns, and that a block particle filter reproduces or exceeds PALL's performance without the bias.

Load-bearing premise

The load-bearing premise is that the positive bias measured in the 40-town measles experiments is caused by the self-normalizing lookahead resampling illustrated in the two-point toy example, and not by the Poisson approximation itself or by implementation details specific to this model.

Editorial extensions

If this is right

The log-likelihood values in the original Table 3 for the measles application should not be treated as reliable evidence for PAL's usefulness on high-dimensional problems.
A plain block particle filter, which applies to a broader class of models, is a sufficient and less biased tool for likelihood evaluation on this metapopulation measles model.
Because the bias scales roughly linearly with the number of towns, conclusions drawn from PAL on larger spatial systems could be even more overstated.
Researchers using lookahead filters for model comparison should check whether log-likelihood estimates move with Monte Carlo effort before interpreting high values as model support.
Vanilla PAL without lookahead fails to beat the ARMA benchmark in these experiments, so the PAL approximation itself does not rescue the high-dimensional measles analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper's specific case, the same self-normalizing-lookahead mechanism could inflate likelihood estimates in other lookahead sequential Monte Carlo methods, making the warning a general one for filter developers.
A practical diagnostic follows directly: report log-likelihood as a function of particle count for any lookahead filter, since a decreasing curve is a red flag for positive bias even when the true likelihood is unknown.
Extending the two-point toy example to a broader class of weight distributions might yield a quantitative bias formula in terms of particle count and weight variance, which could let practitioners correct the estimate.
On the modeling side, these results shift the favorable evidence for the measles models back onto the block particle filter, so the overdispersion and coupling parameters from the PAL analysis should be re-estimated with an unbiased filter before being used.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful core of this paper is the bias finding, and it is solid. E14 versus E15 is the clincher: on the WWR code and parameters, raising the particle count from 5,000 to 100,000 drops the log-likelihood estimate by 834 log units, with a jackknife SE that makes the decrease unambiguous. That is exactly the signature of positive Monte Carlo bias, and it directly undermines the credibility of Table 3 in Whitehouse et al. (2023). The simulated-data experiments with known ground truth (E1, E8) reinforce the point. The paper also deserves credit for shipping code and data on Zenodo and for using the WWR code to generate the PALL numbers, which takes away the usual excuse that the critique is based on a strawman implementation.

The soft spots are real but they do not touch the central bias result. The abstract's claim that 'PAL does not outperform a block particle filter' is not actually established by the experiments shown. E11 uses the BPF with an Euler/Gaussian model and re-optimized case C parameters; E14/E15 use the discrete/binomial case A model with WWR's published parameters. Those are different likelihood functions. Re-optimizing parameters does not make them commensurable. The same issue contaminates E8–E10, where BPF filters the correct C2 coupling while PAL filters the misspecified C1, mixing model error with algorithmic performance. A like-for-like comparison is available in the paper—E13 (PALV at high particle count on case A) gives -74137.62, well below E11's -67761.82—so the comparative headline should have been built on that, or dropped.

Section 3's theoretical explanation is honest about being a toy demonstration for the Rimella et al. lookahead filter rather than a derivation for PALL specifically. That is a limitation, not a fatal one, because the numerical evidence stands independently. The single simulated dataset per treatment is a minor design weakness, not a reason to doubt the bias.

The paper is written by the BPF developers, so there is an obvious conflict of interest in any comparison between PAL and BPF. That does not make the bias evidence circular—no quantity in the bias estimate is fit to the conclusion—but it should be acknowledged in a referee report.

Who is this for? Anyone who relies on the WWR measles results or plans to use lookahead PAL. The bias warning generalizes, even if the exact magnitude does not. I would send this to a serious referee with a request to fix the comparative claims and to add the like-for-like E13 versus E11 discussion. The central finding is reproducible and important, and the paper deserves engagement, not desk rejection.

Referee Report

3 major / 6 minor

Summary. This paper challenges the Poisson approximate likelihood (PAL) filter proposed by Whitehouse et al. (2023) for a spatiotemporal measles metapopulation model. The authors distinguish between the vanilla PAL (PALV) and the lookahead version (PALL) used by WWR, and present a series of numerical experiments on simulated and real data. On a single city with known ground truth, PALL overestimates the log-likelihood while PALV underperforms slightly. On a 40-city system, PALL again overstates the log-likelihood, and increasing the number of particles from 5e3 to 1e5 decreases the PALL estimate by about 834 log units (E14 vs. E15), which signals positive Monte Carlo bias. The paper also argues that PAL does not outperform a block particle filter and offers a toy example to illustrate how self-normalizing resampling in a lookahead filter can produce positive bias. Code and data are provided.

Significance. The numerical demonstration of positive bias in PALL is a significant and credible contribution. The use of multiple independent lines of evidence – known ground truth, comparison with the vanilla filter, and the particle-count dependence – makes the bias claim robust. The paper also contributes a reproducible workflow and careful attention to Monte Carlo standard errors. However, the comparative claim against the block particle filter is not yet supported, because the BPF is evaluated on a different model specification (Euler/Gaussian, case C) than the PALL (discrete/binomial, case A). Furthermore, the theoretical explanation in Section 3 is not directly derived for PALL. These issues limit the paper's current impact but are addressable in revision.

major comments (3)

[Abstract and §2 (Table 2, E11 vs E14/E15)] The claim in the abstract that 'PAL does not outperform a block particle filter' is not established by the experiments as reported. The BPF estimate (E11) is computed for a model with fproc=Euler, fmeas=Gaussian, and case C parameters re-optimized for that model, whereas the PALL estimates (E14/E15) use fproc=discrete, fmeas=binomial, and WWR's case A parameters. These are different statistical models, so the log-likelihood values cannot be compared directly; the gap between E14 and E11 mixes model difference with any algorithmic bias. To support the comparative claim, the paper should provide a same-model comparison, such as running BPF on the discrete/binomial model with the WWR parameters, or rephrase the claim to focus on the demonstrated bias.
[§3] The theoretical explanation of the positive bias is presented for a two-point example of the Rimella et al. lookahead filter, not for PALL. The text asserts that the bias is a property of the lookahead mechanism and therefore transfers, but it does not show that PALL's resampling scheme has the structure required for the positive bias (e.g., proposal probabilities that can be arbitrarily small). The numerical evidence stands on its own, but the theory section should be framed as an illustrative mechanism rather than an explanation of the PALL bias, unless the connection is made precise.
[§2, E8–E10] In the 40-unit simulated-data experiments, the filter is run with coupling C1 on data simulated from C2, so the comparison of λ10 with λ8 (PALL versus BPF ground truth) conflates model misspecification with algorithmic bias. The comparison of λ10 with λ9 (PALL versus PALV, both using C1) is a valid same-model bias estimate, and it could be presented as the primary evidence from that experiment; the comparison to λ8 should be interpreted more cautiously.

minor comments (6)

[§2 (after Table 3)] The sentence 'the decrease of the log-likelihood by λ14−λ15 = -833.83 log units' has a sign error; since λ14 > λ15, the decrease is 833.83, or equivalently λ15 − λ14 = -833.83.
[§2, E2–E3] The statement that E2 and E3 'failed to outperform the ARMA benchmark' compares log-likelihoods from simulated data with benchmark values that appear to be computed on the real data; this comparison is not meaningful unless the ARMA model is fitted to the same simulated data.
[§1] The abbreviations PALL and PALV are used from the abstract onward but are not defined until Section 2; please define them at first use.
[§3] The toy example uses notation (e.g., ε, the resampled weights, the role of the lookahead Y) that is compressed; expanding the derivation would improve clarity.
[Table 1] The notation for parameter values (θ̂*BPF, θ̂PAL) in Table 1 is not explained in the table caption; please specify that these are optimized parameter vectors for each filter.
[§2] Each simulated-data treatment is based on a single dataset; the reported standard errors are Monte Carlo errors, but dataset-to-dataset variability is not quantified. Given the large effect sizes, this does not affect the qualitative conclusion, but it limits the precision of quantitative bias estimates such as 80.55 per unit.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the PAL bias and BPF comparison are established by direct numerical experiments and externally grounded particle-filter consistency, not by fitting the conclusion.

full rationale

The paper's central claims are quantitative: PALL's log-likelihood estimate is positively biased, and this invalidates WWR's Table 3 comparison. The bias evidence is self-contained: E4 vs E5 and E14 vs E15 compare the same algorithm at different particle counts on the same model and parameters, and E5 vs E3 / E10 vs E9 compare PALL to vanilla PALV under the stated assumption that both target the same limiting PAL likelihood. No parameter is fitted to the conclusion; the estimated bias is a measured difference in Monte Carlo estimates. The BPF ground-truth values (E1, E8) rest on the standard unbiasedness/consistency of particle filters (Del Moral 2004) and the block particle filter (Rebeschini & Van Handel 2015), which are external results, and on the paper's own low-variance checks. The authors cite their own BPF work (Ionides et al. 2023, 2024; Ning & Ionides 2023) to motivate BPF as a comparator, but the load-bearing evidence is the experiments in this paper, so the self-citations are not load-bearing. The abstract's comparative claim ('PAL does not outperform a block particle filter') is less directly supported because E11 (BPF) uses Euler/Gaussian and re-optimized parameters while E14/E15 (PALL) use discrete/binomial and WWR case A parameters; however, this is a comparability/model-specification concern, not circularity, since none of the quantities are defined in terms of each other or fitted to the conclusion. Similarly, Section 3's toy example for Rimella et al.'s lookahead filter is an explanatory mechanism applied by analogy to PALL rather than a circular derivation. No step reduces by construction to its own inputs.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new parameters or entities; its argument rests on standard SMC consistency results, the proper scoring rule, and the assumed transfer of the toy bias mechanism to the measles PALL.

assumptions (4)

standard math Log-likelihood is a strictly proper scoring rule: a probabilistic forecast cannot, on average, achieve higher log-likelihood than the exact prediction distribution.
Used in Section 2 to justify measuring bias by comparing filter log-likelihood against exact values from simulated data and by the Jensen's inequality argument on particle number.
standard math The bootstrap/plain particle filter is unbiased and consistent for the likelihood, giving ground-truth log-likelihoods when Monte Carlo variance is small.
Invoked for E1 and E8 ground truth, citing Del Moral (2004).
domain assumption PALL and PALV target the same log-likelihood as J goes to infinity, so lambda5-lambda3 and lambda10-lambda9 estimate the bias of the lookahead version.
Section 2 states both 'target the same quantity in the limit as J goes to infinity'; if the lookahead mechanism changed the target rather than the proposal, the bias estimates would be confounded.
ad hoc to paper The two-point self-normalization toy example in Section 3 captures the bias mechanism of the lookahead filter in the measles model.
Section 3 analyzes a simplified lookahead filter and asserts the property applies to the PALL implementation used by WWR; no formal derivation for the actual model is given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Poisson Approximate Likelihood versus the block particle filter for a spatiotemporal measles model." pith.science (2026). https://pith.science/paper/QTGX2Y2A

@misc{pith2026250709121,
  author       = {Pith},
  title        = {Pith review of: Poisson Approximate Likelihood versus the block particle filter for a spatiotemporal measles model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QTGX2Y2A}},
  note         = {Machine review of arXiv:2507.09121}
}

read the original abstract

Filtering algorithms for high-dimensional nonlinear non-Gaussian partially observed stochastic processes provide access to the likelihood function and hence enable likelihood-based or Bayesian inference for this methodologically challenging class of models. A novel Poisson approximate likelihood (PAL) filter was introduced by Whitehouse et al.\ (2023). PAL employs a Poisson approximation to conditional densities, offering a fast approximation to the likelihood function for a certain subset of partially observed Markov process models. PAL was demonstrated on an epidemiological metapopulation model for measles, specifically, a spatiotemporal model for disease transmission within and between cities. At face value, Table\ 3 of Whitehouse et al.\ (2023) suggests that PAL considerably out-performs previous analysis as well as an ARMA benchmark model. We show that PAL does not outperform a block particle filter and that the lookahead component of PAL was implemented in a way that introduces substantial positive bias in the log-likelihood estimates. Therefore, the results of Table\ 3 of Whitehouse et al.\ (2023) do not accurately represent the true capabilities of PAL.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages

[1]

A., and Ionides, E

Asfaw, K., Park, J., King, A. A., and Ionides, E. L. (2024). spatPomp: An R package for spatiotemporal partially observed Markov process models.Journal of Open Source Software , 9:7008

work page 2024
[2]

Bengtsson, T., Bickel, P., and Li, B. (2008). Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. In Speed, T. and Nolan, D., editors,Probability and Statistics: Essays in Honor of David A. Freedman , pages 316–334. Institute of Mathematical Statistics, Beachwood, OH. Bretó, C., He, D., Ionides, E. L., and King,...

work page 2008
[3]

Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications

DelMoral, P.(2004). Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications

work page 2004
[4]

and Raftery, A

Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation.Journal of the American Statistical Association, 102(477):359–378

work page 2007
[5]

Poisson approximate likelihood compared to the particle filter

Hao, Y., Abkemeier, A. J., and Ionides, E. L. (2024). Poisson approximate likelihood compared to the particle filter. arXiv:2409.12173

work page Pith review arXiv 2024
[6]

L., and King, A

He, D., Ionides, E. L., and King, A. A. (2010). Plug-and-play inference for disease dynamics: Measles in large and small towns as a case study.Journal of the Royal Society Interface , 7:271–283

work page 2010
[7]

L., Asfaw, K., Park, J., and King, A

Ionides, E. L., Asfaw, K., Park, J., and King, A. A. (2023). Bagged filters for partially observed interacting systems. Journal of the American Statistical Association , 118(542):1078–1089

work page 2023
[8]

L., Ning, N., and Wheeler, J

Ionides, E. L., Ning, N., and Wheeler, J. (2024). An iterated block particle filter for inference on coupled dynamic systems with shared and unit-specific parameters.Statistica Sinica, 34:1145–1166

work page 2024

Show all 20 references

[9]

J., and Grenfell, B

Korevaar, H., Metcalf, C. J., and Grenfell, B. T. (2020). Structure, space and size: competing drivers of variation in urban and rural measles transmission.Journal of the Royal Society Interface , 17(168):20200010

2020
[10]

L., King, A

Li, J., Ionides, E. L., King, A. A., Pascual, M., and Ning, N. (2024). Inference on spatiotemporal dynamics for networks of biological populations.Journal of the Royal Society Interface , 21(216):20240217

2024
[11]

and Ionides, E

Ning, N. and Ionides, E. L. (2023). Iterated block particle filter for high-dimensional parameter learning: Beating the curse of dimensionality.Journal of Machine Learning Research , 24:1–76

2023
[12]

and Ionides, E

Park, J. and Ionides, E. L. (2020). Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter.Statistics & Computing , 30:1497–1522

2020
[13]

and Van Handel, R

Rebeschini, P. and Van Handel, R. (2015). Can local particle filters beat the curse of dimensionality?The Annals of Applied Probability, 25(5):2809–2866

2015
[14]

Rimella, L., Jewell, C., and Fearnhead, P. (2023). Approximating optimal SMC proposal distributions in individual-based epidemic models. Statistica Sinica, page SS–2022–0198

2023
[15]

Stocks, T., Britton, T., and Höhle, M. (2018). Model selection and parameter estimation for dynamic epidemic models via iterated filtering: Application to rotavirus in Germany.Biostatistics, 21(3):400–416. 8

2018
[16]

Tan, K., Hooker, G., and Ionides, E. L. (2024). Accelerated inference for partially observed Markov processes using automatic differentiation.arXiv:2407.03085

2024 arXiv
[17]

Wheeler, J., Rosengart, A., Zhuxun, J., Tan, K. H. E., Treutle, N., and Ionides, E. L. (2024). Informing policy via dynamic models: Cholera in Haiti.PLOS Computational Biology , 20:e1012032

2024
[18]

Whitehouse, M., Whiteley, N., and Rimella, L. (2023). Consistent and fast inference in compartmental models of epidemics using Poisson Approximate Likelihoods.Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(4):1173–1203

2023
[19]

Whitehouse, M., Whiteley, N., and Rimella, L. (2025). Correction to: Consistent and fast inference in compartmental models of epidemics using Poisson approximate likelihoods.Journal of the Royal Statistical Society Series B: Statistical Methodology , pre-published online

2025
[20]

N., and Grenfell, B

Xia, Y., Bjørnstad, O. N., and Grenfell, B. T. (2004). Measles metapopulation dynamics: A gravity model for epidemiological coupling and dynamics.American Naturalist, 164(2):267–281. 9

2004

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

A., and Ionides, E

Asfaw, K., Park, J., King, A. A., and Ionides, E. L. (2024). spatPomp: An R package for spatiotemporal partially observed Markov process models.Journal of Open Source Software , 9:7008

work page 2024

[2] [2]

Bengtsson, T., Bickel, P., and Li, B. (2008). Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems. In Speed, T. and Nolan, D., editors,Probability and Statistics: Essays in Honor of David A. Freedman , pages 316–334. Institute of Mathematical Statistics, Beachwood, OH. Bretó, C., He, D., Ionides, E. L., and King,...

work page 2008

[3] [3]

Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications

DelMoral, P.(2004). Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications

work page 2004

[4] [4]

and Raftery, A

Gneiting, T. and Raftery, A. E. (2007). Strictly proper scoring rules, prediction, and estimation.Journal of the American Statistical Association, 102(477):359–378

work page 2007

[5] [5]

Poisson approximate likelihood compared to the particle filter

Hao, Y., Abkemeier, A. J., and Ionides, E. L. (2024). Poisson approximate likelihood compared to the particle filter. arXiv:2409.12173

work page Pith review arXiv 2024

[6] [6]

L., and King, A

He, D., Ionides, E. L., and King, A. A. (2010). Plug-and-play inference for disease dynamics: Measles in large and small towns as a case study.Journal of the Royal Society Interface , 7:271–283

work page 2010

[7] [7]

L., Asfaw, K., Park, J., and King, A

Ionides, E. L., Asfaw, K., Park, J., and King, A. A. (2023). Bagged filters for partially observed interacting systems. Journal of the American Statistical Association , 118(542):1078–1089

work page 2023

[8] [8]

L., Ning, N., and Wheeler, J

Ionides, E. L., Ning, N., and Wheeler, J. (2024). An iterated block particle filter for inference on coupled dynamic systems with shared and unit-specific parameters.Statistica Sinica, 34:1145–1166

work page 2024