pith. sign in

arxiv: 2512.05650 · v4 · pith:GJBX6PJ7new · submitted 2025-12-05 · 📊 stat.ME · stat.CO

Efficient sequential Bayesian inference for state-space epidemic models using ensemble data assimilation

Dhorasso Temfack , Jason Wyse This is my paper

Pith reviewed 2026-05-21 17:59 UTC · model grok-4.3

classification 📊 stat.ME stat.CO
keywords sequential Monte CarloEnsemble Kalman Filterstate-space epidemic modelsBayesian inferencedata assimilationlikelihood approximationmonkeypox
0
0 comments X

The pith

Replacing the inner particle filter in SMC² with an Ensemble Kalman Filter yields fast sequential Bayesian inference for epidemic models while producing comparable posterior estimates.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes eSMC² to make joint state and parameter inference feasible for state-space epidemic models. It keeps the outer SMC sampler over parameters but swaps the expensive inner particle filter for an Ensemble Kalman Filter that approximates each incremental likelihood. State-dependent observation variance and an unbiased Gaussian density estimator are added to handle overdispersed incidence counts. Simulation studies with known ground truth and an application to 2022 US monkeypox data show large reductions in run time with posterior accuracy close to standard SMC². The result is a practical tool for real-time reconstruction of epidemic trajectories and key parameters from noisy surveillance records.

Core claim

The central claim is that an adapted Ensemble Kalman Filter can replace the inner particle filter inside SMC², approximating the observed-data likelihood at each time step with enough fidelity that the resulting posterior for latent states and epidemiological parameters remains comparable to the exact SMC² while cutting computational cost substantially.

What carries the argument

Ensemble SMC² (eSMC²), which substitutes an Ensemble Kalman Filter for the nested particle filter in SMC² and uses state-dependent observation variance together with an unbiased Gaussian density estimator to approximate the incremental likelihood.

If this is right

  • Joint inference of latent epidemic states and parameters becomes feasible in near-real time for routine surveillance.
  • The method can reconstruct full epidemic trajectories from partially observed noisy incidence counts.
  • Key epidemiological quantities such as transmission rates can be estimated sequentially without prohibitive compute.
  • The approach extends naturally to other overdispersed count processes in infectious disease modeling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar ensemble approximations could be tested on non-epidemic state-space models that also involve count observations.
  • Hybrid filters that switch between ensemble and particle methods at different stages might further reduce bias while retaining speed.
  • The technique may support online updating of forecasts as new incidence reports arrive during an ongoing outbreak.

Load-bearing premise

The Gaussian approximation produced by the Ensemble Kalman Filter, after state-dependent variance adaptation and unbiased density estimation, stays accurate enough for the overdispersed count data typical of epidemic incidence.

What would settle it

Run both eSMC² and full SMC² on the same simulated epidemic trajectory with known parameters and check whether the marginal posterior distributions for the transmission rate and reporting probability differ by more than sampling error.

Figures

Figures reproduced from arXiv: 2512.05650 by Dhorasso Temfack, Jason Wyse.

Figure 1
Figure 1. Figure 1: Schematic illustration of the sequential update–prediction cycle in the EnKF. Each time step alternates between an update stage, where observations yt are assimilated to refine the latent state estimate, and a prediction stage, where the ensemble is propagated forward through the system dynamics. Let us first suppose that the parameters θ and a sequence of observations y1:T (in our case, the daily incidenc… view at source ↗
Figure 2
Figure 2. Figure 2: Flowchart of the eSMC2 algorithm. Each parameter particle carries an ensemble of state particles, which are propagated using the EnKF. Weights are updated based on an EnKF-based likelihood approximation. When the ESS falls below a threshold, parameter particles are resampled and rejuvenated via a PMMH step. This procedure is repeated for every data point, allowing sequential Bayesian learning. 9 [PITH_FUL… view at source ↗
Figure 3
Figure 3. Figure 3: Example 1: Filtered estimates of simulated incidence, transmission rate, and effective reproduction number. Solid lines show the posterior mean, with shaded areas representing the 95% credible intervals. Black dots indicate the observed incidence [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example 2: Filtered estimates of simulated incidence, transmission rate, and effective reproduction number. Solid lines show posterior means; shaded areas indicate 95% credible intervals. Observed incidence is shown as black dots. Figures 5 and 6 illustrate the temporal evolution of the posterior means of the inferred pa￾rameters, along with their marginal posterior densities at the final time step, each s… view at source ↗
Figure 5
Figure 5. Figure 5: Example 1: Posterior distributions of α, γ, and νβ for five independent runs. Top row shows filtered means with 95% credible intervals. Bottom row shows marginal posterior distributions at T = 60, with prior distributions overlaid (green dashed lines). Black dashed lines indicate true parameter values [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Example 2: Posterior distributions of α, γ, and νβ for five independent runs. The top row shows filtered means with 95% credible intervals. Bottom row shows marginal posterior distributions at T = 100, with prior distributions overlaid (green dashed lines). Black dashed lines indicate true parameter values. Results in [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Inference of daily incidence and effective reproduction number for the 2022 U.S. monkeypox outbreak. Left: filtered estimates of daily incidence (solid blue line) with reported counts (red dots). Right: inferred effective reproduction number Reff (t). Shaded regions represent 50%, 75%, 90%, and 95% credible intervals. The vertical dashed line indicates the declaration of the national state of emergency. 13… view at source ↗
Figure 8
Figure 8. Figure 8: Posterior distributions of key epidemiological parameters. The left column shows filtered means with credible intervals; the right column displays marginal posterior distributions at the last time step [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Posterior-predictive forecasts at different starting dates. Filtering estimates are shown in blue, and forecast distributions in purple, with corresponding 50%, 75%, 90%, and 95% credible intervals. In-sample observations (prior to the forecast start date) are shown in red, and out-of-sample observations in yellow. Vertical orange dashed lines indicate the forecast start dates [PITH_FULL_IMAGE:figures/ful… view at source ↗
Figure 10
Figure 10. Figure 10: Long-term forecast performance. Again, forecast distribution with credible intervals for the final phase of the outbreak is in purple. In-sample data (red) are shown before the forecast start date (vertical dashed line), while out-of-sample data (yellow) are used for validation [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
read the original abstract

Estimating latent epidemic states and model parameters from partially observed, noisy data remains a major challenge in infectious disease modeling. State-space formulations provide a coherent probabilistic framework for such inference, yet fully Bayesian estimation is often computationally prohibitive because evaluating the observed-data likelihood requires integration over a latent trajectory. The Sequential Monte Carlo squared (SMC$^2$) algorithm offers a principled approach for joint state and parameter inference, combining an outer SMC sampler over parameters with an inner particle filter that estimates the likelihood up to the current time point. Despite its theoretical appeal, this nested particle filter imposes substantial computational cost, limiting routine use in near-real-time outbreak response. We propose Ensemble SMC$^2$ (eSMC$^2$), a computationally efficient variant that replaces the inner particle filter with an Ensemble Kalman Filter (EnKF) to approximate the incremental likelihood at each observation time. While this substitution introduces bias via a Gaussian approximation, we mitigate finite-sample effects using an unbiased Gaussian density estimator and adapt the EnKF for epidemic data through state-dependent observation variance. This makes our approach particularly suitable for overdispersed incidence data commonly encountered in infectious disease surveillance. Simulation experiments with known ground truth and an application to 2022 United States (U.S.) monkeypox incidence data demonstrate that eSMC$^2$ achieves substantial computational gains while producing posterior estimates comparable to SMC$^2$. The method accurately reconstructs epidemic trajectories and estimates key epidemiological parameters, providing an efficient framework for sequential Bayesian inference from imperfect surveillance data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces Ensemble SMC² (eSMC²), a computationally efficient variant of SMC² for joint inference of latent states and parameters in state-space epidemic models. It replaces the inner particle filter with an Ensemble Kalman Filter (EnKF) to approximate incremental likelihoods, incorporating state-dependent observation variance and an unbiased Gaussian density estimator to address overdispersed incidence data. The approach is evaluated via simulation experiments that recover known ground truth and an application to 2022 U.S. monkeypox incidence data, with the central claim being substantial computational gains while yielding posterior estimates comparable to standard SMC².

Significance. If the EnKF approximation bias proves negligible for the targeted epidemic models, the method would offer a practical route to routine sequential Bayesian inference in near-real-time outbreak settings, where full SMC² is often too slow. The paper earns credit for grounding its claims in simulation experiments with known ground truth and a real-data application, which allows direct assessment of both efficiency and accuracy.

major comments (2)
  1. [§3] §3 (proposed method): The claim that the adapted EnKF produces incremental likelihood estimates sufficiently accurate for the outer SMC sampler rests on the Gaussian approximation remaining adequate for discrete, overdispersed count observations. No theoretical error bound or empirical diagnostic (e.g., comparison of approximated vs. exact incremental likelihoods on held-out trajectories) is supplied to quantify how the acknowledged bias affects the parameter posterior; this is load-bearing for the comparability result.
  2. [§5] §5 (simulation experiments): While ground-truth recovery is reported, the experiments do not isolate the contribution of the EnKF approximation error (versus full particle filter) to posterior bias or coverage; without such a diagnostic, it is unclear whether the observed comparability would persist for incidence series exhibiting rapid growth or low counts, where the Gaussian assumption is most strained.
minor comments (2)
  1. [Abstract] The abstract states 'substantial computational gains' without reporting wall-clock ratios, particle/ensemble sizes, or hardware specifications that would allow readers to gauge practical speedup.
  2. [Methods] Notation for the unbiased Gaussian density estimator should be introduced with an explicit equation early in the methods section rather than being referenced only in passing.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which correctly identify the need for stronger quantification of the EnKF approximation error in eSMC². We address each major point below and will revise the manuscript accordingly to improve the supporting evidence for our claims.

read point-by-point responses

  1. Referee: [§3] §3 (proposed method): The claim that the adapted EnKF produces incremental likelihood estimates sufficiently accurate for the outer SMC sampler rests on the Gaussian approximation remaining adequate for discrete, overdispersed count observations. No theoretical error bound or empirical diagnostic (e.g., comparison of approximated vs. exact incremental likelihoods on held-out trajectories) is supplied to quantify how the acknowledged bias affects the parameter posterior; this is load-bearing for the comparability result.

    Authors: We agree that a quantitative assessment of the approximation bias is necessary to support the central comparability claim. A general theoretical error bound for the state-dependent EnKF in nonlinear epidemic models is difficult to obtain and lies outside the scope of the present work. In the revision we will add an empirical diagnostic that directly compares incremental likelihood values produced by the adapted EnKF against those from a high-particle SMC² run on the same held-out simulated trajectories. We will report relative errors and examine how these errors propagate into the outer SMC parameter posterior, thereby providing concrete evidence on the magnitude of the bias for the models considered. revision: yes

  2. Referee: [§5] §5 (simulation experiments): While ground-truth recovery is reported, the experiments do not isolate the contribution of the EnKF approximation error (versus full particle filter) to posterior bias or coverage; without such a diagnostic, it is unclear whether the observed comparability would persist for incidence series exhibiting rapid growth or low counts, where the Gaussian assumption is most strained.

    Authors: We acknowledge that isolating the EnKF approximation’s specific contribution would strengthen the validation. The revised manuscript will expand the simulation section with new experiments that include rapid-growth phases and low-count regimes. In addition, we will run both eSMC² and full SMC² on identical trajectory sets under these conditions and quantify differences in posterior bias and coverage attributable to the EnKF step. This will clarify the robustness of the reported comparability under the most challenging observation regimes. revision: yes

Circularity Check

0 steps flagged

No circularity: eSMC² combines established SMC² and EnKF components via explicit new adaptations tested against ground truth

full rationale

The paper presents eSMC² as a direct algorithmic substitution of the inner particle filter in SMC² by an adapted EnKF, with state-dependent observation variance and an unbiased Gaussian density estimator introduced to address bias for epidemic count data. These adaptations are described explicitly in the method section and validated through simulation experiments with known ground truth plus a real-data application, without any load-bearing step that reduces by definition or self-citation to the target posterior or likelihood estimates. The central claims rest on the empirical comparability of posteriors to full SMC² rather than on any self-referential derivation or fitted quantity renamed as a prediction. No uniqueness theorems, ansatzes smuggled via prior work, or renamings of known results appear in the derivation chain.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Based on the abstract alone, the method inherits standard assumptions of SMC and EnKF (Markovian state transitions, Gaussian filter updates) plus the domain assumption that state-dependent variance adequately captures overdispersion; no explicit free parameters or new entities are named.

pith-pipeline@v0.9.0 · 5796 in / 1052 out tokens · 65753 ms · 2026-05-21T17:59:45.854914+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

7 extracted references · 7 canonical work pages · 1 internal anchor

  1. [1]

    & Kim, S

    Abbas, W., Lee, S. & Kim, S. (2025), ‘Joint estimation of hand-foot-mouth disease model and pre- diction in korea using the ensemble kalman filter’,PLOS Computational Biology21(4), e1012996. Andrieu, C., Doucet, A. & Holenstein, R. (2010), ‘Particle markov chain monte carlo methods’, Journal of the Royal Statistical Society Series B: Statistical Methodolo...

    work page arXiv 2025

  2. [2]

    J., De Angelis, D

    Birrell, P. J., De Angelis, D. & Presanis, A. M. (2018), ‘Evidence synthesis for stochastic epidemic models’,Statistical science: a review journal of the Institute of Mathematical Statistics33(1),

    work page 2018

  3. [3]

    & Dureau, J

    Cazelles, B., Champagne, C. & Dureau, J. (2018), ‘Accounting for non-stationarity in epidemiol- ogy by embedding time-varying parameters in stochastic models’,PLoS computational biology 14(8), e1006211. Chen, Y., Sanz-Alonso, D. & Willett, R. (2022), ‘Autodifferentiable ensemble Kalman filters’,SIAM Journal on Mathematics of Data Science4(2), 801–833. Cho...

    work page 2018

  4. [4]

    Ebeigbe, D., Berry, T., Schiff, S. J. & Sauer, T. (2020), ‘Poisson Kalman filter for disease surveil- lance’,Physical review research2(4), 043028. Evensen, G. (1994), ‘Sequential data assimilation with a nonlinear quasi-geostrophic model us- ing Monte Carlo methods to forecast error statistics’,Journal of Geophysical Research: Oceans 99(C5), 10143–10162. ...

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  5. [5]

    & Smith, A

    Gordon, N., Salmond, D. & Smith, A. (1993), ‘Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation’,IEEE Proceedings F – Radar and Signal Processing140(2), 107–

    work page 1993

  6. [6]

    Katzfuss, M., Stroud, J. R. & Wikle, C. K. (2020), ‘Ensemble Kalman methods for high- dimensional hierarchical dynamic space-time models’,Journal of the American Statistical As- sociation115(530), 866–885. Khalil, M., Sarkar, A., Adhikari, S. & Poirel, D. (2015), ‘The estimation of time-invariant parame- ters of noisy nonlinear oscillatory systems’,Journa...

    work page 2020

  7. [7]

    & Arnold, A

    26 Mitchell, L. & Arnold, A. (2021), ‘Analyzing the effects of observation function selection in ensemble Kalman filtering for epidemic models’,Mathematical biosciences339, 108655. Morzfeld, M., Hodyss, D. & Snyder, C. (2017), ‘What the collapse of the ensemble Kalman filter tells us about particle filters’,Tellus A: Dynamic Meteorology and Oceanography69...

    work page 2021