Assessing the performance of compartmental and renewal models for learning $R_{t}$ using spatially heterogeneous epidemic simulations on real geographies

The time-varying reproduction number ($R_t$) gives an indication of the trajectory of an infectious disease outbreak. Commonly used frameworks for inferring $R_t$ from epidemiological time series include those based on compartmental models (such as the SEIR model) and renewal equation models. These inference methods are usually validated using synthetic data generated from a simple model, often from the same class of model as the inference framework. However, in a real outbreak the transmission processes, and thus the infection data collected, are much more complex. The performance of common $R_t$ inference methods on data with similar complexity to real world scenarios has been subject to less comprehensive validation. We therefore propose evaluating these inference methods on outbreak data generated from a sophisticated, geographically accurate agent-based model. We illustrate this proposed method by generating synthetic data for two outbreaks in Northern Ireland: one with minimal spatial heterogeneity, and one with additional heterogeneity. We find that the simple SEIR model struggles with the greater heterogeneity, while the renewal equation model demonstrates greater robustness to spatial heterogeneity, though is sensitive to the accuracy of the generation time distribution used in inference. Our approach represents a principled way to benchmark epidemiological inference tools and is built upon an open-source software platform for reproducible epidemic simulation and inference.