pith. sign in

arxiv: 2510.18099 · v2 · submitted 2025-10-20 · 📊 stat.ME

Staying on Track: Efficient Trajectory Discovery with Adaptive Batch Sampling

Arindam Fadikar , Abby Stevens , Mickael Binois , Nicholson Collier , David O'Gara , Jonathan Ozik This is my paper

Pith reviewed 2026-05-18 05:18 UTC · model grok-4.3

classification 📊 stat.ME
keywords Bayesian optimizationtrajectory discoveryGaussian processstochastic modelsadaptive samplingepidemic modelingThompson sampling
0
0 comments X

The pith

Bayesian optimization finds data-consistent trajectories by modeling both parameters and random seeds

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces a trajectory-oriented Bayesian optimization technique for stochastic simulation models where the likelihood is hard to compute. It uses a Gaussian process that takes both model parameters and random seeds as inputs to directly work with individual simulation trajectories instead of their averages or summaries. A common random number strategy creates a likelihood function based on the surrogate, which then guides an adaptive Thompson Sampling procedure to focus evaluations on a grid of promising inputs using filtering and Metropolis-Hastings steps. This leads to quicker identification of parameter-random seed pairs that produce trajectories matching observed data in epidemic models. Readers might value this if they need to match specific stochastic paths rather than just average behaviors, as it could reduce the number of expensive simulations required.

Core claim

The authors claim that by extending the Gaussian process surrogate to include random seeds alongside input parameters and employing a common random number approach to establish a trajectory-specific likelihood, their adaptive Thompson Sampling algorithm can refine a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings densification, resulting in improved sampling efficiency and faster discovery of data-consistent trajectories compared to methods that infer only on parameters.

What carries the argument

The adaptive Thompson Sampling algorithm that refines a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings-based densification, enabled by a Gaussian process surrogate incorporating both parameters and random seeds.

Load-bearing premise

The common random number approach defines a surrogate-based likelihood over trajectories that supports effective adaptive Thompson Sampling for refining a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings densification.

What would settle it

Running the method on a known stochastic model and checking if it identifies trajectories that match synthetic data with fewer total simulations than a parameter-only Bayesian optimization baseline.

Figures

Figures reproduced from arXiv: 2510.18099 by Abby Stevens, Arindam Fadikar, David O'Gara, Jonathan Ozik, Mickael Binois, Nicholson Collier.

Figure 1
Figure 1. Figure 1: Deaths and hospitalizations from CityCOVID under 10 parameterizations identified in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Simulated epidemic trajectories generated by the aCRN method for a single experiment configu [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Proportion of trajectories with RMSE below varying thresholds across all experiments varying the [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Number of trajectories with RSME ¡ 15 found using [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Sampling behavior of the two top performing methods, [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Trajectories of deaths (left) and hospitalizations (right) from CityCOVID using aCRN for varying [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Number of trajectories found per simulation budget expenditure with objective function below the [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Mean and standard errors across experimental replicates of proportion below-threshold trajectories [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: rAUC of best performing design for each method under different budget and varying thresholds. [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Proportion of trajectories with RMSE below 30 across all experiments varying the hyper [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Sampling behavior of all the methods along with the initial design, across the joint model [PITH_FULL_IMAGE:figures/full_fig_p022_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Sampling behavior of all the methods along with the initial design, across the two dimensional [PITH_FULL_IMAGE:figures/full_fig_p023_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Simulated epidemic trajectories generated by the aCRN method for a single experiment configu [PITH_FULL_IMAGE:figures/full_fig_p024_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Proportion of trajectories with RMSE below varying thresholds across all experiments varying [PITH_FULL_IMAGE:figures/full_fig_p025_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Mean and standard errors across experimental replicates of proportion below-threshold trajectories [PITH_FULL_IMAGE:figures/full_fig_p025_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: rAUC of best performing design for each method under different budget and varying thresholds. [PITH_FULL_IMAGE:figures/full_fig_p025_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Proportion of trajectories with RMSE below 30 across all experiments varying the hyper [PITH_FULL_IMAGE:figures/full_fig_p026_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Simulated epidemic trajectories generated by the aCRN method for a single experiment configu [PITH_FULL_IMAGE:figures/full_fig_p027_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Proportion of trajectories with RMSE below varying thresholds across all experiments varying [PITH_FULL_IMAGE:figures/full_fig_p027_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Mean and standard errors across experimental replicates of proportion below-threshold trajectories [PITH_FULL_IMAGE:figures/full_fig_p028_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: rAUC of best performing design for each method under different budget and varying thresholds. [PITH_FULL_IMAGE:figures/full_fig_p028_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Proportion of trajectories with RMSE below 30 across all experiments varying the hyper [PITH_FULL_IMAGE:figures/full_fig_p028_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Parameter-replicate tuples corresponding to the trajectories with [PITH_FULL_IMAGE:figures/full_fig_p029_23.png] view at source ↗
read the original abstract

Bayesian optimization (BO) is a powerful framework for estimating parameters of expensive simulation models, particularly in settings where the likelihood is intractable and evaluations are costly. In stochastic models every simulation is run with a specific parameter set and an implicit or explicit random seed, where each parameter set and random seed combination generates an individual realization, or trajectory, sampled from an underlying random process. Existing BO approaches typically rely on summary statistics over the realizations, such as means, medians, or quantiles, potentially limiting their effectiveness when trajectory-level information is desired. We propose a trajectory-oriented BO method that incorporates a Gaussian process surrogate using both input parameters and random seeds as inputs, enabling direct inference at the trajectory level. Using a common random number approach, we define a surrogate-based likelihood over trajectories and introduce an adaptive Thompson Sampling algorithm that refines a fixed-size input grid through likelihood-based filtering and Metropolis-Hastings-based densification. This approach concentrates computation on statistically promising regions of the input space while balancing exploration and exploitation. We apply the method to stochastic epidemic models, a simple compartmental and a more computationally demanding agent-based model, demonstrating improved sampling efficiency and faster identification of data-consistent trajectories relative to parameter-only inference.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a trajectory-oriented Bayesian optimization method for parameter estimation in stochastic simulators. It augments a Gaussian process surrogate with both input parameters and random seeds, uses common random numbers to construct a surrogate-based likelihood directly over trajectories, and applies an adaptive Thompson Sampling procedure that refines a fixed-size input grid via likelihood-based filtering followed by Metropolis-Hastings densification. The method is demonstrated on a simple compartmental epidemic model and a more expensive agent-based model, with claims of improved sampling efficiency and faster recovery of data-consistent trajectories relative to parameter-only baselines.

Significance. If the central algorithmic claims are substantiated, the work provides a practical advance for calibrating stochastic models when trajectory-level fidelity matters more than summary statistics. The seed-augmented GP together with the adaptive grid-refinement strategy (filtering plus MH densification) is a coherent algorithmic contribution that could reduce the number of expensive simulator calls needed in epidemiology and similar domains. The empirical demonstrations on both low- and high-cost models supply useful evidence of practical utility.

major comments (2)
  1. [Method (surrogate likelihood and adaptive TS)] The surrogate-based likelihood construction (described in the method section following the GP definition) is load-bearing for the filtering and densification steps. The manuscript does not provide any quantification of GP predictive error relative to the true stochastic simulator variance on the high-dimensional trajectory outputs; without such diagnostics or bounds, it remains unclear whether the likelihood surface used for Thompson Sampling and MH steps is sufficiently calibrated or whether approximation error systematically biases which grid points survive filtering.
  2. [Experiments and results] In the experimental comparisons (results section), the reported gains in sampling efficiency and trajectory identification are presented relative to parameter-only inference, yet no ablation or diagnostic is given that isolates the contribution of the seed dimension versus the adaptive grid mechanism. This makes it difficult to attribute the observed improvements specifically to the trajectory-level surrogate rather than to the batch-sampling design alone.
minor comments (2)
  1. [Algorithm description] The description of how the fixed-size input grid is initialized and maintained across iterations could be made more explicit, including the precise criterion used to decide which points are filtered out.
  2. [Figures] Figure captions would benefit from stating the exact epidemic compartments or outputs being plotted and whether any uncertainty bands reflect GP posterior variance or simulator replicates.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed review. We address each major comment below and have revised the manuscript to incorporate additional diagnostics and ablations that directly respond to the concerns raised.

read point-by-point responses

  1. Referee: [Method (surrogate likelihood and adaptive TS)] The surrogate-based likelihood construction (described in the method section following the GP definition) is load-bearing for the filtering and densification steps. The manuscript does not provide any quantification of GP predictive error relative to the true stochastic simulator variance on the high-dimensional trajectory outputs; without such diagnostics or bounds, it remains unclear whether the likelihood surface used for Thompson Sampling and MH steps is sufficiently calibrated or whether approximation error systematically biases which grid points survive filtering.

    Authors: We agree that explicit quantification of GP predictive error would strengthen confidence in the surrogate likelihood. In the revised manuscript we have added a new subsection (Section 3.4) and Figure 3 that reports mean squared prediction error of the seed-augmented GP on held-out trajectories for both models. These errors are compared directly to the empirical variance obtained from repeated simulator runs at fixed parameters. The diagnostics show that GP error remains substantially smaller than simulator variance in the high-likelihood regions used for filtering, and we include a short discussion of how the common-random-number construction further limits bias propagation into the Thompson sampling and MH steps. revision: yes

  2. Referee: [Experiments and results] In the experimental comparisons (results section), the reported gains in sampling efficiency and trajectory identification are presented relative to parameter-only inference, yet no ablation or diagnostic is given that isolates the contribution of the seed dimension versus the adaptive grid mechanism. This makes it difficult to attribute the observed improvements specifically to the trajectory-level surrogate rather than to the batch-sampling design alone.

    Authors: The referee correctly notes the lack of component-wise isolation. We have performed the requested ablations and added them to the revised results section (new Table 3 and Supplementary Figure S4). The three conditions are: (i) full method, (ii) seed-augmented GP with non-adaptive fixed grid, and (iii) parameter-only GP with adaptive grid refinement. The new results show that the seed dimension accounts for the majority of the improvement in trajectory fidelity, while the adaptive filtering-plus-MH mechanism contributes the largest reduction in total simulator evaluations. These controlled comparisons now allow clearer attribution of the observed gains. revision: yes

Circularity Check

0 steps flagged

Independent algorithmic contribution using established GP and Thompson Sampling without reduction to fitted inputs

full rationale

The paper proposes a trajectory-oriented BO method that augments standard Gaussian process surrogates with random seeds as additional inputs and applies common random numbers to construct a surrogate likelihood for adaptive Thompson Sampling, grid filtering, and Metropolis-Hastings densification. This algorithmic extension is independent of the inputs and does not reduce any central claim to a self-definition, fitted parameter renamed as prediction, or self-citation chain. The derivation relies on well-established components (GP regression, CRN, TS) whose validity is external to the paper; empirical demonstrations on compartmental and agent-based epidemic models provide separate support. No load-bearing step collapses by construction to the method's own outputs or prior self-citations.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The method rests primarily on standard assumptions from Bayesian optimization and Gaussian processes, with the key addition being the inclusion of random seeds and the adaptive grid refinement strategy.

free parameters (1)
  • fixed-size input grid
    The adaptive algorithm refines a fixed-size grid, but the specific size is a design choice whose impact is not quantified in the abstract.
axioms (1)
  • domain assumption Gaussian process surrogate can model the joint mapping from (parameters, random seed) to full trajectories
    Invoked to enable direct trajectory-level inference and surrogate-based likelihood.

pith-pipeline@v0.9.0 · 5755 in / 1358 out tokens · 53132 ms · 2026-05-18T05:18:38.587835+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

64 extracted references · 64 canonical work pages · 1 internal anchor

  1. [1]

    Emews sweep tutorial

    work page

  2. [2]

    Thompson sampling for contextual bandits with linear payoffs

    Shipra Agrawal and Navin Goyal. Thompson sampling for contextual bandits with linear payoffs. In International conference on machine learning, pages 127–135. PMLR, 2013

    work page 2013

  3. [3]

    Stochastic kriging for simulation metamodeling

    Bruce Ankenman, Barry L Nelson, and Jeremy Staum. Stochastic kriging for simulation metamodeling. Operations research, 58(2):371–382, 2010

    work page 2010

  4. [4]

    Thinking inside the box: A tutorial on grey-box bayesian optimiza- tion

    Raul Astudillo and Peter I Frazier. Thinking inside the box: A tutorial on grey-box bayesian optimiza- tion. In2021 Winter Simulation Conference (WSC), pages 1–15. IEEE, 2021

    work page 2021

  5. [5]

    Analyzing stochastic computer models: A review with opportunities.Statistical Science, 37(1):64–89, 2022

    Evan Baker, Pierre Barbillon, Arindam Fadikar, Robert B Gramacy, Radu Herbei, David Higdon, Jiangeng Huang, Leah R Johnson, Pulong Ma, Anirban Mondal, et al. Analyzing stochastic computer models: A review with opportunities.Statistical Science, 37(1):64–89, 2022

    work page 2022

  6. [6]

    Approximate bayesian computation in population genetics.Genetics, 162(4):2025–2035, 2002

    Mark A Beaumont, Wenyang Zhang, and David J Balding. Approximate bayesian computation in population genetics.Genetics, 162(4):2025–2035, 2002

    work page 2025

  7. [7]

    Random search for hyper-parameter optimization.Journal of Machine Learning Research, 13:281–305, 2012

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization.Journal of Machine Learning Research, 13:281–305, 2012

    work page 2012

  8. [8]

    A Portfolio Approach to Massively Parallel Bayesian Optimization.Journal of Artificial Intelligence Research, 82:137–167, January 2025

    Mickael Binois, Nicholson Collier, and Jonathan Ozik. A Portfolio Approach to Massively Parallel Bayesian Optimization.Journal of Artificial Intelligence Research, 82:137–167, January 2025

    work page 2025

  9. [9]

    Quantifying uncertainty on pareto fronts with gaussian process conditional simulations.European journal of operational research, 243(2):386– 394, 2015

    Micka¨ el Binois, David Ginsbourger, and Olivier Roustant. Quantifying uncertainty on pareto fronts with gaussian process conditional simulations.European journal of operational research, 243(2):386– 394, 2015

    work page 2015

  10. [10]

    Gramacy, and Michael Ludkovski

    Micka¨ el Binois, Robert B. Gramacy, and Michael Ludkovski. Practical heteroscedastic gaussian pro- cess modeling for large simulation experiments.Journal of Computational and Graphical Statistics, 27(4):808–821, 2018

    work page 2018

  11. [11]

    The kalai–smorodinsky solution for many-objective bayesian optimization.Journal of Machine Learning Research, 21:1–42, 2020

    Micka¨ el Binois, Victor Picheny, Pierre Taillandier, and Adel Habbal. The kalai–smorodinsky solution for many-objective bayesian optimization.Journal of Machine Learning Research, 21:1–42, 2020

    work page 2020

  12. [12]

    Hoo: High-dimensional optimization with axis-aligned partitioning.Proceedings of ICML, 2011

    Sebastien Bubeck and Nicol` o Cesa-Bianchi. Hoo: High-dimensional optimization with axis-aligned partitioning.Proceedings of ICML, 2011

    work page 2011

  13. [13]

    Modelling transmission and control of the covid-19 pandemic in australia.Nature Communications, 12(1):1–13, 2021

    Sheryl L Chang, Natasha Harding, Cameron Zachreson, Osborne M Cliff, and Mikhail Prokopenko. Modelling transmission and control of the covid-19 pandemic in australia.Nature Communications, 12(1):1–13, 2021. 16

    work page 2021

  14. [14]

    An empirical evaluation of thompson sampling

    Olivier Chapelle and Lihong Li. An empirical evaluation of thompson sampling. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger, editors,Advances in Neural Information Processing Systems, volume 24. 2011

    work page 2011

  15. [15]

    Contem- porary statistical inference for infectious disease models using stan.Epidemics, 29:100367, 2019

    Arlina Chatzilena, Emily van Leeuwen, Oliver Ratmann, Marc Baguelin, and Nicos Demiris. Contem- porary statistical inference for infectious disease models using stan.Epidemics, 29:100367, 2019

    work page 2019

  16. [16]

    The effects of common random numbers on stochastic kriging metamodels.ACM Transactions on Modeling and Computer Simulation (TOMACS), 22(2):1–20, 2012

    Xi Chen, Bruce E Ankenman, and Barry L Nelson. The effects of common random numbers on stochastic kriging metamodels.ACM Transactions on Modeling and Computer Simulation (TOMACS), 22(2):1–20, 2012

    work page 2012

  17. [17]

    Fast update of conditional simulation ensem- bles.Mathematical Geosciences, 47:771–789, 2015

    C´ eline Chevalier, Xavier Emery, and David Ginsbourger. Fast update of conditional simulation ensem- bles.Mathematical Geosciences, 47:771–789, 2015

    work page 2015

  18. [18]

    John Wiley & Sons, 2012

    Jean-Paul Chiles and Pierre Delfiner.Geostatistics: Modeling Spatial Uncertainty. John Wiley & Sons, 2012

    work page 2012

  19. [19]

    City of Chicago Data Portal.https://data.cityofchicago.org/, 2021

    City of Chicago. City of Chicago Data Portal.https://data.cityofchicago.org/, 2021

    work page 2021

  20. [20]

    Wozniak, Arindam Fadikar, Abby Stevens, and Jonathan Ozik

    Nicholson Collier, Justin M. Wozniak, Arindam Fadikar, Abby Stevens, and Jonathan Ozik. Distributed Model Exploration with EMEWS. In2024 Winter Simulation Conference (WSC), pages 72–86, Orlando, FL, USA, December 2024. IEEE

    work page 2024

  21. [21]

    The frontier of simulation-based inference.Pro- ceedings of the National Academy of Sciences, 117(48):30055–30062, December 2020

    Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference.Pro- ceedings of the National Academy of Sciences, 117(48):30055–30062, December 2020

    work page 2020

  22. [22]

    Distributional encoding for gaussian process regression with qualitative inputs

    S´ ebastien Da Veiga. Distributional encoding for gaussian process regression with qualitative inputs. arXiv preprint arXiv:2506.04813, 2025

    work page arXiv 2025

  23. [23]

    Doyne Farmer, and Sebastian M

    Joel Dyer, Patrick Cannon, J. Doyne Farmer, and Sebastian M. Schmon. Black-box Bayesian inference for agent-based models.Journal of Economic Dynamics and Control, 161:104827, April 2024

    work page 2024

  24. [24]

    Scalable gaussian process optimization.Journal of Machine Learning Research, 2019

    David Eriksson, Samuel Daulton, Colin O’Boyle, and Mauricio Alvarez. Scalable gaussian process optimization.Journal of Machine Learning Research, 2019

    work page 2019

  25. [25]

    Stephen Eubank, Hasan Guclu, V. S. Anil Kumar, Madhav V Marathe, Aravind Srinivasan, Zolt´ an Toroczkai, and Nan Wang. Modelling disease outbreaks in realistic urban social networks.Nature, 429(6988):180–184, 2004

    work page 2004

  26. [26]

    The ensemble kalman filter: theoretical formulation and practical implementation.Ocean Dynamics, 53(4):343–367, 2003

    Geir Evensen. The ensemble kalman filter: theoretical formulation and practical implementation.Ocean Dynamics, 53(4):343–367, 2003

    work page 2003

  27. [27]

    Trajectory-oriented optimization of stochastic epidemiological models

    Arindam Fadikar, Nicholson Collier, Abby Stevens, Jonathan Ozik, Micka¨ el Binois, and Kok Ben Toh. Trajectory-oriented optimization of stochastic epidemiological models. In2023 Winter Simulation Con- ference (WSC), pages 1244–1255, 2023

    work page 2023

  28. [28]

    Calibrating a stochastic, agent-based model using quantile-based emulation.SIAM/ASA Journal on Uncertainty Quantification, 6(4):1685–1706, 2018

    Arindam Fadikar, Dave Higdon, Jiangzhuo Chen, Bryan Lewis, Srinivasan Venkatramanan, and Madhav Marathe. Calibrating a stochastic, agent-based model using quantile-based emulation.SIAM/ASA Journal on Uncertainty Quantification, 6(4):1685–1706, 2018

    work page 2018

  29. [29]

    Towards improved uncertainty quantification of stochastic epidemic models using sequential monte carlo

    Avishek Fadikar, Alexandra Stevens, Nathan Collier, Kevin Toh, Olga Morozova, Alison Hotton, James Clark, David Higdon, and Jonathan Ozik. Towards improved uncertainty quantification of stochastic epidemic models using sequential monte carlo. In2024 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), page 843–852, Los Alam...

    work page 2024

  30. [30]

    Report 9: Impact of non-pharmaceutical interventions (npis) to reduce covid-19 mortality and healthcare demand

    Neil M Ferguson, Daniel Laydon, Gemma Nedjati-Gilani, Natsuko Imai, Kylie Ainslie, Marc Baguelin, Sangeeta Bhatia, Adhiratha Boonyasiri, Zulma Cucunub´ a, Gina Cuomo-Dannenburg, et al. Report 9: Impact of non-pharmaceutical interventions (npis) to reduce covid-19 mortality and healthcare demand. Technical report, Imperial College COVID-19 Response Team, 2020. 17

    work page 2020

  31. [31]

    odin: Ode generation and integration r package version 1.2.6

    Richard FitzJohn. odin: Ode generation and integration r package version 1.2.6. CRAN, 2024

    work page 2024

  32. [32]

    Peter I. Frazier. A tutorial on bayesian optimization, 2018. arXiv:1807.02811

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  33. [33]

    Cambridge University Press, 2023

    Roman Garnett.Bayesian Optimization. Cambridge University Press, 2023

    work page 2023

  34. [34]

    Lawrence

    Javier Gonzalez, Zhiyuan Dai, Philipp Hennig, and Neil D. Lawrence. Batch bayesian optimization via local penalization. InProceedings of the 19th International Conference on Artificial Intelligence and Statistics, 2016

    work page 2016

  35. [35]

    Gramacy.Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences

    Robert B. Gramacy.Surrogates: Gaussian Process Modeling, Design and Optimization for the Applied Sciences. Chapman Hall/CRC, Boca Raton, Florida, 2020

    work page 2020

  36. [36]

    Gramacy, Adam Sauer, and Nathan Wycoff

    Robert B. Gramacy, Adam Sauer, and Nathan Wycoff. Triangulation candidates for bayesian optimiza- tion. InAdvances in Neural Information Processing Systems, volume 35, page 35933–35945. 2022

    work page 2022

  37. [37]

    Combining field data and computer simulations for calibration and prediction.SIAM Journal on Scientific Computing, 26(2):448–466, 2004

    Dave Higdon, Marc Kennedy, James C Cavendish, John A Cafeo, and Robert D Ryne. Combining field data and computer simulations for calibration and prediction.SIAM Journal on Scientific Computing, 26(2):448–466, 2004

    work page 2004

  38. [38]

    M. A. Iglesias, K. J. H. Law, and A. M. Stuart. Ensemble kalman methods for inverse problems.Inverse Problems, 29(4):045001, 2013

    work page 2013

  39. [39]

    Jones, Matthias Schonlau, and William J

    Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expensive black-box functions.Journal of Global Optimization, 13(4):455–492, 1998

    work page 1998

  40. [40]

    Kennedy and A

    M. Kennedy and A. O’Hagan. Bayesian calibration of computer models (with discussion).Journal of the Royal Statistical Society, Series B, 63:425–464, 2001

    work page 2001

  41. [41]

    King, Dao Nguyen, and Edward L

    Aaron A. King, Dao Nguyen, and Edward L. Ionides.Statistical Inference for Partially Observed Markov Processes: The POMP Package. Springer Series in Statistics. Springer, 2016

    work page 2016

  42. [42]

    Augmenting a simulation campaign for hybrid computer model and field data experiments.Technometrics, 66(4):638–650, 2024

    Scott Koermer, Justin Loda, Aaron Noble, and Robert B Gramacy. Augmenting a simulation campaign for hybrid computer model and field data experiments.Technometrics, 66(4):638–650, 2024

    work page 2024

  43. [43]

    Citycovid: A computer simulation of covid-19 spread in a large-urban area

    CM Macal, J Ozik, NT Collier, C Kaligotla, MM MacDonell, C Wang, DJ LePoire, Y Chang, and IJ Martinez-Moyano. Citycovid: A computer simulation of covid-19 spread in a large-urban area. In Proceedings of the 2020 Winter Simulation Conference, 2020

    work page 2020

  44. [44]

    Factorial sampling plans for preliminary computational experiments.Technometrics, 33(6):161–174, 1991

    Max D Morris. Factorial sampling plans for preliminary computational experiments.Technometrics, 33(6):161–174, 1991

    work page 1991

  45. [45]

    Efficient high dimensional bayesian optimization with additivity and quadrature fourier features.Advances in Neural Information Processing Systems, 31, 2018

    Mojmir Mutny and Andreas Krause. Efficient high dimensional bayesian optimization with additivity and quadrature fourier features.Advances in Neural Information Processing Systems, 31, 2018

    work page 2018

  46. [46]

    Collier, Justin M

    Jonathan Ozik, Nicholson T. Collier, Justin M. Wozniak, and Carmine Spagnuolo. From desktop to Large-Scale Model Exploration with Swift/T. In2016 Winter Simulation Conference (WSC), pages 206–220, December 2016

    work page 2016

  47. [47]

    A pop- ulation data-driven workflow for covid-19 modeling and learning.The International Journal of High Performance Computing Applications, 35(5):483–499, 2021

    Jonathan Ozik, Justin M Wozniak, Nicholson Collier, Charles M Macal, and Micka¨ el Binois. A pop- ulation data-driven workflow for covid-19 modeling and learning.The International Journal of High Performance Computing Applications, 35(5):483–499, 2021

    work page 2021

  48. [48]

    Bayesian simulation optimization with common random numbers

    Michael Pearce, Matthias Poloczek, and J¨ urgen Branke. Bayesian simulation optimization with common random numbers. InProceedings of the Winter Simulation Conference, WSC ’19, page 3492–3503. IEEE Press, 2019

    work page 2019

  49. [49]

    Bayesian optimization allowing for common random numbers.Operations Research, 70(6):3457–3472, 2022

    Michael Arthur Leopold Pearce, Matthias Poloczek, and Juergen Branke. Bayesian optimization allowing for common random numbers.Operations Research, 70(6):3457–3472, 2022. 18

    work page 2022

  50. [50]

    Building accurate emulators for stochastic simulations via quantile kriging.Technometrics, 56(4):466–473, 2014

    Matthew Plumlee and Rui Tuo. Building accurate emulators for stochastic simulations via quantile kriging.Technometrics, 56(4):466–473, 2014

    work page 2014

  51. [51]

    Posterior-based proposals for speeding up markov chain monte carlo.Royal Society Open Science, 2(3):140334, 2015

    Christopher M Pooley, Steven C Bishop, and Glenn Marion. Posterior-based proposals for speeding up markov chain monte carlo.Royal Society Open Science, 2(3):140334, 2015

    work page 2015

  52. [52]

    Carl Edward Rasmussen and Christopher K. I. Williams.Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA, 2005

    work page 2005

  53. [53]

    Bayesian cali- bration of stochastic agent based model via random forest.Statistics in medicine, 44(6):e70029, 2025

    Connor Robertson, Cosmin Safta, Nicholson Collier, Jonathan Ozik, and Jaideep Ray. Bayesian cali- bration of stochastic agent based model via random forest.Statistics in medicine, 44(6):e70029, 2025

    work page 2025

  54. [54]

    Group kernels for gaussian process metamodels with categorical inputs.SIAM/ASA Journal on Uncertainty Quantification, 8(2):775–806, 2020

    Olivier Roustant, Esperan Padonou, Yves Deville, Alo¨ ıs Cl´ ement, Guillaume Perrin, Jean Giorla, and Henry Wynn. Group kernels for gaussian process metamodels with categorical inputs.SIAM/ASA Journal on Uncertainty Quantification, 8(2):775–806, 2020

    work page 2020

  55. [55]

    Microsimulation model cali- bration using incremental mixture approximate bayesian computation.The annals of applied statistics, 13(4):2189, 2019

    Carolyn M Rutter, Jonathan Ozik, Maria DeYoreo, and Nicholson Collier. Microsimulation model cali- bration using incremental mixture approximate bayesian computation.The annals of applied statistics, 13(4):2189, 2019

    work page 2019

  56. [56]

    Welch, Toby J

    Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn. Design and analysis of computer experiments.Statistical Science, 4:409–423, 1989

    work page 1989

  57. [57]

    John Wiley & Sons, 2008

    Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, and Stefano Tarantola.Global sensitivity analysis: the primer. John Wiley & Sons, 2008

    work page 2008

  58. [58]

    Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. Practical bayesian optimization of machine learning algorithms. InAdvances in Neural Information Processing Systems, volume 25. 2012

    work page 2012

  59. [59]

    Inferring the spread of covid-19: the role of time-varying reporting rate in epidemiological modelling.Scientific reports, 12(1):10761, 2022

    Adam Spannaus, Theodore Papamarkou, Samantha Erwin, and J Blair Christian. Inferring the spread of covid-19: the role of time-varying reporting rate in epidemiological modelling.Scientific reports, 12(1):10761, 2022

    work page 2022

  60. [60]

    Thompson

    William R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples.Biometrika, 25:285–294, 1933

    work page 1933

  61. [61]

    Approximate bayesian computation scheme for parameter inference and model selection in dynamical systems.Journal of the Royal Society Interface, 6(31):187–202, 2009

    Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, and Michael P H Stumpf. Approximate bayesian computation scheme for parameter inference and model selection in dynamical systems.Journal of the Royal Society Interface, 6(31):187–202, 2009

    work page 2009

  62. [62]

    Hibo: High-dimensional bayesian optimization via mcts-inspired parti- tioning.Journal of Machine Learning Research, 2020

    Zhiwei Wang and Yizhou He. Hibo: High-dimensional bayesian optimization via mcts-inspired parti- tioning.Journal of Machine Learning Research, 2020

    work page 2020

  63. [63]

    J. M. Wozniak, T. G. Armstrong, M. Wilde, D. S. Katz, E. Lusk, and I. T. Foster. Swift/T: Large- Scale Application Composition via Distributed-Memory Dataflow Processing. In2013 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, pages 95–102, Delft, May 2013. IEEE

    work page 2013

  64. [64]

    Swift/t guide

    Justin M Wozniak. Swift/t guide. Technical Report ANL/DSL-TM-377, Argonne National Laboratory, 2018. 19 7 Supplementary Materials 7.1 Experiment Workflows The experiments using the compartmental SIR model (programmed in R) in Section 4 were performed on Argonne Laboratory Computing Resource Center’s Improv cluster using an EMEWS [21] sweep workflow [2]. T...

    work page 2018