{"id":"756260e5-cbc8-4561-bcc8-4a04caa7682a","arxiv_id":"2412.07193","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Graybox Bayesian optimization with Gaussian process surrogates and a decoupled acquisition function calibrates SIQR epidemic models with fewer expensive model evaluations than blackbox Bayesian optimization.","lead":"The paper proposes Bayesian optimization methods that treat the SIQR epidemic model as an expensive simulator and use Gaussian process surrogates to calibrate its parameters. On simulated data and COVID-19 infection counts, the graybox variants reach lower calibration error in fewer optimization rounds than standard blackbox Bayesian optimization.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix proof of Theorem 1 decomposes the decoupled acquisition function as a sum of per-component maxima, but the inner max over x' couples all compartments, so the convergence guarantee for DG-CF is unsupported.","rationale":"The reader's stated weakest assumption is Assumption 1 (η lies in the GP state space), but the Appendix also contains a more immediate mathematical flaw: the decoupled acquisition function's inner maximization over x' does not decompose additively across compartments. This flaw is directly load-bearing because the paper's theoretical contribution, Theorem 1, rests on it. The empirical comparison in the simulated experiments may still be valid, but the convergence guarantee for the proposed DG-CF acquisition function is unsupported as written. This justifies the reader's CONDITIONAL verdict: the paper should be accepted only after the proof is corrected or the convergence claim is withdrawn. I do not see a separate concern that would shift the verdict to ACCEPT or REJECT; the experimental results, with released code, are credible evidence for the pragmatic claim that graybox BO can outperform blackbox BO on this SIQR calibration task. The agreement is partial because the reader identified the decomposition issue in the rationale but did not make it the primary weakest assumption.","tokens_in":14870,"tokens_out":7277,"duration_ms":74621,"concrete_test":"Build a minimal two-compartment counterexample: D=1, T=1, two independent GP posteriors with quadratic means m_1(x) and m_2(x) (e.g., m_1(x) = a - (x-a)^2, m_2(x) = b - (x-b)^2 with a≠b) and constant variances. Compute u^*_{n+1}(1, y(x)) = max_{x'} [ (d_1 - y_1(x'))^2 + (d_2 - y_2(x'))^2 ] / 2 by dense grid search over x', and compare it to the sum of per-compartment maxima assumed in the proof, namely max_{x'} (d_1 - y_1(x'))^2 / 2 + max_{x'} (d_2 - y_2(x'))^2 / 2. If the two values differ, the decomposition identity in the Appendix is false, and Theorem 1 does not follow. This test is analytical and requires no code beyond simple grid evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The convergence proof in the Appendix relies on an invalid interchange of maximization and summation. In the definition of the decoupled acquisition function (Eq. 9), u^*_{n+1}(z, y(x)) = max_{x'} E_{P_{n+1}(y'(x'); z, y(x))}[g(y')], where g(y') is the negative MSE, an additive function over the four compartments: g(y') = (1/T) Σ_i (d_i - y'_i)^2. Therefore u^*_{n+1} is the maximum over x' of a sum of per-compartment terms. The proof, however, defines per-component quantities G_i^n and H_i^n using per-component maxima x_* and x'_*, and then claims α_n(x) = max_{z,x} (1/1^T z) Σ_i z_i G_i^n(x). This assumes max_{x'} Σ_i f_i(x') = Σ_i max_{x'} f_i(x'), which is generally false. For example, with two components f_1(x') = -(x'-a)^2 and f_2(x') = -(x'-b)^2, the left side is maximized at a weighted average of a and b, whereas the right side equals f_1(a)+f_2(b). Since Theorem 1 concludes α_n → 0 by bounding α_n(x_n) ≤ Σ_i G_i^n(x_n) and showing each G_i^n → 0, the entire asymptotic guarantee for the decoupled acquisition function collapses without this decomposition. This is a concrete mathematical error, not merely an unverifiable assumption.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes graybox Bayesian optimization (BO) methods for calibrating compartmental epidemiological SIQR models. It models each compartment output with a Gaussian process, uses the known composite structure of the negative mean-squared-error objective, and introduces a decoupled acquisition function that conditions on predicted outputs from a subset of the GPs. The authors report improved calibration performance, measured by log-MSE, relative to blackbox EI and KG baselines on simulated data, and apply the methods to calibrate SIQR and neural-ODE variants on U.S. and U.K. COVID-19 datasets. An appendix states a convergence theorem for the decoupled acquisition function.","tokens_in":1757,"tokens_out":2105,"duration_ms":94162,"significance":"If the empirical findings hold, the paper presents a practically useful recipe for sample-efficient calibration of expensive epidemiological models, with code provided. The simulated experiments cover complete and incomplete observations, noise, and a nonlinear ground-truth model, and consistently show that the graybox variants outperform blackbox EI and KG within 50 BO iterations. These strengths make the empirical contribution potentially valuable. However, the theoretical convergence claim in the appendix is unsupported due to a mathematically invalid decomposition, and some real-data claims are contradicted by the paper's own table. The conceptual contribution of the function-network graph also needs clarification because the graph as drawn does not encode the SIQR compartment dependencies the text appeals to.","major_comments":[{"comment":"The proof of Theorem 1 assumes that the maximum over x' of a sum of per-compartment expected utilities equals the sum of per-component maxima. Specifically, Eq. (9) defines u^*_{n+1}(z, y(x)) = max_{x'} E_{P_{n+1}(y'(x'); z, y(x))}[g(y')] with g(y') additive over the four compartments, but the proof's final line replaces this with Σ_i z_i G_i^n(x), where each G_i^n uses its own per-component maximizer x'_*. The interchange max_{x'} Σ_i f_i(x') = Σ_i max_{x'} f_i(x') is false in general. Consequently, the bound 0 ≤ α_n(x_n) ≤ Σ_i G_i^n(x_n) is not established, and the claimed convergence α_n → 0 a.s. does not follow. This is a load-bearing error in the paper's theoretical justification; the authors should either provide a correct proof with a common maximizer argument or remove the theorem and clearly state that convergence is not proved.","section":"Appendix, proof of Theorem 1"},{"comment":"Assumption 1 states that the true computer model η is in the state space of the GP prior y ∼ P_n for n ≥ 1, which is needed for Lemma 2 and hence for Theorem 1. The paper gives no argument that SIQR trajectories over the parameter ranges considered lie in the sample path space or RKHS of a Matérn GP, and the term 'state space' is not defined. This assumption is not merely technical; it is a function-class condition that the ODE solutions may fail. At minimum, the assumption should be stated precisely and its plausibility for the SIQR model should be discussed.","section":"Appendix, Assumption 1"},{"comment":"The function-network graph in Fig. 2 contains edges only from x to each y_i and from each y_i to g; there are no edges between compartments. According to Eq. (5), P ai = {y_j : (y_j → y_i) ∈ E}, so every P ai is empty, making the KG-FN model identical to KG-CF under the stated graph. The paper nonetheless reports performance differences between KG-FN and KG-CF and attributes them to the 'ground-truth functional dependency' of the SIQR model. This is internally inconsistent: either the implementation uses additional edges that are not shown (such as S→I or I→Q), or the reported differences arise from another source. Please clarify the actual graph used in the experiments and how it relates to the SIQR ODE structure.","section":"Section III.A, Fig. 2, Eq. (5)"},{"comment":"Table I contradicts the sentence claiming that KG-FN is 'generally better with lower variance.' In the U.K. column, KG-FN has mean smallest-MSE 45.0617×10^4 versus 44.6875×10^4 for KG-CF, so KG-FN is worse on average on that dataset. Also, the real-data experiments compare only KG-CF and KG-FN; no blackbox EI or KG baseline is included. Therefore these experiments do not support the abstract's broad claim that graybox variants outperform blackbox BO on real data. The authors should add blackbox baselines to the real-data study or restrict that claim to the simulated experiments.","section":"Section V, Table I"}],"minor_comments":[{"comment":"The sign of g is inconsistent: Eq. (3) defines g as the negative squared error, while Eq. (4) writes g(y(x)) = (1/T) Σ_i (d_i - y_i(x))^2 without the minus sign. Please align the notation throughout.","section":"Eq. (4) vs. Eq. (3)"},{"comment":"There are several typos, including 'the the commonly adopted' in the Introduction, 'Recoverd' and 'Qurantined' in Figs. 4 and 7, and 'derivation function' in Section IV. A careful proofreading pass is needed.","section":"Throughout"},{"comment":"Reference [1] lists 'Publisher Full Text' instead of a journal or venue name; this citation is incomplete.","section":"References"},{"comment":"The notation g ⊥ v is not defined in the text; please define statistical independence or replace it with a more standard notation such as g ⊥ v meaning 'g is independent of v'.","section":"Section III.B"},{"comment":"Algorithm 1 writes Pn(yi(x)) ← N(µ_i^n(x), σ_i^n(x)), which omits the parent variables P ai used in Eq. (5) and in the acquisition function; this makes the pseudo-code inconsistent with the method description.","section":"Algorithm 1"},{"comment":"The parameter ranges '10^−1 bI(0) ≤ x5 ≤ 10bI(0)' and '10bI(0) ≤ x6 ≤ 10^3 bI(0)' should use explicit multiplication (e.g., 10^{-1} · bI(0)) to avoid ambiguity.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The empirical comparison on simulated data is the strongest part of the paper, and the code availability is a plus. The convergence proof, however, contains a clear mathematical error that cannot be fixed by minor edits; the authors need to either supply a correct proof or explicitly withdraw the theorem. The real-data section would be more convincing with the missing blackbox baselines. The discrepancy between the stated function-network graph and the reported KG-FN results is also a substantive issue that the authors must resolve. Given that the central empirical claim is plausible but the theoretical and presentation problems are load-bearing, major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2412.07193. First, the main empirical claim holds up: graybox composite-function BO beats blackbox EI and KG on SIQR calibration in simulated experiments, the code is available, and the decoupled acquisition is a real, if incremental, extension of Astudillo and Frazier's composite/function-network BO. Second, the convergence proof in the appendix is wrong as written, specifically the last step. The decoupled acquisition in Eq. (9) maximizes over x' an expectation of g(y'), which is a sum over the four compartments. The proof's final decomposition claims alpha_n(x) = max_z (1/1^T z) sum_i z_i G_i^n(x), which requires that the per-component maxima be achieved at the same x'. That is generally false: max over x' of a sum is not the sum of the maxima. Consequently, the bound 0 <= alpha_n(x_n) <= sum_i G_i^n(x_n) does not follow, and Theorem 1's guarantee for DG-CF is unsupported. This is a load-bearing flaw in the appendix, though it does not invalidate the empirical method itself.\n\nWhat is genuinely new and worth credit: the idea of conditioning on a subset of GP nodes (z in {0,1}^4) is absent from the cited composite/function-network BO papers, and this is the first application of that machinery to compartmental model calibration. The experiments are honest and reasonably thorough: three ground-truth settings, complete and incomplete observations, five seeds, and comparisons against EI and KG. The qualitative conclusion that graybox variants improve log-MSE within 50 iterations is supported by the figures.\n\nSoft spots, in proportion: (1) the proof error already noted; this needs to be fixed or explicitly retracted. (2) Assumption 1, requiring the true model to lie in the GP state space, is unverifiable but standard in this literature; it is less concerning than the algebraic error. (3) The real-data section omits blackbox baselines; Table I only reports KG-CF and KG-FN, so the real-world comparison is incomplete. That is a minor gap because the simulated experiments already demonstrate the blackbox comparison.\n\nWho this is for: people doing BO for expensive epidemiological models, and anyone extending composite/function-network BO. It deserves a serious referee and probably a major revision. I would tell the authors to fix the theorem or turn it into a heuristic claim, and to add blackbox results to the real-data table. If the proof is corrected, I would be comfortable citing the empirical results.","headline":"Empirically useful graybox BO for SIQR calibration, but the appendix convergence proof has a max/sum interchange error that invalidates Theorem 1 as stated.","tokens_in":15705,"tokens_out":2806,"would_cite":false,"duration_ms":30249,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By treating the SIQR compartmental structure as a known function network rather than a black box, graybox Bayesian optimization calibrates epidemic models with fewer simulator evaluations and lower log-MSE than standard BO.","keywords":["model calibration","Bayesian optimization","Gaussian process surrogate","graybox optimization","SIQR compartmental model","knowledge gradient","function network","COVID-19 calibration"],"falsifier":"Run the graybox BO procedure on a simulator whose compartment outputs are provably not in the Gaussian process prior's sample paths (for example, a trajectory with a sharp discontinuity induced by a parameter threshold), and check whether $\\alpha_n$ still converges to zero and whether the log-MSE still improves at the claimed rate; if the algorithm stalls while black-box BO continues to improve, Assumption 1 is the load-bearing premise.","tokens_in":14669,"feed_emoji":"🦠","tokens_out":7997,"duration_ms":77544,"temperature":0.7,"pith_summary":"This paper tries to establish that calibrating compartmental epidemic models can be made much more sample-efficient by refusing to treat the simulator as a black box. The authors build Gaussian-process surrogates for each compartment of an SIQR model, compose the known negative-MSE objective on top, and encode the dependencies between compartments as a function network. They also introduce a decoupled acquisition function that conditions a chosen subset of the surrogates on predicted data. On simulated SIQR data and real COVID-19 curves, the graybox variants reach lower log-MSE and converge in fewer Bayesian-optimization iterations than standard black-box EI and KG. If this is right, expensive epidemiological simulators can be calibrated with far fewer simulation runs.","feed_headline":"Graybox BO calibrates epidemic models in fewer simulator runs","feed_subtitle":"Modeling each SIQR compartment with its own Gaussian process beats blackbox BO on synthetic and COVID-19 data.","key_machinery":"The carrying object is the function network $G(V, E)$ combined with a composite-function objective. $V$ contains the parameter node $x$, four GP nodes $y_1,\\dots,y_4$ for S, I, Q, and R, and the metric node $g$; each edge encodes a known dependency, such as I coming from S and R coming from I and Q. The GPs have posterior means and variances conditioned on all previously queried simulator trajectories, and $g(y(x))$ is the known negative MSE, so only the compartment outputs need to be learned. The graybox knowledge-gradient acquisition $\\alpha_n(x, z) = \\frac{1}{\\mathbf{1}^\\top z}\\mathbb{E}_{P_n(y(x))}[u^*_{n+1}(z, y(x))] - u^*_n$ estimates, for each candidate $x$ and conditioning subset $z$, the expected gain in best achievable metric after one more query; it is evaluated by Monte Carlo with $K=8$ samples, $L=128$ inner samples, and the reparametrization trick, giving unbiased estimates and tractable gradients.","core_discovery":"The central claim is that integrating structural knowledge of the compartmental model into the BO acquisition loop improves both final calibration error and convergence speed. Instead of one GP on the scalar objective $f(x)$, the paper places separate GPs on the four SIQR compartment trajectories $y_i(x)$, defines the calibration metric $g(y(x))$ as the known negative mean squared error, and organizes the GPs into a function network whose edges encode which compartments feed which. The acquisition function is then a knowledge-gradient expected improvement computed over these GPs, and its decoupled variant chooses a binary vector $z$ indicating which GPs should be conditioned on predicted data before the next simulator query. Experiments over linear, noisy, and nonlinear ground-truth SIQR models, with and without the susceptible compartment observed, show the graybox methods (KG-CF, KG-FN, DG-CF) dominate black-box EI and KG in log-MSE and iteration budget; on U.S. and U.K. COVID-19 infectious-count data, the function-network variant achieves the lowest mean squared error among the compared methods.","pith_inferences":["The composite-function/function-network machinery is essentially generic: any expensive simulator with a known scalar summary metric can be calibrated this way, with the SIQR model being one tested instance.","The decoupled acquisition suggests a natural adaptive-budget rule: allocate simulator queries to compartments with high posterior variance; the paper's $z$ vector provides the per-compartment information such a rule would need, but the paper does not itself propose that rule.","Because the convergence theorem depends on the GP prior containing the true trajectory, extending the method to highly irregular agent-based model outputs may require nonstationary or warped GP priors, a modification the paper leaves untested.","A stronger real-data test would use multi-compartment observations rather than infectious counts alone, to see whether the function-network advantage over the composite-function-only variant persists when every compartment is observed noisily."],"forward_implications":["Calibrating an ODE-based epidemic model requires only tens of simulator evaluations, because the surrogate GPs and the known metric absorb most of the objective's complexity.","Missing compartments do not destroy calibration: the function network propagates information from observed compartments to the unobserved one.","The decoupled acquisition lets the optimizer spend its simulation budget on the compartments whose GP surrogates are least reliable, rather than refreshing all four evenly.","The same graybox recipe transfers to other compartmental variants and to agent-based models whose input-output dependencies can be written as a known graph."],"supporting_citations":[{"why":"Supplies the composite-function formulation that separates the known metric g from the GP surrogates on compartment outputs.","marker":"[5]"},{"why":"Supplies the function-network graph representation and the convergence lemma the proof invokes.","marker":"[6]"},{"why":"Provides the Monte Carlo Bayesian optimization implementation framework the compared algorithms are built on.","marker":"[7]"},{"why":"Supplies the adjoint gradient method used in the two-stage calibration of the neural-ODE variant.","marker":"[13]"},{"why":"Supplies the Gaussian process posterior inference and knowledge-gradient definitions used in the acquisition functions.","marker":"[16]"},{"why":"Defines the knowledge-gradient policy that the graybox and decoupled acquisition functions generalize.","marker":"[17]"},{"why":"Defines the SIQR compartmental ODE system that all calibration experiments target.","marker":"[25]"},{"why":"Provides the real-world U.S. and U.K. COVID-19 infectious-count data used in the final experiments.","marker":"[27]"}],"fun_headline_variants":["Graybox BO speeds epidemic calibration","Epidemic models calib fast via graybox BO","Graybox BO cuts simulator calls for COVID-19","Function-network BO improves epidemic fitting","Compartment-aware BO calibrates epidemics efficiently"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The convergence guarantee assumes the true SIQR simulator outputs are functions the Gaussian process prior can represent, and the paper never verifies this for the parameter ranges it calibrates.","fun_headline_variants_meta":{"raw":{"variants":["Graybox BO speeds epidemic calibration","Epidemic models calib fast via graybox BO","Graybox BO cuts simulator calls for COVID-19","Function-network BO improves epidemic fitting","Compartment-aware BO calibrates epidemics efficiently"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1462,"prompt_tokens":982,"completion_tokens":480,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":427}},"tokens_in":598,"tokens_out":480,"duration_ms":5319,"temperature":1.0,"reasoning_tokens":427,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:02:35.502405+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the graybox BO procedure on a simulator whose compartment outputs are provably not in the Gaussian process prior's sample paths (for example, a trajectory with a sharp discontinuity induced by a parameter threshold), and check whether $\\alpha_n$ still converges to zero and whether the log-MSE still improves at the claimed rate; if the algorithm stalls while black-box BO continues to improve, Assumption 1 is the load-bearing premise.","supporting_citations":[{"cited_title":"Bayesian optimization of composite functions","cited_arxiv_id":null,"evidence_quote":"Supplies the composite-function formulation that separates the known metric g from the GP surrogates on compartment outputs."},{"cited_title":"Bayesian optimization of function net- works","cited_arxiv_id":null,"evidence_quote":"Supplies the function-network graph representation and the convergence lemma the proof invokes."},{"cited_title":"BOTorch: A framework for efficient Monte-Carlo Bayesian optimization","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo Bayesian optimization implementation framework the compared algorithms are built on."},{"cited_title":"A knowledge- gradient policy for sequential information collection","cited_arxiv_id":null,"evidence_quote":"Defines the knowledge-gradient policy that the graybox and decoupled acquisition functions generalize."},{"cited_title":"Exact properties of SIQR model for COVID-19","cited_arxiv_id":null,"evidence_quote":"Defines the SIQR compartmental ODE system that all calibration experiments target."},{"cited_title":"covid19.analytics: An r package to obtain, analyze and visualize data from the coronavirus disease pandemic","cited_arxiv_id":null,"evidence_quote":"Provides the real-world U.S. and U.K. COVID-19 infectious-count data used in the final experiments."}],"review_version":1}