{"id":"c7f870f3-d862-41d5-b33c-c6d159b74e77","arxiv_id":"2501.00967","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"BOIS linearizes known composite objective functions around Gaussian process predictions to compute closed-form uncertainty estimates, and benchmarks show it matches or beats MC-BO and OP-BO on two process design problems with lower acquisition cost.","lead":"BOIS is a Bayesian optimization variant that approximates complicated process models with local linear shortcuts so it can estimate the best design without heavy Monte Carlo sampling. On two chemical engineering design benchmarks it matches or beats existing grey-box optimizers and spends less time choosing where to sample next.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The decisive weakness is the uncalibrated MC-BO baseline (S=100) and missing significance tests, not the linearization itself: the paper's own text says S=100 made MC-BO noisy, so BOIS's reported edge is not yet established.","rationale":"The reader identified the first-order Taylor approximation as the weakest assumption. That is a fair concern, and the paper itself acknowledges that the Laplace approximation degrades near feasibility bounds. However, the Taylor concern is partially supported by the parity plots in Figures 9 and 12, which compare BOIS moments against MC-BO with S=1,000 on the same GP and show good agreement away from the boundaries. The optimization trials also provide direct evidence that the approximation is adequate for the two systems studied. By contrast, the MC-BO baseline is run with S=100 even though the authors state in Section 5.1 that this sample count made the acquisition function values differ by over 10% between evaluations. That is a self-identified disadvantage built into the main comparator, and it directly affects the headline claim that BOIS matches or beats existing grey-box methods. The lack of significance testing compounds the issue: the reported average improvements are small, and without paired confidence intervals or error bars on the regret curves, the apparent superiority could arise from noise. I would keep the verdict at CONDITIONAL because the paper still has reproducible code, plausible OP-BO comparisons, and credible wall-clock measurements; the concern does not invalidate the framework, but it does mean the central empirical claim needs a fairness check before it can be treated as established. My agreement with the reader is partial: the Taylor assumption is important, but in my reading the uncalibrated MC-BO baseline and missing statistical comparisons are more load-bearing for the specific claim being made.","tokens_in":996,"tokens_out":2312,"duration_ms":85137,"concrete_test":"Re-run both benchmark studies with MC-BO at S=1,000 (or with an adaptive sample count calibrated so the LCB-CF value is stable to within 1%) using the same 25 and 125 trials, initializations, and iteration budgets; compute paired bootstrap 95% confidence intervals for the difference in best log-normalized regret at 50 and 100 iterations. If the BOIS-versus-MC-BO difference interval includes zero or reverses sign, the claim that BOIS performs as well as or better than MC-BO is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 5 states that MC-BO used S=100 samples to build the LCB-CF acquisition function. Section 5.1 then reports that at a given x, different evaluations of the LCB-MCBO AF differed by over 10% and concludes that Monte Carlo needs more samples. That is an admission that the comparator was run in a regime the authors know is underpowered. The central empirical claim, that BOIS performs as well as or better than existing grey-box methods while being cheaper, therefore rests partly on a baseline whose poor performance is an artifact of its sample budget. The measured differences are also small (1.2%, 3.3%, 5.4%, and 1.6% in the two studies) and are reported without error bars, confidence intervals, or paired significance tests; over 25 and 125 trials respectively, we cannot distinguish a real advantage from sampling noise. The Taylor-linearization concern raised by the reader is real but partially mitigated by the parity checks against MC-BO with S=1,000 in Figures 9 and 12, and it is explicitly acknowledged near feasibility bounds. The baseline-calibration issue is not mitigated. If MC-BO with adequate samples closes the gap, the headline reduces to BOIS being competitive with OP-BO and faster per acquisition-function optimization, which is a weaker and less general claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents BOIS, a grey-box Bayesian optimization method for composite objective functions of the form f(x, y(x)), where y is an unknown intermediate vector-valued function modeled by Gaussian processes. The core contribution is a closed-form approximation of the mean and variance of f obtained by a first-order Taylor expansion of f in y around a local reference point, which yields an LCB-style acquisition function whose evaluation avoids Monte Carlo sampling or the augmented-space optimization used by OP-BO. The manuscript extends the authors' earlier BOIS preprint with feasibility handling for intermediate variables and nested GP structures, and benchmarks the method against standard BO, MC-BO, and OP-BO on two chemical process case studies: a reactor/separator/recycle process and a photobioreactor design problem. The reported results indicate that BOIS matches or improves on the solution value of the comparators while requiring less time for acquisition-function optimization, and the authors provide parity plots showing that BOIS moment estimates agree with high-sample MC-BO estimates.","tokens_in":19481,"tokens_out":6301,"duration_ms":54919,"significance":"If the empirical claims are substantiated, the paper makes a useful contribution by offering a computationally efficient way to propagate uncertainty through nonlinear composite functions in BO, which is relevant for exploiting process knowledge in engineering applications. The derivation of the closed-form moments in Eq. (19) is straightforward and clearly presented, and the authors provide reproducible code and data on GitHub, which is a genuine strength. The two case studies are nontrivial and realistic, and the authors are transparent about the known inaccuracy of the Laplace approximation near feasibility bounds. However, the significance is currently tempered by three issues: the MC-BO baseline is run with an admitted underpowered sample size; the performance differences are reported without statistical significance measures; and the moment-accuracy validation is an internal consistency check against MC-BO on the same GP posterior rather than against an external ground truth. These issues leave the central claim 'as well as or better than existing grey-box methods' only partially supported, although the computational advantage over OP-BO is more plausible.","major_comments":[{"comment":"The MC-BO baseline is run with S=100 samples, and the text in Section 5.1 explicitly concedes that 'at a given point x with S=100, different evaluations of the LCB-MCBO AF could return values that differed by over 10%.' This is an admission that the comparator is underpowered, so the reported average improvements over MC-BO (1.2% and 3.3% in Study 1, 1.6% in Study 2) do not establish that BOIS outperforms a properly calibrated MC-BO. Furthermore, these differences are reported as point averages over 25 and 125 trials without error bars, confidence intervals, or paired significance tests, so sampling noise cannot be ruled out. The authors should rerun the MC-BO comparison with a converged sample size (or an adaptive budget) and report the performance differences with appropriate uncertainty quantification, for example paired bootstrap or Wilcoxon tests.","section":"Section 5.1 and Section 5.2"},{"comment":"The moment-accuracy validation compares BOIS's closed-form estimates of m_f(x) and sigma_f(x) against MC-BO estimates computed on the same trained GP posterior of y(x), rather than against true sampled cost values or an independent ground truth. Since the MC-BO reference is itself an estimator of the GP posterior moments (and is noisy at the S=100 baseline), the parity plots demonstrate internal consistency only; they do not validate that the linearization captures the true distribution of the cost function. The authors should validate the moments against direct samples of f(x, y(x)) (e.g., by sampling y from a high-fidelity simulator or the true system) or clearly state that only internal consistency is being checked.","section":"Section 5.1, Figure 9 and Section 5.2, Figure 12"},{"comment":"The closed-form variance in Eq. (19b) uses the GP posterior covariance matrix Σ^ℓ of y, but the paper describes a nested structure in which downstream GP models are evaluated using the current means of upstream GPs, without propagating upstream predictive uncertainty into the downstream inputs. As a result, the covariance used in Eq. (19b) may understate the true uncertainty of f(x, y(x)) in the nested setting, which is precisely the setting used in Study 1 to reduce input dimensionality. This is a potential gap in the uncertainty propagation that should be either justified analytically or quantified empirically, for example by comparing the closed-form variance against a fully propagated Monte Carlo estimate in a nested example.","section":"Section 4, Eqs. (17)-(19)"},{"comment":"The manuscript acknowledges that the Laplace approximation is not accurate near the feasibility bounds of the intermediate functions because the GP's symmetric Gaussian density assigns mass to infeasible values. Since the BOIS acquisition function is built from these moments, the statement that 'this was not an issue' is supported only by the aggregate benchmark results, not by any targeted analysis. The authors should either provide a sensitivity study on a problem whose optimum lies near a feasibility bound, or moderate the claim by explicitly stating that the method's accuracy near bounds remains an open limitation that could affect acquisition decisions in constrained problems.","section":"Section 5.2, near-bound behavior"}],"minor_comments":[{"comment":"The placeholder citation '(see ??)' should be replaced with a proper reference to the literature models for biogas production and CB harvesting.","section":"Section 5.2, first paragraph"},{"comment":"The phrases 'by 1.2% and 3.3% respectively' and 'by and average 5.4%' contain grammatical or typographical errors; the latter should read 'by an average of 5.4%'.","section":"Section 5.1 and Section 5.2"},{"comment":"The reference point notation ŷ_0^ℓ is used in Eqs. (18) and (19) but is not explicitly defined before its first use; the earlier expansion in Eq. (15) uses y_0. Please introduce the notation consistently.","section":"Section 4, Eq. (18)"},{"comment":"The statement that 'ϵ in BOIS was set to ŷ × 10^-3' is ambiguous for vector-valued ŷ; specify whether the scaling is applied elementwise or uses a norm.","section":"Section 5, paragraph 3"},{"comment":"Reference [14] contains a misspelling: 'Baysian Optimization' should be 'Bayesian Optimization'.","section":"References"},{"comment":"The notation m_f,s^ℓ(x) for individual sampled values of f is confusingly close to the notation m_f^ℓ(x) for the estimated mean; consider using f_s(x) for the samples.","section":"Algorithm 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is essentially an extended implementation and benchmark study built on the authors' own BOIS preprint [15]. The central empirical claim would be considerably strengthened by addressing the MC-BO baseline calibration and adding statistical inference; as written, the comparative advantage over MC-BO is not yet established, although the computational advantage over OP-BO is more credible. The manuscript also contains a visible placeholder citation and several typos, suggesting it is not fully polished. The scope is more applied than a typical stat.ML contribution, but the reproducible code and realistic case studies make it suitable for a journal interested in grey-box optimization and process systems engineering."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a solid engineering paper that extends the authors' own BOIS linearization idea with feasibility clipping, a nested intermediate-function structure, and two realistic chemical-process benchmarks. The core algorithm is not new—it is from their earlier preprint—but the implementation details and case studies are. The paper ships code and data, and it is refreshingly honest about where the Laplace approximation struggles.\n\nThe main empirical claim, that BOIS matches or beats existing grey-box methods at lower cost, is plausible but not fully established. The stress-test note is right: the MC-BO baseline used only S=100 samples, and the paper itself says this made the LCB-MCBO acquisition function noisy, with evaluations varying by over 10%. Running a comparator in a regime you already know is underpowered, then reporting 1.2–5.4% improvements without error bars or significance tests, leaves the headline claim shaky. The parity checks against MC-BO with S=1,000 partly rescue the moment-accuracy story, but that comparison is still against another approximate estimator on the same GP, not against true sampled cost values.\n\nThe other weaknesses are more minor. The computational advantage is real for AF evaluation, but the paper's claim that the performance function is evaluated only once conflicts with using finite-difference Jacobians, which require multiple evaluations. Nested uncertainty propagation is not addressed—the independent-GP assumption ignores correlation between intermediates. And near feasibility bounds the linearization is admittedly inaccurate, though the benchmarks suggest it does not hurt the search in practice.\n\nWho is this for? Process-systems engineers using BO on grey-box problems, and BO researchers interested in a careful case study. The contribution is incremental relative to the authors' prior BOIS preprint, but the benchmark results and the feasibility-clipping scheme are worth having in the literature.\n\nMy recommendation: send it to peer review. The method is coherent, the writing is clear, and the work is reproducible. A serious referee should push the authors to fix the MC-BO sample budget, add error bars or paired tests, and correct the computational claims. With those revisions, the empirical claims would be much stronger.","headline":"A well-executed implementation paper whose central performance claim is plausible but not yet proven: the MC-BO baseline is underpowered and the reported gaps lack significance tests.","tokens_in":19973,"tokens_out":1776,"would_cite":false,"duration_ms":18755,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BOIS uses an adaptive linearization of a known composite function to obtain closed-form estimates of performance mean and uncertainty, letting Bayesian optimization exploit structure instead of sampling.","keywords":["Bayesian optimization","grey-box modeling","composite functions","Gaussian processes","adaptive linearization","lower confidence bound","chemical process optimization","interconnected systems"],"falsifier":"On a synthetic composite with deliberately strong nonlinearity in $h$ (for example, $f(x,y)=g(x)+\\exp(y_1)+\\sin(y_2)$), compute the BOIS moments in (19) and compare them with Monte Carlo estimates using $10^4$ samples at 500 points; if the error in $\\sigma_f^\\ell$ grows with the GP variance and the acquisition function preferentially samples where the linearization is poor, the central premise fails. A cheaper check is near-boundary behavior: the paper's parity plots already show a visible bias in $\\sigma$ when the GP mean is clipped to a feasibility bound, so on a problem where the optimum lies near such a bound, one can test whether that bias changes the selected point.","tokens_in":18957,"feed_emoji":"🧪","tokens_out":5917,"duration_ms":50252,"temperature":0.7,"pith_summary":"This paper implements and benchmarks BOIS, a Bayesian optimization strategy for composite objective functions of the form $f(x,y(x))$, where the outer function $f$ is known and the intermediate vector $y(x)$ is learned from data with Gaussian process models. Propagating the Gaussian uncertainty in $y$ through a nonlinear $f$ is generally intractable; existing grey-box methods draw many Monte Carlo samples (MC-BO) or solve a larger constrained problem over an augmented space (OP-BO). BOIS instead linearizes $f$ in $y$ around an adaptively updated reference point, which converts the propagation into closed-form algebraic expressions for the mean and standard deviation of $f$. On two chemical-process benchmarks the paper reports that BOIS matches or beats MC-BO and OP-BO in solution quality while spending far less time optimizing its acquisition function, and that its moment estimates agree with high-sample Monte Carlo at a fraction of the cost.","feed_headline":"Closed-form moments make grey-box Bayesian optimization faster","feed_subtitle":"Local linearization replaces Monte Carlo sampling and beats two grey-box baselines on chemical-process benchmarks.","key_machinery":"The load-bearing mechanism is the adaptive linearization, a local Laplace approximation of $f$. At each candidate $x$, the Gaussian process supplies the mean $\\hat{y}^\\ell$ and covariance $\\hat{\\Sigma}^\\ell$ for $y$; feasibility bounds are enforced by clipping $\\hat{y}^\\ell$ to its allowable range; a reference point $\\hat{y}_0^\\ell$ is chosen within an $\\epsilon$-neighborhood; and the Jacobian $J=\\nabla_y h(x,\\hat{y}_0^\\ell)$ is evaluated. This single linear map turns the GP's Gaussian density on $y$ into an approximate Gaussian density on $f$, giving the closed-form moments in (19) that feed the lower confidence bound acquisition function. The linearization is rebuilt at every candidate point rather than extrapolated from previous points, and it supports nested GP structures in which some intermediate variables depend on others, which the paper uses to shrink the input spaces of the surrogate models.","core_discovery":"The central claim is that for a once-differentiable composite $f(x,y(x)) = g(x)+h(x,y)$, a local first-order Taylor expansion $f \\approx g(x)+h(x,\\hat{y}_0^\\ell)+J^T(y(x)-\\hat{y}_0^\\ell)$, with $J=\\nabla_y h$, is enough to drive grey-box Bayesian optimization. Because Gaussian densities are closed under affine maps, the framework obtains closed-form moments $m_f^\\ell(x) = J^T \\hat{y}^\\ell + g(x) + h(x,\\hat{y}_0^\\ell) - J^T \\hat{y}_0^\\ell$ and $\\sigma_f^\\ell(x) = (J^T \\hat{\\Sigma}^\\ell J)^{1/2}$, where $\\hat{y}^\\ell$ is the GP mean clipped to feasibility bounds and $\\hat{\\Sigma}^\\ell$ is the GP covariance. The paper argues that this makes the acquisition function cheaper to evaluate than Monte Carlo-based LCB-CF, and cheaper to optimize than the OP-BO auxiliary problem because optimization stays in the original $x$-space rather than an augmented $(x,y)$-space. On its two benchmarks this yields solutions as good as or better than MC-BO and OP-BO, with acquisition optimization about 41% faster than MC-BO and about 3.3 times faster than OP-BO in the first case study.","pith_inferences":["The paper's parity experiments suggest that BOIS's advantage over MC-BO is not only speed: with $S=1000$ Monte Carlo samples, MC-BO matches BOIS's accuracy but at roughly two orders of magnitude more compute, so the practical gain transfers to any setting where acquisition evaluation dominates the budget.","For strongly nonlinear $h$, such as exponential or trigonometric terms, the first-order expansion may understate the true variance in regions of high GP uncertainty; an immediate test would compare BOIS's $\\sigma$ to high-precision Monte Carlo on synthetic composites with known curvature.","The near-boundary bias the authors observe motivates a natural extension: replace the symmetric GP with a warped or otherwise non-Gaussian surrogate, or use the analytic moments only away from feasibility bounds while falling back to Monte Carlo near them."],"forward_implications":["If the closed-form moments are accurate, composite-function BO can exploit physics and connectivity without Monte Carlo, making grey-box optimization practical when each system query is expensive.","Because the acquisition function is optimized in the original $x$-space rather than an augmented $(x,y)$-space, BOIS remains cheap as the number of intermediate variables grows, and the paper shows this advantage widening in the higher-dimensional of its two case studies.","Feasibility constraints on intermediates are handled directly through the clipping step (17), so infeasible regions of $y$ do not need to be learned by sampling.","The same propagation step can feed other acquisition functions built from the closed-form moments, such as probability-of-improvement or expected-improvement variants, without changing the linearization machinery."],"supporting_citations":[{"why":"Proposes the BOIS framework and the adaptive linearization idea that this paper implements and benchmarks in detail.","marker":"[15]"},{"why":"Introduces composite-function BO and the Monte Carlo estimation approach that serves as the MC-BO baseline.","marker":"[2]"},{"why":"Proposes the optimism-driven auxiliary problem over an augmented $(x,y)$ space that serves as the OP-BO baseline.","marker":"[43]"},{"why":"Shows how derivative information and feasibility constraints can be exploited in grey-box BO, motivating the feasibility handling in BOIS.","marker":"[27]"},{"why":"Supplies the Monte Carlo machinery used for the MC-BO comparison and moment estimation.","marker":"[6]"}],"fun_headline_variants":["Linearized moments speed grey-box Bayesian optimization","Analytic moments accelerate grey-box Bayesian optimization","Grey-box BO accelerates via local linearization","Closed-form GP moments fast-track grey-box optimization","Faster grey-box BO with analytical moments"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method stands on the assumption that a first-order Taylor expansion of the known function $f(x,y)$ around a single reference value of $y$ stays accurate across the whole region where the Gaussian process is uncertain, which the paper itself notes degrades far from the reference point and near the feasibility bounds of $y$.","fun_headline_variants_meta":{"raw":{"variants":["Linearized moments speed grey-box Bayesian optimization","Analytic moments accelerate grey-box Bayesian optimization","Grey-box BO accelerates via local linearization","Closed-form GP moments fast-track grey-box optimization","Faster grey-box BO with analytical moments"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000434,"raw_usage":{"total_tokens":2305,"prompt_tokens":1137,"completion_tokens":1168,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":753,"completion_tokens_details":{"reasoning_tokens":1100}},"tokens_in":753,"tokens_out":1168,"duration_ms":8718,"temperature":1.0,"reasoning_tokens":1100,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:38:19.530267+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a synthetic composite with deliberately strong nonlinearity in $h$ (for example, $f(x,y)=g(x)+\\exp(y_1)+\\sin(y_2)$), compute the BOIS moments in (19) and compare them with Monte Carlo estimates using $10^4$ samples at 500 points; if the error in $\\sigma_f^\\ell$ grows with the GP variance and the acquisition function preferentially samples where the linearization is poor, the central premise fails. A cheaper check is near-boundary behavior: the paper's parity plots already show a visible bias in $\\sigma$ when the GP mean is clipped to a feasibility bound, so on a problem where the optimum lies near such a bound, one can test whether that bias changes the selected point.","supporting_citations":[{"cited_title":"BOIS: Bayesian Optimization of Interconnected Systems","cited_arxiv_id":"2311.11254","evidence_quote":"Proposes the BOIS framework and the adaptive linearization idea that this paper implements and benchmarks in detail."},{"cited_title":"Astudillo and P","cited_arxiv_id":null,"evidence_quote":"Introduces composite-function BO and the Monte Carlo estimation approach that serves as the MC-BO baseline."},{"cited_title":"Bayesian Optimization of Expensive Nested Grey-Box Functions","cited_arxiv_id":"2306.05150","evidence_quote":"Proposes the optimism-driven auxiliary problem over an augmented $(x,y)$ space that serves as the OP-BO baseline."},{"cited_title":"Paulson and C","cited_arxiv_id":null,"evidence_quote":"Shows how derivative information and feasibility constraints can be exploited in grey-box BO, motivating the feasibility handling in BOIS."},{"cited_title":"Balandat, B","cited_arxiv_id":null,"evidence_quote":"Supplies the Monte Carlo machinery used for the MC-BO comparison and moment estimation."}],"review_version":1}