Constrained Bayesian Optimization under Bivariate Gaussian Process with Application to Cure Process Optimization

Gang Li; Madhura Limaye; Qiong Zhang; Yezhuo Li

REVIEW 4 major objections 5 minor 18 references

Constrained Bayesian Optimization under Bivariate Gaussian Process with Application to Cure Process Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that constrained Bayesian optimization can account for dependence between objective and constraint functions by modeling them with a bivariate Gaussian process and using a closed-form expected constrained improvement…

desk verdict A real but narrow extension of constrained BO to correlated objective and constraint, with a sloppy derivation of the predictive distribution and an application that shows no practical gain; worth refereeing, but expects major revision. read the letter →

arxiv 2506.00174 v1 pith:IBNVQ4ZB submitted 2025-05-30 stat.CO stat.APstat.ML

Yezhuo Li , Qiong Zhang , Madhura Limaye , Gang Li This is my paper

classification stat.COstat.APstat.ML

keywords BayesianoptimizationconstrainedbivariateGaussianprocessexpectedimprovementseparablecovariancecure

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Bayesian optimization for problems with an unknown constraint usually models the objective and the constraint as two independent Gaussian processes. This paper drops that independence assumption and models the pair with a bivariate Gaussian process in which the objective and constraint share a correlation function, then derives a closed-form expected constrained improvement acquisition function. The closed form matters because it lets the optimizer rank candidate points using the joint distribution of objective and constraint without Monte Carlo integration. Applied to a composite cure process, the method reaches essentially the same final deformation as the independence-based method but takes about three times longer to run, which the paper interprets as a sign that the separable covariance function does not capture the true dependence.

What carries the argument

The central object is the bivariate Gaussian process with a separable covariance: $Cov(y(x), y(x'))=\sigma_y^2 R(x,x')$, $Cov(z(x), z(x'))=\sigma_z^2 R(x,x')$, and $Cov(y(x), z(x'))=\rho\,\sigma_y\sigma_z R(x,x')$, so the joint covariance of the stacked outputs is the Kronecker product of a $2\times 2$ dependence matrix and the correlation matrix $R$. This structure gives the marginal conditional means and variances in (11), and the truncated-bivariate-normal expectation in Lemma 1 converts the constrained improvement expectation into the closed form of Proposition 1. The numerical-stability approximations target the same structure when CDF values approach the boundary.

What would settle it

Take a data set generated from a known bivariate Gaussian process with a chosen $\rho$, compute the exact joint predictive distribution of $(y(x), z(x))$ given the observations, and compare the paper's closed-form ECI with a Monte Carlo estimate of $E[\max\{0, y_{\min}-y(x)\}\mathbf{1}\{z(x)\ge c\}]$ under that joint posterior; a systematic gap would show the formula is not the acquisition function of the model.

Watch

Extended reading notes

Core claim

On the paper's own terms, the contribution is Proposition 1: under the bivariate Gaussian process model of Section 3, the expected constrained improvement at a candidate point is $ECI(x) = (t_1(x)+t_2(x))\,t_3(x)$, with $t_1$, $t_2$, $t_3$ explicit expressions built from univariate and bivariate standard normal CDFs and PDFs evaluated at the model parameters. The proof uses a lemma on the expectation of a truncated bivariate normal distribution, and the paper supplies numerical-stability approximations for the cases where the normal CDF is near 0 or 1. The cure-process application compares this acquisition with the independence-based constrained BO and finds comparable final objective values with about three times the computation cost, leading the authors to conclude that the separable covariance assumed in the model may not be accurate for that problem.

Load-bearing premise

The load-bearing premise is that after data have been observed, the objective and constraint at any new point are still jointly normal with the fixed correlation $\rho$ and the parameters used in Proposition 1; the paper derives only the marginal conditional distributions, never the joint conditional distribution, so the formula stands on that unproven joint-normality assumption.

Editorial extensions

If this is right

Constrained Bayesian optimization can be applied when the objective and constraint are believed to be dependent, without giving up a closed-form acquisition function.
At each iteration the next point is chosen by evaluating the formula directly, so the added dependence modeling does not require an inner Monte Carlo loop over the joint distribution.
When the separable covariance is a good description, the formula is an exact ECI; when it is not, the cure-process results suggest it may behave like the independence-based method while costing more.
Practitioners using the method should check whether the separable covariance is appropriate, because the application shows a fidelity gain that did not translate into a better optimum.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A step the paper leaves open is deriving the joint predictive distribution of $(y(x), z(x))$ conditional on the full observed data $y_n$ and $z_n$; Proposition 1 appears to use the unconditional bivariate normal parameters, and the conditional correlation is generally not $\rho$, so the closed form may not be the true ECI of the bivariate GP model.
A direct check would be to simulate from a known bivariate GP and compare the formula against a Monte Carlo estimate of the constrained improvement taken under the joint posterior.
The same truncated-normal lemma should yield an ECI for non-separable covariance functions, which is the direction the paper itself suggests for improving the model.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's contribution is a closed-form expected constrained improvement (ECI) for Bayesian optimization when the objective and constraint are modeled by a bivariate Gaussian process with a separable covariance. That formula is genuinely new relative to Gardner et al.'s independence assumption, and it follows cleanly from standard truncated bivariate normal moments. The stress-test confirms what the reader's report got wrong: under the separable covariance in (8), the conditional correlation between y(x) and z(x) given the observed data remains rho, so Proposition 1 is correct if the μ and σ terms are read as the conditional moments from (11). The paper, however, never states this. Section 3 gives only marginal conditionals, and the proof of Proposition 1 refers vaguely to 'the bivariate normal distribution of (y(x), z(x)) for a fixed x' without saying whether it is the prior or the posterior. That is a real expository gap, and the reader's rejection is understandable, but the gap is fixable with a short derivation.

Where the paper is softest is the empirical side. The case study shows no improvement over the independent baseline and reports about three times the computation cost; the authors honestly admit the separable covariance may not fit the cure process, which is creditworthy, but it means the method's practical value is not demonstrated. There is no synthetic experiment with a known correlation to validate that the closed-form ECI actually exploits dependence. The parameter estimation is also two-stage—means and variances are estimated ignoring rho, then rho is fitted conditionally—and the paper does not discuss the consequences. Finally, Lemma 1 in the appendix has a typo, using c instead of v in a denominator.

This is a minor but legitimate extension, neither as flawed as the reader's report claims nor as polished as it could be. The math is probably right once the conditioning is made explicit, and the paper would be useful to the constrained BO community if it were revised to clarify the predictive distribution, fix the lemma, and add experiments showing where correlated modeling helps.

Recommendation: treat this as a borderline paper that deserves a serious referee. Send it to peer review with the expectation of major revision.

Referee Report

4 major / 5 minor

Summary. The paper proposes a constrained Bayesian optimization method for problems with one objective and one constraint when the two black-box functions may be dependent. It replaces the independent Gaussian process surrogates of Gardner et al. with a bivariate Gaussian process model under a separable covariance structure, and it claims a closed-form expected constrained improvement (ECI) acquisition function in Proposition 1. The authors estimate the model parameters by maximum likelihood and compare the resulting method with the independent-GP approach on a simulated cure-process optimization study, finding similar convergence and roughly three times higher computation time. The main theoretical contribution is the closed-form ECI formula, but the derivation and the sequential updating of the acquisition function are not made explicit, and several technical details in the supporting equations need correction.

Significance. If Proposition 1 were correctly derived for the posterior predictive distribution of the bivariate GP, the paper would provide a useful extension of constrained Bayesian optimization to dependent objective and constraint functions with a closed-form acquisition function, avoiding numerical quadrature. The separable covariance model is a natural first step, and the cure-process application with a pseudo-simulator and 30 replications is a reasonable demonstration setup. The manuscript is not ready in its current form, however: the central theorem is not actually derived for the model's predictive distribution, and the appendix proof is little more than a reference to an unproved lemma. The main result is likely repairable, but the missing joint-posterior derivation is a load-bearing gap that must be addressed before the paper can be recommended for publication.

major comments (4)

[Section 4, Proposition 1 (with Eqs. (7)-(9))] The closed form in Proposition 1 is stated for the bivariate normal distribution with parameters (µ_y, µ_z, σ_y, σ_z, ρ) of Eq. (9), but the ECI in Eq. (7) is defined with the expectation under the conditional distributions y(x)|yn and z(x)|zn, and the end of Section 2 says the acquisition function should be based on the joint conditional distribution. The paper never states the joint conditional law of (y(x), z(x)) given (yn, zn); Eq. (11) gives only the two marginal conditional distributions. Under the separable covariance in Eq. (8), the joint predictive distribution is in fact bivariate normal with the conditional means from Eq. (11) and covariance matrix q(x)Σ0, so the conditional correlation is still ρ, but this derivation is absent from the manuscript. If Proposition 1 is evaluated literally with the unconditional parameters, the resulting ECI(x) depends on the observed responses only through global parameter estimates and is not a valid sequential acquisition function. Please add the joint posterior derivation and restate Proposition 1 with explicit conditional moments.
[Appendix, Lemma 1 and Eq. (15)] Lemma 1 is the only proof step offered for Proposition 1, but it is not proved and its displayed formula contains an apparent typo: the first denominator uses (c − µ_v)/σ_v instead of (v − µ_v)/σ_v. The transition from Lemma 1 to the ECI expression is also not shown; the text only says that the expression can be obtained 'accordingly.' Since Proposition 1 is the main contribution, please provide a complete derivation or a precise citation to a source that contains the exact formula, correct the typo, and verify the sign of the second term in Eq. (15).
[Section 2, Eq. (4) (compare with Eq. (11))] The predictive variance s²(x) in Eq. (4) is written with a minus sign before (1 − 1ᵀR⁻¹r(x))²/(1ᵀR⁻¹1). For the standard Gaussian process model with unknown constant mean, the kriging variance has a plus sign in that term; Eq. (11) uses a plus sign. If Eq. (4) is not a typo, the expected improvement in Eq. (5) is computed with an incorrect predictive variance, and the inconsistency between Eq. (4) and Eq. (11) must be resolved.
[Section 4, paragraph after Proposition 1] The limiting approximations for ECI as Φ((c − µ_z)/σ_z) → 1 are asserted without proof or stated conditions. The implementation is said to be based on these approximations, so either derive them rigorously or present them explicitly as heuristics with a numerical safeguard, and state the range of parameters over which they are reliable.

minor comments (5)

[Abstract and Introduction] There are several grammatical errors, including 'constrained Bayesian Optimization in is developed,' 'we show case the performance,' and 'We include a bivariate Gaussian process model;' these should be corrected throughout.
[Section 2, Eq. (5)] The phrase 'expected contained improvement' appears in Eq. (7) and elsewhere; the standard term is 'expected constrained improvement' or 'expected improvement with constraints.'
[Algorithm 1] The algorithm pseudocode is terse and the 'Preliminary' step does not state how the initial surrogate models are obtained; please specify the initialization procedure and the parameter estimation steps.
[Section 5] The empirical comparison reports only mean curves and confidence bands; a table with final optimization values, standard errors, and computation times would make the claimed 'similar performance' and the threefold runtime increase more precise and reproducible.
[References] Reference [11] is described as an unpublished manuscript; if it is under review or available as a preprint, please provide the appropriate citation details.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the closed-form ECI is derived from an external truncated bivariate normal moment formula with the fitted bivariate GP covariance parameters as model inputs, and the central derivation does not presuppose its own conclusion.

full rationale

Walking the derivation chain, the load-bearing step is Proposition 1's use of Lemma 1, a truncated bivariate normal expectation formula cited to Rosenbaum (1961), to evaluate ECI(x) = E{max(0, ymin - y(x)) I(z(x) >= c)}. Lemma 1 is an external mathematical result and is not derived from the paper's own conclusions. The parameters mu_y, mu_z, sigma_y, sigma_z, and rho in the ECI formula are the covariance parameters of the bivariate Gaussian process in (8)-(10); they are fitted from data, not constructed to reproduce the ECI. The paper's numerical finding that the proposed method performs similarly to the independent-GP constrained BO is empirical and is presented as a limitation, not as a validation that forces the derivation. The only potentially questionable point is that Proposition 1 states the closed form using the unconditional bivariate normal parameters, while in sequential BO the acquisition function should arguably be evaluated under the posterior predictive parameters from (11); that is a possible gap in justification or correctness, but it is not a circular reduction because the claimed formula does not equal its own input by definition. Self-citations such as [11] and [9] are used for the application pseudo-simulator and are not load-bearing for the closed-form derivation. The threshold typo in Lemma 1 (using c instead of v in one denominator) is an editing error, not circularity. Therefore no significant circular reasoning is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central formula depends on a fitted correlation parameter rho and on an assumed bivariate normal predictive distribution whose joint conditional version is not derived. No new physical entities are introduced.

free parameters (1)

rho (correlation between objective and constraint) = estimated by maximum likelihood, value not reported
Introduced in Eq. (8) to model dependence; estimated from data in Eq. (14); the ECI formula in Proposition 1 depends directly on it.

assumptions (3)

domain assumption Bivariate Gaussian process with separable covariance: Cov(y(x), z(x')) = rho sigma_y sigma_z R(x, x')
Assumed in Eq. (8) without empirical justification beyond the motivating application; the whole acquisition function rests on this model.
ad hoc to paper At each candidate point, (y(x), z(x)) has the bivariate normal distribution with the parameters used in Proposition 1
The paper derives only the marginal conditional distributions in (11), and never derives the joint posterior predictive covariance or correlation; Proposition 1 silently uses rho and sigma parameters as if they were the predictive ones.
standard math Truncated bivariate normal moment formula in Lemma 1
Taken from Rosenbaum (1961); used to turn the ECI expectation into the closed form in Proposition 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Constrained Bayesian Optimization under Bivariate Gaussian Process with Application to Cure Process Optimization." pith.science (2026). https://pith.science/paper/IBNVQ4ZB

@misc{pith2026250600174,
  author       = {Pith},
  title        = {Pith review of: Constrained Bayesian Optimization under Bivariate Gaussian Process with Application to Cure Process Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBNVQ4ZB}},
  note         = {Machine review of arXiv:2506.00174}
}

read the original abstract

Bayesian Optimization, leveraging Gaussian process models, has proven to be a powerful tool for minimizing expensive-to-evaluate objective functions by efficiently exploring the search space. Extensions such as constrained Bayesian Optimization have further enhanced Bayesian Optimization's utility in practical scenarios by focusing the search within feasible regions defined by a black-box constraint function. However, constrained Bayesian Optimization in is developed based on the independence Gaussian processes assumption between objective and constraint functions, which may not hold in real-world applications. To address this issue, we use the bivariate Gaussian process model to characterize the dependence between the objective and constraint functions and developed the constrained expected improvement acquisition function under this model assumption. We show case the performance of the proposed approach with an application to cure process optimization in Manufacturing.

Figures

Figures reproduced from arXiv: 2506.00174 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 15 canonical work pages

[1]

Computationally efficient convolved multiple output gaussian processes

Mauricio A Alvarez and Neil D Lawrence. Computationally efficient convolved multiple output gaussian processes. The Journal of Machine Learning Research , 12:1459–1500, 2011

work page 2011
[2]

Springer Science & Business Media, 2008

J¨ urgen Branke.Multiobjective optimization: Interactive and evolutionary approaches , volume 5252. Springer Science & Business Media, 2008

work page 2008
[3]

A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning

Eric Brochu, Vlad M Cora, and Nando De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 , 2010

arXiv 2010
[4]

Peter I. Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811 , 2018

arXiv 2018
[5]

Bayesian optimization with inequality constraints

Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunning- ham. Bayesian optimization with inequality constraints. In ICML, volume 2014, pages 937–945, 2014

work page 2014
[6]

Handbook of spatial statistics

Alan E Gelfand, Peter Diggle, Peter Guttorp, and Montserrat Fuentes. Handbook of spatial statistics. CRC press, 2010

work page 2010
[7]

Development of a theoretical cost model for advanced composite fabrication

Timothy Gutowski, David Hoult, Greg Dillon, Ein-Teck Neoh, Stuart Muter, Eric Kim, and Mawuli Tse. Development of a theoretical cost model for advanced composite fabrication. Composites Manufacturing, 5(4):231–239, 1994

work page 1994
[8]

Efficient global optimization of expensive black-box functions

Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization , 13:455–492, 1998

work page 1998

Show all 18 references

[9]

Sequential selection for minimizing the variance with application to crystallization experiments

Caroline M Kerfonta, Sunuk Kim, Ye Chen, Qiong Zhang, and Mo Jiang. Sequential selection for minimizing the variance with application to crystallization experiments. The American Statistician, pages 1–10, 2024

2024
[10]

Optimization by simulated annealing

Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. Optimization by simulated annealing. science, 220(4598):671–680, 1983. 10

1983
[11]

Numerical simulation informed rapid cure process optimization of composite structures using constrained bayesian optimization

Madhura Limaye, Yezhuo Li, Qiong Zhang, and Gang Li. Numerical simulation informed rapid cure process optimization of composite structures using constrained bayesian optimization. structural and multidisciplinary optimization. Unpublished manuscript, [Clemson University], 2024

2024
[12]

Computational analysis and optimization of cure process of fiber reinforced polymer composites for reducing process-induced warpage

Madhura Limaye, Dwayne Morgan, and Gang Li. Computational analysis and optimization of cure process of fiber reinforced polymer composites for reducing process-induced warpage. Composite structures, 2024

2024
[13]

Bayesian approach adapting stochastic and heuristic methods of global and discrete optimization

Audrius Mockus, Jonas Mockus, and Linas Mockus. Bayesian approach adapting stochastic and heuristic methods of global and discrete optimization. Informatica, 5(1-2):123–166, 1994

1994
[14]

Moments of a truncated bivariate normal distribution

S Rosenbaum. Moments of a truncated bivariate normal distribution. Journal of the Royal Statistical Society Series B: Statistical Methodology , 23(2):405–408, 1961

1961
[15]

Designs for computer experiments

Jerome Sacks, Susannah B Schiller, and William J Welch. Designs for computer experiments. Tech- nometrics, 31(1):41–47, 1989

1989
[16]

Manipulating the multivariate gaussian density

Thomas B Sch¨ on and Fredrik Lindsten. Manipulating the multivariate gaussian density. Div. Au- tomat. Control, Link¨ oping Univ., Linkping, Sweden, Tech. Rep, 4(3.4):4, 2011

2011
[17]

Practical bayesian optimization of machine learning algorithms

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems , 25, 2012

2012
[18]

Scalable bayesian optimization using deep neural networks

Jasper Snoek, Oren Rippel, Kevin Swersky, Ryan Kiros, Nadathur Satish, Narayanan Sundaram, Mostofa Patwary, Mr Prabhat, and Ryan Adams. Scalable bayesian optimization using deep neural networks. In International conference on machine learning , pages 2171–2180. PMLR, 2015. 11

2015

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Computationally efficient convolved multiple output gaussian processes

Mauricio A Alvarez and Neil D Lawrence. Computationally efficient convolved multiple output gaussian processes. The Journal of Machine Learning Research , 12:1459–1500, 2011

work page 2011

[2] [2]

Springer Science & Business Media, 2008

J¨ urgen Branke.Multiobjective optimization: Interactive and evolutionary approaches , volume 5252. Springer Science & Business Media, 2008

work page 2008

[3] [3]

A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning

Eric Brochu, Vlad M Cora, and Nando De Freitas. A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 , 2010

arXiv 2010

[4] [4]

Peter I. Frazier. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811 , 2018

arXiv 2018

[5] [5]

Bayesian optimization with inequality constraints

Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunning- ham. Bayesian optimization with inequality constraints. In ICML, volume 2014, pages 937–945, 2014

work page 2014

[6] [6]

Handbook of spatial statistics

Alan E Gelfand, Peter Diggle, Peter Guttorp, and Montserrat Fuentes. Handbook of spatial statistics. CRC press, 2010

work page 2010

[7] [7]

Development of a theoretical cost model for advanced composite fabrication

Timothy Gutowski, David Hoult, Greg Dillon, Ein-Teck Neoh, Stuart Muter, Eric Kim, and Mawuli Tse. Development of a theoretical cost model for advanced composite fabrication. Composites Manufacturing, 5(4):231–239, 1994

work page 1994

[8] [8]

Efficient global optimization of expensive black-box functions

Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization , 13:455–492, 1998

work page 1998