Uncertainty Estimation of the Optimal Decision with Application to Cure Process Optimization

Gang Li; Madhura Limaye; Qiong Zhang; Yezhuo Li

REVIEW 3 major objections 5 minor 17 references

Uncertainty Estimation of the Optimal Decision with Application to Cure Process Optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By minimizing random draws of a fitted Gaussian process, the framework turns a single recommended setting into an empirical distribution of optimal decisions and uses it in Sobol sensitivity analysis to rank parameter contributions.

desk verdict A practical recipe for propagating GP posterior uncertainty into optimal decisions, with a sensitivity-analysis extension that rests on an independence assumption the decision sample violates. read the letter →

arxiv 2507.21995 v1 pith:JO4COQCB submitted 2025-07-29 stat.AP

Yezhuo Li , Qiong Zhang , Madhura Limaye , Gang Li This is my paper · ORCID

classification stat.AP

keywords GaussianprocesssurrogatedecisionuncertaintyestimationoptimaldistributionSobolsensitivityanalysisblack-boxconstrainedoptimizationcurecomputerexperimentsLatinhypercubesampling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Manufacturers often optimize process parameters using a Gaussian process surrogate trained on a small number of expensive simulation runs. The paper proposes that the uncertainty of the resulting optimal decision can be estimated by drawing many independent realizations of the fitted conditional Gaussian process, minimizing each realization, and collecting the minimizers into an empirical distribution. That distribution is then used as the input distribution for a variance-based sensitivity analysis (Sobol indices), so the reported numbers say how each decision variable contributes to output variability near the optimum rather than across the whole input range. If the approach holds, a practitioner can attach an uncertainty statement to a recommended process setting and a ranking of which parameters deserve follow-up experiments.

What carries the argument

The load-bearing mechanism is the conditional Gaussian process (5), with posterior mean $\hat{y}(x)$ and covariance $\Sigma(x,x')$, combined with the argmin operation: minimizing each independent posterior draw produces the decision uncertainty sample (7). The second stage uses this sample as the input distribution for the Sobol first-order and total sensitivity indices in (9), approximated with the sobolSalt package. For black-box constrained problems, an independent Gaussian process for the constraint function (14) defines feasibility, and the same argmin step is applied only to realizations that satisfy the constraint.

What would settle it

Run the procedure on a test function with a known optimum and compare the empirical distribution of minimizers and the resulting Sobol indices at grid sizes 100, 1000, and 10000 with M fixed; if the distribution shifts substantially with the grid size instead of stabilizing, the finite-grid argmin, not the fitted Gaussian process, is producing the reported decision uncertainty.

Watch

Extended reading notes

Core claim

Using the fitted conditional Gaussian process (5) with posterior mean $\hat{y}(x)$ and covariance $\Sigma(x,x')$, the paper obtains $M$ realizations, evaluates each on a Latin hypercube grid, and takes the minimizer of each realization, producing the decision uncertainty sample $x^*_{(1),N},\ldots,x^*_{(M),N}$. The central claim is that this sample is a valid empirical distribution for the optimal decision $x^*$, and that plugging it into the Sobol first-order and total indices (9) quantifies how each decision variable contributes to output uncertainty near the optimum. In the two-dimensional illustration, the decision-uncertainty view splits the uncertainty contribution roughly one-third to $x_1$ and two-thirds to $x_2$, a different message from the uniform-input analysis. In the composite cure process, the framework yields densities around the two cure-cycle change points and identifies $T_1$ as the most influential parameter for the first change point and $t_2$ for the second.

Load-bearing premise

The method assumes that the finite collection of near-optimal points found by minimizing random model draws can be treated as a genuine sample from the decision distribution and plugged into sensitivity calculations that assume independent inputs; if the grid is too coarse or the decision coordinates are correlated, the uncertainty estimates and parameter rankings may be misleading.

Editorial extensions

If this is right

A single recommended process setting can be replaced by an empirical distribution of near-optimal settings, so decision makers can see which parameters are tightly pinned down by the data and which are interchangeable.
Sobol indices computed under decision uncertainty rank the parameters by contribution to output uncertainty at the optimum, identifying where extra simulation effort or experimental follow-up is most valuable.
The same machinery extends to black-box constrained optimization: independent Gaussian process realizations of the constraint define feasibility, and only feasible realizations of the objective are minimized.
In the cure process application, the second change point has visibly higher uncertainty than the first, and the sensitivity analysis singles out $T_1$ as a priority for follow-up investigation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors do not pursue it, but the same decision-uncertainty sample could be summarized as a credible region around the recommended setting, giving a direct uncertainty budget without computing sensitivity indices.
A natural extension is to replace the Sobol formulas with a variance decomposition that allows dependent inputs, since the coordinates of the optimized decision points are not guaranteed to be independent.
The mechanism transfers to any surrogate from which posterior draws can be sampled cheaply after fitting, provided the finite-grid argmin approximation is controlled.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take. The decision-uncertainty part is a sensible Monte Carlo workflow: fit a GP to simulation data, sample posterior realizations, optimize each realization, and summarize the argmin distribution. That gives practitioners exactly the kind of statement they want about the reliability of recommended settings. The cure-process application is real, the authors ship code and data on GitHub, and the constrained version with two GPs is a reasonable extension. Credit where due: the implementation is careful and the exposition is clear.

The soft spot is the sensitivity analysis. Section 3 says it computes Sobol indices with expectations taken with respect to the empirical distribution of (7), and then uses sobolSalt. But the decision sample has dependent coordinates — the argmins cluster along a low-dimensional manifold in the cure example. Standard Saltelli estimators are derived for independent inputs; permuting columns of a dependent sample produces points that are not draws from the joint distribution, so the sobolSalt output is not a consistent estimate of the Si and STi defined in (9). That means Figures 2 and 5 are not established. This is a load-bearing issue for the sensitivity claims, though not for the uncertainty-estimation part. A second, smaller issue is the finite-LHS argmin: the grid approximation bias is not quantified and it propagates into both the contours and the indices. The abstract's phrase 'sensitivity with respect to the objective function' also overstates things; what is actually done is sensitivity of output variance to decision-variable uncertainty.

The paper would benefit from acknowledging these limitations and either changing the sensitivity method (e.g., dependent-input indices or an interpretability measure that accounts for the manifold) or framing the sobolSalt results as heuristic. The central decision-uncertainty idea remains useful for practitioners.

For peer review, I'd send it out — the application is real and the methodological flaw is fixable. It deserves a serious referee, but the sensitivity claims need major revision.

Referee Report

3 major / 5 minor

Summary. The paper proposes a Gaussian-process-based framework for estimating the uncertainty of the optimal decision in black-box optimization problems. The method fits a GP surrogate to simulation data, draws M independent realizations from the conditional GP, finds the argmin of each realization over a random Latin hypercube grid, and uses the resulting collection of argmins as an empirical distribution of the optimal decision. This distribution is then used as an input distribution in a Sobol sensitivity analysis, implemented with the R package sobolSalt, to rank the contributions of each decision variable to the output uncertainty near the optimum. The approach is illustrated on a two-dimensional toy example and applied to a composite cure process optimization problem with a black-box degree-of-cure constraint.

Significance. The proposed Monte Carlo approach to posterior decision uncertainty is coherent and computationally practical, and the application to a realistic cure process simulation with n=50 runs and a constraint is a useful demonstration. The paper also provides open-source code and data, which supports reproducibility. However, the sensitivity-analysis component, which is a central contribution of the paper, is implemented under an independence assumption that the decision-uncertainty sample does not satisfy; this issue, together with the unquantified finite-grid approximation bias, limits the validity of the reported sensitivity rankings.

major comments (3)

[Section 3, Eq. (9), Figures 2 and 5] The paper states that the decision-uncertainty sample in (7) can be used as the input distribution for Sobol indices computed with the R package sobolSalt. However, sobolSalt implements Saltelli-type estimators that are valid for independent inputs; the sample (7) has dependent coordinates, since each x*_(i),N is the argmin of a single GP realization and the coordinates tend to lie on a low-dimensional manifold (as seen in Figure 4). Permuting columns of this dependent sample, as done by the Saltelli scheme, generates combinations that are not draws from the joint empirical distribution, so the estimates are not consistent for the quantities defined in (9). Consequently, the sensitivity ratios reported in Figures 2 and 5 are not established. The authors should either employ a sensitivity-analysis method that explicitly handles dependent inputs (e.g., Shapley effects or a GP-based index computed under the posterior distribution) or clearly redefine the reported quantities as a heuristic permutation-based measure.
[Section 2, Eq. (7) and Appendix] The optimal decisions in the sample (7) are obtained by minimizing each GP realization over a per-realization Latin hypercube of size N, with N=1000 in the two-dimensional illustration and N=500 in the four-dimensional application. The empirical distribution of x*_(i),N approximates the distribution of the continuous argmin only up to a discretization bias that is not quantified. With N=500 in four dimensions, the grid is sparse, and the bias propagates into the uncertainty contours (Figures 1 and 4) and into the sensitivity indices (Figures 2 and 5). The paper should provide a convergence check (e.g., recomputing the sample and the indices for a range of N values) or an error bound to demonstrate that the reported results are insensitive to N.
[Section 3, Eq. (9)] The Sobol' first-order and total indices in (9) are customarily defined for independent input variables, and their variance-decomposition interpretation depends on that assumption. When p(x1,...,xd) is taken to be the empirical distribution of (7), which has dependent coordinates, the conditional-variance ratios in (9) are still mathematically well-defined but no longer correspond to a unique ANOVA decomposition, and the interpretation that Si and STi measure the contribution of xi's uncertainty to the output uncertainty is not automatically valid. The paper should either derive the interpretation under dependence or restrict the claims accordingly.

minor comments (5)

[Section 1, paragraph 3] The phrase 'To the best of out knowledge' should read 'To the best of our knowledge.'
[Section 4, Figure 4] The sentence 'This figure shows that, the second change point has higher uncertainty than the second change point' contains a typo and should presumably read '...the second change point has higher uncertainty than the first change point.'
[Section 3, paragraph 1] The phrase 'Those sensitivity indices are usually computed based on the independent uniform distributions in each dimension of the decision variable' would be clearer as '...based on independent uniform distributions for each decision variable.'
[Section 2, implementation paragraph] The notation X(i),N for the set of grid points and x*_(i),N for the sampled optimum are visually similar; consider using different symbols (e.g., G_i for the grid) to avoid confusion.
[Section 4, Figure 5] The paper does not state explicitly whether the sensitivity indices in Figure 5 are computed with the true simulation output or with the GP surrogate; please clarify in the caption or text.

Circularity Check

0 steps flagged · score 2.0 of 10

No by-construction circularity: the GP posterior-sampling and plug-in Sobol steps are a Monte Carlo procedure, not a self-defined prediction; the only author-overlap dependency is a minor, non-load-bearing citation for the cure-process setup.

full rationale

The derivation is self-contained. Section 2 fits the GP in (2), forms the conditional GP in (5), draws M independent realizations, and optimizes each on an LHS grid to produce the decision uncertainty sample (7); this is standard posterior predictive Monte Carlo for the argmin. Section 3 treats the empirical distribution of (7) as the input distribution in definition (9) and estimates the indices with sobolSalt; this is a plug-in estimate and any invalidity from using a dependent sample with an independence-based Saltelli estimator is a correctness/validity concern, not a circular reduction. The analytic example in (8) provides an external, non-fitted benchmark. The only same-author citation is Limaye et al. (2025), which supplies the cure-process problem setup (change points, constraints, L-shaped laminate, n=50 data) but does not justify the GP sampling, the argmin algorithm, or the Sobol definitions; hence it is not load-bearing for the central claim. No equation is defined in terms of its own output, no fitted parameter is relabeled as a prediction, and no uniqueness theorem is imported. The low score reflects the minor self-citation in the application section, not a circular derivation.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method rests on standard GP assumptions (stationarity, Matern covariance, known hyperparameters after MLE) and on two assumptions the paper does not justify: objective and constraint are independent GPs, and a finite LHD argmin is a good proxy for the continuous argmin. The sensitivity step quietly assumes standard Sobol formulas remain valid for a dependent empirical input distribution. These are the main structural premises beyond the GP machinery itself.

free parameters (4)

GP correlation length scales ℓ_1,...,ℓ_d = Estimated by MLE; values not reported in the paper
Matern 5/2 correlation parameters estimated from the data and treated as known when sampling posterior realizations; they directly control the spread of the decision uncertainty sample.
GP mean µ and variance σ² = MLE via equations (4); values not reported
Estimated from the data and used as fixed in the conditional GP in (5); hyperparameter uncertainty is not propagated.
Candidate grid size N per realization = 1000 for the illustration, 500 for the cure application
Chosen by hand to discretize the input space; no convergence analysis or sensitivity check is provided.
Number of posterior realizations M = 500 for the illustration, 10000 for the cure application
Chosen by hand for Monte Carlo sampling; no standard errors or confidence intervals are reported for the resulting estimates.

assumptions (5)

domain assumption The simulator output y(x) is a realization of a stationary Gaussian process with constant mean and Matern 5/2 correlation.
Invoked in Section 2, equations (2) and (3); the entire posterior sampling procedure depends on this modeling assumption about the expensive black-box objective.
domain assumption The constraint function z(x) is an independent Gaussian process, independent of the objective GP.
Stated in the Appendix, equation (14); in a single cure simulation, deformation and degree of cure are outputs of the same finite element run and may be correlated, so independence is a strong assumption.
domain assumption The argmin over the finite Latin hypercube candidate set is a good approximation to the argmin over the continuous domain.
Implied in Section 2 after equation (7) and in the Appendix; the finite-grid approximation is not analyzed.
domain assumption Standard Sobol indices are meaningful for the empirical joint distribution of the decision sample.
Section 3, equation (9); the decision sample in (7) is not an independent product distribution, and standard Sobol decompositions assume independence.
standard math Conditional GP formulas and MLE formulas in equations (4) and (5) are correct.
Standard results from Gaussian process regression, used without proof.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Uncertainty Estimation of the Optimal Decision with Application to Cure Process Optimization." pith.science (2026). https://pith.science/paper/JO4COQCB

@misc{pith2026250721995,
  author       = {Pith},
  title        = {Pith review of: Uncertainty Estimation of the Optimal Decision with Application to Cure Process Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JO4COQCB}},
  note         = {Machine review of arXiv:2507.21995}
}

read the original abstract

Decision-making in manufacturing often involves optimizing key process parameters using data collected from simulation experiments. Gaussian processes are widely used to surrogate the underlying system and guide optimization. Uncertainty often inherent in the decisions given by the surrogate model due to limited data and model assumptions. This paper proposes a surrogate model-based framework for estimating the uncertainty of optimal decisions and analyzing its sensitivity with respect to the objective function. The proposed approach is applied to the composite cure process simulation in manufacturing.

Figures

Figures reproduced from arXiv: 2507.21995 by the authors.

**Figure 2.** Comparison of sensitivity analysis between the uniform distribution (left) and the decision [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** A cure cycle with two change points. least 0.96. As a result, the cure process optimization problem is formulated by min y(t1, T1, t2, T2) s.t. z(t1, T1, t2, T2) ≥ 0.96 (10) T1 − 20 t1 ≥ 0 (11) T1 − T2 t2 − t1 ≤ 0 (12) 10 ≤ t1 ≤ 110 125 ≤ T1 ≤ 180 120 ≤ t2 ≤ 200 150 ≤ T2 ≤ 180, where the constraints in (11) and (12) are given by requiring that the slope of S1 is greater than or equal to zero, and the slope of S2 is … view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: The probability density of the empirical distributions of the optimal change points for the [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

**Figure 5.** Figure 5: Comparison of sensitivity analysis between the uniform distribution (left) and the decision [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 16 canonical work pages

[1]

Rapidly cured epoxy/anhydride composites: Effect of residual stress on laminate shear strength,

Agius, S., Joosten, M., Trippit, B., Wang, C., and Hilditch, T. (2016), “Rapidly cured epoxy/anhydride composites: Effect of residual stress on laminate shear strength,” Composites Part A: Applied Science and Manufacturing, 90, 125–136. 13 Convergent Manufacturing Technologies, Inc. (2024), COMPRO – Composite Process Modeling Software,

work page 2016
[2]

Dassault Syst` emes (2024),ABAQUS 2024 User Manual, Dassault Syst` emes Simulia Corp., Providence, RI

Vancouver, BC, computer software. Dassault Syst` emes (2024),ABAQUS 2024 User Manual, Dassault Syst` emes Simulia Corp., Providence, RI

work page 2024
[3]

A tutorial on Bayesian optimization,

Frazier, P. I. (2018), “A tutorial on Bayesian optimization,” arXiv preprint arXiv:1807.02811

arXiv 2018
[4]

(2016), sobolSalt: Monte Carlo Estimation of Sobol’ Indices Based on Saltelli’s Schemes, r package version 1.30.1

Gilquin, L. (2016), sobolSalt: Monte Carlo Estimation of Sobol’ Indices Based on Saltelli’s Schemes, r package version 1.30.1

work page 2016
[5]

Gramacy, R. B. (2020), Surrogates: Gaussian process modeling, design, and optimization for the applied sciences, Chapman and Hall/CRC

work page 2020
[6]

Efficient global optimization of expensive black-box functions,

Jones, D. R., Schonlau, M., and Welch, W. J. (1998), “Efficient global optimization of expensive black-box functions,” Journal of Global optimization, 13, 455–492

work page 1998
[7]

Numerical Simulation Informed Rapid Cure Process Optimization of Composite Structures using Constrained Bayesian Optimization,

Limaye, M., Li, Y., Zhang, Q., and Li, G. (2025), “Numerical Simulation Informed Rapid Cure Process Optimization of Composite Structures using Constrained Bayesian Optimization,”

work page 2025
[8]

Calculations of Sobol indices for the Gaussian process metamodel,

Marrel, A., Iooss, B., Laurent, B., and Roustant, O. (2009), “Calculations of Sobol indices for the Gaussian process metamodel,” Reliability Engineering & System Safety, 94, 742–751

work page 2009

Show all 17 references

[9]

Manufacturing processes for composite materials and components for aerospace applications,

McIlhagger, A., Archer, E., and McIlhagger, R. (2020), “Manufacturing processes for composite materials and components for aerospace applications,” in Polymer composites in the aerospace industry, Elsevier, pp. 59–81

2020
[10]

Latin hypercube sampling as a tool in uncertainty analysis of computer models,

McKay, M. D. (1992), “Latin hypercube sampling as a tool in uncertainty analysis of computer models,” in Proceedings of the 24th conference on Winter simulation, pp. 557–564

1992
[11]

DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization,

Roustant, O., Ginsbourger, D., and Deville, Y. (2012), “DiceKriging, DiceOptim: Two R packages for the analysis of computer experiments by kriging-based metamodeling and optimization,” Journal of statistical software, 51, 1–55

2012
[12]

Statistical science,

Sacks, J., Welch, W., Mitchell, T., and Wynn, H. (1989), “Statistical science,” Design and Analysis of Computer Experiments, 4, 409–423

1989
[13]

J., Williams, B

Santner, T. J., Williams, B. J., Notz, W. I., and Williams, B. J. (2003), The design and analysis of computer experiments, vol. 1, Springer. 14

2003
[14]

Multiple predictor smoothing methods for sensitivity analysis: Description of techniques,

Storlie, C. B. and Helton, J. C. (2008), “Multiple predictor smoothing methods for sensitivity analysis: Description of techniques,” Reliability Engineering & System Safety, 93, 28–54

2008
[15]

Cure cycle optimization for the reduction of processing-induced residual stresses in composite materials,

White, S. R. and Hahn, H. (1993), “Cure cycle optimization for the reduction of processing-induced residual stresses in composite materials,” Journal of Composite Materials, 27, 1352–1378

1993
[16]

Williams, C. K. and Rasmussen, C. E. (2006), Gaussian processes for machine learning, MIT press

2006
[17]

Sequential model-based optimization for continuous inputs with finite decision space,

Zhang, Q. and Hwang, Y. (2020), “Sequential model-based optimization for continuous inputs with finite decision space,” Technometrics, 62, 486–498. 15

2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Rapidly cured epoxy/anhydride composites: Effect of residual stress on laminate shear strength,

Agius, S., Joosten, M., Trippit, B., Wang, C., and Hilditch, T. (2016), “Rapidly cured epoxy/anhydride composites: Effect of residual stress on laminate shear strength,” Composites Part A: Applied Science and Manufacturing, 90, 125–136. 13 Convergent Manufacturing Technologies, Inc. (2024), COMPRO – Composite Process Modeling Software,

work page 2016

[2] [2]

Dassault Syst` emes (2024),ABAQUS 2024 User Manual, Dassault Syst` emes Simulia Corp., Providence, RI

Vancouver, BC, computer software. Dassault Syst` emes (2024),ABAQUS 2024 User Manual, Dassault Syst` emes Simulia Corp., Providence, RI

work page 2024

[3] [3]

A tutorial on Bayesian optimization,

Frazier, P. I. (2018), “A tutorial on Bayesian optimization,” arXiv preprint arXiv:1807.02811

arXiv 2018

[4] [4]

(2016), sobolSalt: Monte Carlo Estimation of Sobol’ Indices Based on Saltelli’s Schemes, r package version 1.30.1

Gilquin, L. (2016), sobolSalt: Monte Carlo Estimation of Sobol’ Indices Based on Saltelli’s Schemes, r package version 1.30.1

work page 2016

[5] [5]

Gramacy, R. B. (2020), Surrogates: Gaussian process modeling, design, and optimization for the applied sciences, Chapman and Hall/CRC

work page 2020

[6] [6]

Efficient global optimization of expensive black-box functions,

Jones, D. R., Schonlau, M., and Welch, W. J. (1998), “Efficient global optimization of expensive black-box functions,” Journal of Global optimization, 13, 455–492

work page 1998

[7] [7]

Numerical Simulation Informed Rapid Cure Process Optimization of Composite Structures using Constrained Bayesian Optimization,

Limaye, M., Li, Y., Zhang, Q., and Li, G. (2025), “Numerical Simulation Informed Rapid Cure Process Optimization of Composite Structures using Constrained Bayesian Optimization,”

work page 2025

[8] [8]

Calculations of Sobol indices for the Gaussian process metamodel,

Marrel, A., Iooss, B., Laurent, B., and Roustant, O. (2009), “Calculations of Sobol indices for the Gaussian process metamodel,” Reliability Engineering & System Safety, 94, 742–751

work page 2009