Sequential Computer Experimental Design for Estimating an Extreme Probability or Quantile

Hao Chen; William J. Welch

REVIEW 3 major objections 6 minor 16 references

Sequential Computer Experimental Design for Estimating an Extreme Probability or Quantile

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that for estimating extreme tail probabilities and quantiles with a Gaussian process surrogate, the discrepancy criterion converges much faster than expected improvement and should be the preferred sequential search rule.

desk verdict A useful empirical comparison of two search criteria for extreme tail estimation that deserves refereeing, but the headline EI-vs-discrepancy claim is only as strong as the unreported EI tuning constant and a one-function comparison. read the letter →

arxiv 1908.05357 v1 pith:YH3EIYYU submitted 2019-08-14 stat.ML cs.LG

Hao Chen , William J. Welch This is my paper · ORCID

classification stat.MLcs.LG

keywords sequentialexperimentaldesignGaussianprocesssurrogatefailureprobabilityestimationextremequantileexpectedimprovementdiscrepancycriterioncomputerexperimentstail

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes practical guidance for estimating a rare failure probability or quantile when the computer model is too expensive for plain Monte Carlo. Its central recommendation is that a Gaussian process surrogate should be trained with an initial design, then extended sequentially using the discrepancy criterion rather than expected improvement. In the short column illustration, the discrepancy criterion reaches accurate probability and quantile estimates after roughly ten added runs, while expected improvement has not converged after twenty and needs about thirty extra points. The paper also recommends a uniform initial design that oversamples distribution tails, stratified sampling for the Monte Carlo set, and Bayesian uncertainty quantification for the surrogate. These recommendations matter because they turn sequential design from a theoretical idea into a workable recipe for engineers estimating tail risk.

What carries the argument

The load-bearing mechanism is the discrepancy criterion, which selects the next evaluation by minimizing $(\hat m_\psi(x)-y_f)^2/v_{\psi,\sigma^2}(x)$, equivalently maximizing $\Pr(Z<-|\hat m_\psi(x)-y_f|/\sqrt{v_{\psi,\sigma^2}(x)})$ with $Z\sim N(0,1)$. Unlike the expected improvement criterion, this form requires no tuning constant $\alpha$ and balances local search near the target contour with global search in regions of high predictive variance. The sequential algorithm wraps this criterion around a Bayesian Gaussian process emulator, whose predictive mean and variance are averaged over Markov chain Monte Carlo samples of the correlation parameters; quantile estimation is handled by treating the current estimated quantile as the target contour at each step.

What would settle it

Run the short column experiment with the expected-improvement criterion across a grid of alpha values covering the range suggested by Ranjan et al. (2008), report the convergence curves and final RMSE for each alpha, and check whether any alpha lets expected improvement match the discrepancy criterion's convergence within ten added runs; if so, the paper's blanket recommendation would fail.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the hypothesis-testing-based discrepancy criterion is the more reliable search rule for sequential estimation of extreme tail probabilities and quantiles. For the short column function, estimates built from the discrepancy criterion effectively converge to the true failure probability of 0.0025 within roughly 11 added runs and stay there across all 10 repeated experiments, whereas expected improvement estimates still wander and require about 30 added runs to converge. The same pattern holds for quantile estimation and for a realistic eight-input floor deflection model. The paper also shows that a uniform initial design, which oversamples the tails of the input distributions, improves accuracy over a random initial design, and that stratifying the Monte Carlo and candidate sets makes the computation more efficient while preserving tail accuracy.

Load-bearing premise

The comparison between the two search criteria assumes that the expected-improvement implementation used a representative value of its tuning constant alpha; the paper never reports the alpha used in the Section 5 experiments, so if that value was atypical the discrepancy criterion's advantage could disappear.

Editorial extensions

If this is right

Practitioners estimating extreme tail probabilities or quantiles from expensive simulations should prefer the discrepancy criterion over expected improvement, since it converged in roughly ten added runs where expected improvement needed about thirty.
A uniform initial design that oversamples the tails of the input distribution is preferable to a purely random initial design, especially for expected improvement, because it places training points closer to the failure region.
Using a stratified Monte Carlo set with weighted stratum estimates can reduce the number of simulation points needed to resolve a tail probability or quantile, as demonstrated in the eight-input floor system example.
Diagnostic plots of the standardized discrepancy across the Monte Carlo set can indicate whether the sequential search has converged, giving practitioners a stopping rule when the true probability or quantile is unknown.
Bayesian estimation of Gaussian process parameters is recommended for these searches because the criteria rely on realistic predictive variances that fully account for parameter uncertainty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Going beyond the paper's comparisons, the reported advantage of the discrepancy criterion could depend on the value of the tuning constant $\alpha$ used in the expected improvement implementation, and the paper does not report that value; sweeping $\alpha$ over the range suggested by the contour-estimation literature would test whether the conclusion is robust.
The discrepancy criterion could be compared against other contour-adaptive rules, such as entropy-based or integrated-risk criteria, on higher-dimensional test functions to see whether its fast convergence generalizes beyond the two examples studied here.
The diagnostic boxplots the paper proposes could be turned into an automatic stopping rule: stop adding runs when the quantiles of the standardized discrepancy distribution stop decreasing by more than a small threshold.
A natural extension would be to apply the same sequential recipe to estimate simultaneous tail probabilities for multiple failure modes or to optimize a design parameter while estimating the tail probability, since the discrepancy criterion naturally targets the relevant contour at each step.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's real contribution is practical: it takes two existing sequential search criteria, EI and the discrepancy criterion, and asks which one actually works when the target is an extreme tail probability or quantile. The answer it gives—discrepancy converges much faster, roughly 10 added runs versus 20-plus for EI on the short column function—is clearly stated and supported by the experiments shown. The paper also addresses two questions practitioners care about: how to choose initial designs and how to build the MC/candidate sets. The recommendation to over-sample the tails in the initial design is sensible, and the stratified sampling scheme for the floor system is a nice piece of applied thinking. The comparison against external Monte Carlo benchmarks (10 million draws for the short column, direct simulation for the floor) is the right way to evaluate these methods, and the paper does not force the estimates by using its own search criterion as truth.

The soft spots are real but not fatal. The biggest one is the one your stress-test note flagged: the EI criterion in equation (13) depends on the confidence parameter alpha, and the paper never says what alpha was used in Figures 3–7 or Tables 1–2. That matters because alpha controls the local-global trade-off, and an unrepresentative choice could make EI look worse than it is. This is an omission, not a sign of dishonesty, but it is load-bearing for the central claim. Second, the direct EI-versus-discrepancy comparison rests on a single test function; the floor system application only runs the discrepancy criterion, so it does not corroborate the comparison. Third, the RMSE tables report point estimates without standard errors or intervals, which is thin for 10 repeats. Fourth, no code or data are shipped, so the empirical claims cannot be reproduced as-is. The stopping diagnostic in Section 7 is informal—it is a boxplot of standardized distances, not a rule—but the paper mostly presents it as a diagnostic aid, so I would treat that as minor.

On balance, the paper is honest, clearly written, and the recommendations are plausible. The citation pattern is fine: the key prior work by Roy and Notz, Ranjan et al., and Jones et al. is acknowledged. The missing alpha and the thin evidence base make the headline claim conditional, but conditional is not invalid. If the authors report alpha, rerun the comparison on at least one more function, and add error bars or code, this could be a solid practical reference for reliability analysis with expensive simulators.

I would send this to peer review. It is a serious empirical study with a clear question, not a desk reject. The referees should push for the missing parameter values and more examples, but the work deserves that effort.

Referee Report

3 major / 6 minor

Summary. The paper proposes and evaluates sequential experimental design strategies for estimating extreme tail probabilities and quantiles of a computer model output using a Gaussian process (GP) surrogate. It compares two existing search criteria, expected improvement (EI) and the hypothesis-testing-based discrepancy criterion, and uses two applications: a short-column limit-state function and an eight-dimensional floor-system computer model. The paper also addresses practical questions: choice of initial design, modeling of the input distribution, stratified construction of Monte Carlo and candidate sets, and a stopping diagnostic. The main empirical findings are that the discrepancy criterion converges much faster than EI, that uniform initial designs that over-sample input-distribution tails are preferable, and that a stratified weighting scheme based on a mixture of empirical lower-tail and upper-tail distributions works well for the floor system.

Significance. If the empirical conclusions hold, the paper provides a useful practical recommendation: the discrepancy criterion in Eq. (16) has no tuning constant, is simpler to implement than EI, and appears to reach accurate estimates with about 10 added runs where EI has not converged after 20. The study is strengthened by external verification against 10 million Monte Carlo draws for the short column and direct computer-model runs for the floor system, by the use of Bayesian uncertainty quantification for GP parameters, and by clear algorithmic descriptions. The main limitations are that the central EI-versus-discrepancy comparison rests on a single test function with only 10 repeats, and that the EI tuning parameter alpha is not reported, making the headline comparison difficult to reproduce or assess.

major comments (3)

[Section 4.2, Eq. (12)-(13); Section 5.1] The value of the tuning constant alpha used in the EI implementation is never stated anywhere in the paper, even though Eq. (12)-(13) show that alpha directly controls the width of the improvement region and hence the balance between local search near the contour and global search driven by predictive variance. Because the headline recommendation that the discrepancy criterion converges faster than EI is based entirely on the experiments in Section 5, the comparison is not reproducible and could be an artifact of an unrepresentative alpha. The authors should report the alpha used for Figures 3-7 and Tables 1-2, and ideally include a sensitivity analysis over a range of alpha values to show that the conclusion is robust.
[Section 5 and Section 8] The central claim that the discrepancy criterion is preferable to EI is supported by only one test function, the short-column function, with 10 repeats; the floor-system application in Section 6 applies only the discrepancy criterion, so it provides no independent replication of the comparison. Section 8 states the recommendation in general terms ('Use the discrepancy criterion'), and although the text adds 'in our study', the evidence base is narrow for a general methodological recommendation. Adding a second test problem that runs both criteria, or substantially tempering the scope of the recommendation, would make the central claim load-bearing rather than illustrative.
[Section 7] The proposed stopping diagnostic plots, for all MC points, the standardized discrepancy -|m_hat(x)-y_f|/sqrt(v_hat(x)), which is exactly the quantity that the discrepancy search criterion optimizes when choosing the next point. A declining trend in these boxplots may therefore be a self-confirming property of the search rule rather than independent evidence that the probability or quantile estimate has converged. The paper should validate the diagnostic against the actual estimation error in a setting with a known truth, or at least explicitly acknowledge this possible circularity.

minor comments (6)

[Section 5, Tables 1 and 2] The RMSE values in Tables 1 and 2 are based on only 10 repeats but are reported without standard errors or any other measure of variability; adding per-repeat errors or bootstrap intervals would help the reader assess whether the large reported differences are statistically stable.
[Section 5.1] The sentence 'Further calculations, not reported here for brevity, show that the EI method requires about 30 extra points after the 20-point initial design, to converge' is not verifiable; it should either be removed or supported by a figure or table.
[Section 6.2] The statement that the fitted 2-parameter and 3-parameter Weibull distributions are 'virtually identical' is qualitative; a numerical comparison of fitted parameters or a distance measure would be more informative.
[Section 6.4] The 'true' 0.999 quantile for the floor system is itself an estimate with a reported standard error of 0.00701; treating 3.88 as the exact truth in Figure 10 is acceptable for plotting, but the uncertainty in this reference value should be acknowledged in the text.
[Section 4.3] The definition of the discrepancy criterion states epsilon > 0, but the text then says 'as we work with epsilon = 0'; this should be clarified, for example by noting that the formula is extended to the boundary case epsilon = 0.
[Figures 6 and 7] The captions describe the dotted line as 'the true quantile, 0', but the context is the lower 0.0025 quantile of a limit-state function; the caption could be more explicit about this being the lower quantile.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central comparison is validated against external Monte Carlo benchmarks and direct computer-model simulation, not against its own inputs.

full rationale

The paper's central claim is that the discrepancy criterion converges faster than EI for extreme probability and quantile estimation. This claim rests on empirical comparisons in Section 5 using the short column function, where the target values are established independently: the failure probability 0.0025 is obtained from 10 million Monte Carlo draws, and the quantile target is the known limit-state value 0. Section 6 similarly establishes the floor-system truth by running the computer model on ten MC sets and averaging. Neither target is defined by the search criteria being compared, so the comparison is not forced by construction. The EI and discrepancy criteria themselves are taken from the literature (Ranjan et al., 2008; Roy and Notz, 2014), and the paper explicitly contrasts them rather than equating one to the other. The self-citations to Chen et al. (2016, 2017) concern Bayesian GP parameter estimation and are methodological choices, not load-bearing definitions of the predicted quantities; moreover, the Bayesian uncertainty estimates are applied symmetrically to both criteria. The Section 7 diagnostic uses a standardized discrepancy statistic to visualize convergence, but this diagnostic is not used to define the target probability or quantile, and the convergence claims are checked against the externally established truths. The absence of a reported EI tuning parameter alpha is a reproducibility or experimental-comparison concern, not a circularity, because alpha does not appear in the definition of the target estimate. Accordingly, no step reduces by definition to its own inputs, and no fitted parameter is renamed as a prediction. The paper is self-contained relative to external benchmarks, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 7 assumptions · 0 invented entities

The central claim rests on standard GP emulation assumptions, on externally simulated benchmarks for the truth, and on an unreported tuning parameter for EI. No new physical or mathematical entities are introduced.

free parameters (3)

alpha (EI improvement width) = not stated
Required by Eq. (12)-(13); controls the width of the improvement region around the contour. The comparison of EI to the discrepancy criterion depends on this tuning choice, which is never reported in the experimental sections.
epsilon (discrepancy criterion constant) = 0
Set to 0 in Section 4.3 to simplify the criterion to a standardized absolute distance; a modeling choice rather than a fitted value.
stratum weights p1 and p2 in sampling distribution H(x) = 0.5, 0.5
Chosen in Section 6.2 to deliberately over-sample the lower tail of the MOE input distribution; they are corrected by weighted stratified estimation in Eq. (20), so they affect efficiency but not the final estimate.

assumptions (7)

domain assumption The deterministic computer output y(x) is modeled as a realization of a stationary Gaussian process with power-exponential correlation (Eqs. (2)-(3)).
Standard for computer experiment emulation but is an untested assumption about the unknown code; Section 3.1.
domain assumption A constant mean GP is sufficient for prediction (citing Chen et al. 2016).
Section 3.1; the choice is supported by prior work but not revalidated here.
domain assumption Correlation parameters are integrated over their posterior using MCMC with priors from Chen et al. (2017).
Section 3.2; the search criteria depend on the resulting predictive variances.
standard math The true failure probability 0.0025 for the short column is obtained by 10 million Monte Carlo samples and treated as exact.
Section 5; the binomial standard error is small but not explicitly quantified.
domain assumption The 'true' 0.999 quantile for the floor system is estimated from 10 MC sets and treated as ground truth.
Section 6.4; the standard error of the mean is 0.00701, so the benchmark has small nonzero uncertainty.
domain assumption For the floor system, the input distribution is the empirical distribution of 580 MOE measurements, with lower-tail over-sampling corrected by stratified weights.
Section 6.2; assumes the 580 boards represent production variability.
domain assumption The EI and discrepancy criteria are derived assuming a normal predictive distribution with known variance, but the implementation uses Bayesian t distributions and averaged variances.
Sections 4.2-4.3; this is a standard approximation for the criteria.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Sequential Computer Experimental Design for Estimating an Extreme Probability or Quantile." pith.science (2026). https://pith.science/paper/YH3EIYYU

@misc{pith2026190805357,
  author       = {Pith},
  title        = {Pith review of: Sequential Computer Experimental Design for Estimating an Extreme Probability or Quantile},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YH3EIYYU}},
  note         = {Machine review of arXiv:1908.05357}
}

read the original abstract

A computer code can simulate a system's propagation of variation from random inputs to output measures of quality. Our aim here is to estimate a critical output tail probability or quantile without a large Monte Carlo experiment. Instead, we build a statistical surrogate for the input-output relationship with a modest number of evaluations and then sequentially add further runs, guided by a criterion to improve the estimate. We compare two criteria in the literature. Moreover, we investigate two practical questions: how to design the initial code runs and how to model the input distribution. Hence, we close the gap between the theory of sequential design and its application.

Figures

Figures reproduced from arXiv: 1908.05357 by the authors.

**Figure 1.** Floor with d = 8 joists (supporting beams). The 8 joists act like springs, i.e., they deflect under a load. MOE1 MOE2 ... MOE8 Inputs Uniform Load Computer Model: Solving a Set of Simultaneous Equations Deflection - y1 Deflection - y2 ... Deflection - y8 Output Maximum Deflection - y [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

**Figure 2.** Computer model of a floor system with d = 8 joists. The code is deterministic, but if we think of the inputs x1, . . . , xd as realizations of random variables X1, . . . , Xd, then the output deflection y is a realization of a random variable Y . Lumber is heterogenous relative to many other engineering materials, even within the same nominal grade, 5 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

**Figure 3.** Probability estimates for the short column function when the initial design is random [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (8 more)

**Figure 4.** Figure 4: Probability estimates for the short column function when the initial design is uniform [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: Search points for estimation of failure probability for the short column function as a [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: Quantile estimates for the short column function when the initial design is random for [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 8.** Figure 8: The relationship between the MOEs of the four middle beams and the response. The [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]

**Figure 9.** Figure 9: Modelling the lower 10% of the MOE data: Histogram of the lower 10% of the data [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]

**Figure 10.** Figure 10: Computer model of the floor system with a uniform initial design of [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

**Figure 11.** Figure 11: Diagnostic plots for the short column function starting with a uniform initial design: [PITH_FULL_IMAGE:figures/full_fig_p024_11.png]

**Figure 12.** Figure 12: Diagnostic plots for the floor system computer model starting with a uniform initial [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

[1]

Reliability-Based Design Op- timization using Eﬃcient Global Reliability Analysis,

Bichon, B. B., Mahadevan, S., and Eldred, M. S. (2009), “Reliability-Based Design Op- timization using Eﬃcient Global Reliability Analysis,” Palm Springs, California, 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, pp. 1–12

work page 2009
[2]

Sulla determinazione empirica delle leggi di probabilita,

Cantelli, F. P. (1933), “Sulla determinazione empirica delle leggi di probabilita,” Giorn. Ist. Ital. Attuari, 4, 221–424

work page 1933
[3]

Analysis Methods for Computer Experiments: How to Assess and What Counts?

Chen, H., Loeppky, J. L., Sacks, J., Welch, W. J., et al. (2016), “Analysis Methods for Computer Experiments: How to Assess and What Counts?” Statistical Science, 31, 40–60

work page 2016
[4]

Flexible Correlation Structure for Accu- rate Prediction and Uncertainty Quantiﬁcation in Bayesian Gaussian Process Emulation of a Computer Model,

Chen, H., Loeppky, J. L., and Welch, W. J. (2017), “Flexible Correlation Structure for Accu- rate Prediction and Uncertainty Quantiﬁcation in Bayesian Gaussian Process Emulation of a Computer Model,” SIAM/ASA Journal on Uncertainty Quantiﬁcation , 5, 598–620

work page 2017
[5]

Sulla determinazione empirica della legge di probabilita,

Glivenko, V. (1933), “Sulla determinazione empirica della legge di probabilita,” Giorn. Ist. Ital. Attuari, 4, 92–99

work page 1933
[6]

A Bayesian Analysis of Kriging,

Handcock, M. S. and Stein, M. L. (1993), “A Bayesian Analysis of Kriging,” Technometrics, 35, 403–410

work page 1993
[7]

Eﬃcient global optimization of expensive black-box functions,

Jones, D. R., Schonlau, M., and Welch, W. J. (1998), “Eﬃcient global optimization of expensive black-box functions,” Journal of Global optimization , 13, 455–492

work page 1998
[8]

Two Basic Problems in Reliability-Based Structural Opti- mization,

Kuschel, N. and Rackwitz, R. (1997), “Two Basic Problems in Reliability-Based Structural Opti- mization,” Mathematical Methods of Operation Research, 46, 309–333

work page 1997

Show all 16 references

[9]

Using Artiﬁcial Censoring to Improve Extreme Tail Quantile Estimates,

Liu, Y., Salib´ an-Barrera, M., Zamar, R. H., and Zidek, J. V. (2018), “Using Artiﬁcial Censoring to Improve Extreme Tail Quantile Estimates,” Applied Statistics, 67, 791–812

2018
[10]

Deﬂections of Uniformly Loaded Floors: A Beam-Spring Analog,

McCutcheon, W. J. (1984), “Deﬂections of Uniformly Loaded Floors: A Beam-Spring Analog,” Tech. Rep. Research Paper FPL449, United States Department of Agriculture, Madison, WI

1984
[11]

A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code,

McKay, M., Beckman, R., and Conover, W. (1979),“A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code,” Technometrics, 21, 239–245

1979
[12]

Sequential Experimental Design for Contour Estimation from Complex Computer Codes,

Ranjan, P., Bingham, D., and Michailidis, G. (2008),“Sequential Experimental Design for Contour Estimation from Complex Computer Codes,” Technometrics, 50, 527–541. 27

2008
[13]

Estimating percentiles in computer experiments: A comparison of sequential-adaptive designs and ﬁxed designs,

Roy, S. and Notz, W. I. (2014), “Estimating percentiles in computer experiments: A comparison of sequential-adaptive designs and ﬁxed designs,” Journal of Statistical Theory and Practice , 8, 12–29

2014
[14]

Design and Analysis of Computer Experiments,

Sacks, J., Welch, W., Mitchell, T., and Wynn, H. (1989), “Design and Analysis of Computer Experiments,” Statistical Science, 4, 409–435

1989
[15]

(2003), The Design and Analysis of Computer Experi- ments, Springer Series in Statistics, Springer Press

Santner, T., Williams, B., and Notz, W. (2003), The Design and Analysis of Computer Experi- ments, Springer Series in Statistics, Springer Press

2003
[16]

Screening the Input Variables to a Computer Model Via Analysis of Variance and Visualization,

Schonlau, M. and Welch, W. J. (2006), “Screening the Input Variables to a Computer Model Via Analysis of Variance and Visualization,” in Screening: Methods for Experimentation in

2006

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Reliability-Based Design Op- timization using Eﬃcient Global Reliability Analysis,

Bichon, B. B., Mahadevan, S., and Eldred, M. S. (2009), “Reliability-Based Design Op- timization using Eﬃcient Global Reliability Analysis,” Palm Springs, California, 50th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, pp. 1–12

work page 2009

[2] [2]

Sulla determinazione empirica delle leggi di probabilita,

Cantelli, F. P. (1933), “Sulla determinazione empirica delle leggi di probabilita,” Giorn. Ist. Ital. Attuari, 4, 221–424

work page 1933

[3] [3]

Analysis Methods for Computer Experiments: How to Assess and What Counts?

Chen, H., Loeppky, J. L., Sacks, J., Welch, W. J., et al. (2016), “Analysis Methods for Computer Experiments: How to Assess and What Counts?” Statistical Science, 31, 40–60

work page 2016

[4] [4]

Flexible Correlation Structure for Accu- rate Prediction and Uncertainty Quantiﬁcation in Bayesian Gaussian Process Emulation of a Computer Model,

Chen, H., Loeppky, J. L., and Welch, W. J. (2017), “Flexible Correlation Structure for Accu- rate Prediction and Uncertainty Quantiﬁcation in Bayesian Gaussian Process Emulation of a Computer Model,” SIAM/ASA Journal on Uncertainty Quantiﬁcation , 5, 598–620

work page 2017

[5] [5]

Sulla determinazione empirica della legge di probabilita,

Glivenko, V. (1933), “Sulla determinazione empirica della legge di probabilita,” Giorn. Ist. Ital. Attuari, 4, 92–99

work page 1933

[6] [6]

A Bayesian Analysis of Kriging,

Handcock, M. S. and Stein, M. L. (1993), “A Bayesian Analysis of Kriging,” Technometrics, 35, 403–410

work page 1993

[7] [7]

Eﬃcient global optimization of expensive black-box functions,

Jones, D. R., Schonlau, M., and Welch, W. J. (1998), “Eﬃcient global optimization of expensive black-box functions,” Journal of Global optimization , 13, 455–492

work page 1998

[8] [8]

Two Basic Problems in Reliability-Based Structural Opti- mization,

Kuschel, N. and Rackwitz, R. (1997), “Two Basic Problems in Reliability-Based Structural Opti- mization,” Mathematical Methods of Operation Research, 46, 309–333

work page 1997