pith. sign in

arxiv: 2412.13731 · v2 · pith:JOA33VRXnew · submitted 2024-12-18 · 📊 stat.CO · stat.ME· stat.ML

Reliability analysis for non-deterministic limit-states using stochastic emulators

Anderson V. Pires , Maliki Moustapha , Stefano Marelli , Bruno Sudret This is my paper

Pith reviewed 2026-05-23 07:24 UTC · model grok-4.3

classification 📊 stat.CO stat.MEstat.ML
keywords reliability analysisstochastic simulatorssurrogate modelsgeneralized lambda modelsstochastic polynomial chaosuncertainty quantificationlimit-state exceedance
0
0 comments X

The pith

Reliability analysis extends to stochastic simulators by training generalized lambda models and stochastic polynomial chaos expansions as surrogates on simulation data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a method for reliability analysis when the underlying model is stochastic, meaning repeated runs with identical inputs yield different outputs. It demonstrates that generalized lambda models and stochastic polynomial chaos expansions can act as emulators that capture this randomness and allow estimation of failure probabilities. The approach is validated on an analytical example with known solution, a beam model, and a wind turbine dataset. A sympathetic reader would care because many engineering systems involve inherent randomness that standard deterministic reliability tools cannot address without prohibitive computation.

Core claim

Reliability analysis for stochastic models is performed by using generalized lambda models and stochastic polynomial chaos expansions to emulate the random response and compute limit-state exceedance probabilities at far lower cost than direct Monte Carlo simulation on the original simulator.

What carries the argument

Generalized lambda models and stochastic polynomial chaos expansions, which are emulators trained to reproduce the distribution of outputs from a stochastic simulator for any fixed input.

If this is right

  • Failure probabilities for systems with non-repeatable behavior become computable without exhaustive Monte Carlo on the original model.
  • Analysis remains feasible when only a finite dataset of simulator runs is available rather than direct access to the code.
  • The same emulators support uncertainty quantification tasks beyond reliability, such as sensitivity analysis on stochastic responses.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method could be tested on simulators whose randomness arises from different sources, such as measurement noise versus intrinsic model variability.
  • Extending the emulators to time-dependent or multi-output stochastic simulators would broaden applicability to dynamic systems.

Load-bearing premise

The chosen emulators accurately reproduce the full output distributions of the stochastic simulator across the input space.

What would settle it

Running the emulators on an analytical stochastic function whose true failure probability is known in closed form and observing that the estimated probability does not converge to the true value as the number of training runs increases.

Figures

Figures reproduced from arXiv: 2412.13731 by Anderson V. Pires, Bruno Sudret, Maliki Moustapha, Stefano Marelli.

Figure 1
Figure 1. Figure 1: R−S function – Box plots comparing convergence behavior obtained from the emulators and from direct MCS for increasing values of N. The analytical probability of failure is depicted by the dashed black line [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: R − S function: Comparison of the conditional failure probability heat maps for different sizes of the experimental design used for GLaM and SPCE models. become more accurate, with only minor differences observed for N = 5,000. For N = 50,000, no distinguishable differences are evident. 5.2 Stochastic simply supported beam Let us consider a simply supported beam subjected to uniform load. We are interested… view at source ↗
Figure 3
Figure 3. Figure 3: Simply supported beam example – Box plots comparing convergence behavior obtained [PITH_FULL_IMAGE:figures/full_fig_p019_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Wind turbine application – (a) Scatter plot depicting the relationship between the [PITH_FULL_IMAGE:figures/full_fig_p021_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Wind turbine application – (a) Empirical mean curve (dotted line) and the corresponding [PITH_FULL_IMAGE:figures/full_fig_p022_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Wind turbine application – Empirical mean curve (dotted line) and the corresponding [PITH_FULL_IMAGE:figures/full_fig_p024_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Wind turbine application – Box plots comparing convergence behavior obtained from [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗
read the original abstract

Reliability analysis is a sub-field of uncertainty quantification that assesses the probability of a system performing as intended under various uncertainties. Traditionally, this analysis relies on deterministic models, where experiments are repeatable, i.e., they produce consistent outputs for a given set of inputs. However, real-world systems often exhibit stochastic behavior, leading to non-repeatable outcomes. These so-called stochastic simulators produce different outputs each time the model is run, even with fixed inputs. This paper formally introduces reliability analysis for stochastic models and addresses it by using suitable surrogate models to lower its typically high computational cost. Specifically, we focus on the recently introduced generalized lambda models and stochastic polynomial chaos expansions. These emulators are designed to learn the inherent randomness of the simulator's response and enable efficient uncertainty quantification at a much lower cost than traditional Monte Carlo simulation. We validate our methodology through three case studies. First, using an analytical function with a closed-form solution, we demonstrate that the emulators converge to the correct solution. Second, we present results obtained from the surrogates using a toy example of a simply supported beam. Finally, we apply the emulators to perform reliability analysis on a realistic wind turbine case study, where only a dataset of simulation results is available.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The manuscript claims that generalized lambda models and stochastic polynomial chaos expansions can serve as effective emulators for performing reliability analysis on stochastic simulators at reduced computational cost compared to Monte Carlo simulation. This is demonstrated through convergence to a closed-form solution in an analytical case, results on a simply supported beam, and application to a wind turbine dataset.

Significance. If the emulators accurately capture response distributions (including tails), the work provides a direct, parameter-free extension of surrogate UQ techniques to the stochastic-simulator setting. Explicit validation against a closed-form analytical solution and the use of reproducible case studies are strengths that support the central claim of efficient and accurate exceedance probability estimation.

major comments (2)
  1. [§4.1] §4.1 (analytical case): while convergence of the estimated failure probability to the closed-form value is shown as a function of training points, no quantitative comparison of the surrogate-based estimator variance versus direct Monte Carlo at equivalent budget is reported; this weakens the efficiency claim for rare-event regimes.
  2. [§5] §5 (wind-turbine case): the stochastic emulator is trained on a fixed dataset, but the manuscript does not report a diagnostic (e.g., QQ-plot or tail-probability error) confirming that the generalized lambda model reproduces the upper tail of the response distribution to sufficient accuracy for the reported 10^{-3}–10^{-4} failure probabilities.
minor comments (3)
  1. [Eq. (12)] Notation for the stochastic PCE coefficients (Eq. 12) is introduced without an explicit statement of the orthogonality measure used when the simulator output is itself random.
  2. [Figure 3] Figure 3 (beam example) lacks error bars on the surrogate probability estimates, making it difficult to judge whether observed differences from Monte Carlo are statistically significant.
  3. [§5] The abstract states three case studies but the wind-turbine section does not specify the size of the available dataset or the train/test split used for emulator fitting.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the positive evaluation and the recommendation of minor revision. We address each major comment below and will incorporate the suggested additions in the revised manuscript.

read point-by-point responses

  1. Referee: [§4.1] §4.1 (analytical case): while convergence of the estimated failure probability to the closed-form value is shown as a function of training points, no quantitative comparison of the surrogate-based estimator variance versus direct Monte Carlo at equivalent budget is reported; this weakens the efficiency claim for rare-event regimes.

    Authors: We agree that a quantitative variance comparison at matched computational budgets would strengthen the efficiency claim, particularly for rare-event estimation. In the revised manuscript we will add, in Section 4.1, a direct comparison of the empirical variance of the failure-probability estimator obtained from the stochastic emulator versus plain Monte Carlo, using the same total number of simulator evaluations for both approaches. revision: yes

  2. Referee: [§5] §5 (wind-turbine case): the stochastic emulator is trained on a fixed dataset, but the manuscript does not report a diagnostic (e.g., QQ-plot or tail-probability error) confirming that the generalized lambda model reproduces the upper tail of the response distribution to sufficient accuracy for the reported 10^{-3}–10^{-4} failure probabilities.

    Authors: We acknowledge that an explicit tail diagnostic would increase confidence in the reported failure probabilities. In the revised Section 5 we will include a QQ-plot of the generalized lambda model against the empirical distribution of the wind-turbine dataset, together with a quantitative assessment of tail-probability error in the range 10^{-3}–10^{-4}. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained via external benchmarks

full rationale

The paper extends standard surrogate techniques (generalized lambda models, stochastic PCE) to stochastic simulators for reliability analysis. Its load-bearing steps consist of emulator construction followed by Monte Carlo estimation of exceedance probabilities; these are validated directly against a closed-form analytical solution in the first case study and against independent simulation datasets in the others. No equation reduces to a fitted parameter renamed as a prediction, no uniqueness theorem is imported from self-citation, and no ansatz is smuggled via prior work by the same authors. The argument chain therefore terminates in external, falsifiable benchmarks rather than internal redefinition.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Based solely on the abstract, the central claim rests on the domain assumption that the chosen emulators can approximate stochastic simulator behavior for probability estimation; no specific free parameters or invented entities are identifiable from the abstract alone.

axioms (1)
  • domain assumption Generalized lambda models and stochastic polynomial chaos expansions can learn and approximate the inherent randomness in simulator outputs for reliability calculations.
    This assumption underpins the use of surrogates to replace direct Monte Carlo simulation on stochastic models.

pith-pipeline@v0.9.0 · 5760 in / 1082 out tokens · 25856 ms · 2026-05-23T07:24:04.661032+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages

  1. [1]

    Nelson, and J

    Ankenman, B., B. Nelson, and J. Staum (2010). Stochastic K riging for simulation metamodeling. Operations Research\/ 58\/ (2), 371--382

    work page 2010

  2. [2]

    Au, S. K. and J. L. Beck (2001). Estimation of small failure probabilities in high dimensions by subset simulation. Probabilistic Engineering Mechanics\/ 16\/ (4), 263--277

    work page 2001

  3. [3]

    Paquette, B

    Barone, M., J. Paquette, B. Resor, and L. Manuel (2012). Decades of wind turbine load simulation. In 50th American Institute of Aeronautics and Astronautics Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition, Nashville, United States of America , pp.\ 1--11

    work page 2012

  4. [4]

    Binois, M., R. B. Gramacy, and M. Ludkovski (2018). Practical heteroscedastic G aussian process modeling for large simulation experiments. Journal of Computational and Graphical Statistics\/ 27\/ (4), 808--821

    work page 2018

  5. [5]

    Deheeger, and M

    Bourinet, J.-M., F. Deheeger, and M. Lemaire (2011). Assessing small failure probabilities by combined subset simulation and support vector machines. Struct. Saf.\/ 33\/ (6), 343--353

    work page 2011

  6. [6]

    Britton, T. (2010). Stochastic epidemic models: A survey. Mathematical Biosciences\/ 225\/ (1), 24--35

    work page 2010

  7. [7]

    Cao, Q. D. and Y. Choe (2019). Cross-entropy based importance sampling for stochastic simulation models. Reliability Engineering & System Safety\/ 191 , 106526

    work page 2019

  8. [8]

    Byon, and N

    Choe, Y., E. Byon, and N. Chen (2015). Importance sampling for reliability evaluation with stochastic simulation models. Technometrics\/ 57\/ (3), 351--361

    work page 2015

  9. [9]

    Lam, and E

    Choe, Y., H. Lam, and E. Byon (2017). Uncertainty quantification of stochastic simulation for black-box computer experiments. Methodology and Computing in Applied Probability\/ 20\/ (4), 1155--1172

    work page 2017

  10. [10]

    Pan, and E

    Choe, Y., Q. Pan, and E. Byon (2016). Computationally efficient uncertainty minimization in wind turbine extreme load assessments. Journal of Solar Energy Engineering\/ 138\/ (4), 041012

    work page 2016

  11. [11]

    Patelli, and M

    De Angelis , M., E. Patelli, and M. Beer (2014). Line Sampling for assessing structural reliability with imprecise failure probabilities . In M. Beer, S.-K. Au, and J. W. Hall (Eds.), Vulnerability, Uncertain. Risk , Liverpool, UK, pp.\ 915--924. ASCE

    work page 2014

  12. [12]

    Ditlevsen, O. and H. O. Madsen (1996). Structural reliability methods . J. Wiley and Sons, Chichester

    work page 1996

  13. [13]

    E uropean S tandard EN 1992-1-1:2004

    European Committee for Standardization (2004). E uropean S tandard EN 1992-1-1:2004. Standard, European Committee for Standardization

    work page 2004

  14. [14]

    Kollia, G

    Freimer, M., G. Kollia, G. S. Mudholkar, and C. T. Lin (1988). A study of the generalized T ukey lambda family. Communications in Statistics -- Theory and Methods\/ 17\/ (10), 3547--3567

    work page 1988

  15. [15]

    Papaioannou, and D

    Geyer, S., I. Papaioannou, and D. Straub (2019). Cross entropy-based importance sampling using G aussian densities revisited. Structural Safety\/ 76 , 15--27

    work page 2019

  16. [16]

    Agrell, E

    Gramstad, O., C. Agrell, E. Bitner-Gregersen, B. Guo, E. Ruth, and E. Vanem (2020). Sequential sampling method using G aussian process regression for estimating extreme structural response. Marine Structures\/ 72 , 102780

    work page 2020

  17. [17]

    Grigoriu, M. (2002). Stochastic Calculus: Applications in Science and Engineering . Springer Science+Business Media

    work page 2002

  18. [18]

    Hao, P., S. Feng, H. Liu, Y. Wang, B. Wang, and B. Wang (2021, October). A novel nested stochastic K riging model for response noise quantification and reliability analysis. Computer Methods in Applied Mechanics and Engineering\/ 384 , 113941

    work page 2021

  19. [19]

    Hastie, T. and R. Tibshirani (1990). Generalized additive models , Volume 43 of Monographs on Statistics and Applied Probability . Chapman and Hall

    work page 1990

  20. [20]

    Heston, S. L. (1993). A closed-form solution for options with stochastic volatility with applications to bond and currency options. Review of Financial Studies\/ 6\/ (2), 327--343

    work page 1993

  21. [21]

    61400-1: Wind turbines part 1: Design requirements

    International Electrotechnical Commission (2005). 61400-1: Wind turbines part 1: Design requirements. Standard, International Electrotechnical Commission

    work page 2005

  22. [22]

    Iooss, B. and M. Ribatet (2009). Global sensitivity analysis of computer models with functional inputs. Reliability Engineering & System Safety\/ 94 , 1194--1204

    work page 2009

  23. [23]

    Jonkman, B. J. (2009). Turbsim user’s guide: Version 1.50. Technical Report NREL/TP-500-46198, National Renewable Energy Laboratory: Golden, Colorado

    work page 2009

  24. [24]

    Jonkman, B. J. and J. M. Jonkman (2013). Addendum to the user’s guides for FAST , A2AD , and AeroDyn released M arch 2010- F ebruary 2013. Technical report, National Renewable Energy Laboratory: Golden, Colorado

    work page 2013

  25. [25]

    Butterfield, W

    Jonkman, J., S. Butterfield, W. Musial, and G. Scott (2009). Definition of a 5-MW reference wind turbine for offshore system development. Technical Report NREL/TP-500-38060, National Renewable Energy Laboratory: Golden, Colorado

    work page 2009

  26. [26]

    Jonkman, J. M. and M. L. J. Buhl (2005). FAST user’s guide - updated A ugust 2005. Technical Report NREL/TP-500-38230, National Renewable Energy Laboratory: Golden, Colorado

    work page 2005

  27. [27]

    Klenke, A. (2013). Wahrscheinlichkeitstheorie . Springer Berlin Heidelberg

    work page 2013

  28. [28]

    Koutsourelakis, P. S., H. J. Pradlwarter, and G. I. Schu\"eller (2004). Reliability of structures in high dimensions, part I : algorithms and applications. Probabilistic Engineering Mechanics\/ 19\/ (4), 409--417

    work page 2004

  29. [29]

    Kurtz, N. and J. Song (2013). Cross-entropy-based adaptive importance sampling using G aussian mixture. Structural Safety\/ 42 , 35--44

    work page 2013

  30. [30]

    Lemaire, M. (2009). Structural reliability . Wiley

    work page 2009

  31. [31]

    Li, S., Y. M. Ko, and E. Byon (2021). Nonparametric importance sampling for wind turbine reliability analysis with stochastic computer models. The Annals of Applied Statistics\/ 15\/ (4), 1850 -- 1871

    work page 2021

  32. [32]

    Marelli, and B

    L\"uthen, N., S. Marelli, and B. Sudret (2023). A spectral surrogate model for stochastic simulators computed from trajectory samples. Computer Methods in Applied Mechanics and Engineering\/ 406\/ (115875), 1--29

    work page 2023

  33. [33]

    L\"uthen, N., X. Zhu, S. Marelli, and B. Sudret (2024a). UQLab user manual -- Generalized Lambda Models . Technical report, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland. Report UQLab-V2.1-120

    work page

  34. [34]

    L\"uthen, N., X. Zhu, S. Marelli, and B. Sudret (2024b). UQLab user manual -- Stochastic polynomial chaos expansions . Technical report, Chair of Risk, Safety and Uncertainty Quantification, ETH Zurich, Switzerland. Report UQLab-V2.1-121

    work page

  35. [35]

    Marelli, and B

    Lüthen, N., S. Marelli, and B. Sudret (2021). Sparse polynomial chaos expansions: Literature survey and benchmark. SIAM / ASA Journal on Uncertainty Quantification\/ 9\/ (2), 593--649

    work page 2021

  36. [36]

    Marelli, S. and B. Sudret (2014). UQLab : A framework for uncertainty quantification in Matlab . In Vulnerability, Uncertainty, and Risk (Proc. 2nd Int. Conf. on Vulnerability, Risk Analysis and Management (ICVRAM2014) , Liverpool, United Kingdom) , pp.\ 2554--2563

    work page 2014

  37. [37]

    Marelli, S. and B. Sudret (2018). An active-learning algorithm that combines sparse polynomial chaos expansions and bootstrap for structural reliability analysis. Structural Safety\/ 75 , 67--74

    work page 2018

  38. [38]

    Iooss, S

    Marrel, A., B. Iooss, S. Da Veiga, and M. Ribatet (2012). Global sensitivity analysis of stochastic computer models with joint metamodels . Statistics and Computing\/ 22 , 833--847

    work page 2012

  39. [39]

    McCullagh, P. and J. Nelder (1989). Generalized linear models , Volume 37 of Monographs on Statistics and Applied Probability . Chapman and Hall

    work page 1989

  40. [40]

    McNeil, A. J., R. Frey, and P. Embrechts (2015). Quantitative risk management: Concepts, techniques and tools . Princeton, NJ: Princeton University Press

    work page 2015

  41. [41]

    Melchers, R. and A. Beck (2018). Structural reliability analysis and prediction . John Wiley & Sons

    work page 2018

  42. [42]

    Melchers, R. E. (1989). Importance sampling in structural systems. Structural Safety\/ 6 , 3--10

    work page 1989

  43. [43]

    Moriarty, P. (2008). Database for validation of design load extrapolation techniques. Wind Energy\/ 11\/ (6), 559--576

    work page 2008

  44. [44]

    Marelli, and B

    Moustapha, M., S. Marelli, and B. Sudret (2022). Active learning for structural reliability: Survey, general framework and benchmark. Structural Safety\/ 96 , 102714

    work page 2022

  45. [45]

    Nanty, and B

    Moutoussamy, V., S. Nanty, and B. Pauwels (2015). Emulators for stochastic simulation codes. ESAIM: Proceedings and Surveys\/ 48 , 116--155

    work page 2015

  46. [46]

    Sandberg, and O

    Olsson, A., G. Sandberg, and O. Dahlblom (2003). On L atin H ypercube sampling for structural reliability analysis. Structural Safety\/ 25 , 47--68

    work page 2003

  47. [47]

    Pan, Q., E. Byon, Y. M. Ko, and H. Lam (2020). Adaptive importance sampling for extreme quantile estimation with stochastic black-box computer models. Naval Research Logistics\/ 67\/ (7), 524--547

    work page 2020

  48. [48]

    Papadimitriou, and D

    Papaioannou, I., C. Papadimitriou, and D. Straub (2016). Sequential importance sampling for structural reliability analysis. Struct. Saf.\/ 62 , 66--75

    work page 2016

  49. [49]

    Nogal, and A

    Teixeira, R., M. Nogal, and A. O'Connor (2021). Adaptive approaches in metamodel-based reliability analysis: A review. Structural Safety\/ 89 , 102019

    work page 2021

  50. [50]

    Wilensky, U. (2015). An Introduction to Agent-Based Modeling: Modeling Natural, Social, and Engineered Complex Systems with Netlogo . The MIT Press

    work page 2015

  51. [51]

    Zhang, G

    Zheng, X., J. Zhang, G. Tang, T. Jiang, W. Peng, and W. Yao (2022). A reliability analysis method based on quantile regression and feedforward neural network. In 12th International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering (QR2MSE 2022) , pp.\ 291--297. Emeishan, China

    work page 2022

  52. [52]

    Zhu, X. and B. Sudret (2020). Replication-based emulation of the response distribution of stochastic simulators using generalized lambda distributions. International Journal for Uncertainty Quantification\/ 10\/ (3), 249--275

    work page 2020

  53. [53]

    Zhu, X. and B. Sudret (2021). Emulation of stochastic simulators using generalized lambda models. SIAM / ASA Journal on Uncertainty Quantification\/ 9\/ (4), 1345--1380

    work page 2021

  54. [54]

    Zhu, X. and B. Sudret (2023). Stochastic polynomial chaos expansions to emulate stochastic simulators . International Journal for Uncertainty Quantification\/ 13\/ (2), 31--52

    work page 2023