Pith. sign in

REVIEW 3 minor 42 references

A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty

T0 review · 0 major / 3 minor · reviewed 2026-05-24 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Partitioning prediction uncertainty by fidelity and cost plus a Believer metric improves adaptive sampling for multi-fidelity Gaussian processes.

desk verdict The paper's main idea is partitioning multi-fidelity GP uncertainty by level and cost plus a Believer term for adaptive sampling, a modest extension that targets a practical choice problem but whose gains are not visible from the abstract alone. read the letter →

arxiv 1907.11739 v1 pith:DJG6VIPX submitted 2019-07-26 stat.ML cs.LG

Sayan Ghosh , Jesper Kristensen , Yiming Zhang , Waad Subber , Liping Wang This is my paper
classification stat.MLcs.LG
keywords multi-fidelityGaussianprocessadaptivesamplingdesignofexperimentspredictiveuncertaintyBelieverconceptfluidizedbed
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper develops an adaptive sampling method for multi-fidelity Gaussian processes to lower predictive uncertainty with fewer expensive evaluations. It extends the design of experiments approach by splitting the uncertainty estimate according to each fidelity level and its execution cost. It further introduces the Believer concept to assess how a candidate sample point would change the uncertainty prediction. The strategy is tested on simple examples and on modeling a fluidized bed process. Readers interested in surrogate modeling for costly simulations would see value in potentially more efficient ways to choose both location and fidelity for new runs.

What carries the argument

Partitioning of prediction uncertainty by fidelity level and cost, combined with the Believer concept for quantifying uncertainty impact of new points.

What would settle it

A head-to-head comparison on the paper's academic examples where the new strategy does not reduce uncertainty more than a cost-augmented acquisition function for the same total computational budget.

Watch

Extended reading notes

Core claim

By partitioning the prediction uncertainty based on the fidelity level and the associated cost of execution, and by utilizing the concept of Believer which quantifies the effect of adding an exploratory design point on the Gaussian process uncertainty prediction, the proposed framework extends the traditional design of experiment for multi-fidelity Gaussian processes and leads to improved sampling decisions.

Load-bearing premise

Partitioning the prediction uncertainty based on fidelity level and cost together with the Believer concept will produce better sampling decisions and greater uncertainty reduction than prior methods that incorporate cost directly into the acquisition function.

Editorial extensions

If this is right

  • Uncertainty is handled separately for each fidelity rather than through a single combined metric.
  • Cost enters the decision via the partition rather than as a direct penalty in acquisition.
  • The Believer allows explicit calculation of uncertainty change from a hypothetical sample.
  • Applied examples include academic test functions and a real fluidized bed thermodynamic model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The approach might extend to other multi-fidelity surrogate models if the uncertainty can be similarly partitioned.
  • Further tests on problems with varying cost ratios between fidelities could show robustness.
  • Combining this sampling with optimization loops would test end-to-end performance gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

0 major / 3 minor

Summary. The paper extends the traditional design-of-experiments framework for multi-fidelity Gaussian processes by partitioning predictive uncertainty according to fidelity level and execution cost, and by incorporating a 'Believer' quantity that measures the uncertainty-reduction effect of adding an exploratory point. The resulting adaptive sampling strategy is demonstrated on academic test functions and one industrial fluidized-bed thermodynamic model.

Significance. If the empirical gains hold, the approach offers a direct way to balance uncertainty reduction against cost without folding cost into a single acquisition function, which could improve sample efficiency in multi-fidelity optimization, calibration, and UQ workflows.

minor comments (3)
  1. Abstract: 'we extent' should read 'we extend'; 'as another factor' should read 'as an additional factor'.
  2. The manuscript would benefit from an explicit statement of the partitioned uncertainty measure (e.g., separate variance terms for each fidelity) and the precise definition of the Believer update before the algorithmic description.
  3. Figure captions and axis labels should be expanded to indicate which fidelity levels and cost values are used in each panel.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive summary, significance assessment, and recommendation of minor revision. No specific major comments were listed in the report, so we have no points requiring direct response or manuscript changes at this stage.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper frames its contribution as an extension of existing multi-fidelity GP adaptive sampling methods via uncertainty partitioning by fidelity/cost and the Believer update rule. No equations or claims in the provided abstract or description reduce a derived quantity to a fitted input by construction, invoke self-citations as load-bearing uniqueness theorems, or smuggle ansatzes. The central claim remains an empirical assertion about improved sampling decisions, supported by external test cases rather than internal redefinition. This is the normal case of a self-contained methodological proposal.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Based solely on the abstract, no free parameters, axioms, or invented entities are explicitly introduced. The work extends standard multi-fidelity Gaussian process concepts without detailing new fitted quantities or unproven assumptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty." pith.science (2026). https://pith.science/paper/DJG6VIPX

@misc{pith2026190711739,
  author       = {Pith},
  title        = {Pith review of: A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DJG6VIPX}},
  note         = {Machine review of arXiv:1907.11739}
}
read the original abstract

Multi-fidelity Gaussian process is a common approach to address the extensive computationally demanding algorithms such as optimization, calibration and uncertainty quantification. Adaptive sampling for multi-fidelity Gaussian process is a changing task due to the fact that not only we seek to estimate the next sampling location of the design variable, but also the level of the simulator fidelity. This issue is often addressed by including the cost of the simulator as an another factor in the searching criterion in conjunction with the uncertainty reduction metric. In this work, we extent the traditional design of experiment framework for the multi-fidelity Gaussian process by partitioning the prediction uncertainty based on the fidelity level and the associated cost of execution. In addition, we utilize the concept of Believer which quantifies the effect of adding an exploratory design point on the Gaussian process uncertainty prediction. We demonstrated our framework using academic examples as well as a industrial application of steady-state thermodynamic operation point of a fluidized bed process

Figures

Figures reproduced from arXiv: 1907.11739 by the authors.

Figure 1
Figure 1. A TYPICAL MULTI-FIDELITY ADAPTIVE SAMPLING PROCESS [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. COMPARISON OF METHODS FOR 1-D FORRESTER FUNCTION WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. COMPARISON OF METHODS FOR 1-D FORRESTER FUNCTION WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: COMPARISON OF METHODS FOR 1-D FORRESTER FUNCTION WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: COMPARISON OF METHODS FOR 4-D PARK FUNCTION WITH COST RATIO OF HIGH-FIDELITY [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: COMPARISON OF METHODS FOR 4-D PARK FUNCTION WITH COST RATIO OF HIGH-FIDELITY [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: COMPARISON OF METHODS FOR 4-D PARK FUNCTION WITH COST RATIO OF HIGH-FIDELITY [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: COMPARISON OF METHODS FOR FLUIDIZED BED PROCESS WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: COMPARISON OF METHODS FOR FLUIDIZED BED PROCESS WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: COMPARISON OF METHODS FOR FLUIDIZED BED PROCESS WITH COST RATIO OF HIGH [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages

  1. [1]

    Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions

    Songqing Shan and G Gary Wang. Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Structural and Multidisciplinary Optimization, 41(2):219–241, 2010

    work page 2010

  2. [2]

    Surrogate-based analysis and optimization

    Nestor V Queipo, Raphael T Haftka, Wei Shyy, Tushar Goel, Rajkumar Vaidyanathan, and P Kevin Tucker. Surrogate-based analysis and optimization. Progress in aerospace sciences, 41(1):1–28, 2005

    work page 2005

  3. [3]

    Special section on multidisci- plinary design optimization: metamodeling in multidisciplinary design optimization: how far have we really come? AIAA journal, 52(4):670–690, 2014

    Felipe AC Viana, Timothy W Simpson, Vladimir Balabanov, and Vasilli Toropov. Special section on multidisci- plinary design optimization: metamodeling in multidisciplinary design optimization: how far have we really come? AIAA journal, 52(4):670–690, 2014

    work page 2014

  4. [4]

    On approaches to combine experimental strength and simulation with application to open-hole-tension configuration

    Yiming Zhang, John Meeker, Jaco Schutte, Nam Kim, and Raphael Haftka. On approaches to combine experimental strength and simulation with application to open-hole-tension configuration. In Proceedings of the American Society for Composites: Thirty-First Technical Conference, 2016

    work page 2016

  5. [5]

    Bayesian uncertainty quantification and information fusion in calphad-based thermodynamic modeling

    Pejman Honarmandi, Thien Chi Duong, S Fatheme Ghoreishi, Dou Allaire, and Raymundo Arroyave. Bayesian uncertainty quantification and information fusion in calphad-based thermodynamic modeling. Acta Materialia, 164:636–647, 2019

    work page 2019

  6. [6]

    Shabouei, W

    M. Shabouei, W. Subber, C. Williams, K. Matouš, and J. Powers. Chemo-thermal model and gaussian process emulator for combustion synthesis of ni/al composites. Combustion and Flame, 2019. Under review, Ref. No.: CNF-D-19-00026

    work page 2019

  7. [7]

    Model reduction for large-scale systems with high-dimensional parametric input space

    Tan Bui-Thanh, Karen Willcox, and Omar Ghattas. Model reduction for large-scale systems with high-dimensional parametric input space. SIAM Journal on Scientific Computing, 30(6):3270–3288, 2008

    work page 2008

  8. [8]

    Multi-fidelity surrogate modeling for application/architecture co-design

    Yiming Zhang, Aravind Neelakantan, Nalini Kumar, Chanyoung Park, Raphael T Haftka, Nam H Kim, and Herman Lam. Multi-fidelity surrogate modeling for application/architecture co-design. In International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems, pages 179–196. Springer, 2017

    work page 2017

Show all 42 references
  1. [9]

    Practical options for selecting data-driven or physics-based prognostics algorithms with reviews

    Dawn An, Nam H Kim, and Joo-Ho Choi. Practical options for selecting data-driven or physics-based prognostics algorithms with reviews. Reliability Engineering & System Safety, 133:223–236, 2015

    2015

  2. [10]

    Crashworthiness-based lightweight design problem via new robust design method considering two sources of uncertainties

    Siliang Zhang, Ping Zhu, and Wei Chen. Crashworthiness-based lightweight design problem via new robust design method considering two sources of uncertainties. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 227(7):1381...

    2013

  3. [11]

    A single-loop kriging surrogate modeling for time-dependent reliability analysis

    Zhen Hu and Sankaran Mahadevan. A single-loop kriging surrogate modeling for time-dependent reliability analysis. Journal of Mechanical Design, 138(6):061406, 2016. 12 A STRATEGY FOR ADAPTIVE SAMPLING OF MULTI-FIDELITY GAUSSIAN PROCESS TO REDUCE PREDICTIVE UNCERTAINTY A PREPRINT

    2016

  4. [12]

    Ex- perimental flapping wing optimization and uncertainty quantification using limited samples

    Anirban Chaudhuri, Raphael T Haftka, Peter Ifju, Kelvin Chang, Christopher Tyler, and Tony Schmitz. Ex- perimental flapping wing optimization and uncertainty quantification using limited samples. Structural and Multidisciplinary Optimization, 51(4):957–970, 2015

    2015

  5. [13]

    Parallel surrogate-assisted global optimization with expensive functions–a survey

    Raphael T Haftka, Diane Villanueva, and Anirban Chaudhuri. Parallel surrogate-assisted global optimization with expensive functions–a survey. Structural and Multidisciplinary Optimization, 54(1):3–13, 2016

    2016

  6. [14]

    V . V . Toropov. Modelling and Approximation Strategies in Optimization: Global and Mid-Range Approximations, Response Surface Methods, Genetic Programming, Low/High Fidelity Models. In J. Blachut and H. Eschenauer, editors, Emerging Methods for Multidisciplinary Optimization,...

    2001

  7. [15]

    Computational Approaches for Aerospace Design: The Pursuit of Excellence

    Andy Keane and Prasanth Nair. Computational Approaches for Aerospace Design: The Pursuit of Excellence. John Wiley & Sons, 2005

    2005

  8. [16]

    Raphael T. Haftka. Combining global and local approximations. AIAA Journal, 29(9):1523–1525, September 1991

    1991

  9. [17]

    A. I. J. Forrester, András Sóbester, and Andy J. Keane. Multi-fidelity optimization via surrogate modelling. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 463(2088):3251–3269, 2007

  10. [18]

    Bayesian multi-source modeling with legacy data

    Sayan Ghosh, Isaac Asher, Jesper Kristensen, You Ling, Kevin Ryan, and Liping Wang. Bayesian multi-source modeling with legacy data. In 2018 AIAA Non-Deterministic Approaches Conference, page 1663, 2018

    2018

  11. [19]

    Multi-source surrogate modeling with bayesian hierarchical regression

    Sayan Ghosh, Ryan Jacobs, and Dimitri N Mavris. Multi-source surrogate modeling with bayesian hierarchical regression. In 17th AIAA Non-Deterministic Approaches Conference, page 1817, 2015

    2015

  12. [20]

    Kennedy and Anthony O’Hagan

    Marc C. Kennedy and Anthony O’Hagan. Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 63(3):425–464, 2001

    2001

  13. [21]

    Multifidelity surrogate based on single linear regression

    Yiming Zhang, Nam H Kim, Chanyoung Park, and Raphael T Haftka. Multifidelity surrogate based on single linear regression. AIAA Journal, 56(12):4944–4952, 2018

    2018

  14. [22]

    Remarks on multi-fidelity surrogates

    Chanyoung Park, Raphael T Haftka, and Nam H Kim. Remarks on multi-fidelity surrogates. Structural and Multidisciplinary Optimization, 55(3):1029–1050, 2017

    2017

  15. [23]

    Review of multi-fidelity models

    M Giselle Fernández-Godino, Chanyoung Park, Nam-Ho Kim, and Raphael T Haftka. Review of multi-fidelity models. arXiv preprint arXiv:1609.07196, 2016

    2016

  16. [24]

    Multi-fidelity Gaussian process regression for computer experiments

    Loic Le Gratiet. Multi-fidelity Gaussian process regression for computer experiments. PhD thesis, Université Paris-Diderot - Paris VII, 2013

    2013

  17. [25]

    An approach to constructing nested space-filling designs for multi-fidelity computer experiments

    Ben Haaland and Peter ZG Qian. An approach to constructing nested space-filling designs for multi-fidelity computer experiments. Statistica Sinica, 20(3):1063, 2010

    2010

  18. [26]

    Difference mapping method using least square support vector regression for variable-fidelity metamodelling.Engineering Optimization, 47(6):719–736, 2015

    Jun Zheng, Xinyu Shao, Liang Gao, Ping Jiang, and Haobo Qiu. Difference mapping method using least square support vector regression for variable-fidelity metamodelling.Engineering Optimization, 47(6):719–736, 2015

    2015

  19. [27]

    Sequential kriging optimization using multiple-fidelity evaluations.Structural and Multidisciplinary Optimization, 32(5):369–382, 2006

    Deng Huang, Theodore T Allen, William I Notz, and R Allen Miller. Sequential kriging optimization using multiple-fidelity evaluations.Structural and Multidisciplinary Optimization, 32(5):369–382, 2006

    2006

  20. [28]

    Multifidelity importance sampling

    Benjamin Peherstorfer, Tiangang Cui, Youssef Marzouk, and Karen Willcox. Multifidelity importance sampling. Computer Methods in Applied Mechanics and Engineering, 300:490–509, 2016

    2016

  21. [29]

    Multifidelity uncertainty propagation via adaptive surrogates in coupled multidisciplinary systems

    Anirban Chaudhuri, Remi Lam, and Karen Willcox. Multifidelity uncertainty propagation via adaptive surrogates in coupled multidisciplinary systems. AIAA Journal, pages 235–249, 2017

    2017

  22. [30]

    A multi-fidelity adaptive sampling method for metamodel-based uncertainty quantification of computer simulations

    Riccardo Pellegrini, Cecilia Leotardi, Umberto Iemma, Emilio F Campana, and Matteo Diez. A multi-fidelity adaptive sampling method for metamodel-based uncertainty quantification of computer simulations. InProceedings of the VII European congress on computational methods in appli...

    2016

  23. [31]

    Kriging is well-suited to parallelize optimization

    David Ginsbourger, Rodolphe Le Riche, and Laurent Carraro. Kriging is well-suited to parallelize optimization. In Computational intelligence in expensive optimization problems, pages 131–162. Springer, 2010

    2010

  24. [32]

    C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006

    2006

  25. [33]

    P. D. Arendt, D. W. Apley, and W. Chen. Quantification of model uncertainty: Calibration, model discrepancy, and identifiability.Journal of Mechanical Design, 134(10):100908–100908, 2012

    2012

  26. [34]

    Alexander I. J. Forrester, Andras Sobester, and Andy J. Keane. Engineering Design via Surrogate Modelling - A Practical Guide. Wiley, 2008

    2008

  27. [35]

    Pattern recognition and machine learning

    Christopher M Bishop. Pattern recognition and machine learning. springer, 2006. 13 A STRATEGY FOR ADAPTIVE SAMPLING OF MULTI-FIDELITY GAUSSIAN PROCESS TO REDUCE PREDICTIVE UNCERTAINTY A PREPRINT

    2006

  28. [36]

    A self-training approach to cost sensitive uncertainty sampling

    Alexander Liu, Goo Jun, and Joydeep Ghosh. A self-training approach to cost sensitive uncertainty sampling. Machine learning, 76(2-3):257–270, 2009

    2009

  29. [37]

    Expected-improvement-based methods for adaptive sampling in multi-objective optimization problems

    Jesper Kristensen, You Ling, Isaac Asher, and Liping Wang. Expected-improvement-based methods for adaptive sampling in multi-objective optimization problems. In ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Confer...

    2016

  30. [38]

    Engineering design via surrogate modelling: a practical guide

    Alexander Forrester, Andras Sobester, and Andy Keane. Engineering design via surrogate modelling: a practical guide. John Wiley & Sons, 2008

    2008

  31. [39]

    Tuning complex computer codes to data and optimal designs

    Jeong Soo Park. Tuning complex computer codes to data and optimal designs. PhD thesis, University of Illlinois, Champaign-Urbana, 1991

    1991

  32. [40]

    A statistical method for tuning a computer code to a data base

    Dennis D Cox, Jeong-Soo Park, and Clifford E Singer. A statistical method for tuning a computer code to a data base. Computational statistics & data analysis, 37(1):77–92, 2001

    2001

  33. [41]

    Sequential design and analysis of high-accuracy and low-accuracy computer codes

    Shifeng Xiong, Peter ZG Qian, and CF Jeff Wu. Sequential design and analysis of high-accuracy and low-accuracy computer codes. Technometrics, 55(1):37–46, 2013

    2013

  34. [42]

    Modeling the steady-state thermodynamic operation point of top-spray fluidized bed processing

    Koen Dewettinck, Alex De Visscher, Lidewij Deroo, and André Huyghebaert. Modeling the steady-state thermodynamic operation point of top-spray fluidized bed processing. Journal of Food Engineering, 39(2):131–143, 1999. 14

    1999

Pith tools

Reviewed May 24, 2026 · model on record in the stance chip above.