pith. sign in

arxiv: 2606.06555 · v1 · pith:PMLUSFGKnew · submitted 2026-06-04 · 💻 cs.NE · cs.LG

Depth over Fidelity in Fixed-Budget Noisy Evolution Strategies

Sichen Wang , Zhipeng Lu This is my paper

Pith reviewed 2026-06-27 23:00 UTC · model grok-4.3

classification 💻 cs.NE cs.LG
keywords noisy evolution strategiesfixed-budget optimizationprobabilistic elite membershipRao-Blackwellizationresidual bootstrappingrank-based updatesdepth-fidelity trade-off
0
0 comments X

The pith

Probabilistic elite membership replaces hard ranks with conditional expected ranks to cut update dispersion while keeping the mean update unchanged in noisy evolution strategies.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that fixed evaluation budgets in noisy evolution strategies create a depth-fidelity trade-off, where denoising rankings uses evaluations that could instead fund more distribution updates. It proposes probabilistic elite membership (PEM) as a way to integrate over ranking uncertainty rather than commit to noisy hard ranks. A sympathetic reader would care because this Rao-Blackwellization preserves the intended mean update direction but reduces variance in the step, allowing deeper search without extra evaluations. The method is instantiated as RB-PEM via residual bootstrapping with an adaptive low-noise switch, and experiments on noisy benchmarks plus RL and hyperparameter tasks show gains precisely when misranking is high.

Core claim

PEM replaces hard rank-based weights in evolution strategies with conditional expected rank weights that integrate over ranking uncertainty. It preserves the conditional mean update while reducing conditional update dispersion, amounting to a Rao-Blackwellization of the noisy rank-based step. The approach is realized through residual bootstrapping (RB-PEM) that caps per-generation overhead and adds an adaptive probe-and-switch mechanism for low-noise regimes.

What carries the argument

Probabilistic elite membership (PEM), which substitutes conditional expected ranks for observed hard ranks to weight the update step.

If this is right

  • More generations become feasible within the same evaluation budget because per-generation overhead stays bounded.
  • Performance gains appear consistently in high-misranking regimes on the COCO bbob-noisy suite.
  • The same gains transfer to external tasks such as RL policy search and hyperparameter optimization under noise.
  • An adaptive probe-and-switch mechanism automatically falls back to conventional behavior when noise is low.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The Rao-Blackwellization framing suggests PEM could be applied to other rank-based noisy optimizers that currently commit to single noisy rankings.
  • If the conditional expectation can be approximated more cheaply than bootstrapping, the depth advantage would increase further.
  • In extremely high-noise regimes the method may still require a minimum number of probes per generation to keep the approximation reliable.

Load-bearing premise

Residual bootstrapping with capped per-generation overhead accurately approximates conditional expected ranks without introducing bias that would degrade performance when misranking is severe.

What would settle it

Run RB-PEM and a standard denoising baseline on a high-misranking noisy benchmark with the same total evaluations; if the denoising baseline achieves lower final error or higher success rate, the advantage of depth via PEM is refuted.

Figures

Figures reproduced from arXiv: 2606.06555 by Sichen Wang, Zhipeng Lu.

Figure 1
Figure 1. Figure 1: RB-PEM improves fixed-budget progress on COCO bbob-noisy. Median (solid) and interquartile range (shaded) of log10(f(ˆx) − f ⋆ ) versus evaluations on four representative high-misranking functions (f110, f113, f116, f125; d = 40, B = 100d, 15 instances each). Right-side markers indicate the number of completed CMA-ES generations (depth). Methods that preserve depth (CMA-ES, RB-PEM) generally achieve lower … view at source ↗
Figure 2
Figure 2. Figure 2: Depth–fidelity trade-off under a fixed budget. Each method is plotted by its average per-candidate evaluation cost (fidelity, x-axis) and number of completed generations (depth, y-axis) on the high-misranking COCO bbob-noisy subset (d = 40, B = 100d, 225 problems = 15 functions × 15 in￾stances). Bubble annotations encode the median final log10 regret (smaller is better). The grey hyperbola marks the budget… view at source ↗
Figure 3
Figure 3. Figure 3: Task-level algorithm ranking (1 = best). Median rank across instances for each method (columns) on each task (rows): 15 high-misranking COCO bbob-noisy functions and six ex￾ternal tasks (d = 40, B = 100d for COCO; task-specific budgets otherwise; darker is better). RB-PEM ranks first on most high￾misranking tasks but underperforms CMA-ES on low-misranking tasks (e.g., CartPole, Pendulum), where the bootstr… view at source ↗
Figure 5
Figure 5. Figure 5: Conditional advantage and single crossing. Estimated ∆(p) = E[LCMA − LRB-PEM | P = p] vs. the probe statistic P (normalized rank disagreement, Eq. (56)) on all 30 COCO bbob-noisy functions (d = 40, B = 500d); positive ∆ favors RB-PEM. The curve crosses zero once near τ = 0.12 (dashed line), giving a threshold rule (RB-PEM when P ≥ τ , else CMA￾ES) that is Bayes-optimal under single crossing (Proposition 5)… view at source ↗
Figure 4
Figure 4. Figure 4: Per-function ranking across all 30 bbob-noisy functions reveals a bimodal misranking structure that defines the high-misranking subset. Mean rank (1 = best) of five methods, sorted by the median probe statistic P (Eq. (56)). The P-values are sharply bimodal: 14 low-misranking functions cluster in P ∈ [0.00, 0.15] (left) and 15 high-misranking ones in P ∈ [0.30, 0.35] (right), separated by a gap of 0.145 (≈… view at source ↗
Figure 7
Figure 7. Figure 7: Mechanism validation on a strongly convex quadratic. (a) Update dispersion ∥∆m(a) − ∆m(b) ∥ 2 grows with two-draw misranking MRD (Pearson r = 0.45). (b) For quadratic objectives, the Jensen gap equals 1 2Var(∆m) exactly (slope 1), confirming that dispersion translates into expected loss under curvature. Protocol: d=40, λ=16, µ=8, σnoise=1.0. 200 independently sampled candidate sets; 256 Monte Carlo draws p… view at source ↗
Figure 8
Figure 8. Figure 8: Estimator ablations on the high-misranking COCO subset (d=40, B=100d, 225 problems). Boxplots show per-instance ∆ log10 regret relative to CMA-ES; the dashed line marks parity. Percentages report win rates (fraction with ∆ > 0). (a) Reevaluation cap Kmax: gains persist even at Kmax=0 (no boundary reevaluations). (b) Bootstrap samples Bboot: stable across 16–64. (c) Noise￾model choice: all variants improve;… view at source ↗
Figure 9
Figure 9. Figure 9: Diagnostics make residual-pool assumptions falsifiable. (a) Boxplots of four diagnostic summaries for good runs (n = 193, RB-PEM wins) vs. bad runs (n = 32, RB-PEM loses) on the COCO high-misranking subset (d = 40, B = 100d). Shape W1 (p = 0.003) and centering stability (p = 0.001) significantly separate the two groups (one-sided Mann–Whitney); drift and scale do not, indicating that shape mismatch and sta… view at source ↗
Figure 10
Figure 10. Figure 10: Empirical validation of sandwich bounds for rank disagreement. (a) Kendall discordance qpair vs. MRD; (b) top-µ disagreement Mtopµ vs. MRD. Grey lines indicate the bounds in Eq. (59). Zero violations. Protocol: d=40, λ=15, µ=7. 750 candidate sets from COCO bbob-noisy (30 functions × 1 instance × 25 candidate sets per function), sampled via CMA-ES evolution. F.5. Interpreting the Rank-Disagreement Probe vi… view at source ↗
Figure 11
Figure 11. Figure 11: Variance ̸= misranking under radial noise. (a) The variance probe is near machine precision while the misranking probe varies widely; triggers: MR 49/54, Var 0/54. (b) Switching decisions: MR-based probe-and-switch outperforms variance-based (MR wins 39/54; p = 0.0007). Protocol: Radial noise σeff (x) = 0.5∥x − x0∥, d ∈ {80, 160, 320}, B = 200d, Bboot=32, Kmax=1. λ follows the CMA-ES default at each d (17… view at source ↗
Figure 12
Figure 12. Figure 12: Probe calibration curves. Empirical Pr(RB-PEM wins | P) versus probe statistic P (quantile bins; 95% Wilson intervals). Shaded regions indicate which algorithm is preferred; the vertical dashed line is the calibrated threshold. The win probability increases monotonically with P, supporting a threshold decision rule. Protocol: d=40, λ=15, µ=7, Bboot=32, Kmax=1. Two budget levels: B=200d and B=500d. 450 pro… view at source ↗
Figure 13
Figure 13. Figure 13: Probe budget vs. ROC. ROC curves for different probe population sizes λ on COCO bbob-noisy (d=40, B=200d). Legend: AUC and accuracy at the displayed operating point. Reliability improves up to λ ≈ 16 and then saturates. Protocol: Bboot=32, Kmax=1. 30 functions × 15 instances = 450 problems per λ value, each run once. λ ∈ {4, 8, 16, 32}; µ=⌊λ/2⌋. Operating point: τ=0.12. 24 [PITH_FULL_IMAGE:figures/full_f… view at source ↗
Figure 14
Figure 14. Figure 14: Threshold sensitivity. (a) Classification accuracy vs. threshold τ : MR exhibits a broad plateau and outperforms Var. (b) Decision regret vs. τ : over-switching dominates at small τ , under-switching at large τ ; the minimum lies within the accuracy plateau, showing robust tuning. Protocol: d=40, λ=15, µ=7, Bboot=32, Kmax=1. Two budget levels: B=200d and B=500d. 450 problems per budget (30 functions × 15 … view at source ↗
Figure 15
Figure 15. Figure 15: Depth–fidelity robustness and baseline sensitivity. (a) Win rate of RB-PEM across budgets and dimensions against UH-CMA-ES, Resample(k=10), and CMA-ES. (b) UH-CMA-ES win rate vs. CMA-ES and vs. Probe-and-Switch (τ=0.12) as a function of maxevals; all values are far below parity, and larger maxevals further degrades performance. Protocol: Bboot=32, Kmax=1. Panel (a): d ∈ {20, 40}, B ∈ {50d, 100d, 200d}, 15… view at source ↗
Figure 16
Figure 16. Figure 16: MLP training on digits0 (nonconvex). Final post-hoc loss (lower is better) across 50 seeds at fixed budget B = 40d. Numbers above boxes indicate the fraction of seeds on which the method improves upon (or ties with) CMA-ES. Left: Bbatch = 4 (extreme noise); Center: Bbatch = 16 (moderate noise); Right: Bbatch = 256 (deterministic). Protocol: d=265, λ=20 (CMA-ES default), µ=10, Bboot=32, Kmax=1. 50 independ… view at source ↗
read the original abstract

Noisy evolution strategies under fixed evaluation budgets face a depth-fidelity trade-off: spending evaluations to denoise intra-generation rankings reduces the number of distribution updates the optimizer can execute. We argue for depth over fidelity and propose probabilistic elite membership (PEM), which replaces hard rank-based weights in evolution strategies with conditional expected rank weights that integrate over ranking uncertainty. PEM preserves the conditional mean update while reducing conditional update dispersion, a Rao-Blackwellization of the noisy rank-based step. We instantiate PEM via residual bootstrapping (RB-PEM) with capped per-generation overhead, complemented by an adaptive probe-and-switch mechanism for low-noise regimes. Across the COCO bbob-noisy suite and external tasks including RL policy search and hyperparameter optimization, RB-PEM achieves consistent gains in high-misranking, budget-constrained settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper argues that in fixed-budget noisy evolution strategies, prioritizing more distribution updates (depth) over denoising individual rankings (fidelity) is preferable. It introduces probabilistic elite membership (PEM), which replaces hard rank-based weights with conditional expected ranks that integrate over ranking uncertainty. This is claimed to preserve the conditional mean update while reducing dispersion, constituting a Rao-Blackwellization of the noisy rank-based step. The approach is instantiated as RB-PEM via residual bootstrapping with capped per-generation overhead and an adaptive probe-and-switch mechanism, with empirical gains reported on the COCO bbob-noisy suite plus RL policy search and hyperparameter optimization tasks.

Significance. If the mean-preservation property holds exactly and the bootstrap approximation introduces no systematic bias, the method would provide a principled way to improve noisy ES performance under tight budgets by enabling more updates without degrading the expected direction of progress. The Rao-Blackwellization framing and empirical consistency across benchmarks would represent a useful contribution to noisy black-box optimization.

major comments (2)
  1. [Abstract] Abstract (paragraph on RB-PEM instantiation): the claim that residual bootstrapping with capped per-generation overhead accurately approximates the conditional expected ranks without introducing bias is load-bearing for the Rao-Blackwellization argument, yet no error analysis, bias bound, or high-misranking regime verification is provided; if the bootstrap deviates from E[rank | observations], the mean-preservation property fails and observed gains cannot be attributed to variance reduction.
  2. [Abstract] Abstract (PEM definition): while the high-level Rao-Blackwellization argument is grounded in standard conditional expectation properties, the manuscript provides no explicit derivation showing that the PEM-weighted update equals the conditional expectation of the noisy rank-based update; without this, the central claim that dispersion is reduced while the mean is preserved remains unverified.
minor comments (1)
  1. [Abstract] The abstract refers to 'COCO bbob-noisy suite' without specifying the exact functions, noise levels, or budget settings used in the experiments; this makes it difficult to assess the scope of the reported gains.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments highlighting the need for stronger theoretical grounding. We address each major comment below and will revise the manuscript to incorporate explicit derivations and approximation analysis.

read point-by-point responses

  1. Referee: [Abstract] Abstract (paragraph on RB-PEM instantiation): the claim that residual bootstrapping with capped per-generation overhead accurately approximates the conditional expected ranks without introducing bias is load-bearing for the Rao-Blackwellization argument, yet no error analysis, bias bound, or high-misranking regime verification is provided; if the bootstrap deviates from E[rank | observations], the mean-preservation property fails and observed gains cannot be attributed to variance reduction.

    Authors: We agree that the manuscript does not include a formal error analysis or bias bounds for the residual bootstrapping approximation in RB-PEM. PEM itself is defined to achieve exact mean preservation via conditional expectation, while RB-PEM is a practical, capped-overhead approximation whose fidelity to the ideal conditional ranks is supported only empirically. We will add a dedicated subsection discussing the bootstrap approximation, its potential bias in high-misranking regimes, and the distinction between exact PEM properties and the RB-PEM instantiation. revision: yes

  2. Referee: [Abstract] Abstract (PEM definition): while the high-level Rao-Blackwellization argument is grounded in standard conditional expectation properties, the manuscript provides no explicit derivation showing that the PEM-weighted update equals the conditional expectation of the noisy rank-based update; without this, the central claim that dispersion is reduced while the mean is preserved remains unverified.

    Authors: The referee is correct that an explicit derivation is absent. By definition, PEM replaces each random rank with its conditional expectation given the observations; the law of total expectation then implies that the PEM-weighted update has the same conditional expectation as the original noisy rank-based update. We will insert a short, self-contained derivation (one paragraph plus two displayed equations) in the methods section to make this step explicit. revision: yes

Circularity Check

0 steps flagged

No circularity; PEM applies standard Rao-Blackwellization to rank weights without reducing to fitted inputs or self-citations

full rationale

The paper's core derivation claims that replacing noisy ranks with their conditional expectations (via PEM) preserves the mean update while reducing dispersion, presented as a direct Rao-Blackwellization of the rank-based step. This follows from standard conditional expectation properties and does not rely on any self-referential definition, fitted parameter renamed as prediction, or load-bearing self-citation. The RB-PEM instantiation uses residual bootstrapping as an approximation method, but the claimed statistical property holds independently of the specific approximation technique. No equations or steps in the provided text reduce the result to its own inputs by construction. The derivation remains self-contained against external statistical benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on modeling ranking uncertainty via bootstrapping and the statistical property that conditional expectation reduces variance without changing the mean; no free parameters or invented physical entities are evident from the abstract.

axioms (1)
  • domain assumption Ranking uncertainty can be integrated via residual bootstrapping to yield conditional expected ranks
    Invoked to define the PEM weights that replace hard ranks.
invented entities (1)
  • probabilistic elite membership (PEM) no independent evidence
    purpose: Replace hard rank-based weights with conditional expected rank weights
    New weighting scheme introduced to achieve Rao-Blackwellization in noisy ES.

pith-pipeline@v0.9.1-grok · 5662 in / 1293 out tokens · 23029 ms · 2026-06-27T23:00:00.127233+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

65 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    Evolutionary Computation , volume =

    Nikolaus Hansen and Andreas Ostermeier , title =. Evolutionary Computation , volume =

  2. [2]

    A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , journal =

    Nikolaus Hansen and Andr. A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , journal =

  3. [3]

    Uncertainty handling

    Verena Heidrich. Uncertainty handling. Genetic and Evolutionary Computation Conference (GECCO 2009) , pages =. 2009 , doi =

    2009

  4. [4]

    Evolutionary Optimization in Uncertain Environments -- A Survey , journal =

    Yaochu Jin and J. Evolutionary Optimization in Uncertain Environments -- A Survey , journal =. 2005 , doi =

    2005

  5. [5]

    Swarm and Evolutionary Computation , volume =

    Pratyusha Rakshit and Amit Konar and Swagatam Das , title =. Swarm and Evolutionary Computation , volume =. 2017 , doi =

    2017

  6. [6]

    E. J. Hughes , title =. Evolutionary Multi-Criterion Optimization (EMO 2001) , series =

    2001

  7. [7]

    Pareto-Front Exploration with Uncertain Objectives , booktitle =

    J. Pareto-Front Exploration with Uncertain Objectives , booktitle =

  8. [8]

    Fieldsend and Richard M

    Jonathan E. Fieldsend and Richard M. Everson , title =. Proceedings of the. 2005 , doi =

    2005

  9. [9]

    Fieldsend and Richard M

    Jonathan E. Fieldsend and Richard M. Everson , title =. IEEE Transactions on Evolutionary Computation , volume =. 2015 , doi =

    2015

  10. [10]

    Comparing Solutions under Uncertainty in Multiobjective Optimization , journal =

    Miha Mlakar and Tea Tu. Comparing Solutions under Uncertainty in Multiobjective Optimization , journal =

  11. [11]

    Gutjahr and Sabine Pichler , title =

    Walter J. Gutjahr and Sabine Pichler , title =. Annals of Operations Research , volume =

  12. [12]

    Gomes and Bart Selman , title =

    Carla P. Gomes and Bart Selman , title =. Artificial Intelligence , volume =

  13. [13]

    Artificial Intelligence , volume =

    Stuart Russell and Eric Wefald , title =. Artificial Intelligence , volume =

  14. [14]

    Learning Dynamic Algorithm Portfolios , journal =

    Matteo Gagliolo and J. Learning Dynamic Algorithm Portfolios , journal =

  15. [15]

    A Racing Algorithm for Configuring Metaheuristics , booktitle =

    Mauro Birattari and Thomas St. A Racing Algorithm for Configuring Metaheuristics , booktitle =

  16. [16]

    2010 , doi =

    Analyzing bandit-based adaptive operator selection mechanisms , journal =. 2010 , doi =

    2010

  17. [17]

    Burke and Michel Gendreau and Matthew R

    Edmund K. Burke and Michel Gendreau and Matthew R. Hyde and Graham Kendall and Gabriela Ochoa and Ender. Hyper-heuristics: a survey of the state of the art , journal =. 2013 , doi =

    2013

  18. [18]

    Tibshirani , title =

    Bradley Efron and Robert J. Tibshirani , title =

  19. [19]

    Optimal Transport: Old and New , series =

    C. Optimal Transport: Old and New , series =

  20. [20]

    Concentration Inequalities: A Nonasymptotic Theory of Independence , publisher =

    St. Concentration Inequalities: A Nonasymptotic Theory of Independence , publisher =

  21. [21]

    van der Vaart and Jon A

    Aad W. van der Vaart and Jon A. Wellner , title =

  22. [22]

    Probability Theory and Related Fields , volume =

    Nicolas Fournier and Arnaud Guillin , title =. Probability Theory and Related Fields , volume =

  23. [23]

    Huber and Elvezio M

    Peter J. Huber and Elvezio M. Ronchetti , title =

  24. [24]

    Graham , title =

    Persi Diaconis and Ronald L. Graham , title =. Journal of the Royal Statistical Society, Series B (Statistical Methodology) , volume =

  25. [25]

    Yurii Nesterov , title =

  26. [26]

    Econometrica , volume =

    Paul Milgrom and Chris Shannon , title =. Econometrica , volume =

  27. [27]

    The Annals of Mathematical Statistics , volume =

    Samuel Karlin and Herman Rubin , title =. The Annals of Mathematical Statistics , volume =

  28. [28]

    Shai Shalev-Shwartz and Shai Ben-David , title =

  29. [29]

    Dataset Shift in Machine Learning , publisher =

  30. [30]

    Roman Vershynin , title =

  31. [31]

    Evolutionary Algorithms in the Presence of Noise: To Sample or Not to Sample , booktitle =

    Hans. Evolutionary Algorithms in the Presence of Noise: To Sample or Not to Sample , booktitle =. 2007 , doi =

    2007

  32. [32]

    Kruisselbrink and Edgar Reehuis and Andr

    Johannes W. Kruisselbrink and Edgar Reehuis and Andr. Using the uncertainty handling. Proceedings of the 13th Annual Genetic and Evolutionary Computation Conference (. 2011 , doi =

    2011

  33. [33]

    Groves and J

    Matthew J. Groves and J. Sequential sampling for noisy optimisation with. Proceedings of the Genetic and Evolutionary Computation Conference (. 2018 , doi =

    2018

  34. [34]

    IEEE Transactions on Systems, Man, and Cybernetics: Systems , volume =

    Zhenhua Li and Shuo Zhang and Xinye Cai and Qingfu Zhang and Xiaomin Zhu and Zhun Fan and Xiuyi Jia , title =. IEEE Transactions on Systems, Man, and Cybernetics: Systems , volume =. 2022 , doi =

    2022

  35. [35]

    Proceedings of the Genetic and Evolutionary Computation Conference (

    Kouhei Nishida and Youhei Akimoto , title =. Proceedings of the Genetic and Evolutionary Computation Conference (. 2016 , doi =

    2016

  36. [36]

    Ono , title =

    Masahiro Nomura and Youhei Akimoto and I. Ono , title =. Proceedings of the Genetic and Evolutionary Computation Conference (. 2023 , doi =

    2023

  37. [37]

    Rendell , title =

    Warren Armstrong and Peter Christen and Eric McCreath and Alistair P. Rendell , title =. Workshop on Integrating Artificial Intelligence and Data Mining (. 2006 , note =

    2006

  38. [38]

    L. Sch. Greedy Restart Schedules: A Baseline for Dynamic Algorithm Selection on Numerical Black-box Optimization Problems , booktitle =. 2025 , doi =

    2025

  39. [39]

    Towards dynamic algorithm selection for numerical black-box optimization: investigating

    Diederick Vermetten and Hao Wang and Thomas B. Towards dynamic algorithm selection for numerical black-box optimization: investigating. Proceedings of the Genetic and Evolutionary Computation Conference (. 2020 , doi =

    2020

  40. [40]

    Applications of Evolutionary Computation - 26th European Conference,

    Diederick Vermetten and Hao Wang and Kevin Sim and Emma Hart , title =. Applications of Evolutionary Computation - 26th European Conference,. 2023 , doi =

    2023

  41. [41]

    Arnold and Hans

    Dirk V. Arnold and Hans. A general noise model and its effects on evolution strategy performance , journal =. 2006 , doi =

    2006

  42. [42]

    A New Approach for Predicting the Final Outcome of Evolution Strategy Optimization Under Noise , journal =

    Hans. A New Approach for Predicting the Final Outcome of Evolution Strategy Optimization Under Noise , journal =. 2005 , doi =

    2005

  43. [43]

    Science China Information Sciences , volume =

    Chao Bian and Chao Qian and Yang Yu and Ke Tang , title =. Science China Information Sciences , volume =. 2021 , doi =

    2021

  44. [44]

    CoRR , volume =

    Timo Budszuhn and Mark Joachim Krallmann and Daniel Horn , title =. CoRR , volume =. 2025 , doi =

    2025

  45. [45]

    IEEE Transactions on Evolutionary Computation , volume =

    Thomas Philip Runarsson and Xin Yao , title =. IEEE Transactions on Evolutionary Computation , volume =. 2000 , doi =

    2000

  46. [46]

    Proceedings of the Genetic and Evolutionary Computation Conference , series =

    Uchida, Kento and Nishihara, Kenta and Shirakawa, Shinichi , title =. Proceedings of the Genetic and Evolutionary Computation Conference , series =. 2024 , pages =

    2024

  47. [47]

    Proceedings of the Genetic and Evolutionary Computation Conference , series =

    Nishida, Kouhei and Akimoto, Youhei , title =. Proceedings of the Genetic and Evolutionary Computation Conference , series =. 2018 , pages =

    2018

  48. [48]

    ACM Transactions on Evolutionary Learning and Optimization , volume =

    Nomura, Masahiro and Akimoto, Youhei and Ono, Isao , title =. ACM Transactions on Evolutionary Learning and Optimization , volume =. 2025 , doi =

    2025

  49. [49]

    Parallel Problem Solving from Nature -- PPSN XVII , series =

    Krause, Oswin , title =. Parallel Problem Solving from Nature -- PPSN XVII , series =. 2022 , pages =

    2022

  50. [50]

    Gaussian Process Surrogate Models for the

    Bajer, Luk. Gaussian Process Surrogate Models for the. Evolutionary Computation , volume =. 2019 , doi =

    2019

  51. [51]

    and Daulton, Samuel and Letham, Benjamin and Wilson, Andrew Gordon and Bakshy, Eytan , title =

    Balandat, Maximilian and Karrer, Brian and Jiang, Daniel R. and Daulton, Samuel and Letham, Benjamin and Wilson, Andrew Gordon and Bakshy, Eytan , title =. Advances in Neural Information Processing Systems , volume =

  52. [52]

    Advances in Neural Information Processing Systems , volume =

    Ament, Sebastian and Daulton, Samuel and Eriksson, David and Balandat, Maximilian and Bakshy, Eytan , title =. Advances in Neural Information Processing Systems , volume =

  53. [53]

    A Tutorial on Bayesian Optimization

    Frazier, Peter I. , title =. arXiv preprint arXiv:1807.02811 , year =

    work page internal anchor Pith review Pith/arXiv arXiv

  54. [54]

    , title =

    Kim, Seong-Hee and Nelson, Barry L. , title =. Proceedings of the Winter Simulation Conference , year =

  55. [55]

    Selecting a Selection Procedure , journal =

    Branke, J. Selecting a Selection Procedure , journal =. 2007 , doi =

    2007

  56. [56]

    , title =

    Frazier, Peter I. , title =. Operations Research , volume =. 2014 , doi =

    2014

  57. [57]

    Jeff and Fan, Weiwei and Luo, Jun , title =

    Hong, L. Jeff and Fan, Weiwei and Luo, Jun , title =. Frontiers of Engineering Management , volume =. 2021 , doi =

    2021

  58. [58]

    Jeff and Jiang, Guangxin and Zhong, Ying , title =

    Hong, L. Jeff and Jiang, Guangxin and Zhong, Ying , title =. INFORMS Journal on Computing , volume =. 2022 , doi =

    2022

  59. [59]

    Efficient Expected Improvement Estimation for Continuous Multiple Ranking and Selection , booktitle =

    Pearce, Michael and Branke, J. Efficient Expected Improvement Estimation for Continuous Multiple Ranking and Selection , booktitle =. 2017 , pages =

    2017

  60. [60]

    Sequential Sampling in Noisy Environments , booktitle =

    J. Sequential Sampling in Noisy Environments , booktitle =. 2004 , doi =

    2004

  61. [61]

    Tohoku Mathematical Journal, Second Series , volume =

    Azuma, Kazuoki , title =. Tohoku Mathematical Journal, Second Series , volume =. 1967 , doi =

    1967

  62. [62]

    Journal of the American Statistical Association , volume =

    Hoeffding, Wassily , title =. Journal of the American Statistical Association , volume =. 1963 , doi =

    1963

  63. [63]

    1991 , isbn =

    Williams, David , title =. 1991 , isbn =

    1991

  64. [64]

    , title =

    Casella, George and Berger, Roger L. , title =. 2002 , isbn =

    2002

  65. [65]

    and Casella, George , title =

    Lehmann, Erich L. and Casella, George , title =. 1998 , isbn =

    1998