Pith. sign in

REVIEW 4 major objections 4 minor 62 references

Accelerating Evolutionary Strategy via Rao-Blackwellizing Realization of Uncertain Input

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that in optimization under input uncertainty, conditioning the evolutionary-strategy gradient estimator on the observed realization of the input—a Rao-Blackwellization—yields an unbiased estimator with no larger variance,

desk verdict Clean Rao-Blackwellization result for ES under input uncertainty, with experiments that outrun the theory; the continuous-control RL estimator is the main soft spot. read the letter →

arxiv 2608.02073 v1 pith:QOLYYVIP submitted 2026-08-03 math.OC cs.LG

So Nakashima , Tetsuya J. Kobayashi This is my paper
classification math.OCcs.LG MSC 90C1590C56
keywords evolutionarystrategyoptimizationunderinputuncertaintyRao-Blackwellizationvariancereductiongradientestimationreinforcementlearningphenotypeestimatorreparameterizationtrick
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Optimization under Input Uncertainty (OIU) is the setting where the input to an objective is sampled from a distribution controlled by parameters, and only the realized input is evaluated. Standard evolutionary strategies use the sampled parameter and the evaluation, but throw away the realized input. This paper proves that conditioning the gradient estimator on the realized input—a Rao-Blackwellization—yields an unbiased estimator with variance no larger than the standard one. The resulting algorithm, PAES, converges faster than ES on BBOB benchmarks and on discrete and continuous reinforcement learning tasks at comparable computation time. The core insight is that the observed realization carries signal that the evaluation alone averages over.

What carries the argument

The phenotype gradient estimator, Eq. (4.8): (1/N)Σ_i W(f(x_i)) ∂log P_F(x_i)/∂ψ, where P_F is the marginal input distribution. Its engine is the identity E_{P_F(θ|x)}[∂log q_ψ(θ)/∂ψ] = ∂log P_F(x)/∂ψ (Lemma 8, 'Fisher's identity'), which lets a conditional expectation be computed as a simple score. Around this, the paper builds a Rao-Blackwellization theorem for any unbiased estimator (Theorem 5), exact analytic updates for exponential-family/conjugate pairs (Proposition 7), and the reparameterization approximation of Eq. (4.36)/(5.14) for non-conjugate RL policies.

What would settle it

Measure the empirical variance of the implemented PAES update (Eq. 5.14) versus ES's update on Swimmer at the initial σ=0.2; if PAES's update variance exceeds ES's at that σ, the small-σ approximation, not the Rao-Blackwellization theorem, is carrying the improvement.

Watch

Extended reading notes

Core claim

The central discovery is that, in OIU, the gradient of the smoothed objective F(ψ)=E[f(x)] can be estimated by the score of the marginal distribution of realized inputs, P_F(x)=∫p_θ(x)q_ψ(θ)dθ, instead of by the score of the population distribution q_ψ(θ). The paper proves this 'phenotype gradient estimator' is unbiased (Theorem 3) and has expected squared norm no larger than the standard 'genotype gradient estimator' (Eq. 4.10). The proof goes through an identity sometimes called Fisher's identity: the conditional expectation of the genotype score given the realized input equals the phenotype score. Thus the phenotype estimator is exactly the Rao-Blackwellization of the genotype estimator (

Load-bearing premise

The variance-reduction guarantee holds for the exact conditional-expectation estimator; for the continuous-RL experiments, the algorithm actually run assumes the population perturbation is small enough that dropping the leftover noise term does not change the result.

Editorial extensions

If this is right

  • For any OIU problem where the realized input can be recorded, replacing the genotype estimator with the phenotype estimator never increases the estimator's mean squared norm, so PAES should match or beat ES at equal population size in terms of gradient reliability.
  • On the 24 BBOB functions with Gaussian input noise (d=40), PAES reaches lower error to optimum than ES on most functions within 5000 generations.
  • On FrozenLake-v1 with the exact Rao-Blackwellized Dirichlet update, PAES reaches a high deterministic reward faster than ES in both generations and wall-clock time.
  • On CartPole, Pendulum, and Swimmer with the reparameterization approximation, PAES converges faster than ES in generation count and elapsed time.
  • Computation per generation is comparable to ES, so the observed speedups are not offset by extra per-step cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the exact variance guarantee is for the exact Rao-Blackwellized estimator; the implemented RL estimator discards an O(σξ) Taylor remainder, so at the larger σ used in Swimmer the practical advantage is an empirical claim, not a direct corollary of Theorem 5.
  • Beyond the paper: the phenotype estimate can be read as 'imitate the high-reward realized inputs', a view that connects PAES to behavioral cloning and could guide a version for partially observed inputs or for combining PAES with off-policy learning.
  • Beyond the paper: the same conditional-expectation trick could be applied inside other OIU solvers (Bayesian optimization, robust gradient methods) whenever the realized input is measurable, potentially yielding similar acceleration outside ES.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies Optimization under Input Uncertainty (OIU), where the objective is over the expectation of f(x) with x ~ p_θ(x), and the realized x is observable. It proposes Phenotype-Accelerated Evolutionary Strategy (PAES), which replaces the standard ES/genotype gradient estimator with a phenotype gradient estimator using the marginal density of x. The main theoretical results (Theorem 3 and its generalization Theorem 5) show that the phenotype estimator is unbiased and has no larger variance than the genotype estimator, via Rao-Blackwellization. The paper derives analytic phenotype gradients for Gaussian OIU and for Dirichlet-multinomial discrete RL, and proposes a reparameterization-trick approximation for continuous RL. Experiments on BBOB functions, FrozenLake, CartPole, Pendulum, and Swimmer report faster convergence for PAES than for ES.

Significance. If scoped to the exact Rao-Blackwellized estimators, the paper provides a clean and useful insight: in OIU problems where the realized input is observed, standard ES discards information that can reduce gradient-estimator variance. The proofs of Theorem 3 and Theorem 5 are standard but correctly executed, and the analytic computations for exponential families and for discrete RL are valuable. The numerical results on BBOB and FrozenLake are consistent with the variance-reduction idea. However, the continuous-control experiments rely on an approximate estimator that the paper itself admits is not Rao-Blackwellized, and the theoretical guarantee does not apply to that estimator as implemented. The paper's broader claim that the observed RL speedups are explained by Rao-Blackwellization is therefore not yet supported.

major comments (4)
  1. [Sec. 5.3, Eq. (5.17)] The estimator used in the continuous-control experiments is not Rao-Blackwellized. Eq. (5.14) retains the noise ξ, and Eq. (5.17) discards an O(σξ) term to obtain a ξ-independent 'leading term.' Since ξ is a standard Gaussian on a high-dimensional policy-parameter space, ||ξ|| scales with the square root of the parameter dimension, so the dropped term is O(σ sqrt(d)), not O(σ). For Swimmer, σ starts at 0.2 and the network has many parameters, so the dropped term can be large. Consequently, Theorem 5 (Eq. 4.17) does not provide a variance-reduction guarantee for the algorithm actually run in Sec. 6.3. The paper's own text acknowledges this ('This estimator is not an exact Rao-Blackwellization'), but Sec. 7 nonetheless claims the experiments are 'consistent with our theoretical results.' The authors should either derive an exact/unbiased conditional estimator or empirically demonstrate tha
  2. [Sec. 5.3, Eq. (5.14); Sec. 6.3.2-6.3.3] The reparameterized estimator (5.14) is derived for discrete action spaces, using empirical counts j(a,s) and the softmax policy π_β(a|s). Pendulum-v1 and Swimmer-v5 have continuous action spaces. The paper does not specify how j(a,s) is defined for continuous actions or how π(a|s) is evaluated in that setting. This makes the algorithms for the two continuous-control environments underspecified and the experimental results not reproducible. The authors must provide the continuous-action formulation, or restrict the claims to discrete action spaces.
  3. [Sec. 2.3, Eq. (2.7)-(2.8); Sec. 4, Theorem 3] The unbiasedness and variance-reduction theorems are stated for a fixed transformation W. In all experiments, W is rank-based and depends on the entire current population (Eqs. 6.1, 6.4, 6.5). For such data-dependent W, the identity E[W(f(x)) ∂ log q_ψ(θ)/∂ψ] = ∂/∂ψ E[W(f(x))] is not valid in the stated form, and the Rao-Blackwell variance inequality does not automatically apply. The paper does not discuss this gap. Since the experiments use exactly these rank-based weights, the theoretical results do not directly cover the evaluated algorithms. A treatment in the IGO framework (where rank-based W defines a valid objective) or an explicit caveat is needed.
  4. [Sec. 5.1 and Sec. 6.1] The Gaussian OIU update rule (5.4) is a natural-gradient estimator using the Fisher information of P_F(x), while the ES baseline in (6.2)-(6.3) uses a different natural gradient (the Fisher information of q_ψ(θ)). The variance-reduction theorem (Theorem 3) applies to the plain gradient estimators (4.4) and (4.8). The paper does not prove that the variance ordering is preserved after multiplying by the inverse Fisher information matrices, nor that comparing the two natural-gradient variants in Figure 6.1 is a test of the theorem. The experimental comparison should be aligned with the theoretical statement, or the theory should be extended to natural gradients.
minor comments (4)
  1. [Sec. 2.2] The variance of a random vector is defined as V_p(v)[v] = E_{p(v)}[||v||^2]; this is a second moment, not a variance. The proofs of Theorem 5 use the centered expression E[(U-u)^2]. Please correct the definition to E[||v - E[v]||^2].
  2. [Sec. 6.3.3 and App. E.2] The Swimmer learning curve in Figure 6.4(c) is plotted against elapsed time, while the text states PAES 'converges faster than ES in terms of generations.' The same inconsistency appears in Appendix E.2. Clarify which quantity is on the horizontal axis.
  3. [Sec. 4.2] Typo: 'gennotype information' should be 'genotype information.'
  4. [Sec. 4.5, Eq. (4.23)-(4.24)] The notation ψ' = (ϕ',ν') is introduced after Eq. (4.24), but the formula already uses it. Moving the definition before the equation would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the Rao-Blackwell variance reduction is proved from standard identities, and the reparameterized RL estimator is explicitly labeled approximate rather than presented as exact theory.

full rationale

The paper's central variance-reduction claims (Theorem 3, Eq. 4.10; Theorem 5, Eq. 4.17) are proved in Appendix C.1 using the standard conditional-expectation decomposition of variance (Eqs. C.4–C.17) and Fisher's identity (Lemma 8, Eqs. C.18–C.24). Proposition 6 shows that the phenotype gradient estimator (4.8) is exactly the Rao-Blackwellization of the genotype gradient estimator (4.4); this is a theorem, not an identity assumed by construction. No parameters are fitted to make the variance comparison hold, and the proof does not rely on the authors' prior work. The Gaussian OIU updates (5.4)–(5.5) follow from known Fisher-information identities for Gaussian families, cited to external references [37, 1, 13]. The paper explicitly acknowledges that the reparameterization-based estimator used in the RL experiments is not an exact Rao-Blackwellization: Section 4.7 states "This estimator is not Rao-Blackwellized since it contains irrelevant random variables ξ on the right hand side," and Section 5.3 states "This estimator is not an exact Rao-Blackwellization, since it contains ξ." The O(σξ) term dropped in Eq. (5.17) is therefore an acknowledged approximation whose accuracy is a regime/correctness concern, not a circular derivation. The self-citations [32, 33, 34] are used for biological analogy and interpretive remarks (e.g., "This update rule resembles ancestral reinforcement learning") and are not load-bearing premises for the variance bound. No uniqueness theorem is imported from the authors, and no fitted input is renamed as a prediction. The main theoretical result is self-contained and externally checkable; the identified limitation is a gap between the exact theorem and the approximate implementation, which is disclosed in the text.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The theory introduces no fitted constants and no new physical or mathematical entities; the genotype-phenotype terminology is an analogy, and PAES is an estimator rather than a postulated entity. The listed free parameters are experimental hyperparameters, while the axioms capture the regularity, observability, conjugacy, and small-noise assumptions that connect the clean theoretical statements to the implemented algorithm.

free parameters (6)
  • rank-based selection fraction p and max weight w_max = p=0.8, w_max=0.1
    Chosen to define the rank-based transformation W in BBOB experiments (Sec 6.1.1); affects the estimator but is not fitted to data.
  • learning rate eta = 0.01 (BBOB, FrozenLake); 0.001 Adam (CartPole, Pendulum, Swimmer)
    Hand-set for each experiment; convergence curves depend on it.
  • population size N = 100 in all experiments
    Hand-selected and fixed across comparisons.
  • input and phenotype noise sigma = 0.1 (BBOB); best sigma=0.01 for ES FrozenLake; 0.1 CartPole; 0.05 to 0.01 Pendulum; 0.2 to 0.05 Swimmer
    The noise scale is both the OIU problem parameter and the small-sigma parameter for the approximation in Eq. 5.17; chosen per benchmark.
  • Dirichlet concentration S = S=1 for PAES on FrozenLake
    Selected as the best hyperparameter by final reward, speed, and stability in Appendix E.1.
  • early stopping thresholds = 475 (CartPole), -200 (Pendulum)
    Stop criteria that cap generation counts; chosen by hand.
assumptions (6)
  • domain assumption Regularity conditions allowing interchange of expectation and differentiation in Eqs. (4.11)-(4.14) and (4.34).
    The paper differentiates under the integral without stating conditions on f, p_theta, or q_psi.
  • domain assumption The realized input x is observable for every evaluation.
    This is the defining premise of PAES; if x is hidden, the phenotype gradient estimator cannot be formed.
  • domain assumption For analytic estimates, p_theta and q_psi form an exponential-family and conjugate pair (Sec 4.5).
    The closed forms in Eqs. (4.23)-(4.29), (5.4)-(5.5), and (5.9) require conjugacy.
  • standard math Rao-Blackwell theorem and Fisher identity (Lemma 8).
    Used to prove Theorem 5 and Proposition 6; the Fisher identity is proved in Appendix C.1.
  • ad hoc to paper The transformation W is a fixed function in the theoretical statements.
    Theorems 3-5 treat W as fixed, but the experiments use rank-based W that depends on all samples (Sec 6.1.1); this mismatch is not discussed.
  • ad hoc to paper Small-sigma first-order approximation in Eq. (5.17).
    The RL estimator drops the O(sigma xi) residual; the paper assumes this residual is negligible for the noise scales used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating Evolutionary Strategy via Rao-Blackwellizing Realization of Uncertain Input." pith.science (2026). https://pith.science/paper/QOLYYVIP

@misc{pith2026260802073,
  author       = {Pith},
  title        = {Pith review of: Accelerating Evolutionary Strategy via Rao-Blackwellizing Realization of Uncertain Input},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QOLYYVIP}},
  note         = {Machine review of arXiv:2608.02073}
}
read the original abstract

We investigate Optimization under Input Uncertainty (OIU), in which the input to the objective function, rather than the objective function itself, is subject to uncertainty. OIU appears in manufacturing processes with production tolerance, control of physical systems with actuation noise, Mixture of Experts, and Reinforcement Learning (RL). Most of the existing approaches solve OIU by using the value of the objective function but discard the information of the realized input, even though the realized input is observable in various applications. The question here is whether the discarded information of the realized input is useful to accelerate the optimization process. We affirmatively answer this question for Evolutionary Strategy (ES) by theoretically showing that the information of the realized input can reduce the variance of the gradient estimator via Rao-Blackwellization. Using the Rao-Blackwellized gradient estimator, we propose Phenotype-Accelerated Evolutionary Strategy (PAES), which is a refinement of ES for OIU. Numerical experiments show that PAES converges faster than the usual ES from simple continuous optimization problems to RL benchmarks.

Figures

Figures reproduced from arXiv: 2608.02073 by the authors.

Figure 1.1
Figure 1.1. Schematic illustration of Optimization under Input Uncertainty (OIU) and its biological analogy. The [PITH_FULL_IMAGE:figures/full_fig_p002_1_1.png] view at source ↗
Figure 4.1
Figure 4.1. Schematic illustration of the application of ES to OIU and its refinement, phenotype-accelerated evolu [PITH_FULL_IMAGE:figures/full_fig_p007_4_1.png] view at source ↗
Figure 6.1
Figure 6.1. The difference of the final objective values to the optimum values on all 24 BBOB functions ( [PITH_FULL_IMAGE:figures/full_fig_p013_6_1.png] view at source ↗
Figures from the paper (5 more)
Figure 6
Figure 6. Figure 6: shows the mean difference between the final objective values and the optimum values for 10 runs of all [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 6.2
Figure 6.2. Figure 6.2: Selected convergence curves (mean ± std, 10 seeds). PAES (orange) vs. ES (blue). 6.2.1 Setup We adopt the FrozenLake-v1 environment [50]. The environment consists of 16 states and 4 actions (left/down/right/up) in a 4 × 4 grid world. For ES, we parameterize the polic…
Figure 6.3
Figure 6.3. Figure 6.3: (a) Learning curves of the updated policy for ES ( [PITH_FULL_IMAGE:figures/full_fig_p015_6_3.png]
Figure 6
Figure 6. Figure 6: shows the learning curves of ES and PAES for CartPole-v1. PAES converges significantly faster than ES in [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 6.4
Figure 6.4. Figure 6.4: Learning curves of ES and PAES for continuous control environments (mean [PITH_FULL_IMAGE:figures/full_fig_p016_6_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 6 canonical work pages

  1. [1]

    Bidirectional relation between cma evolution strategies and natural evolution strategies

    Youhei Akimoto, Yuichi Nagata, Isao Ono, and Shigenobu Kobayashi. Bidirectional relation between cma evolution strategies and natural evolution strategies. In Robert Schaefer, Carlos Cotta, Joanna Kołodziej, and 16 Rao-Blackwellized ES under Input UncertaintyA PREPRINT Günter Rudolph, editors,Parallel Problem Solving from Nature, PPSN XI, pages 154–163, B...

  2. [2]

    Springer, 2016

    Shun-ichi Amari.Information geometry and its applications, volume 194. Springer, 2016

    2016

  3. [3]

    Robust optimization for unconstrained simulation- based problems.Operations Research, 58(1):161–178, 2010

    Dimitris Bertsimas, Omid Nohadani, and Kwong Meng Teo. Robust optimization for unconstrained simulation- based problems.Operations Research, 58(1):161–178, 2010. ISSN 0030364X, 15265463. URLhttp://www. jstor.org/stable/40605967

    arXiv 2010

  4. [4]

    Actuator noise in recombinant evolution strategies on general quadratic fitness models

    Hans-Georg Beyer. Actuator noise in recombinant evolution strategies on general quadratic fitness models. In Kalyanmoy Deb, editor,Genetic and Evolutionary Computation – GECCO 2004, pages 654–665, Berlin, Heidelberg, 2004. Springer Berlin Heidelberg. ISBN 978-3-540-24854-5

    2004

  5. [5]

    Robust optimization – a comprehensive survey.Computer Methods in Applied Mechanics and Engineering, 196(33):3190–3218, 2007

    Hans-Georg Beyer and Bernhard Sendhoff. Robust optimization – a comprehensive survey.Computer Methods in Applied Mechanics and Engineering, 196(33):3190–3218, 2007. ISSN 0045-7825. doi: https: //doi.org/10.1016/j.cma.2007.03.003. URLhttps://www.sciencedirect.com/science/article/pii/ S0045782507001259

    work page doi:10.1016/j.cma.2007.03.003 2007

  6. [6]

    Conditional Expectation and Unbiased Sequential Estimation.The Annals of Mathematical Statistics, 18(1):105 – 110, 1947

    David Blackwell. Conditional Expectation and Unbiased Sequential Estimation.The Annals of Mathematical Statistics, 18(1):105 – 110, 1947. doi: 10.1214/aoms/1177730497. URLhttps://doi.org/10.1214/aoms/ 1177730497

    arXiv 1947

  7. [7]

    Springer, 2005

    Olivier Cappé, Eric Moulines, and Tobias Rydén.Inference in hidden Markov models. Springer, 2005

    2005

  8. [8]

    Wei Chen, J. K. Allen, Kwok-Leung Tsui, and F. Mistree. A procedure for robust design: Minimizing variations caused by noise factors and control factors.Journal of Mechanical Design, 118(4):478–485, 12 1996. ISSN 1050-0472. doi: 10.1115/1.2826915. URLhttps://doi.org/10.1115/1.2826915

    work page doi:10.1115/1.2826915 1996

Show all 62 references
  1. [9]

    Stanley, and Jeff Clune

    Edoardo Conti, Vashisht Madhavan, Felipe Petroski Such, Joel Lehman, Kenneth O. Stanley, and Jeff Clune. Improving exploration in evolution strategies for deep reinforcement learning via a population of novelty- seeking agents. InProceedings of the 32nd International Conferenc...

    2018

  2. [10]

    Duchi, Michael I

    John C. Duchi, Michael I. Jordan, Martin J. Wainwright, and Andre Wibisono. Optimal rates for zero-order convex optimization: The power of two function evaluations.IEEE Trans. Inf. Theor., 61(5):2788–2806, May

  3. [11]

    Flaxman, Adam Tauman Kalai, and H

    Abraham D. Flaxman, Adam Tauman Kalai, and H. Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. InProceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA ’05, page 385–394, USA, 2005. Society for...

    2005

  4. [12]

    Noisy-input entropy search for efficient robust bayesian optimization

    Lukas Fröhlich, Edgar Klenske, Julia Vinogradska, Christian Daniel, and Melanie Zeilinger. Noisy-input entropy search for efficient robust bayesian optimization. In Silvia Chiappa and Roberto Calandra, editors,Proceedings of the Twenty Third International Conference on Artific...

    2020

  5. [13]

    Exponential natural evolution strategies

    Tobias Glasmachers, Tom Schaul, Sun Yi, Daan Wierstra, and Jürgen Schmidhuber. Exponential natural evolution strategies. InProceedings of the 12th Annual Conference on Genetic and Evolutionary Computa- tion, GECCO ’10, page 393–400, New York, NY , USA, 2010. Association for Co...

    2010

  6. [14]

    Understanding the difficulty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Yee Whye Teh and Mike Titterington, editors,Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 ofProceedings o...

    2010

  7. [15]

    Variance reduction techniques for gradient estimates in reinforcement learning

    Evan Greensmith, Peter Bartlett, and Jonathan Baxter. Variance reduction techniques for gradient estimates in reinforcement learning. In T. Dietterich, S. Becker, and Z. Ghahramani, editors,Advances in Neural Information Processing Systems, volume 14. MIT Press, 2001. URLhttps...

    2001

  8. [16]

    Hansen, A

    N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, and D. Brockhoff. COCO: A platform for comparing continuous optimizers in a black-box setting.Optimization Methods and Software, 36:114–144, 2021. doi: https://doi.org/10.1080/10556788.2020.1808977. 17 Rao-Blackwellized ES un...

    2021

  9. [17]

    The CMA evolution strategy: A tutorial, 2016

    Nikolaus Hansen. The CMA evolution strategy: A tutorial, 2016. URLhttps://arxiv.org/abs/1604. 00772

    2016

  10. [18]

    Completely derandomized self-adaptation in evolution strategies

    Nikolaus Hansen and Andreas Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159–195, 06 2001. ISSN 1063-6560. doi: 10.1162/106365601750190398. URLhttps://doi.org/10.1162/106365601750190398

    2001 doi

  11. [19]

    Comparing results of 31 algorithms from the black-box optimization benchmarking bbob-2009

    Nikolaus Hansen, Anne Auger, Raymond Ros, Steffen Finck, and Petr Pošík. Comparing results of 31 algorithms from the black-box optimization benchmarking bbob-2009. InProceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO ’10, page 16...

    2009

  12. [20]

    A largest empty hypersphere metaheuristic for robust op- timisation with implementation uncertainty.Computers & Operations Research, 103:64–80, 2019

    Martin Hughes, Marc Goerigk, and Michael Wright. A largest empty hypersphere metaheuristic for robust op- timisation with implementation uncertainty.Computers & Operations Research, 103:64–80, 2019. ISSN 0305-

    2019

  13. [21]

    Particle swarm metaheuristics for robust optimisa- tion with implementation uncertainty.Computers & Operations Research, 122:104998, 2020

    Martin Hughes, Marc Goerigk, and Trivikram Dokka. Particle swarm metaheuristics for robust optimisa- tion with implementation uncertainty.Computers & Operations Research, 122:104998, 2020. ISSN 0305-

    2020

  14. [22]

    ISBN 9781450300735

    Association for Computing Machinery. ISBN 9781450300735. doi: 10.1145/1830761.1830790. URL https://doi.org/10.1145/1830761.1830790

  15. [23]

    Jordan and R.A

    M.I. Jordan and R.A. Jacobs. Hierarchical mixtures of experts and the em algorithm. InProceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan), volume 2, pages 1339–1344 vol.2,

    1993

  16. [24]

    Kakade, Shai Shalev-Shwartz, and Ambuj Tewari

    Sham M. Kakade, Shai Shalev-Shwartz, and Ambuj Tewari. Efficient bandit algorithms for online multiclass prediction. InProceedings of the 25th International Conference on Machine Learning, ICML ’08, page 440–447, New York, NY , USA, 2008. Association for Computing Machinery. I...

    2008

  17. [25]

    Kingma and Max Welling

    Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In2nd International Conference on Learning Representations (ICLR 2014), 2014. URLhttp://arxiv.org/abs/1312.6114

    2014 arXiv

  18. [26]

    URLhttps://www.sciencedirect.com/science/ article/pii/S0305054820301155

    doi: https://doi.org/10.1016/j.cor.2020.104998. URLhttps://www.sciencedirect.com/science/ article/pii/S0305054820301155

    2020

  19. [27]

    Yaochu Jin and J. Branke. Evolutionary optimization in uncertain environments-a survey.IEEE Transactions on Evolutionary Computation, 9(3):303–317, 2005. doi: 10.1109/TEVC.2005.846356

    2005

  20. [28]

    Springer, 1998

    Erich Leo Lehmann and George Casella.Theory of point estimation. Springer, 1998

    1998

  21. [29]

    Rao-blackwellized stochas- tic gradients for discrete distributions

    Runjing Liu, Jeffrey Regier, Nilesh Tripuraneni, Michael Jordan, and Jon Mcauliffe. Rao-blackwellized stochas- tic gradients for discrete distributions. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors,Proceedings of the 36th International Conference on Machine Learning...

    2019

  22. [30]

    Simple random search of static linear policies is com- petitive for reinforcement learning

    Horia Mania, Aurelia Guy, and Benjamin Recht. Simple random search of static linear policies is com- petitive for reinforcement learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett, editors,Advances in Neural Information Processing Sy...

    2018

  23. [31]

    Monte carlo gradient estimation in machine learning.Journal of Machine Learning Research, 21(132):1–62, 2020

    Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. Monte carlo gradient estimation in machine learning.Journal of Machine Learning Research, 21(132):1–62, 2020. URLhttp://jmlr.org/ papers/v21/19-346.html

    2020

  24. [32]

    Lam, Thang D Bui, George Deligiannidis, and Yee Whye Teh

    Kevin H. Lam, Thang D Bui, George Deligiannidis, and Yee Whye Teh. Rao-blackwellised reparameterisation gradients. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL https://openreview.net/forum?id=L26J2etiCX

    2026

  25. [33]

    Joel Lehman, Jay Chen, Jeff Clune, and Kenneth O. Stanley. Es is more than just a traditional finite-difference approximator. InProceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18, page 450–457, New York, NY , USA, 2018. Association for Computing Mach...

    2018

  26. [34]

    Kobayashi

    So Nakashima and Tetsuya J. Kobayashi. Unifying zeroth-order optimization and genetic algorithms for re- inforcement learning. InProceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’25 Companion, page 311–314, New York, NY , USA, 2025. Associati...

    2025

  27. [35]

    Random gradient-free minimization of convex functions.Foundations of Computational Mathematics, 17(2):527–566, Apr 2017

    Yurii Nesterov and Vladimir Spokoiny. Random gradient-free minimization of convex functions.Foundations of Computational Mathematics, 17(2):527–566, Apr 2017. ISSN 1615-3383. doi: 10.1007/s10208-015-9296-2. URLhttps://doi.org/10.1007/s10208-015-9296-2

    2017 doi

  28. [36]

    Bayesian optimisation under uncertain inputs

    Rafael Oliveira, Lionel Ott, and Fabio Ramos. Bayesian optimisation under uncertain inputs. In Kamalika Chaudhuri and Masashi Sugiyama, editors,Proceedings of the Twenty-Second International Conference on Ar- tificial Intelligence and Statistics, volume 89 ofProceedings of Mac...

    2019

  29. [37]

    Information-geometric optimization algo- rithms: A unifying picture via invariance principles.Journal of Machine Learning Research, 18(18):1–65, 2017

    Yann Ollivier, Ludovic Arnold, Anne Auger, and Nikolaus Hansen. Information-geometric optimization algo- rithms: A unifying picture via invariance principles.Journal of Machine Learning Research, 18(18):1–65, 2017. URLhttp://jmlr.org/papers/v18/14-467.html

    2017

  30. [38]

    Kobayashi

    So Nakashima and Tetsuya J. Kobayashi. Acceleration of evolutionary processes by learning and extended fisher’s fundamental theorem.Phys. Rev. Res., 4:013069, Jan 2022. doi: 10.1103/PhysRevResearch.4.013069. URLhttps://link.aps.org/doi/10.1103/PhysRevResearch.4.013069. 18 Rao-...

    2022 doi

  31. [39]

    Kobayashi

    So Nakashima and Tetsuya J. Kobayashi. Ancestral reinforcement learning: Unifying zeroth-order optimization and genetic algorithms for reinforcement learning, 2024. URLhttps://arxiv.org/abs/2408.09493

    2024 arXiv

  32. [40]

    Information and the accuracy attainable in the estimation of statistical parameters

    C Radhakrishna Rao et al. Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc, 37(3):81–91, 1945

    1945

  33. [41]

    Eigen.Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution

    Ingo Rechenberg and M. Eigen.Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Stuttgart : Frommann-Holzboog, 1973. ISBN 3772803733. URLhttp://lib.ugent. be/catalog/rug01:001280204

    1973

  34. [42]

    Stochastic backpropagation and approximate in- ference in deep generative models

    Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate in- ference in deep generative models. In Eric P. Xing and Tony Jebara, editors,Proceedings of the 31st International Conference on Machine Learning, volume 32 ofProceedings o...

    2014

  35. [43]

    Sahinidis

    Luis Miguel Rios and Nikolaos V . Sahinidis. Derivative-free optimization: a review of algorithms and compari- son of software implementations.Journal of Global Optimization, 56(3):1247–1293, Jul 2013. ISSN 1573-2916. doi: 10.1007/s10898-012-9951-y. URLhttps://doi.org/10.1007/...

    2013 doi

  36. [44]

    Performative prediction

    Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. Performative prediction. In Hal Daumé III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning, volume 119 ofProceedings of Machine Learning Research, pages 7599–76...

    2020

  37. [45]

    Black Box Variational Inference

    Rajesh Ranganath, Sean Gerrish, and David Blei. Black Box Variational Inference. In Samuel Kaski and Jukka Corander, editors,Proceedings of the Seventeenth International Conference on Artificial Intelligence and Statistics, volume 33 ofProceedings of Machine Learning Research,...

    2014

  38. [46]

    Gradient estimation using stochastic com- putation graphs

    John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient estimation using stochastic com- putation graphs. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 28. Curran Associates,...

    2015

  39. [47]

    An optimal algorithm for bandit and zero-order convex optimization with two-point feedback

    Ohad Shamir. An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. Journal of Machine Learning Research, 18(52):1–11, 2017. URLhttp://jmlr.org/papers/v18/16-632. html

    2017

  40. [48]

    MIT press Cambridge, 1998

    Richard S Sutton, Andrew G Barto, et al.Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998

    1998

  41. [49]

    Taguchi and M

    G. Taguchi and M. S. Phadke.Quality Engineering through Design Optimization, pages 77–96. Springer US, Boston, MA, 1989. ISBN 978-1-4684-1472-1. doi: 10.1007/978-1-4684-1472-1_5. URLhttps://doi.org/ 10.1007/978-1-4684-1472-1_5. 19 Rao-Blackwellized ES under Input UncertaintyA PREPRINT

    1989 doi

  42. [50]

    Tim Salimans and David A. Knowles. Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression.Bayesian Analysis, 8(4):837 – 882, 2013. doi: 10.1214/13-BA858. URLhttps://doi. org/10.1214/13-BA858

    2013 doi

  43. [51]

    Evolution strategies as a scalable alternative to reinforcement learning.arXiv preprint arXiv:1703.03864, 2017

    Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. Evolution strategies as a scalable alternative to reinforcement learning.arXiv preprint arXiv:1703.03864, 2017

    2017 arXiv

  44. [52]

    Natural evolution strategies.Journal of Machine Learning Research, 15(27):949–980, 2014

    Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters, and Jürgen Schmidhuber. Natural evolution strategies.Journal of Machine Learning Research, 15(27):949–980, 2014. URLhttp://jmlr.org/papers/ v15/wierstra14a.html

    2014

  45. [53]

    Variance reduction for policy gradient with action-dependent factorized baselines

    Cathy Wu, Aravind Rajeswaran, Yan Duan, Vikash Kumar, Alexandre M Bayen, Sham Kakade, Igor Mor- datch, and Pieter Abbeel. Variance reduction for policy gradient with action-dependent factorized baselines. In International Conference on Learning Representations, 2018. URLhttps:...

    2018

  46. [54]

    Ming Xu, Matias Quiroz, Robert Kohn, and Scott A. Sisson. Variance reduction properties of the reparam- eterization trick. In Kamalika Chaudhuri and Masashi Sugiyama, editors,Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, v...

    2019

  47. [55]

    Can a one-point feedback zeroth-order algorithm achieve linear dimension dependent sample complexity?, 2025

    Haishan Ye and Xiangyu Chang. Can a one-point feedback zeroth-order algorithm achieve linear dimension dependent sample complexity?, 2025. URLhttps://arxiv.org/abs/2508.12228

    2025 arXiv

  48. [56]

    Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032, 2024

    Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, et al. Gymnasium: A standard interface for reinforcement learning environments.arXiv preprint arXiv:2407.17032, 2024

    2024 arXiv

  49. [57]

    S. Tsutsui. A comparative study on the effects of adding perturbations to phenotypic parameters in genetic algorithms with a robust solution searching scheme. InIEEE SMC’99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99C...

    1999

  50. [62]

    ∂ ∂ψ g FW # =E PF(θ|x)

    Mingzhang Yin, Yuguang Yue, and Mingyuan Zhou. ARSM: Augment-REINFORCE-swap-merge estimator for gradient backpropagation through categorical variables. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors,Proceedings of the 36th International Conference on Machine Learning,...

    2019

  51. [548]

    URLhttps://www.sciencedirect.com/science/ article/pii/S0305054818302727

    doi: https://doi.org/10.1016/j.cor.2018.10.013. URLhttps://www.sciencedirect.com/science/ article/pii/S0305054818302727

    2018 doi

  52. [1993]

    doi: 10.1109/IJCNN.1993.716791

    1993

  53. [2010]

    ISBN 978-3-642-15844-5

    Springer Berlin Heidelberg. ISBN 978-3-642-15844-5

  54. [2015]

    doi: 10.1109/TIT.2015.2409256

    ISSN 0018-9448. doi: 10.1109/TIT.2015.2409256. URLhttps://doi.org/10.1109/TIT.2015. 2409256

    2015

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.