pith. sign in

arxiv: 1604.00772 · v2 · pith:4DB4CNBWnew · submitted 2016-04-04 · 💻 cs.LG · stat.ML

The CMA Evolution Strategy: A Tutorial

Nikolaus Hansen (TAO) This is my paper

Pith reviewed 2026-05-15 21:06 UTC · model grok-4.3

classification 💻 cs.LG stat.ML
keywords evolution strategycovariance matrix adaptationcontinuous optimizationstochastic searchderivative-free optimizationnon-convex functionsreal-parameter optimization
0
0 comments X

The pith

The CMA-ES derives its search distribution from intuitive requirements for non-linear non-convex optimization in continuous space.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This tutorial presents the Covariance Matrix Adaptation Evolution Strategy as a stochastic method for optimizing real-valued, non-linear, non-convex functions. It builds the algorithm step by step from basic ideas about how search should proceed when gradients are unavailable and the landscape contains ridges or valleys of different widths. The central step is updating both the mean and the covariance of a multivariate normal sampling distribution using only the ranking of the best candidate solutions. The resulting adaptation makes the search scale-invariant and rotation-invariant without requiring hand-tuned parameters beyond the problem dimension.

Core claim

The CMA-ES is a stochastic method for real-parameter optimization of non-linear, non-convex functions, motivated and derived from intuitive concepts and requirements of non-linear non-convex search in continuous domain.

What carries the argument

Covariance matrix adaptation, which maintains and updates the parameters of a multivariate normal distribution so that its shape reflects the local geometry of the objective function learned from ranked samples.

If this is right

  • The method becomes invariant under linear transformations of the coordinate system.
  • It requires only function evaluations and no derivative information.
  • A small number of strategy parameters can be set automatically from the dimension.
  • Parallel evaluation of the population is possible without changing the update logic.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same ranking-based update logic could be ported to other sampling-based optimizers that lack explicit covariance learning.
  • The approach naturally extends to noisy or multi-objective settings by replacing the ranking with an appropriate selection criterion.
  • In very high dimensions the cubic cost of covariance updates may need replacement by low-rank or diagonal approximations while preserving the core adaptation idea.

Load-bearing premise

The derivation assumes that ranking the best samples from a normal distribution is sufficient to extract the local curvature information needed for effective search.

What would settle it

A benchmark suite where CMA-ES with covariance adaptation shows no faster convergence than a fixed isotropic normal distribution on problems with known elongated valleys would disprove the utility of the adaptation rule.

read the original abstract

This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized, method for real-parameter (continuous domain) optimization of non-linear, non-convex functions. We try to motivate and derive the algorithm from intuitive concepts and from requirements of non-linear, non-convex search in continuous domain.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 1 minor

Summary. This tutorial introduces the CMA Evolution Strategy (CMA-ES), a stochastic method for real-parameter optimization of non-linear, non-convex functions in continuous domains. It motivates and derives the algorithm from intuitive concepts and requirements for effective search in such spaces, presenting the standard covariance matrix adaptation rules without new empirical claims or theorems.

Significance. The tutorial provides a clear, first-principles derivation of an established optimization method, strengthening accessibility for readers with background in probability and linear algebra. This explanatory approach aids understanding of why CMA-ES succeeds on non-convex problems and supports its use in machine learning applications.

minor comments (1)
  1. [Abstract] Abstract: The statement of purpose is clear, but adding a brief note on the tutorial's section structure (e.g., derivation steps followed by pseudocode) would improve reader orientation.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive review and the recommendation to accept the manuscript. The assessment accurately captures the tutorial's purpose as a first-principles derivation of the established CMA-ES algorithm without new empirical claims.

Circularity Check

0 steps flagged

No significant circularity; tutorial derives from stated intuitive requirements

full rationale

The paper is a tutorial that explicitly frames the CMA-ES derivation as arising from intuitive concepts and requirements of non-linear non-convex continuous search rather than from fitted parameters, self-referential definitions, or load-bearing self-citations. No equations reduce by construction to their inputs, no predictions are statistically forced from subsets of data, and no uniqueness theorems are imported from overlapping prior work to forbid alternatives. The central exposition remains self-contained against external benchmarks of the algorithm's established behavior.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

As an expository tutorial on a pre-existing algorithm, the paper introduces no new free parameters, axioms, or invented entities beyond standard descriptions of the CMA-ES.

pith-pipeline@v0.9.0 · 5335 in / 884 out tokens · 20246 ms · 2026-05-15T21:06:33.599420+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • Cost.FunctionalEquation washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    The CMA-ES is a stochastic method for real-parameter optimization of non-linear, non-convex functions, motivated and derived from intuitive concepts and requirements of non-linear non-convex search in continuous domain.

  • Foundation.DimensionForcing dimension_forced unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    Covariance matrix adaptation... rank-μ-update... rank-one-update... cumulation... step-size control

  • Foundation.LedgerCanonicality uniform_scaling_forced unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    We try to motivate and derive the algorithm from intuitive concepts and from requirements of non-linear, non-convex search in continuous domain.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Cited by 46 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Certified Gradient-Based Contact-Rich Manipulation via Smoothing-Error Reachable Tubes

    cs.RO 2026-02 unverdicted novelty 8.0

    A certified gradient-based method for contact-rich manipulation that quantifies smoothing-induced errors via set-valued discrepancies and incorporates them into analytical reachable sets for robust affine feedback policies.

  2. Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

    cs.CL 2023-09 unverdicted novelty 8.0

    Promptbreeder evolves both task prompts and the mutation prompts that improve them using LLMs, outperforming Chain-of-Thought and Plan-and-Solve on arithmetic and commonsense reasoning benchmarks.

  3. Latent Heuristic Search: Continuous Optimization for Automated Algorithm Design

    cs.AI 2026-05 unverdicted novelty 7.0

    Latent Heuristic Search performs continuous optimization over learned embeddings of heuristics, using normalizing flows and LLM prompting to discover competitive solvers for TSP, CVRP, KSP, and OBP.

  4. Low Stage High Order Explicit Runge--Kutta Methods via Q- and D-Conditions: General Theory and Efficient Recursive Construction

    math.NA 2026-05 conditional novelty 7.0

    A Q/D-space reformulation of Butcher simplifying assumptions yields sufficient order conditions and a recursive linear-system construction for explicit Runge-Kutta methods of even order p with s(p)=(p²-2p+8)/4 stages.

  5. EVA-0: Test-Time Model Evolution with Only Two Forward Passes per Sample

    cs.LG 2026-05 unverdicted novelty 7.0

    EVA-0 is a zeroth-order test-time adaptation method that uses scale-invariant loss, anchor-guided optimization, and symmetric two-sided perturbations to enable inference and adaptation in two forward passes, outperfor...

  6. Parameter-Efficient Neuroevolution for Diverse LLM Generation: Quality-Diversity Optimization via Prompt Embedding Evolution

    cs.NE 2026-05 unverdicted novelty 7.0

    QD-LLM evolves prompt embeddings via neuroevolution in a quality-diversity framework, delivering 46% higher coverage and 41% higher QD-score than prior methods on coding and writing benchmarks.

  7. Evolutionary Negative Module Pruning for Better LoRA Merging

    cs.AI 2026-04 conditional novelty 7.0

    ENMP prunes negative LoRA modules via evolutionary search to boost merging performance to new state-of-the-art levels across language and vision tasks.

  8. Simulator Adaptation for Sim-to-Real Learning of Legged Locomotion via Proprioceptive Distribution Matching

    cs.RO 2026-04 unverdicted novelty 7.0

    Proprioceptive distribution matching adapts simulators for legged robot policies by comparing observation and action distributions, reducing sim-to-real gaps with minimal real data and no external sensing.

  9. GeoPAS: Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization

    cs.LG 2026-04 unverdicted novelty 7.0

    GeoPAS represents optimization problems via multi-scale 2D geometric slices fed to a validity-aware CNN that aggregates embeddings for risk-aware solver selection and log-scale performance prediction, outperforming th...

  10. Bootstrapping non-unitary CFTs

    hep-th 2025-12 unverdicted novelty 7.0

    A bootstrap strategy for non-unitary CFTs uses statistical stability of OPE data across cross-ratios to optimize spectra, reproducing A-series minimal models and yielding candidate solutions for c>1.

  11. Exploring Exploration in Bayesian Optimization

    cs.LG 2025-02 unverdicted novelty 7.0

    Introduces observation traveling salesman distance and observation entropy to quantify exploration in Bayesian optimization acquisition functions and links them to empirical performance.

  12. KSOS-BO: Improving Sampling in Bayesian Optimization via Kernel Sum of Squares

    cs.CE 2026-05 unverdicted novelty 6.0

    KSOS-BO improves acquisition function optimization in Bayesian optimization by casting it as a kernel sum of squares semidefinite program, outperforming Sobol, DE, and CMA-ES baselines on 10/15 benchmarks with 81% ave...

  13. Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing

    cs.LG 2026-05 unverdicted novelty 6.0

    Recasts sampling-based nonconvex optimization as smoothed gradient descent to obtain non-asymptotic convergence guarantees and introduces the DIDA annealed algorithm that converges to the global optimum.

  14. Solving Classical and Quantum Spin Glasses with Deep Boltzmann Quantum States

    cond-mat.dis-nn 2026-05 unverdicted novelty 6.0

    Deep Boltzmann Quantum States with natural-gradient optimization and annealing-like training match exact or best-known solutions for large infinite-range Ising spin glasses and solve job shop scheduling instances.

  15. Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection

    cs.CV 2026-05 unverdicted novelty 6.0

    RGSE adapts text embeddings at test time via evolutionary search, using cosine similarity rewards from high-confidence visual proposals to improve open-vocabulary object detection under distribution shifts.

  16. Global Sampling-Based Trajectory Optimization for Contact-Rich Manipulation via KernelSOS

    cs.RO 2026-04 unverdicted novelty 6.0

    Global-MPPI integrates kernel SOS global search with MPPI local refinement and graduated non-convexity smoothing to achieve faster convergence and lower costs on high-dimensional contact-rich manipulation tasks.

  17. Benchmarking Stopping Criteria for Evolutionary Multi-objective Optimization

    cs.NE 2026-04 unverdicted novelty 6.0

    Introduces a single-number performance measure, file-based benchmarking, and efficient text-file storage to evaluate and compare stopping criteria for EMO algorithms.

  18. A Complex-Valued Continuous-Variable Quantum Approximation Optimization Algorithm (CCV-QAOA)

    quant-ph 2026-04 unverdicted novelty 6.0

    CCV-QAOA is a new complex-valued continuous-variable variant of QAOA that solves real and complex multivariate optimization problems via a variational framework.

  19. Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation

    cs.CV 2026-04 unverdicted novelty 6.0

    A flow-matching model derives manipulation strategies from object affordance, adds an adversarial interaction prior, and uses stability simulation to generate natural, effective human-human co-manipulation motions.

  20. Similarity-based Portfolio Construction for Black-box Optimization

    cs.NE 2026-04 unverdicted novelty 6.0

    A k-nearest-neighbor approach constructs problem-specific algorithm portfolios that outperform both single solvers and the virtual best solver in fixed-budget black-box optimization.

  21. On the Generalization Bounds of Symbolic Regression with Genetic Programming

    cs.LG 2026-04 unverdicted novelty 6.0

    Derives a generalization bound for GP-based symbolic regression that decomposes the gap into structure-selection complexity and constant-fitting complexity under tree constraints.

  22. Optimal Majoranas in Mesoscopic Kitaev Chains

    cond-mat.mes-hall 2026-04 unverdicted novelty 6.0

    Microscopic treatment of the hybrid segment in mesoscopic Kitaev chains shows that Andreev bound state parity crossings define optimal sweet spots for localized Majoranas with large gaps.

  23. Trajectory-based actuator identification via differentiable simulation

    cs.RO 2026-04 unverdicted novelty 6.0

    Differentiable simulation enables torque-sensor-free actuator model identification from trajectory data, achieving 1.88x better position tracking than a stand-trained baseline and 46% longer travel in downstream locom...

  24. GeoPAS: Geometric Probing for Algorithm Selection in Continuous Black-Box Optimization

    cs.LG 2026-04 unverdicted novelty 6.0

    GeoPAS uses multi-scale 2D geometric slices of optimization landscapes with validity-mask pooling and a learned-plus-prior composite score to select from 12 solvers, cutting mean relative expected running time from 30...

  25. Sumo: Dynamic and Generalizable Whole-Body Loco-Manipulation

    cs.RO 2026-04 unverdicted novelty 6.0

    Test-time steering of pre-trained whole-body policies via sample-based planning lets legged robots generalize dynamic loco-manipulation to varied heavy objects and tasks without additional training or tuning.

  26. PhDLspec: physical-prior embedded deep learning method for spectroscopic determination of stellar labels in high-dimensional parameter space

    astro-ph.GA 2026-04 unverdicted novelty 6.0

    PhDLspec combines differential spectra from physical stellar models with a transformer to derive approximately 30 stellar parameters from low-resolution spectra hundreds of times faster than traditional calculations.

  27. Depth Augmented and FE Free 3D/2D Liver Registration for Laparoscopic Liver AR

    cs.CV 2026-02 unverdicted novelty 6.0

    A depth-augmented rigid pose refinement combined with a patient-specific statistical deformation model from NICP correspondences achieves 14.73 mm mean TRE for 3D-2D liver registration in controlled laparoscopic settings.

  28. Evolution With Purpose: Hierarchy-Informed Optimization of Whole-Brain Models

    cs.NE 2026-02 unverdicted novelty 6.0

    Hierarchy-informed curricular optimization of heterogeneous whole-brain models enables generalization to new subjects and prediction of behavioral abilities from parameters.

  29. Sample-Efficient Optimisation over the Outputs of Generative Models

    stat.ML 2025-09 unverdicted novelty 6.0

    O3 uses surrogate latent spaces extracted from generative models to perform sample-efficient black-box optimization over their outputs, outperforming direct sampling and original-latent optimization on image and prote...

  30. Learning Evolution via Optimization Knowledge Adaptation

    cs.NE 2025-01 unverdicted novelty 6.0

    OKAEM is a unified learnable evolutionary framework that uses attention-based operators for pre-training on prior knowledge and real-time self-tuning adaptation.

  31. Black-Box Optimization of Mixed Binary-Continuous Variables: Challenges and Opportunities in Evolutionary Model Merging

    cs.NE 2026-05 unverdicted novelty 5.0

    Data flow space model merging is formalized as a mixed binary-continuous black-box optimization problem, where a structured approach respecting variable dependencies achieves 6.7% higher accuracy and 51.4% smaller sea...

  32. Distributed Quantum-Enhanced Optimization: A Topographical Preconditioning Approach for High-Dimensional Search

    quant-ph 2026-04 unverdicted novelty 5.0

    D-QEO framework uses quantum topographical preconditioning on separable functions via small parallel subcircuits to generate seeds that accelerate classical global optimization and avoid exponential failure rates.

  33. Rapid LoRA Aggregation for Wireless Channel Adaptation in Open-Set Radio Frequency Fingerprinting

    eess.SP 2026-04 unverdicted novelty 5.0

    LoRA pretraining per environment plus weighted aggregation at inference cuts EER by 15% and training time by 83% for open-set RFF authentication under varying channels.

  34. What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?

    cs.AI 2025-12 unverdicted novelty 5.0

    An empirical study of JEPA world models identifies architecture, training objective, and planning choices that yield a model outperforming DINO-WM and V-JEPA-2-AC on navigation and manipulation tasks.

  35. Emergence of Internal State-Modulated Swarming in Multi-Agent Patch Foraging System

    nlin.AO 2025-10 unverdicted novelty 5.0

    In a simulated multi-agent foraging environment, evolved neural controllers lead to swarming that is modulated by the agents' internal resource levels, with hidden states encoding resource information.

  36. Machine Learning in the 2HDM2S model for Dark Matter

    hep-ph 2025-09 unverdicted novelty 5.0

    A 2HDM extended by two real scalar singlets is scanned with evolutionary strategies to locate regions satisfying vacuum, unitarity, oblique-parameter, collider and dark-matter constraints.

  37. Sampling-Based Global Optimal Control and Estimation via Semidefinite Programming

    cs.RO 2025-07 unverdicted novelty 5.0

    KernelSOS is shown to be competitive on robot localization and to improve solution quality in high-dimensional trajectory optimization when paired with local solvers.

  38. Diffusion Models are Evolutionary Algorithms

    cs.NE 2024-10 unverdicted novelty 5.0

    Diffusion models are evolutionary algorithms via a denoising-evolution equivalence, yielding Diffusion Evolution that outperforms mainstream EAs on multi-optima tasks.

  39. CC-VPSTO: Chance-Constrained Via-Point-Based Stochastic Trajectory Optimisation for Online Robot Motion Planning under Uncertainty

    cs.RO 2024-02 unverdicted novelty 5.0

    CC-VPSTO formulates stochastic trajectory optimization as a chance-constrained problem, approximates it with Monte Carlo sampling and padding, and integrates it into MPC for online robot motion planning under uncertainty.

  40. Local Online Motor Babbling: Learning Motor Abundance of A Musculoskeletal Robot Arm

    cs.RO 2019-06 unverdicted novelty 5.0

    A method combining goal babbling with CMA-ES-based local online motor babbling is used to learn inverse kinematics and explore motor abundance on a 10-DoF musculoskeletal robot arm.

  41. Artificial Adaptive Intelligence: The Missing Stage Between Narrow and General Intelligence

    cs.AI 2026-05 unverdicted novelty 4.0

    Proposes Artificial Adaptive Intelligence as the regime between narrow and general AI, defined by elimination of human-specified hyperparameters, and introduces an adaptivity index plus parametric minimality principle...

  42. Nested Control Co-design of a Spar Buoy Horizontal-axis Floating Offshore Wind Turbine

    eess.SY 2023-10 unverdicted novelty 4.0

    Nested CCD optimization on a reduced-order spar-buoy FOWT model yields over 11% AEP improvement versus baseline.

  43. Convolutional Reservoir Computing for World Models

    cs.LG 2019-07 unverdicted novelty 4.0

    RCRC uses untrained random CNNs and reservoir computing plus evolution strategies to reach claimed state-of-the-art scores in reinforcement learning tasks while avoiding data storage and heavy training.

  44. Acoustic Model Optimization Based On Evolutionary Stochastic Gradient Descent with Anchors for Automatic Speech Recognition

    cs.CL 2019-07 unverdicted novelty 4.0

    ESGD with anchors guarantees no degradation from the anchor model and reports improved loss and ASR performance on BN50 and SWB300 datasets.

  45. Benchmarking Optimization Algorithms for Automated Calibration of Quantum Devices

    quant-ph 2025-09 unverdicted novelty 3.0

    Simulations show CMA-ES outperforms Nelder-Mead and other algorithms for quantum device calibration across low- and high-dimensional regimes.

  46. Evolution Attack On Neural Networks

    cs.CV 2019-06 unverdicted novelty 2.0

    Covariance matrix adaptive evolution strategy generates effective black-box adversarial examples for neural networks, outperforming other evolution methods tested.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · cited by 45 Pith papers · 1 internal anchor

  1. [1]

    Quality gain analysis of the weighted recombination evolution strategy on general convex quadratic functions

    Akimoto Y , Auger A, Hansen N. Quality gain analysis of the weighted recombination evolution strategy on general convex quadratic functions. In Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms (FOGA ’17) , pages 111–126. ACM, New York, NY , USA, 2017

    work page 2017

  2. [2]

    Diagonal acceleration for covariance matrix adaptation evolution strategies

    Akimoto Y , Hansen N. Diagonal acceleration for covariance matrix adaptation evolution strategies. Evolutionary computation, 28(3):405–435, 2020

    work page 2020

  3. [3]

    A restart CMA evolution strategy with increasing population size

    Auger A, Hansen N. A restart CMA evolution strategy with increasing population size. In Proceedings of the IEEE Congress on Evolutionary Computation, 2005

    work page 2005

  4. [4]

    Weighted multirecombination evolution strategies

    Arnold DV . Weighted multirecombination evolution strategies. Theoretical computer science, 361.1:18–37, 2006

    work page 2006

  5. [5]

    Performance analysis of evolutionary optimization with cumu- lative step length adaptation

    Arnold DV , Beyer HG. Performance analysis of evolutionary optimization with cumu- lative step length adaptation. IEEE Transactions on Automatic Control, 49(4):617–622, 2004

    work page 2004

  6. [6]

    The Theory of Evolution Strategies

    Beyer HG. The Theory of Evolution Strategies. Springer, Berlin, 2001

    work page 2001

  7. [7]

    Qualms regarding the optimality of cumulative path length con- trol in CSA/CMA-evolution strategies

    Beyer HG, Arnold DV . Qualms regarding the optimality of cumulative path length con- trol in CSA/CMA-evolution strategies. Evolutionary Computation, 11(1):19–28, 2003

    work page 2003

  8. [8]

    On self-adaptive features in real-parameter evolutionary algorithms

    Beyer HG, Deb K. On self-adaptive features in real-parameter evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 5(3):250–270, 2001

    work page 2001

  9. [9]

    Multidisciplinary op- timisation in the design of future space launchers

    Collange G, Delattre N, Hansen N, Quinquis I, Schoenauer M. Multidisciplinary op- timisation in the design of future space launchers. In Breitkopf and Coelho, editors, Multidisciplinary Design Optimization in Computational Mechanics , chapter 12, pages 487–496. Wiley, 2010

    work page 2010

  10. [10]

    Augmented Lagrangian, penalty techniques and surrogate mod- eling for constrained optimization with CMA-ES

    Dufoss ´e P, Hansen N. Augmented Lagrangian, penalty techniques and surrogate mod- eling for constrained optimization with CMA-ES. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’21), pages 519–527, 2021

    work page 2021

  11. [11]

    Learning rate adaptation by line search in evolution strategies with recombination

    Gissler A, Auger A, Hansen N. Learning rate adaptation by line search in evolution strategies with recombination. In Proceedings of the Genetic and Evolutionary Compu- tation Conference (GECCO ’22), pages 630–638. ACM, New York, NY , USA, 2022

    work page 2022

  12. [12]

    Exponential natural evolu- tion strategies

    Glasmachers T, Schaul T, Yi S, Wierstra D, Schmidhuber J. Exponential natural evolu- tion strategies. InProceedings of the 12th annual Genetic and Evolutionary Computation Conference, GECCO, pages 393–400. ACM, 2010. 25

    work page 2010

  13. [13]

    Verallgemeinerte individuelle Schrittweitenregelung in der Evolutionsstrate- gie

    Hansen N. Verallgemeinerte individuelle Schrittweitenregelung in der Evolutionsstrate- gie. Mensch und Buch Verlag, Berlin, 1998

    work page 1998

  14. [14]

    Invariance, self-adaptation and correlated mutations in evolution strategies

    Hansen N. Invariance, self-adaptation and correlated mutations in evolution strategies. In Schoenauer M, Deb K, Rudolph G, Yao X, Lutton E, Merelo JJ, Schwefel HP, editors, Parallel Problem Solving from Nature - PPSN VI, pages 355–364. Springer, 2000

    work page 2000

  15. [15]

    The CMA evolution strategy: a comparing review

    Hansen N. The CMA evolution strategy: a comparing review. In Lozano JA, Larranaga P, Inza I, and Bengoetxea E, editors, Towards a new evolutionary computation. Advances on estimation of distribution algorithms, pages 75–102. Springer, 2006

    work page 2006

  16. [16]

    Benchmarking a BI-Population CMA-ES on the BBOB-2009 Function Testbed

    Hansen N. Benchmarking a BI-Population CMA-ES on the BBOB-2009 Function Testbed. In the workshop Proceedings of the Genetic and Evolutionary Computation Conference, GECCO, pages 2389–2395. ACM, 2009

    work page 2009

  17. [17]

    Variable Metrics in Evolutionary Computation

    Hansen N. Variable Metrics in Evolutionary Computation. Habilitation `a diriger des recherches, Universit´e Paris-Sud, 2010

    work page 2010

  18. [18]

    Injecting External Solutions Into CMA-ES

    Hansen N. Injecting External Solutions Into CMA-ES. CoRR, arXiv:1110.4181, 2011

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  19. [19]

    Principled design of continuous stochastic search: From theory to practice

    Hansen N, Auger A. Principled design of continuous stochastic search: From theory to practice. In Y Borenstein and A Moraglio, eds.: Theory and Principled Methods for Designing Metaheustics. Springer, pages 145–180, 2014

    work page 2014

  20. [20]

    How to Assess Step-Size Adaptation Mechanisms in Randomised Search

    Hansen N, Atamna A, Auger A. How to Assess Step-Size Adaptation Mechanisms in Randomised Search. In Parallel Problem Solving from Nature – PPSN XIII, pages 60–69. Springer, 2014

    work page 2014

  21. [21]

    Evaluating the CMA evolution strategy on multimodal test functions

    Hansen N, Kern S. Evaluating the CMA evolution strategy on multimodal test functions. In Xin Yao et al., editors, Parallel Problem Solving from Nature – PPSN VIII , pages 282–291. Springer, 2004

    work page 2004

  22. [22]

    A method for handling uncertainty in evolutionary optimization with an application to feedback control of com- bustion

    Hansen N, Niederberger SPN, Guzzella L, Koumoutsakos P. A method for handling uncertainty in evolutionary optimization with an application to feedback control of com- bustion. IEEE Transactions on Evolutionary Computation, 13(1):180–197, 2009

    work page 2009

  23. [23]

    Reducing the time complexity of the deran- domized evolution strategy with covariance matrix adaptation (CMA-ES)

    Hansen N, M ¨uller SD, Koumoutsakos P. Reducing the time complexity of the deran- domized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation, 11(1):1–18, 2003

    work page 2003

  24. [24]

    Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation

    Hansen N, Ostermeier A. Adapting arbitrary normal mutation distributions in evolution strategies: The covariance matrix adaptation. In Proceedings of the 1996 IEEE Confer- ence on Evolutionary Computation (ICEC ’96), pages 312–317, 1996

    work page 1996

  25. [25]

    Convergence properties of evolution strategies with the deran- domized covariance matrix adaptation: The (µ/µI,λ )-CMA-ES

    Hansen N, Ostermeier A. Convergence properties of evolution strategies with the deran- domized covariance matrix adaptation: The (µ/µI,λ )-CMA-ES. In Proceedings of the 5th European Congress on Intelligent Techniques and Soft Computing, pages 650–654, 1997

    work page 1997

  26. [26]

    Completely derandomized self-adaptation in evolution strate- gies

    Hansen N, Ostermeier A. Completely derandomized self-adaptation in evolution strate- gies. Evolutionary Computation, 9(2):159–195, 2001. 26

    work page 2001

  27. [27]

    Benchmarking a weighted negative covariance matrix update on the BBOB-2010 noiseless testbed

    Hansen N, Ros R. Benchmarking a weighted negative covariance matrix update on the BBOB-2010 noiseless testbed. In Proceedings companion of the 12th annual Genetic and Evolutionary Computation Conference, GECCO, pages 1673–1680. ACM, 2010

    work page 2010

  28. [28]

    Improving evolution strategies through active covariance ma- trix adaptation

    Jastrebski G, Arnold DV . Improving evolution strategies through active covariance ma- trix adaptation. In Proceedings of the 2006 IEEE Congress on Evolutionary Computa- tion, CEC, pages 2814–2821. IEEE, 2006

    work page 2006

  29. [29]

    Learning proba- bility distributions in continuous evolutionary algorithms – a comparative review.Natu- ral Computing, 3:77–112, 2004

    Kern S, M ¨uller SD, Hansen N, B¨uche D, Ocenasek J, Koumoutsakos P. Learning proba- bility distributions in continuous evolutionary algorithms – a comparative review.Natu- ral Computing, 3:77–112, 2004

    work page 2004

  30. [30]

    A review on estimation of distribution algorithms

    Larra ˜naga P. A review on estimation of distribution algorithms. In P. Larra˜naga and J. A. Lozano, editors, Estimation of Distribution Algorithms, pages 80–90. Kluwer Academic Publishers, 2002

    work page 2002

  31. [31]

    Estimation of distribution algorithms based on multivariate normal and Gaussian networks

    Larra ˜naga P, Lozano JA, Bengoetxea E. Estimation of distribution algorithms based on multivariate normal and Gaussian networks. Technical report, Dept. of Computer Science and Artificial Intelligence, University of the Basque Country, 2001. KZAA-IK- 1-01

    work page 2001

  32. [32]

    A simplex method for function minimization.The Computer Journal 7.4:308-313, 1965

    Nelder JA, Mead R. A simplex method for function minimization.The Computer Journal 7.4:308-313, 1965

    work page 1965

  33. [33]

    Evolutionsstrategie ’94

    Rechenberg I. Evolutionsstrategie ’94. Frommann-Holzboog, Stuttgart, Germany, 1994

    work page 1994

  34. [34]

    The Cross-Entropy Method: a unified approach to combina- torial optimization, Monte-Carlo simulation, and machine learning

    Rubenstein RY , Kroese DP. The Cross-Entropy Method: a unified approach to combina- torial optimization, Monte-Carlo simulation, and machine learning. Springer, 2004

    work page 2004

  35. [35]

    Efficient Covariance Matrix Update for Variable Metric Evolution Strategies

    Suttorp T, Hansen N, Igel C. Efficient Covariance Matrix Update for Variable Metric Evolution Strategies. Machine Learning 75(2): 167–197, 2009. 27 A Algorithm Summary: The (µ/µW,λ )-CMA-ES Figure 6 outlines the complete algorithm 28, summarizing (5), (9), (24), (30), (31), and (37). Used symbols, in order of appearance, are: yk ∼ N (0,C), for k = 1,...,λ ...

    work page 2009

  36. [36]

    setting the fitness as ffitness(x) =fmax + ∥x −xfeasible∥ , (62) where fmax is larger than the worst fitness in the feasible population or in the feasible domain (in case of minization) and xfeasible is a constant feasible point, preferably in the middle of the feasible domain

    work page

  37. [37]

    Repair available as for example with box-constraints

    re-sampling any infeasible solution x until it become feasible. Repair available as for example with box-constraints. Simple repair It is possible to simply repair infeasible individuals before the update equations are applied. This is not recommended, because the CMA-ES makes implicit assumptions on the distribution of solution points, which can be heavi...

    work page