Pith. sign in

REVIEW 3 major objections 7 minor 52 references

When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On curriculum-based transfer tasks, direct-encoding neuroevolution, especially NEAT, outperforms PPO and goal-conditioned PPO.

desk verdict Useful benchmarks and an honest head-to-head, but the abstract overclaims 'curriculum' and the transfer story needs a from-scratch control. read the letter →

arxiv 2505.22696 v1 pith:DZ7ZBGHQ submitted 2025-05-28 cs.LG

Eleni Nisioti , Joachim Winther Pedersen , Erwan Plantec , Milton L. Montero , Sebastian Risi This is my paper
classification cs.LG
keywords neuroevolutiontransferlearningcurriculumNEATreinforcementevolutionstrategiesindirectencodingsbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that neuroevolution can outcompete reinforcement learning when skills must be transferred along a curriculum of progressively harder tasks. On two new benchmarks—logic circuits in 'stepping gates' and physics-based robot tasks in 'ecorobot'—the architecture-growing algorithm NEAT advances through levels that PPO and goal-conditioned PPO cannot pass, though PPO solves the hardest logic task when the curriculum is removed. The authors read this as evidence that direct encodings that evolve network structure carry an advantage for transfer, while indirect encodings like HyperNEAT transfer poorly but escape local optima. They also show the advantage is conditional: it disappears when the robot morphology becomes complex, so transfer-friendly evolution and complex control are not yet combined in one method.

What carries the argument

The load-bearing mechanism is the curriculum plus the genotype-to-phenotype map. Each benchmark forces the agent to solve levels in order, with later levels composed of earlier ones: N-parity adds one bit per level, the Simple ALU requires emulating all earlier logic operations, the maze requires visiting stepping stones in sequence, and the obstacle task raises obstacle height level by level. NEAT's incremental topology growth lets early solutions persist and be extended, while PPO's gradient updates overwrite them when the task changes; the paper's ablations show NEAT degrades without a curriculum and PPO improves. The benchmarks make this contrast measurable and reproducible, with stepping gates reducing transfer to logic-circuit composition and ecorobot adding physical continuity to the same idea.

What would settle it

Train PPO directly on the final level of the stepping-stones maze (food reward only, no stones) and on the highest obstacle height, and compare wall-clock time and sample count to NEAT's curriculum-trained runs; if PPO solves the final level with comparable or better efficiency, then NEAT's apparent transfer advantage is not about transferring skills but about curriculum placement.

Watch

Extended reading notes

Core claim

The paper's central claim is that on tasks decomposed into a curriculum, direct-encoding neuroevolution frequently outperforms strong reinforcement learning baselines. In the stepping-gates benchmark, NEAT solves the N-parity curriculum while PPO reaches only the first level, and the Simple ALU shows the same pattern; the ablation that skips the curriculum lets PPO solve 6-parity directly, which the authors take to mean PPO's failure is an inability to progress through non-stationary levels rather than a capacity limit. In the ecorobot stepping-stones maze, NEAT crosses on average five of seven stones while PPO and goal-conditioned PPO stop at the first. The authors also claim the transfer advantage is specific to direct encodings: HyperNEAT, the indirect encoding, loses its edge and instead avoids local optima that trap PPO, while MAP-Elites explores the whole maze but fails to collect stones in order.

Load-bearing premise

The conclusion assumes that passing later levels in these benchmarks actually requires skills carried over from earlier levels, rather than being re-solved from scratch within each level; if later levels are learnable independently with the same effort, the transfer story does not follow.

Editorial extensions

If this is right

  • On curriculum-shaped tasks, NEAT and CMA-ES become a better default than PPO, with NEAT reaching five of seven stepping stones where PPO stalls at the first.
  • Choosing a neuroevolution encoding trades transfer for exploration: direct encodings transfer skills, while the indirect HyperNEAT escapes local optima but fails to build on earlier levels.
  • Removing the curriculum flips the comparison: PPO solves 6-parity directly, while NEAT's performance degrades slightly, so the curriculum is what separates the methods.
  • The transfer advantage does not yet scale: swapping SimpleRob for the ant in the stepping-stones maze makes NEAT stop at the first stone, so complex morphology remains an open problem.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One implication the paper leaves implicit: NEAT's advantage may come from preserving early modules in its evolving genome, so a direct test would be to inspect whether the networks that solve later levels literally contain the sub-circuits from earlier ones.
  • A testable extension: apply the same curriculum logic to non-policy domains, such as incremental language tasks or multi-step tool use, where PPO is not the dominant baseline; if NEAT still wins, the effect is not about control tasks but about curricula generally.
  • The HyperNEAT result suggests a natural hybrid: use an indirect encoding to jump out of local optima, then switch to a direct encoding to accumulate the discovered skills; the paper hints at this in its discussion of evo-devo but does not test it.
  • The benchmarks could serve as a standard measurement axis separating 'transfer under curriculum' from 'deception' and 'locomotion', which are currently conflated in many NE evaluations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper studies whether neuroevolution (NE) can outperform reinforcement learning (RL) on transfer-learning tasks by introducing two new benchmarks: 'stepping gates' (partiy and a simple ALU with progressively harder levels) and 'ecorobot' (a Brax-based environment with maze navigation and obstacle crossing). The authors compare NEAT, HyperNEAT, CMA-ES, MAP-Elites, PPO, and goal-conditioned PPO across these benchmarks with 10 trials per condition and non-parametric significance tests. The main findings are that direct-encoding NE methods, particularly NEAT, tend to solve more curriculum levels than PPO when curricula are present, while indirect encodings such as HyperNEAT generally succeed at avoiding local optima but fail at transfer. The paper also includes ablations removing the curriculum in N-parity and obstacle-height tasks, open-source code, and a visualization website.

Significance. If the results are taken at face value, this study would provide a concrete, reproducible demonstration that certain neuroevolutionary algorithms can outperform or match PPO on curriculum-based transfer tasks, and that direct encodings with architectural search (NEAT) have an advantage over RL in such settings. The paper's strengths include: new open-sourced benchmarks in JAX, explicit p-values with non-parametric tests, 10 trials per condition, and ablations that test the effect of the curriculum. The finding that PPO can solve 6-parity from scratch yet fails on the progressive curriculum is a useful and non-obvious observation. However, the central conceptual claim that both benchmarks test 'transfer across tasks of increasing complexity' is not fully supported by the stepping-stones maze, and the absence of from-scratch controls for several final levels leaves open the possibility that some NE successes are due to re-optimization rather than skill transfer.

major comments (3)
  1. [Section 3.2.2, 'Maze with stepping stones'] The paper describes this task as one of the two ecorobot tasks 'specifically designed to test for transfer learning,' and the abstract claims that 'crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity.' However, the stepping-stones maze is a single maze with a shaped reward that progresses through a sequence of stones within one episode; it contains no separate tasks or levels of increasing difficulty. Thus the success in Figure 5, where NEAT reaches on average five of seven stones, does not by itself constitute evidence of cross-task transfer. To support the transfer framing, the authors need either to provide a from-scratch control on later stones (e.g., train directly on the final sub-goal or on a task that starts at stone k) or to reframe this experiment as testing long-horizon credit assignment or sub-goal sequencing, not transfer.
  2. [Sections 5.1 and 5.2.1, from-scratch controls] The central claim that NEAT and CMA-ES 'transfer' skills across levels is not established for the Simple ALU and the hierarchical-obstacles tasks because no from-scratch control is reported for the final levels of these tasks for those methods. The N-parity ablation in Figure 4 is a good example of the needed control: it shows NEAT degrades when trained directly on 6-parity, indicating the curriculum helps it. But no analogous ablation is shown for Simple ALU or for the full hierarchical-obstacles final level; Figure 8 compares NEAT and PPO on medium- and high-difficulty versions with all obstacles at the same height, which is not the same as the final level of the curriculum (where obstacles have increasing heights). Since CMA-ES uses a fixed topology and no explicit memory mechanism yet solves all N-parity levels, it is plausible that it re-optimizes each level from scratch; the paper needs a control that distinguishes re-optimization from transfer, or it should temper the 'transfer abilities' language.
  3. [Section 4, hyperparameter tuning] The tuning protocol is asymmetric across methods: NEAT and HyperNEAT were tuned only on the 6-parity task, PPO and goal-conditioned PPO were tuned independently for stepping-gates and ecorobot, CMA-ES used defaults from evosax, and MAP-Elites used hyperparameters from a previous study. This asymmetry is a potential confound for the headline claim that 'NE methods ... frequently outperform RL baselines,' because the RL baselines received task-specific tuning while the NE methods did not. While the direction of the bias may in fact favor PPO, the paper does not quantify how sensitive the rankings are to the tuning budget. The authors should either run a small sensitivity analysis with additional configurations for CMA-ES and NEAT on the ecorobot tasks, or explicitly discuss the likely direction and magnitude of this bias in the limitations.
minor comments (7)
  1. [Figure 4 and text in Section 5.1] The phrase 'NEAT's performance degrades slightly' is vague; please report the actual success percentages for NEAT on 6-parity with and without the curriculum, and add error bars or confidence intervals to Figure 4.
  2. [Section 3.2.2, 'Hierarchical obstacles'] It is not clear how the curriculum is implemented for this task: does the agent train on level 1, then level 2 with level 1 still present, or does the task reset each level? Please specify the training protocol and the exact episode composition, since this is central to the transfer interpretation.
  3. [Section 5.2.1] The claim that PPO 'has not learned to adjust its behavior to obstacles of varying size but to ignore them' is based on visual inspection of trajectories. Please provide a quantitative measure (e.g., number of flips, bumpiness metric, obstacle-clearing rate) to support this statement.
  4. [Appendix A.5] The sentence 'We report all relevant hyperparameters in Appendix 5' should refer to the actual appendix/table number (e.g., 'Table 5' or 'Appendix A.5').
  5. [References] References [38] and [39] are both the same PPO paper (Schulman et al., 2017); please consolidate them into a single entry.
  6. [Abstract and Section 4] The abstract says 'Both methods exhibit sub-par transfer learning abilities' about PPO and goal-conditioned PPO, but goal-conditioned PPO is only evaluated in ecorobot, not in stepping gates. Please state in the methods/results where each RL variant is evaluated, or adjust the wording.
  7. [Introduction] The phrase 'a rather concrete flair of transfer learning' appears to be a typo for 'flavor'; please correct it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark comparison and its central claims are measured, not derived from fitted parameters or self-citation chains.

full rationale

The paper's central claim—that direct-encoding neuroevolution methods, especially NEAT, frequently outperform PPO and goal-conditioned PPO on curriculum-based transfer tasks—is supported by direct experimental measurements on newly introduced benchmarks. There is no derivation chain in which an output quantity is defined in terms of the target result. Hyperparameters for NEAT, HyperNEAT, PPO, and goal-conditioned PPO were tuned on a single task (6-parity or locomotion with the ant) and then evaluated on different tasks or levels; this is an upstream tuning choice, not a fitted parameter renamed as a prediction. The ablations that remove the curriculum (direct 6-parity and fixed-obstacle-height tasks) are genuine control experiments rather than circular re-statements of the conclusion. The paper does cite work by its own authors (e.g., [32], [36], [35]), but these citations appear in background discussion about indirect encodings and neuroevolution generally, and they are not load-bearing for the empirical comparisons that constitute the main finding. The statement that a curriculum is 'crucial in both benchmarks' is a design description of the benchmarks, not a conclusion derived from the benchmarks. No equation is shown to equal another by construction, and no prediction is forced by the inputs. The most appropriate finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No parameter-free derivation is claimed; the central claim rests on empirical comparisons. The most important implicit inputs are the validity of the benchmarks as transfer tasks and the adequacy and fairness of the baselines. Hyperparameters are hand-chosen or defaulted and are listed above as free parameters.

free parameters (5)
  • NEAT population size = 5000 (1024 in ecorobot)
    Hand-selected during tuning on the 6-parity task; affects exploration and final success, and was reduced for ecorobot.
  • PPO network architecture = 6x4 neurons (stepping gates), 4x32 neurons (ecorobot)
    Authors state architecture tuning gave the largest benefit; capacity differences relative to evolved networks could affect the comparison.
  • CMA-ES network and defaults = 24 hidden neurons, evosax defaults
    No tuning was performed; the paper acknowledges CMA-ES may benefit from further tuning, so its relative performance is not a strong ceiling.
  • MAP-Elites grid = 50-by-50 tiles
    Chosen to match behavior descriptors (final x,y coordinates); grid resolution influences which behaviors get archived and thus final policies.
  • HyperNEAT substrate layout = single hidden layer, 32 hidden neurons
    Neuron positions on the substrate are chosen manually; this geometry is known to strongly affect indirect-encoding behavior.
assumptions (3)
  • domain assumption The stepping-gates and ecorobot curricula validly measure transfer of learned skills across levels.
    Section 3 states that a curriculum is crucial for evaluating skill transfer; if later levels are solved by independent per-level optimization, the transfer-based interpretation of the results fails.
  • domain assumption The RL baselines are reasonably strong representatives of current RL for these tasks.
    PPO was tuned separately per benchmark, but no exhaustive search was performed; unequal tuning budgets across methods is acknowledged as a limitation in the discussion.
  • domain assumption Ten random trials per condition support the reported statistical comparisons.
    The paper uses Kruskal-Wallis and Mann-Whitney U tests; the small number of trials and lack of explicit multiple-comparison correction add variance to the inference.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?." pith.science (2026). https://pith.science/paper/DZ7ZBGHQ

@misc{pith2026250522696,
  author       = {Pith},
  title        = {Pith review of: When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZ7ZBGHQ}},
  note         = {Machine review of arXiv:2505.22696}
}
read the original abstract

The ability to continuously and efficiently transfer skills across tasks is a hallmark of biological intelligence and a long-standing goal in artificial systems. Reinforcement learning (RL), a dominant paradigm for learning in high-dimensional control tasks, is known to suffer from brittleness to task variations and catastrophic forgetting. Neuroevolution (NE) has recently gained attention for its robustness, scalability, and capacity to escape local optima. In this paper, we investigate an understudied dimension of NE: its transfer learning capabilities. To this end, we introduce two benchmarks: a) in stepping gates, neural networks are tasked with emulating logic circuits, with designs that emphasize modular repetition and variation b) ecorobot extends the Brax physics engine with objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. Crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity. Our empirical analysis shows that NE methods vary in their transfer abilities and frequently outperform RL baselines. Our findings support the potential of NE as a foundation for building more adaptable agents and highlight future challenges for scaling NE to complex, real-world problems.

Figures

Figures reproduced from arXiv: 2505.22696 by the authors.

Figure 1
Figure 1. Illustration of the stepping gates benchmark. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the ecorobot benchmark. An environment consists of a robot and a task. (left) We currently support different Brax robots and a simpler robot, SimpleRob. Robots can be equipped with rangefinders and pie-slice sensors. (right) A task consists in a choice of objects and a reward function. We have implemented tasks that test for different behavioral challenges, with the Stepping stones maze and Hierarchi… view at source ↗
Figure 5
Figure 5. Comparing the ability of methods to progress [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figures from the paper (6 more)
Figure 6
Figure 6. Figure 6: Left: Map of elites in the deceptive maze. Right: Map of elites in the maze with stepping stones. While coverage is optimal in both cases, rewards in the stepping stones maze are low as an optimal trajectory requires traversing through the stepping stones in the right …
Figure 4
Figure 4. Figure 4: Ablation for the N-parity task. When directly solving 6-parity (without going through the intermediate levels) PPO solves the task while NEAT’s performance degrades slightly. Task: SimpleRob in stepping stones maze Reward [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 8
Figure 8. Figure 8: Ablation for the task locomotion with obstacles: [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Effect of complexifying the robot morphology [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Histogram of rewards for multiple evaluation [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 11
Figure 11. Figure 11: Illustration of how the substrate for HyperNEAT [PITH_FULL_IMAGE:figures/full_fig_p010_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

52 extracted references · 25 canonical work pages

  1. [1]

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning. 41–48

    2009

  2. [2]

    Felix Chalumeau, Raphael Boige, Bryan Lim, Valentin Macé, Maxime Allard, Arthur Flajolet, Antoine Cully, and Thomas Pierrot. 2023. Neuroevolution is a Competitive Alternative to Reinforcement Learning for Skill Discovery. https: //doi.org/10.48550/arXiv.2210.03516 arXiv:2210.03516 [cs]

    arXiv doi:10.48550/arxiv.2210.03516 2023

  3. [3]

    Felix Chalumeau, Bryan Lim, Raphael Boige, Maxime Allard, Luca Grillotti, Manon Flageat, Valentin Macé, Guillaume Richard, Arthur Flajolet, Thomas Pierrot, et al. 2024. Qdax: A library for quality-diversity and population-based algorithms with hardware acceleration. Journal of Machine Learning Research 25, 108 (2024), 1–16

    work page 2024

  4. [4]

    Paul Cisek. 2019. Resynthesizing behavior through phylogenetic refinement. Attention, Perception, & Psychophysics 81, 7 (Oct. 2019), 2265–2287. https://doi. org/10.3758/s13414-019-01760-1

    work page doi:10.3758/s13414-019-01760-1 2019

  5. [5]

    Cédric Colas, Tristan Karch, Olivier Sigaud, and Pierre-Yves Oudeyer. 2022. Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforce- ment Learning: a Short Survey. https://doi.org/10.48550/arXiv.2012.09830 arXiv:2012.09830 [cs]

    work page doi:10.48550/arxiv.2012.09830 2022

  6. [6]

    Matthew Crosby, Benjamin Beyret, and Marta Halina. 2019. The Animal-AI Olympics. Nature Machine Intelligence 1, 5 (May 2019), 257–257. https://doi.org/ 10.1038/s42256-019-0050-3 Publisher: Nature Publishing Group

    work page doi:10.1038/s42256-019-0050-3 2019

  7. [7]

    Stanley, and Jeff Clune

    Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, and Jeff Clune

    work page

  8. [8]

    Kai Olav Ellefsen, Jean-Baptiste Mouret, and Jeff Clune. 2015. Neural Modularity Helps Organisms Evolve to Learn New Skills without Forgetting Old Skills.PLOS When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks? Computational Biology 11, 4 (April 2015), e1004128. https://doi.org/10.1371/ journal.pcbi.1004128

    work page 2015

Show all 52 references
  1. [9]

    Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. 2018. Diversity is All You Need: Learning Skills without a Reward Function. https: //doi.org/10.48550/arXiv.1802.06070 arXiv:1802.06070 [cs]

    2018 doi arXiv

  2. [10]

    Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem

    C. Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. 2021. Brax – A Differentiable Physics Engine for Large Scale Rigid Body Simulation. https://doi.org/10.48550/arXiv.2106.13281 arXiv:2106.13281 [cs]

    2021 doi arXiv

  3. [11]

    Greenbury, Ard A

    Sam F. Greenbury, Ard A. Louis, and Sebastian E. Ahnert. 2022. The structure of genotype-phenotype maps makes fitness landscapes navigable. Nature Ecology & Evolution 6, 11 (Nov. 2022), 1742–1752. https://doi.org/10.1038/s41559-022- 01867-z Publisher: Nature Publishing Group

    2022 doi

  4. [12]

    Luca Grillotti and Antoine Cully. 2023. Kheperax: a Lightweight JAX-based Robot Control Environment for Benchmarking Quality-Diversity Algorithms. In Proceedings of the Companion Conference on Genetic and Evolutionary Computa- tion (GECCO ’23 Companion) . Association for Compu...

    2023

  5. [13]

    F. Gruau. 1992. Genetic synthesis of Boolean neural networks with a cell rewrit- ing developmental process. In [Proceedings] COGANN-92: International Work- shop on Combinations of Genetic Algorithms and Neural Networks . IEEE Comput. Soc. Press, Baltimore, MD, USA, 55–74. http...

    1992 doi

  6. [14]

    Frédéric Gruau. 1994. Automatic Definition of Modular Neural Networks. Adaptive Behavior 3, 2 (Sept. 1994), 151–183. https://doi.org/10.1177/ 105971239400300202

    1994

  7. [15]

    Nikolaus Hansen. 2023. The CMA Evolution Strategy: A Tutorial. https: //doi.org/10.48550/arXiv.1604.00772 arXiv:1604.00772 [cs]

    2023 doi arXiv

  8. [16]

    Yizhou Huang, Kevin Xie, Homanga Bharadhwaj, and Florian Shkurti. 2021. Continual Model-Based Reinforcement Learning with Hypernetworks. https: //doi.org/10.48550/arXiv.2009.11997 arXiv:2009.11997 [cs]

    2021 doi arXiv

  9. [17]

    Leslie Pack Kaelbling. 1993. Learning to achieve goals. In IJCAI, Vol. 2. Citeseer, 1094–1098. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi= 6df43f70f383007a946448122b75918e3a9d6682

    1993

  10. [18]

    Khimya Khetarpal, Matthew Riemer, Irina Rish, and Doina Precup. 2022. Towards Continual Reinforcement Learning: A Review and Perspectives. https://doi.org/ 10.48550/arXiv.2012.13490 arXiv:2012.13490 [cs]

    2022 doi arXiv

  11. [19]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2017. Adam: A Method for Stochastic Opti- mization. https://doi.org/10.48550/arXiv.1412.6980 arXiv:1412.6980 [cs]

    2017 doi arXiv

  12. [20]

    Knoll and Sean B

    Andrew H. Knoll and Sean B. Carroll. 1999. Early Animal Evolution: Emerging Views from Comparative Biology and Geology. Science 284, 5423 (June 1999), 2129–2137. https://doi.org/10.1126/science.284.5423.2129

    1999

  13. [21]

    Koza and J.P

    J.R. Koza and J.P. Rice. 1991. Genetic generation of both the weights and ar- chitecture for a neural network. In IJCNN-91-Seattle International Joint Con- ference on Neural Networks , Vol. ii. IEEE, Seattle, WA, USA, 397–404. https: //doi.org/10.1109/IJCNN.1991.155366

    1991

  14. [22]

    Koza and J.P

    J.R. Koza and J.P. Rice. 1992. Genetic Programming. https://mitpress.mit.edu/ 9780262527910/genetic-programming/

    1992

  15. [23]

    John R Koza. [n. d.]. genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems. ([n. d.])

  16. [24]

    Robert Tjarko Lange. 2022. evosax: JAX-based Evolution Strategies. arXiv preprint arXiv:2212.04180 (2022)

    2022 arXiv

  17. [25]

    Joel Lehman and Kenneth O. Stanley. 2011. Abandoning Objectives: Evolution Through the Search for Novelty Alone. Evolutionary Computation 19, 2 (June 2011), 189–223. https://doi.org/10.1162/EVCO_a_00025

    2011 doi

  18. [26]

    Bryan Lim, Maxime Allard, Luca Grillotti, and Antoine Cully. 2022. Accelerated Quality-Diversity through Massive Parallelism. https://doi.org/10.48550/arXiv. 2202.01258 arXiv:2202.01258 [cs]

    2022 doi arXiv

  19. [27]

    Jeremy Lucas and Isabeau Prémont-Schwarz. 2024. Articulated Animal AI: An Environment for Animal-like Cognition in a Limbed Agent. https://doi.org/10. 48550/arXiv.2410.09275 arXiv:2410.09275 [cs]

    2024 doi arXiv

  20. [28]

    Michael Matthews, Michael Beukman, Chris Lu, and Jakob Foerster. 2024. Kinetix: Investigating the Training of General Agents through Open-Ended Physics-Based Control Tasks. https://doi.org/10.48550/arXiv.2410.23208 arXiv:2410.23208 [cs]

    2024 doi arXiv

  21. [29]

    Moriarty and Risto Miikkulainen

    David E. Moriarty and Risto Miikkulainen. 1996. Efficient reinforcement learning through symbiotic evolution. Machine Learning 22, 1 (March 1996), 11–32. https://doi.org/10.1007/BF00114722

    1996 doi

  22. [30]

    Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909 (2015)

    2015 arXiv

  23. [31]

    Elias Najarro, Shyam Sudhakaran, Claire Glanois, and Sebastian Risi. 2022. Hy- perNCA: Growing Developmental Networks with Neural Cellular Automata. http://arxiv.org/abs/2204.11674 arXiv:2204.11674 [cs]

    2022 arXiv

  24. [32]

    Eleni Nisioti, Erwan Plantec, Milton Montero, Joachim Pedersen, and Sebastian Risi. 2024. Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, Melbourne VIC Austral...

    2024

  25. [33]

    NeuroEvolution of Augmenting Topologies

    Evgenia Papavasileiou, Jan Cornelis, and Bart Jansen. 2021. A Systematic Litera- ture Review of the Successors of “NeuroEvolution of Augmenting Topologies”. Evolutionary Computation 29, 1 (March 2021), 1–73. https://doi.org/10.1162/ evco_a_00282

    2021

  26. [34]

    Pugh, Lisa B

    Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. 2016. Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI 3 (July 2016). https://doi.org/10.3389/frobt.2016.00040 Publisher: Frontiers

    2016

  27. [35]

    Sebastian Risi, David Ha, Yujin Tang, and Risto Miikkulainen. 2025. Neuroevo- lution: Harnessing Creativity in AI Model Design . MIT Press, Cambridge, MA. http://www.cs.utexas.edu/users/ai-lab?risi:book25

    2025

  28. [36]

    Sebastian Risi, Joel Lehman, and Kenneth O. Stanley. 2010. Evolving the place- ment and density of neurons in the hyperneat substrate. In Proceedings of the 12th annual conference on Genetic and evolutionary computation (GECCO ’10). Association for Computing Machinery, New Yor...

    2010

  29. [37]

    Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. 2017. Evolution Strategies as a Scalable Alternative to Reinforcement Learning. https: //doi.org/10.48550/arXiv.1703.03864 arXiv:1703.03864 [stat]

    2017 doi arXiv

  30. [39]

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov

  31. [40]

    Sergey Shuvaev, Divyansha Lachi, Alexei Koulakov, and Anthony Zador. 2024. Encoding innate ability through a genomic bottleneck.Proceedings of the National Academy of Sciences 121, 38 (Sept. 2024), e2409160121. https://doi.org/10.1073/ pnas.2409160121 Publisher: Proceedings of...

    2024

  32. [41]

    Stanley, Jeff Clune, Joel Lehman, and Risto Miikkulainen

    Kenneth O. Stanley, Jeff Clune, Joel Lehman, and Risto Miikkulainen. 2019. De- signing neural networks through neuroevolution. Nature Machine Intelligence 1, 1 (Jan. 2019), 24–35. https://doi.org/10.1038/s42256-018-0006-z Publisher: Nature Publishing Group

    2019 doi

  33. [42]

    arXiv:1707.06347 [cs.LG] https://arxiv.org/abs/1707.06347

    Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG] https://arxiv.org/abs/1707.06347

    arXiv

  34. [43]

    Stanley and Risto Miikkulainen

    Kenneth O. Stanley and Risto Miikkulainen. 2002. Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation 10, 2 (June 2002), 99–127. https://doi.org/10.1162/106365602320169811

    2002 doi

  35. [44]

    Stanley and Risto Miikkulainen

    Kenneth O. Stanley and Risto Miikkulainen. 2003. A Taxonomy for Artificial Embryogeny. Artificial Life 9, 2 (April 2003), 93–130. https://doi.org/10.1162/ 106454603322221487

    2003

  36. [45]

    Stanley, David B

    Kenneth O. Stanley, David B. D’Ambrosio, and Jason Gauci. 2009. A Hypercube- Based Encoding for Evolving Large-Scale Neural Networks. Artificial Life 15, 2 (April 2009), 185–212. https://doi.org/10.1162/artl.2009.15.2.15202 Conference Name: Artificial Life

    2009 doi

  37. [46]

    Matthew E Taylor and Peter Stone. [n. d.]. Transfer Learning for Reinforcement Learning Domains: A Survey. ([n. d.])

  38. [47]

    Thomson, Léni Le Goff, Emma Hart, and Edgar Buchanan

    Sarah L. Thomson, Léni Le Goff, Emma Hart, and Edgar Buchanan. 2024. Under- standing Fitness Landscapes in Morpho-Evolution via Local Optima Networks. In Proceedings of the Genetic and Evolutionary Computation Conference . ACM, Melbourne VIC Australia, 114–123. https://doi.org...

    2024

  39. [48]

    Stanley, and Jeff Clune

    Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Ken- neth O. Stanley, and Jeff Clune. 2018. Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforce- ment Learning. https://doi.org/10.48550/arXiv...

    2018 doi arXiv

  40. [49]

    Konstantinos Voudouris, Matthew Crosby, Benjamin Beyret, José Hernández- Orallo, Murray Shanahan, Marta Halina, and Lucy G. Cheke. 2022. Direct Human- AI Comparison in the Animal-AI Environment. Frontiers in Psychology 13 (May 2022). https://doi.org/10.3389/fpsyg.2022.711821 P...

    2022

  41. [50]

    Lishuang Wang, Mengfei Zhao, Enyu Liu, Kebin Sun, and Ran Cheng. 2024. Tensorized NeuroEvolution of Augmenting Topologies for GPU Acceleration. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’24). Association for Computing Machinery, New York, NY,...

    2024

  42. [51]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention Is All You Need. https://doi.org/10.48550/arXiv.1706.03762 arXiv:1706.03762 [cs]

    2023 doi arXiv

  43. [2017]

    https://doi.org/10.48550/arXiv

    Proximal Policy Optimization Algorithms. https://doi.org/10.48550/arXiv. 1707.06347 arXiv:1707.06347 [cs]

    doi arXiv

  44. [2021]

    Nature 590, 7847 (Feb

    First return, then explore. Nature 590, 7847 (Feb. 2021), 580–586. https: //doi.org/10.1038/s41586-020-03157-9 Publisher: Nature Publishing Group

    2021 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.