{"id":"c8972f10-bebe-4af1-a56b-b189e6744f04","arxiv_id":"2505.22696","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On two new curriculum benchmarks, direct-encoding neuroevolution (NEAT) transfers skills across levels better than PPO reinforcement learning, while indirect encodings like HyperNEAT transfer poorly.","lead":"This paper introduces two curriculum benchmarks, one based on logic circuits and one on physics-based robots, and compares how neuroevolution (NE) and reinforcement learning (RL) agents transfer skills across increasing difficulty levels. It reports that direct-encoding NE, especially NEAT, frequently outperforms PPO on these transfer tasks, while all methods struggle when the robot body becomes more complex.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmarks may not require transfer at all: later-level successes could be re-optimization, and the ecorobot 'stepping stones' task is not a curriculum, so the central claim lacks a from-scratch control.","rationale":"The paper is a well-scoped empirical study with open-sourced benchmarks and reproduction code, and the raw observation that NEAT reaches more stepping stones than PPO is not in question. The load-bearing issue is interpretive: the title and abstract frame the contribution as a finding about transfer learning, but the tasks may only require re-optimizing a new level. In stepping gates, PPO solves 6-parity from scratch (Fig. 4), and CMA-ES, which has no architecture growth or explicit memory, solves all levels, suggesting the curriculum may not be necessary for the NE methods either. The paper does not report from-scratch controls for NEAT or CMA-ES on the final levels. In ecorobot, the stepping-stones maze is a single task with a shaped reward rather than a sequence of tasks of increasing complexity; only hierarchical obstacles fits the curriculum description. Thus the abstract's statement that 'both benchmarks' rely on a curriculum is inaccurate. A no-curriculum control—training from scratch on the final level—would settle whether success depends on the preceding levels. If from-scratch performance is comparable, the central claim about transfer is unsupported and the paper reduces to a task-specific NE-vs-RL comparison; if curriculum training is substantially better, the transfer interpretation is corroborated. This check is concrete and feasible given the released repositories, and it directly targets the weakest assumption the reader identified.","tokens_in":14727,"tokens_out":9940,"duration_ms":112575,"concrete_test":"Add a no-curriculum control: for each method, train a fresh population/policy directly on the final level of each curriculum task (6-parity, full Simple ALU, high-obstacle HalfCheetah) using the same evaluation protocol and budget as the curriculum runs, and compare success/reward with Figures 3, 7, and 8. If from-scratch performance matches curriculum performance, the observed advantage is not evidence of transfer; if curriculum training is substantially better, the transfer interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that direct-encoding NE (notably NEAT) outperforms PPO specifically in transfer learning is only as strong as the evidence that the benchmarks measure transfer. In stepping gates, a later level contains earlier functions, but no experiment shows later-level success depends on the earlier solutions: PPO solves 6-parity from scratch (Fig. 4), and CMA-ES, with a fixed topology and no explicit memory mechanism, solves all levels in both tasks (Fig. 3). Without a from-scratch-on-each-level control, those successes may reflect re-optimization on the new level rather than skill transfer. In ecorobot, the main transfer task (maze with stepping stones) is not a curriculum at all: it is a single maze with a shaped reward over a sequence of stones, so the abstract's claim that 'crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity' is not supported by that task; only hierarchical obstacles has distinct levels. The conclusion that NEAT has superior transfer abilities therefore requires showing that curriculum training beats from-scratch training on the final levels, or that the stepping-stones maze genuinely requires cross-task transfer.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies whether neuroevolution (NE) can outperform reinforcement learning (RL) on transfer-learning tasks by introducing two new benchmarks: 'stepping gates' (partiy and a simple ALU with progressively harder levels) and 'ecorobot' (a Brax-based environment with maze navigation and obstacle crossing). The authors compare NEAT, HyperNEAT, CMA-ES, MAP-Elites, PPO, and goal-conditioned PPO across these benchmarks with 10 trials per condition and non-parametric significance tests. The main findings are that direct-encoding NE methods, particularly NEAT, tend to solve more curriculum levels than PPO when curricula are present, while indirect encodings such as HyperNEAT generally succeed at avoiding local optima but fail at transfer. The paper also includes ablations removing the curriculum in N-parity and obstacle-height tasks, open-source code, and a visualization website.","tokens_in":14962,"tokens_out":5349,"duration_ms":53888,"significance":"If the results are taken at face value, this study would provide a concrete, reproducible demonstration that certain neuroevolutionary algorithms can outperform or match PPO on curriculum-based transfer tasks, and that direct encodings with architectural search (NEAT) have an advantage over RL in such settings. The paper's strengths include: new open-sourced benchmarks in JAX, explicit p-values with non-parametric tests, 10 trials per condition, and ablations that test the effect of the curriculum. The finding that PPO can solve 6-parity from scratch yet fails on the progressive curriculum is a useful and non-obvious observation. However, the central conceptual claim that both benchmarks test 'transfer across tasks of increasing complexity' is not fully supported by the stepping-stones maze, and the absence of from-scratch controls for several final levels leaves open the possibility that some NE successes are due to re-optimization rather than skill transfer.","major_comments":[{"comment":"The paper describes this task as one of the two ecorobot tasks 'specifically designed to test for transfer learning,' and the abstract claims that 'crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity.' However, the stepping-stones maze is a single maze with a shaped reward that progresses through a sequence of stones within one episode; it contains no separate tasks or levels of increasing difficulty. Thus the success in Figure 5, where NEAT reaches on average five of seven stones, does not by itself constitute evidence of cross-task transfer. To support the transfer framing, the authors need either to provide a from-scratch control on later stones (e.g., train directly on the final sub-goal or on a task that starts at stone k) or to reframe this experiment as testing long-horizon credit assignment or sub-goal sequencing, not transfer.","section":"Section 3.2.2, 'Maze with stepping stones'"},{"comment":"The central claim that NEAT and CMA-ES 'transfer' skills across levels is not established for the Simple ALU and the hierarchical-obstacles tasks because no from-scratch control is reported for the final levels of these tasks for those methods. The N-parity ablation in Figure 4 is a good example of the needed control: it shows NEAT degrades when trained directly on 6-parity, indicating the curriculum helps it. But no analogous ablation is shown for Simple ALU or for the full hierarchical-obstacles final level; Figure 8 compares NEAT and PPO on medium- and high-difficulty versions with all obstacles at the same height, which is not the same as the final level of the curriculum (where obstacles have increasing heights). Since CMA-ES uses a fixed topology and no explicit memory mechanism yet solves all N-parity levels, it is plausible that it re-optimizes each level from scratch; the paper needs a control that distinguishes re-optimization from transfer, or it should temper the 'transfer abilities' language.","section":"Sections 5.1 and 5.2.1, from-scratch controls"},{"comment":"The tuning protocol is asymmetric across methods: NEAT and HyperNEAT were tuned only on the 6-parity task, PPO and goal-conditioned PPO were tuned independently for stepping-gates and ecorobot, CMA-ES used defaults from evosax, and MAP-Elites used hyperparameters from a previous study. This asymmetry is a potential confound for the headline claim that 'NE methods ... frequently outperform RL baselines,' because the RL baselines received task-specific tuning while the NE methods did not. While the direction of the bias may in fact favor PPO, the paper does not quantify how sensitive the rankings are to the tuning budget. The authors should either run a small sensitivity analysis with additional configurations for CMA-ES and NEAT on the ecorobot tasks, or explicitly discuss the likely direction and magnitude of this bias in the limitations.","section":"Section 4, hyperparameter tuning"}],"minor_comments":[{"comment":"The phrase 'NEAT's performance degrades slightly' is vague; please report the actual success percentages for NEAT on 6-parity with and without the curriculum, and add error bars or confidence intervals to Figure 4.","section":"Figure 4 and text in Section 5.1"},{"comment":"It is not clear how the curriculum is implemented for this task: does the agent train on level 1, then level 2 with level 1 still present, or does the task reset each level? Please specify the training protocol and the exact episode composition, since this is central to the transfer interpretation.","section":"Section 3.2.2, 'Hierarchical obstacles'"},{"comment":"The claim that PPO 'has not learned to adjust its behavior to obstacles of varying size but to ignore them' is based on visual inspection of trajectories. Please provide a quantitative measure (e.g., number of flips, bumpiness metric, obstacle-clearing rate) to support this statement.","section":"Section 5.2.1"},{"comment":"The sentence 'We report all relevant hyperparameters in Appendix 5' should refer to the actual appendix/table number (e.g., 'Table 5' or 'Appendix A.5').","section":"Appendix A.5"},{"comment":"References [38] and [39] are both the same PPO paper (Schulman et al., 2017); please consolidate them into a single entry.","section":"References"},{"comment":"The abstract says 'Both methods exhibit sub-par transfer learning abilities' about PPO and goal-conditioned PPO, but goal-conditioned PPO is only evaluated in ecorobot, not in stepping gates. Please state in the methods/results where each RL variant is evaluated, or adjust the wording.","section":"Abstract and Section 4"},{"comment":"The phrase 'a rather concrete flair of transfer learning' appears to be a typo for 'flavor'; please correct it.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical contribution with good reproducibility practices (open-source code, benchmarks, explicit p-values), but the conceptual framing of the stepping-stones maze as a 'curriculum' is not accurate, and the missing from-scratch controls weaken the central transfer claim. I would ask the authors to add the missing controls or substantially soften the transfer language. The hyperparameter-tuning asymmetry, while acknowledged, should be addressed with at least a small sensitivity analysis. I do not see this as grounds for rejection, but the current framing overstates what can be concluded from the data."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid empirical study with reusable benchmarks, but the abstract oversells the curriculum framing. The stepping stones maze isn't a curriculum; only the hierarchical obstacles task actually has levels. That's a real gap between the prose and the setup.\n\nWhat's new: two JAX benchmarks, stepping gates and ecorobot, both open-sourced. The systematic comparison across NE variants (NEAT, HyperNEAT, CMA-ES, MAP-Elites) and two RL baselines is genuinely useful. The N-parity ablation is the most informative piece: PPO solves 6-parity directly but fails when forced through the curriculum, while NEAT degrades slightly without the curriculum. That's strong evidence that the curriculum itself is what separates the methods.\n\nSoft spots: (1) The abstract says 'crucial in both benchmarks is the presence of a curriculum,' but the stepping stones maze is a single task with a shaped reward over a sequence of stones. It tests credit assignment, not transfer across tasks. The hierarchical obstacles task does have a curriculum, and the results there support the claim, but the paper should be precise about which task tests what. (2) Hyperparameter tuning is asymmetric: NEAT/HyperNEAT tuned on 6-parity, PPO tuned per benchmark, CMA-ES at defaults. That doesn't invalidate the results, but it weakens head-to-head claims. (3) For the stepping stones maze, there's no from-scratch control, so the NEAT advantage might be about optimization difficulty, not transfer. The obstacles ablation partly addresses this, but not for the maze.\n\nThe stress-test note claims the benchmarks may not require transfer at all. That's overblown for the curriculum tasks: the sequential gating means the agent must solve earlier levels before seeing later ones, so earlier solutions are necessarily part of the training history. For the stepping stones maze, the critique lands: it's not a curriculum. But the central claim about NEAT beating PPO on the actual curriculum task (hierarchical obstacles) still holds, so the paper's main conclusion isn't hollow.\n\nThis paper deserves a serious referee. It's not a breakthrough, but it's a well-scoped empirical contribution with open code and a clear negative result about PPO under curricula. A revision should fix the abstract, add a from-scratch control for the maze, and acknowledge the tuning asymmetry more prominently. I'd bring it to the reading group and would cite the benchmarks if I work in NE transfer.","headline":"Useful benchmarks and an honest head-to-head, but the abstract overclaims 'curriculum' and the transfer story needs a from-scratch control.","tokens_in":15506,"tokens_out":3626,"would_cite":true,"duration_ms":33494,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On curriculum-based transfer tasks, direct-encoding neuroevolution, especially NEAT, outperforms PPO and goal-conditioned PPO.","keywords":["neuroevolution","transfer learning","curriculum learning","NEAT","reinforcement learning","evolution strategies","indirect encodings","benchmark"],"falsifier":"Train PPO directly on the final level of the stepping-stones maze (food reward only, no stones) and on the highest obstacle height, and compare wall-clock time and sample count to NEAT's curriculum-trained runs; if PPO solves the final level with comparable or better efficiency, then NEAT's apparent transfer advantage is not about transferring skills but about curriculum placement.","tokens_in":14530,"feed_emoji":"🧬","tokens_out":7028,"duration_ms":71075,"temperature":0.7,"pith_summary":"The paper sets out to establish that neuroevolution can outcompete reinforcement learning when skills must be transferred along a curriculum of progressively harder tasks. On two new benchmarks—logic circuits in 'stepping gates' and physics-based robot tasks in 'ecorobot'—the architecture-growing algorithm NEAT advances through levels that PPO and goal-conditioned PPO cannot pass, though PPO solves the hardest logic task when the curriculum is removed. The authors read this as evidence that direct encodings that evolve network structure carry an advantage for transfer, while indirect encodings like HyperNEAT transfer poorly but escape local optima. They also show the advantage is conditional: it disappears when the robot morphology becomes complex, so transfer-friendly evolution and complex control are not yet combined in one method.","feed_headline":"NEAT leaps five steps where PPO stops at one","feed_subtitle":"Curriculum tasks flip the usual advantage: evolved networks keep old skills while adding new ones.","key_machinery":"The load-bearing mechanism is the curriculum plus the genotype-to-phenotype map. Each benchmark forces the agent to solve levels in order, with later levels composed of earlier ones: N-parity adds one bit per level, the Simple ALU requires emulating all earlier logic operations, the maze requires visiting stepping stones in sequence, and the obstacle task raises obstacle height level by level. NEAT's incremental topology growth lets early solutions persist and be extended, while PPO's gradient updates overwrite them when the task changes; the paper's ablations show NEAT degrades without a curriculum and PPO improves. The benchmarks make this contrast measurable and reproducible, with stepping gates reducing transfer to logic-circuit composition and ecorobot adding physical continuity to the same idea.","core_discovery":"The paper's central claim is that on tasks decomposed into a curriculum, direct-encoding neuroevolution frequently outperforms strong reinforcement learning baselines. In the stepping-gates benchmark, NEAT solves the N-parity curriculum while PPO reaches only the first level, and the Simple ALU shows the same pattern; the ablation that skips the curriculum lets PPO solve 6-parity directly, which the authors take to mean PPO's failure is an inability to progress through non-stationary levels rather than a capacity limit. In the ecorobot stepping-stones maze, NEAT crosses on average five of seven stones while PPO and goal-conditioned PPO stop at the first. The authors also claim the transfer advantage is specific to direct encodings: HyperNEAT, the indirect encoding, loses its edge and instead avoids local optima that trap PPO, while MAP-Elites explores the whole maze but fails to collect stones in order.","pith_inferences":["One implication the paper leaves implicit: NEAT's advantage may come from preserving early modules in its evolving genome, so a direct test would be to inspect whether the networks that solve later levels literally contain the sub-circuits from earlier ones.","A testable extension: apply the same curriculum logic to non-policy domains, such as incremental language tasks or multi-step tool use, where PPO is not the dominant baseline; if NEAT still wins, the effect is not about control tasks but about curricula generally.","The HyperNEAT result suggests a natural hybrid: use an indirect encoding to jump out of local optima, then switch to a direct encoding to accumulate the discovered skills; the paper hints at this in its discussion of evo-devo but does not test it.","The benchmarks could serve as a standard measurement axis separating 'transfer under curriculum' from 'deception' and 'locomotion', which are currently conflated in many NE evaluations."],"forward_implications":["On curriculum-shaped tasks, NEAT and CMA-ES become a better default than PPO, with NEAT reaching five of seven stepping stones where PPO stalls at the first.","Choosing a neuroevolution encoding trades transfer for exploration: direct encodings transfer skills, while the indirect HyperNEAT escapes local optima but fails to build on earlier levels.","Removing the curriculum flips the comparison: PPO solves 6-parity directly, while NEAT's performance degrades slightly, so the curriculum is what separates the methods.","The transfer advantage does not yet scale: swapping SimpleRob for the ant in the stepping-stones maze makes NEAT stop at the first stone, so complex morphology remains an open problem."],"supporting_citations":[{"why":"Defines NEAT, the direct-encoding algorithm whose architecture growth is the main method shown to transfer skills.","marker":"[43]"},{"why":"Defines PPO, the primary RL baseline that stalls at early curriculum levels.","marker":"[38]"},{"why":"Defines HyperNEAT, the indirect encoding whose transfer failure and local-optima success anchor the encoding comparison.","marker":"[42]"},{"why":"Defines MAP-Elites, the diversity-optimization baseline that explores the maze but fails the ordered-stepping task.","marker":"[30]"},{"why":"Defines CMA-ES, the direct-encoding weight-only baseline that solves the logic levels but underperforms NEAT in the maze.","marker":"[15]"},{"why":"Provides the Brax physics engine that ecorobot extends with walls, obstacles, food, and new reward functions.","marker":"[10]"},{"why":"Defines goal-conditioned RL, the variant the paper uses to try to alleviate PPO's non-stationarity problem.","marker":"[5]"},{"why":"Defines curriculum learning, the paradigm the two benchmarks instantiate to test transfer of skills.","marker":"[1]"}],"fun_headline_variants":["Neuroevolution outperforms PPO when tasks scale stepwise","Evolved networks transfer across curriculum, RL fails to keep up","NEAT leaps curriculum steps, PPO stuck at first level","Direct encoding NE transfers, indirect loses edge","Curriculum makes neuroevolution beat reinforcement learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion assumes that passing later levels in these benchmarks actually requires skills carried over from earlier levels, rather than being re-solved from scratch within each level; if later levels are learnable independently with the same effort, the transfer story does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Neuroevolution outperforms PPO when tasks scale stepwise","Evolved networks transfer across curriculum, RL fails to keep up","NEAT leaps curriculum steps, PPO stuck at first level","Direct encoding NE transfers, indirect loses edge","Curriculum makes neuroevolution beat reinforcement learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1371,"prompt_tokens":928,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":367}},"tokens_in":544,"tokens_out":443,"duration_ms":5309,"temperature":1.0,"reasoning_tokens":367,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:09:50.906159+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train PPO directly on the final level of the stepping-stones maze (food reward only, no stones) and on the highest obstacle height, and compare wall-clock time and sample count to NEAT's curriculum-trained runs; if PPO solves the final level with comparable or better efficiency, then NEAT's apparent transfer advantage is not about transferring skills but about curriculum placement.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines goal-conditioned RL, the variant the paper uses to try to alleviate PPO's non-stationarity problem."}],"review_version":1}