Pith. sign in

REVIEW 2 major objections 1 minor 29 references

Improving Evaluation of Recombination-based Cartesian Genetic Programming

T0 review · 2 major / 1 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hyperparameter optimization improves performance of recombination-based Cartesian Genetic Programming.

desk verdict Hyperparameter tuning revives some recombination operators in CGP, but the evaluation setup needs scrutiny for overfitting. read the letter →

arxiv 2605.28353 v1 pith:S23LTU63 submitted 2026-05-27 cs.NE cs.AIcs.SC

Duy Long Tran , Anja Jankovic , Marie Anastacio , Holger Hoos , Roman Kalkreuth This is my paper · ORCID
classification cs.NEcs.AIcs.SC
keywords CartesianGeneticProgrammingrecombinationoperatorshyperparameteroptimizationsymbolicregressionSRBenchsubgraphcrossoverphenotypicevolutionaryalgorithms
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper examines two recombination operators, subgraph crossover and discrete phenotypic recombination, in Cartesian Genetic Programming on the SRBench symbolic regression platform. It applies hyperparameter optimization to the representations that use these operators within the TinyverseGP framework. Results show performance gains relative to earlier evaluations that avoided recombination. A reader would care because the work suggests that long-standing reliance on mutation alone may reflect untuned parameters rather than fundamental limits of the recombination methods.

What carries the argument

Hyperparameter optimization of representations that employ subgraph crossover and discrete phenotypic recombination in Cartesian Genetic Programming.

What would settle it

A replication that performs the identical hyperparameter optimization on the same two operators but records no performance improvement on SRBench, or that shows the chosen parameters overfit the benchmark data.

Watch

Extended reading notes

Core claim

Our work demonstrates that hyperparameter optimisation can lead to improvements in performance for recombination-based Cartesian Genetic Programming. This is achieved by testing subgraph crossover and discrete phenotypic recombination on SRBench after tuning hyperparameters for the respective representations using the TinyverseGP implementations.

Load-bearing premise

The hyperparameter optimization was performed without selection bias or overfitting and the TinyverseGP implementations faithfully represent the two recombination operators under test.

Editorial extensions

If this is right

  • Recombination operators can deliver performance gains in Cartesian Genetic Programming once hyperparameters are tuned.
  • Earlier conclusions against recombination may have rested on evaluations that did not optimize the underlying representations.
  • Symbolic regression tasks on SRBench can benefit from the tuned recombination-based variants.
  • The TinyverseGP framework provides usable implementations for conducting such operator-specific tuning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same optimization approach could be tested on other genetic programming variants or benchmark suites to check generality.
  • Future comparisons of mutation versus recombination in CGP should include hyperparameter tuning for both to avoid biased results.
  • If the gains persist under stricter validation, hybrid mutation-plus-recombination schedules may become standard in CGP practice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The manuscript evaluates two recombination operators (subgraph crossover and discrete phenotypic recombination) for Cartesian Genetic Programming on the SRBench symbolic regression benchmark, using TinyverseGP implementations. It claims that hyperparameter optimization of the respective representations with these operators produces performance improvements, challenging the traditional view that recombination yields no gains in CGP.

Significance. If substantiated with proper controls, the result would indicate that recombination-based CGP can be competitive when hyperparameters are tuned, potentially broadening the set of viable genetic operators in the field and motivating further operator development.

major comments (2)
  1. [Abstract] Abstract: the central claim that hyperparameter optimisation 'can lead to improvements in performance' is asserted without any reported data, tables, statistical tests, baseline comparisons, or experimental details, so the demonstration cannot be evaluated from the text.
  2. [Experimental procedure] Experimental procedure (hyperparameter optimisation section): no information is given on whether tuning used a held-out validation split, cross-validation, or was performed directly on the final SRBench test problems; without this, any reported gains are at risk of reflecting selection bias or overfitting rather than genuine operator improvement.
minor comments (1)
  1. The implementations are taken from TinyverseGP; the manuscript should explicitly confirm that these match the operator definitions in the cited source papers to ensure the comparison is faithful.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments and the opportunity to clarify our work. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that hyperparameter optimisation 'can lead to improvements in performance' is asserted without any reported data, tables, statistical tests, baseline comparisons, or experimental details, so the demonstration cannot be evaluated from the text.

    Authors: We acknowledge that the abstract is concise and does not embed specific numerical results or statistical details. The full manuscript reports these elements in the results section, including performance tables, baseline comparisons on SRBench, and statistical tests. We will revise the abstract to briefly reference the key observed improvements and their statistical support for better self-containment. revision: yes

  2. Referee: [Experimental procedure] Experimental procedure (hyperparameter optimisation section): no information is given on whether tuning used a held-out validation split, cross-validation, or was performed directly on the final SRBench test problems; without this, any reported gains are at risk of reflecting selection bias or overfitting rather than genuine operator improvement.

    Authors: We agree this detail should have been explicit. Hyperparameter tuning was performed on held-out validation splits drawn from the SRBench problems (separate from the final test sets) to mitigate overfitting risk. We will revise the hyperparameter optimisation section to document the exact procedure, including validation split usage and any cross-validation steps employed. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical hyperparameter study with no derivation chain

full rationale

The paper reports an empirical study applying hyperparameter optimization to two recombination operators in Cartesian Genetic Programming and evaluating on SRBench. The abstract and description contain no equations, no fitted parameters presented as predictions, and no load-bearing self-citations or uniqueness theorems. The central claim (that hyperparameter optimization yields performance improvements) is an experimental outcome, not a quantity that reduces to its own inputs by construction. No steps match any of the enumerated circularity patterns.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract provides no information on free parameters, axioms or invented entities. The central claim rests entirely on an empirical observation from hyperparameter optimization experiments whose details are not supplied.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Evaluation of Recombination-based Cartesian Genetic Programming." pith.science (2026). https://pith.science/paper/S23LTU63

@misc{pith2026260528353,
  author       = {Pith},
  title        = {Pith review of: Improving Evaluation of Recombination-based Cartesian Genetic Programming},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S23LTU63}},
  note         = {Machine review of arXiv:2605.28353}
}
read the original abstract

Cartesian Genetic Programming has traditionally been using mutation as its main and often sole genetic operator to drive evolutionary search. Despite advancements in recent years, recombinationbased approaches have long been avoided, due to apparent lack of performance gains. This study examines two recently suggested recombination-based operators, subgraph crossover and discrete phenotypic recombination on SRBench, a benchmarking platform for symbolic regression. Using the implementations provided in the TinyverseGP framework, we perform hyperparameter optimisation of the respective representations with these two operators. Our work demonstrates that hyperparameter optimisation can lead to improvements in performance for recombination-based Cartesian Genetic Programming.

Figures

Figures reproduced from arXiv: 2605.28353 by the authors.

Figure 1
Figure 1. Exemplification of the CGP encoding. performance of CGP with discrete phenotypic recombination, in￾cluding as a baseline the mutation-only CGP variant (i.e., without crossover). We observed that the median performance of discrete phenotypic recombination is better than that of subgraph crossover when configured for the symbolic regression problems that we considered. While previous approaches have relied mostly on r… view at source ↗
Figure 2
Figure 2. Median fitness of the best found configuration [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 3 canonical work pages

  1. [1]

    Malan, Jason H

    T. Bartz-Beielstein, C. Doerr, D. van den Berg, J. Bossek, S. Chandrasekaran, T. Ef- timov, A. Fischbach, P. Kerschke, W. La Cava, M. López-Ibáñez, K. M. Malan, J. H. Moore, B. Naujoks, P. Orzechowski, V. Volz, M. Wagner, and T. Weise. 2020. Bench- marking in Optimization: Best Practice and Open Issues. arXiv:2007.03488 [cs.NE] Improving Evaluation of Rec...

    work page arXiv 2020

  2. [2]

    Bergstra and Y

    J. Bergstra and Y. Bengio. 2012. Random Search for Hyper-Parameter Optimiza- tion.Journal of Machine Learning Research13 (2012), 281–305

    2012

  3. [3]

    W. G. La Cava, P. Orzechowski, B. Burlacu, F. O. de França, M. Virgolin, Y. Jin, M. Kommenda, and J. H. Moore. 2021. Contemporary Symbolic Regression Methods and their Relative Performance. InProceedings of the 35th International Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks

    2021

  4. [4]

    Clegg, J

    J. Clegg, J. A. Walker, and J. F. Miller. 2007. A New Crossover Technique for Cartesian Genetic Programming. InProceedings of the Genetic and Evolutionary Computation Conference (GECCO’07). 1580–1587

    2007

  5. [5]

    N. L. Cramer. 1985. A Representation for the Adaptive Generation of Simple Sequential Programs. InProceedings of the 1st International Conference on Genetic Algorithms. 183–187

    1985

  6. [6]

    Dickmanns, J

    D. Dickmanns, J. Schmidhuber, and A. Winklhofer. 1987.Der genetische Algorith- mus: Eine Implementierung in Prolog. Fortgeschrittenenpraktikum. Institut für Informatik, Technische Universität Mänchen

    1987

  7. [7]

    P. I. Frazier. 2018. A Tutorial on Bayesian Optimization.CoRRabs/1807.02811 (2018). arXiv:1807.02811

    work page Pith review arXiv 2018

  8. [8]

    R. Garnett. 2023.Bayesian Optimization. Cambridge University Press

    2023

Show all 29 references
  1. [9]

    S Imai Aldeia, H

    G. S Imai Aldeia, H. Zhang, G. Bomarito, M. Cranmer, A. Fonseca, B. Burlacu, W. G. La Cava, and F. O. de França. 2025. Call for Action: Towards the next generation of symbolic regression benchmarks. InProceedings of the Genetic and Evolutionary Computation Conference (GECCO’25...

    2025

  2. [10]

    Kalkreuth

    R. Kalkreuth. 2020. A Comprehensive Study on Subgraph Crossover in Cartesian Genetic Programming. InProceedings of the 12th International Joint Conference on Computational Intelligence (IJCCI 2020). 59–70

    2020

  3. [11]

    Kalkreuth

    R. Kalkreuth. 2022. Towards Discrete Phenotypic Recombination in Cartesian Genetic Programming. InProceedings of the International Conference on Parallel Problem Solving from Nature (PPSN’22). 63–77

    2022

  4. [12]

    Kalkreuth, F

    R. Kalkreuth, F. O. de França, J. Dierkes, M. Anastacio, A. Jankovic, Z. Vasícek, and H. H. Hoos. 2025. TinyverseGP: Towards a Modular Cross-domain Bench- marking Framework for Genetic Programming. InProceedings of the Genetic and Evolutionary Computation Conference Companion ...

    2025

  5. [13]

    Kalkreuth, G

    R. Kalkreuth, G. Rudolph, and A. Droschinsky. 2017. A New Subgraph Crossover for Cartesian Genetic Programming. InGenetic Programming - 20th European Con- ference, EuroGP 2017, Amsterdam, The Netherlands, April 19-21, 2017, Proceedings, Vol. 10196. 294–310

    2017

  6. [14]

    Kocherovsky and W

    M. Kocherovsky and W. Banzhaf. 2024. Crossover Destructiveness in Cartesian versus Linear Genetic Programming. InProceedings of the Artificial Life Conference (ALIFE 2024), Vol. ALIFE 2024: Proceedings of the 2024 Artificial Life Conference

    2024

  7. [15]

    J. R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection.MA: MIT Press.(1992), 1–840

    1992

  8. [16]

    Kronberger, B

    G. Kronberger, B. Burlacu, M. Kommenda, S. M. Winkler, and M. Affenzeller. 2024. Basics of Symbolic Regression. InSymbolic Regression. 21–34

    2024

  9. [17]

    Niklas Lavesson and Paul Davidsson. 2006. Quantifying the Impact of Learning Algorithm Parameter Tuning. InProceedings of the 21st National Conference on Artificial Intelligence(AAAI). 395–400

    2006

  10. [18]

    Lindauer, K

    M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, D. Deng, C. Benjamins, T. Ruhkopf, R. Sass, and F. Hutter. 2022. SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization.Journal of Machine Learning Research 23 (2022), 1–9

    2022

  11. [19]

    S. Luke. 1998. ECJ Evolutionary Computation Library. http://cs.gmu.edu/$\ sim$eclab/projects/ecj/

    1998

  12. [20]

    J. F. Miller. 2011. Cartesian Genetic Programming. InCartesian Genetic Program- ming. 17–34

    2011

  13. [21]

    J. F. Miller and P. Thomson. 2000. Cartesian Genetic Programming. InGenetic Programming, European Conference, Edinburgh, Scotland, UK, April 15-16, 2000, Proceedings, Vol. 1802. 121–132

    2000

  14. [22]

    Q. U. Nguyen, X. H. Nguyen, M. O’Neill, R. I. McKay, and E. Galván López. 2011. Semantically-based Crossover in Genetic Programming: Application to Real- Valued Symbolic Regression.Genetic Programming and Evolvable Machines12, 2 (2011), 91–119

    2011

  15. [23]

    R. S. Olson, W. La Cava, P. Orzechowski, R. J. Urbanowicz, and J. H. Moore

  16. [24]

    PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison.BioData Mining10, 1 (2017), 1–36

    2017

  17. [25]

    J. D. Romano, T. T. Le, W. La Cava, J. T. Gregg, D. J. Goldberg, P. Chakraborty, N. L. Ray, D. Himmelstein, W. Fu, and J. H. Moore. 2021. PMLB v1.0: An Open Source Dataset Collection for Benchmarking Machine Learning Methods.arXiv preprint arXiv:2012.00058v2(2021), 1–4

    2021

  18. [26]

    E. O. Scott and S. Luke. 2019. ECJ at 20: Toward a General Metaheuristics Toolkit. InProceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2019, Prague, Czech Republic, July 13-17, 2019. 1391–1398

    2019

  19. [27]

    Snoek, H

    J. Snoek, H. Larochelle, and R. P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. InProceedings of the 26th Conference on Advances in Neural Information Processing Systems (NeurIPS), Vol. 25

    2012

  20. [28]

    Vladislavleva, G

    E. Vladislavleva, G. Smits, and D. den Hertog. 2009. Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming.IEEE Transactions on Evolutionary Computation13, 2 (2009), 333–349

    2009

  21. [29]

    D. R. White, J. McDermott, M. Castelli, L. Manzoni, B. W. Goldman, G. Kron- berger, W. Jaskowski, U.-M. O’Reilly, and S. Luke. 2013. Better GP Benchmarks: Community Survey Results and Proposals.Genetic Programming and Evolvable Machines14, 1 (2013), 3–29. GECCO Companion ’26, ...

    2013

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.