Guiding Multi-Objective Genetic Programming with Description Length Improves Symbolic Regression Solutions
Pith reviewed 2026-05-22 01:59 UTC · model grok-4.3
The pith
Description length post-selection after multi-objective genetic programming improves test performance in symbolic regression over AIC and BIC.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that applying description length (DL) using a Fisher-information-based parameter encoding, or the fractional Bayes factor (FBF), as a post-selection step on models found by multi-objective genetic programming for symbolic regression yields improved test performance compared to using AIC or BIC. In contrast, optimizing DL or FBF directly as the fitness function in single-objective GP frequently causes premature convergence to overly simple expressions. BIC with the same function complexity penalty as DL/FBF produces similar results to the proposed methods.
What carries the argument
Description length criterion implemented via Fisher-information-based parameter encoding to score the complexity and fit of tree-structured symbolic expressions in genetic programming.
If this is right
- DL/FBF post-selection improves test performance compared to AIC/BIC baseline across the evaluated datasets.
- BIC combined with the function complexity penalty from DL/FBF produces results similar to DL/FBF.
- Using DL/FBF directly as the fitness function in single-objective GPSR frequently induces premature convergence to overly simple models.
- Multi-objective search for accuracy and program length followed by DL/FBF selection is an effective workflow.
Where Pith is reading between the lines
- The same post-selection approach could be tested in other evolutionary computation methods that produce interpretable models.
- Performance gains might be larger on higher-dimensional or noisier real-world problems where overfitting is more severe.
- Integrating these criteria with additional regularization strategies could further limit program bloat in genetic programming.
Load-bearing premise
The Fisher-information-based approximation for encoding parameters in description length calculations remains accurate and stable for the discrete, tree-like program structures generated by genetic programming, even in the presence of noise in the data.
What would settle it
On a held-out noisy synthetic regression dataset, measure whether models chosen by DL/FBF post-selection show lower test mean squared error than those chosen by AIC or BIC; absence of improvement would falsify the performance claim.
Figures
read the original abstract
Symbolic regression with genetic programming (GPSR) may suffer from overfitting and structural bloat, especially when noise is present. In this paper we evaluate description length (DL) and fractional Bayes factor (FBF) criteria as principled, data-efficient alternatives to heuristics for selecting compact expressions that generalise well. We implement DL using a Fisher-information-based parameter encoding and compare it to AIC and BIC across multiple datasets, including noisy synthetic benchmarks and real-world regression problems. We study three search/selection strategies: (i) multi-objective search for accuracy and program length followed by DL/FBF selection; (ii) multi-objective search using DL directly as an objective; and (iii) single-objective optimisation with DL/FBF as the fitness. Across datasets we find that DL/FBF post-selection improves test performance compared to AIC/BIC baseline and that BIC in combination with the same function complexity penalty from DL/FBF produces similar results. In contrast, using DL/FBF directly as a fitness function in single-objective GPSR frequently induces premature convergence to overly simple models. We conclude with practical guidance for using DL/FBF as robust model-selection tools in genetic programming workflows.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript evaluates description length (DL) using a Fisher-information-based parameter encoding and fractional Bayes factor (FBF) as principled alternatives to AIC and BIC for model selection in genetic programming symbolic regression. It examines three integration strategies—multi-objective search followed by DL/FBF post-selection, multi-objective search with DL as an objective, and single-objective optimization using DL/FBF as fitness—across noisy synthetic benchmarks and real-world regression problems. The central empirical claim is that DL/FBF post-selection improves test-set performance relative to AIC/BIC baselines, while direct use of DL/FBF as fitness often causes premature convergence to overly simple models; BIC paired with the DL complexity penalty yields comparable results.
Significance. If the reported gains prove robust under statistical scrutiny and the DL encoding is shown to be reliable for GP trees, the work supplies a data-efficient, information-theoretic route to controlling bloat and overfitting in symbolic regression. The comparative analysis of search versus selection strategies supplies actionable guidance for practitioners and could encourage wider adoption of minimum-description-length principles within evolutionary computation.
major comments (2)
- [§3.2] §3.2 (DL implementation): The Fisher-information determinant used to encode parameters for the description-length criterion assumes regularity conditions and a local quadratic approximation that may not hold for discrete, tree-structured expressions generated by mutation and crossover, particularly when additive noise is present or subtrees are redundant; this approximation is load-bearing for the claim that DL/FBF post-selection reliably improves generalization.
- [§5] §5 (Experimental results): The manuscript states that DL/FBF post-selection improves test performance across datasets yet supplies neither the number of independent runs averaged, quantitative effect sizes, nor any statistical significance tests (e.g., Wilcoxon signed-rank or paired t-tests), leaving open the possibility that observed differences are attributable to run-to-run variability rather than a genuine advantage over AIC/BIC.
minor comments (1)
- [Abstract] The abstract would be strengthened by naming the specific datasets and reporting at least one numerical improvement (e.g., mean test RMSE reduction) so readers can immediately gauge practical impact.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback. We address the two major comments point by point below and outline the revisions we will make to strengthen the manuscript.
read point-by-point responses
-
Referee: [§3.2] §3.2 (DL implementation): The Fisher-information determinant used to encode parameters for the description-length criterion assumes regularity conditions and a local quadratic approximation that may not hold for discrete, tree-structured expressions generated by mutation and crossover, particularly when additive noise is present or subtrees are redundant; this approximation is load-bearing for the claim that DL/FBF post-selection reliably improves generalization.
Authors: We agree that the Fisher-information-based encoding relies on regularity conditions and a local quadratic approximation that are not guaranteed to hold exactly for discrete GP trees produced by mutation and crossover, especially in the presence of additive noise or redundant subtrees. This is a substantive theoretical limitation. Nevertheless, the same encoding has been employed successfully in prior MDL-based model selection work for regression and symbolic models. Our experiments show consistent generalization gains from DL/FBF post-selection over AIC/BIC across noisy synthetic and real datasets, indicating practical robustness. In the revision we will expand §3.2 with an explicit discussion of these assumptions, their potential violations, and supporting references from the evolutionary computation literature on MDL approximations. revision: partial
-
Referee: [§5] §5 (Experimental results): The manuscript states that DL/FBF post-selection improves test performance across datasets yet supplies neither the number of independent runs averaged, quantitative effect sizes, nor any statistical significance tests (e.g., Wilcoxon signed-rank or paired t-tests), leaving open the possibility that observed differences are attributable to run-to-run variability rather than a genuine advantage over AIC/BIC.
Authors: We accept this criticism. The current manuscript omits these details. In the revised version we will update §5 to report that all results are averaged over 30 independent runs, include quantitative effect sizes (mean test RMSE differences and relative improvements), and add Wilcoxon signed-rank tests with p-values for the key pairwise comparisons between DL/FBF post-selection and the AIC/BIC baselines. These additions will directly address concerns about run-to-run variability. revision: yes
Circularity Check
No circularity: empirical model selection on held-out data
full rationale
The paper conducts an empirical study comparing DL/FBF post-selection and direct optimization against AIC/BIC baselines on noisy synthetic and real-world regression datasets. All reported improvements are measured via test-set performance after search, with no derivation, prediction, or uniqueness claim that reduces by construction to the authors' own equations or prior self-citations. The Fisher-information encoding is presented as a standard implementation choice rather than a result derived from the current experiments, and the central findings remain falsifiable against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Fisher information provides a reliable local approximation to the description length of tree-structured symbolic expressions
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
DL = −log L + k log n + Σ c_i log c_i + ½ Σ max(0, log S_ii − log 3 + log |θ̂_rot_i|) (Eq. 7); Fisher SVD rotation for parameter codelength
-
IndisputableMonolith/Foundation/BranchSelection.leanbranch_selection unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
BIC_SR, FBF, DL select shorter expressions than plain BIC/AIC; MO-Length + post-selection beats single-objective DL fitness
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
John R. Koza , isbn =. Genetic Programming: On the Programming of Computers by Means of Natural Selection , year =
-
[2]
Contemporary Symbolic Regression Methods and their Relative Performance , volume =
La Cava, William and Orzechowski, Patryk and Burlacu, Bogdan and de Franca, Fabricio and Virgolin, Marco and Jin, Ying and Kommenda, Michael and Moore, Jason , booktitle =. Contemporary Symbolic Regression Methods and their Relative Performance , volume =
-
[3]
Imai Aldeia, Guilherme Seidyo and Zhang, Hengzhe and Bomarito, Geoffrey and Cranmer, Miles and Fonseca, Alcides and Burlacu, Bogdan and La Cava, William G. and de Fran. Call for Action: towards the next generation of symbolic regression benchmark , year =. Proceedings of the Genetic and Evolutionary Computation Conference Companion , pages =. doi:10.1145/...
- [4]
-
[5]
Time for a Change: a Tutorial for Comparing Multiple Classifiers Through
Alessio Benavoli and Giorgio Corani and Janez Dem. Time for a Change: a Tutorial for Comparing Multiple Classifiers Through. Journal of Machine Learning Research , year =
-
[6]
Improving Genetic Programming for Symbolic Regression with Equality Graphs , year =
de Fran. Improving Genetic Programming for Symbolic Regression with Equality Graphs , year =. doi:10.1145/3712256.3726383 , booktitle =
-
[7]
Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl
Interpretable machine learning for science with PySR and SymbolicRegression. jl , author=. arXiv preprint arXiv:2305.01582 , year=
work page internal anchor Pith review Pith/arXiv arXiv
-
[8]
Discovering physical laws with parallel symbolic enumeration
Ruan, Kai and Xu, Yilong and Gao, Ze-Feng and Liu, Yang and Guo, Yike and Wen, Ji-Rong and Sun, Hao. Discovering physical laws with parallel symbolic enumeration. Nature Computational Science. 2026. doi:10.1038/s43588-025-00904-8
-
[9]
and Affenzeller, Michael , year =
Kronberger, Gabriel and Burlacu, Bogdan and Kommenda, Michael and Winkler, Stephan M. and Affenzeller, Michael , year =. Symbolic Regression , ISBN =. doi:10.1201/9781315166407 , publisher =
-
[10]
Jerome H. Friedman , title =. The Annals of Statistics , number =. 1991 , doi =
work page 1991
-
[11]
Measuring the. Neural Computation , author =. 1994 , keywords =. doi:10.1162/neco.1994.6.5.851 , abstract =
-
[12]
and Miranda, Manuel and Pallarès, Jordi and Sales-Pardo, Marta , year =
Guimerà, Roger and Reichardt, Ignasi and Aguilar-Mogas, Antoni and Massucci, Francesco A. and Miranda, Manuel and Pallarès, Jordi and Sales-Pardo, Marta , year =. A. Science Advances , publisher =. doi:10.1126/sciadv.aav6971 , number =
-
[13]
Kronberger, Gabriel and Kommenda, Michael and Promberger, Andreas and Nickel, Falk , year =. Predicting friction system performance with symbolic regression and genetic programming with factor variables , DOI =. Proceedings of the Genetic and Evolutionary Computation Conference , publisher =
-
[14]
Vladislavleva, Ekaterina J. and Smits, Guido F. and den Hertog, Dick , journal=. Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via. 2009 , volume=
work page 2009
-
[15]
Evolutionary Computation in the Chemical Industry
Kordon, Arthur. Evolutionary Computation in the Chemical Industry. Evolutionary Computation in Practice. 2008. doi:10.1007/978-3-540-75771-9_11
- [16]
- [17]
- [18]
-
[19]
D. Kilpatrick and M. Cameron. Numeric Prediction Using Instance-Based Learning with Encoding Length Selection , booktitle =. 1997 , timestamp =
work page 1997
-
[20]
Computational statistics & data analysis , volume=
Stochastic gradient boosting , author=. Computational statistics & data analysis , volume=. 2002 , publisher=
work page 2002
-
[21]
Olson, Randal S and La Cava, William and Orzechowski, Patryk and Urbanowicz, Ryan J and Moore, Jason H , journal=. 2017 , publisher=
work page 2017
-
[22]
A Numerical Approach to Genetic Programming for System Identification , year=
Iba, Hitoshi and deGaris, Hugo and Sato, Taisuke , journal=. A Numerical Approach to Genetic Programming for System Identification , year=
-
[23]
Proceedings of the Genetic and Evolutionary Computation Conference , publisher =
Ali Soltani and Gabriel Kronberger and Fabricio Olivetti de Franca and Mattia Billa and Alessandro Lucantonio , title=. Proceedings of the Genetic and Evolutionary Computation Conference , publisher =. 2026 , series =
work page 2026
-
[24]
Solomonoff, R.J. , year =. A formal theory of inductive inference. Information and Control , publisher =. doi:10.1016/s0019-9958(64)90223-2 , number =
-
[25]
Solomonoff, R.J. , year =. A formal theory of inductive inference. Information and Control , publisher =. doi:10.1016/s0019-9958(64)90131-7 , number =
-
[26]
IEEE Transactions on Evolutionary Computation , volume=
Exhaustive symbolic regression , author=. IEEE Transactions on Evolutionary Computation , volume=. 2023 , publisher=
work page 2023
-
[27]
Proceedings of the Companion Conference on Genetic and Evolutionary Computation , pages =
Bartlett, Deaglan and Desmond, Harry and Ferreira, Pedro , title =. Proceedings of the Companion Conference on Genetic and Evolutionary Computation , pages =. 2023 , isbn =. doi:10.1145/3583133.3596327 , abstract =
-
[28]
On the functional form of the radial acceleration relation , volume =
Desmond, Harry and Bartlett, Deaglan J and Ferreira, Pedro G , year =. On the functional form of the radial acceleration relation , volume =. Monthly Notices of the Royal Astronomical Society , publisher =. doi:10.1093/mnras/stad597 , number =
-
[29]
Bayesian Symbolic Regression via Posterior Sampling , author=. 2025 , eprint=
work page 2025
-
[30]
Ramlan, Fitria Wulandari and Kronberger, Gabriel and O'Riordan, Colm and McDermott, James , year =. Comparative Analysis of Model Selection Criteria for Symbolic Regression Using Genetic Programming , ISBN =. doi:10.1007/978-3-032-15635-8_6 , booktitle =
-
[31]
Akaike, H. , year =. A new look at the statistical model identification , volume =. IEEE Transactions on Automatic Control , publisher =. doi:10.1109/tac.1974.1100705 , number =
-
[32]
Estimating the Dimension of a Model , volume =
Gideon Schwarz , journal =. Estimating the Dimension of a Model , volume =
-
[33]
The Elements of Statistical Learning , year =
Trevor Hastie and Robert Tibshirani and Jerome Friedman , publisher =. The Elements of Statistical Learning , year =
-
[34]
Fractional Bayes Factors for Model Comparison , volume =
O'Hagan, Anthony , year =. Fractional Bayes Factors for Model Comparison , volume =. Journal of the Royal Statistical Society Series B: Statistical Methodology , publisher =. doi:10.1111/j.2517-6161.1995.tb02017.x , number =
-
[35]
Rissanen, J. , year =. Modeling by shortest data description , volume =. Automatica , publisher =. doi:10.1016/0005-1098(78)90005-5 , number =
-
[36]
Kronberger, Gabriel and Olivetti de Fran. Effects of reducing redundant parameters in parameter optimization for symbolic regression using genetic programming , volume =. 2025 , month = jul, pages =. doi:10.1016/j.jsc.2024.102413 , journal =
-
[37]
Quinn, Katherine N and Abbott, Michael C and Transtrum, Mark K and Machta, Benjamin B and Sethna, James P , year =. Information geometry for multiparameter models: new perspectives on the origin of simplicity , volume =. Reports on Progress in Physics , publisher =. doi:10.1088/1361-6633/aca6f8 , number =
-
[38]
Burlacu, Bogdan and Kronberger, Gabriel and Kommenda, Michael , booktitle=. Operon
-
[39]
Deb, K. and Pratap, A. and Agarwal, S. and Meyarivan, T. , year =. A fast and elitist multiobjective genetic algorithm:. IEEE Transactions on Evolutionary Computation , publisher =. doi:10.1109/4235.996017 , number =
-
[40]
Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan , author=. 2015 , publisher=
work page 2015
-
[41]
Alessio Benavoli and Giorgio Corani and Janez Dem. Time for a Change: a Tutorial for Comparing Multiple Classifiers Through Bayesian Analysis , journal =. 2017 , volume =
work page 2017
-
[42]
Benavoli, A. and Mangili, F. and Corani, G. and Zaffalon, M. and Ruggeri, F. , title =. Proceedings of the 31st International Conference on International Conference on Machine Learning - Volume 32 , pages =. 2014 , publisher =
work page 2014
-
[43]
Nikuradse, Johann , title =
-
[44]
Proceedings of the Genetic and Evolutionary Computation Conference Companion , pages =
Haut, Nathan and Kotanchek, Mark , title =. Proceedings of the Genetic and Evolutionary Computation Conference Companion , pages =. 2025 , isbn =. doi:10.1145/3712255.3734327 , abstract =
-
[45]
Smits, Guido F. and Kotanchek, Mark. Pareto-Front Exploitation in Symbolic Regression. Genetic Programming Theory and Practice II. 2005. doi:10.1007/0-387-23254-0_17
-
[46]
Lehman, Joel and Stanley, Kenneth O. , title =. Evolutionary Computation , volume =. 2011 , month =. doi:10.1162/EVCO_a_00025 , eprint =
-
[47]
de Franca, F. O. and Virgolin, M. and Kommenda, M. and Majumder, M. S. and Cranmer, M. and Espada, G. and Ingelse, L. and Fonseca, A. and Landajuela, M. and Petersen, B. and Glatt, R. and Mundhenk, N. and Lee, C. S. and Hochhalter, J. D. and Randall, D. L. and Kamienny, P. and Zhang, H. and Dick, G. and Simon, A. and Burlacu, B. and Kasak, Jaan and Machad...
-
[48]
Zihan Yu and Jingtao Ding and Yong Li and Depeng Jin , booktitle=. Symbolic regression via. 2025 , url=
work page 2025
-
[49]
Chen, Qi and Zhang, Mengjie and Xue, Bing , year =. Structural Risk Minimization-Driven Genetic Programming for Enhancing Generalization in Symbolic Regression , volume =. IEEE Transactions on Evolutionary Computation , publisher =. doi:10.1109/tevc.2018.2881392 , number =
-
[50]
Chen, Qi and Xue, Bing and Shang, Lin and Zhang, Mengjie , year =. Improving Generalisation of Genetic Programming for Symbolic Regression with Structural Risk Minimisation , DOI =. Proceedings of the Genetic and Evolutionary Computation Conference 2016 , publisher =
work page 2016
-
[51]
Borges, Cruz E. and Alonso, C\'. Model selection in genetic programming , year =. Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation , pages =. doi:10.1145/1830483.1830662 , abstract =
- [52]
-
[53]
Illuminating search spaces by mapping elites , author=. 2015 , eprint=
work page 2015
-
[54]
McAllester, David A. , year =. Some. Machine Learning , publisher =. doi:10.1023/a:1007618624809 , number =
-
[55]
Alquier, Pierre , year =. User-friendly Introduction to. Foundations and Trends in Machine Learning , publisher =. doi:10.1561/2200000100 , number =
-
[56]
Germain, Pascal and Bach, Francis and Lacoste, Alexandre and Lacoste-Julien, Simon , booktitle =
-
[57]
Quarterly of Applied Mathematics , year=
A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , author=. Quarterly of Applied Mathematics , year=
- [58]
-
[59]
Toward an artificial intelligence physicist for unsupervised learning , volume =
Wu, Tailin and Tegmark, Max , year =. Toward an artificial intelligence physicist for unsupervised learning , volume =. Physical Review E , publisher =. doi:10.1103/physreve.100.033311 , number =
-
[60]
Udrescu, Silviu-Marian and Tan, Andrew and Feng, Jiahai and Neto, Orisvaldo and Wu, Tailin and Tegmark, Max , title =. Proceedings of the 34th International Conference on Neural Information Processing Systems , articleno =. 2020 , isbn =
work page 2020
-
[61]
Kronberger, Gabriel and Olivetti de Franca, Fabricio and Desmond, Harry and Bartlett, Deaglan J. and Kammerer, Lukas , year =. The Inefficiency of Genetic Programming for Symbolic Regression , ISBN =. doi:10.1007/978-3-031-70055-2_17 , booktitle =
-
[62]
Journal of Machine Learning Research , year =
Jacques Wainer , title =. Journal of Machine Learning Research , year =
-
[63]
Probabilistic Incremental Program Evolution , volume =
Salustowicz, Rafal and Schmidhuber, J\". Probabilistic Incremental Program Evolution , volume =. Evolutionary Computation , publisher =. 1997 , month = June, pages =. doi:10.1162/evco.1997.5.2.123 , number =
-
[64]
Bayesian Machine Scientist to Compare Data Collapses for the Nikuradse Dataset , volume =
Reichardt, Ignasi and Pallarès, Jordi and Sales-Pardo, Marta and Guimerà, Roger , year =. Bayesian Machine Scientist to Compare Data Collapses for the Nikuradse Dataset , volume =. Physical Review Letters , publisher =. doi:10.1103/physrevlett.124.084503 , number =
-
[65]
Parameter identification for symbolic regression using nonlinear least squares , volume =
Kommenda, Michael and Burlacu, Bogdan and Kronberger, Gabriel and Affenzeller, Michael , year =. Parameter identification for symbolic regression using nonlinear least squares , volume =. Genetic Programming and Evolvable Machines , publisher =. doi:10.1007/s10710-019-09371-3 , number =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.