Pith. sign in

REVIEW 3 major objections 6 minor 27 references

A New Deterministic Technique for Symbolic Regression

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper claims that symbolic regression can be performed deterministically by growing a single expression tree whose every replacement step is guaranteed to lower the mean squared error, and that this returns test results as good as a…

desk verdict A sound algebraic core for deterministic symbolic regression, overclaimed by a heuristic constant search and an under-powered experiment. read the letter →

arxiv 1908.06754 v4 pith:ES4AV7KZ submitted 2019-08-16 cs.LG stat.ML

classification cs.LGstat.ML
keywords symbolicregressiondeterministicalgorithmexpressiontreesemanticspacemeansquarederrorinterpretablemodelsconstantsearchdivisionbyzero
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a symbolic-regression method that replaces the usual random evolutionary search with a single expression tree that is grown step by step. The central claim is that each replacement of a subtree, whether by a constant, a variable, or a one-operator expression involving one of those, is guaranteed to reduce the mean squared error, so the construction never worsens and no population of candidates is needed. Because the process is deterministic, one run per configuration suffices, and because the user can cap the number of nodes, the returned model is a compact equation a human can inspect rather than a black box or a giant tree. The authors demonstrate the method on the law of universal gravitation, where it rediscovers the known formula, and on the Boston housing benchmark, where its test error is not significantly different from a neural network's in their experiments.

What carries the argument

The carrier of the argument is the assignment to every node of a scalar equation of the form $\frac{1}{N}\sum_{i=1}^N \left(\frac{a_i o_i - b_i}{c_i o_i - d_i}\right)^2$, together with a set $S$ of forbidden output vectors inherited from divisions higher up the tree. In the $N$-dimensional semantic space, one coordinate per data pattern, so each model is a point, these equations turn improvement into geometry: for each node the set of strictly better subtrees is an ellipsoid (or a related shape), and the searches look for constants, variables, and simple one-operation branches inside it. The propagation rules for $+, -, \times, \div$ translate a parent equation into the two child equations and update $S$, so each candidate replacement can be evaluated locally without recomputing the whole tree. The constant search closes the loop analytically in the listed special cases and otherwise falls back to evaluating the zero candidates $z_i=b_i/a_i$.

What would settle it

Run the fallback constant search on a small synthetic node whose error terms are chosen so that the true minimizer of the equation lies far from every zero $b_i/a_i$ and every pole $d_i/c_i$; if a dense one-dimensional search finds an improving constant while all $N$ candidate zeros return negative reductions, then the claim that accepted changes are always improvements is false.

Watch

Extended reading notes

Core claim

The paper's central claim is that symbolic regression can be posed as deterministic descent on a single expression tree. For any node, the contribution of that node to the global mean squared error has the rational form $\frac{1}{N}\sum_{i=1}^N \left(\frac{a_i o_i - b_i}{c_i o_i - d_i}\right)^2$ with coefficient vectors $a_i,b_i,c_i,d_i$; the four arithmetic operations propagate these coefficients down the tree, and each propagation also carries a set $S$ of forbidden semantics that trace division-by-zero conditions to the leaves. Improving the tree means finding, at some node, a substitutable subtree whose semantic lies inside the current error ellipsoid of that node, which is exactly a positive reduction in that equation. The available replacements are constants, variables, one-operation constant-variable forms, and constant-plus-subtree forms; constants are refined by a closed-form minimum whenever the coefficients fall into the listed special cases, and otherwise by a candidate-zero heuristic. The authors conclude that the method can return test results as good as an artificial neural network, in low computational time, with a user-set bound on expression size.

Load-bearing premise

The load-bearing premise is that, in the general case, the fallback constant search, testing only the $N$ zero candidates $z_i=b_i/a_i$ after discarding values near poles, finds a constant that reduces the error whenever any constant reduces it, because if this heuristic misses, the algorithm stops prematurely and the guaranteed-improvement claim collapses.

Editorial extensions

If this is right

  • If the central claim holds, symbolic regression becomes reproducible: the same dataset and parameters always return the same expression, so no averaging over random seeds is needed.
  • The recursion-local evaluation means a bounded-complexity tree can be grown efficiently, and the final model is an ordinary arithmetic expression that can be embedded in spreadsheets or any programming language without a machine-learning library.
  • Because the user can cap node count, the technique offers explicit control of overfitting by expression size, complementing or replacing implicit regularization.
  • The worked gravitation example implies the method can build constants from scratch, including very small or very large ones, rather than relying on random constant generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that its per-step guarantee is a descent property, not a global-optimality property; a natural extension is to run the same deterministic grower from several different initial constants or to stop on a validation set, and measure how often escape from local minima matters.
  • If the semantic-space geometry is as general as the root-equation derivation suggests, the same node-replacement test could be applied to any model that outputs one value per pattern, allowing the tree to assemble ensembles that mix closed-form expressions with opaque predictors.
  • The per-node ellipsoid test also yields a by-product the paper does not exploit: for every variable it reports how close that variable's semantic point is to the current improving region, which could be read as an interpretable feature-relevance score during construction.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a deterministic symbolic regression method that builds a single expression tree over {+,-,*,/}. Starting from the constant equal to the mean target, the algorithm searches at each node for a replacement subtree—a constant, a variable, a constant-variable expression, or a constant-expression expression—that reduces mean squared error (MSE). To make this search local, the paper propagates from the root a rational error expression (Eq. 6) with coefficient vectors a_i,b_i,c_i,d_i, using algebraic rules per operator (Eqs. 7-20), together with a set S of forbidden semantics intended to prevent division by zero in ancestors. The paper claims each executed modification is guaranteed to improve MSE and that the tree grows until it fits the data, and reports experiments on the Boston housing dataset, including a comparison to a multilayer perceptron, concluding performance is statistically indistinguishable. A worked example on Newton's law of gravitation is also given.

Significance. The algebraic core—propagating the MSE functional through arithmetic operators as rational functions and using positivity of Eq. 26 as the acceptance test—is a genuinely interesting and largely correct framework; the special-case constant solutions (Eqs. 29-34) are derived correctly, and the Newton's law example is a convincing demonstration that the method can recover known structure. However, the advertised guarantees are stronger than what is proved: the general-case constant search is a heuristic over finite zero candidates, and the empirical comparison selects hyperparameters on the test set. The paper is transparent about the former in Section 6, but this concession directly contradicts the 'guaranteed improvement' and 'grown until it fits' claims. With the claims appropriately scaled down and the evaluation redone with a validation protocol, the framework could be a useful addition to the symbolic regression literature.

major comments (3)
  1. [Section 2.1, Eqs. (26)-(28), pp. 10-12] The general-case constant search is a heuristic, not an exact solver. In the fallback case, the algorithm evaluates only the N zero candidates z_i=b_i/a_i, excludes values close to poles or forbidden by S, and selects the candidate with lowest Eq. 27. The paper itself notes that 'the minimum value of equation 6 will not be any of these values' and Section 6 lists exact minimization of Eq. 6 for general c_i,d_i as future work. No proof is given that if a constant k exists with positive reduction in Eq. 26, one of the z_i also yields positive reduction. Since every search (constant, constant-variable, constant-expression) and the constant-optimization pass ultimately depends on this constant search, a miss can terminate the entire iterative process prematurely. Consequently, the advertised properties in the Abstract and Section 1—'each change ... guaranteed to improve' and 'grown until it fits the data'—are not established. The narrow claim that executed changes reduce MSE is true by construction when Eq. 26 is positive, but the method's completeness as a search is an unproven heuristic.
  2. [Section 4, Tables 4-5 and the ANN comparison, pp. 21-25] The hyperparameter comparison (minimum MSE reduction, maximum nodes) is made using the test-set MSE, and the 'best test' configurations in Tables 4 and 5 are chosen from the same test folds. The ANN baseline is likewise selected at its best test epoch (epoch 4347 in Table 6). There is no held-out validation set or nested cross-validation. Because model selection and evaluation are done on the same test data, the paired t-test reported on p. 25 does not support the generalization claim that the system 'is able to return test results as good as an ANN.' A validation-based selection procedure or nested cross-validation is necessary to make this claim.
  3. [Section 2, S-set propagation rules, pp. 8-10 and Tables 1-2] The invariant that the propagated set S exactly captures all ancestor division-by-zero domains is only illustrated by examples, not proved. The rules involving Inf and NaN (e.g., 's_i=Inf/y_i=NaN when y_i=0' and 's_i=x_i/Inf=0') are stated informally. Since the guarantee that the tree 'will always be correct, with no divisions by 0' (p. 3) and the validity of every semantic check depend on this invariant, the authors should supply a formal inductive proof for the four operators, including the handling of Inf/NaN, or provide a counterexample and restrict the claims accordingly.
minor comments (6)
  1. [Section 2, root initialization] There is an inconsistency between the initial value of d_i: the text near Eq. 3 sets d_i=-1, while the paragraph introducing the S set (p. 8) sets d_i=1. Since the MSE is squared, both yield the same value at the root, but the sign matters in the propagation rules; please make this consistent.
  2. [Table 3] The exponent of the constant selected in iteration 4 (3.932029293203675e19) appears inconsistent with the exponent 1e-19 in the iteration 3 result; please check and correct the table.
  3. [Section 2.1, zero-candidate step] The computation of z_i=b_i/a_i does not specify the behavior when a_i=0, nor how to handle c_i=0 when computing poles d_i/c_i; please define these cases explicitly.
  4. [Section 2.8, reproducibility] The manuscript states that the source code 'will be provided,' but no repository or link is included in this version; please include it to support reproducibility.
  5. [Section 4, empirical scope] The empirical evaluation uses a single dataset (Boston housing); the conclusion that the method is 'as good as an ANN' would be considerably more convincing with additional benchmark datasets.
  6. [Section 2, Eq. (23)] The expression x3− 2/(2·x3− 1) is ambiguous without parentheses; please insert parentheses to clarify the intended precedence.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: the derivation chain is self-contained algebra from the MSE definition, and the experimental claims rest on external benchmarks.

full rationale

The paper's derivation starts from the definition of MSE (Eq. 3) and propagates per-node equations (Eq. 6) through the arithmetic operators using algebraic identities (Eqs. 7-22). The special-case constant minimizers (Eqs. 29-34) are obtained by differentiating Eq. 28 and solving the resulting closed-form conditions, and the initial constant is derived from Eq. 35 as the mean target value. Every accepted tree replacement is explicitly required to produce a positive value in Eq. 26, so the claim that executed changes improve the MSE is verified by construction rather than being a fitted result renamed as a prediction. The only notable gap is in Section 2.1's general-case constant search, where the paper states that the true minimum 'will not be any of these values' and instead evaluates a finite set of zero candidates; this is an unproven completeness heuristic, not a circularity, because the algorithm still checks Eq. 26 before accepting a change and does not define its output in terms of the benchmark target. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation; the Newton-law example is generated from the known law and the Boston-housing comparison is external empirical evaluation. Therefore the derivation is self-contained and no circular step is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The axioms capture the main unproved assumptions: the rational error form, the S-set domain tracking, the approximate constant search, and the greedy search convergence. Free parameters are the user-chosen hyperparameters plus the unstated pole-proximity threshold, several of which were tuned on the test set.

free parameters (5)
  • Minimum MSE improvement threshold (delta) = 10^-6 (selected after grid search)
    Hyperparameter controlling when a search counts as successful; grid-searched over 10^-1 to 10^-10 and selected based on test-set performance in Section 4.
  • Maximum number of nodes (N_max) = 20 to 200 in grid; best test around 105-200
    Complexity limit; grid-searched on the Boston housing test set.
  • Search strategy = Strategies 1-4, best 3 and 4
    High-level choice of which searches run in which order; selected from results on the same test set.
  • Candidate zero closeness threshold for constant search = Not specified
    In Section 2.1, values 'too close' to poles are excluded; no quantitative threshold is given, an implicit free parameter.
  • ANN hyperparameters (comparison baseline) = 2 hidden layers (10,3), Adam lr=0.01, 50000 epochs
    The MLP baseline is tuned by the authors; its best test epoch is selected from the test set, another selection on test data.
assumptions (5)
  • domain assumption The Euclidean distance (RSSE) in semantic space is an appropriate loss; minimizing it is equivalent to minimizing MSE.
    Section 2, Eqs. 1-3. This is a standard regression loss choice, not an ad hoc assumption.
  • standard math Every node's contribution to the root MSE can be represented by the rational form (a_i o_i - b_i)/(c_i o_i - d_i), and the propagation rules preserve this form.
    Section 2, Eqs. 6-22. Algebraically standard for +,-,*,/ but relies on exact arithmetic; denominators may be zero.
  • ad hoc to paper The set S of prohibited semantics correctly captures all division-by-zero domains of ancestor nodes under the propagation rules.
    Section 2, 'Special care has to be taken...' The NaN/Inf arithmetic is invented for this paper; its correctness is assumed, not proved.
  • ad hoc to paper The approximate constant search over zero candidates b_i/a_i (after excluding poles) finds a positive-reduction constant whenever one exists.
    Section 2.1. This is an unproven heuristic; if the true optimal constant is not near a zero candidate, the algorithm may stop prematurely.
  • domain assumption Greedy single-node substitutions with positive local MSE reduction eventually produce a globally good expression.
    Section 2.8 and the search process. There is no proof of convergence or of global optimality; local search may get stuck in local optima.
invented entities (1)
  • S-set with Inf/NaN markers
    purpose: Tracking values that would lead to division by zero in ancestor nodes of the expression tree.
    Introduced in Section 2 to keep constructed trees in the domain of all division operators. It is an internal algorithmic structure with no external falsifiable handle.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A New Deterministic Technique for Symbolic Regression." pith.science (2026). https://pith.science/paper/ES4AV7KZ

@misc{pith2026190806754,
  author       = {Pith},
  title        = {Pith review of: A New Deterministic Technique for Symbolic Regression},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES4AV7KZ}},
  note         = {Machine review of arXiv:1908.06754}
}
read the original abstract

This paper describes a new method for Symbolic Regression that allows to find mathematical expressions from a dataset. This method has a strong mathematical basis. As opposed to other methods such as Genetic Programming, this method is deterministic, and does not involve the creation of a population of initial solutions. Instead of it, a simple expression is being grown until it fits the data. The experiments performed show that the results are as good as other Machine Learning methods, in a very low computational time. Another advantage of this technique is that the complexity of the expressions can be limited, so the system can return mathematical expressions that can be easily analysed by the user, in opposition to other techniques like GSGP.

Figures

Figures reproduced from arXiv: 1908.06754 by the authors.

Figure 1
Figure 1. Example of a tree distance = vuutX N i=1 (oi − ti) 2 (1) This equation corresponds to the square root of the SSE (Sum Squared Error) of the model. Thus, finding a better model (i.e., a model with a lower SSE) is equivalent to finding a model closer to the target point. Therefore, a N-dimensional sphere is being created, and any model inside this sphere will be a better model (i.e., with a lower SSE). This sphere has… view at source ↗
Figure 2
Figure 2. Example of N-dimensional sphere generation [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of calculating new shapes for the sum operation (a), and multiplication operation (b) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Example of a tree with invalid operations [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Examples of a) constant search, b) variable search, c) constant-variable search, d) constant-expression search [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: MSE results in training and test for the housing problem [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: MSE results in training and test for the housing problem [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Averages heights for the housing problem [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: Average time for the housing problem 20 40 60 80 100 0 10 20 30 40 Strategy 1 Strategy 2 Strategy 3 Strategy 4 Maximum number of nodes Tim e Time with minimum reduction 1.0e-6 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Average computational time for different maximum number of nodes values and a minimum MSE reduction [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Results of the MLP training process usual, while the training results are still improving, the test results reach a point in which begin to worsen. Finally, the training process is stopped when the maximum number of epochs is reached. In this point, the ANN has overfi…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 25 canonical work pages

  1. [1]

    Simon S. Haykin. Neural networks and learning machines. Pearson Education, Upper Saddle River, NJ, third edition, 2009

  2. [2]

    John R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA, 1992

  3. [3]

    Langdon, and Nicholas Freitag McPhee

    Riccardo Poli, William B. Langdon, and Nicholas Freitag McPhee. A Field Guide to Genetic Programming. Lulu Enterprises, UK Ltd, 2008

  4. [4]

    Springer US, Boston, MA, 2007

    Basic Concepts of Linear Genetic Programming, pages 13–34. Springer US, Boston, MA, 2007

  5. [5]

    T. Perkis. Stack-based genetic programming. In Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence, pages 148–153 vol.1, June 1994

  6. [6]

    Cartesian genetic programming

    Julian Miller and Andrew Turner. Cartesian genetic programming. In Proceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation , GECCO Companion ’15, pages 179–198, New York, NY , USA, 2015. ACM

  7. [7]

    Positional Cartesian Genetic Programming

    Dennis G. Wilson, Julian F. Miller, Sylvain Cussat-Blanc, and Hervé Luga. Positional cartesian genetic program- ming. CoRR, abs/1810.04119, 2018

  8. [8]

    On crossover success rate in genetic programming with offspring selection

    Gabriel Kronberger, Stephan Winkler, Michael Affenzeller, and Stefan Wagner. On crossover success rate in genetic programming with offspring selection. In Leonardo Vanneschi, Steven Gustafson, Alberto Moraglio, Ivanoe De Falco, and Marc Ebner, editors, Genetic Programming, pages 232–243, Berlin, Heidelberg, 2009. Springer Berlin Heidelberg

Show all 27 references
  1. [9]

    Medial crossovers for genetic programming

    Krzysztof Krawiec. Medial crossovers for genetic programming. In Alberto Moraglio, Sara Silva, Krzysztof Krawiec, Penousal Machado, and Carlos Cotta, editors, Genetic Programming, pages 61–72, Berlin, Heidelberg,

  2. [10]

    McConaghy and G

    T. McConaghy and G. G. E. Gielen. Template-free symbolic performance modeling of analog circuits via canonical-form functions and genetic programming. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 28(8):1162–1175, Aug 2009

  3. [11]

    Applying genetic programming to civil engineering in the improvement of models, codes and norms

    Juan Pérez, Mónica Miguélez Rico, Juan Rabuñal, and Fernando Abella. Applying genetic programming to civil engineering in the improvement of models, codes and norms. pages 452–460, 10 2008

  4. [12]

    Alberto Moraglio, Krzysztof Krawiec, and Colin G. Johnson. Geometric semantic genetic programming. In Carlos A. Coello Coello, Vincenzo Cutello, Kalyanmoy Deb, Stephanie Forrest, Giuseppe Nicosia, and Mario Pavone, editors, Parallel Problem Solving from Nature - PPSN XII, page...

  5. [13]

    Geometric semantic genetic programming is overkill

    Tomasz Pawlak. Geometric semantic genetic programming is overkill. volume 9594, 03 2016

  6. [14]

    Joao Francisco B. S. Martins, Luiz Otávio Vilas Boas Oliveira, Luis Fernando Miranda, Felipe Casadei, and Gisele L. Pappa. Solving the exponential growth of symbolic regression trees in geometric semantic genetic programming. CoRR, abs/1804.06808, 2018

  7. [15]

    Geometric semantic genetic programming for financial data

    James Mcdermott, Alexandros Agapitos, Anthony Brabazon, and Michael O’Neill. Geometric semantic genetic programming for financial data. pages 215–226, 04 2014

  8. [16]

    Z. Zhu, A. K. Nandi, and M. W. Aslam. Adapted geometric semantic genetic programming for diabetes and breast cancer classification. In 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–5, Sep. 2013

  9. [17]

    FFX: Fast, Scalable, Deterministic Symbolic Regression Technology, pages 235–260

    Trent Mcconaghy. FFX: Fast, Scalable, Deterministic Symbolic Regression Technology, pages 235–260. 01 2011

  10. [18]

    Q. Chen, B. Xue, and M. Zhang. Generalisation and domain adaptation in gp with gradient descent for symbolic regression. In 2015 IEEE Congress on Evolutionary Computation (CEC), pages 1137–1144, May 2015

  11. [19]

    Predicting friction system performance with symbolic regression and genetic programming with factor variables

    Gabriel Kronberger, Michael Kommenda, Andreas Promberger, and Falk Nickel. Predicting friction system performance with symbolic regression and genetic programming with factor variables. pages 1278–1285, 07 2018

  12. [20]

    The average height of binary trees and other simple trees

    Philippe Flajolet and Andrew Odlyzko. The average height of binary trees and other simple trees. Journal of Computer and System Sciences, 25(2):171 – 213, 1982

  13. [21]

    Joo Manuel Paiva Cardoso, Jos Gabriel de Figueiredo Coutinho, and Pedro C. Diniz. Embedded Computing for High Performance: Efficient Mapping of Computations Using Customization, Code Transformations and Compilation. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1st e...

  14. [22]

    Julia: A fresh approach to numerical computing

    Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65–98, 2017

  15. [23]

    I. Newton. Philosophiae naturalis principia mathematica. J. Societatis Regiae ac Typis J. Streater, 1687

  16. [24]

    Harrison and Daniel L

    Otto R. Harrison and Daniel L. Rubinfeld. Hedonic prices and the demand for clean air. 1978

  17. [25]

    Python Machine Learning

    Sebastian Raschka. Python Machine Learning. Packt Publishing, Birmingham, UK, 2015

  18. [26]

    Snyman and D.N

    J.A. Snyman and D.N. Wilke. Practical Mathematical Optimization: Basic Optimization Theory and Gradient- Based Algorithms. Springer Optimization and Its Applications. Springer International Publishing, 2018. 29

  19. [2012]

    Springer Berlin Heidelberg

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.