Pith. sign in

REVIEW 3 major objections 5 minor 27 references

DeepPySR -- A Symbolic Regression Framework with Dynamic Pruning, Pareto Selection, and Hierarchical Composition for Real-World Scientific Discovery

T0 review · 3 major / 5 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DeepPySR turns evolutionary symbolic regression into a practical tool for high-dimensional medical and social data by pruning features during search, scoring formulas with an exponential accuracy-complexity rule, and stacking layers to reso

desk verdict Solid PySR extension for high-d biomedical SR; the real soft spot is optimistic multi-config selection without held-out data, not the algorithms themselves. read the letter →

arxiv 2607.08150 v1 pith:ML56KCO7 submitted 2026-07-09 cs.LG

Fuling Chen , Kevin Vinsen , Phillip Melton , Rae-Chi Huang This is my paper · ORCID
classification cs.LG
keywords symbolicregressioninterpretablemachinelearningdynamicvariablepruningParetoselectionhierarchicalcompositionbiomedicalpredictionmulticollinearityclassimbalance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Symbolic regression promises glass-box equations instead of black-box models, but it usually fails on real biomedical and social datasets that are high-dimensional, multicollinear, or class-imbalanced. DeepPySR extends evolutionary symbolic regression with three coordinated pieces: a schedule that ramps up pruning of irrelevant variables as search progresses, an exponential score that picks one formula from the Pareto front by jointly rewarding accuracy and penalizing complexity, and a multi-layer architecture that discovers hierarchical intermediate expressions. On physics recovery tasks and seven real-world datasets, the method reports higher accuracy than plain PySR and conventional baselines while returning short formulas that match known clinical and social risk factors. A sympathetic reader would care because the approach aims to make transparent, auditable equations usable where post-hoc explanations of black boxes are unstable or insufficient.

What carries the argument

The Dynamic Variable Pruning Schedule (DVPS) is a mutation whose weight ramps linearly with iteration, replacing variable leaves with constants so the population self-organizes around causal features; Exponential Pareto Selection (EPS) then scores hall-of-fame equations by R^{2}^ρ · exp(−λ(c−1)); multi-layer composition discovers latent symbolic sub-expressions that resolve intraclass correlation.

What would settle it

Re-run the same 27-configuration grid and multi-layer pipeline on an independent external cohort for body fat, heart disease, or Raine BMI; if the R^{2}/F1 gaps versus PySR and tree ensembles disappear or reverse under identical operator sets and fold structure, the claimed superiority of DVPS/EPS/multi-layer fails.

Watch

Extended reading notes

Core claim

Integrating dynamic variable pruning during evolution, exponential Pareto selection of a single formula, and multi-layer hierarchical composition lets symbolic regression handle high-dimensional, multicollinear, and imbalanced real-world data and recover interpretable equations that match or beat both vanilla PySR and standard machine-learning baselines on body-fat, heart-disease, student-performance, and longitudinal BMI prediction tasks.

Load-bearing premise

That five-fold cross-validation over a large internal hyperparameter grid, with no separate held-out cohort on the public datasets, fairly measures how much better the new components generalize than plain PySR.

Editorial extensions

If this is right

  • High-dimensional clinical and social datasets can be searched for short analytic formulas without separate pre-selection of features.
  • A single tunable exponential score can replace manual inspection of Pareto fronts for deploying one communicable equation.
  • Multi-layer symbolic composition can expose gene–environment and trajectory interactions that single-layer SR and black-box models leave opaque.
  • Imbalanced classification tasks can be handled by formula structure alone, without synthetic oversampling, while retaining clinical precision.
  • Domain experts receive equations whose variables and functional form can be checked against known risk factors rather than post-hoc attributions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same pruning-plus-hierarchical pattern could be applied to other observational sciences (epidemiology, ecology, education) where multicollinearity and sparse but causal covariates are the norm.
  • If EPS reliably sits at the knee of the Pareto front, it may serve as a default selection rule for any evolutionary SR engine, not only this codebase.
  • The recovered Raine BMI interaction of polygenic score with early BMI trajectory offers a concrete hypothesis that could be tested for mediation or intervention in other birth cohorts.
  • Fixed layer depth and operator-set dependence remain open limits; automated depth selection or dimensional constraints would be natural next controls.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. DeepPySR extends evolutionary symbolic regression (PySR/SymbolicRegression.jl) with three components aimed at high-dimensional, multicollinear, and imbalanced scientific data: a Dynamic Variable Pruning Schedule (DVPS) that ramps a prune-variable mutation during search (Eqs. 16–17, Alg. 1); Exponential Pareto Selection (EPS) that scores hall-of-fame formulas by R²^ρ·exp(−λ(c−1)) (Eq. 18); and a multi-layer hierarchical composition that discovers latent symbolic subexpressions and substitutes them (Eq. 19). The paper evaluates exact recovery on four Feynman equations and predictive performance plus formula interpretability on seven real-world datasets (body fat, heart disease, wine, student grades, stroke, diabetes, Raine BMI longitudinal with 125 features). Reported gains over PySR include body fat R² 0.794 vs 0.702, heart disease F1 0.898 vs 0.787, student math R² 0.964 vs 0.948, and growing Raine BMI advantage with age, with formulas argued to align with known clinical/epidemiological factors.

Significance. If the gains are attributable to DVPS/EPS/multi-layer rather than extra search effort or optimistic selection, the work would be a useful practical advance for applying evolutionary SR beyond low-d physics benchmarks to biomedical and social data where glass-box equations are valued. Strengths include open code (deeppysr.jl + Python wrapper), multi-dataset evaluation with fold-level means±SE and Wilcoxon tests, convergence curves isolating pruning (Fig. 1), and domain-plausible recovered formulas (e.g., Raine gene–early-BMI interaction, heart-disease risk factors). The integration of pruning inside the evolutionary loop and a tunable, post-hoc Pareto scorer are concrete engineering contributions that address real barriers to SR adoption.

major comments (3)
  1. Methods §4.6 and Tables 3–5: The central superiority claims rest on 5-fold CV over a 27-configuration grid (T_start, T_ramp, α) with EPS (ρ, λ) scored on the same hall-of-fame fronts, and no external held-out test set for public datasets. “DeepPySR (best)” rows therefore risk optimistic selection of the lucky configuration rather than a fixed method. Nested selection plus n=5 (Wilcoxon min p=0.031) weakens the inference that DVPS/EPS/multi-layer—not search budget or fold luck—drive the reported lifts (body fat +0.092 R², heart F1 +0.111, Raine age-27 +0.215). A nested CV, fixed default config, or true held-out split is needed to support the abstract/table claims of outperformance.
  2. §2.1–2.4 and Fig. 1: Ablations isolating DVPS, EPS, and multi-layer are deferred to Supplementary Note S3 and are not quantified in the main text against a matched PySR budget. Convergence curves use identical adaptive parsimony but still leave open whether multi-layer depth and the 27-config search inflate gains. Main-text ablations (or at least a single fixed-config DeepPySR vs PySR column) are load-bearing for attributing gains to the three named contributions rather than overall search effort.
  3. §4.6 experimental design: SMOTE is applied only to baselines on stroke/diabetes while SR models run without it; the paper argues this is conservative, yet it confounds direct F1/accuracy comparison (Table 4). Either report both with and without SMOTE for all methods or restrict primary claims to metrics that do not depend on this asymmetry (e.g., formula structure and recall at fixed accuracy).
minor comments (5)
  1. Abstract and Table 5: Abstract quotes a single Raine BMI R² 0.525 vs 0.370; main text reports age-specific values (0.727–0.425). Clarify whether 0.525 is a mean across ages and report it consistently.
  2. Eq. (18): The floor max(10^{-4}, R²_i) and the role of r2-weight vs ρ are slightly underspecified relative to the “r2-weight = 1.0” used in the Raine formula; align notation.
  3. Table 1 vs §2.3: Stroke N is listed as 5,110 in Table 1 but 43,400 in the classification results text; reconcile sample size and preprocessing.
  4. Fig. 1d–e relationship circle plots are useful but lack a short caption definition of “connection score” and layer colouring in the main text.
  5. §4.4 multi-layer: Default EPS leaf threshold 2.0 and fixed depth are stated without sensitivity; a brief note on robustness would help.

Circularity Check

1 steps flagged · score 1.0 of 10

Empirical SR methods paper with no derivation-by-construction; only a non-load-bearing self-citation on KAN limits.

  1. self citation load bearing [Discussion, paragraph on KAN]
    "A recent study on longitudinal BMI prediction [10] demonstrates that KAN succeeds, but only on a curated 20-30-feature input that required substantial domain-guided feature selection prior to modelling. This prerequisite is unavailable in truly exploratory settings, and our Raine BMI task (125 raw features, no pre-selection) is specifically designed to stress-test."

    Reference [10] has substantial author overlap with the present paper and is used to frame KAN’s failure mode. It is not load-bearing for the central claims (DVPS/EPS/multi-layer outperform PySR on the reported tables); those rest on the paper’s own experiments. Flagged only as minor self-citation, not as a forced uniqueness or definitional reduction.

full rationale

DeepPySR is an algorithmic/empirical contribution (DVPS mutation schedule, EPS scoring of hall-of-fame equations, multi-layer composition). Claims of superiority are experimental comparisons under 5-fold CV against PySR and baselines on Feynman ground-truth equations and seven real-world datasets; recovered formulas are evolutionary fits by design, not first-principles predictions. Feynman recovery is checked against known closed forms. Biological readings of the Raine formula (gene–environment, perinatal adversity) are post-hoc domain interpretation of discovered structure, not circular proofs. The sole self-citation of note is [10] (overlapping authors), used only to note that KAN needs curated features—not as a uniqueness theorem or load-bearing premise for DVPS/EPS/multi-layer. Selection of best grid/EPS configs on the same CV folds is a methodological risk (optimistic reporting), not circularity under this analyzer’s definition. No equation reduces to its inputs by construction; no uniqueness is imported; no ansatz is smuggled. Score 1 for the minor non-load-bearing self-reference only.

Assumptions & free parameters 5 free parameters · 4 assumptions · 3 invented entities

Load-bearing content is algorithmic and empirical. Claims rest on evolutionary search succeeding under a fixed operator grammar, on hyperparameters chosen by grid search, and on CV design choices—not on new physical entities. Free parameters are the search and selection knobs that define which formula is reported as ‘best’ or ‘interpretable.’ Domain assumptions include sufficiency of the operator set and that held-out fold loss is a valid pruning/selection signal.

free parameters (5)
  • DVPS schedule (T_start, T_ramp, M, w0)
    Grid-searched pruning onset, ramp, and cap control when variables are removed; reported wins depend on these choices (T_start∈{25,50,75}, T_ramp∈{50,100,150}, M=0.7).
  • EPS (ρ, λ) and r2-weight
    User-tunable accuracy exponent and complexity decay pick the single formula from the Pareto front; ‘interpretable’ row also uses complexity ≤40.
  • adaptive parsimony scaling α and base parsimony β
    α∈{1,10,50} grid-searched; β=0.001 fixed; both shape which trees survive evolution.
  • SR search budget (populations, individuals, iterations, max tree size 40)
    100×200 populations, 500 iterations (100 for large classification), max 40 nodes per call—compute and expressivity limits that affect recovery (e.g., Feynman I.9.18).
  • multi-layer depth and EPS leaf threshold (default 2.0)
    Fixed layer budget and score cutoff decide when hierarchical decomposition stops; hierarchical claims depend on this.
assumptions (4)
  • domain assumption Expression trees over a fixed grammar {+,-,×,÷,piecewise,exp,log,sin,√} can approximate the target scientific relationships.
    Methods §4.1–4.6; failure modes (I.9.18 structure) are acknowledged when the grammar is awkward.
  • domain assumption Held-out formula loss during evolution is a valid signal for pruning variables and ranking formulas (vs marginal feature importance).
    Core justification for DVPS retaining rare perinatal features on Raine (§2.4, Discussion).
  • ad hoc to paper 5-fold CV without an external test set is adequate to claim outperformance on public datasets.
    Explicitly stated in §4.6 as a design choice due to small N and lack of external cohorts.
  • standard math Standard evolutionary multi-population SR (PySR backend) is a sound base optimizer.
    Builds on Cranmer PySR / SymbolicRegression.jl [7].
invented entities (3)
  • Dynamic Variable Pruning Schedule (DVPS)
    purpose: Time-ramped prune-variable mutation to drop irrelevant inputs during search without separate preprocessing.
    Methodological construct defined by w_prune(t)=w0·τ(t)·M; independent evidence is empirical ablation/convergence curves, not an external physical object.
  • Exponential Pareto Selection (EPS)
    purpose: score = [max(1e-4,R²)]^ρ · exp(-λ(c-1)) to pick one formula from the hall of fame.
    Scoring rule invented for this framework; ‘best’ formulas are defined by this score after search.
  • Multi-layer hierarchical symbolic composition
    purpose: BFS latent targets and symbolic substitution to decompose ICC and hierarchical structure.
    Architectural procedure; circle plots and nested formulas are outputs of this procedure, not independently measured latent variables.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepPySR -- A Symbolic Regression Framework with Dynamic Pruning, Pareto Selection, and Hierarchical Composition for Real-World Scientific Discovery." pith.science (2026). https://pith.science/paper/ML56KCO7

@misc{pith2026260708150,
  author       = {Pith},
  title        = {Pith review of: DeepPySR -- A Symbolic Regression Framework with Dynamic Pruning, Pareto Selection, and Hierarchical Composition for Real-World Scientific Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ML56KCO7}},
  note         = {Machine review of arXiv:2607.08150}
}
abstract

Symbolic regression (SR) discovers analytical equations from data, yielding glass-box models with directly interpretable formulas, unlike black-box methods that rely on unstable post-hoc tools such as SHAP or LIME. This transparency is crucial in clinical medicine and social science, but SR faces three challenges: high-dimensional inputs, principled selection of Pareto-front formulae, and data irregularities such as multicollinearity and class imbalance. We introduce DeepPySR, which addresses these issues with a dynamic variable-pruning schedule to remove irrelevant features during search, an exponential Pareto selection criterion that eliminates trade-offs between accuracy and complexity, and a multi-layer architecture for hierarchical symbolic composition. On four Feynman physics benchmarks and seven biomedical and social-science datasets, DeepPySR outperforms PySR and baselines on body fat (R$^2$: 0.794 vs.\ 0.702), heart disease (F1: 0.898 vs.\ 0.787), student performance (R$^2$: 0.964 vs.\ 0.948), and Raine BMI (R$^2$: 0.525 vs.\ 0.370), producing interpretable formulas aligned with domain risk factors.

Figures

Figures reproduced from arXiv: 2607.08150 by the authors.

Figure 1
Figure 1. (a–c) Convergence curves (MSE vs. iteration, log scale). Both models use identical hyperparameters (adaptive parsimony scaling= 10, all others at default); losses differ from the grid-searched results in [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019

    Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead.Nature Machine Intelligence, 1(5):206–215, 2019

    work page 2019

  2. [2]

    A unified approach to interpreting model predictions

    Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems, volume 30, 2017

    work page 2017

  3. [3]

    why should I trust you?

    Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “why should I trust you?”: Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144, 2016

    work page 2016

  4. [4]

    On the robustness of interpretability methods

    David Alvarez-Melis and Tommi Jaakkola. On the robustness of interpretability methods. In Workshop on Human Interpretability in Machine Learning (WHI), ICML, 2018

    work page 2018

  5. [5]

    Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods

    Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 180–186, 2020

    work page 2020

  6. [6]

    MIT Press, 1994

    John R Koza.Genetic Programming II: Automatic Discovery of Reusable Programs. MIT Press, 1994. 18

    work page 1994

  7. [7]

    Interpretable Machine Learning for Science with PySR and SymbolicRegression.jl

    Miles Cranmer. Interpretable machine learning for science with PySR and SymbolicRegression.jl. arXiv preprint arXiv:2305.01582, 2023

    work page Pith review arXiv 2023

  8. [8]

    AI Feynman: A physics-inspired method for symbolic regression.Science Advances, 6(16):eaay2631, 2020

    Silviu-Marian Udrescu and Max Tegmark. AI Feynman: A physics-inspired method for symbolic regression.Science Advances, 6(16):eaay2631, 2020

    work page 2020

Show all 27 references
  1. [9]

    KAN: Kolmogorov-arnold networks.arXiv preprint arXiv:2404.19756, 2024

    Ziming Liu, Yixuan Wang, Sachin Vaidya, Fabian Ruehle, James Halverson, Marin Soljaˇ ci´ c, Thomas Y Hou, and Max Tegmark. KAN: Kolmogorov-arnold networks.arXiv preprint arXiv:2404.19756, 2024

    2024 arXiv

  2. [10]

    Fuling Chen, Phillip E Melton, Kevin Vinsen, Trevor Mori, Lawrence Beilin, and Rae-Chi Huang. Longitudinal prediction of bmi using explainable ai: integrating polygenic scores, maternal, early-life and familial factors: Epidemiology and population health.International Journal ...

    2026

  3. [11]

    Interpretable policies for reinforcement learning by genetic programming.Engineering Applications of Artificial Intelligence, 76:158–169, 2018

    Daniel Hein, Steffen Udluft, and Thomas A Runkler. Interpretable policies for reinforcement learning by genetic programming.Engineering Applications of Artificial Intelligence, 76:158–169, 2018

    2018

  4. [12]

    Improving model-based genetic programming for symbolic regression of small expressions.Evolutionary Computation, 29(2):211–237, 2021

    Marco Virgolin, Tanja Alderliesten, Cees Witteveen, and Peter A N Bosman. Improving model-based genetic programming for symbolic regression of small expressions.Evolutionary Computation, 29(2):211–237, 2021

    2021

  5. [13]

    Integration of neural network-based symbolic regression in deep learning for scientific discovery.IEEE Transactions on Neural Networks and Learning Systems, 32(9): 4166–4177, 2020

    Samuel Kim, Peter Y Lu, Srijon Mukherjee, Michael Gilbert, Li Jing, Vladimir ˇCeperi´ c, and Marin Soljaˇ ci´ c. Integration of neural network-based symbolic regression in deep learning for scientific discovery.IEEE Transactions on Neural Networks and Learning Systems, 32(9): ...

    2020

  6. [14]

    Individual comparisons by ranking methods.Biometrics Bulletin, 1(6):80–83, 1945

    Frank Wilcoxon. Individual comparisons by ranking methods.Biometrics Bulletin, 1(6):80–83, 1945

    1945

  7. [15]

    Fitting percentage of body fat to simple body measurements.Journal of Statistics Education, 4(1), 1996

    Roger W Johnson. Fitting percentage of body fat to simple body measurements.Journal of Statistics Education, 4(1), 1996

    1996

  8. [16]

    International application of a new probability algorithm for the diagnosis of coronary artery disease.American Journal of Cardiology, 64(5):304–310, 1989

    Robert Detrano, Andras Janosi, Walter Steinbrunn, Matthias Pfisterer, Johann-Jakob Schmid, Sarbjit Sandhu, Kern H Guppy, Sheryl Lee, and Victor Froelicher. International application of a new probability algorithm for the diagnosis of coronary artery disease.American Journal of...

    1989

  9. [17]

    Cohort profile: The western australian pregnancy cohort (raine) study—generation 2

    Leon Straker, Jeneva Mountain, Angela Jacques, Scott White, Anne Smith, Louis Landau, et al. Cohort profile: The western australian pregnancy cohort (raine) study—generation 2. International Journal of Epidemiology, 46(5):1384–1385j, Oct 2017

    2017

  10. [18]

    Maternal and fetal genetic effects on birth weight and their relevance to cardio-metabolic risk factors.Nature Genetics, 51(5):804–814, 2019

    Nicole M Warrington et al. Maternal and fetal genetic effects on birth weight and their relevance to cardio-metabolic risk factors.Nature Genetics, 51(5):804–814, 2019

    2019

  11. [19]

    Prenatal stress, development, health and disease risk: A 2015 Delphi consensus and call for action.Psychoneuroendocrinology, 62: 366–375, 2015

    Sonja Entringer, Claudia Buss, and Pathik D Wadhwa. Prenatal stress, development, health and disease risk: A 2015 Delphi consensus and call for action.Psychoneuroendocrinology, 62: 366–375, 2015

    2015

  12. [20]

    Cohort profile: Pregnancy and childhood epigenetics (PACE) consortium.International Journal of Epidemiology, 47(1):22–23, 2018

    Janine F Felix, Bonnie R Joubert, Andrea A Baccarelli, et al. Cohort profile: Pregnancy and childhood epigenetics (PACE) consortium.International Journal of Epidemiology, 47(1):22–23, 2018. 19

    2018

  13. [21]

    Fetal origins of coronary heart disease.BMJ, 311(6998):171–174, 1995

    David JP Barker. Fetal origins of coronary heart disease.BMJ, 311(6998):171–174, 1995

    1995

  14. [22]

    Modeling wine preferences by data mining from physicochemical properties.Decision Support Systems, 47(4):547–553, 2009

    Paulo Cortez, Ant´ onio Cerdeira, Fernando Almeida, Telmo Matos, and Jos´ e Reis. Modeling wine preferences by data mining from physicochemical properties.Decision Support Systems, 47(4):547–553, 2009

    2009

  15. [23]

    Using data mining to predict secondary school student performance

    Paulo Cortez and Alice Silva. Using data mining to predict secondary school student performance. InProceedings of 5th Annual Future Business Technology Conference, pages 5–12, 2008

    2008

  16. [24]

    Behavioral risk factor surveillance system survey data

    Centers for Disease Control and Prevention. Behavioral risk factor surveillance system survey data. Technical report, U.S. Department of Health and Human Services, 2015

    2015

  17. [25]

    XGBoost: A scalable tree boosting system

    Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 785–794, 2016

    2016

  18. [26]

    Random forests.Machine Learning, 45:5–32, 2001

    Leo Breiman. Random forests.Machine Learning, 45:5–32, 2001

    2001

  19. [27]

    Extremely randomized trees.Machine Learning, 63:3–42, 2006

    Pierre Geurts, Damien Ernst, and Louis Wehenkel. Extremely randomized trees.Machine Learning, 63:3–42, 2006. 20

    2006

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.