REVIEW 4 major objections 5 minor 35 references
Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Unit-aware genetic programming automatically constructs interpretable, dimensionally consistent features that improve classification accuracy on particle-collision data by up to 3.8 percentage points.
desk verdict A genuinely useful grammar-guided GP idea for physics feature construction, but the paper's own statistics do not back the 'significant gain on three datasets' claim as printed. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a context-free grammar whose nonterminals are physical types—Energy, Angle, Float, squared Energy—and whose production rules permit only combinations that respect dimensional analysis: an Energy can be added to an Energy, divided by a Float, or obtained by taking the square root of a squared Energy, but an Energy and an Angle can never be added. Every candidate feature is a derivation from this grammar, so its expression is guaranteed to be independent of the system of measurement. A second component, a probability transition matrix over operators, biases derivations toward recurrent physics patterns, such as a square root applied to a sum of squares; the matrix is set by hand for the experiments and compared with uniform probabilities. Evolution uses standard genetic programming operators—initialization, mutation, crossover—restricted so that all offspring remain grammar-compliant, and fitness is the cross-validated accuracy of a gradient-boosted classifier trained on the constructed feature set.
What would settle it
Re-run the three experiments with the type tags deliberately corrupted—mark one Energy feature as Float, or collapse all types to a single Float type. If the accuracy gains over baseline survive, the grammar's dimensional constraint is not what produces them; if the best known physics feature becomes unreachable and accuracy drops, the constraint is load-bearing.
Extended reading notes
Core claim
The central discovery is that enforcing dimensional consistency through a context-free grammar does more than filter out invalid expressions: it concentrates the search on features that are simultaneously physically meaningful and competitive for classification. On the first dataset, the grammar-guided method finds a single feature that outperforms the invariant mass; on the second dataset it delivers a statistically significant accuracy gain over unconstrained genetic programming; on the third dataset the gain is smaller, which the paper attributes to a lack of basic angular variables. The paper also shows that a hand-set probability transition matrix over grammar rules—for instance, favouring square roots of sums of squares—steers evolution toward formulas whose components physicists recognize, such as transverse-energy balances and cosine differences of angles, while keeping final accuracy comparable to the unguided grammar method. When several features are constructed jointly, the probability-guided variant matches the unguided one while remaining more interpretable.
Load-bearing premise
Every input feature must be tagged with the correct physical type (Energy, Angle, Float, squared Energy, etc.) and the type system must be complete enough to express the genuinely best feature; the paper does not publish its tag assignments and states the grammar figure is simplified.
Editorial extensions
If this is right
- Physicists gain an automated source of candidate observables whose expressions visibly carry their physical dimensions, so each candidate can be assessed by an expert before use.
- Because the constructed features are independent of the measurement system, they can be reused across detector calibrations and simulation setups.
- The transition-matrix guidance behaves as a performance-versus-interpretability dial: constructing several features at once lets the guided method match the unguided one while producing recognizable physics components.
- Since the grammar is defined over physical types rather than dataset-specific variable names, the same machinery can be re-instantiated for other unit-carrying datasets.
- The gains persist across different fitness functions—gradient-boosted trees, decision trees, and k-nearest neighbours—which indicates the constructed features themselves, not one particular wrapper, drive the improvement.
Reading between the lines
- A natural stress test would re-run the three datasets with deliberately mistyped features: if the gains survive when all types are collapsed to a single Float, the grammar is not the active ingredient; if they vanish, the type tags are load-bearing.
- Because the transition matrix is hand-set, its entries could instead be estimated from a corpus of physics formulas; such a data-driven prior would make the method portable and could be compared directly against the hand-tuned matrix on the same benchmarks.
- Unit-consistent constructed features are also natural inputs for tasks beyond classification, such as anomaly detection or regression in experimental physics, where readable observables matter equally.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a genetic programming (GP) approach to automatic feature construction for high-energy physics, in which a context-free grammar enforces dimensional consistency of the evolved expressions. Two variants are studied: GGGP, which uses only the grammar, and PGGGP, which additionally uses a hand-set transition probability matrix over grammar operators. The method is evaluated on three physics datasets (Higgs, DVCS, and τ→μμμ) with XGBoost as the main classifier, and the paper claims significant accuracy gains over simple unconstrained GP and over a PSO feature-construction baseline, while also claiming interpretability of the constructed features, validated by HEP experts.
Significance. If the claims are sustained, the work is a practical contribution to a real need: automatic construction of physically meaningful, dimensionally consistent observables for collider analyses, with interpretability for domain experts. The main strengths are the use of grammars to enforce dimensional consistency, the exploration of several fitness functions and classifiers, the comparison across three different experimental setups, and the explicit attempt to interpret the evolved formulas. The paper also makes a useful design point: the manually chosen transition matrix can trade off single-feature performance for interpretability. However, the headline empirical claim of a "significant gain" is not currently supported by the reported statistics, and the reproducibility of the search space is limited by the absence of the full grammar and type assignments.
major comments (4)
- [IV-C, Table II] The first p-value reported for the Higgs dataset (10^-17) is irreconcilable with the means and standard deviations shown in the same table. For a comparison between the best constrained method (GGGP, 2.23±0.68) and simple GP (1.92±0.11) with n=20, a Welch t-test gives t≈2.0 and p≈0.06, not 10^-17. Please report the exact test statistic, degrees of freedom, and the precise comparison performed, and re-examine the abstract's "significant gain" claim in light of the corrected test.
- [IV-D] The paper promotes a single best GGGP feature on the Higgs dataset with a +3.82% accuracy gain and compares it to the invariant-mass feature (+2.91%), concluding "we overcome the invariant mass." This is a selected maximum over the runs, whereas Table II reports a mean of 2.23±0.68 for GGGP on the same task. The expected gain is therefore not +3.82%, and the comparison overstates the method's typical performance. Please report the mean and standard deviation of the gain over runs, or provide a proper paired comparison against the invariant-mass feature, and adjust the wording.
- [Abstract and Conclusion] The claim that the method brings a significant gain on "three physics datasets" is not supported by the results in Table II. On the τμ3 dataset, GGGP (0.43±0.28) and PGGGP (0.54±0.37) are not better than simple GP (0.58±0.36), and no p-value is reported comparing the constrained methods to simple GP for this dataset. The conclusion itself contains a contradiction: it first says the method significantly improves accuracy "for two high-energy physics datasets," then says "for the three datasets, our interpretable GGGP-based feature construction algorithm brings a significant improvement." The claim must be restricted to the datasets and comparisons where the statistical evidence actually supports it.
- [III-A and dataset description] The paper does not provide the physical type assignment for the 17, 36, and 46 input features of the three datasets, nor the complete grammar used in the experiments; Figure 1 is explicitly stated to be a simplified version. Since the method's search space is defined by these type tags and grammar productions, a mistyped feature or an incomplete type set could exclude the best possible feature, and the experiments cannot be reproduced without this information. Please publish the full grammar, the type map for each dataset, and the exact transition matrix used.
minor comments (5)
- [II-C] The text contains a typo: "Welchs t-test" should be "Welch's t-test."
- [IV-C, Table II] The caption says the first p-value compares the best of PGGGP and GGGP to simple GP, but the table only shows a first p-value for the Higgs row; the corresponding cells for DVCS and τμ3 are missing. Please either report those values or clarify that they were not computed.
- [IV-D, Equation (1)] Equation (1) is difficult to read because of formatting artifacts such as "missingtE" and "missingtE2" and the placement of exponents and square roots. Please typeset the formula cleanly, and also check the similar garbled notation in Table I and around the discussion of the five-feature set.
- [IV-B] The paper says results are means over "at least 20 independent runs" but does not state how the random seeds were generated or whether the same seeds were used across compared methods. This information would help assess the variance and the paired/unpaired nature of the significance tests.
- [IV-D] The interpretability discussion would benefit from a more precise statement of how the HEP expert validation was conducted, since the abstract mentions expert validation but the body only describes qualitative agreement with physics intuition.
Circularity Check
No load-bearing circularity; central accuracy claim is benchmark-based and independent. Minor self-confirmation in the interpretability claim from hand-set transition probabilities.
-
other
[Section III-B (Transition matrix definition), with the conclusion claim in Section V]
"The probabilities are chosen manually to emulate the frequencies of operators in HEP formulas."
The hand-set transition matrix is the direct cause of whatever HEP-likeness PGGGP outputs exhibit, and the later conclusion that PGGGP features are 'definitely better' with respect to interpretability for physics experts (Section V) is then supported by that same HEP-likeness. No independent expert survey or interpretability metric is supplied, so the interpretability claim restates the design choice rather than being confirmed by external evidence. The accuracy comparisons are unaffected: they are measured on held-out test sets against simple GP and PSO, so the central empirical claim is not circular.
full rationale
The paper's headline result is an empirical comparison of test-set accuracy gains on three external physics benchmarks (Higgs, DVCS, tau-mu3), measured against a baseline XGBoost and against simple GP and PSO. Those numbers are not derived from the grammar, the transition matrix, or any fitted parameter; they come from held-out evaluations. No self-citations appear in the reference list, and no uniqueness theorem is imported from the authors. The grammar and the manually chosen transition matrix do restrict the search space, but restrictions are inputs, not predictions; they do not by themselves produce the reported gains. The only mild circularity is in the interpretability discussion: PGGGP's transition probabilities were hand-set to favor HEP-like operator patterns, and the paper then cites the HEP-likeness of the resulting features as evidence that PGGGP is more interpretable. That is a self-confirming qualitative claim, but it is not load-bearing for the accuracy results. The apparent inconsistency between the reported p-value and the published means/standard deviations in Table II is a statistical-correctness concern, not a circularity concern.
Assumptions & free parameters
free parameters (5)
- transition probability matrix =
not fully specified; Table I shows a snippet
- mutation and crossover rates =
0.6 each
- population size and generation count =
500, 150
- tournament size =
3
- maximum tree depth =
not reported
assumptions (4)
- domain assumption Each raw feature can be assigned a physical type such as Energy, Angle, Float, or Energy-squared, and only same-type or grammar-allowed combinations are physically meaningful.
- domain assumption Monte-Carlo simulated events used for training are representative enough that accuracy gains on the test split transfer to real experimental data.
- ad hoc to paper The manually chosen transition probabilities reflect typical HEP formula frequencies and improve search, rather than merely encoding the output.
- ad hoc to paper The grammar restricts the search space in a way that does not remove the optimal features.
Cite this review
Pith. "Pith review of Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics." pith.science (2026). https://pith.science/paper/RJYCLOBT
@misc{pith2026190808005,
author = {Pith},
title = {Pith review of: Consistent Feature Construction with Constrained Genetic Programming for Experimental Physics},
year = {2026},
howpublished = {\url{https://pith.science/paper/RJYCLOBT}},
note = {Machine review of arXiv:1908.08005}
}
read the original abstract
A good feature representation is a determinant factor to achieve high performance for many machine learning algorithms in terms of classification. This is especially true for techniques that do not build complex internal representations of data (e.g. decision trees, in contrast to deep neural networks). To transform the feature space, feature construction techniques build new high-level features from the original ones. Among these techniques, Genetic Programming is a good candidate to provide interpretable features required for data analysis in high energy physics. Classically, original features or higher-level features based on physics first principles are used as inputs for training. However, physicists would benefit from an automatic and interpretable feature construction for the classification of particle collision events. Our main contribution consists in combining different aspects of Genetic Programming and applying them to feature construction for experimental physics. In particular, to be applicable to physics, dimensional consistency is enforced using grammars. Results of experiments on three physics datasets show that the constructed features can bring a significant gain to the classification accuracy. To the best of our knowledge, it is the first time a method is proposed for interpretable feature construction with units of measurement, and that experts in high-energy physics validate the overall approach as well as the interpretability of the built features.
Figures
Reference graph
Works this paper leans on
-
[1]
Irrelevant Features and the Subset Selection Problem,
G. H. John, R. Kohavi, and K. Pfleger, “Irrelevant Features and the Subset Selection Problem,” in Machine Learning Proceedings 1994 . Elsevier, 1994, pp. 121–129
work page 1994
-
[2]
Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning,
L. H. Gilpin, D. Bau, B. Z. Yuan, A. Bajwa, M. Specter, and L. Kagal, “Explaining Explanations: An Approach to Evaluating Interpretability of Machine Learning,” arXiv:1806.00069 [cs, stat] , May 2018, arXiv: 1806.00069
arXiv 2018
-
[3]
V . N. Vapnik, The Nature of Statistical Learning Theory . New York, NY , USA: Springer-Verlag New York, Inc., 1995
1995
-
[4]
Deep learning of representations: Looking forward,
Y . Bengio, “Deep learning of representations: Looking forward,” CoRR, vol. abs/1305.0445, 2013
arXiv 2013
-
[5]
Feature Construction Methods: A Survey,
P. Sondhi, “Feature Construction Methods: A Survey,” p. 8, 2009
work page 2009
-
[6]
Feature generation using general constructor functions,
S. Markovitch and D. Rosenstein, “Feature generation using general constructor functions,” Machine Learning , vol. 49, no. 1, pp. 59–98, Oct 2002
work page 2002
-
[7]
Cognito: Automated feature engineering for supervised learning,
U. Khurana, D. Turaga, H. Samulowitz, and S. Parthasrathy, “Cognito: Automated feature engineering for supervised learning,” in 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) , Dec 2016, pp. 1304–1307
work page 2016
-
[8]
Feature engineering for predictive modeling using reinforcement learning,
U. Khurana, H. Samulowitz, and D. S. Turaga, “Feature engineering for predictive modeling using reinforcement learning,” CoRR, vol. abs/1709.07150, 2017
arXiv 2017
Show all 35 references
-
[9]
PSO for feature construction and binary classification
B. Xue, M. Zhang, Y . Dai, and W. N. Browne, “PSO for feature construction and binary classification.” ACM Press, 2013, p. 137
2013
-
[10]
New Representations in PSO for Feature Construction in Classification,
Y . Dai, B. Xue, and M. Zhang, “New Representations in PSO for Feature Construction in Classification,” in Applications of Evolutionary Computation, ser. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg, Apr. 2014, pp. 476–488
2014
-
[11]
A Survey on the Application of Genetic Programming to Classification,
P. Espejo, S. Ventura, and F. Herrera, “A Survey on the Application of Genetic Programming to Classification,” IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 40, no. 2, pp. 121–144, Mar. 2010
2010
-
[12]
J. R. Koza, Genetic programming: on the programming of computers by means of natural selection, ser. Complex adaptive systems. Cambridge, Mass: MIT Press, 1992
1992
-
[13]
Genetic Programming-based Construction of Features for Machine Learning and Knowledge Discovery Tasks,
K. Krawiec, “Genetic Programming-based Construction of Features for Machine Learning and Knowledge Discovery Tasks,” p. 15, 2002
2002
-
[14]
Genetic Programming for Attribute Construction in Data Mining,
F. E. B. Otero, M. M. S. Silva, A. A. Freitas, and J. C. Nievola, “Genetic Programming for Attribute Construction in Data Mining,” in Proceedings of the 6th European Conference on Genetic Programming , ser. EuroGP’03. Berlin, Heidelberg: Springer-Verlag, 2003, pp. 384– 393
2003
-
[15]
Genetic Programming with a Genetic Al- gorithm for Feature Construction and Selection,
M. G. Smith and L. Bull, “Genetic Programming with a Genetic Al- gorithm for Feature Construction and Selection,” Genetic Programming and Evolvable Machines , vol. 6, no. 3, pp. 265–281, Sep. 2005
2005
-
[16]
Genetic Programming and Class-Wise Orthogonal Transformation for Dimension Reduction in Classification Problems,
K. Neshatian and M. Zhang, “Genetic Programming and Class-Wise Orthogonal Transformation for Dimension Reduction in Classification Problems,” in Genetic Programming , ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2008, pp. 242–253
2008
-
[17]
Feature generation using genetic programming with application to fault classification,
H. Guo, L. B. Jack, and A. K. Nandi, “Feature generation using genetic programming with application to fault classification,” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) , vol. 35, no. 1, pp. 89–99, Feb. 2005
2005
-
[18]
A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming,
K. Neshatian, M. Zhang, and P. Andreae, “A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming,” IEEE Transactions on Evolutionary Computa- tion, vol. 16, no. 5, pp. 645–661, Oct. 2012
2012
-
[19]
On Prediction of Epileptic Seizures by Computing Multiple Genetic Programming Artificial Fea- tures,
H. Firpi, E. Goodman, and J. Echauz, “On Prediction of Epileptic Seizures by Computing Multiple Genetic Programming Artificial Fea- tures,” inGenetic Programming, ser. Lecture Notes in Computer Science. Springer Berlin Heidelberg, 2005, pp. 321–330
2005
-
[20]
Multiple feature construction in classification on high-dimensional data using GP
B. Tran, M. Zhang, and B. Xue, “Multiple feature construction in classification on high-dimensional data using GP.” IEEE, Dec. 2016, pp. 1–8
2016
-
[21]
Selecting and constructing features using grammatical evolution,
D. Gavrilis, I. G. Tsoulos, and E. Dermatas, “Selecting and constructing features using grammatical evolution,” Pattern Recognition Letters , vol. 29, no. 9, pp. 1358–1365, Jul. 2008
2008
-
[22]
Strongly Typed Genetic Programming,
D. J. Montana, “Strongly Typed Genetic Programming,” Evol. Comput., vol. 3, no. 2, pp. 199–230, Jun. 1995
1995
-
[23]
Type Inheri- tance in Strongly Typed Genetic Programming,
T. D. Haynes, D. A. Schoenefeld, and R. L. Wainwright, “Type Inheri- tance in Strongly Typed Genetic Programming,” in Advances in Genetic Programming 2, chapter 18 . MIT Press, 1996, pp. 359–376
1996
-
[24]
Grammar- based Genetic Programming: a survey,
R. McKay, N. Hoai, P. Whigham, Y . Shan, and M. O’Neill, “Grammar- based Genetic Programming: a survey,” Genetic Programming and Evolvable Machines, vol. 11, pp. 365–396, Sep. 2010
2010
-
[25]
Advances in Genetic Programming,
F. Gruau, “Advances in Genetic Programming,” P. J. Angeline and K. E. Kinnear, Jr., Eds. Cambridge, MA, USA: MIT Press, 1996, pp. 377– 394
1996
-
[26]
Dimensionally Aware Genetic Program- ming,
M. Keijzer and V . Babovic, “Dimensionally Aware Genetic Program- ming,” in Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - Volume 2 , ser. GECCO’99, San Francisco, CA, USA, 1999, pp. 1069–1076
1999
-
[27]
Grammar-guided genetic programming and dimensional consistency: application to non-parametric identification in mechanics,
A. Ratle and M. Sebag, “Grammar-guided genetic programming and dimensional consistency: application to non-parametric identification in mechanics,” Applied Soft Computing , vol. 1, no. 1, pp. 105–118, Jun. 2001
2001
-
[28]
Grammatical evolution,
M. O’Neill and C. Ryan, “Grammatical evolution,” IEEE Transactions on Evolutionary Computation , vol. 5, no. 4, pp. 349–358, Aug. 2001
2001
-
[29]
Enhancing discrimination power with genetic feature construction: A grammatical evolution approach,
P. Miquilini, R. C. Barros, V . V . d. Melo, and M. P. Basgalupp, “Enhancing discrimination power with genetic feature construction: A grammatical evolution approach,” in 2016 IEEE Congress on Evolution- ary Computation (CEC) , Jul. 2016, pp. 3824–3831
2016
-
[30]
MBCGP-FE: A modified balanced cartesian genetic programming feature extractor,
S. Yazdani, J. Shanbehzadeh, and E. Hadavandi, “MBCGP-FE: A modified balanced cartesian genetic programming feature extractor,” Knowledge-Based Systems, vol. 135, pp. 89–98, Nov. 2017
2017
-
[31]
Distilling Free-Form Natural Laws from Experimental Data,
M. Schmidt and H. Lipson, “Distilling Free-Form Natural Laws from Experimental Data,” Science, vol. 324, no. 5923, pp. 81–85, Apr. 2009
2009
-
[32]
A Survey of Probabilistic Model Building Genetic Programming,
Y . Shan, R. I. McKay, D. Essam, and H. A. Abbass, “A Survey of Probabilistic Model Building Genetic Programming,” in Scalable Optimization via Probabilistic Modeling . Berlin, Heidelberg: Springer Berlin Heidelberg, 2006, vol. 33, pp. 121–160
2006
-
[33]
Avoiding the Bloat with Stochastic Grammar- based Genetic Programming,
A. Ratle and M. Sebag, “Avoiding the Bloat with Stochastic Grammar- based Genetic Programming,” arXiv:cs/0602022, Feb. 2006, arXiv: cs/0602022
2006 arXiv
-
[34]
Learning to discover: the Higgs boson machine learning challenge - Documentation,
C. Adam-Bourdarios, G. Cowan, C. Germain, I. Guyon, B. Kegl, and D. Rousseau, “Learning to discover: the Higgs boson machine learning challenge - Documentation,” 2014
2014
-
[35]
Search for the lepton flavour violating decay τ −→ µ−µ+µ−,
LHCb collaboration, “Search for the lepton flavour violating decay τ −→ µ−µ+µ−,” Journal of High Energy Physics , vol. 2015, no. 2, Feb. 2015, arXiv: 1409.8548
2015 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.