{"id":"be0f8137-b636-402e-b6db-5f6c41c13383","arxiv_id":"2505.01262","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Modular GP-GOMEA, which evolves multiple tree templates that can hierarchically call each other as subexpressions, generally beats single-template GP-GOMEA in accuracy on tested benchmarks and recovers modular ground-truth expressions more often.","lead":"This paper introduces Modular GP-GOMEA, a genetic programming variant that evolves several small expression trees in parallel, letting each tree call earlier trees as reusable subexpressions so the final formula can be far larger than any individual template. A generalist might read it because it targets a central tension in explainable AI, accuracy versus interpretability, by decomposing learned formulas into smaller, reusable pieces.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world support for 'generally outperforms' is contradicted by the paper's own Discussion and lacks per-dataset evidence.","rationale":"The reader's weakest assumption correctly identifies benchmark fairness and population-size selection as risks, and the reader's rationale notes the abstract/Discussion inconsistency. I go further: the inconsistency is not merely a wording issue but a direct contradiction in the evidence for the central claim. The synthetic experiments are strong within their scope, so the paper should not be rejected outright. The appropriate verdict remains CONDITIONAL, but the conditions should include providing per-dataset real-world results, significance testing, and clarifying the scope of 'generally outperforms.' My concrete test targets exactly that gap. I do not see a more fundamental algorithmic flaw in the method as described; the FOS construction, GOM procedure, and subexpression semantics are coherent, and the synthetic recovery results are plausible. The main weakness is that the broad real-world claim is not supported by the data shown and is explicitly walked back by the authors in their own Discussion.","tokens_in":14727,"tokens_out":9788,"duration_ms":101292,"concrete_test":"Produce a table of per-dataset mean and standard deviation of best R2 for the 4x4 modular configuration versus the 7x1 non-modular configuration (and, ideally, the 6x1 single-tree configuration) over the 30 runs on Airfoil, Bike, Concrete, Dow Chemical, and Tower. Run a paired significance test on per-run best R2 values (e.g., Wilcoxon signed-rank test) for each dataset and for the pooled comparison. If the modular configuration is not significantly better on most real-world datasets, or if the per-dataset differences are inconsistent in sign, revise the abstract and conclusion to state that modular GP-GOMEA is competitive with, but not generally better than, single-template GP-GOMEA on real-world data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two pillars: synthetic recovery and real-world R2. The synthetic pillar is internally consistent: on five datasets deliberately built from reusable subexpressions, modular GP-GOMEA recovers ground truth more often and reaches higher R2 than single-template GP-GOMEA. The real-world pillar is not. The Abstract and Conclusions say Modular GP-GOMEA 'achieved higher R2 across five real-world datasets' and 'generally outperforms single-template GP-GOMEA,' but Section 4 states 'Modular GP-GOMEA performs on par with standard GP-GOMEA on real-world datasets.' These statements directly contradict each other. The only real-world evidence is Figure 8b, an average over five datasets with standard-deviation whiskers; no per-dataset R2 table is provided, and no paired significance test is reported. The population size used in Section 3.5 (4096) was chosen because it was the best-tried modular size under a 1-hour budget for the synthetic problems, not because it suits the real-world data or the non-modular baseline. In the population-size experiment (Figure 3), non-modular GP-GOMEA's R2 increases monotonically with population size up to 32768 under the 6-hour budget; if the real-world comparison had used a larger non-modular population, the 'generally outperforms' result could reverse. The authors themselves further weaken the headline by observing that modularity's benefit 'primarily' comes from a larger flexible template structure rather than from re-use of subexpressions as functions. Thus, the evidence supports 'outperforms on synthetic modular benchmarks and is comparable on real-world datasets,' not the abstract's unqualified 'generally outperforms.'","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Modular GP-GOMEA, a symbolic-regression variant in which an individual is a vector of fixed-size tree templates, with later trees allowed to call earlier trees as subexpressions through newly introduced subexpression and argument nodes. The central claims are that this representation generally outperforms single-template GP-GOMEA in R2 and that on synthetic datasets with reusable subexpressions it recovers ground-truth expressions faster than the non-modular baseline. The paper reports experiments on five synthetic datasets deliberately constructed from reusable subexpressions and five real-world datasets, along with population-size, template-depth, and tree-count studies, plus a qualitative interpretability example on the Bike Daily dataset. The synthetic recovery results are internally consistent, but the real-world generalization claim is contradicted by the paper's own Discussion and is supported only by averaged results without per-dataset statistics or significance tests.","tokens_in":15035,"tokens_out":6242,"duration_ms":62053,"significance":"If the claims were fully supported, the modular representation would be a meaningful extension of GP-GOMEA: it allows hierarchical subexpression reuse while retaining small fixed templates, and the synthetic experiments show substantially higher ground-truth recovery rates than the single-template baseline. The paper also contributes a set of synthetic benchmarks with known modular ground truths and a useful qualitative analysis of an evolved subexpression. These are real strengths. However, the headline claim of 'generally outperforms' is not established by the evidence as presented: the real-world comparison rests on aggregated R2 averages with no paired testing, and the authors themselves state that the benefit primarily comes from a more flexible template structure rather than from functional reuse. The paper would be significantly stronger if the claims were narrowed to the synthetic regime and if the real-world results were reported per dataset with appropriate statistical analysis.","major_comments":[{"comment":"The Abstract's claim that Modular GP-GOMEA 'generally outperforms single-template GP-GOMEA' and Section 5's statement that it 'achieved higher R2 across five real-world datasets' are directly contradicted by Section 4, which states that 'Modular GP-GOMEA performs on par with standard GP-GOMEA on real-world datasets.' The only real-world evidence is Figure 8b, which shows averages over five datasets with standard-deviation whiskers; no per-dataset R2 table and no paired significance test are provided. Given that the real-world datasets are central to the generality claim, the authors must either provide per-dataset paired statistics or revise the abstract and conclusions to match the more modest Discussion statement.","section":"Abstract, Section 4, Section 5, Figure 8b"},{"comment":"The population size used for the real-world comparisons in Section 3.5 is 4096, chosen because it was the best-tried modular size under a 1-hour budget for synthetic problems, as shown in Figure 4 and acknowledged in the Discussion: it was 'the best-tested one for 2 out of 5 synthetic problems and was not tested for real-world problems.' This is a potential bias against the non-modular baseline, because Figure 3 shows that non-modular GP-GOMEA's average R2 has a general upward trend with population size up to 32768. To support a fair comparison, the authors should either tune the population size for each configuration and dataset or demonstrate that the qualitative conclusion is insensitive to this choice.","section":"Section 3.3, Section 3.5, Discussion"},{"comment":"All five synthetic ground truths are deliberately constructed from reusable subexpressions, so the modular representation is given an advantage by construction on these benchmarks. This is reasonable for demonstrating the mechanism, but it does not support the broad 'generally outperforms' claim. Moreover, the paper's own analysis in Section 3.5 states that the 4x2 configuration, which has sufficient structure to represent synthetic expression 1, attains R2=0.68 versus 0.87 for 4x4 and 0.82 for 7x1, and that this 'indicates that subexpression re-use alone does not explain the performance gap.' This is an honest caveat, but it means the paper has not isolated the contribution of modular reuse from the contribution of a larger or differently shaped template. An ablation that controls total template nodes, search effort, or both is needed before claiming that the modular representation itself is the source of the improvement.","section":"Section 3.2, Section 3.5"},{"comment":"No statistical significance tests are reported anywhere in the paper. The claims of superiority rest on average R2 values with standard-deviation whiskers, and for the real-world results in Figure 8b the whiskers overlap substantially across configurations. A paired test over the 30 repetitions, or at least a per-dataset table with confidence intervals, is necessary to support 'generally outperforms' and 'achieved higher R2.' Without this, the observed differences could be within run-to-run noise.","section":"Section 3.1, Figure 8"}],"minor_comments":[{"comment":"The text mentions 'concical tree initialization methods'; this appears to be a typo, likely for 'canonical' or 'conical'.","section":"Section 2.1"},{"comment":"In the paragraph on population size, 'non-modular configuration of GP-GOMEOA' should read 'GP-GOMEA'.","section":"Section 3.3"},{"comment":"The caption contains typos: 'indivdiual' should be 'individual' and 'evalutations' should be 'evaluations'.","section":"Figure 7 caption"},{"comment":"The distinction between 'subexpressions used' and 'subexpressions re-used' is not defined explicitly before the table; a short definition or a pointer to Section 2.2 would improve clarity.","section":"Section 3.5, Table 3"},{"comment":"The appendix reports R2 with and without inputs in the subexpression operator set only as a figure; providing the per-dataset numerical values would make the claim more reproducible.","section":"Appendix A, Figure 11"}],"recommendation":"major_revision","confidential_remarks":"The manuscript contains an internal inconsistency: the abstract and conclusions claim real-world superiority, while the Discussion explicitly says performance is on par on real-world datasets. This must be resolved before publication. The synthetic experiments are the strongest part of the paper and should be foregrounded, with the real-world claim either substantiated through per-dataset paired statistics or dropped. The paper is within the scope of the journal and the core idea is worth publishing after a revision that honestly calibrates the claims and adds the missing statistical evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Joe, here's my quick read.\n\nThe modular representation is a genuine extension of GP-GOMEA: multiple trees, hierarchical subexpression calls, per-tree FOS learning, and a flattened FOS for GOM. That is a clean way to get around the fixed-template limitation, and the synthetic results are the real strength. On the five ground-truth problems with reusable subexpressions, Modular GP-GOMEA recovers the exact expression much more often and gets higher R2 than single-template GP-GOMEA and a template-constrained GP. The authors also do the right thing by admitting the limits: subexpression reuse is less common than expected, much of the gain comes from having a larger template, and population size was not tuned per problem.\n\nThe soft spot is the real-world claim. The abstract and conclusions say 'generally outperforms' and 'higher R2 across five real-world datasets,' but Section 4 says 'on par.' That mismatch matters. The only real-world evidence is an averaged plot in Figure 8b with no per-dataset table and no significance test. And the population size used there (4096) was chosen from the synthetic experiments under a 1-hour budget, not for the real-world sets or the non-modular baseline. Since non-modular R2 keeps climbing with population size in Figure 3, a larger population could flip the comparison. The paper would be honest if it claimed 'comparable on real-world data with a larger template.'\n\nMinor issues: no code or data released, no statistical tests anywhere, and the synthetic benchmarks are purposely designed to favor modularity. None of this kills the paper, but it means the headline claim needs scope-limiting.\n\nBottom line: I would send this to review. The method is clearly described, the synthetic evidence is solid, and the authors are candid about the limitations. The revision should reconcile the abstract with the discussion and add per-dataset real-world results. I'd cite the method and bring it to our reading group.","headline":"Strong synthetic results for a genuinely modular GP representation, but the paper's own discussion contradicts its real-world performance claim.","tokens_in":15571,"tokens_out":3329,"would_cite":true,"duration_ms":30491,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modular GP-GOMEA represents a symbolic regression model as multiple small trees that call each other hierarchically, and the paper shows this generally outperforms single-template GP-GOMEA in $R^2$ while recovering modular ground-truth…","keywords":["symbolic regression","genetic programming","GP-GOMEA","modularity","automatically defined functions","hierarchical subexpressions","linkage learning","interpretability"],"falsifier":"Run both methods on a set of problems whose ground truths contain no reusable subexpressions, giving each method the same total node budget and a per-dataset tuned population size; if modular GP-GOMEA does not match or beat single-template GP-GOMEA in $R^2$ and recovery rate on most of those problems, the central claim is falsified.","tokens_in":14526,"feed_emoji":"🌳","tokens_out":9232,"duration_ms":79847,"temperature":0.7,"pith_summary":"Modular GP-GOMEA is a symbolic regression method that replaces the single fixed tree template used by GP-GOMEA with a vector of small tree templates, letting later trees call earlier trees as reusable subexpressions. The paper claims that this representation generally achieves higher $R^2$ than single-template GP-GOMEA, recovers known modular ground-truth expressions more often and faster, and keeps each subexpression small enough to interpret on its own. The reason this matters is that symbolic regression has long traded accuracy against interpretability; if a large effective expression can be built from small reused parts, the trade-off shifts. The paper reports gains on five synthetic and five real-world datasets, while noting that part of the gain comes from extra structure rather than from functional reuse.","feed_headline":"Modular GP-GOMEA beats fixed-template GP-GOMEA","feed_subtitle":"Callable subexpressions recover modular ground truths faster and raise R2 on real-world data.","key_machinery":"The central object is the multi-tree template individual: a vector of fixed-size tree templates in which only the last tree is the output, while earlier trees can be called as functions through subexpression nodes, and argument nodes tie the caller's inputs to the called tree's parameters. Search proceeds by learning a separate linkage-tree Family of Subsets for each tree, flattening the per-tree FOSes into one shuffled FOS, and applying gene-pool optimal mixing with donor solutions; whole-subexpression swaps are allowed for non-output trees while the output tree never swaps wholesale. This machinery lets a small fixed template per subexpression produce a larger effective expression through hierarchical calls.","core_discovery":"On its own terms, this paper establishes that expressing a candidate symbolic regression model as multiple hierarchically callable fixed-size trees improves both accuracy and recovery speed over GP-GOMEA's single fixed template. In the proposed multi-tree representation, only the last tree is the output; any earlier tree can appear as a subexpression node in a later tree, with argument nodes passing the caller's inputs into the called tree, and cycles are prevented by allowing tree $i$ to call only trees $j<i$. Across five synthetic datasets whose ground truths reuse subexpressions, the modular configuration reaches higher $R^2$ and recovers the exact expression more often than single-template GP-GOMEA and a template-constrained GP baseline, and it does so in less wall-clock time per generation. On five real-world datasets, the modular configuration achieves higher $R^2$ as the number of trees and their depth grow, though the paper's own analysis shows that subexpression reuse as functions is less frequent than expected and that much of the benefit is attributable to having more available structure.","pith_inferences":["A sharper test of the representation would hold total node capacity fixed across modular and single-template variants; the paper's data suggest the advantage could shrink when the single template is given as many nodes.","On non-modular ground truths, modular GP-GOMEA may lose its edge; the synthetic benchmarks are modular by construction, so the 'generally outperforms' claim should be read as limited to problems with reusable structure.","Adding parsimony pressure aimed at subexpression reuse, for instance by making reuse an archive objective, is a plausible next step the paper identifies as future work; it could turn rare functional reuse into a systematic behavior.","The qualitative subexpression analysis could support a user study: if humans understand expressions with reused subexpressions faster and more accurately than flat expanded expressions, the interpretability argument would have direct evidence."],"forward_implications":["Symbolic regression models no longer need their final shape fixed in advance; a deep or unbalanced expression can be assembled from shallow templates, so the user does not have to guess the right template depth or width.","On problems with genuinely reusable subexpressions, the method should find exact or near-exact expressions faster, which directly supports scientific discovery settings where the goal is to recover a known law.","The multi-tree representation creates a natural unit for human inspection: a reader can understand a small subexpression once and then treat it as a feature wherever it is reused.","Increasing the number of trees or template depth improves $R^2$ up to a point, after which the larger individual slows generation throughput within a fixed time budget; future configurations should be chosen with the time budget in mind.","Even without parsimony pressure, runs on real-world datasets show at least one reused subexpression on average, indicating that some real regression problems are decomposable in a way this representation can capture."],"supporting_citations":[{"why":"Supplies the base GP-GOMEA algorithm, the fixed-tree-template representation, linkage-tree FOS learning, and gene-pool optimal mixing that Modular GP-GOMEA extends.","marker":"[24]"},{"why":"Prior GOMEA variant with Cartesian genetic programming that first enabled automatic discovery of duplicate subexpressions.","marker":"[6]"},{"why":"Embedded CGP, which adds functional subexpressions that take different inputs; Modular GP-GOMEA extends this to hierarchical calls.","marker":"[26]"},{"why":"Introduces hierarchically callable automatically defined functions, the modularity concept this representation realizes within GP-GOMEA.","marker":"[10]"},{"why":"Provides the genetic programming baseline, half-and-half initialization, and the ADF comparison points used in the experiments.","marker":"[11]"},{"why":"Basis for coefficient mutation during gene-pool optimal mixing, used in all runs.","marker":"[25]"},{"why":"Source of the 25 coefficient bins and higher-arity operator handling adopted in the experimental setup.","marker":"[20]"},{"why":"Adaptively gridded elitist archive used to maintain the multi-objective trade-off front between expression size and $R^2$.","marker":"[16]"},{"why":"Defines optimal mixing evolutionary algorithms, the variation principle on which GP-GOMEA's GOM is built.","marker":"[22]"}],"fun_headline_variants":["Modular GP-GOMEA speeds up recovery of modular ground truths","Multi-tree GP-GOMEA beats single-template on R2 and speed","Callable subexpressions boost GP-GOMEA's symbolic regression","Modular trees find smaller, more accurate expressions faster","GP-GOMEA with modular templates recovers truths faster"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that modular GP-GOMEA generally outperforms single-template GP-GOMEA rests on a comparison in which every synthetic ground truth is built from reusable subexpressions and the real-world runs use a population size tuned on only two of the synthetic problems, so the advantage may not carry over to non-modular problems or to a tuned baseline.","fun_headline_variants_meta":{"raw":{"variants":["Modular GP-GOMEA speeds up recovery of modular ground truths","Multi-tree GP-GOMEA beats single-template on R2 and speed","Callable subexpressions boost GP-GOMEA's symbolic regression","Modular trees find smaller, more accurate expressions faster","GP-GOMEA with modular templates recovers truths faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001296,"raw_usage":{"total_tokens":5299,"prompt_tokens":967,"completion_tokens":4332,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":4259}},"tokens_in":583,"tokens_out":4332,"duration_ms":32792,"temperature":1.0,"reasoning_tokens":4259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:21:50.285013+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both methods on a set of problems whose ground truths contain no reusable subexpressions, giving each method the same total node budget and a per-dataset tuned population size; if modular GP-GOMEA does not match or beat single-template GP-GOMEA in $R^2$ and recovery rate on most of those problems, the central claim is falsified.","supporting_citations":[{"cited_title":"Evolu- tionary computation 29(2), 211–237 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the base GP-GOMEA algorithm, the fixed-tree-template representation, linkage-tree FOS learning, and gene-pool optimal mixing that Modular GP-GOMEA extends."},{"cited_title":"In: International Conference on Parallel Problem Solving from Nature","cited_arxiv_id":null,"evidence_quote":"Prior GOMEA variant with Cartesian genetic programming that first enabled automatic discovery of duplicate subexpressions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Embedded CGP, which adds functional subexpressions that take different inputs; Modular GP-GOMEA extends this to hierarchical calls."},{"cited_title":"In: Foundations of Genetic Algorithms, vol","cited_arxiv_id":null,"evidence_quote":"Introduces hierarchically callable automatically defined functions, the modularity concept this representation realizes within GP-GOMEA."},{"cited_title":"Statistics and computing 4, 87–112 (1994)","cited_arxiv_id":null,"evidence_quote":"Provides the genetic programming baseline, half-and-half initialization, and the ADF comparison points used in the experiments."},{"cited_title":"In: Proceedings of the Genetic and Evolutionary Computation Conference Companion","cited_arxiv_id":null,"evidence_quote":"Basis for coefficient mutation during gene-pool optimal mixing, used in all runs."},{"cited_title":"Improving the efficiency of GP-GOMEA for higher-arity operators","cited_arxiv_id":"2402.09854","evidence_quote":"Source of the 25 coefficient bins and higher-arity operator handling adopted in the experimental setup."},{"cited_title":"In: Parallel Problem Solving from Nature-PPSN XII: 12th International Conference, Taormina, Italy, September 1-5, 2012, Pro- ceedings, Part II 12","cited_arxiv_id":null,"evidence_quote":"Adaptively gridded elitist archive used to maintain the multi-objective trade-off front between expression size and $R^2$."},{"cited_title":"In: Proceed- ings of the 13th annual conference on Genetic and evolutionary computation","cited_arxiv_id":null,"evidence_quote":"Defines optimal mixing evolutionary algorithms, the variation principle on which GP-GOMEA's GOM is built."}],"review_version":1}