{"id":"5301575e-0c68-454b-98df-168037e988c1","arxiv_id":"2501.14768","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"An enhanced evolutionary equation-discovery framework, EPDE, is validated against SINDy on noisy ODE and PDE benchmarks, with mixed success rates.","lead":"This paper presents an updated version of EPDE, an evolutionary algorithm that discovers differential equations from noisy data by building terms from elementary functions rather than a fixed library. Its authors compare the method with the widely used SINDy framework on four benchmark equations.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 7 does not support 'notably surpass': at 1–2.5% noise SINDy has higher positive-discovery rates, and EPDE's 5% success is one run out of 40 credited via a per-experiment 'one positive' column.","rationale":"The paper has real engineering value: open-source code, multi-objective search, dynamic term generation, and the Chebyshev/ANN preprocessing choices are plausible contributions. But the central comparative claim is specifically that EPDE 'notably surpasses' SINDy and has 'superior noise robustness'. Table 7 contradicts that claim under the standard per-run metric: at 1% and 2.5% noise, SINDy's positive-discovery rate exceeds EPDE's, and EPDE's 5% edge is one successful run out of 40 total runs, reported via the nonstandard 'one positive in experiment' column. The paper's own Section 4 statement that high noise makes the wrong equation optimal explains why the true structure appears only rarely and undermines the noise-robustness narrative. The reader's weakest assumption is close but framed around search reliability and fitness ranking; I would sharpen it to the asymmetric success metric and the best-of-N credit given to EPDE but not to SINDy. Since the framework itself may still be a useful contribution, a conditional verdict pending corrected evaluation and toned-down claims is appropriate; no verdict change is needed beyond what the reader already recommended.","tokens_in":15352,"tokens_out":6228,"duration_ms":64678,"concrete_test":"Recompute the aggregate comparison from the provided GitHub repository using a per-run success definition for both methods: for each of the four benchmarks and each noise level (0, 0.5, 1, 2.5, 5%), generate at least 40 independent noise realizations; run SINDy once per realization with its reported sparsity parameter and EPDE once per realization; count the fraction of realizations whose output matches the ground-truth equation within the tolerance used in Tables 1–6. Rebuild Table 7 with these per-run fractions for both methods and report the raw per-run counts. If EPDE's per-run success is not above SINDy's at 1% and 2.5%, and if the 5% advantage disappears once best-of-N credit is removed, the headline claim should be downgraded or conditioned on the per-experiment best-of-10 reporting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 7 is the load-bearing evidence for the central claim that EPDE 'notably surpasses' SINDy in noise robustness, and the table does not support it. At 1% and 2.5% noise, SINDy's positive-discovery rate is 25% while EPDE's per-run rate is 10% and 2.5% respectively; EPDE is credited with a 25% 'one positive in experiment' rate at those noise levels, meaning at least one of ten runs succeeded in exactly one of the four benchmarks. At 5%, EPDE's Pos. column (2.5%) and one-positive column (25%) correspond to a single successful run out of ten runs in one benchmark, i.e., one of 40 total runs across all experiments. The paper's own Section 4 concedes that 'high noise contamination leads to the wrong equation being optimal from an optimization criteria standpoint,' which means the few EPDE successes are stochastic outliers against the objective, not evidence of systematic superiority. The comparison is also asymmetric: SINDy is deterministic and gets one output, while EPDE gets ten stochastic runs and is scored by the best run. The central comparative claim therefore rests on a metric that rewards lucky runs. This is an internal mismatch between the reported numbers and the headline assertion, not a disagreement with external consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the EPDE evolutionary equation-discovery framework by adding multi-objective optimization, improved handling of differential equation systems, a neural-network-based differentiation/preprocessing route, and automated hyperparameter management. The authors validate the method on Burgers, Korteweg-de Vries, Van der Pol, and Lotka-Volterra systems with noise levels from 0% to 5%, comparing term-recovery rates, coefficient errors, and full-equation recovery against pySINDy. The central claim is that stochastic, dynamically generated term libraries allow EPDE to notably surpass deterministic sparse-regression methods such as SINDy in noise robustness.","tokens_in":15599,"tokens_out":4140,"duration_ms":41189,"significance":"If the central claim were supported, the paper would make a useful contribution by demonstrating an evolutionary alternative to library-based sparse regression, with public code and benchmark comparisons. The framework's multi-objective formulation, dynamic term generation, and treatment of missing differentials are potentially valuable directions. However, the headline comparative claim is not supported by the paper's own summary table, and the evaluation procedure is asymmetric between the deterministic and stochastic algorithms. The contribution is therefore promising but needs a substantially revised and more careful empirical assessment before the stated conclusions can be accepted.","major_comments":[{"comment":"The reported positive-discovery rates contradict the claim that EPDE 'notably surpasses' SINDy. At 1% and 2.5% noise, SINDy recovers the correct equation in 25% of cases while EPDE's per-run rate is 10% and 2.5%, respectively; at 5% noise, EPDE's 2.5% positive rate and 25% one-positive rate correspond to a single successful run in one of the four benchmark experiments. The authors should either revise the comparative claim or provide a statistically justified reason why best-of-ten stochastic scoring is the appropriate basis for 'notably surpass'.","section":"Table 7, rows for 1%, 2.5%, and 5% noise"},{"comment":"The comparison is asymmetric: SINDy is deterministic and produces one model after hyperparameter selection, while EPDE is given ten stochastic runs and scored by the best run through the 'one positive in experiment' column. This metric rewards lucky runs and, with ten runs per experiment, cannot distinguish systematic superiority from chance. Please report the full distribution of outcomes over the ten runs, or use a matched criterion such as median or mean success rate, success in at least k of ten runs, or a statistical test with confidence intervals.","section":"Section 4, scoring procedure"},{"comment":"The statement that 'the algorithm can reliably converge to the correct equation ... only with noise levels equal to or lower than 1%' is not supported by the table. At 1% noise, EPDE recovers u'_t in 90% of runs but uu'_x in only 10%, and the reported coefficient for u'_t is far from the ground-truth value. The text should distinguish term-level inclusion from full-equation recovery and define 'reliably' operationally, since the table shows a rapid loss of full-structure recovery even at 1% noise.","section":"Section 3.1.1, Table 1"},{"comment":"The concession that 'high noise contamination leads to the wrong equation being optimal from an optimization criteria standpoint' directly undermines the noise-robustness claim in the regime where EPDE is presented as superior. If the true equation is not optimal under the fitness criterion, the paper should explain how a practitioner can identify the rare successful run without already knowing the true equation, or present a criterion under which the true equation is optimal on noisy data.","section":"Section 4, last paragraph"}],"minor_comments":[{"comment":"The introduction of Section 2 refers to 'Subsection 2.2' twice; the second reference should be to Subsection 2.3.","section":"Section 2, organization"},{"comment":"The definition of F_i repeats the same grid-node expression twice; it should list a range over grid nodes rather than duplicating the first entry.","section":"Equation (6)"},{"comment":"The confidence intervals for u^2 u'_t and u'_t are reported as 33.6e5 and 11.9e5, which are implausibly large relative to the coefficient estimates; please explain the source of this dispersion.","section":"Table 3, 0.5% noise row"},{"comment":"The notation 'Deriv. ord=(2,3)' is ambiguous; for the KdV experiments, the maximal derivative order should be stated explicitly as 3.","section":"Appendix A"},{"comment":"Several phrases appear to be typographical errors, including 'P of f spr' (likely 'offspring') and 'autonomous tokes' (likely 'tokens').","section":"Section 2.3"},{"comment":"The coefficient-error column is labeled 'best' for EPDE but the rule for selecting this best run is not specified; please state how the best run is chosen and report the variability across runs.","section":"Table 7"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a real engineering contribution: the EPDE framework now combines multi-objective evolutionary search, dynamic term generation, and neural/Chebyshev derivative preprocessing, with code on GitHub and a systematic comparison against pySINDy on four benchmark equations. Second, the paper's central claim—that EPDE notably surpasses SINDy in noise robustness—is not supported by its own numbers. Read Table 7 and you'll see SINDy actually reports higher positive-discovery rates at 1% and 2.5% noise (25% vs 10% and 2.5%). The 5% 'success' for EPDE is a single run out of forty, inflated by the 'one positive in experiment' column. That column is transparent but it rewards a lucky run, not systematic behavior.\n\nWhat's deserved: the integration is genuinely new compared to the authors' earlier EPDE and multi-objective papers. The comparison details, the run statistics, and the appendix with hyperparameters are useful. The authors are also honest about limitations: the method needs manual parameter setting, is slow, and they concede in Section 4 that high noise leads the wrong equation to be optimal. That concession is in tension with the abstract's 'robust equation extraction' claim, but it's there.\n\nSoft spots: (1) The headline overstates the evidence. For Burgers at 1% noise, they say the algorithm 'reliably converges,' but the key nonlinear term uu'_x is found in only 10% of runs. (2) The comparison is asymmetric: SINDy is deterministic and gets one shot; EPDE gets ten stochastic runs and the authors count the best. A fair comparison would report median or worst-case performance, or at least the full distribution. (3) The evolutionary search hyperparameters (populations of 8–12, 55–85 epochs) are small; there is no sensitivity analysis showing the results are robust to those choices. None of these flaws makes the framework worthless; they make the noise-robustness claim unproven.\n\nWho should read it: anyone working on equation discovery, especially on the practical side of evolutionary search and derivative preprocessing. It deserves peer review—the code and the benchmark comparison merit scrutiny—but the revision should re-analyze Table 7, either dropping the 'notably surpass' claim or providing a comparison that supports it. If I were the editor, I'd send it to review with a strong request for major revision.","headline":"A useful evolutionary equation-discovery framework, but its central noise-robustness claim is contradicted by the paper's own Table 7.","tokens_in":16181,"tokens_out":3034,"would_cite":false,"duration_ms":27077,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Evolutionary search beats SINDy on noisy equation discovery.","keywords":["equation discovery","evolutionary optimization","multi-objective optimization","SINDy","partial differential equations","noise robustness","symbolic regression","ordinary differential equations"],"falsifier":"A direct test would be to take a noisy dataset for a known equation, enumerate all candidate equations of the same length and token families that the evolutionary search could produce, and compute their discrepancy fitness; if any wrong equation scores better than the true one, then the algorithm's occasional successes depend on lucky search rather than on the fitness criterion, and the claimed noise robustness would fail in that regime.","tokens_in":1384,"feed_emoji":"🧬","tokens_out":3678,"duration_ms":64642,"temperature":0.7,"pith_summary":"This paper tries to establish that a stochastic, evolutionary approach to differential equation discovery can outperform the widely used sparse-regression method SINDy, especially when the data are noisy. The key idea is that instead of fitting a fixed library of candidate terms, the algorithm evolves equations from basic building blocks like derivatives and elementary functions, letting the term set adapt during the search. The authors enhance the EPDE framework with multi-objective optimization, improved handling of equation systems, and neural-network-based derivative estimation, then test it on Burgers, Korteweg-de Vries, Van der Pol, and Lotka-Volterra equations. If correct, this would make equation discovery less reliant on prior knowledge about the equation's form and more robust to experimental noise, at the price of much longer computation time.","feed_headline":"Evolutionary search beats SINDy on noisy equation discovery","feed_subtitle":"By building terms from elementary functions and differentials, EPDE adapts its search instead of relying on a fixed library.","key_machinery":"The engine of the method is an equation encoding as a tree whose leaves are tokens: derivatives and elementary functions such as trigonometric or polynomial factors, grouped into families. Terms are formed by multiplying tokens, and the equation is a linear combination of terms; a sparsity constant and complexity metric control parsimony. The search uses a multi-objective evolutionary algorithm based on dominance and decomposition, where each population member is associated with a weight vector in the objective space and a penalty-based intersection (PBI) value balances convergence and diversity. Fitness is computed either from the L2 discrepancy of the differential operator on the data grid or from the error of a neural-network-solved candidate equation, with weighted regression for coefficient estimation.","core_discovery":"The central claim is that dynamic term generation coupled with multi-objective evolutionary search distinguishes significant equation terms from noise more effectively than deterministic sparse regression on a predefined library. In the experiments, EPDE recovered the correct governing equation at noise levels up to 5% for the Lotka-Volterra system and with a higher frequency than SINDy across most tested cases, while SINDy reliably succeeded only on noiseless data. The paper also reports that when the correct structure is found, EPDE's coefficient estimates are comparable or slightly better in relative error, but that this robustness comes with a dramatic increase in runtime, from milliseconds to tens of seconds or more.","pith_inferences":["If the central claim holds, equation discovery becomes feasible for experimental datasets where a functional form is unknown and noise is non-negligible, such as ecology or climate data.","A natural test of the mechanism would be to compare EPDE against a weak-form or ensemble variant of SINDy, which were not included in the paper's comparison.","The reported 'wrong equation is optimal' failure at high noise suggests the fitness landscape, rather than the search algorithm alone, is the limiting factor; improving the discrepancy metric could extend the noise ceiling.","The evolutionary search could be hybridized with sparse regression as a local refinement step to reduce the runtime that currently limits practical use."],"forward_implications":["The term library adapts to the problem, so equations whose forms are not anticipated by a fixed candidate set can, in principle, be discovered.","At moderate noise levels, the evolutionary approach can recover correct structures more often than SINDy, including systems of ODEs.","When the correct structure is found, coefficient errors are comparable to or better than those of sparse regression.","The multi-objective formulation yields a Pareto set of candidate equations, letting a researcher inspect the trade-off between equation complexity and data fit.","The main practical drawback is computational cost, so SINDy may remain preferable when a reliable library is known and the equation is first-order in time."],"supporting_citations":[{"why":"Introduces SINDy, the sparse regression baseline that the paper's comparison is built against.","marker":"[1]"},{"why":"Describes the earlier EPDE framework and its sparse-regression-based component, which this paper extends.","marker":"[12]"},{"why":"Provides the Python implementation of SINDy used for the experimental comparison.","marker":"[18]"},{"why":"Supplies the fast non-dominated sorting procedure used to divide populations into dominance levels.","marker":"[20]"},{"why":"Establishes the dominance-and-decomposition multi-objective algorithm that the evolutionary search adapts.","marker":"[21]"},{"why":"Defines the normal-boundary intersection method for generating weight vectors that partition the objective space.","marker":"[22]"},{"why":"Offers the neural-network-based solver used to evaluate solution-based fitness for candidate equations.","marker":"[19]"},{"why":"Provides evidence that multi-objective optimization outperforms single-objective optimization in equation discovery, motivating the design choice.","marker":"[14]"}],"fun_headline_variants":["Evolutionary search finds equations SINDy misses in noise","EPDE: dynamic terms, not fixed library, wins on noisy data","No library? No problem: EPDE adapts to noisy physics data","EPDE robust to 5% noise; SINDy needs clean data","EPDE: term building from scratch beats SINDy on noisy equations"],"cache_read_input_tokens":18176,"weakest_assumption_plain":"That a few dozen generations of evolution over a small population can reliably locate the true equation in the combinatorially large term space, and that the chosen fitness function ranks that true equation as optimal even on noisy data.","fun_headline_variants_meta":{"raw":{"variants":["Evolutionary search finds equations SINDy misses in noise","EPDE: dynamic terms, not fixed library, wins on noisy data","No library? No problem: EPDE adapts to noisy physics data","EPDE robust to 5% noise; SINDy needs clean data","EPDE: term building from scratch beats SINDy on noisy equations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000672,"raw_usage":{"total_tokens":2996,"prompt_tokens":813,"completion_tokens":2183,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":429,"completion_tokens_details":{"reasoning_tokens":2089}},"tokens_in":429,"tokens_out":2183,"duration_ms":15189,"temperature":1.0,"reasoning_tokens":2089,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:27:54.662155+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to take a noisy dataset for a known equation, enumerate all candidate equations of the same length and token families that the evolutionary search could produce, and compute their discrepancy fitness; if any wrong equation scores better than the true one, then the algorithm's occasional successes depend on lucky search rather than on the fitness criterion, and the claimed noise robustness would fail in that regime.","supporting_citations":[{"cited_title":"Maslyaev, A","cited_arxiv_id":null,"evidence_quote":"Describes the earlier EPDE framework and its sparse-regression-based component, which this paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the dominance-and-decomposition multi-objective algorithm that the evolutionary search adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the normal-boundary intersection method for generating weight vectors that partition the objective space."},{"cited_title":"Hvatov, Automated differential equation solver based on the para- metric approximation optimization, Mathematics 11 (8) (2023) 1787","cited_arxiv_id":null,"evidence_quote":"Offers the neural-network-based solver used to evaluate solution-based fitness for candidate equations."},{"cited_title":"Maslyaev, A","cited_arxiv_id":null,"evidence_quote":"Provides evidence that multi-objective optimization outperforms single-objective optimization in equation discovery, motivating the design choice."}],"review_version":1}