{"id":"289169de-83e1-4c9b-944e-42c064b3598a","arxiv_id":"2507.12404","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A neural network ranks features and symbolic regression fits a three-term formula (μ/t, μ/RA, LUMO) for OER activity in 23 perovskites, reporting a validation MAE of 20.6 meV.","lead":"Using 23 measured perovskite samples, the authors combine a neural network with symbolic regression to rediscover the known μ/t activity descriptor and to propose a new formula that also includes LUMO energy. The workflow is clear and the writing is clean, but the new claims rest on a 5-sample validation set that was used both to select features and to pick the best of 432 formulas.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Validation-set reuse during feature and formula selection makes the reported 20.6 meV validation MAE an optimistically biased estimate; the LUMO descriptor claim is not established.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the 5-sample validation set is used for permutation importance, for selecting the three features, and for choosing the best symbolic formula, and then the same set is used to report the final validation MAE. This creates a clear selection-bias pathway. I agree with the reader's assessment that the Phase I reproduction of the known μ/t descriptor is credible and provides some independent support for the framework, but the Phase II novelty claims—LUMO as a genuine electronic descriptor and a 20.6 meV validation MAE indicating improved generalizability—are not supported by the reported protocol. The proposed held-out test would directly settle whether the 20.6 meV value is unbiased; if it fails, the paper would need to be revised to present Phase II as exploratory hypothesis generation rather than validation. Because the reader already assigned a CONDITIONAL verdict and the concern reinforces rather than changes that assessment, no adjustment to the reader's verdict is needed.","tokens_in":11019,"tokens_out":4308,"duration_ms":50858,"concrete_test":"Hold out the 5 validation samples completely until the end. On the 18 training samples only, re-run the full Phase II protocol: neural-network training, permutation importance (using internal CV or a training-only split), selection of μ/t, μ/RA, and LUMO, and gplearn selection among 432 hyperparameter combinations using only training-based validation. Then fit the final formula on the 18 samples and evaluate once on the untouched 5 samples. If the held-out MAE is substantially above 20.6 meV (e.g., >30 meV), the reported validation MAE and the LUMO claim are artifacts of validation-set reuse. As a complementary control, repeat the same pipeline with LUMO values randomly permuted across the 18 samples; if a similarly low 'best' validation MAE and LUMO selection occur under permutation, selection bias alone explains the result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that the Phase II formula with μ/t, μ/RA, and LUMO achieves improved accuracy and identifies LUMO as a key electronic descriptor—rests on a statistical protocol that uses the same 5-sample validation set for multiple selection steps. In Phase II, permutation feature importance is computed on the validation set (Methods), which determines that LUMO is a key feature. Then gplearn is run with only the three chosen features, and among the 432 hyperparameter combinations, the formula with the lowest validation MAE is selected (Results: 'The better-performing formula achieved a training MAE of 0.0221 eV and a validation MAE of 0.0206 eV'). Table S3 shows exactly two formulas contained all three features and the better one was chosen by validation MAE. With n=5 validation points, selecting the better of two candidate formulas that were themselves produced by a 432-formula search can lower the validation MAE substantially below the true generalization error. The 0.2 meV improvement over the Phase I formula (20.8 vs 20.6 meV) is well within the standard error of a mean absolute error computed on 5 samples. Additionally, the physical interpretation of LUMO is not independently validated: the Matminer AtomicOrbitals LUMO is a composition-weighted average of atomic LUMO levels, not a solid-state eigenvalue, and the Sabatier mechanism is invoked post hoc without electronic-structure calculations. Therefore the statistical premise required for the headline claim—unbiased generalization of the 20.6 meV validation MAE and genuine predictive contribution of LUMO—is not met by the reported protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a two-phase machine-learning workflow for discovering interpretable symbolic formulas that predict oxygen evolution reaction (OER) activity (VRHE) of oxide perovskites from a small dataset (23 samples). Phase I uses seven structural features, adds engineered composites (μ/t, μ/RA, RA·t), and derives a formula (μ/RA)^0.5 + μ/t + 0.775 with reported training and validation MAEs of 22.8 and 20.8 meV. Phase II expands to 164 Matminer features, filters to 46, identifies LUMO energy as a top descriptor via permutation importance, and reports a final formula μ/t + μ/RA + E_LUMO + (−E_LUMO)^0.5 + 0.780 with training and validation MAEs of 22.1 and 20.6 meV, claiming improved accuracy and strong physical interpretability. The paper also benchmarks the neural network against Ridge, random forest, XGBoost, and PCA+XGBoost.","tokens_in":11214,"tokens_out":5183,"duration_ms":54692,"significance":"If the statistical foundation were sound, this paper would be a valuable demonstration of combining neural-network feature ranking with symbolic regression for data-scarce materials problems. The workflow is clearly described, the benchmark against traditional ML models is useful, and the idea of an electronic descriptor alongside structural descriptors is interesting. However, the core claim - that LUMO is a genuine descriptor and that the Phase II formula achieves improved validation accuracy - rests on a statistical protocol that reuses the same five-sample validation set for feature selection, formula selection, and final performance reporting. The reported validation MAEs are therefore better interpreted as selections, not predictions. The physical interpretation of LUMO is also not independently validated, as it uses an atomic composition-weighted proxy rather than a solid-state electronic level. With explicit acknowledgment of these limitations or additional validation, the work could be reframed as a promising hypothesis-generation pipeline, but the current claims are not supported by the evidence presented.","major_comments":[{"comment":"The permutation feature importance analysis used to select LUMO as a key descriptor was performed on the same five-sample validation set that later serves as the evidence for the final formula's accuracy (Methods: 'The analysis was performed on the validation set using R² as the scoring metric'). This creates a selection-dependence loop: the validation set is used to rank features, then to choose the symbolic-regression formula, and then to report the headline validation MAE of 20.6 meV. With n = 5, a 0.2 meV difference between Phase I and Phase II validation MAEs is far smaller than the sampling uncertainty of the mean absolute error, so the abstract's claim of 'improved accuracy' for the Phase II formula is not supported. The paper should either report an independent holdout (e.g., leave-one-out or nested cross-validation that includes the feature and formula selection steps) or clearly label the reported validation numbers as selection results rather than generalization estimates.","section":"Methods, 'Neural Network Training'"},{"comment":"The final formula was selected from 432 gplearn hyperparameter configurations by validation MAE, and among the two formulas that contained all three features, the one with lower validation MAE (0.0206 vs. 0.0283 eV, Table S3) was chosen. Because the same five validation points were used for this selection, the reported validation MAE is an optimistically biased estimate of the formula's performance on unseen materials. No correction for this selection bias is provided, and Table S3 does not report any confidence interval or per-point error distribution. The claim that the formula 'achieves the lowest validation error' is therefore not evidence that it generalizes; at minimum, the paper must quantify the bias (e.g., through a bootstrap over the selection procedure) or treat the formula as a hypothesis to be tested on new data.","section":"Results, 'Refined Formula and Performance'"},{"comment":"The paper interprets the LUMO feature as a solid-state electronic energy level that controls binding of OER intermediates via the Sabatier principle (Figure 5b). However, as the text itself states, the feature is Matminer's AtomicOrbitals LUMO, a composition-weighted average of atomic LUMO levels rather than a solid-state eigenvalue. No electronic-structure calculation is performed to support the claim that this atomic proxy corresponds to the perovskite LUMO level. The non-monotonic relationship in Figure 5a is based on 23 points and an optimal window (-0.34 to -0.32 eV) that is inferred from the same data, so the mechanistic narrative is post hoc. The identification of LUMO as a 'key electronic descriptor' is thus not established; the authors should either validate the proxy against first-principles calculations or moderate the claim to say that the composition-weighted atomic LUMO is a promising empirical feature.","section":"Results, 'Physical Interpretability of LUMO'"}],"minor_comments":[{"comment":"Several entries in Tables S1 and S2 contain typographic artifacts such as repeated superscripts (e.g., '0.50.5') and missing parentheses (e.g., formula I), making the formulas hard to read; please regenerate the tables with a proper equation renderer.","section":"Supplementary Tables S1 and S2"},{"comment":"The caption refers to 'the single green point' without defining the color scheme for the points; please add a legend or explain the color mapping in the caption.","section":"Figure 5d"},{"comment":"The sentence 'suggesting to increase t, decrease μ, and adjust components to acquire optimal LUMO energy' is incomplete as a design rule; clarify how the formula's coefficients imply these recommendations, given the non-monotonic LUMO term.","section":"Discussion, 'Interpretation of Descriptors'"},{"comment":"The description of the hyperparameter grid states p_crossover ranged from 0.5 to 0.925 in increments of 0.025, but it does not specify how the remaining probability mass is split among the three mutation operators (subtree, hoist, point); please provide the exact grid, as the reproducibility of the 432-formula search depends on it.","section":"Methods, 'Symbolic Regression with gplearn'"}],"recommendation":"major_revision","confidential_remarks":"The manuscript presents an interesting and clearly written pipeline, but the central statistical flaw is substantial: the five-sample validation set is used for feature selection, formula selection, and final performance reporting, making the headline validation MAE an optimistically biased estimate. I am not convinced that the LUMO descriptor claim is established beyond a hypothesis-generation level. The paper may benefit from being reframed as a proposal for a workflow with the LUMO descriptor as a candidate, or from adding an independent validation set. The work fits the scope of physics.data-an, but the authors should be asked to address the selection-bias issue directly before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this paper's Phase I is a solid, useful reproduction of Weng et al.'s μ/t descriptor with a small, plausible improvement. The Phase II claim that LUMO energy is a key electronic descriptor with 'improved accuracy' is not supported by the protocol as reported. The headline validation MAE of 20.6 meV is optimistically biased because the same 5 validation samples were used for permutation feature importance, formula selection among 432 gplearn candidates, and final evaluation. That is not a hidden flaw; the Methods state it plainly (permutation importance 'was performed on the validation set'). The paper just does not acknowledge the resulting selection bias. With n=5, the 0.2 meV improvement over Phase I's 20.8 meV is within the noise of a mean absolute error. The two candidate formulas in Table S3 were compared on those same 5 points, so the 'better' one is the one that fits the validation set best, not necessarily the one that generalizes best.\n\nWhat is genuinely new: the specific Phase II formula including E_LUMO and (−E_LUMO)^0.5 is not in the cited prior literature, and the idea that a composition-weighted atomic LUMO proxy could matter for OER activity is worth a test. But the evidence here is not that test. The Matminer LUMO is not a solid-state eigenvalue; the Sabatier-based explanation is invoked post hoc. The physical story is a hypothesis, not a finding.\n\nWhat the paper does well: the workflow is clearly described, the baseline comparisons are transparent, and Phase I honestly reproduces prior work rather than ignoring it. The framework of NN-guided feature selection followed by symbolic regression is reasonable for small-data materials problems. But the statistical handling of the tiny validation set is the load-bearing wall, and it has a crack running through the central claim.\n\nWho this is for: materials informatics readers who want a worked example of a two-phase descriptor-discovery pipeline. It will be useful as a cautionary case study in selection bias if the authors revise the evaluation protocol (e.g., nested CV or a true held-out set) or reframe the LUMO claim as hypothesis generation. I would not cite the LUMO result as established. But the paper deserves a serious referee: the framework is relevant, the issue is fixable, and a good referee could push the authors to produce trustworthy numbers.","headline":"A clearly described two-phase pipeline that honestly reproduces the known μ/t descriptor, but the LUMO claim rests on validation-set reuse and the reported improvement is noise.","tokens_in":11924,"tokens_out":1731,"would_cite":false,"duration_ms":20623,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a two-phase neural-network-guided symbolic regression pipeline can discover accurate, interpretable descriptors for oxygen evolution reaction activity in oxide perovskites, finding that a formula using μ/t, μ/RA…","keywords":["oxygen evolution reaction","oxide perovskites","symbolic regression","neural network feature importance","LUMO energy descriptor","descriptor discovery","small-data machine learning","overpotential prediction"],"falsifier":"Re-run the pipeline with the five validation samples completely excluded from feature ranking and formula selection, or with leave-one-out cross-validation across all 23 samples, and check whether a LUMO-containing formula is still chosen and whether its out-of-sample error stays near 20.6 meV. Independently, compare the composition-weighted atomic LUMO proxy against measured or density-functional-theory-calculated solid-state LUMO levels for the same perovskites; if the proxy does not track the true LUMO, the proposed electronic interpretation would need revision.","tokens_in":10689,"feed_emoji":"⚡","tokens_out":6789,"duration_ms":71666,"temperature":0.7,"pith_summary":"The paper claims that a two-phase machine-learning pipeline can find compact, interpretable formulas for oxygen evolution reaction (OER) activity in oxide perovskites even when only 23 samples are available. In the first phase, a neural network ranks a small set of structural descriptors and engineered composite features such as μ/t and μ/RA; symbolic regression then recovers and slightly improves the known μ/t descriptor. In the second phase, the feature space is expanded to 164 candidates, reduced to 46, and the neural network identifies LUMO energy as a new electronic descriptor. A symbolic formula combining μ/t, μ/RA, and LUMO energy predicts overpotential with training and validation mean absolute errors of 22.1 and 20.6 meV, respectively. If correct, this means interpretable descriptors can be extracted without sacrificing accuracy in data-scarce materials problems.","feed_headline":"Three descriptors predict perovskite OER activity to ~21 meV","feed_subtitle":"The winning formula pairs two perovskite geometry ratios with LUMO energy, revealing an optimal electronic window.","key_machinery":"The machinery is a two-phase pipeline. A regularized three-layer neural network is trained on the available samples, and permutation feature importance—computed by shuffling each feature and measuring the drop in validation performance—identifies which descriptors matter. Composite features are then engineered from the top structural descriptors, and in the expanded phase the descriptor space is filtered by variance and correlation thresholds before symbolic regression searches over candidate formulas. The central object that carries the final result is the three-descriptor formula $\\frac{\\mu}{t}+\\frac{\\mu}{R_A}+E_{LUMO}+(-E_{LUMO})^{0.5}+0.780$, interpreted through an optimal LUMO window that reflects intermediate binding strength on the catalyst surface.","core_discovery":"On the paper's own terms, the central discovery is that OER overpotential for oxide perovskites can be described by an explicit formula, $\\frac{\\mu}{t}+\\frac{\\mu}{R_A}+E_{LUMO}+(-E_{LUMO})^{0.5}+0.780$, where $\\mu$ is the octahedral factor, $t$ the tolerance factor, $R_A$ the A-site ionic radius, and $E_{LUMO}$ a composition-weighted estimate of the lowest unoccupied molecular orbital energy. The LUMO term is not treated as a mere fitting variable: measured overpotential is non-monotonic in LUMO energy, with the best samples falling in an optimal window around $-0.34$ to $-0.32$ eV, which the authors explain through the Sabatier principle as balanced adsorption and desorption of reaction intermediates. The same workflow reproduces and improves the earlier μ/t rule when restricted to seven structural features, and the expanded-feature phase shows that the LUMO term can be discovered from compositional information alone.","pith_inferences":["The paper leaves untested whether the 20.6 meV validation error is optimistic, because the same five validation samples participate in both feature ranking and formula selection; an independent test set or nested validation would be needed to certify the number.","Because the LUMO feature is a composition-weighted average of atomic LUMO values rather than a computed solid-state level, the apparent electronic descriptor may inherit its predictive power from electronegativity and valence-orbital information; testing against explicit electronic-structure calculations would show whether the mechanism is genuinely electronic.","The framework is likely to transfer to other data-scarce structure-property problems, but its main advantage over direct symbolic regression should shrink as sample sizes grow and plain symbolic regression becomes more stable."],"forward_implications":["If the central claim is correct, LUMO energy becomes a physically meaningful electronic descriptor for OER activity in oxide perovskites, not just a statistical correlate.","The formula gives a concrete screening rule: favor low μ/t, low μ/RA, and LUMO energies in roughly the -0.34 to -0.32 eV window to minimize overpotential.","The two-phase approach demonstrates a general recipe for small-data descriptor discovery: use a neural network to cut the feature space before symbolic regression, rather than running symbolic regression on all features.","Phase I independently confirms and slightly improves the previously reported μ/t descriptor, suggesting the pipeline can recover known physics before finding new descriptors."],"supporting_citations":[{"why":"Provides the 23-sample OER dataset, the seven initial descriptors, and the earlier μ/t formula that Phase I reproduces and improves.","marker":"[18]"},{"why":"Supplies the genetic-programming symbolic regression method used to search candidate formulas.","marker":"[20]"},{"why":"Supplies the Sabatier-principle reasoning used to interpret the optimal LUMO window as balanced intermediate binding.","marker":"[22]"},{"why":"Supplies the featurizers that generate the expanded 164-feature space from which LUMO emerges.","marker":"[24]"},{"why":"Provides the permutation feature importance scoring used to rank descriptors and guide the symbolic regression input.","marker":"[27]"},{"why":"Motivates symbolic regression as a route to interpretable scientific formulas from data.","marker":"[23]"}],"fun_headline_variants":["LUMO plus geometry ratios: explicit perovskite OER formula to ~21 meV","Neural-guided symbolic regression reveals perovskite OER descriptor","Mu/t, mu/RA, LUMO: new perovskite OER descriptor predicts activity","Data-scarce perovskite OER deciphered by NN-guided formula search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is statistical: the same five held-out samples are used to rank features, to select the best of 432 symbolic formulas, and then to report the final validation error, so the 20.6 meV figure assumes this procedure does not bias the estimate.","fun_headline_variants_meta":{"raw":{"variants":["LUMO plus geometry ratios: explicit perovskite OER formula to ~21 meV","Neural-guided symbolic regression reveals perovskite OER descriptor","Mu/t, mu/RA, LUMO: new perovskite OER descriptor predicts activity","Data-scarce perovskite OER deciphered by NN-guided formula search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2342,"prompt_tokens":995,"completion_tokens":1347,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":1264}},"tokens_in":611,"tokens_out":1347,"duration_ms":14000,"temperature":1.0,"reasoning_tokens":1264,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:47:49.972527+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline with the five validation samples completely excluded from feature ranking and formula selection, or with leave-one-out cross-validation across all 23 samples, and check whether a LUMO-containing formula is still chosen and whether its out-of-sample error stays near 20.6 meV. Independently, compare the composition-weighted atomic LUMO proxy against measured or density-functional-theory-calculated solid-state LUMO levels for the same perovskites; if the proxy does not track the true LUMO, the proposed electronic interpretation would need revision.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the 23-sample OER dataset, the seven initial descriptors, and the earlier μ/t formula that Phase I reproduces and improves."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the genetic-programming symbolic regression method used to search candidate formulas."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Sabatier-principle reasoning used to interpret the optimal LUMO window as balanced intermediate binding."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the featurizers that generate the expanded 164-feature space from which LUMO emerges."},{"cited_title":"Pedregosa, G","cited_arxiv_id":null,"evidence_quote":"Provides the permutation feature importance scoring used to rank descriptors and guide the symbolic regression input."},{"cited_title":"Schmidt and H","cited_arxiv_id":null,"evidence_quote":"Motivates symbolic regression as a route to interpretable scientific formulas from data."}],"review_version":1}