{"id":"22a52fa3-0f97-408a-a0e9-3b2ac0cbc0c3","arxiv_id":"2412.05947","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A SISSO-based active-learning workflow with bagging and Monte-Carlo feature dropout found 12 acid-stable oxides from 1470 candidates in 30 DFT-HSE06 iterations.","lead":"An active-learning workflow that uses SISSO symbolic regression with ensemble uncertainty estimates guided expensive DFT-HSE06 calculations to find acid-stable oxides for water splitting. It found 12 acid-stable materials from 1470 candidates in 30 iterations, compared to 2 by random selection.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 1's POF uses τ=0.00 while acid-stable is defined as ΔG≤0.1 eV/atom, so the acquisition objective is misaligned with the reported discovery target; the 12-in-30 efficiency claim depends on this arbitrary threshold choice.","rationale":"The paper's central demonstration is that a SISSO-based active-learning workflow with bagging plus Monte-Carlo feature dropout identifies 12 acid-stable oxides from 1470 candidates in 30 iterations, compared with 2 for random selection. For this comparison to be meaningful, the acquisition score must target the same property that is counted as success. Eq. 1 defines POF with τ=0.00, while the acid-stable label is ΔG_pbx^OER ≤ 0.1 eV/atom, as stated just before Figure 2. The text says POF quantifies the probability that an oxide is acid-stable, but acid-stable is not ΔG ≤ 0. Because σ varies across candidates, shifting τ to 0.1 is not a monotone relabeling; candidate rankings can change. Therefore the reported number of discoveries is sensitive to an apparently arbitrary threshold choice. This concern is more load-bearing than the Gaussian assumption alone, because even a perfectly calibrated Gaussian POF with the wrong target threshold optimizes the wrong objective. A rerun with τ=0.1, ideally repeated over seeds, would settle whether the efficiency claim is robust. I do not see a fatal flaw: the workflow and DFT data are described transparently, and the ensemble comparison is a reasonable contribution. The threshold mismatch is a concrete, fixable inconsistency that should be addressed before accepting the efficiency claim as stated. The reader's conditionality remains appropriate, hence the verdict is unchanged.","tokens_in":10059,"tokens_out":4133,"duration_ms":41495,"concrete_test":"Rerun the AL campaign exactly as described but with τ=0.1 eV/atom in Eq. 1, keeping k=10, MC dropout, the same initial 250 and candidate 1470 materials, and repeat with at least 5 random seeds for both POF and random selection. If the number of acid-stable oxides found in 30 iterations, or the gap relative to random selection, changes materially, the central efficiency claim is weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central efficiency claim is that POF-guided active learning finds 12 acid-stable oxides in 30 iterations. But POF in Eq. 1 is evaluated with τ=0.00, whereas an oxide is labeled acid-stable only if ΔG_pbx^OER ≤ 0.1 eV/atom. These thresholds are different, and the difference is not cosmetic: for a candidate with predicted mean 0.05 eV/atom and σ=0.03 eV/atom, POF(τ=0) ≈ 0.05 while POF(τ=0.1) ≈ 0.95. Because σ varies across candidates, switching to the stated stability threshold changes the POF ranking, not just by a constant shift. Thus the set of materials acquired, and hence the number of stable oxides found, is partly dependent on choosing τ=0 rather than the threshold used to define success. The paper gives no justification for this mismatch, and the random-selection baseline does not repair it. The Gaussian/calibration issue noted in the paper (mean z=1.76) compounds the problem, but the threshold mismatch is sufficient on its own to undermine the quantitative comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops an active-learning (AL) workflow based on ensembles of SISSO symbolic-regression models. To quantify prediction uncertainty, the authors compare three ensemble strategies (bagging, model-complexity bagging, and bagging with Monte-Carlo dropout of primary features) and find that the latter yields the lowest MAE and the most balanced miscalibration scores on held-out data. They then use this ensemble in an AL campaign with a probability-of-feasibility (POF) acquisition function to discover acid-stable oxides from a pool of 1470 candidates, using DFT-HSE06 as the ground-truth evaluator. In 30 AL iterations, they report identifying 12 acid-stable oxides, whereas random selection finds only 2. They also provide SISSO-derived descriptor maps of oxide stability and compare PBE and HSE06 predictions for the discovered materials.","tokens_in":10314,"tokens_out":3225,"duration_ms":31788,"significance":"If the findings are robust, this is a valuable demonstration of symbolic regression in closed-loop materials discovery, going beyond interpolation-based AL by offering interpretable descriptors and uncertainty estimates. The ensemble comparison is carefully done with 30 independent trials, and the public availability of code, data, and a tutorial strengthens reproducibility. The practical result—discovering acid-stable oxides that are missed by GGA-based screening—is of interest to the electrocatalysis community. However, the central efficiency claim rests on several choices that are not fully justified, notably the threshold τ used in the acquisition function and the lack of repeated AL campaigns; these need to be addressed to establish the quantitative advantage claimed.","major_comments":[{"comment":"Equation (1) defines the acquisition probability with τ = 0.00, but the paper labels an oxide as acid-stable only if ΔG_pbx^OER ≤ 0.1 eV/atom (stated in the paragraph after Eq. (1) and in Figure 2). Since σ_ESISSO varies across candidates, this threshold mismatch does not merely shift all POF values by a constant; it changes the ranking of candidates and therefore the set of materials acquired in each AL iteration. The reported efficiency gain (12 acid-stable oxides found by POF vs. 2 by random selection in 30 iterations) is thus contingent on an arbitrary threshold choice. The authors should either justify τ = 0, rerun the AL campaign with τ = 0.1, or at least show that the conclusions are insensitive to τ over a plausible range.","section":"Results (POF definition)"},{"comment":"The central claim that POF-guided AL identifies 12 acid-stable oxides in 30 iterations is based on a single AL campaign. The ensemble comparison in Figure 1 is repeated over 30 independent trials, but the AL results in Figure 2 are not; given the stochasticity of bootstrap sampling and Monte-Carlo feature dropout, a different random seed could yield a different trajectory and a different number of discoveries. The authors should report the distribution of outcomes over multiple campaigns (e.g., 5–10 seeds) or at least provide a sensitivity analysis with respect to the initial training set and the random seed.","section":"Results (AL campaigns)"},{"comment":"The ensemble method used for acquisition has a mean miscalibration score z = 1.76, indicating overconfident uncertainty estimates. Equation (1) treats σ_ESISSO as a Gaussian standard deviation, but the calibration analysis shows that POF values are not true probabilities. The paper acknowledges this limitation but does not test whether the AL efficiency actually derives from the uncertainty term. A useful control would be an acquisition strategy based only on the ensemble mean prediction (e.g., selecting the lowest predicted ΔG_pbx^OER) to separate the exploitation and exploration contributions; if that baseline performs comparably to POF, the uncertainty-driven exploration claim would need to be moderated.","section":"Results (uncertainty calibration)"}],"minor_comments":[{"comment":"The filled and open square markers are difficult to distinguish in grayscale; consider using different shapes or colors with accessible palettes.","section":"Figure 2"},{"comment":"The acronym 'POF' is defined as 'probability of feasibility', but the term 'feasibility' is not standard for a stability threshold; consider 'probability of stability' or define the target event more explicitly.","section":"Results (POF definition)"},{"comment":"In the operator set in Eq. (2), the division φ1/φ2 is included but there is no statement about how zero denominators are handled; please clarify this implementation detail.","section":"Methods (SISSO)"},{"comment":"The composition and selection criteria of the initial 250-oxide training set are deferred to the Supplementary Material; a brief statement in the main text would help the reader judge potential bias in the training distribution.","section":"Results (initial dataset)"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the journal's scope and the authors have made a strong effort in code/data sharing. The main concerns are the threshold mismatch in the acquisition function and the lack of repeated AL campaigns; both are addressable in a revision. The control experiment suggested in major comment 3 would also clarify whether the uncertainty term adds real value. I do not see a need to reject the paper at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nRead this one if you care about active learning for materials discovery. The paper's real contribution is a new ensemble method for SISSO—bagging plus Monte-Carlo feature dropout—that produces uncertainty estimates for a symbolic-regression model, and it demonstrates the method on a genuine screening problem: HSE06 Pourbaix stability of 1470 ternary oxides. The workflow finds 12 acid-stable oxides in 30 iterations, and the authors release their code, data, and a tutorial. That is a concrete, reproducible package.\n\nThe methodological novelty is legitimate. Single SISSO models are deterministic and give no error bars; the ensemble approach fixes that and improves held-out MAE from 0.34 to 0.26 eV/atom. The application is also well chosen: ternary oxides are rarely screened with HSE06 because of cost, and the paper shows five of the twelve hits are predicted unstable by PBE but stable by HSE06, which strengthens the argument for high-quality data. The descriptor maps are a nice bonus.\n\nThe soft spots are real but not fatal. The stress-test note is correct: Eq. (1) defines POF with τ=0, while an oxide is labeled acid-stable only if ΔG≤0.1 eV/atom. Because σ differs across candidates, this is not a constant shift—it changes the acquisition ranking and therefore the set of materials queried in 30 iterations. The random baseline does not fix this. The efficiency claim (12 vs 2) depends on that threshold choice, and the authors give no justification for the mismatch. Also, there is only one AL campaign per strategy, so the comparison has no variance estimate. The miscalibration (mean z=1.76) is acknowledged but it means the acquisition strategy is being driven by overconfident uncertainties. All of these are addressable: rerun with τ=0.1, repeat campaigns with different seeds, and compare against a threshold-matched acquisition function.\n\nThe core discovery is solid: the 12 materials are confirmed by independent HSE06 calculations, so they stand on their own. What is weakened is the quantitative efficiency claim, not the qualitative demonstration.\n\nSend this to review. A good referee will push on the threshold mismatch and the missing repeats, but the work is serious, reproducible, and useful for the materials-informatics community. I'd cite it.","headline":"A genuinely useful SISSO-based active-learning workflow with a real screening application, but the headline efficiency claim rests on a threshold mismatch that needs fixing.","tokens_in":10850,"tokens_out":2193,"would_cite":true,"duration_ms":20711,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An active-learning workflow using ensembles of SISSO symbolic-regression models finds 12 acid-stable oxides out of 1470 candidates in only 30 hybrid-DFT evaluations, versus 2 for random selection.","keywords":["active learning","symbolic regression","SISSO","acid-stable oxides","electrocatalysis","Pourbaix decomposition energy","uncertainty quantification","materials discovery"],"falsifier":"A decisive test would be to rerun the same 30-iteration campaign with an acquisition function that only exploits the ensemble mean, ignoring the uncertainty term; if the mean-only strategy also finds 12 acid-stable oxides, then the calibrated uncertainty estimates are not essential to the claimed gain.","tokens_in":9817,"feed_emoji":"🧪","tokens_out":6996,"duration_ms":63919,"temperature":0.7,"pith_summary":"This paper argues that active-learning materials discovery does not need to know the few governing parameters in advance, because symbolic regression can find them on the fly. The authors build ensembles of SISSO models, adding Monte-Carlo dropout of input features to ordinary bagging, and show that this reduces prediction error and tames the overconfidence of single SISSO or bagged models. Using the ensemble's probability of feasibility as the acquisition function, the workflow finds 12 acid-stable oxides for electrocatalytic water splitting out of 1470 candidates in just 30 high-cost hybrid-DFT evaluations, whereas random selection finds 2. If the approach holds, high-quality but expensive calculations can be spent where they matter, and the resulting descriptor maps give a physical explanation for the search.","feed_headline":"AI-guided search finds 12 acid-stable oxides in 30 rounds","feed_subtitle":"Symbolic-regression ensembles guide costly DFT to six times more hits than random screening","key_machinery":"The load-bearing mechanism is an ensemble construction the paper calls bagging with Monte-Carlo dropout of primary features: each ensemble member is a SISSO model trained on a bootstrap sample of the data with a random 20% subset of the 14 primary features retained. Averaging these models gives the mean prediction $\\Delta G_{\\text{pbx,ESISSO}}$, and their spread gives the uncertainty estimate $\\sigma_{\\text{ESISSO}}$ used in the acquisition function $\\text{POF} = F\\!\\left(\\frac{\\tau - \\Delta G_{\\text{pbx,ESISSO}}}{\\sigma_{\\text{ESISSO}}}\\right)$, the Gaussian cumulative probability that a candidate is acid-stable. Together they turn SISSO into a closed-loop discovery engine that selects one oxide per iteration for hybrid-DFT evaluation and retrains.","core_discovery":"The central claim is that bagging with Monte-Carlo dropout of primary features creates SISSO ensembles whose mean prediction and standard deviation are good enough to steer active learning. On a held-out test set the ensemble reaches a mean absolute error of 0.26 eV/atom, down from 0.34 for a single SISSO model and 0.29 for plain bagging, and its miscalibration score drops from 2.76 for bagging to 1.76, still overconfident but markedly improved. The acquisition function, probability of feasibility, is the Gaussian cumulative probability that a candidate's Pourbaix decomposition free energy lies below the stability threshold. In 30 iterations this strategy identifies 12 acid-stable oxides, several of which are missed by PBE-level screening, and the SISSO descriptor map shows the selected materials clustering in the low-energy region.","pith_inferences":["I infer that Monte-Carlo feature dropout works because it prevents the ensemble from over-agreeing on a single descriptor set; a similar effect could be obtained by sampling over the SISSO hyperparameters q and D, which the paper does not test.","I infer that the POF criterion will be most valuable in early iterations when the surrogate is least certain; after the descriptor space is well mapped, pure exploitation may become nearly as efficient, an effect visible in the shrinking error bars in the paper's Figure 2.","I infer that the 12 oxides, which are rich in Mo, Ta, and W, suggest a design rule worth testing in the laboratory: acid stability at pH 0 and 1.23 V correlates with these oxophilic early-transition-metal frameworks, and the descriptor map could be used to propose new ternary compositions.","I infer that the overconfidence remaining, with a mean z of 1.76, means the workflow's hit rate could be improved further by replacing the Gaussian cumulative-distribution assumption in POF with a heavier-tailed or nonparametric calibration, a modification the paper leaves open."],"forward_implications":["If correct, hybrid-DFT screening of oxide stability becomes practical for thousands of materials instead of dozens.","The same workflow can be applied to other properties whose governing parameters are unknown, as long as primary features and an ensemble of symbolic-regression models can be defined.","The descriptor maps produced by SISSO give interpretable axes for the materials space, showing which chemical motifs, such as Mo, Ta, and W, favor acid stability.","Because five of the twelve discovered oxides are misclassified by PBE, the result implies that cheaper exchange-correlation functionals can mislead high-throughput screens and that ensemble uncertainty flags where higher-level calculations are needed.","The workflow reduces the number of expensive calculations by roughly a factor of six relative to random selection in this pool, a direct measure of its efficiency."],"supporting_citations":[{"why":"Supplies the SISSO symbolic-regression method that generates low-dimensional descriptor expressions for the target property.","marker":"[13]"},{"why":"Provides the SISSO++ implementation used for all SISSO analysis in the workflow.","marker":"[15]"},{"why":"Defines the Pourbaix diagram framework used to compute the acid-stability target property.","marker":"[18]"},{"why":"Introduces the HSE06 hybrid functional used to generate the high-quality DFT labels for stability.","marker":"[19]"},{"why":"Enables efficient all-electron hybrid-functional calculations for the large set of materials and competing phases.","marker":"[20]"},{"why":"Defines the miscalibration score z used to compare uncertainty estimates across ensemble methods.","marker":"[27]"},{"why":"Provides the calibration-after-bootstrap approach that motivates the ensemble uncertainty quantification.","marker":"[28]"},{"why":"Gives prior identification of some acid-stable oxides, used as a comparison for the newly discovered materials.","marker":"[33]"},{"why":"Provides a previous high-throughput screening of acid-stable oxides whose stability thresholds and findings are compared with this work.","marker":"[35]"}],"fun_headline_variants":["Symbolic-regression ensemble finds 12 acid-stable oxides in 30 rounds","Ensemble SISSO active learning yields 12 acid-stable oxides","SISSO-guided search finds 12 acid-stable oxides in 30 iterations","Uncertainty-aware symbolic regression finds 12 acid-stable oxides"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ranking of candidates by probability of feasibility assumes that each prediction error is Gaussian with standard deviation equal to the ensemble's spread, and that this spread reliably orders which candidates the model knows least about; the paper's own calibration analysis shows the ensemble is still overconfident, so the search's efficiency rests on this unproven ranking property.","fun_headline_variants_meta":{"raw":{"variants":["Symbolic-regression ensemble finds 12 acid-stable oxides in 30 rounds","Ensemble SISSO active learning yields 12 acid-stable oxides","SISSO-guided search finds 12 acid-stable oxides in 30 iterations","Uncertainty-aware symbolic regression finds 12 acid-stable oxides"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000389,"raw_usage":{"total_tokens":2062,"prompt_tokens":971,"completion_tokens":1091,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":1009}},"tokens_in":587,"tokens_out":1091,"duration_ms":10522,"temperature":1.0,"reasoning_tokens":1009,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:10:18.001054+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be to rerun the same 30-iteration campaign with an acquisition function that only exploits the ensemble mean, ignoring the uncertainty term; if the mean-only strategy also finds 12 acid-stable oxides, then the calibrated uncertainty estimates are not essential to the claimed gain.","supporting_citations":[{"cited_title":"Ouyang, S","cited_arxiv_id":null,"evidence_quote":"Supplies the SISSO symbolic-regression method that generates low-dimensional descriptor expressions for the target property."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SISSO++ implementation used for all SISSO analysis in the workflow."},{"cited_title":"Pourbaix, Atlas of electrochemical equilibria in aque- ous solutions, NACE (1966)","cited_arxiv_id":null,"evidence_quote":"Defines the Pourbaix diagram framework used to compute the acid-stability target property."},{"cited_title":"Efficient All-electron Hybrid Density Functionals for Atomistic Simulations Beyond 10,000 Atoms","cited_arxiv_id":"2403.10343","evidence_quote":"Enables efficient all-electron hybrid-functional calculations for the large set of materials and competing phases."},{"cited_title":"Pernot, Calibration in machine learning uncertainty quantification: beyond consistency to target adaptivity, APL Machine Learning 1 (2023)","cited_arxiv_id":null,"evidence_quote":"Defines the miscalibration score z used to compare uncertainty estimates across ensemble methods."},{"cited_title":"Palmer, S","cited_arxiv_id":null,"evidence_quote":"Provides the calibration-after-bootstrap approach that motivates the ensemble uncertainty quantification."},{"cited_title":"Wang, Y.-R","cited_arxiv_id":null,"evidence_quote":"Gives prior identification of some acid-stable oxides, used as a comparison for the newly discovered materials."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides a previous high-throughput screening of acid-stable oxides whose stability thresholds and findings are compared with this work."}],"review_version":1}