{"id":"0824ae74-2a44-45b2-9cb8-780d5daca5d8","arxiv_id":"2608.09257","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Recursion coefficients from a tight-binding bond model, derived from DFT diatomic molecules, improve machine-learning predictions of zincblende versus rocksalt stability for octet binaries.","lead":"A machine-learning study adds tight-binding recursion coefficients, which encode bond information from DFT dimers, as features for predicting whether AB-type octet compounds prefer rocksalt or zincblende structures. The bond-informed features lower the prediction error on a benchmark of 82 compounds compared with earlier atomic-only descriptors, but the evaluation pipeline has data-leakage caveats.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pre-split sequential feature selection leaks test targets into model construction, so the reported 0.029 eV test RMSE is not an unbiased generalization estimate; a nested CV check is needed before the quantitative outperformance claim is accepted.","rationale":"The reader's weakest_assumption is exactly the concern I would flag. Section II E discloses that SFS is performed once on the complete dataset before the outer splits, so the held-out folds have influenced feature selection. This is load-bearing because the headline claim is quantitative (0.080 eV to 0.029 eV), and the bias is not neutral between the proposed models and the reference: the new models use data-driven SFS on their own feature sets, whereas the reference uses fixed features. Additional secondary issues compound the optimism: D2 outlier removal uses target information, invalid splits are excluded from the averaged metrics, and the best recursion order is chosen from test performance. The concrete nested-SFS test would determine whether the concern actually lands; if the nested RMSE is much higher, the quantitative superiority claim would need to be softened, though the directional finding may survive because even the AtomF-only KRR baseline outperforms the reference. The paper has independent support in that the recursion features come from BOPfox with TB parameters derived from DFT dimers rather than from the formation-energy target, and the improved performance is consistent across datasets and feature ladders. I therefore agree with the reader's conditional verdict and recommend no change to it.","tokens_in":19003,"tokens_out":5057,"duration_ms":54471,"concrete_test":"Re-run the full workflow on D1 and D2 with SFS moved inside the cross-validation: for each of the 150 ShuffleSplits, perform SFS on the 73/70 training compounds only, refit the KRR ensemble on the selected features, then score the 9/8 held-out test compounds. Report mean test RMSE over all 150 splits both with and without the R2<0 filter, for AtomF and for AtomF+RecN (N=1,...,8). If the nested mean test RMSE for the best AtomF+Rec6 model rises materially above the reported 0.029 eV (for example above 0.05 eV), the headline quantitative improvement is inflated by selection leakage; comparing the AtomF-only nested result will show whether the relative benefit of recursion features survives under an unbiased protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the claim that the reported cross-validated errors are honest estimates of generalization, i.e., that the 150-split protocol measures a pipeline that has not seen the held-out targets. Section II E breaks this: 'The SFS is performed once on the complete dataset, prior to the train-test splits described below, and the selected features are then kept fixed for all subsequent splits.' Because SFS scores candidate feature subsets by 10-fold KRR MSE, the full target vector, including the compounds that later form each 9- or 8-sample test fold, participates in choosing the features. With 82 or 78 samples and a large candidate feature space (AtomF plus recursion summaries up to order 8), the selection can overfit target noise, and the subsequent test RMSE is optimistically biased. The bias is not symmetric across the comparison: the AtomF+RecN models use data-driven SFS, while the Ghiringhelli et al. reference uses fixed features, so part of the reported improvement may be a selection-leakage artifact rather than a genuine generalization gain. Two further full-data choices push in the same direction: the isolation-forest outlier removal for D2 uses target values before defining splits, and the selection of Rec4/Rec6 as the 'best' feature set is made on the same test folds. The protocol therefore does not support the conclusion's quantitative '0.080 eV to 0.029 eV' framing. A nested SFS check would settle whether the qualitative directional finding, that recursion features help, survives without the leakage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a feature representation for octet AB binary compounds that combines elemental atomic features with recursion coefficients of a tight-binding bond model, where the pairwise Hamiltonians are downfolded from DFT dimer eigenstates. The authors evaluate kernel-ridge and SISSO ensemble models on the 82-compound dataset of Ghiringhelli et al. and on an outlier-reduced 78-compound version, using 150 repeated 90/10 splits and sequential feature selection. They report that adding recursion features up to order 4 (dataset D1) or order 6 (dataset D2) lowers the mean test RMSE from 0.139 eV to 0.046 eV and from 0.080 eV to 0.029 eV, respectively, relative to a reference model based on the three features of Ghiringhelli et al.","tokens_in":19417,"tokens_out":5399,"duration_ms":53483,"significance":"The core idea is attractive: recursion coefficients provide a physically motivated, target-independent description of the local bonding environment, and the paper asks a genuinely useful question about whether bond-level domain knowledge helps small-data stability prediction. The consistent monotonic improvement with recursion order on both datasets, the use of the same DFT data as Ref. 3, and the detailed per-compound analysis of failing splits are strengths. If the reported errors survive a nested feature-selection and outlier-removal protocol, the result would be a practically valuable demonstration of chemistry-aware descriptors for materials informatics. At present, however, the quantitative outperformance claim rests on a validation protocol that leaks test information into model construction, so the manuscript needs revision before the central quantitative claim can be accepted.","major_comments":[{"comment":"Section II E states that 'The SFS is performed once on the complete dataset, prior to the train-test splits described below.' Because SFS scores candidate feature subsets by 10-fold KRR MSE computed on all 82/78 compounds, the held-out test targets participate in choosing the features. The reported test RMSE is therefore not an unbiased estimate of generalization, and the comparison is asymmetric: the AtomF+RecN models use data-driven SFS, while the Ghiringhelli et al. reference uses fixed features, so part of the reported improvement from 0.080 eV to 0.029 eV could be an artifact of selection leakage. Please move SFS inside each outer training split or otherwise demonstrate that the selected features are insensitive to held-out targets, and report the re-estimated RMSE.","section":"II E and Fig. 4"},{"comment":"The outlier-removal procedure in Section II D uses the target variable in its first stage and the full descriptor set including recursion features in its second stage, both before any train/test split; the test folds of dataset D2 are therefore defined on a population already cleaned with test information. In addition, Section II E says that splits with negative R2 'were ruled out when averaging the metrics reported below,' which biases the reported means if the reference and the proposed models fail on different fractions of splits. Please report aggregate metrics over all 150 splits and use a nested or training-only outlier-removal rule, or quantitatively show that these choices do not affect the conclusion.","section":"II D and II E"},{"comment":"The selection of Rec4 on D1 and Rec6 on D2 as the best recursion order is made by inspecting test-set performance on the very same 150 splits that define the reported errors. This is model selection on the test set and can bias the reported 'best' RMSE downward. Also, the reference model's D1 test RMSE of 0.139±0.088 eV differs substantially from the 0.08 eV originally reported for the same three-feature descriptor in Ref. 3, and the manuscript does not explain whether the reference was re-evaluated under the same split protocol or what causes this discrepancy; please clarify this in the revision.","section":"III A-B, Figs. 5-6"}],"minor_comments":[{"comment":"The x-axis labels in Figures 5 and 6 are truncated, with entries such as 'AtomF+Rec' missing the recursion order, which makes it difficult to verify the claimed monotonic trend; please provide complete labels or a legend.","section":"Figs. 5-6"},{"comment":"The flowchart places the SFS step outside the outer split loop, which visually encodes the leakage problem identified above; the figure should be updated to reflect the corrected validation protocol.","section":"Fig. 4"},{"comment":"Equation (2) uses inconsistent notation for the averaged recursion coefficient (as_n + 3ap_n), mixing subscripts and superscripts; please harmonize the notation throughout.","section":"Eq. (2)"},{"comment":"The orbital-resolved (s/p) KRR curves in the supplementary figures are shown without error bars or summary statistics, unlike the other curves; please report the underlying values in a table.","section":"Supplementary Figures S1-S6"},{"comment":"The data availability statement only promises that data and notebooks will be made available to reviewers and public upon acceptance; for reproducibility, please provide a permanent repository link or an anonymized copy in the revision.","section":"Data availability"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and the empirical trend is consistent, but the validation protocol is the main obstacle to accepting the quantitative claims. I would support a major revision rather than rejection because the leakage concerns can be addressed by nested cross-validation and a more transparent reporting of all splits. I also recommend that the editor require the data and code to be actually available during the revision, since the current statement only promises reviewer access."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something real: it takes cheap, physics-based recursion features from DFT-dimer-derived tight-binding Hamiltonians and shows, in a systematic way, that they improve the classic octet binary RS/ZB stability benchmark. The recursion features are not fitted to the 82 formation energies, which is a genuine strength. The monotonic improvement as recursion order increases, on both the full and cleaned datasets and against an AtomF-only baseline, is a credible directional signal. The authors also do a lot of things right: they report invalid-split behavior, analyze per-compound failure enrichment, and provide enough detail to reproduce the workflow.\n\nThat said, the headline numbers are not honest generalization estimates. The SFS feature selection is run once on the complete dataset before the train/test splits are defined, so the held-out folds have already influenced which features are kept. With 82 samples and a large candidate space, this can overfit target noise. The AtomF+RecN models are more exposed to this than the fixed-feature Ghiringhelli reference, so part of the reported improvement over 0.080 eV may be an artifact. The same direction: D2 outlier removal uses the full target vector before splitting, and calling Rec6 the \"best\" means selecting on the test folds. The exclusion of invalid splits from averaged metrics is a smaller issue on D2 (0.1% of splits) but still a form of cherry-picking.\n\nThe reference-model numbers also need explanation. Their re-evaluation of the Ghiringhelli features gives 0.139 eV test RMSE, well above the originally reported 0.080 eV. If the protocol differs, fine, but the paper should say why. Without that, the comparison baseline is unstable.\n\nAll of this pushes the quantitative claims in the same direction: the 0.029 eV RMSE is optimistic. But I would be surprised if the directional finding collapsed. The AtomF baseline uses the same SFS leakage and still underperforms the recursion-augmented models, so the qualitative conclusion is probably robust. The paper needs a nested SFS check before the magnitudes are taken seriously.\n\nThis deserves a serious referee. The feature idea is practical, the authors are honest about the invalid splits, and the methodological fix is straightforward. I would read a revised version and would want to see whether the improvement survives nested CV. Until then, I would not cite the numeric claims.","headline":"Bond-informed recursion features are a credible advance for RS/ZB stability prediction, but the headline RMSE numbers are inflated by pre-split feature selection and should not be taken at face value.","tokens_in":19917,"tokens_out":2781,"would_cite":false,"duration_ms":31853,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adding bond-aware recursion coefficients to atomic features more than halves the machine-learning error for predicting rocksalt-versus-zincblende stability of octet AB compounds.","keywords":["octet binary compounds","rocksalt","zincblende","recursion coefficients","tight-binding","machine learning","formation energy","sequential feature selection"],"falsifier":"Repeat the same 150-split ensemble protocol with sequential feature selection performed separately inside every training fold; if the AtomF+Rec6 test RMSE on the 78-compound dataset rises substantially above 0.029 eV, the claimed improvement over the reference is partly an artifact of selection leakage.","tokens_in":18834,"feed_emoji":"⚫️","tokens_out":12311,"duration_ms":106303,"temperature":0.7,"pith_summary":"The paper tries to establish that a machine-learning model can predict which of two competing crystal structures, rocksalt or zincblende, an octet AB binary compound prefers more accurately when the feature vector encodes the actual A-B bond rather than only atomic properties. The bond knowledge is injected through recursion coefficients computed from a tight-binding bond model whose pairwise Hamiltonians are downfolded from density-functional-theory eigenstates of diatomic molecules. On a refined set of 78 octet binaries, a kernel-ridge ensemble using atomic features plus six recursion orders reaches a mean test RMSE of 0.029 eV, against about 0.080 eV for the earlier three-feature descriptor. On the full 82-compound dataset, the best bond-aware model reaches 0.046 eV in this paper's re-evaluation, versus 0.139 eV for the reference. If the result holds, cheap coarse-grained electronic-structure features can replace expensive fingerprints for small-data materials problems.","feed_headline":"Recursion features cut octet-stability error by more than half","feed_subtitle":"Tight-binding bond information cuts test RMSE from 0.080 to 0.029 eV for 78 binary AB compounds.","key_machinery":"The central object is the recursion representation of the local electronic density of states: a Lanczos algorithm transforms the tight-binding Hamiltonian into a tridiagonal matrix whose diagonal and off-diagonal entries $a_n$ and $b_n$ encode increasingly distant coordination shells as a continued fraction. The chemistry enters through pairwise Hamiltonians taken from downfolded DFT eigenstates of diatomic molecules, so each A-B pair carries bond-specific domain knowledge. The paper converts these coefficients into per-compound features by aggregating site-resolved values (max, min, average) and taking differences and ratios between rocksalt and zincblende, producing the feature ladder AtomF+RecN. The argument is carried by an ensemble protocol: 20 independently trained KRR or SISSO (sure-independence-screening-and-sparsifying-operator) models per split, combined by inverse-MSE weighting, evaluated on 150 repeated 90/10 splits, with sequential feature selection (SFS) reducing each feature set before training.","core_discovery":"The paper's central claim is that explicitly incorporating interatomic-bond information into the feature vector systematically improves prediction of the formation-energy difference $\\Delta E_{AB}=E_{\\mathrm{RS}}-E_{\\mathrm{ZB}}$ between rocksalt and zincblende/wurtzite structures for the 82 octet $AB$ compounds used in Ref. [3]. The bond information is carried by recursion coefficients $a_n$ and $b_n$, obtained by Lanczos tridiagonalization of a tight-binding bond model with chemistry-specific Hamiltonians transferred from DFT eigenstates of diatomic molecules. Features are built by comparing site-resolved recursion coefficients between the two structures, via max/min/average aggregation and RS-ZB differences and ratios, concatenated with atomic features (AtomF) in increasing recursion order (Rec1 through Rec8). Using a 150-split ensemble of kernel-ridge regression (KRR) and SISSO (a compressed-sensing symbolic-regression method) models, the paper shows the bond-aware KRR ensemble beats both the previous three-feature symbolic-regression model and symbolic descriptors built from the same primary features. The best results are AtomF+Rec6 on the 78-compound refined dataset (test RMSE 0.029 eV) and AtomF+Rec4 on the full dataset (test RMSE 0.046 eV).","pith_inferences":["Because the recursion features do not require self-consistent DFT on the target structure, the same recipe could plausibly transfer to other structural competitions, such as wurtzite versus zincblende or layered versus cubic polymorphs, reusing the same per-element Hamiltonians.","The sharp plateau beyond Rec6 suggests a practical rule of thumb for small binary datasets: six recursion levels suffice, which could be tested as a design guideline for larger feature-engineering studies.","The quantitative margin may be partly optimistic because sequential feature selection was run once on the full dataset before the splits; a strictly nested feature-selection rerun would reveal whether the reported RMSEs hold or need revision.","A strong transfer test would be to apply the best AtomF+Rec6 model to newly computed octet binaries outside the original 82, such as pairs involving heavier p-block or post-transition elements, and check whether the sub-0.03 eV accuracy persists."],"forward_implications":["On the refined 78-compound dataset, adding recursion features up to order 6 lowers the mean test RMSE from about 0.080 eV for the reference descriptor to 0.029 eV, and the worst-case test MAE from 0.155 eV to 0.049 eV.","On the full 82-compound dataset, the best bond-aware model reaches a mean test RMSE of 0.046 eV against 0.139 eV for the reference model re-evaluated in this paper's ensemble, so the improvement does not depend on removing the four outliers.","The 3D SISSO ensemble built from the same features also improves on the reference but levels off around 0.062-0.074 eV test RMSE on the refined dataset, indicating that the gain is primarily due to the recursion features themselves rather than to the symbolic-regression wrapper.","Permutation feature importance places lower-order recursion features such as $(a_3)^{\\max}_r$ alongside ionization potentials and HOMO energies, suggesting the model is exploiting bond-strength differences between the two structures.","Recursion orders 7 and 8 add little and can slightly raise mean RMSE, so the first six recursion levels capture essentially all the bonding information useful for these simple binaries."],"supporting_citations":[{"why":"Supplies the 82-compound dataset of RS-ZB formation-energy differences and the three-feature reference descriptor that all bond-aware models are compared against.","marker":"[3]"},{"why":"Provides the pairwise tight-binding Hamiltonians downfolded from DFT eigenstates of diatomic molecules, which inject bond-specific domain knowledge into the recursion features.","marker":"[23]"},{"why":"Computes the recursion coefficients for the tight-binding bond model, producing the RecN feature vectors.","marker":"[24]"},{"why":"Supplies the recursive-solution formalism that produces the local density of states and the $a_n$, $b_n$ coefficients used as features.","marker":"[10]"},{"why":"Isolation forest identifies the four outliers whose removal defines the refined 78-compound dataset used for the headline result.","marker":"[30]"},{"why":"Provides the sequential feature selection implementation that reduces each feature set before the ensemble models are trained.","marker":"[31]"},{"why":"The SISSO compressed-sensing method is the symbolic-regression baseline whose descriptors are built from the same primary features for comparison.","marker":"[34]"}],"fun_headline_variants":["Bond-informed ML cuts octet stability error 64%","Recursion features halve octet stability prediction error","Bond knowledge halves octet structure stability error","Tight-binding recursion features boost octet stability ML","ML with bond features cuts octet stability error in half"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the reported cross-validated errors are honest, even though the features are selected once on the full dataset before the train/test splits are generated, so the held-out samples have already influenced which features were chosen.","fun_headline_variants_meta":{"raw":{"variants":["Bond-informed ML cuts octet stability error 64%","Recursion features halve octet stability prediction error","Bond knowledge halves octet structure stability error","Tight-binding recursion features boost octet stability ML","ML with bond features cuts octet stability error in half"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000638,"raw_usage":{"total_tokens":3018,"prompt_tokens":1100,"completion_tokens":1918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":1840}},"tokens_in":716,"tokens_out":1918,"duration_ms":15343,"temperature":1.0,"reasoning_tokens":1840,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:14:09.353760+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the same 150-split ensemble protocol with sequential feature selection performed separately inside every training fold; if the AtomF+Rec6 test RMSE on the 78-compound dataset rises substantially above 0.029 eV, the claimed improvement over the reference is partly an artifact of selection leakage.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 82-compound dataset of RS-ZB formation-energy differences and the three-feature reference descriptor that all bond-aware models are compared against."},{"cited_title":"Haydock, Recursive solution of the Schr¨ odinger equation, Comput","cited_arxiv_id":null,"evidence_quote":"Supplies the recursive-solution formalism that produces the local density of states and the $a_n$, $b_n$ coefficients used as features."},{"cited_title":"Raschka, Mlxtend: Providing machine learning and data science utilities and extensions to python’s scientific computing stack, Journal of Open Source Software3, 638 (2018)","cited_arxiv_id":null,"evidence_quote":"Provides the sequential feature selection implementation that reduces each feature set before the ensemble models are trained."},{"cited_title":"Ouyang, S","cited_arxiv_id":null,"evidence_quote":"The SISSO compressed-sensing method is the symbolic-regression baseline whose descriptors are built from the same primary features for comparison."}],"review_version":2}