{"id":"2891125d-bce0-479f-adb0-2f112b12b554","arxiv_id":"2608.03805","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Quantile gradient boosting on 393 DFT alloys predicts refractory CCA elastic properties with R² 0.89-0.97, but reported 100% Born stability and Re-based design rules are undercut by internal inconsistencies and miscalibrated intervals.","lead":"An alloy-design study trains a gradient-boosting model on 393 simulated refractory alloys and reports accurate elastic-property predictions plus design rules. The paper's headline claims about mechanical stability and element importance are contradicted by its own data, so results should be read with caution.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3.3 contradicts the headline design rule: #2 candidate CrMo3 contains no Re, so 'Re appears in 100% of top-10' cannot be true.","rationale":"The reader's Weakest Assumption concerns representativeness of the 59-alloy test set for trillion-scale screening. While that is a valid concern, I found a stronger, more direct problem: the paper's own Table 3.3 falsifies the Re-100% design rule. Since any top-10 list contains the top-5 list, a Re-free alloy at rank 2 caps Re frequency at 90%, so the abstract/conclusion claim cannot be correct as stated. This internal inconsistency is load-bearing for the screening/design-rule portion of the central claim, which is the main contribution beyond R2 numbers. It also raises a reproducibility issue: the top-10 list used for elemental analysis is not shown and cannot be inferred from the reported tables. A computational rerun of the screening would settle whether the 100% figure is a typo or whether a different candidate set was used. Since this reinforces the reader's REJECT verdict rather than moving it, I recommend UNCHANGED. I do not see a need to attack the raw R2/MAE numbers or the Born-stability metrics; those are secondary once the headline design rules are internally contradicted.","tokens_in":14465,"tokens_out":6102,"duration_ms":65561,"concrete_test":"Reproduce the top-10 candidate list by applying the published QGBR model (13 features from §2.4, hyperparameters from §2.5) to the same 59 test alloys and separately to all 393 alloys in the Zhang dataset, sorting by predicted C44 and taking the top 10. Count Re-containing entries in each list. If CrMo3 (or any other non-Re alloy with C44 > 131 GPa) appears, the 'Re in 100% of top-10' claim is refuted. Also compare the resulting lists with the set used for the §3.5 elemental analysis to determine whether the analysis was performed on a different, undocumented candidate set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central screening/design-rule claim—'rhenium appears in 100% of top-performing alloys' (Abstract; §3.5; Conclusions)—is internally contradicted by Table 3.3, which is explicitly labeled 'the five highest-ranked alloys by predicted C44': MoReW (139.09), CrMo3 (133.22), Cr2ReV (131.66), CrReV (131.43), CrReW2 (131.03). CrMo3 contains no Re. Any top-10 list must include the top-5, so at most 9 of the top-10 can contain Re; the reported 100% Re frequency is arithmetically impossible. Either the elemental analysis used a different, unreported candidate list, or the frequency is miscalculated. In either case, the design rule 'Re is essential' and the associated recommendation of MoReW lose their stated support. This is a direct internal inconsistency, not merely a question of extrapolating from 59 test alloys; however §3.5 also shows that the screening maps are built only from the 59 test alloys, so the provenance of the top-10 list is not established. The error is load-bearing because the Re-100% rule is a headline quantitative result and is used to motivate the VEC=6.0–6.4 and Re-Mo-Cr design guidelines.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents a quantile gradient boosting regression framework, trained on 393 DFT-computed BCC refractory complex concentrated alloys from a 10-element space, to predict six elastic properties (B, G, Hv, C11, C12, C44) and derive E and ν from B and G. The authors report R² = 0.89–0.97 on a 59-alloy test set, claim 100% Born stability satisfaction on test predictions, and derive screening design rules: Re appears in 100% of top-10 alloys, optimal VEC = 6.0–6.4, δ < 10%, and Re–Mo–Cr ternary systems maximize shear rigidity. The top candidate is MoReW with predicted C44 = 139 GPa. The screening and design-rule conclusions are drawn from the 59 test alloys in Section 3.5.","tokens_in":14799,"tokens_out":5083,"duration_ms":55806,"significance":"If the claims held, the framework could be a useful screening tool for refractory CCAs. The paper has genuine strengths: it uses a public DFT dataset, reports learning curves and cross-validation, attempts experimental validation on 18 alloys, and provides quantile-based uncertainty intervals. However, the headline UQ and design-rule claims are compromised by internal inconsistencies and overstatement. The 'Re appears in 100% of top-10' rule is arithmetically contradicted by the paper's own Table 3.3, and the 'physics-informed constraints' are post-hoc diagnostics rather than enforcement. The screening is performed on the 59 test alloys, not on the trillion-scale composition space claimed in the abstract. These issues affect the central contributions, so the paper needs substantial revision before its conclusions can be accepted.","major_comments":[{"comment":"The statement 'rhenium appears in 100% of top-performing alloys' is internally contradicted by Table 3.3, which lists the five highest-ranked alloys by predicted C44: MoReW, CrMo3, Cr2ReV, CrReV, CrReW2. CrMo3 contains no Re. Since the top-5 are necessarily part of the top-10, at most 9 of the top-10 can contain Re. Either the elemental analysis was performed on a different, unreported candidate list, or the frequency is miscalculated. This invalidates the design rule 'Re is essential' and the associated Re–Mo–Cr ternary recommendation. The authors must provide the actual top-10 list and recompute the frequencies.","section":"Abstract; §3.5; Conclusions"},{"comment":"The paper repeatedly claims 'well-calibrated 90% prediction intervals with 78–90% coverage' for the framework. Table 3.1 reports PICP for ν (Poisson's ratio) as 1.7%. Since ν is one of the eight properties the paper says it predicts, this single value contradicts the 78–90% coverage claim. The footnote 'PICP for ν is not directly comparable due to non-Gaussian distribution' is not sufficient: either derive meaningful intervals for ν or restrict the UQ claim to the six directly modeled properties and report the ν coverage honestly as a failure.","section":"Table 3.1; Abstract; Introduction"},{"comment":"The abstract and conclusions state that physics-informed Born stability constraints ensure '100% compliance' on test predictions. However, §2.6.2 says compliance was 'evaluated post-prediction' and §3.3 says the model achieves perfect compliance 'without explicit constraint enforcement.' Thus the 100% figure is a post-hoc observation on 59 test points, not evidence that constraints are enforced during training or applied during prediction. This distinction matters because, as the authors note, all 393 training alloys are Born-stable; the model may simply be interpolating. Please reframe the claim as a diagnostic rather than a property of the framework.","section":"§2.6.2; §3.3"},{"comment":"The compositional screening and design rules are derived exclusively from the 59 test alloys. Figure 3.3 and the elemental frequencies are summaries of this small test set, not a screening of the trillion-scale composition space. The conclusion that the framework 'enables screening of 10^15 possible compositions' is not demonstrated. To support this claim, the authors need to apply the model to unseen compositions outside the 393-alloy dataset and describe how candidates are selected. As written, the design rules are descriptive statistics of a handful of already-computed alloys.","section":"§3.5; Conclusions"},{"comment":"MoReW is presented as the top candidate with predicted C44 = 139 GPa, but the paper never reports the actual DFT C44 for MoReW from the test set. Since MoReW is in the test set, its true value is available and should be compared with the prediction. Without this comparison, the 'discovery' of MoReW as a top candidate could be an artifact of ranking noisy test predictions; the 90% PI width is approximately 40 GPa, which is substantial. Please report the actual C44 and the rank of MoReW (and the other top-5 candidates) in the test set.","section":"§3.4; §4.6"}],"minor_comments":[{"comment":"The heading 'Physics-Informed Constraints' is misleading when the criteria are only evaluated after prediction. Consider renaming to 'Post-hoc physics validation' to match the actual method.","section":"§2.6"},{"comment":"CrMo3 is listed with C44 = 133.22 GPa in Table 3.3 but as 128 GPa in §4.8 ('CrMo3 (C44 = 128 GPa, 8% reduction)'). Reconcile this inconsistency.","section":"§4.8 vs Table 3.3"},{"comment":"The design rule states 'δ < 10%', but the high-C44 region in Figure 3.3 is described as δ = 3–9%. Clarify whether the recommended δ range is the narrower 3–9% or the broader <10%.","section":"§3.5"},{"comment":"Use ν instead of µ for Poisson's ratio for consistency with the text. Also, the footnote about ν PICP should be replaced with a substantive explanation of why coverage fails, rather than dismissing it as 'not directly comparable.'","section":"Table 3.1"}],"recommendation":"major_revision","confidential_remarks":"The referee report from the reader recommends rejection, and I agree that the central claims are overstated. However, I see the main problems as correctable: the Re-frequency error can be fixed by reporting the actual top-10 list, the ν PICP issue can be addressed by honest reporting, and the screening claims can be reframed as test-set analysis. The regression methodology itself appears sound, and the paper contains useful benchmarking. I therefore recommend major revision rather than outright rejection, with the expectation that the authors either substantiate or remove the overstated claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nMy read: the regression core is decent. QGBR on Zhang et al.'s 393-alloy dataset gives R2 0.89–0.97 on a 59-alloy test split with reasonable MAEs, and the separate experimental validation on 18 alloys not used in training is a genuine positive. That part is credible and a modest step beyond Zhang's earlier RF results, though a same-split baseline is needed before claiming improvement.\n\nThe soft spots are where the paper moves from prediction to design rules. The stress-test note is correct: Table 3.3 lists CrMo3 as the #2 candidate, and CrMo3 contains no Re. So \"rhenium appears in 100% of top-10 candidates\" cannot be true if that top-10 includes the top-5. Either the elemental analysis used a different list or the count is wrong. That 100% Re frequency is a headline abstract and conclusion claim, so this is load-bearing, not cosmetic. The same overreach applies to the VEC 6.0–6.4 rule: all those candidates come from a narrow slice of a 59-alloy test set, so generalizing to 10^15 compositions is unjustified.\n\nTwo more concerns, proportionate. First, the abstract claims well-calibrated 90% prediction intervals with 78–90% coverage, but the derived property nu has a PICP of 1.7%, dismissed in a footnote as \"not directly comparable.\" That is not calibration; it means uncertainty for nu is badly underestimated, and if one of eight properties fails, the blanket UQ claim needs qualification. Second, the 100% Born stability compliance is a post-hoc observation on 59 points, not an enforced constraint—the authors admit this later—so calling it a \"physics-informed constraint\" oversells it. And MoReW's predicted C44 is never checked against a DFT value, so the \"discovery\" may just be interpolation within the training distribution.\n\nWho this is for: someone working on ML for alloy screening will find the modeling and experimental validation useful, and the paper is a good case study in how screening claims can outrun a small test set. It deserves a serious referee because the core modeling is competently done and the flaws are fixable: recompute the elemental frequencies, report same-split baselines, and either validate MoReW with DFT or soften the discovery claim. I would accept it for peer review with major revision, but I would not cite the design rules as established.","headline":"Solid regression work undercut by an internal contradiction in the headline design rule and an overclaimed screening result; worth refereeing after major revision.","tokens_in":15270,"tokens_out":2654,"would_cite":false,"duration_ms":28951,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":["81.05.Bx","71.15.Mb"],"model":"deepseek-v4-flash","headline":"A physics-informed machine learning model predicts eight mechanical properties of refractory complex concentrated alloys with R² 0.89–0.97 and 100% Born stability compliance on held-out tests.","keywords":["refractory complex concentrated alloys","physics-informed machine learning","quantile gradient boosting","Born stability criteria","elastic constants","compositional screening","valence electron concentration","alloy design"],"falsifier":"Measure the elastic constants of arc-melted MoReW by resonant ultrasound spectroscopy and compare C44 to the predicted 139 GPa; a discrepancy well beyond the 8–12 GPa DFT/experiment scatter would undermine the screening. Also compute or measure C44 for a VEC ≈ 6.4 alloy without rhenium (e.g., CrMo3) to test whether the predicted shear penalty is real.","tokens_in":14366,"feed_emoji":"🔬","tokens_out":5473,"duration_ms":58731,"temperature":0.7,"pith_summary":"The paper argues that combining quantile gradient boosting with Born stability checks—applied to 393 DFT-computed BCC alloys—produces a model accurate enough to screen the roughly 10^15 possible refractory complex concentrated alloy compositions. It claims the model predicts six elastic and hardness properties directly, derives two more from them, and satisfies all mechanical stability criteria on every held-out prediction. Screening then identifies MoReW as the top shear-resistant candidate (predicted C44 = 139 GPa) and yields quantitative design rules: VEC between 6.0 and 6.4, atomic size mismatch below 10%, and Re–Mo–Cr ternary systems maximize shear rigidity. If these claims hold, expensive experimental or DFT searches can be replaced by fast, risk-aware screening with a short list of synthesis-ready candidates.","feed_headline":"ML model predicts eight alloy properties with zero Born violations","feed_subtitle":"Screening 393 DFT alloys yields VEC 6.0–6.4, delta < 10%, and MoReW as top shear candidate.","key_machinery":"The key machinery is quantile gradient boosting regression (QGBR), an ensemble of regression trees that estimates conditional quantiles at 0.05, 0.50, and 0.95, giving both median predictions and 90% prediction intervals. The physics enters through 13 features including valence electron concentration (VEC) and atomic size mismatch (δ), through deriving E and ν from predicted B and G via standard elasticity relations rather than fitting them independently, and through post-prediction verification of the Born stability criteria for cubic crystals. The model does not enforce the Born criteria in its loss function; it instead relies on a fully Born-stable DFT training set and smooth composition–","core_discovery":"The central claim is that a quantile gradient boosting model, trained on 13 composition-derived descriptors from 393 DFT-computed single-phase BCC alloys, can predict bulk modulus, shear modulus, Vickers hardness, and elastic constants C11, C12, and C44 with test-set R² values of 0.89–0.97, while Young's modulus and Poisson's ratio derived from the predicted B and G remain thermodynamically consistent. All 59 held-out test predictions satisfy the Born stability criteria (C11–C12 > 0, C44 > 0, C11 + 2C12 > 0), with no violations. Compositional screening in VEC–δ space reveals a high-shear-rigidity zone near VEC 5.8–6.4 and δ 3–9%, and elemental analysis of the top-ranked candidates shows rhen","pith_inferences":["The 100% Born compliance is observed within the sampled composition window, not proven as a general guarantee; a stress test using out-of-distribution compositions near the BCC/FCC boundary or with δ > 10% would show whether the model's stability holds beyond its training region.","Because rhenium's dominance in the top ten is a correlation within 393 DFT alloys, an experimental comparison of MoReW against a Re-free mimic (e.g., CrMo3) would test whether the predicted ~8% shear penalty for removing rhenium is real and persists at service temperatures.","The VEC 6.0–6.4 optimum sits near the BCC/FCC phase boundary (VEC ≈ 6.87), so the highest-C44 candidates may be elastically stable yet thermodynamically metastable; pairing this ML screen with CALPHAD or formation-energy calculations would sharpen the design rules.","A natural next step is active learning: use the model's prediction intervals to select the most uncertain VEC–δ regions for new DFT calculations, retrain, and see whether the VEC 6.0–6.4 window shifts as the dataset grows beyond 393 alloys."],"forward_implications":["MoReW, already synthesized as a single-phase BCC alloy, now has a quantitative target: measuring C44 near the predicted 139 GPa would directly confirm the screening pipeline.","The proposed design rules (VEC = 6.0–6.4, δ < 10%, Re–Mo–Cr ternary preference) give experimentalists a narrow composition window to explore instead of the full trillion-scale space.","Re-free alternatives identified in the paper, such as CrMo3, MoW, and CrMoW ternaries, retain roughly 88–92% of the top C44 at much lower cost, offering practical substitutes where rhenium is too expensive.","The 90% prediction intervals allow risk-stratified screening: high-confidence predictions can proceed directly to synthesis, while wide-interval candidates can be sent for DFT refinement first.","The same workflow—compositional descriptors, quantile boosting, and Born checks—can be transferred to other alloy families and properties, provided a matching DFT or experimental dataset exists."],"supporting_citations":[{"why":"Supplies the entire 393-alloy DFT dataset of BCC refractory alloys plus a baseline neural-network study for comparison.","marker":"[9]"},{"why":"Provides the general-purpose ML framework and descriptor protocol that the feature engineering follows.","marker":"[10]"},{"why":"Prior ML-guided HEA design with ~15% Born stability violations, the contrast motivating this paper's physics-informed approach.","marker":"[11]"},{"why":"Prior ML solid-solution prediction with ~10% stability violations, another baseline for Born compliance.","marker":"[12]"},{"why":"Establishes the VEC threshold (6.87) separating BCC from FCC, grounding the VEC 6.0–6.4 design window.","marker":"[21]"},{"why":"States the Born stability criteria used to verify mechanical stability of all predictions.","marker":"[26]"},{"why":"Provides independent experimental elastic constants used to validate the model on 18 refractory CCAs.","marker":"[22]"},{"why":"Reports experimental synthesis of equiatomic MoReW as single-phase BCC, making the predicted C44 value experimentally testable.","marker":"[31]"}],"fun_headline_variants":["Physics-informed ML hits 100% Born stability on test alloys","Rhenium in every top refractory alloy from ML screen","ML identifies Re-Mo-Cr as key to shear-resistant alloys","MoReW tops ML screening for shear rigidity in refractory CCAs"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The design rules and top-candidate ranking assume the 59 held-out alloys represent the trillion-scale BCC composition space; if that sample is not representative, the rules may not generalize beyond this dataset.","fun_headline_variants_meta":{"raw":{"variants":["Physics-informed ML hits 100% Born stability on test alloys","Rhenium in every top refractory alloy from ML screen","ML identifies Re-Mo-Cr as key to shear-resistant alloys","MoReW tops ML screening for shear rigidity in refractory CCAs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000889,"raw_usage":{"total_tokens":3750,"prompt_tokens":901,"completion_tokens":2849,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":645,"completion_tokens_details":{"reasoning_tokens":2777}},"tokens_in":645,"tokens_out":2849,"duration_ms":21738,"temperature":1.0,"reasoning_tokens":2777,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:04:48.060204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the elastic constants of arc-melted MoReW by resonant ultrasound spectroscopy and compare C44 to the predicted 139 GPa; a discrepancy well beyond the 8–12 GPa DFT/experiment scatter would undermine the screening. Also compute or measure C44 for a VEC ≈ 6.4 alloy without rhenium (e.g., CrMo3) to test whether the predicted shear penalty is real.","supporting_citations":[{"cited_title":"Zhang, X","cited_arxiv_id":null,"evidence_quote":"Supplies the entire 393-alloy DFT dataset of BCC refractory alloys plus a baseline neural-network study for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the general-purpose ML framework and descriptor protocol that the feature engineering follows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior ML-guided HEA design with ~15% Born stability violations, the contrast motivating this paper's physics-informed approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior ML solid-solution prediction with ~10% stability violations, another baseline for Born compliance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"States the Born stability criteria used to verify mechanical stability of all predictions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides independent experimental elastic constants used to validate the model on 18 refractory CCAs."},{"cited_title":"Senkov, S","cited_arxiv_id":null,"evidence_quote":"Reports experimental synthesis of equiatomic MoReW as single-phase BCC, making the predicted C44 value experimentally testable."}],"review_version":1}