{"id":"7ff3918b-7624-4bb7-a163-0714d3842ddc","arxiv_id":"2501.05164","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An Extra Trees model trained on one-hot encoded compositions predicts superconducting transition temperatures of liquid metal alloys with R2=0.9519 and identifies In0.5Sn0.5 as the best printable candidate at 7.01 K.","lead":"This paper trains tree-based machine learning models on the SuperCon superconductor database to predict the critical temperature of liquid metal alloys, reporting R2=0.9519 for an Extra Trees model. It then screens thousands of binary and ternary metal combinations and points to In0.5Sn0.5, with a predicted Tc of 7.01 K, as the best printable liquid metal alloy candidate.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The top candidate In0.5Sn0.5 is likely in the SuperCon training set, so the 7.01 K 'prediction' may be retrieval rather than generalization; a composition holdout test is needed.","rationale":"The reader's weakest assumption was that one-hot composition encoding omits structural and pressure information, making predicted Tc unreliable for materials whose superconductivity depends on those factors. That is a real limitation, and the paper itself acknowledges it in the Discussion. However, I see a more immediately load-bearing concern for the paper's specific headline result: the top candidate In0.5Sn0.5 is a known superconductor and is very likely already in the SuperCon training set. If so, the 7.01 K value is not a forward prediction but a retrieval of a previously measured Tc, and the claim that machine learning 'identifies' this material is misleading. The paper provides no evidence that this candidate is absent from training data, and the random-split evaluation does not measure generalization to new compositions. This concern is concrete and testable: hold out all In-Sn entries and see whether the model still predicts 7.01 K. I partially agree with the reader because the descriptor limitation and the leakage concern are distinct; the leakage concern is more central to the claimed discovery aspect, while the descriptor limitation affects broader applicability. I would keep the verdict conditional because the paper could address the issue by releasing code/data and performing the holdout test, but if the test shows the prediction collapses, the central claim would need rejection. The external Ga-In-Sn check and the reproduction of known tree-model performance on SuperCon are positive points, but they do not resolve the candidate-in-training-set problem.","tokens_in":11629,"tokens_out":4475,"duration_ms":46064,"concrete_test":"Parse mdr_clean.csv to determine whether In0.5Sn0.5 or any In-Sn composition with x near 0.5 is present, and record the experimental Tc values for those entries. Then retrain the Extra Trees model with all In-Sn entries (and, separately, all printable liquid-metal alloy compositions) held out from training, retune hyperparameters on the remaining data, and predict Tc for In0.5Sn0.5. If the predicted Tc drops substantially below 7.01 K or is no longer the maximum among printable alloys, the claimed identification is a training-set artifact. Additionally, rerun the model evaluation using a grouped split by unique formula, so no composition appears in both training and test sets, and report the resulting R2 and RMSE.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that an Extra Trees model trained on one-hot composition features identifies In0.5Sn0.5 as the printable liquid-metal alloy with the highest Tc, 7.01 K. For this to be a valid identification, the model must generalize to compositions it has not memorized. The paper uses a random 90/10 train/test split and never reports whether In0.5Sn0.5 or nearby In_xSn_1-x compositions appear in the cleaned SuperCon dataset. SuperCon is the source of the training data, and the paper itself cites a synthesis study of InSn alloy superconductors (ref. 32), strongly suggesting such entries are present. If In0.5Sn0.5 is in the training set, the quoted 7.01 K is an interpolated or averaged experimental Tc, so the 'identification' is a retrieval of a known label, not a predictive discovery. Random splits also allow near-duplicate formulas to appear in both train and test, inflating the reported R2=0.9519 and making it an unreliable estimate of performance on new compositions. The one-hot descriptor's lack of structural information, flagged in the Discussion, is a related limitation, but the more immediate and testable flaw is this potential training-set leakage. Without a composition-based holdout, the paper's headline screening result is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper applies tree-based machine learning regression to the SuperCon database, using a one-hot encoding of chemical composition as the sole descriptor, to predict superconducting critical temperatures (Tc). After preprocessing, the authors compare eight tree-based models and report that an Extra Trees model achieves R2 = 0.9519 and RMSE = 6.2624 K on a random 9:1 train/test split. The model is then used to screen liquid-metal (LM) alloys, identifying In0.5Sn0.5 as the printable LM alloy with the highest predicted Tc at 7.01 K. The authors further screen 2,145 binary and 45,760 ternary alloys formed from 66 metal elements. The central claims are the high predictive accuracy of the Extra Trees model and the identification of In0.5Sn0.5 as a promising high-Tc printable liquid-metal superconductor.","tokens_in":11900,"tokens_out":3840,"duration_ms":37725,"significance":"If the model genuinely generalizes to new compositions, the work would provide a low-cost screening tool for printable liquid-metal superconductors and a large database of predicted Tc values for binary and ternary alloys. The external comparison with Ga-In-Sn experimental data (errors within 1 K) is a useful sanity check, and the systematic comparison of eight tree-based models is a reasonable methodological contribution. However, the headline 'identification' of In0.5Sn0.5 is not established as a predictive discovery: the candidate is a known In-Sn alloy composition that may already exist in the training set, and the one-hot composition descriptor discards structural and pressure information that the authors themselves acknowledge as important. The screening results are thus better described as interpolation/retrieval within the known composition space unless a composition-holdout validation is provided.","major_comments":[{"comment":"The identification of In0.5Sn0.5 as the highest-Tc printable LM alloy (7.01 K) is not supported as a predictive discovery because the paper never reports whether In_xSn_1-x compositions, including In0.5Sn0.5 itself, appear in the cleaned mdr_clean.csv dataset or in the training partition. Since the model is trained on SuperCon data and In-Sn alloy superconductors are known (ref. 32 reports their synthesis), the 7.01 K value may simply be an interpolation or average of experimental Tc values already in the training set. The authors should perform a composition-based holdout test — for example, training on all data except entries containing In and/or Sn, or at least except In-Sn compositions — and report the predicted Tc for In0.5Sn0.5 under that model. They should also state explicitly whether the exact formula In0.5Sn0.5 is present in the training data. Without this analysis, the central screening claim is unverified.","section":"§3, Fig. 4(c); §2 Data preprocessing"},{"comment":"The one-hot composition descriptor contains only elemental proportions and carries no information about crystal structure, pressure, or synthesis conditions. The authors acknowledge this in the Discussion ('one-hot coding used in this study only records the proportion of elements in the formula. In the absence of structural information'). This is not merely a peripheral limitation: the screening protocol applies the model to arbitrary alloy compositions, including hypothetical ones that may lie far from the training distribution, and the prediction for In0.5Sn0.5 is made under the implicit assumption that composition alone determines Tc. The paper should either restrict the screening claims to compositions within the chemical and structural domain of the training data, or add a similarity/distance check to flag extrapolations. As written, the uncertainty of the 7.01 K prediction is understated.","section":"§4, Discussion point 1; §2 Model comparison"},{"comment":"The claim that 1,000 random composition samples per alloy family is sufficient to find the maximum Tc is inferred from the stability of the maximum for one ternary family (Ga-In-Sn). This does not establish that 1k samples is sufficient for all 2,145 binary and 45,760 ternary families, some of which may have sharper Tc optima or more complex composition dependence. A sensitivity analysis on several representative families, or an adaptive sampling criterion with a convergence check, is needed to support the 'highest Tc' claims for the full screening. Without this, the reported maxima for many alloy families may be underestimated.","section":"§3, Fig. 4(b)"}],"minor_comments":[{"comment":"Equation (5) as written has the numerator and denominator of the R2 expression reversed relative to the standard definition R2 = 1 - SS_res/SS_tot. Please correct this typo, since the reported R2 values are otherwise consistent with the standard calculation.","section":"§3, Eq. (5)"},{"comment":"The abstract states '45,670 ternary alloys' while Section 3 and Figure 4 report 45,760 ternary alloys; the numbers should be reconciled.","section":"Abstract vs. §3"},{"comment":"The conclusion refers to 'mdr_celan.csv' instead of 'mdr_clean.csv'; please correct the typo.","section":"§5, Conclusion"},{"comment":"The quantity in Eq. (6) is the per-sample absolute error, not the mean absolute error (MAE). Rename it (e.g., 'absolute error') or define MAE as the mean over predictions to avoid confusion with the standard metric.","section":"§3, Eq. (6)-(7)"},{"comment":"Reference 32 (Mochiku et al., Synthesis of InSn alloy superconductor below room temperature) appears in the reference list but is not cited in the main text. If it is intended to support the existence of In-Sn superconductors in the training set or the relevance of In0.5Sn0.5, it should be cited where appropriate.","section":"References"},{"comment":"The color maps and element labels in Figures 4(c.ii), 4(c.iii), 4(d.ii), and 4(d.iii) are too small to read in the provided resolution; higher-resolution or separate panels would improve clarity.","section":"Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript applies well-established tree-based regression methods to a materials science screening problem. The main scientific concern is whether the headline candidate In0.5Sn0.5 is genuinely predicted or simply retrieved from the training set; the composition-holdout analysis is essential and is within the scope of a revision. The paper's own Discussion acknowledges the structural-information limitation, which further tempers the strength of the screening claims. I would not recommend rejection because the methodology is sound and the external Ga-In-Sn comparison provides some evidence of practical utility, but the current evidence does not support the central 'identification' claim as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take: this is a solid screening paper in a crowded subfield, not a breakthrough. What is genuinely new: the first explicit ML screen targeting liquid-metal alloys for printed superconducting devices, a cleaned SuperCon-derived dataset, and a large screening output (2,145 binaries, 45,760 ternaries). The Extra Trees model with one-hot formula features is borrowed from earlier Tc-prediction work, so the method itself is not new. What the paper does well: preprocessing is described in enough detail to follow, the 9:1 split and five-fold CV are standard, and the comparison against Ga-In-Sn experimental values (error within 1 K) is a useful sanity check. Those are real strengths.\n\nThe main problem is the screening claim. In0.5Sn0.5 is a known In-Sn superconductor (ref. 32 is cited), SuperCon is the training source, and the paper never checks whether that formula or near neighbors are in the training set. On a random 9:1 split, near-duplicate formulas can appear in both train and test, so R2=0.95 is not evidence of generalization to new compositions. The 7.01 K 'identification' may simply be retrieval of a known experimental Tc. A composition-based holdout—train on formulas, test on disjoint compositions—is the obvious fix and should be reported. Second soft spot: one-hot composition alone omits structure and pressure, which the Discussion explicitly concedes; for broad screening of nonexistent alloys, predicted Tc values are extrapolations, not candidates with known phases. Third: the model cannot distinguish superconductors from non-superconductors, yet the conclusion reports all screened alloys as having Tc > 0 K; that overstates what the model can say. Fourth: no code or data released; 'available on request' is weaker than a repository with the cleaned dataset and scripts.\n\nThese are addressable revisions, not fatal flaws. The external Ga-In-Sn check and the honest limitation paragraphs show the authors know where the method is fragile. I would send it to peer review: a competent referee can ask for composition holdout, comparison of In0.5Sn0.5 with ref. 32's experimental value, and release of the cleaned dataset. I wouldn't cite it as a discovery, but as a screening study with a reproducible pipeline it could be useful to the liquid-metal community. My verdict: conditional, leaning accept after revision.","headline":"A competent screening paper with a modest new application, but the headline In0.5Sn0.5 result is probably retrieval from training data, not a validated prediction.","tokens_in":12484,"tokens_out":2543,"would_cite":false,"duration_ms":24092,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an Extra Trees model trained on one-hot chemical compositions predicts the critical temperature of liquid-metal alloy superconductors with $R^2=0.9519$ and identifies In$_{0.5}$Sn$_{0.5}$ as the highest-$T_c$…","keywords":["liquid metal","machine learning","critical temperature","superconductor","Extra Trees","alloy screening","composition encoding"],"falsifier":"Synthesize In$_{0.5}$Sn$_{0.5}$ at normal pressure and measure resistivity versus temperature: if the superconducting onset is not near 7.01 K, or if the alloy does not superconduct at all, the composition-only prediction is falsified for the paper's headline candidate. A broader version would measure a sample of the 2,145 binary and 45,760 ternary alloys the model ranks highly and compare predicted and observed $T_c$ values.","tokens_in":11343,"feed_emoji":"🧊","tokens_out":11177,"duration_ms":97651,"temperature":0.7,"pith_summary":"Superconducting materials are limited by how hard it is to predict their critical temperature, and the paper tries to show that a fast, formula-only machine-learning model can close that gap for liquid-metal alloys. The authors clean a large literature-derived dataset of known superconductors, train eight tree-based regressors, and report that an Extra Trees model reproduces transition temperatures with $R^2=0.9519$ and an RMSE of 6.2624 K. They then use the model to screen printable liquid-metal alloys, identifying In$_{0.5}$Sn$_{0.5}$ as the best candidate with a predicted $T_c$ of 7.01 K, and extend the screen to thousands of binary and ternary alloys across 66 metals. If the claim holds, alloy discovery for printed superconducting wires and flexible electronics becomes a quick computational ranking instead of slow trial and error.","feed_headline":"Machine learning picks In0.5Sn0.5 as liquid-metal superconductor","feed_subtitle":"Composition-only tree model hits R²=0.95 and predicts a printable alloy that superconducts at 7.01 K.","key_machinery":"The object that carries the argument is the one-hot composition vector: a formula is turned into a long fixed-length vector whose entries are the atomic fractions of each permitted element, so the stoichiometry is the only descriptor. The regression mechanism is Extra Trees, an ensemble of randomized decision trees that averages many trees built with random thresholds, chosen after a grid search among eight tree-based models. The paper argues that tree ensembles are well matched to this tabular regression task because they preserve feature orientation, learn irregular target functions, need little regularization, and train quickly on the roughly 21,000-sample dataset. A five-fold cross-validated hyperparameter search selects the Extra Trees configuration used for all alloy screening.","core_discovery":"The paper's central claim is that a material's superconducting critical temperature can be read off its chemical formula: encode the atomic fractions of the elements as a one-hot vector, feed it to an Extra Trees regressor, and the model reproduces known transition temperatures and ranks untested liquid-metal alloys. The authors build this model on a cleaned dataset of 20,995 records, after removing missing, high-pressure, mislabeled, and illegal-element entries, converting mass-percentage formulas to atomic ratios, and adding known non-superconductors with $T_c = 0$. They report the Extra Trees model outperforms seven other tree-based models, reaching $R^2 = 0.9519$ and an RMSE of 6.2624 K on a held-out split. Screening printable liquid-metal alloys from eight elements, the model assigns the highest $T_c$, 7.01 K, to In$_{0.5}$Sn$_{0.5}$, and it is then applied to 2,145 binary and 45,760 ternary alloys across 66 metals. The paper stresses that the model predicts $T_c$ values, not whether a given composition is a superconductor.","pith_inferences":["The paper leaves implicit that the same model could screen quaternary and higher alloys once the combinatorial search becomes efficient; it notes the current search algorithm is the limiting factor.","The paper's own limitation statement implies the 7.01 K value is a composition-only estimate; adding crystal-structure or pressure information could change the ranking.","A testable follow-up is to check whether prediction residuals cluster in composition neighborhoods, which would reveal families where structure or synthesis conditions override composition.","If this recipe transfers beyond superconductors, the one-hot-plus-Extra-Trees pipeline could rank other composition-dependent properties of liquid-metal alloys."],"forward_implications":["Alloy screening becomes a formula-only pipeline: tens of thousands of binary and ternary combinations can be ranked in minutes without density-functional calculations or experiments.","Printable liquid-metal electronics gain a specific lead candidate, In$_{0.5}$Sn$_{0.5}$, with a predicted $T_c$ of 7.01 K, worth testing for superconducting wires and flexible circuits.","Within a known alloy family, the model can guide composition tuning: the paper reports Ga-In-Sn predictions within about 1 K of experimental values.","Tree-based regression is a competitive baseline for $T_c$ prediction from composition, matching or beating neural-network results on this mid-sized dataset.","The predicted $T_c$ map over 66 metals gives experimentalists a prioritized list of thousands of plausible alloy superconductors below 40 K."],"supporting_citations":[{"why":"Supplies the eight printable liquid-metal alloy elements (Ga, Bi, In, Sn, Zn, Ag, Sb, Cu) that define the screening target.","marker":"[30]"},{"why":"Supplies experimental $T_c$ values for Ga-In-Sn alloys, including a 6.6 K maximum, used to check the model's predictions.","marker":"[31]"},{"why":"Reports synthesis of an InSn alloy superconductor below room temperature, supporting In$_{0.5}$Sn$_{0.5}$ as a plausible candidate.","marker":"[32]"},{"why":"Gives the argument that tree-based models outperform deep learning on tabular data, which motivates the choice of Extra Trees.","marker":"[25]"},{"why":"Source of non-superconducting records with $T_c = 0$ added to the dataset, teaching the model that some compositions do not superconduct.","marker":"[9]"},{"why":"Additional source of non-superconductor data added to the training set.","marker":"[10]"},{"why":"Provides the composition-only bagged-tree baseline that this work extends and improves with Extra Trees.","marker":"[14]"}],"fun_headline_variants":["Tree model predicts In0.5Sn0.5 as top superconductor","Extra Trees model finds In0.5Sn0.5 for liquid-metal Tc","Composition-only ML ranks liquid-metal alloys for Tc","Tree-based ML predicts 7.01 K Tc for In0.5Sn0.5","AI identifies In0.5Sn0.5 as highest-Tc liquid metal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline assumes that the superconducting transition temperature is fully determined by the proportions of the chemical elements, with no influence from crystal structure, pressure, or synthesis conditions; the paper's own discussion notes that one-hot coding records only element proportions and lacks structural information.","fun_headline_variants_meta":{"raw":{"variants":["Tree model predicts In0.5Sn0.5 as top superconductor","Extra Trees model finds In0.5Sn0.5 for liquid-metal Tc","Composition-only ML ranks liquid-metal alloys for Tc","Tree-based ML predicts 7.01 K Tc for In0.5Sn0.5","AI identifies In0.5Sn0.5 as highest-Tc liquid metal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000346,"raw_usage":{"total_tokens":1918,"prompt_tokens":988,"completion_tokens":930,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":827}},"tokens_in":604,"tokens_out":930,"duration_ms":8219,"temperature":1.0,"reasoning_tokens":827,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:13:37.566356+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Synthesize In$_{0.5}$Sn$_{0.5}$ at normal pressure and measure resistivity versus temperature: if the superconducting onset is not near 7.01 K, or if the alloy does not superconduct at all, the composition-only prediction is falsified for the paper's headline candidate. A broader version would measure a sample of the 2,145 binary and 45,760 ternary alloys the model ranks highly and compare predicted and observed $T_c$ values.","supporting_citations":[{"cited_title":"& Liu, J","cited_arxiv_id":null,"evidence_quote":"Supplies the eight printable liquid-metal alloy elements (Ga, Bi, In, Sn, Zn, Ag, Sb, Cu) that define the screening target."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies experimental $T_c$ values for Ga-In-Sn alloys, including a 6.6 K maximum, used to check the model's predictions."},{"cited_title":"& Matsushita, Y","cited_arxiv_id":null,"evidence_quote":"Reports synthesis of an InSn alloy superconductor below room temperature, supporting In$_{0.5}$Sn$_{0.5}$ as a plausible candidate."},{"cited_title":"& Varoquaux, G","cited_arxiv_id":null,"evidence_quote":"Gives the argument that tree-based models outperform deep learning on tabular data, which motivates the choice of Extra Trees."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of non-superconducting records with $T_c = 0$ added to the dataset, teaching the model that some compositions do not superconduct."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Additional source of non-superconductor data added to the training set."},{"cited_title":"& Dordevic, S","cited_arxiv_id":null,"evidence_quote":"Provides the composition-only bagged-tree baseline that this work extends and improves with Extra Trees."}],"review_version":1}