{"id":"60b67f07-e888-4375-8b20-eea9038b3ec4","arxiv_id":"2507.08155","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"QSVM and hybrid QNN models achieve around 90% validation accuracy on Mg stacking fault energy tasks, using a small dataset and no classical baseline.","lead":"This paper applies two standard quantum machine learning models, a quantum support vector machine and a quantum neural network, to predict stacking fault energies and ductilizing solutes in magnesium alloys. It reports validation scores around 90% on a small three-feature dataset, but without classical baselines or a held-out test set.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported ~90% validation scores are not statistically resolvable: with only a handful of validation samples per split, the 0.92-vs-0.88 gaps and the entanglement claim may reflect single-sample noise.","rationale":"The paper is a benchmark-style study: it applies QSVM and hybrid QNN to a small magnesium stacking-fault dataset and reports validation scores. Good faith reading shows the authors provide code and package versions, which is a real strength. The central claim, however, is that these models achieve approximately 90% validation accuracy and that this indicates practical applicability. The weakest link is the statistical basis of that claim. The reader's weakest_assumption identifies exactly this: the validation sets are tiny, so the reported accuracies and their comparisons are not stable. My stress-test pass confirms and sharpens this concern. First, the 80/20 split on a dataset of only a handful of solutes yields validation folds with a handful of samples, making a one-sample error move accuracy by 0.2–0.25; thus differences of 0.01–0.04 between configurations are below the resolution of the evaluation. Second, hyperparameters are selected by looking at the same validation scores, so the 'optimal' accuracy is the maximum over many noisy estimates, not an unbiased predictor of future performance. Third, the paper reports no confidence intervals, no classical baseline, and no held-out test set, so the central applicability claim lacks a quantitative anchor. The internal inconsistencies in the Conclusion (misreporting the QNN classification depth, duplicating the QSVR result, omitting QNN regression) reinforce that the reported numbers are not robust enough to carry the stated conclusions. I recommend keeping the reader's CONDITIONAL verdict: the empirical observations may be reproducible, but the central claims about optimality, entanglement superiority, and practical applicability require additional statistical validation and an independent test set. The proposed concrete test—computing exact binomial confidence intervals from the per-split predictions and comparing against baselines—would directly settle whether the 90% scores and the 0.92-vs-0.88 distinctions are meaningful.","tokens_in":9345,"tokens_out":4267,"duration_ms":50383,"concrete_test":"Run the provided code with the claimed optimal hyperparameters (QSVC, C=1, reps=3, full entanglement) over the same 20 shuffled 80/20 splits, but record the validation-set size and the per-split number of correct predictions. Compute the exact 95% binomial confidence interval for the pooled accuracy (total correct / total validation samples). Also run a majority-class baseline and a classical linear SVM on identical splits. If the CI lower bound is below 0.8, or if it overlaps the baseline accuracy, or if the CI for the full-entanglement configuration overlaps the CI for the linear-entanglement configuration, then the reported 0.92 optimum and the 'higher entanglement yields improved results' claim (Section 3.1) are not statistically meaningful; the paper would need to report error bars and an independent test set to support its central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that QSVM and hybrid QNN models yield validation scores around 90% and are applicable to real materials problems (Section 3.1, Conclusion)—rests entirely on validation accuracy numbers computed from a very small dataset. The paper lists only a handful of solutes (Pr, Sc, Ti, Zr, Nd, Os, Co, etc.), and the protocol in Section 3.1 uses an 80/20 split, so each validation fold contains only about 3–5 samples if the total dataset has ~15–25 entries. With such tiny validation sets, a single sample corresponds to a 0.20–0.25 accuracy swing, so the reported differences (0.92 vs 0.91 vs 0.88, or 0.88 vs 0.81 in QSVR) are smaller than the resolution of one validation point. The scores are also averaged over shuffled splits and then used to select the best hyperparameters from a grid (C, reps, entanglement); this maximum-of-grid selection is optimistically biased and cannot support claims of optimality without a separate test set or correction. No error bars, confidence intervals, or classical baselines (e.g., majority-class or linear SVM) are reported, so the statement in the conclusion that 'promising accuracies … indicate their applicability' is not supported by the evidence shown. The internal inconsistencies in the Conclusion (stating QNN classification optimum at l=1 with 0.91 when Section 3.3 reports l=2 with 0.905 and l=1 with 0.810; repeating the QSVR result and omitting QNN regression) further undermine confidence that the headline numbers were carefully verified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript applies quantum support vector machines (QSVM) and hybrid estimator quantum neural networks (QNN) to a small materials-science dataset: three elemental features (bulk modulus, atomic volume, electronegativity) are used to predict stacking fault energies in magnesium alloys and to classify solutes as ductilizing or not. The authors scan a hyperparameter grid (C, reps/depth, entanglement pattern, epsilon) and report validation scores around 0.88-0.92 for both classification and regression. They conclude that the selected QML models are promising for real materials-science problems and that higher-entanglement circuits perform better.","tokens_in":9711,"tokens_out":3899,"duration_ms":43013,"significance":"If the reported numbers were statistically robust independent predictions, this would be a useful small-scale demonstration that QML workflows can be applied to a concrete materials problem with real data, and the paper's code availability with pinned package versions is a genuine strength. The manuscript also systematically varies ansatz depth and entanglement patterns, which is informative for practitioners. However, the central quantitative claims are not supported by the evaluation protocol: the validation sets are tiny, no error bars or confidence intervals are given, hyperparameters are selected on the same validation data used to report the 'optimal' scores, and no classical baselines are provided. These issues make the headline accuracies and the entanglement conclusions unreliable in their current form.","major_comments":[{"comment":"The paper never states the dataset size, but the solute list (Pr, Sc, Ti, Zr, Nd, Os, Co, etc.) suggests roughly 15-25 entries. With the 80/20 split described in §3.1, each validation fold contains only about 3-5 samples, so a single misclassified or correctly classified sample changes the reported accuracy by 0.20-0.25. The differences highlighted in the text (0.92 vs 0.91 in QSVC, 0.88 vs 0.81 in QSVR, 0.905 vs 0.810 in QNN classification) are therefore within the resolution of a single validation point, and the claims in §3.1 and §3.2 that 'a higher level of entanglement yields improved results' and that entanglement is 'important' are not supported. The authors should report N, the per-fold predictions, and error bars or confidence intervals, and should consider leave-one-out or repeated stratified cross-validation with a separate held-out test set.","section":"§3.1-§3.4"},{"comment":"The quoted 'optimal performance' is obtained by maximizing accuracy over a hyperparameter grid evaluated on the validation data. Table 1 alone defines 4×3×5 = 60 combinations, and the text says results are averaged over three tests; selecting the maximum of this grid is an optimistic estimate, not an independent predictive score. No held-out test set is used, and no classical baselines (linear SVM, RBF SVM, random forest, majority-class classifier) are reported. Consequently, the conclusion in §4 that 'the promising accuracies of these four models indicate their applicability in real materials science problems' is not established. The authors should evaluate the final selected models on a true test set or use nested cross-validation, and should compare against simple classical models to show whether QML adds any predictive value.","section":"§3.1, §4"},{"comment":"The Conclusion is internally inconsistent with the Results. Section 3.3 reports hybrid QNN classification accuracies of 0.810 for l=1, 0.905 for l=2, and 0.857 for l=3, and states the optimum is l=2. Section 4 instead states 'an optimal accuracy of 0.91 is achieved ... with a depth of l=1.' In addition, Section 4 repeats the QSVR regression result twice and omits the hybrid QNN regression results (R2 = 0.875, 0.865, 0.864 reported in §3.4). These inconsistencies undermine confidence that the headline numbers were carefully verified. The authors should harmonize the reported optimum with the corresponding figure and table, and list all four task/model results exactly once.","section":"§3.3, §4"},{"comment":"The regression and classification outcomes are not compared against the trivial baseline or the data distribution. For classification, the dataset is small and the label threshold (SFE of pure Mg, 19 mJ/m²) may produce an imbalanced class distribution; a model that always predicts the majority class could already achieve high accuracy. For regression, reporting R2 = 0.875 without the sample size or a null-model comparison (e.g., predicting the mean SFE) makes it impossible to judge whether the hybrid QNN captures real structure or simply memorizes a few points. Adding these baselines is essential for the paper's central claim of applicability.","section":"§3.1-§3.4"}],"minor_comments":[{"comment":"Table 1 lists C in [0.1, 1, 10, 100], but the text says 'We examined a range of C values corresponding to powers of 10, ranging from 0.01 to 100.' Please reconcile the stated range with the table.","section":"§3.1"},{"comment":"Equation (4) is not the standard SVM formulation: the text says the target is 'minimizing the reciprocal inverse magnitude of the weight vector w,' which would be minimizing 1/||w||. In a maximum-margin SVM one maximizes 1/||w|| (or equivalently minimizes ||w||²). Please correct the equation and the surrounding wording.","section":"§2.1.1"},{"comment":"Two GitHub repositories with different cross-validation schemes are listed (five-fold versus twenty-fold). Please specify which repository and which fold count were used to generate each figure, since the text describes only five-fold cross-validation.","section":"Code Availability"},{"comment":"Figures 3 and 4 lack explicit axis labels and legends: the caption says 'validation scores with various C, reps, and entanglement' but does not identify which panel corresponds to which hyperparameter. Figure 5 would also benefit from a labeled colorbar or explicit accuracy values on each bar.","section":"Figures 3-5"},{"comment":"There are several typos and wording issues: 'anzates' in §3.5, 'vary different' in §1, and 'unit-less' should be 'dimensionless.' A careful proofread is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful, reproducible code base and addresses a real materials dataset, but the evaluation protocol is currently too weak to support the main quantitative claims. I see this as fixable within the manuscript's scope: the authors could re-run the analysis with proper test splits or leave-one-out CV, report exact N and per-fold results, add error bars and classical baselines, and soften the claims accordingly. The internal inconsistency between §3.3 and §4 should be corrected without fail. I would not recommend rejection at this stage, but the revision needs to be substantive rather than cosmetic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a straightforward application of two standard QML models (QSVM and a hybrid Estimator QNN) to a small magnesium alloy dataset, with code and pinned package versions. That reproducibility artifact is real and useful: anyone can rerun the sweep. The body reports the raw validation numbers (0.92, 0.88, 0.905, R2=0.875) plainly, and I don't see evidence of cherry-picked figures.\n\nThe novelty is thin. It's a hyperparameter scan of known circuits on a new dataset—no new ansatz, no theory, no claim of quantum advantage. That's acceptable for an application note, but the evaluation has to carry the weight, and here it doesn't. The validation sets are tiny: with only a few solutes in the data, an 80/20 split leaves maybe 3–5 validation samples. One sample is a 0.2–0.25 accuracy swing, so the gaps between 0.92 and 0.88 are within single-sample noise. Hyperparameters are selected on the same validation data, making the 'optimal' scores fitted values rather than independent predictions. There are no error bars and no classical baselines, so the conclusion that these accuracies 'indicate their applicability' is not supported.\n\nThe conclusion also misreports its own numbers: it gives the QNN classifier's optimum as l=1 with 0.91, but the body says l=2 with 0.905 and l=1 with 0.810; and it lists QSVR twice. That suggests the headline numbers were not carefully cross-checked.\n\nThe central empirical claim may be true—the models likely do reach those scores on this data—but the paper doesn't demonstrate anything beyond that, and the entanglement-superiority remark is especially fragile.\n\nWho this is for: practitioners implementing QSVM or hybrid QNN on a small tabular dataset who want a concrete, working starting point. It's not a paper that moves the QML-for-materials needle.\n\nRecommendation: send it to peer review, but with the expectation of heavy revision. A proper test set, error bars, classical baselines, and a corrected conclusion would turn this into a solid application note. As is, it's a workshop-grade empirical record.","headline":"A small, honestly reported QML-on-materials demo whose headline accuracies are likely real but statistically under-supported.","tokens_in":10226,"tokens_out":2770,"would_cite":false,"duration_ms":28809,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantum SVM and hybrid QNN models can be tuned to roughly 90% validation accuracy on a real magnesium-alloy dataset.","keywords":["quantum machine learning","quantum support vector machine","quantum neural network","stacking fault energy","magnesium alloys","hyperparameter optimization","hybrid quantum-classical model","QSVM"],"falsifier":"Re-evaluate the reported optimal configurations using leave-one-out cross-validation on the same dataset. If the QSVM classification accuracy (reported 0.92) falls below chance-plus-a-few-points, or if the hybrid QNN regression $R^2$ (0.875) collapses when a single outlier solute (Nd or Pr) is removed, the central claim of stable ~90% performance would be undermined.","tokens_in":9127,"feed_emoji":"⚛️","tokens_out":5972,"duration_ms":55403,"temperature":0.7,"pith_summary":"The paper tries to show that two quantum machine-learning algorithms, a quantum support vector machine (QSVM) and a hybrid quantum neural network (QNN), can be tuned to reach validation scores around 90% on a real materials-science dataset. The dataset is small: three elemental features (bulk modulus, atomic volume, electronegativity) for solutes in magnesium, with stacking fault energies as labels. Classification labels solutes as ductilizing (SFE below 19 mJ/m$^2$) or not; regression predicts the SFE value itself. By sweeping hyperparameters such as the SVM cost $C$, circuit depth reps, and entanglement type, the authors find optimal settings for each task and report that highly entangled circuits generally outperform linear ones. If the results hold, they indicate that QML models can be applied to practical materials problems rather than only synthetic benchmarks.","feed_headline":"Quantum models hit ~90% on magnesium alloy predictions","feed_subtitle":"QSVM and hybrid quantum neural nets predict stacking fault energies and ductilizing solutes from three element properties.","key_machinery":"The argument is carried by two quantum circuits. For QSVM, a ZZFeatureMap encodes the three classical features into qubit states and computes a quantum kernel $K(x,x')=|\\langle\\Phi(x)|\\Phi(x')\\rangle|^2$ that replaces the classical SVM kernel; the circuit depth (reps), entanglement pattern (circular, full, or linear), and cost parameter $C$ are tuned. For the hybrid QNN, the same feature map is followed by a RealAmplitudes ansatz, and the whole circuit is wrapped as a layer in a PyTorch neural network via TorchConnector, with the number of repeats $l$ as the main depth parameter. The paper's central observation is that these circuits, after hyperparameter search, produce validation scores around 90% on the magnesium stacking-fault dataset.","core_discovery":"The central claim is that QSVM and hybrid QNN models, with carefully selected hyperparameters, achieve validation scores of roughly 0.88 to 0.92 on both classification and regression tasks for magnesium stacking fault energies. The paper reports optimal validation scores of 0.92 for QSVM classification (with $C=1$, reps=3, and full entanglement), 0.88 for QSVM regression (reps=1 with circular or full entanglement), about 0.91 for hybrid QNN classification, and $R^2=0.875$ for hybrid QNN regression. The authors conclude that complex entanglement, either circular or fully entangled, is superior to linear entanglement, and that these accuracies indicate the models are applicable to real materials-science problems.","pith_inferences":["A direct extension is to compare QSVM and hybrid QNN against classical SVM and classical neural networks on identical train/validation splits; without that baseline, the 90% scores show feasibility but not quantum advantage.","The same workflow could be applied to larger multi-component alloy datasets where the number of features exceeds three, forcing the circuits onto more qubits and making entanglement topology a more consequential hyperparameter.","Re-running the reported configurations with leave-one-out cross-validation or repeated shuffled splits with confidence intervals would test whether the 0.92 versus 0.88 differences are statistically meaningful given the small solute set."],"forward_implications":["QSVM and hybrid QNN models can be applied to real materials-science data with only three features and a small number of solutes, reaching roughly 90% validation accuracy.","Full or circular entanglement consistently outperforms linear entanglement, suggesting that entangled feature spaces carry useful information for this problem.","The optimal hyperparameters differ between classification and regression even for the same data and algorithm, so task-specific tuning is necessary.","The hybrid QNN regression model identifies Nd and Pr as outliers with low SFEs and Os and Co as outliers with high SFEs, while still classifying the latter qualitatively correctly as non-ductilizing.","These optimized quantum models can predict targeted solutes based on elemental volume, electronegativity, and bulk modulus."],"supporting_citations":[{"why":"Supplies the magnesium stacking-fault-energy dataset (features and labels) used for all training and validation.","marker":"[25]"},{"why":"Provides the ab initio stacking-fault-energy calculations in Mg and Mg-Y that underpin the dataset values.","marker":"[24]"},{"why":"Designs QSVM models for alloy design with quantum circuits, the approach adapted here.","marker":"[15]"},{"why":"Introduces quantum feature maps and kernels, the theoretical basis of the QSVM kernel.","marker":"[33]"},{"why":"Provides the quantum software environment used to implement circuits, feature maps, and QNN.","marker":"[28]"},{"why":"Supplies the classical SVM machinery that the QSVM wraps with the quantum kernel.","marker":"[32]"}],"fun_headline_variants":["Quantum ML hits 90% on magnesium alloy tasks","QSVM and hybrid quantum nets score ~90% in materials","Quantum models predict ductile magnesium solutes at 90%","Hybrid quantum ML reaches 90% on stacking fault energies","Quantum classifiers top 90% for magnesium ductility"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 90% validation scores are assumed to be stable even though the validation sets contain only a few samples, so a one-sample change could move a score by several points and the paper provides no error bars or separate test set.","fun_headline_variants_meta":{"raw":{"variants":["Quantum ML hits 90% on magnesium alloy tasks","QSVM and hybrid quantum nets score ~90% in materials","Quantum models predict ductile magnesium solutes at 90%","Hybrid quantum ML reaches 90% on stacking fault energies","Quantum classifiers top 90% for magnesium ductility"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1205,"prompt_tokens":853,"completion_tokens":352,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":469,"tokens_out":352,"duration_ms":4133,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:26:09.524661+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-evaluate the reported optimal configurations using leave-one-out cross-validation on the same dataset. If the QSVM classification accuracy (reported 0.92) falls below chance-plus-a-few-points, or if the hybrid QNN regression $R^2$ (0.875) collapses when a single outlier solute (Nd or Pr) is removed, the central claim of stable ~90% performance would be undermined.","supporting_citations":[{"cited_title":"Sandlöbes, Z","cited_arxiv_id":null,"evidence_quote":"Supplies the magnesium stacking-fault-energy dataset (features and labels) used for all training and validation."},{"cited_title":"Pei, L.-F","cited_arxiv_id":null,"evidence_quote":"Provides the ab initio stacking-fault-energy calculations in Mg and Mg-Y that underpin the dataset values."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Designs QSVM models for alloy design with quantum circuits, the approach adapted here."},{"cited_title":"Havlíˇcek, A","cited_arxiv_id":null,"evidence_quote":"Introduces quantum feature maps and kernels, the theoretical basis of the QSVM kernel."},{"cited_title":"Javadi-Abhari, M","cited_arxiv_id":null,"evidence_quote":"Provides the quantum software environment used to implement circuits, feature maps, and QNN."},{"cited_title":"Pedregosa, G","cited_arxiv_id":null,"evidence_quote":"Supplies the classical SVM machinery that the QSVM wraps with the quantum kernel."}],"review_version":1}