{"id":"44ff7455-016e-4065-b2fa-3c3b2c2f1395","arxiv_id":"2411.13670","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A graph convolutional network trained on hybrid MC/MD simulation snapshots predicts the potential energy of a NiCoCr medium-entropy alloy with R2 above 0.93 and MAPE around 2-5%.","lead":"The paper trains graph neural networks to predict the potential energy of NiCoCr medium-entropy alloy configurations generated by hybrid Monte Carlo/molecular dynamics simulations across annealing temperatures. It tests whether a graph network can serve as a fast surrogate for atomistic simulations of chemically ordered alloys, potentially accelerating searches for new alloy designs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Velocity as a node feature may let the GCNN infer temperature instead of learning the position-only EAM energy; a permutation test is needed to rule out this confound.","rationale":"The reader's weakest assumption identified both the 12-neighbor truncation and the velocity feature as questionable. I focus on the velocity feature as the single most load-bearing concern because it directly violates the physical input–output relationship of the EAM potential: potential energy depends solely on atomic positions. If the model uses velocity, it is not learning a configuration-to-energy mapping but rather a temperature or MC-stage proxy, which would invalidate the central claim of 'energy mapping' and the broader assertion of capturing LCO. The proposed permutation test is decisive because it directly tests whether the model's predictions depend on velocity while holding the configuration fixed. If predictions change, the model has learned a spurious dependence; if they do not, the concern is empirically refuted. The paper provides code on GitHub, but the test can be run on the existing trained model, so it is a low-cost, high-information check. The reader's verdict of CONDITIONAL is appropriate; my concern does not move it, as the condition should include this velocity test. I considered the architectural confusion (input channels equal to 13500 in Sections 3.5.1–3.5.2) but that is a reproducibility issue rather than a direct confound of the energy-mapping claim; the velocity feature is more fundamental. The 12-neighbor truncation is also a valid concern but is less likely to be the primary driver of performance, since first-neighbor environments dominate EAM energy in dense FCC alloys; the velocity shortcut is the most plausible explanation for the reported accuracy. Thus, the verdict remains CONDITIONAL pending this specific test.","tokens_in":12122,"tokens_out":7423,"duration_ms":58385,"concrete_test":"Take the trained case-study-3 model and, for a fixed set of test configurations, randomly permute the velocity features across snapshots while keeping positions and atom types unchanged. If the predicted potential energies change by more than the reported MAPE (or if R2 drops substantially), the model is relying on velocity, demonstrating that it is not a position-only energy function. If predictions remain essentially unchanged (within MAPE), velocity is not the load-bearing feature and the concern is resolved. A complementary check is to retrain the model using only atom-type features and compare held-out accuracy; a significant drop would confirm the velocity shortcut.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the GCNN maps atomic configurations to EAM potential energy. However, Section 3.4 specifies node features as 'atom type and absolute velocity', and Section 3.5 describes training on these features. The EAM potential in Eq. 2 is a function of atomic positions only; no velocity term appears. Including velocity allows the model to exploit the strong correlation between the velocity distribution (instantaneous temperature) and the annealing-temperature-dependent potential-energy level. In case study 3, the model is tested on temperatures not used in training, so velocity can act as a direct indicator of the target temperature. High R2 values may then reflect temperature interpolation rather than generalization to unseen atomic arrangements. This undermines the stated purpose of an energy-mapping surrogate and the claim that the model captures local chemical order. The reported R2 = 0.93–0.98 and MAPE = 0.018–0.050 across case studies are consistent with this shortcut, because the velocity feature is available at inference time but is not a function of the configuration whose energy is being predicted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a graph convolutional neural network (GCNN) framework to predict the potential energy of a NiCoCr medium-entropy alloy from hybrid Monte-Carlo/molecular dynamics (MC/MD) simulation data. Atomic configurations from LAMMPS dump files are converted into graphs with atoms as nodes, edges to the 12 nearest neighbors, and node features consisting of atom type and absolute velocity. The model is trained on per-atom potential energy labels from the EAM potential. Three case studies are presented: training separate models for individual annealing temperatures (450K, 650K, 950K), training one model on a combined set of temperatures (450K, 650K, 850K), and training on a set of temperatures (350K, 650K, 850K, 1150K) to test on unseen temperatures. Reported R2 values range from 0.93 to 0.98 and MAPE from 0.018 to 0.050 for the first two cases, while the third case shows degraded performance at higher temperatures. The abstract claims strong performance on both training data and unseen configurations.","tokens_in":12354,"tokens_out":4829,"duration_ms":48652,"significance":"If validated, the proposed GCNN would provide a fast surrogate for the EAM potential in MC/MD energy mapping, potentially enabling efficient exploration of local chemical order in MEAs and HEAs. However, the current evidence is undermined by two critical methodological issues: the inclusion of absolute velocity as a node feature, which can act as a temperature proxy and decouple predictions from the actual atomic configuration, and the absence of a clearly described held-out test split in the first two case studies. The third case study, which does use held-out temperatures, reports degraded performance, weakening the central claim. The authors do provide data and code availability on GitHub, which is a strength for reproducibility. Substantial methodological revision is needed before the claims can be accepted.","major_comments":[{"comment":"The node features include absolute velocity, but the EAM potential in Eq. (2) is purely a function of atomic positions. Velocity is not a position-derived feature and directly encodes the instantaneous temperature via the Maxwell-Boltzmann distribution. In the hybrid MC/MD annealing simulations, potential energy is strongly correlated with temperature, so the model can output the correct energy level by reading the velocity feature rather than learning the configurational energy surface. In case study 3, test temperatures are not in the training set, making velocity a direct temperature tag. To support the claim that the model captures local chemical order, the authors should remove velocity from the features or perform a permutation test that shuffles velocities across configurations and show that prediction quality does not degrade.","section":"Section 3.4, Eq. (2)"},{"comment":"The first two case studies report high R2 and low MAPE values, but the text does not describe any held-out train/test split within each annealing temperature. The description states that the parameters with the lowest MSE are stored, implying that the metrics are computed on the training data themselves. The abstract's claim of performance on 'unseen configurations' is therefore not supported for these cases. The authors should specify a proper split (e.g., reserving a fraction of the 300 MC frames per temperature for validation and testing) and report metrics on the test set, not just the training set.","section":"Sections 3.5.1, 3.5.2, 4.2, 4.3"},{"comment":"The architecture description is internally inconsistent and incomplete. The first GCNConv layer is stated to have an input channel count equal to the number of atoms (13500), yet Section 3.4 says that node features are atom type and absolute velocity, which would be a 2-dimensional feature vector per node. Additionally, GCNConv outputs per-node features, but the target is a single scalar potential energy per graph; no graph-level readout (e.g., global pooling) is described between the final GCNConv layer and the fully connected layer. Please clarify the exact data flow, including how per-node outputs are aggregated into the scalar energy prediction.","section":"Section 3.5.1, Fig. 1"},{"comment":"The third case study is the only one that tests on genuinely unseen configurations (held-out annealing temperatures), and the text admits that R2 drops as annealing temperature increases and that the model 'struggles to predict the higher energy levels.' This directly contradicts the abstract and conclusion, which claim satisfactory results on unseen configurations. The authors should report the actual R2 and MAPE values for each temperature in both the training and test sets, and temper the summary claims to reflect the observed degradation.","section":"Section 4.4"},{"comment":"The graph representation includes edges only to the 12 nearest neighbors (the first coordination shell in FCC). However, the EAM potential (Eq. 2) includes a pair potential and an embedding functional that depend on the electron density, which may extend beyond the first shell for the chosen potential. Truncating the neighborhood to 12 neighbors could discard physically relevant interactions. The authors should justify this truncation or perform a sensitivity analysis with a larger neighbor count.","section":"Section 3.4"}],"minor_comments":[{"comment":"The figure caption lists '1350K' but the text uses '1150K' as the highest annealing temperature; please make the notation consistent.","section":"Fig. 2(c)"},{"comment":"The statement that the first GCNConv layer has 'input channels equal to the number of atoms in the system (which is 13500)' is inconsistent with the node feature matrix described in Section 3.4; this is likely a typo and should be corrected to the node feature dimension.","section":"Section 3.5.1"},{"comment":"The text says R2 values range from 0 to 1, but R2 can be negative for poorly fitting models; the wording should be adjusted for mathematical accuracy.","section":"Section 3.6"},{"comment":"The justification for including velocity as a feature relies on reference [51], but that work uses velocity in a different context; a physical argument for why velocity is relevant to a position-dependent energy function is needed.","section":"Section 3.4"},{"comment":"The data availability statement says all data and codes are available on GitHub, but the repository link is in the author contributions section; please include the URL in the Data Availability section as well.","section":"Data Availability"}],"recommendation":"major_revision","confidential_remarks":"The velocity feature is a serious confound that, together with the missing holdout evaluation in the first two case studies, makes the reported R2 values unreliable as evidence of learning the configurational energy surface. The authors' own third case study shows clear degradation on unseen temperatures, which should be front and center rather than hidden in the limitations. I recommend major revision, not rejection, because the issues are fixable by re-running the experiments without velocity and with proper train/test splits."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is an honest, incremental application of Noda et al.'s GCNN to a new alloy system, with code and data provided. The central claim—that the model predicts potential energy of hybrid MC/MD configurations—is plausible but not well supported because the node features include absolute velocity, which can act as a temperature proxy, and because case studies 1 and 2 appear to be evaluated on training data.\n\nWhat's new: the application to NiCoCr MEA with hybrid MC/MD data, the 12-nearest-neighbor graph construction for LCO, and the three case studies including a held-out temperature test. The acknowledgment of the debt to Noda et al. is exemplary, and the GitHub repository is a real plus. Case 3, which truly holds out temperatures, is the most informative and shows expected degradation at high temperatures—they note this. That is honest.\n\nThe soft spots are real. First, velocity is not a function of atomic positions, and the EAM potential is purely positional. Since the velocity distribution is tied to the annealing temperature, the model can infer the energy level from the temperature without learning the structural mapping. A permutation test or training without velocity would be needed to rule out this shortcut. Second, case studies 1 and 2 do not describe a held-out test set; they report R2 on configurations used to select the lowest-MSE parameters. The abstract overclaims 'unseen configurations' when only case 3 actually tests that, with mixed results. Third, the 12-neighbor cutoff may miss EAM embedding contributions beyond the first shell, though that is secondary.\n\nThis is not a bad paper—it is a straightforward surrogate study with reproducible artifacts. But the velocity confound cuts to the heart of the physical interpretation. I would not cite it as a reliable energy model without the confound addressed.\n\nRecommendation: send to peer review, but require the authors to justify or remove the velocity feature, provide explicit train/test splits for all case studies, and soften the abstract's claims. A permutation test using shuffled velocities would be the cleanest fix.","headline":"A transparent but methodologically shaky GNN energy surrogate for NiCoCr; the velocity feature is a likely temperature proxy and the reported R2 is not fully trustworthy.","tokens_in":12885,"tokens_out":3334,"would_cite":false,"duration_ms":923729,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph convolutional network predicts potential energy in NiCoCr alloy simulations with R2 up to 0.98.","keywords":["Graph neural network","Medium-entropy alloy","Hybrid Monte-Carlo molecular dynamics","Potential energy prediction","Local chemical order","NiCoCr","EAM potential","Machine learning surrogate"],"falsifier":"Build two atomic configurations of NiCoCr that have identical species and velocities on every atom and identical 12-nearest-neighbor environments but differ in the arrangement of atoms in the second coordination shell; if the GCNN assigns nearly identical energies to both while the EAM potential gives noticeably different energies, the 12-neighbor graph is not a faithful energy mapping.","tokens_in":11955,"feed_emoji":"⚛️","tokens_out":6038,"duration_ms":55715,"temperature":0.7,"pith_summary":"The paper claims that a graph convolutional neural network can predict the potential energy of a NiCoCr medium-entropy alloy during hybrid Monte-Carlo/molecular dynamics simulations, using a graph where each atom is a node connected to its 12 nearest neighbors and node features are atom type and absolute velocity. Across three case studies, the model reaches R2 values between 0.93 and 0.98 and mean absolute percentage errors between 0.018 and 0.050, including for configurations at annealing temperatures not seen during training. If correct, the GCNN serves as a fast surrogate for the embedded-atom-method potential and captures the local chemical order that controls properties of the alloy. The significance is a graph-based route to predicting energy-driven properties of medium- and high-entropy alloys from simulation snapshots.","feed_headline":"Graph neural network maps alloy energy across annealing temperatures","feed_subtitle":"Trained on 12-neighbor atomic graphs, the model captures NiCoCr's local chemical order from MC/MD runs.","key_machinery":"The central object is the graph representation of an alloy configuration: 13,500 nodes, one per atom, with node features (atom type, absolute velocity) and edges to the 12 nearest neighbors plus self-loops, with no edge features. The GCNN stacks GCNConv layers using the normalized adjacency with self-loops, followed by ReLU and dropout, and is trained with mean squared error to regress the per-atom normalized total potential energy. The 12-neighbor edge construction is the mechanism intended to encode the local chemical order measured by Warren-Cowley parameters, and the velocity feature is included on the basis of prior use in GNN energy prediction.","core_discovery":"The discovery is that the energy profile of hybrid MC/MD simulations of the equiatomic NiCoCr MEA can be mapped by a GCNN that reads only atom identity, absolute velocity, and a 12-nearest-neighbor edge list. The model reproduces the potential-energy-versus-MC-step curves for individual annealing temperatures (450K, 650K, 950K) with R2 of 0.98, 0.93, and 0.96; trained on a combined set (450K, 650K, 850K) it predicts all three with R2 of 0.97, 0.97, and 0.96; and trained on 350K, 650K, 850K, 1150K it generalizes to held-out temperatures with the best performance at lower temperatures and a documented drop at higher temperatures where energy plateaus quickly.","pith_inferences":["Editorial inference: the inclusion of absolute velocity as a node feature means the model may be learning a temperature or kinetic-energy proxy in addition to the potential-energy surface; retraining with velocity withheld would reveal how much of the accuracy is structural.","Editorial inference: because edges are truncated at 12 nearest neighbors and unweighted, contributions to the EAM embedding energy from atoms beyond the first coordination shell are invisible to the model, so the mapping is likely an effective first-shell model rather than a complete EAM surrogate.","Editorial inference: a natural stress test is to apply the trained GCNN as the energy evaluator inside the Metropolis acceptance step of a fresh MC/MD run; if the MC trajectories remain stable and consistent, the surrogate is usable for accelerated sampling."],"forward_implications":["If the central claim holds, one trained GCNN can replace repeated EAM energy evaluations during MC/MD annealing at the temperatures it was trained on, at a small fraction of the cost.","A single model trained on a few annealing temperatures covers intermediate temperatures with R2 >= 0.96, so the approach transfers across thermal conditions within the training range.","Generalization to unseen annealing temperatures is demonstrated for lower temperatures, establishing that graph representations of MEA configurations encode transferable structure-energy relationships.","Because the graph has no edge features, the framework is directly applicable to other MEAs and HEAs without per-system feature engineering.","The observed degradation at high unseen temperatures identifies the data-density limitation that future work would need to address."],"supporting_citations":[{"why":"Supplies the EAM interatomic potential that defines the potential-energy labels the model is trained to predict.","marker":"[40]"},{"why":"Provides the graph-convolutional energy-profile prediction method that this work adapts to MEAs.","marker":"[51]"},{"why":"LAMMPS runs the hybrid MC/MD simulations that generate the dump files and energy labels.","marker":"[46]"},{"why":"Atomsk creates the random equiatomic FCC NiCoCr structure used as the starting configuration.","marker":"[45]"},{"why":"Gives the Metropolis acceptance probability used in the swap attempts of the hybrid MC/MD scheme.","marker":"[47]"},{"why":"Defines the Warren-Cowley parameter used to quantify the local chemical order the graph is meant to capture.","marker":"[49]"},{"why":"Provides the OVITO-Python modifier used to compute Warren-Cowley parameters from simulation outputs.","marker":"[50]"}],"fun_headline_variants":["GNN maps alloy energy from MC/MD neighbor graphs","Graph nets predict MEA energy across temperatures","GCNN reads 12-atom neighborhoods to map alloy energy","Neural graphs capture local order to predict MEA energy","AI decodes alloy energy from atomic arrangements"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that each atom's total potential energy under the EAM potential is fully determined by its atom type, its absolute velocity, and the species of its 12 nearest neighbors, even though the EAM embedding function sums over neighbors within a longer cutoff.","fun_headline_variants_meta":{"raw":{"variants":["GNN maps alloy energy from MC/MD neighbor graphs","Graph nets predict MEA energy across temperatures","GCNN reads 12-atom neighborhoods to map alloy energy","Neural graphs capture local order to predict MEA energy","AI decodes alloy energy from atomic arrangements"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000248,"raw_usage":{"total_tokens":1562,"prompt_tokens":977,"completion_tokens":585,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":509}},"tokens_in":593,"tokens_out":585,"duration_ms":7117,"temperature":1.0,"reasoning_tokens":509,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:59:54.543965+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build two atomic configurations of NiCoCr that have identical species and velocities on every atom and identical 12-nearest-neighbor environments but differ in the arrangement of atoms in the second coordination shell; if the GCNN assigns nearly identical energies to both while the EAM potential gives noticeably different energies, the 12-neighbor graph is not a faithful energy mapping.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the EAM interatomic potential that defines the potential-energy labels the model is trained to predict."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the graph-convolutional energy-profile prediction method that this work adapts to MEAs."},{"cited_title":"Plimpton, Fast parallel algorithms for short-range molecular dynamics, J","cited_arxiv_id":null,"evidence_quote":"LAMMPS runs the hybrid MC/MD simulations that generate the dump files and energy labels."},{"cited_title":"Hirel, Atomsk: A tool for manipulating and converting atomic data files, Comput","cited_arxiv_id":null,"evidence_quote":"Atomsk creates the random equiatomic FCC NiCoCr structure used as the starting configuration."},{"cited_title":"Chen, Z.H","cited_arxiv_id":null,"evidence_quote":"Gives the Metropolis acceptance probability used in the swap attempts of the hybrid MC/MD scheme."},{"cited_title":"Cowley, An approximate theory of order in alloys, Phys","cited_arxiv_id":null,"evidence_quote":"Defines the Warren-Cowley parameter used to quantify the local chemical order the graph is meant to capture."}],"review_version":1}