REVIEW 2 major objections 5 minor 55 references
Robust data-driven approach for predicting the configurational energy of high entropy alloys
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A pair-interaction model fitted with Bayesian regression and BIC shell selection predicts the configurational energy of refractory high-entropy alloys with a held-out RMSE around 0.6 meV.
desk verdict Solid incremental methods paper for sparse-data HEAs configurational energy; accuracy holds on random configs but thermodynamic claims outrun the validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the effective pair interaction (EPI) model, an Ising-like Hamiltonian that maps a configuration to an energy through the probabilities $P^{X|Y}_m$ of finding element $X$ in the $m$-th coordination shell around element $Y$, for each independent pair $X\neq Y$. Three pieces carry the argument: Bayesian regularized regression with conjugate gamma priors on the noise and coefficient precisions, which supplies stable point estimates and uncertainty quantification for the pair coefficients; the BIC shell-selection criterion, which decides how many of the shells to keep; and ensemble sampling, which draws configurations from several supercell sizes so the training data include both short-range and long-range order.
What would settle it
Refit Eq. (8) with an additive supercell-size offset and with triplet correlation features on the same DFT data; if either change materially lowers the held-out RMSE or shifts the BIC-selected shell count for these alloys, the pair-only, size-independent assumption behind the 0.6 meV claim is falsified.
Extended reading notes
Core claim
The central claim is that the configurational energy of a multicomponent alloy can be written, to good accuracy, as a sum of chemically distinct pair probabilities in the first few coordination shells, $E = N\sum_{X\neq Y,m} J^{X,Y}_m P^{X|Y}_m$, and that the coefficients $J^{X,Y}_m$ can be estimated reliably by Bayesian $\ell^2$-regularized regression even when the number of DFT configurations is small. To set model complexity, the paper minimizes the Bayesian information criterion expressed through the residual sum of squares, $\mathrm{BIC}_{\mathrm{RSS}} = n_d \log(\mathrm{RSS}/n_d) + k\log(n_d)$, and shows that the selected shell count grows sensibly with dataset size. With the BIC-selected shells and an ensemble sampling strategy that mixes supercells of 16–160 atoms, the fitted models reach testing RMSE about 0.6 meV on the three refractory alloys, and the uncertainty in the pair interactions drops sharply once a few hundred configurations are available.
Load-bearing premise
The model only works if the configurational energy is fully determined by two-body pair probabilities with coefficients that do not depend on supercell size, so significant triplet or many-body interactions, or a size-dependent offset in the DFT energies, would bias the fitted coefficients and the reported 0.6 meV accuracy.
Editorial extensions
If this is right
- A Monte Carlo simulation of order-disorder transitions can replace direct DFT calls with the fitted EPI Hamiltonian, because the surrogate predicts large-supercell energies from small-supercell training data to within about 1 meV.
- The BIC shell count supplies a data-size-dependent recipe—2–3 shells for small datasets, 5–6 for medium ones, 6–9 for large ones—so model complexity no longer has to be set arbitrarily.
- Ensemble sampling across supercell sizes should be part of similar data-driven alloy models, since it lowers both the mean and the standard deviation of the prediction error compared with training on a single small supercell.
- Increasing the number of training configurations from 100 to 400 reduces the variance of the fitted pair interactions by more than an order of magnitude, so the framework can tell users how much confidence to place in each energy estimate.
- For NbMoTaWV and NbMoTaWTi the best model retains more coordination shells than for NbMoTaW, indicating that nearest-neighbor-only pair models would systematically underfit those alloys.
Reading between the lines
- Because the EPI model is pair-only, a direct extension would be to add triplet correlation features and compare BIC; if triplets systematically lower held-out RMSE on these alloys, the 0.6 meV claim would need to be weakened.
- The BIC-selected shell count can be read as a measured interaction range, which suggests a testable prediction: independent electronic-structure calculations should find longer-ranged or more frustrated effective interactions in NbMoTaWV and NbMoTaWTi than in NbMoTaW.
- A stricter stress test than the paper reports is to train the model on one refractory alloy and predict another; success would indicate the pair coefficients capture transferable ordering physics, while failure would mean they encode chemistry-specific fits.
- The paper stops short of propagating the Bayesian parameter uncertainties into Monte Carlo free energies; doing so would reveal whether a 0.6 meV energy error is small enough for accurate order-disorder transition temperatures.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayesian regularized regression framework, combined with an effective pair interaction (EPI) model and Bayesian information criterion (BIC) based feature selection, to predict the configurational energy of refractory high entropy alloys from sparse first-principles data. The method is demonstrated on NbMoTaW, NbMoTaWV, and NbMoTaWTi, using DFT energies from supercells of 16--128 or 20--160 atoms, with training data drawn from the three smaller supercells and testing on the largest supercell. The reported held-out test RMSE is around 0.6 meV for all three alloys. The paper also introduces an ensemble sampling strategy that combines configurations from different supercell sizes, analyzes the uncertainty and correlations of the fitted pair interaction parameters, and shows that BIC-based selection of the number of coordination shells reduces the risk of overfitting and underfitting when data are limited.
Significance. If the claims hold, the paper offers a practical recipe for constructing surrogate Hamiltonians for multicomponent alloys with quantified parameter uncertainty, which is valuable because conventional cluster expansion becomes combinatorially intractable for high entropy alloys. The work is concrete: the algorithm steps are clearly enumerated, the DFT data generation is described, and the held-out test evaluation gives a quantitative accuracy statement. The explicit use of BIC for shell truncation and the comparison of Bayesian regression against ordinary least squares are useful methodological contributions. The main limitation is that the validation is restricted to random configurations, so the stated suitability for Monte Carlo simulations of order-disorder transitions is not directly demonstrated.
major comments (2)
- [§2.2, Eq. (8) and §3.1, ensemble sampling] The regression model in Eq. (8) assumes that a single set of pair interaction coefficients J_m^{X,Y} describes the configurational energy across all supercell sizes, but the training data combine DFT energies from supercells of 16, 32, 64, and 128 atoms (or 20, 40, 80, and 160 atoms). If the per-atom DFT energies carry any supercell-size-dependent reference offset, for example from different Brillouin-zone sampling or from the definition of the configurational energy itself, then the fitted coefficients and the BIC-selected number of shells would be biased. The manuscript does not discuss this possibility or include a size-dependent offset term in the model. The authors should either add a supercell-size indicator or intercept in the regression, or demonstrate explicitly that no such offset is present for these materials and this DFT setup.
- [§3.1 and Step 6 (Section 2.5)] All training and testing configurations are described as randomly drawn, and the reported RMSE values therefore characterize prediction accuracy only for configurations near random disorder. The paper's stated purpose, however, is to feed the fitted Hamiltonian into Monte Carlo simulations for modeling thermodynamics and order-disorder transitions, where the sampled configurations develop short-range order and may include ordered states. The EPI model of Eq. (8) contains pair interactions only, and the paper does not validate the model on ordered or strongly short-range-ordered configurations. The central claim of robustness for thermodynamic applications is thus an extrapolation beyond the tested distribution. I recommend adding validation on ordered supercells or on configurations with strong SRO, or explicitly limiting the accuracy claim to random configurations.
minor comments (5)
- [§2.2, Eq. (9)] Equation (9) writes E = JP + ε with no constant term, while Eq. (5) contains a concentration-dependent constant J0 that is said to be discarded. For absolute DFT energies, the regression must include an intercept or the energies must be centered; the manuscript should state which convention is used.
- [§3.3, Figures 12--14] The text refers to 'RMSE results' for different shell numbers but does not explicitly state whether these RMSEs are computed on the held-out test supercell or on training data. Please clarify that the comparison in Figures 12--14 uses the same held-out test set as Table 1, and that BIC selection is performed using only training data.
- [Figure 8 caption] The caption of Figure 8 reads 'NbMoTaTi' but the text and Table 1 refer to NbMoTaWTi; please correct the caption.
- [§2.3, Eq. (14)] In Eq. (14), the notation N(E|JP, λ2) implies λ2 is the variance, but later in Eq. (16) λ2 is treated as a gamma-distributed precision parameter. Please make the variance/precision convention consistent.
- [§3.1, Table 1] The column headers 'TrainingεR (meV)' and 'TestingεR (meV)' are missing spaces; consider formatting as 'Training εR (meV)' for readability.
Circularity Check
No construction-level circularity: the reported predictions are held-out, and the self-citations are to methodology rather than to the claimed result.
full rationale
The central result, the configurational-energy surrogate with testing RMSE near 0.6 meV, is obtained by fitting the effective pair interaction model, Eq. (8), to DFT energies from smaller supercells and then evaluating on 200 held-out configurations from the largest supercell (Section 3.1, Table 1). This is a genuine out-of-sample test, not a re-statement of the training fit. The BIC feature selection in Section 2.4 also operates on the training data, and the subsequent comparisons among fixed shell numbers and the BIC-selected m are evaluated on the same held-out set, so the claimed advantage of feature selection is not forced by construction. The EPI model and several Bayesian/Latin-hypercube tools are drawn from the authors' prior work (refs [49]-[52], [54]), and the EPI form is introduced as an ansatz in Section 2.2; however, the paper does not rely on those citations to supply the tested predictions, and the cited methods have independent standing. The skeptical concern that validation uses only randomly drawn configurations while Monte Carlo use requires accuracy on ordered or short-range-ordered states is a generalizability and model-form risk, not a circularity of the derivation. No equation reduces to its own input, no fitted parameter is renamed as a prediction, and no load-bearing uniqueness claim is imported from the authors' own work. Thus the paper shows no significant circularity; at most there is a minor self-citation to the EPI modeling framework that is not load-bearing for the out-of-sample accuracy claim.
Assumptions & free parameters
free parameters (3)
- Gamma prior hyperparameters alpha1, alpha2, beta1, beta2 =
1e-8 for all four
- Coordination shell truncation m for the main EPI fit =
6 for the main results; up to 13 considered in BIC comparisons
- RMSE acceptance threshold epsilon_bar =
1 meV
assumptions (4)
- domain assumption Configurational energy is a linear function of the pair probabilities P^{X|Y}_m (Eq. 8) with the same effective pair interactions across supercell sizes.
- domain assumption DFT (LSMS) total energies are the ground truth and errors are i.i.d. Gaussian, epsilon ~ N(0, sigma^2).
- standard math BIC with the Laplace approximation is an adequate approximation to the model evidence for selecting coordination shells.
- standard math The gamma priors with alpha1=alpha2=beta1=beta2=1e-8 are non-informative and lead to a proper posterior.
Cite this review
Pith. "Pith review of Robust data-driven approach for predicting the configurational energy of high entropy alloys." pith.science (2026). https://pith.science/paper/NHR3DDCZ
@misc{pith2026190803665,
author = {Pith},
title = {Pith review of: Robust data-driven approach for predicting the configurational energy of high entropy alloys},
year = {2026},
howpublished = {\url{https://pith.science/paper/NHR3DDCZ}},
note = {Machine review of arXiv:1908.03665}
}
read the original abstract
High entropy alloys (HEAs) have been increasingly attractive as promising next-generation materials due to their various excellent properties. It's necessary to essentially characterize the degree of chemical ordering and identify order-disorder transitions through efficient simulation and modeling of thermodynamics. In this study, a robust data-driven framework based on Bayesian approaches is proposed and demonstrated on the accurate and efficient prediction of configurational energy of high entropy alloys. The proposed effective pair interaction (EPI) model with ensemble sampling is used to map the configuration and its corresponding energy. Given limited data calculated by first-principles calculations, Bayesian regularized regression not only offers an accurate and stable prediction but also effectively quantifies the uncertainties associated with EPI parameters. Compared with the arbitrary determination of model complexity, we further conduct a physical feature selection to identify the truncation of coordination shells in EPI model using Bayesian information criterion. The results achieve efficient and robust performance in predicting the configurational energy, particularly given small data. The developed methodology is applied to study a series of refractory HEAs, i.e. NbMoTaW, NbMoTaWV and NbMoTaWTi where it is demonstrated how dataset size affects the confidence we can place in statistical estimates of configurational energy when data are sparse.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
O. N. Senkov, G. Wilks, J. Scott, D. B. Miracle, Mechanical properties of nb25mo25ta25w25 and v20nb20mo20ta20w20 refractory high entropy alloys, Inter- metallics 19 (2011) 698–706
work page 2011
- [3]
-
[4]
Z. Li, K. G. Pradeep, Y. Deng, D. Raabe, C. C. Tasan, Metastable high-entropy dual- phase alloys overcome the strength–ductility trade-off, Nature 534 (2016) 227
2016
-
[5]
Tsai, J.-W
M.-H. Tsai, J.-W. Yeh, High-entropy alloys: a critical review, Materials Research Letters 2 (2014) 107–123
2014
-
[6]
D. B. Miracle, O. N. Senkov, A critical review of high entropy alloys and related concepts, Acta Materialia 122 (2017) 448–511
2017
-
[7]
M. C. Gao, J.-W. Yeh, P. K. Liaw, Y. Zhang, High-entropy alloys: fundamentals and applications, Springer, 2016
work page 2016
- [8]
Show all 55 references
-
[9]
B. S. Murty, J.-W. Yeh, S. Ranganathan, P. Bhattacharjee, High-entropy alloys, Else- vier, 2019. 24
2019
-
[10]
Eisenbach, Z
M. Eisenbach, Z. Pei, X. Liu, First-principles study of order-disorder transitions in multicomponent solid-solution alloys, Journal of Physics: Condensed Matter 31 (2019) 273002
2019
-
[11]
Widom, Modeling the structure and thermodynamics of high-entropy alloys, Journal of Materials Research 33 (2018) 2881–2898
M. Widom, Modeling the structure and thermodynamics of high-entropy alloys, Journal of Materials Research 33 (2018) 2881–2898
2018
-
[12]
M. C. Gao, P. Gao, J. A. Hawk, L. Ouyang, D. E. Alman, M. Widom, Computational modeling of high-entropy alloys: Structures, thermodynamics and elasticity, Journal of Materials Research 32 (2017) 3627–3641
2017
-
[13]
Ikeda, B
Y. Ikeda, B. Grabowski, F. K¨ ormann, Ab initio phase stabilities and mechanical prop- erties of multicomponent alloys: A comprehensive review for high entropy alloys and compositionally complex alloys, Materials Characterization 147 (2019) 464–511
2019
-
[14]
Y. Ye, Q. Wang, J. Lu, C. Liu, Y. Yang, High-entropy alloy: challenges and prospects, Materials Today 19 (2016) 349–362
2016
-
[15]
D. Ma, B. Grabowski, F. K¨ ormann, J. Neugebauer, D. Raabe, Ab initio thermodynamics of the cocrfemnni high entropy alloy: Importance of entropy contributions beyond the configurational one, Acta Materialia 100 (2015) 90–97
2015
-
[16]
Huang, A
W. Huang, A. Urban, Z. Rong, Z. Ding, C. Luo, G. Ceder, Construction of ground-state preserving sparse lattice models for predictive materials simulations, npj Computational Materials 3 (2017) 30
2017
-
[17]
S. N. Khan, M. Eisenbach, Density-functional monte-carlo simulation of cuzn order- disorder transition, Physical Review B 93 (2016) 024203
2016
-
[18]
Kikuchi, A theory of cooperative phenomena, Physical review 81 (1951) 988
R. Kikuchi, A theory of cooperative phenomena, Physical review 81 (1951) 988
1951
-
[19]
J. M. Sanchez, F. Ducastelle, D. Gratias, Generalized cluster description of multi- component systems, Physica A: Statistical Mechanics and its Applications 128 (1984) 334–350
1984
-
[20]
van de Walle, G
A. van de Walle, G. Ceder, Automating first-principles phase diagram calculations, Journal of Phase Equilibria 23 (2002) 348
2002
-
[21]
L. J. Nelson, G. L. Hart, F. Zhou, V. Ozoli¸ nˇ s, et al., Compressive sensing as a paradigm for building physics models, Physical Review B 87 (2013) 035125. 25
2013
-
[22]
Mueller, G
T. Mueller, G. Ceder, Bayesian approach to cluster expansions, Physical Review B 80 (2009) 024103
2009
-
[23]
V. Blum, G. L. Hart, M. J. Walorski, A. Zunger, Using genetic algorithms to map first- principles results to model hamiltonians: Application to the generalized ising model for alloys, Physical Review B 72 (2005) 165113
2005
-
[24]
G. L. Hart, V. Blum, M. J. Walorski, A. Zunger, Evolutionary approach for determining first-principles hamiltonians, Nature materials 4 (2005) 391
2005
-
[25]
A. Seko, Y. Koyama, I. Tanaka, Cluster expansion method for multicomponent sys- tems based on optimal selection of structures for density-functional theory calculations, Physical Review B 80 (2009) 165122
2009
-
[26]
A. R. Natarajan, A. Van der Ven, Machine-learning the configurational energy of multicomponent crystalline solids, npj Computational Materials 4 (2018) 56
2018
-
[27]
J. H. Chang, D. Kleiven, M. Melander, J. Akola, J. M. Garcia-Lastra, T. Vegge, Clease: A versatile and user-friendly implementation of cluster expansion method, Journal of Physics: Condensed Matter 31 (2019) 325901
2019
-
[28]
˚Angqvist, W
M. ˚Angqvist, W. A. Mu˜ noz, J. M. Rahm, E. Fransson, C. Durniak, P. Rozyczko, T. H. Rod, P. Erhart, Icet–a python library for constructing and sampling alloy cluster ex- pansions, Advanced Theory and Simulations (2019) 1900015
2019
-
[29]
Jiang, B
C. Jiang, B. P. Uberuaga, Efficient ab initio modeling of random multicomponent alloys, Physical review letters 116 (2016) 105501
2016
-
[30]
M. I. Jordan, T. M. Mitchell, Machine learning: Trends, perspectives, and prospects, Science 349 (2015) 255–260
2015
-
[31]
LeCun, Y
Y. LeCun, Y. Bengio, G. Hinton, Deep learning, nature 521 (2015) 436
2015
-
[32]
K. T. Butler, D. W. Davies, H. Cartwright, O. Isayev, A. Walsh, Machine learning for molecular and materials science, Nature 559 (2018) 547
2018
-
[33]
Mueller, A
T. Mueller, A. G. Kusne, R. Ramprasad, Machine learning in materials science: Recent progress and emerging applications, Reviews in Computational Chemistry 29 (2016) 186–273
2016
-
[34]
Sanchez-Lengeling, A
B. Sanchez-Lengeling, A. Aspuru-Guzik, Inverse molecular design using machine learn- ing: Generative models for matter engineering, Science 361 (2018) 360–365. 26
2018
-
[35]
C. Kim, G. Pilania, R. Ramprasad, From organized high-throughput data to phe- nomenological theory using machine learning: the example of dielectric breakdown, Chemistry of Materials 28 (2016) 1304–1311
2016
-
[36]
Raccuglia, K
P. Raccuglia, K. C. Elbert, P. D. Adler, C. Falk, M. B. Wenny, A. Mollo, M. Zeller, S. A. Friedler, J. Schrier, A. J. Norquist, Machine-learning-assisted materials discovery using failed experiments, Nature 533 (2016) 73
2016
-
[37]
Carrasquilla, R
J. Carrasquilla, R. G. Melko, Machine learning phases of matter, Nature Physics 13 (2017) 431
2017
-
[38]
Huang, P
W. Huang, P. Martin, H. L. Zhuang, Machine-learning phase prediction of high-entropy alloys, Acta Materialia 169 (2019) 225–236
2019
-
[39]
Kostiuchenko, F
T. Kostiuchenko, F. K¨ ormann, J. Neugebauer, A. Shapeev, Impact of lattice relaxations on phase transitions in a high-entropy alloy studied by machine-learning potentials, npj Computational Materials 5 (2019) 55
2019
-
[40]
Fujimura, A
K. Fujimura, A. Seko, Y. Koyama, A. Kuwabara, I. Kishida, K. Shitara, C. A. Fisher, H. Moriwake, I. Tanaka, Accelerated materials design of lithium superionic conductors based on first-principles calculations and machine learning algorithms, Advanced Energy Materials 3 (2013) 980–985
2013
-
[41]
Pilania, C
G. Pilania, C. Wang, X. Jiang, S. Rajasekaran, R. Ramprasad, Accelerating materials property predictions using machine learning, Scientific reports 3 (2013) 2810
2013
-
[42]
L. Ward, A. Agrawal, A. Choudhary, C. Wolverton, A general-purpose machine learning framework for predicting properties of inorganic materials, npj Computational Materials 2 (2016) 16028
2016
-
[43]
Dragoni, T
D. Dragoni, T. D. Daff, G. Cs´ anyi, N. Marzari, Achieving dft accuracy with a machine- learning interatomic potential: Thermomechanics and defects in bcc ferromagnetic iron, Physical Review Materials 2 (2018) 013808
2018
-
[44]
Chmiela, A
S. Chmiela, A. Tkatchenko, H. E. Sauceda, I. Poltavsky, K. T. Sch¨ utt, K.-R. M¨ uller, Machine learning of accurate energy-conserving molecular force fields, Science advances 3 (2017) e1603015
2017
-
[45]
V. L. Deringer, G. Cs´ anyi, Machine learning based interatomic potential for amorphous carbon, Physical Review B 95 (2017) 094203. 27
2017
-
[46]
Z. Li, J. R. Kermode, A. De Vita, Molecular dynamics with on-the-fly machine learning of quantum-mechanical forces, Physical review letters 114 (2015) 096405
2015
-
[47]
Aldegunde, N
M. Aldegunde, N. Zabaras, J. Kristensen, Quantifying uncertainties in first-principles alloy thermodynamics using cluster expansions, Journal of Computational Physics 323 (2016) 17–44
2016
-
[48]
Kristensen, N
J. Kristensen, N. J. Zabaras, Bayesian uncertainty quantification in the evaluation of alloy properties with the cluster expansion method, Computer Physics Communications 185 (2014) 2885–2892
2014
-
[49]
Zhang, M
J. Zhang, M. D. Shields, On the quantification and efficient propagation of imprecise probabilities resulting from small datasets, Mechanical Systems and Signal Processing 98 (2018) 465–483
2018
-
[50]
X. Liu, J. Zhang, M. Eisenbach, Y. Wang, Machine learning modeling of high entropy alloy: the role of short-range order, arXiv preprint arXiv:1906.02889 (2019)
2019 arXiv
-
[51]
Zhang, M
J. Zhang, M. D. Shields, Efficient monte carlo resampling for probability measure changes from bayesian updating, Probabilistic Engineering Mechanics 55 (2019) 54–66
2019
-
[52]
Zhang, M
J. Zhang, M. D. Shields, The effect of prior probabilities on quantification and propa- gation of imprecise probabilities resulting from small datasets, Computer Methods in Applied Mechanics and Engineering 334 (2018) 483–506
2018
-
[53]
Y. Wang, G. Stocks, W. Shelton, D. Nicholson, Z. Szotek, W. Temmerman, Order-n multiple scattering approach to electronic structure calculations, Physical review letters 75 (1995) 2867
1995
-
[54]
M. D. Shields, J. Zhang, The generalization of latin hypercube sampling, Reliability Engineering & System Safety 148 (2016) 96–108
2016
-
[55]
Mehta, M
P. Mehta, M. Bukov, C.-H. Wang, A. G. Day, C. Richardson, C. K. Fisher, D. J. Schwab, A high-bias, low-variance introduction to machine learning for physicists, Physics Reports (2019). 28
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.