REVIEW 4 major objections 5 minor 26 references
Amending CALPHAD databases using a neural network for predicting mixing enthalpy of liquids
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read A neural network trained on 91 known binary liquid alloys can predict the mixing enthalpy of unseen binary systems with estimated uncertainty below 1 kJ/mol, and the resulting Redlich-Kister parameters can be written directly into CALPHAD…
desk verdict Useful ML pipeline for filling missing binary liquid Hmix in steel CALPHAD databases, with honest LOOCV, but the <1 kJ/mol uncertainty claim for extrapolated systems is asserted rather than demonstrated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a neural network regression model mapping a descriptor vector of a binary liquid at a given composition to its mixing enthalpy $H_{\mathrm{mix}}$. The descriptors combine 151 atomic features, 181 composition-based features, plus Miedema and Yang alloy features extracted with Matminer, chosen to encode electronegativity differences, orbital overlaps, and semi-empirical enthalpic trends. The model is trained at a fixed temperature, 1873 K, on 2000 equidistant composition points per binary system extracted from the MatCalc open database with pycalphad, augmented by experimental enthalpy data. Validation uses 10-fold cross-validation and leave-one-system-out cross-validation to test genuine extrapolation; the final predictions are fitted to a fourth-order Redlich-Kister polynomial $H_{\mathrm{mix}} = x_1 x_2 \sum_{k=1}^{4} L_k (x_1 - x_2)^{k-1}$, whose parameters are tabulated for direct insertion into a CALPHAD database file.
What would settle it
Measure the mixing enthalpy of a predicted-but-unseen binary system such as Al-Sn or Al-Sb by high-temperature calorimetry at 1873 K and compare the curve to the model's prediction; if the deviation exceeds the claimed 1 kJ/mol uncertainty at any composition, the central uncertainty claim is falsified for that system.
Extended reading notes
Core claim
On its own terms, the paper claims that a feedforward neural network, trained on 91 binary liquid-phase systems from an open steel database supplemented with experimental mixing enthalpies, learns composition-dependent interaction behavior well enough to predict mixing enthalpy curves for binary systems it has never seen. The key demonstration is leave-one-out validation: when a whole binary system such as Fe-Cu is quarantined, the network reproduces the CALPHAD curve, including demixing, as long as some other binary containing each element (Fe and Cu) remains in training. The two failures, Fe-Sn and Fe-Sb, occur precisely when no other Sn- or Sb-containing system is available. The paper further reports that a model trained on all data achieves a mean absolute error of 16.52 J/mol on held-out points and R²=0.99, and that uncertainty from removing a second correlated system stays below about 0.5 kJ/mol, leading to the claim that extrapolated predictions for the 140 missing binaries carry errors under 1 kJ/mol.
Load-bearing premise
The central claim rests on the premise that interaction knowledge from binaries containing an element transfers to unseen binaries with that element—a premise the paper shows fails when a quarantined element disappears from training (Fe-Sn and Fe-Sb), so the blanket 1 kJ/mol claim for all 140 missing binaries assumes this transfer works for every element, including sparse As, Sn, and Sb.
Editorial extensions
If this is right
- All 140 missing binary liquid systems among the 22 considered elements become available as fitted Redlich-Kister parameters, so a CALPHAD database can be amended without new calorimetric measurements.
- Systems relevant to scrap recycling, such as Fe-Sn, Fe-Sb, Fe-As, Al-Sn, and Al-Sb, receive a first thermodynamic description at 1873 K, useful for initial phase-stability screening.
- The same workflow can be extended to temperature-dependent training data, since the constant-temperature restriction is presented as a proof of concept rather than a fundamental limit.
- The predicted parameters are formatted to be read directly into existing thermodynamic database files, making the output usable by standard CALPHAD software.
- If the uncertainty claim holds, the model provides quantitatively reliable interpolation across the binary composition matrix, complementing prior work on ternary extrapolation from known binaries.
Reading between the lines
- The paper's own LOOCV failures for Fe-Sn and Fe-Sb imply that the 140 'missing' predictions should be stratified by how many correlated binaries exist for the involved elements; binaries pairing two sparse elements (e.g., As-Sn, As-Sb, Sn-Sb) are the least supported, and a calorimetric spot-check on one of them would be the sharpest test of the 1 kJ/mol claim.
- Because the descriptor set includes Miedema and Yang features that already encode semi-empirical estimates of mixing enthalpy, part of the apparent predictive accuracy may be inherited from those physics-informed features rather than learned from the thermodynamic database; ablating those features and retraining would separate the two contributions.
- The uncertainty-quantification procedure suggests a natural active-learning loop: remove the binary whose removal most degrades predictions for a target system, run an experiment there, add it to the training set, and repeat to build a complete 22-element matrix with minimal calorimetric effort.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a neural-network model trained on CALPHAD-derived and experimental liquid-phase mixing-enthalpy data for 91 binary systems of 22 elements at 1873 K, using Matminer elemental and composition features. The authors report a 10-fold cross-validated MAE of 16.52 J/mol, a leave-one-system-out test on Fe-X binaries, and an uncertainty analysis that removes a primary and one correlated secondary system. They then use the model to predict Hmix for all missing binaries, extract Redlich-Kister parameters, and claim that the uncertainty of the predicted mixing enthalpy is below 1 kJ/mol, so that the parameters can be reintegrated into a CALPHAD database.
Significance. If the extrapolation claim were fully supported, this would be a valuable tool for augmenting CALPHAD databases with first-pass descriptions of unassessed binary systems, especially for tramp elements relevant to steel recycling. The paper has genuine strengths: the LOOCV protocol is the right kind of test for unseen systems; the authors openly report the Fe-Sn and Fe-Sb failures; the data and code are shared through a Zenodo repository; and the Redlich-Kister extraction addresses practical use by the CALPHAD community. However, the evidence as presented supports a more limited claim than the abstract makes: reliable extrapolation is demonstrated only for systems whose elements appear elsewhere in the training set, and the sub-1 kJ/mol uncertainty bound is asserted for systems that were not tested against any held-out data.
major comments (4)
- [Sec. 3.1 and Fig. 3] The headline MAE of 16.52 J/mol is reported for the best of 10 folds in a point-level k-fold split, so every binary system appears in the training set. This metric cannot support the abstract's claim that the model predicts mixing enthalpy for systems not present in the training dataset. The paper should either report metrics from the LOOCV as the primary out-of-system measure or clearly label the k-fold result as an interpolation test.
- [Sec. 3.2 and Sec. 3.4, Fig. 7] The LOOCV is performed only on Fe-X systems, and it fails outright for Fe-Sn and Fe-Sb because no other Sn- or Sb-containing system remains in training. Yet the final deliverable includes missing binaries such as Al-Sn and Al-Sb, where the only training information about Sn or Sb is the single Fe-Sn or Fe-Sb system. The paper does not provide a held-out test for this regime, so the transferability that is load-bearing for those predictions is not demonstrated. At minimum, the abstract and Sec. 3.4 should be reworded to state that reliable extrapolation is conditional on the element being represented elsewhere in the training set.
- [Sec. 3.3 and abstract] The uncertainty quantification removes a primary and one correlated secondary system and compares the resulting predictions with CALPHAD values from the same database family used for training. This measures internal consistency of the database, not physical accuracy, and it is shown only for Cu-based systems with many Fe-Z training systems. The abstract's statement that 'the estimated uncertainty of the model is below 1 kJ/mol for the predicted mixing enthalpy' is therefore not supported for the genuinely extrapolated predictions. The paper should either provide a validated error estimate for missing systems (e.g., against experimental data for at least a few held-out systems) or replace the uncertainty claim with a clearly labeled consistency measure.
- [Sec. 3.4, final paragraph] The authors concede that As-X, Sn-X, and Sb-X systems 'seem to perform relatively worse' and then state 'we believe' that extrapolated errors will be below 1 kJ/mol. This is an explicit admission that the central uncertainty claim is not demonstrated for the systems that matter most for the proposed application. The manuscript should either provide quantitative evidence for the error on such systems or remove the unqualified sub-1 kJ/mol claim from the abstract and summary.
minor comments (5)
- [Abstract] The phrase 'amended with several direct experimental reports' is vague; the paper should specify which systems and how many experimental data points were added.
- [Eq. (1)] The Redlich-Kister expansion is written with indices k=1...n, whereas the CALPHAD convention typically starts at k=0; this may confuse readers comparing with standard database files.
- [Fig. 6 caption] The caption abbreviates RMSE as 'RSME'; please correct the typo.
- [Sec. 2.1 and Fig. 1] The heatmap uses green and red fields, but the caption does not state the color convention explicitly; please add a legend or explicit sentence.
- [Sec. 2.4 and Table 1] The precision of the tabulated RK parameters is much higher than the claimed physical accuracy; the footnote is helpful but could be strengthened by adding an explicit statement of the implied uncertainty in the parameter values.
Circularity Check
No significant circularity: the model is trained on external CALPHAD/experimental Hmix data and tested on genuinely held-out systems, while the weaker 1 kJ/mol extrapolated-uncertainty claim is an explicit belief, not a circular derivation.
full rationale
The paper's derivation chain is: extract Hmix values from an external open CALPHAD database plus literature experiments, featurize with Matminer element- and composition-based descriptors, train a neural network to predict Hmix, then fit Redlich-Kister polynomials to the NN output. The target quantity for unseen binaries is not used to construct the features (Miedema and Yang descriptors are derived from elemental properties, not from the target Hmix values), and for quarantined systems the target values are withheld during training. The LOOCV in Sec. 3.2 is a genuine held-out-system test, and its failures for Fe-Sn and Fe-Sb, explicitly attributed to the disappearance of Sn/Sb from the training set, show the test is not vacuous. The uncertainty quantification in Sec. 3.3 compares predictions with CALPHAD values from the same database family used for training, so it measures internal consistency rather than physical accuracy; however, the paper does not present this comparison as a formal derivation of the extrapolated error bound. In Sec. 3.4 the authors explicitly say 'we believe that even for the extrapolated predictions, the difference in error between the true mixing enthalpy and the predicted mixing enthalpy should be less than 1 kJ/mol', and they also flag that As-X, Sn-X, and Sb-X systems 'seem to perform relatively worse'. That is an honest limitation and a correctness/robustness risk, not a circular step. Self-citations such as the MatCalc database [10] are data sources, not load-bearing uniqueness arguments, and no equation is defined in terms of the quantity it claims to predict; no fitted parameter is renamed as a prediction. Therefore no definitional or constructional circularity is present.
Assumptions & free parameters
free parameters (4)
- NN architecture hyperparameters =
not stated in text; in Supplementary Table S1
- Redlich-Kister order n =
4
- Temperature =
1873 K
- Feature set =
332 Matminer descriptors
assumptions (5)
- domain assumption The 91 CALPHAD binary assessments used as ground truth are accurate enough for training and validation.
- domain assumption Element-pair interaction knowledge transfers across binaries containing a common element.
- domain assumption The 332 Matminer features, including Miedema and Yang descriptors, capture the physical determinants of liquid mixing enthalpy.
- domain assumption A fourth-order Redlich-Kister polynomial adequately represents Hmix(x) for every predicted binary system.
- standard math Statistical learning assumptions such as independence and smoothness hold for the extracted data points.
Cite this review
Pith. "Pith review of Amending CALPHAD databases using a neural network for predicting mixing enthalpy of liquids." pith.science (2026). https://pith.science/paper/WWZWAOFV
@misc{pith2026250418366,
author = {Pith},
title = {Pith review of: Amending CALPHAD databases using a neural network for predicting mixing enthalpy of liquids},
year = {2026},
howpublished = {\url{https://pith.science/paper/WWZWAOFV}},
note = {Machine review of arXiv:2504.18366}
}
read the original abstract
In order to establish the thermodynamic stability of a system, knowledge of its Gibbs free energy is essential. Most often, the Gibbs free energy is predicted within the CALPHAD framework using models employing thermodynamic properties, such as the mixing enthalpy, heat capacity, and activity coefficients. Here, we present a deep-learning approach capable of predicting the mixing enthalpy of liquid phases of binary systems that were not present in the training dataset. Therefore, our model allows for a system-informed enhancement of the thermodynamic description to unknown binary systems based on information present in the available thermodynamic assessment. Thereby, significant experimental efforts in assessing new systems can be spared. We use an open database for steels containing 91 binary systems to generate our initial training (and validation) and amend it with several direct experimental reports. The model is thoroughly tested using different strategies, including a test of its predictive capabilities. The model shows excellent predictive capabilities outside of the training dataset as soon as some data containing species of the predicted system is included in the training dataset. The estimated uncertainty of the model is below 1 kJ/mol for the predicted mixing enthalpy. Subsequently, we used our model to predict the enthalpy of mixing of all binary systems not present in the original database and extracted the Redlich-Kister parameters, which can be readily reintegrated into the thermodynamic database file.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
H. L. Lukas, S. G. Fries, B. Sundman, Computational Thermodynamics: The CALPHAD Method, Cambridge University Press, 2007
work page 2007
-
[2]
U. Thiedemann, M. R¨ osner-Kuhn, D. M. Matson, G. Kuppermann, K. Drewes, M. C. Flemings, M. G. Frohberg, Mixing enthalpy measurements in the liquid ternary system iron-nickel-chromium and its binaries, Steel Research 69 (1) (1998) 3–7. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10. 1002/srin.199801599, doi:https://doi.org/10.1002/srin.199801599. URL ht...
-
[3]
M. Mezbahul-Islam, A. Mostafa, M. Medraj, Essential magnesium alloys binary phase diagrams, Journal of MaterialsIn Press (2014). URL http://www.hindawi.com/journals/jma/aip/704283/
work page 2014
-
[4]
P. Laiu, Y. Yang, M. Lupo Pasini, J. Y. Choi, D. Shin, A neural network approach to predict gibbs free energy of ternary solid solutions, Journal of Phase Equilibria and Diffusion 43 (6) (2022) 916–930. doi:10.1007/s11669-022-01010-2 . URL https://doi.org/10.1007/s11669-022-01010-2
-
[5]
G. Deffrennes, B. Hallstedt, T. Abe, Q. Bizot, E. Fischer, J.-M. Joubert, K. Terayama, R. Tamura, Data-driven study of the enthalpy of mixing in the liquid phase, Calphad 87 (2024) 102745. doi: https://doi.org/10.1016/j.calphad.2024.102745. URL https://www.sciencedirect.com/science/article/pii/S0364591624000877
arXiv 2024
- [6]
-
[7]
S. Dworak, J. Fellner, M. Beermann, M. H¨ auselmann, J. Schenk, S. Michelic, J. Cejka, A. Sakic, J. Mayer, K. Steininger, Stahlrecycling–potenziale und herausforderungen f¨ ur innovatives und nach- haltiges recycling, ¨Osterreichische Wasser-und Abfallwirtschaft 75 (1) (2023) 97–107
work page 2023
-
[8]
I. Dumitrescu, S. Seetharaman, M. Selleby, The calphad method as a tool to assess the thermodynamic properties of high-entropy alloys, Journal of Alloys and Compounds 654 (2016) 549–560. 13
work page 2016
Show all 26 references
-
[9]
E. F. Koch, B. Predel, Database on thermodynamic properties of alloys and compounds, Journal of Phase Equilibria 13 (4) (1992) 399–409
1992
-
[10]
Povoden-Karadeniz, Open databases for MatCalc, https://www.matcalc.at/index.php/ databases/open-databases, [Online; accessed 11-Dec-2024] (2024)
E. Povoden-Karadeniz, Open databases for MatCalc, https://www.matcalc.at/index.php/ databases/open-databases, [Online; accessed 11-Dec-2024] (2024)
2024
-
[11]
R. A. Otis, Z.-K. Liu, Pycalphad: Calphad-based computational tools for thermodynamic modeling, Journal of Open Research Software 5 (1) (2017) 1–8
2017
-
[12]
L. A. Dreval, P. G. Agraval, M. A. Turchanin, Enthalpy of mixing of liquid cu-fe-zr alloys at 1873 k (1600 °c), Metallurgical and Materials Transactions B 46 (5) (2015) 2234–2245. doi: 10.1007/s11663-015-0383-1 . URL https://doi.org/10.1007/s11663-015-0383-1
2015 doi
-
[13]
Tanaka, N
T. Tanaka, N. A. Gokcen, Z. Morita, Relationship between enthalpy of mixing and excess entropy in liquid binary alloys/ zusammenhang zwischen mischungsenthalpien und¨Uberschußentropien in fl¨ ussigen bin¨ aren legierungen, International Journal of Materials Research 81 (1) (19...
1990 doi
-
[14]
M. A. Turchanin, P. G. Agraval, I. V. Nikolaenko, Thermodynamics of alloys and phase equilib- ria in the copper-iron system, Journal of Phase Equilibria 24 (4) (2003) 307–319. doi:10.1361/ 105497103770330280. URL https://doi.org/10.1361/105497103770330280
2003 doi
-
[15]
Iguchi, Y
Y. Iguchi, Y. Tozaki, M. Kakizaki, T. Fuwa, S. Ban-ya, A calorimetric study of heats of mixing of liquid iron alloys, Tetsu To Hagane-journal of The Iron and Steel Institute of Japan 67 (7) (1981) 925–932. doi:10.2355/tetsutohagane1955.67.7_925
1981 doi
-
[16]
Vincely, Supplementary data: Amending calphad databases using a neural network for predicting mixing enthalpy of liquids (Apr
C. Vincely, Supplementary data: Amending calphad databases using a neural network for predicting mixing enthalpy of liquids (Apr. 2025). doi:10.5281/zenodo.15194567. URL https://doi.org/10.5281/zenodo.15194567
2025 doi
-
[17]
L. Ward, A. Dunn, A. Faghaninia, N. E. R. Zimmermann, S. Bajaj, Q. Wang, J. Montoya, J. Chen, K. Bystrom, M. Dylla, K. Chard, I. Foster, A. Jain, K. A. Persson, C. Wolverton, Matminer: An open source toolkit for materials data mining, Computational Materials Science 152 (2018) 60–69
2018
-
[18]
D. Jha, L. Ward, A. Paul, W.-K. Liao, C. Wolverton, A. Choudhary, A. Agrawal, Elemnet: Deep learning the chemistry of materials from only elemental composition, Scientific Reports 8 (1) (2018) 17593
2018
-
[19]
Y. Liu, T. Zhao, W. Ju, S. Shi, Materials discovery and design using machine learning, Journal of Materiomics 3 (3) (2017) 159–177
2017
-
[20]
Pedregosa, G
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, Scikit-learn: Machine learning in Python, Journal of Machine Learning Res...
2011
-
[21]
Abadi, A
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man´ e, R. Monga, S. Moore, D. Murray, C. Olah, M....
2015
-
[22]
Chollet, et al., Keras (2015)
F. Chollet, et al., Keras (2015). URL https://github.com/fchollet/keras
2015
-
[23]
O’Malley, E
T. O’Malley, E. Bursztein, J. Long, F. Chollet, H. Jin, L. Invernizzi, et al., Kerastuner, https:// github.com/keras-team/keras-tuner (2019)
2019
-
[24]
Bergstra, Y
J. Bergstra, Y. Bengio, Random search for hyper-parameter optimization, in: Journal of Machine Learning Research, Vol. 13, 2012, pp. 281–305
2012
-
[25]
Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Sta- tistical Society: Series B (Methodological) 36 (2) (1974) 111–133
M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Sta- tistical Society: Series B (Methodological) 36 (2) (1974) 111–133. 14
1974
-
[26]
L. Ward, A. Agrawal, A. Choudhary, C. Wolverton, A general-purpose machine learning framework for predicting properties of inorganic materials, npj Computational Materials 2 (1) (2016) 16028. 15
2016
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.