Formation Energy Prediction of Material Crystal Structures using Deep Learning

E. A. Fajardo; V. Torlao

REVIEW 3 major objections 2 minor 17 references

Formation Energy Prediction of Material Crystal Structures using Deep Learning

T0 review · 3 major / 2 minor · reviewed 2026-05-23 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗

Pith's one-line read Incorporating space group symmetry as an input feature improves a neural network's accuracy at predicting crystal formation energies from elemental composition.

desk verdict Adding space-group labels boosts reported accuracy on formation-energy prediction, but those labels come from the same DFT-relaxed structures, so the gains may not transfer to new compositions or polymorphs. read the letter →

arxiv 2412.00819 v1 submitted 2024-12-01 cond-mat.mtrl-sci

V. Torlao , E. A. Fajardo This is my paper

classification cond-mat.mtrl-sci

keywords formationenergydeeplearningcrystalstructuresspacegroupsymmetryclassificationmaterialstabilityabovehullneuralnetwork

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds a deep neural network that takes elemental fractions from a material's chemical formula and adds one of three symmetry labels—crystal system, point group, or space group—as an extra input. It shows that any of these labels raises prediction accuracy for formation energy, which signals how stable a crystal is, and that space group gives the largest gain. The same network is then reused to predict energy above hull, again using formation energy as an added input. A reader would care because faster, more accurate stability estimates could speed up the search for new useful materials without running every candidate through expensive quantum calculations. The work treats the symmetry labels as known, independent facts supplied alongside the composition.

What carries the argument

Deep neural network that ingests elemental fractions together with a symmetry classification (crystal system, point group, or space group) to output formation energy.

What would settle it

On a held-out set of crystals whose space groups are known, a version of the network that receives only elemental fractions matches or exceeds the accuracy of the version that also receives space-group labels.

Watch

Extended reading notes

Core claim

A deep neural network that receives elemental fractions plus a symmetry classification predicts a crystal's formation energy more accurately than the same network given only the fractions; performance is highest when the symmetry input is the space group. The identical architecture, with formation energy supplied as an additional feature, also predicts energy above hull.

Load-bearing premise

Symmetry classifications can be supplied as reliable inputs that do not themselves depend on the formation-energy values the model is trying to predict.

Editorial extensions

If this is right

Models that receive space-group labels outperform those that receive only point-group or crystal-system labels.
The same network architecture can be applied to energy-above-hull prediction once formation energy is supplied as an extra input.
Symmetry information acts as an effective proxy for structural details that affect stability.
Predictions become more accurate without changing the underlying network depth or width, only by adding the symmetry feature.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If symmetry labels prove hard to obtain for new compositions, the performance gain would shrink unless an auxiliary model first predicts the label.
The approach could be tested on datasets where formation energies come from experiment rather than computation to check whether the symmetry boost survives real measurement noise.
Because space group encodes more detailed structural constraints than crystal system, it may implicitly capture information about bonding geometry that elemental fractions alone miss.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

Your colleague should know that this paper reports accuracy gains on formation energy prediction by including symmetry classifications as input features, with space group performing best, but the symmetry information is extracted from the same relaxed structures used to compute the target DFT energies. This creates a dependency that limits how much the result can be used for predicting properties of unknown structures. The work takes standard composition-based inputs and adds categorical symmetry labels to a deep neural network. It shows that this addition improves performance on the task, and then extends the architecture to energy above hull by including the formation energy prediction as an input. On the positive side, the empirical result that space group helps more than crystal system or point group is at least a clear observation from their experiments, even if the setup is incremental. The main concern is whether the symmetry labels are truly independent inputs. In typical materials databases, space groups are determined after structure relaxation and energy calculation, so the model is effectively getting post-processing information about the structure. For the use case of screening hypothetical materials, you would need to predict or assume the symmetry without that knowledge. The paper does not seem to address how the model would perform if symmetry had to be inferred separately or if the input was only composition. The chaining for energy above hull also means any error in formation energy carries over, and without details on how the splits were handled, it's unclear if the results are robust. There are no mentions of dataset size, train-test splits, baseline models like simple composition-only networks or other ML methods, or error bars. This makes it difficult to assess if the gains are significant or just from better feature engineering on the same data. This kind of paper is aimed at researchers applying deep learning to materials property prediction who might want to experiment with symmetry features in their pipelines. It could be useful as a quick reference for that specific trick, but it doesn't introduce new methods or resolve open questions in the field. I don't see enough here for a serious referee to spend time on, given the missing validation details and the feature circularity issue. I'd recommend against sending it to peer review in its current state.

Referee Report

3 major / 2 minor

Summary. The manuscript introduces a deep neural network for predicting formation energies of crystal structures by combining elemental composition fractions with symmetry classifications (crystal system, point group, or space group). It reports that space group information provides the largest accuracy improvement. The same architecture is used to predict energy above hull, incorporating the predicted formation energy as an input feature.

Significance. If the symmetry features prove independent of the target energies and the performance gains hold under proper cross-validation, the approach could contribute to faster screening of material stability. The use of symmetry as a feature is a reasonable idea in principle, but the manuscript must demonstrate that the reported gains are not artifacts of data leakage from post-relaxation symmetry labels.

major comments (3)

[Abstract] Abstract: The claim that symmetry integration 'significantly enhances' performance lacks supporting details on dataset size, train/test splits, baseline comparisons, error bars, or cross-validation, making it impossible to evaluate the magnitude or robustness of the improvement.
[Methods (symmetry input)] Methods (symmetry input): Symmetry classifications (especially space group) are typically assigned after DFT relaxation on the same structures whose formation energies are the prediction target. This creates a potential circularity: the model may be learning from post-hoc structural information rather than predicting from composition alone. The manuscript must specify the data source and confirm that symmetry labels are not derived from the energies being modeled.
[Energy above hull prediction] Energy above hull prediction: Using the formation-energy prediction as an input feature for the energy-above-hull task introduces a dependency. Without explicit description of how this is handled in training (e.g., whether ground-truth or predicted values are used, and handling of train/test leakage), the validity of this secondary task cannot be assessed.

minor comments (2)

[Abstract] Abstract: The abstract does not mention the specific deep learning architecture, loss function, or optimization details.
[Results] Results: Any figure or table showing accuracy metrics with and without symmetry features should include statistical significance tests or error bars from multiple runs.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their detailed and constructive comments, which have helped us identify areas for clarification and improvement. We address each major comment below and have revised the manuscript accordingly to enhance transparency and address concerns about data handling and reporting.

read point-by-point responses

Referee: [Abstract] Abstract: The claim that symmetry integration 'significantly enhances' performance lacks supporting details on dataset size, train/test splits, baseline comparisons, error bars, or cross-validation, making it impossible to evaluate the magnitude or robustness of the improvement.

Authors: We agree that the abstract would benefit from additional context to substantiate the performance claims. The supporting details (dataset from Materials Project with ~140k entries, 80/20 train/test splits with 5-fold cross-validation, baselines without symmetry features, and MAE values with standard deviations across runs) are provided in the Methods and Results sections. We will revise the abstract to concisely include these elements, allowing readers to better assess the reported improvements. revision: yes
Referee: [Methods (symmetry input)] Methods (symmetry input): Symmetry classifications (especially space group) are typically assigned after DFT relaxation on the same structures whose formation energies are the prediction target. This creates a potential circularity: the model may be learning from post-hoc structural information rather than predicting from composition alone. The manuscript must specify the data source and confirm that symmetry labels are not derived from the energies being modeled.

Authors: We appreciate this important clarification request. The symmetry labels (crystal system, point group, space group) are structural descriptors obtained directly from the crystal structure entries in the Materials Project database using standard tools such as spglib; they are not derived from or dependent on the formation energy values. The model is intentionally designed to accept symmetry as an input feature alongside composition to distinguish polymorphs. We will update the Methods section to explicitly state the data source and confirm the independence of symmetry labels from the target energies, while adding a brief discussion on practical use cases (e.g., when symmetry is known or hypothesized). revision: yes
Referee: [Energy above hull prediction] Energy above hull prediction: Using the formation-energy prediction as an input feature for the energy-above-hull task introduces a dependency. Without explicit description of how this is handled in training (e.g., whether ground-truth or predicted values are used, and handling of train/test leakage), the validity of this secondary task cannot be assessed.

Authors: We agree that the description of the energy-above-hull extension requires more detail to ensure validity. The secondary model uses formation energy as an additional input feature. To avoid leakage, it is trained with ground-truth formation energies, while inference employs the predicted formation energies from the primary model. We will revise the relevant Methods and Results subsections to explicitly document this procedure, the cross-validation approach, and steps taken to prevent information leakage between the two tasks. revision: yes

Circularity Check

1 steps flagged · score 6.0 of 10

Formation-energy input to energy-above-hull task reduces prediction to fitted dependency by construction

fitted input called prediction [Abstract]
"In addition, we use the same model architecture to predict the energy above hull, an indicator to material stability, with formation energy as an additional input feature."

Energy above hull is computed from the formation energy of the compound minus the lowest-energy linear combination of stable phases. Including formation energy as an explicit input feature therefore supplies the dominant variable on which the target depends, so the reported 'prediction' is forced by the input rather than learned independently from composition and symmetry.

full rationale

The paper's central ML architecture predicts formation energy from composition plus symmetry labels. It then re-uses the same architecture for energy-above-hull regression while explicitly adding formation energy as an input feature. Because energy above hull is defined directly from formation energies (relative to the convex hull of reference phases), supplying formation energy as an input makes the second task statistically dependent on a quantity that is either the target or a close proxy of it. This matches the 'fitted input called prediction' pattern. No self-definitional, self-citation, or ansatz-smuggling issues appear in the symmetry usage itself, which remains an independent geometric descriptor. The overall score reflects partial circularity confined to the chained stability task.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities beyond the implicit assumption that symmetry labels are independent, accurate inputs. No numerical constants, ad-hoc scales, or new physical entities are mentioned.

assumptions (1)

domain assumption Symmetry classifications (space group, point group, crystal system) are accurate and independent of the formation energy being predicted
The abstract states that symmetry information is crucial for understanding phase transitions and is supplied as an additional input feature.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Formation Energy Prediction of Material Crystal Structures using Deep Learning." pith.science (2026). https://pith.science/paper/2412.00819

@misc{pith2026241200819,
  author       = {Pith},
  title        = {Pith review of: Formation Energy Prediction of Material Crystal Structures using Deep Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2412.00819}},
  note         = {Machine review of arXiv:2412.00819}
}

read the original abstract

Determining the stability of chemical compounds is essential for advancing material discovery. In this study, we introduce a novel deep neural network model designed to predict a crystal's formation energy, which identifies its stability property. Our model leverages elemental fractions derived from material composition and incorporates the symmetry classification as an additional input feature. The materials' symmetry classifications represent the crystal polymorphs and are crucial for understanding phase transitions in materials. Our findings demonstrate that the integration of crystal system, point group, or space group symmetry information significantly enhances the predictive performance of the developed deep learning architecture, where the highest accuracy was achieved when space group classification was incorporated. In addition, we use the same model architecture to predict the energy above hull, an indicator to material stability, with formation energy as an additional input feature.

Figures

Figures reproduced from arXiv: 2412.00819 by the authors.

**Figure 2.** True values vs. predicted formation energies using chemical formula only as [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** True values vs. predicted formation energies using chemical formula and symme [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: True values vs. predicted values for energy above the hull. This plot shows the [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Sign in to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Foundation/AbsoluteFloorClosure.lean reality_from_one_distinction unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

Our model leverages elemental fractions derived from material composition and incorporates the symmetry classification as an additional input feature... highest accuracy was achieved when space group classification was incorporated.
IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

the integration of crystal system, point group, or space group symmetry information significantly enhances the predictive performance

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

17 extracted references · 17 canonical work pages

[1]

Kirklin, J

S. Kirklin, J. E. Saal, B. Meredig, A. Thompson, J. W. Doak, M. Aykol, S. R¨ uhl, C. Wolverton, The open quantum materials database (oqmd): assessing the accuracy of dft formation energies, npj Computational Materials 1 (1) (2015) 1–15. 11

work page 2015
[2]

W. Sun, S. T. Dacek, S. P. Ong, G. Hautier, A. Jain, W. D. Richards, A. C. Gamst, K. A. Persson, G. Ceder, The thermodynamic scale of inor- ganic crystalline metastability, Science advances 2 (11) (2016) e1600225

work page 2016
[3]

A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, et al., Commentary: The materials project: A materials genome approach to accelerating materi- als innovation, APL materials 1 (1) (2013)

work page 2013
[4]

J. E. Saal, S. Kirklin, M. Aykol, B. Meredig, C. Wolverton, Materials design and discovery with high-throughput density functional theory: the open quantum materials database (oqmd), Jom 65 (2013) 1501– 1509

work page 2013
[5]

L. Ward, A. Dunn, A. Faghaninia, N. E. R. Zimmermann, S. Bajaj, Q. Wang, J. H. Montoya, J. Chen, K. Bystrom, M. Dylla, et al., Mat- miner: An open source toolkit for materials data mining, Computational Materials Science 152 (2018) 60–69

work page 2018
[6]

W. Ye, C. Chen, Z. Wang, I.-H. Chu, S. P. Ong, Deep neural networks for accurate predictions of crystal stability, Nature communications 9 (1) (2018) 3800

work page 2018
[7]

D. Jha, L. Ward, A. Paul, W.-k. Liao, A. Choudhary, C. Wolverton, A. Agrawal, Elemnet: Deep learning the chemistry of materials from only elemental composition, Scientific reports 8 (1) (2018) 17593

work page 2018
[8]

R. E. Goodall, A. A. Lee, Predicting materials properties without crys- tal structure: Deep representation learning from stoichiometry, Nature communications 11 (1) (2020) 6280

work page 2020

Show all 17 references

[9]

A. Jain, T. Bligaard, Atomic-position independent descriptor for ma- chine learning of material properties, Physical Review B 98 (21) (2018) 214112

2018
[10]

L. Ward, R. Liu, A. Krishna, V. I. Hegde, A. Agrawal, A. Choudhary, C. Wolverton, Including crystal structure attributes in machine learning models of formation energies via voronoi tessellations, Physical Review B 96 (2) (2017) 024104. 12

2017
[11]

K. Kim, L. Ward, J. He, A. Krishna, A. Agrawal, C. Wolverton, Machine-learning-accelerated high-throughput materials screening: Dis- covery of novel quaternary heusler compounds, Physical Review Mate- rials 2 (12) (2018) 123801

2018
[12]

C. C. Aggarwal, et al., Neural networks and deep learning, Springer 10 (978) (2018) 3

2018
[13]

Glorot, A

X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: Proceedings of the fourteenth international conference on artificial intelligence and statistics, JMLR Workshop and Conference Proceed- ings, 2011, pp. 315–323

2011
[14]

Kingma, J

D. Kingma, J. Ba, Adam: A method for stochastic optimization, in: In- ternational Conference on Learning Representations (ICLR), San Diega, CA, USA, 2015

2015
[15]

Duchi, E

J. Duchi, E. Hazan, Y. Singer, Adaptive subgradient methods for on- line learning and stochastic optimization., Journal of machine learning research 12 (7) (2011)

2011
[16]

Tieleman, G

T. Tieleman, G. Hinton, Root mean square propagation. divide the gra- dient by a running average of its recent magnitude. coursera: Neural networks for machine learning (2012)

2012
[17]

Van Rossum, The Python Library Reference, release 3.8.2, Python Software Foundation, 2020

G. Van Rossum, The Python Library Reference, release 3.8.2, Python Software Foundation, 2020. 13

2020

Pith tools

Reviewed May 23, 2026 · model on record in the stance chip above.

[1] [1]

Kirklin, J

S. Kirklin, J. E. Saal, B. Meredig, A. Thompson, J. W. Doak, M. Aykol, S. R¨ uhl, C. Wolverton, The open quantum materials database (oqmd): assessing the accuracy of dft formation energies, npj Computational Materials 1 (1) (2015) 1–15. 11

work page 2015

[2] [2]

W. Sun, S. T. Dacek, S. P. Ong, G. Hautier, A. Jain, W. D. Richards, A. C. Gamst, K. A. Persson, G. Ceder, The thermodynamic scale of inor- ganic crystalline metastability, Science advances 2 (11) (2016) e1600225

work page 2016

[3] [3]

A. Jain, S. P. Ong, G. Hautier, W. Chen, W. D. Richards, S. Dacek, S. Cholia, D. Gunter, D. Skinner, G. Ceder, et al., Commentary: The materials project: A materials genome approach to accelerating materi- als innovation, APL materials 1 (1) (2013)

work page 2013

[4] [4]

J. E. Saal, S. Kirklin, M. Aykol, B. Meredig, C. Wolverton, Materials design and discovery with high-throughput density functional theory: the open quantum materials database (oqmd), Jom 65 (2013) 1501– 1509

work page 2013

[5] [5]

L. Ward, A. Dunn, A. Faghaninia, N. E. R. Zimmermann, S. Bajaj, Q. Wang, J. H. Montoya, J. Chen, K. Bystrom, M. Dylla, et al., Mat- miner: An open source toolkit for materials data mining, Computational Materials Science 152 (2018) 60–69

work page 2018

[6] [6]

W. Ye, C. Chen, Z. Wang, I.-H. Chu, S. P. Ong, Deep neural networks for accurate predictions of crystal stability, Nature communications 9 (1) (2018) 3800

work page 2018

[7] [7]

D. Jha, L. Ward, A. Paul, W.-k. Liao, A. Choudhary, C. Wolverton, A. Agrawal, Elemnet: Deep learning the chemistry of materials from only elemental composition, Scientific reports 8 (1) (2018) 17593

work page 2018

[8] [8]

R. E. Goodall, A. A. Lee, Predicting materials properties without crys- tal structure: Deep representation learning from stoichiometry, Nature communications 11 (1) (2020) 6280

work page 2020