REVIEW 4 major objections 5 minor 111 references
CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A coordinate-free symmetry string is enough to predict crystal properties and obey thermodynamics.
desk verdict Genuinely new crystal representation and a promising physics-integration idea, but the Cv/U improvement claim is overstated and needs collision statistics before it convinces. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is SCOPE, a string built as `[generator string] | [Wyckoff symbols] | [composition]`. The generator string encodes each space group as a minimal set of symmetry operations using 14 matrices labeled by letters and 10 translation components, so all 230 space groups fit in about 4 kilobytes; the Wyckoff symbols name symmetry-equivalent sites by multiplicity and letter; the composition closes the string with element fractions. This representation is invariant under symmetry-preserving transformations and intentionally drops atomic coordinates and lattice parameters. CLOUD is a masked transformer encoder that is pretrained by reconstructing masked tokens in SCOPE strings, then fine-tuned with a prediction head on the `[CLS]` embedding; CLOUD-DEBYE replaces direct property regression with prediction of the Debye temperature $\Theta$, and a differentiable Debye-model integral converts $\Theta$ into temperature-dependent $C_v$ and $U$. The representation does the work of exposing global symmetry and long-range periodicity to the attention mechanism, while the Debye layer does the work of enforcing thermodynamic consistency and temperature dependence.
What would settle it
Find two DFT-relaxed crystals with identical SCOPE strings—same space group, same Wyckoff assignments, same composition—but measurably different heat capacity arising from different lattice parameters or free atomic coordinates. Since the model's input is identical for both, its predictions are identical, so at least one prediction is off by at least half the measured difference; a benchmark of many such pairs would directly quantify how much the omitted coordinates cost.
Extended reading notes
Core claim
In the paper's own terms, the central claim is that SCOPE—a symmetry-consistent string that records the space-group generator operations, the occupied Wyckoff positions, and the composition—is a sufficient input representation for a large pretrained transformer to predict formation energies, band gaps, elastic moduli, and thermodynamic properties at accuracy competitive with graph neural networks that consume full 3D coordinates. The model outperforms earlier coordinate-free and structure-agnostic baselines on most of eight regression tasks, matches a strong graph-based model on out-of-distribution stability screening, and is competitive with or better than structure-based models on unconventional crystals. The paper then makes a stronger, physics-specific claim: if the model is used to predict the Debye temperature $\Theta$ rather than the heat capacity directly, and $\Theta$ is fed through a differentiable Debye model, the resulting CLOUD-DEBYE predictor achieves lower MAE/MAD than descriptor-augmented graph networks on $C_v$ and $U$, satisfies the Dulong–Petit high-temperature limit and the $T^3$ low-temperature limit, and extrapolates measured heat capacities from 0 K toward the melting point after training on 300 K data only.
Load-bearing premise
The load-bearing premise is that a material's space group, Wyckoff position assignments, and composition determine the properties being predicted; because SCOPE discards lattice parameters and the adjustable coordinates inside Wyckoff sites, several physically distinct relaxed crystals can map to the same string and the model is forced to predict identical values for all of them.
Editorial extensions
If this is right
- Because the input is coordinate-free, property screening no longer requires DFT-relaxed coordinates or costly graph construction: a space group, Wyckoff assignment, and composition from a CIF file are enough to predict a broad spectrum of properties.
- The fitted scaling law with roughly equal exponents for data and parameters implies that adding pretraining data and model capacity in tandem should keep improving downstream accuracy predictably, following the same recipe used for large language models.
- On phonon thermodynamics, the Debye-fused model outperforms graph networks with and without global descriptors, so long-range and global properties are better handled by symmetry-explicit attention than by local message passing.
- Temperature-dependent heat capacity and internal energy can be predicted at arbitrary temperatures from labels collected at one temperature, because the Debye layer supplies the temperature dependence rather than the data.
- On unconventional crystals—defects, large cells, low-dimensional systems—the symmetry-string model is competitive with the best structure-based transformer, suggesting the representation generalizes beyond ideal ordered bulk crystals.
Reading between the lines
- A direct test of the representation's ceiling: two relaxed crystals with identical SCOPE strings but different lattice parameters or Wyckoff free coordinates must receive identical predictions; any property sensitive to those omitted degrees, such as exact phonon dispersion, would be provably unlearnable in the current encoding.
- The same differentiable-physics pattern could be reapplied to other structure-level laws—for instance the Einstein model, Gruneisen parameter relations, or equation-of-state forms—with CLOUD supplying the material-specific parameter and the law supplying the temperature, pressure, or volume dependence.
- Because the low fitted entropy term is partly a property of the rule-generated SCOPE grammar, representation design may be a more efficient route to better scientific foundation models than pouring in more tokens; comparing an augmented SCOPE that adds a few canonicalized lattice or shape invariants would isolate the value of the lost coordinates.
- One could test whether attention's strong weighting of space-group tokens is genuinely physical: ablate the generator-string portion while keeping Wyckoff and composition, and measure the drop in heat-capacity accuracy; if the drop is large, the symmetry encoding is doing the work.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces CLOUD, a BERT-style transformer for crystal property prediction, together with SCOPE, a coordinate-free string representation of crystals built from space-group generator strings, occupied Wyckoff symbols, and composition. CLOUD is pre-trained with masked language modeling on about 6.3 million structures from OPTIMADE and fine-tuned on MatBench, MatBench Discovery, UnconvBench, and a phonon thermodynamics dataset. The paper also proposes CLOUD-DEBYE, in which CLOUD predicts a Debye temperature and the Debye model supplies U(T) and C_v(T), enabling end-to-end training on 300 K phonon data. The authors report competitive MAE/MAD on MatBench, an AUC of 0.81 on WBM, strong UnconvBench results, fitted Hoffmann-style scaling-law exponents, and temperature extrapolation curves for four compounds including one explicitly held-out material.
Significance. If the claims are supported, the paper would show that a compact symmetry-based sequence representation can be pre-trained at scale, that such a representation is competitive across several materials benchmarks, and that coupling a transformer to a differentiable physical law yields thermodynamically consistent temperature-dependent heat capacities without per-temperature labels. The strengths of the manuscript are its reproducibility-oriented presentation: public code and data links, five-fold cross-validation with standard deviations, comparisons with a wide range of baselines, and a concrete attention analysis. The CLOUD-DEBYE idea of predicting a physical intermediate quantity (the Debye temperature) inside a differentiable model is attractive and potentially useful for screening.
major comments (4)
- [Table 1 and CLOUD-DEBYE results] The statement that CLOUD-DEBYE 'significantly' outperforms the descriptor-hybridized GNNs is not supported by the reported statistics. For C_v, CLOUD-DEBYE-FT gives MAE/MAD = 0.057 ± 0.001 while de-CGCNN and de-MEGNet give 0.058 ± 0.004 and 0.058 ± 0.005; for U, the values are 0.055 ± 0.003 versus 0.057 ± 0.005. These differences are comparable to or smaller than the reported standard deviations. The authors should either provide a paired significance test over the five folds or soften the claim from 'significantly reduces' to 'comparable or marginally lower.'
- [Supplementary S1 and Discussion] The manuscript acknowledges that SCOPE omits lattice parameters and free Wyckoff coordinates and therefore represents an ensemble of structures. This is not a generic limitation for the CLOUD-DEBYE central claim, because the Debye temperature in Eqs. (9)-(11) depends on volume V, effective interplanar spacing d, and sound velocity v, all of which are omitted degrees of freedom. Any two test structures with the same SCOPE string receive identical inputs and therefore identical Theta, U(T), and C_v(T). The paper does not quantify how often such collisions occur in the 1,512-structure Gong et al. dataset or how large the within-collision variance in C_v and U is. The authors should report collision statistics and within-group spreads; if collisions are frequent or the within-group spread is large, the Table 1 MAE/MAD values must be interpreted with respect to the benchmark's structure distribution rather than as evidence that the model has learned the structural determinants of Theta.
- [Figure 5 and temperature extrapolation] The temperature-extrapolation demonstration is weaker than stated. Only CaTiO3 is explicitly described as outside both the training and test sets; for Al2O3, Li2O, and GaN the manuscript does not state whether they occur in the Gong et al. training data. Moreover, the temperature dependence is imposed by the Debye model (Eqs. 7-8) rather than learned, so the extrapolation success tests the adequacy of the Debye approximation for these compounds and the accuracy of the learned Theta for the input SCOPE string, not a learned temperature dependence. The authors should report quantitative deviations from the experimental and DFT curves, state the training-set membership of all four compounds, and check for SCOPE-string overlap with training entries for CaTiO3.
- [Scaling Analysis and Eq. (17)] The scaling-law claim is under-documented. The paper reports five fitted parameters (A, B, E, alpha, beta) to four decimal places but does not state the number of (N, D, L) configurations used, the data subsampling scheme, or confidence intervals for the parameters. Since alpha and beta drive the claim that CLOUD scales in the same regime as Hoffmann's law (a = 0.45, b = 0.55), the authors should provide the empirical scaling points, parameter uncertainties, and a sensitivity analysis with respect to the Huber threshold and initialization.
minor comments (5)
- [Abstract and Methods (Datasets)] The pre-training set is described as containing 'DFT-relaxed crystals,' but the OPTIMADE sources include experimental databases such as COD; the description should be qualified accordingly.
- [Methods (Datasets)] The deduplication rule 'keeping the structure with the smallest volume per volume' appears to contain a typo and should presumably read 'smallest volume per atom' or similar. In addition, deduplicating on chemical formula plus space group alone may discard distinct Wyckoff configurations and lattice parameters, which is exactly the information SCOPE otherwise aims to encode.
- [Attention analysis] The t-test against a null value of 0.5 is not the correct random-attention baseline for the reported p1 and p2 statistics, because space-group tokens occupy about 64% of the sequence on average; for k = 1 the random expectation for p1 would be about 0.64, not 0.5. The qualitative conclusion may survive, but the statistical test should be specified correctly.
- [Figure 5] The figure compares computed C_v curves with experimental C_p curves near the melting point, where the C_p - C_v correction can be substantial; the authors should either report the correction or restrict the comparison to the temperature range where the approximation C_p approximately equals C_v is quantitatively justified.
- [Table S3] Wrenformer results are listed without standard deviations; the text should clarify whether these are single-split leaderboard values and should avoid direct error-bar comparisons with five-fold results.
Circularity Check
No significant circularity: external benchmarks carry the claims; Debye coupling is an explicit physics prior, and the admitted SCOPE incompleteness is a limitation rather than a circular step.
full rationale
CLOUD's property-prediction claims are supported by fine-tuning on external benchmarks (MatBench, MatBench Discovery, UnconvBench) and by baseline results obtained from MatBench leaderboards; no target quantity is used to define its own input representation, so the central derivation is not circular. CLOUD-DEBYE uses the Debye model (Eqs. 7-11) as a fixed differentiable law; the learned intermediate is the Debye temperature, which is trained end-to-end from Cv/U labels at 300 K rather than from Theta labels. Temperature dependence is therefore imposed by the physics law rather than learned from multi-temperature data, exactly as the paper states; this is an inductive-bias design choice, not a hidden recapitulation of the output by construction, and for held-out materials Theta remains a genuine structure-to-physics prediction. The acknowledged lossiness of SCOPE (free Wyckoff coordinates and lattice parameters omitted, Discussion and Supplementary S1) means structures sharing a SCOPE string receive identical inputs and hence identical predictions; that is a representation-completeness limitation and a potential benchmark-collision risk, but it is not a circular reduction of a derived result to an input. The only author-overlapping citations ([30] and [49]) are used as empirical baselines or related work, not as load-bearing uniqueness theorems or smuggled ansatze, and the MatInFormer results are independently available on the MatBench leaderboard. The scaling-law analysis fits Hoffmann's functional form to the model's own pre-training losses; this is descriptive curve fitting, and the near-agreement of the fitted exponents with Hoffmann's is an observed analogy rather than a result derived from those exponents.
Assumptions & free parameters
free parameters (2)
- Debye temperature Theta =
per-structure, learned by CLOUD
- Scaling-law coefficients A, B, E, alpha, beta =
A=90.7464, B=21.9854, E=0.2296, alpha=0.4339, beta=0.3518
assumptions (5)
- domain assumption Debye model adequately approximates phonon internal energy and heat capacity for the materials studied
- domain assumption Hoffmann scaling law L(N,D)=A/N^alpha+B/D^beta+E captures pretraining loss
- standard math Space-group generator strings from Ref. [60] correctly encode all 230 space groups
- domain assumption pymatgen SpacegroupAnalyzer assigns space groups and Wyckoff positions correctly
- domain assumption DFT-computed Cv and U labels from Gong et al. [27] are accurate reference values
Cite this review
Pith. "Pith review of CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning." pith.science (2026). https://pith.science/paper/6JBE3CHD
@misc{pith2026250617345,
author = {Pith},
title = {Pith review of: CLOUD: A Scalable and Physics-Informed Foundation Model for Crystal Representation Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/6JBE3CHD}},
note = {Machine review of arXiv:2506.17345}
}
read the original abstract
The prediction of crystal properties is essential for understanding structure-property relationships and accelerating the discovery of functional materials. However, conventional approaches relying on experimental measurements or density functional theory (DFT) calculations are often resource-intensive, limiting their scalability. Machine learning (ML) models offer a promising alternative by learning complex structure-property relationships from data, enabling faster predictions. Yet, existing ML models often rely on labeled data, adopt representations that poorly capture essential structural characteristics, and lack integration with physical principles--factors that limit their generalizability and interpretability. Here, we introduce CLOUD (Crystal Language mOdel for Unified and Differentiable materials modeling), a transformer-based framework trained on a novel Symmetry-Consistent Ordered Parameter Encoding (SCOPE) that encodes crystal symmetry, Wyckoff positions, and composition in a compact, coordinate-free string representation. Pre-trained on over six million crystal structures, CLOUD is fine-tuned on multiple downstream tasks and achieves competitive performance in predicting a wide range of material properties, demonstrating strong scaling performance. Furthermore, as proof of concept of differentiable materials modeling, CLOUD is applied to predict the phonon internal energy and heat capacity, which integrates the Debye model to preserve thermodynamic consistency. The CLOUD-DEBYE framework enforces thermodynamic consistency and enables temperature-dependent property prediction without requiring additional data. These results demonstrate the potential of CLOUD as a scalable and physics-informed foundation model for crystalline materials, unifying symmetry-consistent representations with physically grounded learning for property prediction and materials discovery.
Reference graph
Works this paper leans on
-
[1]
Materials Science and Engineering: R: Reports63(1), 1–30 (2008)
Wang, M., Pan, N.: Predictions of effective physical properties of complex mul- tiphase materials. Materials Science and Engineering: R: Reports63(1), 1–30 (2008)
2008
-
[2]
Scientific reports3(1), 2810 (2013)
Pilania, G., Wang, C., Jiang, X., Rajasekaran, S., Ramprasad, R.: Accelerating materials property predictions using machine learning. Scientific reports3(1), 2810 (2013)
2013
-
[3]
ACS central science4(8), 996–1006 (2018)
Ahmad, Z., Xie, T., Maheshwari, C., Grossman, J.C., Viswanathan, V.: Machine learning enabled computational screening of inorganic solid electrolytes for sup- pression of dendrite formation in lithium metal anodes. ACS central science4(8), 996–1006 (2018)
2018
-
[4]
Nature reviews materials3(5), 5–20 (2018)
Tabor, D.P., Roch, L.M., Saikin, S.K., Kreisbeck, C., Sheberla, D., Montoya, J.H., Dwaraknath, S., Aykol, M., Ortiz, C., Tribukait, H.,et al.: Accelerating the discovery of materials for clean energy in the era of smart automation. Nature reviews materials3(5), 5–20 (2018)
2018
-
[5]
npj Computational Materials8(1), 84 (2022)
Pyzer-Knapp, E.O., Pitera, J.W., Staar, P.W., Takeda, S., Laino, T., Sanders, D.P., Sexton, J., Smith, J.R., Curioni, A.: Accelerating materials discovery using artificial intelligence, high performance computing and robotics. npj Computational Materials8(1), 84 (2022)
2022
-
[6]
Oxford University Press, Oxford (2014)
Giustino, F.: Materials Modelling Using Density Functional Theory: Properties and Predictions. Oxford University Press, Oxford (2014)
2014
-
[7]
Nature communications13(1), 5454 (2022)
Dave, A., Mitchell, J., Burke, S., Lin, H., Whitacre, J., Viswanathan, V.: Autonomous optimization of non-aqueous li-ion battery electrolytes via robotic experimentation and machine learning coupling. Nature communications13(1), 5454 (2022)
2022
-
[8]
MRS Bulletin47(10), 1036–1044 (2022) https://doi.org/10.1557/ s43577-022-00424-0 27
Annevelink, E., Kurchin, R., Muckley, E., Kavalsky, L., Hegde, V.I., Sulzer, V., Zhu, S., Pu, J., Farina, D., Johnson, M., Gandhi, D., Dave, A., Lin, H., Edelman, A., Ramsundar, B., Saal, J., Rackauckas, C., Shah, V., Meredig, B., Viswanathan, V.: Automat: Automated materials discovery for electrochemi- cal systems. MRS Bulletin47(10), 1036–1044 (2022) ht...
2022
Show all 111 references
-
[9]
Nature chemistry1(1), 37–46 (2009)
Nørskov, J.K., Bligaard, T., Rossmeisl, J., Christensen, C.H.: Towards the computational design of solid catalysts. Nature chemistry1(1), 37–46 (2009)
2009
-
[10]
Elsevier, Amsterdam (1999)
Grimvall, G.: Thermophysical Properties of Materials. Elsevier, Amsterdam (1999)
1999
-
[11]
Integrating Materials and Manufacturing Innovation7, 43–51 (2018)
Kauwe, S.K., Graser, J., Vazquez, A., Sparks, T.D.: Machine learning prediction of heat capacity for solid inorganics. Integrating Materials and Manufacturing Innovation7, 43–51 (2018)
2018
-
[12]
Springer, New York (2012)
Shankar, R.: Principles of Quantum Mechanics. Springer, New York (2012)
2012
-
[13]
Physical Chemistry Chemical Physics22(20), 11174–11196 (2020)
McCullough, K., Williams, T., Mingle, K., Jamshidi, P., Lauterbach, J.: High- throughput experimentation meets artificial intelligence: a new pathway to catalyst discovery. Physical Chemistry Chemical Physics22(20), 11174–11196 (2020)
2020
-
[14]
Journal of the American Chemical Society146(29), 20009–20018 (2024)
Chen, C., Nguyen, D.T., Lee, S.J., Baker, N.A., Karakoti, A.S., Lauw, L., Owen, C., Mueller, K.T., Bilodeau, B.A., Murugesan, V.,et al.: Accelerating compu- tational materials discovery with machine learning and cloud high-performance computing: from large-scale screening to e...
2024
-
[15]
Crystallography Reviews10(1), 17–22 (2004)
Hellenbrandt, M.: The inorganic crystal structure database (icsd)—present and future. Crystallography Reviews10(1), 17–22 (2004)
2004
-
[16]
Chem1(4), 617–627 (2016)
Davies, D.W., Butler, K.T., Jackson, A.J., Morris, A., Frost, J.M., Skelton, J.M., Walsh, A.: Computational screening of all stoichiometric inorganic materials. Chem1(4), 617–627 (2016)
2016
-
[17]
ACS Omega (2024)
Phuthi, M.K., Yao, A.M., Batzner, S., Musaelian, A., Guan, P., Kozinsky, B., Cubuk, E.D., Viswanathan, V.: Accurate surface and finite-temperature bulk properties of lithium metal at large scales using machine learning interaction potentials. ACS Omega (2024)
2024
-
[18]
In: International Conference on Machine Learning, pp
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural mes- sage passing for quantum chemistry. In: International Conference on Machine Learning, pp. 1263–1272 (2017). PMLR
2017
-
[19]
Physical review letters120(14), 145301 (2018)
Xie, T., Grossman, J.C.: Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters120(14), 145301 (2018)
2018
-
[20]
Nature communications 11(1), 6280 (2020) 28
Goodall, R.E., Lee, A.A.: Predicting materials properties without crystal struc- ture: Deep representation learning from stoichiometry. Nature communications 11(1), 6280 (2020) 28
2020
-
[21]
npj Computational Materials7(1), 185 (2021)
Choudhary, K., DeCost, B.: Atomistic line graph neural network for improved materials property predictions. npj Computational Materials7(1), 185 (2021)
2021
-
[22]
arXiv preprint arXiv:2302.14102 (2023)
Ruff, R., Reiser, P., St¨ uhmer, J., Friederich, P.: Connectivity optimized nested graph networks for crystal structures. arXiv preprint arXiv:2302.14102 (2023)
2023 arXiv
-
[23]
Advanced Science9(18), 2200164 (2022)
Ihalage, A., Hao, Y.: Formula graph self-attention network for representation- domain independent materials discovery. Advanced Science9(18), 2200164 (2022)
2022
-
[24]
Npj Computational Materials7(1), 77 (2021)
Wang, A.Y.-T., Kauwe, S.K., Murdock, R.J., Sparks, T.D.: Composition- ally restricted attention-based network for materials property predictions. Npj Computational Materials7(1), 77 (2021)
2021
-
[25]
The Journal of Chemical Physics148(24) (2018)
Sch¨ utt, K.T., Sauceda, H.E., Kindermans, P.-J., Tkatchenko, A., M¨ uller, K.-R.: Schnet–a deep learning architecture for molecules and materials. The Journal of Chemical Physics148(24) (2018)
2018
-
[26]
In: International Conference on Machine Learning, pp
Satorras, V.G., Hoogeboom, E., Welling, M.: E (n) equivariant graph neural net- works. In: International Conference on Machine Learning, pp. 9323–9332 (2021). PMLR
2021
-
[27]
Science Advances9(45), 3245 (2023)
Gong, S., Yan, K., Xie, T., Shao-Horn, Y., Gomez-Bombarelli, R., Ji, S., Gross- man, J.C.: Examining graph neural networks for crystal structures: limitations and opportunities for capturing periodicity. Science Advances9(45), 3245 (2023)
2023
-
[28]
Science Advances 8(30), 4117 (2022)
Goodall, R.E., Parackal, A.S., Faber, F.A., Armiento, R., Lee, A.A.: Rapid dis- covery of stable materials by coordinate-free coarse graining. Science Advances 8(30), 4117 (2022)
2022
-
[29]
Nature Communications14(1), 7027 (2023)
Xiao, H., Li, R., Shi, X., Chen, Y., Zhu, L., Chen, X., Wang, L.: An invertible, invariant crystal representation for inverse design of solid-state materials using generative deep learning. Nature Communications14(1), 7027 (2023)
2023
-
[30]
arXiv preprint arXiv:2308.16259 (2023)
Huang, H., Magar, R., Xu, C., Farimani, A.B.: Materials informatics trans- former: A language model for interpretable materials properties prediction. arXiv preprint arXiv:2308.16259 (2023)
2023 arXiv
-
[31]
Communications Materials3(1), 1–9 (2022) https://doi.org/10.1038/s43246-022-00283-x
Fujinuma, N., DeCost, B., Hattrick-Simpers, J., Lofland, S.E.: Why big data and compute are not necessarily the path to big materials science. Communications Materials3(1), 1–9 (2022) https://doi.org/10.1038/s43246-022-00283-x
2022 doi
-
[32]
CoRRabs/2108.07258(2021) 2108.07258
Bommasani, R., Hudson, D.A., Adeli, E., Altman, R.B., Arora, S., Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N.S., Chen, A.S., Creel, K., Davis, J.Q., Dem- szky, D., Donahue, C., Doumbouya, M....
2021 arXiv
-
[33]
Advances in neural information processing systems30(2017)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems30(2017)
2017
-
[34]
arXiv preprint arXiv:1810.04805 (2018)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
2018 arXiv
-
[35]
arXiv preprint arXiv:2005.14165 (2020)
Brown, T.B.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
2020 arXiv
-
[36]
npj Computational Materials11(1), 84 (2025)
Lu, W., Luu, R.K., Buehler, M.J.: Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities. npj Computational Materials11(1), 84 (2025)
2025
-
[37]
arXiv preprint arXiv:2001.08361 (2020)
Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., Amodei, D.: Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)
2020 arXiv
-
[38]
arXiv preprint arXiv:2203.15556 (2022)
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D.d.L., Hendricks, L.A., Welbl, J., Clark, A., et al.: Training compute- optimal large language models. arXiv preprint arXiv:2203.15556 (2022)
2022 arXiv
-
[39]
arXiv preprint arXiv:2410.01243 (2024)
Nayak, A.K., Varshney, L.R.: An information theory of compute-optimal size scaling, emergence, and plateaus in language models. arXiv preprint arXiv:2410.01243 (2024)
2024 arXiv
-
[40]
Nature Machine Intelligence5(11), 1297–1305 (2023)
Frey, N.C., Soklaski, R., Axelrod, S., Samsi, S., Gomez-Bombarelli, R., Coley, C.W., Gadepally, V.: Neural scaling of deep chemical models. Nature Machine Intelligence5(11), 1297–1305 (2023)
2023
-
[41]
Nature, 1–6 (2023)
Merchant, A., Batzner, S., Schoenholz, S.S., Aykol, M., Cheon, G., Cubuk, E.D.: Scaling deep learning for materials discovery. Nature, 1–6 (2023)
2023
-
[42]
arXiv preprint arXiv:2307.04340 (2023)
Antunes, L.M., Butler, K.T., Grau-Crespo, R.: Crystal structure generation with autoregressive large language modeling. arXiv preprint arXiv:2307.04340 (2023)
2023 arXiv
-
[43]
arXiv preprint arXiv:2402.04379 (2024) 30
Gruver, N., Sriram, A., Madotto, A., Wilson, A.G., Zitnick, C.L., Ulissi, Z.: Fine-tuned language models generate stable inorganic materials as text. arXiv preprint arXiv:2402.04379 (2024) 30
2024 arXiv
-
[44]
CoRR abs/2010.09885(2020) 2010.09885
Chithrananda, S., Grand, G., Ramsundar, B.: Chemberta: Large-scale self-supervised pretraining for molecular property prediction. CoRR abs/2010.09885(2020) 2010.09885
2020 arXiv
-
[45]
Nature Machine Intelligence4(12), 1256–1264 (2022)
Ross, J., Belgodere, B., Chenthamarakshan, V., Padhi, I., Mroueh, Y., Das, P.: Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence4(12), 1256–1264 (2022)
2022
-
[46]
Advances in Neural Information Processing Systems36, 1452–1475 (2023)
Chen, D., Zhu, Y., Zhang, J., Du, Y., Li, Z., Liu, Q., Wu, S., Wang, L.: Uncover- ing neural scaling laws in molecular representation learning. Advances in Neural Information Processing Systems36, 1452–1475 (2023)
2023
-
[47]
Journal of Applied Physics 135(16) (2024)
Huang, H., Barati Farimani, A.: Multimodal learning of heat capacity based on transformers and crystallography pretraining. Journal of Applied Physics 135(16) (2024)
2024
-
[48]
Physical Review Materials7(2), 023803 (2023)
Gurunathan, R., Choudhary, K., Tavazza, F.: Rapid prediction of phonon struc- ture and properties using the atomistic line graph neural network (alignn). Physical Review Materials7(2), 023803 (2023)
2023
-
[49]
Nature Communications15(1), 8649 (2024)
Zhu, S., Ramsundar, B., Annevelink, E., Lin, H., Dave, A., Guan, P.-W., Gering, K., Viswanathan, V.: Differentiable modeling and optimization of non-aqueous li-based battery electrolyte solutions using geometric deep learning. Nature Communications15(1), 8649 (2024)
2024
-
[50]
Scripta Materialia207, 114217 (2022)
Guan, P.-W.: Differentiable thermodynamic modeling. Scripta Materialia207, 114217 (2022)
2022
-
[51]
Cell Reports Physical Science4(10) (2023)
Dold, D., Egmond, D.A.: Differentiable graph-structured models for inverse design of lattice materials. Cell Reports Physical Science4(10) (2023)
2023
-
[52]
Computational Materials Science253, 113828 (2025)
Al Hassanieh, S., Reinhart, W.F., Beese, A.M.: Efficient material model parameter optimization in finite element analysis with differentiable physics. Computational Materials Science253, 113828 (2025)
2025
-
[53]
Digital Discovery3(8), 1509–1533 (2024)
Evans, M.L., Bergsma, J., Merkys, A., Andersen, C.W., Andersson, O.B., Beltr´ an, D., Blokhin, E., Boland, T.M., Balderas, R.C., Choudhary, K.,et al.: Developments and applications of the optimade api for materials discovery, design, and data exchange. Digital Discovery3(8), 1...
2024
-
[54]
John Wiley & Sons, Hoboken (2018)
Kittel, C., McEuen, P.: Introduction to Solid State Physics. John Wiley & Sons, Hoboken (2018)
2018
-
[55]
Springer, Dordrecht (2014)
Malgrange, C., Ricolleau, C., Schlenker, M.: Symmetry and Physical Properties of Crystals. Springer, Dordrecht (2014)
2014
-
[56]
Advances in Neural Information Processing Systems35, 11423–11436 (2022)
Batatia, I., Kovacs, D.P., Simm, G., Ortner, C., Cs´ anyi, G.: Mace: Higher order 31 equivariant message passing neural networks for fast and accurate force fields. Advances in Neural Information Processing Systems35, 11423–11436 (2022)
2022
-
[57]
arXiv preprint arXiv:2306.12059 (2023)
Liao, Y.-L., Wood, B., Das, A., Smidt, T.: Equiformerv2: Improved equiv- ariant transformer for scaling to higher-degree representations. arXiv preprint arXiv:2306.12059 (2023)
2023 arXiv
-
[58]
arXiv preprint arXiv:2403.11857 (2024)
Yan, K., Fu, C., Qian, X., Qian, X., Ji, S.: Complete and efficient graph transformers for crystal material property prediction. arXiv preprint arXiv:2403.11857 (2024)
2024 arXiv
-
[59]
arXiv preprint arXiv:2402.03992 (2024)
Jiao, R., Huang, W., Liu, Y., Zhao, D., Liu, Y.: Space group constrained crystal generation. arXiv preprint arXiv:2402.03992 (2024)
2024 arXiv
-
[60]
Cambridge University Press, Cambridge (2012)
De Graef, M., McHenry, M.E.: Structure of Materials: an Introduction to Crys- tallography, Diffraction and Symmetry. Cambridge University Press, Cambridge (2012)
2012
-
[61]
Wyckoff, R.W.G.: The Analytical Expression of the Results of the Theory of Space-groups vol. 318. Carnegie institution of Washington, Washington, D.C. (1922)
1922
-
[62]
arXiv preprint arXiv:1910.14659 (2019)
Salazar, J., Liang, D., Nguyen, T.Q., Kirchhoff, K.: Masked language model scoring. arXiv preprint arXiv:1910.14659 (2019)
2019 arXiv
-
[63]
In: International Conference on Machine Learning, pp
Bao, H., Dong, L., Wei, F., Wang, W., Yang, N., Liu, X., Wang, Y., Gao, J., Piao, S., Zhou, M.,et al.: Unilmv2: Pseudo-masked language models for unified language model pre-training. In: International Conference on Machine Learning, pp. 642–652 (2020). PMLR
2020
-
[64]
arXiv preprint arXiv:2012.14388 (2020)
Yang, Z., Yang, Y., Cer, D., Law, J., Darve, E.: Universal sentence repre- sentation learning with conditional masked language model. arXiv preprint arXiv:2012.14388 (2020)
2020 arXiv
-
[65]
npj Computational Materials6(1), 138 (2020)
Dunn, A., Wang, Q., Ganose, A., Dopp, D., Jain, A.: Benchmarking materials property prediction methods: the matbench test set and automatminer reference algorithm. npj Computational Materials6(1), 138 (2020)
2020
-
[66]
arXiv preprint arXiv:2308.14920 (2023)
Riebesell, J., Goodall, R.E., Jain, A., Benner, P., Persson, K.A., Lee, A.A.: Mat- bench discovery–an evaluation framework for machine learning crystal stability prediction. arXiv preprint arXiv:2308.14920 (2023)
2023 arXiv
-
[67]
APL materials 1(1) (2013) 32
Jain, A., Ong, S.P., Hautier, G., Chen, W., Richards, W.D., Dacek, S., Cholia, S., Gunter, D., Skinner, D., Ceder, G., et al.: Commentary: The materials project: A materials genome approach to accelerating materials innovation. APL materials 1(1) (2013) 32
2013
-
[68]
npj Computational Materials7(1), 12 (2021)
Wang, H.-C., Botti, S., Marques, M.A.: Predicting stable crystalline compounds using chemical similarity. npj Computational Materials7(1), 12 (2021)
2021
-
[69]
Chemistry of Materials 31(9), 3564–3572 (2019)
Chen, C., Ye, W., Zuo, Y., Zheng, C., Ong, S.P.: Graph networks as a universal machine learning framework for molecules and crystals. Chemistry of Materials 31(9), 3564–3572 (2019)
2019
-
[70]
Battery Energy, 70004 (2024)
Wang, H., Sun, J., Liang, J., Zhai, L., Tang, Z., Li, Z., Zhai, W., Wang, X., Gao, W., Gong, S.: Crystograph: A comprehensive predictive model for crystal material properties and the benchmark. Battery Energy, 70004 (2024)
2024
-
[71]
arXiv preprint arXiv:2005.00928 (2020)
Abnar, S., Zuidema, W.: Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928 (2020)
2020 arXiv
-
[72]
Mathematics of computation35(151), 773–782 (1980)
Nocedal, J.: Updating quasi-newton matrices with limited storage. Mathematics of computation35(151), 773–782 (1980)
1980
-
[73]
arXiv preprint arXiv:2408.00165 (2024)
Wang, Y., Cho, K.: Non-convolutional graph neural networks. arXiv preprint arXiv:2408.00165 (2024)
2024 arXiv
-
[74]
arXiv preprint arXiv:2006.13318 (2020)
Cai, C., Wang, Y.: A note on over-smoothing for graph neural networks. arXiv preprint arXiv:2006.13318 (2020)
2020 arXiv
-
[75]
arXiv preprint arXiv:2303.10993 (2023)
Rusch, T.K., Bronstein, M.M., Mishra, S.: A survey on oversmoothing in graph neural networks. arXiv preprint arXiv:2303.10993 (2023)
2023 arXiv
-
[76]
arXiv preprint arXiv:2006.05205 (2020)
Alon, U., Yahav, E.: On the bottleneck of graph neural networks and its practical implications. arXiv preprint arXiv:2006.05205 (2020)
2020 arXiv
-
[77]
arXiv preprint arXiv:2111.14522 (2021)
Topping, J., Di Giovanni, F., Chamberlain, B.P., Dong, X., Bronstein, M.M.: Understanding over-squashing and bottlenecks on graphs via curvature. arXiv preprint arXiv:2111.14522 (2021)
2021 arXiv
-
[78]
npj Computational Materials2(1), 1–7 (2016)
Ward, L., Agrawal, A., Choudhary, A., Wolverton, C.: A general-purpose machine learning framework for predicting properties of inorganic materials. npj Computational Materials2(1), 1–7 (2016)
2016
-
[79]
Journal of research of the National Bureau of Standards57(2), 67–82 (1956)
Furukawa, G.T., Douglas, T.B., McCoskey, R.E., Ginnings, D.C.: Thermal prop- erties of aluminum oxide from 0 to 1200 k. Journal of research of the National Bureau of Standards57(2), 67–82 (1956)
1956
-
[80]
Stull, D.R.: JANAF Thermochemical Tables... vol. 1. Clearinghouse, Washing- ton, D.C. (1965)
1965
-
[81]
The Journal of Chemical Thermodynamics31(12), 1573–1583 (1999) 33
Woodfield, B.F., Shapiro, J.L., Stevens, R., Boerio-Goates, J., Putnam, R.L., Helean, K.B., Navrotsky, A.: Molar heat capacity and thermodynamic functions for catio3. The Journal of Chemical Thermodynamics31(12), 1573–1583 (1999) 33
1999
-
[82]
Thermochimica acta401(2), 169–173 (2003)
Leitner, J., Strejc, A., Sedmidubsky, D., Ruzicka, K.: High temperature enthalpy and heat capacity of gan. Thermochimica acta401(2), 169–173 (2003)
2003
-
[83]
arXiv preprint arXiv:2412.16551 (2024)
Loew, A., Sun, D., Wang, H.-C., Botti, S., Marques, M.A.: Universal machine learning interatomic potentials are ready for phonons. arXiv preprint arXiv:2412.16551 (2024)
2024 arXiv
-
[84]
Thermochimica Acta497(1-2), 7–13 (2010)
Leitner, J., Voˇ nka, P., Sedmidubsk` y, D., Svoboda, P.: Application of neumann– kopp rule for the estimation of heat capacity of mixed oxides. Thermochimica Acta497(1-2), 7–13 (2010)
2010
-
[85]
Industrial & engineering chemistry research35(1), 343–348 (1996)
Mostafa, A.G., Eakman, J.M., Montoya, M.M., Yarbro, S.L.: Prediction of heat capacities of solid inorganic salts from group contributions. Industrial & engineering chemistry research35(1), 343–348 (1996)
1996
-
[86]
Computational Materials Science68, 314–319 (2013)
Ong, S.P., Richards, W.D., Jain, A., Hautier, G., Kocher, M., Cholia, S., Gunter, D., Chevrier, V.L., Persson, K.A., Ceder, G.: Python materials genomics (pymat- gen): A robust, open-source python library for materials analysis. Computational Materials Science68, 314–319 (2013)
2013
-
[87]
org: A web ecosystem of databases, software and tools
Esters, M., Oses, C., Divilov, S., Eckert, H., Friedrich, R., Hicks, D., Mehl, M.J., Rose, F., Smolyanyuk, A., Calzolari, A.,et al.: aflow. org: A web ecosystem of databases, software and tools. Computational Materials Science216, 111808 (2023)
2023
-
[88]
Science advances7(49), 7948 (2021)
Schmidt, J., Pettersson, L., Verdozzi, C., Botti, S., Marques, M.A.: Crystal graph attention networks for the prediction of stable materials. Science advances7(49), 7948 (2021)
2021
-
[89]
2D Materials8(4), 044002 (2021)
Gjerding, M.N., Taghizadeh, A., Rasmussen, A., Ali, S., Bertoldo, F., Deilmann, T., Knøsgaard, N.R., Kruse, M., Larsen, A.H., Manti, S.,et al.: Recent progress of the computational 2d materials database (c2db). 2D Materials8(4), 044002 (2021)
2021
-
[90]
Nucleic acids research40(D1), 420–427 (2012)
Graˇ zulis, S., Daˇ skeviˇ c, A., Merkys, A., Chateigner, D., Lutterotti, L., Quiros, M., Serebryanaya, N.R., Moeck, P., Downs, R.T., Le Bail, A.: Crystallography open database (cod): an open-access collection of crystal structures and platform for world-wide collaboration. Nu...
2012
-
[91]
npj computational materials6(1), 173 (2020)
Choudhary, K., Garrity, K.F., Reid, A.C., DeCost, B., Biacchi, A.J., Hight Walker, A.R., Trautt, Z., Hattrick-Simpers, J., Kusne, A.G., Centrone, A., et al.: The joint automated repository for various integrated simulations (jarvis) for data-driven materials design. npj comput...
2020
-
[92]
Scientific data7(1), 299 (2020) 34
Talirz, L., Kumbhar, S., Passaro, E., Yakutovich, A.V., Granata, V., Gargiulo, F., Borelli, M., Uhrin, M., Huber, S.P., Zoupanos, S.,et al.: Materials cloud, a platform for open computational science. Scientific data7(1), 299 (2020) 34
2020
-
[93]
https://paulingfile.com/
Team, M.: Pauling File: Inorganic Materials Database. https://paulingfile.com/. Accessed: 2024-09-04 (2024)
2024
-
[94]
arXiv preprint arXiv:2404.02849 (2024)
Krajewski, A.M., Siegel, J.W., Liu, Z.-K.: Efficient structure-informed featuriza- tion and property prediction of ordered, dilute, and random atomic structures. arXiv preprint arXiv:2404.02849 (2024)
2024 arXiv
-
[95]
Advances in Applied Ceramics116(8), 428–433 (2017)
Fuentes-Cobas, L.E., Chateigner, D., Fuentes-Montero, M.E., Pepponi, G., Grazulis, S.: The representation of coupling interactions in the material prop- erties open database (mpod). Advances in Applied Ceramics116(8), 428–433 (2017)
2017
-
[96]
Mrs Bulletin43(9), 676–682 (2018)
Draxl, C., Scheffler, M.: Nomad: The fair concept for big data-driven materials science. Mrs Bulletin43(9), 676–682 (2018)
2018
-
[97]
Journal of Open Source Software5(54), 2563 (2020)
Evans, M.L., Morris, A.J.: matador: a python library for analysing, curating and performing high-throughput density-functional theory calculations. Journal of Open Source Software5(54), 2563 (2020)
2020
-
[98]
Machine Learning Meets Quantum Physics, 377–395 (2020)
Armiento, R.: Database-driven high-throughput calculations and machine learn- ing models for materials design. Machine Learning Meets Quantum Physics, 377–395 (2020)
2020
-
[99]
Jom65, 1501–1509 (2013)
Saal, J.E., Kirklin, S., Aykol, M., Meredig, B., Wolverton, C.: Materials design and discovery with high-throughput density functional theory: the open quantum materials database (oqmd). Jom65, 1501–1509 (2013)
2013
-
[100]
iScience23(11), 101681 (2020) https://doi.org/10.1016/j.isci
Grygorenko, O.O., Radchenko, D.S., Dziuba, I., Chuprina, A., Gubina, K.E., Moroz, Y.S.: Generating multibillion chemical space of readily accessible screen- ing compounds. iScience23(11), 101681 (2020) https://doi.org/10.1016/j.isci. 2020.101681
2020
-
[101]
Journal of Chemical Information and Modeling62(3), 553–566 (2022) https://doi.org/10.1021/acs.jcim.1c01378 https://doi.org/10.1021/acs.jcim.1c01378
Bellmann, L., Penner, P., Gastreich, M., Rarey, M.: Comparison of combinatorial fragment spaces and its application to ultralarge make- on-demand compound catalogs. Journal of Chemical Information and Modeling62(3), 553–566 (2022) https://doi.org/10.1021/acs.jcim.1c01378 https...
2022 doi
-
[102]
Scientific Data (2020)
Patel, H., Ihlenfeldt, W.-D., Judson, P.N., Moroz, Y.S., Pevzner, Y., Peach, M.L., Delann´ ee, V., Tarasova, N.I., Nicklaus, M.C.: Savi, in silico generation of billions of easily synthesizable compounds through expert-system type rules. Scientific Data (2020)
2020
-
[103]
Gavezzotti, A.: Are crystal structures predictable? Accounts of chemical research 27(10), 309–314 (1994)
1994
-
[104]
arXiv preprint arXiv:2312.03687 (2023)
Zeni, C., Pinsler, R., Z¨ ugner, D., Fowler, A., Horton, M., Fu, X., Shysheya, S., 35 Crabb´ e, J., Sun, L., Smith, J., et al.: Mattergen: a generative model for inorganic materials design. arXiv preprint arXiv:2312.03687 (2023)
2023 arXiv
-
[105]
arXiv preprint arXiv:2410.12771 (2024)
Barroso-Luque, L., Shuaibi, M., Fu, X., Wood, B.M., Dzamba, M., Gao, M., Rizvi, A., Zitnick, C.L., Ulissi, Z.W.: Open materials 2024 (omat24) inorganic materials dataset and models. arXiv preprint arXiv:2410.12771 (2024)
2024 arXiv
-
[106]
Nature Machine Intelligence5(9), 1031–1041 (2023)
Deng, B., Zhong, P., Jun, K., Riebesell, J., Han, K., Bartel, C.J., Ceder, G.: Chgnet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nature Machine Intelligence5(9), 1031–1041 (2023)
2023
-
[107]
Journal of Chemical Theory and Computation (2024)
Park, Y., Kim, J., Hwang, S., Han, S.: Scalable parallel algorithm for graph neu- ral network interatomic potentials in molecular dynamics simulations. Journal of Chemical Theory and Computation (2024)
2024
-
[108]
Nature Computational Science2(11), 718–728 (2022)
Chen, C., Ong, S.P.: A universal graph deep learning interatomic potential for the periodic table. Nature Computational Science2(11), 718–728 (2022)
2022
-
[109]
npj Computational Materials8(1), 211 (2022)
Gibson, J., Hire, A., Hennig, R.G.: Data-augmentation for graph neural net- work learning of the relaxed energies of unrelaxed structures. npj Computational Materials8(1), 211 (2022)
2022
-
[110]
Materials Today51, 126–135 (2021)
Zuo, Y., Qin, M., Chen, C., Ye, W., Li, X., Luo, J., Ong, S.P.: Accelerating mate- rials discovery with bayesian optimization and graph deep learning. Materials Today51, 126–135 (2021)
2021
-
[111]
Ward, L., Liu, R., Krishna, A., Hegde, V.I., Agrawal, A., Choudhary, A., Wolver- ton, C.: Including crystal structure attributes in machine learning models of formation energies via voronoi tessellations. Physical Review B96(2), 024104 (2017) 36 S1 Limitations and Future Work ...
2017
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.