Pith. sign in

REVIEW 4 major objections 5 minor 71 references

Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A contrastive model trained on published crystal structures and their paper titles lets users search materials by plain-text descriptions like 'superconductor'.

desk verdict CLaSP is a sensible application of CLIP-style contrastive learning to crystals using publication texts; the missing deduplication in the train/test split is the main soft spot, but the paper deserves serious peer review. read the letter →

arxiv 2501.12919 v2 pith:UCQYEUD2 submitted 2025-01-22 cs.LG cond-mat.mtrl-sci

Yuta Suzuki , Tatsunori Taniai , Ryo Igarashi , Kotaro Saito , Naoya Chiba , Yoshitaka Ushiku , Kanta Ono This is my paper · ORCID
classification cs.LGcond-mat.mtrl-sci
keywords CLaSPcontrastivelearningcrystalstructureembeddingmaterialsinformaticstext-basedretrievalcrossmodalrepresentationzero-shotscreening
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces CLaSP, a training method that aligns crystal structures with the words used to describe them in publications. It builds a shared embedding space from over 400,000 crystal structures and their paper titles, so a user can query a database with a phrase such as 'superconductor' and retrieve structures that were never labelled with that phrase. The authors show that this literature-driven supervision captures property- and functionality-level similarities that structure-only embedding methods miss. If the approach works at scale, materials researchers could search unannotated structural databases by the properties they care about, without needing hand-curated property labels.

What carries the argument

The central object is the CLaSP crossmodal embedding space, trained by minimizing a large-margin cosine contrastive loss between a crystal encoder and a text encoder. The crystal encoder is a CGCNN graph neural network with its property-regression head replaced by a linear projection to a 768-dimensional vector; the text encoder is a frozen SciBERT model followed by a multilayer perceptron. During pre-training the positive pairs are crystal structures and their publication titles, and during fine-tuning they are structures and LLM-generated keywords distilled from titles and abstracts. The loss pulls paired structure-text embeddings together while pushing all other pairs in the batch apart.

What would settle it

A controlled test would take materials with a well-known property that their publication titles and abstracts never mention, then check whether text-based retrieval for that property still ranks those materials above random; the paper's own 'ferromagnetic' result, a pre-trained ROC-AUC of 0.4281, shows such a test can fail and would settle whether title-based supervision is comprehensive.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that publication metadata, specifically paper titles and abstracts via LLM-generated keywords, can serve as a comprehensive source of property and functionality supervision for crystal structures. CLaSP learns a joint embedding space in which structures and texts are matched through a large-margin cosine contrastive loss. In zero-shot screening of 40,604 test structures, queries such as 'superconductor' reach a ROC-AUC of 0.9431 before fine-tuning, and keyword-based fine-tuning raises the mean ROC-AUC from 0.7121 to 0.7804 while also organizing the embedding space into semantically coherent material clusters.

Load-bearing premise

The load-bearing premise is that paper titles and abstracts give a comprehensive and representative description of a material's properties and functionalities; if they systematically omit a property, as the paper finds for 'ferromagnetic', the learned embedding cannot recover it.

Editorial extensions

If this is right

  • Text-based screening lets a researcher retrieve candidate materials by writing a property description, no matter whether the target structures carry any stored text labels.
  • Because the crystal encoder is trained without property labels, the approach can be applied to large unannotated structure databases such as newly simulated or measured structures.
  • The learned embedding space doubles as a materials map: clusters correspond to intuitive families like organometallic complexes, oxides, and intermetallics.
  • The paper's bandgap validation shows the text-driven clusters reflect quantitative physical properties, not just vocabulary.
  • LLM-generated keywords from abstracts improve retrieval over titles alone, indicating that richer text sources should yield further gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same contrastive scheme could be trained on full-text articles, citation contexts, or Wikipedia-style descriptions, which would likely fix the blind spots the paper found for terms that titles omit, such as 'ferromagnetic'.
  • Inference: the embedding space could be reused for open-vocabulary classification or captioning of crystal structures, where the model generates textual descriptions from structures instead of retrieving by text.
  • Inference: the method's success depends on how consistently the literature uses property words; a systematic audit of keyword prevalence versus known structural property labels would show where text-driven supervision saturates.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CLaSP (Contrastive Language–Structure Pre-training), a two-stage contrastive learning framework that aligns crystal-structure embeddings (a CGCNN encoder) with text embeddings (a frozen SciBERT encoder followed by an MLP). Pre-training uses pairs of COD crystal structures and their publication titles; fine-tuning uses LLM-generated property/functionality keywords derived from title–abstract pairs. The authors evaluate the method by zero-shot text-based retrieval of test-set structures, reporting ROC-AUC and AP for six property keywords (e.g., superconductor, ferroelectric), and by t-SNE visualization of the learned embedding space. They also compare against a structure-only baseline (CMML) and analyze the effect of the CosFace loss margin and scale. The central claim is that bibliographic text provides sufficient supervision to learn a crossmodal space that supports intuitive, text-driven screening of unannotated materials.

Significance. If the reported results hold, CLaSP would be a valuable contribution to materials informatics: it offers a scalable route to property- and functionality-sensitive structure embeddings without expensive first-principles labels, and it enables interpretable, text-queryable materials retrieval. The paper is strengthened by the release of code, pretrained weights, and the constructed dataset, and by the use of a held-out test split for retrieval evaluation. The qualitative visualizations (Sec. 4.3) provide intuitive evidence of semantic organization. However, the quantitative evaluation currently has gaps—most notably the absence of train/test deduplication for the COD data, the lack of a conventional text-search baseline, and the reliance on a single training run—that make the headline ROC-AUC numbers difficult to interpret as evidence of genuine generalization to unannotated structures.

major comments (4)
  1. [Sec. 7.3 and Table 1] The random 8:1:1 split of 406,048 COD entries is performed without any deduplication by chemical composition, space group, or structural similarity. COD is known to contain multiple entries for the same or nearly identical crystals (polymorphs, re-refinements, the same compound reported in different papers). Because CGCNN embeddings of near-identical structures are nearly identical, a test structure whose near-duplicate appeared in training with a title containing 'superconductor' could be retrieved correctly by structural memorization rather than by learning a general structure–property mapping. This would inflate the PT ROC-AUC of 0.9431 for 'superconductor' and other high scores in Table 1. The discussion in Sec. 5.1 addresses title-label noise (e.g., the low 'ferromagnetic' score) but does not address this leakage mechanism. The authors should either deduplicate the dataset (e.g., by composition and space group, or by structural similarity) and re-evaluate, or provide a clear analysis showing that near-duplicate train/test pairs do not drive the reported performance.
  2. [Sec. 4.1 and Sec. 4.4] The retrieval evaluation compares CLaSP only against the CMML baseline, which is a structure-only self-supervised method with a proxy concept-embedding procedure. The paper does not include a conventional text-search baseline, such as retrieving test structures by directly matching the query keyword against titles or against TF-IDF representations of titles/abstracts. Such a baseline would quantify the added value of the learned crossmodal embedding over simply searching the existing textual metadata—the very setting the paper claims to improve upon (Sec. 1: 'conventional text-based search systems falter when textual metadata are absent'). Adding this baseline is necessary to substantiate the claim that CLaSP enables screening of structures that lack textual annotations.
  3. [Sec. 7.1 and Table 1] All retrieval scores are reported for a single training run with no error bars, confidence intervals, or significance tests. The number of positives in the test set is small (e.g., 20 true positives for 'thermoelectric', 24 for 'electroluminescence'), so the ROC-AUC and AP estimates are subject to high variance. The authors should report results across multiple random seeds (or at least bootstrap confidence intervals) to establish that the differences between pre-trained and fine-tuned models, and between CLaSP and the baseline, are not due to chance.
  4. [Sec. 4.1 and Sec. 7.3] The bandgap sanity check (Fig. 4) uses bandgaps predicted by Crystalformer rather than experimentally measured or DFT-computed values. While this is a reasonable illustration, the text in Sec. 4.1 states that 'these distributions successfully reflect the expected bandgap ranges'—this is only as valid as the Crystalformer predictions themselves. The authors should either caveat this more explicitly or, where possible, validate a small subset against external measurements. This is a supporting result, not the central claim, but it should be framed as a model-based sanity check rather than an independent validation.
minor comments (5)
  1. [Sec. 7.3] The section heading 'Data preprocessing and spliting' contains a typo ('spliting' should be 'splitting').
  2. [Sec. 7.6] The sentence 'see Sec. 7.6 for for methodological details' has a duplicated 'for'.
  3. [Table 1 caption] The caption says 'pre-trained (PT) and fine-tined (FT)'—'fine-tined' should be 'fine-tuned'.
  4. [Sec. 4.1] The AP evaluation on a balanced subset with randomly downsampled negatives is non-standard; the paper should state explicitly that AP is computed on this balanced subset and note that the absolute AP values are therefore not comparable to AP computed on the full test set.
  5. [Sec. 7.1] The text says the model was trained for '2000 epochs' with a global batch size of 16,384; given 324,838 training pairs, this corresponds to roughly 40,000 optimizer steps, which is plausible, but the authors should confirm that 'epochs' is the intended unit rather than 'steps'.

Circularity Check

0 steps flagged · score 1.0 of 10

Minor self-citations present but not load-bearing; the retrieval evaluation is held-out and the derivation is non-circular.

full rationale

The paper's central claim, that CLaSP learns crossmodal structure-text embeddings enabling text-based screening, is supported by held-out retrieval experiments rather than by construction. Section 7.3 reports a random 8:1:1 split of 406,048 COD entries, with training, validation, and test sets kept separate; Table 1 evaluates retrieval on over 40,000 test structures whose labels are derived from publication titles. Because the contrastive loss in Eq. (1) is optimized only on training pairs, a high test ROC-AUC requires generalization to unseen structures and does not reduce to a fitted training quantity. Fine-tuning uses LLM-generated keywords from train-split title-abstract pairs, while test labels are title keywords, so the reported improvement is an empirical result rather than a label-leakage artifact. Hyperparameters were selected on the validation set, which is standard model selection rather than circular prediction. The paper's self-citations are not load-bearing: CMML [11] is used as a baseline for comparison, and Crystalformer [8] is used as an external bandgap oracle with pre-trained weights trained on JARVIS-DFT, independent of CLaSP's fitted values. The t-SNE cluster coherence analysis is self-consistent with the training objective but is presented as a visualization sanity check, not as a quantitative derivation. The absence of structure deduplication is a potential data-leakage threat to validity, but it is not circularity of the derivation chain, since the retrieval scores are not equal to the training objective by definition. Overall, no step in the paper's derivation reduces to its own inputs.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The approach's core assumption is that bibliographic text is a proxy for material properties; this is plausible but unproven, and the paper itself identifies biases.

free parameters (5)
  • Loss scale s = 3.0
    Selected from grid search over {1.0,1.5,2.0,2.5,3.0,3.5} based on validation ROC-AUC (Table 2).
  • Loss margin m = 0.5
    Selected from grid search over {0,0.3,0.5} based on validation ROC-AUC (Table 2).
  • Pre-training epochs = 2000
    Chosen by hand; no ablation reported for this choice.
  • Fine-tuning epochs = 50
    Chosen by hand; no ablation reported.
  • Global batch size = 16384
    Largest feasible on 8 A100 GPUs; not optimized.
assumptions (6)
  • domain assumption Publication titles and abstracts convey material properties and functionalities.
    Central premise of the method (Sec. 1 and 2); the paper acknowledges title noise for 'ferromagnetic' and 'semiconductor' (Sec. 5.1).
  • ad hoc to paper LLM-generated keywords are valid material property descriptions.
    Keywords generated by Llama 3 were not validated by domain experts (Sec. 5.2).
  • ad hoc to paper Title keyword occurrence is a reliable ground-truth label for retrieval evaluation.
    Used to label test materials (Sec. 4.1); noisy, e.g., materials with ferromagnetism may be omitted from titles.
  • domain assumption The COD dataset is representative of crystal structures and their property diversity.
    Paper notes 80% of entries from a few journals (Sec. 5.2), limiting generalizability.
  • domain assumption Frozen SciBERT is a sufficient text encoder for materials language.
    Relies on SciBERT's pretrained representations without fine-tuning.
  • domain assumption CGCNN can learn expressive structure embeddings from scratch.
    Baseline architecture used without architectural modifications beyond output head.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science." pith.science (2026). https://pith.science/paper/UCQYEUD2

@misc{pith2026250112919,
  author       = {Pith},
  title        = {Pith review of: Bridging Text and Crystal Structures: Literature-driven Contrastive Learning for Materials Science},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UCQYEUD2}},
  note         = {Machine review of arXiv:2501.12919}
}
read the original abstract

Understanding structure-property relationships is an essential yet challenging aspect of materials discovery and development. To facilitate this process, recent studies in materials informatics have sought latent embedding spaces of crystal structures to capture their similarities based on properties and functionalities. However, abstract feature-based embedding spaces are human-unfriendly and prevent intuitive and efficient exploration of the vast materials space. Here we introduce Contrastive Language--Structure Pre-training (CLaSP), a learning paradigm for constructing crossmodal embedding spaces between crystal structures and texts. CLaSP aims to achieve material embeddings that 1) capture property- and functionality-related similarities between crystal structures and 2) allow intuitive retrieval of materials via user-provided description texts as queries. To compensate for the lack of sufficient datasets linking crystal structures with textual descriptions, CLaSP leverages a dataset of over 400,000 published crystal structures and corresponding publication records, including paper titles and abstracts, for training. We demonstrate the effectiveness of CLaSP through text-based crystal structure screening and embedding space visualization.

Figures

Figures reproduced from arXiv: 2501.12919 by the authors.

Figure 1
Figure 1. Contrastive learning paradigm of CLaSP in two stages. (1) Pre-training using pairs of crystal structures and publication titles. (2) Fine-tuning using pairs of crystal structures and keywords that are generated from the titles and abstracts using an LLM. image and description text pairs. By learning to align two embedding spaces across the two modalities, CLIP enables crossmodal retrieval between images and texts, a… view at source ↗
Figure 2
Figure 2. Example of crystal structures with publication titles and generated keywords. Panels (a)–(d) showcase dataset entries [27, 28, 29, 30] whose titles or keywords contain ‘ferroelectric’, ‘electroluminescence’, ‘superconductor’, and ‘thermoelectric’, respectively. encoders by minimizing the large margin cosine loss function [24]: L = − 1 N X N i=1 log exp(s(cos(ci , ti) − m)) exp(s(cos(ci , ti) − m)) + PN j=1,j̸=i exp(… view at source ↗
Figure 3
Figure 3. ROC curves of keyword-based crystal structure retrieval. (a) The zero-shot results with only pre-training show good performance, and (b) fine-tuning leads to further improvements. (c) The baseline method CMML [11], which is trained solely on crystal structure information and does not utilize any text descriptions, lags behind our proposed approach that leverages both structure and textual data. The test set consists… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Violin plots of bandgaps for crystals retrieved via keyword searches. The distributions reflect the expected bandgap ranges for narrow bandgap materials and insulators, successfully demonstrating the retrieval of materials with targeted properties. 7 [PITH_FULL_IMAGE:…
Figure 5
Figure 5. Figure 5: t-SNE visualization of crystal structure embeddings. (a) World map of COD materials. The embeddings are grouped into 20 clusters and assigned keywords that represent the paper titles associated with the clusters. (b, c) Heat maps showing cosine similarities between the…
Figure 6
Figure 6. Figure 6: A Jensen–Shannon divergence matrix for cluster coherence and separation analysis. The rows and columns represent the 20 clusters from the map in Fig. 5a. Each matrix element represents the ‘distance’ between two clusters, evaluated based on the symmetric Jensen–Shannon…
Figure 7
Figure 7. Figure 7: t-SNE visualizations of crystal structure embeddings comparing CMML [11] and CLaSP. Material entries with publication titles that include the keywords ‘superconductor’ (top row) or ‘metal-organic framework’ (bottom row) are highlighted in red. 11 [PITH_FULL_IMAGE:figu…
Figure 8
Figure 8. Figure 8: Top 20 journals contributing to the COD dataset. candidates, echoing the evolution of vision–language models [60, 61]. Taken together, these capabilities promise a more intuitive and efficient exploration of the vast materials design space. 7 Method details Below, we p…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

71 extracted references · 25 canonical work pages

  1. [1]

    Callister, W. D. & Rethwisch, D. G.Materials Science and Engeneering(John Wiley and Sons, 2010)

    work page 2010

  2. [2]

    & McHenry, M

    De Graef, M. & McHenry, M. E.Structure of Materials. An Introduction to Crystallography, Diffraction and Symmetry (Cambridge University Press, 2012)

    work page 2012

  3. [3]

    & Grossman, J

    Xie, T. & Grossman, J. C. Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties.Phys. Rev. Lett.120, 145301 (2018). doi:10.1103/PhysRevLett.120. 145301

    doi:10.1103/physrevlett.120 2018

  4. [4]

    & Ong, S

    Chen, C., Ye, W., Zuo, Y ., Zheng, C. & Ong, S. P. Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals.Chem. Mater .31, 3564–3572 (2019).doi:10.1021/acs.chemmater.9b01294

    doi:10.1021/acs.chemmater.9b01294 2019

  5. [5]

    & Ong, S

    Chen, C. & Ong, S. P. A universal graph deep learning interatomic potential for the periodic table.Nat. Comput. Sci.2, 718–728 (2022).doi:10.1038/s43588-022-00349-3

    doi:10.1038/s43588-022-00349-3 2022

  6. [6]

    InThe 40th International Conference on Machine Learning (ICML 2023), vol

    Lin, Y .et al.Efficient Approximations of Complete Interatomic Potentials for Crystal Property Prediction. InThe 40th International Conference on Machine Learning (ICML 2023), vol. 202 ofProceedings of Machine Learning Research, 21260–21287 (2023). Online:https://proceedings.mlr.press/v202/lin23m.html

    work page 2023

  7. [7]

    Yan, K., Liu, Y ., Lin, Y . & Ji, S. Periodic Graph Transformers for Crystal Material Property Prediction. InAdvances in Neural Information Processing Systems 25 (NeurIPS 2022), 15066–15080 (2022).arXiv:2209.11807

    arXiv 2022

  8. [8]

    InThe Twelfth International Conference on Learning Representations (ICLR 2024)(2024)

    Taniai, T.et al.Crystalformer: Infinitely Connected Attention for Periodic Structure Encoding. InThe Twelfth International Conference on Learning Representations (ICLR 2024)(2024). Online: https://openreview. net/forum?id=fxQiecl9HB

    work page 2024

Show all 71 references
  1. [9]

    & Ono, K

    Ito, Y ., Taniai, T., Igarashi, R., Ushiku, Y . & Ono, K. Rethinking the role of frames for SE(3)-invariant crystal structure modeling. InThe Thirteenth International Conference on Learning Representations (ICLR 2025)(2025). Online:https://openreview.net/forum?id=gzxDjnvBDa

    2025

  2. [10]

    & Grossman, J

    Xie, T. & Grossman, J. C. Hierarchical visualization of materials space with graph convolutional neural networks. J. Chem. Phys.149, 174111 (2018).doi:10.1063/1.5047803

    2018 doi

  3. [11]

    & Ono, K

    Suzuki, Y ., Taniai, T., Saito, K., Ushiku, Y . & Ono, K. Self-supervised learning of materials concepts from crystal structures via deep neural networks.Mach. Learn.: Sci. Technol.3, 045034 (2022). doi:10.1088/2632-2153/ aca23d

    2022 doi

  4. [12]

    Li, Q.et al.Global Mapping of Structures and Properties of Crystal Materials.J. Chem. Inf. Model.63, 3814–3826 (2023).doi:10.1021/acs.jcim.3c00224

    2023 doi

  5. [13]

    Mater .10, 1–14 (2024).doi:10.1038/s41524-024-01231-8

    Qu, J.et al.Leveraging language representation for materials exploration and discovery.npj Comput. Mater .10, 1–14 (2024).doi:10.1038/s41524-024-01231-8

    2024 doi

  6. [14]

    InThe 38th International Conference on Machine Learning (ICML 2021), vol

    Radford, A.et al.Learning Transferable Visual Models from Natural Language Supervision. InThe 38th International Conference on Machine Learning (ICML 2021), vol. 139 ofProceedings of Machine Learning Research, 8748–8763 (2021). Online:https://proceedings.mlr.press/v139/radford21a.html

    2021

  7. [15]

    & Sun, M

    Zeng, Z., Yao, Y ., Liu, Z. & Sun, M. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals.Nat. Commun.13, 862 (2022). doi:10.1038/ s41467-022-28494-3. 16

    2022

  8. [16]

    Bioinform.23, bbac461 (2022).doi:10.1093/bib/bbac461

    Wang, J.et al.Multi-modal chemical information reconstruction from images and texts for exploring the near-drug space.Brief. Bioinform.23, bbac461 (2022).doi:10.1093/bib/bbac461

    2022 doi

  9. [17]

    Liu, S.et al.Multi-modal molecule structure–text model for text-based retrieval and editing.Nat. Mach. Intell.5, 1447–1457 (2023).doi:10.1038/s42256-023-00759-6

    2023 doi

  10. [18]

    & Klambauer, G

    Seidl, P., Vall, A., Hochreiter, S. & Klambauer, G. Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language.CoRR(2023).arXiv:2303.03363

    2023 arXiv

  11. [19]

    InThe NeurIPS 2023 Workshop on AI for Accelerated Materials Design (AI4Mat 2023)(2023)

    Takeda, S.et al.Multi-modal Foundation Model for Material Design. InThe NeurIPS 2023 Workshop on AI for Accelerated Materials Design (AI4Mat 2023)(2023). Online: https://openreview.net/forum?id= EiT2bLsfM9

    2023

  12. [20]

    Kaufman, B.et al.COATI: Multimodal Contrastive Pretraining for Representing and Traversing Chemical Space. J. Chem. Inf. Model.64, 1145–1157 (2024).doi:10.1021/acs.jcim.3c01753

    2024 doi

  13. [21]

    Online:https://www.cell.com/newton/abstract/S2950-6360(25)00008-8

    Moro, V .et al.Multimodal foundation models for material property prediction and discovery.Newton1(2025). Online:https://www.cell.com/newton/abstract/S2950-6360(25)00008-8

    2025

  14. [22]

    & Itakura, T

    Ozawa, K., Suzuki, T., Tonogai, S. & Itakura, T. Graph-text contrastive learning of inorganic crystal structure toward a foundation model of inorganic materials.STAM Methods0, 2406219 (2024). doi:10.1080/27660400. 2024.2406219

    2024

  15. [23]

    Gražulis, S.et al.Crystallography Open Database – an open-access collection of crystal structures.J. Appl. Crystallogr .42, 726–729 (2009).doi:10.1107/S0021889809016690

    2009 doi

  16. [24]

    InThe 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), 5265–5274 (2018)

    Wang, H.et al.CosFace: Large Margin Cosine Loss for Deep Face Recognition. InThe 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2018), 5265–5274 (2018). doi:10.1109/CVPR. 2018.00552

    2018

  17. [25]

    & Cohan, A

    Beltagy, I., Lo, K. & Cohan, A. SciBERT: A Pretrained Language Model for Scientific Text. InThe 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), 3615–3620 (2019).d...

    2019 doi

  18. [26]

    Grattafiori, A.et al.The Llama 3 Herd of Models.CoRR(2024).arXiv:2407.21783

    2024 arXiv

  19. [27]

    Pasi´nska, K.et al.A paraelectric–ferroelectric phase transition of an organically templated zinc oxalate coordina- tion polymer.Dalton Trans.47, 11308–11312 (2018).doi:10.1039/C8DT02859A

    2018 doi

  20. [28]

    Dalton Trans.47, 7463–7470 (2018).doi:10.1039/C8DT01310A

    Li, C.-A.et al.A PEG/copper(I) halide cluster as an eco-friendly catalytic system for C–N bond formation. Dalton Trans.47, 7463–7470 (2018).doi:10.1039/C8DT01310A

    2018 doi

  21. [29]

    V ., Petˇríˇcek, V ., Khasanova, N

    Mironov, A. V ., Petˇríˇcek, V ., Khasanova, N. R. & Antipov, E. V . New insight on bismuth cuprates with incommensurate modulated structures.Acta Cryst B72, 395–403 (2016). doi:10.1107/S2052520616005643

    2016 doi

  22. [30]

    O., Baranets, S., Childs, A

    Ogunbunmi, M. O., Baranets, S., Childs, A. B. & Bobev, S. The Zintl phases AIn2As2 (A = Ca, Sr, Ba): New topological insulators and thermoelectric material candidates.Dalton Trans.50, 9173–9184 (2021). doi: 10.1039/D1DT01521D

    2021 doi

  23. [31]

    T., Sauceda, H

    Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. SchNet – A deep learning architecture for molecules and materials.J. Chem. Phys.148, 241722 (2018).doi:10.1063/1.5019779

    2018 doi

  24. [32]

    & DeCost, B

    Choudhary, K. & DeCost, B. Atomistic Line Graph Neural Network for improved materials property predictions. npj Comput. Mater .7, 185 (2021).doi:10.1038/s41524-021-00650-1

    2021 doi

  25. [33]

    Louis, S.-Y .et al.Graph convolutional neural networks with global attention for improved materials property prediction.Phys. Chem. Chem. Phys.22, 18141–18148 (2020).doi:10.1039/D0CP01474E

    2020 doi

  26. [34]

    Jain, A.et al.Commentary: The Materials Project: A materials genome approach to accelerating materials innovation.APL Materials1, 011002 (2013).doi:10.1063/1.4812323

    2013 doi

  27. [35]

    Mater .6, 1–13 (2020).doi:10.1038/s41524-020-00440-1

    Choudhary, K.et al.The joint automated repository for various integrated simulations (JARVIS) for data-driven materials design.npj Comput. Mater .6, 1–13 (2020).doi:10.1038/s41524-020-00440-1

    2020 doi

  28. [36]

    Deng, B.et al.CHGNet as a pretrained universal neural network potential for charge-informed atomistic modelling. Nat. Mach. Intell.5, 1031–1041 (2023).doi:10.1038/s42256-023-00716-3

    2023 doi

  29. [37]

    Batatia, I.et al.A foundation model for atomistic materials chemistry.CoRR(2024).arXiv:2401.00096

    2024 arXiv

  30. [38]

    CoRR(2024).arXiv:2405.04967

    Yang, H.et al.MatterSim: A Deep Learning Atomistic Model Across Elements, Temperatures and Pressures. CoRR(2024).arXiv:2405.04967

    2024 arXiv

  31. [39]

    Chanussot, L.et al.Open Catalyst 2020 (OC20) Dataset and Community Challenges.ACS Catalysis11, 6059–6072 (2021).doi:10.1021/acscatal.0c04525. 17

    2021 doi

  32. [40]

    Tran, R.et al.The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts.ACS Catalysis 13, 3066–3084 (2023).doi:10.1021/acscatal.2c05426

    2023 doi

  33. [41]

    Zhou, Q.et al.Learning atoms for materials discovery.Proceedings of the National Academy of Sciences (PNAS) 115, E6411–E6417 (2018).doi:10.1073/pnas.1801181115

    2018 doi

  34. [42]

    & Shatruk, M

    Ryan, K., Lengyel, J. & Shatruk, M. Crystal Structure Prediction via Deep Learning.J. Am. Chem. Soc.140, 10158–10168 (2018). PMID: 29874459,doi:10.1021/jacs.8b03913

    2018 doi

  35. [43]

    & Dean, J

    Mikolov, T., Chen, K., Corrado, G. & Dean, J. Efficient Estimation of Word Representations in Vector Space. CoRR(2013).arXiv:1301.3781

    2013 arXiv

  36. [44]

    Sutskever, I., Vinyals, O. & Le, Q. V . Sequence to Sequence Learning with Neural Networks. InAdvances in Neural Information Processing Systems 27 (NIPS 2014), 3104–3112 (2014).arXiv:1409.3215

    2014 arXiv

  37. [45]

    InAdvances in Neural Information Processing Systems 30 (NIPS 2017), 6000–6010 (2017).arXiv:1706.03762

    Vaswani, A.et al.Attention Is All you Need. InAdvances in Neural Information Processing Systems 30 (NIPS 2017), 6000–6010 (2017).arXiv:1706.03762

    2017 arXiv

  38. [46]

    & Toutanova, K

    Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. InThe 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2019...

    2019 doi

  39. [47]

    & Sutskever, I

    Radford, A., Narasimhan, K., Salimans, T. & Sutskever, I. Improving Language Understanding by Gener- ative Pre-Training. Tech. Rep., OpenAI (2018). Online: https://cdn.openai.com/research-covers/ language-unsupervised/language_understanding_paper.pdf

    2018

  40. [48]

    Radford, A.et al.Language Models are Unsupervised Multitask Learners. Tech. Rep., OpenAI (2019). Online: https://cdn.openai.com/better-language-models/language_models_are_unsupervised_ multitask_learners.pdf

    2019

  41. [49]

    InAdvances in Neural Information Processing Systems 33 (NeurIPS 2020), 1877–1901 (2020).arXiv:2005.14165

    Brown, T.et al.Language Models are Few-Shot Learners. InAdvances in Neural Information Processing Systems 33 (NeurIPS 2020), 1877–1901 (2020).arXiv:2005.14165

    2020 arXiv

  42. [50]

    OpenAIet al.GPT-4 Technical Report.CoRR(2024).arXiv:2303.08774

    2024 arXiv

  43. [51]

    arXiv:2302.13971

    Touvron, H.et al.LLaMA: Open and Efficient Foundation Language Models.CoRR(2023). arXiv:2302.13971

    2023 arXiv

  44. [52]

    arXiv:2307.09288

    Touvron, H.et al.Llama 2: Open Foundation and Fine-Tuned Chat Models.CoRR(2023). arXiv:2307.09288

    2023 arXiv

  45. [53]

    Nature571, 95–98 (2019).doi:10.1038/s41586-019-1335-8

    Tshitoyan, V .et al.Unsupervised word embeddings capture latent knowledge from materials science literature. Nature571, 95–98 (2019).doi:10.1038/s41586-019-1335-8

    2019 doi

  46. [54]

    arXiv:2108.07258

    Bommasani, R.et al.On the Opportunities and Risks of Foundation Models.CoRR(2022). arXiv:2108.07258

    2022 arXiv

  47. [55]

    InThe 38th International Conference on Machine Learning (ICML 2021), vol

    Jia, C.et al.Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision. InThe 38th International Conference on Machine Learning (ICML 2021), vol. 139 ofProceedings of Machine Learning Research, 4904–4916 (2021). Online:https://proceedings.mlr.pr...

    2021

  48. [56]

    InThe 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), 248–255 (2009).doi:10.1109/CVPR.2009.5206848

    Deng, J.et al.ImageNet: A large-scale hierarchical image database. InThe 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), 248–255 (2009).doi:10.1109/CVPR.2009.5206848

    2009

  49. [57]

    InThe 13th European Conference on Computer Vision (ECCV 2014), 740–755 (2014).doi:10.1007/978-3-319-10602-1_48

    Lin, T.-Y .et al.Microsoft COCO: Common Objects in Context. InThe 13th European Conference on Computer Vision (ECCV 2014), 740–755 (2014).doi:10.1007/978-3-319-10602-1_48

    2014 doi

  50. [58]

    InAdvances in Neural Information Processing Systems 33 (NeurIPS 2020), 9459–9474 (2020).arXiv:2005.11401

    Lewis, P.et al.Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. InAdvances in Neural Information Processing Systems 33 (NeurIPS 2020), 9459–9474 (2020).arXiv:2005.11401

    2020 arXiv

  51. [59]

    & Chen, D

    Xia, M., Gao, T., Zeng, Z. & Chen, D. Sheared llama: Accelerating language model pre-training via structured pruning.CoRR(2024).arXiv:2310.06694

    2024 arXiv

  52. [60]

    & Hoi, S

    Li, J., Li, D., Savarese, S. & Hoi, S. BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models.CoRR(2023).arXiv:2301.12597

    2023 arXiv

  53. [61]

    & Chen, M

    Ramesh, A., Dhariwal, P., Nichol, A., Chu, C. & Chen, M. Hierarchical Text-Conditional Image Generation with CLIP Latents.CoRR(2022).arXiv:2204.06125

    2022 arXiv

  54. [62]

    Kingma, D. P. & Ba, J. Adam: A Method for Stochastic Optimization. InThe Third International Conference on Learning Representations (ICLR 2015)(2015).arXiv:1412.6980

    2015 arXiv

  55. [63]

    & Hutter, F

    Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. InThe Seventh International Conference on Learning Representations (ICLR 2019)(2019). Online:https://openreview.net/forum?id=Bkg6RiCqY7. 18

    2019

  56. [64]

    InThe 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2024), vol

    Ansel, J.et al.PyTorch 2: Faster Machine Learning through Dynamic Python Bytecode Transformation and Graph Compilation. InThe 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2024), vol. 2, 929–947 (2024).doi:10...

    2024

  57. [65]

    & Lenssen, J

    Fey, M. & Lenssen, J. E. Fast Graph Representation Learning with PyTorch Geometric. InThe ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds(2019).arXiv:1903.02428

    2019 arXiv

  58. [66]

    In The 29th Symposium on Operating Systems Principles (SOSP 2023), 611–626 (2023).doi:10.1145/3600006

    Kwon, W.et al.Efficient Memory Management for Large Language Model Serving with PagedAttention. In The 29th Symposium on Operating Systems Principles (SOSP 2023), 611–626 (2023).doi:10.1145/3600006. 3613165

    2023 doi

  59. [67]

    & Izumi, F.VESTA 3for three-dimensional visualization of crystal, volumetric and morphology data

    Momma, K. & Izumi, F.VESTA 3for three-dimensional visualization of crystal, volumetric and morphology data. J. Appl. Crystallogr .44, 1272–1276 (2011).doi:10.1107/S0021889811038970

    2011 doi

  60. [68]

    & Hinton, G

    van der Maaten, L. & Hinton, G. Visualizing Data using t-SNE.J. Mach. Learn. Res.9, 2579–2605 (2008). Online:http://jmlr.org/papers/v9/vandermaaten08a.html

    2008

  61. [69]

    G., Stražar, M

    Poliˇcar, P. G., Stražar, M. & Zupan, B. openTSNE: A Modular Python Library for t-SNE Dimensionality Reduction and Embedding.J. Stat. Softw.109, 1–30 (2024).doi:10.18637/jss.v109.i03

    2024 doi

  62. [70]

    & Vassilvitskii, S

    Arthur, D. & Vassilvitskii, S. k-means++: The Advantages of Careful Seeding. InThe Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2007), 1027–1035 (2007)

    2007

  63. [71]

    A Statistical Interpretation of Term Specificity and Its Application in Retrieval.Journal of Documentation28, 11–21 (1972)

    Spärck Jones, K. A Statistical Interpretation of Term Specificity and Its Application in Retrieval.Journal of Documentation28, 11–21 (1972). Online:https://doi.org/10.1108/eb026526. 19

    1972 doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.