Pith. sign in

REVIEW 6 cited by

From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.16802 v2 pith:EQNKHU6K submitted 2023-10-25 cs.LG

Nima Shoghi , Adeesh Kolluru , John R. Kitchin , Zachary W. Ulissi , C. Lawrence Zitnick , Brandon M. Wood This is my paper · ORCID
classification cs.LG
keywords pre-trainingchemicaldomainsmodelspredictionpropertytaskstraining
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
abstract

Foundation models have been transformational in machine learning fields such as natural language processing and computer vision. Similar success in atomic property prediction has been limited due to the challenges of training effective models across multiple chemical domains. To address this, we introduce Joint Multi-domain Pre-training (JMP), a supervised pre-training strategy that simultaneously trains on multiple datasets from different chemical domains, treating each dataset as a unique pre-training task within a multi-task framework. Our combined training dataset consists of $\sim$120M systems from OC20, OC22, ANI-1x, and Transition-1x. We evaluate performance and generalization by fine-tuning over a diverse set of downstream tasks and datasets including: QM9, rMD17, MatBench, QMOF, SPICE, and MD22. JMP demonstrates an average improvement of 59% over training from scratch, and matches or sets state-of-the-art on 34 out of 40 tasks. Our work highlights the potential of pre-training strategies that utilize diverse data to advance property prediction across chemical domains, especially for low-data tasks. Please visit https://nima.sh/jmp for further information.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DEQuify your force field: More efficient simulations using deep equilibrium models

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Recasting EquiformerV2 as a deep equilibrium model with warm-started fixed points gives faster and often more accurate force predictions on standard MD benchmarks, though energy accuracy on OC20 is worse.

  2. Distillation of atomistic foundation models across architectures and chemical domains

    physics.comp-ph 2025-06 conditional novelty 6.0 of 10

    Distillation of atomistic foundation models via synthetic data yields 10x-100x faster student potentials with near-teacher accuracy.

  3. From Molecules to Mixtures: Learning Representations of Olfactory Mixture Similarity using Inductive Biases

    cs.LG 2025-01 conditional novelty 6.0 of 10

    POMMix, a graph-based model with attention and cosine similarity heads, extends the Principal Odor Map to predict human perceptual similarity of odor mixtures, reporting a test correlation of 0.78 on a compiled datase...

  4. Implicit Delta Learning of High Fidelity Neural Network Potentials

    physics.chem-ph 2024-12 conditional novelty 6.0 of 10

    IDLe, a multi-task training strategy with fidelity-specific prediction heads on a shared representation, matches high-fidelity NNP energy accuracy while using up to 50x less high-fidelity QM data.

  5. LAMBench: A Benchmark for Large Atomistic Models

    physics.comp-ph 2025-04 conditional novelty 5.0 of 10

    LAMBench evaluates ten large atomistic models on out-of-distribution accuracy, property prediction, fine-tuning, speed, and stability, finding a large gap to a universal potential and DPA-3.1-3M at the top.

  6. Towards Faster and More Compact Foundation Models for Molecular Property Prediction

    cs.LG 2025-04 conditional novelty 4.0 of 10

    Removing the two deepest interaction blocks from the JMP-L molecular foundation model reduces size by 32% and speeds inference by 1.3x while keeping downstream accuracy nearly unchanged.

Pith tools