Pith. sign in

REVIEW 8 cited by

Uni-Mol2: Exploring Molecular Pretraining Model at Scale

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.14969 v2 pith:XL5MQ33L submitted 2024-06-21 cs.LG cs.AI

Uni-Mol2: Exploring Molecular Pretraining Model at Scale

Xiaohong Ji , Zhen Wang , Zhifeng Gao , Hang Zheng , Linfeng Zhang , Guolin Ke , Weinan E This is my paper
classification cs.LG cs.AI
keywords pretrainingmodelmolecularsizeuni-mol2levelmodelsparameters
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

In recent years, pretraining models have made significant advancements in the fields of natural language processing (NLP), computer vision (CV), and life sciences. The significant advancements in NLP and CV are predominantly driven by the expansion of model parameters and data size, a phenomenon now recognized as the scaling laws. However, research exploring scaling law in molecular pretraining models remains unexplored. In this work, we present Uni-Mol2 , an innovative molecular pretraining model that leverages a two-track transformer to effectively integrate features at the atomic level, graph level, and geometry structure level. Along with this, we systematically investigate the scaling law within molecular pretraining models, characterizing the power-law correlations between validation loss and model size, dataset size, and computational resources. Consequently, we successfully scale Uni-Mol2 to 1.1 billion parameters through pretraining on 800 million conformations, making it the largest molecular pretraining model to date. Extensive experiments show consistent improvement in the downstream tasks as the model size grows. The Uni-Mol2 with 1.1B parameters also outperforms existing methods, achieving an average 27% improvement on the QM9 and 14% on COMPAS-1D dataset.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SC3: The Multi-Solvent Solubility Challenge and Benchmark

    physics.chem-ph 2026-06 unverdicted novelty 7.0

    SC3 is a new multi-solvent solubility benchmark with reproducible curation, Gold/Silver/Bronze tiers, and metrics revealing current models perform 5 times worse than the recalibrated aleatoric limit of 0.106 log S.

  2. DPA4: Pushing the Accuracy-Cost Frontier of Interatomic Potentials with EMFA SO(2) Convolution

    physics.chem-ph 2026-06 unverdicted novelty 7.0

    DPA4 is a new SE(3)-equivariant interatomic potential with EMFA SO(2) convolution that sets new accuracy-cost records on Matbench Discovery and SPICE benchmarks using fewer parameters than prior models.

  3. S1-Omni: A Unified Multimodal Reasoning Model for Scientific Understanding, Prediction, and Generation

    cs.AI 2026-07 conditional novelty 6.0

    A single 32B multimodal model with task-specific decoders handles roughly 200 scientific tasks across molecules, materials, proteins, spectra, and images, and outperforms general LLMs on most of 66 evaluated tasks.

  4. A Unified Generative Framework for Scalable Chemical Reaction Network Exploration

    physics.chem-ph 2026-06 unverdicted novelty 6.0

    ByteCRN combines reaction enumeration with a unified generative rectified flow model for transition state generation and validation to achieve 10-100x acceleration and prune 70-90% of reactions in CRN exploration.

  5. Suiren-1.0 Technical Report: A Family of Molecular Foundation Models

    physics.chem-ph 2026-03 unverdicted novelty 6.0

    Suiren-1.0 is a family of three molecular foundation models (Base, Dimer, ConfAvg) pre-trained on 70M+ DFT samples and distilled to achieve claimed state-of-the-art performance on quantum property prediction tasks fro...

  6. Retrieval-Augmented Multimodal Learning for Enzyme-Substrate Interaction Prediction Under Low-Homology Shift

    cs.LG 2026-06 unverdicted novelty 5.0

    RAMMESI combines multimodal cross-attention with retrieval of neighboring enzymes to improve ESI prediction robustness under sequence-identity distribution shift on two benchmarks.

  7. MSAlign: Aligning Molecule and Mass Spectra Foundation Models for Metabolite Identification

    cs.LG 2026-05 conditional novelty 5.0

    MSAlign aligns frozen DreaMS and ChemBERTa models with MLPs and candidate-based contrastive learning to outperform prior methods on molecule retrieval from MS/MS spectra while quantifying distribution shift in data splits.

  8. DrugPlayGround: Benchmarking Large Language Models and Embeddings for Drug Discovery

    cs.LG 2026-02 unverdicted novelty 5.0

    DrugPlayGround is a new benchmark framework for evaluating LLMs on text-based descriptions of physiochemical drug characteristics, synergism, drug-protein interactions, and physiological responses.