REVIEW 35 cited by
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations
read the original abstract
Equivariant Transformers such as Equiformer have demonstrated the efficacy of applying Transformers to the domain of 3D atomistic systems. However, they are limited to small degrees of equivariant representations due to their computational complexity. In this paper, we investigate whether these architectures can scale well to higher degrees. Starting from Equiformer, we first replace $SO(3)$ convolutions with eSCN convolutions to efficiently incorporate higher-degree tensors. Then, to better leverage the power of higher degrees, we propose three architectural improvements -- attention re-normalization, separable $S^2$ activation and separable layer normalization. Putting this all together, we propose EquiformerV2, which outperforms previous state-of-the-art methods on large-scale OC20 dataset by up to $9\%$ on forces, $4\%$ on energies, offers better speed-accuracy trade-offs, and $2\times$ reduction in DFT calculations needed for computing adsorption energies. Additionally, EquiformerV2 trained on only OC22 dataset outperforms GemNet-OC trained on both OC20 and OC22 datasets, achieving much better data efficiency. Finally, we compare EquiformerV2 with Equiformer on QM9 and OC20 S2EF-2M datasets to better understand the performance gain brought by higher degrees.
Forward citations
Cited by 35 Pith papers
-
A Priori Sampling of Transition States with Guided Diffusion
ASTRA reframes transition-state search as guided diffusion inference that samples the isodensity surface between metastable basins and converges to first-order saddles via score differences and physical forces.
-
Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models
OMat24 releases a new open dataset of 110M+ DFT calculations and EquiformerV2 models achieving SOTA on Matbench Discovery with F1>0.9 for stability and 20 meV/atom accuracy for formation energies.
-
EpiFormer: Learning Antigen-Antibody Interactions for Epitope Prediction via Geometric Deep Learning
EpiFormer improves epitope prediction F1 score by over 40% via early-fusion cross-attention in GNN layers and sparsity-aware objectives, while recovering known biology as emergent behavior.
-
DPA4: Pushing the Accuracy-Cost Frontier of Interatomic Potentials with EMFA SO(2) Convolution
DPA4 is a new SE(3)-equivariant interatomic potential with EMFA SO(2) convolution that sets new accuracy-cost records on Matbench Discovery and SPICE benchmarks using fewer parameters than prior models.
-
TriSearch: Learning to Optimize Triangulations via Bistellar Flips
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperformi...
-
Bridging Atomistic Simulation and Experimental Processing Timescales with Goal-Directed Deep Reinforcement Learning
An E(3)-equivariant deep RL framework lets an O2 agent discover kinetically plausible diffusion and dissociation pathways in disordered Si/a-SiO2 without hand-crafted reaction coordinates or collective variables.
-
Fast contracted Clebsch--Gordan tensor products for equivariant graph neural networks
An O(L^3) algorithm computes contracted Clebsch-Gordan tensor products for equivariant ML potentials using a structured angular grid and spherical Poisson bracket to handle parity-odd terms at fixed CP rank.
-
Knowing when to trust machine-learned interatomic potentials
PROBE recasts MLIP uncertainty quantification as selective classification by training a compact discriminative classifier on frozen per-atom backbone embeddings, yielding a reliability probability that tracks actual e...
-
Pushing the limits of unconstrained machine-learned interatomic potentials
Unconstrained non-equivariant and direct-force neural interatomic potentials scale to 730M parameters and match or beat equivariant state-of-the-art models on several atomistic benchmarks.
-
OmniMol: Transferring Particle Physics Knowledge to Molecular Dynamics with Point-Edge Transformers
OmniMol transfers a billion-jet pre-trained PET foundation model from HEP to molecular dynamics via an interaction-matrix attention bias, delivering strong performance on the oMol dataset with minimal fine-tuning and ...
-
Equivariant Volumetric Grasping
A novel tri-plane equivariant volumetric grasp model adapts GIGA and IGD planners with flow matching and deformable attention to achieve higher real-time performance than non-equivariant baselines.
-
Dyna-Mat: End-to-end benchmarking of foundation machine learning interatomic potentials in finite-temperature ensembles
Dyna-Mat-v1.0 benchmarks 15 foundation MLIPs on finite-T MD observables, finding average force-error correlation with RDF/VDOS but systematic pressure failures and near-Pareto optimality of latest cross-trained models.
-
E3DGS: Unified Geometric-Photometric Equivariance for 3D Gaussian Splatting via Color-as-Geometry Embedding
3D Gaussian view-dependent colors are repacked as 3×3 matrices so geometry and color rotate together, giving exact rotation-equivariant recognition and world modeling in 3DGS.
-
High-order tensor neural network for iteration-free structure relaxation
HotRelax is a high-order tensor message-passing neural network trained on paired unrelaxed-relaxed structures to predict relaxed crystal geometries in a single forward pass without force labels or iteration.
-
Toward Controllable Catalyst Inverse Design via Large-Scale Autoregressive Pretraining
A GPT-style model pretrained on 133M catalyst structures generates valid structures conditioned on categorical and continuous properties, achieving 98% structural validity and up to 4-fold screening efficiency gains.
-
CatalyticMLLM: A Graph-Text Multimodal Large Language Model for Catalytic Materials
A unified graph-text multimodal LLM integrates property prediction and inverse design for catalytic materials into one model and shared space, forming a closed-loop workflow.
-
TSAgent: An Agentic Workflow for Autonomous Transition State Search
TSAgent automates transition state searches at DFT accuracy via an agentic loop, reaching 83% success on 100 OC20NEB examples and 70% on 10 held-out cases versus 73% for human experts.
-
Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials
A new benchmark finds that state-of-the-art ML interatomic potentials struggle with compositional generalization, producing errors an order of magnitude higher on unseen molecular combinations than on training-like cases.
-
Selectivity- and Activity-Aware Catalyst Descriptors for CO$_2$ Hydrogenation on Alloy Nanocatalysts using Machine-Learned Force Fields
A facet-resolved adsorption energy distribution method with ML force fields identifies active and methanol-selective alloy nanocatalyst surfaces for CO2 hydrogenation.
-
Suiren-1.0 Technical Report: A Family of Molecular Foundation Models
Suiren-1.0 is a family of three molecular foundation models (Base, Dimer, ConfAvg) pre-trained on 70M+ DFT samples and distilled to achieve claimed state-of-the-art performance on quantum property prediction tasks fro...
-
Faster Molecular Dynamics with Neural Network Potentials via Distilled Multiple Time-Stepping and Non-Conservative Forces
Distilled non-conservative force models in a multi-time-step integrator accelerate neural-network-potential molecular dynamics by up to 5.6x on tested systems without degrading sampling accuracy.
-
UBio-MolFM: A Universal Molecular Foundation Model for Bio-Systems
UBio-MolFM achieves ab initio-level fidelity on large out-of-distribution biomolecular systems using a new multi-fidelity dataset, E2Former-V2 architecture, and three-stage curriculum learning.
-
From Evaluation to Design: Using Potential Energy Surface Smoothness Metrics to Guide Machine Learning Interatomic Potential Architectures
A bond-deformation benchmark plus a force-smoothness metric is proposed to detect PES artifacts and guide MLIP architecture design, with improvements shown on a new Transformer-style model.
-
Global Plane Waves From Local Gaussians: Periodic Charge Densities in a Blink
ELECTRAFI predicts periodic electron densities by analytically Fourier-transforming a Gaussian mixture, reaching near-SOTA accuracy with up to 633× faster inference and ~20% end-to-end DFT speedups.
-
E2Former-V2: On-the-Fly Equivariant Attention with Linear Activation Memory
E2Former-V2 combines SO(2) sparsification and a fused streaming Triton kernel to cut equivariant attention memory to linear in system size, claiming ~20x faster kernels and 100k-atom inference.
-
Platonic Transformers: A Solid Choice For Equivariance
Platonic Transformers achieve exact equivariance to translations plus discrete Platonic-solid rotations by lifting features into multiple reference frames and sharing one RoPE attention across them, with a linear-time...
-
Bayesian E(3)-Equivariant Interatomic Potential with Iterative Restratification of Many-body Message Passing
Bayesian E(3)-equivariant MLPs with joint energy-force NLL loss achieve competitive accuracy while enabling uncertainty-guided active learning, OOD detection, and calibration.
-
Open Materials 2024 (OMat24) Inorganic Materials Dataset and Models
OMat24 provides over 110 million DFT calculations and EquiformerV2 models that reach state-of-the-art performance on material stability and formation energy prediction.
-
CatalyticMLLM: A Graph-Text Multimodal Large Language Model for Catalytic Materials
CatalyticMLLM unifies property prediction and inverse design for catalytic materials inside one graph-text multimodal LLM and reports better performance than decoupled baselines.
-
CatalyticMLLM: A Graph-Text Multimodal Large Language Model for Catalytic Materials
QE-Catalytic-V2 unifies property prediction and inverse design of catalytic materials inside one graph-text multimodal LLM and reports better performance than decoupled baselines on relaxed-energy prediction and struc...
-
Benchmarking Chemically Scalable Machine-Learning Interatomic Potentials for Large-Scale Simulations of Multicomponent Alloys
GRACE-FS is generally more accurate and transferable while UNEP-v1 is much faster and supports uncertainty-aware multi-million-atom shock simulations of multicomponent alloys.
-
Music102: An $D_{12}$-equivariant transformer for chord progression accompaniment
Music102 integrates D12-equivariance into a transformer for chord progression accompaniment and shows gains over Music101 on POP909.
-
REViT: Roto-reflection Equivariant Convolutional Vision Transformer
REViT introduces a discrete roto-reflection equivariant convolutional vision transformer claimed to outperform prior equivariant networks on image classification.
-
Benchmarking Chemically Scalable Machine-Learning Interatomic Potentials for Large-Scale Simulations of Multicomponent Alloys
GRACE MLIPs train faster and predict alloy properties more accurately than NEP, but NEP's 60-fold speed advantage enables reliable million-atom simulations of shock propagation when paired with ensemble uncertainty qu...
-
Six Open Questions in Machine-Learned Interatomic Potential Foundation Models
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.