Lie-Algebra Attention defines tokens as bare matrix Lie group elements and computes attention scores from the negative squared algebra norm of the relative pose log(g_i^{-1} g_j).
hub
Equiformer: Equivariant graph attention transformer for 3d atomistic graphs
21 Pith papers cite this work, alongside 65 external citations. Polarity classification is still indexing.
21
Pith papers citing it
65
external citations · Pith
abstract
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like translational invariance and rotational equivariance are considered. In this paper, we demonstrate that Transformers can generalize well to 3D atomistic graphs and present Equiformer, a graph neural network leveraging the strength of Transformer architectures and incorporating SE(3)/E(3)-equivariant features based on irreducible representations (irreps). First, we propose a simple and effective architecture by only replacing original operations in Transformers with their equivariant counterparts and including tensor products. Using equivariant operations enables encoding equivariant information in channels of irreps features without complicating graph structures. With minimal modifications to Transformers, this architecture has already achieved strong empirical results. Second, we propose a novel attention mechanism called equivariant graph attention, which improves upon typical attention in Transformers through replacing dot product attention with multi-layer perceptron attention and including non-linear message passing. With these two innovations, Equiformer achieves competitive results to previous models on QM9, MD17 and OC20 datasets.
hub tools
citation-role summary
background 1
method 1
citation-polarity summary
representative citing papers
EAGOR reformulates embodied 360-degree directional reasoning as recursive Bayesian estimation on a spherical manifold using spherical harmonics, achieving training-free, rotation-equivariant target tracking.
EpiFormer improves epitope prediction F1 score by over 40% via early-fusion cross-attention in GNN layers and sparsity-aware objectives, while recovering known biology as emergent behavior.
DPA4 is a new SE(3)-equivariant interatomic potential with EMFA SO(2) convolution that sets new accuracy-cost records on Matbench Discovery and SPICE benchmarks using fewer parameters than prior models.
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperforming prior samplers in 3D and 4D.
An O(L^3) algorithm computes contracted Clebsch-Gordan tensor products for equivariant ML potentials using a structured angular grid and spherical Poisson bracket to handle parity-odd terms at fixed CP rank.
A composition-weighted symbolic regression framework learns analytical expressions and elemental weightings from composition to predict materials properties with accuracy competitive to black-box models while producing explicit, constraint-enforcing formulas.
h-MINT improves ligand-protein binding affinity prediction by 2-4% and virtual screening metrics by 1-3% via overlapping fragment tokenization and hierarchical modeling.
A novel tri-plane equivariant volumetric grasp model adapts GIGA and IGD planners with flow matching and deformable attention to achieve higher real-time performance than non-equivariant baselines.
MFASSL adds mirror-paired views, a lightweight Mirror-Fusion Attention module, and reflection-consistency losses to improve SSL on bilateral data with ~2.7% extra parameters.
ElemeNet is a unified ML software package for molecular property prediction across elements 1-100 with built-in uncertainty quantification and competitive benchmarks on diverse chemistry datasets.
EvoStruct integrates evolutionary priors from a protein language model with structural priors from an E(3)-equivariant GNN to raise amino acid recovery by 16% and diversity by 2.3x on CHIMERA-Bench while cutting perplexity 43%.
Fine-tuning CHGNet on OMDB data and performing site substitution on QMOF prototypes yields novel highly magnetic MOFs.
EQUIMF is a unified equivariant framework that jointly generates discrete topologies and continuous geometries in molecular graphs via synchronized MeanFlow dynamics for efficient few-step sampling.
Bayesian E(3)-equivariant MLPs with joint energy-force NLL loss achieve competitive accuracy while enabling uncertainty-guided active learning, OOD detection, and calibration.
HASGO combines harmony search with universal MLIPs and multi-head replay fine-tuning to locate operando surface reconstructions, demonstrated by identifying the square-pyramidal O5 subsurface motif on Ag(100) during ethylene epoxidation.
Music102 integrates D12-equivariance into a transformer for chord progression accompaniment and shows gains over Music101 on POP909.
REViT introduces a discrete roto-reflection equivariant convolutional vision transformer claimed to outperform prior equivariant networks on image classification.
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
citing papers explorer
-
The Token Is a Group Element: On Lie-Algebra Attention over Matrix Lie Groups
Lie-Algebra Attention defines tokens as bare matrix Lie group elements and computes attention scores from the negative squared algebra norm of the relative pose log(g_i^{-1} g_j).
-
EAGOR: Embodied Reasoning in Omni-direction
EAGOR reformulates embodied 360-degree directional reasoning as recursive Bayesian estimation on a spherical manifold using spherical harmonics, achieving training-free, rotation-equivariant target tracking.
-
EpiFormer: Learning Antigen-Antibody Interactions for Epitope Prediction via Geometric Deep Learning
EpiFormer improves epitope prediction F1 score by over 40% via early-fusion cross-attention in GNN layers and sparsity-aware objectives, while recovering known biology as emergent behavior.
-
DPA4: Pushing the Accuracy-Cost Frontier of Interatomic Potentials with EMFA SO(2) Convolution
DPA4 is a new SE(3)-equivariant interatomic potential with EMFA SO(2) convolution that sets new accuracy-cost records on Matbench Discovery and SPICE benchmarks using fewer parameters than prior models.
-
TriSearch: Learning to Optimize Triangulations via Bistellar Flips
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperforming prior samplers in 3D and 4D.
-
Fast contracted Clebsch--Gordan tensor products for equivariant graph neural networks
An O(L^3) algorithm computes contracted Clebsch-Gordan tensor products for equivariant ML potentials using a structured angular grid and spherical Poisson bracket to handle parity-odd terms at fixed CP rank.
-
Composition-Weighted Symbolic Regression for General-Purpose Property Prediction
A composition-weighted symbolic regression framework learns analytical expressions and elemental weightings from composition to predict materials properties with accuracy competitive to black-box models while producing explicit, constraint-enforcing formulas.
-
h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network
h-MINT improves ligand-protein binding affinity prediction by 2-4% and virtual screening metrics by 1-3% via overlapping fragment tokenization and hierarchical modeling.
-
Equivariant Volumetric Grasping
A novel tri-plane equivariant volumetric grasp model adapts GIGA and IGD planners with flow matching and deformable attention to achieve higher real-time performance than non-equivariant baselines.
-
Mirror-Fusion Attention for Reflection-Aware Self-Supervised Representation Learning
MFASSL adds mirror-paired views, a lightweight Mirror-Fusion Attention module, and reflection-consistency losses to improve SSL on bilateral data with ~2.7% extra parameters.
-
ElemeNet: Multiscale Molecular Machine Learning with Uncertainty Quantification Across the Periodic Table
ElemeNet is a unified ML software package for molecular property prediction across elements 1-100 with built-in uncertainty quantification and competitive benchmarks on diverse chemistry datasets.
-
EvoStruct: Bridging Evolutionary and Structural Priors for Antibody CDR Design via Protein Language Model Adaptation
EvoStruct integrates evolutionary priors from a protein language model with structural priors from an E(3)-equivariant GNN to raise amino acid recovery by 16% and diversity by 2.3x on CHIMERA-Bench while cutting perplexity 43%.
-
Generation of magnetic metal-organic frameworks
Fine-tuning CHGNet on OMDB data and performing site substitution on QMOF prototypes yields novel highly magnetic MOFs.
-
Equivariant Efficient Joint Discrete and Continuous MeanFlow for Molecular Graph Generation
EQUIMF is a unified equivariant framework that jointly generates discrete topologies and continuous geometries in molecular graphs via synchronized MeanFlow dynamics for efficient few-step sampling.
-
Bayesian E(3)-Equivariant Interatomic Potential with Iterative Restratification of Many-body Message Passing
Bayesian E(3)-equivariant MLPs with joint energy-force NLL loss achieve competitive accuracy while enabling uncertainty-guided active learning, OOD detection, and calibration.
-
Scalable Prediction of Complex Surface Reconstructions under Operating Conditions via Harmony-Search-Based Global Optimization
HASGO combines harmony search with universal MLIPs and multi-head replay fine-tuning to locate operando surface reconstructions, demonstrated by identifying the square-pyramidal O5 subsurface motif on Ag(100) during ethylene epoxidation.
-
Music102: An $D_{12}$-equivariant transformer for chord progression accompaniment
Music102 integrates D12-equivariance into a transformer for chord progression accompaniment and shows gains over Music101 on POP909.
-
REViT: Roto-reflection Equivariant Convolutional Vision Transformer
REViT introduces a discrete roto-reflection equivariant convolutional vision transformer claimed to outperform prior equivariant networks on image classification.
-
Six Open Questions in Machine-Learned Interatomic Potential Foundation Models
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
- AgForce Enables Antigen-conditioned Generative Antibody Design
- ConTact: Contact-First Antibody CDR Design via Explicit Interface Reasoning