Lie-Algebra Attention defines tokens as bare matrix Lie group elements and computes attention scores from the negative squared algebra norm of the relative pose log(g_i^{-1} g_j).
hub
Equiformer: Equivariant graph attention transformer for 3d atomistic graphs
21 Pith papers cite this work, alongside 65 external citations. Polarity classification is still indexing.
21
Pith papers citing it
65
external citations · Pith
abstract
Despite their widespread success in various domains, Transformer networks have yet to perform well across datasets in the domain of 3D atomistic graphs such as molecules even when 3D-related inductive biases like translational invariance and rotational equivariance are considered. In this paper, we demonstrate that Transformers can generalize well to 3D atomistic graphs and present Equiformer, a graph neural network leveraging the strength of Transformer architectures and incorporating SE(3)/E(3)-equivariant features based on irreducible representations (irreps). First, we propose a simple and effective architecture by only replacing original operations in Transformers with their equivariant counterparts and including tensor products. Using equivariant operations enables encoding equivariant information in channels of irreps features without complicating graph structures. With minimal modifications to Transformers, this architecture has already achieved strong empirical results. Second, we propose a novel attention mechanism called equivariant graph attention, which improves upon typical attention in Transformers through replacing dot product attention with multi-layer perceptron attention and including non-linear message passing. With these two innovations, Equiformer achieves competitive results to previous models on QM9, MD17 and OC20 datasets.
hub tools
citation-role summary
background 1
method 1
citation-polarity summary
representative citing papers
EAGOR reformulates embodied 360-degree directional reasoning as recursive Bayesian estimation on a spherical manifold using spherical harmonics, achieving training-free, rotation-equivariant target tracking.
EpiFormer improves epitope prediction F1 score by over 40% via early-fusion cross-attention in GNN layers and sparsity-aware objectives, while recovering known biology as emergent behavior.
TriSearch is an RL framework that optimizes triangulations of polytopes using bistellar flips with a circuit-supported subtriangulation action representation, generalizing zero-shot to larger instances and outperforming prior samplers in 3D and 4D.
An O(L^3) algorithm computes contracted Clebsch-Gordan tensor products for equivariant ML potentials using a structured angular grid and spherical Poisson bracket to handle parity-odd terms at fixed CP rank.
A composition-weighted symbolic regression framework learns analytical expressions and elemental weightings from composition to predict materials properties with accuracy competitive to black-box models while producing explicit, constraint-enforcing formulas.
h-MINT improves ligand-protein binding affinity prediction by 2-4% and virtual screening metrics by 1-3% via overlapping fragment tokenization and hierarchical modeling.
A novel tri-plane equivariant volumetric grasp model adapts GIGA and IGD planners with flow matching and deformable attention to achieve higher real-time performance than non-equivariant baselines.
MFASSL adds mirror-paired views, a lightweight Mirror-Fusion Attention module, and reflection-consistency losses to improve SSL on bilateral data with ~2.7% extra parameters.
ElemeNet is a unified ML software package for molecular property prediction across elements 1-100 with built-in uncertainty quantification and competitive benchmarks on diverse chemistry datasets.
EvoStruct integrates evolutionary priors from a protein language model with structural priors from an E(3)-equivariant GNN to raise amino acid recovery by 16% and diversity by 2.3x on CHIMERA-Bench while cutting perplexity 43%.
Fine-tuning CHGNet on OMDB data and performing site substitution on QMOF prototypes yields novel highly magnetic MOFs.
EQUIMF is a unified equivariant framework that jointly generates discrete topologies and continuous geometries in molecular graphs via synchronized MeanFlow dynamics for efficient few-step sampling.
Bayesian E(3)-equivariant MLPs with joint energy-force NLL loss achieve competitive accuracy while enabling uncertainty-guided active learning, OOD detection, and calibration.
HASGO combines harmony search with universal MLIPs and multi-head replay fine-tuning to locate operando surface reconstructions, demonstrated by identifying the square-pyramidal O5 subsurface motif on Ag(100) during ethylene epoxidation.
Music102 integrates D12-equivariance into a transformer for chord progression accompaniment and shows gains over Music101 on POP909.
REViT introduces a discrete roto-reflection equivariant convolutional vision transformer claimed to outperform prior equivariant networks on image classification.
This perspective article develops a definition of foundational MLIPs and poses six open questions that the authors believe will define future research in machine-learned interatomic potentials.
citing papers explorer
-
h-MINT: Modeling Pocket-Ligand Binding with Hierarchical Molecular Interaction Network
h-MINT improves ligand-protein binding affinity prediction by 2-4% and virtual screening metrics by 1-3% via overlapping fragment tokenization and hierarchical modeling.