Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

InertialAR turns 3D molecules into canonical token sequences, letting an autoregressive transformer generate valid, stable molecules at state-of-the-art rates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 06:57 UTC pith:VHNZRC53

load-bearing objection Impressive empirical results and a clever method assembly, but the inference-time frame protocol is unspecified and needs to be fixed before the invariance claims hold. the 3 major comments →

arxiv 2510.27497 v2 pith:VHNZRC53 submitted 2025-10-31 cs.LG cs.AI

InertialAR: Autoregressive 3D Molecule Generation with Inertial Frames

Haorui Li , Weitao Du , Yuqiang Li , Hongyu Guo , Shengchao Liu This is my paper
classification cs.LG cs.AI
keywords autoregressive generation3D molecule generationcanonical tokenizationinertial framegeometric positional encodingdiffusion lossclass-conditioned generationB3LYP
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that 3D molecule generation can be cast as autoregressive next-token prediction without discretizing geometry. It claims a canonical tokenization—principal-axis (inertial) alignment plus deterministic canonical atom ordering—maps any molecule to a unique, SE(3)- and permutation-invariant sequence of atom tokens. On top of this it builds a geometry-aware Transformer (GeoRoPE) and a hierarchical decoder that predicts atom type by cross-entropy and 3D coordinates by diffusion loss. If the claims hold, a standard Transformer architecture matches or beats diffusion baselines on QM9, GEOM-Drugs, and a 1M-molecule B3LYP subset, and gives strong class-conditional control. A careful reader would care because it suggests a scalable, variable-length path to molecule generation that avoids expensive iterative sampling.

Core claim

The central discovery claimed is that the two obstructions to AR molecular generation—SE(3) symmetry and atom-index permutation—can both be removed by preprocessing rather than by equivariant architecture. After centering on the center of mass and diagonalizing the inertia tensor, the paper fixes residual axis-direction ambiguity with a fourth anchor atom, then reorders atoms by a deterministic canonical rule; the result is a unique 1D sequence of (atom type, 3D coordinate) tokens. The model then predicts the next token with a hierarchical head: cross-entropy for type, denoising-based diffusion loss for coordinates, with attention made geometry-aware by rotary 3D positional encoding plus a l

What carries the argument

The load-bearing object is the canonical tokenization: (1) an inertial frame built from the center of mass and eigenvectors of the inertia tensor, with signs fixed by requiring a fourth, off-plane anchor atom to land in the first x-y quadrant, and (2) a deterministic canonical atom-ordering rule that breaks atom-index permutation symmetry. Around this, GeoRoPE combines a 3D rotary positional encoding (whose attention inner product depends only on coordinate differences) with a Nyström low-rank encoding that injects approximate pairwise distances into attention. A hierarchical AR head completes the machinery: atom type via cross-entropy, continuous coordinates via diffusion loss on a Gaussian

Load-bearing premise

The load-bearing premise is that the canonical tokenization, defined on the complete molecule, remains the right representation for the partially generated molecule at each autoregressive step—but the paper never states which frame is used during sequential decoding, and a partial molecule's inertial frame can differ from the full molecule's by a rotation.

What would settle it

Decode a known molecule starting from a random rotation of the first atom without re-aligning to the canonical frame, and compare validity/stability to decoding from the canonical frame; a large drop would confirm the train/test frame mismatch. Separately, compute the inertia-tensor eigenvectors for a linear molecule like CO2 and check whether the fourth-node rule can even be applied; if the tokenization is not unique for such molecules, the invariance claim is conditional at best.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Autoregressive transformers, not just diffusion models, can generate chemically valid 3D molecules, including molecule stability that diffusion struggles with at scale.
  • Canonicalization moves SE(3) and permutation symmetry out of the network, so the backbone needs no equivariant layers; the same tokenization trick could extend other sequence models to 3D point clouds.
  • Variable-length generation falls out naturally from next-token prediction, avoiding diffusion's fixed-size constraints.
  • Class-conditioned generation with classifier-free guidance reaches high functional-group hit rates (83.3% average vs 25.7% for EDM), pointing to controllable molecule editing by tuning guidance scale.
  • On B3LYP-1M, the method more than doubles atom stability over EDM and raises molecule stability from 0.8% to 24.2%, evidence the approach scales to chemically diverse, million-scale data.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves unspecified which frame represents partially generated molecules at inference. A natural test is to compare decoding from a partial-molecule inertial frame against decoding in the full-molecule frame; if the latter is needed, the model must implicitly learn frame prediction, which is neither described nor evaluated.
  • Linear molecules (e.g., CO2) have no atom off the x-z and y-z planes, so the fourth-anchor sign disambiguation is undefined for them; this suggests the 'unique tokenization' claim is better interpreted as holding for non-linear, non-degenerate molecules, and symmetric molecules may map to several equivalent frames.
  • The GeoRoPE recipe—rotary encoding for relative position plus a low-rank Nyström distance term—is a general way to make attention geometry-aware, and would transfer to any 3D point-cloud generation task beyond molecules.
  • If the empirical gains are driven mainly by canonicalization rather than by the geometry-aware attention, a direct ablation (full model vs. no Nyström term vs. no RoPE-3D) would isolate which component carries the effect; the paper does not report this split.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes InertialAR, an autoregressive Transformer for 3D molecule generation. Molecules are tokenized by first aligning coordinates to a full-molecule inertial frame and fixing axis signs with a fourth atom, then ordering atoms by RDKit canonical ordering. Tokens are atom-type/cartesian-coordinate pairs; the architecture uses GeoRoPE (per-axis rotary positional encoding plus a Nyström approximation of pairwise RBF distances) and a hierarchical decoder that predicts atom type by cross-entropy and coordinates by diffusion loss. Experiments cover unconditional generation on QM9, GEOM-Drugs, and a 1M-molecule B3LYP subset, and class-conditional generation on QM9 using functional-group labels. The paper reports state-of-the-art results on 3 of 6 QM9/GEOM-Drugs metrics, all 4 B3LYP metrics, and all 5 conditional metrics.

Significance. If the mechanism is sound, the paper makes a useful empirical contribution: it shows that a Transformer-based autoregressive model with continuous-coordinate diffusion loss can compete with diffusion models on 3D molecule generation and scale to a million-molecule dataset. The evaluation uses external benchmarks, and the B3LYP results are notably strong. However, the paper's central theoretical claim — a unique, SE(3)- and permutation-invariant tokenization that supports left-to-right generation — is not fully specified and is at least partially contradicted by the text. The empirical numbers are therefore not yet backed by a complete algorithmic description. The significance is conditional on resolving the train/inference frame mismatch and the degeneracy issues.

major comments (3)
  1. [§3.1 vs. §3.3, Eq. (3) and Eq. (15)] The canonical tokenization is computed from the complete molecule: center of mass, inertia tensor, and RDKit ordering all use all N atoms (§3.1), but decoding is sequential, sampling c_{i+1} from p(c_{i+1}|t_{i+1}, h_i) where h_i encodes only the prefix a_{≤i} (§3.3, Eq. (15)). The paper never states which frame is used to represent a partially generated molecule at inference. If the partial molecule's own inertial frame is used, training prefixes live in the full-molecule frame while inference prefixes live in a rotated/translated frame, so the SE(3)-invariance argument for the tokenization does not transfer to the conditional distributions used in generation. If the full-molecule frame is assumed, the model must know future atoms to compute the frame, which is impossible in left-to-right decoding. The same problem applies to canonical ordering: RDKit canonical ranks depend on the compl
  2. [§3.1, Theorem 1 and Section D] The claimed uniqueness of the inertial-frame tokenization is contradicted by the text itself. The paper concedes that symmetric molecules such as CO2 and CH4 have degenerate inertia eigenvalues, making the inertial frame non-unique. Moreover, Theorem 1 requires a fourth node not on the y-z or x-z planes; for linear molecules (including the paper's own CO2 example) no such atom exists, and for any two-atom or collinear prefix the inertia tensor is degenerate. The appendix proof (Theorem 2) also changes the condition to a point coplanar/nonplanar with the basis vectors and contains apparent typos in Eq. (32)–(33). As stated, the theorem does not establish a unique frame for all molecules in the evaluation datasets, so the strong claim of a unique SE(3)- and permutation-invariant token sequence is unsupported. The authors should either restrict the claim to generic asymmetric molecules or p
  3. [§4.1, Table 2] The abstract and conclusion claim state-of-the-art performance on B3LYP, but Table 2 compares InertialAR against only two baselines (EDM and Geo2Seq). With only one diffusion baseline and one autoregressive baseline, the claim that InertialAR is state-of-the-art across all 4 B3LYP metrics is not established. This is a claim about the empirical headline result, not just presentation. The authors should either soften the wording or add more baselines (including recent 3D generation models that can be trained on the 1M subset).
minor comments (6)
  1. [Abstract vs. §1 and §3] The abstract says 8 of 10 unconditional metrics; the introduction and contributions say 7 of 10. The table counts also differ from the body text. Please make the counts consistent.
  2. [§3.2, Eq. (14)] Eq. (14) writes the attention score as exactly RBF(||c_i - c_j||), but z_i^T z_j is only the Nyström approximation of the RBF kernel. The equality should be an approximation, and the number of anchors m and the RBF width σ are not specified anywhere; these are free choices that affect the reported results.
  3. [§3.1 and §4] The terminology 'SE(3)-equivariance' is used for a tokenization that is intended to be invariant. Please distinguish invariance of the sequence from equivariance of coordinates; this affects the reader's understanding of what the model learns.
  4. [Appendix D] There are inconsistencies between Theorem 1 in the main text and Theorem 2 in the appendix ('not on the y-z plane or x-z plane' vs. 'coplanar with the three basis vectors'), and the proof has typographical issues in the matrix equations. These should be corrected and the theorem restated cleanly.
  5. [§4] Reproducibility details are missing: model dimensions, number of layers, diffusion noise schedule, number of denoising steps at inference, CFG guidance scale s per experiment, and the exact procedure for selecting the B3LYP-1M training subset. The code link is welcome, but the paper should state these values.
  6. [§4.2, Table 3] The 'Hit Rate' metric is not formally defined. It should state whether it is computed over all generated molecules or only valid molecules, and how the functional-group pattern is matched to the generated structure. This is important because the headline conditional claim is based on this metric.

Circularity Check

0 steps flagged

No circular derivation; external benchmarks carry the claims; only a minor same-group attribution (Guo et al. 2025) is present and it is not load-bearing.

full rationale

InertialAR's headline results are empirical numbers against external baselines on QM9, GEOM-Drugs, and B3LYP, so there is no reduction of a prediction to a fitted input. The canonical tokenization is defined constructively in §3.1 (inertial frame construction, RDKit-based reordering), and the uniqueness theorem is stated and proved in Appendix D rather than imported from prior work. The inertial-frame idea is credited to Guo et al. 2025, whose author list overlaps with the present paper (Hongyu Guo and Shengchao Liu), but the main text re-derives the construction and does not use that citation as the evidence for the method; this is a minor same-group attribution, not a load-bearing self-citation. The hierarchical AR + diffusion-loss scheme follows Cheng et al. 2025b and Li et al. 2024a, which are external or non-overlapping, and the loss equations (16)-(18) are stated directly. The paper's most significant internal gap — the frame protocol for partially generated molecules at inference is not specified, so the claimed SE(3)-invariance may not transfer from full-molecule canonicalization to sequential decoding — is a correctness/executability concern, not a circular reduction: no equation defines the next-token distribution in terms of the full-molecule canonical frame in a way that forces the empirical scores. The paper also honestly concedes non-unique inertial frames for symmetric molecules such as CO2 and CH4, which limits the uniqueness claim but is a limitation rather than circularity. No circular step meets the evidence bar of an equation equal to its own input by construction or a fitted parameter renamed as a prediction; the score reflects only the minor same-group self-citation.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

No new physical entities are postulated; the 'fourth node' is a tokenizer-internal construct with no external falsifiable handle, and the 'Molecule Class ID' scheme re-labels existing molecules rather than postulating new ones. The paper's central claims (unique SE(3)- and permutation-invariant tokenization; SOTA generation at scale) rest on canonicalization properties that fail or are unproven for symmetric/linear molecules (acknowledged for degenerate eigenvalues in §3.1; the fourth-node rule is proven only against 180-degree axis-reflection competitors in Appendix D, not against arbitrary rotations within degenerate eigenspaces), on an implicit train/inference frame-consistency assumption that is never stated, and on several unreported hyperparameters (sigma, m, theta, CFG scale, denoising steps) that the empirical scores depend on.

free parameters (6)
  • Nyström RBF width sigma = not reported
    Appears in K_ij = exp(-||c_i - c_j||^2 / 2 sigma^2) in §3.2; controls the distance kernel injected into attention and must be chosen by hand; no value given.
  • Number of Nyström anchors m = not reported
    Anchor count for the low-rank approximation in §3.2; anchors are sampled from the molecule's own atoms but the sampling rule is not specified.
  • RoPE-3D frequency schedule theta = not reported
    Frequency parameters for the x/y/z rotary blocks in Eq. (6); the theta schedule is not given, although the coordinate scale (Ångström) makes it crucial.
  • CFG guidance scale s = not reported (per-class)
    §3.3 and §4.2: CFG is applied to both noise and logits; conditional results depend on s, whose per-class values and search protocol are not reported (Fig. 3 shows editing by incrementing s by 0.3).
  • Diffusion denoising steps at inference = not reported
    §3.3: coordinates are generated by iterative denoising; the step count and noise scheduler are not stated, so the claimed efficiency advantage over diffusion models cannot be checked.
  • B3LYP-1M training subset selection = unspecified
    §4.1: 'We use a subset of 1M molecules for training' with no sampling rule given, so the dataset difficulty is not reproducible.
axioms (6)
  • domain assumption Nyström anchor Gram matrix A is positive definite so the Cholesky factor L exists
    §3.2 and Appendix C: requires m anchor RBF vectors with positive eigenvalues; degenerate anchor sets (near-duplicate atoms) would break the Cholesky decomposition, and the paper does not justify this for all molecules.
  • domain assumption RDKit canonical ordering is deterministic and stable
    §3.1 Step 2: the atom order is produced by RDKit's canonical ranking with a tie-breaking perturbation; the paper assumes the same molecule always maps to the same order across runs and versions.
  • ad hoc to paper The fourth-node sign rule resolves frame sign ambiguity for all molecules
    §3.1 Theorem 1 requires a fourth atom not on the y-z or x-z planes; for linear molecules (the paper's own CO2 example) all atoms lie on at least two of the coordinate planes, so no such node exists. Degenerate inertia tensors (CO2, CH4) additionally leave the eigenbasis itself non-unique.
  • ad hoc to paper The full-molecule inertial frame is available at generation time
    §3.1 computes the frame from all N atoms; §3.3 generates atoms one at a time; the paper never states how a partial prefix is framed at inference, so training and test token distributions are not shown to match.
  • domain assumption Geometry-based bond-type assignment is valid
    §4.1: bond types are determined from molecular geometries based on pairwise atomic distances and atom identities (standard protocol from Hoogeboom et al. 2022); reliability at the B3LYP scale (broader chemistry, molecular weight up to 1000) is assumed.
  • standard math Standard transformer and RoPE mathematics
    RoPE properties (R_m^T R_n = R_{n-m}) and the Nyström low-rank approximation are taken as given from prior literature (Su et al. 2024; Williams & Seeger 2000).

pith-pipeline@v1.3.0-alltime-deepseek · 21420 in / 29482 out tokens · 244976 ms · 2026-08-04T06:57:05.588257+00:00 · methodology

0 comments
read the original abstract

Transformer-based autoregressive models have emerged as a unifying paradigm across modalities such as text and images, but their extension to 3D molecule generation remains underexplored. The gap stems from two fundamental challenges: (1) how to tokenize molecules into a canonical 1D sequence of tokens that is invariant to both SE(3) transformations and atom index permutations, and (2) how to design an architecture capable of modeling hybrid atom-based tokens that couple discrete atom types with continuous 3D coordinates. To address these challenges, we introduce InertialAR. It first performs generation-oriented canonical tokenization by aligning each molecule to a canonical inertial frame and reordering atoms, thereby converting arbitrary 3D structures into a unique, SE(3)- and permutation-invariant sequence of tokens for autoregressive generation. Built upon this canonical tokenization, we propose geometric positional encoding (GeoPE), which endows Transformer attention with 3D geometric awareness. Finally, InertialAR utilizes a hierarchical autoregressive paradigm to decode the next atom, consecutively predicting the atom type and 3D coordinates via Diffusion Loss. Experimentally, InertialAR achieves state-of-the-art performance on 8 of the 10 evaluation metrics for unconditional generation across QM9, GEOM-Drugs, and B3LYP. Moreover, it significantly outperforms baselines in controllable generation for targeted chemical functionality, attaining state-of-the-art results across all 5 metrics. Code is available at github.com/HaoruiLi46/InertialAR.

Figures

Figures reproduced from arXiv: 2510.27497 by Haorui Li, Hongyu Guo, Shengchao Liu, Weitao Du, Yuqiang Li.

Figure 1
Figure 1. Figure 1: Overview of InertialAR: (a) canonical tokenization, (b) geometric rotary positional encoding (GeoRoPE), and (c) hierarchical autoregressive paradigm. Our Contributions. To address these challenges, we propose InertialAR, a novel AR model for 3D molecule generation. InertialAR rests on two key innovations. First, it leverages a canonical tok￾enization strategy that uses an inertial frame to align 3D molecul… view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of introducing a fourth node as the anchor node. We define the sign of the x-y-z axis to make sure that x4 is in the first quadrant, and there are four cases as illustrated in the four subfigures. More concretely, we introduce a two-step canonical tokenization, as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of molecule editing by tuning the CFG guidance scale s. atoms satisfying correct valency), Molecule Stability (proportion of molecules in which all atoms are stable), Validity (fraction of chemically valid molecules as verified by RDKit), and Uniqueness (fraction of non-duplicate molecules among generated samples). All metrics are computed following evaluation protocols established in prior w… view at source ↗
Figure 4
Figure 4. Figure 4: Overview of mapping 3D molecules to their Molecule Class IDs. The task is then to generate molecules conditioned on a specified functional group configuration. Concretely, we select the 5 most fre￾quent Molecule Class IDs as conditioning targets. In addition to the metrics used for unconditional genera￾tion, we introduce a critical new metric for class-conditional generation, Hit Rate, which measures the p… view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of existing SE(3)-equivariant graph neural networks and InertialAR. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Overview of how 3D molecules are mapped to their Molecule Class IDs. In chemistry and biology, class-conditional generation is highly useful, as ”molecule classes” can correspond to key attributes such as chemical functionality or physicochemical characteristics, en￾abling the targeted design or editing of molecules for drug discovery and materials science. How￾ever, commonly used datasets, such as QM9 and… view at source ↗
Figure 7
Figure 7. Figure 7: (a, b) show two potential rotational alignments between two coordinate systems (axes). (c, d) show that only one unique rotation is possible for four non-coplanar points. Proof. For three vectors, we can easily find a counter-example , as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Illustration of the geometric representation of a complex number. If we transform the matrix as follows:  a −b b a  = p a 2 + b 2 " √ a a2+b 2 √ −b a2+b 2 √ b a2+b 2 √ a a2+b 2 # = ||z||  cos(θ) − sin(θ) sin(θ) cos(θ)  . (52) Then we can see that this matrix is indeed the rotation transformation on the 2D plane, as shown in [PITH_FULL_IMAGE:figures/full_fig_p025_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation

    cs.LG 2025-12 unverdicted novelty 7.0

    NEAT achieves state-of-the-art 3D molecular generation on QM9 and GEOM-Drugs via a neighborhood-guided autoregressive set transformer that ensures atom-level permutation invariance and offers a significant speed advantage.

  2. Atomic Design Transformer: Scaffold-Conditioned 3D Molecule Generation via xTB-Reward Reinforcement Learning

    physics.comp-ph 2026-07 conditional novelty 6.0

    A plain causal transformer that tokenizes atom positions in local frames generates 3D molecules directly; RL against an xTB relaxation reward lifts topology-preserving valid yield from ~50% to ~95%.

Reference graph

Works this paper leans on

56 extracted references · 34 linked inside Pith · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Accurate structure prediction of biomolecular interactions with alphafold 3

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3. Nature, 630 0 (8016): 0 493--500, 2024

    2024

  3. [3]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    Crystal structure generation with autoregressive large language modeling

    Luis M Antunes, Keith T Butler, and Ricardo Grau-Crespo. Crystal structure generation with autoregressive large language modeling. Nature Communications, 15 0 (1): 0 1--16, 2024

    2024

  5. [5]

    Geom, energy-annotated molecular conformations for property prediction and molecular generation

    Simon Axelrod and Rafael G \'o mez-Bombarelli. Geom, energy-annotated molecular conformations for property prediction and molecular generation. Scientific Data, 9 0 (1): 0 185, 2022. doi:10.1038/s41597-022-01288-4. URL https://doi.org/10.1038/s41597-022-01288-4

    doi:10.1038/s41597-022-01288-4 2022

  6. [6]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    Pith/arXiv arXiv 2020

  8. [8]

    Stiefel flow matching for moment-constrained structure elucidation, 2025 a

    Austin Cheng, Alston Lo, Kin Long Kelvin Lee, Santiago Miret, and Alán Aspuru-Guzik. Stiefel flow matching for moment-constrained structure elucidation, 2025 a . URL https://arxiv.org/abs/2412.12540

    Pith/arXiv arXiv 2025

  9. [9]

    Cheng, Chong Sun, and Alán Aspuru-Guzik

    Austin H. Cheng, Chong Sun, and Alán Aspuru-Guzik. Scalable autoregressive 3d molecule generation, 2025 b . URL https://arxiv.org/abs/2505.13791

    Pith/arXiv arXiv 2025

  10. [10]

    Transformer-xl: Attentive language models beyond a fixed-length context

    Zihang Dai. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02860, 2019

    Pith/arXiv arXiv 1901

  11. [11]

    Taming transformers for high-resolution image synthesis, 2021

    Patrick Esser, Robin Rombach, and Björn Ommer. Taming transformers for high-resolution image synthesis, 2021. URL https://arxiv.org/abs/2012.09841

    Pith/arXiv arXiv 2021

  12. [12]

    Proxelgen: Generating proteins as 3d densities, 2025

    Felix Faltings, Hannes Stark, Regina Barzilay, and Tommi Jaakkola. Proxelgen: Generating proteins as 3d densities, 2025. URL https://arxiv.org/abs/2506.19820

    Pith/arXiv arXiv 2025

  13. [13]

    Unigem: A unified approach to generation and property prediction for molecules, 2025

    Shikun Feng, Yuyan Ni, Yan Lu, Zhi-Ming Ma, Wei-Ying Ma, and Yanyan Lan. Unigem: A unified approach to generation and property prediction for molecules, 2025. URL https://arxiv.org/abs/2410.10516

    Pith/arXiv arXiv 2025

  14. [14]

    Language models can generate molecules, materials, and protein binding sites directly in three dimensions as xyz, cif, and pdb files, 2023

    Daniel Flam-Shepherd and Alán Aspuru-Guzik. Language models can generate molecules, materials, and protein binding sites directly in three dimensions as xyz, cif, and pdb files, 2023. URL https://arxiv.org/abs/2305.05708

    Pith/arXiv arXiv 2023

  15. [15]

    Fragment and geometry aware tokenization of molecules for structure-based drug design using language models, 2024

    Cong Fu, Xiner Li, Blake Olson, Heng Ji, and Shuiwang Ji. Fragment and geometry aware tokenization of molecules for structure-based drug design using language models, 2024. URL https://arxiv.org/abs/2408.09730

    Pith/arXiv arXiv 2024

  16. [16]

    Directional message passing for molecular graphs, 2022

    Johannes Gasteiger, Janek Groß, and Stephan Günnemann. Directional message passing for molecular graphs, 2022. URL https://arxiv.org/abs/2003.03123

    Pith/arXiv arXiv 2022

  17. [17]

    Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules

    Niklas Gebauer, Michael Gastegger, and Kristof Sch\" u tt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alch\' e -Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceeding...

    2019

  18. [18]

    Assembleflow: Rigid flow matching with inertial frames for molecular assembly

    Hongyu Guo, Yoshua Bengio, and Shengchao Liu. Assembleflow: Rigid flow matching with inertial frames for molecular assembly. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=jckKNzYYA6

    2025

  19. [19]

    Deberta: Decoding-enhanced bert with disentangled attention

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654, 2020

    Pith/arXiv arXiv 2006

  20. [20]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. URL https://arxiv.org/abs/2207.12598

    Pith/arXiv arXiv 2022

  21. [21]

    Equivariant diffusion for molecule generation in 3d, 2022

    Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling. Equivariant diffusion for molecule generation in 3d, 2022. URL https://arxiv.org/abs/2203.17003

    Pith/arXiv arXiv 2022

  22. [22]

    Elucidating the design space of diffusion-based generative models, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models, 2022. URL https://arxiv.org/abs/2206.00364

    Pith/arXiv arXiv 2022

  23. [23]

    Rdkit: Open-source cheminformatics software

    Greg Landrum. Rdkit: Open-source cheminformatics software. 2016. URL https://github.com/rdkit/rdkit/releases/tag/Release_2016_09_4

    2016

  24. [24]

    Autoregressive image generation without vector quantization, 2024 a

    Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization, 2024 a . URL https://arxiv.org/abs/2406.11838

    Pith/arXiv arXiv 2024

  25. [25]

    Geometry informed tokenization of molecules for language model generation

    Xiner Li, Limei Wang, Youzhi Luo, Carl Edwards, Shurui Gui, Yuchao Lin, Heng Ji, and Shuiwang Ji. Geometry informed tokenization of molecules for language model generation. arXiv preprint arXiv:2408.10120, 2024 b

    Pith/arXiv arXiv 2024

  26. [26]

    Equiformer: Equivariant graph attention transformer for 3d atomistic graphs, 2023

    Yi-Lun Liao and Tess Smidt. Equiformer: Equivariant graph attention transformer for 3d atomistic graphs, 2023. URL https://arxiv.org/abs/2206.11990

    Pith/arXiv arXiv 2023

  27. [27]

    Beyond atoms: Enhancing molecular pretrained representations with 3d space modeling, 2025 a

    Shuqi Lu, Xiaohong Ji, Bohang Zhang, Lin Yao, Siyuan Liu, Zhifeng Gao, Linfeng Zhang, and Guolin Ke. Beyond atoms: Enhancing molecular pretrained representations with 3d space modeling, 2025 a . URL https://arxiv.org/abs/2503.10489

    Pith/arXiv arXiv 2025

  28. [28]

    Uni-3dar: Unified 3d generation and understanding via autoregression on compressed spatial tokens, 2025 b

    Shuqi Lu, Haowei Lin, Lin Yao, Zhifeng Gao, Xiaohong Ji, Weinan E, Linfeng Zhang, and Guolin Ke. Uni-3dar: Unified 3d generation and understanding via autoregression on compressed spatial tokens, 2025 b . URL https://arxiv.org/abs/2503.16278

    arXiv 2025

  29. [29]

    Pubchemqc b3lyp/6-31g*//pm6 dataset: the electronic structures of 86 million molecules using b3lyp/6-31g* calculations, 2023

    Maho Nakata and Toshiyuki Maeda. Pubchemqc b3lyp/6-31g*//pm6 dataset: the electronic structures of 86 million molecules using b3lyp/6-31g* calculations, 2023. URL https://arxiv.org/abs/2305.18454

    Pith/arXiv arXiv 2023

  30. [30]

    Scalable diffusion models with transformers, 2023

    William Peebles and Saining Xie. Scalable diffusion models with transformers, 2023. URL https://arxiv.org/abs/2212.09748

    Pith/arXiv arXiv 2023

  31. [31]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  32. [32]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. Advances in neural information processing systems, 20, 2007

    2007

  33. [33]

    Quantum chemistry structures and properties of 134 kilo molecules

    Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules. Scientific data, 1 0 (1): 0 1--7, 2014

    2014

  34. [34]

    Fuchs, Ingmar Posner, and Max Welling

    Victor Garcia Satorras, Emiel Hoogeboom, Fabian B. Fuchs, Ingmar Posner, and Max Welling. E(n) equivariant normalizing flows, 2022 a . URL https://arxiv.org/abs/2105.09016

    Pith/arXiv arXiv 2022

  35. [35]

    E(n) equivariant graph neural networks, 2022 b

    Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E(n) equivariant graph neural networks, 2022 b . URL https://arxiv.org/abs/2102.09844

    Pith/arXiv arXiv 2022

  36. [36]

    Equivariant message passing for the prediction of tensorial properties and molecular spectra

    Kristof Sch \"u tt, Oliver Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra. In International Conference on Machine Learning, pp.\ 9377--9388. PMLR, 2021

    2021

  37. [37]

    Schütt, Pieter-Jan Kindermans, Huziel E

    Kristof T. Schütt, Pieter-Jan Kindermans, Huziel E. Sauceda, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: A continuous-filter convolutional neural network for modeling quantum interactions, 2017. URL https://arxiv.org/abs/1706.08566

    Pith/arXiv arXiv 2017

  38. [38]

    Schütt, Oliver T

    Kristof T. Schütt, Oliver T. Unke, and Michael Gastegger. Equivariant message passing for the prediction of tensorial properties and molecular spectra, 2021. URL https://arxiv.org/abs/2102.03150

    Pith/arXiv arXiv 2021

  39. [39]

    Self-attention with relative position representations

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155, 2018

    Pith/arXiv arXiv 2018

  40. [40]

    Benchmarking graphormer on large-scale molecular modeling datasets, 2023

    Yu Shi, Shuxin Zheng, Guolin Ke, Yifei Shen, Jiacheng You, Jiyan He, Shengjie Luo, Chang Liu, Di He, and Tie-Yan Liu. Benchmarking graphormer on large-scale molecular modeling datasets, 2023. URL https://arxiv.org/abs/2203.04810

    Pith/arXiv arXiv 2023

  41. [41]

    Road to transformer upgrades: 4

    Jianlin Su. Road to transformer upgrades: 4. rotational positional encoding for 2d positions, May 2021. URL https://spaces.ac.cn/archives/8397

    2021

  42. [42]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 0 127063, 2024

    2024

  43. [43]

    Autoregressive model beats diffusion: Llama for scalable image generation, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation, 2024. URL https://arxiv.org/abs/2406.06525

    Pith/arXiv arXiv 2024

  44. [45]

    Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds, 2018 b

    Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation- and translation-equivariant neural networks for 3d point clouds, 2018 b . URL https://arxiv.org/abs/1802.08219

    Pith/arXiv arXiv 2018

  45. [46]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction, 2024. URL https://arxiv.org/abs/2404.02905

    Pith/arXiv arXiv 2024

  46. [47]

    Llama: Open and efficient foundation language models, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023. URL https://arxiv.org/abs/2302.13971

    Pith/arXiv arXiv 2023

  47. [48]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  48. [49]

    Midi: Mixed graph and 3d denoising diffusion for molecule generation

    Clement Vignac, Nagham Osman, Laura Toni, and Pascal Frossard. Midi: Mixed graph and 3d denoising diffusion for molecule generation. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp.\ 560--576. Springer, 2023

    2023

  49. [50]

    Using the nystr \"o m method to speed up kernel machines

    Christopher Williams and Matthias Seeger. Using the nystr \"o m method to speed up kernel machines. Advances in neural information processing systems, 13, 2000

    2000

  50. [51]

    Diffusion-based molecule generation with informative prior bridges, 2022

    Lemeng Wu, Chengyue Gong, Xingchao Liu, Mao Ye, and Qiang Liu. Diffusion-based molecule generation with informative prior bridges, 2022. URL https://arxiv.org/abs/2209.00865

    Pith/arXiv arXiv 2022

  51. [52]

    Geometric latent diffusion models for 3d molecule generation

    Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Ermon, and Jure Leskovec. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning, pp.\ 38592--38610. PMLR, 2023

    2023

  52. [53]

    Invariant tokenization of crystalline materials for language model enabled generation

    Keqiang Yan, Xiner Li, Hongyi Ling, Kenna Ashen, Carl Edwards, Raymundo Arr \'o yave, Marinka Zitnik, Heng Ji, Xiaofeng Qian, Xiaoning Qian, et al. Invariant tokenization of crystalline materials for language model enabled generation. Advances in Neural Information Processing Systems, 37: 0 125050--125072, 2024

    2024

  53. [54]

    Nystr \"o m method vs random fourier features: A theoretical and empirical comparison

    Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, and Zhi-Hua Zhou. Nystr \"o m method vs random fourier features: A theoretical and empirical comparison. Advances in neural information processing systems, 25, 2012

    2012

  54. [55]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should not add it explicitly Type <Return> for now, but then later remove the command n...

  55. [56]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@first@sw \@firstoftwo \@ifundefined NAT@b*@#2 \@firstoftwo @num @NAT@ctr \@secondoft...

  56. [57]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibsetup #1 @NAT@ctr @ @openbib .11em \@plus.33em \@minus.07em 4000 4000 `\.\@m @bibit...

    Pith/arXiv arXiv 2023