Pith. sign in

REVIEW 3 major objections 5 minor 12 references

Do we need equivariant models for molecule generation?

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Rotation-augmented CNNs can learn molecular equivariance, but matching explicit symmetry in generation takes a 111M-parameter model.

desk verdict A useful empirical map of when augmented CNNs match equivariant models, but the rotation operator is unspecified and the equivariant baseline is too small to support the headline. read the letter →

arxiv 2507.09753 v1 pith:Q3ZT63NF submitted 2025-07-13 cs.LG q-bio.QM

Ewa M. Nowara , Joshua Rackers , Patricia Suriana , Pan Kessel , Max Shen , Andrew Martin Watkins , Michael Maser This is my paper
classification cs.LGq-bio.QM
keywords equivariancerotationaugmentationmoleculegeneration3Dconvolutionalnetworksvoxelgridswalk-jumpsamplinglearnedgenerative
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Most 3D molecular generative models build rotation symmetry directly into their architecture with equivariant graph neural networks, on the assumption that explicit equivariance is essential, even though such models are complex, hard to train, and scale poorly. This paper asks whether a plain 3D CNN trained with random rotation augmentation can learn the same behavior instead. The answer depends on the task: for denoising and property prediction, even small augmented models match the equivariance level of an explicitly equivariant network, and a derived loss decomposition separates prediction error from equivariance error to make the result quantitative. For generation, however, only the full 111M-parameter VoxMol model produces stable outputs when the seed molecule is rotated; smaller models and models trained on less data diverge, losing generative equivariance. The paper concludes that learned equivariance is possible in principle but demands redundant capacity that explicit equivariance would save, and suggests auxiliary objectives that align latent embeddings across rotations could close the gap.

What carries the argument

The load-bearing object is Theorem 3.2, the loss decomposition of Eq. (2), which splits the rotation-augmented training loss into a prediction-error term plus an equivariance-error term involving a Hessian-weighted variance of the rotated outputs, and which becomes exact for MSE loss in Eq. (3). That decomposition justifies the operational criterion in Eq. (4): a model is approximately equivariant when the reconstruction of a rotated input $R(\hat{x}_0)$ matches the rotated reconstruction of the unrotated input. The experiments run this criterion over VoxMol, a U-Net 3D CNN that denoises $64^3$ voxel grids at $0.25$ Å resolution and generates molecules by walk-jump sampling, compared against E3NN, an SE(3)-equivariant steerable CNN that serves as the explicit-equivariance baseline. A third component is the latent-embedding cosine-similarity analysis, which shows rotated molecules embed differently and thereby diagnoses why the augmented CNN pays a large parameter overhead.

What would settle it

Repeat the reconstruction and seeded-generation experiments with two rotation families: exact grid symmetries (multiples of $90^\circ$ about grid axes, which a Cartesian grid can represent exactly) versus arbitrary angles that require interpolation and resampling. If the equivariance error is much lower for grid-aligned rotations, part of the claimed learned equivariance is a voxelization artifact rather than model behavior. Alternatively, train the 7M-parameter VoxMol on finer grids at $0.125$ Å resolution with the same rotation augmentation: if generative equivariance then appears without the 111M-parameter budget, the paper's capacity requirement is a resolution artifact, while if it still fails, the capacity claim survives.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that rotation augmentation alone teaches a non-equivariant 3D CNN approximate rotational equivariance for molecule reconstruction: the equivariance error, measured by comparing the reconstruction of a rotated input with the rotated reconstruction of the unrotated input, stays below the ground-truth reconstruction floor, and this holds with as little as 1% of the training data and within a few hundred epochs of training. The central discovery is that reconstruction equivariance does not transfer to generation. In seeded walk-jump sampling, only the full-size rotation-augmented VoxMol (111M parameters) produces generation metrics and chemical-property distributions that are nearly identical whether the seed lead is rotated or not, matching the explicitly equivariant E3NN baseline; smaller VoxMol variants (28M and 7M parameters), the unaugmented model, and models trained on half the data lose stability and shift property distributions when the seed is rotated. The paper also shows that latent embeddings of the same molecule under different rotations have low cosine similarity even when reconstructions agree, concluding that the CNN learns redundant parallel representations instead of recognizing rotated inputs as equivalent, which it offers as the reason augmented CNNs need so many more parameters.

Load-bearing premise

All the equivariance measurements assume that rotating a $64^3$ voxel grid at $0.25$ Å resolution, followed by peak-finding to recover atoms, preserves molecular geometry faithfully enough that any difference between rotated and unrotated outputs is caused by the model's missing equivariance rather than by voxel discretization or resampling artifacts, yet the paper never states how rotations are applied to the grids.

Editorial extensions

If this is right

  • Rotation augmentation can replace architectural equivariance for reconstruction and property prediction, so molecule-generation pipelines without built-in symmetry remain viable for those stages.
  • Generative robustness under rotated seeds is a separate property that emerges only at full model scale; deploying a smaller augmented model in a lead-optimization loop could silently change its outputs, so rotation checks are needed before use.
  • The loss decomposition gives a practical diagnostic: computing the equivariance-error term on a validation set tells a practitioner whether an augmented model is safe to trust under rotation.
  • Auxiliary losses that pull the latent embeddings of rotated molecules together, which the paper proposes, are a direct follow-up that could reduce the parameter cost of learned equivariance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If latent alignment is the bottleneck, a moderate CNN trained with an invariance-penalty auxiliary objective could plausibly beat both the 111M-parameter augmented CNN and the equivariant baseline on generation robustness; the paper floats this idea but does not run it.
  • The 1%-data success for reconstruction equivariance may partly reflect that isotropic Gaussian noise is rotation-invariant by construction; anisotropic or spatially correlated noise would likely expose the augmented CNN's equivariance as much shallower than an explicitly equivariant model's.
  • Because the paper never specifies how arbitrary rotations are applied to a Cartesian grid, a natural extension is to compare grid-aligned rotations (multiples of $90^\circ$) with interpolated arbitrary angles; a large gap in equivariance error would mean voxelization, not the model, carries part of the apparent equivariance.
  • The redundant-representation finding suggests a general regularization principle: when a task demands invariance, a model trained only on the target loss will spend capacity on duplicate input representations unless the loss penalizes that redundancy, a lesson that may extend beyond molecules to any augmented generative system.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper asks whether explicitly equivariant GNNs are necessary for 3D molecular generation by comparing a voxel-based CNN (VoxMol) with an E3NN equivariant baseline. It derives a loss decomposition that separates prediction error from equivariance error, then empirically evaluates reconstruction, seeded generation, and property prediction on the GEOM-Drugs dataset. The main findings are that CNNs trained with rotation augmentation easily learn reconstruction equivariance, but that generative equivariance under rotated seed molecules is robust only for the full-size VoxMol model, while smaller, un-augmented, or data-limited models diverge. The paper concludes with a hypothesis that latent-alignment auxiliary losses could improve robustness without large models.

Significance. If the empirical claims were fully supported, this would be a valuable systematic study of learned versus built-in equivariance in molecular generative models, with practical implications for model selection and scaling. The loss decomposition in Theorem 3.2 is a useful conceptual tool, and the use of a public dataset together with ablations over model size, data fraction, and training duration is a methodological strength. However, the central quantitative claims are currently weakened by an unspecified and potentially non-orthogonal rotation operator on voxel grids, a mismatched equivariant baseline (0.5M versus 111M parameters), and a lack of statistical testing on the generation differences.

major comments (3)
  1. [Section 3.1, Eqs. (3)-(4)] The decomposition in Eq. (3) is exact only when R is an orthogonal transformation of the input vector space, since it uses the step ||R(g_R - y)||^2 = ||g_R - y||^2. The experiments in Sections 4.1 and 4.2 apply rotations to 64^3 voxel grids, but the rotation operator is never specified. Resampling a Cartesian grid by a general rotation is not orthogonal and generally does not satisfy the group law R(θ1)R(θ2)=R(θ1+θ2), so the equality behind Eq. (3) does not hold for the actual transformation used. Consequently, the equivariance error in Eq. (4) conflates model non-equivariance with voxel discretization and interpolation artifacts, and the same confound enters the generation experiments of Section 4.2. Please specify the rotation implementation; if arbitrary angles are used, either rotate continuous atomic coordinates before voxelization, or restrict evaluation to exact voxel-preserving rotations (such as multiples of 90 degrees), and quantify the interpolation error separately.
  2. [Section 4.1, Abstract, Appendix A.1] The abstract claims that CNNs can 'match the performance of equivariant models,' but the only equivariant baseline, E3NN, has 0.5M parameters versus 111M for VoxMol and is explicitly reported as not matching VoxMol on denoising or generation metrics (Appendix A.1). Thus the experiments support at most that a small equivariant model underperforms a much larger augmented CNN, not that equivariant models are unnecessary at comparable capacity. Please either train an equivariant model of comparable capacity, or restrict the claim to the baselines actually evaluated.
  3. [Section 4.2, Tables 1, 2] The assertion that smaller, un-augmented, or data-limited models 'diverge, losing generative equivariance' rests on differences that appear comparable to the reported standard deviations. For example, Table 1 shows VoxMol (No Aug.) with stable 66±25 unrotated versus 61±23 rotated, and uniqueness 75±19 versus 83±17; the KL divergences in Table 2 (0.01-0.07) are reported without confidence intervals. No statistical test is provided. Please report paired tests (e.g., bootstrap or permutation) over the 100 seed molecules, or otherwise show that the rotated/unrotated differences exceed within-model variability.
minor comments (5)
  1. [Throughout] There are several typos and formatting inconsistencies: 'processnig' in Section 1, 'demostrating' in the caption of Figure 6, and inconsistent spacing in 'V oxMol' throughout the text and tables.
  2. [Section 3.1, Theorem 3.2] The proof sketch says 'This follows by a second-order Taylor expansion,' but the regularity conditions on the loss l are not stated. Please specify the differentiability assumptions or note that the approximation is informal.
  3. [Section 3.2, Eq. (7)] The derivation of p(Ry)=p(y) uses invariance of the Lebesgue measure under rotations; it would be clearer to state explicitly that det R = 1 implies the Jacobian factor is one before performing the substitution x = R x~.
  4. [Section 4.2] The metrics 'stable mol' and 'stable atom' are used in Tables 1 and 4 but are not defined in the main text; please define them or cite the original definitions.
  5. [Section 4.2] The description of the second generation experiment says the seed molecule is 'noised with the same noise' as the rotated experiment, but the procedure for fixing the noise is not fully specified. Please describe how the random noise is controlled across runs.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the loss decomposition is a mathematical identity and the empirical claims are measured on held-out rotations; only a non-load-bearing self-citation is present.

full rationale

The paper's central theoretical tool, Theorem 3.2 / Eq. (3), is a bias-variance decomposition of the augmented MSE loss and is not fitted to the conclusions; it holds as an exact identity for orthogonal rotations. The empirical claims (reconstruction equivariance across model size, dataset size, and training duration; generation robustness in Tables 1-2; property-prediction rotation robustness in Table 3) are evaluated on held-out rotations of GEOM-Drugs and do not reduce to the paper's inputs. Self-citations (Pinheiro et al. 2023, Nowara et al. 2024, Gerken and Kessel 2024) import architectures, the walk-jump sampling procedure, and background on emergent equivariance, but the central results are re-measured here, and the E3NN baseline is external; hence these citations are not load-bearing. The unspecified voxel-grid rotation operator is a measurement-validity concern (interpolation may break the orthogonality of R, so Eq. (4) may conflate discretization artifacts with model equivariance), not a circularity, because the metric is a definition rather than a derived prediction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard assumptions about rotation-invariant data and noise, plus an unstated rotation-invariance condition in the general loss decomposition. No free parameters are fitted to derive the equivariance results; model hyperparameters are standard training choices. No new physical or model entities are introduced; the 'redundant parallel representations' concept is an interpretational label for existing latent vectors, not a new entity.

assumptions (5)
  • domain assumption The noisy data distribution p(y) is rotation-equivariant, p(Ry)=p(y), because the molecular data distribution p(x) and isotropic Gaussian noise are rotation-equivariant.
    Invoked in Section 3.2 (equations 6-7) to derive that the denoised distribution is equivariant. This assumes GEOM-Drugs conformers have no preferred global orientation.
  • standard math The loss decomposition in Theorem 3.2 requires the loss to be rotation-invariant, l(Rz, Ry)=l(z,y), so that the expectation can be rewritten in terms of R^{-1}f(Rx). The general approximate form does not state this condition.
    Theorem 3.2 in Section 3.1. For MSE the identity is exact because the Euclidean norm is rotation-invariant, but the general form silently assumes this property.
  • standard math The Dirac delta identity delta(Rz)=delta(z)/|det R| for det R=1 is used to show the denoised distribution is equivariant.
    Section 3.2, equation (9). This is a standard property of the delta function and is valid for SO(3) rotations.
  • domain assumption Rotating a 64^3 voxel grid at 0.25 angstrom resolution, and the subsequent peak-finding post-processing, preserve molecular geometry well enough that residual errors can be attributed to model non-equivariance rather than to discretization artifacts.
    Used throughout Section 4. The paper never specifies how rotations are applied to the voxel grids, and arbitrary rotations cannot be exactly represented on a Cartesian grid.
  • domain assumption Walk-jump sampling (neural empirical Bayes) is a valid generative procedure for this task, with Langevin dynamics hyperparameters chosen appropriately.
    Section 3.3. The framework is taken from Saremi and Hyvarinen (2019) and Pinheiro et al. (2023), and is not re-derived here.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do we need equivariant models for molecule generation?." pith.science (2026). https://pith.science/paper/Q3ZT63NF

@misc{pith2026250709753,
  author       = {Pith},
  title        = {Pith review of: Do we need equivariant models for molecule generation?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q3ZT63NF}},
  note         = {Machine review of arXiv:2507.09753}
}
read the original abstract

Deep generative models are increasingly used for molecular discovery, with most recent approaches relying on equivariant graph neural networks (GNNs) under the assumption that explicit equivariance is essential for generating high-quality 3D molecules. However, these models are complex, difficult to train, and scale poorly. We investigate whether non-equivariant convolutional neural networks (CNNs) trained with rotation augmentations can learn equivariance and match the performance of equivariant models. We derive a loss decomposition that separates prediction error from equivariance error, and evaluate how model size, dataset size, and training duration affect performance across denoising, molecule generation, and property prediction. To our knowledge, this is the first study to analyze learned equivariance in generative tasks.

Figures

Figures reproduced from arXiv: 2507.09753 by the authors.

Figure 1
Figure 1. Experimental set up of reconstruction equivariance to rotations of voxelized molecules. We compute the reconstruction error between reconstructed molecules from a rotated input xˆn to the rotated reconstruction of an unrotated input R(ˆx0) [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Rotational equivariance with different models compared to reconstruction upper bound computed between reconstructed and ground truth molecule (dotted blue lines). unrotated input R(ˆx0). A model is equivariant if these out￾puts are identical or differ only by a small, bounded error [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. Effect of the number of training epochs. Equivariance is learned in very early epochs although the overall reconstruction errors are higher in earlier epochs. latent space. However, we do observe that VoxMol trained with rotation augmentations embeds the rotated molecules closer together compared to VoxMol trained with no augmentations. In contrast, the explicitly equivariant E3NN model produces identical latent emb… view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Effect of training set size. Equivariance is already learned with limited data although the overall reconstruction errors increase with decreasing amount of training data. subset) in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Cosine similarity between latent embeddings of the same molecules under different rotations. Latent embeddings are not rotation equivariant as rotated molecules are embedded differently. However, VoxMol trained with rotation augmentations embeds rotated molecules close…
Figure 7
Figure 7. Figure 7: Experimental set up of generation equivariance. We compare generated molecules when the seed lead molecule is rotated to when the seed molecule is not rotated but noised with the same noise [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Generated molecules with and without rotating the seed molecule obtained with E3NN and VoxMol trained with rotations. seed molecule. In contrast, models trained with no aug￾mentations (“VoxMol (No Aug.)”), smaller model capacity (“VoxMol (28M)” and “VoxMol (7M)”), or r…
Figure 9
Figure 9. Figure 9: Distribution of chemical properties of generated molecules with and without rotating the seed molecule obtained with VoxMol trained with rotations. Model Setup Spearman (GT) ↑ Spearman (Rot) ↑ MAE (GT) ↓ MAE (Rot) ↓ VoxMol (111 M) Encoder-only 0.83 0.95 0.054 0.021 Vox…
Figure 10
Figure 10. Figure 10: Rotational equivariance for the 111 M model [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Rotational equivariance for the E3NN model. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: Rotational equivariance for the 111 M trained with no rotation augmentations [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: Rotational equivariance for the 111 M model with randomly initialized weights before training. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_13.png]
Figure 14
Figure 14. Figure 14: Rotational equivariance for the 28 M model [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: Rotational equivariance for the 7 M model. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: Cosine similarity between latent embeddings of the same molecules under different rotations for the E3NN model. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: Cosine similarity between latent embeddings of the same molecules under different rotations for the 111 M model with randomly initialized weights before training. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_17.png]
Figure 18
Figure 18. Figure 18: Cosine similarity between latent embeddings of the same molecules under different rotations for the 28 M model [PITH_FULL_IMAGE:figures/full_fig_p017_18.png]
Figure 19
Figure 19. Figure 19: Cosine similarity between latent embeddings of the same molecules under different rotations for the 7 M model. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 8 canonical work pages

  1. [2]

    M., Bruna, J., Cohen, T., and Veli ˇckovi´c, P

    Bronstein, M. M., Bruna, J., Cohen, T., and Veli ˇckovi´c, P. Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint arXiv:2104.13478,

    arXiv

  2. [5]

    Gerken, J. E. and Kessel, P. Emergent equivariance in deep ensembles. arXiv preprint arXiv:2403.03103,

    arXiv

  3. [8]

    O., Jamasb, A., Mahmood, O., Sresht, V ., and Saremi, S

    Pinheiro, P. O., Jamasb, A., Mahmood, O., Sresht, V ., and Saremi, S. Structure-based drug design by denoising voxel grids. arXiv preprint arXiv:2405.03961,

    arXiv

  4. [11]

    Appendix A.1

    10 Do we need equivariant models for molecule generation? A. Appendix A.1. Model Architecture We train two models to evaluate learned equivariance. VoxMol is based on a U-Net model with 3D CNN layers with 4 levels of resolution and self-attention on the lowest two resolutions (Pinheiro et al., 2023). V oxMol was trained until convergence for 239 epochs wi...

    work page 2023

  5. [12]

    and we use the official implementation of Diaz et al. (2023). We tune the network hyperparameters for the best denoising performance, however, the E3NN model is not able to match the performance of V oxMol on denoising or generation metrics. We trained E3NN until convergence for 138 epochs and learning rate of 1e-2. Predictive Head We add a lightweight Mu...

    work page 2023

  6. [2016]

    O., Mahajan, S

    Nowara, E., Pinheiro, P. O., Mahajan, S. P., Mahmood, O., Watkins, A. M., Saremi, S., and Maser, M. Nebula: Neural empirical bayes under latent representations for efficient and controllable design of molecular libraries. In ICML 2024 AI for Science Workshop,

    work page 2024

  7. [2018]

    Soft Random Sampling: A Theoretical and Empirical Analysis

    Cui, X., Mittal, A., Lu, S., Zhang, W., Saon, G., and Kings- bury, B. Soft random sampling: A theoretical and empiri- cal analysis. arXiv preprint arXiv:2311.12727,

    work page Pith review arXiv

  8. [2019]

    MiDi: Mixed Graph and 3D Denoising Diffusion for Molecule Generation

    9 Do we need equivariant models for molecule generation? Vignac, C., Osman, N., Toni, L., and Frossard, P. Midi: Mixed graph and 3d denoising diffusion for molecule generation. arXiv preprint arXiv:2302.09048,

    work page Pith review arXiv

Show all 12 references
  1. [2020]

    Ragoza, M., Masuda, T., and Koes, D. R. Learning a contin- uous representation of 3d molecular structures with deep generative models. arXiv preprint arXiv:2010.08687 ,

    2010 arXiv

  2. [2022]

    Equiv- ariant energy-guided sde for inverse molecular design

    Bao, F., Zhao, M., Hao, Z., Li, P., Li, C., and Zhu, J. Equiv- ariant energy-guided sde for inverse molecular design. arXiv preprint arXiv:2209.15408,

    arXiv

  3. [2023]

    Diaz, I., Geiger, M., and McKinley, R. I. An end-to-end se (3)-equivariant segmentation network. arXiv preprint arXiv:2303.00351,

    arXiv

  4. [2024]

    C., Pederson, R., Underkoffler, C., Panjwani, Z., Wang-Henderson, M., Mardirossian, N., Katcher, M

    Kaufman, B., Williams, E. C., Pederson, R., Underkoffler, C., Panjwani, Z., Wang-Henderson, M., Mardirossian, N., Katcher, M. H., Strater, Z., Grandjean, J.-M., et al. Latent diffusion for conditional generation of molecules. bioRxiv, pp. 2024–08,

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.