pith. sign in

arxiv: 2605.13297 · v1 · pith:IJWCJCPDnew · submitted 2026-05-13 · 💻 cs.LG

PaMM: Periodic Motif Memory for Atomistic Models with an Explicit Local-Structure Interface

Ryan Dong This is my paper

Pith reviewed 2026-05-14 19:28 UTC · model grok-4.3

classification 💻 cs.LG
keywords periodic motifsatomistic modelingequivariant networkslocal structureinductive biascrystal predictionedge encodingmachine learning potentials
0
0 comments X

The pith

Explicit pair and triplet motif lookup tables improve energy and force accuracy in periodic atomistic models at intermediate training steps.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Current equivariant models for crystals encode repeating local coordination patterns only implicitly through dense edge features. PaMM adds explicit memory tables that store pair motifs keyed by element types and radial bins and triplet motifs keyed by element types and angular bins, then fuses those lookups back into the edge representation. In controlled runs on the OMAT dataset using the UMA-S architecture, both gate-only and affine-equipped versions of PaMM lower mean absolute errors for energy and forces at the 10k-step and 20k-step checkpoints. Ablations show that the gains shrink when the tables are replaced by random buckets or single-motif variants, indicating that the benefit comes from the structured organization of motifs rather than extra capacity alone. The result holds across held-out source families within the dataset, supporting the claim that explicit motif memory supplies a helpful inductive bias for periodic atomistic modeling under these training conditions.

Core claim

PaMM augments the UMA eSCN-MD edge encoder with hashed lookup tables for pair motifs keyed by (Z_j, Z_i, b_r) and triplet motifs keyed by (Z_j, Z_i, Z_k, b_θ). These tables are fused with the baseline edge features through lightweight gate-only or affine-equipped modules. In matched UMA-S + OMAT experiments, the gate-only variant records the lowest energy MAE and the affine variant the lowest force MAE at both 10k and 20k steps, while pair-only, triplet-only, random-bucket, and capacity-matched MLP controls produce smaller gains. Within-OMAT24 source-family splits likewise show small consistent improvements, establishing that explicit pair/triplet motif memory functions as a useful inductive

What carries the argument

PaMM periodic motif memory that hashes pair motifs by (Z_j, Z_i, b_r) and triplet motifs by (Z_j, Z_i, Z_k, b_θ) into fixed-size tables and fuses them with edge features via gate or affine modules.

If this is right

  • At fixed intermediate budgets of 10k and 20k steps, both PaMM variants outperform the plain UMA-S baseline on energy and force MAEs.
  • Combined pair-plus-triplet tables produce larger gains than pair-only, triplet-only, or random-bucket alternatives.
  • Small but consistent improvements appear across held-out generation families within the OMAT24 source split.
  • The motif tables provide an inspectable local-structure interface that remains compatible with the existing equivariant encoder.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the same pattern persists at full convergence on other datasets, explicit motif memory could lower the total training compute needed for periodic systems.
  • The approach may transfer to other equivariant architectures that currently rely on implicit edge features.
  • Inspectable motif tables could support post-hoc analysis of which local geometries the model treats as similar across different crystals.

Load-bearing premise

The measured gains at the 10k-step and 20k-step checkpoints are produced by the structured motif tables rather than by incidental capacity increases or optimization differences.

What would settle it

Full-convergence training runs in which the PaMM variants show no final MAE advantage over the baseline, or in which random-bucket controls match the structured-motif performance, would falsify the claim that explicit motif memory supplies a useful inductive bias.

Figures

Figures reproduced from arXiv: 2605.13297 by Ryan Dong.

Figure 1
Figure 1. Figure 1: PaMM augments UMA eSCN-MD with an explicit motif interface. Pair motifs are keyed by atom types and discretized distances, while triplet motifs are keyed by local angular config￾urations. The retrieved pair/triplet memories are concatenated with the baseline edge representation and then consumed by lightweight fusion modules. In the affine-equipped variant, the same memory also drives a per-layer modulatio… view at source ↗
Figure 2
Figure 2. Figure 2: Anytime validation and test curves under the matched protocol. Both PaMM variants improve on the plain UMA-S baseline before full convergence. Gate-only is strongest on energy near the locked budget, while the affine-equipped variant is strongest on force [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Anytime validation and test curves under the matched protocol. Both PaMM variants improve on the plain UMA-S baseline before full convergence. Gate-only is strongest on energy near the locked budget, while the affine-equipped variant is strongest on force. A.3 Companion-Wave Robustness Results We also completed a companion wave designed to stress the paper’s central comparison without changing its scope. I… view at source ↗
Figure 4
Figure 4. Figure 4: Held-out OMAT24 source-family comparison at the matched checkpoint. Bars com￾pare the plain baseline and the evaluated PaMM checkpoint on each source family. The PaMM checkpoint is consistently better across all five source families on both energy and force MAE. No-gate Pair-only Triplet-only Random-bucket MLP control 0.0 0.1 0.2 0.3 0.4 0.5 Test Energy MAE 0.4601 0.4673 0.4627 0.4623 0.5089 No-gate Pair-o… view at source ↗
Figure 5
Figure 5. Figure 5: Visual summary of the aligned 10k-step controls. Pair-only and triplet-only remain weaker than the full pair+triplet memory path, while the parameter-matched MLP control is the clearest negative reference. A.4 Anytime and Source-Family Visualizations A.5 Visual Summary of the Controls A.6 External and OOD Boundary Checks The appendix is the right place for external evidence that is informative but less tig… view at source ↗
Figure 6
Figure 6. Figure 6: Frequent pair motifs recur across the analyzed OMAT24 validation slice. A relatively small subset of pair types occupies a large fraction of the motif mass. 0 1 2 3 Layer aimd_1000 Source Family PaMM Gates Are Selectively Stronger Across Layers and Families 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 Mean Gate [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Gate usage is structured across layers and source families. The mean scalar gate magnitude is not uniform, suggesting that the memory path is used selectively rather than as a static extra feature block. 1 2-3 4-7 8-15 16-31 32-63 64-127 128-255 256-511 512-1023 1024-2047 4096-8191 Motif Frequency Bin 0.05 0.04 0.03 0.02 0.01 0.00 M e a n Error (P a M M - B aselin e) -0.036 -0.020 -0.031 -0.035 -0.052 -0.0… view at source ↗
Figure 8
Figure 8. Figure 8: More frequent motifs tend to align with larger PaMM gains in the analysis slice. The plotted quantity is the mean error change (PaMM − baseline), so lower is better. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Bucket-count sweep at the matched 10k-step budget. Increasing the number of pair/triplet memory buckets from 2k to 8k improves both validation and test metrics, while the move from 8k to 16k gives only marginal additional benefit. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
read the original abstract

Periodic crystals repeatedly instantiate similar local coordination motifs across translated cells and chemically related structures, but current equivariant atomistic models usually encode these patterns only implicitly in dense edge features. We introduce PaMM, a periodic motif memory that augments the UMA eSCN-MD edge encoder with explicit pair and triplet lookup features. Pair motifs are keyed by $(Z_j, Z_i, b_r)$ and triplet motifs by $(Z_j, Z_i, Z_k, b_\theta)$, hashed into fixed-size tables and fused with the baseline edge representation through lightweight gate-only and affine-equipped variants. We evaluate PaMM in a matched UMA-S OMAT setting and focus on a narrow question: whether explicit motif memory helps at a fixed intermediate training budget. At the 10k-step checkpoint, both PaMM variants improve over the plain baseline; gate-only gives the best energy MAE, while the affine-equipped variant gives the best force MAE. A matched 20k follow-up keeps the same operating-point picture. Aligned controls show that the gain weakens for pair-only, triplet-only, random-bucket, and parameter-matched MLP alternatives, suggesting that the benefit is tied to structured pair/triplet organization rather than generic added capacity. A within-OMAT24 source-family evaluation also shows small but consistent gains across held-out generation families. We therefore make a focused claim: in the studied UMA-S + OMAT regime, explicit pair/ triplet motif memory is a useful inductive bias for periodic atomistic modeling. We do not claim broad cross-dataset transfer, a uniquely preferred fusion variant, or strong scientific interpretability beyond a more inspectable local-structure interface.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces PaMM, a periodic motif memory module that augments the UMA eSCN-MD edge encoder with explicit hashed lookup tables for pair motifs keyed by (Z_j, Z_i, b_r) and triplet motifs keyed by (Z_j, Z_i, Z_k, b_θ). It evaluates this addition in a matched UMA-S + OMAT setting at fixed 10k- and 20k-step training budgets, reporting MAE gains over the plain baseline as well as pair-only, triplet-only, random-bucket, and parameter-matched MLP controls, and concludes that explicit pair/triplet motif memory supplies a useful inductive bias for periodic atomistic modeling in this regime.

Significance. If the gains prove robust, the work supplies targeted empirical evidence that explicit local-structure memory can serve as an effective inductive bias in equivariant atomistic models, improving performance at intermediate training budgets without altering the core architecture. The use of matched controls and the narrow, falsifiable scope of the claim (fixed-budget UMA-S + OMAT) are strengths that would make the result a useful reference point for future architecture design in materials ML.

major comments (2)
  1. [Results] Evaluation at fixed checkpoints: The MAE improvements at the 10k- and 20k-step checkpoints are reported without standard deviations, multiple random seeds, or error bars. Given that the gains are described as modest, this omission prevents assessment of whether they exceed normal run-to-run fluctuation (abstract and results sections).
  2. [Experimental Setup] Training dynamics: No learning curves to full convergence are provided, so it is unclear whether the observed advantage of the motif memory persists beyond the intermediate checkpoints or is specific to early-stage optimization behavior (experimental setup and evaluation sections).
minor comments (1)
  1. [Methods] The description of the gate-only versus affine-equipped fusion variants would be clearer with an explicit equation or small diagram reference in the methods.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major point below, indicating where revisions will be made.

read point-by-point responses

  1. Referee: [Results] Evaluation at fixed checkpoints: The MAE improvements at the 10k- and 20k-step checkpoints are reported without standard deviations, multiple random seeds, or error bars. Given that the gains are described as modest, this omission prevents assessment of whether they exceed normal run-to-run fluctuation (abstract and results sections).

    Authors: We agree that the absence of error bars from multiple seeds limits assessment of the modest gains. In the revised manuscript we will rerun the 10k- and 20k-step evaluations with three independent random seeds and report mean MAE values together with standard deviations. revision: yes

  2. Referee: [Experimental Setup] Training dynamics: No learning curves to full convergence are provided, so it is unclear whether the observed advantage of the motif memory persists beyond the intermediate checkpoints or is specific to early-stage optimization behavior (experimental setup and evaluation sections).

    Authors: The manuscript's scope is deliberately restricted to fixed intermediate training budgets, as stated in the abstract and introduction; we make no claim about behavior at full convergence. To address the request we will add a clarifying sentence in the evaluation section and include full learning curves (to 100k steps) for the main PaMM variants versus baseline in the supplementary material. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical performance comparison at fixed budgets

full rationale

The paper presents an empirical study comparing PaMM-augmented UMA-S models against baselines and ablations (pair-only, triplet-only, random-bucket, parameter-matched MLP) on held-out OMAT data at fixed 10k- and 20k-step checkpoints. No mathematical derivation, uniqueness theorem, or ansatz is invoked that reduces the reported MAE gains to fitted inputs or self-citations by construction. The central claim is explicitly scoped to the studied regime and grounded in direct experimental controls rather than any self-referential reduction.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The approach rests on the assumption that common local motifs in periodic crystals can be usefully discretized into fixed-size hashed tables without losing critical information, and that lightweight fusion is sufficient to integrate them with existing edge encoders. No new physical entities are postulated.

free parameters (1)
  • table size for pair and triplet hashes
    Fixed-size tables are chosen; exact dimensions are not stated in the abstract but constitute a design hyperparameter.
axioms (1)
  • domain assumption Periodic crystals repeatedly instantiate similar local coordination motifs across translated cells
    Stated in the opening sentence; treated as background for the motif-memory design.

pith-pipeline@v0.9.0 · 5599 in / 1374 out tokens · 29122 ms · 2026-05-14T19:28:35.463373+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

10 extracted references · 10 canonical work pages

  1. [1]

    2025 , eprint=

    UMA: A Family of Universal Models for Atoms , author=. 2025 , eprint=

    work page 2025

  2. [2]

    2026 , eprint=

    Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models , author=. 2026 , eprint=

    work page 2026

  3. [3]

    Nature Communications , volume=

    E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials , author=. Nature Communications , volume=

    work page

  4. [4]

    2022 , eprint=

    MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields , author=. 2022 , eprint=

    work page 2022

  5. [5]

    2023 , eprint=

    EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degree Representations , author=. 2023 , eprint=

    work page 2023

  6. [6]

    Lawrence Zitnick and Zachary Ulissi , title =

    Lowik Chanussot and Abhishek Das and Siddharth Goyal and Thibaut Lavril and Muhammed Shuaibi and Morgane Riviere and Kevin Tran and Javier Heras-Domingo and Caleb Ho and Weihua Hu and Aini Palizhati and Anuroop Sriram and Brandon Wood and Junwoong Yoon and Devi Parikh and C. Lawrence Zitnick and Zachary Ulissi , title =. ACS Catalysis , volume =. 2021 , doi =

    work page 2021

  7. [7]

    Wood and Siddharth Goyal and Abhishek Das and Javier Heras-Domingo and Adeesh Kolluru and Ammar Rizvi and Nima Shoghi and Anuroop Sriram and F

    Richard Tran and Janice Lan and Muhammed Shuaibi and Brandon M. Wood and Siddharth Goyal and Abhishek Das and Javier Heras-Domingo and Adeesh Kolluru and Ammar Rizvi and Nima Shoghi and Anuroop Sriram and F. The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysts , journal =. 2023 , doi =

    work page 2022

  8. [8]

    npj Computational Materials , volume=

    AdsorbML: a leap in efficiency for adsorption energy calculations using generalizable machine learning potentials , author=. npj Computational Materials , volume=

    work page

  9. [9]

    The Journal of Chemical Physics , volume=

    Transfer learning using attentions across atomic systems with graph neural networks (TAAG) , author=. The Journal of Chemical Physics , volume=

    work page

  10. [10]

    Shoghi, A

    From molecules to materials: Pre-training large generalizable models for atomic property prediction , author=. arXiv preprint arXiv:2310.16802 , year=

    work page arXiv