Pith. sign in

REVIEW 2 major objections 2 minor 13 references

AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning

T0 review · 2 major / 2 minor · reviewed 2026-06-29 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-composition reinforcement learning agent constructs valid 3D isomers from scratch and generalizes to find up to ten times more on unseen chemical formulas than single-composition baselines.

desk verdict AtomComposer claims order-of-magnitude gains on unseen formulas via multi-composition RL from scratch, but the generalization evidence needs the full experimental splits and results to be convincing. read the letter →

arxiv 2605.28287 v1 pith:FDND5ZDX submitted 2026-05-27 cs.LG cond-mat.mtrl-sci

Bjarke Hastrup , Francois Cornet , Tejs Vegge , Arghya Bhowmik This is my paper · ORCID
classification cs.LGcond-mat.mtrl-sci
keywords reinforcementlearningmolecularisomerschemicalspaceexplorationonline3Dgenerationgeneralizationstoichiometricconstraints
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents AtomComposer as an autonomous agent that builds 3D molecular isomers under given stoichiometric constraints using only online reinforcement learning and no pretraining data. It trains the agent across many different chemical formulas at once rather than fixing it to one composition. This scheme produces substantially more valid isomers when the agent is later tested on formulas it has never encountered. If the result holds, molecular discovery can shift from models that require large curated datasets to agents that explore chemical space directly through interaction.

What carries the argument

The multi-composition training scheme, which trains the agent across diverse chemical formulas at once so that learned policies generalize instead of overfitting to any single stoichiometry.

What would settle it

Run the multi-composition agent on a held-out set of formulas never seen during training and measure whether the count of valid isomers remains within a factor of two of the single-composition baselines rather than reaching an order-of-magnitude improvement.

Watch

Extended reading notes

Core claim

AtomComposer is a self-guided reinforcement learning agent that autonomously assembles valid three-dimensional isomers while respecting stoichiometric constraints. It is trained exclusively online with energy- and validity-based rewards under a multi-composition scheme that exposes the agent to many formulas simultaneously. This yields up to an order of magnitude more valid isomers on unseen test formulas than existing single-composition reinforcement-learning baselines that use per-step energy rewards.

Load-bearing premise

Training the agent on multiple compositions at once is sufficient to produce broad generalization that works on entirely new chemical formulas without overfitting.

Editorial extensions

If this is right

  • Molecular generation no longer requires large pre-curated datasets that introduce bias.
  • Exploration of chemical configuration space can proceed from scratch via online interaction.
  • A single trained agent can address many different stoichiometric targets without retraining.
  • The same online reinforcement learning loop scales to larger or more complex composition spaces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same multi-composition principle could be tested on discovering molecules with targeted properties beyond validity and energy.
  • If the agent learns reusable construction rules, it may transfer to related tasks such as crystal structure prediction under different constraints.
  • The approach suggests that constraint-satisfaction problems in other discrete configuration spaces could benefit from simultaneous training on varied instances.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces AtomComposer, a reinforcement learning agent trained from scratch to autonomously construct valid 3D molecular isomers under stoichiometric constraints. It proposes a multi-composition training scheme using energy- and validity-based rewards that enables generalization across diverse chemistry, claiming the agent discovers up to an order of magnitude more valid isomers on unseen test formulas than single-composition RL baselines trained with per-step energy rewards.

Significance. If the generalization results hold under rigorous OOD evaluation, the work would demonstrate a viable path for dataset-free exploration of chemical configuration space via online RL, addressing biases in pre-curated training data and potentially enabling broader chemical discovery.

major comments (2)
  1. [Abstract, §3] Abstract and §3 (Methods): The headline claim of order-of-magnitude gains on 'unseen test formulas' is load-bearing for the central generalization thesis, yet the manuscript provides no explicit description of the multi-composition sampling procedure, the element sets or stoichiometry ranges used in training episodes versus test, or the precise operationalization of 'unseen' (e.g., novel element combinations, larger atom counts, or merely held-out stoichiometries). Without these details, it is impossible to distinguish genuine OOD generalization from reduced overfitting within statistically similar chemical spaces.
  2. [§4] §4 (Experiments): The comparison to 'existing single-composition reinforcement-learning baselines' requires a clear statement of whether those baselines were also evaluated under the same multi-composition regime or retrained per formula; if the latter, the performance gap may be attributable to training protocol differences rather than the multi-composition scheme itself.
minor comments (2)
  1. [§3] Notation for the validity and energy reward functions should be introduced with explicit equations rather than prose descriptions to allow reproducibility.
  2. [Figures] Figure captions should include the exact number of independent runs and error bars used to generate the reported performance statistics.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which have helped us improve the clarity of our manuscript regarding the multi-composition training and baseline comparisons. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract, §3] Abstract and §3 (Methods): The headline claim of order-of-magnitude gains on 'unseen test formulas' is load-bearing for the central generalization thesis, yet the manuscript provides no explicit description of the multi-composition sampling procedure, the element sets or stoichiometry ranges used in training episodes versus test, or the precise operationalization of 'unseen' (e.g., novel element combinations, larger atom counts, or merely held-out stoichiometries). Without these details, it is impossible to distinguish genuine OOD generalization from reduced overfitting within statistically similar chemical spaces.

    Authors: We agree that these details are essential and were insufficiently described in the original submission. We have revised §3 to explicitly detail the multi-composition sampling procedure (including how compositions are sampled per episode), the element sets (C, H, O, N and extensions) and stoichiometry ranges used in training versus test, and the definition of 'unseen' as novel element combinations and larger atom counts outside the training distribution. This revision clarifies the OOD evaluation. revision: yes

  2. Referee: [§4] §4 (Experiments): The comparison to 'existing single-composition reinforcement-learning baselines' requires a clear statement of whether those baselines were also evaluated under the same multi-composition regime or retrained per formula; if the latter, the performance gap may be attributable to training protocol differences rather than the multi-composition scheme itself.

    Authors: We agree this distinction must be stated clearly. The baselines were retrained per formula, consistent with their single-composition design. Our method's advantage stems from training one agent across multiple compositions. We have revised §4 to explicitly describe the protocols for our agent and the baselines, and added discussion noting that the gap reflects the multi-composition scheme rather than protocol alone. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; empirical RL performance claim is self-contained

full rationale

The paper reports an empirical result: an RL agent trained online with external energy/validity rewards discovers more valid isomers on held-out formulas than single-composition baselines. No derivation chain, equations, or self-citations are presented that reduce the performance claim to fitted parameters or prior author work by construction. The multi-composition scheme is an experimental training protocol whose generalization is tested directly against baselines; it does not contain self-definitional, fitted-input, or uniqueness-imported steps. The result is therefore not circular.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

Review is abstract-only; no implementation details available to enumerate free parameters, axioms, or invented entities. The approach implicitly relies on standard RL assumptions and molecular validity definitions.

assumptions (1)
  • domain assumption Reinforcement learning with energy and validity rewards can guide construction of stable 3D molecular isomers.
    Central to the online training scheme described in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning." pith.science (2026). https://pith.science/paper/FDND5ZDX

@misc{pith2026260528287,
  author       = {Pith},
  title        = {Pith review of: AtomComposer: Discovering Chemical Space from First Principles with Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FDND5ZDX}},
  note         = {Machine review of arXiv:2605.28287}
}
read the original abstract

Discovering novel stable molecules without training data remains a grand scientific challenge. Current molecular generative models are trained on large, pre-curated datasets, which introduce biases and limit exploration of novel chemistry. In contrast, we propose a new paradigm: autonomous, generalized agents capable of mapping vast, unknown chemical spaces without any pretraining. For the first time, we present AtomComposer, a self-guided agent that autonomously constructs valid 3D isomers under stoichiometric constraints and is trained exclusively online using reinforcement learning. Unlike existing approaches that generally overfit to a specific chemical formula, we establish a multi-composition training scheme that enables a broad generalization across diverse chemistry, guided by energy- and validity-based rewards. Our agent can discover up to an order of magnitude more valid isomers on unseen test formulas than existing single-composition reinforcement-learning baselines trained with per-step energy rewards. These results fulfill the promise of online reinforcement learning as a powerful paradigm for scalable, from-scratch exploration of chemical configuration space.

Figures

Figures reproduced from arXiv: 2605.28287 by the authors.

Figure 1
Figure 1. The AtomComposer multi-composition training and evaluation workflow. AtomCom￾poser constructs isomer generation tasks by extracting chemical formulas from a reference dataset and introduces new terminal rewards based on validity and total energy. We evaluate the RL agents’ isomer discovery capabilities at just a single checkpoint, as well as cumulatively across the entire discovery campaign. Crucially, no 3D structu… view at source ↗
Figure 2
Figure 2. illustrates our training and evaluation scheme. Through linear combinations of the 3 fundamental reward components (A, V, F) introduced in Box 2.1(a+b), we define 5 distinct reward functions A, AV, F, FV, and AFV, each corresponding to a separate agent that is trained independently three times using different random seeds (the linear coefficients are shown in [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Learning Curves (in-sample). (a) Validity and (b) unrelaxed Relative Atomic Energy (RAE) of continuously collected training rollouts, plotted against total number of single-atom placements (environment steps). The RAE metric quantifies the excess energy relative to the average energies of QM7 molecules with the same chemical formula (see page 26 for detailed metric definitions). Shading represents ±1 standard deviat… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Q1 visualizations. Histograms of formation energy per atom (center column) together with top 3 best scoring molecules (lowest energy) after structural relaxation for Agent A on the left and Agent AV on the right. Agent A samples molecules of significantly lower energie…
Figure 5
Figure 5. Figure 5: Q2: Out-of-sample agent comparison. We report discovery metrics (top row) and geometry metrics (bottom row) in the multi-bag evaluation setting outlined in Fig. 2b (see page 26 for detailed metric definitions). Each point reflects a weighted average across 20 test bags…
Figure 6
Figure 6. Figure 6: Q3. Cumulative discovery campaign. (a) Number of novel SMILES discovered during training. (b) Number of QM7 SMILES rediscovered. (c)-(d) Total expansion and rediscovery relative to the size of QM7. Although the agents are able to discover many novel molecules and expan…
Figure 7
Figure 7. Figure 7: Functional group analysis. Normalized frequency difference of the 50 most common functional groups in the QM7 dataset between rediscovered molecules and the full QM7 reference set. To avoid rare functional groups dominating the extremes of the distribution, we report t…
Figure 8
Figure 8. Figure 8: Rediscovery energy distributions. The figure shows the formation energy distribution of all QM7 training molecules (grey), together with the energy distribution of rediscovered molecules for the two agents A and AV, with mean values shown vertically. Despite rediscover…
Figure 9
Figure 9. Figure 9: Q4. Property-directed finetuning. (a) Validity and (b) dipole moment magnitude of training rollouts during finetuning, plotted against environment steps. A reward coefficient schedule gradually introduces the dipole moment reward alongside the existing atomization ener…
Figure 10
Figure 10. Figure 10: Q4. Complete finetuning evaluation results. Each row corresponds to one of the 10 evaluated carbonate and ether formulas, with the two formulas highlighted in the main text (H4C3O3 and H6C4O3) shown the first and third row. For each formula, three columns are shown: (…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

  1. [1]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    work page Pith review arXiv

  2. [2]

    Ameya Daigavane, Song Kim, Mario Geiger, and Tess Smidt

    URL https://openreview.net/ forum?id=Snqhqz4LdK. Ameya Daigavane, Song Kim, Mario Geiger, and Tess Smidt. Symphony: Symmetry-equivariant point-centered spherical harmonics for molecule generation.arXiv preprint arXiv:2311.16199,

    work page arXiv

  3. [3]

    Molminer: Transformer architecture for fragment- based autoregressive generation of molecular stories.arXiv preprint arXiv:2411.06608,

    Raul Ortega Ochoa, Tejs Vegge, and Jes Frellsen. Molminer: Transformer architecture for fragment- based autoregressive generation of molecular stories.arXiv preprint arXiv:2411.06608,

    work page Pith review arXiv

  4. [4]

    Scalable fragment-based 3d molecular design with reinforcement learning.arXiv preprint arXiv:2202.00658,

    Daniel Flam-Shepherd, Alexander Zhigalin, and Alán Aspuru-Guzik. Scalable fragment-based 3d molecular design with reinforcement learning.arXiv preprint arXiv:2202.00658,

    work page arXiv

  5. [5]

    An empirical investigation of the challenges of real-world reinforcement learning,

    Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. An empirical investigation of the challenges of real-world reinforcement learning. arXiv preprint arXiv:2003.11881,

    work page arXiv 2003

  6. [6]

    Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Control

    Riashat Islam, Peter Henderson, Maziar Gomrokchi, and Doina Precup. Reproducibility of bench- marked deep reinforcement learning tasks for continuous control.arXiv preprint arXiv:1708.04133,

    work page Pith review arXiv

  7. [7]

    A systematic survey of chemical pre-trained models.arXiv preprint arXiv:2210.16484,

    22 Jun Xia, Yanqiao Zhu, Yuanqi Du, and Stan Z Li. A systematic survey of chemical pre-trained models.arXiv preprint arXiv:2210.16484,

    work page arXiv

  8. [8]

    A hitchhiker’s guide to statistical comparisons of reinforcement learning algorithms.arXiv preprint arXiv:1904.06979,

    Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer. A hitchhiker’s guide to statistical comparisons of reinforcement learning algorithms.arXiv preprint arXiv:1904.06979,

    work page arXiv 1904

Show all 13 references
  1. [9]

    Mars: Markov molecular sampling for multi-objective drug discovery.arXiv preprint arXiv:2103.10432,

    Yutong Xie, Chence Shi, Hao Zhou, Yuwei Yang, Weinan Zhang, Yong Yu, and Lei Li. Mars: Markov molecular sampling for multi-objective drug discovery.arXiv preprint arXiv:2103.10432,

  2. [10]

    URL https://doi.org/10.1021/ci300415d

    doi: 10.1021/ci300415d. URL https://doi.org/10.1021/ci300415d. PMID: 23088335. Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules.Scientific Data, 1,

    doi

  3. [11]

    doi: 10.26434/chemrxiv-2022-v5p6m-v3. Conor F Hayes, Roxana R ˘adulescu, Eugenio Bargiacchi, Johan Källström, Matthew Macfarlane, Mathieu Reymond, Timothy Verstraeten, Luisa M Zintgraf, Richard Dazeley, Fredrik Heintz, et al. A practical guide to multi-objective reinforcement ...

    2022 doi

  4. [12]

    Safe rlhf: Safe reinforcement learning from human feedback.arXiv preprint arXiv:2310.12773,

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback.arXiv preprint arXiv:2310.12773,

    arXiv

  5. [13]

    High-dimensional continuous control using generalized advantage estimation.arXiv preprint arXiv:1506.02438,

    John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation.arXiv preprint arXiv:1506.02438,

    arXiv

Pith tools

Reviewed June 29, 2026 · model on record in the stance chip above.