Pith. sign in

REVIEW 4 major objections 5 minor 69 references

LLMol claims that a two-stage supervised fine-tuning plus RLVR training with GRPO generates molecules that outperform all baselines on logP targeting, drug-likeness maximization, and similarity-constrained optimization.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 13:35 UTC pith:2CQ4YTAB

load-bearing objection The paper's own tables contradict its headline claims: LLMol does not consistently outperform baselines, and the imported baseline numbers make the comparisons unsupported. the 4 major comments →

arxiv 2607.19044 v1 pith:2CQ4YTAB submitted 2026-07-21 cs.LG

Adopting Reinforcement Learning with Verifiable Rewards for Molecular Generation

Mingxuan Ouyang , Hao Lan , Wanyu Lin This is my paper
classification cs.LG
keywords molecular generationreinforcement learning with verifiable rewardsGRPOlarge language modelsSMILES/SELFIESproperty targetingQEDpenalized logP
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes LLMol, a framework that first supervised fine-tunes a large language model on simplified SELFIES strings, then applies reinforcement learning with verifiable rewards (RLVR) using RDKit-based property checkers, optimized by GRPO to avoid a critic network. It aims to establish that this combination lets a general-purpose LLM generate molecules that hit narrow logP windows (46.91% success vs 0.46% with SFT alone), achieve state-of-the-art penalized logP (31.12) and QED (0.948), and improve P-logP under similarity constraints (17.50 at δ=0.4) in about one hour of training. If true, it provides a parameter-efficient, reward-model-free route to goal-directed molecular design that can be extended to multi-objective tasks. The evidence rests on comparing LLMol's numbers with baselines copied from prior papers, so the framework's value can only be fully validated by a unified protocol.

Core claim

The paper's central discovery is that molecular generation can be cast as a goal-conditioned sequence prediction task and optimized with RLVR, where the reward is computed by deterministic verifiers (RDKit) rather than a learned model. Stage 1 fine-tunes a 4-billion-parameter LLM with LoRA on simplified SELFIES to learn chemical syntax; Stage 2 uses GRPO to update the policy from group-normalized advantages over sampled molecules, with a KL penalty to a frozen reference. This yields a jump from 0.46% to 46.91% success on the logP interval (-2.5,-2.0), top P-logP of 31.12 and QED of 0.948, and mean P-logP improvement of 17.50 at Tanimoto threshold 0.4. The authors claim this consistently surp

What carries the argument

The central machinery is the pairing of RLVR with GRPO. RLVR replaces learned reward models with rule-based verification (property ranges, similarity, validity) computed via RDKit, giving deterministic objective rewards. GRPO removes the critic network by normalizing rewards within each group of sampled molecules to define token-level advantages, and a KL penalty keeps the policy near the SFT reference. Task-specific reward shaping (smooth exponential decay for logP boundaries, squared QED, hybrid log compression for P-logP, weighted similarity+property for constrained tasks) makes sparse verifiable signals dense enough for stable optimization.

Load-bearing premise

The entire comparison rests on treating baseline numbers copied from earlier papers as directly comparable to LLMol's own runs, with no enforced common protocol for sample counts, temperatures, filtering, or length limits.

What would settle it

Re-run LLMol and all baselines (GCPN, MolDQN, MOLGEN, LIMO, etc.) under identical conditions—same number of generated molecules, same temperature 0.8, same max completion length 128, same validity filtering—and recompute the logP success rate, P-logP/QED scores, and similarity-constrained improvements. If LLMol's advantage over the best baseline disappears or reverses, the central claim is refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If LLMol is right, goal-directed molecular generation no longer requires a learned reward model or critic; deterministic property checkers suffice.
  • The two-stage paradigm generalizes across tasks: logP targeting, QED/P-logP maximization, and similarity-constrained optimization all use the same backbone with task-specific prompts, LoRA adapters, and reward functions.
  • The method trains in about one hour (versus 8-24 hours for several baselines), making it practical for iterative design loops.
  • The observed 0.46% SFT-only success mirrors the dataset's natural distribution, implying RLVR is what drives targeted property satisfaction.
  • The group-relative advantage mechanism stabilizes training on binary rewards, potentially enabling other discrete sequence generation tasks with verifiable objectives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported 46.91% logP success may not transfer to other narrow property bands, since the reward's smoothing width δ and the task-specific prompts are tuned to this interval; a test on a different logP window would reveal whether the gain is general.
  • The superiority over baselines is fragile because baseline numbers were taken from different papers without a common sampling protocol; a fair re-run with identical sample counts and decoding settings could shrink or erase the gap.
  • The framework's modular reward design suggests a straightforward extension to multi-property optimization (e.g., logP + QED + similarity) by adding weighted terms; the paper leaves that combination untested.
  • If the method's advantage survives a unified benchmark, the same RLVR+GRPO recipe could be applied to other string-to-structure domains (e.g., reaction prediction, retrosynthesis) with deterministic verifiers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes LLMol, a two-stage framework for goal-directed molecular generation. In Stage 1, a pretrained LLM (Qwen-3 4B) is supervised fine-tuned with LoRA on simplified SELFIES strings to learn chemical syntax. In Stage 2, the model is optimized with Group Relative Policy Optimization (GRPO) using Reinforcement Learning with Verifiable Rewards (RLVR), where reward functions are computed deterministically with RDKit for tasks such as logP targeting, QED/P-logP maximization, and similarity-constrained optimization. The central claim, stated in the abstract and conclusion, is that LLMol consistently outperforms existing methods with higher success rates and improved efficiency. The paper reports results on ZINC250K and compares against a range of baseline methods, with baseline numbers largely imported from prior papers [14] and [30].

Significance. If substantiated, the application of RLVR with GRPO to molecular generation would be a useful practical contribution: the reward design is deterministic and reproducible, the training pipeline is conceptually clean, and the use of simplified SELFIES is a reasonable design choice. The claim of state-of-the-art performance is, however, the main selling point of the paper, and it is directly contradicted by the paper's own tables. The framework itself may be worth studying, but the experimental evidence as presented does not support the headline performance claims. No code, data, or reproducibility artifacts are provided, which further limits the verifiability of the results.

major comments (4)
  1. [Table III and §V-D.2] The text in §V-D.2 states that LLMol achieves a logP-targeting success rate of 46.91%, 'surpassing all prior baselines by a substantial margin.' However, Table III reports GCPN at 85.50% success for the same task. This is an internal contradiction: the paper's own table shows that LLMol is substantially worse than GCPN. This invalidates the 'consistently outperforms' claim in the abstract and conclusion as written.
  2. [Table V and §V-D.4] §V-D.4 claims that 'LLMol outperforms baseline models under both similarity thresholds.' Table V shows that at δ=0.6, MOLGEN achieves a mean penalized logP improvement of 12.08 (SD 0.82), while LLMol achieves 10.60 (SD 13.00). Thus LLMol is worse on the primary metric at δ=0.6, and its very large standard deviation further undermines the reliability of the reported mean. The claim of superiority under both thresholds is not supported by the table.
  3. [Table IV and §V-D.3] In §V-D.3, the authors state that after RL, LLMol 'outperforming all prior methods' on penalized logP. Table IV shows MARS at 45.0 P-logP, well above LLMol's 31.12. The LL (length limit) column distinguishes these methods, and within the LL group LLMol is best, but the text does not make this qualification. The unqualified claim of outperforming all prior methods is contradicted by the table. The authors should either restrict the claim to the LL subgroup or provide a valid comparison with non-LL methods.
  4. [§V-B, §V-C, and Tables III–V] All baseline numbers in Tables III, IV, and V are taken from [14] and [30], as noted in the table captions, but no common evaluation protocol is enforced or reported. The paper does not specify the number of generated samples, decoding temperature, sampling strategy, filtering, or randomness seeds for LLMol. Without a controlled, apples-to-apples comparison, the claim that LLMol 'outperforms' baselines is not statistically supported. At minimum, the authors need to run the baselines under the same protocol or clearly restrict the comparison to settings that are known to be comparable.
minor comments (5)
  1. [§V-D.3] The text reports that '1.20% of generated molecules surpassing a QED score of 0.9' but this statistic does not appear in Table IV or elsewhere. Please provide the supporting data or remove the claim.
  2. [§IV-D, Eq. (7)–(8)] The role of δ in the logP targeting reward is described as 'the width of the smooth transition region,' but the decay function ψ(·) depends on the minimum distance to either boundary. Clarify whether δ is a fixed band around each boundary and how it relates to the success metric used in Table III.
  3. [Eq. (6)] The unbiased KL estimator is written in a way that may confuse readers: the expected value sign is missing. Please write it as an expectation over πθ or clarify the notation.
  4. [General] No code or data availability statement is included. Given the reproducibility emphasis of RLVR, a public code release would materially strengthen the paper.
  5. [Table II] LLMol† has a novelty of 100% and validity of 100%, but its diversity (0.8473) is lower than all listed baselines. The text acknowledges this, but the 'strong ability to model molecular distributions' claim should be carefully phrased given the diversity gap.

Circularity Check

0 steps flagged

No significant circularity: reward-evaluation identity is the explicit RL objective, and the few self-citations are not load-bearing.

full rationale

LLMol's training objective (Eq. 2) directly maximizes a verifiable reward v(c,m), and the task rewards in Eqs. 7-13 are deterministic functions of RDKit-computed property values (logP, QED, P-logP, Tanimoto similarity). The reported evaluation metrics are the same underlying property quantities. This is not circular: the paper is not claiming to 'predict' these properties from first principles or to derive a result whose inputs secretly contain the outputs; it is explicitly performing goal-directed optimization against known, external property calculators. Optimizing a stated objective and then measuring that objective is the standard correctness check for a generative RL method, not a hidden tautology. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in through self-citation. The self-citations in the paper ([3]-[5], [50]) appear only in related-work context and are not used to justify the central claim. The paper's imported baseline numbers and the apparent contradictions in Tables III-V (e.g., GCPN 85.50 vs. LLMol 46.91 in logP targeting; MOLGEN 12.08 vs. LLMol 10.60 at δ=0.6) are serious correctness and comparability concerns, but they are not circularity: they concern whether the empirical comparison is valid, not whether the derivation reduces to its own inputs. On the circularity dimension, the derivation chain is self-contained and no specific circular step can be exhibited.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The ledger shows that the framework relies on established components and domain assumptions rather than new physical entities. The most fragile item is not the RL math but the comparability of imported baseline numbers. Several reward-shaping constants (delta, lambda, epsilon) are left unreported, and the simplified SELFIES tokenization is asserted without full equivalence proof. No fabricated physical entities are introduced.

free parameters (5)
  • delta (logP smooth transition width)
    Appears in Eq. 7-8; controls the width of the near-boundary reward shoulder, directly affecting logP success. Value never reported in hyperparameter Table VI.
  • lambda (similarity-property trade-off weight)
    Eq. 13 weights Tanimoto similarity vs property score; no value is reported, yet it controls the Table V results.
  • beta (KL penalty coefficient) = 0.2
    Table VI; controls how far the policy can drift from the SFT reference model. Standard GRPO hyperparameter, not fitted to the benchmark.
  • GRPO clipping epsilon
    Eq. 4; the PPO-style clipping bound is not reported in the hyperparameter table.
  • G (group size) = 16
    Table VI; sets the number of samples per prompt used to compute relative advantages and controls variance.
axioms (5)
  • domain assumption RDKit-computed property scores (logP, QED, P-logP, Tanimoto) are faithful, correct, and comparable across methods
    Used throughout Section IV-D as the verifiable reward ground truth; no independent validation of the scores or of their comparability across different model families is provided.
  • domain assumption Simplified SELFIES (square brackets removed, spaces inserted) encodes the same molecules and remains RDKit-parseable
    Section III-A presents this representation change as a tokenization efficiency win without proving equivalence or safety for all molecules beyond the ZINC250K average length table.
  • domain assumption Random 9:1 split of ZINC250K and novelty defined as 'not in the training set' is a meaningful metric
    Section V-A/C uses this novelty definition. Generated molecules drawn from the test split would count as novel, inflating the reported 100% novelty.
  • ad hoc to paper Baseline results in Tables III-V from [14] and [30] were obtained under protocols comparable to LLMol's
    All baseline numbers are quoted from previous papers without re-running; no common sampling protocol, temperature, length limits, or seeds are enforced. This assumption is load-bearing for the outperformance claim.
  • domain assumption A general-purpose 4B LLM fine-tuned with LoRA on 200k molecules can learn chemical syntax and distribution
    Section V-A uses Qwen-3 4B as the base model; the claim that no molecular pretraining or graph inductive bias is needed is empirical and only supported by the paper's own validity numbers.

pith-pipeline@v1.3.0-alltime-deepseek · 17419 in / 12417 out tokens · 129361 ms · 2026-08-01T13:35:10.704580+00:00 · methodology

0 comments
read the original abstract

Leveraging large language models (LLMs) for molecular generation has shown remarkable potential in chemical and drug design. Current methods primarily rely on supervised training or fine-tuning with limited datasets, which are insufficient to capture complex molecular design objectives. While some approaches attempt to guide generation toward specific goals, they often lack direct optimization mechanisms, making it difficult to align generated molecules with desired properties. To tackle these challenges, we propose \textbf{LLMol}, a principled reinforcement learning framework that directly incorporates verifiable rewards for targeted molecule generation. The key insight is to formulate molecular design as a goal-conditioned sequence prediction task, where verifiable rewards serve as explicit supervision to drive generation toward desired objectives. LLMol follows a two-stage training paradigm combining supervised learning and reinforcement learning. In the first stage, large language models are supervised fine-tuned to capture chemical syntax and molecular distributions. In the second stage, we introduce Reinforcement Learning with Verifiable Rewards (RLVR), which directly integrates property-based reward signals to guide molecular generation toward task-specific objectives. To address the high variance and instability common in discrete sequence optimization, we adopt Group Relative Policy Optimization (GRPO), a stable on-policy algorithm that smooths reward signals and improves training robustness. This framework enables LLMol to effectively handle a range of molecular design tasks, including single-property targeting (e.g., penalized logP, QED) and structure-constrained optimization. Experimental results demonstrate that LLMol consistently outperforms existing methods, achieving higher success rates and improved efficiency across diverse molecular benchmarks.

Figures

Figures reproduced from arXiv: 2607.19044 by Hao Lan, Mingxuan Ouyang, Wanyu Lin.

Figure 1
Figure 1. Figure 1: An illustration of the proposed LLMol framework for one molecular generation task. Stage 1 performs supervised fine-tuning on a pretrained backbone [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Six sampled molecules with the highest QED scores discovered by LLMol. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: An illustration of constrained optimization results based on P-logP and QED scores. The top row shows reference molecules from the test dataset, [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Training curves across different tasks. Each plot shows the metric trend over training steps. The top-left figure shows the supervised fine-tuning (SFT) [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

69 extracted references · 19 linked inside Pith

  1. [1]

    Crystal diffusion variational autoencoder for periodic material generation,

    T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, and T. S. Jaakkola, “Crystal diffusion variational autoencoder for periodic material generation,” in International Conference on Learning Representations, 2022

    2022

  2. [2]

    Geodiff: A geometric diffusion model for molecular conformation generation,

    M. Xu, L. Yu, Y . Song, C. Shi, S. Ermon, and J. Tang, “Geodiff: A geometric diffusion model for molecular conformation generation,” arXiv preprint arXiv:2203.02923, 2022

    Pith/arXiv arXiv 2022

  3. [3]

    Accelerating 3d molecule generation via jointly geometric optimal transport,

    H. Hong, W. Lin, and K. C. Tan, “Accelerating 3d molecule generation via jointly geometric optimal transport,” inThe Thirteenth International Conference on Learning Representations, 2025

    2025

  4. [4]

    Diffusion-driven domain adaptation for generating 3d molecules,

    ——, “Diffusion-driven domain adaptation for generating 3d molecules,” arXiv preprint arXiv:2404.00962, 2024

    arXiv 2024

  5. [5]

    Crystalline material discovery in the era of artificial intelligence,

    Z. Wang, H. Hua, W. Lin, M. Yang, and K. C. Tan, “Crystalline material discovery in the era of artificial intelligence,”arXiv preprint arXiv:2408.08044, 2024

    arXiv 2024

  6. [6]

    Estimation of the size of drug-like chemical space based on gdb-17 data,

    P. G. Polishchuk, T. I. Madzhidov, and A. Varnek, “Estimation of the size of drug-like chemical space based on gdb-17 data,”Journal of computer- aided molecular design, vol. 27, pp. 675–679, 2013

    2013

  7. [7]

    Impact of high-throughput screening in biomedical research,

    R. Macarron, M. N. Banks, D. Bojanic, D. J. Burns, D. A. Cirovic, T. Garyantes, D. V . Green, R. P. Hertzberg, W. P. Janzen, J. W. Paslay et al., “Impact of high-throughput screening in biomedical research,” Nature reviews Drug discovery, vol. 10, no. 3, pp. 188–195, 2011

    2011

  8. [8]

    Strategy to discover diverse optimal molecules in the small molecule universe,

    C. Rupakheti, A. Virshup, W. Yang, and D. N. Beratan, “Strategy to discover diverse optimal molecules in the small molecule universe,” Journal of chemical information and modeling, vol. 55, no. 3, pp. 529– 537, 2015

    2015

  9. [9]

    Medgan: optimized generative adversarial network with graph convolutional networks for novel molecule design,

    B. Macedo, I. Ribeiro Vaz, and T. Taveira Gomes, “Medgan: optimized generative adversarial network with graph convolutional networks for novel molecule design,”Scientific Reports, vol. 14, no. 1, p. 1212, 2024

    2024

  10. [10]

    Equivariant flow matching with hybrid probability transport for 3d molecule generation,

    Y . Song, J. Gong, M. Xu, Z. Cao, Y . Lan, S. Ermon, H. Zhou, and W.- Y . Ma, “Equivariant flow matching with hybrid probability transport for 3d molecule generation,”Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  11. [11]

    Auto-encoding variational bayes,

    D. P. Kingma, M. Wellinget al., “Auto-encoding variational bayes,” 2013

    2013

  12. [12]

    Automatic chemical design using a data-driven continuous representation of molecules,

    R. G ´omez-Bombarelli, J. N. Wei, D. Duvenaud, J. M. Hern ´andez- Lobato, B. S ´anchez-Lengeling, D. Sheberla, J. Aguilera-Iparraguirre, T. D. Hirzel, R. P. Adams, and A. Aspuru-Guzik, “Automatic chemical design using a data-driven continuous representation of molecules,”ACS central science, vol. 4, no. 2, pp. 268–276, 2018

    2018

  13. [13]

    Junction tree variational autoen- coder for molecular graph generation,

    W. Jin, R. Barzilay, and T. Jaakkola, “Junction tree variational autoen- coder for molecular graph generation,” inInternational conference on machine learning. PMLR, 2018, pp. 2323–2332

    2018

  14. [14]

    Limo: 10 Latent inceptionism for targeted molecule generation,

    P. Eckmann, K. Sun, B. Zhao, M. Feng, M. K. Gilson, and R. Yu, “Limo: 10 Latent inceptionism for targeted molecule generation,”Proceedings of machine learning research, vol. 162, p. 5777, 2022

    2022

  15. [15]

    Objective-reinforced generative adversarial networks (organ) for sequence generation models,

    G. L. Guimaraes, B. Sanchez-Lengeling, C. Outeiral, P. L. C. Farias, and A. Aspuru-Guzik, “Objective-reinforced generative adversarial networks (organ) for sequence generation models,”arXiv preprint arXiv:1705.10843, 2017

    Pith/arXiv arXiv 2017

  16. [16]

    drugan: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico,

    A. Kadurin, S. Nikolenko, K. Khrabrov, A. Aliper, and A. Zhavoronkov, “drugan: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico,”Molecular pharmaceutics, vol. 14, no. 9, pp. 3098–3104, 2017

    2017

  17. [17]

    Graph convolutional policy network for goal-directed molecular graph generation,

    J. You, B. Liu, Z. Ying, V . Pande, and J. Leskovec, “Graph convolutional policy network for goal-directed molecular graph generation,”Advances in neural information processing systems, vol. 31, 2018

    2018

  18. [18]

    Graphdf: A discrete flow model for molecular graph generation,

    Y . Luo, K. Yan, and S. Ji, “Graphdf: A discrete flow model for molecular graph generation,” inInternational conference on machine learning. PMLR, 2021, pp. 7192–7203

    2021

  19. [19]

    Graphaf: a flow-based autoregressive model for molecular graph generation,

    C. Shi, M. Xu, Z. Zhu, W. Zhang, M. Zhang, and J. Tang, “Graphaf: a flow-based autoregressive model for molecular graph generation,”arXiv preprint arXiv:2001.09382, 2020

    Pith/arXiv arXiv 2001

  20. [20]

    Constrained graph variational autoencoders for molecule design,

    Q. Liu, M. Allamanis, M. Brockschmidt, and A. Gaunt, “Constrained graph variational autoencoders for molecule design,”Advances in neural information processing systems, vol. 31, 2018

    2018

  21. [21]

    Retrieval-based controllable molecule generation,

    Z. Wang, W. Nie, Z. Qiao, C. Xiao, R. Baraniuk, and A. Anandku- mar, “Retrieval-based controllable molecule generation,”arXiv preprint arXiv:2208.11126, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y . Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y . Zhang, ...

    Pith/arXiv arXiv 2023

  23. [23]

    Language models can learn complex molecular distributions,

    D. Flam-Shepherd, K. Zhu, and A. Aspuru-Guzik, “Language models can learn complex molecular distributions,”Nature Communications, vol. 13, no. 1, p. 3293, 2022

    2022

  24. [24]

    Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules,

    D. Weininger, “Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules,”Journal of chemical information and computer sciences, vol. 28, no. 1, pp. 31–36, 1988

    1988

  25. [25]

    A systematic study of key elements underlying molecular property prediction,

    J. Deng, Z. Yang, H. Wang, I. Ojima, D. Samaras, and F. Wang, “A systematic study of key elements underlying molecular property prediction,”Nature Communications, vol. 14, no. 1, p. 6395, 2023

    2023

  26. [26]

    A review of molecular representation in the age of machine learning,

    D. S. Wigh, J. M. Goodman, and A. A. Lapkin, “A review of molecular representation in the age of machine learning,”Wiley Interdisciplinary Reviews: Computational Molecular Science, vol. 12, no. 5, p. e1603, 2022

    2022

  27. [27]

    Chemformer: a pre- trained transformer for computational chemistry,

    R. Irwin, S. Dimitriadis, J. He, and E. J. Bjerrum, “Chemformer: a pre- trained transformer for computational chemistry,”Machine Learning: Science and Technology, vol. 3, no. 1, p. 015022, 2022

    2022

  28. [28]

    Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective,

    J. Li, Y . Liu, W. Fan, X.-Y . Wei, H. Liu, J. Tang, and Q. Li, “Empowering molecule discovery for molecule-caption translation with large language models: A chatgpt perspective,”IEEE transactions on knowledge and data engineering, 2024

    2024

  29. [29]

    Conversational drug editing using retrieval and domain feedback,

    S. Liu, J. Wang, Y . Yang, C. Wang, L. Liu, H. Guo, and C. Xiao, “Conversational drug editing using retrieval and domain feedback,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  30. [30]

    Domain- agnostic molecular generation with chemical feedback,

    Y . Fang, N. Zhang, Z. Chen, L. Guo, X. Fan, and H. Chen, “Domain- agnostic molecular generation with chemical feedback,” inThe Twelfth International Conference on Learning Representations, 2024

    2024

  31. [31]

    Molecularrnn: Generat- ing realistic molecular graphs with optimized properties,

    M. Popova, M. Shvets, J. Oliva, and O. Isayev, “Molecularrnn: Generat- ing realistic molecular graphs with optimized properties,”arXiv preprint arXiv:1905.13372, 2019

    Pith/arXiv arXiv 1905

  32. [32]

    Mars: Markov molecular sampling for multi-objective drug discovery,

    Y . Xie, C. Shi, H. Zhou, Y . Yang, W. Zhang, Y . Yu, and L. Li, “Mars: Markov molecular sampling for multi-objective drug discovery,”arXiv preprint arXiv:2103.10432, 2021

    Pith/arXiv arXiv 2021

  33. [33]

    T\” ulu 3: Pushing frontiers in open language model post-training,

    N. Lambert, J. Morrison, V . Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V . Miranda, A. Liu, N. Dziri, S. Lyuet al., “T\” ulu 3: Pushing frontiers in open language model post-training,”arXiv preprint arXiv:2411.15124, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,”Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    2022

  35. [35]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

    Pith/arXiv arXiv 2021

  36. [36]

    Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling,

    G. Landrumet al., “Rdkit: A software suite for cheminformatics, computational chemistry, and predictive modeling,”Greg Landrum, vol. 8, no. 31.10, p. 5281, 2013

    2013

  37. [37]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models,

    Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y . Li, Y . Wuet al., “Deepseekmath: Pushing the limits of mathematical reasoning in open language models,”arXiv preprint arXiv:2402.03300, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Learning multimodal graph-to-graph translation for molecular optimization,

    W. Jin, K. Yang, R. Barzilay, and T. Jaakkola, “Learning multimodal graph-to-graph translation for molecular optimization,”arXiv preprint arXiv:1812.01070, 2018

    Pith/arXiv arXiv 2018

  39. [39]

    Deep learn- ing for molecular design—a review of the state of the art,

    D. C. Elton, Z. Boukouvalas, M. D. Fuge, and P. W. Chung, “Deep learn- ing for molecular design—a review of the state of the art,”Molecular Systems Design & Engineering, vol. 4, no. 4, pp. 828–849, 2019

    2019

  40. [40]

    Autoencoders, unsupervised learning, and deep architectures,

    P. Baldi, “Autoencoders, unsupervised learning, and deep architectures,” inProceedings of ICML workshop on unsupervised and transfer learn- ing. JMLR Workshop and Conference Proceedings, 2012, pp. 37–49

    2012

  41. [41]

    Efficient multi-objective molecular optimization in a continuous latent space,

    R. Winter, F. Montanari, A. Steffen, H. Briem, F. No´e, and D.-A. Clevert, “Efficient multi-objective molecular optimization in a continuous latent space,”Chemical science, vol. 10, no. 34, pp. 8016–8024, 2019

    2019

  42. [42]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020

    2020

  43. [43]

    Reinforced ad- versarial neural computer for de novo molecular design,

    E. Putin, A. Asadulaev, Y . Ivanenkov, V . Aladinskiy, B. Sanchez- Lengeling, A. Aspuru-Guzik, and A. Zhavoronkov, “Reinforced ad- versarial neural computer for de novo molecular design,”Journal of chemical information and modeling, vol. 58, no. 6, pp. 1194–1204, 2018

    2018

  44. [44]

    De novo generation of hit-like molecules from gene expression signa- tures using artificial intelligence,

    O. M ´endez-Lucio, B. Baillif, D.-A. Clevert, D. Rouqui´e, and J. Wichard, “De novo generation of hit-like molecules from gene expression signa- tures using artificial intelligence,”Nature communications, vol. 11, no. 1, p. 10, 2020

    2020

  45. [45]

    Hierarchical generation of molecular graphs using structural motifs,

    W. Jin, R. Barzilay, and T. Jaakkola, “Hierarchical generation of molecular graphs using structural motifs,” inInternational conference on machine learning. PMLR, 2020, pp. 4839–4848

    2020

  46. [46]

    Genetic algorithms are strong baselines for molecule generation,

    A. Tripp and J. M. Hern ´andez-Lobato, “Genetic algorithms are strong baselines for molecule generation,”arXiv preprint arXiv:2310.09267, 2023

    Pith/arXiv arXiv 2023

  47. [47]

    Augmenting genetic algorithms with deep neural networks for exploring the chemical space,

    A. Nigam, P. Friederich, M. Krenn, and A. Aspuru-Guzik, “Augmenting genetic algorithms with deep neural networks for exploring the chemical space,”arXiv preprint arXiv:1909.11655, 2019

    Pith/arXiv arXiv 1909

  48. [48]

    Regression transformer enables concurrent sequence regression and generation for molecular language modelling,

    J. Born and M. Manica, “Regression transformer enables concurrent sequence regression and generation for molecular language modelling,” Nature Machine Intelligence, vol. 5, no. 4, pp. 432–444, 2023

    2023

  49. [49]

    Translation between molecules and natural language,

    C. Edwards, T. Lai, K. Ros, G. Honke, K. Cho, and H. Ji, “Translation between molecules and natural language,”arXiv preprint arXiv:2204.11817, 2022

    Pith/arXiv arXiv 2022

  50. [50]

    Hierarchical deep re- inforcement learning for multi-robot cooperation in partially observable environment,

    Z. Liang, J. Cao, W. Lin, J. Chen, and H. Xu, “Hierarchical deep re- inforcement learning for multi-robot cooperation in partially observable environment,” in2021 IEEE third international conference on cognitive machine intelligence (CogMI). IEEE, 2021, pp. 272–281

    2021

  51. [51]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jainet al., “Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collab- oration 0,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6892–6903

    2024

  52. [52]

    Scaling laws for reward model overoptimization,

    L. Gao, J. Schulman, and J. Hilton, “Scaling laws for reward model overoptimization,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 10 835–10 866

    2023

  53. [53]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Biet al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  54. [54]

    Deep reinforcement learning for de novo drug design,

    M. Popova, O. Isayev, and A. Tropsha, “Deep reinforcement learning for de novo drug design,”Science advances, vol. 4, no. 7, p. eaap7885, 2018

    2018

  55. [55]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  56. [56]

    Reinforcement learning with verifiable rewards: Grpo’s effective loss, dynamics, and success amplification,

    Y . Mroueh, “Reinforcement learning with verifiable rewards: Grpo’s effective loss, dynamics, and success amplification,”arXiv preprint arXiv:2503.06639, 2025

    arXiv 2025

  57. [57]

    Self- referencing embedded strings (selfies): A 100% robust molecular string representation,

    M. Krenn, F. H ¨ase, A. Nigam, P. Friederich, and A. Aspuru-Guzik, “Self- referencing embedded strings (selfies): A 100% robust molecular string representation,”Machine Learning: Science and Technology, vol. 1, no. 4, p. 045024, 2020

    2020

  58. [58]

    Zinc 15–ligand discovery for everyone,

    T. Sterling and J. J. Irwin, “Zinc 15–ligand discovery for everyone,” Journal of chemical information and modeling, vol. 55, no. 11, pp. 2324– 2337, 2015. 11

    2015

  59. [59]

    Quantifying the chemical beauty of drugs,

    G. R. Bickerton, G. V . Paolini, J. Besnard, S. Muresan, and A. L. Hopkins, “Quantifying the chemical beauty of drugs,”Nature chemistry, vol. 4, no. 2, pp. 90–98, 2012

    2012

  60. [60]

    Optimization of molecules via deep reinforcement learning,

    Z. Zhou, S. Kearnes, L. Li, R. N. Zare, and P. Riley, “Optimization of molecules via deep reinforcement learning,”Scientific reports, vol. 9, no. 1, p. 10752, 2019

    2019

  61. [61]

    Extended-connectivity fingerprints,

    D. Rogers and M. Hahn, “Extended-connectivity fingerprints,”Journal of chemical information and modeling, vol. 50, no. 5, pp. 742–754, 2010

    2010

  62. [62]

    Quiet-star: Language models can teach themselves to think before speaking,

    E. Zelikman, G. Harik, Y . Shao, V . Jayasiri, N. Haber, and N. D. Goodman, “Quiet-star: Language models can teach themselves to think before speaking,”arXiv preprint arXiv:2403.09629, 2024

    Pith/arXiv arXiv 2024

  63. [63]

    Training chain- of-thought via latent-variable inference,

    M. D. Hoffman, D. Phan, D. Dohan, S. Douglas, T. A. Le, A. Parisi, P. Sountsov, C. Sutton, S. Vikram, and R. A. Saurous, “Training chain- of-thought via latent-variable inference,” inNeurIPS, 2023

    2023

  64. [64]

    Rlef: Grounding code llms in execution feedback with reinforcement learning,

    J. Gehring, K. Zheng, J. Copet, V . Mella, Q. Carbonneaux, T. Cohen, and G. Synnaeve, “Rlef: Grounding code llms in execution feedback with reinforcement learning,”arXiv preprint arXiv:2410.02089, 2024

    Pith/arXiv arXiv 2024

  65. [65]

    Moleculenet: a benchmark for molecular machine learning,

    Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V . Pande, “Moleculenet: a benchmark for molecular machine learning,”Chemical science, vol. 9, no. 2, pp. 513– 530, 2018

    2018

  66. [66]

    Qwen3 technical report,

    Q. Team, “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  67. [67]

    Transformers: State- of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowiczet al., “Transformers: State- of-the-art natural language processing,” inProceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45

    2020

  68. [68]

    Pytorch: An imperative style, high-performance deep learning library,

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antigaet al., “Pytorch: An imperative style, high-performance deep learning library,”Advances in neural information processing systems, vol. 32, 2019

    2019

  69. [69]

    Virtual compound libraries in computer-assisted drug discovery,

    N. van Hilten, F. Chevillard, and P. Kolb, “Virtual compound libraries in computer-assisted drug discovery,”Journal of chemical information and modeling, vol. 59, no. 2, pp. 644–651, 2019. VI. APPENDIX A. Baseline Descriptions JT-V AE[13] is a V AE-based generative model that first generates a scaffold junction tree and then assembles its nodes into a com...

    2019