REVIEW 3 major objections 5 minor 23 references
PaccMann$^{RL}$: Designing anticancer drugs from transcriptomic data via reinforcement learning
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A generative model conditions molecule design on a cancer cell's transcriptomic profile and uses reinforcement learning to bias generation toward compounds with high predicted potency, shifting the share of predicted-effective molecules…
desk verdict A well-built proof-of-concept for omics-conditioned drug generation whose efficacy gains are self-referential to the critic; peer-reviewable with a demand for external validation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is a fused latent space: a denoising VAE (called PVAE) maps a tumor's gene-expression profile to a 128-dimensional latent code, and a stack-augmented GRU VAE (called SVAE) maps SMILES strings of molecules to a same-dimensional latent space; the two codes are combined by addition, and the molecular decoder then generates a SMILES string conditioned on the profile. This conditional generator is treated as a reinforcement-learning agent whose actions are SMILES characters, and the reward is $f(\mathrm{IC}_{50}) = \exp(-\mathrm{IC}_{50}/5)$, where $\mathrm{IC}_{50}$ is the log-micromolar inhibitory concentration predicted by the PaccMann critic for the compound–profile pair. REINFORCE policy gradients update the generator's weights, with gradient clipping and pretraining preserving chemical syntax. The additive fusion is what lets a transcriptome, not a target protein, steer which region of chemical space is explored.
What would settle it
Synthesize a panel of the generated 'effective' compounds, for instance 100 per cancer site, and measure their actual $\mathrm{IC}_{50}$ against the corresponding cell lines in a viability assay. If the fraction with measured $\mathrm{IC}_{50}$ below $1\,\mu\mathrm{M}$ is comparable to the 1–4% baseline rate of random drug-like molecules rather than the claimed 17–30%, the central claim that RL optimization discovered genuinely potent compounds would be refuted.
Extended reading notes
Core claim
According to the paper, a conditional generator built from two pretrained variational autoencoders—one encoding transcriptomic profiles, one encoding SMILES strings—can be fine-tuned with policy gradients so that, starting from random points in chemical space, it produces molecules whose predicted cellular $\mathrm{IC}_{50}$ against a chosen cancer profile is markedly lower than those of an unbiased generator. The authors report 17–30% of optimized candidates receiving predicted $\mathrm{IC}_{50}$ below $1\,\mu\mathrm{M}$ across breast, lung, prostate, and autonomic-ganglia cancer profiles, versus 1–4% at baseline, with the top candidates showing high Tanimoto structural similarity to known drugs for those cancer types: Doxorubicin for breast, Embelin for lung, Vorapaxar for prostate, and Fulvestrant for neuroblastoma. They also report that these effects generalize to cell lines held out during optimization, and that the generated molecules retain comparable drug-likeness, solubility, and synthetic-accessibility scores. The paper is explicit that these potencies come from the critic's predictions, not from wet-lab assays, and notes the critic was trained only on known anticancer compounds.
Load-bearing premise
The pipeline's results stand or fall on the assumption that the PaccMann critic's predicted $\mathrm{IC}_{50}$ values are a valid proxy for real drug potency, even though the critic was trained only on known anticancer compounds and never saw clearly inactive molecules; if that predictive model is biased, the reinforcement learning would just optimize molecules to satisfy the bias.
Editorial extensions
If this is right
- Molecule generation becomes conditionable by disease state rather than by a single protein target, so candidate libraries can be generated for an individual patient's expression profile.
- Because the generator itself never sees anticancer drugs—only the critic does—the structural motifs it rediscovers for each cancer type are learned from the reward signal, suggesting the framework can propose novel scaffolds rather than merely tweak known drugs.
- The same architecture can be extended by adding more latent codes, such as mutations, copy-number changes, or drug scaffolds, through the same permutation-invariant fusion operation.
- If the critic's predictions hold in assays, the pipeline's yield of predicted-effective candidates (17–30%) is far above the baseline 1–4%, implying that in silico screening could substantially shrink the number of compounds needing experimental testing.
Reading between the lines
- The strongest unstated test is wet-lab: synthesizing a few hundred of the generated compounds and measuring true $\mathrm{IC}_{50}$ on the target cell lines would settle whether the 17–30% 'effective' share is real potency or an artifact of critic bias.
- If the critic is biased toward structural families already overrepresented among known anticancer drugs, the reinforcement learning would amplify those families regardless of true efficacy; a fair control would compare generated compounds against random ChEMBL-like molecules matched for Tanimoto similarity to known drugs.
- The framework implicitly argues that target-free drug design from transcriptomes can bypass the failed target hypotheses documented for many clinical candidates; adding multi-objective rewards that penalize predicted toxicity or reward synthesizability is a natural next step the paper points to but does not test.
- Because the two VAEs are trained independently and combined additively, the same recipe could condition generation on any measurable cellular state—proteomic, metabolomic, or epigenetic—whenever a critic can be trained for that context.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces PaccMannRL, a reinforcement-learning framework for de novo anticancer drug design conditioned on transcriptomic profiles. The conditional generator combines two pretrained variational autoencoders: one encoding gene expression profiles (PVAE) and one encoding SMILES strings (SVAE); their latent codes are summed, and the fused model is fine-tuned with REINFORCE using PaccMann, a multimodal drug-sensitivity predictor, as a fixed critic whose predicted IC50 provides the reward (Eq. 1). The authors report that optimized generation shifts the distribution of predicted IC50 toward lower values for four cancer types (breast, lung, prostate, autonomic ganglia), with 17–30% of generated molecules predicted effective (IC50 < 1 µM) versus 1–4% for the baseline SVAE, while maintaining 87–94% SMILES validity. They further show that site-specific generated compounds have nearest neighbors among known anticancer agents and that distributional properties such as QED, ESOL, and SCScore resemble those of real cancer drugs, despite not being explicitly optimized. The paper includes public code, pretrained models, and interactive visualizations.
Significance. If the central claims were externally validated, this would be a valuable demonstration of conditioning molecular generation on disease context rather than on a single protein target, and the open-source release of code and pretrained models is a clear strength. The methodological core—combining two pretrained VAEs with a fixed critic in an RL loop—is sound as an in silico optimization procedure, and the distribution-shift results genuinely show that the generator learns to exploit the critic. However, the significance as a drug-discovery contribution is presently bounded by the fact that the evaluation metric is the same critic that supplies the reward, and by the acknowledged absence of a negative training set for that critic. The nearest-neighbor analysis provides an independent, but only structural, plausibility check. The paper is best read as a proof-of-concept for profile-conditioned generative optimization in silico, and the claims should be re-scoped accordingly.
major comments (3)
- [Section 2.1 and Eq. (1)] The central quantitative result is circular: the reward in Eq. (1) is R(sT) = f(C(CT, Xc)), a function of the critic's predicted IC50, and the effectiveness fractions in Figure 2 and the text are computed from that same critic output. The observed shift from 1–4% to 17–30% of compounds below the 1 µM threshold therefore demonstrates that REINFORCE successfully maximizes the training objective, but it does not by itself provide evidence about real cellular potency. Throughout the abstract and Section 2.1, 'effective' should be read as 'predicted effective by PaccMann'; the authors should either add an independent validation (e.g., an in vitro assay, an external sensitivity dataset, or docking against a relevant target) or explicitly re-scope the headline claims to predicted efficacy.
- [Appendix S7] The critic lacks a negative training set, as the authors acknowledge. The control experiment—2.17% of 1000 ChEMBL molecules predicted effective versus 15.2% of GDSC drugs—shows some discrimination, but ChEMBL molecules are not confirmed inactive compounds, and most GDSC drugs were seen during critic training. Consequently, the control does not establish that the absolute IC50 predictions near the 1 µM threshold are calibrated. Because both the reward function and the main evaluation depend on this critic, the absolute 'effective' threshold used throughout Section 2.1 is not externally anchored. Please either validate the critic's absolute predictions against a dataset with confirmed inactive compounds or consistently qualify the results as predicted efficacy.
- [Section 2.1, 'Investigation of nearest neighbors'] The nearest-neighbor analysis is independent of the critic, but structural similarity to a known anticancer drug is not evidence of efficacy, and the authors themselves note that drugs sharing a mechanism of action can have low Tanimoto similarity (e.g., Crizotinib and Erlotinib, τ = 0.11). The analysis should therefore be presented as a chemical-space plausibility check rather than as validation of anticancer activity. The sentence 'This result is remarkable...' and the related conclusions in the Discussion should be moderated accordingly.
minor comments (5)
- [Section 2.1] In the text, 'some high structural similarity' should read 'share high structural similarity.'
- [Figure 2] The column header 'Profile-specific Compunds' contains a typo: 'Compunds' should be 'Compounds.'
- [Abstract and Introduction] The abstract's novelty claim ('the first generative model') should be reconciled with the description of Méndez-Lucio et al. (2020) in the Introduction, which also conditions de novo generation on gene expression signatures; please qualify the claim.
- [Appendix S2] The sentence beginning 'A reward function f was used...' contains a stray double period ('(see Figure S2).. It is computed'); please correct the punctuation.
- [Appendix S8, Figure S9 caption] The caption contains a spurious space: 'compound gen rated against nervous system cancer' should be 'compound generated against nervous system cancer.'
Circularity Check
Main efficacy result is circular: PaccMann's predicted IC50 is both the reward in Eq. (1) and the evaluation metric in Section 2.1, so the reported 17–30% 'effective' rates reflect the optimization objective rather than validated potency.
-
fitted input called prediction
[Section 2.1 (evaluation), Fig. 1B caption, and Eq. (1) in the Methods]
"The IC50 efficacy as predicted by the critic, is interpreted as reward and is subject to maximization during the RL based optimization. ... For the evaluation, all generated compounds with a predicted IC50 value below 1 µM were considered to be effective. ... the reward R(sT ) = f(C(CT,Xc)) is the output of the critic C scaled by a reward function f. ... The weights of C are fixed."
The reported 'effective' fractions (17–30% vs 1–4%) are computed by applying the fixed critic C to generated molecules, while Eq. (1) trains the generator to maximize R(sT)=f(C(CT,Xc)), the same predicted-IC50 output. A REINFORCE policy-gradient optimizer mechanically shifts samples toward regions where its fixed critic scores higher, so the distribution shift in Fig. 2 is the training objective itself, not an independent measurement of potency. Nothing in the main evaluation assays the molecules or compares C against real IC50 values for generated compounds; the independent checks are Tanimoto similarity to known drugs and in silico chemical scores, which do not establish efficacy. Thus the central 'effective anticancer compound' claim reduces by construction to the reward function.
full rationale
The paper's central quantitative result—that RL optimization raises the fraction of 'effective' generated compounds from 1–4% to 17–30%—is computed with the PaccMann critic, and the same critic's output is the reward in Eq. (1). Because the generator is trained by REINFORCE to maximize f(C(CT,Xc)), the observed distribution shift toward lower predicted IC50 is the objective being optimized; it would be surprising if it did not occur. This is a direct case of a prediction that is forced by construction: the evaluation metric is the training reward. Appendix S7 explicitly concedes that PaccMann 'lacks a negative training set which would help extending its generalization across the space of known anticancer compounds,' so the absolute calibration of the 1 µM effectiveness threshold is not established against confirmed inactive compounds; the reported 2.17% vs 15.2% control shows some separation but does not validate absolute IC50 values for novel molecules. The independent evidence—SMILES validity, Tanimoto nearest-neighbor matches to known anticancer drugs, and chemical property distributions—is real but does not establish cellular efficacy; the authors themselves note that high Tanimoto similarity is not necessary for efficacy. The critic is cited from the authors' prior work (Manica et al., 2019), but the circularity does not depend on the citation's provenance; it depends on the identity of the reward and the reported outcome. A score of 6 reflects partial circularity: the central prediction reduces by construction, while auxiliary validations remain independent.
Assumptions & free parameters
free parameters (2)
- alpha (reward scaling) =
5
- Effective IC50 threshold =
1 µM
assumptions (4)
- domain assumption The critic PaccMann's predicted IC50 is a valid proxy for the actual inhibitory effect of a compound on a cell line.
- domain assumption Transcriptomic profiles from TCGA human samples are compatible with cell line profiles from GDSC and CCLE for the purpose of transfer.
- ad hoc to paper Adding the latent codes of the profile and the molecule (Zc + Zp) produces a valid conditioning for molecular generation.
- domain assumption Structural similarity (Tanimoto on ECFP fingerprints) correlates with similarity in drug sensitivity patterns.
Cite this review
Pith. "Pith review of PaccMann$^{RL}$: Designing anticancer drugs from transcriptomic data via reinforcement learning." pith.science (2026). https://pith.science/paper/6XADCEVN
@misc{pith2026190905114,
author = {Pith},
title = {Pith review of: PaccMann$^RL$: Designing anticancer drugs from transcriptomic data via reinforcement learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/6XADCEVN}},
note = {Machine review of arXiv:1909.05114}
}
read the original abstract
With the advent of deep generative models in computational chemistry, in silico anticancer drug design has undergone an unprecedented transformation. While state-of-the-art deep learning approaches have shown potential in generating compounds with desired chemical properties, they disregard the genetic profile and properties of the target disease. Here, we introduce the first generative model capable of tailoring anticancer compounds for a specific biomolecular profile. Using a RL framework, the transcriptomic profiles of cancer cells are used as a context for the generation of candidate molecules. Our molecule generator combines two separately pretrained variational autoencoders (VAEs) - the first VAE encodes transcriptomic profiles into a smooth, latent space which in turn is used to condition a second VAE to generate novel molecular structures on the given transcriptomic profile. The generative process is optimized through PaccMann, a previously developed drug sensitivity prediction model to obtain effective anticancer compounds for the given context (i.e., transcriptomic profile). We demonstrate how the molecule generation can be biased towards compounds with high predicted inhibitory effect against individual cell lines or specific cancer sites. We verify our approach by investigating candidate drugs generated against specific cancer types and find the highest structural similarity to existing compounds with known efficacy against these cancer types. We envision our approach to transform in silico anticancer drug design by leveraging the biomolecular characteristics of the disease in order to increase success rates in lead compound discovery.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[2]
The model was trained with a batch size of 64 for a maximum of 2000 epochs
adding noise to gene expression values (ε∼N (0, 0.1)). The model was trained with a batch size of 64 for a maximum of 2000 epochs. SVAE. The model was trained on molecules provided in SMILES notation, the longest molecules had 1423 tokens. Both encoder and decoder consisted of two layers of bidirectional GRU (hidden size of 128, dropout of 0.1 at the first...
work page 2000
-
[5]
S5 Results for gene expression profile VAE and SMILES VAE Profile VAE (PVAE) The pretraining results of the PVAE are presented in Figure S4A, B and C. As shown in Figure S4B, the reconstructed gene expression profiles (GEP), shown in blue, as well as the generated GEPs (green) accurately mimic the distribution of the original GEPs (red). Furthermore, the sam...
work page 2018
-
[8]
US Patent 6,680,047. Juan Lao, Julia Madani, Teresa Pu´ ertolas, Mar´ ıa´Alvarez, Alba Hern´ andez, Roberto Pazo-Cid,´Angel Artal, and Antonio Ant´ on Torres. Liposomal doxorubicin in the treatment of breast cancer patients: a review. Journal of drug delivery, 2013,
work page 2013
-
[10]
Visualization of Very Large High-Dimensional Data Sets as Minimum Spanning Trees
Daniel Probst and Jean-Louis Reymond. Visualization of very large high-dimensional data sets as minimum spanning trees. arXiv preprint arXiv:1908.10410 ,
work page Pith review arXiv 1908
-
[12]
Ketan T Savjani, Anuradha K Gajjar, and Jignasa K Savjani
Accessed: 2020-02-02. Ketan T Savjani, Anuradha K Gajjar, and Jignasa K Savjani. Drug solubility: importance and enhancement techniques. ISRN pharmaceutics, 2012,
work page 2020
-
[13]
URL https://www.biorxiv.org/content/early/2020/01/25/2020.01.24.918953
doi: 10.1101/2020.01.24.918953. URL https://www.biorxiv.org/content/early/2020/01/25/2020.01.24.918953. Pavithra Shivakumar and Michael Krauthammer. Structural similarity assessment for drug sensitivity prediction in cancer. In BMC bioinformatics, volume 10, page S17. Springer,
-
[15]
Gradient surgery for multi-task learning
Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. arXiv preprint arXiv:2001.06782 ,
arXiv 2001
-
[16]
XQ Zhang, CY Yang, XF Rao, and JP Xiong
doi: 10.1002/pros.20920. XQ Zhang, CY Yang, XF Rao, and JP Xiong. Plumbagin shows anti-cancer activity in human breast cancer cells by the upregulation of p53 and p21 and suppression of g1 cell cycle regulators. European journal of gynaecological oncology, 37(1):30–35,
Show all 23 references
-
[17]
16 PaccMannRL: Designing anticancer drugs from transcriptomic data via reinforcement learning - Appendix S1 SMILES VAE architecture with StackGRU To enable neural networks to count, Joulin and Mikolov (2015) introduced stack-augmented RNN. Stack-RNNs complement RNNs with a diff...
2015
-
[22]
#"$! % % !
of a subset of 1000 generated molecules with the training and test data from ChEMBL. Figure S4F presents the distributions of the highest Tanimoto similarity between each generated compound and all compounds in training and test dataset respectively. Only a negligible fraction...
2013
-
[23]
The predicted synthesis consists of four sequential reactions with a total 10 commercially available reactants (green)
is shown for a compound proposed against nervous system cancer (top middle). The predicted synthesis consists of four sequential reactions with a total 10 commercially available reactants (green). 8 Information about the retrosynthesis Created On: 2020-02-04T12:27:22.365000 Mo...
2020
-
[50]
Similar optimization parameters as PVAE were used
The latent space of SVAE had the same dimensionality as the PVAE (128) to enable the addition of encodings. Similar optimization parameters as PVAE were used. This model further utilized teacher forcing Williams and Zipser (1989), i.e., the model’s output is conditioned on the...
1989
-
[128]
To further regularize the PVAE, denoising methods were employed by
2 We minimized the variational loss, consisting of the reconstruction loss and KL divergence, using Adam optimizer (β1 = 0.9, β2 = 0.999, ε = 1e−8) and a decreasing learning rate starting at 0 .001 Kingma and Ba (2014). To further regularize the PVAE, denoising methods were em...
2014
-
[1969]
Learning to smile (s)
Stanis law Jastrzebski, Damian Le´ sniak, and Wojciech Marian Czarnecki. Learning to smile (s). arXiv preprint arXiv:1602.06289,
-
[2004]
Joaquin Dopazo
doi: 10.1021/ci034243x. Joaquin Dopazo. Genomics and transcriptomics in drug discovery. Drug discovery today, 19(2):126–132,
-
[2009]
David Weininger
doi: 10.1038/nrd2898. David Weininger. Smiles, a chemical language and information system
-
[2010]
doi: 10.1021/ci100050t
ISSN 1549-9596. doi: 10.1021/ci100050t. URL https://doi.org/10.1021/ci100050t. RXN. Ibm rxn for chemistry. https://rxn.res.ibm.com/,
-
[2013]
doi: 10.1093/nar/gkt1031
ISSN 0305-1048. doi: 10.1093/nar/gkt1031. URL https://doi.org/10.1093/nar/gkt1031. G Richard Bickerton, Gaia V Paolini, J´ er´ emy Besnard, Sorel Muresan, and Andrew L Hopkins. Quantifying the chemical beauty of drugs. Nature chemistry, 4(2):90,
-
[2014]
Auto-encoding variational bayes
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 ,
-
[2017]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,
-
[2018]
Generating sentences from a continuous space
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew M Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349 ,
-
[2019]
PMID: 31618586
doi: 10.1021/acs.molpharmaceut.9b00520. PMID: 31618586. Oscar M´ endez-Lucio, Benoit Baillif, Djork-Arn´ e Clevert, David Rouqui´ e, and Joerg Wichard. De novo generation of hit-like molecules from gene expression signatures using artificial intelligence. Nature Communications,...
-
[2020]
Latent molecular optimization for targeted therapeutic design
Tristan Aumentado-Armstrong. Latent molecular optimization for targeted therapeutic design. arXiv preprint arXiv:1809.02032,
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.