{"id":"f06e2bb1-cec8-4712-8539-9c85db6c7f68","arxiv_id":"2502.07237","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"SPO fine-tunes an LLM to rewrite drug molecules into analogs that score higher on docking, drug-likeness, solubility, and synthesizability, beating baselines on two protein targets.","lead":"This paper presents DrugImproverGPT, an LLM that edits existing drug molecules into improved analogs. Its new SPO fine-tuning algorithm scores each candidate against the original molecule and reports better surrogate-based results than current baselines on cancer and SARS-CoV-2 targets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported docking gains are measured with the same fitted surrogate used in the SPO reward; without direct OEDOCK validation, the central drug-optimization claim may rest on surrogate optimism.","rationale":"The reader's weakest_assumption identifies exactly the concern I find most load-bearing: evaluation and reward share the same surrogate docking model. I read the paper in good faith and acknowledge its strengths: it introduces a large docking dataset, a plausible RL fine-tuning pipeline, an ablation showing partial-molecule improvement helps, and consistent gains across several surrogate-based metrics. However, the empirical claim of the paper is not just that the SPO algorithm maximizes its own reward; it is that the method produces better drug candidates. That claim requires the surrogate to faithfully approximate true docking scores on the generated molecules. The paper does not provide this evidence. The validation R2 values in Appendix A.5 are computed on a random split of the docking dataset, not on the distribution of molecules produced by the fine-tuned policy, which is precisely where overoptimization is most likely to occur. The fact that the same surrogate is used in Eq. (11) as the docking critic means the policy is trained to exploit the surrogate, and the evaluation then reports the same exploited objective. This is not a mathematical inconsistency in the paper's equations, but it is a correctness risk in the central application claim. I do not see a need to move the verdict from CONDITIONAL: the concern is addressable by direct docking validation, as requested. My recommended verdict is therefore UNCHANGED relative to the reader's verdict, with the same conditions: run OEDOCK/FRED on generated molecules, report unfiltered metrics, and release code and data. I also note the theoretical section is weaker than claimed because Lemma 5.1 depends on the unverified strict-improvement assumption in (19), but this is secondary to the surrogate issue and would not change the verdict.","tokens_in":20492,"tokens_out":2078,"duration_ms":20735,"concrete_test":"Take the exact generated molecules used to compute Table 1 for 3CLPro and RTCB (including the original molecules and the REINVENT 4, Molsearch, MIMOSA, and DrugEx v3 outputs), run the actual OEDOCK/FRED docking program that generated the dataset, and recompute the docking column and AvgNorm Reward from the true docking scores. If DRUGIMPROVER's true docking improvement over the original and over baselines disappears or reverses, the central drug-optimization claim fails. A second, weaker check is to retrain the surrogate on a split that excludes all generated molecules and compare its predictions to OEDOCK scores on a held-out sample of generated molecules.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DRUGIMPROVER generates drug analogs with higher docking scores than the original and than all baselines. Table 1's docking column, however, is not produced by the OEDOCK/FRED docking program described in Appendix B; it is produced by a transformer surrogate (Appendix A.5) with validation R2 = 0.842 for 3CLPro and 0.73 for RTCB. Critically, this same surrogate is used as the docking critic inside the SPO reward in Eq. (11), where CDocking is defined as \"we employ a docking surrogate model ... to output this score.\" Thus the function being optimized during RL and the function being reported in evaluation are the same fitted model. If the surrogate is systematically optimistic on the chemical region explored by the policy, the policy can appear to improve docking while the true OEDOCK scores of the generated molecules are no better, or worse, than the originals. This risk is amplified because generated molecules are novel SMILES that may lie outside the surrogate's training distribution, and because the reported results are filtered to molecules with Tanimoto similarity above 0.6. The moderate validation R2 on a random split does not establish calibration on optimized, out-of-distribution outputs. This is the weakest load-bearing link: all headline comparative statements about docking improvement and the \"AvgNorm Reward\" metric inherit the surrogate's fidelity. The two theoretical lemmas are conditional on the unverified BON-improvement assumption (19), but even if those proofs are correct, they do not address whether the surrogate reward corresponds to real docking quality. The paper's contribution would still stand if the surrogate were independently validated on the generated molecules, but as written the empirical core of the paper is not independently confirmed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DRUGIMPROVER, an LLM-based framework for drug optimization, and SPO, a structured policy optimization algorithm that fine-tunes the LLM using an advantage preference comparing generated analogs with the original drug across multiple critics (druglikeness, solubility, synthesizability, docking). The authors report improvements over several baselines on SARS-CoV-2 (3CLPro) and cancer (RTCB) benchmarks, provide an ablation of the partial-molecule component, and present two theoretical lemmas supporting the SPO objective. They also release a dataset of one million docked compounds and plan to release code.","tokens_in":20773,"tokens_out":5859,"duration_ms":52117,"significance":"If the empirical claims hold, the paper would be a useful contribution to RL-based drug optimization: it addresses a practically important task (optimizing existing drugs rather than de novo design), introduces a plausible policy-gradient objective with partial-molecule rollouts, and releases a large docking dataset that could benefit the community. The paper also deserves credit for comparing against six REINVENT 4 variants and several other baselines, and for reporting wall-clock times. However, the central docking-improvement claim currently rests on evaluation with the same fitted surrogate that supplies the reward, and the theoretical lemmas are conditional on an unverified best-of-N assumption; these issues need to be resolved before the claims can be accepted as stated.","major_comments":[{"comment":"The reported docking improvements are circular: the docking score used as a reward component in Eq. (11) is the transformer surrogate described in Appendix A.5, and the same surrogate appears to produce the Docking column in Table 1. With validation R2 = 0.842 for 3CLPro and 0.73 for RTCB, the surrogate may be optimistic on the chemical region explored by the policy, so the headline claim that DRUGIMPROVER generates analogs with better docking than the originals and baselines is not established. Please evaluate the generated molecules with the actual OEDOCK/FRED docking program, or at minimum with an independent surrogate not used in the reward, and report surrogate calibration on the generated distribution.","section":"§4.2, Eq. (11); Table 1; Appendix A.5"},{"comment":"All reported results are restricted to generated molecules with Tanimoto similarity above 0.6. This filter can exclude the most aggressively optimized molecules and may differentially affect methods that explore lower similarity; without unfiltered statistics or a justification that the filter is not responsible for the observed gains, the comparative claims are vulnerable. Please report the fraction of generated molecules excluded and provide unfiltered results for all metrics.","section":"Table 1 caption; §6.1"},{"comment":"The partial-molecule advantage r^AP_BON(j) is under-specified. It is not stated how the best-of-N rollouts are drawn when the partial SMILES is invalid or cannot be completed, how the N candidates are formed, or whether the X and Y partial sequences are aligned by token position; moreover, Eq. (5) defines BON as an arg-max over rewards while Eq. (15) writes an expectation over BON completions. Please clarify the exact sampling procedure used in the experiments, since this term is central to the SPO update in Eq. (17).","section":"Eq. (15); Definition 4.1"},{"comment":"The proof of Lemma 5.1 appears to contain an inequality with the wrong direction: from J(π) ≥ J0(π) and J0(π*) ≥ J(π) it does not follow that J0(π*) − J(π) ≥ (1/2)(J0(π*) − J0(π)); the latter would require an additional bound that is not proved. In addition, the lemma is conditional on the BON strict-improvement assumption in Eq. (19), which is not verified for the trained policy. Please correct the proof or state precisely under what verified conditions SPO and the standard RL objective share the same optimizers.","section":"§5, Lemma 5.1; Appendix C"},{"comment":"The reward weighting is inconsistent as written. If the list C(yT) contains the four non-similarity critics, then the total weight on those critics is 4(1−β)/5, so the coefficients do not sum to 1 except at β = 1; if C includes Tanimoto, then Tanimoto is double-counted. Please state the intended normalization and give the exact reward formula used to produce Tables 1 and 2.","section":"Eq. (11)"}],"minor_comments":[{"comment":"The pretraining loss is written as λ·NLL(X,Y)/((1−λ)·Similarity(X,Y)); this form is unusual and likely a typo for a weighted sum such as λ·NLL + (1−λ)·(1−Similarity) or similar. Please clarify the exact loss used.","section":"Eq. (10)"},{"comment":"There is a discrepancy between the hyperparameter table ('# of Molecules Optimized 256') and the text in Appendix A.7 ('A selection of 1280 molecules from each of the RTCB and 3CLPro datasets ... is used for SPO finetuning'). Please reconcile these numbers.","section":"Appendix A.7, Table 5"},{"comment":"The handling of invalid generated SMILES is listed ambiguously as '[0, −Rc(X)]' with two alternatives described in the text; please specify which option was used for the results in Tables 1 and 2 and report the invalid-SMILES rate per method.","section":"Table 5"},{"comment":"The prose contains several grammatical and typographical errors (e.g., 'introduce a novel reinforcement learning algorithm', 'while retains the beneficial chemical properties'), which should be corrected in a thorough language edit.","section":"Abstract and §1"},{"comment":"Figure 2 shows Tanimoto similarity over epochs without error bars or standard deviations despite the text referring to five experimental runs; please add uncertainty information or state why it is omitted.","section":"Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The surrogate-circularity issue is the main correctness risk and should be treated as a requirement for acceptance: the authors should rerun evaluation with the actual docking program or a genuinely independent surrogate. The theory section also needs a corrected proof or a weakened claim. I do not recommend rejection because the issues are addressable within the manuscript's scope and the released dataset plus framework are potentially valuable to the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the thing to know: the headline docking improvements in Table 1 are computed with a fitted transformer surrogate, not with OEDOCK/FRED on the generated molecules. That same surrogate is the docking critic inside the SPO reward (Eq. 11). So the function being optimized is the function being reported. The validation R2 values (0.842, 0.73) come from random splits, which does not tell you whether the surrogate is calibrated on the optimized, out-of-distribution SMILES the policy produces. If it is optimistic there, the central drug-optimization claim fails even though the table numbers are real. This is the load-bearing soft spot, and it is not a minor one.\n\nWhat is genuinely new: the SPO objective (Eq. 15) and gradient (Eq. 17) are a new assembly of known pieces, and the release of a 1M-compound docking dataset for 5 cancer and 24 COVID receptors is a substantial, reusable asset. The experimental sweep is broad — six REINVENT 4 variants, Molsearch, MIMOSA, DrugEx v3 — and the ablation shows the partial-molecule term helps. That part is solid.\n\nThe other soft spots are less severe but worth naming. Evaluation is restricted to molecules with Tanimoto similarity >0.6; we never see unfiltered metrics, so we don't know how many generated molecules fall outside that filter or how the method behaves there. The theory (Lemmas 5.1–5.2) is conditional on assumption (19) that BON strictly improves; that is an unverified behavioral property, and the proof sketch essentially assumes it. The definition of rAP_BON(j) around Eq. (15) is under-specified: how the prefix interacts with the BON rollout is left vague, and the invalid-SMILES advantage has two possible values (0 or −Rc(X)), a free parameter that can shift results. Code and data are promised but not shipped, so the reproducibility is currently promise-only.\n\nWho gets value from this? Someone working on RL fine-tuning of generative molecular models will find the SPO recipe interesting and will want the docking dataset. Someone looking for a validated drug optimization method should not rely on the docking claims yet.\n\nRecommendation: send it to peer review, but with a clear request: run OEDOCK/FRED on a few thousand generated molecules, report unfiltered metrics, and tighten the definitions. If the docking gains survive direct scoring, this is a solid paper. If not, the surrogate-guided optimization may still be useful, but the claims need to be scaled back accordingly.","headline":"A plausible RL fine-tuning recipe and a valuable docking dataset, but the docking claim is not independently validated because the same fitted surrogate serves as both reward and evaluator.","tokens_in":21403,"tokens_out":2539,"would_cite":true,"duration_ms":24227,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning a drug-optimization LLM with Structured Policy Optimization makes it generate analogs that outperform the original molecule and competing baselines on normalized multi-objective reward.","keywords":["drug optimization","large language models","reinforcement learning","Structured Policy Optimization","advantage preference","docking score surrogate","SMILES generation","drug repurposing"],"falsifier":"Rescore a random sample of the molecules generated for 3CLPro and RTCB with the actual docking program, and compare their scores with the original drugs' scores. If the true docking scores of the generated set are not better than the originals, or are systematically worse than the surrogate predicted, the paper's central claim is refuted.","tokens_in":20243,"feed_emoji":"💊","tokens_out":15705,"duration_ms":125952,"temperature":0.7,"pith_summary":"This paper tries to establish that a large language model, fine-tuned with a new reinforcement-learning algorithm called Structured Policy Optimization (SPO), can take an existing drug molecule and generate analogs that score better than the original across several drug-relevant objectives while staying chemically similar. The practical payoff would be substantial: adapting an approved drug to a fast-evolving virus or a drug-resistant cancer is far cheaper and faster than inventing a new drug from scratch, and most generative models either build molecules from nothing or reproduce biases from their training data. The paper reports that on the SARS-CoV-2 protease 3CLPro and the human cancer target RTCB, the generated analogs beat the original drug and all compared baselines on average normalized reward, with Tanimoto similarity held above 0.6. It also releases a dataset of one million compounds with docking scores against multiple viral and cancer protein targets, so other methods can be fine-tuned and benchmarked on the same grounds. The central mechanism is an advantage preference: the policy is pushed to increase a generated molecule's normalized multi-objective score relative to the input molecule, and the reward signal is additionally densified by scoring partial molecules.","feed_headline":"Drug-optimizing LLM beats baselines on viral and cancer targets","feed_subtitle":"SPO's advantage-preference reward makes generated analogs score higher than the original and the leading baselines.","key_machinery":"The load-bearing object is the advantage-preference reward $r_{AP}(Y_{1:T}, X) = R_c(Y_{1:T}) - R_c(X)$, where $R_c$ is a min-max normalized sum of four critics (drug-likeness, solubility, synthesizability, docking score) plus a Tanimoto-similarity term relative to the input drug. SPO uses this signed difference as the policy-gradient coefficient, so sequences that beat the original are reinforced and sequences that fall short are suppressed. To handle sparse rewards, it also samples a random partial prefix, completes the prefix with roll-in/roll-out using best-of-N decoding, and forms a blended advantage $R_{AP}$ that averages the partial-molecule advantage and the full-sequence advantage. The theoretical lemmas show that, when best-of-N strictly improves suboptimal molecules, maximizing the SPO objective is equivalent to maximizing the standard reward, and that the blended gradient decomposes into per-prefix terms, which is the formal sense in which the reward signal is densified.","core_discovery":"The paper's central claim is that SPO fine-tuning turns a pretrained sequence-to-sequence LLM into a drug-optimization engine: given a source drug, it outputs valid chemical analogs with higher normalized reward than the source and higher than every baseline tested, while preserving similarity to the source. In the main experiments, the method's average normalized reward is 0.601 on 3CLPro versus 0.524 for the original and 0.575 for the best baseline variant, and 0.694 on RTCB versus 0.538 for the original and 0.642 for the best baseline. The complete comparison also shows higher drug-likeness, better synthesizability, and improved solubility on both targets; the docking component improves on RTCB (-9.462 vs. -8.538) but not on 3CLPro (-8.163 vs. -8.687), which is why the authors phrase the headline result as outperforming across most performance metrics. An ablation attributes part of the gain to the partial-molecule term, which raises validity and most rewards compared to optimizing only complete sequences.","pith_inferences":["Editorial inference: since the same learned surrogate supplies both the reward and the reported docking numbers, the most direct test of the paper's central claim is to re-score the generated molecules with the actual docking program; the paper does not report such a check.","Editorial inference: the advantage-preference construction does not depend on SMILES tokenization, so it should transfer to graph-based molecular generative models or synthesis-planning policies with only the critics changed.","Editorial inference: the best-of-N completion term makes partial-molecule advantage estimates optimistic by construction, so part of the measured densification gain could come from sampling bias rather than from better policy learning; separating the two would require comparing best-of-N-completed rewards against full-sequence rewards under the same policy.","Editorial inference: if surrogate optimism is concentrated in exactly the region the policy explores, interleaving SPO updates with occasional true docking scores would turn the method into an active-learning loop and could correct for the drift."],"forward_implications":["If the central claim is right, drug optimization becomes a sequence-to-sequence LLM task: input a known drug and get back a structurally similar analog with better computed properties, without enumerating chemical space.","The partial-molecule advantage term densifies the reward signal, so SPO should train more stably and converge faster on sparse-reward molecular tasks than full-sequence-only policy gradients.","The released docking dataset and surrogate models give the field a shared benchmark for fine-tuning and comparing generative drug-optimization methods on viral and cancer targets.","Because the reward is a weighted sum of arbitrary critics, the same algorithm extends to other objectives such as toxicity or metabolic stability whenever those critics can be computed for generated molecules.","Reported validity of 0.844 on 3CLPro and 0.964 on RTCB indicates the fine-tuned policy keeps producing syntactically plausible SMILES while optimizing, so the gains are not just a side effect of generating nonsense strings."],"supporting_citations":[{"why":"Supplies the TOP-PK sampling and the normalized multi-critic reward construction that SPO adapts for drug optimization.","marker":"[42]"},{"why":"The six baseline variants of the current leading generative drug-design pipeline that the method is compared against.","marker":"[44]"},{"why":"A graph-based multi-constraint molecule-sampling baseline in the main comparison.","marker":"[20]"},{"why":"A scaffold-constrained transformer reinforcement-learning baseline in the main comparison.","marker":"[43]"},{"why":"A search-based multi-objective molecule-generation baseline in the main comparison.","marker":"[65]"},{"why":"Provides the roll-in/roll-out scheduling used to estimate rewards for partial molecules in the SPO gradient.","marker":"[57]"},{"why":"Defines best-of-N sampling, the completion-and-ranking procedure inside the partial-molecule advantage.","marker":"[22]"},{"why":"The docking program that produced the ground-truth docking scores for the released dataset and surrogate training labels.","marker":"[34]"}],"fun_headline_variants":["SPO-tuned LLM improves drug scores on cancer and COVID targets","Drug LLM fine-tuned via SPO outperforms on most drug metrics","RL-trained LLM generates drug analogs with higher rewards","SPO boosts drug optimization LLM over baselines on viral and cancer proteins","New RL method enhances drug analogs while preserving key properties"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the learned docking surrogate's predicted scores are accurate for the newly generated molecules; the same surrogate is used both inside the reward and in the reported evaluation, so if it is systematically optimistic for generated analogs, the main improvement claim fails.","fun_headline_variants_meta":{"raw":{"variants":["SPO-tuned LLM improves drug scores on cancer and COVID targets","Drug LLM fine-tuned via SPO outperforms on most drug metrics","RL-trained LLM generates drug analogs with higher rewards","SPO boosts drug optimization LLM over baselines on viral and cancer proteins","New RL method enhances drug analogs while preserving key properties"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000339,"raw_usage":{"total_tokens":1892,"prompt_tokens":986,"completion_tokens":906,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":816}},"tokens_in":602,"tokens_out":906,"duration_ms":8100,"temperature":1.0,"reasoning_tokens":816,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:23:23.146513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rescore a random sample of the molecules generated for 3CLPro and RTCB with the actual docking program, and compare their scores with the original drugs' scores. If the true docking scores of the generated set are not better than the originals, or are systematically worse than the surrogate predicted, the paper's central claim is refuted.","supporting_citations":[{"cited_title":"Entropy-reinforced planning with large language models for de novo drug discovery","cited_arxiv_id":null,"evidence_quote":"Supplies the TOP-PK sampling and the normalized multi-critic reward construction that SPO adapts for drug optimization."},{"cited_title":"Reinvent 4: Modern ai–driven generative molecule design","cited_arxiv_id":null,"evidence_quote":"The six baseline variants of the current leading generative drug-design pipeline that the method is compared against."},{"cited_title":"Mimosa: Multi-constraint molecule sampling for molecule optimization","cited_arxiv_id":null,"evidence_quote":"A graph-based multi-constraint molecule-sampling baseline in the main comparison."},{"cited_title":"Drugex v3: scaffold-constrained drug design with graph transformer-based reinforcement learning","cited_arxiv_id":null,"evidence_quote":"A scaffold-constrained transformer reinforcement-learning baseline in the main comparison."},{"cited_title":"Molsearch: search-based multi-objective molecular generation and property optimization","cited_arxiv_id":null,"evidence_quote":"A search-based multi-objective molecule-generation baseline in the main comparison."},{"cited_title":"Scaling laws for reward model overoptimization","cited_arxiv_id":null,"evidence_quote":"Defines best-of-N sampling, the completion-and-ranking procedure inside the partial-molecule advantage."},{"cited_title":"Posit: flexible shape-guided docking for pose prediction","cited_arxiv_id":null,"evidence_quote":"The docking program that produced the ground-truth docking scores for the released dataset and surrogate training labels."}],"review_version":1}