{"id":"3ede3630-1981-49a3-abb9-806f46b4e744","arxiv_id":"2505.20131","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MolEditRL uses structure-aware graph diffusion plus RL fine-tuning to edit molecules toward desired properties while preserving scaffold similarity, reporting SOTA on its own MolEdit-Instruct benchmark.","lead":"A new molecular editing model combines discrete graph diffusion with reinforcement learning to modify molecules while keeping their core structure intact. It reports large gains over text-based baselines on a newly built 3-million-example dataset, though no code or data link is provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on a self-built benchmark whose test set and templates come from the same MMP/BRICS pipeline used for training, and baselines are not adapted to that pipeline; an external evaluation is needed before the 74% margin can be taken as general editing ability.","rationale":"The strongest claim is comparative: MolEditRL significantly outperforms SOTA on editing accuracy and structural fidelity. For that claim to hold, the evaluation benchmark must be informative rather than tautological. Appendix C shows that training and test pairs are generated by the same MMP/BRICS pipeline with high-similarity filtering and template prompts, so the model is trained on the exact distribution it is later evaluated on. Baselines are not given the same adaptation. This makes external validity the load-bearing assumption. The reader's weakest_assumption identifies the same concern, and the proposed external evaluation would settle it. This is not an accusation of data leakage; it is a request for evidence that the benchmark measures general editing ability. If the external check passes, the conditional verdict can be upgraded; if it fails, the headline claim is unsupported. Until then, CONDITIONAL is the appropriate verdict, so no change to the reader's verdict is needed.","tokens_in":23395,"tokens_out":6650,"duration_ms":76420,"concrete_test":"Run a controlled external-benchmark evaluation: fine-tune all baselines on the MolEdit-Instruct training split using the same template prompts, then evaluate MolEditRL and the adapted baselines on an independent third-party test set (e.g., MuMOInstruct or MolOpt-Instructions test splits) with identical RDKit/TDC scoring and three seeds. If MolEditRL's Acc_all(0.65) margin over the best adapted baseline is no longer significant, the central SOTA claim is a benchmark artifact rather than general editing performance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparative claim requires MolEdit-Instruct to be a fair and representative testbed. The paper's own construction in Appendix C couples training and test distributions: source–target pairs are selected by MMP/BRICS fragmentation, filtered for high Tanimoto similarity and large property shifts, then paired with template prompts from Table 4. The test set is drawn from this same pipeline, so MolEditRL is trained to maximize likelihood on exactly the distribution used for evaluation. The baselines are not adapted to this distribution: GeLLM3O-P(6) was trained on six properties but tested on ten, and DrugAssist, BioT5, and REINVENT4 are used essentially as released without template-specific fine-tuning. Tables 1, 5, 6, and 7 report point estimates without error bars, and the 'unseen property' results in Appendix E still use self-constructed pairs. Under these conditions, the reported SOTA margins and the 74% improvement may reflect training/evaluation distribution alignment rather than general structure-preserving editing capability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MolEditRL, a two-stage molecular editing framework. Stage one pretrains a RoBERTa-based transformer with a structure-aware attention bias over a discrete masked graph diffusion process, reconstructing target molecular graphs from source graphs plus natural language instructions. Stage two fine-tunes the diffusion policy with a KL-regularized reinforcement learning objective, using property rewards computed by RDKit and TDC oracles, with gradient estimation via x0-parameterization. To evaluate the method, the authors construct MolEdit-Instruct, a dataset of roughly 3 million source–target molecular pairs over 10 chemical properties and 20 single-property editing tasks plus multi-property combinations, generated by MMP/BRICS fragmentation with template prompts. Experiments report that MolEditRL outperforms BioT5, DrugAssist, GeLLM3O, MolGen, and REINVENT4 on editing accuracy at Tanimoto thresholds 0.65 and 0.15, achieves lower FCD, and does so with substantially fewer parameters, including a claimed 74% improvement in editing success rate.","tokens_in":23607,"tokens_out":5221,"duration_ms":78882,"significance":"If the comparative evaluation is valid, the paper makes a useful contribution: it explicitly models the discrete graph structure of molecules during editing, introduces a structure-preserving RL fine-tuning stage with a KL penalty to the pretrained prior, releases a large public dataset, and reports consistently strong numbers across single- and multi-property tasks. The use of standard oracles (RDKit, TDC), the relatively small model footprint, and the qualitative scaffold-preservation results are concrete strengths. However, the central SOTA claim rests on a self-constructed benchmark whose test distribution is generated by the same pipeline as the training data and on which the baselines are not adapted, and all reported results are single-run point estimates. These issues are load-bearing for the headline 'significantly outperforms' claim, so the conclusions are plausible but not yet established.","major_comments":[{"comment":"The central SOTA claim relies entirely on MolEdit-Instruct, which is constructed by the authors from MMP/BRICS pairs and the template prompts in Table 4. Because the test set is sampled from the same pipeline and prompt templates as the training set, while the baselines (DrugAssist, BioT5, REINVENT4, MolGen, GeLLM3O) are used without template-specific adaptation—and GeLLM3O-P(6) was trained on six properties but evaluated on ten—the comparison conflates model capability with train/test distribution alignment. The reported 74% margin in the Abstract may therefore be a benchmark artifact. I would need an evaluation on at least one independently constructed benchmark (e.g., MolOpt-Instructions or MuMOInstruct) or an external validation set, plus baseline fine-tuning or adapter training on the MolEdit-Instruct templates, before the headline comparative claim is supported.","section":"§4.1, Appendix C"},{"comment":"All quantitative results are single-run point estimates; no error bars, confidence intervals, or multiple seeds are reported anywhere in the main text or appendix. The phrases 'significantly outperforms' (Abstract, Section 1) and 'consistently achieves the highest' (Section 4.4) are used without a statistical test. Given the noise inherent in RL fine-tuning and stochastic diffusion sampling, the reported margins over DrugAssist and GeLLM3O cannot be assessed for robustness. Please report at least three seeds with standard deviations for all main metrics, and a paired significance test for the principal comparisons.","section":"Tables 1, 5, 6, 7; Figures 3–5"},{"comment":"The reward function is underspecified. Eq. (8) defines r only as 'property improvements through chemical-rule-based oracles', and Table 2 lists only 'Reward Calculation (0, 0.2, 1.0)' with no mapping to the binary oracles. It is not clear whether r includes a Tanimoto-similarity term, how a direction/magnitude threshold for each property is applied, how multi-property rewards are aggregated, or how invalid molecules are penalized. Because the same oracles and thresholds appear to underlie the Accall evaluation metrics, the RL objective and the evaluation metric may be aligned by construction; please spell out the exact reward formula and confirm that the evaluation thresholds are not used to shape the reward during training.","section":"§3.4, Eq. (8), Table 2"},{"comment":"The gradient derivation is difficult to follow and appears internally inconsistent as written. Eq. (9) places the KL penalty outside the reward expectation, but Eq. (10) includes both an advantage-weighted term and a negative KL derivative; Eq. (14) uses the uncentered reward r while Eq. (10) uses the normalized advantage A-hat. Since this objective is the core training mechanism, please state the exact surrogate loss used in the implementation, clarify the sign convention of the KL term, and explain how the x0-parameterization in Eqs. (12)–(13) is converted into the reward-weighted cross-entropy loss in Eq. (14).","section":"Eqs. (9)–(14)"}],"minor_comments":[{"comment":"The '74% improvement' and '98% fewer parameters' claims need a concrete reference: which baseline and which metric define the 74%, and what is the parameter count of the comparison model?","section":"Abstract, Section 1"},{"comment":"The caption reads 'FCD at a fixed threshold of 0.15', but FCD is not thresholded by Tanimoto similarity; this appears to be a typo that should be corrected.","section":"Figure 5(b) caption"},{"comment":"The explanation of the ± notation is confusing: 'positive values for property reduction and negative values for property increase' inverts the usual arrow convention (↑/↓). Please clarify the sign convention for each property direction.","section":"Table 3"},{"comment":"The baseline description notes that GeLLM3O-P(6) was trained on six properties but is evaluated on ten. If the baseline cannot be fine-tuned to the full property set, this limitation should be stated in the table caption so readers can weigh the comparison fairly.","section":"Section 4.2"},{"comment":"The 'unseen property' experiments are not zero-shot: MolEditRL is fine-tuned with property-specific oracles for BBBP, HIA, and hERG. The text should be reworded to say 'adaptation to new reward oracles' rather than implying generalization without any task-specific training.","section":"Appendix E"},{"comment":"The attention bias indexing assumes contiguous source and target atom positions; please clarify how padding, variable-length graphs, and graphs with different node orderings are handled in the implementation.","section":"Eq. (3)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is technically interesting and the method is well-motivated, but the evaluation infrastructure is the main risk. The authors would benefit from engaging with an external benchmark and reporting seed variance; without that, the SOTA claim is vulnerable to the self-construction critique. I see no citation or novelty-disclosure concern beyond the benchmark issue. The paper fits the scope of the journal if the empirical claims are hardened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nThe thing to know: MolEditRL is a competent two-stage molecular editing system, and MolEdit-Instruct is a potentially useful dataset. The 74% improvement headline, though, is only as credible as the self-built benchmark it's measured on, and the paper doesn't give enough to check it.\n\nWhat's actually new is modest but real. The structure-aware attention bias—source adjacency injected at all layers, target adjacency at the first layer, then propagated—is a simple and sensible way to bias discrete graph diffusion toward scaffold preservation. Combining that with KL-regularized x0-parameterized policy-gradient fine-tuning on an instruction-conditioned diffusion model is not in the cited prior work. The dataset itself is a genuine asset if it ships: 3M MMP/BRICS pairs, 10 properties, single- and multi-property prompts. That alone is worth something.\n\nThe methods section is coherent and the mechanisms check out. Using GDPO-style gradient estimation with a KL term to keep the fine-tuned policy near the pretrained prior is a sound way to balance reward and structure. I believe the system does what the authors say.\n\nThe soft spot is the evaluation, and it's load-bearing. The test pairs are generated by the same MMP/BRICS pipeline as training, and baselines are not adapted to those templates: GeLLM3O-P(6) is tested on ten properties after training on six; DrugAssist, BioT5, and REINVENT4 are essentially used as released. In that setting, the large margins in Tables 1–7 could reflect train/test distribution alignment more than general editing ability. Every number is a point estimate—no seeds, no error bars. The \"unseen property\" experiments still use pairs constructed the same way, and only MolEditRL is fine-tuned to those tasks. Also, the reward in Eq. 8 is the property oracle alone; structural preservation is carried by the KL prior, not by an explicit similarity term. That is a defensible design choice, but it means the structural-fidelity claim is softer than the name implies. Finally, the abstract says the dataset is on Hugging Face, but I see no URL or code link anywhere in the text.\n\nI'd send this to review rather than desk-reject. The system is sensible, and the dataset could be a real contribution. But the referee needs to push on benchmark fairness, request multiple seeds, released code/data, an external test set, and properly adapted baselines before the SOTA claim can be taken at face value.\n\nBest,","headline":"Sensible graph-diffusion + RL editing system and a large new dataset, but the headline 74% margin sits on a self-built testbed that needs external confirmation.","tokens_in":24146,"tokens_out":3291,"would_cite":false,"duration_ms":33054,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MolEditRL claims that discrete graph diffusion pretraining followed by KL-regularized reinforcement learning yields state-of-the-art structure-preserving molecular editing with far fewer parameters than large language models.","keywords":["molecular editing","discrete graph diffusion","reinforcement learning","structure preservation","property optimization","instruction following","benchmark dataset","drug discovery"],"falsifier":"Run MolEditRL and the compared baselines on an independently constructed test set whose source molecules have low Tanimoto similarity to every MolEdit-Instruct training molecule, with property changes and instructions drawn from a different source; if the reported margin over the best baseline shrinks or disappears, the claim of general structure-preserving editing capability would be falsified.","tokens_in":23160,"feed_emoji":"🧪","tokens_out":9455,"duration_ms":92054,"temperature":0.7,"pith_summary":"MolEditRL sets out to show that molecular editing is best done on the molecular graph itself, not on a text string. The paper builds a compact transformer that reads a source molecule, a target molecule, and a natural-language instruction together, injects bond connectivity directly into attention, and is trained in two stages: first a discrete diffusion pass that reconstructs targets from masked graphs, then a reinforcement-learning pass that rewards property changes while a KL term pulls the output back toward the pretrained structure-aware distribution. To evaluate this, the authors construct MolEdit-Instruct, three million source–target pairs spanning ten chemical properties and twenty single- and multi-property editing tasks. On that benchmark they report the highest accuracy at strict and relaxed similarity thresholds across all tasks, the lowest distributional distance, a 74% improvement in editing success rate over leading baselines, and a parameter count 98% smaller than the largest comparison model. The caring point is that precise, localized molecular edits may not need a very large language model at all.","feed_headline":"Graph diffusion plus RL edits molecules while keeping scaffolds intact","feed_subtitle":"It reports a 74% editing success gain over baselines using 2% of the parameters.","key_machinery":"The workhorse is the Structure-Preserving Editing Network, a transformer with a graph-aware attention bias: at every layer the bias re-injects the source molecule's adjacency, at the first layer it injects the target adjacency, and at later layers it reuses the previous layer's attention, so the model's information flow is constrained by molecular topology rather than by token order alone. Around that network sits a two-stage training loop. Stage one is x0-parameterized discrete diffusion: a forward process masks atoms and bonds of the target graph, and the network learns to predict the clean graph conditioned on source graph and instruction, with an auxiliary cross-entropy term that keeps the instruction tokens aligned. Stage two is editing-aware reinforcement learning: the denoising trajectory is treated as an MDP, rewards from chemical property oracles arrive only at the final molecule, and a policy-gradient objective with x0-parameterization turns those rewards into a weighted cross-entropy loss over atoms and bonds, regularized by the KL divergence to the pretrained structure-aware prior. The KL term is the mechanism that prevents reward chasing from destroying the scaffold.","core_discovery":"The central claim is that a discrete graph diffusion model fine-tuned with a KL-regularized, editing-aware reinforcement-learning objective can edit molecules both accurately and structure-preservingly, outperforming larger sequence-based and instruction-tuned language models. MolEditRL encodes instruction tokens and source/target atom tokens into one sequence, adds learnable biases that carry bond-level connectivity into the attention scores, and pretrains the resulting network to reverse a masking process that corrupts atom and bond types of the target graph. Fine-tuning then frames each reverse-diffusion step as an action in a Markov decision process, scores the final molecule with deterministic chemical property oracles, normalizes rewards per batch, and optimizes a reward-weighted cross-entropy loss over atoms and bonds with an extra KL term toward the pretrained prior; gradients are taken at a fixed stride of denoising steps to keep training stable. Across the MolEdit-Instruct benchmark the paper reports the best editing accuracy at Tanimoto thresholds 0.65 and 0.15 for every single- and multi-property task, the best Fréchet ChemNet Distance, and a 74% relative gain in editing success rate while using about 2% of the parameters of the largest compared model.","pith_inferences":["Editorial inference: because the benchmark pairs are generated by fragment-based matched molecular pairs, the reported gains may be concentrated in local edits such as substituent swaps; testing on edits that require ring-system changes would clarify whether the structure-preservation claim holds for larger topological rewrites.","Editorial inference: the recipe—graph-conditioned diffusion pretraining plus KL-regularized policy fine-tuning with deterministic oracle rewards—is portable to other discrete structured prediction tasks wherever a cheap oracle exists.","Editorial inference: a head-to-head ablation that keeps the RL objective identical but replaces the graph decoder with a SELFIES or SMILES decoder would isolate how much of the 74% gain comes from the graph representation itself rather than from the reward design."],"forward_implications":["Editing at a strict Tanimoto threshold of 0.65 is reported highest for MolEditRL on every single- and multi-property task, so if correct, instruction-following edits that stay close to the source scaffold are achievable without a large language model.","The same pretrained backbone adapts to three held-out properties by swapping in new reward oracles, so new optimization objectives can be added after deployment without retraining the diffusion model.","Because the model uses about 2% of the parameters of the largest compared model, practical molecular editing systems could run on much smaller infrastructure while maintaining or improving fidelity.","KL regularization against the pretrained denoiser keeps validity and similarity stable through RL fine-tuning, which is the property that makes the two-stage recipe usable for multi-objective tasks with three or four simultaneous constraints."],"supporting_citations":[{"why":"Supplies the x0-parameterized discrete denoising machinery used for both pretraining and gradient estimation.","marker":"[2]"},{"why":"Provides the cheminformatics toolkit used to compute validity, property values, and structural similarity in rewards and metrics.","marker":"[3]"},{"why":"Defines the stepwise diffusion-policy-optimization baseline whose instability motivates MolEditRL's full-trajectory KL-regularized objective.","marker":"[4]"},{"why":"Instruction-tuned LLM baseline for multi-property optimization that MolEditRL is compared against.","marker":"[8]"},{"why":"SELFIES-based chemical-feedback generator used as a baseline that achieves full validity but fails structural-similarity constraints.","marker":"[9]"},{"why":"Supplies the curated property oracles used to compute editing rewards.","marker":"[13]"},{"why":"Provides the x0-parameterized graph diffusion policy optimization formulation that underlies the RL gradient estimate.","marker":"[23]"},{"why":"Cross-modal SELFIES/text baseline that represents the sequence-based paradigm MolEditRL is designed to beat.","marker":"[28]"},{"why":"Earlier unified text-graph diffusion work that informs the structure-aware attention design of the editing network.","marker":"[35]"},{"why":"Supplies the LLM dialogue editing baseline and the matched-molecular-pair construction procedure that MolEdit-Instruct follows.","marker":"[37]"}],"fun_headline_variants":["Discrete diffusion + RL improves molecular edit success by 74%","Structure-preserving molecular editing with 98% fewer parameters","Graph diffusion and RL cut molecular editing params by 98%","74% better molecular editing via graph diffusion and RL","MolEditRL: discrete diffusion and RL keep molecular scaffolds intact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the paper's own MolEdit-Instruct benchmark, built from matched molecular pairs and template-generated prompts, fairly represents molecular editing, so that the advantage measured on it will transfer to editing tasks whose molecules and instructions were not generated by the same pipeline.","fun_headline_variants_meta":{"raw":{"variants":["Discrete diffusion + RL improves molecular edit success by 74%","Structure-preserving molecular editing with 98% fewer parameters","Graph diffusion and RL cut molecular editing params by 98%","74% better molecular editing via graph diffusion and RL","MolEditRL: discrete diffusion and RL keep molecular scaffolds intact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001097,"raw_usage":{"total_tokens":4598,"prompt_tokens":984,"completion_tokens":3614,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":3530}},"tokens_in":600,"tokens_out":3614,"duration_ms":26686,"temperature":1.0,"reasoning_tokens":3530,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:58:30.097134+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run MolEditRL and the compared baselines on an independently constructed test set whose source molecules have low Tanimoto similarity to every MolEdit-Instruct training molecule, with property changes and instructions drawn from a different source; if the reported margin over the best baseline shrinks or disappears, the claim of general structure-preserving editing capability would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the x0-parameterized discrete denoising machinery used for both pretraining and gradient estimation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the cheminformatics toolkit used to compute validity, property values, and structural similarity in rewards and metrics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SELFIES-based chemical-feedback generator used as a baseline that achieves full validity but fails structural-similarity constraints."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the x0-parameterized graph diffusion policy optimization formulation that underlies the RL gradient estimate."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the LLM dialogue editing baseline and the matched-molecular-pair construction procedure that MolEdit-Instruct follows."}],"review_version":1}