{"id":"b4a0eb2e-8310-497c-92ce-8b332ea32126","arxiv_id":"2506.00009","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"MolTextNet is a 2.5 million molecule-text dataset whose GPT-4o-mini descriptions are grounded in ChEMBL35; downstream gains are reported but may be confounded by label leakage.","lead":"The authors built MolTextNet, a dataset pairing 2.5 million drug-like molecules with detailed computer-written descriptions of their structure, properties, and synthesis. A language model wrote the long descriptions, and pretraining a molecule-text model on them produced modest gains in property prediction and structure retrieval.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central empirical claim is not yet supported: MolTextNet's pretraining text embeds ChEMBL bioactivity outcomes, and the paper never removes OGB test molecules from pretraining, so the Tables 4-5 gains may be label leakage rather than learning.","rationale":"The Reader's REJECT verdict is justified. The most load-bearing threat to the central claim is leakage, not architecture or scale. The paper's own pipeline documentation provides strong circumstantial evidence: it feeds complete ChEMBL35 assay dictionaries into the generated text and threshold activity values into verbal classes. OGB benchmarks are drawn from public assay and property collections, so exact molecule overlap is likely. Because the GIN is pretrained to align graph embeddings with text embeddings, label information in the text is transferable to graph-only fine-tuning. The paper's comparisons to PubChem-300K and ChEBI-20 do not control for this because those datasets are derived from PubChem and lack per-molecule bioassay outcomes; hence any advantage could come from label exposure rather than better molecular understanding. The weakness is empirical and testable. I found no internal inconsistency in the dataset construction; the issue is missing decontamination, which is a correctness risk high enough to block acceptance of the empirical claim. I agree with the Reader's weakest assumption, so no change to the REJECT verdict is needed. A secondary caveat is that the zero-shot retrieval evaluation is only seven qualitative cases with queries that mirror pretraining structural counts, but the property-prediction leakage is the load-bearing concern.","tokens_in":15547,"tokens_out":6838,"duration_ms":70187,"concrete_test":"Run a decontamination check: (1) Compute canonical InChIKey overlap between all OGBG scaffold-split test molecules and the 2,474,590 MolTextNet molecules; report overlap per task. (2) For each overlapping molecule, strip from its MolTextNet description every sentence containing bioactivity results, assay descriptions, pChEMBL/activity words, and any downstream-relevant property values, leaving structural and synthesis text intact. (3) Retrain the MolTextNet-300K GIN-ModernBERT CLIP model on this masked corpus and repeat the exact Section 5.1 fine-tuning protocol. If the Table 4-5 gains over ChEBI-20 and PubChem-300K vanish or fall within one standard deviation, label leakage explains the headline result; if the gains persist, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that CLIP pretraining on MolTextNet produces a GNN encoder with better downstream property-prediction performance (Tables 4-5). This requires the pretraining text to be independent of the OGB test labels. That condition is not met as far as the paper shows. Section 3.1 and A.2.2 state that the text for every molecule is built from ChEMBL35's complete assay dictionary including standardized activity values and pChEMBL categories rendered as \"inactive\"/\"slightly active\"/\"active\". Section 5.1 evaluates on OGBG classification and regression tasks whose labels are the same bioactivity and property endpoints that ChEMBL35 records, including HIV inhibition, ToxCast/Tox21, BACE, and lipophilicity-type data. The paper never reports an overlap analysis between MolTextNet's 2.47M molecules and the OGB test scaffolds, nor any removal of overlapping molecules or masking of their assay text. In CLIP training, the GIN is optimized to encode from molecular structure the content of the paired text, so a test molecule whose own ChEMBL assay outcome appears in its description gives the GIN a direct shortcut to the downstream label before the MLP head is ever trained. The same contamination also weakens the zero-shot retrieval demonstration, because the query counts (amide, ketone, etc.) are echoed in the pretraining structural annotations. The dataset resource may still be useful, but the paper's headline validation does not currently rule out label leakage, so the empirical support for the central claim is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MolTextNet, a dataset of approximately 2.5 million molecule-text pairs constructed from ChEMBL35. A synthetic text generation pipeline uses GPT-4o-mini to convert structural features (functional groups, scaffolds, bond counts), computed physicochemical properties, bioactivity assay data (including pChEMBL values and assay descriptions), and synthesis-related scores (SCScore, SAScore, USPTO reaction conditions) into structured scientific descriptions. The authors evaluate the dataset by pretraining CLIP-style GNN-ModernBERT models and fine-tuning the graph encoder on seven OGB classification tasks and three OGB regression tasks, reporting improved AUC and R² relative to encoders pretrained on ChEBI-20 and PubChem-300K. They also report zero-shot structure retrieval on OGBG-MolHIV queries about functional-group counts. The central claims are that MolTextNet is a more informative and larger molecule-text resource than prior datasets and that its use in pretraining improves downstream property prediction and retrieval.","tokens_in":15881,"tokens_out":6792,"duration_ms":67103,"significance":"If the empirical claims are valid, MolTextNet would be a valuable resource for multimodal molecular learning: it is an order of magnitude larger than existing molecule-text datasets, covers diverse chemical space, and its descriptions include structure, property, and synthesis information. The synthetic generation pipeline is clearly described, the dataset is released publicly, and the authors provide extensive appendices with processing details. However, the headline validation results are threatened by a likely label-leakage confound: the pretraining text contains the same ChEMBL bioactivity outcomes that are the basis of several OGB downstream labels, and no overlap analysis or decontamination is reported. Without addressing this, the improved property-prediction performance cannot be attributed to learned molecular understanding. The retrieval experiments are also less compelling because the queries echo structural annotations directly included in the pretraining text.","major_comments":[{"comment":"The pretraining text for each molecule includes its ChEMBL bioactivity outcomes: Section A.2.2 states that pChEMBL values are mapped into three categories ('not active', 'slightly active', 'active') and included in the text, and Section 3.1 further incorporates assay descriptions from ChEMBL35. Downstream evaluation in Section 5.1 uses OGBG classification and regression tasks (HIV, BACE, ToxCast, Tox21, SIDER, ClinTox, MolLipo, etc.) whose labels are derived from ChEMBL-type bioactivity and property endpoints. The manuscript reports no analysis of overlap between the 2.47M MolTextNet molecules and the OGB test molecules, and no decontamination (removing overlapping molecules or masking their bioactivity text). Because CLIP pretraining explicitly trains the graph encoder to predict the paired text content from the molecular graph, a test molecule whose own assay value appears in its description gives the encoder a direct shortcut to the downstream label; the fine-tuned MLP head can then exploit this. Thus the improvements in Tables 4 and 5 may reflect label leakage rather than generalizable molecular representations. Please quantify the molecule-level overlap between MolTextNet and the OGB train/validation/test splits, and re-run the fine-tuning after decontamination (for example, excluding all OGB benchmark molecules from pretraining or masking their assay text), or otherwise demonstrate that the reported gains persist in an independent evaluation.","section":"§3.1, §A.2.2, §5.1, Tables 4–5"},{"comment":"The structure-retrieval experiment is described as 'zero-shot', but the queries are statements of functional-group counts (e.g., 'The molecule has one Amide group', 'The molecule has eight Carbonyl groups'). These exact structural annotations are part of the pretraining text, as described in Sections 3.1 and A.2.2 ('we extract ... functional group frequencies ... and convert these into descriptive sentences'). Since the test molecules are from OGBG-MolHIV, which is a ChEMBL-derived benchmark, and since MolTextNet is built from all of ChEMBL35, it is highly likely that the test molecules and their full structural descriptions were seen during pretraining. The observed retrieval accuracy may therefore reflect memorization of the paired text rather than compositional zero-shot generalization. Please evaluate retrieval on molecules that are not present in the pretraining set, or otherwise mask the structural annotation fields for the test molecules, so that the result can meaningfully be called zero-shot.","section":"§5.2, §A.6"},{"comment":"The quality-control section describes only format-level checks: description length, repetition, and consistency of compound name and SMILES. It does not verify that the LLM-generated text faithfully preserves the numeric bioactivity values, physicochemical properties, or synthesis scores from the annotations, nor that the text is free of hallucinations. Since the generated text is used as the ground-truth modality in CLIP pretraining, any systematic fabrication or numeric corruption in the text would undermine the claimed 'factual alignment with reference data' (Section 3.1) and could confound the downstream gains. Please add a quantitative audit of a random sample (for example, comparing extracted numbers and activity categories in the generated text against the original ChEMBL records) and report the corresponding accuracy.","section":"§3.3"}],"minor_comments":[{"comment":"Several entries report a standard deviation of exactly 0.000 (e.g., MolTextNet-50K on BACE, 0.841±0.000), while other entries have large standard deviations (e.g., PubChem-300K on SIDER, 0.602±0.078). Please report the number of random seeds and the variance estimation procedure, and consider whether the 'consistently achieves' wording is supported by the confidence intervals.","section":"Table 4"},{"comment":"The captions say the model 'consistently retrieves the right structure', but the exact evaluation metric is not stated in Section 5.2; the appendix shows top-3 images. Please define whether retrieval is scored by exact SMILES match, by scaffold match, or by presence of the queried functional groups, and report top-1/top-3 accuracy with quantitative numbers.","section":"§5.2, Figures 5–13"},{"comment":"The post-generation filter rejects outputs 'fewer than 100 characters', but the prompt requests 100–500 words. A 100-character output is far below the requested length, so this threshold appears extremely permissive; please report the distribution of generated lengths after filtering and the fraction of regenerated samples.","section":"§3.3"},{"comment":"The bar chart shows percentages such as 40.8%, 35.0%, and 63.1% without labeling the denominator or the exact keyword categories. Please clarify whether these are the fraction of descriptions containing at least one keyword from the category, and add axis labels and error bars if applicable.","section":"Figure 4"},{"comment":"The paper does not specify the exact GPT-4o-mini model snapshot, sampling temperature, or other decoding parameters used during generation. Please provide these details for reproducibility. Additionally, Table 3 shows a maximum of 24,603 ModernBERT tokens for MolTextNet, which far exceeds the 1,536-token limit stated in Section 5; clarify how truncation was applied and what fraction of descriptions were truncated.","section":"§3.2, Table 3"}],"recommendation":"major_revision","confidential_remarks":"The reviewer's label-leakage concern is real and well grounded in the manuscript text: the pretraining text explicitly encodes ChEMBL bioactivity outcomes, and the downstream OGB benchmarks are known to be ChEMBL-derived. However, the issue is addressable within the scope of a revision: the authors can perform an overlap analysis and decontamination (removing OGB benchmark molecules from pretraining or masking their bioactivity text) and re-run the fine-tuning. If the decontaminated results still show gains, the paper would be strong. If they do not, the current claims would not hold. I therefore recommend major revision rather than outright rejection, but I emphasize that the requested experiments are essential and not optional. I also note that the manuscript has no limitations section; the authors should add one discussing the leakage risk and the measures taken."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: MolTextNet is a real dataset contribution. 2.5 million molecule-text pairs, descriptions over 10x longer than prior datasets, and a transparent pipeline grounded in ChEMBL35, RDKit descriptors, SCScore/SAScore, and USPTO reaction data. The authors also release the dataset. That is concrete value, and I expect the community will use it regardless of what happens with this paper.\n\nWhat is new: the scale and depth are a clear step up from PubChem-300K and ChEBI-20. The structured integration of bioactivity, physicochemical properties, structural motifs, and synthesis metrics into one coherent LLM-generated description is genuinely useful for multimodal molecular learning. The quality control section is honest about filtering and chunking. The topic analysis is a nice touch, even if somewhat descriptive.\n\nThe soft spot is not small: the validation section as written does not support the claim that pretraining on MolTextNet yields generalizable molecular understanding. The text for each molecule includes the molecule's own ChEMBL bioactivity outcomes—pChEMBL categories like \"active\"/\"inactive\", standardized IC50 values, and assay descriptions. The OGB property-prediction benchmarks (HIV, ToxCast, Tox21, BACE, etc.) are built from the same ChEMBL or closely related assay data. The paper never reports an overlap analysis between MolTextNet's molecules and the OGB test scaffolds, nor any masking of bioactivity text for test molecules. So Tables 4 and 5 gains may simply reflect the GIN learning to extract the answer from the paired text during CLIP pretraining, then transferring that shortcut to fine-tuning labels. That is label leakage in its most direct form.\n\nThe zero-shot structure retrieval demonstration is also weaker than it looks. The queries are functional-group counts, and the pretraining text contains exactly those structural annotations. The model can succeed by lexical pattern matching rather than learned molecular understanding. I would not count that as evidence of generalization.\n\nThe saving grace is that the flaw is fixable. The authors need to (1) identify which pretraining molecules overlap with downstream test molecules, (2) remove those molecules or redact their bioactivity text, and (3) re-run the fine-tuning and retrieval experiments. A version where all bioactivity outcomes are masked from pretraining text would directly test whether the model learns structure-property relationships or just memorizes labels. Without that, the empirical claims are unverified.\n\nThe citation pattern and related work look fine. The writing is clear and the pipeline is reproducible in principle.\n\nWho this is for: anyone building multimodal molecular models, especially for property prediction or structure-text retrieval. The paper deserves a serious referee because the resource is substantial and the main flaw is methodological and fixable. I would send it to peer review, but with an expectation of major revision and re-evaluation before acceptance.","headline":"A genuinely useful molecule-text dataset whose headline validation is compromised by label leakage; the resource is worth engaging, but the empirical claims need decontamination.","tokens_in":16407,"tokens_out":1713,"would_cite":true,"duration_ms":18282,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new two-million-pair molecule-text dataset improves graph-language models for molecular property prediction and zero-shot structure retrieval.","keywords":["molecule-text dataset","multimodal molecular learning","CLIP-style pretraining","molecular property prediction","zero-shot structure retrieval","synthetic text generation","graph neural networks","drug discovery"],"falsifier":"Compute the overlap between the bioactivity values and assay descriptions in MolTextNet and the molecules and labels in the test splits of the ten property-prediction benchmarks; then retrain the same model with every overlapping molecule's activity text masked or removed and compare AUC and R2. If the advantages shrink or disappear, label leakage from pretraining text is the explanation.","tokens_in":15349,"feed_emoji":"🧪","tokens_out":10090,"duration_ms":93607,"temperature":0.7,"pith_summary":"MolTextNet is a dataset of roughly 2.5 million molecule-text pairs, built by prompting a large language model to turn structured annotations into scientific descriptions. The annotations cover molecular structure, computed physicochemical properties, bioactivity measurements from over a million assays, and synthesis-complexity scores with reaction conditions. The paper claims that this scale and textual density fix the bottleneck that has limited graph-language molecular models, namely the lack of informative paired text, and demonstrates the claim by pretraining a contrastive graph-text model. After fine-tuning, the graph encoder pretrained on a 300K sample of MolTextNet outperforms encoders pretrained on prior molecule-text corpora on all ten standard property-prediction tasks, and it answers zero-shot structure queries that the prior corpora cannot. The practical stake is that a single pretraining resource could support property prediction, structure retrieval, and eventually text-guided molecular design.","feed_headline":"MolTextNet: 2.5M molecule-text pairs lift property prediction","feed_subtitle":"Graph-language models pretrained on it outperform prior molecule-text datasets on every benchmark tested.","key_machinery":"The load-bearing mechanism is an annotation-grounded text-generation pipeline followed by contrastive alignment. For each molecule, structural descriptors, computed physicochemical properties, bioactivity values normalized and categorized by pChEMBL thresholds, and synthesis-complexity scores are packed into a structured dictionary. A large language model is prompted to rewrite that dictionary into a 100–500 word academic paragraph that maps scaffolds and functional groups to physicochemical behavior, bioactivity, and synthetic feasibility, with numeric values preserved in <number> tags so quantitative information is not lost in language. The resulting pairs are then used to train a graph encoder and a long-context text encoder with a contrastive CLIP-style objective, so that a molecule graph and its description are pulled together in embedding space and pushed apart from other pairs; it is this aligned embedding space that transfers to property-prediction fine-tuning and zero-shot retrieval.","core_discovery":"On its own terms, the paper's claim is that the informativeness of molecule-text data, not model architecture alone, is what unlocks multimodal molecular learning. MolTextNet's descriptions average about 253 words, more than ten times the average of the two existing benchmark corpora, and are generated to reason across structure, properties, and synthesis rather than merely concatenate fields. In the paper's validation, a graph neural network pretrained with a CLIP-style contrastive objective against a long-context text encoder on MolTextNet-300K achieves the highest fine-tuned AUC on all seven classification tasks and the highest R2 (lowest RMSE) on all three regression tasks, compared with the same encoder pretrained on the 30K and 300K prior corpora; classification AUC improves by 1–2% and regression R2 by about 6%. The same pretrained model correctly retrieves the target structure for all seven functional-group queries in a zero-shot setting, while both baseline corpora fail all seven. The authors read these results as evidence that MolTextNet's structure-property-synthesis descriptions teach a graph encoder transferable molecular representations.","pith_inferences":["An unstated risk the paper does not address: if any test-split molecule's own assay outcome appears in its MolTextNet description, the reported AUC and R2 gains could partly reflect label leakage rather than molecular understanding; masking those numeric values and retraining would settle it.","The retrieval evaluation uses simple functional-group count queries, which a model could answer from keywords alone; an extension would test compositional queries such as 'a basic molecule with two aromatic rings and low permeability' to see whether the alignment encodes relations, not just vocabulary.","Because the descriptions are written by one LLM from one template, the model may learn template-specific linguistic patterns; an unstated testable extension is whether the same pretraining recipe works when descriptions come from a different generator or from mixed sources.","The paper leaves text-conditioned generation untested, but the dataset's structured counts and synthesis text are exactly the kind of constraints an inverse-design model could consume, so a natural extension is molecule generation from MolTextNet descriptions."],"forward_implications":["More pretraining data from the same pipeline helps: the 300K sample beats the 50K sample on nearly every benchmark, so the full 2.5M corpus is the resource the paper positions for further scaling.","Long-context text encoders are part of the recipe: truncating the same descriptions to 512 tokens with a shorter-context encoder reduces or erases the scaling advantage, so dataset design and encoder capacity are coupled.","Zero-shot structure queries by natural language become feasible after pretraining: all seven tested functional-group queries retrieve the correct structure at top-1, whereas prior corpora retrieve none.","The paper argues the same paired resource can support synthesis-related prediction, because every description embeds synthesis scores and, where available, reaction conditions.","Fine-tuning from MolTextNet pretraining transfers across both classification and regression benchmarks, so the learned graph representations are not specialized to one task family."],"supporting_citations":[{"why":"Source database supplying the molecules, bioactivity records, assay definitions, and computed properties used to build the dataset.","marker":"[26]"},{"why":"Prior 300K molecule-text corpus that serves as the main baseline for pretraining comparisons.","marker":"[7]"},{"why":"Prior 30K molecule-text corpus used as the smaller baseline.","marker":"[5]"},{"why":"Provides the ten property-prediction tasks and the scaffold-based splits used for downstream evaluation.","marker":"[8]"},{"why":"Supplies the contrastive learning objective that aligns graph and text embeddings.","marker":"[18]"},{"why":"Supplies the long-context text encoder used in the evaluated CLIP-style models.","marker":"[23]"},{"why":"Supplies the graph isomorphism network architecture used as the molecule encoder.","marker":"[25]"},{"why":"The large language model used to generate the synthetic descriptions from structured annotations.","marker":"[1]"},{"why":"Supplies the neural synthesis-complexity score included in the synthesis annotations.","marker":"[2]"},{"why":"Supplies reaction conditions from chemical patents that are incorporated into synthesis descriptions where matches exist.","marker":"[17]"}],"fun_headline_variants":["2.5M molecule-text pairs sharpen property predictions","MolTextNet: richer text, sharper multimodal learning","Tenfold longer molecule texts lift graph-language models","Zero-shot structure retrieval solved with MolTextNet","Informativeness, not architecture, drives multimodal gain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pretraining text contains no information about the molecules that the downstream evaluation then asks the model to predict; if a test molecule's own assay outcome appears in its generated description, the reported gains could come from memorization rather than learning chemistry.","fun_headline_variants_meta":{"raw":{"variants":["2.5M molecule-text pairs sharpen property predictions","MolTextNet: richer text, sharper multimodal learning","Tenfold longer molecule texts lift graph-language models","Zero-shot structure retrieval solved with MolTextNet","Informativeness, not architecture, drives multimodal gain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000245,"raw_usage":{"total_tokens":1544,"prompt_tokens":959,"completion_tokens":585,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":511}},"tokens_in":575,"tokens_out":585,"duration_ms":5808,"temperature":1.0,"reasoning_tokens":511,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:05:43.605811+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the overlap between the bioactivity values and assay descriptions in MolTextNet and the molecules and labels in the test splits of the ten property-prediction benchmarks; then retrain the same model with every overlapping molecule's activity text masked or removed and compare AUC and R2. If the advantages shrink or disappear, label leakage from pretraining text is the explanation.","supporting_citations":[{"cited_title":"Zdrazil, E","cited_arxiv_id":null,"evidence_quote":"Source database supplying the molecules, bioactivity records, assay definitions, and computed properties used to build the dataset."},{"cited_title":"Edwards, C","cited_arxiv_id":null,"evidence_quote":"Prior 30K molecule-text corpus used as the smaller baseline."},{"cited_title":"Radford, J","cited_arxiv_id":null,"evidence_quote":"Supplies the contrastive learning objective that aligns graph and text embeddings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the neural synthesis-complexity score included in the synthesis annotations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies reaction conditions from chemical patents that are incorporated into synthesis descriptions where matches exist."}],"review_version":1}