REVIEW 3 major objections 5 minor 27 references
MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learning
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A new two-million-pair molecule-text dataset improves graph-language models for molecular property prediction and zero-shot structure retrieval.
desk verdict A genuinely useful molecule-text dataset whose headline validation is compromised by label leakage; the resource is worth engaging, but the empirical claims need decontamination. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is an annotation-grounded text-generation pipeline followed by contrastive alignment. For each molecule, structural descriptors, computed physicochemical properties, bioactivity values normalized and categorized by pChEMBL thresholds, and synthesis-complexity scores are packed into a structured dictionary. A large language model is prompted to rewrite that dictionary into a 100–500 word academic paragraph that maps scaffolds and functional groups to physicochemical behavior, bioactivity, and synthetic feasibility, with numeric values preserved in <number> tags so quantitative information is not lost in language. The resulting pairs are then used to train a graph encoder and a long-context text encoder with a contrastive CLIP-style objective, so that a molecule graph and its description are pulled together in embedding space and pushed apart from other pairs; it is this aligned embedding space that transfers to property-prediction fine-tuning and zero-shot retrieval.
What would settle it
Compute the overlap between the bioactivity values and assay descriptions in MolTextNet and the molecules and labels in the test splits of the ten property-prediction benchmarks; then retrain the same model with every overlapping molecule's activity text masked or removed and compare AUC and R2. If the advantages shrink or disappear, label leakage from pretraining text is the explanation.
Extended reading notes
Core claim
On its own terms, the paper's claim is that the informativeness of molecule-text data, not model architecture alone, is what unlocks multimodal molecular learning. MolTextNet's descriptions average about 253 words, more than ten times the average of the two existing benchmark corpora, and are generated to reason across structure, properties, and synthesis rather than merely concatenate fields. In the paper's validation, a graph neural network pretrained with a CLIP-style contrastive objective against a long-context text encoder on MolTextNet-300K achieves the highest fine-tuned AUC on all seven classification tasks and the highest R2 (lowest RMSE) on all three regression tasks, compared with the same encoder pretrained on the 30K and 300K prior corpora; classification AUC improves by 1–2% and regression R2 by about 6%. The same pretrained model correctly retrieves the target structure for all seven functional-group queries in a zero-shot setting, while both baseline corpora fail all seven. The authors read these results as evidence that MolTextNet's structure-property-synthesis descriptions teach a graph encoder transferable molecular representations.
Load-bearing premise
The load-bearing premise is that the pretraining text contains no information about the molecules that the downstream evaluation then asks the model to predict; if a test molecule's own assay outcome appears in its generated description, the reported gains could come from memorization rather than learning chemistry.
Editorial extensions
If this is right
- More pretraining data from the same pipeline helps: the 300K sample beats the 50K sample on nearly every benchmark, so the full 2.5M corpus is the resource the paper positions for further scaling.
- Long-context text encoders are part of the recipe: truncating the same descriptions to 512 tokens with a shorter-context encoder reduces or erases the scaling advantage, so dataset design and encoder capacity are coupled.
- Zero-shot structure queries by natural language become feasible after pretraining: all seven tested functional-group queries retrieve the correct structure at top-1, whereas prior corpora retrieve none.
- The paper argues the same paired resource can support synthesis-related prediction, because every description embeds synthesis scores and, where available, reaction conditions.
- Fine-tuning from MolTextNet pretraining transfers across both classification and regression benchmarks, so the learned graph representations are not specialized to one task family.
Reading between the lines
- An unstated risk the paper does not address: if any test-split molecule's own assay outcome appears in its MolTextNet description, the reported AUC and R2 gains could partly reflect label leakage rather than molecular understanding; masking those numeric values and retraining would settle it.
- The retrieval evaluation uses simple functional-group count queries, which a model could answer from keywords alone; an extension would test compositional queries such as 'a basic molecule with two aromatic rings and low permeability' to see whether the alignment encodes relations, not just vocabulary.
- Because the descriptions are written by one LLM from one template, the model may learn template-specific linguistic patterns; an unstated testable extension is whether the same pretraining recipe works when descriptions come from a different generator or from mixed sources.
- The paper leaves text-conditioned generation untested, but the dataset's structured counts and synthesis text are exactly the kind of constraints an inverse-design model could consume, so a natural extension is molecule generation from MolTextNet descriptions.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces MolTextNet, a dataset of approximately 2.5 million molecule-text pairs constructed from ChEMBL35. A synthetic text generation pipeline uses GPT-4o-mini to convert structural features (functional groups, scaffolds, bond counts), computed physicochemical properties, bioactivity assay data (including pChEMBL values and assay descriptions), and synthesis-related scores (SCScore, SAScore, USPTO reaction conditions) into structured scientific descriptions. The authors evaluate the dataset by pretraining CLIP-style GNN-ModernBERT models and fine-tuning the graph encoder on seven OGB classification tasks and three OGB regression tasks, reporting improved AUC and R² relative to encoders pretrained on ChEBI-20 and PubChem-300K. They also report zero-shot structure retrieval on OGBG-MolHIV queries about functional-group counts. The central claims are that MolTextNet is a more informative and larger molecule-text resource than prior datasets and that its use in pretraining improves downstream property prediction and retrieval.
Significance. If the empirical claims are valid, MolTextNet would be a valuable resource for multimodal molecular learning: it is an order of magnitude larger than existing molecule-text datasets, covers diverse chemical space, and its descriptions include structure, property, and synthesis information. The synthetic generation pipeline is clearly described, the dataset is released publicly, and the authors provide extensive appendices with processing details. However, the headline validation results are threatened by a likely label-leakage confound: the pretraining text contains the same ChEMBL bioactivity outcomes that are the basis of several OGB downstream labels, and no overlap analysis or decontamination is reported. Without addressing this, the improved property-prediction performance cannot be attributed to learned molecular understanding. The retrieval experiments are also less compelling because the queries echo structural annotations directly included in the pretraining text.
major comments (3)
- [§3.1, §A.2.2, §5.1, Tables 4–5] The pretraining text for each molecule includes its ChEMBL bioactivity outcomes: Section A.2.2 states that pChEMBL values are mapped into three categories ('not active', 'slightly active', 'active') and included in the text, and Section 3.1 further incorporates assay descriptions from ChEMBL35. Downstream evaluation in Section 5.1 uses OGBG classification and regression tasks (HIV, BACE, ToxCast, Tox21, SIDER, ClinTox, MolLipo, etc.) whose labels are derived from ChEMBL-type bioactivity and property endpoints. The manuscript reports no analysis of overlap between the 2.47M MolTextNet molecules and the OGB test molecules, and no decontamination (removing overlapping molecules or masking their bioactivity text). Because CLIP pretraining explicitly trains the graph encoder to predict the paired text content from the molecular graph, a test molecule whose own assay value appears in its description gives the encoder a direct shortcut to the downstream label; the fine-tuned MLP head can then exploit this. Thus the improvements in Tables 4 and 5 may reflect label leakage rather than generalizable molecular representations. Please quantify the molecule-level overlap between MolTextNet and the OGB train/validation/test splits, and re-run the fine-tuning after decontamination (for example, excluding all OGB benchmark molecules from pretraining or masking their assay text), or otherwise demonstrate that the reported gains persist in an independent evaluation.
- [§5.2, §A.6] The structure-retrieval experiment is described as 'zero-shot', but the queries are statements of functional-group counts (e.g., 'The molecule has one Amide group', 'The molecule has eight Carbonyl groups'). These exact structural annotations are part of the pretraining text, as described in Sections 3.1 and A.2.2 ('we extract ... functional group frequencies ... and convert these into descriptive sentences'). Since the test molecules are from OGBG-MolHIV, which is a ChEMBL-derived benchmark, and since MolTextNet is built from all of ChEMBL35, it is highly likely that the test molecules and their full structural descriptions were seen during pretraining. The observed retrieval accuracy may therefore reflect memorization of the paired text rather than compositional zero-shot generalization. Please evaluate retrieval on molecules that are not present in the pretraining set, or otherwise mask the structural annotation fields for the test molecules, so that the result can meaningfully be called zero-shot.
- [§3.3] The quality-control section describes only format-level checks: description length, repetition, and consistency of compound name and SMILES. It does not verify that the LLM-generated text faithfully preserves the numeric bioactivity values, physicochemical properties, or synthesis scores from the annotations, nor that the text is free of hallucinations. Since the generated text is used as the ground-truth modality in CLIP pretraining, any systematic fabrication or numeric corruption in the text would undermine the claimed 'factual alignment with reference data' (Section 3.1) and could confound the downstream gains. Please add a quantitative audit of a random sample (for example, comparing extracted numbers and activity categories in the generated text against the original ChEMBL records) and report the corresponding accuracy.
minor comments (5)
- [Table 4] Several entries report a standard deviation of exactly 0.000 (e.g., MolTextNet-50K on BACE, 0.841±0.000), while other entries have large standard deviations (e.g., PubChem-300K on SIDER, 0.602±0.078). Please report the number of random seeds and the variance estimation procedure, and consider whether the 'consistently achieves' wording is supported by the confidence intervals.
- [§5.2, Figures 5–13] The captions say the model 'consistently retrieves the right structure', but the exact evaluation metric is not stated in Section 5.2; the appendix shows top-3 images. Please define whether retrieval is scored by exact SMILES match, by scaffold match, or by presence of the queried functional groups, and report top-1/top-3 accuracy with quantitative numbers.
- [§3.3] The post-generation filter rejects outputs 'fewer than 100 characters', but the prompt requests 100–500 words. A 100-character output is far below the requested length, so this threshold appears extremely permissive; please report the distribution of generated lengths after filtering and the fraction of regenerated samples.
- [Figure 4] The bar chart shows percentages such as 40.8%, 35.0%, and 63.1% without labeling the denominator or the exact keyword categories. Please clarify whether these are the fraction of descriptions containing at least one keyword from the category, and add axis labels and error bars if applicable.
- [§3.2, Table 3] The paper does not specify the exact GPT-4o-mini model snapshot, sampling temperature, or other decoding parameters used during generation. Please provide these details for reproducibility. Additionally, Table 3 shows a maximum of 24,603 ModernBERT tokens for MolTextNet, which far exceeds the 1,536-token limit stated in Section 5; clarify how truncation was applied and what fraction of descriptions were truncated.
Circularity Check
MolTextNet's property-prediction validation may reduce to label leakage: the pretraining text embeds ChEMBL bioactivity outcomes for the same endpoints evaluated on OGB, with no decontamination reported.
-
fitted input called prediction
[Section 3.1 (Property Annotations), Section 5.1 (Downstream Task 1), Appendix A.2.2]
"We use all molecules from ChEMBL35... For experimental annotations, ChEMBL35 has over 1.7 million assays with 21 million associated bioactivity records, covering binding affinity, biological function, ADME, and toxicity... We use the pChEMBL, i.e., negative logarithm of activity (e.g., IC50, EC50, Ki), and categorize molecules based on thresholds: <5 as "inactive", 5-8 as "slightly active", and >8 as "active". ... We evaluate pretrained GIN encoders on standard molecular property prediction benchmarks from the OGB benchmarks [8]."
MolTextNet is built from the complete ChEMBL35 assay dictionary, and the generated text for each molecule explicitly includes its bioactivity values converted to active/inactive categories. The CLIP pretraining objective trains the GIN to align the molecular graph with this text, so for any molecule the encoder is optimized to map structure to the label information contained in the text. The downstream validation then fine-tunes the same GIN on OGB property-prediction tasks whose labels are bioactivity/property endpoints of the same kind (HIV, ToxCast, BACE, lipophilicity, etc.). The paper never reports removing OGB test molecules from the 2.47M pretraining set, masking their assay text, or analyzing overlap.
full rationale
The dataset construction and retrieval experiments are otherwise largely self-contained; there is no load-bearing self-citation chain or imported uniqueness theorem. The zero-shot structure retrieval task queries functional-group counts that are explicitly part of the pretraining structural annotations, so it functions as an in-distribution alignment check rather than a circular derivation. The main circularity is the missing independence between MolTextNet's pretraining text and the downstream OGB labels in Section 5.1. Because the headline claim ("Pretraining CLIP-style models ... on MolTextNet yields improved performance") rests on Tables 4-5, and because the pretraining text for any overlapping molecule already contains the bioactivity category being predicted, the validation can reduce to the label being an input to pretraining. The paper provides no overlap statistics, no removal of test molecules, and no ablation in which assay text is masked, so the reported gains are not demonstrably due to learned structure-property understanding. This warrants a partial-circularity score of 6. The dataset resource itself may still be useful for other purposes, but the central empirical validation as presented does not rule out leakage.
Assumptions & free parameters
free parameters (3)
- pChEMBL activity thresholds =
5 and 8
- Minimum generated description length =
100 characters
- ModernBERT token limit for MolTextNet =
1536 tokens
assumptions (4)
- domain assumption ChEMBL35 annotations are accurate and complete enough to ground molecule-text descriptions.
- domain assumption GPT-4o-mini generated text is factually aligned with the input annotations.
- ad hoc to paper Bioactivity can be meaningfully reduced to three pChEMBL bins (<5, 5-8, >8).
- domain assumption SMARTS-defined functional groups and 2D graph features capture structure-property relationships relevant for downstream tasks.
Cite this review
Pith. "Pith review of MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learning." pith.science (2026). https://pith.science/paper/MQPMPZUK
@misc{pith2026250600009,
author = {Pith},
title = {Pith review of: MolTextNet: A Two-Million Molecule-Text Dataset for Multimodal Molecular Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/MQPMPZUK}},
note = {Machine review of arXiv:2506.00009}
}
read the original abstract
Small molecules are essential to drug discovery, and graph-language models hold promise for learning molecular properties and functions from text. However, existing molecule-text datasets are limited in scale and informativeness, restricting the training of generalizable multimodal models. We present MolTextNet, a dataset of 2.5 million high-quality molecule-text pairs designed to overcome these limitations. To construct it, we propose a synthetic text generation pipeline that integrates structural features, computed properties, bioactivity data, and synthetic complexity. Using GPT-4o-mini, we create structured descriptions for 2.5 million molecules from ChEMBL35, with text over 10 times longer than prior datasets. MolTextNet supports diverse downstream tasks, including property prediction and structure retrieval. Pretraining CLIP-style models with Graph Neural Networks and ModernBERT on MolTextNet yields improved performance, highlighting its potential for advancing foundational multimodal modeling in molecular science. Our dataset is available at https://huggingface.co/datasets/liuganghuggingface/moltextnet.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
C. W. Coley, L. Rogers, W. H. Green, and K. F. Jensen. Scscore: synthetic complexity learned from a reaction corpus.Journal of chemical information and modeling, 58(2):252–261, 2018
work page 2018
-
[3]
K. Degtyarenko, P. De Matos, M. Ennis, J. Hastings, M. Zbinden, A. McNaught, R. Alcántara, M. Darsow, M. Guedj, and M. Ashburner. Chebi: a database and ontology for chemical entities of biological interest.Nucleic acids research, 36(suppl_1):D344–D350, 2007
work page 2007
-
[4]
C. Edwards, T. Lai, K. Ros, G. Honke, K. Cho, and H. Ji. Translation between molecules and natural language.arXiv preprint arXiv:2204.11817, 2022
arXiv 2022
-
[5]
C. Edwards, C. Zhai, and H. Ji. Text2mol: Cross-modal molecule retrieval with natural language queries. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 595–607, 2021
work page 2021
-
[6]
P. Ertl and A. Schuffenhauer. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions.Journal of cheminformatics, 1:1–11, 2009
work page 2009
-
[7]
Y . Fang, X. Liang, N. Zhang, K. Liu, R. Huang, Z. Chen, X. Fan, and H. Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models.arXiv preprint arXiv:2306.08018, 2023
arXiv 2023
-
[8]
W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec. Open graph benchmark: Datasets for machine learning on graphs.arXiv preprint arXiv:2005.00687, 2020
arXiv 2005
Show all 27 references
-
[9]
W. Hu, B. Liu, J. Gomes, M. Zitnik, P. Liang, V . Pande, and J. Leskovec. Strategies for pre-training graph neural networks.arXiv preprint arXiv:1905.12265, 2019
1905 arXiv
-
[10]
S. Kim, J. Chen, T. Cheng, A. Gindulyte, J. He, S. He, Q. Li, B. A. Shoemaker, P. A. Thiessen, B. Yu, et al. Pubchem in 2021: new data content and improved web interfaces.Nucleic acids research, 49(D1):D1388–D1395, 2021
2021
-
[11]
G. Liu, E. Inae, T. Zhao, J. Xu, T. Luo, and M. Jiang. Data-centric learning from unlabeled graphs with diffusion model.Advances in neural information processing systems, 36:21039– 21057, 2023
2023
-
[12]
G. Liu, S. Seal, J. Arevalo, Z. Liang, A. E. Carpenter, M. Jiang, and S. Singh. Learning molecular representation in a cell.ArXiv, pages arXiv–2406, 2024
2024
-
[13]
G. Liu, M. Sun, W. Matusik, M. Jiang, and J. Chen. Multimodal large language models for inverse molecular design with retrosynthetic planning.arXiv preprint arXiv:2410.04223, 2024
2024 arXiv
-
[14]
G. Liu, J. Xu, T. Luo, and M. Jiang. Graph diffusion transformers for multi-conditional molecular generation.arXiv preprint arXiv:2401.13858, 2024
2024 arXiv
-
[15]
G. Liu, T. Zhao, E. Inae, T. Luo, and M. Jiang. Semi-supervised graph imbalanced regression. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1453–1465, 2023
2023
-
[16]
S. Liu, W. Nie, C. Wang, J. Lu, Z. Qiao, L. Liu, J. Tang, C. Xiao, and A. Anandkumar. Multi- modal molecule structure–text model for text-based retrieval and editing.Nature Machine Intelligence, 5(12):1447–1457, 2023
2023
-
[17]
D. Lowe. Chemical reactions from us patents (1976–sep2016), 2017. Accessed: 2025-05-08
1976
-
[18]
Radford, J
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 10
2021
-
[19]
Rdkit cookbook, 2024
RDKit Project. Rdkit cookbook, 2024. Accessed: 2025-05-08
2024
-
[20]
Schneuing, C
A. Schneuing, C. Harris, Y . Du, K. Didi, A. Jamasb, I. Igashov, W. Du, C. Gomes, T. L. Blundell, P. Lio, et al. Structure-based drug design with equivariant diffusion models.Nature Computational Science, 4(12):899–909, 2024
2024
-
[21]
Taori, I
R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto. Alpaca: A strong, replicable instruction-following model, 2023. Accessed: 2025-05-08
2023
-
[22]
Y . Wang, Y . Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi. Self-instruct: Aligning language models with self-generated instructions.arXiv preprint arXiv:2212.10560, 2022
2022 arXiv
-
[23]
Warner, A
B. Warner, A. Chaffin, B. Clavié, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, et al. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference.arXiv preprint...
2024 arXiv
-
[24]
Z. Wu, B. Ramsundar, E. N. Feinberg, J. Gomes, C. Geniesse, A. S. Pappu, K. Leswing, and V . Pande. Moleculenet: a benchmark for molecular machine learning.Chemical science, 9(2):513–530, 2018
2018
-
[25]
K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks?arXiv preprint arXiv:1810.00826, 2018
2018 arXiv
-
[26]
Zdrazil, E
B. Zdrazil, E. Felix, F. Hunter, E. J. Manners, J. Blackshaw, S. Corbett, M. de Veij, H. Ioan- nidis, D. M. Lopez, J. F. Mosquera, et al. The chembl database in 2023: a drug discovery platform spanning multiple bioactivity data types and time periods.Nucleic acids research, 52...
2023
-
[27]
not active
Z. Zeng, Y . Yao, Z. Liu, and M. Sun. A deep-learning system bridging molecule structure and biomedical text with comprehension comparable to human professionals.Nature communica- tions, 13(1):862, 2022. 11 A Technical Appendices and Supplementary Material A.1 More Details on ...
2022
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.