REVIEW 6 minor 22 references
Improving reproducibility of cheminformatics workflows with chembl-downloader
T0 review · 0 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read chembl-downloader makes ChEMBL-derived datasets reproducible by pinning the database version in code.
desk verdict A small, well-scoped software paper that meets its reproducibility claim for ChEMBL data access; the soft spots are documentation-level, not load-bearing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is a three-layer Python API around the ChEMBL FTP server. The low-level layer maps resource types to deterministic local files, resolving latest or a chosen release, downloading, decompressing, and caching; this is the layer that makes a ChEMBL version part of the code rather than a hidden external fact. The mid-level layer parses those files into queryable structures, including a SQLite database, tabular molecule lists, fingerprint files, and XML, while keeping utilities small so users can extend them. The high-level layer supplies ready-made SQL queries for routinely needed extractions, such as bioactivities for a given assay or target and compounds linked to a publication or patent.
What would settle it
Run the same generation script, with the same package version and the same pinned ChEMBL release, on two machines that have never cached the data, and compare the resulting files byte-for-byte; any difference falsifies the reproducibility claim. A complementary test is to try regenerating a dataset pinned to a release after the ChEMBL FTP server has dropped that release, which shows whether version-pinning survives archive removal.
Extended reading notes
Core claim
The central claim is that chembl-downloader provides reproducible acquisition, access, and manipulation of ChEMBL data through its FTP server. Concretely, a script can request either the latest or a specified version of a ChEMBL resource—relational database dumps, molecule lists, precomputed fingerprints, the monomer library, or UniProt target mappings—download it, decompress and cache it, then read it through convenient data structures and pre-formatted queries. Because the version is an explicit parameter, running the same script at two different times or on two different machines should produce the same derived dataset, and changing the version parameter should regenerate it against a newer release. Three case studies demonstrate the workflow: reconstructing the dataset of a published deep-learning study, finding 4,266 candidate ChEMBL–ChEBI identifier mappings for curation, and making the code of popular cheminformatics blogs reproducible and version-agnostic.
Load-bearing premise
The reproducibility promise depends on the ChEMBL FTP server continuing to host every requested version of every resource under a stable, versioned layout; if the server removes an old release, renames files, or changes a version's contents after the fact, a pinned script can no longer reconstruct the same dataset.
Editorial extensions
If this is right
- A published dataset can include its generating script; a reader who runs it obtains the same ChEMBL snapshot and can audit every filtering and aggregation step.
- Bumping a single version parameter regenerates a dataset on a newer ChEMBL release, so obsolete static datasets can be refreshed without redesigning the pipeline.
- Reproducible bulk access reduces the need to download, set up, and connect to a local ChEMBL database, lowering the barrier to working with full-database data.
- Common queries—bioactivities by assay or target, compounds in a publication or patent, compound names—come with ready-made queries, making the package useful without deep database expertise.
- Because the same versioned files are used across scripts, results from different groups become comparable in principle, even if their analyses differ.
Reading between the lines
- If adopted widely, the de facto standard for publishing ChEMBL-derived data could shift from a frozen supplementary file to a version-pinned script, making reproducibility checks a routine part of peer review.
- The same version-pinned download-and-cache pattern could extend to other public bioinformatics resources with versioned releases, turning reproducibility into a general tool property rather than a per-project effort.
- A testable prediction is that routine use of such a tool would shrink the average gap between a published dataset's ChEMBL version and the current release, because updating a dataset would cost one line instead of a rebuild.
- Pinning versions also enables stability analysis: rerunning one analysis across successive ChEMBL releases shows which conclusions are sensitive to new data, a form of sensitivity testing that static datasets cannot support.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents chembl-downloader, a Python package that provides versioned bulk access to ChEMBL data via the ChEMBL FTP server. The author argues that many ChEMBL-derived datasets are published without generation code and quickly become stale, and proposes the package as a way to make derived datasets transparent, updatable, and reproducible. The paper describes low-level download and preprocessing of SQLite, SDF, TSV, fingerprint, monomer XML, and UniProt mapping resources; mid-level parsing utilities; high-level task-specific SQL queries; and three case-study notebooks (re-creating a Deep Confidence dataset, finding ChEMBL-ChEBI identifier mappings via Gilda, and updating blog/example code to use the package). Availability is via GitHub, PyPI, Zenodo, and readthedocs.
Significance. If the package works as described, it addresses a real and timely reproducibility gap in cheminformatics, and the evidence in the manuscript is stronger than for many software notes: source code, documentation, a Zenodo DOI, Jupyter notebooks, and reports of independent use (including by members of the ChEMBL group) are all public. The paper makes no fitted-parameter or circular claims, and the central reproducibility claim is falsifiable by executing the provided notebooks on a pinned ChEMBL version. The main weakness is documentary: the manuscript itself gives no code example or explicit scope statement about the FTP server's retention of old releases. These are local clarifications rather than flaws in the central claim.
minor comments (6)
- [Title] The title contains a typo: 'withchembl-downloader' should read 'with chembl-downloader'.
- [Summary] In the 'Summary' section, clarify that 'a given version' remains reproducible only while ChEMBL's FTP server retains that version under a stable path; version pinning cannot compensate for upstream removal of old releases.
- [Case studies] The 'Case studies' section should report, for each notebook, the ChEMBL version used and the chembl-downloader version or commit hash, so that readers can re-run the notebooks without guessing the environment.
- [Case studies] The first case study uses 'reproduce the dataset generation' while also noting intentional changes in aggregation (arithmetic vs geometric mean); please replace 'reproduce' with 're-create' or explicitly state that the notebook is a modified version of the original workflow.
- [Availability and usage] The 'Availability and usage' section would benefit from a minimal code snippet (e.g., one call that downloads a ChEMBL release and loads a SQLite table), since the prose currently describes interfaces without showing the API.
- [References] The reference to Landrum (n.d.) is incomplete; provide the URL and an access date.
Circularity Check
No significant circularity: the reproducibility claim is grounded in external ChEMBL data, public code repositories, and independent adoption rather than in self-referential derivation.
full rationale
This paper makes no mathematical derivation, fitted-parameter prediction, or uniqueness claim. Its central claim is that the chembl-downloader package provides reproducible acquisition, access, and manipulation of versioned ChEMBL data via the ChEMBL FTP server. The relevant evidence is external to the paper itself: the package is available on PyPI, archived on Zenodo, documented on ReadTheDocs, and demonstrated in Jupyter notebooks; the case studies reference external resources such as the RDKit Blog, Practical Cheminformatics, and Is Life Worth Living? repositories; and adoption is supported by citations of scholarly articles including work by the ChEMBL group itself. The author's own prior packages (pystow, Gilda, Biomappings) appear as implementation components or as tools in case studies, but they do not define the success predicate of the reproducibility claim. The reproducibility claim is appropriately scoped: the paper states that acquisition and pre-processing of 'either the latest or a given version of most resources in the ChEMBL FTP server' is the low-level function, and it does not claim bitwise reproducibility of arbitrary downstream transformations. Dependence on the external FTP server's continued availability and stable layout is a stated scope condition rather than a hidden premise, and version pinning is available for users who need a specific ChEMBL release. No step reduces to its own input by construction, no fitted quantity is renamed as a prediction, and no load-bearing argument depends on an unverified self-citation. The absence of a pinned package version or commit hash in the manuscript is a documentation limitation, not a circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption The ChEMBL FTP server provides the resources chembl-downloader expects, including relational database dumps, SDF files, TSV files, fingerprint files, monomer XML, and UniProt mappings.
- domain assumption The Python package and its dependencies (pystow, pandas, RDKit, chemfp) can be installed and operate correctly in a user's environment.
- domain assumption The ChEMBL FTP server can serve previous versions of the data, not only the latest release.
Cite this review
Pith. "Pith review of Improving reproducibility of cheminformatics workflows with chembl-downloader." pith.science (2026). https://pith.science/paper/B77JALUS
@misc{pith2026250717783,
author = {Pith},
title = {Pith review of: Improving reproducibility of cheminformatics workflows with chembl-downloader},
year = {2026},
howpublished = {\url{https://pith.science/paper/B77JALUS}},
note = {Machine review of arXiv:2507.17783}
}
read the original abstract
Many modern cheminformatics workflows derive datasets from ChEMBL, but few of these datasets are published with accompanying code for their generation. Consequently, their methodologies (e.g., selection, filtering, aggregation) are opaque, reproduction is difficult, and interpretation of results therefore lacks important context. Further, such static datasets quickly become out-of-date. For example, the current version of ChEMBL is v35 (as of December 2024), but ExCAPE-DB uses v20, Deep Confidence uses v23, the consensus dataset from Isigkeit _et al._ (2022) uses v28, and Papyrus uses v30. Therefore, there is a need for tools that provide reproducible bulk access to the latest (or a given) version of ChEMBL in order to enable researchers to make their derived datasets more transparent, updatable, and trustworthy. This article introduces `chembl-downloader`, a Python package for the reproducible acquisition, access, and manipulation of ChEMBL data through its FTP server. It can be downloaded under the MIT license from https://github.com/cthoyt/chembl-downloader and installed from PyPI with `pip install chembl-downloader.`
Reference graph
Works this paper leans on
-
[1]
Béquignon, O. J. M., Bongers, B. J., Jespers, W., IJzerman, A. P., Water, B. van der, & Westen, G. J. P. van. (2023). Papyrus: A large-scale curated dataset aimed at bioactivity predictions. Journal of Cheminformatics, 15(1), 3. https://doi.org/10.1186/s13321-022-00672-x
-
[2]
Cortés-Ciriano, I., & Bender, A. (2019). Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Prediction Errors for Deep Neural Networks . J. Chem. Inf. Model., 59(3), 1269--1281. https://doi.org/10.1021/acs.jcim.8b00542
-
[3]
Dalke, A. (2019). The chemfp project . J. Cheminform., 11(1), 76. https://doi.org/10.1186/s13321-019-0398-8
-
[4]
Davies, M., Nowotka, M., Papadatos, G., Dedman, N., Gaulton, A., Atkinson, F., Bellis, L., & Overington, J. P. (2015). ChEMBL web services: streamlining access to drug discovery data and utilities . Nucleic Acids Research, 43(W1), W612--W620. https://doi.org/10.1093/nar/gkv352
-
[5]
Domingo-Fernández, D., Gadiya, Y., Mubeen, S., Healey, D., Norman, B. H., & Colluru, V. (2023). Exploring the known chemical space of the plant kingdom: Insights into taxonomic patterns, knowledge gaps, and bioactive regions. Journal of Cheminformatics, 15(1), 107. https://doi.org/10.1186/s13321-023-00778-w
-
[6]
Gadiya, Y., Gribbon, P., Hofmann-Apitius, M., & Zaliani, A. (2023). Pharmaceutical patent landscaping: A novel approach to understand patents from the drug discovery perspective. Artificial Intelligence in the Life Sciences, 3, 100069. https://doi.org/https://doi.org/10.1016/j.ailsci.2023.100069
arXiv 2023
-
[7]
L., Patricia Bento, A., Chambers, J., Mendez, D., Mutowo, P., Atkinson, F., Bellis, L
Gaulton, A., Hersey, A., Nowotka, M. L., Patricia Bento, A., Chambers, J., Mendez, D., Mutowo, P., Atkinson, F., Bellis, L. J., Cibrian-Uhalte, E., Davies, M., Dedman, N., Karlsson, A., Magarinos, M. P., Overington, J. P., Papadatos, G., Smit, I., & Leach, A. R. (2017). The ChEMBL database in 2017 . Nucleic Acids Res., 45(D1), D945--D954. https://doi.org/...
-
[8]
Gorostiola González, M., Béquignon, O. J. M., Manners, E., Gaulton, A., Mutowo, P., Dawson, E., Zdrazil, B., Leach, A. R., IJzerman, A. P., Heitman, L. H., & al., et. (2024). Excuse me, there is a mutant in my bioactivity soup! A comprehensive analysis of the genetic variability landscape of bioactivity databases and its effect on activity modelling. Chem...
Show all 22 references
-
[9]
M., Hoyt, C
Gyori, B. M., Hoyt, C. T., & Steppi, A. (2022). Gilda: biomedical entity text normalization with machine-learned disambiguation as a service . Bioinformatics Advances. https://doi.org/10.1093/bioadv/vbac034
2022 doi
-
[10]
Hastings, J., Owen, G., Dekker, A., Ennis, M., Kale, N., Muthukrishnan, V., Turner, S., Swainston, N., Mendes, P., & Steinbeck, C. (2016). ChEBI in 2016: Improved services and an expanding collection of metabolites . Nucleic Acids Res., 44(D1), D1214--D1219. https://doi.org/10...
2016 doi
-
[11]
P., & Young, R
Hill, A. P., & Young, R. J. (2010). Getting physical in drug discovery: a contemporary perspective on solubility and hydrophobicity . Drug Discov. Today, 15(15), 648--655. https://doi.org/https://doi.org/10.1016/j.drudis.2010.05.016
2010 doi
-
[12]
T., Hoyt, A
Hoyt, C. T., Hoyt, A. L., & Gyori, B. M. (2023). Prediction and Curation of Missing Biomedical Identifier Mappings with Biomappings . Bioinformatics. https://doi.org/10.1093/bioinformatics/btad130
2023 doi
-
[13]
Isigkeit, L., Chaikuad, A., & Merk, D. (2022). A Consensus Compound/Bioactivity Dataset for Data-Driven Drug Design and Chemogenomics . Molecules, 27(8). https://doi.org/10.3390/molecules27082513
2022 doi
-
[14]
M., Martin, M., Le Novère, N., Parkinson, H., Birney, E., & Jenkinson, A
Jupp, S., Malone, J., Bolleman, J., Brandizi, M., Davies, M., Garcia, L., Gaulton, A., Gehant, S., Laibe, C., Redaschi, N., Wimalaratne, S. M., Martin, M., Le Novère, N., Parkinson, H., Birney, E., & Jenkinson, A. M. (2014). The EBI RDF platform: linked open data for the life ...
2014 doi
-
[15]
Landrum, G. A. (n.d.). RDKit: Open-source cheminformatics. http://www.rdkit.org
-
[16]
A., & Riniker, S
Landrum, G. A., & Riniker, S. (2024). Combining IC50 or ki values from different sources is a source of significant noise. Journal of Chemical Information and Modeling, 64(5), 1560--1567. https://doi.org/10.1021/acs.jcim.4c00049
2024 doi
-
[17]
McKinney, W. (2010). D ata S tructures for S tatistical C omputing in P ython. In S. van der Walt & J. Millman (Eds.), P roceedings of the 9th P ython in S cience C onference (pp. 56--61). https://doi.org/10.25080/Majora-92bf1922-00a
2010 doi
-
[18]
Nisonoff, H., Wang, Y., & Listgarten, J. (2023). Coherent blending of biophysics-based knowledge with bayesian neural networks for robust protein property prediction. ACS Synthetic Biology, 12(11), 3242--3251. https://doi.org/10.1021/acssynbio.3c00217
2023 doi
-
[19]
G., Heitman, L
Schoenmaker, L., Sastrokarijo, E. G., Heitman, L. H., Beltman, J. B., Jespers, W., & Westen, G. J. P. van. (2025). Towards assay-aware bioactivity model(er)s: Getting a grip on biological context. ChemRxiv. https://doi.org/10.26434/chemrxiv-2025-vnd2c
2025 doi
-
[20]
F., Engkvist, O., Carlsson, L., Wegner, J., Ceulemans, H., Georgiev, I., Jeliazkov, V., Kochev, N., Ashby, T
Sun, J., Jeliazkova, N., Chupakin, V., Golib-Dzib, J. F., Engkvist, O., Carlsson, L., Wegner, J., Ceulemans, H., Georgiev, I., Jeliazkov, V., Kochev, N., Ashby, T. J., & Chen, H. (2017). ExCAPE-DB: An integrated large scale dataset facilitating Big Data analysis in chemogenomi...
2017 doi
-
[21]
J., Blackshaw, J., Corbett, S., Veij, M
Zdrazil, B., Felix, E., Hunter, F., Manners, E. J., Blackshaw, J., Corbett, S., Veij, M. de, Ioannidis, H., Lopez, D. M., Mosquera, J. F., Magarinos, M. P., Bosc, N., Arcila, R., Kizilören, T., Gaulton, A., Bento, A. P., Adasme, M. F., Monecke, P., Landrum, G. A., & Leach, A. ...
2023 doi
-
[22]
Zhang, H., Wu, J., Liu, S., & Han, S. (2024). A pre-trained multi-representation fusion network for molecular property prediction. Information Fusion, 103, 102092. https://doi.org/https://doi.org/10.1016/j.inffus.2023.102092 CSLReferences document
2024
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.