Pith. sign in

REVIEW 6 minor 22 references

Improving reproducibility of cheminformatics workflows with chembl-downloader

T0 review · 0 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read chembl-downloader makes ChEMBL-derived datasets reproducible by pinning the database version in code.

desk verdict A small, well-scoped software paper that meets its reproducibility claim for ChEMBL data access; the soft spots are documentation-level, not load-bearing. read the letter →

arxiv 2507.17783 v1 pith:B77JALUS submitted 2025-07-23 q-bio.QM

classification q-bio.QM
keywords ChEMBLreproducibilitycheminformaticsdataversioningbioactivityPythonpackageFTPaccess
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

chembl-downloader is a Python package that gives cheminformatics workflows a reproducible route into ChEMBL, the large public database of bioactive molecule activities. The paper's claim is that by exposing version-pinned downloads of ChEMBL's FTP resources, the package lets researchers generate derived datasets whose methodology—selection, filtering, aggregation—can be re-run by anyone, and which can be upgraded to a newer ChEMBL release later. That matters because most published ChEMBL-derived datasets ship without their generating code and quickly fall behind the database: several widely used datasets were built from releases between v20 and v30, while the current release is v35. The paper argues this opacity and staleness harm interpretation, and introduces chembl-downloader as a practical fix.

What carries the argument

The mechanism that carries the argument is a three-layer Python API around the ChEMBL FTP server. The low-level layer maps resource types to deterministic local files, resolving latest or a chosen release, downloading, decompressing, and caching; this is the layer that makes a ChEMBL version part of the code rather than a hidden external fact. The mid-level layer parses those files into queryable structures, including a SQLite database, tabular molecule lists, fingerprint files, and XML, while keeping utilities small so users can extend them. The high-level layer supplies ready-made SQL queries for routinely needed extractions, such as bioactivities for a given assay or target and compounds linked to a publication or patent.

What would settle it

Run the same generation script, with the same package version and the same pinned ChEMBL release, on two machines that have never cached the data, and compare the resulting files byte-for-byte; any difference falsifies the reproducibility claim. A complementary test is to try regenerating a dataset pinned to a release after the ChEMBL FTP server has dropped that release, which shows whether version-pinning survives archive removal.

Watch

Extended reading notes

Core claim

The central claim is that chembl-downloader provides reproducible acquisition, access, and manipulation of ChEMBL data through its FTP server. Concretely, a script can request either the latest or a specified version of a ChEMBL resource—relational database dumps, molecule lists, precomputed fingerprints, the monomer library, or UniProt target mappings—download it, decompress and cache it, then read it through convenient data structures and pre-formatted queries. Because the version is an explicit parameter, running the same script at two different times or on two different machines should produce the same derived dataset, and changing the version parameter should regenerate it against a newer release. Three case studies demonstrate the workflow: reconstructing the dataset of a published deep-learning study, finding 4,266 candidate ChEMBL–ChEBI identifier mappings for curation, and making the code of popular cheminformatics blogs reproducible and version-agnostic.

Load-bearing premise

The reproducibility promise depends on the ChEMBL FTP server continuing to host every requested version of every resource under a stable, versioned layout; if the server removes an old release, renames files, or changes a version's contents after the fact, a pinned script can no longer reconstruct the same dataset.

Editorial extensions

If this is right

  • A published dataset can include its generating script; a reader who runs it obtains the same ChEMBL snapshot and can audit every filtering and aggregation step.
  • Bumping a single version parameter regenerates a dataset on a newer ChEMBL release, so obsolete static datasets can be refreshed without redesigning the pipeline.
  • Reproducible bulk access reduces the need to download, set up, and connect to a local ChEMBL database, lowering the barrier to working with full-database data.
  • Common queries—bioactivities by assay or target, compounds in a publication or patent, compound names—come with ready-made queries, making the package useful without deep database expertise.
  • Because the same versioned files are used across scripts, results from different groups become comparable in principle, even if their analyses differ.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If adopted widely, the de facto standard for publishing ChEMBL-derived data could shift from a frozen supplementary file to a version-pinned script, making reproducibility checks a routine part of peer review.
  • The same version-pinned download-and-cache pattern could extend to other public bioinformatics resources with versioned releases, turning reproducibility into a general tool property rather than a per-project effort.
  • A testable prediction is that routine use of such a tool would shrink the average gap between a published dataset's ChEMBL version and the current release, because updating a dataset would cost one line instead of a rebuild.
  • Pinning versions also enables stability analysis: rerunning one analysis across successive ChEMBL releases shows which conclusions are sensitive to new data, a form of sensitivity testing that static datasets cannot support.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 6 minor

Summary. This paper presents chembl-downloader, a Python package that provides versioned bulk access to ChEMBL data via the ChEMBL FTP server. The author argues that many ChEMBL-derived datasets are published without generation code and quickly become stale, and proposes the package as a way to make derived datasets transparent, updatable, and reproducible. The paper describes low-level download and preprocessing of SQLite, SDF, TSV, fingerprint, monomer XML, and UniProt mapping resources; mid-level parsing utilities; high-level task-specific SQL queries; and three case-study notebooks (re-creating a Deep Confidence dataset, finding ChEMBL-ChEBI identifier mappings via Gilda, and updating blog/example code to use the package). Availability is via GitHub, PyPI, Zenodo, and readthedocs.

Significance. If the package works as described, it addresses a real and timely reproducibility gap in cheminformatics, and the evidence in the manuscript is stronger than for many software notes: source code, documentation, a Zenodo DOI, Jupyter notebooks, and reports of independent use (including by members of the ChEMBL group) are all public. The paper makes no fitted-parameter or circular claims, and the central reproducibility claim is falsifiable by executing the provided notebooks on a pinned ChEMBL version. The main weakness is documentary: the manuscript itself gives no code example or explicit scope statement about the FTP server's retention of old releases. These are local clarifications rather than flaws in the central claim.

minor comments (6)
  1. [Title] The title contains a typo: 'withchembl-downloader' should read 'with chembl-downloader'.
  2. [Summary] In the 'Summary' section, clarify that 'a given version' remains reproducible only while ChEMBL's FTP server retains that version under a stable path; version pinning cannot compensate for upstream removal of old releases.
  3. [Case studies] The 'Case studies' section should report, for each notebook, the ChEMBL version used and the chembl-downloader version or commit hash, so that readers can re-run the notebooks without guessing the environment.
  4. [Case studies] The first case study uses 'reproduce the dataset generation' while also noting intentional changes in aggregation (arithmetic vs geometric mean); please replace 'reproduce' with 're-create' or explicitly state that the notebook is a modified version of the original workflow.
  5. [Availability and usage] The 'Availability and usage' section would benefit from a minimal code snippet (e.g., one call that downloads a ChEMBL release and loads a SQLite table), since the prose currently describes interfaces without showing the API.
  6. [References] The reference to Landrum (n.d.) is incomplete; provide the URL and an access date.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reproducibility claim is grounded in external ChEMBL data, public code repositories, and independent adoption rather than in self-referential derivation.

full rationale

This paper makes no mathematical derivation, fitted-parameter prediction, or uniqueness claim. Its central claim is that the chembl-downloader package provides reproducible acquisition, access, and manipulation of versioned ChEMBL data via the ChEMBL FTP server. The relevant evidence is external to the paper itself: the package is available on PyPI, archived on Zenodo, documented on ReadTheDocs, and demonstrated in Jupyter notebooks; the case studies reference external resources such as the RDKit Blog, Practical Cheminformatics, and Is Life Worth Living? repositories; and adoption is supported by citations of scholarly articles including work by the ChEMBL group itself. The author's own prior packages (pystow, Gilda, Biomappings) appear as implementation components or as tools in case studies, but they do not define the success predicate of the reproducibility claim. The reproducibility claim is appropriately scoped: the paper states that acquisition and pre-processing of 'either the latest or a given version of most resources in the ChEMBL FTP server' is the low-level function, and it does not claim bitwise reproducibility of arbitrary downstream transformations. Dependence on the external FTP server's continued availability and stable layout is a stated scope condition rather than a hidden premise, and version pinning is available for users who need a specific ChEMBL release. No step reduces to its own input by construction, no fitted quantity is renamed as a prediction, and no load-bearing argument depends on an unverified self-citation. The absence of a pinned package version or commit hash in the manuscript is a documentation limitation, not a circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim is a software functionality claim, not a mathematical derivation. It rests on the availability and stability of the ChEMBL FTP server and on the correctness of the package's dependency stack. There are no free parameters, no fitted values, and no invented entities.

assumptions (3)
  • domain assumption The ChEMBL FTP server provides the resources chembl-downloader expects, including relational database dumps, SDF files, TSV files, fingerprint files, monomer XML, and UniProt mappings.
    The package's functionality depends on the external data source's stable layout and contents; this is stated in the 'At a low-level' paragraph of the Summary.
  • domain assumption The Python package and its dependencies (pystow, pandas, RDKit, chemfp) can be installed and operate correctly in a user's environment.
    The low- and mid-level utilities are described as relying on these tools; their correct operation is assumed for the package to work.
  • domain assumption The ChEMBL FTP server can serve previous versions of the data, not only the latest release.
    The paper claims access to 'a given version' of ChEMBL resources; the availability of historical versions on the FTP server is not documented in the text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving reproducibility of cheminformatics workflows with chembl-downloader." pith.science (2026). https://pith.science/paper/B77JALUS

@misc{pith2026250717783,
  author       = {Pith},
  title        = {Pith review of: Improving reproducibility of cheminformatics workflows with chembl-downloader},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B77JALUS}},
  note         = {Machine review of arXiv:2507.17783}
}
read the original abstract

Many modern cheminformatics workflows derive datasets from ChEMBL, but few of these datasets are published with accompanying code for their generation. Consequently, their methodologies (e.g., selection, filtering, aggregation) are opaque, reproduction is difficult, and interpretation of results therefore lacks important context. Further, such static datasets quickly become out-of-date. For example, the current version of ChEMBL is v35 (as of December 2024), but ExCAPE-DB uses v20, Deep Confidence uses v23, the consensus dataset from Isigkeit _et al._ (2022) uses v28, and Papyrus uses v30. Therefore, there is a need for tools that provide reproducible bulk access to the latest (or a given) version of ChEMBL in order to enable researchers to make their derived datasets more transparent, updatable, and trustworthy. This article introduces `chembl-downloader`, a Python package for the reproducible acquisition, access, and manipulation of ChEMBL data through its FTP server. It can be downloaded under the MIT license from https://github.com/cthoyt/chembl-downloader and installed from PyPI with `pip install chembl-downloader.`

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 15 canonical work pages

  1. [1]

    Béquignon, O. J. M., Bongers, B. J., Jespers, W., IJzerman, A. P., Water, B. van der, & Westen, G. J. P. van. (2023). Papyrus: A large-scale curated dataset aimed at bioactivity predictions. Journal of Cheminformatics, 15(1), 3. https://doi.org/10.1186/s13321-022-00672-x

  2. [2]

    Cortés-Ciriano, I., & Bender, A. (2019). Deep Confidence: A Computationally Efficient Framework for Calculating Reliable Prediction Errors for Deep Neural Networks . J. Chem. Inf. Model., 59(3), 1269--1281. https://doi.org/10.1021/acs.jcim.8b00542

  3. [3]

    Dalke, A. (2019). The chemfp project . J. Cheminform., 11(1), 76. https://doi.org/10.1186/s13321-019-0398-8

  4. [4]

    Davies, M., Nowotka, M., Papadatos, G., Dedman, N., Gaulton, A., Atkinson, F., Bellis, L., & Overington, J. P. (2015). ChEMBL web services: streamlining access to drug discovery data and utilities . Nucleic Acids Research, 43(W1), W612--W620. https://doi.org/10.1093/nar/gkv352

  5. [5]

    H., & Colluru, V

    Domingo-Fernández, D., Gadiya, Y., Mubeen, S., Healey, D., Norman, B. H., & Colluru, V. (2023). Exploring the known chemical space of the plant kingdom: Insights into taxonomic patterns, knowledge gaps, and bioactive regions. Journal of Cheminformatics, 15(1), 107. https://doi.org/10.1186/s13321-023-00778-w

  6. [6]

    Gadiya, Y., Gribbon, P., Hofmann-Apitius, M., & Zaliani, A. (2023). Pharmaceutical patent landscaping: A novel approach to understand patents from the drug discovery perspective. Artificial Intelligence in the Life Sciences, 3, 100069. https://doi.org/https://doi.org/10.1016/j.ailsci.2023.100069

  7. [7]

    L., Patricia Bento, A., Chambers, J., Mendez, D., Mutowo, P., Atkinson, F., Bellis, L

    Gaulton, A., Hersey, A., Nowotka, M. L., Patricia Bento, A., Chambers, J., Mendez, D., Mutowo, P., Atkinson, F., Bellis, L. J., Cibrian-Uhalte, E., Davies, M., Dedman, N., Karlsson, A., Magarinos, M. P., Overington, J. P., Papadatos, G., Smit, I., & Leach, A. R. (2017). The ChEMBL database in 2017 . Nucleic Acids Res., 45(D1), D945--D954. https://doi.org/...

  8. [8]

    Gorostiola González, M., Béquignon, O. J. M., Manners, E., Gaulton, A., Mutowo, P., Dawson, E., Zdrazil, B., Leach, A. R., IJzerman, A. P., Heitman, L. H., & al., et. (2024). Excuse me, there is a mutant in my bioactivity soup! A comprehensive analysis of the genetic variability landscape of bioactivity databases and its effect on activity modelling. Chem...

Show all 22 references
  1. [9]

    M., Hoyt, C

    Gyori, B. M., Hoyt, C. T., & Steppi, A. (2022). Gilda: biomedical entity text normalization with machine-learned disambiguation as a service . Bioinformatics Advances. https://doi.org/10.1093/bioadv/vbac034

  2. [10]

    Hastings, J., Owen, G., Dekker, A., Ennis, M., Kale, N., Muthukrishnan, V., Turner, S., Swainston, N., Mendes, P., & Steinbeck, C. (2016). ChEBI in 2016: Improved services and an expanding collection of metabolites . Nucleic Acids Res., 44(D1), D1214--D1219. https://doi.org/10...

  3. [11]

    P., & Young, R

    Hill, A. P., & Young, R. J. (2010). Getting physical in drug discovery: a contemporary perspective on solubility and hydrophobicity . Drug Discov. Today, 15(15), 648--655. https://doi.org/https://doi.org/10.1016/j.drudis.2010.05.016

  4. [12]

    T., Hoyt, A

    Hoyt, C. T., Hoyt, A. L., & Gyori, B. M. (2023). Prediction and Curation of Missing Biomedical Identifier Mappings with Biomappings . Bioinformatics. https://doi.org/10.1093/bioinformatics/btad130

  5. [13]

    Isigkeit, L., Chaikuad, A., & Merk, D. (2022). A Consensus Compound/Bioactivity Dataset for Data-Driven Drug Design and Chemogenomics . Molecules, 27(8). https://doi.org/10.3390/molecules27082513

  6. [14]

    M., Martin, M., Le Novère, N., Parkinson, H., Birney, E., & Jenkinson, A

    Jupp, S., Malone, J., Bolleman, J., Brandizi, M., Davies, M., Garcia, L., Gaulton, A., Gehant, S., Laibe, C., Redaschi, N., Wimalaratne, S. M., Martin, M., Le Novère, N., Parkinson, H., Birney, E., & Jenkinson, A. M. (2014). The EBI RDF platform: linked open data for the life ...

  7. [15]

    Landrum, G. A. (n.d.). RDKit: Open-source cheminformatics. http://www.rdkit.org

  8. [16]

    A., & Riniker, S

    Landrum, G. A., & Riniker, S. (2024). Combining IC50 or ki values from different sources is a source of significant noise. Journal of Chemical Information and Modeling, 64(5), 1560--1567. https://doi.org/10.1021/acs.jcim.4c00049

  9. [17]

    McKinney, W. (2010). D ata S tructures for S tatistical C omputing in P ython. In S. van der Walt & J. Millman (Eds.), P roceedings of the 9th P ython in S cience C onference (pp. 56--61). https://doi.org/10.25080/Majora-92bf1922-00a

  10. [18]

    Nisonoff, H., Wang, Y., & Listgarten, J. (2023). Coherent blending of biophysics-based knowledge with bayesian neural networks for robust protein property prediction. ACS Synthetic Biology, 12(11), 3242--3251. https://doi.org/10.1021/acssynbio.3c00217

  11. [19]

    G., Heitman, L

    Schoenmaker, L., Sastrokarijo, E. G., Heitman, L. H., Beltman, J. B., Jespers, W., & Westen, G. J. P. van. (2025). Towards assay-aware bioactivity model(er)s: Getting a grip on biological context. ChemRxiv. https://doi.org/10.26434/chemrxiv-2025-vnd2c

  12. [20]

    F., Engkvist, O., Carlsson, L., Wegner, J., Ceulemans, H., Georgiev, I., Jeliazkov, V., Kochev, N., Ashby, T

    Sun, J., Jeliazkova, N., Chupakin, V., Golib-Dzib, J. F., Engkvist, O., Carlsson, L., Wegner, J., Ceulemans, H., Georgiev, I., Jeliazkov, V., Kochev, N., Ashby, T. J., & Chen, H. (2017). ExCAPE-DB: An integrated large scale dataset facilitating Big Data analysis in chemogenomi...

  13. [21]

    J., Blackshaw, J., Corbett, S., Veij, M

    Zdrazil, B., Felix, E., Hunter, F., Manners, E. J., Blackshaw, J., Corbett, S., Veij, M. de, Ioannidis, H., Lopez, D. M., Mosquera, J. F., Magarinos, M. P., Bosc, N., Arcila, R., Kizilören, T., Gaulton, A., Bento, A. P., Adasme, M. F., Monecke, P., Landrum, G. A., & Leach, A. ...

  14. [22]

    Zhang, H., Wu, J., Liu, S., & Han, S. (2024). A pre-trained multi-representation fusion network for molecular property prediction. Information Fusion, 103, 102092. https://doi.org/https://doi.org/10.1016/j.inffus.2023.102092 CSLReferences document

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.