{"id":"f49db493-84b0-4091-8c90-00fca43a4cc3","arxiv_id":"2507.17783","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"chembl-downloader is a Python package that provides reproducible, versioned access to ChEMBL data, demonstrated by case studies that regenerate prior datasets and suggest new identifier mappings.","lead":"This paper presents chembl-downloader, a Python tool that fetches ChEMBL data in a repeatable way and lets researchers regenerate their datasets. It matters because many published bioactivity datasets are built from hidden or outdated downloads, and this tool makes them checkable and updateable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified; the reproducibility claim is appropriately scoped and well supported by public code, packaging, and adoption evidence.","rationale":"This is a short software note whose strongest claim is modest: with chembl-downloader, a researcher can fetch a specified ChEMBL release and access it through common formats and pre-built queries. The reader's weakest assumption about external FTP stability is real, but it is an operational dependency rather than a flaw in the package's argument: no software wrapper can guarantee that an upstream host retains files forever, and the paper explicitly scopes the package to the FTP server. The code is publicly available, archived on Zenodo, installable from PyPI, and cited by independent groups, which provides credible support for the central claim. The absence of a pinned package version in the manuscript is worth noting, but a user of the package can specify versions in their own workflow, and the paper's central argument does not depend on bitwise reproduction of its illustrative notebooks. I therefore find no load-bearing concern that would change the accept verdict.","tokens_in":4426,"tokens_out":7530,"duration_ms":87372,"concrete_test":"As a worthwhile verification step, set up a clean environment, install a specific chembl-downloader version (for example, the version matching the Zenodo archive), pin a ChEMBL release such as 35, and run the first case-study notebook end-to-end. Compare the generated SQLite query outputs and reported dataset statistics with the notebook's posted outputs; if they match, the availability and reproducibility claims are confirmed, and if they differ, the manuscript should document the exact package version and dependency environment.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that chembl-downloader provides reproducible acquisition, access, and manipulation of versioned ChEMBL data through the ChEMBL FTP server. I examined the places where such a claim could fail. First, dependence on the FTP server's layout and retention is explicitly part of the package's design, so it is a stated scope condition rather than a hidden assumption; version pinning is available for users who need to reproduce a specific release. Second, the paper does not claim that arbitrary cheminformatics transformations are deterministic across library versions; it claims reproducible data acquisition and access, with case studies as illustrative workflows rather than guarantees of end-to-end bitwise reproducibility. Third, the implementation artifacts (GitHub, PyPI, Zenodo, notebook case studies, and external uptake) support the availability and usability claims. The lack of a pinned package version or commit hash in the manuscript is a documentation weakness, but it is not fatal to the central claim because a user can pin versions in their own scripts and environments. No internally inconsistent, circular, or unsupported step was identified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents chembl-downloader, a Python package that provides versioned bulk access to ChEMBL data via the ChEMBL FTP server. The author argues that many ChEMBL-derived datasets are published without generation code and quickly become stale, and proposes the package as a way to make derived datasets transparent, updatable, and reproducible. The paper describes low-level download and preprocessing of SQLite, SDF, TSV, fingerprint, monomer XML, and UniProt mapping resources; mid-level parsing utilities; high-level task-specific SQL queries; and three case-study notebooks (re-creating a Deep Confidence dataset, finding ChEMBL-ChEBI identifier mappings via Gilda, and updating blog/example code to use the package). Availability is via GitHub, PyPI, Zenodo, and readthedocs.","tokens_in":4541,"tokens_out":7531,"duration_ms":81536,"significance":"If the package works as described, it addresses a real and timely reproducibility gap in cheminformatics, and the evidence in the manuscript is stronger than for many software notes: source code, documentation, a Zenodo DOI, Jupyter notebooks, and reports of independent use (including by members of the ChEMBL group) are all public. The paper makes no fitted-parameter or circular claims, and the central reproducibility claim is falsifiable by executing the provided notebooks on a pinned ChEMBL version. The main weakness is documentary: the manuscript itself gives no code example or explicit scope statement about the FTP server's retention of old releases. These are local clarifications rather than flaws in the central claim.","major_comments":[],"minor_comments":[{"comment":"The title contains a typo: 'withchembl-downloader' should read 'with chembl-downloader'.","section":"Title"},{"comment":"In the 'Summary' section, clarify that 'a given version' remains reproducible only while ChEMBL's FTP server retains that version under a stable path; version pinning cannot compensate for upstream removal of old releases.","section":"Summary"},{"comment":"The 'Case studies' section should report, for each notebook, the ChEMBL version used and the chembl-downloader version or commit hash, so that readers can re-run the notebooks without guessing the environment.","section":"Case studies"},{"comment":"The first case study uses 'reproduce the dataset generation' while also noting intentional changes in aggregation (arithmetic vs geometric mean); please replace 'reproduce' with 're-create' or explicitly state that the notebook is a modified version of the original workflow.","section":"Case studies"},{"comment":"The 'Availability and usage' section would benefit from a minimal code snippet (e.g., one call that downloads a ChEMBL release and loads a SQLite table), since the prose currently describes interfaces without showing the API.","section":"Availability and usage"},{"comment":"The reference to Landrum (n.d.) is incomplete; provide the URL and an access date.","section":"References"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper delivers what it advertises: chembl-downloader is a thin, practical Python package that wraps pystow and common parsing libraries to give reproducible, version-pinned access to ChEMBL FTP resources. That is genuinely useful for a real pain point, and the paper says so without overselling. It is an engineering contribution, not a new algorithm, and it is honest about that.\n\nWhat is new here is the integration, not the components. RDKit, chemfp, pandas, and the ChEMBL FTP already exist; the package's contribution is making the acquisition step scriptable, cached, and version-aware. The three case studies are sensible demonstrations, and the fact that several independent groups (including ChEMBL's own) use the package in their code is real evidence of uptake. The author's other packages appear in the case studies, but that does not force the central claim—the reproducibility promise is about ChEMBL data, not about those tools.\n\nThe soft spots are minor and mostly about scope. First, the reproducibility guarantee depends on the ChEMBL FTP server keeping its layout and old versions available. The paper states that as a design condition, so it is an explicit scope limit rather than a hidden flaw; users who need bitwise exact reproduction can pin versions. Second, the manuscript does not pin a specific package version or commit hash, which makes it slightly harder to recreate the exact case-study outputs. That is a documentation weakness, not a fatal one, because users can pin in their own environments. Third, the case studies are described in the text and pointed to as notebooks; I did not execute them during this review. Given the package is public and the notebooks are in the repository, this is acceptable for a tool paper, though independent execution would strengthen it.\n\nThe citation pattern looks fair: the relevant prior art on ChEMBL access, RDKit, chemfp, and pandas is there, and the stale-dataset examples (ExCAPE-DB, Deep Confidence, Papyrus) are accurate. No circular reasoning or unsupported claims jumped out. The paper is clearly not a field-reshaping discovery, but it is exactly the kind of infrastructure paper that makes other people's work more trustworthy.\n\nWho is this for? Any cheminformatics researcher who builds datasets from ChEMBL and wants their scripts to be reproducible and updateable. It deserves a serious referee and, after minor revisions to pin versions and clarify the scope of the reproducibility claim, should be accepted. I would bring it to a reading group only if the group cares about research data infrastructure; otherwise, it is a useful citation rather than a lively debate topic.","headline":"A small, well-scoped software paper that meets its reproducibility claim for ChEMBL data access; the soft spots are documentation-level, not load-bearing.","tokens_in":765,"tokens_out":1112,"would_cite":true,"duration_ms":24607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"chembl-downloader makes ChEMBL-derived datasets reproducible by pinning the database version in code.","keywords":["ChEMBL","reproducibility","cheminformatics","data versioning","bioactivity data","Python package","FTP data access"],"falsifier":"Run the same generation script, with the same package version and the same pinned ChEMBL release, on two machines that have never cached the data, and compare the resulting files byte-for-byte; any difference falsifies the reproducibility claim. A complementary test is to try regenerating a dataset pinned to a release after the ChEMBL FTP server has dropped that release, which shows whether version-pinning survives archive removal.","tokens_in":4176,"feed_emoji":"🧪","tokens_out":8744,"duration_ms":77902,"temperature":0.7,"pith_summary":"chembl-downloader is a Python package that gives cheminformatics workflows a reproducible route into ChEMBL, the large public database of bioactive molecule activities. The paper's claim is that by exposing version-pinned downloads of ChEMBL's FTP resources, the package lets researchers generate derived datasets whose methodology—selection, filtering, aggregation—can be re-run by anyone, and which can be upgraded to a newer ChEMBL release later. That matters because most published ChEMBL-derived datasets ship without their generating code and quickly fall behind the database: several widely used datasets were built from releases between v20 and v30, while the current release is v35. The paper argues this opacity and staleness harm interpretation, and introduces chembl-downloader as a practical fix.","feed_headline":"Pin any ChEMBL release and re-run its datasets exactly","feed_subtitle":"A Python package fetches versioned ChEMBL dumps over FTP, so derived datasets can be regenerated and updated.","key_machinery":"The mechanism that carries the argument is a three-layer Python API around the ChEMBL FTP server. The low-level layer maps resource types to deterministic local files, resolving latest or a chosen release, downloading, decompressing, and caching; this is the layer that makes a ChEMBL version part of the code rather than a hidden external fact. The mid-level layer parses those files into queryable structures, including a SQLite database, tabular molecule lists, fingerprint files, and XML, while keeping utilities small so users can extend them. The high-level layer supplies ready-made SQL queries for routinely needed extractions, such as bioactivities for a given assay or target and compounds linked to a publication or patent.","core_discovery":"The central claim is that chembl-downloader provides reproducible acquisition, access, and manipulation of ChEMBL data through its FTP server. Concretely, a script can request either the latest or a specified version of a ChEMBL resource—relational database dumps, molecule lists, precomputed fingerprints, the monomer library, or UniProt target mappings—download it, decompress and cache it, then read it through convenient data structures and pre-formatted queries. Because the version is an explicit parameter, running the same script at two different times or on two different machines should produce the same derived dataset, and changing the version parameter should regenerate it against a newer release. Three case studies demonstrate the workflow: reconstructing the dataset of a published deep-learning study, finding 4,266 candidate ChEMBL–ChEBI identifier mappings for curation, and making the code of popular cheminformatics blogs reproducible and version-agnostic.","pith_inferences":["If adopted widely, the de facto standard for publishing ChEMBL-derived data could shift from a frozen supplementary file to a version-pinned script, making reproducibility checks a routine part of peer review.","The same version-pinned download-and-cache pattern could extend to other public bioinformatics resources with versioned releases, turning reproducibility into a general tool property rather than a per-project effort.","A testable prediction is that routine use of such a tool would shrink the average gap between a published dataset's ChEMBL version and the current release, because updating a dataset would cost one line instead of a rebuild.","Pinning versions also enables stability analysis: rerunning one analysis across successive ChEMBL releases shows which conclusions are sensitive to new data, a form of sensitivity testing that static datasets cannot support."],"forward_implications":["A published dataset can include its generating script; a reader who runs it obtains the same ChEMBL snapshot and can audit every filtering and aggregation step.","Bumping a single version parameter regenerates a dataset on a newer ChEMBL release, so obsolete static datasets can be refreshed without redesigning the pipeline.","Reproducible bulk access reduces the need to download, set up, and connect to a local ChEMBL database, lowering the barrier to working with full-database data.","Common queries—bioactivities by assay or target, compounds in a publication or patent, compound names—come with ready-made queries, making the package useful without deep database expertise.","Because the same versioned files are used across scripts, results from different groups become comparable in principle, even if their analyses differ."],"supporting_citations":[{"why":"Defines the ChEMBL database and its schema, the data source that chembl-downloader acquires.","marker":"Gaulton et al., 2017"},{"why":"Describes ChEMBL's current release infrastructure and data types, grounding the package's resource inventory.","marker":"Zdrazil et al., 2023"},{"why":"Introduces the ChEMBL web services and API that the paper contrasts as inefficient for bulk data access.","marker":"Davies et al., 2015"},{"why":"Presents ExCAPE-DB, a widely used static dataset built on ChEMBL v20, an example of the staleness problem.","marker":"Sun et al., 2017"},{"why":"Provides the Deep Confidence dataset and methodology that the first case study reproduces.","marker":"Cortés-Ciriano & Bender, 2019"},{"why":"Presents Papyrus, a curated dataset built on ChEMBL v30, another example of a static snapshot.","marker":"Béquignon et al., 2023"}],"fun_headline_variants":["Reproduce any ChEMBL dataset with one pip install","Pin ChEMBL versions and replay your exact dataset","FTP-fresh ChEMBL: versioned dumps, reproducible workflows","ChEMBL data on demand: version-pinned and reproducible"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reproducibility promise depends on the ChEMBL FTP server continuing to host every requested version of every resource under a stable, versioned layout; if the server removes an old release, renames files, or changes a version's contents after the fact, a pinned script can no longer reconstruct the same dataset.","fun_headline_variants_meta":{"raw":{"variants":["Reproduce any ChEMBL dataset with one pip install","Pin ChEMBL versions and replay your exact dataset","FTP-fresh ChEMBL: versioned dumps, reproducible workflows","ChEMBL data on demand: version-pinned and reproducible"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000348,"raw_usage":{"total_tokens":1904,"prompt_tokens":949,"completion_tokens":955,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":884}},"tokens_in":565,"tokens_out":955,"duration_ms":7473,"temperature":1.0,"reasoning_tokens":884,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:52:27.869423+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same generation script, with the same package version and the same pinned ChEMBL release, on two machines that have never cached the data, and compare the resulting files byte-for-byte; any difference falsifies the reproducibility claim. A complementary test is to try regenerating a dataset pinned to a release after the ChEMBL FTP server has dropped that release, which shows whether version-pinning survives archive removal.","supporting_citations":[{"cited_title":"F., Engkvist, O., Carlsson, L., Wegner, J., Ceulemans, H., Georgiev, I., Jeliazkov, V., Kochev, N., Ashby, T","cited_arxiv_id":null,"evidence_quote":"Presents ExCAPE-DB, a widely used static dataset built on ChEMBL v20, an example of the staleness problem."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Deep Confidence dataset and methodology that the first case study reproduces."}],"review_version":1}