Pith. sign in

REVIEW 3 major objections 4 minor 47 references

Can AI Agents Design and Implement Drug Discovery Pipelines?

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A multi-agent AI system matched a human expert on a virtual drug-screening task, scoring 33.5% to 33.6% in 10 hours.

desk verdict Worth engaging as a benchmark paper, but the near-expert claim rests on a best-of-three run; the average o3 performance is below the top human team. read the letter →

arxiv 2504.19912 v1 pith:6HO5SPC5 submitted 2025-04-28 cs.AI cs.MA

Khachik Smbatyan , Tsolak Ghukasyan , Tigran Aghajanyan , Hovhannes Dabaghyan , Sergey Adamyan , Aram Bughdaryan , Vahagn Altunyan , Gagik Navasardyan
show 8 more authors
Aram Davtyan Anush Hakobyan Aram Gharibyan Arman Fahradyan Artur Hakobyan Hasmik Mnatsakanyan Narek Ginoyan Garik Petrosyan
This is my paper · ORCID
classification cs.AIcs.MA
keywords AIagentsmulti-agentsystemsdrugdiscoveryvirtualscreeningbenchmarkLLMactivelearningdocking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper attempts to show that a general-purpose multi-agent AI system can plan, write, and execute a drug-discovery pipeline from scratch, not just solve one predictive task. It introduces DO Challenge, a benchmark in which an agent must find the 1,000 highest-scoring molecules among one million docked conformations with a limited label budget, and reports that its Deep Thought system scored 33.5% in the 10-hour setup, nearly matching the top human expert's 33.6% and more than doubling the best human competition team's 16.4%. The result matters because it locates the frontier of autonomous AI in scientific discovery in end-to-end strategic decision-making, not only model accuracy. The paper also reports that the system is unstable across runs and remains far behind an unrestricted expert solution at 77.8%, so it frames the result as promising but not yet transformative.

What carries the argument

The load-bearing object is the DO Score, defined in Eq. (2) as the mean probability of binding to the therapeutic target 6G3C minus the maximum binding probability to three ADMET-related anti-targets (1W0F, 8YXA, 8ZYQ), computed from docking poses and two logistic-regression classifiers. This single scalar label defines the ground truth behind the benchmark's top-1,000 list, so the surrounding protocol—a label-query budget of 100,000 out of one million structures, three submissions of 3,000 structures each, and an overlap metric—tests how efficiently an agent can reconstruct that ranking. The other carrying component is Deep Thought itself, a multi-agent system of role-specialized LLM agents whose code-writing-and-execution loop converts a task prompt into a submitted solution.

What would settle it

Run the Deep Thought cfg-10 configuration with o3 as the primary agent 100 times under identical conditions and plot the score distribution; if the 33.5% result is a rare upper-tail event and the typical score is near the reported 14.03% mean, the claimed match with the human expert is best-of-luck, not representative agent behavior.

Watch

Extended reading notes

Core claim

The central claim, on the paper's terms, is that an LLM-based multi-agent system can compete with human ML experts on a constrained virtual screening task. In the DO Challenge benchmark, Deep Thought's best time-limited configuration with o3 as the primary agent achieved a 33.5% overlap with the true top 1,000 molecules, effectively tied with the top human expert's 33.6% and far ahead of the best DO Challenge 2025 team's 16.4%. The same configuration averaged only 14.03% with a standard deviation of 17.11% across three runs, and the paper documents failure modes ranging from ignoring available tools to exhausting the label budget. In the time-unrestricted setup, the best expert reached 77.8% while Deep Thought reached 33.5%, though a differently configured Deep Thought scored 50.3% in the post-challenge extension. The paper concludes that autonomous agents show real potential but still fall short of expert-designed solutions.

Load-bearing premise

The whole comparison rests on the assumption that the synthetic DO Score, a single docking-derived number, is a meaningful proxy for drug candidacy; if that label does not reflect real drug-likeness, the agent-versus-human scores say little about actual drug discovery.

Editorial extensions

If this is right

  • If the benchmark is accepted, a general LLM-based multi-agent system can carry out an end-to-end ML drug-screening workflow—data exploration, model selection, code writing, execution, and submission—without task-specific hints.
  • The near-tie with the time-limited human expert implies that automated planning can reach the level of an experienced ML practitioner on this kind of constrained task.
  • The gap in the unrestricted setup implies that the remaining bottleneck is not basic feasibility but sustained strategy, model quality, and hyperparameter discipline.
  • The ablation results imply that the choice of LLM in the primary agent role is decisive: frontier models produced competitive code while smaller models mostly failed to finish.
  • The four correlated success factors—strategic structure selection, spatial-relational networks, position non-invariance, and strategic submission—offer a concrete design checklist for future scientific agents.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the same o3 configuration averaged 14.03% with a 17.11% standard deviation, benchmark comparisons should be based on score distributions over repeated runs, not best-of-three maxima, before declaring an agent competitive with a human expert.
  • Editorial inference: if the DO Score proxies real drug-likeness even roughly, the active-learning-plus-surrogate strategy Deep Thought discovered should transfer to other label-expensive search problems, such as materials or reaction-condition discovery, where the same planning loop can be pointed at a different oracle.
  • Editorial inference: the unrestricted expert's advantage suggests a concrete remedy—equipping the agent with position-aware architectures and a top-k weighted loss, since those were the decisive components in the 77.8% solution.
  • Editorial inference: the near-zero contribution of the Research agent group in runs with advanced models hints that web research is not the bottleneck for this task; execution reliability and budget discipline are.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces the DO Challenge, a benchmark for autonomous AI agents in a virtual-screening-style drug discovery task. Agents receive one million unlabeled molecular conformations, may query at most 100,000 DO Score labels, make up to three 3,000-molecule submissions, and are evaluated by overlap with the true top-1000. The DO Score is defined as a docking-derived binding probability to a JAK2 target minus the maximum binding probability over three ADMET anti-targets. The paper reports results from the human DO Challenge 2025 competition, two in-house expert solutions, and the Deep Thought multi-agent LLM system, together with ablation studies over LLM roles and agent configurations. The headline claim is that Deep Thought in a 10-hour setup achieved 33.5% overlap, nearly identical to the best human expert's 33.6% and far above the best competition team's 16.4%; the paper also reports an unrestricted expert result of 77.8%. The benchmark and code are released on Zenodo and GitHub.

Significance. If the claims were robust, the paper would be a valuable contribution to agentic AI evaluation for scientific discovery: it provides an integrated, resource-constrained task requiring planning, implementation, and execution, and it includes unusually transparent run-level reporting of agent variance, failure modes, and LLM ablations. The public release of the benchmark and the Deep Thought code is a concrete strength, as is the explicit canary string for leakage prevention. However, the central quantitative claim is weakened by best-of-three selection, and the benchmark's construct validity as a drug-discovery proxy rests on a limited validation. With appropriate re-framing and strengthened validation, the benchmark and the agent study are still publishable and useful.

major comments (3)
  1. [4.2.4, Table 5, Supplementary F Table F13; Abstract and Section 4] The headline comparison rests on selecting the maximum of three runs for the o3 configuration. Table F13 reports the three cfg-10 o3 scores as 1.4%, 33.5%, and 7.2%, and Table 5 reports the same configuration's average as 14.03% with standard deviation 17.11%. The abstract and the Section 4 results text present 'Deep Thought achieved 33.5%' and describe this as 'nearly identical' to the expert's 33.6%, without stating that 33.5% is the best of three runs and that the expected score under the same protocol is below the top competition team's 16.4%. A capability claim ('the best run matched expert performance') is defensible, but the current wording supports a stronger performance claim than the data justify. Please either report the mean as the primary measure, or explicitly and consistently label the 33.5% as the best of three runs throughout the abstract, Tables 1 and 2, and the conclusion.
  2. [3.2, Eq. (2), Table A2] The external validity of the DO Score is a load-bearing assumption for the benchmark's interpretation, and the paper's own validation is weak. Table A2 shows that among 107 DUD-E JAK2 binders, DO Score ranking places only 9 in the top 1000, with EF1% = 8.41, while ranking by the single-target Score6G3C places 19 binders with EF1% = 27.10. Thus the multi-objective score enriches known binders less than the single-target score, and no validation is provided for the anti-target selectivity or for drug-likeness more broadly. The conclusion that the benchmark tests 'drug discovery pipelines' is therefore not firmly established. I recommend either substantially strengthening the validation of Eq. (2) as a drug-candidacy proxy, or explicitly reframing the benchmark as a synthetic virtual-screening task whose labels are defined by the authors' own scoring function.
  3. [4, Supplementary D, Tables 1 and 2] The human expert comparator is a single run per expert, produced by authors from the same organization, with no reported variance or reproducibility check. The 33.6% expert result is therefore as fragile as the agent's best run, and the 'nearly identical to expert' conclusion compares a selected maximum from the agent against a single expert run. This is a further reason the headline comparison is not robust. Please report the expert results as single demonstrations, avoid language implying a stable expert performance level, and state the single-run limitation explicitly when the agent-vs-expert comparison is made.
minor comments (4)
  1. [Table 5 vs. Supplementary F] The configuration labels are inconsistent: Table 5 lists the o3 Software Engineer configuration as cfg-11 with scores 1.4/33.5/7.2, while Supplementary F Table F13 identifies the same o3 runs as cfg-10 and cfg-11 is described as 'without Reviewer' in Section 4.2.4. Similarly, Table 5 lists Gemini 2.0 Flash as cfg-7 and Claude 3.5 Haiku as cfg-9, whereas Supplementary F assigns cfg-7 to Claude 3.5 Haiku and cfg-8 to Gemini 2.0 Flash. Please reconcile all configuration identifiers between the main text and the supplementary tables.
  2. [4.1] The number of competition teams is stated inconsistently: the abstract and Section 4 say 20 human teams, while Section 4.1 says 24 teams were selected and 20 ultimately engaged. Please make these numbers consistent.
  3. [3.3] The dataset description says 500,000 molecules were randomly sampled and filtered, then 'a sample of 200,000 molecules was retained,' but it is not clear how the 200,000 were selected from the filtered set or how the 1,000,000 conformations correspond to the 200,000 molecules. Clarify the sampling procedure.
  4. [Supplementary A, Table A2] The enrichment validation description should state explicitly whether the 107 DUD-E binders were added to the 1M dataset as extra molecules or docked poses, and how the 'TOP 1000 Hits' overlap is defined when multiple poses of the same molecule may appear; this affects interpretation of the enrichment factors.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the paper's headline comparisons are empirical benchmark outcomes, not derivations from fitted inputs.

full rationale

The claimed derivations are benchmark construction and empirical evaluation, not fitted predictions. The DO Score in Eq. 2 is generated from docking simulations against external protein targets (6G3C, 1W0F, 8YXA, 8ZYQ) and logistic-regression classifiers trained on the external DEKOIS2 dataset. Deep Thought's scores come from allowed label queries on the benchmark server, not from fitting any parameter to the top-1000 answer. The enrichment validation in Table A2 uses JAK2 binders from the external DUD-E set. The only overlapping-author citation, [45], appears in Supplementary D to describe feature choices for Expert 2's GNN solution and is not load-bearing for the central claim about the agentic system. The best-of-three o3 result (33.5% vs. average 14.03% ± 17.11%) is a real statistical robustness concern, but selecting the best of several stochastic runs is not equivalent to fitting an input and predicting it back, so it does not make the claim circular. No step satisfies the quoted-reduction standard for circularity, and the paper is transparent about instability in its abstract and ablations.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the benchmark's ground-truth labels, which are generated by docking simulations and trained binary classifiers rather than by direct measurement. The paper's own validated evidence for the meaningfulness of DO Score is one enrichment test on JAK2 binders (Table A2). No new physical entities, forces, or dimensions are introduced.

free parameters (1)
  • DEKOIS2-trained logistic regression coefficients for binding probability classifiers = not reported
    These fitted classifiers define Score_target,c, which enters the DO Score in Eq. 2 and therefore determines the benchmark's ground-truth labels. The paper does not report the coefficients or model weights, so the label generation is not independently reproducible.
assumptions (3)
  • domain assumption AutoDock Vina and Balto docking scores, ProLIF interaction counts, and DEKOIS2-trained logistic regression classifiers provide valid estimates of protein-ligand binding probability.
    The DO Score in Eq. 2 is built from these components; if they do not reflect real binding, the top-1000 molecules are not meaningful drug candidates. Invoked in Section 3.2.
  • domain assumption The four selected protein structures (6G3C, 1W0F, 8YXA, 8ZYQ) capture the relevant therapeutic and ADMET axes for drug candidates.
    DO Score is defined as therapeutic-target binding minus maximum ADMET binding; the choice of these four proteins determines what the benchmark rewards. Described in Section 3.2 and Supplementary A.
  • domain assumption Protein-ligand docking in a shared reference frame provides labels whose ranking is a reasonable virtual screening proxy.
    The task instructs agents that translation and rotation of a structure changes its DO Score, encoding a strong modeling assumption about pose-dependent scoring. Stated in the task description in Supplementary A and used throughout the benchmark.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can AI Agents Design and Implement Drug Discovery Pipelines?." pith.science (2026). https://pith.science/paper/6HO5SPC5

@misc{pith2026250419912,
  author       = {Pith},
  title        = {Pith review of: Can AI Agents Design and Implement Drug Discovery Pipelines?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6HO5SPC5}},
  note         = {Machine review of arXiv:2504.19912}
}
read the original abstract

The rapid advancement of artificial intelligence, particularly autonomous agentic systems based on Large Language Models (LLMs), presents new opportunities to accelerate drug discovery by improving in-silico modeling and reducing dependence on costly experimental trials. Current AI agent-based systems demonstrate proficiency in solving programming challenges and conducting research, indicating an emerging potential to develop software capable of addressing complex problems such as pharmaceutical design and drug discovery. This paper introduces DO Challenge, a benchmark designed to evaluate the decision-making abilities of AI agents in a single, complex problem resembling virtual screening scenarios. The benchmark challenges systems to independently develop, implement, and execute efficient strategies for identifying promising molecular structures from extensive datasets, while navigating chemical space, selecting models, and managing limited resources in a multi-objective context. We also discuss insights from the DO Challenge 2025, a competition based on the proposed benchmark, which showcased diverse strategies explored by human participants. Furthermore, we present the Deep Thought multi-agent system, which demonstrated strong performance on the benchmark, outperforming most human teams. Among the language models tested, Claude 3.7 Sonnet, Gemini 2.5 Pro and o3 performed best in primary agent roles, and GPT-4o, Gemini 2.0 Flash were effective in auxiliary roles. While promising, the system's performance still fell short of expert-designed solutions and showed high instability, highlighting both the potential and current limitations of AI-driven methodologies in transforming drug discovery and broader scientific research.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 25 canonical work pages

  1. [1]

    and Bridgland, A., 2021

    Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A. and Bridgland, A., 2021. Highly accurate protein structure prediction with AlphaFold. nature, 596(7873), pp.583-589

    work page 2021

  2. [2]

    and Liu, Z., 2023

    Chan, C.M., Chen, W., Su, Y., Yu, J., Xue, W., Zhang, S., Fu, J. and Liu, Z., 2023. Chateval: Towards better llm-based evaluators through multi-agent debate. arXiv preprint arXiv:2308.07201

    arXiv 2023

  3. [3]

    and Tu, Z., 2024, November

    Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S. and Tu, Z., 2024, November. Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 17889-17904)

    work page 2024

  4. [4]

    and Batenburg, K.J., 2025

    Plaat, A., van Duijn, M., van Stein, N., Preuss, M., van der Putten, P. and Batenburg, K.J., 2025. Agentic Large Language Models, a survey. arXiv preprint arXiv:2503.23037

    arXiv 2025

  5. [5]

    and Dernoncourt, F., 2024

    Nguyen, D., Lai, V.D., Yoon, S., Rossi, R.A., Zhao, H., Zhang, R., Mathur, P., Lipka, N., Wang, Y., Bui, T. and Dernoncourt, F., 2024. Dynasaur: Large language agents beyond predefined actions. arXiv preprint arXiv:2411.01747. 16

    arXiv 2024

  6. [6]

    and Ha, D., 2024

    Lu, C., Lu, C., Lange, R.T., Foerster, J., Clune, J. and Ha, D., 2024. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292

    arXiv 2024

  7. [7]

    and Hwang, S.J., 2024

    Baek, J., Jauhar, S.K., Cucerzan, S. and Hwang, S.J., 2024. Researchagent: Iterative research idea generation over scientific literature with large language models. arXiv preprint arXiv:2404.07738

    arXiv 2024

  8. [8]

    and Saab, K., 2025

    Gottweis, J., Weng, W.H., Daryin, A., Tu, T., Palepu, A., Sirkovic, P., Myaskovsky, A., Weissenberger, F., Rong, K., Tanno, R. and Saab, K., 2025. Towards an AI co-scientist. arXiv preprint arXiv:2502.18864

    arXiv 2025

Show all 47 references
  1. [9]

    and Heidecke, J., 2025

    Miserendino, S., Wang, M., Patwardhan, T. and Heidecke, J., 2025. SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?. arXiv preprint arXiv:2502.12115

    2025 arXiv

  2. [10]

    and Hwang, S.J., 2024

    Trirat, P., Jeong, W. and Hwang, S.J., 2024. Automl-agent: A multi-agent llm framework for full-pipeline automl. arXiv preprint arXiv:2410.02958

    2024 arXiv

  3. [11]

    and Gomes, G., 2023

    Boiko, D.A., MacKnight, R., Kline, B. and Gomes, G., 2023. Autonomous chemical research with large language models. Nature, 624(7992), pp.570-578

    2023

  4. [12]

    and Zou, J., 2024

    Swanson, K., Wu, W., Bulaong, N.L., Pak, J.E. and Zou, J., 2024. The virtual lab: AI agents design new SARS-CoV-2 nanobodies with experimental validation. bioRxiv, pp.2024-11

    2024

  5. [13]

    and Kohli, P., 2024

    Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M.P., Dupont, E., Ruiz, F.J., Ellenberg, J.S., Wang, P., Fawzi, O. and Kohli, P., 2024. Mathematical discoveries from program search with large language models. Nature, 625(7995), pp.468-475

    2024

  6. [14]

    and Barsoum, E., 2025

    Schmidgall, S., Su, Y., Wang, Z., Sun, X., Wu, J., Yu, X., Liu, J., Liu, Z. and Barsoum, E., 2025. Agent laboratory: Using llm agents as research assistants. arXiv preprint arXiv:2501.04227

    2025 arXiv

  7. [15]

    and Ha, D., 2025

    Lange, R.T., Prasad, A., Sun, Q., Faldor, M., Tang, Y. and Ha, D., 2025. The AI CUDA Engineer: Agentic CUDA Kernel Discovery, Optimization and Composition

    2025

  8. [16]

    and Hupkes, D., 2025

    Nathani, D., Madaan, L., Roberts, N., Bashlykov, N., Menon, A., Moens, V., Budhiraja, A., Magka, D., Vorotilov, V., Chaurasia, G. and Hupkes, D., 2025. Mlgym: A new framework and benchmark for advancing ai research agents. arXiv preprint arXiv:2502.14499

    2025 arXiv

  9. [17]

    and Weng, L., 2024

    Chan, J.S., Chowdhury, N., Jaffe, O., Aung, J., Sherburn, D., Mays, E., Starace, G., Liu, K., Maksin, L., Patwardhan, T. and Weng, L., 2024. Mle-bench: Evaluating machine learning agents on machine learning engineering. arXiv preprint arXiv:2410.07095

    2024 arXiv

  10. [18]

    and Narasimhan, K., 2023

    Jimenez, C.E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O. and Narasimhan, K., 2023. Swe-bench: Can language models resolve real-world github issues?. arXiv preprint arXiv:2310.06770

    2023 arXiv

  11. [19]

    and Leskovec, J., 2024, July

    Huang, Q., Vora, J., Liang, P. and Leskovec, J., 2024, July. MLAgentBench: evaluating language agents on machine learning experimentation. In Proceedings of the 41st International Conference on Machine Learning (pp. 20271-20309)

    2024

  12. [20]

    and Ericheva, E., 2024

    Wijk, H., Lin, T., Becker, J., Jawhar, S., Parikh, N., Broadley, T., Chan, L., Chen, M., Clymer, J., Dhyani, J. and Ericheva, E., 2024. Re-bench: Evaluating frontier ai r&d capabilities of language model agents against human experts. arXiv preprint arXiv:2411.15114

    2024 arXiv

  13. [21]

    and Yang, M., 2024

    Xu, F.F., Song, Y., Li, B., Tang, Y., Jain, K., Bao, M., Wang, Z.Z., Zhou, X., Guo, Z., Cao, M. and Yang, M., 2024. Theagentcompany: benchmarking llm agents on consequential real world tasks. arXiv preprint arXiv:2412.14161

    2024 arXiv

  14. [22]

    and Kambhampati, S., 2023

    Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S. and Kambhampati, S., 2023. Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change. Advances in Neural Information Processing Systems, 36, pp.38975-38987

    2023

  15. [23]

    and Glaese, A., BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

    Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H.W., Passos, A.T., Fedus, W. and Glaese, A., BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

  16. [24]

    and Dey, V., 2024

    Chen, Z., Chen, S., Ning, Y., Zhang, Q., Wang, B., Yu, B., Li, Y., Liao, Z., Wei, C., Lu, Z. and Dey, V., 2024. Scienceagentbench: Toward rigorous assessment of language agents for data-driven scientific discovery. arXiv preprint arXiv:2410.05080

    2024 arXiv

  17. [25]

    and Narayanan, A., 2024

    Siegel, Z.S., Kapoor, S., Nagdir, N., Stroebl, B. and Narayanan, A., 2024. CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark. arXiv preprint arXiv:2409.11363. 17

    2024 arXiv

  18. [26]

    and Heidecke, J., 2025

    Starace, G., Jaffe, O., Sherburn, D., Aung, J., Chan, J.S., Maksin, L., Dias, R., Mays, E., Kinsella, B., Thompson, W. and Heidecke, J., 2025. PaperBench: Evaluating AI's Ability to Replicate AI Research. arXiv preprint arXiv:2504.01848

    2025 arXiv

  19. [27]

    and Zitnik, M

    Huang, K., Fu, T., Gao, W., Zhao, Y., Roohani, Y., Leskovec, J., Coley, C., Xiao, C., Sun, J. and Zitnik, M. (2021) 'Therapeutics data commons: Machine learning datasets and tasks for drug discovery and development', NeurIPS Datasets and Benchmarks

    2021

  20. [28]

    and Lai, H., 2023, June

    Ji, Y., Zhang, L., Wu, J., Wu, B., Li, L., Huang, L.K., Xu, T., Rong, Y., Ren, J., Xue, D. and Lai, H., 2023, June. Drugood: Out-of-distribution dataset curator and benchmark for ai-aided drug discovery–a focus on affinity prediction problems with noise annotations. In Proceed...

    2023

  21. [29]

    and Zeng, J., 2024

    Tian, T., Li, S., Zhang, Z., Chen, L., Zou, Z., Zhao, D. and Zeng, J., 2024. Benchmarking compound activity prediction for real-world drug discovery applications. Communications Chemistry, 7(1), p.127

    2024

  22. [30]

    and Vaucher, A.C., 2019

    Brown, N., Fiscato, M., Segler, M.H. and Vaucher, A.C., 2019. GuacaMol: benchmarking models for de novo molecular design. Journal of chemical information and modeling, 59(3), pp.1096-1108

    2019

  23. [31]

    and Pande, V., 2018

    Wu, Z., Ramsundar, B., Feinberg, E.N., Gomes, J., Geniesse, C., Pappu, A.S., Leswing, K. and Pande, V., 2018. MoleculeNet: a benchmark for molecular machine learning. Chemical science, 9(2), pp.513-530

    2018

  24. [32]

    and Hernández-Lobato, J.M., 2020, November

    Simm, G., Pinsler, R. and Hernández-Lobato, J.M., 2020, November. Reinforcement learning for molecular design guided by quantum mechanics. In International Conference on Machine Learning (pp. 8959-8969). PMLR

    2020

  25. [33]

    and Boeckler, F.M., 2013

    Bauer, M.R., Ibrahim, T.M., Vogel, S.M. and Boeckler, F.M., 2013. Evaluation and optimization of virtual screening workflows with DEKOIS 2.0–a public library of challenging docking benchmark sets. Journal of chemical information and modeling, 53(6), pp.1447-1462

    2013

  26. [34]

    and Olson, A.J., 2010

    Trott, O. and Olson, A.J., 2010. AutoDock Vina: improving the speed and accuracy of docking with a new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2), pp.455-461

    2010

  27. [35]

    and Shoichet, B.K., 2012

    Mysinger, M.M., Carchia, M., Irwin, J.J. and Shoichet, B.K., 2012. Directory of Useful Decoys, Enhanced (DUD-E): Better Ligands and Decoys for Better Benchmarking. Journal of Medicinal Chemistry, 55(14)

    2012

  28. [36]

    and Liu, T.Y., 2017

    Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q. and Liu, T.Y., 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30

    2017

  29. [37]

    and Eck, M.J., 2019

    McNally, R., Li, Q., Li, K., Dekker, C., Vangrevelinghe, E., Jones, M., Chène, P., Machauer, R., Radimerski, T. and Eck, M.J., 2019. Discovery and structural characterization of ATP-site ligands for the wild-type and V617F mutant JAK2 pseudokinase domain. ACS Chemical Biology,...

    2019

  30. [38]

    and Jhoti, H., 2004

    Williams, P.A., Cosme, J., Vinkovic, D.M., Ward, A., Angove, H.C., Day, P.J., Vonrhein, C., Tickle, I.J. and Jhoti, H., 2004. Crystal structures of human cytochrome P450 3A4 bound to metyrapone and progesterone. Science, 305(5684), pp.683-686

    2004

  31. [39]

    and Doi, Y., 2024

    Kawai, A., Yamasaki, K., Otagiri, M. and Doi, Y., 2024. Interaction of Cephalosporins with Human Serum Albumin: A Structural Study. Journal of Medicinal Chemistry, 67(16), pp.14175-14183

    2024

  32. [40]

    and Murata, T., 2024

    Miyashita, Y., Moriya, T., Kato, T., Kawasaki, M., Yasuda, S., Adachi, N., Suzuki, K., Ogasawara, S., Saito, T., Senda, T. and Murata, T., 2024. Improved higher resolution cryo-EM structures reveal the binding modes of hERG channel inhibitors. Structure, 32(11), pp.1926-1935

    2024

  33. [41]

    Berman, J

    H.M. Berman, J. Westbrook, Z. Feng, G. Gilliland, T.N. Bhat, H. Weissig, I.N. Shindyalov, P.E. Bourne, The Protein Data Bank (2000) Nucleic Acids Research 28: 235-242 https://doi.org/10.1093/nar/28.1.235

    2000 doi

  34. [42]

    and Ke, G., 2024

    Ji, X., Wang, Z., Gao, Z., Zheng, H., Zhang, L. and Ke, G., 2024. Uni-Mol2: Exploring Molecular Pretraining Model at Scale. arXiv preprint arXiv:2406.14969

    2024 arXiv

  35. [43]

    and Ghanem, B., 2020

    Li, G., Xiong, C., Thabet, A. and Ghanem, B., 2020. Deepergcn: All you need to train deeper gcns. arXiv preprint arXiv:2006.07739

    2020 arXiv

  36. [44]

    and Lenssen, J.E., 2019

    Fey, M. and Lenssen, J.E., 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428. 18

    2019 arXiv

  37. [45]

    and Petrosyan, G., 2025

    Ghukasyan, T., Altunyan, V., Bughdaryan, A., Aghajanyan, T., Smbatyan, K., Papoian, G.A. and Petrosyan, G., 2025. Smart distributed data factory volunteer computing platform for active learning-driven molecular data acquisition. Scientific Reports, 15(1), p.7122

    2025

  38. [46]

    rdkit/rdkit: 2022_03_1 (Q1 2022) Release

    Greg Landrum (2022) “rdkit/rdkit: 2022_03_1 (Q1 2022) Release”. Zenodo. doi: 10.5281/zenodo.6388425. 19 Supplementary materials A. DO Challenge DO Challenge task definition Table A1 provides the textual description of the task that was given to AI agents and human developers. ...

    2022 doi

  39. [47]

    # Rest of the code remains the same

    Use the available search tools to understand the codebase and the query. 2. Implement the solution using all tools available to you 3. Verify the solution if possible with tests. 4. Avoid creating jupyter notebooks or CI/CD pipelines. 5. While creating requirements file, make ...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.