Pith. sign in

REVIEW 4 major objections 6 minor 95 references

Exploring Modularity of Agentic Systems for Drug Discovery

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM-based agentic systems for drug discovery are not modular: swapping the LLM, agent type, or system prompt changes performance unpredictably, depending on the question and model combination.

desk verdict A careful, clearly-scoped case study that makes a plausible non-modularity point for LLM agents in drug discovery; trust the framing, but treat the LLM-as-a-judge as an unvalidated instrument. read the letter →

arxiv 2506.22189 v2 pith:IUHH2S72 submitted 2025-06-27 cs.LG cs.CLcs.MA

Laura van Weesep , Samuel Genheden , Ola Engkvist , Jens Sjölund This is my paper
classification cs.LGcs.CLcs.MA
keywords LLMagentmodularityLLM-as-a-judgesmolagentsCodeToolCallingdrugdiscoverycheminformaticssystemprompts
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether the parts of an LLM-based agentic system for drug discovery—the backbone LLM, the agent type, and the system prompt—are interchangeable. Its answer is no: on a 26-question cheminformatics task set, scores shift dramatically with each swap, and the direction of the shift depends on the specific question and model combination. That matters because drug-discovery teams are starting to build workflows on these agents; if modularity fails, every new model release or prompt tweak requires re-evaluating the whole system, not just the swapped part.

What carries the argument

The experimental grid is the central object: seven LLMs crossed with two agent types and three system prompts, evaluated on 26 industry-style cheminformatics questions using 17 RDKit- and PubChem-based tools. The two agent types are CodeAgent, which writes and executes Python code, and ToolCallingAgent, which emits JSON tool calls, both running in the smolagents framework's ReAct-style loop. Scoring is done by an LLM-as-a-judge (GPT-4o) that first summarizes the agent's answer, then assigns a score from 0 to 100 against an expected answer, with five repetitions per question to smooth variance.

What would settle it

Run the same 26-question evaluation with a judge from a different model family (for example, Claude-3.5-Sonnet or Llama-3.1-70B grading the same agent answers) and compare the model rankings. If the top-three ordering changes or the gap between Claude/GPT and Llama collapses, the observed non-modularity is an artifact of the judge rather than a property of the agentic systems. A second independent check is to have human chemists grade the same 26 answers and compute the correlation with GPT-4o's scores.

Watch

Extended reading notes

Core claim

The paper's central discovery is that modularity fails across every axis tested. On a 26-question cheminformatics benchmark scored by GPT-4o as judge, Claude-3.5-Sonnet, Claude-3.7-Sonnet, and GPT-4o beat Llama-3.1-8B, Llama-3.1-70B, GPT-3.5-Turbo, and Nova-Micro; code-generating agents beat tool-calling agents on average, but the advantage reverses or vanishes on specific questions and models; and changing the system prompt helps some model-question pairs while hurting others. The authors conclude that components of an agentic system cannot simply be replaced without re-engineering.

Load-bearing premise

The whole comparison rests on GPT-4o as the judge giving scores that truly reflect answer quality; the paper only sanity-checks this on 10 questions with no human grading, so a biased judge would make the non-modularity finding an artifact.

Editorial extensions

If this is right

  • A team adopting a new, stronger LLM into an existing drug-discovery agent cannot assume the workflow will improve uniformly; the new model must be re-tested across the same question set and agent type.
  • Reporting only average benchmark scores is misleading here: the question-level heatmaps show reversals that averages hide, so evaluation reports should include per-question and per-model breakdowns.
  • Prompt engineering does not transfer: a 'cleaner' or 'domain-relevant' system prompt that helps one LLM can hurt another, so prompts should be treated as part of the configuration space, not as a fixed layer.
  • The choice between CodeAgent and ToolCallingAgent is not a one-time decision; because the better agent depends on the question and model, the system should either support both or have a selection mechanism.
  • Agentic systems for drug discovery should be evaluated as full configurations (LLM by agent type by prompt), not as standalone components, before being used in real discovery workflows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same non-modularity should be expected in other tool-using agent domains (e.g., general coding or bioinformatics), because the interaction between an LLM's parametric memory and the syntax of tool calls is not chemistry-specific.
  • Inference: because the judge is itself an LLM, some of the reported score variance may be judge variance rather than agent variance; a judge from a different model family could compress or reorder the observed gaps.
  • Inference: a direct test of prompt-level modularity would hold the system fixed and rephrase only the question, as the oestrogen/progesterone example does, to quantify how much of the variance is due to wording rather than component choice.
  • Inference: question-level features, such as whether the compound is in PubChem or whether a SMILES string is supplied, could predict when modularity breaks; the paper's tool-call analysis for questions 5 and 8 is a first step toward such a predictor.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper investigates whether LLM-based agentic systems for drug discovery are modular, i.e., whether components such as the backbone LLM, agent type, and system prompt can be interchanged without degrading performance. The authors build a smolagents-based system with 17 cheminformatics tools and evaluate seven LLMs (GPT-4o, GPT-3.5-Turbo, Claude-3.5/3.7-Sonnet, Llama-3.1-8B/70B, Nova-Micro) in both CodeAgent and ToolCallingAgent modes across three system prompts on 26 industry-style questions. Performance is measured by an LLM-as-a-judge (GPT-4o) that assigns a 0–100 score. The main findings are that Claude-3.5-Sonnet, Claude-3.7-Sonnet, and GPT-4o significantly outperform the other models (paired one-sided Wilcoxon with Bonferroni correction), that CodeAgent beats ToolCallingAgent on average but with strong question- and model-dependent reversals, and that prompt changes have inconsistent effects. The paper concludes that agentic systems are not modular and that components should not be swapped without re-evaluation.

Significance. If the LLM-judge scores can be trusted, the paper provides useful empirical evidence for a phenomenon—non-modularity of agentic systems—that is often assumed away in system-building practice. The strengths include repeated runs with per-question detail, a reasonable paired statistical test for the model comparison, an explicit acknowledgment of the judge limitations, and an informative question-level analysis of tool usage (Appendices A.7 and A.10). The contribution is, however, entirely contingent on the validity of the GPT-4o judge, which is the same model as one of the evaluated agents; the preliminary cross-judge check in Appendix A.9 is too small and too aggregated to establish reliability. The paper's conclusions would be on firmer ground with human validation or a more extensive and statistically quantified judge comparison. As it stands, the evidence supports a descriptive claim about scores assigned by GPT-4o but not yet a robust claim about the intrinsic performance of the agentic system.

major comments (4)
  1. [§3.1, Appendix A.9] The central claim of non-modularity is derived exclusively from scores assigned by GPT-4o, which is also one of the evaluated models. The cross-judge check in Appendix A.9 uses only 10 questions and 3 runs, employs two judges (Claude-3.5-Sonnet and Llama-3.1-8B), reports only averaged trends without any inter-judge agreement metric (e.g., correlation, Cohen's kappa), and does not compare against human grading. Because the judge's reliability is load-bearing for every headline result (Figures 3–5), the paper should either provide stronger validation or consistently qualify all claims in the abstract and body as “according to GPT-4o” rather than only in the conclusion.
  2. [§3.2] The claim that CodeAgent outperforms ToolCallingAgent on average “for each of the models” (by 2.5 to 12.9 units) is made without any measure of uncertainty or significance test, despite the availability of five repetitions per question. A paired test or confidence intervals on the difference would be needed to support this claim, particularly because the heatmap in Figure 4 shows large per-question variability and the paper's own narrative stresses question-dependent reversals.
  3. [§3.3 and Figure 5] The prompt engineering analysis reports only mean differences in LLM scores and concludes that effects are inconsistent across questions and models. No statistical analysis is provided, so claims such as “the default prompt performs the best for some questions but worst for others” are based on visual inspection. While this is a secondary result, adding at least a simple effect-size quantification or a paired comparison would strengthen the non-modularity argument.
  4. [Reproducibility] No code or data are made available, and the expected answers used in the judge prompt (Appendix A.6) are not listed for the 26 questions. Without these, independent re-scoring is impossible, which is particularly problematic in a study whose main endpoint is an unvalidated subjective score. Releasing the full score matrix would allow readers to assess judge stability and re-run the analyses.
minor comments (6)
  1. [Abstract and §3.1] The abstract and Section 3.1 state that “Claude-3.5-Sonnet, Claude-3.7-Sonnet and GPT-4o outperform alternative language models” but omit the qualifier “as evaluated by the GPT-4o LLM judge” that appears in the conclusion; this should be consistent throughout.
  2. [Appendix A.6] There is a typo in the prompt text: “Agent's Anser” should be “Agent's Answer”.
  3. [Appendix A.10] The SMILES string in question 8 is printed as “C1OCc21c1ccc3S=CC=Cc23?” with an extra “1”; this appears to be a typo (the correct SMILES is likely “C1OCc2c1ccc3S=CC=Cc23”).
  4. [Figure 2] The flow diagram is somewhat ambiguous: the arrows do not clearly indicate that the same LLMs are used for both agent types and that the judge scores the adjusted answer; a clearer diagram would help readers follow the experimental design.
  5. [§2.1 and §3.4] The smolagents version 1.14.0 is mentioned only in Section 3.4; it should be stated in Section 2.1 for reproducibility.
  6. [General] The term “LLM score” is used inconsistently; it should be made explicit throughout that this is the judge's score, not the agent's internal confidence or a chemical property.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the modularity comparisons are empirical measurements under an explicit LLM-judge protocol, not derivations from their own inputs.

full rationale

The paper's central claim is an empirical comparison of agentic configurations: different LLMs, agent types, and system prompts are run on 26 questions and scored by a fixed LLM-as-a-judge protocol. None of the headline results is obtained by fitting a parameter to a subset of the data and then predicting a closely related quantity; the scores are external measurements of agent outputs against provided expected answers. The GPT-4o judge being one of the evaluated models creates a potential measurement-bias risk, but this is not circularity in the defined sense: the scores are not constructed to equal any input quantity, the conclusion is explicitly conditional on 'as evaluated by an LLM-as-a-judge system,' and Appendix A.9 offers a preliminary cross-judge robustness check. The paper contains no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation; the smolagents framework and prior CodeAgent-vs-ToolCallingAgent results are used as context, not as the basis of the paper's derivation. Therefore, no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claim depends on the reliability of the LLM judge, the representativeness of the 26-question set, and the correctness of the smolagents-based tool orchestration. No free parameters are fitted: the design choices (questions, prompts, significance thresholds) are experimental specifications rather than fitted values.

assumptions (3)
  • domain assumption LLM-as-a-judge scores are a valid proxy for answer quality.
    The evaluation uses GPT-4o as judge without human validation; a small cross-judge sanity check (Appendix A.9) is preliminary.
  • domain assumption The 26 hand-authored questions and expected answers are correct and representative.
    Stated in Section 2.2 as industry-representative, but the set is small and the answers are not externally verified.
  • domain assumption The smolagents framework and the listed tools behave as described, and the agent implementations are correct.
    The paper uses smolagents v1.14.0 (Section 3.4) but does not ship code; the Llama ToolCallingAgent failure (Appendix A.4) shows environment-specific issues.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Modularity of Agentic Systems for Drug Discovery." pith.science (2026). https://pith.science/paper/IUHH2S72

@misc{pith2026250622189,
  author       = {Pith},
  title        = {Pith review of: Exploring Modularity of Agentic Systems for Drug Discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IUHH2S72}},
  note         = {Machine review of arXiv:2506.22189}
}
read the original abstract

Large-language models (LLMs) and agentic systems present exciting opportunities to accelerate drug discovery. In this study, we examine the modularity of LLM-based agentic systems for drug discovery, i.e., whether parts of the system such as the LLM and type of agent are interchangeable, a topic that has received limited attention in drug discovery. We compare the performance of different LLMs and the effectiveness of tool-calling agents versus code-generating agents. Our case study, comparing performance in orchestrating tools for chemistry and drug discovery using an LLM-as-a-judge score, shows that Claude-3.5-Sonnet, Claude-3.7-Sonnet and GPT-4o outperform alternative language models such as Llama-3.1-8B, Llama-3.1-70B, GPT-3.5-Turbo, and Nova-Micro. Although we confirm that code-generating agents outperform the tool-calling ones on average, we show that this is highly question- and model-dependent. Furthermore, the impact of replacing system prompts is dependent on the question and model, underscoring that even in this particular domain one cannot just replace components of the system without re-engineering. Our study highlights the necessity of further research into the modularity of agentic systems to enable the development of reliable and modular solutions for real-world problems.

Figures

Figures reproduced from arXiv: 2506.22189 by the authors.

Figure 1
Figure 1. Molecule with its SMILES notation. Brackets correspond to branches, = to a double bond. The progress in LLMs and their reason￾ing has been remarkable [34]. While the rea￾soning frameworks enhance logical coherence, they lack chemical intuition [4]. Chemical lan￾guage is often challenging for LLMs [14], es￾pecially in SMILES (Simplified Molecular In￾put Line Entry System) [31] strings, which en￾code molecular structu… view at source ↗
Figure 2
Figure 2. Overview of our experimental setup. We experiment with seven different [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Average with standard deviation LLM score over 26 questions with five [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: LLM score difference be￾tween CodeAgent and ToolCall￾ingAgent across 26 questions and 5 LLMs (average over 5 runs). Looking beyond average performance, there are a few questions where models that perform worse overall match or even surpass the top-performing ones (see …
Figure 5
Figure 5. Figure 5: Difference in LLM score for different system prompts across 26 questions [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

95 extracted references · 66 canonical work pages

  1. [1]

    D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes. Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    2023

  2. [2]

    F. Cai, J. Bai, T. Tang, J. Luo, T. Zhu, L. Liu, and F. Luo. MolLangBench: A Com- prehensive Benchmark for Language-Prompted Molecular Structure Recognition, Editing, and Generation.arXiv preprint arXiv:2505.15054, 2025

    arXiv 2025

  3. [3]

    Costello and J

    S. Costello and J. Roodenburg. Acquiescence response bias—Yeasaying and higher education. The Educational and Developmental Psychologist, 32(2):105–119, 2015

    2015

  4. [4]

    D. Cui, S. Long, and Q. Li. Can reasoning power significantly improve the knowledge of large language models for chemistry?–Based on conversations with Deepseek and ChatGPT.ChemRxiv, 2025

    2025

  5. [5]

    B. Gao, Y. Huang, Y. Liu, W. Xie, W.-Y. Ma, Y.-Q. Zhang, and Y. Lan. Pharma- gents: Building a virtual pharma with large language model agents.arXiv preprint arXiv:2503.22164, 2025

    arXiv 2025

  6. [6]

    A. E. Ghareeb, B. Chang, L. Mitchener, A. Yiu, C. J. Szostkiewicz, J. M. Laurent, M. T. Razzak, A. D. White, M. M. Hinks, and S. G. Rodriques. Robin: A multi- agent system for automating scientific discovery.arXiv preprint arXiv:2505.13400, 2025

    arXiv 2025

  7. [7]

    Grougan, M

    M. Grougan, M. A. Hix, and A. R. Walker. Librarian of Alexandria: An Extensible LLM-based Chemical Data Extraction Pipeline.ChemRxiv, 2025

    2025

  8. [8]

    J. Gu, X. Jiang, Z. Shi, H. Tan, X. Zhai, C. Xu, W. Li, Y. Shen, S. Ma, H. Liu, et al. A survey on LLM-as-a-judge.arXiv preprint arXiv:2411.15594, 2024

    arXiv 2024

Show all 95 references
  1. [9]

    T. Guo, B. Nan, Z. Liang, Z. Guo, N. Chawla, O. Wiest, X. Zhang, et al. What can large language models do in chemistry? a comprehensive benchmark on eight tasks. Advances in Neural Information Processing Systems, 36:59662–59688, 2023

    2023

  2. [10]

    Hosseini and H

    S. Hosseini and H. Seilani. The role of agentic AI in shaping a smart future: A systematic review. Array, page 100399, 2025

    2025

  3. [11]

    Aitomia:YourIntelligent Assistant for AI-Driven Atomistic and Quantum Chemical Simulations

    J.Hu,H.Nawaz,Y.Rui,L.Chi,A.Ullah,andP.O.Dral. Aitomia:YourIntelligent Assistant for AI-Driven Atomistic and Quantum Chemical Simulations. arXiv preprint arXiv:2505.08195, 2025

    2025

  4. [12]

    Huang, S

    K. Huang, S. Zhang, H. Wang, Y. Qu, Y. Lu, Y. Roohani, R. Li, L. Qiu, J. Zhang, Y. Di, et al. Biomni: A General-Purpose Biomedical AI Agent. bioRxiv, pages 2025–05, 2025

    2025

  5. [13]

    Inoue, T

    Y. Inoue, T. Song, and T. Fu. DrugAgent: Explainable Drug Repurposing Agent with Large Language Model-based Reasoning. arXiv preprint arXiv:2408.13378, 2024

    2024 arXiv

  6. [14]

    Y. Jang, J. Kim, and S. Ahn. Improving Chemical Understanding of LLMs via SMILES Parsing. arXiv preprint arXiv:2505.16340, 2025

    2025 arXiv

  7. [15]

    Johansen, I

    K. Johansen, I. Lundström, and B. Liedberg. Sensitivity deviation: instrumen- tal linearity errors that influence concentration analyses and kinetic evaluation of biomolecular interactions. Biosensors and Bioelectronics, 15(9-10):503–509, 2000

    2000

  8. [16]

    H. Kim, Y. Jang, and S. Ahn. MT-Mol: Multi Agent System with Tool-based Reasoning for Molecular Optimization.arXiv preprint arXiv:2505.20820, 2025

    2025 arXiv

  9. [17]

    S. Kim, J. Chen, T. Cheng, A. Gindulyte, J. He, S. He, Q. Li, B. A. Shoemaker, P. A. Thiessen, B. Yu, et al. PubChem 2025 update. Nucleic Acids Research, 53(D1):D1516–D1525, 2025

    2025

  10. [18]

    Klinger, C

    R. Klinger, C. Kolářik, J. Fluck, M. Hofmann-Apitius, and C. M. Friedrich. Detec- tion of IUPAC and IUPAC-like chemical names.Bioinformatics, 24(13):i268–i276, 2008. 10 van Weesep et al

    2008

  11. [19]

    G. Landrum. Rdkit: Cheminformatics and Machine Learning Software RDKIT, 2013

    2013

  12. [20]

    K. Li, Z. Wu, S. Wang, and W. Hu. DrugPilot: LLM-based Parameterized Rea- soning Agent for Drug Discovery.arXiv preprint arXiv:2505.13940, 2025

    2025 arXiv

  13. [21]

    J. Liu, A. Liu, X. Lu, S. Welleck, P. West, R. L. Bras, Y. Choi, and H. Hajishirzi. Generated Knowledge Prompting for Commonsense Reasoning. InAnnual Meeting of the Association for Computational Linguistics, 2021

    2021

  14. [22]

    A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. D. White, and P. Schwaller. Aug- menting large language models with chemistry tools.Nature Machine Intelligence, 6(5):525–535, 2024

    2024

  15. [23]

    Mirza, N

    A. Mirza, N. Alampara, S. Kunchapu, M. Ríos-García, B. Emoekabu, A. Krishnan, T. Gupta, M. Schilling-Wilhelmi, M. Okereke, A. Aneesh, et al. A framework for evaluating the chemical knowledge and reasoning abilities of large language models against the expertise of chemists.Nat...

    2025

  16. [24]

    N. T. Runcie, C. M. Deane, and F. Imrie. Assessing the Chemical Intelligence of Large Language Models.arXiv preprint arXiv:2505.07735, 2025

    2025 arXiv

  17. [25]

    Saito, A

    K. Saito, A. Wachi, K. Wataoka, and Y. Akimoto. Verbosity bias in preference labeling by large language models.arXiv preprint arXiv:2310.10076, 2023

    2023 arXiv

  18. [26]

    Scialom, P.-A

    T. Scialom, P.-A. Dray, P. Gallinari, S. Lamprier, B. Piwowarski, J. Staiano, and A. Wang. QuestEval: Summarization asks for fact-based evaluation.arXiv preprint arXiv:2103.12693, 2021

    2021 arXiv

  19. [27]

    D. R. Serrano, F. C. Luciano, B. J. Anaya, B. Ongoren, A. Kara, G. Molina, B. I. Ramirez, S. A. Sánchez-Guirales, J. A. Simon, G. Tomietto, et al. Artificial intelligence (AI) applications in drug discovery and drug delivery: Revolutionizing personalized medicine. Pharmaceutic...

    2024

  20. [28]

    Szymański, M

    P. Szymański, M. Markowicz, and E. Mikiciuk-Olasik. Adaptation of high- throughput screening in drug discovery—toxicological screening tests. Interna- tional journal of molecular sciences, 13(1):427–452, 2011

    2011

  21. [29]

    E. Wang, S. Schmidgall, P. F. Jaeger, F. Zhang, R. Pilgrim, Y. Matias, J. Barral, D. Fleet, and S. Azizi. Txgemma: Efficient and agentic LLMs for therapeutics. arXiv preprint arXiv:2504.06196, 2025

    2025 arXiv

  22. [30]

    X. Wang, Y. Chen, L. Yuan, Y. Zhang, Y. Li, H. Peng, and H. Ji. Executable code actions elicit better LLM agents. InForty-first International Conference on Machine Learning, 2024

    2024

  23. [31]

    Weininger

    D. Weininger. SMILES, a chemical language and information system. 1. Intro- duction to methodology and encoding rules.Journal of chemical information and computer sciences, 28(1):31–36, 1988

    1988

  24. [32]

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  25. [33]

    G. Ye, X. Cai, H. Lai, X. Wang, J. Huang, L. Wang, W. Liu, and X. Zeng. DrugAs- sist:Alargelanguagemodelformoleculeoptimization. Briefings in Bioinformatics, 26(1):bbae693, 2025

    2025

  26. [34]

    Zhang, S

    Y. Zhang, S. A. Khan, A. Mahmud, H. Yang, A. Lavin, M. Levin, J. Frey, J. Dunn- mon, J. Evans, A. Bundy, S. Dzeroski, J. Tegner, and H. Zenil. Exploring the role of large language models in the scientific method: from hypothesis to discovery. npj Artificial Intelligence, 1(1):...

    2025

  27. [35]

    How many hydrogen donors does aspirin have?

  28. [36]

    How many hydrogen donors doesCC(=O)OC1=CC=CC=C1C(=O)O have?

  29. [37]

    How many hydrogen acceptors doesC1OCc2c1ccc3S=CC=Cc23 have?

  30. [38]

    Does ibuprofen have a higher logP value than glucose?

  31. [39]

    What is the molecular weight of caffeine?

  32. [40]

    What is the pKa of the cyanogroup in acetaminophen?

  33. [41]

    What is the pKa of the cyanogroup inCO2?

  34. [42]

    What is the molecular weight ofC1OCc2c1ccc3S=CC=Cc23?

  35. [43]

    Is testosterone an aromatic compound?

  36. [44]

    What is the molecular formula of glucose?

  37. [45]

    What is the number of hydrogen bond donors inNC(=O)Oc1ccccc1C=O?

  38. [46]

    Does guanine have more hydrogen bond donors or acceptors?

  39. [47]

    Does enzalutamide (CC1(C(=O)N(C(=S)N1C2=CC(=C(C=C2)C(=O)NC)F)C3= CC(=C(C=C3)C#N)C(F)(F)F)C) contain a cyano group (CC#N)?

  40. [48]

    Howmanyether( COC)groupsdoesglucose( C([C@@H]1[C@H]([C@@H]([C@H] (C(O1)O)O)O)O)O) contain?

  41. [49]

    Howmanycarboxylicacid( COOH)groupsdoestestosterone( C[C@]12CC[C@H] 3[C@H]([C@@H]1CC[C@@H]2O)CCC4=CC(=O)CC[C@]34C) contain?

  42. [50]

    What is the number of aromatic carbons in indole (c1ccc2[nH]ccc2c1)?

  43. [51]

    What type of LogP can you calculate?

  44. [52]

    What is the Crippen LogP of octanol?

  45. [53]

    What is the xlogP of aspirin?

  46. [54]

    What is the number of rotatable bonds in ethene?

  47. [55]

    How many rotatable bonds doesCC(C)C1CCC1 have?

  48. [56]

    Based on the number of rotatable bonds, is tabersonine or poloppin a more flexible compound?

  49. [57]

    How many atoms does a water molecule have?

  50. [58]

    How many atoms more does oestrogen have than progesterone?

  51. [59]

    How many nitrogens are there in Wedeloside?

  52. [60]

    12 van Weesep et al

    Give me the number of hydrogens, carbons, and sulfur atoms in quercitrin. 12 van Weesep et al. A.2 Tools

  53. [61]

    It returns a list of compounds

    pubchem_tool: This tool uses PubChemPy to retrieve information about a compound. It returns a list of compounds

  54. [62]

    mol_from_smiles: This tool uses RDKit to convert a SMILES string to a molecule object

  55. [63]

    smiles_from_mol: This tool uses RDKit to convert a molecule object to a SMILES string

  56. [64]

    NumHDonors: This tool uses RDKit to calculate the number of hydrogen bond donors in a molecule

  57. [65]

    NumHAcceptors: This tool uses RDKit to calculate the number of hydrogen bond acceptors in a molecule

  58. [66]

    If this tool is called by the agent, specify that this calculates the Crippen LogP, don’t just say LogP

    Crippen_LogP: This tool uses RDKit to calculate the Crippen LogP of a molecule. If this tool is called by the agent, specify that this calculates the Crippen LogP, don’t just say LogP

  59. [67]

    MolWt: This tool uses RDKit to calculate the molecular weight of a molecule

  60. [68]

    MolImage: This tool uses RDKit to generate an image of a molecule and save it to a file

  61. [69]

    NumRotatableBonds: This tool uses RDKit to calculate the number of ro- tatable bonds in a molecule

  62. [70]

    N_atoms: This tool uses RDKit to calculate the number of atoms in a molecule

  63. [71]

    AromaticAtoms: This tool uses RDKit to get the indices of aromatic atoms in a molecule

  64. [72]

    atom_counts: This tool uses RDKit to count the number of each type of atom in a molecule

  65. [73]

    get_attibutes: This function takes an object and a list of attributes, and returns a dictionary with the attribute names as keys and their values as values

  66. [74]

    get_attribute_value:Thisfunctiontakesanobjectandanattributename, and returns the value of the attribute

  67. [75]

    get_smiles_from_pcp_compound: This function takes a PubChemPy com- pound object and returns its canonical SMILES representation

  68. [76]

    create_smarts_pattern:ThisfunctiontakesaSMILESstringoramolecule object and returns its SMARTS pattern

  69. [77]

    mes- sage

    find_substructure_match: This function takes a molecule object and a SMARTS pattern, and returns the indices of the atoms that match the pat- tern. Exploring Modularity of Agentic Systems for Drug Discovery 13 A.3 Model Details Model Name Implementation Version GPT-3.5-Turbo A...

    2019

  70. [78]

    Always provide a ’ Thought : ’ sequence , and a ’ Code :\ n ‘ ‘ ‘ py ’ sequence ending with ’ ‘ ‘ ‘ < end_code > ’ sequence , else you will fail

  71. [79]

    Use only va ria bl es that you have defined !

  72. [80]

    What is the place where James Bond lives ?

    Always use the right ar gu me nt s for the tools . DO NOT pass the ar gu me nt s as a dict as in ’ answer = wiki ({ ’ query ’: " What is the place where James Bond lives ?"}) ’, but use the arg um en ts directly as in ’ answer = wiki ( query =" What is the place where James Bo...

  73. [81]

    Take care to not chain too many s e q u e n t i a l tool calls in the same code block , e s p e c i a l l y when the output format is u n p r e d i c t a b l e . For instance , a call to search has an u n p r e d i c t a b l e return format , so do not have another tool call t...

  74. [82]

    Call a tool only when needed , and never re - do a tool call that you p r e v i o u s l y did with the exact same p a r a m e t e r s

  75. [83]

    Don ’ t name any new variable with the same name as a tool : for instance don ’ t name a variable ’ final_answer ’

  76. [84]

    Never create any notional va ria bl es in our code , as having these in your logs will derail you from the true v ar iab le s

  77. [85]

    You can use imports in your code , but only from the fo ll ow in g list of modules : {{ a u t h o r i z e d _ i m p o r t s }}

  78. [86]

    The state persists between code e x e c u t i o n s : so if in one step you ’ ve created v ar ia ble s or imported modules , these will all persist

  79. [87]

    Now Begin ! 22 van Weesep et al

    Don ’ t give up ! You ’ re in charge of solving the task , not pro vi di ng d i r e c t i o n s to solve it . Now Begin ! 22 van Weesep et al. A.6 LLM-as-a-judge e v a l u a t i o n _ p r o m p t = """ Please give a score for how well the agent p erf or me d on the task . You ...

  80. [88]

    S um ma riz e and get adjusted answer to the task based on the agent ’ s answer

  81. [89]

    Compare the final answer to the expected answer

  82. [90]

    First de ter mi ne if the agent ’ s answer is correct or not

    Give a score between 0 and 100 based on how well the agent ’ s answer matches the expected answer . First de ter mi ne if the agent ’ s answer is correct or not . If the agent ’ s answer is correct , give a score of 100. If the agent ’ s answer is incorrect , give a score base...

  83. [91]

    "" prompt = f

    If the agent ’ s answer is not correct , please provide a reason for the score . Example : Task : Find the SMILES for the compound with CID 2244 Agent ’ s Answer : The SMILES for the compound with CID 2244 is C1 = CC = C ( C = C1 ) C (=0) 0 Adjusted Answer : C1 = CC - C ( C = ...

  84. [92]

    Does enzalutamide contain a cyano group?

  85. [93]

    How many ether groups does glucose contain?

  86. [94]

    How many carboxylic acid groups does testosterone contain

  87. [95]

    How many atoms more does oestrogen have than progesterone?

    What is the number of aromatic carbons in indole? While performance is clearly affected (see Supplementary Figure 5a), there is no consistent improvement or decline across all questions, suggesting that the impact varies depending on the specific chemical compound covered by t...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.