pith. sign in

arxiv: 2606.02618 · v1 · pith:JX3KDL7Bnew · submitted 2026-05-27 · 💻 cs.CE · cs.AI· cs.MA· physics.chem-ph

Closed-Loop Molecular Design with Calibrated Deference

Newman Cheng , Gordon Broadbent IV , Jason Dong , Syed Mohammed Ali Hussaini , Farman Ullah , Morris Sharp , Gabrielle Barnes , Nanlin Guo
show 6 more authors
Deyu Zou Karin Strauss William Chappell David G. Kwabi Bichlien H. Nguyen Jake A. Smith
This is my paper

Pith reviewed 2026-06-29 09:15 UTC · model grok-4.3

classification 💻 cs.CE cs.AIcs.MAphysics.chem-ph
keywords closed-loop molecular designAI agentredox flow batterycalibrated deferencebelief-state graphmechanistic hypothesisAORFB negolyteion pairing
0
0 comments X

The pith

An AI agent with a belief-state graph and recursive planning loop can generate mechanistic hypotheses to diagnose and fix its own design failures in closed-loop molecular campaigns.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces CLIO, which maintains a continuously updated belief-state graph inside a recursive plan-then-act loop. In a three-round human-AI campaign to improve an aqueous organic redox flow battery negolyte, CLIO proposed candidates that raised redox potential by 130 mV, then diagnosed unexpected poor reversibility as phosphonate-potassium ion pairing and prescribed a sulfonate replacement that restored reversibility while retaining a 90 mV gain. A sympathetic reader would care because the agent contributed to proposal, interpretation, and redesign rather than only to property prediction.

Core claim

CLIO couples a continuously-updated belief-state graph with a recursive plan-then-act loop to produce calibrated deference: the capacity to recognize when its own tools or assumptions are failing, adapt its strategy, and generate mechanistic hypotheses that guide experimental revision, as shown when it traced a reversibility regression in a phosphonate candidate to ion pairing and prescribed a sulfonate fix that improved performance.

What carries the argument

The belief-state graph inside a recursive plan-then-act loop, which continuously updates the agent's model of the problem and enables it to revise both strategy and molecular proposals when experiments contradict prior assumptions.

If this is right

  • CLIO can lead both proposal and interpretation steps while working with chemists who handle synthesis and characterization.
  • The agent can identify and correct performance regressions that standard property predictors miss.
  • Over multiple design-make-test rounds the agent can converge on candidates that deliver both higher redox potential and acceptable reversibility.
  • The same architecture can close the loop by prescribing concrete structural replacements that maintain prior gains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same belief-state-plus-recursive-loop pattern could be tested in other molecular domains where unexpected side reactions or solubility issues appear after initial property optimization.
  • If the architecture generalizes, it might reduce the total number of synthesis rounds needed by surfacing mechanistic explanations earlier than trial-and-error iteration alone.
  • The approach raises the question of how much of the observed performance stems from the explicit graph structure versus the language model's implicit knowledge, which could be probed by ablating the graph in future experiments.

Load-bearing premise

The agent's ability to generate and act on mechanistic hypotheses comes from the belief-state graph and recursive loop rather than from the underlying language model or from human chemist input.

What would settle it

A controlled comparison in which the same language model without the belief-state graph and recursive loop fails to produce discriminating diagnostics or successful redesigns in an equivalent AORFB campaign would falsify the claim that those structures produce the observed calibrated deference.

read the original abstract

We present Cognitive Loop via In-Situ Optimization (CLIO), an agent that couples a continuously-updated belief-state graph with a recursive plan-then-act loop. The result is a reasoning agent that can contribute something qualitatively different, which we term \emph{calibrated deference}: the capacity to recognize when its own tools or assumptions are failing, to adapt its strategy in response, and to generate mechanistic hypotheses that guide experimental revision. We tested CLIO in a closed-loop human-AI campaign to design an aqueous organic redox flow battery (AORFB) negolyte, with CLIO leading proposal and interpretation in close partnership with chemists who synthesized, characterized, and weighed in on design choices. Across 17 candidates over three rounds, CLIO converged on a top phosphonate candidate; characterization confirmed a 130~mV improvement in redox potential over the literature baseline. Characterization then revealed unexpectedly poor electrochemical reversibility -- a regression no property predictor had flagged. CLIO generated competing mechanistic hypotheses, prioritized discriminating diagnostics, traced the failure to phosphonate-potassium ion pairing, and prescribed a sulfonate replacement. The resulting compound showed substantially improved electrochemical reversibility and maintained a 90~mV improvement in redox potential, closing the design-make-test-redesign loop.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 1 minor

Summary. The paper presents Cognitive Loop via In-Situ Optimization (CLIO), an agent coupling a continuously-updated belief-state graph with a recursive plan-then-act loop to enable 'calibrated deference'—recognizing tool/assumption failures, adapting strategy, and generating mechanistic hypotheses. In a closed-loop human-AI campaign for an aqueous organic redox flow battery negolyte, CLIO led proposals and interpretation across 17 candidates in three rounds, converging on a phosphonate with 130 mV redox improvement; upon observing poor reversibility, it generated hypotheses, prioritized diagnostics, traced failure to phosphonate-K+ pairing, and prescribed a sulfonate replacement yielding improved reversibility while retaining a 90 mV gain.

Significance. If the attribution to the specific architecture holds and the experimental outcomes are robust, the work could advance AI-driven discovery by showing agents that contribute mechanistic hypothesis generation and adaptive iteration in real chemistry campaigns, beyond standard property prediction or prompting. The concrete closure of a design-make-test-redesign loop provides a tangible case study for calibrated deference in materials applications.

major comments (3)
  1. [Abstract] Abstract: The central claim that CLIO's generation of competing mechanistic hypotheses, prioritization of diagnostics, and tracing of the phosphonate-K+ pairing failure stems specifically from the belief-state graph plus recursive plan-then-act architecture is not isolated from base LLM capabilities or human chemist input; the manuscript supplies no ablation studies, baseline comparisons against standard LLM prompting without the graph, agent reasoning traces, or quantification of human steering.
  2. [Abstract] Abstract: No internal mechanism details, statistical controls, error analysis, or quantitative metrics (e.g., for the reported 130 mV and 90 mV redox improvements or reversibility changes) are provided to support the experimental outcomes or the claim of mechanistic hypothesis generation.
  3. [Abstract] Abstract: The description of how the belief-state graph is constructed, continuously updated, or used within the recursive loop is absent at a level that would allow evaluation of whether it produces qualitatively new behavior; this is load-bearing for the novelty of calibrated deference.
minor comments (1)
  1. [Abstract] Abstract: The notation '130~mV' and '90 mV' should be standardized for clarity and consistency with standard scientific formatting.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thoughtful and constructive report. The comments correctly identify several areas where the manuscript would benefit from additional detail and clarification. Below we respond point-by-point to the three major comments and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that CLIO's generation of competing mechanistic hypotheses, prioritization of diagnostics, and tracing of the phosphonate-K+ pairing failure stems specifically from the belief-state graph plus recursive plan-then-act architecture is not isolated from base LLM capabilities or human chemist input; the manuscript supplies no ablation studies, baseline comparisons against standard LLM prompting without the graph, agent reasoning traces, or quantification of human steering.

    Authors: We agree that the manuscript does not contain ablation studies or direct comparisons against base LLM prompting, and therefore cannot quantitatively isolate the contribution of the belief-state graph and recursive loop from general LLM capabilities or human input. The work is presented as an integrated case study of a closed-loop campaign rather than a controlled benchmark of the architecture. In the revised manuscript we will add a limitations paragraph that explicitly acknowledges this gap, include additional excerpts from the agent reasoning traces (currently only summarized), and provide a clearer accounting of the points at which human chemists provided input versus where CLIO generated hypotheses and diagnostics autonomously. We will also outline planned future ablation experiments. revision: partial

  2. Referee: [Abstract] Abstract: No internal mechanism details, statistical controls, error analysis, or quantitative metrics (e.g., for the reported 130 mV and 90 mV redox improvements or reversibility changes) are provided to support the experimental outcomes or the claim of mechanistic hypothesis generation.

    Authors: The referee is correct that the abstract and main text currently lack error bars, replicate statistics, and quantitative reversibility metrics. The experimental values (130 mV and 90 mV) are taken from single representative cyclic voltammograms shown in the supplementary information; no formal error analysis or statistical controls are reported. In revision we will move the key electrochemical data into the main text with error estimates from replicate measurements, add peak-current-ratio values to quantify reversibility changes, and expand the description of how the mechanistic hypotheses were generated and tested within the agent loop. revision: yes

  3. Referee: [Abstract] Abstract: The description of how the belief-state graph is constructed, continuously updated, or used within the recursive loop is absent at a level that would allow evaluation of whether it produces qualitatively new behavior; this is load-bearing for the novelty of calibrated deference.

    Authors: We accept that the current description of belief-state graph construction, update rules, and integration with the recursive plan-then-act loop is insufficient for readers to assess its role in producing calibrated deference. The methods section provides a high-level overview but omits implementation specifics such as node/edge update logic and query mechanisms. In the revised manuscript we will add a dedicated subsection with a diagram of the graph structure, pseudocode for the update and planning steps, and concrete examples of how the graph was modified during the three experimental rounds. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical outcome with no derivation chain

full rationale

The paper describes an agent architecture (belief-state graph + recursive plan-then-act) and reports its use in an experimental AORFB design campaign. No equations, parameters, or mathematical derivations appear. The central claim concerns experimental outcomes (redox potential improvements, reversibility fixes) rather than any quantity defined in terms of the agent's own outputs. No self-citations, uniqueness theorems, or ansatzes are invoked. The attribution of 'calibrated deference' to the architecture is a hypothesis about system behavior, not a self-referential reduction by construction. This is a standard non-circular empirical report.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Review limited to abstract; implementation details, training data, and internal algorithms unavailable. The central claim rests on the unexamined premise that the described loop produces reliable mechanistic hypotheses.

axioms (1)
  • domain assumption The belief-state graph accurately captures and updates uncertainty in chemical property predictions
    Invoked as the foundation for the agent's ability to detect failures and generate hypotheses.
invented entities (1)
  • CLIO agent no independent evidence
    purpose: To implement calibrated deference via belief graph and recursive loop
    New system introduced in the paper; no independent evidence of its internal behavior provided beyond the abstract narrative.

pith-pipeline@v0.9.1-grok · 5808 in / 1369 out tokens · 35770 ms · 2026-06-29T09:15:34.002817+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

38 extracted references · 8 canonical work pages · 1 internal anchor

  1. [1]

    Preprint at https://arxiv.org/abs/2503.24047

    Ren, S.et al.Towards Scientific Intelligence: A Survey of LLM-based Scientific Agents (2025). Preprint at https://arxiv.org/abs/2503.24047

    work page arXiv 2025

  2. [2]

    EMNLP17733–17750 (2025)

    Zheng, T.et al.From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery.Proc. EMNLP17733–17750 (2025)

    2025

  3. [3]

    M.et al.Augmenting large language models with chemistry tools

    Bran, A. M.et al.Augmenting large language models with chemistry tools. Nature Machine Intelligence6, 525–535 (2024)

    2024

  4. [4]

    A., MacKnight, R., Kline, B

    Boiko, D. A., MacKnight, R., Kline, B. & Gomes, G. Autonomous chemical research with large language models.Nature624, 570–578 (2023)

    2023

  5. [5]

    BioDiscoveryAgent: An AI agent for designing genetic perturbation experiments.arXiv preprint arXiv:2405.17631, 2024

    Roohani, Y.et al.BioDiscoveryAgent: An AI Agent for Designing Genetic Per- turbation Experiments.International Conference on Learning Representations (2025). Preprint at https://arxiv.org/abs/2405.17631. 15

    work page arXiv 2025

  6. [6]

    E.et al.A multi-agent system for automating scientific discovery

    Ghareeb, A. E.et al.A multi-agent system for automating scientific discovery. Nature(2026)

    2026

  7. [7]

    & Chappell, W

    Cheng, N., Broadbent, G. & Chappell, W. Cognitive Loop via In-Situ Optimiza- tion: Self-Adaptive Reasoning for Science (2025). Preprint at https://arxiv.org/ abs/2508.02789

    work page arXiv 2025

  8. [8]

    ACS Appl

    Singh, S.et al.Sulfonated Benzo[c]cinnolines for Alkaline Redox-Flow Batteries. ACS Appl. Energy Mater.8, 7904–7911 (2025)

    2025

  9. [9]

    & Schuffenhauer, A

    Ertl, P. & Schuffenhauer, A. Estimation of synthetic accessibility score of drug- like molecules based on molecular complexity and fragment contributions.Journal of Cheminformatics1, 8 (2009)

    2009

  10. [10]

    Zhang, Z.et al.A multimodal robotic platform for multi-element electrocatalyst discovery.Nature647, 390–396 (2025)

    2025

  11. [11]

    ArXiv:2106.05234

    Ying, C.et al.Do transformers really perform bad for graph representation? NIPS’21: Proceedings of the 35th International Conference on Neural Information Processing Systems28877–28888 (2021). ArXiv:2106.05234

    work page arXiv 2021

  12. [12]

    URL https://www.rdkit.org

    Landrum, G.et al.RDKit: Open-source cheminformatics software (2024). URL https://www.rdkit.org. https://github.com/rdkit/rdkit

    2024

  13. [13]

    Preprint at https://arxiv.org/abs/2412.05269

    Maziarz, K.et al.Chemist-aligned retrosynthesis by ensembling diverse inductive bias models (2024). Preprint at https://arxiv.org/abs/2412.05269

    work page arXiv 2024

  14. [14]

    Deep research tool for agents (2025)

    Microsoft. Deep research tool for agents (2025). URL https://learn.microsoft. com/en-us/azure/foundry-classic/agents/how-to/tools-classic/deep-research. Accessed: 2026-05-16

    2025

  15. [15]

    π-stacking

    Xiao, Q., LeVine, M. S. & Iverson, B. L. Rethinking the terms “π-stacking” and “π–πstacking” again: A proposal to clarify the language of aromatic interactions. Journal of the American Chemical Society148, 15331–15340 (2026)

    2026

  16. [16]

    & Costentin, C.Elements of Molecular and Biomolecular Elec- trochemistry: An Electrochemical Approach to Electron Transfer Chemistry2nd edn (John Wiley & Sons, 2019)

    Sav´ eant, J.-M. & Costentin, C.Elements of Molecular and Biomolecular Elec- trochemistry: An Electrochemical Approach to Electron Transfer Chemistry2nd edn (John Wiley & Sons, 2019)

    2019

  17. [17]

    & Lajunen, L

    Popov, K., R¨ onkk¨ om¨ aki, H. & Lajunen, L. H. J. Critical evaluation of stability constants of phosphonic acids (IUPAC technical report).Pure Appl. Chem.74, 2227 (2002)

    2002

  18. [18]

    A benchmark of expert-level academic questions to assess AI capabilities.Nature649, 1139–1146 (2026)

    Center for AI Safety, Scale AI & HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities.Nature649, 1139–1146 (2026). 16

    2026

  19. [19]

    L., Pak, J

    Swanson, K., Wu, W., Bulaong, N. L., Pak, J. E. & Zou, J. The Virtual Lab of AI agents designs new SARS-CoV-2 nanobodies.Nature646, 716–723 (2025)

    2025

  20. [20]

    FastAPI (2018)

    Ram´ ırez, S. FastAPI (2018). URL https://fastapi.tiangolo.com. https://github. com/fastapi/fastapi

    2018

  21. [21]

    Preprint at https: //doi.org/10.26434/chemrxiv.15002385/v1

    Martinez-Baez, E.et al.Mixed Computational/Experimental Screening for Aqueous Organic Redox Flow Battery Negolytes (2026). Preprint at https: //doi.org/10.26434/chemrxiv.15002385/v1

    work page doi:10.26434/chemrxiv.15002385/v1 2026

  22. [22]

    & Irwin, J

    Sterling, T. & Irwin, J. J. ZINC 15 – ligand discovery for everyone.Journal of Chemical Information and Modeling55, 2324–2337 (2015)

    2015

  23. [23]

    diffusional

    Dickinson, E. J. F., Limon-Petersen, J. G., Rees, N. V. & Compton, R. G. How much supporting electrolyte is required to make a cyclic voltammetry experiment quantitatively “diffusional”? A theoretical and experimental investigation.J. Phys. Chem. C113, 11157–11171 (2009)

    2009

  24. [24]

    Preprint at https://arxiv.org/abs/2502.12845

    Ran, N.et al.ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyond (2025). Preprint at https://arxiv.org/abs/2502.12845

    work page arXiv 2025

  25. [25]

    AlphaEvolve: A coding agent for scientific and algorithmic discovery

    Novikov, A.et al.AlphaEvolve: A coding agent for scientific and algorithmic discovery (2025). Preprint at https://arxiv.org/abs/2506.13131

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  26. [26]

    Gottweis, J.et al.Accelerating scientific discovery with Co-Scientist.Nature (2026)

    2026

  27. [27]

    Huang, K.et al.Biomni: A General-Purpose Biomedical AI Agent.bioRxiv (2025)

    2025

  28. [28]

    & Coley, C

    Gao, W., Fu, T., Sun, J. & Coley, C. W. Sample efficiency matters: A benchmark for practical molecular optimization.Advances in Neural Information Processing Systems35, 21342–21357 (2022). Supplementary Information Contents

    2022

  29. [29]

    Design prompt (Section S1)

  30. [30]

    Calibrated deference: extended discussion (Section S2)

  31. [31]

    Comparison with related agentic and optimization systems (Section S3)

  32. [32]

    CLIO hypothesis inventory (Section S4)

  33. [33]

    CLIO for strictly numerical optimization (Section S5)

  34. [34]

    Experimental characterization of ExLLM structures (Section S6)

  35. [35]

    Electrochemical characterization (Section S7)

  36. [36]

    Spectroelectrochemistry (Section S8) 17

  37. [37]

    Solubility studies (Section S9)

  38. [38]

    Aim to shift reduction potential negative vs. parent (∼+0.7 V vs. SHE) toward−1.2 to−0.3 V vs. SHE

    Synthetic procedures (Section S10) 18 S1 Design prompt Given the undecorated scaffold compound —C12=CC=CC=C1N=NC3=C2C=CC=C3— design derivative organic molecules that function as aqueous anolytes for redox flow batteries. The molecules must undergo a reversible reduction with a reduction potential between−1.2 V and−0.3 V vs. SHE, must remain chemically and...

    2048