Pith. sign in

REVIEW 3 major objections 36 references

A game-theoretic multi-agent loop trains a 7B chemistry model that cuts hallucinations by 79% while matching GPT-4o mini.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 07:58 UTC pith:R4556MAO

load-bearing objection Solid open chemistry-LLM systems paper with real artifacts and a useful ablation; the game-theory story and exact 79% hallucination cut are oversold relative to the evidence. the 3 major comments →

arxiv 2607.08403 v1 pith:R4556MAO submitted 2026-07-09 cs.AI

Game Theory Driven Multi-Agent Framework Mitigates Language Model Hallucination

Runzhe Liu , Biquan Bie , Zihao Wang , Yuchao Ma , Yexin Liu , Xinghai Li , Harry Yang , Wenbo Yang
show 2 more authors
Jinzhe Cao Shengyang Tao
This is my paper
classification cs.AI
keywords large language modelshallucination mitigationmulti-agent systemsgame theorychemistry reasoningchain-of-thought synthesisadaptive trainingOmniChem
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Lightweight language models often fail in chemistry because they imitate fluent text instead of following physical rules, so they produce confident but false answers. This paper presents G-Frame, an adaptive multi-agent system that uses team-game collaboration to generate carefully checked data and Bayesian updating to steer training and resource use. The loop produces a large specialized corpus of reasoning chains and question-answer pairs, which is then used to train OmniChem, a 7-billion-parameter chemistry model. OmniChem reaches performance parity with GPT-4o mini on custom and public chemistry benchmarks and shows a 79.46% drop in hallucinations relative to its base model. The authors further show the model can propose molecular designs and practical synthesis routes, arguing the same closed-loop method can be reused for other rule-heavy scientific domains.

Core claim

G-Frame combines hierarchical team-game decomposition of generation tasks with Bayesian adaptive control of concurrency and training to force internalization of chemical constraints. The resulting closed loop synthesizes 363,045 chains-of-thought and 199,589 QA pairs and yields OmniChem-7B, which matches GPT-4o mini on ThChem and ChemBench while reducing the hallucination rate by 79.46% versus its base architecture.

What carries the argument

G-Frame: a three-layer multi-agent architecture in which short team-game collaborations among role-constrained executive agents refine intermediate states (lowering generation entropy), while a decisional agent runs a Bayesian game that updates workflow and hyperparameter policies from real-time feedback, forming an automated data-to-training closed loop.

Load-bearing premise

The paper treats the team-game and Bayesian formalization itself as what forces the model to internalize physical rules, rather than the gains mainly coming from cleaner multi-agent data and better adaptive training schedules.

What would settle it

Run an otherwise identical multi-agent data-cleaning, CoT/QA-synthesis, and adaptive-training pipeline that uses ordinary role prompts and simple feedback control, with no game-theoretic language or Bayesian update equations; if the same 79% hallucination drop and benchmark scores appear, the game-theoretic framing is not the causal mechanism.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 0 minor

Summary. The manuscript introduces G-Frame, a hierarchical multi-agent system that couples team-game role decomposition for data cleaning/augmentation/CoT–QA synthesis with Bayesian adaptive control of concurrency and training hyperparameters. Using this pipeline the authors clean a chemistry corpus, synthesize 363,045 CoT traces and 199,589 QA pairs, and train OmniChem-7B. They report parity with GPT-4o mini on author-built ThChem and on ChemBench, a 79.46% reduction in hallucinations versus the base model on ChemJudge (LLM-as-judge with human override), and qualitative demonstrations in BODIPY design, solubility optimization, and lidocaine retrosynthesis. Ablation Table 5 is used to argue that domain pretraining, synthetic fine-tuning, and adaptive training are all necessary.

Significance. If the closed-loop multi-agent synthesis plus adaptive training genuinely yields a 7B chemistry model that matches GPT-4o mini while sharply reducing factual errors, the work would be a useful engineering template for domain LLMs under data-sovereignty constraints. Strengths include open code, model, and datasets; a full factorial ablation (Table 5); human expert override on ChemJudge discrepancies; and concrete molecular-design case studies with DFT coordinates deposited. The game-theoretic formalization (SI §B, Eqs. 3–9) is presented as the causal mechanism for constraint internalization; that claim, if substantiated, would be of broader interest beyond chemistry.

major comments (3)
  1. Abstract / §3 / ChemJudge: The headline 79.46% hallucination reduction is load-bearing for the central claim, yet the manuscript never reports the absolute base-model versus OmniChem deduction totals (or per-question scores) that produce this percentage. ChemJudge is 471 author-curated questions scored by Gemini 3.1 Pro with human override only on discrepancies. Without published raw scores, inter-annotator agreement, and at least one independent public chemistry hallucination suite (or a non-author judge), the reduction figure cannot be independently verified and remains tied to an author-controlled metric.
  2. §2 and SI §B, Eqs. (3)–(6): The paper asserts that team-game role constraints and verifier weights S(zi) structurally reduce DKL(Ptrue ∥ Pteam) relative to monolithic generation, thereby forcing internalization of axiomatic chemical constraints. No measurement of DKL, of intermediate S(zi) distributions, or of entropy on held-out chemical text is provided. Ablation Table 5 shows that domain pretraining + synthetic CoT/QA + adaptive training matter for scores, but no arm isolates hierarchical game structure from ordinary multi-agent prompting, better corpus quality, and adaptive learning-rate/concurrency schedules. The causal mechanism claim therefore remains an interpretation of an engineering pipeline rather than a measured result.
  3. §3 / Methods (ThChem): Performance parity with GPT-4o mini rests substantially on ThChem 1.0/2.0, which are author-constructed and not independently validated. ChemBench provides a useful external check (49.82%), but the paper should report confidence intervals or multiple seeds, and should clarify how much of the claimed parity survives when ThChem is down-weighted or replaced by fully public suites. Without that, the strongest comparative claim is only partially secured.

Circularity Check

0 steps flagged

No load-bearing circular derivation; empirical claims rest on measured benchmarks (including external ChemBench) and human-verified ChemJudge, while the game-theoretic formalization is descriptive rather than a tautological reduction.

full rationale

The paper's central results (OmniChem-7B parity with GPT-4o mini on ThChem/ChemBench; 79.46% hallucination reduction vs base on ChemJudge; molecular design/synthesis examples) are empirical measurements, not quantities derived from the SI equations that reduce to their own inputs by construction. SI Eqs. (1)–(9) formalize autoregressive generation, team-game latent paths with verifier weights S(zi), and Bayesian POMDP utility maximization as a hierarchical optimization story; they assert that role constraints yield lower DKL(Ptrue∥Pteam) but never compute or fit DKL, S(zi), or the utility on held-out data and then re-label the same quantity as a prediction. Adaptive training uses ChemJudge-style LLM-referee payoffs (DeepSeek-R1) to update hyperparameters, while final ChemJudge numbers use Gemini 3.1 Pro plus human expert ground-truth override on discrepancies; this is mild self-reinforcement of an author-curated metric, not equivalence by definition, and is further checked by the independent ChemBench suite, ablation Table 5, open artifacts, and external DFT validation of design proposals. No uniqueness theorems, ansatzes, or load-bearing results are imported solely via overlapping-author citations. Standard multi-agent distillation + adaptive SFT is described with game-theoretic language, but the reported numbers do not collapse to the formalization or to fitted parameters renamed as predictions. Score remains near zero per the default expectation for self-contained empirical engineering papers.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 4 invented entities

The central performance claim rests less on pure math than on engineering choices: multi-agent role prompts as ‘utility functions,’ Bayesian belief updates over system state for concurrency/hyperparameters, LLM judges as payoff signals, and standard LLM training recipes. Free parameters (LRs, mix ratios, concurrency priors, temperatures) and domain assumptions about OCR/cleaning targets and judge reliability are load-bearing. Invented named systems (G-Frame, OmniChem, ChemJudge, ThChem) are the paper’s operational objects; independent evidence is partial via open weights/data and external ChemBench.

free parameters (6)
  • pretraining learning rate = 3e-5
    AdamW initial LR set to 3e-5; chosen training hyperparameter that shapes the domain-pretrained base.
  • SFT learning rate = 5e-5
    Full-parameter SFT uses 5e-5; directly affects instruction/CoT acquisition and forgetting dynamics in ablations.
  • domain:general corpus mix ratios = 8:2 then 4:6
    Initial 8:2 then third-round 4:6 chemistry:general mix; hand-chosen schedule claimed to mitigate catastrophic forgetting.
  • generation temperature for CoT/QA synthesis = 0.6
    Temperature 0.6 used during synthetic data generation; affects diversity/quality of the 363k/199k corpora.
  • adaptive concurrency prior and update policy = prior then feedback-updated (e.g., 16→8 example)
    Initial concurrency and Bayesian posterior adjustments from GPU/queue/KV feedback are free control parameters of the claimed efficiency/quality loop.
  • ChemJudge per-question scoring rule = 10 - n_hallucinations
    10 points minus 1 per hallucination (0 if irrelevant) defines the 79.46% reduction metric; scoring design is author-chosen.
axioms (5)
  • domain assumption Autoregressive LLMs produce hallucinations when generated distribution diverges from a low-entropy true domain distribution (DKL > ε).
    SI §B Eqs. 1–3; standard framing used to motivate the whole method.
  • ad hoc to paper Decomposing generation into short role-constrained agent steps with verifier weights reduces KL to the true distribution versus monolithic generation.
    SI Eqs. 4–6; asserted mechanism of the team game without direct KL measurements on chemistry outputs.
  • ad hoc to paper Macro control of workflow/hyperparameters is well-modeled as a POMDP solved by Bayesian belief updates maximizing a global utility with crash penalty.
    SI Eqs. 7–9 and Fig. 6; maps engineering adaptive control onto Bayesian game language.
  • domain assumption LLM-as-judge scores (with human override) are a valid operational measure of chemical hallucination rate.
    Methods ChemJudge; load-bearing for the 79.46% claim.
  • domain assumption Supplying full/core literature text as context during synthesis sufficiently grounds CoT/QA to limit teacher-model hallucination.
    Methods fine-tuning data construction; key data-quality premise.
invented entities (4)
  • G-Frame independent evidence
    purpose: Named hierarchical multi-agent framework combining team-game execution and Bayesian adaptive decisions for data and training.
    Core system contribution; evidence is implementation and ablations, not independent prior existence.
  • OmniChem (7B) independent evidence
    purpose: Resulting chemistry reasoning model trained via G-Frame.
    Primary empirical artifact; released on Hugging Face.
  • ChemJudge no independent evidence
    purpose: 471-question hallucination/accuracy evaluation and training payoff signal using LLM judges.
    Author-defined metric suite; partially validated by human experts but not a community standard benchmark.
  • ThChem 1.0/2.0 no independent evidence
    purpose: Author-constructed chemistry reasoning benchmarks, including unanswerable items in 2.0.
    Custom evals central to parity claims; external validity depends on expert construction claims.

pith-pipeline@v1.1.0-grok45 · 22246 in / 3729 out tokens · 44167 ms · 2026-07-10T07:58:21.663508+00:00 · methodology

0 comments
read the original abstract

The application of lightweight Large Language Models in rule-based scientific domains remains severely limited by their tendency to mimic linguistic patterns rather than reproduce axiomatic reasoning, causing frequent hallucinations. Here, we show that G-Frame, an adaptive multi-agent framework integrating Bayesian and team game principles, establishes an automated closed-loop for high-quality data synthesis and model training. By forcing the internalization of domain constraints through structured reasoning, we synthesized a specialized corpus of 363,045 chains-of-thought and 199,589 question-answer pairs. The resulting 7B model OmniChem achieves performance parity with GPT 4o mini on custom benchmarks and ChemBench while exhibiting a 79.46% reduction in hallucinations relative to its base architecture. We further demonstrate the advanced capabilities of OmniChem in molecular design and synthesis planning. This work establishes a scalable paradigm utilizing adaptive multi-agents to overcome inherent reasoning deficiencies, offering a feasible pathway for accelerating knowledge discovery in specialized scientific fields.

Figures

Figures reproduced from arXiv: 2607.08403 by Biquan Bie, Harry Yang, Jinzhe Cao, Runzhe Liu, Shengyang Tao, Wenbo Yang, Xinghai Li, Yexin Liu, Yuchao Ma, Zihao Wang.

Figure 1
Figure 1. Figure 1: a. G-Frame achieves adaptive strategies through two distinct modes: (i) team games and (ii) Bayesian games. b. The bar chart illustrates the number of open-source databases designated with chemical labels on Hugging Face as of May 15, 2025. Notably, a substantial portion of these datasets contains invalid entries or pertains to other domains; thus, the amount of genuinely usable data for chemistry is less … view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of G-Frame and its team-game and Bayesian-game mechanisms. The framework couples worker-agent role constraints with a decisional agent that updates prior and posterior decisions from real-time feedback. Prompt templates for team and Bayesian games are shown in panel b. 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: a. The performance of G-Frame compared to a single LLM on the SQuAD 2.0 benchmark is shown. b. The efficacy of G-Frame in concurrency optimization is demonstrated. c. A qualitative comparison of the question-answering performance for the three models pre-training. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: a. Scores of different models on the ThChem1.0, ThChem2.0, and ChemJudge benchmarks. The ThChem benchmarks use a 100-point scale, whereas ChemJudge is evaluated based on the hallucination rate. A dash (-) indicates that the model does not apply to this test. b. Flowchart illustrating the workflow for a deep researcher and domain knowledge QA Expert, with a visually enhanced knowledge graph shown as an exam… view at source ↗
Figure 5
Figure 5. Figure 5: Pseudocode for the G-Frame team game. Input: Sprior: Prior knowledge of the system state. T: Joint type space of Task Agents. Uglobal: The global utility function to be maximized. Output: A sequence of optimized decisions applied throughout the task lifecycle. 1 Function Dynamic Bayesian Adaptive Decision(Sprior, T, Uglobal) 2 // Step 1: Initialization 3 Decisional Agent <- Get Decisional Agent () 4 dcurre… view at source ↗
Figure 6
Figure 6. Figure 6: Pseudocode for the G-Frame Bayesian game. 11 [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: KV Cache usage and pending process statistics in synchronous mode. D Evaluation Protocols and Quantitative Tables ThChem Benchmark ThChem is a benchmark meticulously constructed to accurately assess the comprehensive capabilities of large language models in the domain of chemistry. Its content extensively covers core branches of the discipline, including inorganic, organic, analytical, and physical chemist… view at source ↗
Figure 8
Figure 8. Figure 8: KV Cache usage and pending process statistics at a fixed concurrency level of 50. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: KV Cache usage and pending process statistics at a fixed concurrency level of 100. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: KV Cache usage and pending process statistics at a fixed concurrency level of 200. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: KV Cache usage and pending process statistics at a fixed concurrency level of 500. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: KV Cache usage and pending process statistics under G-Frame adaptive concurrency control. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Token consumption statistics for the test tasks. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Original knowledge graph generated by the QA Expert module for the domain-specific chemistry example. E Chemical Design Calculations Here, we present a design case study in which OmniChem was tasked with creating BODIPY derivatives with targeted red-light absorption properties. Design of BODIPY Molecules with Enhanced Water Solubility Here, we demonstrate an example of using OmniChem to design BODIPY mole… view at source ↗
Figure 15
Figure 15. Figure 15: Simulated UV-Vis-NIR absorption spectrum and corresponding oscillator strengths for the red-absorbing BODIPY design [PITH_FULL_IMAGE:figures/full_fig_p030_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Molecular orbital isosurface visualization for HOMO. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Molecular orbital isosurface visualization for HOMO+1 [PITH_FULL_IMAGE:figures/full_fig_p031_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Molecular orbital isosurface visualization for LUMO. 31 [PITH_FULL_IMAGE:figures/full_fig_p031_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

36 extracted references · 36 canonical work pages · 8 internal anchors

  1. [1]

    Wang, H. et al. Scientific discovery in the age of artificial intelligence. Nature 620, 47–60 (2023).https://doi.org/10.1038/s41586-023-06221-2

    work page doi:10.1038/s41586-023-06221-2 2023

  2. [2]

    MacKnight, R. et al. Rethinking chemical research in the age of large language models. Nat. Comput. Sci. (2025).https://doi.org/10.1038/s43588-025-00811-y

    work page doi:10.1038/s43588-025-00811-y 2025

  3. [3]

    Zhang, J., Chen, X., Ye, X., Yang, Y. & Ai, B. Large Language Model in Materials Science: Roles, Challenges, and Strategic Outlook. Adv. Intell. Discovery n/a, 202500085 (2025). https://doi.org/10.1002/aidi.202500085

    work page doi:10.1002/aidi.202500085 2025

  4. [4]

    Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems 30 (2017)

    work page 2017

  5. [5]

    Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings Bioinf. 23 (2022).https://doi.org/10.1093/bib/bbac409

    work page doi:10.1093/bib/bbac409 2022

  6. [6]

    Zhang, D. et al. ChemLLM: A chemical large language model. arXiv preprint arXiv:2402.06852 (2024)

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Qiu, P. et al. Towards building multilingual language model for medicine. Nat. Commun. 15, 8384 (2024).https://doi.org/10.1038/s41467-024-52417-z

    work page doi:10.1038/s41467-024-52417-z 2024

  8. [8]

    M., Schwaller, P., Ortega-Guerrero, A

    Jablonka, K. M., Schwaller, P., Ortega-Guerrero, A. & Smit, B. Leveraging large language models for predictive chemistry. Nat. Mach. Intell. 6, 161–169 (2024).https://doi.org/ 10.1038/s42256-023-00788-1

    work page doi:10.1038/s42256-023-00788-1 2024

  9. [9]

    Li, Y. et al. Large language model-based biological age prediction in large-scale populations. Nat. Med. (2025).https://doi.org/10.1038/s41591-025-03856-8

    work page doi:10.1038/s41591-025-03856-8 2025

  10. [10]

    Role play with large language models

    Shanahan, M., McDonell, K. & Reynolds, L. Role play with large language models. Nature 623, 493–498 (2023).https://doi.org/10.1038/s41586-023-06647-8

    work page doi:10.1038/s41586-023-06647-8 2023

  11. [11]

    Hu, E. J. et al. Lora: Low-rank adaptation of large language models. ICLR 1, 3 (2022)

    work page 2022

  12. [12]

    Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023). 16

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  13. [13]

    Ouyang, L. et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35, 27730–27744 (2022)

    work page 2022

  14. [14]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

    Sanh, V., Debut, L., Chaumond, J. & Wolf, T. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  15. [15]

    Tu, T. et al. Towards conversational diagnostic artificial intelligence. Nature 642, 442–450 (2025).https://doi.org/10.1038/s41586-025-08866-7

    work page doi:10.1038/s41586-025-08866-7 2025

  16. [16]

    M., Gebru, T., McMillan-Major, A

    Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. in Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. 610–623

    work page 2021

  17. [17]

    Dziri, N. et al. Faith and fate: Limits of transformers on compositionality (2023). arXiv preprint arXiv:2305.18654 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  18. [18]

    Huang, L. et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Trans. Inf. Syst. 43, 1–55 (2025)

    work page 2025

  19. [19]

    Can Large Language Models Be an Alternative to Human Evaluations?

    Chiang, C.-H. & Lee, H.-y. Can large language models be an alternative to human evalua- tions? arXiv preprint arXiv:2305.01937 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  20. [20]

    How Language Model Hallucinations Can Snowball

    Zhang, M., Press, O., Merrill, W., Liu, A. & Smith, N. A. How language model hallucinations can snowball. arXiv preprint arXiv:2305.13534 (2023)

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  21. [21]

    TruthfulQA: Measuring How Models Mimic Human Falsehoods

    Lin, S., Hilton, J. & Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 (2021)

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  22. [22]

    Wei, J., Kim, N., Tay, Y. & Le, Q. V. Inverse scaling can become u-shaped. arXiv preprint arXiv:2211.02011 (2022)

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  23. [23]

    & Kim, J

    Kang, Y. & Kim, J. ChatMOF: an artificial intelligence system for predicting and generating metal-organic frameworks using large language models. Nat. Commun. 15, 4705 (2024). https://doi.org/10.1038/s41467-024-48998-4

    work page doi:10.1038/s41467-024-48998-4 2024

  24. [24]

    Pan, Y. et al. Chat-microreactor: A large-language-model-based assistant for designing continuous flow systems. Chem. Eng. Sci. 311, 121567 (2025).https://doi.org/10.1016/ j.ces.2025.121567

    work page arXiv 2025

  25. [25]

    Ruan, Y. et al. An automatic end-to-end chemical synthesis development platform powered by large language models. Nat. Commun. 15, 10160 (2024).https://doi.org/10.1038/ s41467-024-54457-x

    work page 2024

  26. [26]

    Wang, J. et al. Discovery of antimicrobial peptides with notable antibacterial potency by an LLM-based foundation model. Sci. Adv. 11, eads8932 (2025).https://doi.org/10.1126/ sciadv.ads8932

    work page 2025

  27. [27]

    R., Bruun, T

    Shanker, V. R., Bruun, T. U. J., Hie, B. L. & Kim, P. S. Unsupervised evolution of protein and antibody complexes with a structure-informed language model. Science 385, 46–53 (2024).https://doi.org/10.1126/science.adk8946

    work page doi:10.1126/science.adk8946 2024

  28. [28]

    & Eichler, A

    Kaiser, J., Lauscher, A. & Eichler, A. Large language models for human-machine collabora- tive particle accelerator tuning through natural language. Sci. Adv. 11, eadr4173 (2025). https://doi.org/10.1126/sciadv.adr4173

    work page doi:10.1126/sciadv.adr4173 2025

  29. [29]

    Liu, N. et al. Harnessing large language models for data-scarce learning of polymer properties. Nat. Comput. Sci. 5, 245–254 (2025).https://doi.org/10.1038/s43588-025-00768-y 17

    work page doi:10.1038/s43588-025-00768-y 2025

  30. [30]

    Zhang, Y. et al. Large language models to accelerate organic chemistry synthesis. Nat. Mach. Intell. 7, 1010–1022 (2025).https://doi.org/10.1038/s42256-025-01066-y

    work page doi:10.1038/s42256-025-01066-y 2025

  31. [31]

    Xiong, J. et al. Bridging chemistry and artificial intelligence by a reaction de- scription language. Nat. Mach. Intell. 7, 782–793 (2025).https://doi.org/10.1038/ s42256-025-01032-8

    work page 2025

  32. [32]

    Zheng, Y. et al. Large language models for scientific discovery in molecular prop- erty prediction. Nat. Mach. Intell. 7, 437–447 (2025). https://doi.org/10.1038/ s42256-025-00994-z

    work page 2025

  33. [33]

    & Ren, Z

    Liu, P., Tao, J. & Ren, Z. A quantitative analysis of knowledge-learning preferences in large language models in molecular science. Nat. Mach. Intell. 7, 315–327 (2025).https: //doi.org/10.1038/s42256-024-00977-6

    work page doi:10.1038/s42256-024-00977-6 2025

  34. [34]

    M. Bran, A. et al. Augmenting large language models with chemistry tools. Nat. Mach. Intell. 6, 525–535 (2024).https://doi.org/10.1038/s42256-024-00832-8

    work page doi:10.1038/s42256-024-00832-8 2024

  35. [35]

    A., MacKnight, R., Kline, B

    Boiko, D. A., MacKnight, R., Kline, B. & Gomes, G. Autonomous chemical research with large language models. Nature 624, 570–578 (2023).https://doi.org/10.1038/ s41586-023-06792-0

    work page 2023

  36. [36]

    LLM as a Judge

    Mirza, A. et al. A framework for evaluating the chemical knowledge and reasoning abilities of large language models against the expertise of chemists. Nat. Chem. 17, 1027–1034 (2025). https://doi.org/10.1038/s41557-025-01815-x 18 A Supplementary Information This appendix contains the supplementary evidence needed to support the arXiv version of the manusc...

    work page doi:10.1038/s41557-025-01815-x 2025