Pith. sign in

REVIEW 4 major objections 5 minor 144 references

Tool access cuts prediction error on novel materials from ~1.5 eV to under 0.005 eV, the authors report.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 16:53 UTC pith:ADNWDB63

load-bearing objection The abstract claims a headline result the body never reports; the real contribution is the honest Figure 9 comparison and the open platform. the 4 major comments →

arxiv 2512.11935 v2 pith:ADNWDB63 submitted 2025-12-12 cs.AI cond-mat.mtrl-sci

AGAPI-Agents: An Open-Access Agentic AI Platform for Accelerated Materials Design on AtomGPT.org

Jaehyung Lee , Justin Ely , Kent Zhang , Akshaya Ajith , Charles Rhys Campbell , Kamal Choudhary This is my paper
classification cs.AI cond-mat.mtrl-sci
keywords agentic AImaterials designlarge language modelstool useJARVIS-DFTdefect engineeringopen-source LLMsREST API
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AGAPI-Agents is an open-access platform that pairs eight open-source large language models with 18 materials-science API endpoints, letting an agent plan, execute, and summarize multi-step design workflows. The paper's central claim is that tool access is essential when materials are not memorized by the LLM or the ML models: on test sets of 57 defective supercells and 60 hypothetical compositions, the tool-augmented agent reaches a mean absolute error below 0.005 eV, while the tool-free agent fails with errors above 1.25 eV. On well-known benchmark materials, however, tool augmentation helps only for bulk modulus and actually hurts for bandgap, superconducting Tc, SLME, and dielectric constant. A three-way residual decomposition shows the agent pipeline reproduces the underlying JARVIS-DFT values to numerical precision, meaning the degradation on those benchmarks is inherited from the database's functional approximations rather than from agentic malfunction. The paper argues for adaptive tool selection and presents working 10-step autonomous workflows as evidence that open-source agents can democratize and reproducibility.

Core claim

The paper establishes that an agentic AI system built entirely on open-source components can execute autonomous multi-step materials workflows, and it quantifies when tool access matters. The load-bearing evidence is a set of 'memorization-resistant' structures—57 defective supercells and 60 hypothetical compositions—that presumably cannot be recalled from parametric knowledge. On these, the tool-augmented agent achieves an MAE below 0.005 eV, whereas the tool-free agent's MAE is 1.25 to 1.86 eV. Alongside this, a three-evaluation residual decomposition on standard electronic-structure benchmarks demonstrates that the agent pipeline reproduces JARVIS-DFT entries to numerical precision, isola

What carries the argument

The central object is the AGAPI Agent-Planner-Executor-Summarizer architecture, which delegates reasoning to an LLM and execution to a REST API layer of materials tools (ALIGNN for property prediction, ALIGNN-FF for relaxation, SlaKoNet for tight-binding band structure, XRD simulation, supercell and defect generators, and database queries). The memorization-resistant test sets are the critical mechanism for isolating true tool value: if those structures are genuinely novel to both the LLM and the ML models, then the <0.005 eV vs 1.25+ eV comparison isolates the contribution of tool access from parametric recall. The three-way residual decomposition is the second key mechanism, separating pip

Load-bearing premise

The 57 defective supercells and 60 hypothetical compositions are genuinely unseen by both the LLM and the ML models; if any of them closely resembles training data, the <0.005 eV result is not evidence for tool necessity.

What would settle it

Release the 57 supercells and 60 hypothetical compositions, and check for near-duplicates in the JARVIS-DFT/ALIGNN training data and in the LLM pretraining corpus. Independently, reproduce the tool-free agent run with temperature 0 and no retrieval on those exact structures; if the error drops well below 1.25 eV or if any structure appears in the training sets, the central claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the <0.005 eV result is robust, tool access becomes a prerequisite for reliable generative and inverse materials design, where novel structures are common.
  • The observed degradation on well-known benchmarks shows that future agents should gate tool use by database reliability and property-specific confidence, not always retriieve.
  • Open-source LLMs plus pinned model versions offer a reproducible alternative to commercial APIs for scientific agent workflows.
  • Autonomous 10-step operations (search, supercell, substitution, relaxation, XRD, property prediction, band structure) suggest that complex materials pipelines can run without manual stitching.
  • The separation of reasoning from execution means new tools and new LLMs can be swapped independently, making the platform extensible beyond the current tool set.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The entire <0.005 eV claim rests on the memorization-resistant sets being genuinely novel; the paper gives no construction details, no examples, and no deduplication analysis, so the claim is unverified until the structures are released and checked against training corpora.
  • The same benchmark setup could be extended to test whether a stronger open-source model narrows the tool-free error gap, or whether the gap is invariant to LLM scale.
  • A natural next step is an adaptive tool-gating policy that predicts which property a tool will help, using the pattern in Figure 9; this would directly address the four degraded properties.
  • The architecture is domain-agnostic: the same planner-executor pattern with REST endpoints could be ported to chemistry, biology, or metadata-driven materials discovery.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces AGAPI, an open-access agentic AI platform that couples multiple open-source LLMs with REST endpoints for materials databases, ML property predictors, force fields, tight-binding band structures, XRD simulation, and protein tools. It describes an Agent-Planner-Executor-Summarizer architecture, demonstrates single- and multi-step workflows (e.g., defect engineering, heterostructure construction, XRD analysis), and reports benchmark results. The abstract makes two central quantitative claims: (i) a three-evaluation residual decomposition separates agent pipeline fidelity from DFT functional bias on JARVIS-Leaderboard electronic-structure test sets, with bulk modulus and bandgap reproduced to numerical precision; and (ii) on "memorization-resistant" test sets (57 defective supercells, 60 hypothetical compositions), tool-augmented MAE is below 0.005 eV versus 1.25–1.86 eV tool-free, confirming tools are indispensable for novel materials. The main-text evaluation in Section 2.6 (Figure 9) instead reports property-dependent effects on five JARVIS-Leaderboard properties, with tool access improving only bulk modulus and degrading bandgap, Tc, SLME, and dielectric constant.

Significance. If fully supported, the platform would be a valuable open-source contribution: it offers reproducible version-pinned LLM inference, a broad set of materials-science tools behind a public API, and honest evidence that tool access is not universally beneficial. The workflow demonstrations (e.g., the 10-step defect pipeline and GaN/AlN interface construction) are concrete and plausible. However, the paper's headline claim — that tool access reduces MAE to below 0.005 eV on materials unseen by parametric models — is stated only in the abstract and is not supported anywhere in the body or supplement. That claim, if true, would be a strong and falsifiable result; as written, it is unverifiable and sits in tension with Figure 9, which shows tool access often hurting performance. The missing evaluation is load-bearing for the paper's central message, not a presentation detail.

major comments (4)
  1. [Abstract (third paragraph) vs. Sections 2–4] The abstract's central quantitative claim — "On memorization-resistant test sets (57 defective supercells, 60 hypothetical compositions), tool-augmented MAE is below 0.005 eV versus 1.25 to 1.86 eV tool-free" — is never described in the main text or supplementary information. There is no construction protocol, no example structure or composition, no reference values, no deduplication analysis against JARVIS-DFT/ALIGNN training data or LLM pretraining corpora, and no statement of what the "tool-augmented" prediction actually returns (database retrieval vs. ML prediction on a modified structure). The reader cannot verify whether the test sets exist, whether they are novel, or whether the reported MAE is computed from direct label lookup. This omission removes the evidence for the paper's strongest claim.
  2. [Section 2.6, Figure 9] The only systematic evaluation present in the manuscript shows that tool access improves bulk modulus (MAE 7.876 → 5.732 GPa) but degrades bandgap, superconducting Tc, SLME, and dielectric constant, with Tc MAE increasing more than fivefold. No sample sizes, error bars, or per-property test-set sizes are given, and the mechanism of "with tools" is undefined — it could mean retrieving JARVIS-DFT entries for the same test structures, which would make agreement with JARVIS-DFT definitional rather than an empirical finding. Because the abstract's <0.005 eV result is absent from this section, the body's own evidence does not support the conclusion that tools are indispensable on novel materials; it supports the opposite for well-known properties.
  3. [Abstract, first paragraph; no Methods section] The "three-evaluation residual decomposition" mentioned in the abstract is not presented in Section 4 (Methods) or in Results. No equations, protocol, or quantitative decomposition are given to separate "agent pipeline fidelity" from "inherited DFT functional bias." This decomposition is needed to interpret the claim that reproducing JARVIS-DFT entries to numerical precision means experimental-reference degradation is functional bias. Without it, the paper's explanatory framework is an assertion rather than a demonstrated result.
  4. [General reproducibility] No dataset, code, or scripts are provided for the memorization-resistant evaluation or for the Figure 9 metrics. The GitHub/Colab links in the Data Availability section point to the platform and example notebooks, not to the evaluation harness, test-set construction, or raw outputs. A sub-0.005 eV MAE is orders of magnitude below ALIGNN's typical band-gap errors on JARVIS-DFT, so unless the tool directly retrieves ground-truth labels from a database containing the test structures, the number is physically implausible. The lack of artefact sharing makes the headline claim unreproducible as well as unsupported.
minor comments (5)
  1. [Section 2.6] Typo "comapred" for "compared" in the opening sentence.
  2. [Section 3 (Discussion)] Typo "temeprature" in the limitations paragraph; also the sentence "If the datasets contain wrong results, such as underestimated bandgaps, the agentic AI itself can be low to no usage" is unclear and should be rewritten.
  3. [Throughout] Inconsistent rendering of "JARVIS-DFT" (some places "JAR VIS-DFT" or "JAR VIS"), and the reference list contains a likely typo "CrsystaLLM" [26] for "CrystaLLM."
  4. [Supplementary Information, ag31] The print statement in the ALIGNN-FF example outputs `print(ag24)` though the variable is defined as `ag31`; this appears to be a copy-paste error and would not display the intended result.
  5. [Figure 9] Panels (d) and (e) plot only a narrow experimental range (SLME 18–34%, dielectric 6–20), and the dielectric constant axis is labeled 'Exp. x' rather than a physical property name. This reduces readability and should be fixed.

Circularity Check

0 steps flagged

No demonstrated circularity: the body's central evaluation is a residual decomposition against external JARVIS-Leaderboard references, and the abstract-only memorization-resistant numbers are unsupported but not shown to be definitional.

full rationale

The paper does not contain a derivation chain in which a fitted input is renamed as a prediction, nor a load-bearing self-citation that substitutes for evidence. The only quasi-quantitative evaluation in the body (Section 2.6, Fig. 9) compares tool-augmented and tool-free agent predictions against experimental references on the JARVIS-Leaderboard ES test sets. The statement that 'the agent reproduces JARVIS-DFT entries to numerical precision' is a pipeline-fidelity check: if the tool is the JARVIS-DFT database query endpoint, reproducing the database entry is expected, and the paper explicitly uses this only to attribute experimental-reference error to DFT functional bias rather than to claim a novel prediction. That decomposition is logically coherent and is benchmarked against external experimental values, not against the paper's own fitted outputs. The self-citations to JARVIS-DFT, ALIGNN, SlaKoNet, and AtomGPT are normal use of prior artifacts, and no uniqueness theorem or ansatz is imported from the authors' prior work to force the central conclusion. The abstract's claim about 'memorization-resistant test sets (57 defective supercells, 60 hypothetical compositions)' with MAE below 0.005 eV is never described in the body or supplement, so it is currently unverifiable; this is a correctness/reproducibility failure, not a demonstrated circularity. Without construction details, one cannot exhibit a specific equation or step showing that the tool-augmented MAE equals its own input by construction. Per the hard rules, absence of method is not equivalent to circularity, and speculation about whether the tool retrieved the reference labels is not permitted. No circular step can be quoted, so the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper introduces no new free parameters or invented physical entities. It relies on four domain assumptions, the most fragile being the existence and novelty of the memorization-resistant test sets, which appear only in the abstract. The evaluation depends heavily on the authors' own JARVIS/ALIGNN ecosystem, so a reader must accept those tools as reference-grade to interpret the with/without-tool comparison.

axioms (4)
  • domain assumption JARVIS-DFT/ALIGNN outputs are treated as valid references for evaluating agent predictions.
    Section 2.6 uses JARVIS-Leaderboard ES test sets and ALIGNN as the tool layer; no independent DFT or experimental validation is reported for retrieved values.
  • ad hoc to paper Memorization-resistant test sets (57 defective supercells, 60 hypothetical compositions) are not in LLM or ALIGNN training data.
    Only stated in the abstract; no construction or deduplication details are provided, so this assumption is neither evidenced nor checkable.
  • domain assumption Tool-free LLM answers provide a valid baseline of parametric knowledge under identical instructions.
    The comparison in Fig 9 assumes disabling tools only removes retrieval, but prompt phrasing, temperature, and model determinism are not fully controlled across conditions.
  • domain assumption Experimental reference values in Fig 9 are correct and directly comparable.
    Parity plots use experimental values from JARVIS-Leaderboard without listing per-point sources or uncertainty.

pith-pipeline@v1.3.0-alltime-deepseek · 36913 in / 9926 out tokens · 87949 ms · 2026-08-03T16:53:04.113564+00:00 · methodology

0 comments
read the original abstract

Agentic AI systems increasingly connect large language models (LLMs) to external scientific tools, yet whether and when tool access improves prediction accuracy remains uncharacterized. We present AGAPI (AtomGPT.org API), an open access platform integrating eight open-source LLMs with 18 REST endpoints (28 agent tools, 50 web apps) spanning materials databases, force fields, tight-binding band structures, X-ray diffraction, and protein structure. A three-evaluation residual decomposition on JARVIS-Leaderboard electronic-structure test sets separates agent pipeline fidelity from inherited density functional theory (DFT) functional bias. For bulk modulus and bandgap the agent reproduces JARVIS-DFT entries to numerical precision, so the experimental-reference degradation is functional bias, not agentic malfunction. On memorization-resistant test sets (57 defective supercells, 60 hypothetical compositions), tool-augmented mean absolute error (MAE) is below 0.005 eV versus 1.25 to 1.86 eV tool-free, confirming tools are indispensable where parametric knowledge is unavailable. We further demonstrate autonomous multi-step workflows including 10-operation defect-engineering pipelines. AGAPI is available at https://github.com/atomgptlab/agapi.

Figures

Figures reproduced from arXiv: 2512.11935 by Akshaya Ajith, Charles Rhys Campbell, Jaehyung Lee, Justin Ely, Kamal Choudhary, Kent Zhang.

Figure 1
Figure 1. Figure 1: Overview of the AGAPI-Agents workflow illustrating its architecture for au￾tonomous materials design. The system integrates a large language model (LLM) as the central brain that interprets user intent, selects appropriate tools, and orchestrates multi-step scientific work￾flows. Through a unified interface, the LLM can call diverse backend models, access materials databases, and interact with domain-speci… view at source ↗
Figure 2
Figure 2. Figure 2: AtomGPT.org chatbot with tool calling enabled. Without tool calling, chatbots can hallucinate materials science information. With tools enabled, the LLM grounds all responses in actual API calls to databases and computational models, significantly reducing hallucination through retrieval￾augmented generation. Benchmarking focused on tokens-per-second (TPS) generation speed, a critical metric for interactiv… view at source ↗
Figure 3
Figure 3. Figure 3: Comprehensive benchmarking of token generation speeds and load testing for LLMs integrated within AGAPI. (a) Model performance comparison showing mean token generation speeds (tokens/s) measured over repeated test runs, using llama-3.2-90b-vision-instruct as baseline. GPT-OSS models (20B and 120B) achieve 3.93× and 3.39× speedup respectively. (b) Response time distribution under simulated load of 1000 conc… view at source ↗
Figure 4
Figure 4. Figure 4: Overview of AGAPI’s core functional modules and capabilities. The framework inte￾grates multiple tools and databases to enable autonomous materials discovery and analysis including: (1) JARVIS database access for materials data retrieval; (2) Property prediction using ALIGNN for formation energy, bandgap, elastic constants, and other properties; (3) Force field predictions via ALIGNN-FF for structure relax… view at source ↗
Figure 5
Figure 5. Figure 5: Screenshot of the AGAPI (AtomGPT.org API) interactive documentation. Gener￾ated automatically from the OpenAPI 3.1 specification, the interface provides a unified, REST-compatible API layer for multi-agent scientific workflows. Each expandable block represents an available endpoint, with GET routes supporting lightweight information retrieval and metadata queries, and POST routes enabling full inference, c… view at source ↗
Figure 6
Figure 6. Figure 6: Example of a real-time API call to the AtomGPT.org AGAPI endpoint for the ALIGNN machine-learning model. The request URL queries the /alignn/query endpoint with the material identifier JVASP-1151, returning a structured JSON response containing multiple pre￾dicted physical properties. The POSCAR visualization (left) illustrates the atomic structure supplied to ALIGNN, showing the geometric environment used… view at source ↗
Figure 7
Figure 7. Figure 7: Example database search workflow. Natural language query for materials containing specific elements is translated to API call returning structured results with JARVIS-IDs, formulas, space groups, formation energies, bulk moduli, and bandgaps. The agent automatically formats results in a human-readable table while preserving all metadata for downstream analysis. For property-based screening, queries request… view at source ↗
Figure 8
Figure 8. Figure 8: Example of comprehensive semiconductor defect analysis workflow. The workflow executes 10 sequential operations: database search, structure retrieval, supercell construction, atomic substitution, structure optimization, XRD pattern generation, property predictions, band structure cal￾culation, and result synthesis following the prompt in schematic [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Parity plots comparing materials property predictions with (green squares) and without (orange [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

144 extracted references · 21 linked inside Pith

  1. [1]

    Scientific discovery in the age of artificial intelligence.Nature, 620(7972):47–60, 2023

    Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al. Scientific discovery in the age of artificial intelligence.Nature, 620(7972):47–60, 2023

    2023

  2. [2]

    Recent advances and applications of deep learning methods in materials science.npj Computational Materials, 8(1):59, 2022

    Kamal Choudhary, Brian DeCost, Chi Chen, Anubhav Jain, Francesca Tavazza, Ryan Cohn, Cheol Woo Park, Alok Choudhary, Ankit Agrawal, Simon JL Billinge, et al. Recent advances and applications of deep learning methods in materials science.npj Computational Materials, 8(1):59, 2022

    2022

  3. [3]

    The fair guiding principles for scientific data management and stewardship.Scientific data, 3(1):1–9, 2016

    Mark D Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E Bourne, et al. The fair guiding principles for scientific data management and stewardship.Scientific data, 3(1):1–9, 2016. 13

    2016

  4. [4]

    Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities.Information Fusion, 50:71–91, 2019

    Marinka Zitnik, Francis Nguyen, Bo Wang, Jure Leskovec, Anna Goldenberg, and Michael M Hoff- man. Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities.Information Fusion, 50:71–91, 2019

    2019

  5. [5]

    Chatgpt for good? on opportunities and challenges of large language models for education.Learning and individual differences, 103:102274, 2023

    Enkelejda Kasneci, Kathrin Seßler, Stefan K¨ uchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan G¨ unnemann, Eyke H¨ ullermeier, et al. Chatgpt for good? on opportunities and challenges of large language models for education.Learning and individual differences, 103:102274, 2023

    2023

  6. [6]

    A comprehensive overview of large language models

    Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. ACM Transactions on Intelligent Systems and Technology, 16(5):1–72, 2025

    2025

  7. [7]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon.Digital discovery, 2(5):1233–1250, 2023

    Kevin Maik Jablonka, Qianxiang Ai, Alexander Al-Feghali, Shruti Badhwar, Joshua D Bocarsly, Andres M Bran, Stefan Bringuier, L Catherine Brinson, Kamal Choudhary, Defne Circi, et al. 14 examples of how llms can transform materials science and chemistry: a reflection on a large language model hackathon.Digital discovery, 2(5):1233–1250, 2023

    2023

  9. [9]

    Leveraging large language models for predictive chemistry.Nature Machine Intelligence, 6(2):161–169, 2024

    Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, and Berend Smit. Leveraging large language models for predictive chemistry.Nature Machine Intelligence, 6(2):161–169, 2024

    2024

  10. [10]

    Chemnlp: a natural language-processing-based library for materials chemistry text data.The Journal of Physical Chemistry C, 127(35):17545–17555, 2023

    Kamal Choudhary and Mathew L Kelley. Chemnlp: a natural language-processing-based library for materials chemistry text data.The Journal of Physical Chemistry C, 127(35):17545–17555, 2023

    2023

  11. [11]

    Kamal Choudhary. Chatgpt material explorer: Design and implementation of a custom gpt assistant for materials science applications.Integrating Materials and Manufacturing Innovation, 14(3):276– 283, 2025

    2025

  12. [12]

    Kamal Choudhary. Diffractgpt: Atomic structure determination from x-ray diffraction patterns using a generative pretrained transformer.The Journal of Physical Chemistry Letters, 16(8):2110–2119, 2025

    2025

  13. [13]

    Atomgpt: Atomistic generative pretrained transformer for forward and inverse materials design.The Journal of Physical Chemistry Letters, 15(27):6909–6917, 2024

    Kamal Choudhary. Atomgpt: Atomistic generative pretrained transformer for forward and inverse materials design.The Journal of Physical Chemistry Letters, 15(27):6909–6917, 2024

    2024

  14. [14]

    Kamal Choudhary. Microscopygpt: Generating atomic-structure captions from microscopy images of 2d materials with vision-language transformers.The Journal of Physical Chemistry Letters, 16:7028– 7035, 2025

    2025

  15. [15]

    Crystal structure generation with autoregressive large language modeling.Nature Communications, 15(1):10570, 2024

    Luis M Antunes, Keith T Butler, and Ricardo Grau-Crespo. Crystal structure generation with autoregressive large language modeling.Nature Communications, 15(1):10570, 2024

    2024

  16. [16]

    Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy.Nature, 630(8017):625–630, 2024

    2024

  17. [17]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025

    2025

  18. [18]

    Towards mitigating hallucination in large language models via self-reflection.arXiv preprint arXiv:2310.06271, 2023

    Ziwei Ji, Tiezheng Yu, Yan Xu, Nayeon Lee, Etsuko Ishii, and Pascale Fung. Towards mitigating hallucination in large language models via self-reflection.arXiv preprint arXiv:2310.06271, 2023

    Pith/arXiv arXiv 2023

  19. [19]

    Towards understanding sycophancy in language models.arXiv preprint arXiv:2310.13548, 2023

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. Towards understanding sycophancy in language models.arXiv preprint arXiv:2310.13548, 2023

    Pith/arXiv arXiv 2023

  20. [20]

    Holistic evaluation of language models.arXiv preprint arXiv:2211.09110, 2022

    Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models.arXiv preprint arXiv:2211.09110, 2022. 14

    Pith/arXiv arXiv 2022

  21. [21]

    Galactica: A large language model for science

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Language models can generate molecules, materials, and protein binding sites directly in three dimensions as xyz, cif, and pdb files.arXiv preprint arXiv:2305.05708, 2023

    Daniel Flam-Shepherd and Al´ an Aspuru-Guzik. Language models can generate molecules, materials, and protein binding sites directly in three dimensions as xyz, cif, and pdb files.arXiv preprint arXiv:2305.05708, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    Chemcrow: Augmenting large-language models with chemistry tools.arXiv preprint arXiv:2304.05376, 2023

    Andres M Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller. Chemcrow: Augmenting large-language models with chemistry tools.arXiv preprint arXiv:2304.05376, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models.Nature, 624(7992):570–578, 2023

    2023

  25. [25]

    Llm-prop: Pre- dicting physical and electronic properties of crystalline solids from their text descriptions.arXiv preprint arXiv:2310.14029, 2023

    Andre Niyongabo Rubungo, Craig Arnold, Barry P Rand, and Adji Bousso Dieng. Llm-prop: Pre- dicting physical and electronic properties of crystalline solids from their text descriptions.arXiv preprint arXiv:2310.14029, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    Fine-tuned language models generate stable inorganic materials as text.arXiv preprint arXiv:2402.04379, 2024

    Nate Gruver, Anuroop Sriram, Andrea Madotto, Andrew Gordon Wilson, C Lawrence Zitnick, and Zachary Ulissi. Fine-tuned language models generate stable inorganic materials as text.arXiv preprint arXiv:2402.04379, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    From ai for science to agentic science: A survey on autonomous scientific discovery.arXiv preprint arXiv:2508.14111, 2025

    Jiaqi Wei, Yuejin Yang, Xiang Zhang, Yuhan Chen, Xiang Zhuang, Zhangyang Gao, Dongzhan Zhou, Guangshuai Wang, Zhiqiang Gao, Juntai Cao, et al. From ai for science to agentic science: A survey on autonomous scientific discovery.arXiv preprint arXiv:2508.14111, 2025

    arXiv 2025

  28. [28]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems, 36:68539–68551, 2023

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess ` ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools.Advances in Neural Information Processing Systems, 36:68539–68551, 2023

    2023

  30. [30]

    John Wiley & Sons, 2005

    Lin Padgham and Michael Winikoff.Developing intelligent agent systems: A practical guide. John Wiley & Sons, 2005

    2005

  31. [31]

    Atomagents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence.arXiv preprint arXiv:2407.10022, 2024

    Alireza Ghafarollahi and Markus J Buehler. Atomagents: Alloy design and discovery through physics-aware multi-modal multi-agent artificial intelligence.arXiv preprint arXiv:2407.10022, 2024

    Pith/arXiv arXiv 2024

  32. [32]

    Au- tonomous universal research assistant (aura): Agentic ai meets nanohub’s fair workflows and data

    Juan Carlos Verduzco Gastelum, Mohnish Harwani, Daniel Mejia, and Alejandro Strachan. Au- tonomous universal research assistant (aura): Agentic ai meets nanohub’s fair workflows and data. 2025

    2025

  33. [33]

    Llamp: Large language model made powerful for high-fidelity materials knowledge retrieval and distillation.arXiv preprint arXiv:2401.17244, 2024

    Yuan Chiang, Elvis Hsieh, Chia-Hong Chou, and Janosh Riebesell. Llamp: Large language model made powerful for high-fidelity materials knowledge retrieval and distillation.arXiv preprint arXiv:2401.17244, 2024

    Pith/arXiv arXiv 2024

  34. [34]

    Scitoolagent: a knowledge-graph-driven scientific agent for multitool integration.Nature Computational Science, pages 1–11, 2025

    Keyan Ding, Jing Yu, Junjie Huang, Yuchen Yang, Qiang Zhang, and Huajun Chen. Scitoolagent: a knowledge-graph-driven scientific agent for multitool integration.Nature Computational Science, pages 1–11, 2025

    2025

  35. [35]

    Towards agentic science for advancing scientific discovery.Nature Machine Intelligence, pages 1–3, 2025

    Hongliang Xin, John R Kitchin, and Heather J Kulik. Towards agentic science for advancing scientific discovery.Nature Machine Intelligence, pages 1–3, 2025

    2025

  36. [36]

    Pricing4apis: A rigorous model for restful api pricings.Computer Standards & Interfaces, 91:103878, 2025

    Rafael Fresno-Aranda, Pablo Fernandez, Antonio Gamez-Diaz, Amador Duran, and Antonio Ruiz- Cortes. Pricing4apis: A rigorous model for restful api pricings.Computer Standards & Interfaces, 91:103878, 2025

    2025

  37. [37]

    Cost transparency of enterprise ai adoption

    Soogand Alavi, Salar Nozari, and Andrea Luangrath. Cost transparency of enterprise ai adoption. arXiv preprint arXiv:2511.11761, 2025

    arXiv 2025

  38. [38]

    ” O’Reilly Media, Inc.”, 2011

    Mark Masse.REST API design rulebook: designing consistent RESTful web service interfaces. ” O’Reilly Media, Inc.”, 2011. 15

    2011

  39. [39]

    The joint automated repository for various integrated simulations (jarvis) for data-driven materials design

    Kamal Choudhary, Kevin F Garrity, Andrew CE Reid, Brian DeCost, Adam J Biacchi, Angela R Hight Walker, Zachary Trautt, Jason Hattrick-Simpers, A Gilad Kusne, Andrea Centrone, et al. The joint automated repository for various integrated simulations (jarvis) for data-driven materials design. npj computational materials 6. 2020

    2020

  40. [40]

    Recent progress in the jarvis infrastructure for next-generation data-driven materials design.Applied Physics Reviews, 10(4), 2023

    Daniel Wines, Ramya Gurunathan, Kevin F Garrity, Brian DeCost, Adam J Biacchi, Francesca Tavazza, and Kamal Choudhary. Recent progress in the jarvis infrastructure for next-generation data-driven materials design.Applied Physics Reviews, 10(4), 2023

    2023

  41. [41]

    The jarvis infrastructure is all you need for materials design.arXiv preprint arXiv:2503.04133, 2025

    Kamal Choudhary. The jarvis infrastructure is all you need for materials design.arXiv preprint arXiv:2503.04133, 2025

    Pith/arXiv arXiv 2025

  42. [42]

    Atomistic line graph neural network for improved materials property predictions.npj Computational Materials, 7(1):185, 2021

    Kamal Choudhary and Brian DeCost. Atomistic line graph neural network for improved materials property predictions.npj Computational Materials, 7(1):185, 2021

    2021

  43. [43]

    Unified graph neural network force-field for the periodic table: solid state applications

    Kamal Choudhary, Brian DeCost, Lily Major, Keith Butler, Jeyan Thiyagalingam, and Francesca Tavazza. Unified graph neural network force-field for the periodic table: solid state applications. Digital Discovery, 2(2):346–355, 2023

    2023

  44. [44]

    Kamal Choudhary. Slakonet: A unified slater-koster tight-binding framework using neural network infrastructure for the periodic table.The Journal of Physical Chemistry Letters, 16(43):11109–11119, 2025

    2025

  45. [45]

    openai-agents-python: A lightweight, powerful framework for multi-agent workflows

    OpenAI. openai-agents-python: A lightweight, powerful framework for multi-agent workflows. https://github.com/openai/openai-agents-python, 2025. Accessed: 2025-12-11

    2025

  46. [46]

    Agentic AI for Science (AAI4Science) Hackathon 2025.https://github.com/ atomgptlab/aai4science, 2025

    AtomGPTLab. Agentic AI for Science (AAI4Science) Hackathon 2025.https://github.com/ atomgptlab/aai4science, 2025. GitHub repository

    2025

  47. [47]

    A survey on large language model based autonomous agents

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18(6):186345, 2024

    2024

  48. [48]

    Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey.IEEe Access, 2025

    Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B Divya. Agentic ai: Autonomous intelligence for complex goals–a comprehensive survey.IEEe Access, 2025

    2025

  49. [49]

    Commentary: The materials project: A materials genome approach to accelerating materials innovation.APL materials, 1(1), 2013

    Anubhav Jain, Shyue Ping Ong, Geoffroy Hautier, Wei Chen, William Davidson Richards, Stephen Dacek, Shreyas Cholia, Dan Gunter, David Skinner, Gerbrand Ceder, et al. Commentary: The materials project: A materials genome approach to accelerating materials innovation.APL materials, 1(1), 2013

    2013

  50. [50]

    Aflow: An automatic framework for high-throughput materials discovery.Computational Materials Science, 58:218–226, 2012

    Stefano Curtarolo, Wahyu Setyawan, Gus L W Hart, Michal Jahnatek, Roman V Chepulskii, Richard H Taylor, Shidong Wang, Junkai Xue, Kesong Yang, Ohad Levy, et al. Aflow: An automatic framework for high-throughput materials discovery.Computational Materials Science, 58:218–226, 2012

    2012

  51. [51]

    The open quantum materials database (oqmd): assessing the accuracy of dft formation energies.npj Computational Materials, 1(1):1–15, 2015

    Scott Kirklin, James E Saal, Bryce Meredig, Alex Thompson, Jeff W Doak, Muratahan Aykol, Stephan R¨ uhl, and Chris Wolverton. The open quantum materials database (oqmd): assessing the accuracy of dft formation energies.npj Computational Materials, 1(1):1–15, 2015

    2015

  52. [52]

    Optimade, an api for exchanging materials data.Scientific data, 8(1):217, 2021

    Casper W Andersen, Rickard Armiento, Evgeny Blokhin, Gareth J Conduit, Shyam Dwaraknath, Matthew L Evans, ´Ad´ am Fekete, Abhijith Gopakumar, Saulius Graˇ zulis, Andrius Merkys, et al. Optimade, an api for exchanging materials data.Scientific data, 8(1):217, 2021

    2021

  53. [53]

    Protein database searches for multiple alignments.Pro- ceedings of the National Academy of Sciences, 87(14):5509–5513, 1990

    Stephen F Altschul and David J Lipman. Protein database searches for multiple alignments.Pro- ceedings of the National Academy of Sciences, 87(14):5509–5513, 1990

    1990

  54. [54]

    Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models.Nucleic acids research, 50(D1):D439–D444, 2022

    Mihaly Varadi, Stephen Anyango, Mandar Deshpande, Sreenath Nair, Cindy Natassia, Galabina Yor- danova, David Yuan, Oana Stroe, Gemma Wood, Agata Laydon, et al. Alphafold protein structure database: massively expanding the structural coverage of protein-sequence space with high-accuracy models.Nucleic acids research, 50(D1):D439–D444, 2022

    2022

  55. [55]

    Chips-ff: Evaluating universal machine learning force fields for material properties.ACS Materials Letters, 7(6):2105–2114, 2025

    Daniel Wines and Kamal Choudhary. Chips-ff: Evaluating universal machine learning force fields for material properties.ACS Materials Letters, 7(6):2105–2114, 2025. 16

    2025

  56. [56]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  57. [57]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Pith/arXiv arXiv 2023

  58. [58]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  59. [59]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Pith/arXiv arXiv 2023

  60. [60]

    Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi` ere, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    Pith/arXiv arXiv 2024

  61. [61]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025

    Pith/arXiv arXiv 2025

  62. [62]

    GPT-OSS-20B: An open-source 20-billion-parameter large language model, 2024

    OpenAI. GPT-OSS-20B: An open-source 20-billion-parameter large language model, 2024. Computer software

    2024

  63. [63]

    Phi-4 technical report.arXiv preprint arXiv:2412.08905, 2024

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S´ ebastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report.arXiv preprint arXiv:2412.08905, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Temperature and compositional dependence of the energy band gap of algan alloys.Applied Physics Letters, 87(24), 2005

    N Nepal, J Li, ML Nakarmi, JY Lin, and HX Jiang. Temperature and compositional dependence of the energy band gap of algan alloys.Applied Physics Letters, 87(24), 2005

    2005

  65. [65]

    Jarvis-leaderboard: a large scale benchmark of materials design methods.npj Computational Materials, 10(1):93, 2024

    Kamal Choudhary, Daniel Wines, Kangming Li, Kevin F Garrity, Vishu Gupta, Aldo H Romero, Jaron T Krogel, Kayahan Saritas, Addis Fuhr, Panchapakesan Ganesh, et al. Jarvis-leaderboard: a large scale benchmark of materials design methods.npj Computational Materials, 10(1):93, 2024

    2024

  66. [66]

    Wtu- eval: A whether-or-not tool usage evaluation benchmark for large language models.arXiv preprint arXiv:2407.12823, 2024

    Kangyun Ning, Yisong Su, Xueqiang Lv, Yuanzhe Zhang, Jian Liu, Kang Liu, and Jinan Xu. Wtu- eval: A whether-or-not tool usage evaluation benchmark for large language models.arXiv preprint arXiv:2407.12823, 2024

    Pith/arXiv arXiv 2024

  67. [67]

    Smartcal: An approach to self-aware tool-use evaluation and calibration.arXiv preprint arXiv:2412.12151, 2024

    Yuanhao Shen, Xiaodan Zhu, and Lei Chen. Smartcal: An approach to self-aware tool-use evaluation and calibration.arXiv preprint arXiv:2412.12151, 2024

    Pith/arXiv arXiv 2024

  68. [68]

    Advancing tool-augmented large language models via meta-verification and reflection learning

    Zhiyuan Ma, Jiayu Liu, Xianzhen Luo, Zhenya Huang, Qingfu Zhu, and Wanxiang Che. Advancing tool-augmented large language models via meta-verification and reflection learning. InProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 2078– 2089, 2025

    2078

  69. [69]

    Ollama: Local large language model runtime, 2023

    Ollama Team. Ollama: Local large language model runtime, 2023. Computer software

    2023

  70. [70]

    Whats the capital of US?

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. InProceedings of the 29th symposium on operating systems principles, pages 611–626, 2023. 17 Supplementary Information AGAPI-Agents: An Open-Access Age...

    2023

  71. [71]

    γ‑Al₂O₃ is the most widely used form for catalysis because of its high surface area (~200 m² g⁻¹)

    Crystalline Polymorphs Polymorph Space Group Lattice (Å) Density (g cm⁻³) Band Gap (eV) Melting Point (°C) Typical Uses Typical Synthesis α‑Al₂O₃ (corundum, sapphire) R(\bar{3})c a = 4.759, c = 12.991 3.97 9.0-9.3 2050- 2070 Optical windows, substrates for GaN, high‑temperature ceramics High‑temperature solid‑state reaction, Czochralski, hydrothermal γ‑Al...

    2050

  72. [72]

    Glass‑ceramics contain nanocrystalline α‑Al₂O₃ embedded in a glassy matrix, combining high hardness with low thermal expansion

    Amorphous and Glassy Forms Form Typical Composition Density (g cm⁻³) Band Gap (eV) Typical Uses Typical Synthesis Amorphous Al₂O₃ (glass) 1:1 Al:O, sometimes with Si, Ti, or B 3.5-3.8 8.5-9.0 Optical coatings, dielectric layers, protective coatings Melt‑quench, sol‑gel, ALD, sputtering Al₂O₃‑based glass‑ceramics α‑Al₂O₃ nuclei in glass matrix 3.8-4.0 9.0 ...

  73. [73]

    ALD is the most common route for conformal Al₂O₃ thin films; the typical precursor is trimethylaluminum (TMA) with water or ozone

    Nanostructured and Low‑Dimensional Forms Nanostructure Morphology Typical Size Typical Uses Typical Synthesis Al₂O₃ nanoparticles Spherical, 5-200 nm 5-200 nm Catalyst support, composite reinforcement, biomedical Sol‑gel, hydrothermal, precipitation Al₂O₃ nanowires / nanorods 1-10 µm long, 10- 100 nm diameter 10-100 nm Nanoscale electronics, Hydrothermal,...

  74. [74]

    Composite materials combine the high hardness of Al₂O₃ with the toughness or thermal conductivity of the secondary phase

    Doped and Composite Variants Doped/Composite Dopant Effect on Properties Typical Applications Cr‑doped Al₂O₃ (ruby) Cr³⁺ Red color, optical gainLaser media, ruby lasers Fe‑doped Al₂O₃ (sapphire) Fe³⁺ Greenish tint, magnetic Optical windows, magnetic sensors Ti‑doped Al₂O₃ Ti⁴⁺ Improved toughness, lower thermal expansion High‑temperature ceramics Y‑doped A...

  75. [75]

    Computational Studies of Al₂O₃ Method Typical Accuracy Common Findings DFT‑LDA Underestimates lattice constants by ~1-2 % Good for relative energetics of polymorphs DFT‑GGA (PBE) Lattice constants accurate to ~1 %; band gaps underestimated by ~2-3 eV Used for structural optimization DFT‑Hybrid (HSE06, PBE0) Band gaps within 0.1-0.2 eV of experiment Prefer...

  76. [76]

    Structure optimization with GGA (PBE) → obtain lattice constants

  77. [77]

    Band‑gap correction with HSE06 or GW

  78. [78]

    Defect calculations (O vacancies, Al vacancies, dopants) with DFT‑+U

  79. [79]

    Surface and interface modeling (e.g., Al₂O₃/SiC) with slab calculations

  80. [80]

    Thermodynamic stability via convex hull analysis (Materials Project)

Showing first 80 references.