REVIEW 2 major objections 1 minor 1 cited by
NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science
T0 review · 2 major / 1 minor · reviewed 2026-05-24 · grok-4.3
Pith's one-line read NIMS-OS is a Python library that combines AI methods with robotic controllers to run materials exploration experiments in a fully autonomous closed loop.
desk verdict NIMS-OS is a new open Python library that wires existing AI modules to robotic hardware for materials experiments, but the paper shows almost no data on whether the loop actually runs without intervention. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
NIMS-OS orchestration system that assembles interchangeable AI modules and robotic experiment controllers into an autonomous loop.
What would settle it
An actual run of the electrolyte exploration in which the system encounters an error or hardware issue that stops progress and cannot be resolved without a person stepping in.
Extended reading notes
Core claim
NIMS-OS realizes a closed loop of robotic experiments and artificial intelligence without human intervention for automated materials exploration. It uses various combinations of modules to operate autonomously, with each module acting as an AI for materials exploration or a controller for robotic experiments. As AI techniques, Bayesian optimization, boundless objective-free exploration, phase diagram construction, and random exploration methods can be used. A system called NIMS automated robotic electrochemical experiments is available as a set of robotic experimental equipment. Visualization tools for the results are also included, which allows users to check the optimization results in the
Load-bearing premise
The separate AI and robotic modules can be combined and will continue to function autonomously once started, with no unhandled edge cases or hardware failures that require human intervention during operation.
Editorial extensions
If this is right
- New electrolytes can be explored through repeated cycles of AI-guided robotic testing with no human present.
- Additional AI methods or robotic setups can be incorporated simply by writing and registering new modules.
- Users can monitor optimization progress in real time through the built-in visualization tools.
- Control of the entire autonomous system is available through the developed GUI application.
Reading between the lines
- The modular design could allow the same orchestration approach to be applied to other experimental domains if equivalent controllers are written.
- Long unattended runs might increase the total number of experiments completed per unit time compared with manual scheduling.
- Integration of additional AI techniques beyond those already listed could be tested by adding them as modules and measuring exploration efficiency.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces NIMS-OS, a Python library that orchestrates combinations of AI modules (PHYSBO for Bayesian optimization, BLOX, PDC for phase diagram construction, and RE) with the NAREE robotic controller to realize a closed loop of robotic experiments and AI for automated materials exploration without human intervention. It includes real-time visualization tools, a GUI application, and an extensible module interface, with a demonstration on electrolyte exploration; the code is released publicly on GitHub.
Significance. An open-source orchestration framework that integrates multiple AI exploration methods with robotic hardware could support reproducible automation in materials discovery workflows, provided the autonomous integration is validated. The public code release is a clear strength that enables community use and extension.
major comments (2)
- [Abstract and demonstration section] Abstract and demonstration description: the central claim that NIMS-OS 'realize[s] a closed loop ... without human intervention' and operates 'autonomously' rests on an unshown demonstration of the electrolyte exploration. No runtime logs, intervention counts, error rates, failure-mode analysis, or quantitative performance metrics are reported, leaving the reliability of the combined AI-robotic process unsubstantiated.
- [System architecture and module description] Architecture and module integration description: the text states that modules 'can be used' and 'newly created modules ... can be added easily' but provides no concrete specification of exception handling, hardware fault recovery, or inter-module compatibility checks that would be required to sustain autonomous operation over extended runs.
minor comments (1)
- The manuscript would benefit from a table or diagram explicitly mapping the data flow between each AI module (PHYSBO, BLOX, etc.) and the NAREE controller, including input/output formats.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript. We address each major comment below and indicate where revisions will be made to strengthen the paper.
read point-by-point responses
-
Referee: [Abstract and demonstration section] Abstract and demonstration description: the central claim that NIMS-OS 'realize[s] a closed loop ... without human intervention' and operates 'autonomously' rests on an unshown demonstration of the electrolyte exploration. No runtime logs, intervention counts, error rates, failure-mode analysis, or quantitative performance metrics are reported, leaving the reliability of the combined AI-robotic process unsubstantiated.
Authors: The demonstration illustrates the workflow for electrolyte exploration using the available modules, with the full implementation and execution details provided in the open GitHub repository for community verification. We agree that additional quantitative details on autonomy would better substantiate the claims. In the revised manuscript we will expand the demonstration section to include a summary of key run metrics, such as iteration count and observed behavior during the exploration. revision: yes
-
Referee: [System architecture and module description] Architecture and module integration description: the text states that modules 'can be used' and 'newly created modules ... can be added easily' but provides no concrete specification of exception handling, hardware fault recovery, or inter-module compatibility checks that would be required to sustain autonomous operation over extended runs.
Authors: The manuscript focuses on the modular interface and extensibility. We concur that explicit descriptions of exception handling and fault recovery are valuable for demonstrating sustained autonomy. We will add a dedicated subsection on error handling, hardware fault recovery procedures, and inter-module checks in the revised system architecture description. revision: yes
Circularity Check
No circularity: software description paper with no derivations or predictions
full rationale
This paper is a description of NIMS-OS software that combines existing AI modules (PHYSBO, BLOX, PDC, RE) and a robotic controller (NAREE) for automated experiments. No mathematical derivations, equations, fitted parameters, predictions, or uniqueness theorems are presented anywhere in the manuscript. The central claim is an engineering description of module integration and a demonstration run, with no load-bearing logical chain that could reduce to self-definition, fitted inputs, or self-citations. The reader's assessment of 0.0 is correct; circularity analysis does not apply.
Assumptions & free parameters
Cite this review
Pith. "Pith review of NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science." pith.science (2026). https://pith.science/paper/2304.13927
@misc{pith2026230413927,
author = {Pith},
title = {Pith review of: NIMS-OS: An automation software to implement a closed loop between artificial intelligence and robotic experiments in materials science},
year = {2026},
howpublished = {\url{https://pith.science/paper/2304.13927}},
note = {Machine review of arXiv:2304.13927}
}
read the original abstract
NIMS-OS (NIMS Orchestration System) is a Python library created to realize a closed loop of robotic experiments and artificial intelligence (AI) without human intervention for automated materials exploration. It uses various combinations of modules to operate autonomously. Each module acts as an AI for materials exploration or a controller for a robotic experiments. As AI techniques, Bayesian optimization (PHYSBO), boundless objective-free exploration (BLOX), phase diagram construction (PDC), and random exploration (RE) methods can be used. Moreover, a system called NIMS automated robotic electrochemical experiments (NAREE) is available as a set of robotic experimental equipment. Visualization tools for the results are also included, which allows users to check the optimization results in real time. Newly created modules for AI and robotic experiments can be added easily to extend the functionality of the system. In addition, we developed a GUI application to control NIMS-OS.To demonstrate the operation of NIMS-OS, we consider an automated exploration for new electrolytes. NIMS-OS is available at https://github.com/nimsos-dev/nimsos.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Optimal Resource Utilization for Autonomous Laboratory Orchestrators
Constraint programming optimizes experiment schedules for an autonomous MOF synthesis platform, with status dependencies enabling robust execution.
Reference graph
Works this paper leans on
-
[1]
The materials genome initiative: One year on
White A. The materials genome initiative: One year on. MRS Bulletin. 2 012;37(8):715–
-
[2]
Available from: https://doi.org/10.1557/mrs.2012.194
-
[3]
Machine learning in materials informatics: recent applications and prospects,
Ramprasad R, Batra R, Pilania G, et al. Machine learning in materials in formatics: recent applications and prospects. npj Computational Materials. 2017;3( 1):54. Available from: https://doi.org/10.1038/s41524-017-0056-5
-
[4]
Recent advances and applications of machine learning in solid-state materials science,
Schmidt J, Marques MRG, Botti S, et al. Recent advances and ap plications of ma- chine learning in solid-state materials science. npj Computational Ma terials. 2019;5(1):83. Available from: https://doi.org/10.1038/s41524-019-0221-0
-
[5]
A review of the recent progress in battery informatics
Ling C. A review of the recent progress in battery informatics. n pj Computational Mate- rials. 2022;8(1):33. Available from: https://doi.org/10.1038/s41524-022-00713-x
-
[6]
Black-box optimization for automated discovery
Terayama K, Sumita M, Tamura R, et al. Black-box optimization for automated discovery. Accounts of Chemical Research. 2021;54(6):1334–13 46. Available from: https://doi.org/10.1021/acs.accounts.0c00713
-
[7]
Combo: An efficient bayesian optimiz ation library for materials science
Ueno T, Rhone TD, Hou Z, et al. Combo: An efficient bayesian optimiz ation library for materials science. Materials Discovery. 2016;4:18–21. Av ailable from: https://www.sciencedirect.com/science/article/pii/S2352924516300035
work page 2016
-
[8]
Homma K, Liu Y, Sumita M, et al. Optimization of a heterogeneous te rnary Li3PO4–Li3BO3–Li2SO4 mixture for li-ion conductivity by machine learning. The Journal of Physical Chemistry C. 2020;124(24):12865–12870 . Available from: https://doi.org/10.1021/acs.jpcc.9b11654
Show all 42 references
-
[9]
Ultranarrow-band wavelen gth-selective thermal emission with aperiodic multilayered metamaterials designed b y bayesian optimization
Sakurai A, Yada K, Simomura T, et al. Ultranarrow-band wavelen gth-selective thermal emission with aperiodic multilayered metamaterials designed b y bayesian optimization. ACS Central Science. 2019;5(2):319–326. A vailable from: https://doi.org/10.1021/acscentsci.8b00802. 25
2019 doi
-
[10]
A data fusion approach to optimiz e compo- sitional stability of halide perovskites
Sun S, Tiihonen A, Oviedo F, et al. A data fusion approach to optimiz e compo- sitional stability of halide perovskites. Matter. 2021;4(4):1305–13 22. Available from: https://www.sciencedirect.com/science/article/pii/S2590238521000084
2021
-
[11]
Machine learning-driven op timization in powder manufacturing of ni-co based superalloy
Tamura R, Osada T, Minagawa K, et al. Machine learning-driven op timization in powder manufacturing of ni-co based superalloy. Materials & Design. 2021;1 98:109290. Available from: https://www.sciencedirect.com/science/article/pii/S0264127520308261
2021
-
[12]
Data-driven automated r obotic exper- iments accelerate discovery of multi-component electrolyte for re chargeable Li- O2 batteries
Matsuda S, Lambard G, Sodeyama K. Data-driven automated r obotic exper- iments accelerate discovery of multi-component electrolyte for re chargeable Li- O2 batteries. Cell Reports Physical Science. 2022;3(4):100832. Ava ilable from: https://www.sciencedirect.com/science/articl...
2022
-
[13]
Automated crystal struct ure analysis based on blackbox optimisation
Ozaki Y, Suzuki Y, Hawai T, et al. Automated crystal struct ure analysis based on blackbox optimisation. npj Computational Materials. 2020;6(1):7 5. Available from: https://doi.org/10.1038/s41524-020-0330-9
2020 doi
-
[14]
Automatic rietveld refine ment by robotic process automation with rietan-fp
Tamura R, Sumita M, Terayama K, et al. Automatic rietveld refine ment by robotic process automation with rietan-fp. Science and Technology of Advanced Materials: Methods. 2022; 2(1):435–444. Available from: https://doi.org/10.1080/27660400.2022.2146470
2022 doi
-
[15]
MDTS: automatic complex materia ls design using monte carlo tree search
Dieb TM, Ju S, Yoshizoe K, et al. MDTS: automatic complex materia ls design using monte carlo tree search. Science and Technology of Advanced Mat erials. 2017;18(1):498–
2017
-
[16]
Available from: https://doi.org/10.1080/14686996.2017.1344083
2017 doi
-
[17]
Self-learning entropic population annealing for interpretable materials design
Li J, Zhang J, Tamura R, et al. Self-learning entropic population annealing for interpretable materials design. Digital Discovery. 2022;1:295–302. Available from: http://dx.doi.org/10.1039/D1DD00043H
2022 doi
-
[18]
Designing metamaterials with quantum a nnealing and factorization machines
Kitai K, Guo J, Ju S, et al. Designing metamaterials with quantum a nnealing and factorization machines. Physical Review Research. 2020;2(1):013 319. Available from: https://doi.org/10.1103/PhysRevResearch.2.013319
2020 doi
-
[19]
Benchmark test of black-box optimiza- tion using d-wave quantum annealer
Koshikawa AS, Ohzeki M, Kadowaki T, et al. Benchmark test of black-box optimiza- tion using d-wave quantum annealer. Journal of the Physical Socie ty of Japan. 2021; 26 90(6):064001. Available from: https://doi.org/10.7566/JPSJ.90.064001
2021 doi
-
[20]
Continuous black-box optimiza tion with an ising machine and random subspace coding
Izawa S, Kitai K, Tanaka S, et al. Continuous black-box optimiza tion with an ising machine and random subspace coding. Phys Rev Res. 2022;4:023062 . Available from: https://link.aps.org/doi/10.1103/PhysRevResearch.4.023062
2022 doi
-
[21]
The automa- tion of science
King RD, Rowland J, Oliver SG, et al. The automa- tion of science. Science. 2009;324(5923):85–89. Available from: https://www.science.org/doi/abs/10.1126/science.1165620
2009 doi
-
[22]
Robotic laboratory automation
Boyd J. Robotic laboratory automation. Science. 2002;295(55 54):517–518. Available from: https://www.science.org/doi/abs/10.1126/science.295.5554.517
2002 doi
-
[23]
The first 110 years of laboratory automation: Techno logies, applications, and the creative scientist
Olsen K. The first 110 years of laboratory automation: Techno logies, applications, and the creative scientist. SLAS Technology. 2012;17(6):469–480 . Available from: https://doi.org/10.1177/2211068212455631
2012 doi
-
[24]
Impact of high-throug hput screening in biomedical research
Macarron R, Banks MN, Bojanic D, et al. Impact of high-throug hput screening in biomedical research. Nature Reviews Drug Discovery. 2011;10(3) :188–195. Available from: https://doi.org/10.1038/nrd3368
2011 doi
-
[25]
Self-driving labor atory for accelerated discovery of thin-film materials
MacLeod BP, Parlane FGL, Morrissey TD, et al. Self-driving labor atory for accelerated discovery of thin-film materials. Science Advances. 2020;6(20):eaa z8867. Available from: https://www.science.org/doi/abs/10.1126/sciadv.aaz8867
2020 doi
-
[26]
Autonomous materia ls synthesis by machine learning and robotics
Shimizu R, Kobayashi S, Watanabe Y, et al. Autonomous materia ls synthesis by machine learning and robotics. APL Materials. 2020;8(11):111110. A vailable from: https://doi.org/10.1063/5.0020370
2020 doi
-
[27]
Autonomous discovery o f battery electrolytes with robotic experimentation and machine lear n- ing
Dave A, Mitchell J, Kandasamy K, et al. Autonomous discovery o f battery electrolytes with robotic experimentation and machine lear n- ing. Cell Reports Physical Science. 2020;1(12):100264. Available fr om: https://www.sciencedirect.com/science/article/pii/S2666386420302861
2020
-
[28]
A mobile robotic chemis t
Burger B, Maffettone PM, Gusev VV, et al. A mobile robotic chemis t. Nature. 2020; 583(7815):237–241. Available from: https://doi.org/10.1038/s41586-020-2442-2 . 27
2020 doi
-
[29]
Chemos: An orchestratio n software to democratize autonomous discovery
Roch LM, H¨ ase F, Kreisbeck C, et al. Chemos: An orchestratio n software to democratize autonomous discovery. PLOS ONE. 2020;15(4):1–18. Available from: https://doi.org/10.1371/journal.pone.0229862
2020 doi
-
[30]
Bayesian optimization pa ckage: PHYSBO
Motoyama Y, Tamura R, Yoshimi K, et al. Bayesian optimization pa ckage: PHYSBO. Computer Physics Communications. 2022;278:108405. Ava ilable from: https://www.sciencedirect.com/science/article/pii/S0010465522001242
2022
-
[31]
Pushing property limits in m aterials discovery via boundless objective-free exploration
Terayama K, Sumita M, Tamura R, et al. Pushing property limits in m aterials discovery via boundless objective-free exploration. Chemical Science. 2020 ;11:5959–5968. Available from: http://dx.doi.org/10.1039/D0SC00982B
2020 doi
-
[32]
Efficient construction met hod for phase diagrams using uncertainty sampling
Terayama K, Tamura R, Nose Y, et al. Efficient construction met hod for phase diagrams using uncertainty sampling. Physical Review Materials. 2019;3:03380 2. Available from: https://link.aps.org/doi/10.1103/PhysRevMaterials.3.033802
2019 doi
-
[33]
Python Materials Genomics (py - matgen): A robust, open-source python library for materials ana ly- sis
Ong SP, Richards WD, Jain A, et al. Python Materials Genomics (py - matgen): A robust, open-source python library for materials ana ly- sis. Computational Materials Science. 2013;68:314–319. Available fr om: https://linkinghub.elsevier.com/retrieve/pii/S0927025612006295
2013
-
[34]
A general-purpose mac hine learning frame- work for predicting properties of inorganic materials
Ward L, Agrawal A, Choudhary A, et al. A general-purpose mac hine learning frame- work for predicting properties of inorganic materials. npj Computa tional Materi- als. 2016;2(1):1–7. Number: 1 Publisher: Nature Publishing Group; A vailable from: https://www.nature.com/article...
2016
-
[35]
RDKit ; Available from: https://www.rdkit.org/
-
[36]
Efficiently searching extr eme me- chanical properties via boundless objective-free exploration and minimal first- principles calculations
Ojih J, Al-Fahdi M, Rodriguez AD, et al. Efficiently searching extr eme me- chanical properties via boundless objective-free exploration and minimal first- principles calculations. npj Computational Materials. 2022;8(1):143 . Available from: https://doi.org/10.1038/s41524-022-00836-1
2022 doi
-
[37]
Machine-learning-based p hase dia- gram construction for high-throughput batch experiments
Tamura R, Deffrennes G, Han K, et al. Machine-learning-based p hase dia- gram construction for high-throughput batch experiments. Scie nce and Tech- 28 nology of Advanced Materials: Methods. 2022;2(1):153–161. Availa ble from: https://doi.org/10.1080/27660400.2022.2076548
2022 doi
-
[38]
Experimental establis hment of phase dia- grams guided by uncertainty sampling: An application to the depositio n of Zn–Sn–P films by molecular beam epitaxy
Katsube R, Terayama K, Tamura R, et al. Experimental establis hment of phase dia- grams guided by uncertainty sampling: An application to the depositio n of Zn–Sn–P films by molecular beam epitaxy. ACS Materials Letters. 2020;2(6):57 1–575. Available from: https://doi.org/10.10...
2020 doi
-
[39]
Topological alternation from st ructurally adaptable to mechanically stable crosslinked polymer
Hu WH, Chen TT, Tamura R, et al. Topological alternation from st ructurally adaptable to mechanically stable crosslinked polymer. Science and Technology of A dvanced Materials. 2022;23(1):66–75. Available from: https://doi.org/10.1080/14686996.2021.2025426
2022 doi
-
[40]
High-throughput combinat orial screen- ing of multi-component electrolyte additives to improve the perform ance of Li metal secondary batteries
Matsuda S, Nishioka K, Nakanishi S. High-throughput combinat orial screen- ing of multi-component electrolyte additives to improve the perform ance of Li metal secondary batteries. Scientific Reports. 2019;9(1):621 1. Available from: https://doi.org/10.1038/s41598-019-42766-x
2019 doi
-
[41]
Effect of vinylene carbonate as additiv e to elec- trolyte for lithium metal anode
Ota H, Shima K, Ue M, et al. Effect of vinylene carbonate as additiv e to elec- trolyte for lithium metal anode. Electrochimica Acta. 2004;49(4):56 5–572. Available from: https://www.sciencedirect.com/science/article/pii/S0013468603007266
2004
-
[42]
Fluoroethylene carbonate a d- ditives to render uniform li deposits in lithium metal batteries
Zhang XQ, Cheng XB, Chen X, et al. Fluoroethylene carbonate a d- ditives to render uniform li deposits in lithium metal batteries. Ad- vanced Functional Materials. 2017;27(10):1605989. Available from : https://onlinelibrary.wiley.com/doi/abs/10.1002/adfm.201605989. 29
2017 doi
Reviewed May 24, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.