Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks

Ian Abshire; Nathan S Johnson

REVIEW 3 major objections 4 minor 37 references

Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Agentic microscopy benchmarks can qualify and regression-test agents, but task-held-out surrogate models perform near chance, so no global configuration ranking transfers to unseen tasks.

desk verdict A genuinely useful benchmark and honest negative result, but the load-bearing 'near chance' AUC lacks error bars on a small, version-drifting dataset, so the strength of the transferability claim is not yet secured. read the letter →

arxiv 2608.05266 v1 pith:6TAKVUXH submitted 2026-08-05 cs.AI cond-mat.mtrl-scics.LG

Nathan S Johnson , Ian Abshire This is my paper · ORCID

classification cs.AIcond-mat.mtrl-scics.LG

keywords agenticmicroscopyLLMagentsbenchmarkgeneralizationtask-held-outvalidationretrieval-augmentedgenerationself-drivinglaboratoryModelContextProtocolsurrogatemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether benchmark performance on a fixed set of microscopy tasks can be used to choose an agent configuration that will also work on tasks the agent has never seen. The authors built a trace-logging benchmark of 53 tests, ran 105 agent configurations across 1,949 test runs, and trained surrogate models under two validation schemes. When known tests stayed in training, the random-forest surrogate reached an ROC-AUC of about 0.78; when entire test identities were held out, both surrogates fell to about 0.55, near chance. The paper concludes that this benchmark suite is useful for qualification, regression testing, diagnosis, and direct comparison, but it cannot support a task-independent global configuration model. If true, this matters because teams building autonomous microscopes and beamlines need to know what a pass rate is and is not evidence for.

What carries the argument

The central mechanism is grouped cross-validation with task-held-out folds: all runs sharing a test identity are assigned to the same fold, test identity is removed from the design matrix, and the surrogate must rank configurations purely from configuration features on unseen task identities. This contrasts with configuration-held-out validation, in which known test identities remain in training and the model mostly learns intrinsic task difficulty. Supporting this are the composite pass criteria (scientific correctness, trace checks, artifact checks, operational limits), the stable configuration hash that defines an agent design, and RAG retrieval identifiers that make each retrieved context fragment auditable in relation to pass or fail outcomes.

What would settle it

Run a randomized factorial sweep with a fixed software version, balanced repeats, and task-held-out folds: if the surrogate's held-out ROC-AUC rises well above 0.55 (for example, above 0.7) on a fresh collection of unrelated microscopy tasks, the paper's claim of no transferable configuration signal would be overturned.

Watch

Extended reading notes

Core claim

The central claim is that the heterogeneous benchmark suite does not support a task-independent global configuration model. Configuration variables—number of agents, LLM, temperature, RAG parameters, context limits—affected latency, token use, cost, and failure mode, and test identity was the strongest predictor in the configuration-held-out surrogate. Once complete tests were removed from training and test-identity features were dropped, L2 logistic regression and random forest both achieved ROC-AUC around 0.55. The paper therefore asserts that benchmark pass rates remain valid for qualification, regression testing, diagnosis, and direct comparison of defined workflows, while a separate, prospective comparison can still identify a good configuration for the tested workflows, with a one-agent pruned-RAG configuration performing best while a no-RAG baseline performed approximately as well.

Load-bearing premise

The load-bearing premise is that the grouped task-held-out cross-validation of this historical, unbalanced dataset—with non-independent repeats and changing software versions—accurately measures transfer to new tasks rather than being distorted by temporal drift or confounding.

Editorial extensions

If this is right

Benchmarks should be used for qualification, regression testing, diagnosis, and direct comparison, and should not be treated as an optimization oracle for unseen tasks.
Multi-agent topologies did not consistently improve pass rates and generally increased tokens, calls, and time, so added agents need a demonstrated payoff.
A simple no-RAG baseline should be part of every agent comparison, since additional context and architectural complexity gave no consistent advantage.
Future benchmark-driven optimization should move toward task-aware configuration selection—describing workflow family, required tools, sequence length, and quantitative versus trace endpoint—rather than one global ranking.
Repeated scientific operations should be moved from free-form LLM code into deterministic, checked tools, since the least reliable step was ad hoc Python written during the workflow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If test identity dominates as the paper's feature analysis shows, similar task-held-out near-chance results should be expected for other scientific agent benchmarks (synthesis, synchrotron beamlines, imaging), and benchmark reports should publish task descriptors to test transfer.
Because the dataset is historical and unbalanced with changing model, software, and hardware versions, a randomized factorial benchmark with fixed versions and controlled repeats could separate true configuration signal from temporal drift; this is the paper's own stated limitation, extended here as the next experiment.
The RAG pruning result suggests that context databases should be maintained by software-maintenance criteria (stale, contradictory, topology-inappropriate entries) rather than by outcome association alone, since retrieval is observational and confounded by task and workflow state.
A practical extension is to build task-aware surrogate models that include workflow descriptors; if such descriptors restored held-out AUC, the conclusion would shift from 'no global model' to 'the right task representation supports transfer'.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know about arXiv:2608.05266. First, it's a serious, data-rich study of agentic microscope control—105 configurations, 53 tests, 1,949 runs, and 49,109 logged RAG retrievals—and it ships code and data. Second, its central negative claim, that benchmark pass rates do not support a global configuration model for unseen tasks, is plausible but not quantitatively nailed down: the task-held-out ROC-AUC of 0.55 is reported without confidence intervals or a permutation null, on a dataset the authors themselves describe as historical, unbalanced, and with version drift. On that evidence, 'near chance' could be noise or weak-but-real transfer.

What's actually new: the explicit split between configuration-held-out and task-held-out validation for microscope agents. The config-held-out RF gets ~0.78 AUC because test identity dominates; task-held-out drops to ~0.55. That decomposition is exactly the right way to ask whether a benchmark can be used for optimization, and I don't know prior work that does it for instrument agents. The RAG retrieval-to-outcome linkage with stable identifiers is also a real contribution—it turns the context window into an auditable object and shows which entries are stale or harmful. The direct comparisons (one vs two vs three agents, token/cost/failure mode differences) are useful engineering data, and the physical validation on a real Versa 730 with quantitative criteria is more than most papers in this space do.

Soft spots, in proportion. The missing uncertainty quantification on Table 1 is the main one. With 53 test groups and up to 5 folds, the cluster-level AUC variance is large; a 0.55 point estimate could easily have a 95% CI that includes 0.5, which would turn 'no transferable signal' into 'we couldn't detect one.' That distinction matters for the paper's conclusion. The authors are admirably honest about the dataset limitations in Section 3.3, but they don't translate those limitations into error bars or a permutation test, which is the one analysis that would settle the question. Minor point: the prospective comparison picks candidates using the same 53-test benchmark, so it's a direct comparison, not a validation of the surrogate; the authors say as much, so it's not a flaw, just a boundary.

The paper is carefully hedged, the reasoning is clear, and the limitations are stated rather than hidden. The citation pattern looks appropriate. If I were refereeing, I'd ask for confidence intervals or a permutation null on the task-held-out AUC, and for a discussion of temporal confounding that the version drift could introduce. If the CI excludes 0.5, the conclusion can stay; if not, the claim weakens to 'we did not find transfer,' which is still a legitimate result but a different one.

This deserves a serious referee. I'd bring it to reading group—it's a good template for how to think about benchmark generalization in autonomous labs.

Referee Report

3 major / 4 minor

Summary. The paper presents a trace-logging and benchmark framework for agentic microscopy control built on one-, two-, and three-agent LLM graphs connected to six MCP servers. Over 105 configurations, 1,949 test runs, and 49,109 RAG retrievals across 53 benchmark tests, the authors show that direct comparisons reveal clear differences in token use, latency, cost, and failure modes, and that a one-agent pruned-RAG configuration was the best prospective candidate, though a no-RAG baseline performed approximately as well. The central negative claim is about transfer: a random-forest surrogate reaches ROC-AUC ≈ 0.78 under configuration-held-out validation, but falls to 0.55 under task-held-out validation (0.53 for logistic regression), which the authors interpret as showing that the current heterogeneous test suite does not support a task-independent global configuration model. The paper also reports RAG-retrieval auditing, physical validation on a ZEISS Versa 730, and a careful discussion of what the benchmark can and cannot support.

Significance. If the central negative claim is secure, this is a valuable and timely negative result for the nascent field of LLM-controlled scientific instruments: it would caution against using aggregate benchmark pass rates to optimize agent configurations for unseen tasks, while preserving the benchmark's value for qualification, regression testing, and diagnosis. The paper's strengths are its large traced dataset, its use of two model classes (random forest and L2 logistic regression), an explicit grouped cross-validation scheme, a sensitivity analysis removing near-universal tests, prospective physical validation, and openly released code and data. The claim is not circular: the task-held-out evaluation removes complete test identities, so the negative result does not arise from fitting the conclusion. However, the statistical support for the near-chance task-held-out AUC is currently incomplete, and the central claim rests on a small number of quantitative point estimates.

major comments (3)

[Table 1 / §5.10] The task-held-out ROC-AUC values (0.53 for logistic regression, 0.55 for random forest) are reported as point estimates without confidence intervals, standard errors, or a permutation null. With 53 test groups, up to five folds, and variable class balance, the sampling uncertainty around these AUC values is plausibly large; if the 95% confidence interval includes 0.5, the evidence that configuration carries no transferable signal is weak, whereas a narrow interval excluding 0.5 would support a different framing ('weak but detectable transfer'). Please provide cluster-resampled confidence intervals (e.g., bootstrap by test identity) and a permutation null for the task-held-out analysis, and report the number of folds and groups actually used.
[§2.3 / §5.10] The sensitivity analysis that removes near-universally passing tests is reported only as 'did not materially improve the task-held-out result,' with no numerical AUC values. Because this analysis directly addresses whether the negative result is dominated by uninformative easy tests, please report the resulting task-held-out AUCs (and their uncertainties) for the reduced test set, or state explicitly which tests were removed and why the numbers were omitted.
[§3.3 / §5.2] The manuscript acknowledges that the dataset is historical and unbalanced, that model, prompt, framework, code, software, and hardware versions changed during the study, and that repeated runs are not fully independent. Under these conditions, the task-held-out null result could be confounded by temporal drift if version changes correlate with both configuration and outcome. The paper should address this possibility concretely, for example by testing whether the task-held-out AUC changes when the analysis is restricted to a period of stable software versions, or by including version/time as a covariate, or at minimum by discussing the direction and magnitude of plausible bias.

minor comments (4)

[Code and Data Availability] The repository URL appears as plain text 'github.com/natertott/agentic microscopy benchmarks XRM' with spaces; please format it as a proper clickable URL with the correct repository path.
[Figure 5 caption] The caption states 'Error bars shows the Wilson 95% confidence interval'; this should be 'Error bars show'.
[§2.5 / §3.1] The phrase describing the no-RAG baseline as performing 'approximately as well' on the standardized benchmark is not quantified; please report the actual pass rates and confidence intervals so readers can judge the practical difference between the best candidate and the baseline.
[§5.10] The text says grouped cross-validation used 'up to five folds'; please clarify whether the number of folds was fixed at five, whether it varied by group availability, and whether a single split or repeated/randomized grouping was used.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the central negative claim rests on task-held-out cross-validation, not on the conclusion being built into the inputs.

full rationale

The paper's central claim—that the current heterogeneous test suite does not support a task-independent global configuration model—is derived from a grouped task-held-out cross-validation (Sections 2.3 and 5.10) in which complete test identities are removed from training and test-identity features are dropped from the design matrix. The resulting near-chance ROC-AUC values (0.53 for logistic regression, 0.55 for random forest) are measured outcomes on held-out task groups, not restatements of the conclusion; no equation defines the target result into the inputs. The configuration-held-out AUC of 0.78 is explicitly interpreted as interpolation on already represented tests, and the paper does not use the prospective candidate comparison as evidence of transfer; Section 2.5 states that a configuration can perform best in a direct comparison while the surrogate still fails to rank configurations on unseen task identities. The label 'prospective' for the standardized-benchmark comparison is somewhat imprecise because those 53 tests overlap the surrogate training data, but the authors explicitly disclaim any generalization conclusion from that comparison, so it is not a circular step in the central derivation. The single self-citation (reference [10]) is background context about prior systems and is not load-bearing; the methods rely on external standards such as ReAct and the Model Context Protocol, and the physical validation tasks are independent of the surrogate dataset. The limitations the paper itself lists in Section 3.3—historical and unbalanced data, version drift across model/prompt/framework/software/hardware, non-independent repeated runs, and a multi-agent implementation incompatibility—are genuine threats to the precision of the near-chance AUC estimate, but they are statistical validity concerns, not circularity. Therefore no significant circular step is present.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claims are empirical, so no constants are fitted to derive a result. The main implicit costs are the domain assumptions listed above: composite pass rate as a scalar endpoint, test-suite representativeness, and statistical validity under non-independence. No new physical or conceptual entities are postulated.

free parameters (1)

Surrogate model hyperparameters (random forest and L2 logistic regression) = not reported
The task-held-out and configuration-held-out AUC values depend on unstated hyperparameter choices. Agreement between the two model classes reduces concern, but exact values are absent from the paper.

assumptions (3)

domain assumption The composite binary pass/fail outcome is a valid scalar endpoint for comparing agent configurations.
The conclusion that no global configuration model exists is based on predicting the composite pass/fail outcome defined in Section 5.5. If component checks carried transferable signal that pooling hides, the negative conclusion could change.
domain assumption The 53-test suite is sufficiently heterogeneous to represent the space of microscopy tasks relevant to the conclusion.
The generalization result is explicitly bounded to 'the current heterogeneous test suite.' A narrower or differently composed suite might show apparent transfer, so the assumption that this suite is a meaningful testbed is load-bearing.
domain assumption Grouped cross-validation and Fisher exact tests produce reliable statistics despite non-independent repeats.
Section 5.2 states that repeated runs of the same task and configuration were not fully independent, which can bias AUC estimates and p-values. The paper acknowledges this but does not quantify its effect on the central 0.55 AUC result.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks." pith.science (2026). https://pith.science/paper/6TAKVUXH

@misc{pith2026260805266,
  author       = {Pith},
  title        = {Pith review of: Agentic self-driving microscopy benchmarks support qualification but do not necessarily generalize to unseen tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6TAKVUXH}},
  note         = {Machine review of arXiv:2608.05266}
}

read the original abstract

Large language model agents are increasingly being developed to control a wide range of scientific characterization tools including microscopes and synchrotron beamlines. Research into agentic control of physical infrastructure is nascent and there are few well-established paradigms for how to engineer an agentic system. There are many choices to make when designing a microscopy agent, including the choice of LLM, the number of agents to use, agent responsibilities and delegation rules, retrieval-augmented generation parameters, and more. When designing and optimizing an agentic microscope controller, researchers not only want to ensure that the agent can correctly perform known tasks but also that the agent can generalize to new tasks that it has not encountered before. In this study, we develop a benchmark and trace-logging framework that reveals a) how different choices of agent architecture impact performance at microscopy tasks and b) the limitations of benchmarks for predicting if a particular agent will perform well on unseen microscopy tasks. The framework was used to evaluate one-, two-, and three-agent graph topologies, five LLMs, RAG and context parameters, and operational constraints across 53 microscopy benchmark tests. In total, 105 agent configurations, 1,949 individual test runs, and 49,109 RAG retrievals were recorded. Direct comparisons showed clear differences in latency, token use, cost, and failure mode between configurations. However, surrogate models trained on agent architecture and test results did not reliably predict an agent's performance on new, unseen tasks. These results show that these benchmarks are useful for qualification, regression testing, diagnosis, and direct comparison, but the current heterogeneous test suite does not support a task-independent global configuration model.

Figures

Figures reproduced from arXiv: 2608.05266 by the authors.

**Figure 2.** Direct comparison of one-, two-, and three-agent topologies on the common 15-test screening [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 1.** We evaluated 105 configurations across 53 benchmark tests and linked every run to its configu [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗

Figures from the paper (6 more)

**Figure 3.** Figure 3: Pearson correlations between selected configuration variables and benchmark outcomes. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]

**Figure 4.** Figure 4: Association between retrieval of individual RAG entries and composite benchmark pass rate. The [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Prospective comparison across the standardized 53-test benchmark and five quantitative valida [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Output from a successful physical validation workflow. The agent completed the workflow through [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]

**Figure 7.** Figure 7: Selection of three-dimensional tomography datasets acquired using the same integrated agent [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Context construction and execution flow for supervisor and ReAct worker agents. The one-agent [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages

[1]

Accelerating scientific discovery with Co-Scientist.Nature, 655:487–496, May 2026

Juraj Gottweis et al. Accelerating scientific discovery with Co-Scientist.Nature, 655:487–496, May 2026

work page 2026
[2]

Szostkiewicz, Dmytro Shved, Gavin J

Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, Dmytro Shved, Gavin J. Gyimesi, Jon M. Laurent, Samantha M. Wright, Muhammed T. Razzak, Andrew D. White, Silvia C. Finnemann, Michaela M. Hinks, and Samuel G. Rodriques. A multi-agent system for automating scientific discovery.Nature, 655:497–505, May 2026

work page 2026
[3]

Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D

Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. Augmenting large language models with chemistry tools.Nature Machine Intelligence, 6:525–535, 2024

work page 2024
[4]

Accurate predictions on small data with a tabular foundation model.Nature, 637:319–326, 2025

Noah Hollmann, Samuel M¨ uller, Lennart Purucker, Arjun Krishnakumar, Max K¨ orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model.Nature, 637:319–326, 2025

work page 2025
[5]

Gilad Kusne, Ji-Cheng Zhao, and Ichiro Takeuchi

Haotong Liang, Chuangye Wang, Heshan Yu, Dylan Kirsch, Rohit Pant, Austin McDannald, A. Gilad Kusne, Ji-Cheng Zhao, and Ichiro Takeuchi. Real-time experiment-theory closed-loop interaction for autonomous materials science.Science Advances, 11(27):eadu7426, 2025

work page 2025
[6]

Abolhasani and E

M. Abolhasani and E. Kumacheva. The rise of self-driving labs in chemical and materials sciences. Nature Synthesis, 2:483–492, 2023

work page 2023
[7]

Wu, and Al´ an Aspuru-Guzik

Martin Seifrid, Robert Pollice, Andr´ es Aguilar-Granda, Zamyla Morgan Chan, Kazuhiro Hotta, Cher Tian Ser, Jenya Vestfrid, Tony C. Wu, and Al´ an Aspuru-Guzik. Autonomous chemical exper- iments: Challenges and perspectives on establishing a self-driving lab.Accounts of Chemical Research, 55(17):2454–2466, 2022. PMID: 35948428

work page 2022
[8]

Schmid, Sterling G

Gary Tom, Stefan P. Schmid, Sterling G. Baird, Yang Cao, Kourosh Darvish, Han Hao, Stanley Lo, Sergio Pablo-Garc ´ ıa, Ella M. Rajaonson, Marta Skreta, Naruki Yoshikawa, Samantha Corapi, Gun Deniz Akkoc, Felix Strieth-Kalthoff, Martin Seifrid, and Al´ an Aspuru-Guzik. Self-driving laboratories for chemistry and materials science.Chemical Reviews, 124(16):...

work page 2024

Show all 37 references

[9]

Perspectives for self-driving labs in synthetic biology.Current Opinion in Biotechnology, 79:102881, 2023

Hector G Martin, Tijana Radivojevic, Jeremy Zucker, Kristofer Bouchard, Jess Sustarich, Sean Peisert, Dan Arnold, Nathan Hillson, Gyorgy Babnigg, Jose M Marti, Christopher J Mungall, Gregg T Beckham, Lucas Waldburger, James Carothers, ShivShankar Sundaram, Deb Agarwal, Blake A...

2023
[10]

Nathan S. Johnson. Multi-agent systems for autonomous laboratory instrument operation, August 2025

2025
[11]

Bridging electron microscopy and materials analysis with an autonomous agentic platform.Science Advances, 12(14):eaed0583, 2026

Guangyao Chen, Wenhao Yuan, and Fengqi You. Bridging electron microscopy and materials analysis with an autonomous agentic platform.Science Advances, 12(14):eaed0583, 2026

2026
[12]

Yager, and Esther H

Shray Mathur, Noah van der Vleuten, Kevin G. Yager, and Esther H. R. Tsai. VISION: a modular AI assistant for natural human–instrument interaction at scientific user facilities.Machine Learning: Science and Technology, 6(2):025051, June 2025

2025
[13]

Cherukara

Ming Du, Yanqi Luo, Srutarshi Banerjee, Michael Wojcik, Jelena Popovic, and Mathew J. Cherukara. Eaa: Automating materials characterization with vision language model agents, 2026

2026
[14]

Vriza, M

A. Vriza, M. H. Prince, T. Zhou, H. Chan, and M. J. Cherukara. Operating advanced scientific instru- ments with AI agents that learn on the job.npj Computational Materials, 12:160, 2026

2026
[15]

Wall, Alexander J

Morgan K. Wall, Alexander J. Pattison, Edward S. Barnard, Stephanie M. Ribet, and Peter Ercius. TEM Agent: enhancing transmission electron microscopy with modern AI tools.npj Computational Materials, 2026. Published online 10 June 2026. 18

2026
[16]

Spm-bench: Benchmarking large language models for scanning probe microscopy, 2026

Peiyao Xiao, Xiaogang Li, Chengliang Xu, Jiayi Wang, Ben Wang, Zichao Chen, Zeyu Wang, Kejun Yu, Yueqian Chen, Xulin Liu, Wende Xiao, Bing Zhao, and Hu Wei. Spm-bench: Benchmarking large language models for scanning probe microscopy, 2026

2026
[17]

Kalinin, Maxim Ziatdinov, Jacob Hinkle, Stephen Jesse, Ayana Ghosh, Kyle P

Sergei V. Kalinin, Maxim Ziatdinov, Jacob Hinkle, Stephen Jesse, Ayana Ghosh, Kyle P. Kelley, An- drew R. Lupini, Bobby G. Sumpter, and Rama K. Vasudevan. Automated and autonomous experiments in electron and scanning probe microscopy.ACS Nano, 15(8):12604–12627, 2021. PMID: 34269558

2021
[18]

Rack, Yu-Chen Liu, Jan-Chi Yang, and Sergei V

Yu Liu, Roger Proksch, Jason Bemis, Utkarsh Pratiush, Astita Dubey, Mahshid Ahmadi, Reece Emery, Philip D. Rack, Yu-Chen Liu, Jan-Chi Yang, and Sergei V. Kalinin. Machine learning-based reward-driven tuning of scanning probe microscopy: Toward fully automated microscopy.ACS Na...

2025
[19]

Smedskjaer, Katrin Wondraczek, Lothar Won- draczek, Nitya Nand Gosvami, and N

Indrajeet Mandal, Jitendra Soni, Mohd Zaki, Morten M. Smedskjaer, Katrin Wondraczek, Lothar Won- draczek, Nitya Nand Gosvami, and N. M. Anoop Krishnan. Evaluating large language model agents for automation of atomic force microscopy.Nature Communications, 16:9104, 2025

2025
[20]

Balachandran, Dezhen Xue, and Ruihao Yuan

Turab Lookman, Prasanna V. Balachandran, Dezhen Xue, and Ruihao Yuan. Active learning in mate- rials science with emphasis on adaptive sampling using uncertainties for targeted design.npj Computa- tional Materials, 5:21, 2019

2019
[21]

Deringer, Albert P

Volker L. Deringer, Albert P. Bart´ ok, Noam Bernstein, David M. Wilkins, Michele Ceriotti, and G´ abor Cs´ anyi. Gaussian process regression for materials and molecules.Chemical Reviews, 121(16):10073– 10141, 2021. PMID: 34398616

2021
[22]

Wu, Al´ an Aspuru-Guzik, and Eugenia Kumacheva

Huachen Tao, Tianyi Wu, Matteo Aldeghi, Tony C. Wu, Al´ an Aspuru-Guzik, and Eugenia Kumacheva. Nanoparticle synthesis assisted by machine learning.Nature Reviews Materials, 6:701–716, 2021

2021
[23]

Deep learning and computer vision techniques for microcirculation analysis: A review.Patterns, 4(1):100641, 2023

Maged Helmy, Trung Tuyen Truong, Eric Jul, and Paulo Ferreira. Deep learning and computer vision techniques for microcirculation analysis: A review.Patterns, 4(1):100641, 2023

2023
[24]

Deep learning methods for microstructural image anal- ysis: The state-of-the-art and future perspectives.Integrating Materials and Manufacturing Innovation, 13(3):703–731, 2024

Khaled Alrfou, Tian Zhao, and Amir Kordijazi. Deep learning methods for microstructural image anal- ysis: The state-of-the-art and future perspectives.Integrating Materials and Manufacturing Innovation, 13(3):703–731, 2024

2024
[25]

Overview: Machine learning for segmentation and classification of complex steel microstructures.Metals, 14(5):553, 2024

Martin M¨ uller, Marie Stiefel, Bj¨ orn-Ivo Bachmann, Dominik Britz, and Frank M¨ ucklich. Overview: Machine learning for segmentation and classification of complex steel microstructures.Metals, 14(5):553, 2024

2024
[26]

Corrao, Phillip M

Adam A. Corrao, Phillip M. Maffettone, Bruce Ravel, Thomas A. Caswell, Stuart I. Campbell, Howie Joress, Stuart Wilkins, and Daniel Olds. A modular framework for collaborative human-ai, multi-modal and multi-beamline synchrotron experiments, 2025

2025
[27]

Retrieval meets long context large language models

Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, and Bryan Catanzaro. Retrieval meets long context large language models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, edito...

2024
[28]

From retrieval to response: Tracing the impact of embedding quality in rag systems.IEEE Access, 13:212773–212781, 2025

Orlando Amaral Cejas, Yuejun Guo, and Qiang Tang. From retrieval to response: Tracing the impact of embedding quality in rag systems.IEEE Access, 13:212773–212781, 2025

2025
[29]

Risks of AI scientists: prioritizing safeguarding over autonomy.Nature Communications, 16:8317, 2025

Xiangru Tang, Qiao Jin, Kunlun Zhu, Tongxin Yuan, Yichi Zhang, Wangchunshu Zhou, Meng Qu, Yilun Zhao, Jian Tang, Zhuosheng Zhang, Arman Cohan, Dov Greenbaum, Zhiyong Lu, and Mark Gerstein. Risks of AI scientists: prioritizing safeguarding over autonomy.Nature Communications, 1...

2025
[30]

Heurekabench: A benchmarking framework for ai co-scientist, 2026

Siba Smarak Panigrahi, Jovana Videnovi´ c, and Maria Brbi´ c. Heurekabench: A benchmarking framework for ai co-scientist, 2026. 19

2026
[31]

Ingolf- sson, David Lenz, Hanqi Guo, Hongfeng Yu, Teja Leburu, Michael Molash, Bei Wang, Tom Peterka, Chaoli Wang, and Shusen Liu

Kuangshi Ai, Haichao Miao, Kaiyuan Tang, Nathaniel Gorski, Jianxin Sun, Guoxi Liu, Helgi I. Ingolf- sson, David Lenz, Hanqi Guo, Hongfeng Yu, Teja Leburu, Michael Molash, Bei Wang, Tom Peterka, Chaoli Wang, and Shusen Liu. Scivisagentbench: A benchmark for evaluating scientifi...

2026
[32]

M.T. Postek. Critical issues in scanning electron microscope metrology.Journal of Research of the National Institute of Standards and Technology, 99:641–671, Sep-Oct 1994

1994
[33]

Groenewald

Annemari Groenewald and Willem A. Groenewald. Development of a universal medical x-ray imaging phantom prototype.Journal of Applied Clinical Medical Physics, 17(6):356–365, 2016

2016
[34]

Christie, Wenhan Zheng, William Johnson, Eric K

Liam B. Christie, Wenhan Zheng, William Johnson, Eric K. Marecki, James Heidrich, Jun Xia, and Kwang W. Oh. Review of imaging test phantoms.Journal of Biomedical Optics, 28(8):080903, 2023

2023
[35]

ReAct: Synergizing reasoning and acting in language models

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, 2023

2023
[36]

Model context protocol specification.https:// modelcontextprotocol.io/specification/2025-06-18/, 2025

Model Context Protocol Contributors. Model context protocol specification.https:// modelcontextprotocol.io/specification/2025-06-18/, 2025. Version 2025-06-18. Accessed 2026- 05-11

2025
[37]

Introducing the model context protocol.https://www.anthropic.com/news/ model-context-protocol, November 2024

Anthropic. Introducing the model context protocol.https://www.anthropic.com/news/ model-context-protocol, November 2024. Accessed 2026-05-11. 20

2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [1]

Accelerating scientific discovery with Co-Scientist.Nature, 655:487–496, May 2026

Juraj Gottweis et al. Accelerating scientific discovery with Co-Scientist.Nature, 655:487–496, May 2026

work page 2026

[2] [2]

Szostkiewicz, Dmytro Shved, Gavin J

Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J. Szostkiewicz, Dmytro Shved, Gavin J. Gyimesi, Jon M. Laurent, Samantha M. Wright, Muhammed T. Razzak, Andrew D. White, Silvia C. Finnemann, Michaela M. Hinks, and Samuel G. Rodriques. A multi-agent system for automating scientific discovery.Nature, 655:497–505, May 2026

work page 2026

[3] [3]

Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D

Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. Augmenting large language models with chemistry tools.Nature Machine Intelligence, 6:525–535, 2024

work page 2024

[4] [4]

Accurate predictions on small data with a tabular foundation model.Nature, 637:319–326, 2025

Noah Hollmann, Samuel M¨ uller, Lennart Purucker, Arjun Krishnakumar, Max K¨ orfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model.Nature, 637:319–326, 2025

work page 2025

[5] [5]

Gilad Kusne, Ji-Cheng Zhao, and Ichiro Takeuchi

Haotong Liang, Chuangye Wang, Heshan Yu, Dylan Kirsch, Rohit Pant, Austin McDannald, A. Gilad Kusne, Ji-Cheng Zhao, and Ichiro Takeuchi. Real-time experiment-theory closed-loop interaction for autonomous materials science.Science Advances, 11(27):eadu7426, 2025

work page 2025

[6] [6]

Abolhasani and E

M. Abolhasani and E. Kumacheva. The rise of self-driving labs in chemical and materials sciences. Nature Synthesis, 2:483–492, 2023

work page 2023

[7] [7]

Wu, and Al´ an Aspuru-Guzik

Martin Seifrid, Robert Pollice, Andr´ es Aguilar-Granda, Zamyla Morgan Chan, Kazuhiro Hotta, Cher Tian Ser, Jenya Vestfrid, Tony C. Wu, and Al´ an Aspuru-Guzik. Autonomous chemical exper- iments: Challenges and perspectives on establishing a self-driving lab.Accounts of Chemical Research, 55(17):2454–2466, 2022. PMID: 35948428

work page 2022

[8] [8]

Schmid, Sterling G

Gary Tom, Stefan P. Schmid, Sterling G. Baird, Yang Cao, Kourosh Darvish, Han Hao, Stanley Lo, Sergio Pablo-Garc ´ ıa, Ella M. Rajaonson, Marta Skreta, Naruki Yoshikawa, Samantha Corapi, Gun Deniz Akkoc, Felix Strieth-Kalthoff, Martin Seifrid, and Al´ an Aspuru-Guzik. Self-driving laboratories for chemistry and materials science.Chemical Reviews, 124(16):...

work page 2024