Agentic-J: An AI Agent for Biological Microscopy Image Analysis
Pith reviewed 2026-06-28 11:44 UTC · model grok-4.3
The pith
A multi-agent AI assistant lets biologists describe microscopy analysis tasks in natural language and receive executable, traceable ImageJ scripts.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Agentic-J is a containerised multi-agent AI assistant primarily for ImageJ/Fiji that enables biologists to specify analysis tasks in natural language. The system generates executable scripts organised into a documented project structure so every analysis decision is traceable and the workflow can be reproduced or shared. Specialised sub-agents handle plugin management, code generation, debugging, quality assurance, and statistical reporting.
What carries the argument
The multi-agent architecture with sub-agents dedicated to plugin management, code generation, debugging, quality assurance, and statistical reporting.
If this is right
- Biologists without programming skills can perform advanced analyses like multi-condition quantification.
- Analysis workflows become fully reproducible and shareable through documented project structures.
- Every decision in the analysis process is traceable back to the generated code and documentation.
- Integration of heterogeneous tools and domain knowledge is automated by the agent system.
Where Pith is reading between the lines
- This design could be adapted to other biological analysis platforms beyond ImageJ.
- Automated quality assurance might help standardise statistical practices across labs.
- Natural language interfaces could lower the barrier for collaborative analysis in research teams.
Load-bearing premise
Natural language prompts and the multi-agent architecture can reliably produce correct, executable ImageJ scripts for diverse biological microscopy tasks without frequent human correction or hidden failures.
What would settle it
Running the system on a range of standard biological microscopy datasets and finding that the generated scripts frequently fail to execute or produce incorrect results requiring manual fixes.
read the original abstract
Biological image analysis increasingly demands integration across heterogeneous tools, programming environments, and domain knowledge that few researchers can command simultaneously. We present Agentic-J, a containerised, multi-agent AI assistant, primarily for ImageJ/Fiji that enables biologists to specify analysis tasks in natural language, from nuclei segmentation and cell tracking to multi-condition quantification. The agent generates executable scripts organised into a documented project structure, so every analysis decision is traceable and the workflow can be reproduced or shared. The specialised sub-agents handle plugin management, code generation, debugging, quality assurance, and statistical reporting. In this paper we introduce the system's design, demonstrate real biological microscopy image analysis workflows, and detailed the technical implementation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Agentic-J, a containerised multi-agent AI system for ImageJ/Fiji that lets biologists describe microscopy image analysis tasks (nuclei segmentation, cell tracking, multi-condition quantification) in natural language. Specialised sub-agents manage plugins, generate and debug code, perform QA, and produce statistical reports, outputting documented, traceable, and reproducible project structures.
Significance. If the multi-agent pipeline reliably converts natural-language prompts into correct, executable ImageJ scripts across diverse tasks without frequent human intervention, the system would lower the barrier for non-programmers to conduct reproducible biological image analysis and could accelerate adoption of advanced quantification methods. The containerised design and emphasis on traceability are practical strengths.
major comments (2)
- [Abstract] Abstract: the manuscript states that the system 'enables biologists to specify analysis tasks in natural language' and 'demonstrate[s] real biological microscopy image analysis workflows,' yet supplies no success rates, intervention counts, task-coverage statistics, error rates, or comparison against manual scripting baselines. This leaves the central claim that the sub-agent architecture reliably produces correct scripts untested.
- [Demonstrations] Demonstrations section (implied by the abstract's claim of real workflows): the absence of any quantitative validation (e.g., fraction of prompts that required human correction, plugin-compatibility failure modes, or agreement with ground-truth segmentations) makes it impossible to assess whether the weakest assumption—that NL prompts plus the multi-agent loop succeed without hidden failures—holds.
minor comments (1)
- [Abstract] Abstract: the sentence 'detailed the technical implementation' is grammatically incorrect and should read 'details the technical implementation.'
Simulated Author's Rebuttal
We thank the referee for these comments, which correctly identify a gap in quantitative evaluation. We will revise the manuscript to address both points by adding systematic metrics on system performance.
read point-by-point responses
-
Referee: [Abstract] Abstract: the manuscript states that the system 'enables biologists to specify analysis tasks in natural language' and 'demonstrate[s] real biological microscopy image analysis workflows,' yet supplies no success rates, intervention counts, task-coverage statistics, error rates, or comparison against manual scripting baselines. This leaves the central claim that the sub-agent architecture reliably produces correct scripts untested.
Authors: We agree that the abstract overstates reliability without supporting numbers. The original manuscript presents the system architecture and illustrative workflows rather than a benchmark study. In revision we will add a dedicated evaluation section reporting success rates, human intervention counts, task coverage, and common failure modes across a held-out prompt set, and will tone down the abstract accordingly. revision: yes
-
Referee: [Demonstrations] Demonstrations section (implied by the abstract's claim of real workflows): the absence of any quantitative validation (e.g., fraction of prompts that required human correction, plugin-compatibility failure modes, or agreement with ground-truth segmentations) makes it impossible to assess whether the weakest assumption—that NL prompts plus the multi-agent loop succeed without hidden failures—holds.
Authors: The demonstrations were provided as concrete examples of end-to-end usage rather than as a controlled study. We accept that this leaves the reliability claim untested. The revised version will expand the demonstrations section with quantitative results, including the fraction of prompts requiring correction, observed plugin-compatibility issues, and, where ground-truth annotations exist, agreement metrics with manual or other-tool segmentations. revision: yes
Circularity Check
No circularity: system description with no derivations or self-referential claims
full rationale
The paper is a design and implementation description of a multi-agent AI system for ImageJ/Fiji image analysis. It introduces architecture, sub-agents for plugin management/code generation/debugging/QA, and demonstrates workflows, but contains no equations, fitted parameters, predictions, uniqueness theorems, or derivation chains. No load-bearing steps reduce to self-citation or input-by-construction. The central claims are about system capabilities and reproducibility, which are presented as engineering outcomes rather than derived results. This matches the default expectation of no significant circularity for non-mathematical system papers.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Nature Methods 9(7), 676–682 (2012) https://doi.org/10.1038/nmeth.2019
Schindelin, J., Arganda-Carreras, I., Frise, E., Kaynig, V., Longair, M., Pietzsch, T., Preibisch, S., Rueden, C., Saalfeld, S., Schmid, B., Tinevez, J.-Y., White, D.J., Hartenstein, V., Eliceiri, K., Toman- cak, P., Cardona, A.: Fiji: an open-source platform for biological-image analysis. Nature Methods 9(7), 676–682 (2012) https://doi.org/10.1038/nmeth.2019
-
[2]
Nature Methods9(7), 671–675 (2012) https://doi.org/10.1038/nmeth.2089
Schneider, C.A., Rasband, W.S., Eliceiri, K.W.: NIH Image to ImageJ: 25 years of image analysis. Nature Methods9(7), 671–675 (2012) https://doi.org/10.1038/nmeth.2089
-
[3]
PLOS Biology17(6), 3000340 (2019) https://doi.org/10.1371/journal.pbio.3000340
Rueden, C.T., Ackerman, J., Arena, E.T., Eglinger, J., Cimini, B.A., Goodman, A., Carpenter, A.E., Eliceiri, K.W.: Scientific community image forum: A discussion forum for scientific image software. PLOS Biology17(6), 3000340 (2019) https://doi.org/10.1371/journal.pbio.3000340
-
[4]
bioRxiv (2024) https://doi.org/10.1101/2024.04.19.590278
Haase, R., Tischer, C., H´ erich´ e, J.-K., Scherf, N.: Benchmarking large language models for bio-image analysis code generation. bioRxiv (2024) https://doi.org/10.1101/2024.04.19.590278
-
[5]
https://arxiv.org/abs/2503.13399
Burgess, J., Nirschl, J.J., Bravo-S´ anchez, L., Lozano, A., Gupte, S.R., Galaz-Montoya, J.G., Zhang, Y., Su, Y., Bhowmik, D., Coman, Z., Hasan, S.M., Johannesson, A., Leineweber, W.D., Nair, M.G., Yarlagadda, R., Zuraski, C., Chiu, W., Cohen, S., Hansen, J.N., Leonetti, M.D., Liu, C., Lundberg, E., Yeung-Levy, S.: MicroVQA: A Multimodal Reasoning Benchma...
arXiv 2025
-
[6]
Beurer-Kellner, L., Buesser, B., Cret ¸u, A.-M., Debenedetti, E., Dobos, D., Fabian, D., Fischer, M., Froelicher, D., Grosse, K., Naeff, D., Ozoani, E., Paverd, A., Tram` er, F., Volhejn, V.: Design Patterns for Securing LLM Agents against Prompt Injections. arXiv (2025). https://doi.org/10. 48550/ARXIV.2506.08837
arXiv 2025
-
[7]
https://www.imagejpipelinebuilder.com Accessed 2026-05-22
Buswinka, C.: ImageJ Pipeline Builder (IJPB): Natural-language Image Analysis Automation for ImageJ. https://www.imagejpipelinebuilder.com Accessed 2026-05-22
2026
-
[8]
Chan Zuckerberg Biohub San Francisco
Theodoro, I., Royer Lab: napari-mcp: An MCP Server for AI-powered Napari Control. Chan Zuckerberg Biohub San Francisco. https://github.com/royerlab/napari-mcp Accessed 2026-05-22
2026
-
[9]
Nature Methods21, 1368–1370 (2024) https://doi.org/10.1038/s41592-024-02370-y
Lei, W., Fuster-Barcel´ o, C., Reder, G., Mu˜ noz-Barrutia, A., Ouyang, W.: BioImage.IO Chatbot: a community-driven AI assistant for integrative computational bioimaging. Nature Methods21, 1368–1370 (2024) https://doi.org/10.1038/s41592-024-02370-y
-
[10]
bioRxiv (2025) https://doi.org/10.1101/2025.05.30.656746
Huang, K., Zhang, S., Wang, H., Qu, Y., Lu, Y., Roohani, Y., Li, R., Qiu, L., Li, G., Zhang, J., Yin, D., Marwaha, S., Carter, J.N., Zhou, X., Wheeler, M., Bernstein, J.A., Wang, M., He, P., Zhou, J., Snyder, M., Cong, L., Regev, A., Leskovec, J.: Biomni: A general-purpose biomedical ai agent. bioRxiv (2025) https://doi.org/10.1101/2025.05.30.656746
-
[11]
https://platform.openai.com/docs/api-reference
OpenAI: OpenAI API. https://platform.openai.com/docs/api-reference. Accessed: 2026-05-26 (2026)
2026
-
[12]
https://openrouter.ai/docs
OpenRouter: OpenRouter API: A Unified Interface for LLMs. https://openrouter.ai/docs. Accessed: 2026-05-26 (2026)
2026
-
[13]
napari: a multi-dimensional image viewer for python,
Sofroniew, N., Lambert, T., Bokota, G., Nunez-Iglesias, J., Sobolewski, P., Sweet, A., Gaifas, L., Evans, K., Burt, A., Doncila Pop, D., Yamauchi, K., Weber Mendon¸ ca, M., Rodr´ ıguez-Guerra, J., Liu, L., Buckley, G., Vierdag, W.-M., Anderson, A., Monko, T., Willing, C., Rodr´ ıguez-Reza, C.M., Royer, L., Can Solak, A., Harrington, K.I.S., Abramo, J., Ah...
-
[14]
Scientific Reports13(1) (2023) https://doi.org/10.1038/s41598-023-50227-9
Holme, B., Bjørnerud, B., Pedersen, N.M., Ballina, L.R., Wesche, J., Haugsten, E.M.: Automated tracking of cell migration in phase contrast images with celltraxx. Scientific Reports13(1) (2023) https://doi.org/10.1038/s41598-023-50227-9
-
[15]
Ershov, D., Phan, M.-S., Pylv¨ an¨ ainen, J.W., Rigaud, S.U., Le Blanc, L., Charles-Orszag, A., Conway, J.R.W., Laine, R.F., Roy, N.H., Bonazzi, D., Dum´ enil, G., Jacquemet, G., Tinevez, J.-Y.: Bringing trackmate into the era of machine-learning and deep-learning (2021) https://doi.org/10.1101/2021. 09.03.458852
-
[16]
JCI Insight8(14) (2023) https://doi.org/10.1172/jci.insight.169020
Baars, I., Jaedtka, M., Dewitz, L.-A., Fu, Y., Franz, T., Mohr, J., Gintschel, P., Berlin, H., Degen, A., Freier, S., Rygol, S., Schraven, B., Kahlfuß, S., Zandbergen, G., M¨ uller, A.J.: Leishmania major drives host phagocyte death and cell-to-cell transfer depending on intracellular pathogen proliferation rate. JCI Insight8(14) (2023) https://doi.org/10...
-
[17]
IEEE Transactions on Image Processing7(1), 27–41 (1998)
Th´ evenaz, P., Ruttimann, U.E., Unser, M.: A pyramid approach to subpixel registration based on intensity. IEEE Transactions on Image Processing7(1), 27–41 (1998)
1998
-
[18]
Schmied, C., Nelson, M.S., Avilov, S., Bakker, G.-J., Bertocchi, C., Bischof, J., Boehm, U., Brocher, J., Carvalho, M.T., Chiritescu, C., Christopher, J., Cimini, B.A., Conde-Sousa, E., Ebner, M., Ecker, R., Eliceiri, K., Fernandez-Rodriguez, J., Gaudreault, N., Gelman, L., Grunwald, D., Gu, T., Halidi, N., Hammer, M., Hartley, M., Held, M., Jug, F., Kapo...
-
[19]
Contemporary Educational Technology17(2), 567 (2025) https://doi.org/10
Kiran, K., Saat, R.M., Demeester, L., Lew, M.D.N., Neo, W.L., Pausawasdi, N., Nopparatjamjomras, T.R.: Technology-enhanced personalized learning: Lessons from online teaching at three south- east asian universities. Contemporary Educational Technology17(2), 567 (2025) https://doi.org/10. 30935/cedtech/15946
2025
-
[20]
https://bioimagebook.github.io/
Bankhead, P.: Introduction to Bioimage Analysis. https://bioimagebook.github.io/. Accessed May 2026 (2023)
2026
-
[21]
https://doi.org/10.5281/zenodo.10465773
Haase, R., Lampert, M., Witz, G., Korten, T., Ankenbrand, M.J., Kugler, E., Zoccoler, M., Fernandes, M., Taylor, S., Ray, A.: haesleinhuepf/BioImageAnalysisNotebooks: Gpt-2024.1.6. https://doi.org/10.5281/zenodo.10465773 . CC-BY 4.0 & BSD-3. https://haesleinhuepf.github.io/ BioImageAnalysisNotebooks/intro.html 25
-
[22]
Social Sciences &; Humanities Open12, 101751 (2025) https://doi.org/10.1016/j.ssaho
N´ emeth, R., T´ atrai, A., Szab´ o, M., Zaletnyik, P.T., Tam´ asi, a.: Exploring the use of retrieval- augmented generation models in higher education: A pilot study on artificial intelligence-based tutoring. Social Sciences &; Humanities Open12, 101751 (2025) https://doi.org/10.1016/j.ssaho. 2025.101751
-
[23]
Nature Communications11(1) (2020) https://doi.org/10.1038/ s41467-020-14529-0
Roels, J., Vernaillen, F., Kremer, A., Gon¸ calves, A., Aelterman, J., Luong, H.Q., Goossens, B., Philips, W., Lippens, S., Saeys, Y.: An interactive imagej plugin for semi-automated image denoising in electron microscopy. Nature Communications11(1) (2020) https://doi.org/10.1038/ s41467-020-14529-0
2020
-
[24]
PLOS Computational Biology20(9), 1012375 (2024) https://doi.org/10.1371/journal.pcbi.1012375
Rokem, A.: Ten simple rules for scientific code review. PLOS Computational Biology20(9), 1012375 (2024) https://doi.org/10.1371/journal.pcbi.1012375
-
[25]
https://www
Riverbank Computing: PyQt: Python Bindings for the Qt Application Framework. https://www. riverbankcomputing.com/software/pyqt/. Accessed: 2026-05-26 (2026)
2026
-
[26]
Accessed: 2026
AI, L.: Deepagents. Accessed: 2026. https://github.com/langchain-ai/deepagents
2026
-
[27]
https://sites.imagej.net/sites.json
ImageJ Project: ImageJ Update Sites Metadata Index (sites.json). https://sites.imagej.net/sites.json. Data repository file; accessed May 2026 (2026)
2026
-
[28]
Bioinformatics32(22), 3532–3534 (2016) https://doi.org/10.1093/ bioinformatics/btw413
Legland, D., Arganda-Carreras, I., Andrey, P.: Morpholibj: integrated library and plugins for mathe- matical morphology with imagej. Bioinformatics32(22), 3532–3534 (2016) https://doi.org/10.1093/ bioinformatics/btw413
2016
-
[29]
In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018
Schmidt, U., Weigert, M., Broaddus, C., Myers, G.: Cell detection with star-convex polygons. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018. Lecture Notes in Computer Science, vol. 11071, pp. 265–273. Springer, Cham, Switzerland (2018). https://doi.org/ 10.1007/978-3-030-00934-2 30
-
[30]
Frontiers in Computer Science4 (2022) https://doi.org/10.3389/fcomp.2022.777728
Arzt, M., Deschamps, J., Schmied, C., Pietzsch, T., Schmidt, D., Tomancak, P., Haase, R., Jug, F.: Labkit: Labeling and segmentation toolkit for big image data. Frontiers in Computer Science4 (2022) https://doi.org/10.3389/fcomp.2022.777728
-
[31]
https://github.com/openclaw/openclaw
OpenClaw Team: OpenClaw. https://github.com/openclaw/openclaw
-
[32]
AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
Yang, C.: AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use (2026). https://doi.org/10.48550/arXiv.2605.04785
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2605.04785 2026
-
[33]
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., K¨ uttler, H., Lewis, M., Yih, W.-t., Rockt¨ aschel, T., Riedel, S., Kiela, D.: Retrieval-Augmented Generation for Knowledge- Intensive NLP Tasks. arXiv (2020). https://doi.org/10.48550/ARXIV.2005.11401
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2005.11401 2020
-
[34]
Cormack, G.V., Clarke, C.L.A., Buettcher, S.: Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR ’09, pp. 758–759. Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/...
-
[35]
Accessed: 2026-05-18
Anthropic: Agent Skills: a standardized way to give AI agents new capabilities and expertise. Accessed: 2026-05-18. https://agentskills.io/home (2026)
2026
-
[36]
Shankar, S., Zamfirescu-Pereira, J.D., Hartmann, B., Parameswaran, A.G., Arawjo, I.: Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences. arXiv 26 (2024). https://doi.org/10.48550/ARXIV.2404.12272
-
[37]
Stureborg, R., Alikaniotis, D., Suhara, Y.: Large Language Models are Inconsistent and Biased Evaluators (2024)
2024
-
[38]
Queen’s University Belfast, Belfast, UK (2014)
Bankhead, P.: Analyzing Fluorescence Microscopy Images with ImageJ. Queen’s University Belfast, Belfast, UK (2014). No DOI or ISBN issued – self-published handbook. Semantic Scholar Cor- pusID:18910833. Originally drafted at the Nikon Imaging Center, Heidelberg University (2010– 2012); later evolved into the online book at https://bioimagebook.github.io. ...
2014
-
[39]
(eds.): Bioimage Data Analysis Workflows – Advanced Components and Methods
Miura, K., Sladoje, N. (eds.): Bioimage Data Analysis Workflows – Advanced Components and Methods. Learning Materials in Biosciences. Springer, Cham, Switzerland (2022). https://doi.org/ 10.1007/978-3-030-76394-7 . Open-access; companion to the earlier Miura & Sladoje (2020) volume. https://link.springer.com/book/10.1007/978-3-030-76394-7
-
[40]
PLOS Biology21(6), 3002167 (2023) https://doi.org/10.1371/journal.pbio.3002167
Senft, R.A., Diaz-Rohrer, B., Colarusso, P., Swift, L., Jamali, N., Jambor, H., Pengo, T., Brideau, C., Montero Llopis, P., Uhlmann, V., Kirk, J., Gonzales, K.A., Bankhead, P., Evans III, E.L., Eli- ceiri, K.W., Cimini, B.A.: A biologist’s guide to planning and performing quantitative bioimaging experiments. PLOS Biology21(6), 3002167 (2023) https://doi.o...
-
[41]
A biologist’s guide to planning and performing quantitative bioimaging experiments
COBA-NIH and contributors: MicroscopyForBeginnersReferenceGuide: Companion website to “A biologist’s guide to planning and performing quantitative bioimaging experiments”. https://www.bioimagingguide.org/welcome.html. GitHub: https://github.com/COBA-NIH/ MicroscopyForBeginnersReferenceGuide; BSD-3 License. Companion to [40] (2023)
2023
-
[42]
National Institutes of Health, USA, (2012)
Ferreira, T., Rasband, W.S.: ImageJ User Guide – IJ 1.46r. National Institutes of Health, USA, (2012). National Institutes of Health, USA. Revised October 2012. https://imagej.net/ij/docs/guide/ user-guide.pdf
2012
-
[43]
Nowell, C.J.: Fiji Training Manual (v6.5). Monash University. Monash Institute of Pharmaceuti- cal Science; CC-BY 4.0 (2023). https://doi.org/10.26180/21901989 . https://bridges.monash.edu/ articles/educational resource/Fiji Training Manual v6 5 /21901989
-
[44]
https://neubias.github.io/ training-resources/
NEUBIAS Community: Bioimage Analysis Training Resources. https://neubias.github.io/ training-resources/. GitHub: https://github.com/NEUBIAS/training-resources; CC-BY 4.0. Water- shed module: https://neubias.github.io/training-resources/watershed/index.html (2019–2026)
2019
-
[45]
Bioinformatics32(22), 3532–3534 (2016) https://doi.org/10.1093/ bioinformatics/btw413
Legland, D., Arganda-Carreras, I., Andrey, P.: MorphoLibJ: integrated library and plugins for mathe- matical morphology with ImageJ. Bioinformatics32(22), 3532–3534 (2016) https://doi.org/10.1093/ bioinformatics/btw413
2016
-
[46]
F1000Research Slides 9:1378 (2020)
Legland, D., Arganda-Carreras, I., Andrey, P.: Mathematical morphology with morphoLibJ [NEU- BIAS Academy@Home webinar slides]. F1000Research Slides 9:1378 (2020). https://doi.org/10. 7490/f1000research.1118396.1 . https://f1000research.com/slides/9-1378
2020
-
[47]
F1000Research Slides 9:625
Boudier, T.: Introduction to 3D Analysis with 3D ImageJ Suite [NEUBIAS Academy@Home webinar slides]. F1000Research Slides 9:625. Slides also available at https://f1000research.com/slides/9-625; video at https://www.youtube.com/watch?v=OPC2kP-5By4 (2020). https://imagej.net/media/ plugins/3d-imagej-suite/neubias-3dsuite.pdf
2020
-
[48]
Bioinformatics29(14), 1840–1841 27 (2013) https://doi.org/10.1093/bioinformatics/btt276
Ollion, J., Cochennec, J., Loll, F., Escud´ e, C., Boudier, T.: TANGO: a generic tool for high- throughput 3D image analysis for studying nuclear organization. Bioinformatics29(14), 1840–1841 27 (2013) https://doi.org/10.1093/bioinformatics/btt276 . Primary citation requested by the 3D ImageJ Suite authors. 28
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.