Detecting immune cells with label-free two-photon autofluorescence and deep learning
Pith reviewed 2026-05-19 09:19 UTC · model grok-4.3
The pith
A convolutional neural network classifies immune cell types from label-free two-photon autofluorescence images.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors trained a SqueezeNet convolutional neural network on label-free multiphoton microscopy images of immune cells and showed that it can classify cell types based on their autofluorescence from metabolic proteins. For binary classification in mixed samples using 5,075 cells, the model reached 0.89 ROC-AUC and 0.95 PR-AUC. For six-class classification on 3,424 isolated cells, it achieved an F1 score of 0.689. Tests showed the predictions depend equally on the NADH and FAD channels and are not misled by extracellular material.
What carries the argument
A low-complexity SqueezeNet CNN that takes two-channel autofluorescence images from NADH and FAD as input and outputs cell type classifications.
If this is right
- Such models could enable direct detection of specific immune cells in unstained images during in vivo endomicroscopy.
- The approach computationally augments the specificity of label-free multiphoton microscopy.
- Both autofluorescence channels contribute similarly to accurate classification.
- The model maintains performance in mixed cell samples, suggesting it handles realistic imaging conditions.
Where Pith is reading between the lines
- Extending the training data to include cells from more donors or disease states could test broader applicability.
- Integrating this classification into real-time imaging systems might allow immediate feedback during procedures.
- Similar deep learning augmentation could be applied to other label-free optical techniques beyond multiphoton microscopy.
Load-bearing premise
The autofluorescence signals from different immune cell types are distinct and consistent enough that a neural network can learn reliable patterns for classification on unseen samples.
What would settle it
Retraining and testing the same architecture on a completely new dataset of immune cells imaged under different conditions or from different subjects, and finding performance drops to near-random levels, would falsify the claim.
Figures
read the original abstract
Label-free imaging has gained broad interest because of its potential to omit elaborate staining procedures which is especially relevant for in vivo use. Label-free multiphoton microscopy (MPM), for instance, exploits two-photon excitation of natural autofluorescence (AF) from native, metabolic proteins, making it ideal for in vivo endomicroscopy. Deep learning (DL) models have been widely used in other optical imaging technologies to predict specific target annotations and thereby digitally augment the specificity of these label-free images. However, this computational specificity has only rarely been implemented for MPM. In this work, we used a data set of label-free MPM images from a series of different immune cell types (5,075 individual cells for binary classification in mixed samples and 3,424 cells for a multi-class classification task) and trained a convolutional neural network (CNN) to classify cell types based on this label-free AF as input. A low-complexity squeezeNet architecture was able to achieve reliable immune cell classification results (0.89 ROC-AUC, 0.95 PR-AUC, for binary classification in mixed samples; 0.689 F1 score, 0.697 precision, 0.748 recall, and 0.683 MCC for six-class classification in isolated samples). Perturbation tests confirmed that the model is not confused by extracellular environment and that both input AF channels (NADH and FAD) are about equally important to the classification. In the future, such predictive DL models could directly detect specific immune cells in unstained images and thus, computationally improve the specificity of label-free MPM which would have great potential for in vivo endomicroscopy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes training a low-complexity SqueezeNet CNN on label-free two-photon autofluorescence images (NADH and FAD channels) to classify immune cell types. It reports 0.89 ROC-AUC and 0.95 PR-AUC for binary classification on 5,075 cells in mixed samples, and 0.689 F1 score (0.697 precision, 0.748 recall, 0.683 MCC) for six-class classification on 3,424 cells in isolated samples, with perturbation tests confirming both channels contribute and the model is not driven by extracellular material.
Significance. If the autofluorescence signatures prove cell-type specific and stable, the work could advance label-free in vivo endomicroscopy by adding computational specificity without staining. The empirical design with concrete held-out metrics and explicit perturbation tests for channel importance and extracellular confusion is a positive feature that supports the central claim within the reported dataset.
major comments (2)
- [Results] The evaluation uses held-out cells but provides no donor-wise or preparation-wise cross-validation. This is load-bearing for the generalizability claim because isolation method, activation state, and donor variability could introduce batch-correlated features that the model exploits instead of intrinsic NADH/FAD metabolic patterns. A leave-one-donor-out protocol or equivalent would directly test whether the reported 0.89 ROC-AUC and 0.689 F1 generalize to new biological samples.
- [Results] The six-class F1 of 0.689 is only moderately above chance levels for six categories; without per-class precision/recall or a confusion matrix it is unclear which cell types drive the errors and whether the moderate aggregate score undermines the claim of reliable multi-class classification from AF alone.
minor comments (2)
- The abstract and summary would be strengthened by stating the number of donors or biological replicates and the exact train/validation/test split strategy (random cell-level vs. donor-stratified).
- Figure legends or methods should clarify image preprocessing steps (e.g., normalization, patch extraction) to allow reproduction of the input to SqueezeNet.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed review. We address each major comment point-by-point below and indicate the revisions we will make to the manuscript.
read point-by-point responses
-
Referee: [Results] The evaluation uses held-out cells but provides no donor-wise or preparation-wise cross-validation. This is load-bearing for the generalizability claim because isolation method, activation state, and donor variability could introduce batch-correlated features that the model exploits instead of intrinsic NADH/FAD metabolic patterns. A leave-one-donor-out protocol or equivalent would directly test whether the reported 0.89 ROC-AUC and 0.689 F1 generalize to new biological samples.
Authors: We agree that donor- and preparation-wise cross-validation would strengthen the generalizability claim. Our current results use random held-out cell splits across the pooled dataset of mixed and isolated samples. In the revised manuscript we will add a preparation-wise evaluation (training on mixed samples and testing on isolated samples, and vice versa) and report the resulting metrics. If donor identifiers are available in the underlying dataset we will also implement leave-one-donor-out cross-validation; otherwise we will explicitly note this as a limitation of the present study. revision: partial
-
Referee: [Results] The six-class F1 of 0.689 is only moderately above chance levels for six categories; without per-class precision/recall or a confusion matrix it is unclear which cell types drive the errors and whether the moderate aggregate score undermines the claim of reliable multi-class classification from AF alone.
Authors: We thank the referee for this observation. While an F1 of 0.689 is substantially higher than the random baseline of ~0.167 for six classes, we agree that aggregate metrics alone are insufficient. In the revised manuscript we will add a confusion matrix together with per-class precision, recall, and F1 scores for the six-class task so that readers can directly see which cell types are well classified and which contribute most to the errors. revision: yes
Circularity Check
No circularity: standard empirical ML evaluation on held-out cell images
full rationale
The manuscript describes collection of label-free two-photon autofluorescence images from immune cells, followed by supervised training of a SqueezeNet CNN and reporting of classification metrics (ROC-AUC, F1, etc.) on held-out test cells. No equations, first-principles derivations, or predictions are presented that reduce to fitted parameters or self-citations by construction. Performance numbers are direct empirical outputs from data splits and perturbation tests; the pipeline contains no load-bearing self-referential steps.
Axiom & Free-Parameter Ledger
free parameters (1)
- SqueezeNet model weights
axioms (1)
- domain assumption Autofluorescence patterns from NADH and FAD are sufficiently distinct and consistent for different immune cell types to enable classification
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
A low-complexity squeezeNet architecture was able to achieve reliable immune cell classification results (0.89 ROC-AUC... both input AF channels (NADH and FAD) are about equally important
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Reconstructing cell cycle and disease progression using deep learning,
P. Eulenberg, N. Köhler, T. Blasi,et al., “Reconstructing cell cycle and disease progression using deep learning,” Nat. communications 8, 463 (2017)
work page 2017
-
[2]
Evaluating the utility of brightfield image data for mechanism of action prediction,
P. J. Harrison, A. Gupta, J. Rietdijk,et al., “Evaluating the utility of brightfield image data for mechanism of action prediction,” PLOS Comput. Biol.19, e1011323 (2023)
work page 2023
-
[3]
X. Yang, D. Chen, Q. Sun,et al., “A live-cell image-based machine learning strategy for reducing variability in psc differentiation systems,” Cell discovery9, 53 (2023)
work page 2023
-
[4]
H. Su, Z. Yin, S. Huh, and T. Kanade, “Cell segmentation in phase contrast microscopy images via semi-supervised classification over optics-related features,” Med. image analysis17, 746–765 (2013)
work page 2013
-
[5]
H. Niioka, S. Asatani, A. Yoshimura,et al., “Classification of c2c12 cells at differentiation by convolutional neural network of deep learning using phase contrast images,” Hum. cell31, 87–93 (2018)
work page 2018
-
[6]
M. Kang and J. Kim, “Cancer cell classification based on morphological features of 3d phase contrast microscopy using deep neural network,” IEEE Access (2025)
work page 2025
-
[7]
Cell membrane tracking in living brain tissue using differential interference contrast microscopy,
J. Lee, I. Kolb, C. R. Forest, and C. J. Rozell, “Cell membrane tracking in living brain tissue using differential interference contrast microscopy,” IEEE Trans. on Image Process.27, 1847–1861 (2017)
work page 2017
-
[8]
An automatic cell segmentation method for differential interference contrast microscopy,
A. Kuijper and B. Heise, “An automatic cell segmentation method for differential interference contrast microscopy,” in 2008 19th International Conference on Pattern Recognition,(IEEE, 2008), pp. 1–4
work page 2008
-
[9]
F. Pan, Y. Wu, K. Cui,et al., “Accurate detection and instance segmentation of unstained living adherent cells in differential interference contrast images,” Comput. Biol. Med.182, 109151 (2024)
work page 2024
-
[10]
Bacterialcellidentificationindifferentialinterferencecontrast microscopy images,
B.Obara,M.A.Roberts,J.P.Armitage,andV.Grau,“Bacterialcellidentificationindifferentialinterferencecontrast microscopy images,” BMC bioinformatics14, 1–13 (2013)
work page 2013
-
[11]
T. Ogawa, K. Ochiai, T. Iwata,et al., “Different cell imaging methods did not significantly improve immune cell image classification performance,” Plos one17, e0262397 (2022)
work page 2022
-
[12]
C. L. Curl, C. J. Bellair, T. Harris,et al., “Refractive index measurement in viable cells using quantitative phase- amplitude microscopy and confocal microscopy,” Cytom. Part A: The J. Int. Soc. for Anal. Cytol.65, 88–92 (2005)
work page 2005
-
[13]
Quantitative phase imaging with broadband fields,
Z. Wang and G. Popescu, “Quantitative phase imaging with broadband fields,” Appl. Phys. Lett.96 (2010)
work page 2010
-
[14]
Artificial intelligence-enabled quantitative phase imaging methods for life sciences,
J. Park, B. Bai, D. Ryu,et al., “Artificial intelligence-enabled quantitative phase imaging methods for life sciences,” Nat. Methods20, 1645–1660 (2023)
work page 2023
-
[15]
Quantitative phase imaging: recent advances and expanding potential in biomedicine,
T. L. Nguyen, S. Pradeep, R. L. Judson-Torres,et al., “Quantitative phase imaging: recent advances and expanding potential in biomedicine,” ACS nano16, 11516–11544 (2022)
work page 2022
-
[16]
Automatic classification of red blood cell morphology based on quantitative phase imaging,
M. Jiang, M. Shao, X. Yang,et al., “Automatic classification of red blood cell morphology based on quantitative phase imaging,” Int. J. Opt.2022, 1240020 (2022)
work page 2022
-
[17]
V. K. Lam, T. Nguyen, V. Bui,et al., “Quantitative scoring of epithelial and mesenchymal qualities of cancer cells using machine learning and quantitative phase imaging,” J. biomedical optics25, 026002–026002 (2020)
work page 2020
-
[18]
Quantitative phase microscopy spatial signatures of cancer cells,
D. Roitshtain, L. Wolbromsky, E. Bal,et al., “Quantitative phase microscopy spatial signatures of cancer cells,” Cytom. Part A91, 482–493 (2017)
work page 2017
-
[19]
V. Ayyappan, A. Chang, C. Zhang,et al., “Identification and staging of b-cell acute lymphoblastic leukemia using quantitative phase imaging and machine learning,” ACS sensors5, 3281–3289 (2020)
work page 2020
-
[20]
Label-freesinglecellphenotypingtodeterminetumorcellheterogeneity in pancreatic cancer in real-time,
K.Wittenzellner,M.Lengl,S.Röhrl, etal.,“Label-freesinglecellphenotypingtodeterminetumorcellheterogeneity in pancreatic cancer in real-time,” JCI insight (2025)
work page 2025
-
[21]
T. Go, H. Byeon, and S. J. Lee, “Label-free sensor for automatic identification of erythrocytes using digital in-line holographic microscopy and machine learning,” Biosens. Bioelectron.103, 12–18 (2018)
work page 2018
-
[22]
Automated stain-free holographic image-based phenotypic classification of elliptical cancer cells,
K. Jaferzadeh, S. Son, A. Rehman,et al., “Automated stain-free holographic image-based phenotypic classification of elliptical cancer cells,” Adv. Photonics Res.4, 2200043 (2023)
work page 2023
-
[23]
M. E. Kandel, Y. R. He, Y. J. Lee,et al., “Phase imaging with computational specificity (pics) for measuring dry mass changes in sub-cellular compartments,” Nat. communications11, 6256 (2020)
work page 2020
-
[24]
Nonlinear magic: multiphoton microscopy in the biosciences,
W. R. Zipfel, R. M. Williams, and W. W. Webb, “Nonlinear magic: multiphoton microscopy in the biosciences,” Nat. biotechnology21, 1369–1377 (2003)
work page 2003
-
[25]
Q. Yu and A. A. Heikal, “Two-photon autofluorescence dynamics imaging reveals sensitivity of intracellular nadh concentration and conformation to cell physiology at the single-cell level,” J. Photochem. Photobiol. B: Biol.95, 46–57 (2009)
work page 2009
-
[26]
Optical imaging unveiling metabolic dynamics in cells and organisms during aging and diseases,
L. Shi and J. Villazon, “Optical imaging unveiling metabolic dynamics in cells and organisms during aging and diseases,” Med-X3, 1–24 (2025)
work page 2025
-
[27]
L. Shi, L. Lu, G. Harvey,et al., “Label-free fluorescence spectroscopy for detecting key biomolecules in brain tissue from a mouse model of alzheimer’s disease,” Sci. reports7, 2599 (2017)
work page 2017
-
[28]
Optical imaging of metabolic dynamics in animals,
L. Shi, C. Zheng, Y. Shen,et al., “Optical imaging of metabolic dynamics in animals,” Nat. communications9, 2995 (2018)
work page 2018
-
[29]
Label-freeinvivohistopathologyofexperimentalcolitisvia3-channel multiphoton endomicroscopy,
L.Kreiß,O.-M.Thoma,A.Dilipkumar, etal.,“Label-freeinvivohistopathologyofexperimentalcolitisvia3-channel multiphoton endomicroscopy,” Gastroenterology159, 832–834 (2020)
work page 2020
-
[30]
L. Kreiss, O.-M. Thoma, S. Lemire,et al., “Label-free characterization and quantification of mucosal inflammation in common murine colitis models with multiphoton imaging,” Inflamm. Bowel Dis.28, 1637–1646 (2022)
work page 2022
-
[31]
A. Schmalzl, T. Leupold, L. Kreiss,et al., “Interferon regulatory factor 1 (irf-1) promotes intestinal group 3 innate lymphoid responses during citrobacter rodentium infection,” Nat. Commun.13, 5730 (2022)
work page 2022
-
[32]
K. Sommer, K. Heidbreder, L. Kreiss,et al., “Anti-𝛽7 integrin treatment impedes the recruitment on non-classical monocytes to the gut and delays macrophage-mediated intestinal wound healing,” Clin. Transl. Med.13, e1233 (2023)
work page 2023
-
[33]
A. Habibalahi, A. G. Anwer, A. Knab,et al., “Multispectral autofluorescence for label free classification of immune cell type and activation/polarization status,” Scand. J. Immunol.101, e70004 (2025)
work page 2025
-
[34]
V. S. Shah, J. Hou, V. Vinarsky,et al., “Autofluorescence imaging permits label-free cell type assignment and reveals the dynamic formation of airway secretory cell associated antigen passages (saps),” Elife12, e84375 (2023)
work page 2023
-
[35]
S. Lemire, O.-M. Thoma, L. Kreiss,et al., “Natural nadh and fad autofluorescence as label-free biomarkers for discriminating subtypes and functional states of immune cells,” Int. J. Mol. Sci.23 (2022)
work page 2022
-
[36]
Metabolic profiling of single cells by exploiting nadh and fad fluorescence via flow cytometry,
A. H. Abir, L. Weckwerth, A. Wilhelm,et al., “Metabolic profiling of single cells by exploiting nadh and fad fluorescence via flow cytometry,” Mol. Metab.87, 101981 (2024)
work page 2024
-
[37]
Y. Zhang, K. de Haan, Y. Rivenson,et al., “Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue,” Light. Sci. & Appl.9, 78 (2020)
work page 2020
-
[38]
Virtualhistologicalstainingofunlabelledtissue-autofluorescenceimages via deep learning,
Y.Rivenson,H.D.Wang,Z.S.Wei, etal.,“Virtualhistologicalstainingofunlabelledtissue-autofluorescenceimages via deep learning,” Nat. Biomed. Eng.3, 466–477 (2019)
work page 2019
-
[39]
Digital staining in optical microscopy using deep learning-a review,
L. Kreiss, S. Jiang, X. Li,et al., “Digital staining in optical microscopy using deep learning-a review,” PhotoniX4, 34 (2023)
work page 2023
-
[40]
U. Gehlsen, M. Szaszak, A. Gebert,et al., “Non-invasive multi-dimensional two-photon microscopy enables optical fingerprinting (tpof) of immune cells,” J. Biophotonics8, 466–479 (2015)
work page 2015
-
[41]
T. W. Bocklitz, F. S. Salah, N. Vogler,et al., “Pseudo-he images derived from cars/tpef/shg multimodal imaging in combination with raman-spectroscopy as a pathological screening tool,” BMC cancer16, 1–11 (2016)
work page 2016
-
[42]
P. Pradhan, T. Meyer, M. Vieth,et al., “Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning,” Biomed. Opt. Express12, 2280–2298 (2021)
work page 2021
-
[43]
Fiber-optic scanning two-photon fluorescence endoscope,
M. T. Myaing, D. J. MacDonald, and X. Li, “Fiber-optic scanning two-photon fluorescence endoscope,” Opt. letters 31, 1076–1078 (2006)
work page 2006
-
[44]
H. Bao, J. Allen, R. Pattie,et al., “Fast handheld two-photon fluorescence microendoscope with a 475𝜇m× 475 𝜇m field of view for in vivo imaging,” Opt. letters33, 1333–1335 (2008)
work page 2008
-
[45]
Minimallyinvasivehigh-speedimagingofsarcomere contractile dynamics in mice and humans,
M.E.Llewellyn,R.P.Barretto,S.L.Delp,andM.J.Schnitzer,“Minimallyinvasivehigh-speedimagingofsarcomere contractile dynamics in mice and humans,” Nature454, 784–788 (2008)
work page 2008
-
[46]
Compact and flexible raster scanning multiphoton endoscope capable of imaging unstained tissue,
D. R. Rivera, C. M. Brown, D. G. Ouzounov,et al., “Compact and flexible raster scanning multiphoton endoscope capable of imaging unstained tissue,” Proc. National Acad. Sci.108, 17598–17603 (2011)
work page 2011
-
[47]
In vivo imaging of unstained tissues using long gradient index lens multiphoton endoscopic systems,
D. M. Huland, C. M. Brown, S. S. Howard,et al., “In vivo imaging of unstained tissues using long gradient index lens multiphoton endoscopic systems,” Biomed. optics express3, 1077–1085 (2012)
work page 2012
-
[48]
Label-free multiphoton endomicroscopy for minimally invasive in vivo imaging,
A. Dilipkumar, A. Al-Shemmary, L. Kreiß,et al., “Label-free multiphoton endomicroscopy for minimally invasive in vivo imaging,” Adv. Sci.6, 1801735 (2019)
work page 2019
-
[49]
Visualizing unstained neurons in living brain slices by infrared dic- videomicroscopy,
H.-U. Dodt and W. Zieglgänsberger, “Visualizing unstained neurons in living brain slices by infrared dic- videomicroscopy,” Brain research537, 333–336 (1990)
work page 1990
-
[50]
Distinctive image features from scale-invariant keypoints,
D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. journal computer vision60, 91–110 (2004)
work page 2004
- [51]
-
[52]
F. Iandola,Exploring the design space of deep convolutional neural networks at large scale(University of California, Berkeley, 2016)
work page 2016
-
[53]
Adam: A method for stochastic optimization,
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” (2017)
work page 2017
-
[54]
Averaging weights leads to wider optima and better generalization,
P. Izmailov, D. Podoprikhin, T. Garipov,et al., “Averaging weights leads to wider optima and better generalization,” (2019)
work page 2019
-
[55]
A multiple instance learning approach for detecting covid-19 in peripheral blood smears,
C. L. Cooke, K. Kim, S. Xu,et al., “A multiple instance learning approach for detecting covid-19 in peripheral blood smears,” PLOS Digit. Health1, e0000078 (2022)
work page 2022
-
[56]
A. Somani, A. A. Sekh, I. S. Opstad,et al., “Virtual labeling of mitochondria in living cells using correlative imaging and physics-guided deep learning,” Biomed. Opt. Express13, 5495–5516 (2022)
work page 2022
-
[57]
Chapter 2 - principles of immunophenotyping,
F. Naeim, P. Nagesh Rao, S. X. Song, and R. T. Phan, “Chapter 2 - principles of immunophenotyping,” inAtlas of Hematopathology (Second Edition),F. Naeim, P. Nagesh Rao, S. X. Song, and R. T. Phan, eds. (Academic Press, 2018), pp. 29–56, second edition ed
work page 2018
-
[58]
L. Haber, K. Olson, M. P. Kelly,et al., “Generation of t-cell-redirecting bispecific antibodies with differentiated profiles of cytokine release and biodistribution by cd3 affinity tuning,” Sci. Reports11, 14397 (2021)
work page 2021
-
[59]
Applications and challenges of ai and microscopy in life science research: A review,
H. Buckchash, G. K. Verma, and D. K. Prasad, “Applications and challenges of ai and microscopy in life science research: A review,” arXiv preprint arXiv:2501.13135 (2025). Fig. 5. Histograms of class labels and learning curves from binary classification of cells in mixture (a) and from multi-class classification from isolated cells (b). Supplemental mater...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.