REVIEW 3 major objections 4 minor 27 references
CytoFM: The first cytology foundation model
T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read CytoFM, the first cytology-specific foundation model, learns transferable cell representations that beat non-cytology foundation models on two of three downstream cytology classification tasks, including a dataset completely unseen during…
desk verdict A genuinely first cytology foundation model with a plausible transfer signal, but the same-dataset evaluation splits and missing error bars on the unseen test make the 'two of three' claim weaker than the abstract suggests. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is CytoFM, a ViT-Base feature extractor pretrained with iBOT. iBOT trains a teacher-student pair with two objectives: self-distillation, where the student's class token must match the teacher's predictions on a differently augmented view, and masked image modeling, where the student predicts patch tokens for masked regions from the teacher's unmasked embeddings. This pairing lets one frozen encoder represent both coarse context, such as cellularity and cell clusters, and fine detail, such as nuclear boundaries and mitotic activity. For evaluation, patch features are pooled by an attention-based multiple instance learning head, so predictions are made at the image or slide level without patch-level labels.
What would settle it
A direct test: retrain CytoFM with all MLBC and FNAC2019 images removed from the pretraining corpus and rerun the same downstream protocols. If the reported MLBC and FNAC2019 margins over UNI and iBOT-ImageNet shrink or disappear, then the apparent advantage comes from training/test distribution overlap rather than from cytology-specific representation learning; the HiCervix result alone would then be the only clean transfer evidence.
Extended reading notes
Core claim
CytoFM is a frozen ViT-Base encoder obtained by continuing iBOT pretraining on a multi-institutional, multi-organ cytology corpus. On a held-out split of the MLBC cervical dataset, it reaches 0.930 ± 0.05 accuracy and 0.993 ± 0.01 AUROC (area under the ROC curve), significantly above UNI (0.895/0.986) and iBOT-ImageNet (0.879/0.983). On the completely unseen HiCervix dataset, it reaches 0.844 accuracy and 0.968 AUROC, ahead of UNI (0.800/0.952) and iBOT-ImageNet (0.803/0.956). On FNAC2019 breast malignancy classification it scores 0.908 accuracy and 0.979 AUROC, the lowest of the three models, which the authors attribute to breast cytology being only a small fraction of the pretraining data. The paper reads these results, together with attention maps that highlight nuclei, mitotic figures, and cell boundaries, as evidence that cytology-specific self-supervised pretraining captures features that general-purpose encoders miss.
Load-bearing premise
The evaluation assumes that splitting the same dataset into disjoint pretraining and test portions measures generalization; because the pretraining and held-out FNAC2019 and MLBC images come from the same acquisition and labeling process, shared staining and scanner characteristics could inflate CytoFM's scores on those two tasks.
Editorial extensions
If this is right
- CytoFM's frozen features place it ahead of both baselines on cervical cell typing (MLBC accuracy 0.930, AUROC 0.993), so practitioners using a cytology-specific backbone may not need to train separate encoders for each cervical cytology task.
- On a dataset the model never saw (HiCervix), CytoFM still leads (accuracy 0.844, AUROC 0.968), suggesting the learned features transfer across institutions and scanners without stain normalization or fine-tuning.
- On breast FNAC2019, where breast patches were only a small part of pretraining, CytoFM trails iBOT-ImageNet; the paper's explanation implies that balancing organ representation in pretraining should improve cross-organ performance.
- The reported gains justify further investment in larger, more balanced cytology pretraining corpora and in probing which cytology-specific features, such as mitotic activity and nuclear boundaries, drive the downstream improvements.
Reading between the lines
- A stronger test of the 'cytology-specific' claim would evaluate CytoFM on an organ absent from pretraining, such as urine or lung cytology; HiCervix is unseen but still cervical, so it mainly proves cross-institution transfer within the dominant organ.
- The two partially held-out evaluations could be rerun with pretraining ablations that remove MLBC and FNAC2019 entirely, separating the effects of dataset overlap from genuine transfer.
- If the HiCervix margin holds under such ablations, cytology foundation models could become the default frozen backbones for rare cytology tasks where labels are too scarce to train a supervised model from scratch.
- The attention maps point to a concrete clinical extension the paper does not pursue: using CytoFM's attention as a screening signal for mitotic activity, which could be validated against expert annotations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript describes CytoFM, a ViT-Base encoder pretrained with iBOT on approximately 1.4 million image patches from seven public cytology datasets and one private thyroid dataset. The pretrained encoder is frozen and its features are aggregated with an attention-based multiple instance learning (ABMIL) framework for three downstream tasks: malignancy classification on FNAC2019 and cell-type classification on MLBC and HiCervix. The authors compare CytoFM against iBOT-ImageNet and UNI and report that CytoFM outperforms both baselines on MLBC and HiCervix but underperforms on FNAC2019. They also provide UMAP visualizations of feature embeddings and attention maps as qualitative evidence that CytoFM learns cytologically relevant features.
Significance. If the empirical claims hold, CytoFM would be a useful contribution as the first cytology-specific self-supervised foundation model. The comparison against a histopathology foundation model (UNI) and a natural-image model (iBOT-ImageNet) is appropriate and informative. Strengths of the paper include the use of multiple public datasets, the inclusion of a completely unseen evaluation dataset (HiCervix), significance testing over 100 random splits for two of the three tasks, and the qualitative UMAP and attention-map analyses. However, the evaluation protocol does not yet fully support the central 'two of three' generalization claim: the two partially held-out datasets use same-dataset image-level splits rather than patient-level splits, and the clean out-of-distribution result on HiCervix is reported from a single fixed split without confidence intervals or a significance test. These issues are fixable with additional experiments and statistical reporting, but they are load-bearing for the main claim.
major comments (3)
- [§3.1, Table 2] The held-out portions of FNAC2019 and MLBC are formed by splitting the same datasets used for pretraining into 'two non-overlapping sets' at the image or patch level, not at the patient level. MLBC is described as containing 963 images from 460 patients, so the current split can place images from the same patient or the same scanner/stain protocol into both the pretraining set and the test set. Self-supervised pretraining can memorize site-specific acquisition artifacts, which would inflate CytoFM's performance on MLBC (accuracy 0.930 versus 0.895 for UNI and 0.879 for iBOT-ImageNet) relative to an unseen dataset. This is the load-bearing weakness identified by the stress-test concern, and it does land: the MLBC advantage cannot be cleanly attributed to transferable cytology knowledge without a patient-level split or an equivalent control. Please report results with patient-level splits for FNAC2019 and MLBC, or otherwise demonstrate that the image-level split does not create distributional overlap.
- [§3.2, Table 2] The HiCervix result is the cleanest out-of-distribution evidence for the generalization claim, yet it is reported from a single fixed train/validation/test split with no confidence intervals, error bars, or significance test. The observed accuracy gap (0.844 for CytoFM versus 0.803 for iBOT-ImageNet and 0.800 for UNI) is promising, but it cannot by itself carry the 'two of three' claim. Please add bootstrap confidence intervals or repeated-seed variability estimates and perform a statistical comparison against the baselines on the same fixed split. Without this, the HiCervix gain is not quantitatively established.
- [§2.2, Figure 2; §4] The private thyroid dataset is filtered with the custom ThyVGG model using a top-1500 patch threshold, and the paper acknowledges in §4 that breast cytology contributes only 2,544 images while cervical cytology contributes close to a million. Since no downstream thyroid evaluation or ablation is presented, the claim that CytoFM learns representations generalizing across organs is not currently supported by the experiments. Either add a thyroid evaluation or soften the cross-organ generalization claim in the introduction and discussion.
minor comments (4)
- [§2.1] The spelling 'Papanicolau' should be 'Papanicolaou' in the description of the FNAC2019 dataset.
- [§4] The method name 'HeirSwin' appears to be a typo for 'HierSwin'; please correct it and ensure the reference matches the cited work.
- [§3.2, Figure 3] The attention maps in Figure 3 are selected examples and may not be representative; a quantitative analysis, such as attention-agreement with cell-level annotations, would strengthen the claim that CytoFM attends to cytologically relevant features.
- [General] The manuscript does not mention a plan to release code, model weights, or the private thyroid dataset; given the emphasis on reproducibility and downstream reuse, a statement about data and code availability would be helpful.
Circularity Check
No significant circularity: CytoFM's claims are empirical comparisons with disclosed pretraining; the same-dataset held-out splits are a generalization limitation, not a definitional or fitted-input reduction.
full rationale
This paper is an empirical model-comparison study, not a derivation, and I find no step in which a claimed prediction reduces by construction to a fitted parameter, a self-definition, or a load-bearing self-citation. CytoFM is pretrained with iBOT, a self-supervised method, on unlabeled cytology patches; the downstream ABMIL classifiers are trained separately on labels. No downstream label is used to fit the feature extractor, and no fitted quantity is renamed as a prediction. The held-out FNAC2019 and MLBC portions come from the same datasets used in pretraining, which raises a legitimate distributional-leakage concern about generalization, but this is an evaluation-design weakness rather than circular reasoning: the paper states the datasets are 'split into two non-overlapping sets' (Section 3.1), the compared models iBOT-ImageNet and UNI are evaluated under the same protocol, and the HiCervix dataset is 'completely unseen during model training' (Section 3.2). ImageNet-1K initialization is disclosed in Section 2.3. The only self-referential element is 'our ThyVGG model' used for patch filtering of the private thyroid data; that is a preprocessing utility and is not load-bearing for the central representation-learning claim. There are no uniqueness theorems imported from the authors, no ansatz smuggled in via self-citation, and no renaming of a known result as a new organization. The paper even acknowledges its limitation on FNAC2019 and compares HiCervix performance to an existing state of the art. Under the stated rules, absence of a quoted reduction means the correct circularity score is 0.
Assumptions & free parameters
free parameters (2)
- Top-1500 patch threshold for thyroid filtering =
1500
- Magnification estimate based on nucleus pixel size =
not reported
assumptions (3)
- domain assumption Patch-based analysis at 40x magnification captures the cell-level information needed for cytology classification.
- domain assumption Self-supervised pretraining with iBOT on unlabeled cytology patches yields features that transfer to downstream tasks.
- domain assumption The held-out splits from the same datasets (FNAC2019, MLBC) are representative independent test sets.
Cite this review
Pith. "Pith review of CytoFM: The first cytology foundation model." pith.science (2026). https://pith.science/paper/36FIWZME
@misc{pith2026250413402,
author = {Pith},
title = {Pith review of: CytoFM: The first cytology foundation model},
year = {2026},
howpublished = {\url{https://pith.science/paper/36FIWZME}},
note = {Machine review of arXiv:2504.13402}
}
read the original abstract
Cytology is essential for cancer diagnostics and screening due to its minimally invasive nature. However, the development of robust deep learning models for digital cytology is challenging due to the heterogeneity in staining and preparation methods of samples, differences across organs, and the limited availability of large, diverse, annotated datasets. Developing a task-specific model for every cytology application is impractical and non-cytology-specific foundation models struggle to generalize to tasks in this domain where the emphasis is on cell morphology. To address these challenges, we introduce CytoFM, the first cytology self-supervised foundation model. Using iBOT, a self-supervised Vision Transformer (ViT) training framework incorporating masked image modeling and self-distillation, we pretrain CytoFM on a diverse collection of cytology datasets to learn robust, transferable representations. We evaluate CytoFM on multiple downstream cytology tasks, including breast cancer classification and cell type identification, using an attention-based multiple instance learning framework. Our results demonstrate that CytoFM performs better on two out of three downstream tasks than existing foundation models pretrained on histopathology (UNI) or natural images (iBOT-Imagenet). Visualizations of learned representations demonstrate our model is able to attend to cytologically relevant features. Despite a small pre-training dataset, CytoFM's promising results highlight the ability of task-agnostic pre-training approaches to learn robust and generalizable features from cytology data.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
- [1]
-
[2]
HiCervix: An Extensive Hierarchical Dataset and Benchmark for Cervi- cal Cytology Classification
De Cai, Jie Chen, Junhan Zhao, Yuan Xue, Sen Yang, Wei Yuan, Min Feng, Haiyan Weng, Shuguang Liu, Yulong Peng, Junyou Zhu, Kanran Wang, Christopher Jackson, Hongping Tang, Junzhou Huang, and Xiyue Wang. HiCervix: An Extensive Hierarchical Dataset and Benchmark for Cervi- cal Cytology Classification. IEEE transactions on medical imaging, 43(12):4344–4355, 2024. 4
work page 2024
-
[3]
Chen, Chengkuan Chen, Yicong Li, Tiffany Y
Richard J. Chen, Chengkuan Chen, Yicong Li, Tiffany Y . Chen, Andrew D. Trister, Rahul G. Krishnan, and Faisal Mahmood. Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning, 2022. arXiv:2206.02647 [cs]. 2
arXiv 2022
-
[4]
Richard J. Chen, Tong Ding, Ming Y . Lu, Drew F. K. Williamson, Guillaume Jaume, Andrew H. Song, Bowen Chen, Andrew Zhang, Daniel Shao, Muhammad Shaban, Mane Williams, Lukas Oldenburg, Luca L. Weishaupt, Judy J. Wang, Anurag Vaidya, Long Phi Le, Georg Ger- ber, Sharifa Sahai, Walt Williams, and Faisal Mahmood. To- wards a general-purpose foundation model ...
work page 2024
-
[5]
Ex- ploring Foundation Models Fine-Tuning for Cytology Clas- sification, 2024
Manon Dausort, Tiffanie Godelaine, Maxime Zanella, Karim El Khoury, Isabelle Salmon, and Beno ˆıt Macq. Ex- ploring Foundation Models Fine-Tuning for Cytology Clas- sification, 2024. arXiv:2411.14975 [eess]. 2
arXiv 2024
-
[6]
Kovalsky, Serge Assaad, Jonathan Cohen, Danielle Elliott Range, Avani A
David Dov, Shahar Z. Kovalsky, Serge Assaad, Jonathan Cohen, Danielle Elliott Range, Avani A. Pendse, Ricardo Henao, and Lawrence Carin. Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images. Medical Image Analysis, 67:101814,
-
[7]
Towards Artificial Intelligence Applications in Next Generation Cytopathology
Enrico Giarnieri and Simone Scardapane. Towards Artificial Intelligence Applications in Next Generation Cytopathology. Biomedicines, 11(8), 2023. 1
work page 2023
-
[8]
Pixel-wise segmen- tation of cells in digitized Pap smear images.Scientific Data, 11(1):733, 2024
Balazs Harangi, Gergo Bogacsovics, Janos Toth, Ilona Ko- vacs, Erzsebet Dani, and Andras Hajdu. Pixel-wise segmen- tation of cells in digitized Pap smear images.Scientific Data, 11(1):733, 2024. 3
work page 2024
Show all 27 references
-
[9]
An efficient framework based on large foun- dation model for cervical cytopathology whole slide image screening, 2024
Jialong Huang, Gaojie Li, Shichao Kan, Jianfeng Liu, and Yixiong Liang. An efficient framework based on large foun- dation model for cervical cytopathology whole slide image screening, 2024. arXiv:2407.11486 [cs]. 2
2024 arXiv
-
[10]
Mahanta, Himakshi Borah, and Chandana Ray Das
Elima Hussain, Lipi B. Mahanta, Himakshi Borah, and Chandana Ray Das. Liquid based-cytology Pap smear dataset for automated multi-class diagnosis of pre-cancerous and cervical cancer lesions. Data in Brief, 30:105589, 2020. 3
2020
-
[11]
USFM: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis
Jing Jiao, Jin Zhou, Xiaokang Li, Menghua Xia, Yi Huang, Lihong Huang, Na Wang, Xiaofan Zhang, Shichong Zhou, Yuanyuan Wang, and Yi Guo. USFM: A universal ultrasound foundation model generalized to tasks and organs towards label efficient image analysis. Medical Image Analysis...
2024
-
[12]
Landau and Liron Pantanowitz
Michael S. Landau and Liron Pantanowitz. Artificial in- telligence in cytopathology: a review of the literature and overview of commercial landscape. Journal of the American Society of Cytopathology, 8(4):230–241, 2019. 1
2019
-
[13]
L. J. Layfield, E. E. Mooney, B. Glasgow, S. Hirschowitz, and A. Coogan. What constitutes an adequate smear in fine- needle aspiration cytology of the breast? Cancer, 81(1): 16–21, 1997. 1
1997
-
[14]
Local Label Point Correction for Edge Detection of Overlapping Cervical Cells
Jiawei Liu, Huijie Fan, Qiang Wang, Wentao Li, Yandong Tang, Danbo Wang, Mingyi Zhou, and Li Chen. Local Label Point Correction for Edge Detection of Overlapping Cervical Cells. Frontiers in Neuroinformatics, 16, 2022. Publisher: Frontiers. 3
2022
-
[15]
Whole-slide imaging in cytopathology: state of the art and future directions
Stefano Marletta, Darren Treanor, Albino Eccher, and Liron Pantanowitz. Whole-slide imaging in cytopathology: state of the art and future directions. Diagnostic Histopathology, 27(11):425–430, 2021. 1
2021
-
[16]
DINOv2: Learning Robust Visual Features without Supervision, 2024
Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mah- moud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michae...
2024 arXiv
-
[17]
Conventional Cervical Cytology Image Dataset with Cell Outline Annotations
Antonina Pater, Krzysztof Siemion, Karol Deptuch, Łukasz Roszkowiak, Jakub ˙Zak, Katarzyna Jakubowska, Stanisław Sulkowski, Marek Baltaziak, Mariusz Koda, and Anna Ko- rzy´nska. Conventional Cervical Cytology Image Dataset with Cell Outline Annotations. In2023 International Sy...
2023
-
[18]
Plissiti, P
Marina E. Plissiti, P. Dimitrakopoulos, G. Sfikas, Christophoros Nikou, O. Krikoni, and A. Charchanti. Sipakmed: A New Dataset for Feature and Image Based Classification of Normal and Pathological Cervical Cells in Pap Smear Images. In 2018 25th IEEE International Conference o...
2018
-
[19]
Renshaw, Dawn Underwood, Ghada Aramoni, Beverly Cash, Maureen Croyle, Dave Deeds, Sandra Dolar, Stephen Gmitro, Nancy Ray, Debbie Sabo, Julie A
Andrew A. Renshaw, Dawn Underwood, Ghada Aramoni, Beverly Cash, Maureen Croyle, Dave Deeds, Sandra Dolar, Stephen Gmitro, Nancy Ray, Debbie Sabo, Julie A. Shorie, Bridgette Springer, Dana Weber Moffsinger, and Tarik M. Elsheikh. Time consumed by microscopic and nonmicro- scopi...
2016
-
[20]
Mahanta, and Anup Kumar Das
Amartya Ranjan Saikia, Kangkana Bora, Lipi B. Mahanta, and Anup Kumar Das. Comparative assessment of CNN ar- chitectures for classification of breast FNAC images. Tissue and Cell, 57:8–14, 2019. 3
2019
-
[21]
Kunz, Matthew C
Eugene V orontsov, Alican Bozkurt, Adam Casson, George Shaikovski, Michal Zelechowski, Kristen Severson, Eric Zimmermann, James Hall, Neil Tenenholtz, Nicolo Fusi, 8 Ellen Yang, Philippe Mathieu, Alexander van Eck, Donghun Lee, Julian Viret, Eric Robert, Yi Kan Wang, Jeremy D....
2024
-
[22]
Celeste Welch, Chenhao Lu, C
E. Celeste Welch, Chenhao Lu, C. James Sung, Cunxian Zhang, Anubhav Tripathi, and Joyce Ou. BMT: A Cross- Validated ThinPrep Pap Cervical Cytology Dataset for Ma- chine Learning Model Training and Validation. Scientific Data, 11(1):1444, 2024. 2, 3
2024
-
[23]
Summers, Shaoxiong Liu, and Jianhua Yao
Ling Zhang, Le Lu, Isabella Nogues, Ronald M. Summers, Shaoxiong Liu, and Jianhua Yao. DeepPap: Deep Convolu- tional Networks for Cervical Cell Classification. IEEE Jour- nal of Biomedical and Health Informatics, 21(6):1633–1643,
-
[24]
LESS: Label- efficient multi-scale learning for cytological whole slide im- age screening
Beidi Zhao, Wenlong Deng, Zi Han (Henry) Li, Chen Zhou, Zuhua Gao, Gang Wang, and Xiaoxiao Li. LESS: Label- efficient multi-scale learning for cytological whole slide im- age screening. Medical Image Analysis, 94:103109, 2024. 1, 2
2024
-
[25]
iBOT: Image BERT Pre- Training with Online Tokenizer, 2022
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong. iBOT: Image BERT Pre- Training with Online Tokenizer, 2022. arXiv:2111.07832 [cs]. 2, 5 9
2022 arXiv
-
[2017]
arXiv:1801.08616 [cs]. 2
-
[2018]
ISSN: 2381-8549. 2, 3
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.