Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Frozen vision foundation models plus a lightweight attentive probe can match or beat fully supervised deep learning in microscopy classification using as few as 100 annotated pixels.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 17:46 UTC pith:OTPMPMFU

load-bearing objection A genuinely useful benchmark with a practical message, but the domain-specific VFM rankings need a pretraining-overlap check. the 3 major comments →

arxiv 2603.19802 v2 pith:OTPMPMFU submitted 2026-03-20 cs.CV

Evaluating Vision Foundation Models for Pixel and Object Classification in Microscopy

Carolin Teuber , Anwai Archit , Tobias Boothe , Peter Ditte , Jochen Rink , Constantin Pape This is my paper
classification cs.CV
keywords vision foundation modelspixel classificationobject classificationmicroscopyattentive probingrandom forestlabel efficiencysemantic segmentation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether vision foundation models (large pretrained image encoders such as SAM and DINOv3) can replace hand-crafted features in the interactive pixel and object classification workflows commonly used for microscopy. On five diverse datasets, it finds that VFM embeddings consistently outperform classical filter-bank features when fed into a random forest, and that training a small attention probe (DeAP for pixels, ObAP for objects) on top of frozen VFM features is dramatically more label-efficient: 100 annotated pixels often match or exceed a random forest trained on 100,000 pixels, and on several datasets probe-based models surpass fully supervised U-Net and ResNet baselines. The authors conclude that VFMs provide a practical path to label-efficient classification in microscopy, with the caveat that the best VFM differs by learning strategy.

Core claim

The central claim is that frozen VFM image encoders are strong feature extractors for both dense pixel classification and object-level classification in microscopy, and that attentive probing — a small cross-attention module with Gaussian locality masks trained on the frozen features — extracts far more signal from very few labels than either a random forest on the same features or a fully supervised model trained from scratch. Concretely, DeAP (Dense Attentive Probing) with 100 annotated pixels matches or exceeds an RF with 100,000 pixels on LIVECell and CRC, and the best DeAP/ObAP models beat the fully supervised U-Net/ResNet on LIVECell, PanNuke, and CRC. The paper also finds that domain-

What carries the argument

The central object is the vision foundation model (VFM) image encoder, kept frozen and used as a feature extractor for microscopy images. On top of it sit two learning strategies: (1) a random forest trained on per-pixel or per-object embeddings (with AnyUp for resolution restoration), and (2) attentive probing — DeAP for pixels and its object-level extension ObAP — which initialize learnable queries (on a grid or at object centers) that attend to the frozen feature volume through a Gaussian-masked cross-attention with a learnable bandwidth σ, feeding into a small MLP/CNN head. The attention mask enforces spatial locality, and the learnable σ adapts the receptive field to object size.

Load-bearing premise

The conclusion that domain-specific VFMs are generally best for these tasks assumes their pretraining data did not include the evaluation datasets; the paper never states whether µSAM or PathoSAM were trained on LIVECell, PanNuke, CRC, or HBM images.

What would settle it

Check the public training-data documentation for µSAM and PathoSAM and determine whether any images or cells from LIVECell, PanNuke, CRC, or HBM appear in their pretraining sets. If they do, re-run the benchmarking with VFMs pretrained on disjoint data (or a fresh microscopy dataset) and see whether the domain-specific advantage disappears. Another direct test: evaluate on a dataset type not represented in any VFM pretraining and see whether VFM features still beat classical features at 100 pixels.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Interactive microscopy tools can be built on VFM features, giving users much larger label-efficiency than classical filter-bank features while keeping the lightweight random forest classifier.
  • At 100 annotated pixels, attentive probing can match or beat a random forest trained on 100,000 pixels, which suggests that near-interactive deep-learning classification is possible in practice.
  • On LIVECell, PanNuke, and CRC, the best attentive probing models exceed fully supervised U-Net or ResNet baselines, showing that frozen VFM features carry enough signal to avoid end-to-end training.
  • The choice of VFM depends on the learning strategy: domain-specific models (µSAM, PathoSAM) win with random forests, while SAM2 wins with attentive probing; DINOv3 lags both, so not all VFMs transfer equally to microscopy.
  • Because only the small adapter is trained, adding VFM-based classification to existing software is computationally light at inference and compatible with interactive annotation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves open whether the domain-specific VFMs (µSAM, PathoSAM) saw any of the evaluation datasets (LIVECell, PanNuke, CRC, HBM) during pretraining; if they did, their edge over general VFMs could reflect data leakage rather than domain adaptation, and this should be checked before generalizing the conclusion.
  • The preprocessing that sums 50+ fluorescent channels into a single channel for the CODEX datasets likely discards most of the biologically relevant signal; a channel-aware foundation model may close the remaining gap to fully supervised baselines, which the authors themselves note.
  • The label-efficiency numbers suggest a hybrid workflow: users annotate quickly with a random forest on VFM features, then fine-tune an attentive probe for higher accuracy, a combination the paper mentions but does not implement or test.
  • DeAP/ObAP training still takes GPU time comparable to a fully supervised network, so the 'interactive' advantage currently belongs only to the random forest variant; making the probe trainable on CPU is a concrete engineering target that would extend interactivity to the high-accuracy regime.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper evaluates whether vision foundation models (VFMs) can improve pixel and object classification in microscopy, which currently rely heavily on hand-crafted features and shallow learning. The authors compare features from general-purpose VFMs (SAM, SAM2, DINOv3) and domain-specific VFMs (µSAM, PathoSAM) when used as input to a random forest, and also evaluate dense attentive probing (DeAP) for pixel classification and a new object-guided attentive probing (ObAP) for object classification. Five public microscopy datasets are used with fixed splits. The central claims are: (1) VFM-based features consistently outperform ilastik's hand-crafted features for both pixel and object classification; (2) domain-specific VFMs generally perform best when combined with random forests, while SAM2 performs best with attentive probing; (3) DeAP and ObAP consistently outperform the random forest baselines, often matching or exceeding fully supervised U-Net/ResNet baselines at very low annotation budgets. The paper includes ablations on feature upsampling and object feature aggregation, and reports runtimes to assess interactivity.

Significance. If the results hold, the paper provides a practically useful benchmark and design guidance for integrating VFMs into interactive microscopy analysis tools such as ilastik. The study covers five diverse public datasets, uses fixed splits, includes multiple repeats for random forest baselines, and compares against strong classical and supervised baselines. These are clear strengths. The central claims are well-motivated and the paper is generally clearly written. However, the main limitations are that the attentive probing results (DeAP, ObAP) are based on single training runs, and that the potential overlap between the pretraining data of µSAM/PathoSAM and the evaluation datasets is never addressed. Both issues are load-bearing for the paper's strongest conclusions about consistency and domain-specific advantage. The study would be significantly strengthened by addressing these points, but the overall direction and empirical findings are valuable to the community.

major comments (3)
  1. [Sec. 4.2 (paragraph 'With Random Forest') and Sec. 4.3 (paragraph 'With Random Forest')] The conclusion that 'the domain-specific one generally performs best' depends on µSAM and PathoSAM being domain-adapted rather than having seen the evaluation data. The manuscript never states the pretraining corpora for these two models. The only related statement is that µSAM was not trained on data similar to Planari (Sec. 4.2); no such statement is given for LIVECell, PanNuke, CRC, or HBM. Since µSAM is trained on microscopy and PathoSAM on histopathology, their strong performance on these datasets could be due to overlap between pretraining and evaluation data, making the benchmark ranking circular. This is a load-bearing concern for the main generalization claim. Please specify the pretraining data for both models, and if overlap exists, either exclude the affected datasets from the headline comparisons or include an analysis that separates memorization from transfer. A concrete te
  2. [Sec. 4.2 ('With DeAP') and Sec. 4.3 ('With ObAP')] DeAP and ObAP are trained only once ('DeAP training is not repeated due to its long runtime'; 'we train each ObAP model only once due to long training'), yet the text makes strong, unqualified claims: 'DeAP consistently outperforms Random Forest-based methods across all experiments' and 'SAM2 achieves the highest overall performance'. With a single seed, these observations could be outliers, especially at very low annotation budgets (100 pixels/objects). In contrast, the random forest and U-Net/ResNet baselines have 5–25 repeats. Please provide multiple seeds (e.g., 3–5) for the key comparisons and report means and variances, or at minimum soften the claims to reflect that statistical significance was not assessed. This is necessary to support the 'consistent' and 'highest' language in the abstract and results.
  3. [Sec. 3.2 and Sec. 4.2 (RF vs. DeAP comparison)] The comparison between random forest and DeAP conflates feature resolution with learning strategy. The RF pipeline resizes embeddings to 256×256 (Sec. 3.2), whereas DeAP processes images at 1024×1024 with a query grid at 1/8 resolution (Sec. 3.2, 'Pixel Classification with DeAP'). Consequently, the reported advantages of DeAP over RF could be driven by higher-resolution features rather than by the attentive probing mechanism itself. A controlled comparison would require evaluating the RF on features at the same resolution as DeAP (or DeAP at the RF resolution). Please add an ablation that isolates the effect of feature resolution on the RF baseline, or explicitly discuss this confound and its implications for the 'DeAP outperforms RF' claim.
minor comments (5)
  1. [Abstract] The abstract lists SAM3 and KRONOS among the evaluated models, but the Methods and Results sections only include SAM, SAM2, DINOv3, µSAM, and PathoSAM. Either evaluate SAM3 and KRONOS or remove them from the abstract to avoid inconsistency.
  2. [Table 1] The LIVECell row reports training/validation/test splits of 3158/569/1512 images, which sum to 5239, while Sec. 4.1 states the dataset contains 'over 3158 images'. Please reconcile these numbers; if the dataset has more images, state the correct total.
  3. [App. 5.5, Table 7] The ablation shows that mean-only aggregation is best on PanNuke, while mean+area is best on LIVECell. The paper then adopts mean+area for all experiments. This is a reasonable choice, but the selection should be acknowledged as a potential source of overfitting to LIVECell; reporting the variance across the five folds would help assess its stability.
  4. [Sec. 4.2] The sentence 'Random forests are trained on 102,10^3,10^4, and 10^5 pixels' contains a typo: '102' should be '10^2' (i.e., 100).
  5. [Fig. 9] The y-axis is not labeled. The caption says 'Differences in mean F1 scores', but the reader cannot tell whether these are absolute differences, percentage points, or relative improvement. Add an axis label and units.

Circularity Check

0 steps flagged

No circular derivation; empirical benchmark with frozen VFM features and held-out test splits. Caveats about pretraining overlap and configuration selection are correctness risks, not circularity.

full rationale

This is an empirical evaluation rather than a derivation chain. The VFM encoders are frozen pretrained models, the random forests and DeAP/ObAP probes are trained on sparse labels, and all reported F1 scores are computed on fixed held-out test pixels/objects. No equation in the paper makes a predicted quantity equal to an input by construction, and no fitted parameter is renamed as a prediction. The main circularity-adjacent concern is that the domain-specific models (µSAM, PathoSAM) are developed by the same group and their pretraining corpora are not documented relative to the evaluation datasets; the paper only states that µSAM was not trained on Planari-like data. If µSAM or PathoSAM were pretrained on LIVECell, PanNuke, CRC, or HBM, the 'domain-specific model performs best' conclusion could partly reflect data leakage. That is a generalizability/validity threat, not a case where the result reduces to its input by definition. Likewise, the choice of AnyUp upsampling and of mean+area object-feature aggregation was made using the same datasets (App. 5.5), which can introduce selection bias, but the final comparisons still measure genuine classifier behavior on held-out labels. Self-citations to µSAM, PathoSAM, FeatureForest, and the SAM3-in-microscopy study are references to externally available models/prior studies rather than load-bearing uniqueness or ansatz arguments. Hence no specific circular step can be quoted, and the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

No theoretical derivation is claimed; every empirical component rests on external pretrained features and standard classifiers. The most consequential unstated assumptions are that domain-specific VFMs do not overlap with evaluation data and that automatic instance masks used for object classification are accurate enough. The paper also makes post-hoc configuration choices (AnyUp, mean+area features) using the same datasets reported as main results.

axioms (5)
  • domain assumption Frozen VFM encoders produce dense features that are informative for pixel- and object-level classification after upsampling.
    The entire method relies on using VFM embeddings as input features; supported empirically on five datasets, but not proven. Sec. 3.2.
  • domain assumption Object classification can be separated from segmentation: instance masks are available or can be generated automatically.
    The object classification method requires instance masks; for CRC and Planari masks are generated with µSAM/SAM, and errors propagate. Sec. 3.3, Sec. 4.1.
  • domain assumption Summing all channels of CODEX images into one channel preserves enough information for classification.
    The paper states this 'does not exploit channel-specific information' and that it contributed to low quality on CRC/HBM; Discussion, Sec. 5 and App. 5.2.
  • domain assumption AnyUp upsampling generalizes across VFMs and image domains without retraining.
    AnyUp is pretrained and applied without fine-tuning; the choice is justified mainly by a gain on LIVECell. Sec. 3.1, App. 5.5.
  • domain assumption The chosen datasets, pixel/object budgets, class-balanced sampling, and fixed splits represent realistic interactive microscopy use.
    The evaluation protocol is designed to mimic interactive annotation, but no user-in-the-loop study is performed; Appendix 5.1 acknowledges this.

pith-pipeline@v1.3.0-alltime-deepseek · 15257 in / 11251 out tokens · 110152 ms · 2026-08-02T17:46:52.732312+00:00 · methodology

0 comments
read the original abstract

Deep learning underlies most modern approaches and tools in computer vision, including biomedical imaging. However, for interactive semantic segmentation (often called pixel classification in this context) and interactive object-level classification (object classification), feature-based shallow learning remains widely used. This is due to the diversity of data in this domain, the lack of large pretraining datasets, and the need for computational and label efficiency. In contrast, state-of-the-art tools for many other vision tasks in microscopy - most notably cellular instance segmentation - already rely on deep learning and have recently benefited substantially from vision foundation models (VFMs), particularly SAM. Here, we investigate whether VFMs can also improve pixel and object classification compared to current approaches. To this end, we evaluate several VFMs, including general-purpose models (SAM, SAM2, SAM3, DINOv3) and domain-specific ones ($\mu$SAM, PathoSAM, KRONOS), in combination with shallow learning and attentive probing on five diverse and challenging datasets. Our results demonstrate consistent improvements over hand-crafted features and provide a clear pathway toward practical improvements. Furthermore, our study establishes a benchmark for VFMs in microscopy and informs future developments in this area.

Figures

Figures reproduced from arXiv: 2603.19802 by Anwai Archit, Carolin Teuber, Constantin Pape, Jochen Rink, Peter Ditte, Tobias Boothe.

Figure 1
Figure 1. Figure 1: Left [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: For a given image, the VFM predicts dense embeddings. Pixel classification: sparse annotations are projected onto the embedding space, the resulting per-pixel fea￾tures and labels are used to train a random forest (RF). During inference, the classifier predicts per-pixel labels, shown as semantic segmentation. Object classification: in￾stance masks are used to aggregate embeddings to obtain object-level fe… view at source ↗
Figure 3
Figure 3. Figure 3: Object-guided attentive probing (ObAP): A VFM (frozen) extracts a feature volume from the input image. Instead of a regular grid of queries, one query is ini￾tialized per object via its center, encoded by a fixed sinusoidal positional encoding. The object queries attend to the feature volume via Gaussian-masked cross-attention with a learnable bandwidth parameter σ, yielding one token per object, which are… view at source ↗
Figure 4
Figure 4. Figure 4: Pixel classification with a random forest across different training set sizes. Error bars based on 25 runs (5 folds, 5 repeats), 5 repeats for the U-Net, which is trained on all data. Bottom right: training (solid lines) and inference (dashed lines) times in seconds, comparing random forest (SAM features) and U-Net on LIVECell. details) as input to a random forest. Second, a U-Net, consisting of four down￾… view at source ↗
Figure 5
Figure 5. Figure 5: Pixel classification with DeAP (solid lines) compared to random forest (dash￾dotted lines). Bottom right: training (solid lines) and inference (dashed lines) times in seconds, comparing DeAP (SAM) with U-Net on LIVECell. With DeAP: We further evaluate pixel classification with DeAP (Sec. 3.1) [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Object classification with a random forest, comparing VFMs, classical features and a ResNet for different training set sizes. Error bars are derived from 25 runs (five folds, five repeats). Bottom right: training (solid lines) and inference times (dash￾dotted lines) of the ResNet and random forest (SAM features) on LIVECell [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance of Object-Guided Attentive Probing for object classification (solid lines) in comparison to random forest-based object classification (dash-dotted lines). Performance is reported in F1 Score and the number of objects used for training is shown on the x-axis. The bottom right plot shows training (solid lines) and inference (dashed lines) times in seconds, comparing Dense Attentive Probing with S… view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative results for pixel- and object classification across datasets using domain-specific SAM features. For object classification, models were trained using 100 annotated objects; pixel classification models were trained with 10000 annotated pixels. For object classification, the semantic segmentations are obtained by projecting the predicted object classes onto instances (ground-truth instance masks … view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of standard upsampling via interpolation and AnyUp upsampling for embedding-based feature extraction. Differences in mean F1 scores across training runs are shown for all datasets. AnyUp provides a noticeable performance gain on LIVECell, with no significant differences observed on the other datasets [PITH_FULL_IMAGE:figures/full_fig_p028_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Nature methods22(3), 579–591 (2025)

    Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., Freitag, M., Teuber, C., Spitzner, M., Tapia Contreras, C., et al.: Segment anything for microscopy. Nature methods22(3), 579–591 (2025)

    2025

  2. [2]

    In: Medical Imaging with Deep Learning (2026)

    Archit, A., Pape, C.: Revisiting foundation models for cell instance segmentation. In: Medical Imaging with Deep Learning (2026)

    2026

  3. [3]

    Bioinformatics33(15), 2424–2426 (2017)

    Arganda-Carreras, I., Kaynig, V., Rueden, C., Eliceiri, K.W., Schindelin, J., Car- dona, A., Sebastian Seung, H.: Trainable weka segmentation: a machine learning tool for microscopy pixel classification. Bioinformatics33(15), 2424–2426 (2017)

    2017

  4. [4]

    Frontiers in computer science4, 777728 (2022)

    Arzt, M., Deschamps, J., Schmied, C., Pietzsch, T., Schmidt, D., Tomancak, P., Haase, R., Jug, F.: Labkit: labeling and segmentation toolkit for big image data. Frontiers in computer science4, 777728 (2022)

    2022

  5. [5]

    Cell187(12), 3120– 3140.e29 (Jun 2024).https://doi.org/10.1016/j.cell.2024.04.013,http: //dx.doi.org/10.1016/j.cell.2024.04.013

    Bandyopadhyay, S., Duffy, M.P., Ahn, K.J., Sussman, J.H., Pang, M., Smith, D., Duncan, G., Zhang, I., Huang, J., Lin, Y., Xiong, B., Imtiaz, T., Chen, C.H., Thadi, A., Chen, C., Xu, J., Reichart, M., Martinez, Z., Diorio, C., Chen, C., Pillai, V., Snaith, O., Oldridge, D., Bhattacharyya, S., Maillard, I., Carroll, M., Nelson, C., Qin, L., Tan, K.: Mapping...

    2024

  6. [6]

    Scientific re- ports7(1), 1–7 (2017)

    Bankhead, P., Loughrey, M.B., Fernández, J.A., Dombrowski, Y., McArt, D.G., Dunne, P.D., McQuaid, S., Gray, R.T., Murray, L.J., Coleman, H.G., et al.: Qupath: Open source software for digital pathology image analysis. Scientific re- ports7(1), 1–7 (2017)

    2017

  7. [7]

    Nature methods16(12), 1226–1232 (2019)

    Berg, S., Kutra, D., Kroeger, T., Straehle, C.N., Kausler, B.X., Haubold, C., Schiegg, M., Ales, J., Beier, T., Rudy, M., et al.: Ilastik: interactive machine learn- ing for (bio) image analysis. Nature methods16(12), 1226–1232 (2019)

    2019

  8. [8]

    arXiv preprint arXiv:2504.13181 (2025)

    Bolya, D., Huang, P.Y., Sun, P., Cho, J.H., Madotto, A., Wei, C., Ma, T., Zhi, J., Rajasegaran, J., Rasheed, H., et al.: Perception encoder: The best visual em- beddings are not at the output of the network. arXiv preprint arXiv:2504.13181 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    arXiv preprint arXiv:2511.16719 (2025)

    Carion, N., Gustafson, L., Hu, Y.T., Debnath, S., Hu, R., Suris, D., Ryali, C., Alwala,K.V.,Khedr,H.,Huang,A.,etal.:Sam3:Segmentanythingwithconcepts. arXiv preprint arXiv:2511.16719 (2025)

    Pith/arXiv arXiv 2025

  10. [10]

    Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., Joulin, A.: Emerging properties in self-supervised vision transformers (2021).https:// doi.org/10.48550/ARXIV.2104.14294,https://arxiv.org/abs/2104.14294

    Pith/arXiv arXiv doi:10.48550/arxiv.2104.14294 2021

  11. [11]

    Nature medicine30(3), 850–862 (2024)

    Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Song, A.H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al.: Towards a general-purpose foundation model for computational pathology. Nature medicine30(3), 850–862 (2024)

    2024

  12. [12]

    Couairon, P., Chambon, L., Serrano, L., Haugeard, J.E., Cord, M., Thome, N.: Jafar: Jack up any feature at any resolution (2025).https://doi.org/10.48550/ ARXIV.2506.11136,https://arxiv.org/abs/2506.11136

    doi:10.48550/arxiv.2506.11136 2025

  13. [13]

    URL https://arxiv

    Deitke, M., Clark, C., Lee, S., Tripathi, R., Yang, Y., Park, J., Salehi, M., Muen- nighoff, N., Lo, K., Soldaini, L., et al.: Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models, 2024. URL https://arxiv. org/abs/2409.17146 (2024)

    Pith/arXiv arXiv 2024

  14. [14]

    In: 2009 IEEE Conference on Computer Vision and VFMs for Pixel and Object Classification in Microscopy 17 Pattern Recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and VFMs for Pixel and Object Classification in Microscopy 17 Pattern Recognition. p. 248–255. IEEE (Jun 2009).https://doi.org/10.1109/ cvpr.2009.5206848,http://dx.doi.org/10.1109/CVPR.2009.5206848

    arXiv 2009

  15. [15]

    https://doi.org/10.48550/ARXIV.2010.11929,https://arxiv.org/abs/2010

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale (2020). https://doi.org/10.48550/ARXIV.2010.11929,https://arxiv.org/abs/2010. 11929

    Pith/arXiv arXiv doi:10.48550/arxiv.2010.11929 2020

  16. [16]

    Nature Methods18(9), 1038–1045 (Aug 2021).https://doi.org/10

    Edlund, C., Jackson, T.R., Khalid, N., Bevan, N., Dale, T., Dengel, A., Ahmed, S., Trygg, J., Sjögren, R.: Livecell—a large-scale dataset for label-free live cell seg- mentation. Nature Methods18(9), 1038–1045 (Aug 2021).https://doi.org/10. 1038/s41592-021-01249-6,http://dx.doi.org/10.1038/s41592-021-01249-6

    doi:10.1038/s41592-021-01249-6 2021

  17. [17]

    org/abs/2403.10516

    Fu, S., Hamilton, M., Brandt, L., Feldman, A., Zhang, Z., Freeman, W.T.: Featup: A model-agnostic framework for features at any resolution (2024),https://arxiv. org/abs/2403.10516

    Pith/arXiv arXiv 2024

  18. [18]

    Gamper, J., Koohbanani, N.A., Benes, K., Graham, S., Jahanifar, M., Khurram, S.A., Azam, A., Hewitt, K., Rajpoot, N.: Pannuke dataset extension, insights and baselines (2020),https://arxiv.org/abs/2003.10778

    Pith/arXiv arXiv 2020

  19. [19]

    arXiv preprint arXiv:2502.00408 (2025)

    Griebel, T., Archit, A., Pape, C.: Segment anything for histopathology. arXiv preprint arXiv:2502.00408 (2025)

    Pith/arXiv arXiv 2025

  20. [20]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 16000–16009 (2022)

    2022

  21. [21]

    He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    2016

  22. [22]

    bioRxiv pp

    Hinderling,L.,Witz,G.,Schwob,R.,Stojiljković,A.,Dobrzyński,M.,Vladymyrov, M., Frei, J., Grädel, B., Frismantiene, A., Pertz, O.: Convpaint-interactive pixel classification using pretrained neural networks. bioRxiv pp. 2024–09 (2024)

    2024

  23. [23]

    Nature methods18(2), 203–211 (2021)

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods18(2), 203–211 (2021)

    2021

  24. [24]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015–4026 (2023)

    2023

  25. [25]

    arxiv 2023

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al.: Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arxiv 2023. arXiv preprint arXiv:2303.05499 (2023)

    Pith/arXiv arXiv 2023

  26. [26]

    Transactions on Machine Learning Research (2025), https://openreview.net/forum?id=neMAx4uBlh, expert Certification

    Lüddecke, T., Ecker, A.S.: Characterizing vision backbones for dense prediction with dense attentive probing. Transactions on Machine Learning Research (2025), https://openreview.net/forum?id=neMAx4uBlh, expert Certification

    2025

  27. [27]

    Nature Methods pp

    Marks, M., Israel, U., Dilip, R., Li, Q., Yu, C., Laubscher, E., Iqbal, A., Pradhan, E., Ates, A., Abt, M., et al.: Cellsam: a foundation model for cell segmentation. Nature Methods pp. 1–9 (2025)

    2025

  28. [28]

    arXiv preprint arXiv:2304.07193 (2023)

    Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., et al.: Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    Pith/arXiv arXiv 2023

  29. [29]

    BioRxiv pp

    Pachitariu,M.,Rariden,M.,Stringer,C.:Cellpose-sam:superhumangeneralization for cellular segmentation. BioRxiv pp. 2025–04 (2025)

    2025

  30. [30]

    Teuber et al

    Psomas, B., Christopoulos, D., Baltzi, E., Kakogeorgiou, I., Aravanis, T., Ko- modakis, N., Karantzalos, K., Avrithis, Y., Tolias, G.: Attention, please! revisiting 18 C. Teuber et al. attentive probing through the lens of efficiency. arXiv preprint arXiv:2506.10178 (2025)

    arXiv 2025

  31. [31]

    In: International conference on machine learning

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PmLR (2021)

    2021

  32. [32]

    arXiv preprint arXiv:2408.00714 (2024)

    Ravi, N., Gabeur, V., Hu, Y.T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., et al.: Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714 (2024)

    Pith/arXiv arXiv 2024

  33. [33]

    arXiv preprint arXiv:2401.14159 (2024)

    Ren, T., Liu, S., Zeng, A., Lin, J., Li, K., Cao, H., Chen, J., Huang, X., Chen, Y., Yan, F., et al.: Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159 (2024)

    Pith/arXiv arXiv 2024

  34. [34]

    In: International Conference on Medical image computing and computer-assisted intervention

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)

    2015

  35. [35]

    Ryali, C., Hu, Y.T., Bolya, D., Wei, C., Fan, H., Huang, P.Y., Aggarwal, V., Chowdhury, A., Poursaeed, O., Hoffman, J., Malik, J., Li, Y., Feichtenhofer, C.: Hi- era:Ahierarchicalvisiontransformerwithoutthebells-and-whistles(2023).https: //doi.org/10.48550/ARXIV.2306.00989,https://arxiv.org/abs/2306.00989

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.00989 2023

  36. [36]

    In: International conference on medical image computing and computer- assisted intervention

    Schmidt, U., Weigert, M., Broaddus, C., Myers, G.: Cell detection with star-convex polygons. In: International conference on medical image computing and computer- assisted intervention. pp. 265–273. Springer (2018)

    2018

  37. [37]

    cancerimagingarchive.net/collection/crc_ffpe-codex_cellneighs/

    Schürch, C.M., Bhate, S., Barlow, G., Phillips, D., Noti, L., Zlobec, I., Chu, P., Black, S., Demeter, J., McIlwain, D., Samusik, N., Goltsev, Y., Nolan, G.: High- dimensional imaging of colorectal carcinoma and other tumors with 50+ mark- ers (2020).https://doi.org/10.7937/TCIA.2020.FQN0- 0326,https://www. cancerimagingarchive.net/collection/crc_ffpe-cod...

    work page doi:10.7937/tcia.2020.fqn0- 2020

  38. [38]

    npj Imaging3(1), 32 (2025)

    Seifi, M., Dalle Nogare, D., Battagliotti, J.M., Galinova, V., Rao, A.K., Jouneau, P.H., Archit, A., Pape, C., Decelle, J., et al.: Featureforest: the power of foundation models, the usability of random forests. npj Imaging3(1), 32 (2025)

    2025

  39. [39]

    arXiv preprint arXiv:2506.03373 (2025)

    Shaban, M., Chang, Y., Qiu, H., Yeo, Y.Y., Song, A.H., Jaume, G., Wang, Y., Weishaupt, L.L., Ding, T., Vaidya, A., et al.: A foundation model for spatial pro- teomics. arXiv preprint arXiv:2506.03373 (2025)

    Pith/arXiv arXiv 2025

  40. [40]

    arXiv preprint arXiv:2508.101042(4), 5 (2025)

    Siméoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M.a.d.: Dinov3. arXiv preprint arXiv:2508.101042(4), 5 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Nature methods18(1), 100–106 (2021)

    Stringer, C., Wang, T., Michaelos, M., Pachitariu, M.: Cellpose: a generalist algo- rithm for cellular segmentation. Nature methods18(1), 100–106 (2021)

    2021

  42. [42]

    scikit-image: Image processing in Python

    van der Walt, S., Schönberger, J.L., Nunez-Iglesias, J., Boulogne, F., Warner, J.D., Yager, N., Gouillart, E., Yu, T., contributors, t.s.i.: scikit-image: Image processing inpython(2014).https://doi.org/10.48550/ARXIV.1407.6245,https://arxiv. org/abs/1407.6245

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1407.6245 2014

  43. [43]

    48550/ARXIV.2510.12764,https://arxiv.org/abs/2510.12764

    Wimmer, T., Truong, P., Rakotosaona, M.J., Oechsle, M., Tombari, F., Schiele, B., Lenssen, J.E.: Anyup: Universal feature upsampling (2025).https://doi.org/10. 48550/ARXIV.2510.12764,https://arxiv.org/abs/2510.12764

    doi:10.48550/arxiv.2510.12764 2025

  44. [44]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Xie, R., Pang, K., Bader, G.D., Wang, B.: Maester: masked autoencoder guided segmentation at pixel resolution for accurate, self-supervised subcellular structure recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3292–3301 (2023)

    2023

  45. [45]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language im- age pre-training. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 11975–11986 (2023) VFMs for Pixel and Object Classification in Microscopy 19 Appendix 5.1 Qualitative Results Fig. 8 presents qualitative examples of the predictions obtained wit...

    2023