Pith. sign in

REVIEW 3 major objections 7 minor 12 references

Is Self-Supervision Enough? Benchmarking Foundation Models Against End-to-End Training for Mitotic Figure Classification

T0 review · 3 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A benchmark of mitotic figure classification finds that an end-to-end trained ResNet50 beats classifiers built on frozen histopathology foundation-model embeddings at every training-set size, and that the foundation models are not more…

desk verdict A useful benchmark with an honest scope, but the headline claim overreaches because the comparison stacks fine-tuning and augmentation on top of the pretraining comparison. read the letter →

arxiv 2412.06365 v2 pith:6W5ENLX6 submitted 2024-12-09 cs.CV

Jonathan Ganz , Jonas Ammeling , Emely Rosbach , Ludwig Lausser , Christof A. Bertram , Katharina Breininger , Marc Aubreville This is my paper · ORCID
classification cs.CV
keywords mitoticfigureclassificationfoundationmodelshistopathologyself-supervisedlearninglinearprobingdomainshiftdigitalpathologybenchmarking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Foundation models for histopathology promise that self-supervised pretraining on vast unlabeled data produces embeddings good enough that a simple classifier can solve downstream tasks with little labeled data and robust to new domains. The paper tests that promise for mitotic figure classification, a key tumor-prognosis biomarker, using two public datasets and classifiers trained on frozen embeddings from several foundation models. The result is negative: an end-to-end trained ResNet50, pretrained only on ImageNet and then fine-tuned on the task, outperforms every foundation-model-based classifier at every training-set size tested, from 0.1% to 100% of the data. The foundation-model classifiers also do not generalize better across the five domains of the MIDOG dataset, showing comparable or larger domain-shift drops. If correct, the paper establishes that for this task extensive self-supervised pretraining is not a substitute for task-specific supervision.

What carries the argument

The machinery that carries the comparison is linear probing on frozen embeddings: each foundation model turns a $224 \times 224$ pixel patch, extracted at 0.25 microns per pixel, into a vector (the class token, or for Virchow and Virchow2 the concatenation of class token and mean patch token), and a single linear layer is trained on those vectors. This isolates what the pretrained representations contribute, since only the final layer learns from labeled mitotic figure data. The evaluation protocol is equally load-bearing: identical train/validation/test splits across models, five-fold Monte Carlo cross-validation, training-set sizes from 0.1% to 100%, and a leave-one-domain-out scheme on the five MIDOG domains that measures in-domain and out-of-domain AUROC separately.

What would settle it

Fine-tune one of the best-performing foundation models, such as H-optimus-0, on the MIDOG training domains and test on the held-out domains; if its out-of-domain AUROC reaches or exceeds the end-to-end baseline's 0.74, the claim that self-supervised pretraining cannot replace task-specific supervision for this task would be refuted.

Watch

Extended reading notes

Core claim

The paper's central discovery is that frozen self-supervised histopathology embeddings do not carry the day for mitotic figure classification. Using shared train/validation/test splits and five-fold Monte Carlo cross-validation, the authors trained linear classifiers on embeddings from the compared foundation models and compared them with ImageNet-pretrained feature extractors and with a ResNet50 stem fine-tuned end to end. The end-to-end baseline reached mean AUROC of 0.87 on the CCMCT whole-slide dataset (canine cutaneous mast cell tumors) and 0.89 on MIDOG at full data, while the best foundation-model-based classifiers remained below that at every training-set size. In the cross-domain experiment, the end-to-end baseline again led both in-domain (0.87) and out-of-domain (0.74) AUROC, versus 0.79 and 0.66 for the best FM linear probe, and both model types suffered large and uneven domain-shift drops. The paper concludes that the two advertised benefits of foundation models for this task, reduced labeled-data need and increased domain robustness, did not materialize under linear probing.

Load-bearing premise

The load-bearing assumption is that a single linear layer trained on frozen $224 \times 224$ embeddings at 0.25 microns per pixel is a fair and representative way to use foundation models; if fine-tuning the models or adjusting resolution per model changes the ranking, the paper's conclusion would no longer hold.

Editorial extensions

If this is right

  • For mitotic figure classification, adopting a frozen foundation-model embedding plus a linear layer will not save labeling effort: the end-to-end-trained network achieves higher AUROC at every training-set size from 0.1% to 100%.
  • Foundation-model embeddings do not remove the need for domain-specific data: on the five-domain MIDOG benchmark, their out-of-domain AUROC drops to 0.66, only slightly above the ImageNet baselines and below the end-to-end model's 0.74.
  • The fact that all FM embeddings beat the ImageNet feature extractors shows the self-supervised representations do capture histology-specific information; the shortfall is specific to the frozen-embedding, linear-probe mode of use.
  • The observed performance curves had not converged at the largest dataset size, so larger training sets could still narrow the gap between FM-based and end-to-end classifiers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This suggests a stronger test of the paper's question would be fine-tuning the foundation models themselves; the authors note this could improve results but require much more compute, and it remains open whether the gap would close.
  • Because all FMs were given $224 \times 224$ patches at 0.25 microns per pixel regardless of the magnification they were trained on, per-model patch resolution might change the ranking, an untested confound.
  • This result cautions against assuming that any histopathology foundation model will transfer to a new, visually demanding task; similar linear-probing benchmarks on other rare-cell classification tasks would show whether the finding generalizes beyond mitotic figures.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper benchmarks linear classifiers trained on frozen embeddings from public histopathology foundation models (Prov-GigaPath, H-optimus-0, Virchow, Virchow2, UNI, Phikon; the abstract says five, but Table 1 lists six) against ImageNet-pretrained feature extractors and an end-to-end-trained ResNet50 for mitotic figure classification. The authors evaluate performance as a function of training set size on MIDOG and CCMCT and evaluate cross-domain robustness on MIDOG. Their main findings are that the end-to-end baseline outperforms all FM-based classifiers at every data size and that FM-based classifiers are not more robust to domain shift. The paper concludes that the assumptions that foundation-model embeddings reduce labeling needs and improve domain robustness are incorrect for this task.

Significance. This paper addresses a timely and clinically relevant question: whether large self-supervised histopathology models can substitute for task-specific end-to-end training in mitotic figure classification. Its strengths are the use of public datasets and public foundation models, the systematic variation of training set size, and the inclusion of a domain-shift experiment. A careful, protocol-matched comparison would be a valuable negative result for the computational pathology community. However, as presented, the experimental protocol confounds the source of pretraining with fine-tuning, data augmentation, and training compute, so the headline claim is broader than what the evidence actually supports. The paper is reproducible in principle because all datasets and models are public, but the protocol limitations currently limit its impact.

major comments (3)
  1. [Section 3, paragraphs 1 and 3] The central comparison conflates the source of pretraining with three other factors. The FM embeddings are evaluated with a single linear layer on frozen features with no data augmentation, whereas the end-to-end ResNet50 is fully trainable, is trained with color jitter, Gaussian blurring, flipping, and rotations, and uses a one-cycle learning-rate schedule. This means the observed performance gap could be due to fine-tuning capacity, augmentation, or training compute rather than to the supervision signal alone. The discussion in Section 5 already concedes that fine-tuning the FMs might improve results, yet the abstract and title draw the much stronger conclusion that self-supervision is not enough. At a minimum, the authors should rephrase the claim to refer to linear probing of frozen embeddings, or better, provide a matched evaluation such as fine-tuning the FMs with the same augmentation pipeline or evaluating a frozen ResNet50 linear probe under identical conditions.
  2. [Section 3, dataset-size experiment] The dataset-size experiment draws the test set as 20% of all annotations at the annotation level. The CCMCT dataset comprises only 32 whole-slide images, so this type of split can place patches from the same WSI in both training and test sets. This leaks slide-level staining and tissue appearance into training, which may differentially benefit the end-to-end model because it has more trainable parameters and can more easily memorize slide-specific artifacts. This could inflate the reported advantage of the end-to-end baseline on CCMCT. The authors should use a slide-level split for CCMCT and an ROI-level split for MIDOG, or explicitly justify why an annotation-level split is appropriate for the generalization claim.
  3. [Section 3, patch extraction paragraph] All FM embeddings are computed from 224x224 patches at 0.25 microns per pixel, but the foundation models were pretrained at different resolutions and patch sizes (for example, Virchow and UNI use 0.5 microns per pixel). The single-resolution protocol therefore probably places some FMs outside their native operating range, while the end-to-end ResNet50 is trained and tested on the same extracted patches. Without a sensitivity analysis or a justification that the chosen resolution is appropriate for each FM, the comparison is not at each model's best operating point, and the conclusion that FMs underperform the baseline may be partly an artifact of suboptimal embedding extraction.
minor comments (7)
  1. [Abstract] The abstract says 'five publicly available FMs' but Table 1 lists six foundation models; please correct the count.
  2. [Section 4, Figure 1] Figure 1 does not show error bars or confidence intervals even though the text reports standard deviations; adding error bars would make the comparative claims much easier to assess.
  3. [Section 5] The sentence 'The positive impact of the FMs’ pretraining- on the given tasks suggests a comparison...' contains an apparent typo ('pretraining-') and is difficult to parse; please rewrite it.
  4. [Section 5] The phrase 'better then' should be 'better than'.
  5. [Section 3, DINOv2 description] DINOv2 is described as training by 'contrasting differently augmented views'; a more precise description would mention self-distillation, since DINOv2 does not use a contrastive loss.
  6. [Section 3, linear probing protocol] The linear probing implementation is attributed to Chen et al. [3], but the original linear evaluation protocol is usually cited to the relevant self-supervised learning paper (e.g., DINOv2 or SimCLR); please add the appropriate reference.
  7. [Section 2, CCMCT dataset] Please clarify how the 44,880 MF and 27,965 look-alike annotations were derived from the original 262,481 annotations, and whether any patches were discarded because they were out of slide bounds or otherwise unusable at the chosen resolution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper is an empirical benchmark with no derivation chain that reduces to its inputs.

full rationale

This paper is an empirical comparison study, not a derivation. It evaluates five publicly available foundation models via linear probing against ImageNet-pretrained feature extractors and an end-to-end-trained ResNet50 on two public mitotic figure datasets, measuring AUROC across training-set sizes and domain splits. The central claim, that the end-to-end baseline outperforms the FM-based classifiers, is a direct report of measured performance in Tables and Figures, not a quantity derived from a fitted parameter or from an assumption that already contains the conclusion. The authors cite their own prior work (e.g., the MIDOG challenge paper and the Virchow papers), but these citations are used to describe datasets, embedding conventions, and previously reported methods, not to justify the benchmark result. The linear-probing protocol is adopted from Chen et al. and Vorontsov et al., which is standard practice and does not smuggle in the target claim. The only substantive concern is a protocol mismatch: FM embeddings are frozen and unaugmented, while the end-to-end model is fully trainable with augmentation, so the comparison confounds pretraining source with trainable capacity and augmentation. That is a fairness or correctness risk, not circular reasoning, because the conclusion is still empirically determined rather than true by construction. There is no fitted input renamed as a prediction, no uniqueness theorem imported from the authors, and no ansatz disguised as external support. Accordingly, the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

This is an empirical benchmark, so there are no invented entities and no fitted parameters in the model sense. The listed free parameters are experimental design choices that the central claim depends on. The axioms concern the fairness and scope of the comparison.

free parameters (3)
  • Patch extraction size and resolution = 224x224 at 0.25 mpp
    Chosen by the authors for all models; may not match the pretraining resolution of each foundation model and could bias the comparison.
  • Maximum learning rate for end-to-end training = 1e-4
    Used with the one-cycle policy; a standard but arbitrary choice that affects the baseline's performance.
  • Data augmentation pipeline = random color jitter, Gaussian blur, flipping, rotations
    Applied only to the end-to-end model, not to the linear probes; this asymmetry can favor the baseline.
assumptions (3)
  • domain assumption Linear probing is a representative way to use foundation models
    The paper evaluates FMs only through a single linear layer on frozen embeddings, and the conclusion 'no benefit' is limited to this usage. The authors acknowledge fine-tuning might help.
  • domain assumption The fixed patch extraction protocol is fair to all models
    All models receive 224x224 patches at 0.25 mpp, while some FMs were pretrained at other resolutions or magnifications; this could disadvantage them.
  • domain assumption AUROC is an appropriate metric for this comparison
    AUROC is used for all conclusions, but precision-recall or F1 could rank models differently given class imbalance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Is Self-Supervision Enough? Benchmarking Foundation Models Against End-to-End Training for Mitotic Figure Classification." pith.science (2026). https://pith.science/paper/6W5ENLX6

@misc{pith2026241206365,
  author       = {Pith},
  title        = {Pith review of: Is Self-Supervision Enough? Benchmarking Foundation Models Against End-to-End Training for Mitotic Figure Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6W5ENLX6}},
  note         = {Machine review of arXiv:2412.06365}
}
read the original abstract

Foundation models (FMs), i.e., models trained on a vast amount of typically unlabeled data, have become popular and available recently for the domain of histopathology. The key idea is to extract semantically rich vectors from any input patch, allowing for the use of simple subsequent classification networks potentially reducing the required amounts of labeled data, and increasing domain robustness. In this work, we investigate to which degree this also holds for mitotic figure classification. Utilizing two popular public mitotic figure datasets, we compared linear probing of five publicly available FMs against models trained on ImageNet and a simple ResNet50 end-to-end-trained baseline. We found that the end-to-end-trained baseline outperformed all FM-based classifiers, regardless of the amount of data provided. Additionally, we did not observe the FM-based classifiers to be more robust against domain shifts, rendering both of the above assumptions incorrect.

Figures

Figures reproduced from arXiv: 2412.06365 by the authors.

Figure 1
Figure 1. Results of the investigation of the influence of training set size on model performance. The percentages refer to the total size of the data set. The mean in-domain and out-of-domain performances over the different domains for all models are given in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Cross-domain AUROC results for the end-to-end trained ResNet50 baseline model and the classifier based on H-optimus-0 embeddings. Like in the first experiment, the end-to-end baseline clearly outperforms all other models, both in terms of in-domain performance but also in terms of out-of-domain performance (see [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 9 canonical work pages

  1. [1]

    Do- main generalization across tumor types, laboratories, and species—Insights from the 2022 editionoftheMitosisDomainGeneralizationChallenge.MedicalImageAnalysis94(2024), p

    Aubreville M, Stathonikos N, Donovan TA, Klopfleisch R, Ammeling J, Ganz J et al. Do- main generalization across tumor types, laboratories, and species—Insights from the 2022 editionoftheMitosisDomainGeneralizationChallenge.MedicalImageAnalysis94(2024), p. 103155

    work page 2024

  2. [2]

    Scientific Reports 14.1 (2024), p

    GanzJ,MarzahlC,AmmelingJ,RosbachE,RichterB,PugetCetal.Informationmismatch in PHH3-assisted mitosis annotation leads to interpretation shifts in H&E slide analysis. Scientific Reports 14.1 (2024), p. 26273

    work page 2024

  3. [3]

    Towards a general- purpose foundation model for computational pathology

    Chen RJ, Ding T, Lu MY, Williamson DF, Jaume G, Song AH et al. Towards a general- purpose foundation model for computational pathology. Nature Medicine 30.3 (2024), pp. 850–862

    work page 2024

  4. [4]

    A foundation model for clinical-grade computational pathology and rare cancers detection

    Vorontsov E, Bozkurt A, Casson A, Shaikovski G, Zelechowski M, Severson K et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature Medicine (2024), pp. 1–12

    work page 2024

  5. [5]

    Vir- chow 2: Scaling Self-Supervised Mixed Magnification Models in Pathology

    Zimmermann E, Vorontsov E, Viret J, Casson A, Zelechowski M, Shaikovski G et al. Vir- chow 2: Scaling Self-Supervised Mixed Magnification Models in Pathology. arXiv preprint arXiv:2408.00738 (2024)

    arXiv 2024

  6. [6]

    Scaling self-supervised learning for histopathology with masked image modeling

    Filiot A, Ghermi R, Olivier A, Jacob P, Fidon L, Mac Kain A et al. Scaling self-supervised learning for histopathology with masked image modeling. medRxiv (2023), pp. 2023–07

    work page 2023

  7. [7]

    SaillardC,JenattonR,Llinares-LópezF,MarietZ,CahanéD,DurandEetal.H-optimus-0. 2024

    work page 2024

  8. [8]

    A whole-slide foundation model for digital pathology from real-world data

    Xu H, Usuyama N, Bagga J, Zhang S, Rao R, Naumann T et al. A whole-slide foundation model for digital pathology from real-world data. Nature (2024)

    work page 2024

Show all 12 references
  1. [9]

    Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region

    Aubreville M, Bertram CA, Marzahl C, Gurtner C, Dettwiler M, Schmidt A et al. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region. Scientific reports 10.1 (2020), p. 16447

    2020

  2. [10]

    BertramCA,AubrevilleM,MarzahlC,MaierA,KlopfleischR.Alarge-scaledatasetformi- toticfigureassessmentonwholeslideimagesofcaninecutaneousmastcelltumor.Scientific data 6.1 (2019), p. 274

    2019

  3. [11]

    arXiv preprint arXiv:2304.07193 (2023)

    OquabM,DarcetT,MoutakanniT,VoH,SzafraniecM,KhalidovVetal.Dinov2:Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023)

    2023 arXiv

  4. [12]

    Super-convergence: Very fast training of neural networks using large learning rates

    Smith LN, Topin N. Super-convergence: Very fast training of neural networks using large learning rates. arXiv. arXiv preprint arXiv:1708.07120 6 (2017)

    2017 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.