Pith. sign in

REVIEW 3 major objections 4 minor 15 references

Classifying Mitotic Figures in the MIDOG25 Challenge with Deep Ensemble Learning and Rule Based Refinement

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read An ensemble of three neural networks separates atypical from normal mitotic figures with 84.02% balanced accuracy on the MIDOG25 preliminary test set; a rule-based refinement module raises specificity but lowers sensitivity and overall perf

desk verdict A solid, honest challenge entry whose main claim rests on an unverified label-harmonization assumption across three merged datasets. read the letter →

arxiv 2508.20919 v1 pith:JPXWUPWW submitted 2025-08-28 cs.CV

classification cs.CV
keywords mitoticfiguresatypicalmitosisdeepensemblelearningConvNeXtrule-basedrefinementhistopathologyimageclassificationMIDOG25challengebalancedaccuracy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a two-way classification problem in tumor pathology: deciding whether a mitotic figure, a cell caught in the act of dividing, is atypical or normal, a distinction that is labor-intensive and subjective when done by eye. The authors build an ensemble of three pretrained ConvNeXtBase image classifiers on roughly 15,700 mitotic-figure crops merged from three public datasets, and report a balanced accuracy of 84.02% on the MIDOG25 preliminary test set. They also test a rule-based refinement module that uses cell shape and orientation to adjust the ensemble's scores. The refinement raises specificity from 75.09% to 80.97% but lowers sensitivity from 92.96% to 85.92%, so the plain ensemble remains the better overall model. The paper's central claim is that deep ensembles are a strong baseline for atypical-mitosis classification, and that this refinement idea needs a more reliable cell detector before it can help.

What carries the argument

The mechanism is a two-branch inference pipeline. Branch one is the deep ensemble: three ConvNeXtBase networks, trained in a standardized medical-image classification framework with class-weighted focal loss and three-fold cross-validation, each output a softmax vector, and the mean vector is thresholded at 0.5. Branch two, the rule-based refinement module, is a modifier rather than a classifier: after Macenko stain normalization, a pretrained general-purpose cell detector finds cells near the image center; hand-written rules then shift the ensemble score up or down depending on how many cells are found, whether a single cell is ring-like, round, or oval, and whether two cells are parallel o

What would settle it

Take the final MIDOG25 test set and rerun the exact ensemble with the same architecture, training data, and threshold; if its balanced accuracy is statistically indistinguishable from or below 75%, the 84.02% result was specific to the preliminary set. If, on the same set, the RBR with a mitosis-specific cell detector beats the ensemble, the paper's conclusion that refinement cannot improve overall performance is wrong.

Watch

Extended reading notes

Core claim

At its core, the paper reports an empirical finding: on the MIDOG25 preliminary test set, an ensemble of three ConvNeXtBase networks distinguishes atypical from normal mitotic figures with 84.02% balanced accuracy, with sensitivity of 92.96% and specificity of 75.09%. When a rule-based refinement module is switched on, specificity rises to 80.97% but sensitivity falls to 85.92%, for a balanced accuracy of 83.44%. The authors therefore conclude that the deep ensemble alone is the stronger classifier, and that tying morphological heuristics to an unreliable general-purpose cell detector is what prevents the refinement from helping.

Load-bearing premise

The test accuracy is only meaningful if all three merged datasets label 'atypical' and 'normal' the same way, so that one binary classifier has a consistent ground truth to learn and be scored against.

Editorial extensions

If this is right

  • A three-model ConvNeXtBase ensemble is enough to establish a strong baseline for atypical-versus-normal mitosis classification, so future challenge submissions can be compared against an 84% balanced-accuracy bar.
  • The reported error pattern—high sensitivity, low specificity—means the ensemble will rarely miss atypical figures but will over-flag normals; clinical deployment would need to decide which error is more costly.
  • The six-point specificity gain from the rule-based module shows morphological cues carry useful signal, but the module's current form is not worth its sensitivity loss.
  • The paper's own conclusion is that a mitosis-specific cell detector, replacing the general-purpose one, is the prerequisite for making rule-based refinement viable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 0.5 threshold is an unexamined dial: lowering it would shift the plain ensemble toward the same specificity as the RBR module without incurring the module's detection errors, so a threshold sweep is the cheapest test of the refinement's value.
  • Because the evaluation split was patient-stratified but within the same challenge datasets, the 84.02% figure likely overstates performance on a new center, stain, or scanner; the relative ranking of ensemble vs RBR could also change under domain shift.
  • The hand-set rule weights (0.2–0.6) encode the developers' trust in heuristics rather than measured importance; tuning these weights on a held-out set might recover some of the lost sensitivity.
  • The mismatch between the two stain-normalization methods (Reinhard for the ensemble, Macenko for the RBR) is never isolated; raw color differences could explain part of the sensitivity drop that is currently attributed to the rules.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a two-stage pipeline for classifying atypical versus normal mitotic figures (AMF/NMF) in H&E-stained histopathology images: an ensemble of three ConvNeXtBase models trained with the AUCMEDI framework, optionally combined with a rule-based refinement (RBR) module. The RBR module adjusts the deep-learning score based on StarDist cell detection and morphological features such as cell shape and orientation. On the MIDOG25 preliminary test set, the ensemble alone achieves a balanced accuracy of 84.02% (sensitivity 92.96%, specificity 75.09%, AUC 92.84); adding RBR changes these to 83.44%, 85.92%, 80.97%, and 89.17%, respectively. The authors conclude that deep ensembles provide strong baselines for AMF classification, while the current RBR approach does not improve overall performance but may improve specificity with further research.

Significance. If the result holds, the paper provides a strong, reproducible baseline for the MIDOG25 AMF/NMF classification task, which is of practical interest for tumor grading. The authors make their code publicly available, describe training details sufficiently for re-implementation, and honestly report that the RBR module degrades overall performance. These are strengths. The central claim, however, rests on the validity of the test metric, which depends on label consistency across three merged datasets, and the lack of uncertainty quantification weakens the strength of the conclusions.

major comments (3)
  1. [Section II – Data] The paper merges AMi-Br, MIDOG25, and Octopath into a single 15,689-MF training set with no label-harmonization analysis. The AMi-Br dataset is known to contain subtypes (indeed, 'AMF other' is used in the RBR rules), and if the three datasets use different morphological criteria for AMF, the binary target is ill-defined. Consequently, the balanced accuracy of 84.02% in Table I may reflect dataset-specific artifacts rather than true AMF recognition. Please report per-dataset class distributions, cite or reproduce the challenge's label definitions, and provide evidence (e.g., inter-annotator statistics or confirmation from the challenge organizers) that AMF/NMF labels are interchangeable across the three sources.
  2. [Section III – Table I] All metrics are point estimates on a single preliminary test set, with no confidence intervals or significance tests. This is especially problematic for the RBR comparison: the specificity increase from 75.09% to 80.97% and sensitivity drop from 92.96% to 85.92% could plausibly be within sampling variability. Bootstrapped 95% confidence intervals, or at least a paired test (e.g., McNemar for the binary decisions, DeLong for AUC), are needed before asserting that 'RBR can increase specific metrics.' The current presentation does not support a statistical distinction between the two configurations.
  3. [Section II – Rule-Based Refinement] The RBR modification weights (e.g., 0.6 for clear parallel orientation, 0.2 for a ring-like shape) are chosen ad hoc, and the paper acknowledges that they reflect trust in the analysis algorithms rather than measured significance. Since the RBR conclusion is based on these specific weights, a sensitivity analysis (varying the weights or showing the decision boundary) is needed to establish that the specificity improvement is robust to the choice of parameters. Without it, the statement that RBR can increase specificity should be limited to 'with these weights, on this test set.'
minor comments (4)
  1. [Abstract] Typo: 'extend' should be 'extended.' Also, capitalization of 'Mitotic Figures' is inconsistent with 'NMFs' and 'MFs' later in the text.
  2. [Section II – Training] Typo: 'tree models' should be 'three models.' Also, there is a missing period after the torchstain reference: 'in the torchstain framework [10]. was used.'
  3. [Section II – Data split] It would be helpful to clarify whether the final submission trained on all available data includes any images from the same patients as the preliminary test set, since the earlier patient-stratified split is abandoned. The test set is external, but potential patient overlap should be excluded or explicitly discussed.
  4. [Figures] Figure 2 caption is minimal; consider labeling the green path (AUCMEDI only) and the RBR path directly in the figure so the reader can follow the inference flow without switching between caption and text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported accuracy is an external benchmark result, and no fitted parameter is relabeled as a prediction.

full rationale

The paper's central claim is an empirical result: a deep ensemble reaches 84.02% balanced accuracy on the MIDOG25 preliminary test set (Section III, Table I). The model is trained on challenge-provided training data and evaluated on a held-out test set provided by the challenge; the accuracy is computed from those external labels and is not obtained by construction from any fitted parameter. The rule-based refinement (RBR) weights are manually defined heuristics in Section II (e.g., parallel orientation increases NMF probability by 0.6, ring-like shape by 0.2) and are not optimized against the test set; indeed, the paper reports that RBR decreased balanced accuracy, which is the opposite of a fitted-to-test result. The AUCMEDI framework citation [11] is a self-citation by two of the authors, but it is used as software tooling for model training, not as a load-bearing scientific premise, uniqueness theorem, or source of the claim being made. The only substantive concern—label harmonization across AMi-Br, MIDOG25, and Octopath—is a possible correctness or validity threat to the label definition, not a circularity: the model still learns from supplied labels and is evaluated on independently supplied test labels. No equation reduces to its own input, no fitted quantity is renamed a prediction, and no self-citation chain forces the outcome. Therefore the paper is self-contained as an empirical benchmark report and receives score 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper's central claim relies primarily on standard transfer learning assumptions and the integrity of the external test set. The only explicit hand-tuned parameters are the RBR modification weights, which are acknowledged as heuristic.

free parameters (2)
  • RBR modification weights = 0.6 (parallel cells), 0.2 (ring-like shape), and other unspecified weights
    Hand-tuned in Section II to adjust NMF or AMF probabilities based on StarDist morphological analysis. These weights are arbitrary and are not justified by a data-fitting procedure.
  • Focal loss class weights = Computed from class distributions in training folds
    Mentioned in Section II as computed based on class distributions; these are fitted to training data but are standard practice to handle class imbalance.
assumptions (4)
  • domain assumption The Reinhard and Macenko stain normalization algorithms reduce stain variation without removing diagnostic content.
    Used in Section II for preprocessing all images. If this fails, the classifier would not generalize across scanners and staining protocols.
  • domain assumption Pre-trained ConvNeXtBase weights on ImageNet transfer effectively to H&E-stained histopathology images.
    The paper initializes from pre-trained weights (Section II). This is a common assumption in transfer learning that is not verified in the paper.
  • domain assumption The StarDist '2D versatile he' model detects mitotic cells with enough accuracy for morphological analysis.
    Used in the RBR pipeline (Section II). The paper later notes this assumption is violated, leading to RBR errors.
  • domain assumption The MIDOG25 preliminary test set is representative of the challenge's target domain and is free of leakage from the training data.
    The final model is trained on all data and evaluated on the preliminary test set (Section II). If this assumption fails, the reported accuracies are not meaningful.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Classifying Mitotic Figures in the MIDOG25 Challenge with Deep Ensemble Learning and Rule Based Refinement." pith.science (2026). https://pith.science/paper/JPXWUPWW

@misc{pith2026250820919,
  author       = {Pith},
  title        = {Pith review of: Classifying Mitotic Figures in the MIDOG25 Challenge with Deep Ensemble Learning and Rule Based Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JPXWUPWW}},
  note         = {Machine review of arXiv:2508.20919}
}
read the original abstract

Mitotic figures (MFs) are relevant biomarkers in tumor grading. Differentiating atypical MFs (AMFs) from normal MFs (NMFs) remains difficult, as manual annotation is time-consuming and subjective. In this work an ensemble of ConvNeXtBase models was trained with AUCMEDI and extend with a rule-based refinement (RBR) module. On the MIDOG25 preliminary test set, the ensemble achieved a balanced accuracy of 84.02%. While the RBR increased specificity, it reduced sensitivity and overall performance. The results show that deep ensembles perform well for AMF classification. RBR can increase specific metrics but requires further research.

Figures

Figures reproduced from arXiv: 2508.20919 by the authors.

Figure 1
Figure 1. Training and validation workflow of the proposed classification [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Inference pipeline of the proposed classification algorithm. If only [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages

  1. [1]

    Grading of invasive breast carcinoma: the way forward,

    C. Van Dooijeweert, P. J. Van Diest, and I. O. Ellis, “Grading of invasive breast carcinoma: the way forward,” Virchows Archiv , vol. 480, no. 1, pp. 33–43, Jan. 2022. [Online]. Available: https://link.springer.com/10.1007/s00428-021-03141-2

  2. [2]

    Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors,

    C. A. Bertram, T. A. Donovan, and A. Bartel, “Mitotic activity: A systematic literature review of the assessment methodology and prognostic value in canine tumors,” V eterinary Pathology , vol. 61, no. 5, pp. 752–764, Sep. 2024. [Online]. Available: https://journals.sagepub.com/doi/10.1177/03009858241239565

  3. [3]

    Improving mitotic cell counting accuracy and efficiency using phosphohistone-H3 (PHH3) antibody counterstained with haematoxylin and eosin as part of breast cancer grading,

    A. Ibrahim et al., “Improving mitotic cell counting accuracy and efficiency using phosphohistone-H3 (PHH3) antibody counterstained with haematoxylin and eosin as part of breast cancer grading,” Histopathology, vol. 82, no. 3, pp. 393–406, Feb. 2023. [Online]. Available: https://onlinelibrary.wiley.com/doi/10.1111/his.14837

  4. [4]

    The characteristics and clinical significance of atypical mitosis in breast cancer,

    A. Lashen et al., “The characteristics and clinical significance of atypical mitosis in breast cancer,” Modern Pathology , vol. 35, no. 10, pp. 1341–1348, Oct. 2022. [Online]. Available: https: //linkinghub.elsevier.com/retrieve/pii/S0893395222002496

  5. [5]

    Mitosis Domain Generalization Challenge 2025,

    J. Ammeling et al., “Mitosis Domain Generalization Challenge 2025,” Mar. 2025. [Online]. Available: https://zenodo.org/doi/10.5281/zenodo. 15077361

  6. [6]

    Histologic Dataset of Normal and Atypical Mitotic Figures on Human Breast Cancer (AMi-Br),

    C. A. Bertram et al., “Histologic Dataset of Normal and Atypical Mitotic Figures on Human Breast Cancer (AMi-Br),” in Bildverarbeitung f¨ur die Medizin 2025 , C. Palm, K. Breininger, T. Deserno, H. Handels, A. Maier, K. H. Maier-Hein, and T. M. Tolxdorff, Eds. Wiesbaden: Springer Fachmedien Wiesbaden, 2025, pp. 113– 118, series Title: Informatik aktuell. ...

  7. [7]

    A dataset of atypical vs normal mitoses classification for MIDOG - 2025,

    V . Weiss et al., “A dataset of atypical vs normal mitoses classification for MIDOG - 2025,” Jul. 2025. [Online]. Available: https://zenodo.org/doi/10.5281/zenodo.16044804

  8. [8]

    OMG-Octo Atypical: A refinement of the original OMG-Octo database to incorporate atypical mitoses,

    Z. Shen, M. A. Hawkins, E. Baer, K. Br ¨autigam, and C.-A. Collins Fekete, “OMG-Octo Atypical: A refinement of the original OMG-Octo database to incorporate atypical mitoses,” Aug. 2025. [Online]. Available: https://zenodo.org/doi/10.5281/zenodo.16780587

Show all 15 references
  1. [9]

    Color transfer between images,

    E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, “Color transfer between images,” IEEE Computer Graphics and Applications , vol. 21, no. 5, pp. 34–41, Jul. 2001. [Online]. Available: https: //ieeexplore.ieee.org/document/946629

  2. [10]

    torchstain,

    C. A. Barbano and A. Pedersen, “torchstain,” Aug. 2022. [Online]. Available: https://doi.org/10.5281/zenodo.6979540

  3. [11]

    Standardized Medical Image Classification across Medical Disciplines,

    S. Mayer, D. M ¨uller, and F. Kramer, “Standardized Medical Image Classification across Medical Disciplines,” Oct. 2022, arXiv:2210.11091 [eess]. [Online]. Available: http://arxiv.org/abs/2210.11091

  4. [12]

    A ConvNet for the 2020s,

    Z. Liu et al., “A ConvNet for the 2020s,” Mar. 2022, arXiv:2201.03545 [cs]. [Online]. Available: http://arxiv.org/abs/2201.03545

  5. [13]

    Focal Loss for Dense Object Detection,

    T.-Y . Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal Loss for Dense Object Detection,” Feb. 2018, arXiv:1708.02002 [cs]. [Online]. Available: http://arxiv.org/abs/1708.02002

  6. [14]

    A method for normalizing histology slides for quantitative analysis,

    M. Macenko et al., “A method for normalizing histology slides for quantitative analysis,” in 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro , Jun. 2009, pp. 1107–1110, iSSN: 1945-8452. [Online]. Available: https://ieeexplore. ieee.org/document/5193250

  7. [15]

    Nuclei Instance Segmentation and Classification in Histopathology Images with Stardist,

    M. Weigert and U. Schmidt, “Nuclei Instance Segmentation and Classification in Histopathology Images with Stardist,” in 2022 IEEE International Symposium on Biomedical Imaging Challenges (ISBIC) . Kolkata, India: IEEE, Mar. 2022, pp. 1–4. [Online]. Available: https://ieeexplor...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.