{"id":"d86ca632-5f38-4c6b-a86d-a84613f86772","arxiv_id":"2509.02630","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A Faster R-CNN plus classifier ensemble achieved high recall (0.95) but low precision (0.13) on MIDOG 2025, and no optimization variant recovered the loss.","lead":"This report describes a two-stage mitotic figure detection pipeline, Faster R-CNN plus a three-classifier ensemble, submitted to the MIDOG 2025 pathology challenge. The system detected nearly all true mitoses but produced many false positives, and the authors report that later optimization attempts made results worse.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Best-submission architecture is unspecified; internal pipeline recall (0.0488) contradicts the official result (0.9528), so the claim that classifier filtering degrades performance is unsupported.","rationale":"The reader's CONDITIONAL verdict is appropriate. The most load-bearing concern is not primarily annotation pooling but attribution: the internal evaluation in §C shows a catastrophic recall drop from detection-only to full pipeline, which the paper uses to conclude that classifier filtering degrades performance. But the official best submission achieved recall 0.9528, roughly 20× higher. If the full pipeline produced that official score, then the internal and official evaluations are incompatible; if the best submission was detection-only, then the abstract's framing of a two-stage approach is misleading and the low-precision conclusion applies to a different system. The paper never provides the six-submission breakdown that would resolve this. External scores are credible but cannot validate conclusions about internal architectural components. This is a concrete, fixable omission: a revised version listing each submission variant and reconciling the two recall numbers could support the lessons. The concern is real but does not itself constitute demonstrated invalidity, so the reader's CONDITIONAL verdict remains appropriate.","tokens_in":4271,"tokens_out":4249,"duration_ms":48164,"concrete_test":"Obtain the MIDOG 2025 challenge submission logs (or, failing that, the exact checkpoint/config/Docker image used for the best submission) and reproduce inference on the §C internal test set. List all six variants with detector training set and whether the classifier ensemble was enabled. If the best-scoring variant had the ensemble disabled, the two-stage central claim collapses; if it had the ensemble enabled, reconcile the internal recall of 0.0488 with the official recall of 0.9528 by identifying the evaluation-protocol or threshold difference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central lesson—that two-stage classification filtering is counterproductive and false-positive suppression is the binding constraint—rests on §C, where detection alone achieves F1=0.0831 (recall 0.9842) but the full pipeline drops to F1=0.0646 (recall 0.0488). Yet the official best submission is reported as Faster R-CNN trained solely on MIDOG++, with recall 0.9528 and precision 0.1267. These numbers cannot describe the same system: if the best submission included the classifier ensemble, the official recall is ~20× the internal recall; if it did not, the headline result is not the two-stage pipeline described in the abstract. The manuscript never enumerates the six submitted variants, never maps scores to variants, and never reconciles the internal and official numbers. The external challenge scores are credible measurements, but they are uninterpretable as evidence about the two-stage architecture without knowing which variant generated them. This attribution gap is more load-bearing than cross-dataset annotation compatibility: even perfect annotation compatibility would not tell the reader whether the best result came from the full pipeline or from a detection-only model.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports the authors' participation in the MIDOG 2025 mitotic figure detection challenge. The proposed system is a two-stage pipeline: a Faster R-CNN detector generates candidate boxes, and an ensemble of three CNN classifiers (DenseNet-121, EfficientNet-v2, InceptionResNet-v2) filters false positives. The authors present internal evaluation results on a pooled dataset and official MIDOG 2025 scores for their submissions. Their best official submission achieved F1=0.2237 (recall 0.9528, precision 0.1267) and is attributed to a Faster R-CNN trained solely on MIDOG++. The final submission achieved F1=0.2031. Internal experiments show that the full pipeline drastically reduces recall (0.0488) compared with detection alone (0.9842), leading the authors to conclude that classifier-based false-positive filtering is counterproductive and that false-positive suppression is the main bottleneck in cross-domain mitosis detection.","tokens_in":4579,"tokens_out":5386,"duration_ms":56145,"significance":"If the attribution of the official scores to a specific architecture were clear, this would be a valuable challenge report: it documents a reproducible observation that a high-recall detector can be obtained across domains but at the cost of very low precision, and that sequential classifier filtering can hurt. The paper is transparent about its failures and provides many implementation details and hyperparameters. However, the central lesson is currently not defensible from the reported numbers because the manuscript does not identify which of the six submitted variants produced the official scores, and the internal pipeline metrics are inconsistent with the official best score. The paper's main contribution is a candid but poorly specified negative result that needs substantial revision.","major_comments":[{"comment":"The best official result (F1=0.2237, R=0.9528, P=0.1267) is attributed to \"a Faster R-CNN trained solely on MIDOG++\", but it is not stated whether this submission includes the three-classifier ensemble. This is not a pedantic point: Methods C reports that the full pipeline has recall 0.0488 and F1 0.0646, while detection alone has recall 0.9842. Official recall 0.9528 is about 20x the internal full-pipeline recall, so the official best almost certainly did not use the full pipeline as described. If it did, the internal and official numbers are irreconcilable; if it did not, the paper's headline result is not the two-stage method in the abstract. The authors must enumerate the six variants, state which architecture produced each official score, and reconcile the internal and official recall values.","section":"Results; Methods C"},{"comment":"The internal evaluation numbers for the detector are inconsistent. Methods A says detection on the PooledDB test set achieved recall 0.9820, precision 0.0578, F1 0.1091. Methods C says \"detection alone\" achieved F1 0.0831 (recall 0.9842). These two descriptions of the same component cannot both be correct on the same test set. The discrepancy may reflect different test sets (e.g., PooledDB test vs challenge validation), but the manuscript does not define the evaluation split used in Methods C. Without a clear protocol, the claimed degradation from adding the classifier cannot be interpreted.","section":"Methods A; Methods C"},{"comment":"The conclusion that \"subsequent optimization attempts were counterproductive\" is based on comparing the best submission (F1 0.2237) with the final submission (F1 0.2031). No details are given for the other four submissions, nor for what changed in \"dataset restructuring and classifier tuning.\" With only two endpoints and no error bars, the 9.2% difference could be within run-to-run variability or due to specific changes. The paper needs a table listing all six submissions with their configurations and official scores to support this claim.","section":"Results"},{"comment":"The detector and classifiers are trained on pooled datasets (MIDOG++, MITOS-CMC, MITOS-CCMCT, MiDeSeC, ICPR 2012) under the implicit assumption that mitotic-figure annotations are label-compatible across sources. The paper does not test this assumption (e.g., per-source recall/precision or a domain-adversarial check). Given that the best official submission used a MIDOG++-only detector, the role of pooling in the submitted variants is unclear. This omission leaves open the possibility that the observed low precision is partly an annotation/protocol artifact rather than purely a domain-shift phenomenon.","section":"Methods A/B"}],"minor_comments":[{"comment":"Typo: \"omplexity\" should be \"complexity\".","section":"Abstract"},{"comment":"The footer \"bioRχiv\" appears to be an artifact; it should name the actual journal or repository.","section":"Footer"},{"comment":"The reference list has typographical issues, e.g., \"Y ale\" and \"Y essica\" for author names. Please proofread.","section":"Bibliography"},{"comment":"No citations or URLs are provided for the datasets MITOS-CMC, MITOS-CCMCT, MiDeSeC, ICPR 2012, and MIDOG++. Please supply references.","section":"Methods A/B"},{"comment":"It is unclear whether classifier training patches were extracted from the same PooledDB split used for detector training, or whether detection-generated boxes from the validation set were used. Please specify to rule out train/test leakage.","section":"Methods B"}],"recommendation":"major_revision","confidential_remarks":"The official MIDOG 2025 scores are externally credible, but the central claim is currently unverifiable because the manuscript does not map the six submitted variants to their official scores. This is fixable with a variant table and a reconciliation of the internal and official metrics. The cross-dataset annotation compatibility concern is real but secondary. The acknowledgment of AI-assisted translation is fine, though the editor may wish to confirm all authors approved the final text."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one genuinely new thing here is the official MIDOG 2025 test-set scores for this team's submitted variants. Those numbers are externally scored, so they are real measurements, and the paper is refreshingly honest: it says outright that the work was a first challenge attempt, that time constraints caused inconsistent training protocols, and that the final performance was below competitive level. That counts for something. The high-recall/low-precision pattern it reports is also plausible and matches what people already suspect about mitotic detection across domains.\n\nBut the main lesson the abstract draws—that two-stage classifier filtering is counterproductive and that false-positive suppression is the binding constraint—does not hold together on the numbers as reported. Section C says the full pipeline drops to recall 0.0488 (F1 0.0646), while the official best submission is reported as recall 0.9528 (F1 0.2237). Those cannot be the same system. If the best submission included the classifier ensemble, the official recall is about twenty times the internal recall. If it did not include the ensemble, then the headline result is not the two-stage pipeline the paper describes. The manuscript never maps the six submitted variants to their scores and never reconciles this gap. That is a load-bearing omission, and it is more serious than the cross-dataset annotation compatibility concern the reader raised. Even if pooling is fine, you still wouldn't know which architecture produced the official result.\n\nThere are smaller soft spots: the three classifiers were trained on different datasets with different protocols, and the pooling assumes annotation compatibility across sources without any per-source checks. But those are secondary. The core problem is the attribution gap between the internal and official evaluations.\n\nWho is this for? Future MIDOG contestants and developers of mitotic detection pipelines. It is a data point, not a methodological advance; the two-stage Faster R-CNN plus classifier ensemble is the MDFS-style recipe the paper itself cites. The paper deserves a serious referee rather than a desk reject, because the measurements are real and the flaw is fixable by simply reporting which variant produced which number. But as it stands, I would not cite it for the claim that classifier filtering degrades performance, and I would not treat its lessons as established until the variant-to-score mapping is clarified.\n\nRecommendation: send it to peer review, but the reviewers should push hard on the reconciliation of the internal and official numbers. If that gap remains, the central narrative should be retracted or heavily qualified.","headline":"A credible challenge report with real external scores, but the central lesson about classifier filtering is undermined by an unexplained 20x gap between the internal and official recall numbers.","tokens_in":5033,"tokens_out":1761,"would_cite":false,"duration_ms":20186,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage mitotic-figure detector finds nearly every mitosis but drowns in look-alike false positives.","keywords":["mitotic figure detection","domain generalization","computational pathology","Faster R-CNN","ensemble classification","false positive suppression","MIDOG challenge","histopathology"],"falsifier":"Evaluate the PooledDB-trained detector separately on each constituent dataset (MIDOG++, MITOS-CMC, MITOS-CCMCT, MiDeSeC, ICPR 2012): if per-source precision varies widely, the pooled precision is a dataset artifact. Separately, feed ground-truth mitotic patches directly into the ensemble classifier: if recall on ground-truth patches is high but recall in the full pipeline is low, the bottleneck is bounding-box misalignment, not classifier discrimination.","tokens_in":4182,"feed_emoji":"🔬","tokens_out":3231,"duration_ms":37575,"temperature":0.7,"pith_summary":"This paper reports the authors' entry in the MIDOG 2025 challenge, which asks algorithms to detect mitotic figures across diverse tissue domains and scanners. Their pipeline pairs a Faster R-CNN candidate detector with an ensemble of three image classifiers meant to remove false positives, and their best submission achieved an F1-score of 0.2237, with recall at 0.9528 but precision at only 0.1267. The core claim is that candidate generation is no longer the bottleneck: the detector essentially finds all true mitoses, but the system is overwhelmed by imposters that look like mitoses, and every attempt at classifier-based filtering made things worse by removing true positives. If correct, this refocuses the field from detection sensitivity to false-positive suppression, and warns that naive sequential filtering can be counterproductive.","feed_headline":"Mitotic figure detector hits 95% recall yet 12.7% precision","feed_subtitle":"MIDOG 2025 entry shows false-positive suppression, not detection, is the bottleneck in cross-domain mitosis finding.","key_machinery":"A two-stage pipeline: Faster R-CNN detects candidate bounding boxes, and an ensemble of DenseNet-121, EfficientNet-v2, and InceptionResNet-v2 classifiers averages softmax probabilities to filter candidates. Training combines a pooled dataset (MIDOG++, MITOS-CMC, MITOS-CCMCT) and additional sets (MiDeSeC, ICPR 2012), with D4 rotation, defocus, and RandStainNA stain augmentations. Knowledge distillation from InceptionResNet-v2 improves the DenseNet-121 classifier from F1 0.57 to 0.73, but the ensemble still cannot reconcile candidate-box distribution with classifier training patches.","core_discovery":"For the MIDOG 2025 mitotic figure detection task, a Faster R-CNN trained only on the MIDOG++ dataset produced recall of 0.9528 and precision of 0.1267, an F1-score of 0.2237 that was the team's best among six submitted variants. On the internal pooled dataset, detection alone reached recall 0.9842 but precision 0.0578, and adding the three-classifier ensemble collapsed recall to 0.0488 while only modestly improving precision. The authors conclude that high-recall candidate detection is achievable across domains, but that the fundamental limitation is distinguishing true mitotic figures from morphologically similar imposters, and that sequential classifier filtering is not an effective remedy","pith_inferences":["The precision bottleneck might be partly an artifact of cross-dataset labeling differences: if the pooled datasets use different inclusion criteria for 'true' mitoses, the detector is trained on inconsistent labels and the measured precision is understated; per-source evaluation would test this.","The dramatic recall drop after classifier filtering (0.9842 to 0.0488) could reflect bounding-box misalignment: training classifiers on centered annotation patches while testing on detector-generated boxes with imperfect centers may systematically penalize true positives.","A testable extension is to calibrate the ensemble threshold per tissue domain: since precision varied little across submissions but recall varied substantially, a simple per-domain or per-scanner threshold could raise F1 without retraining.","The authors' lesson that 'optimization attempts were counterproductive' may generalize to other small-object detection challenges where the base detector already saturates recall; in such cases, precision-targeted losses or hard-negative mining are more promising than additional classifiers."],"forward_implications":["If the central claim holds, future domain-robust mitosis detection systems should prioritize precision-oriented training and false-positive suppression over further recall improvements.","Sequential detection-then-classification pipelines need joint fine-tuning or feature-level integration rather than independent filtering, because the classifier sees different patches than the detector produces.","High recall alone is not a useful clinical metric in this setting; the precision collapse implies that a pathologist would spend most of their time reviewing false positives.","The finding that adding more training data and classifiers decreased performance suggests that naive pooling of heterogeneous mitosis datasets can hurt rather than help domain generalization.","The performance decline of the final submitted variant indicates that overfitting to validation or to specific data sources is a serious risk in challenge-oriented optimization."],"supporting_citations":[{"why":"The 1000 Mitoses Project establishes inter-observer variability in mitotic figure recognition, motivating the need for automated detection.","marker":"[1]"},{"why":"The MIDOG 2021 challenge defines the scanner-agnostic domain generalization benchmark that this work participates in.","marker":"[2]"},{"why":"The MIDOG 2022 challenge extends the task to multi-domain generalization, providing the evaluation context for MIDOG 2025.","marker":"[3]"},{"why":"RandStainNA supplies the stain augmentation method used to improve robustness to staining variation across domains.","marker":"[4]"},{"why":"Knowledge distillation is the technique that substantially improved the DenseNet-121 classifier, a component of the ensemble.","marker":"[8]"}],"fun_headline_variants":["Mitotic figure detection: recall 95%, precision 12.7% — the gap","MIDOG 2025: high recall, low precision — false positives are the hurdle","Why 95% recall isn't enough: the mitosis detector's precision problem","Domain-shift mitosis detector: recall soars, precision stalls at 12.7%","The precision gap in MIDOG 2025: detection works, discrimination fails"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The detector and classifiers are trained on pooled datasets whose mitotic-figure labeling protocols, scanner profiles, and inclusion criteria are assumed to be equivalent, so if label noise or annotation-style mismatch differs across sources, the low precision could reflect dataset inconsistency rather than a true domain-generalization limit.","fun_headline_variants_meta":{"raw":{"variants":["Mitotic figure detection: recall 95%, precision 12.7% — the gap","MIDOG 2025: high recall, low precision — false positives are the hurdle","Why 95% recall isn't enough: the mitosis detector's precision problem","Domain-shift mitosis detector: recall soars, precision stalls at 12.7%","The precision gap in MIDOG 2025: detection works, discrimination fails"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000375,"raw_usage":{"total_tokens":1838,"prompt_tokens":750,"completion_tokens":1088,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":979}},"tokens_in":494,"tokens_out":1088,"duration_ms":10751,"temperature":1.0,"reasoning_tokens":979,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:19:26.096642+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the PooledDB-trained detector separately on each constituent dataset (MIDOG++, MITOS-CMC, MITOS-CCMCT, MiDeSeC, ICPR 2012): if per-source precision varies widely, the pooled precision is a dataset artifact. Separately, feed ground-truth mitotic patches directly into the ensemble classifier: if recall on ground-truth patches is high but recall in the full pipeline is low, the bottleneck is bounding-box misalignment, not classifier discrimination.","supporting_citations":[{"cited_title":"The 1000 mitoses project: A Consensus-Based international collaborative study on mitotic figures classification","cited_arxiv_id":null,"evidence_quote":"The 1000 Mitoses Project establishes inter-observer variability in mitotic figure recognition, motivating the need for automated detection."},{"cited_title":"Mitosis domain generalization challenge 2022","cited_arxiv_id":null,"evidence_quote":"The MIDOG 2022 challenge extends the task to multi-domain generalization, providing the evaluation context for MIDOG 2025."},{"cited_title":"RandStainNA: Learning Stain- Agnostic Features from Histology Slides by Bridging Stain Augmentation and Normaliza- tion, page 212–221","cited_arxiv_id":null,"evidence_quote":"RandStainNA supplies the stain augmentation method used to improve robustness to staining variation across domains."},{"cited_title":"Distilling the knowledge in a neural network, 2015","cited_arxiv_id":null,"evidence_quote":"Knowledge distillation is the technique that substantially improved the DenseNet-121 classifier, a component of the ensemble."}],"review_version":1}