REVIEW 4 major objections 5 minor 15 references
MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read MammoMix claims that training separate detectors on each mammography dataset and fusing their outputs through a learned gate plus calibrated confidence gives more accurate and reliable lesion detection than a single model trained on the…
desk verdict A reasonable MoE-for-mammography engineering idea whose headline claim is unsupported because the full MammoMix system was never run and reliability was never measured. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is three domain-specialized YOLOS detectors, one trained on CSAW, one on DDSM, and one on DMID, where YOLOS is a Vision Transformer object detector that treats image patches as tokens with learnable detection tokens. A separate CNN gating network classifies an input image's dataset origin and routes it to the corresponding expert, while MoCAE calibrates each expert's confidence by training a Random Forest regressor that maps a ResNet-18 image embedding plus raw confidence score to an expected IoU value, explicitly assigning zero IoU to out-of-domain validation images. The calibrated predictions from all experts are then fused through Soft NMS and Score Voting, which together consolidate overlapping detections into refined boxes.
What would settle it
Measure the gating network's accuracy at predicting dataset origin on held-out validation images; if it is near chance, or if out-of-domain detections are not uniformly zero-IoU in a representative mixed test set, the routing and calibration premises fail.
Extended reading notes
Core claim
The central discovery is that expert specialization plus calibrated ensemble fusion, implemented as MammoMix, improves breast lesion detection over a monolithic model (MoMo) trained on combined data. On CSAW, Simple MoE reaches mAP@50-95 of 0.3287 versus MoMo's 0.2458; on DMID, Simple MoE reaches 0.3348 versus MoMo's 0.2660. The calibration-augmented variant MoCAE achieves the best small-lesion AP on CSAW (0.3119) and the best large-lesion AP on DMID (0.4910). On DDSM, however, the best MammoMix variant only matches MoMo (0.1973 vs. 0.1710) and DETR outranks all MammoMix variants (0.1995).
Load-bearing premise
The system depends on the gating network being able to tell which dataset an image comes from just by looking at it; if mammograms from different centers are not visually distinguishable, routing fails and the mixture collapses into a plain ensemble.
Editorial extensions
If this is right
- On diverse or small datasets such as CSAW and DMID, domain-specialized experts with gating outperform a single YOLOS trained on the combined data, giving a recipe for multi-source medical detection without monolithic retraining.
- Per-expert Random Forest calibration that maps image embeddings and raw confidence to expected IoU yields confidence scores that better mirror true detection quality, improving small-lesion detection on CSAW.
- Fusion through Soft NMS and Score Voting consolidates partial detections from multiple experts, improving localization over any single expert.
- For more homogeneous datasets, a strong single detector (such as DETR on DDSM) can still match or beat the MoE approach, so the advantage of MammoMix is conditional on data heterogeneity.
Reading between the lines
- A stronger test of MammoMix would require a held-out dataset that looks visually similar to one training domain but contains different lesion distributions; if the gate routes by appearance while the expert was trained on different pathology, the system could be confidently wrong.
- The practice of assigning IoU=0 to out-of-domain validation images quietly encodes the assumption that cross-domain detections are worthless, which could depress calibrated scores below their true value in real deployments with overlapping domains.
- The same mixture-of-calibrated-experts recipe could transfer to other multi-center medical imaging tasks, such as chest X-ray or retinal imaging, provided each site's annotations are available for training and calibration.
- The gains from calibration on small lesions suggest that miscalibration is a major failure mode specifically for small-object detection, a direction worth isolating in future studies.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MammoMix, a Mixture-of-Experts framework for mammogram lesion detection. It trains three YOLOS experts on three datasets (CSAW, DDSM, DMID), a CNN gate intended to route each image to the appropriate expert, and a MoCAE calibration module that fits Random Forest regressors to map confidence scores to expected IoU, followed by Soft NMS and Score Voting fusion. The experiments compare a monolithic combined-training model (MoMo), a simple hard-routing MoE (Simple MoE), and the calibration-based ensemble (MoCAE) against DETR and RT-DETRv2 baselines on the three datasets, reporting mAP at multiple IoU thresholds. The paper claims that MammoMix improves both detection accuracy and reliability across heterogeneous datasets.
Significance. If the integrated MammoMix system performed as claimed, it would be a useful contribution to multi-domain medical object detection, where per-domain experts, learned routing, and calibrated fusion could improve generalization across imaging protocols. The paper uses three public mammography datasets, describes the components in reasonable detail, and makes code available. However, the significance is undercut by the fact that the full MammoMix pipeline—experts, CNN gate, calibrators, and fusion—is never evaluated as a single system, and no calibration or reliability metric is reported. The numerical results in Table 2 also contain unexplained duplicates between the 'Individually' and 'Simple MoE' rows, which raises concerns about the reliability of the reported experiments. The central claims of the paper are therefore not supported by the evidence presented.
major comments (4)
- [Section 6, Table 2; Abstract] The integrated MammoMix system described in the abstract and Figure 1—three experts, a CNN gate, three Random Forest calibrators, Soft NMS, and Score Voting operating together—is never evaluated. Table 2 reports results only for MoMo, Simple MoE, and MoCAE. Simple MoE (Section 4.3) performs hard argmax selection of one expert, while MoCAE (Section 4.4) runs all experts but does not use the CNN gate. No experiment combines the gate with calibrated fusion, so the manuscript provides no experimental result for the actual MammoMix framework. This is a load-bearing omission because the central claim that MammoMix outperforms baselines in accuracy is unsupported by the reported experiments.
- [Section 6; Abstract] The abstract and conclusion claim that MammoMix improves both average precision and reliability, but no calibration or reliability metric appears anywhere in the paper. The reported metric is mAP at various IoU thresholds, which is an accuracy measure. Expected calibration error, reliability diagrams, Brier score, or any direct assessment of whether confidence scores reflect true precision are absent. The statement in Section 6 that MoCAE 'improves model reliability' is therefore an unsupported claim.
- [Table 2; Section 4.3] The rows labeled 'Individually' and 'Simple MoE' are numerically identical on CSAW (0.3292, 0.7185, 0.2106, 0.3210 for mAP50-95, mAP50, small, medium) and on DMID (0.3364, 0.6249, 0.0000 for those metrics). If Simple MoE is a hard gate that selects the dataset-specific expert, then on per-dataset test splits the output would equal the individual expert only if the gate classifies the dataset origin with perfect accuracy on the test set. Any realistic gate error would produce small numerical differences, as seen on DDSM. The exact duplicates therefore imply either an implausible oracle gate or duplication of table entries; this needs a detailed explanation and corrected results.
- [Section 4.4] The calibration regressors assign an IoU of 0 to every out-of-domain validation image when constructing the training set, regardless of whether the expert actually produces a correct detection on that image. This modeling assumption forces the calibrator to output low calibrated scores for any expert on images from other datasets, even when the expert detects a lesion correctly. In a mixed-domain test set with real out-of-domain positives, this systematically suppresses true positives from the minority expert and biases MoCAE's fusion. The paper provides no evidence that this assumption holds, and it directly affects the validity of the MoCAE results.
minor comments (5)
- [Figure 2 caption] The caption contains the typo 'across dataset'; it should read 'across datasets'.
- [Throughout] The acronym is spelled inconsistently as both 'MoCaE' and 'MoCAE' (e.g., Section 4.4 header and Section 6 text). Please standardize.
- [Table 2] The header line 'Subset mAP DETR baseline YOLOS' is confusing; the columns should be clearly grouped by dataset and model variant.
- [Section 6, DDSM paragraph] The sentence 'DETR baseline outperformed all other approaches in terms of mAP@50-95 (0.1995)' is correct per Table 2, but the later sentence 'MoE variants remained competitive' understates that MoCAE is substantially worse (0.1357) on this dataset; the discussion could be more balanced.
- [Section 4.4] The notation for the calibrated score uses both ˜s and ˜s^e_i; please define once and use consistently.
Circularity Check
Out-of-domain calibration targets are set to IoU=0 by definition, so the claimed low-confidence behavior for out-of-domain experts is constructed rather than measured; all other derivation steps are self-contained.
-
fitted input called prediction
[Section 4.4 (MoCaE), calibration-dataset bullet list and following paragraph, pp. 8–9]
"For out-of-domain images from other datasets (Dk,k̸=e), we assign an IoU of 0, as these images lack ground truth annotations relevant to the expert’s training domain, reflecting the expert’s likely poor performance due to domain mismatch. ... This calibration strategy ensures the calibrator learns to assign low confidence to out-of-domain predictions, preventing overconfident errors."
The Random Forest calibrator is fit to validation targets in which every out-of-domain sample is assigned IoU=0 by definition, not by measurement. The paper then states that the calibrator 'learns to assign low confidence to out-of-domain predictions,' but that behavior is a direct consequence of the label construction rather than an empirical discovery. When the abstract and conclusion claim that MammoMix improves 'reliability' and that MoCAE 'adjusts confidence scores to reflect true predictive uncertainty,' the out-of-domain portion of that reliability is encoded in the training target and re-emerges as the calibrated output.
full rationale
Most of the pipeline is not circular. Each YOLOS expert is trained on its own dataset with external ground-truth boxes; the Simple MoE gate is a hard argmax over a dataset-origin classifier and its test numbers reflect actual detector outputs; MoCAE's in-domain calibration uses measured max-IoU targets; Soft NMS and Score Voting are standard fusion operations; and the only self-citation [15] is a related-work mention that is not load-bearing. The one constructed step is the out-of-domain calibration label: the paper assigns IoU=0 to all validation images outside an expert's training domain, fits a Random Forest to these labels, and then states that the calibrator 'learns to assign low confidence to out-of-domain predictions.' That behavior is a restatement of the label construction, not an empirical measurement. Because the abstract and conclusion claim improved 'reliability' and 'true predictive uncertainty,' the out-of-domain portion of that reliability claim reduces to the authors' own zero-assignment. Separately, the actual integrated MammoMix (CNN gate plus MoCAE plus fusion) is never run, and no calibration metric is reported; those are correctness and evidence gaps rather than circularity. The reported accuracy results are largely independent of the circular label choice, so the circularity is partial.
Assumptions & free parameters
free parameters (4)
- IoU assignment for out-of-domain calibration samples =
0
- Soft NMS sigma_nms =
0.08
- Random Forest estimator count =
300
- Augmentation hyperparameters =
alpha 50, sigma 5, rotation +/-10 degrees, scale +/-0.2, brightness/contrast +/-0.2, noise std 0.05
assumptions (4)
- domain assumption Dataset origin is visually identifiable from mammogram image appearance.
- domain assumption A Random Forest regressor can map image embeddings plus confidence scores to expected maximum IoU with sufficient accuracy.
- domain assumption YOLOS-base is a suitable detector backbone for all three datasets and outperforms DETR on this task.
- domain assumption Standard object-detection metrics with COCO-size categories transfer meaningfully to mammography lesions.
invented entities (1)
-
MammoMix framework (as a distinct integrated method)
Cite this review
Pith. "Pith review of MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection." pith.science (2026). https://pith.science/paper/HSZ4DPWE
@misc{pith2026260810437,
author = {Pith},
title = {Pith review of: MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/HSZ4DPWE}},
note = {Machine review of arXiv:2608.10437}
}
read the original abstract
Breast lesion detection in mammography remains a challenging task due to variations in image quality, lesion appearance, and population demographics across datasets. While current object detectors such as YOLO and DETR achieve strong results on individual datasets, their performance often degrades when trained on or applied across heterogeneous sources. To address this, we propose MammoMix, a novel framework based on Mixture-of-Experts (MoE) paradigm for robust and generalizable lesion detection. In MammoMix, each expert model is trained on a specific domain, allowing it to specialize in distinct characteristics of its source data. A gating mechanism adaptively weighs contributions from each expert based on input image, combining their outputs to enable domain-adaptive inference. To improve reliability, we further incorporate a calibration module, MoCAE, which adjusts confidence scores to reflect true predictive uncertainty. We evaluate MammoMix on 3 public mammography datasets: CSAW, DDSM, and DMID, covering diverse clinical settings. Results show that MammoMix outperforms baseline detectors in both average precision and reliability, particularly on datasets with greater variability. Our findings demonstrate that expert specialization and calibrated ensemble fusion significantly enhance model generalization and robustness. MammoMix offers a promising step toward dependable AI-assisted breast cancer screening across real-world clinical domains.
Figures
Reference graph
Works this paper leans on
-
[13]
arXiv preprint arXiv:2309.14976 (2023), https://doi.org/10.48550/arxiv.2309.14976
Oksuz,K.,Kuzucu,S.,Joy,T.,Dokania,P.K.:Mocae:Mixtureofcalibratedexperts significantly improves object detection. arXiv preprint arXiv:2309.14976 (2023), https://doi.org/10.48550/arxiv.2309.14976
-
[1]
In: European conference on computer vision
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European conference on computer vision. pp. 213–229. Springer (2020),https://doi.org/10.1007/ 978-3-030-58452-8_13
work page 2020
-
[2]
Radiology294(2), 265–272 (2020)
Dembrower, K., Liu, Y., Azizpour, H., Eklund, M., Smith, K., Lindholm, P., Strand, F.: Comparison of a deep learning risk score and standard mammographic density score for breast cancer risk prediction. Radiology294(2), 265–272 (2020)
work page 2020
-
[3]
arXiv preprint arXiv:2010.11929 (2020),https://doi.org/10.48550/arxiv.2010.11929
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020),https://doi.org/10.48550/arxiv.2010.11929
-
[4]
Fang, Y., Liao, B., Wang, X., Fang, J., Qi, J., Wu, R., Niu, J., Liu, W.: You only look at one sequence: Rethinking transformer in vision through object detec- tion. Advances in Neural Information Processing Systems34, 26183–26197 (2021), https://doi.org/10.48550/arxiv.2106.00666
-
[5]
Artificial Intelligence in Medicine132, 102386 (2022)
Garrucho, L., Kushibar, K., Jouide, S., Diaz, O., Igual, L., Lekadir, K.: Domain generalization in deep learning based mass detection in mammography: A large- scale multi-center study. Artificial Intelligence in Medicine132, 102386 (2022)
work page 2022
-
[6]
CA: a cancer journal for clinicians72(6), 524–541 (2022)
Giaquinto, A.N., Sung, H., Miller, K.D., Kramer, J.L., Newman, L.A., Minihan, A., Jemal, A., Siegel, R.L.: Breast cancer statistics, 2022. CA: a cancer journal for clinicians72(6), 524–541 (2022)
work page 2022
-
[7]
Cochrane database of systematic reviews1(6) (2013)
Gøtzsche, P.C., Jørgensen, K.J.: Screening for breast cancer with mammography. Cochrane database of systematic reviews1(6) (2013)
work page 2013
Show all 15 references
-
[8]
In: Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020)
Hamed, G., Marey, M.A.E.R., Amin, S.E.S., Tolba, M.F.: Deep learning in breast cancer detection and classification. In: Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). pp. 322–333. Springer (2020)
2020
-
[9]
The Lancet Digital Health 2(3), e138–e148 (2020)
Kim, H.E., Kim, H.H., Han, B.K., Kim, K.H., Han, K., Nam, H., Lee, E.H., Kim, E.K.: Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. The Lancet Digital Health 2(3), e138–e148 (2020)
2020
-
[10]
Scientific data4(1), 1–9 (2017)
Lee, R.S., Gimenez, F., Hoogi, A., Miyake, K.K., Gorovoy, M., Rubin, D.L.: A curated mammography data set for use in computer-aided detection and diagnosis research. Scientific data4(1), 1–9 (2017)
2017
-
[11]
arXiv preprint arXiv:2407.17140 (2024),https://doi.org/10.48550/arxiv.2407.17140
Lv, W., Zhao, Y., Chang, Q., Huang, K., Wang, G., Liu, Y.: Rt-detrv2: Improved baseline with bag-of-freebies for real-time detection transformer. arXiv preprint arXiv:2407.17140 (2024),https://doi.org/10.48550/arxiv.2407.17140
-
[12]
Artificial Intelligence Review42(2), 275–293 (2014)
Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artificial Intelligence Review42(2), 275–293 (2014)
2014
-
[14]
Biomedical Engineering Letters14(2), 317–330 (2024)
Oza, P., Oza, U., Oza, R., Sharma, P., Patel, S., Kumar, P., Gohel, B.: Digital mammography dataset for breast cancer diagnosis research (dmid) with breast mass segmentation analysis. Biomedical Engineering Letters14(2), 317–330 (2024)
2024
-
[15]
Ultrasound in Medicine & Biology51(6), 999–1007 (2025)
Zhang, C., Zheng, Y., McAviney, J., Ling, S.H.: Ssat-swin: Deep learning-based spinal ultrasound feature segmentation for scoliosis using self-supervised swin transformer. Ultrasound in Medicine & Biology51(6), 999–1007 (2025)
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.