Pith. sign in

REVIEW 4 major objections 5 minor 15 references

MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read MammoMix claims that training separate detectors on each mammography dataset and fusing their outputs through a learned gate plus calibrated confidence gives more accurate and reliable lesion detection than a single model trained on the…

desk verdict A reasonable MoE-for-mammography engineering idea whose headline claim is unsupported because the full MammoMix system was never run and reliability was never measured. read the letter →

arxiv 2608.10437 v1 pith:HSZ4DPWE submitted 2026-08-11 cs.CV

classification cs.CV
keywords Mixture-of-ExpertsMammographyLesiondetectionYOLOSConfidencecalibrationDomaingeneralizationObjectBreastcancerscreening
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to show that when mammography images come from several different sources, a single object detector trained on all of them together loses accuracy, while a mixture of experts—one detector per source—can do better. The authors build MammoMix, which pairs a per-source YOLOS detector with a gating network that decides which expert to trust, and a calibration module that rescales each expert's confidence to match its true precision before the boxes are fused. The result, they argue, is improved average precision and more reliable confidence scores on datasets with high variability, like CSAW and DMID. On the more homogeneous DDSM set, a strong single detector remains competitive, so the benefit is conditional on how heterogeneous the data are.

What carries the argument

The machinery is three domain-specialized YOLOS detectors, one trained on CSAW, one on DDSM, and one on DMID, where YOLOS is a Vision Transformer object detector that treats image patches as tokens with learnable detection tokens. A separate CNN gating network classifies an input image's dataset origin and routes it to the corresponding expert, while MoCAE calibrates each expert's confidence by training a Random Forest regressor that maps a ResNet-18 image embedding plus raw confidence score to an expected IoU value, explicitly assigning zero IoU to out-of-domain validation images. The calibrated predictions from all experts are then fused through Soft NMS and Score Voting, which together consolidate overlapping detections into refined boxes.

What would settle it

Measure the gating network's accuracy at predicting dataset origin on held-out validation images; if it is near chance, or if out-of-domain detections are not uniformly zero-IoU in a representative mixed test set, the routing and calibration premises fail.

Watch

Extended reading notes

Core claim

The central discovery is that expert specialization plus calibrated ensemble fusion, implemented as MammoMix, improves breast lesion detection over a monolithic model (MoMo) trained on combined data. On CSAW, Simple MoE reaches mAP@50-95 of 0.3287 versus MoMo's 0.2458; on DMID, Simple MoE reaches 0.3348 versus MoMo's 0.2660. The calibration-augmented variant MoCAE achieves the best small-lesion AP on CSAW (0.3119) and the best large-lesion AP on DMID (0.4910). On DDSM, however, the best MammoMix variant only matches MoMo (0.1973 vs. 0.1710) and DETR outranks all MammoMix variants (0.1995).

Load-bearing premise

The system depends on the gating network being able to tell which dataset an image comes from just by looking at it; if mammograms from different centers are not visually distinguishable, routing fails and the mixture collapses into a plain ensemble.

Editorial extensions

If this is right

  • On diverse or small datasets such as CSAW and DMID, domain-specialized experts with gating outperform a single YOLOS trained on the combined data, giving a recipe for multi-source medical detection without monolithic retraining.
  • Per-expert Random Forest calibration that maps image embeddings and raw confidence to expected IoU yields confidence scores that better mirror true detection quality, improving small-lesion detection on CSAW.
  • Fusion through Soft NMS and Score Voting consolidates partial detections from multiple experts, improving localization over any single expert.
  • For more homogeneous datasets, a strong single detector (such as DETR on DDSM) can still match or beat the MoE approach, so the advantage of MammoMix is conditional on data heterogeneity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A stronger test of MammoMix would require a held-out dataset that looks visually similar to one training domain but contains different lesion distributions; if the gate routes by appearance while the expert was trained on different pathology, the system could be confidently wrong.
  • The practice of assigning IoU=0 to out-of-domain validation images quietly encodes the assumption that cross-domain detections are worthless, which could depress calibrated scores below their true value in real deployments with overlapping domains.
  • The same mixture-of-calibrated-experts recipe could transfer to other multi-center medical imaging tasks, such as chest X-ray or retinal imaging, provided each site's annotations are available for training and calibration.
  • The gains from calibration on small lesions suggest that miscalibration is a major failure mode specifically for small-object detection, a direction worth isolating in future studies.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MammoMix, a Mixture-of-Experts framework for mammogram lesion detection. It trains three YOLOS experts on three datasets (CSAW, DDSM, DMID), a CNN gate intended to route each image to the appropriate expert, and a MoCAE calibration module that fits Random Forest regressors to map confidence scores to expected IoU, followed by Soft NMS and Score Voting fusion. The experiments compare a monolithic combined-training model (MoMo), a simple hard-routing MoE (Simple MoE), and the calibration-based ensemble (MoCAE) against DETR and RT-DETRv2 baselines on the three datasets, reporting mAP at multiple IoU thresholds. The paper claims that MammoMix improves both detection accuracy and reliability across heterogeneous datasets.

Significance. If the integrated MammoMix system performed as claimed, it would be a useful contribution to multi-domain medical object detection, where per-domain experts, learned routing, and calibrated fusion could improve generalization across imaging protocols. The paper uses three public mammography datasets, describes the components in reasonable detail, and makes code available. However, the significance is undercut by the fact that the full MammoMix pipeline—experts, CNN gate, calibrators, and fusion—is never evaluated as a single system, and no calibration or reliability metric is reported. The numerical results in Table 2 also contain unexplained duplicates between the 'Individually' and 'Simple MoE' rows, which raises concerns about the reliability of the reported experiments. The central claims of the paper are therefore not supported by the evidence presented.

major comments (4)
  1. [Section 6, Table 2; Abstract] The integrated MammoMix system described in the abstract and Figure 1—three experts, a CNN gate, three Random Forest calibrators, Soft NMS, and Score Voting operating together—is never evaluated. Table 2 reports results only for MoMo, Simple MoE, and MoCAE. Simple MoE (Section 4.3) performs hard argmax selection of one expert, while MoCAE (Section 4.4) runs all experts but does not use the CNN gate. No experiment combines the gate with calibrated fusion, so the manuscript provides no experimental result for the actual MammoMix framework. This is a load-bearing omission because the central claim that MammoMix outperforms baselines in accuracy is unsupported by the reported experiments.
  2. [Section 6; Abstract] The abstract and conclusion claim that MammoMix improves both average precision and reliability, but no calibration or reliability metric appears anywhere in the paper. The reported metric is mAP at various IoU thresholds, which is an accuracy measure. Expected calibration error, reliability diagrams, Brier score, or any direct assessment of whether confidence scores reflect true precision are absent. The statement in Section 6 that MoCAE 'improves model reliability' is therefore an unsupported claim.
  3. [Table 2; Section 4.3] The rows labeled 'Individually' and 'Simple MoE' are numerically identical on CSAW (0.3292, 0.7185, 0.2106, 0.3210 for mAP50-95, mAP50, small, medium) and on DMID (0.3364, 0.6249, 0.0000 for those metrics). If Simple MoE is a hard gate that selects the dataset-specific expert, then on per-dataset test splits the output would equal the individual expert only if the gate classifies the dataset origin with perfect accuracy on the test set. Any realistic gate error would produce small numerical differences, as seen on DDSM. The exact duplicates therefore imply either an implausible oracle gate or duplication of table entries; this needs a detailed explanation and corrected results.
  4. [Section 4.4] The calibration regressors assign an IoU of 0 to every out-of-domain validation image when constructing the training set, regardless of whether the expert actually produces a correct detection on that image. This modeling assumption forces the calibrator to output low calibrated scores for any expert on images from other datasets, even when the expert detects a lesion correctly. In a mixed-domain test set with real out-of-domain positives, this systematically suppresses true positives from the minority expert and biases MoCAE's fusion. The paper provides no evidence that this assumption holds, and it directly affects the validity of the MoCAE results.
minor comments (5)
  1. [Figure 2 caption] The caption contains the typo 'across dataset'; it should read 'across datasets'.
  2. [Throughout] The acronym is spelled inconsistently as both 'MoCaE' and 'MoCAE' (e.g., Section 4.4 header and Section 6 text). Please standardize.
  3. [Table 2] The header line 'Subset mAP DETR baseline YOLOS' is confusing; the columns should be clearly grouped by dataset and model variant.
  4. [Section 6, DDSM paragraph] The sentence 'DETR baseline outperformed all other approaches in terms of mAP@50-95 (0.1995)' is correct per Table 2, but the later sentence 'MoE variants remained competitive' understates that MoCAE is substantially worse (0.1357) on this dataset; the discussion could be more balanced.
  5. [Section 4.4] The notation for the calibrated score uses both ˜s and ˜s^e_i; please define once and use consistently.

Circularity Check

1 steps flagged · score 6.0 of 10

Out-of-domain calibration targets are set to IoU=0 by definition, so the claimed low-confidence behavior for out-of-domain experts is constructed rather than measured; all other derivation steps are self-contained.

  1. fitted input called prediction [Section 4.4 (MoCaE), calibration-dataset bullet list and following paragraph, pp. 8–9]
    "For out-of-domain images from other datasets (Dk,k̸=e), we assign an IoU of 0, as these images lack ground truth annotations relevant to the expert’s training domain, reflecting the expert’s likely poor performance due to domain mismatch. ... This calibration strategy ensures the calibrator learns to assign low confidence to out-of-domain predictions, preventing overconfident errors."

    The Random Forest calibrator is fit to validation targets in which every out-of-domain sample is assigned IoU=0 by definition, not by measurement. The paper then states that the calibrator 'learns to assign low confidence to out-of-domain predictions,' but that behavior is a direct consequence of the label construction rather than an empirical discovery. When the abstract and conclusion claim that MammoMix improves 'reliability' and that MoCAE 'adjusts confidence scores to reflect true predictive uncertainty,' the out-of-domain portion of that reliability is encoded in the training target and re-emerges as the calibrated output.

full rationale

Most of the pipeline is not circular. Each YOLOS expert is trained on its own dataset with external ground-truth boxes; the Simple MoE gate is a hard argmax over a dataset-origin classifier and its test numbers reflect actual detector outputs; MoCAE's in-domain calibration uses measured max-IoU targets; Soft NMS and Score Voting are standard fusion operations; and the only self-citation [15] is a related-work mention that is not load-bearing. The one constructed step is the out-of-domain calibration label: the paper assigns IoU=0 to all validation images outside an expert's training domain, fits a Random Forest to these labels, and then states that the calibrator 'learns to assign low confidence to out-of-domain predictions.' That behavior is a restatement of the label construction, not an empirical measurement. Because the abstract and conclusion claim improved 'reliability' and 'true predictive uncertainty,' the out-of-domain portion of that reliability claim reduces to the authors' own zero-assignment. Separately, the actual integrated MammoMix (CNN gate plus MoCAE plus fusion) is never run, and no calibration metric is reported; those are correctness and evidence gaps rather than circularity. The reported accuracy results are largely independent of the circular label choice, so the circularity is partial.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central results rest on four hand-chosen hyperparameters or target assignments (out-of-domain IoU 0, sigma_nms 0.08, RF 300 trees, augmentation ranges), on the undemonstrated assumption that gating can classify visual domain, and on the untested assumption that the calibration regressor accurately predicts IoU. No new physical or architectural entity is introduced beyond the naming of MammoMix. The most consequential assumption is the out-of-domain IoU 0 label, which directly encodes the belief that experts are useless outside their training domain and then is used to prove that calibrated fusion works in mixed domains.

free parameters (4)
  • IoU assignment for out-of-domain calibration samples = 0
    In Section 4.4, the calibrator for each expert is trained with out-of-domain validation images labeled with IoU 0 instead of measured overlap, because ground-truth annotations from other datasets are considered irrelevant. This hand-set target directly shapes all calibrated confidence values and therefore the fusion ranking.
  • Soft NMS sigma_nms = 0.08
    Chosen manually (Section 5) to control Gaussian decay in Soft NMS and score voting. The paper reports no sensitivity analysis, though ensemble output depends strongly on this hyperparameter.
  • Random Forest estimator count = 300
    Selected for the calibration regressors (Section 5) without ablations; calibration curves depend on the regressor capacity.
  • Augmentation hyperparameters = alpha 50, sigma 5, rotation +/-10 degrees, scale +/-0.2, brightness/contrast +/-0.2, noise std 0.05
    All augmentation strengths are hand-set (Section 3.2). They affect the trained experts' specialization and the reported accuracy, and none are ablated.
assumptions (4)
  • domain assumption Dataset origin is visually identifiable from mammogram image appearance.
    The gating CNN (Section 4.3) is trained to classify which dataset an image comes from and routes to the corresponding expert. If images from different centers are not visually separable, the gate cannot select the right expert and the whole specialization argument fails.
  • domain assumption A Random Forest regressor can map image embeddings plus confidence scores to expected maximum IoU with sufficient accuracy.
    Section 4.4 assumes calibrated score = E[IoU | score, embedding] is well approximated by a Random Forest trained on validation predictions. No calibration-error metric is reported, so this premise is unverified.
  • domain assumption YOLOS-base is a suitable detector backbone for all three datasets and outperforms DETR on this task.
    Section 4.1 selects YOLOS based on 'initial experiments' that are not reported in detail. The paper later reports DETR outperforms all MammoMix variants on DDSM, which partially contradicts this choice.
  • domain assumption Standard object-detection metrics with COCO-size categories transfer meaningfully to mammography lesions.
    The paper uses mAP at IoU thresholds and small/medium/large size bins (Section 5) as the primary evaluation; it assumes these aggregates reflect clinical detection quality.
invented entities (1)
  • MammoMix framework (as a distinct integrated method)
    purpose: The title and abstract present MammoMix as a novel framework combining MoE gating plus MoCAE calibration.
    The experiments never instantiate a fused MammoMix model. The paper evaluates MoMo, Simple MoE, and MoCAE separately (Sections 4.2 to 4.4), and no experiment combines the learned gate with the calibrated ensemble. The named entity is therefore a paper-level construct without an experimental realization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection." pith.science (2026). https://pith.science/paper/HSZ4DPWE

@misc{pith2026260810437,
  author       = {Pith},
  title        = {Pith review of: MammoMix: Leveraging Mixture of Experts for Robust Mammogram Breast Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSZ4DPWE}},
  note         = {Machine review of arXiv:2608.10437}
}
read the original abstract

Breast lesion detection in mammography remains a challenging task due to variations in image quality, lesion appearance, and population demographics across datasets. While current object detectors such as YOLO and DETR achieve strong results on individual datasets, their performance often degrades when trained on or applied across heterogeneous sources. To address this, we propose MammoMix, a novel framework based on Mixture-of-Experts (MoE) paradigm for robust and generalizable lesion detection. In MammoMix, each expert model is trained on a specific domain, allowing it to specialize in distinct characteristics of its source data. A gating mechanism adaptively weighs contributions from each expert based on input image, combining their outputs to enable domain-adaptive inference. To improve reliability, we further incorporate a calibration module, MoCAE, which adjusts confidence scores to reflect true predictive uncertainty. We evaluate MammoMix on 3 public mammography datasets: CSAW, DDSM, and DMID, covering diverse clinical settings. Results show that MammoMix outperforms baseline detectors in both average precision and reliability, particularly on datasets with greater variability. Our findings demonstrate that expert specialization and calibrated ensemble fusion significantly enhance model generalization and robustness. MammoMix offers a promising step toward dependable AI-assisted breast cancer screening across real-world clinical domains.

Figures

Figures reproduced from arXiv: 2608.10437 by the authors.

Figure 1
Figure 1. Overview of the MammoMix framework: (a) Expert models specialized on in￾dividual datasets got selected by a CNN gate (b) Calibration module aligns image embeddings and confidence scores with IoU by employing the predictive reliability from Random Forest; then fusion via Soft NMS and Score Voting. tector that serves as the backbone for all approaches evaluated in this study: MoMo, Simple MoE, and MoCaE. We selected Y… view at source ↗
Figure 2
Figure 2. Inference Breakdown Across Multiple Expert Models accross dataset This highlights MoCAE’s ability to provide more balanced and reliable predic￾tions across lesion scales [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

  1. [13]

    arXiv preprint arXiv:2309.14976 (2023), https://doi.org/10.48550/arxiv.2309.14976

    Oksuz,K.,Kuzucu,S.,Joy,T.,Dokania,P.K.:Mocae:Mixtureofcalibratedexperts significantly improves object detection. arXiv preprint arXiv:2309.14976 (2023), https://doi.org/10.48550/arxiv.2309.14976

  2. [1]

    In: European conference on computer vision

    Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European conference on computer vision. pp. 213–229. Springer (2020),https://doi.org/10.1007/ 978-3-030-58452-8_13

  3. [2]

    Radiology294(2), 265–272 (2020)

    Dembrower, K., Liu, Y., Azizpour, H., Eklund, M., Smith, K., Lindholm, P., Strand, F.: Comparison of a deep learning risk score and standard mammographic density score for breast cancer risk prediction. Radiology294(2), 265–272 (2020)

  4. [3]

    arXiv preprint arXiv:2010.11929 (2020),https://doi.org/10.48550/arxiv.2010.11929

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020),https://doi.org/10.48550/arxiv.2010.11929

  5. [4]

    Advances in Neural Information Processing Systems34, 26183–26197 (2021), https://doi.org/10.48550/arxiv.2106.00666

    Fang, Y., Liao, B., Wang, X., Fang, J., Qi, J., Wu, R., Niu, J., Liu, W.: You only look at one sequence: Rethinking transformer in vision through object detec- tion. Advances in Neural Information Processing Systems34, 26183–26197 (2021), https://doi.org/10.48550/arxiv.2106.00666

  6. [5]

    Artificial Intelligence in Medicine132, 102386 (2022)

    Garrucho, L., Kushibar, K., Jouide, S., Diaz, O., Igual, L., Lekadir, K.: Domain generalization in deep learning based mass detection in mammography: A large- scale multi-center study. Artificial Intelligence in Medicine132, 102386 (2022)

  7. [6]

    CA: a cancer journal for clinicians72(6), 524–541 (2022)

    Giaquinto, A.N., Sung, H., Miller, K.D., Kramer, J.L., Newman, L.A., Minihan, A., Jemal, A., Siegel, R.L.: Breast cancer statistics, 2022. CA: a cancer journal for clinicians72(6), 524–541 (2022)

  8. [7]

    Cochrane database of systematic reviews1(6) (2013)

    Gøtzsche, P.C., Jørgensen, K.J.: Screening for breast cancer with mammography. Cochrane database of systematic reviews1(6) (2013)

Show all 15 references
  1. [8]

    In: Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020)

    Hamed, G., Marey, M.A.E.R., Amin, S.E.S., Tolba, M.F.: Deep learning in breast cancer detection and classification. In: Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020). pp. 322–333. Springer (2020)

  2. [9]

    The Lancet Digital Health 2(3), e138–e148 (2020)

    Kim, H.E., Kim, H.H., Han, B.K., Kim, K.H., Han, K., Nam, H., Lee, E.H., Kim, E.K.: Changes in cancer detection and false-positive recall in mammography using artificial intelligence: a retrospective, multireader study. The Lancet Digital Health 2(3), e138–e148 (2020)

  3. [10]

    Scientific data4(1), 1–9 (2017)

    Lee, R.S., Gimenez, F., Hoogi, A., Miyake, K.K., Gorovoy, M., Rubin, D.L.: A curated mammography data set for use in computer-aided detection and diagnosis research. Scientific data4(1), 1–9 (2017)

  4. [11]

    arXiv preprint arXiv:2407.17140 (2024),https://doi.org/10.48550/arxiv.2407.17140

    Lv, W., Zhao, Y., Chang, Q., Huang, K., Wang, G., Liu, Y.: Rt-detrv2: Improved baseline with bag-of-freebies for real-time detection transformer. arXiv preprint arXiv:2407.17140 (2024),https://doi.org/10.48550/arxiv.2407.17140

  5. [12]

    Artificial Intelligence Review42(2), 275–293 (2014)

    Masoudnia, S., Ebrahimpour, R.: Mixture of experts: a literature survey. Artificial Intelligence Review42(2), 275–293 (2014)

  6. [14]

    Biomedical Engineering Letters14(2), 317–330 (2024)

    Oza, P., Oza, U., Oza, R., Sharma, P., Patel, S., Kumar, P., Gohel, B.: Digital mammography dataset for breast cancer diagnosis research (dmid) with breast mass segmentation analysis. Biomedical Engineering Letters14(2), 317–330 (2024)

  7. [15]

    Ultrasound in Medicine & Biology51(6), 999–1007 (2025)

    Zhang, C., Zheng, Y., McAviney, J., Ling, S.H.: Ssat-swin: Deep learning-based spinal ultrasound feature segmentation for scoliosis using self-supervised swin transformer. Ultrasound in Medicine & Biology51(6), 999–1007 (2025)

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.