{"id":"07f32ae2-d80f-4eed-ad50-a059bfbaf088","arxiv_id":"2508.06137","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A benchmark of seven models and four image preprocessing methods on the CBIS-DDSM mammography dataset reports near-perfect CNN accuracy and up to 13-point transformer gains with HOG/AHE, wrapped in a proposed explainable ensemble called MammoFormer.","lead":"This paper benchmarks seven deep learning models and four image preprocessing methods for mammography breast cancer detection, reporting near-perfect accuracy and up to 13 percentage point gains for transformers paired with the right preprocessing. It packages the results into an ensemble called MammoFormer with multiple explainability tools, but provides no code, no data split details, and no external validation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.2's 'artificial testing samples' and missing patient-level split mean the reported 96–99.9% accuracies may not reflect real mammography; the transformer-vs-CNN and deployability claims rest on this unvalidated evaluation set.","rationale":"The reader's weakest_assumption identifies exactly the dataset validity issue: the evaluation set is unbalanced, contains synthetic additions, and lacks a documented patient-level split. The paper's own Section 3.2 states that 'artificial testing samples' were developed, and the final balanced dataset of 1,000 images per class was composed via augmentation. No train/test split methodology is provided, and the number of participants (1,566) versus studies (6,775) in CBIS-DDSM makes patient leakage a real risk. If the test set is contaminated, all quantitative results—and hence the central claim that transformers with feature enhancement can equal or beat CNNs—are unsupported. The paper also admits in Section 3.9 that external validation is needed before clinical deployment, contradicting the 'clinically deployable' framing. This is the most load-bearing concern because it directly undermines the evidence base for the headline claim; other issues (architecture table inconsistencies, unexplained XAI quantification) are secondary. A concrete test—recomputation on a patient-disjoint, original-image-only test set—would settle whether the reported accuracies survive. Since the reader already rejected for this reason, no verdict change is needed.","tokens_in":16419,"tokens_out":3944,"duration_ms":39906,"concrete_test":"Ask the authors to release the exact dataset composition and the train/test partition with participant IDs. Then recompute Table 4 on a patient-disjoint test set containing only original CBIS-DDSM images (no synthetic/artificial samples, no augmentation in test), with class prevalence reflecting clinical reality. If ViT+AHE, Swin+HOG, or ResNet50 accuracy changes by more than the reported <0.3% standard deviation—or if any synthetic images were in the original test set—the claimed 'equal or better than CNN' result fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that the evaluation measures performance on real mammograms. Section 3.2 explicitly states: \"Initial image technical issues prompted our team to develop artificial testing samples during preprocessing.\" It also says the final dataset was balanced to 1,000 images per class via data balancing and augmentation, and that synthetic mammogram images supplemented the training set. The paper never documents a patient-level train/test split; CBIS-DDSM has only 1,566 participants but 6,775 studies, so random image-level splits can leak the same patient into train and test. If the test set includes synthetic or augmented images, the reported 99.9% CNN and 96.3–99.0% transformer accuracies do not measure real-world diagnostic performance, and the central claim that transformers with suitable enhancements can equal or beat CNNs is unsupported. Additionally, Section 3.9 admits external validation is required before clinical deployment, directly undercutting the abstract's \"clinically deployable ensemble system\" claim. This is an internal validity gap, not a disagreement with consensus.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 'MammoFormer,' a framework for binary benign/malignant classification of mammograms that combines seven CNN, transformer, and hybrid architectures with four image preprocessing enhancements (original, negative, AHE, HOG) and five XAI techniques. The central claim is that architecture-specific feature enhancement enables transformer models to match or exceed CNN accuracy (ViT with AHE reaching 98.3%, Swin with HOG gaining 13.0%), and that the resulting weighted-voting ensemble is clinically deployable. Evaluation uses CBIS-DDSM images balanced to 1,000 images per class, with Table 4 reporting accuracies/precision/recall/F1 for each architecture-enhancement combination.","tokens_in":16731,"tokens_out":4638,"duration_ms":50552,"significance":"If the reported results were valid, the paper would offer a useful systematic comparison of architecture-enhancement pairings and a multi-perspective XAI workflow for mammography, and it would provide evidence that transformers can approach CNN performance with appropriate preprocessing. The explicit comparison of seven architectures under four preprocessing settings and the inclusion of five XAI methods are strengths, as is the internal consistency of the headline numbers in Table 4 with the abstract. However, the significance is conditional on the evaluation protocol, which is not documented in a way that supports the central claims; the stated use of artificial testing samples and the absence of a patient-level split make the reported accuracies insufficient to support conclusions about real mammography or clinical deployability.","major_comments":[{"comment":"The evaluation set is not demonstrably a real mammography test set. Section 3.2 states that 'Initial image technical issues prompted our team to develop artificial testing samples during preprocessing' and that the final dataset was balanced to 1,000 images per class via augmentation and synthetic generation. No train/test split or patient-level grouping is documented; CBIS-DDSM has only 1,566 participants but 6,775 studies, so random image-level partitions can leak views from the same patient into both training and test. If synthetic or augmented images appear in the test set, the 99.9% CNN and 96.3–99.0% transformer accuracies in Table 4 do not measure performance on real mammograms. This directly undermines the abstract's central claim that transformers with suitable enhancements equal or beat CNNs on mammography.","section":"§3.2 Dataset and §4 Results"},{"comment":"The text claims 'Table 4 also reports standard deviations, which remain below 0.3% for CNN-based and hybrid models across all enhancements,' but Table 4 contains no standard deviations, confidence intervals, or number of repeated runs; it reports only point estimates. This unquantified robustness claim is load-bearing for recommending specific model–enhancement pairs, especially where differences are small (e.g., ViT AHE 98.3 vs. HOG 99.0; CNN/ResNet at 99.9). The paper also does not state how many independent training runs were performed.","section":"§4 Result Discussion / Table 4"},{"comment":"The abstract and §1 claim MammoFormer is 'a clinically deployable ensemble system,' but §3.9 states that 'external validation on diverse datasets before any clinical deployment' is required, and §5 lists clinical validation as future work. This is an internal contradiction. The current evaluation on a single curated dataset with artificial samples cannot support the deployability claim, and the manuscript itself concedes the missing external validation.","section":"§3.9 Ethics and Clinical Integration vs. Abstract"},{"comment":"The paper asserts that synthetic mammogram images 'accurately mimic' the distinguishing characteristics of benign and malignant presentations, but no fidelity metrics, radiologist evaluation, or distributional comparison are provided. If synthetic images are present in the training set and possibly the test set, the burden is on the authors to show that they are representative of real mammograms; otherwise, the reported accuracies may reflect the synthetic distribution rather than clinical data.","section":"§3.2 Synthetic image realism"}],"minor_comments":[{"comment":"The abstract highlights ViT with AHE at 98.3%, while the conclusion highlights ViT with HOG at 99.0%. Both values appear in Table 4, but the paper should state which pairing is recommended and why, to avoid apparent inconsistency.","section":"Abstract vs. §5 Conclusion"},{"comment":"The phrase 'a 83.3% accuracy increase for Swin Transformer' is confusing; it likely means the absolute accuracy with HOG is 83.3%? Actually Table 4 shows Swin HOG 96.3% and the increase over original is 13.0 percentage points. The sentence should be corrected (e.g., '96.3% accuracy, a 13.0-point improvement over original').","section":"§4 Result Discussion"},{"comment":"Sections 2.1 and 2.2 contain nearly identical paragraphs on HATNet, Abimouloud et al., and TEBLS, with duplicated text. One should be removed or condensed.","section":"§2.1 and §2.2"},{"comment":"References [11] and [16] are the same Lee et al. paper (Transformer-based Deep Neural Network for Breast Cancer Classification on Digital Breast Tomosynthesis Images), and reference [4] has an incomplete author list ('Hussain, et al.' with 'S. Hussain' in text). Please deduplicate and complete the entries.","section":"References"},{"comment":"The Integrated Gradients equation is malformed in the rendering; the integral and gradient notation need to be typeset correctly.","section":"§3.7, Eq. (4)"},{"comment":"The aggregated average accuracies (95.6% for original, 92.2% for negative, 92.0% for AHE) are not reproducible from Table 4 if all seven models are averaged; the paper should state which subset of models is included in each average, or the values should be recalculated.","section":"§4 Aggregated results"}],"recommendation":"reject","confidential_remarks":"The manuscript reads like a student project report that was expanded into a journal article. The self-citations to the first author's prior work are extensive and used to position the novelty, but the primary problem is not novelty—it is that the evaluation cannot support the stated claims because artificial testing samples are used and no patient-level split is documented. This is a load-bearing internal validity gap, not a disagreement with the field's consensus. I recommend rejection rather than major revision because fixing this requires a new evaluation on a properly split, real-image test set, which is beyond a local revision of the current manuscript. I would also flag the contradiction between the 'clinically deployable' framing and the authors' own statement that external validation is required."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the architecture–enhancement matrix is the one real contribution; everything else is packaging. The paper is a systematic sweep of seven architectures and four preprocessing methods on CBIS-DDSM, plus five XAI techniques. That mapping is genuinely useful if you are trying to decide, say, whether to feed HOG features to a Swin Transformer. Credit where due: the authors cover a lot of combinations, and the pattern they report—ViT varying from 54.3% to 99.0% depending on preprocessing—is the kind of concrete result that deserves an explanation.\n\nBut the evaluation protocol is not described well enough to support the headline claims. Section 3.2 says 'Initial image technical issues prompted our team to develop artificial testing samples during preprocessing,' and that synthetic mammograms supplemented the training set. The paper never says whether those artificial samples are in the test set. CBIS-DDSM has only 1,566 participants for 6,775 studies, so without a patient-level split, the same patient can be in train and test. The stress-test note is right: if synthetic or augmented images are in the test set, the 99.9% CNN and 96–99% transformer accuracy numbers are not measuring real mammograms.\n\nThe abstract also calls MammoFormer 'clinically deployable,' but Section 3.9 says external validation is required before any clinical deployment. That is a direct contradiction. There are smaller inconsistencies too: Table 2 lists EfficientNetB3, MobileNetV2, and a Hybrid ViT-CNN, while Table 4 reports DenseTrans, ConvNeXt, and no EfficientNet; the XAI quantitative claims (83% correspondence, 76% distributed attention) are stated without methodology; and no code or data is shared.\n\nThese are not minor blemishes—the evaluation protocol is the load-bearing wall. The central claim that transformers with suitable enhancements can match CNNs may well be true, but this manuscript does not provide the evidence. That said, the matrix itself is internally consistent and worth a serious look if the authors can clarify the split, the synthetic data handling, and release code.\n\nWho this is for: someone who wants a quick scan of which preprocessing tends to help which architecture on CBIS-DDSM. Not for anyone about to deploy a clinical system. I would not desk-reject it—I would send it to a referee asking specifically about evaluation design. As written, it is a reject; with a tightened protocol and data release, the benchmark could be a modest but useful contribution.","headline":"A useful architecture–enhancement matrix undermined by an undocumented evaluation protocol and an overclaim about clinical deployment.","tokens_in":17220,"tokens_out":4373,"would_cite":false,"duration_ms":44191,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MammoFormer claims transformer models can match or beat CNNs on mammograms when each architecture is paired with the right image enhancement, and packages the pairing with explainability into a clinical screening framework.","keywords":["Breast Cancer","Deep Learning","Mammography","Explainable AI (XAI)","Vision Transformers","Swin Transformer","CBIS-DDSM","Feature Enhancement"],"falsifier":"Re-run the evaluation on CBIS-DDSM with a strict patient-exclusive split (no images of the same participant in both training and test) and with every synthetic image removed from the test set, then compare ViT+AHE and Swin+HOG against ResNet-50. If the transformer accuracies drop below the reported 98.3% and 96.3% while CNN accuracy stays near 99.9%, or if the ordering inverts, the paper's central claim that suitable enhancement makes transformers match CNNs is falsified. The simpler pilot: report accuracy on the real-image-only test subset and compare it with the headline numbers.","tokens_in":16354,"feed_emoji":"🩻","tokens_out":11272,"duration_ms":117597,"temperature":0.7,"pith_summary":"The paper's central claim is that transformer models do not intrinsically underperform convolutional networks on mammography; they only appear to when they are fed the wrong image representation. On the authors' evaluation set, the same Vision Transformer moves from 54.3% to 99.0% accuracy depending on preprocessing, Swin Transformer moves from 51.7% to 96.3%, and HOG is the most reliable enhancement overall at 98.4% average accuracy, nearly matching the 99.9% of CNN-based models. MammoFormer is the packaging of this finding: a tiered clinical framework with ResNet-50 on original images as the primary screen, ViT and Swin as global-context validators on contrast- and gradient-enhanced images, weighted voting between them, and five XAI methods for human review. The reason to care is that transformer global-context modeling plus explainability would remove two barriers—long-range dependency failure and black-box opacity—that currently block AI mammography from clinical adoption.","feed_headline":"Preprocessing swing: ViT accuracy 54–99% on mammograms","feed_subtitle":"MammoFormer's architecture-specific enhancement closes the CNN-transformer accuracy gap and adds explainability.","key_machinery":"The load-bearing mechanism is architecture–enhancement pairing: each model family is matched to the preprocessing that exposes the features its mechanism can use. HOG (histogram of oriented gradients) emphasizes edge and gradient structure, which suits Swin's shifted-window, patch-based attention and gives the reported 13.0% improvement; AHE (adaptive histogram equalization) normalizes dense-tissue contrast, which suits ViT's global patch comparison and restores it to 98.3%. Around that pairing, MammoFormer wraps a weighted-voting ensemble—ResNet-50 on original images as the high-accuracy primary tier, ViT and Swin as complementary contextual validators—and five XAI attributions (Integrated","core_discovery":"On its own terms, MammoFormer claims to close the CNN-transformer gap in mammography by systematic architecture-specific enhancement. It evaluates seven architectures (custom CNN, ResNet-50, ViT, Swin, DenseTransformer, ConvMixer, ConvNeXt) under four preprocessings (original, negative, AHE, HOG) on a balanced CBIS-DDSM subset enriched with synthetic images. The finding is that pure transformers are highly preprocessing-sensitive—ViT ranges from 54.3% to 99.0%, Swin from 51.7% to 96.3%—while CNN-based models stay at 99.9% across settings. Matching enhancement to architecture gives ViT 98.3% with AHE and Swin 96.3% with HOG, a 13.0% improvement; HOG is the most reliable overall at 98.4% avera","pith_inferences":["Beyond the paper, the reported 44.7-point range in ViT accuracy across preprocessings implies that any published transformer-vs-CNN comparison on medical images is only interpretable when the enhancement protocol is fixed and reported; preprocessing effects can swamp architectural differences.","Because the paper notes it 'developed artificial testing samples' and that one participant can carry multiple patient IDs, a patient-exclusive split with synthetic images kept out of the test set is the natural next experiment before the clinical claims are taken at face value—this is the paper's own external-validation call made concrete.","The architecture–enhancement matching principle could extend to ultrasound and MRI as the authors propose, but the specific pairings (HOG for Swin, AHE for ViT) would need to be re-derived per modality rather than transferred unchanged.","If the numbers survive external validation, the tiered ensemble suggests a cost-conscious clinical workflow: high-confidence CNN reads pass through, and transformer+XAI resources are spent only on disagreement or ambiguity—an allocation testable for sensitivity, specificity, and radiologist workload."],"forward_implications":["A screening workflow can put ResNet-50 on original images first (99.9% accuracy), with ViT and Swin validators triggered only for ambiguous cases—this is the deployment MammoFormer describes.","Transformer models on mammograms no longer need to be written off: on the paper's data, preprocessing alone moves ViT from 54.3% to 99.0% and Swin from 51.7% to 96.3%.","HOG is the most generally reliable preprocessing (98.4% average accuracy), so gradient-based input representations should be a candidate for any transformer-based medical image pipeline.","Multi-perspective explainability, with Occlusion best aligned to radiologist workflow, is integral rather than optional to the framework's clinical claims.","The design rule that emerges is architecture-specific enhancement pairing rather than applying one preprocessing to all models."],"supporting_citations":[{"why":"Supplies HATNet, the end-to-end transformer backbone that matches pathologist performance and links attention maps to known tissue structures, establishing the transformer baseline MammoFormer builds on.","marker":"[1]"},{"why":"Shows ViT-based transfer learning can reach perfect classification metrics on standardized mammography datasets, the strongest prior evidence that transformers can excel in this domain.","marker":"[2]"},{"why":"Provides the multi-modal transformer (MMT) with AUROC 0.943 on 1.3 million exams and the caution that transformer fusion requires precise architectural design, motivating MammoFormer's tiered ensemble.","marker":"[6]"},{"why":"Demonstrates four-image transformer models outperform multi-view CNNs (AUC 0.818 vs 0.784), the key cited evidence that transformers add value in mammography.","marker":"[7]"},{"why":"Introduces MV-Swin-T, a multi-view Swin transformer for mammography, supporting the paper's use of Swin as a global-context validator.","marker":"[9]"},{"why":"Shows Swin transformers reach AUC 0.934 on digital breast tomosynthesis and outperform ResNet101 and vanilla ViT, justifying Swin's inclusion and its enhancement sensitivity.","marker":"[10]"},{"why":"TEBLS combines Swin transformer blocks with Grad-CAM for breast lesion segmentation, the cited precedent that transformer explainability is feasible and that MammoFormer's XAI layer extends.","marker":"[13]"},{"why":"Documents the gap MammoFormer targets: regular ViTs underperform CNNs (93.4% vs 95.59%) until given unlabeled data and token sampling, the closest prior statement of the transformer optimization problem.","marker":"[15]"}],"fun_headline_variants":["MammoFormer: matching preprocessing to transformer architecture lifts accuracy to 98%","ViT jumps from 54% to 99% on mammograms with AHE in MammoFormer","MammoFormer closes CNN-transformer gap with architecture-specific boosts","HOG lifts Swin by 13% in MammoFormer mammography detection","Explainable MammoFormer: transformers rival CNNs with right preprocessing"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that the balanced, synthetically augmented mammogram set used for evaluation—including the 'artificial testing samples' the authors created and without a documented patient-level train/test split—represents real screening conditions closely enough that the reported 96–99.9% accuracies reflect genuine model ability rather than the way the test set was assembled.","fun_headline_variants_meta":{"raw":{"variants":["MammoFormer: matching preprocessing to transformer architecture lifts accuracy to 98%","ViT jumps from 54% to 99% on mammograms with AHE in MammoFormer","MammoFormer closes CNN-transformer gap with architecture-specific boosts","HOG lifts Swin by 13% in MammoFormer mammography detection","Explainable MammoFormer: transformers rival CNNs with right preprocessing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001406,"raw_usage":{"total_tokens":5562,"prompt_tokens":830,"completion_tokens":4732,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":4627}},"tokens_in":574,"tokens_out":4732,"duration_ms":37008,"temperature":1.0,"reasoning_tokens":4627,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:54:14.409961+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the evaluation on CBIS-DDSM with a strict patient-exclusive split (no images of the same participant in both training and test) and with every synthetic image removed from the test set, then compare ViT+AHE and Swin+HOG against ResNet-50. If the transformer accuracies drop below the reported 98.3% and 96.3% while CNN accuracy stays near 99.9%, or if the ordering inverts, the paper's central claim that suitable enhancement makes transformers match CNNs is falsified. The simpler pilot: report accuracy on the real-image-only test subset and compare it with the headline numbers.","supporting_citations":[{"cited_title":"Ayana, et al., ―Vision -Transformer-Based Transfer Lear n- ing for Mammogram Classification,‖ Diagnostics (Basel), vol","cited_arxiv_id":null,"evidence_quote":"Shows ViT-based transfer learning can reach perfect classification metrics on standardized mammography datasets, the strongest prior evidence that transformers can excel in this domain."},{"cited_title":"Leveraging Transformers to Improve Breast Cancer Classification and Risk Assessment with Multi-modal and Longitudinal Data","cited_arxiv_id":"2311.03217","evidence_quote":"Provides the multi-modal transformer (MMT) with AUROC 0.943 on 1.3 million exams and the caution that transformer fusion requires precise architectural design, motivating MammoFormer's tiered ensemble."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates four-image transformer models outperform multi-view CNNs (AUC 0.818 vs 0.784), the key cited evidence that transformers add value in mammography."},{"cited_title":"Kassis, et al., ―Detection of breast cancer in digital breast tomosynthesis with vision transformers,‖ Scientific Reports, vol","cited_arxiv_id":null,"evidence_quote":"Shows Swin transformers reach AUC 0.934 on digital breast tomosynthesis and outperform ResNet101 and vanilla ViT, justifying Swin's inclusion and its enhancement sensitivity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TEBLS combines Swin transformer blocks with Grad-CAM for breast lesion segmentation, the cited precedent that transformer explainability is feasible and that MammoFormer's XAI layer extends."},{"cited_title":"Wang, et al., ―S emi-supervised vision transformer with adaptive token sampling for breast cancer classification,‖ Frontiers in Pharmacology, vol","cited_arxiv_id":null,"evidence_quote":"Documents the gap MammoFormer targets: regular ViTs underperform CNNs (93.4% vs 95.59%) until given unlabeled data and token sampling, the closest prior statement of the transformer optimization problem."}],"review_version":1}