A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans

Daniel Steinberg; Justin Yiu; Kushank Arora; Rohit Ghiya

REVIEW 2 major objections 1 minor 1 references

A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans

T0 review · 2 major / 1 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ResNet50, in a systematic comparison, excels at knee MRI ROI detection, with CNN transfer learning emerging as the most effective approach on the MRNet dataset.

desk verdict The abstract and the full text are two different papers; as submitted, the MRI study does not exist, so it cannot go to peer review. read the letter →

arxiv 2508.14151 v2 pith:UVAZLFPC submitted 2025-08-19 eess.IV cs.AIcs.CV

Justin Yiu , Kushank Arora , Daniel Steinberg , Rohit Ghiya This is my paper · ORCID

classification eess.IVcs.AIcs.CV

keywords kneeMRIdeeplearningtransferResNet50visiontransformerU-NetexplainableAIGrad-CAM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to determine which deep learning architecture best locates regions of interest in knee MRI scans and whether explainable AI methods can make the detections interpretable to clinicians. It compares CNN transfer learning (ResNet50, InceptionV3), vision transformers, and U-Net variants with MLP classifiers on the MRNet dataset, measuring classification with AUC and reconstruction with PSNR/SSIM. The authors find that ResNet50 consistently leads in classification and ROI identification, and that Grad-CAM gives the most clinically meaningful explanations. If the result holds, it suggests that for moderate-sized medical imaging datasets, CNN transfer learning remains a stronger default than transformers, and that interpretability tools like Grad-CAM are a practical complement.

What carries the argument

The primary mechanism is transfer learning with pretrained CNN backbones, with ResNet50 carrying most of the weight. The U-Net + MLP hybrids are meant to supply spatial reconstruction features to the classifier, while Grad-CAM provides the attribution maps that localize the region of interest.

What would settle it

A head-to-head comparison on a second, independent knee MRI dataset (ideally multi-center with radiologist-annotated ROIs) where a vision transformer or U-Net hybrid matches or beats ResNet50 in AUC and clinician-rated localization accuracy would undercut the claim that CNN transfer learning is the most effective approach.

Watch

Extended reading notes

Core claim

The central claim is that, on the MRNet knee MRI dataset, CNN-based transfer learning—especially ResNet50—outperforms transformer-based models and hybrid U-Net + MLP designs in both slice classification and region-of-interest identification. The authors support this with AUC comparisons, reconstruction quality metrics (PSNR/SSIM), and qualitative Grad-CAM and saliency visualizations. They also claim that Grad-CAM explanations are the most clinically meaningful across all tested architectures. This is presented as an empirical result, not a theoretical one, and is limited to the dataset and experimental setup used.

Load-bearing premise

The assumption that the MRNet dataset and the chosen metrics (AUC, PSNR/SSIM, qualitative visualizations) are representative of real-world knee MRI ROI detection, so that ResNet50's advantage generalizes beyond this single dataset.

Editorial extensions

If this is right

For similar medical imaging datasets of moderate size, starting with a pretrained CNN like ResNet50 is a reliable baseline.
Transformer models may require larger-scale pretraining to catch up, as the authors note.
Grad-CAM can be used alongside CNN classifiers to produce ROI visualizations that clinicians can inspect.
Reconstruction-based features from U-Net hybrids do not yet translate into better classification performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The conclusion that CNNs beat transformers is contingent on dataset scale; with larger pretraining, transformers may close the gap, which the authors themselves flag.
The finding that Grad-CAM explanations are most clinically meaningful is based on qualitative inspection; a formal clinical study with radiologist evaluation would be needed to verify.
Because only MRNet is used, transferability to other joints or MRI protocols is untested; a multi-site evaluation would be a natural next step.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this before reading: the PDF on arXiv does not match its own abstract. The abstract describes a knee MRI deep-learning benchmark; the body is a status report on an X-ray CCD camera for the AXIS mission. It's a submission/upload mismatch, but for peer review that's fatal: there is no paper here to evaluate.

What's actually in the body is a reasonable technical update on detector design, with CCD and ASIC test data and clear figures. If that's the intended submission, it's a legitimate hardware report. But it has nothing to do with the MRI claims.

The MRI abstract, taken alone, is a routine benchmark comparison: standard CNNs and ViTs, Grad-CAM, MRNet dataset. No new method, no theory, single dataset, no numbers in the abstract. Even if the full text matched, it would be a modest empirical study. As submitted, the central claim that ResNet50 excels is entirely unsupported because the supporting sections are not in the document. The X-ray content is not a response to the MRI abstract, so the issue isn't a missing reference or a circular step—it's the absence of the actual study.

The reader's UNVERDICTED verdict is right. I'd go further: this is incoherent on its own terms, not just unverified. A serious referee would have nothing to engage with, and any review would be of a paper that isn't there.

Who is this for? A desk editor deciding whether to send out: this should go back to the authors to resolve the mismatch, not to reviewers. The MRI study may deserve a referee if it ever exists; the X-ray report might deserve one for a hardware venue. But as it stands, neither claim can be reviewed.

Recommendation: desk reject or return to authors. If a corrected version arrives with the right PDF, the MRI benchmark might merit a quick look, but I would not hold high expectations.

Referee Report

2 major / 1 minor

Summary. The abstract of arXiv:2508.14151 announces a systematic evaluation of deep learning models and xAI methods for region-of-interest detection in knee MRI scans, claiming that ResNet50 and CNN-based transfer learning perform best on the MRNet dataset using AUC, PSNR/SSIM, and qualitative visualizations. The full text of the manuscript, however, is a completely different paper titled "The high-speed X-ray camera on AXIS: design and performance updates," describing CCD detectors, ASIC readout, and the AXIS mission. There is no mention of MRI, ResNet50, U-Net, Grad-CAM, MRNet, or any of the claimed experiments. The submitted manuscript therefore contains no supporting content for its abstract's claims.

Significance. If the study described in the abstract were actually present, it could provide a useful empirical comparison of CNN and transformer architectures with explainability methods for knee MRI ROI detection, and the claim that CNN transfer learning outperforms transformers on a modest dataset would be a plausible, falsifiable contribution. However, the manuscript as submitted contains none of that study. No dataset, architectures, training details, evaluation metrics, or results appear anywhere in the full text. Consequently, the scientific significance cannot be assessed; the submitted content does not support its own stated contribution. The paper in its current form cannot be reviewed as a research contribution to the field.

major comments (2)

[Full Text (entire manuscript body)] The full text is an unrelated paper about the AXIS high-speed X-ray camera (CCDs, ASIC readout, and mission status). The abstract claims a knee MRI deep learning study with ResNet50, InceptionV3, ViT, U-Net variants, Grad-CAM, saliency maps, AUC, PSNR/SSIM, and the MRNet dataset. None of these appear in the full text. This is not a minor editorial issue: the central claim of the manuscript has no supporting content in the manuscript itself.
[Full Text vs. Abstract] Because the full text contains no experiments, no dataset description, no model implementations, no training procedure, and no quantitative results, the abstract's statement that "ResNet50 consistently excels" and that "CNN-based transfer learning emerges as the most effective approach" is entirely unverifiable. The manuscript cannot reproduce, check, or even locate the claimed study. Any evaluation of soundness is impossible.

minor comments (1)

[Title and metadata] The title "A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans" does not match the content of the submitted full text. The full text also carries its own abstract and keywords, indicating that two distinct papers are conflated, likely due to a submission error.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the abstract's MRI/deep-learning claims have no supporting derivation in the manuscript, which is an unrelated X-ray camera paper.

full rationale

The paper as submitted has no derivation chain connecting the abstract's claims to any equations, fitted parameters, or self-citations. The abstract describes a systematic evaluation of ResNet50, InceptionV3, ViT, U-Net variants, Grad-CAM, and saliency maps on knee MRI from the MRNet dataset. The full text supplied is a completely different paper, 'The high-speed X-ray camera on AXIS: design and performance updates,' concerning CCDs, ASIC readout, and X-ray test facilities. There is no mention of MRI, ResNet50, U-Net, Grad-CAM, MRNet, AUC, PSNR, or SSIM in the body text. Consequently, there is no specific reduction from a purported result back to its input by construction, no fitted parameter renamed as a prediction, and no load-bearing self-citation that forces the conclusion. The lack of supporting content is a serious manuscript-level integrity and completeness issue, but it is not circularity under the required definition: no equation or fitted value is available to examine. Accordingly, the circularity score is 0, with the caveat that the abstract's scientific claims are entirely unverified by the submitted full text.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

Only the abstract is available for review because the manuscript text is a different paper. The assumptions listed are the minimal domain assumptions required by the abstract.

assumptions (2)

domain assumption The MRNet dataset is representative of knee MRI ROI detection tasks.
The abstract uses MRNet for all evaluations, so the generalization of the conclusions depends on this assumption.
domain assumption AUC, PSNR, and SSIM are appropriate evaluation metrics for ROI detection and reconstruction quality.
The abstract reports these metrics without justifying their clinical relevance or task appropriateness.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans." pith.science (2026). https://pith.science/paper/UVAZLFPC

@misc{pith2026250814151,
  author       = {Pith},
  title        = {Pith review of: A Systematic Study of Deep Learning Models and xAI Methods for Region-of-Interest Detection in MRI Scans},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UVAZLFPC}},
  note         = {Machine review of arXiv:2508.14151}
}

read the original abstract

Magnetic Resonance Imaging (MRI) is an essential diagnostic tool for assessing knee injuries. However, manual interpretation of MRI slices remains time-consuming and prone to inter-observer variability. This study presents a systematic evaluation of various deep learning architectures combined with explainable AI (xAI) techniques for automated region of interest (ROI) detection in knee MRI scans. We investigate both supervised and self-supervised approaches, including ResNet50, InceptionV3, Vision Transformers (ViT), and multiple U-Net variants augmented with multi-layer perceptron (MLP) classifiers. To enhance interpretability and clinical relevance, we integrate xAI methods such as Grad-CAM and Saliency Maps. Model performance is assessed using AUC for classification and PSNR/SSIM for reconstruction quality, along with qualitative ROI visualizations. Our results demonstrate that ResNet50 consistently excels in classification and ROI identification, outperforming transformer-based models under the constraints of the MRNet dataset. While hybrid U-Net + MLP approaches show potential for leveraging spatial features in reconstruction and interpretability, their classification performance remains lower. Grad-CAM consistently provided the most clinically meaningful explanations across architectures. Overall, CNN-based transfer learning emerges as the most effective approach for this dataset, while future work with larger-scale pretraining may better unlock the potential of transformer models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

1 extracted references · 1 canonical work pages

[1]

PA, USA gSouthwest Research Institute, San Antonio, TX, USA hNASA Goddard Space Flight Center, Greenbelt, MD, USA iDepartment of Astronomy, University of Maryland, College Park, MD, USA ABSTRACT AXIS, a Probe mission concept now in a Phase A study, will provide transformative studies of high-energy astrophysical phenomena thanks to its high-resolution X-r...

work page Pith review arXiv 2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

PA, USA gSouthwest Research Institute, San Antonio, TX, USA hNASA Goddard Space Flight Center, Greenbelt, MD, USA iDepartment of Astronomy, University of Maryland, College Park, MD, USA ABSTRACT AXIS, a Probe mission concept now in a Phase A study, will provide transformative studies of high-energy astrophysical phenomena thanks to its high-resolution X-r...

work page Pith review arXiv 2025