{"id":"4f08b84f-83d9-414a-9058-b6a7158a8708","arxiv_id":"2411.15802","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MST wraps DINOv2 slice features in a transformer aggregator; it beats a 3D ResNet in AUC on breast and knee MRI, shows a non-significant trend on chest CT, and produces more accurate attention-based localization.","lead":"A method called Medical Slice Transformer (MST) adapts the 2D self-supervised model DINOv2 to 3D medical scans by analyzing one slice at a time and then combining slice features with a small transformer. It reports higher diagnostic accuracy and more precise saliency maps than a 3D ResNet across breast MRI, chest CT, and knee MRI.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Patient-level data leakage in train/test split could invalidate the breast and chest AUC comparisons; the paper does not state that splitting was stratified by patient.","rationale":"The paper's strongest claim is the AUC superiority on three datasets. The explainability claim is also important, but even if it held, the diagnostic claim is the quantitative core. The ambiguity in the data split unit creates a concrete risk that the breast and chest results are inflated by patient leakage. This is a correctness issue, not a matter of consensus. The test is straightforward because code is provided. If the split is patient-level, the concern is resolved and the reader's CONDITIONAL verdict could stand. If not, the central claim collapses to a single dataset (knee), and the abstract's statement that MST achieved higher AUC across all three datasets would be misleading. I therefore identify this as the single most load-bearing concern, disagreeing with the reader's focus on explainability, though that concern is also valid. The proposed test is objective and directly checks the code's data-splitting logic.","tokens_in":12736,"tokens_out":9995,"duration_ms":86687,"concrete_test":"Check the data-splitting code in the public GitHub repository (https://github.com/mueller-franzes/MST) to determine whether the 80/20 train-validation/test split for the DUKE and LIDC-IDRI experiments is stratified at the patient level or at the individual side/nodule level. If the split is performed per side (DUKE) or per nodule (LIDC), rerun the experiments with a patient-level split (e.g., using GroupShuffleSplit on patient IDs) and recompute the AUC differences and DeLong P-values. If the breast AUC difference drops below significance or the effect sizes shrink substantially, the central claim is not robust.","verdict_should_be":"REJECT","load_bearing_attack":"The central diagnostic claim rests on AUC comparisons between MST and a 3D ResNet on two datasets where the unit of analysis may not be the patient. For DUKE, the paper states that each MRI study was separated into left and right breast image stacks, so the classification unit is a lateralized stack. For LIDC-IDRI, the paper reports 1,625 nodules from 722 patients, so the classification unit is a nodule. The Methods say 'we divided the samples into training-validation and test sets with an 80% to 20% split, stratified by label distribution' without stating that the split was performed at the patient level. If the split is at the stack/nodule level, the same patient can contribute data to both training and test sets. Because images from the same patient share scanner properties, anatomy, and disease status, this creates a form of data leakage. It can inflate AUC and violate the independence assumption underlying DeLong's test, potentially making the reported P=0.02 for breast (and the P-values for chest) unreliable. If the breast result no longer holds under a patient-level split, only the knee result (which used a separate public validation set) remains significant, and the claim that MST outperforms ResNet 'across all three datasets' is not supported. The paper does not explicitly describe a patient-stratified split.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the Medical Slice Transformer (MST), a framework that adapts the 2D self-supervised foundation model DINOv2 to 3D medical image classification by processing individual slices with a 2D encoder and aggregating slice features with a lightweight Transformer. The authors evaluate MST against a 3D ResNet50 on three public datasets (breast MRI, chest CT, knee MRI) for diagnosis, reporting higher AUC for MST on all three, with P-values of 0.02, 0.13, and 0.001 respectively. They also present a radiologist's subjective evaluation of saliency maps, reporting that MST attention maps are more often correct in highlighting the slice and lesion core than ResNet Grad-CAM maps. The paper includes ablation studies on the slice aggregation mechanism, positional encoding, backbone size, freezing, and register tokens, and provides public code and data links.","tokens_in":13030,"tokens_out":3233,"duration_ms":29188,"significance":"If the central claims hold, the paper would be a useful contribution: it demonstrates a surprisingly simple and parameter-efficient way to repurpose 2D self-supervised vision transformers for 3D medical imaging, with favorable diagnostic performance and potentially more interpretable attention maps than post-hoc Grad-CAM. The strengths include the use of multiple public datasets, a controlled preprocessing pipeline, head-to-head comparison with a standard 3D CNN, and a thorough ablation study. The code is publicly available, which supports reproducibility. However, the significance is tempered by two load-bearing issues: the chest CT comparison is not statistically significant, and the train/test split for the breast and chest datasets may not be patient-stratified, which could invalidate the significance tests. The explainability claim rests on a single-reader subjective assessment without inter-rater reliability.","major_comments":[{"comment":"The train/test split is described as an 80/20 split stratified by label distribution, but the manuscript does not state that the split was performed at the patient level. For DUKE, each MRI study is separated into left and right breast stacks, and for LIDC-IDRI the classification unit is the individual nodule (1,625 nodules from 722 patients). If the split is applied to stacks or nodules rather than patients, the same patient can contribute samples to both training and test sets. This creates data leakage that can inflate AUC and violates the independence assumption underlying DeLong's test, potentially making the reported P=0.02 for breast unreliable. Please clarify the splitting unit; if it is not patient-level, re-run the experiments with patient-stratified splitting and report whether the breast and chest results still hold.","section":"Methods, Model Implementation and Training; Methods, Dataset Collection and Preprocessing"},{"comment":"The chest CT comparison yields P=0.13, which is not statistically significant at the conventional 0.05 threshold. The abstract and conclusion state that \"MST achieved higher AUC values compared to ResNet across all three datasets,\" which overstates the evidence: only the knee result is significant as reported, and the breast result is significant only if the patient-leakage issue is resolved. Please qualify the claim by reporting the chest result as a non-significant trend, and consider presenting effect sizes and confidence intervals for all three comparisons.","section":"Results, MST Outperforms Standard CNN Architectures; Abstract; Discussion"},{"comment":"The explainability evaluation is based on a single radiologist's blinded review of 50 saliency maps per dataset, with no inter-rater reliability measure (e.g., Cohen's kappa) and no quantitative localization metric (e.g., overlap with lesion masks). Because attention maps and Grad-CAM images are visually distinct, the rater may be effectively unblinded. This evidence is insufficient to support the claim of \"substantial improvement in model explainability.\" Please add multiple readers with agreement statistics or a quantitative evaluation against ground-truth lesion locations, or temper the conclusion to reflect the single-reader nature of the assessment.","section":"Results, MST Provides Better Model Explainability; Methods, Saliency Maps"}],"minor_comments":[{"comment":"The table captions state \"Results are shown as mean accuracy ± standard deviation,\" but the values reported (e.g., 0.94±0.01 for breast MRI) are evidently AUC values, matching the AUC results in the text and Figure 2. Please correct the captions to say \"AUC\" or clarify the metric.","section":"Tables 2 and 3"},{"comment":"The sentence \"The color coding toward blue indicates low attention, while the spectrum toward red indicates high attention\" appears in multiple figure captions; consider stating it once in the main text and referring to it in captions.","section":"Methods, Saliency Maps"},{"comment":"Reference 22 includes several authors of this paper. While this is a legitimate related work, the self-citation is noticeable; please ensure the citation is necessary and not inflated by self-reference.","section":"References"},{"comment":"The split description would benefit from specifying the random seed and whether the stratification included any additional variables beyond the label (e.g., patient, scanner), which would help reproducibility.","section":"Methods, Model Implementation and Training"}],"recommendation":"major_revision","confidential_remarks":"The patient-level leakage concern is the most serious issue; if the current split is not patient-stratified, the breast and chest AUC comparisons may be invalid. The explainability claim also needs stronger evidence. The paper is otherwise well-structured and the code release is a plus. This is a borderline case; major revision with re-analysis could make it suitable, but if the leakage is confirmed, the 'all three datasets' claim collapses and the paper would need substantial reframing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Colleague],\n\nQuick take: this paper is worth reading for its clean ablation design, but the headline claim that MST beats 3D ResNet 'across all three datasets' does not hold up as stated. The knee result is solid. The breast and chest results are questionable.\n\nWhat's new: the paper adapts DINOv2 to 3D volumes by feeding 2D slices through a DINOv2 backbone and passing the CLS tokens through a small slice transformer. The systematic evaluation across three public datasets (breast MRI, chest CT, knee MRI) with code and data available is a real contribution. The ablations (linear vs. transformer aggregation, positional encoding, backbone size, registers, freezing) give useful practical guidance. The fine-tuning finding – freezing DINOv2 collapses AUC to ~0.62-0.66 – is a clear, honest negative result.\n\nThe soft spots are not minor. First, for DUKE and LIDC, the classification unit is not the patient: DUKE splits each study into left and right stacks, and LIDC works on individual nodules (1,625 from 722 patients). The Methods say the split was 'stratified by label distribution' but never say it was patient-level. If the split is at stack/nodule level, the same patient can be in both training and test, inflating AUC and breaking the independence assumption behind DeLong's test. That could make the breast P=0.02 and the chest P=0.13 unreliable. Only the knee dataset is clean, because it uses the public validation set as a separate test set. This is a load-bearing problem for the paper's central claim.\n\nSecond, the explainability evaluation relies on a single radiologist's ratings of 150 saliency maps, with no inter-rater variability and a likely unblinding issue: attention maps look very different from Grad-CAM heatmaps. The large gap (136 vs 37 slice-correct) is suggestive, but the 'substantial improvement' claim needs more than one reader.\n\nThird, the paper does not compare against prior slice-transformer approaches (M3T, Medical Transformer, Alp et al.) using the same backbones, so we don't know how much DINOv2 specifically adds beyond a generic 2D encoder.\n\nBottom line: the authors did a lot right – public code, public data, thoughtful ablations – but the comparative accuracy claim is not yet supported for two of the three datasets. I'd send it to review, but with a clear request: re-run the breast and chest experiments with a patient-level split and report multi-reader saliency agreement.\n\nYes, bring it to the reading group to talk about data leakage in medical imaging.","headline":"Useful architecture paper whose headline accuracy claim is compromised by a likely patient-level data leak in two of three datasets.","tokens_in":13574,"tokens_out":4744,"would_cite":false,"duration_ms":38536,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A slice transformer adapts 2D DINOv2 to 3D scans and beats 3D ResNet","keywords":["Medical Slice Transformer","DINOv2","3D medical image analysis","self-supervised learning","explainability","saliency maps","MRI","CT"],"falsifier":"Have a second panel of radiologists rate the identical saliency maps with model identity hidden; if the 136 of 150 versus 37 of 150 slice-correctness split does not reproduce, the explainability claim is falsified. A quantitative check would compute overlap between MST combined attention maps and expert lesion segmentations on the same volumes.","tokens_in":12570,"feed_emoji":"🩻","tokens_out":6907,"duration_ms":57327,"temperature":0.7,"pith_summary":"The paper claims that the Medical Slice Transformer (MST), which applies the 2D self-supervised model DINOv2 to each slice of a 3D volume and then aggregates the slices with a Transformer, outperforms a standard 3D ResNet on three diagnostic tasks: breast MRI, chest CT, and knee MRI. The reported AUC improvements are 0.94 vs 0.91 for breast, 0.95 vs 0.92 for chest, and 0.85 vs 0.69 for knee, with the breast and knee differences reaching statistical significance. The same attention mechanism that drives classification also produces saliency maps: multiplying the slice-level attention of the Transformer with the within-slice attention of DINOv2 highlights both the relevant slice and the core of the lesion. A blinded radiologist rated these MST maps as correctly locating the lesion slice in 136 of 150 cases and the lesion core in 57 of 150 cases, against 37 and 0 for ResNet's Grad-CAM maps. If confirmed, the approach matters because it lets 2D self-supervised models that need no 3D annotations carry the diagnostic and localization burden in volumetric imaging.","feed_headline":"Slice transformer lifts DINOv2 to 3D scans, beating ResNet","feed_subtitle":"On breast, lung, and knee volumes, MST raises AUC and localizes lesions far more often than Grad-CAM","key_machinery":"The central object is the Medical Slice Transformer (MST), a two-stage architecture in which a pre-trained 2D image encoder (DINOv2) converts each slice of a 3D volume into a feature vector, and a sequence Transformer aggregates those vectors with a classification token. The mechanism carrying the argument is the combined attention map: the Transformer's normalized attention over slices tells which slices contain the finding, the DINOv2 encoder's patch attention tells where within the slice, and their product produces a saliency map aligned to the input volume. This single mechanism simultaneously enables classification and explainability, which is what lets the paper claim that MST addresses both the annotation-scarcity and black-box problems in 3D medical imaging.","core_discovery":"On its own terms, the paper's central discovery is that the MST framework — a DINOv2 encoder that maps each 2D slice to a feature vector plus a single-layer Transformer that reads the sequence of slice vectors — transfers DINOv2's strong 2D representations to 3D volumes and makes the model's reasoning visible. The classification token of the slice Transformer yields the diagnosis, and its attention over slices, multiplied by DINOv2's patch-level attention, yields a combined saliency map that localizes findings. The paper reports that MST beats 3D ResNet50 in AUC on all three datasets and that its saliency maps are markedly more precise, with Grad-CAM rated as pointing at lesion cores in 0 of 150 cases versus 57 of 150 for MST. Ablations show the Transformer aggregation, not the DINOv2 features alone, is what carries the gain: replacing the Transformer with a linear layer or averaging feature vectors lowers AUC, and freezing DINOv2's weights collapses performance on breast and chest.","pith_inferences":["I read the chest CT result (P=0.13) as the paper's weak statistical link; the burden of proof for the diagnostic claim rests mainly on breast and knee, and a replication with more chest cases would sharpen the picture.","A natural extension the paper does not test is using the combined attention map as a weakly supervised segmentation, e.g., comparing it to expert outlines with an overlap metric; the paper only measures slice- and core-level correctness.","Because the explainability rating involved a single radiologist, an obvious replication is an inter-observer study with several radiologists; if ratings vary widely, the explainability advantage would need to be redefined.","The ablation showing frozen DINOv2 collapses to 0.62 AUC on breast suggests that fine-tuning the 2D encoder to medical data does much of the work; a separate experiment varying the amount of fine-tuning data would separate the contribution of pretraining from the contribution of the Transformer aggregation."],"forward_implications":["2D self-supervised models pretrained on natural images can be lifted to 3D medical diagnosis without 3D pretraining, reducing the need for large annotated volumetric datasets.","A single forward pass yields both a diagnosis and a coarse lesion localization, so the explainability signal does not require a separate segmentation or detection model.","The slice Transformer is the load-bearing component: replacing it with linear aggregation or feature averaging lowers AUC, and freezing the DINOv2 backbone degrades performance sharply on breast MRI and chest CT.","On the knee MRI task the gain is large (0.85 vs 0.69 AUC), suggesting the benefit is greatest where the 3D context across slices carries diagnostic information.","Attention-based saliency maps point to lesion cores in 57 of 150 cases where Grad-CAM points to none, which if reproducible would give clinicians a localization cue that convolution-based heatmaps do not provide."],"supporting_citations":[{"why":"Supplies the DINOv2 2D self-supervised feature extractor that MST adapts to 3D volumes.","marker":"(17)"},{"why":"Supplies the 3D ResNet50 architecture used as the convolutional baseline for comparison.","marker":"(28)"},{"why":"Supplies Grad-CAM, the baseline saliency method whose maps are compared against MST's attention maps.","marker":"(8)"},{"why":"Supplies the scaled-dot-product attention mechanism used for the within-slice and slice-level saliency maps.","marker":"(12)"},{"why":"Supplies the Transformer sequence-aggregation architecture used by the slice Transformer.","marker":"(11)"},{"why":"Supplies the breast MRI dataset and labels for the breast cancer detection task.","marker":"(25)"},{"why":"Supplies the chest CT dataset and nodule labels for the lung nodule dignity task.","marker":"(26)"},{"why":"Supplies the knee MRI dataset and meniscus tear labels for the knee task.","marker":"(27)"},{"why":"Supplies the statistical test used to compare the correlated AUC values between models.","marker":"(41)"}],"fun_headline_variants":["DINOv2 goes 3D: slice transformer tops ResNet on MRI, CT","Adapting DINOv2 to 3D: MST beats ResNet in diagnosis and maps","Slice transformer with DINOv2 lifts 3D diagnostics and saliency","MST: DINOv2 Transformer for 3D scans beats ResNet on all sets","From DINOv2 to 3D: slice transformer ups AUC and explainability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The explainability claim rests on one radiologist's blinded ratings of 150 saliency maps; if other radiologists would not reproduce those ratings, the claim of improved explainability loses its support.","fun_headline_variants_meta":{"raw":{"variants":["DINOv2 goes 3D: slice transformer tops ResNet on MRI, CT","Adapting DINOv2 to 3D: MST beats ResNet in diagnosis and maps","Slice transformer with DINOv2 lifts 3D diagnostics and saliency","MST: DINOv2 Transformer for 3D scans beats ResNet on all sets","From DINOv2 to 3D: slice transformer ups AUC and explainability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000526,"raw_usage":{"total_tokens":2655,"prompt_tokens":1180,"completion_tokens":1475,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":796,"completion_tokens_details":{"reasoning_tokens":1361}},"tokens_in":796,"tokens_out":1475,"duration_ms":8482,"temperature":1.0,"reasoning_tokens":1361,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:52:19.328259+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have a second panel of radiologists rate the identical saliency maps with model identity hidden; if the 136 of 150 versus 37 of 150 slice-correctness split does not reproduce, the explainability claim is falsified. A quantitative check would compute overlap between MST combined attention maps and expert lesion segmentations on the same volumes.","supporting_citations":[],"review_version":1}