{"id":"c9d40bf3-5360-4872-aa0b-ba971b9d74d3","arxiv_id":"2501.15733","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"Applying a video vision transformer (ViViT) to 3D brain MRI slices treated as video frames yields 98.6% three-class accuracy on ADNI1 Complete 3Yr 3T, outperforming the paper's CNN-BiLSTM and ViT-BiLSTM baselines.","lead":"A new deep learning model, ViTranZheimer, treats 3D brain MRI scans as video sequences and reports 98.6% accuracy in classifying Alzheimer's disease, mild cognitive impairment, and normal cognition on an ADNI dataset. The paper is a proof-of-concept that video vision transformers can be applied to volumetric brain images, but the small dataset, unclear evaluation protocol, and lack of released code limit its impact.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation-protocol contradiction and missing significance test make the 98.6% accuracy claim unverifiable: §2.1 says a single 60/20/20 split, §3.1 says repeated 10-fold CV, baselines are trained under different budgets, and no statistical test backs the 'significant advantage'.","rationale":"I read the paper in good faith: the contribution is a plausible method (applying ViViT to 3D MRI volumes), and the reported per-class rates in Figure 4 are roughly compatible with 98.6% on the full 351-scan set. The paper's central claim, however, is an empirical accuracy comparison with a significance statement. What must be true for that claim to hold is a valid, reproducible evaluation protocol and comparable model training. The manuscript itself undermines this: the split description in Section 2.1 contradicts the cross-validation description in Section 3.1; the number of test samples and implied error counts differ; the relationship between the validation set and checkpoint selection is unspecified; the baselines are trained with different epochs, batch sizes, and pretraining; and no significance test is shown despite the conclusion's claim. These are not external disagreements with consensus; they are internal, verifiable gaps in the evidence. A single reconstruction of the exact partitions would settle the issue: if the numbers reproduce under a clear protocol, the claim stands; if not, the 98.6% result is an artifact of the unspecified evaluation. I therefore agree with the reader's REJECT verdict and recommend no change.","tokens_in":14357,"tokens_out":8014,"duration_ms":70783,"concrete_test":"Ask the authors for the exact subject-level data partition (subject IDs per train/test/validation fold and the random seed), then re-run ViTranZheimer and both baselines under a single pre-specified protocol: 10-fold stratified cross-validation at the subject level, with no validation-based early stopping on the test fold, or a fixed 60/20/20 split if that is the claimed design. Report per-fold accuracies and apply a paired significance test (e.g., 5x2 CV or McNemar on the agreed test set). If the ViTranZheimer mean does not remain near 98.6% or its advantage over ViT-BiLSTM is not statistically significant, the paper's central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that ViTranZheimer reaches 98.6% multiclass accuracy (NC/MCI/AD) on ADNI1 Complete 3Yr 3T, statistically significantly above CNN-BiLSTM and ViT-BiLSTM. The load-bearing condition is that this accuracy is an unbiased estimate from one unambiguous protocol with no leakage and comparable baselines. That condition is not met. Section 2.1 states the 351 scans were randomly split into 60% training, 20% testing, and 20% validation sets, but Section 3.1 says repeated 10-fold stratified cross-validation and testing were adopted and then each sample of all the datasets was only used for the test once, which describes a single 10-fold CV, not a repeated 10-fold CV and not a 60/20/20 split. The two protocols imply different test sizes (about 70 vs 351 samples) and different error counts for 98.6% (about 1 vs 5 errors), so the reported number cannot be interpreted without knowing which was used. Model selection is also unevaluable: the checkpoint is saved when validation loss improves over 1500 epochs, but the text never says whether that validation set is the fixed 20% split or a fold within CV, and if the test set is also used to select the checkpoint the accuracy is optimistic. Baseline comparisons are not controlled: CNN-BiLSTM/ViT-BiLSTM use pretrained CNN/ViT feature extractors, 100 epochs, and batch size 25, while ViTranZheimer is trained end-to-end from scratch for 1500 epochs with batch size 128. Finally, the Conclusion asserts a statistically significant advantage, but no significance test, confidence interval, or paired comparison appears in Sections 3-5. Without these details, 98.6% is not a reproducible, comparable estimate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes ViTranZheimer, a video vision transformer for three-class classification (NC/MCI/AD) of 3D brain MRI from the ADNI1 Complete 3Yr 3T dataset, treating MRI slices as video frames. It reports 98.6% accuracy for ViTranZheimer versus 96.479% for CNN-BiLSTM and 97.465% for ViT-BiLSTM, and concludes that this is a statistically significant advantage. The paper describes preprocessing, the ViViT architecture with spatio-temporal attention, training hyperparameters, and comparisons with baselines and prior work.","tokens_in":14776,"tokens_out":5888,"duration_ms":52516,"significance":"If the result were reliably established, applying video transformers to whole MRI volumes would be a useful contribution to Alzheimer's disease diagnosis. The paper has strengths: it uses a publicly available standard ADNI dataset, proposes an end-to-end trainable model, and provides architectural details and equations. However, the evaluation is undermined by contradictory protocol descriptions, uncontrolled baseline training settings, and the absence of any statistical test for the claimed significance. As it stands, the headline accuracy is not a trustworthy estimate and the main claim of superiority is not supported.","major_comments":[{"comment":"The evaluation protocol is described inconsistently. Section 2.1 states that the 351 scans were 'randomly split into 60% training, 20% testing, and 20% validation sets,' while Section 3.1 states that 'Repeated 10-fold stratified cross-validation and testing were adopted' and then says 'each sample of all the datasets was only used for the test once,' which is a single 10-fold cross-validation rather than a repeated one. These protocols imply different test set sizes (about 70 scans for the 60/20/20 split versus 351 scans for 10-fold CV) and therefore different error counts for the reported 98.6% accuracy. The manuscript must specify one unambiguous protocol and report results consistently with it.","section":"Section 2.1 vs Section 3.1"},{"comment":"The model selection procedure is not clearly tied to the evaluation protocol. The text says the model checkpoint is saved when validation loss improves, but it is not stated whether that validation set is the fixed 20% split described in Section 2.1 or a fold within cross-validation. If any portion of the test set was used to select the checkpoint, the reported accuracy is optimistically biased. The authors should define exactly how the training, validation, and test partitions are created and how checkpoint selection interacts with the reported accuracy.","section":"Section 2.4 and Section 3.1"},{"comment":"The baseline comparisons are not controlled. CNN-BiLSTM and ViT-BiLSTM use ImageNet-pretrained feature extractors, are trained for 100 epochs with batch size 25, and are described as processing slices, whereas ViTranZheimer is trained from scratch for 1500 epochs with batch size 128 and operates on the whole volume. Table 4 gives hyperparameters only for the CNN-BiLSTM baseline and does not specify the ViT-BiLSTM configuration. The reported accuracy differences (1-2 percentage points) could be due to training budget, pretraining, or input representation rather than architectural superiority. A matched-budget comparison or a sensitivity analysis is needed to support the claim that ViTranZheimer outperforms the baselines.","section":"Section 3.1 and Table 4"},{"comment":"The conclusion asserts a 'statistically significant advantage' for ViTranZheimer, but no statistical test is reported anywhere in Section 3.2. With 351 samples and accuracies of 98.6% versus 97.465%, the difference may not be significant; the authors should provide a paired test (e.g., McNemar's test on the same folds) or confidence intervals for the accuracy difference. Without such a test, the significance claim is unsupported.","section":"Section 5 and Section 3.2"}],"minor_comments":[{"comment":"Table 6 labels the proposed method as 'Slice-based' even though Section 2.3 describes processing the entire 3D volume through tubelets; this labeling is inconsistent and should be corrected.","section":"Table 6"},{"comment":"The discussion contains the typo 'using 3CNN' where '3D CNN' is intended; please correct this.","section":"Section 4"},{"comment":"The confusion matrices in Figure 4 are too small to read in the manuscript; they should be enlarged or replotted for legibility.","section":"Figure 4"},{"comment":"The replication statement says code and data are 'available on request'; for reproducibility, the authors should provide code, trained model weights, and the exact data split or cross-validation indices.","section":"Replication of results"}],"recommendation":"major_revision","confidential_remarks":"The evaluation problems are substantial but addressable. If the authors can clarify the protocol, add proper statistical tests, and run controlled baseline comparisons, the paper might become viable. The idea of treating MRI slices as video frames is reasonable, but the current evidence does not support the headline accuracy or the significance claim. I would not reject solely on the architecture novelty; the issues concern the empirical validation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a reasonable \"video transformer as 3D-MRI classifier\" demo, but the 98.6% accuracy claim cannot be trusted as written because the evaluation protocol is contradictory and the baselines are not trained under matched conditions.\n\nWhat is genuinely useful: the paper treats the 32 selected coronal slices as frames and feeds them to ViViT, which is a sensible way to let a transformer see the whole volume at once. The ADNI1 Complete 3Yr 3T subset is public and described with enough demographic detail. The comparison with CNN-BiLSTM and ViT-BiLSTM, both from the authors' prior work, is a reasonable continuation of that line. They also report precision, recall, F1 and confusion matrices, not just accuracy. The discussion correctly notes that Table 6 comparisons across the literature are not apples-to-apples.\n\nThe soft spots are load-bearing. Section 2.1 says the 351 scans were split 60/20/20 into train/test/validation. Section 3.1 says \"repeated 10-fold stratified cross-validation and testing\" and then says each sample was used for the test once. Those are different protocols: the first gives about 70 test samples, the second gives all 351 as test across folds. The reported 98.6% with a ±1.4 standard deviation suggests the cross-validation reading, but the split description is never reconciled. Also, the model checkpoint is saved on validation loss, but the reader cannot tell whether that validation set is the fixed 20% or a fold, and if the test fold is also used for checkpoint selection, the accuracy is optimistic.\n\nThe baselines are not controlled. CNN-BiLSTM and ViT-BiLSTM use ImageNet-pretrained feature extractors, 100 epochs, batch size 25, while ViTranZheimer is trained from scratch for 1500 epochs with batch size 128. Different training budgets alone can produce a few points of accuracy difference. The paper calls the advantage \"statistically significant\" in the conclusion, but no significance test, confidence interval, or paired comparison appears anywhere in Sections 3-5. The word \"significant\" carries no statistical meaning here.\n\nThere is also no code or data released, only \"available on request,\" which is weak for a paper whose only real contribution is an empirical accuracy number.\n\nBottom line: the core idea is worth trying, but in its current form the evaluation is too muddy to support the central claim. A serious referee could ask for a consistent evaluation protocol, matched training budgets, and a proper significance test; with those, the paper could be a solid incremental contribution. I'd send it to review rather than desk-reject, but I would not cite it as it stands.","headline":"A reasonable video-transformer-on-3D-MRI demo whose headline accuracy claim is undercut by a contradictory evaluation protocol, unmatched baseline training, and an unsupported 'significant' conclusion.","tokens_in":15369,"tokens_out":2271,"would_cite":false,"duration_ms":19716,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that treating 3D brain MRI scans as videos and applying a video vision transformer classifies Alzheimer's disease, mild cognitive impairment, and normal cognition with 98.6% accuracy on the ADNI1 Complete 3Yr 3T dataset.","keywords":["Alzheimer's disease","mild cognitive impairment","video vision transformer","ViViT","3D brain MRI","multiclass classification","spatio-temporal attention","ADNI"],"falsifier":"Train the same three architectures on the same ADNI1 Complete 3Yr 3T data using subject-level cross-validation, ensuring no scan from the same subject appears in both training and test sets, and report test accuracy. If ViTranZheimer's accuracy falls to within the baseline's standard deviation, or if the original split contains scans from the same subject in both train and test, the claimed advantage collapses. A simpler check: compute a paired test or 95% confidence interval over the 10 repeat runs; if the intervals overlap, the 'statistically significant' claim is not supported by the reported numbers.","tokens_in":14196,"feed_emoji":"🧠","tokens_out":6873,"duration_ms":57357,"temperature":0.7,"pith_summary":"This paper introduces ViTranZheimer, a deep learning model that reinterprets a 3D brain MRI scan as a video—each slice a frame—and applies a video vision transformer to classify the scan as normal cognition, mild cognitive impairment, or Alzheimer's disease. The authors report 98.6% accuracy on the ADNI1 Complete 3Yr 3T dataset, outperforming two hybrid baselines, CNN-BiLSTM (96.479%) and ViT-BiLSTM (97.465%). The central idea is that treating the whole 3D volume as a spatio-temporal sequence lets the model capture inter-slice relationships end-to-end, rather than extracting slice features separately and then modeling sequences. If true, this gives clinicians a non-invasive, automated tool for early AD detection, which matters because current treatments target early-stage disease.","feed_headline":"Video transformer hits 98.6% accuracy on Alzheimer's MRI","feed_subtitle":"Treating 3D brain scans as video helps an AI separate Alzheimer's, MCI, and normal aging.","key_machinery":"The central mechanism is tubelet-based spatio-temporal attention. The 3D MRI volume of shape $32 \\times 64 \\times 64$ is partitioned into non-overlapping 3D tubelets of size $32 \\times 16 \\times 16$; each tubelet is flattened, linearly embedded, and summed with positional encodings. The resulting token sequence enters a transformer encoder with 16 heads and 16 layers, where multi-head self-attention computes relationships across all tubelets, capturing both within-slice spatial patterns and between-slice dependencies. A CLS token aggregates global information for classification. This replaces the two-stage pipeline of independent slice feature extraction followed by a recurrent sequence model with a single end-to-end spatio-temporal model.","core_discovery":"ViTranZheimer uses the ViViT video vision transformer to classify 3D T1-weighted brain MRI volumes. Instead of splitting the brain into independent 2D slices and feeding them to a CNN or ViT plus a BiLSTM, it divides the entire volume into 3D tubelets that span both space and adjacent slices, embeds them, and applies multi-head spatio-temporal self-attention. The final CLS token is passed to a softmax head for three-way classification of normal cognition, mild cognitive impairment, and Alzheimer's disease. On the ADNI1 Complete 3Yr 3T dataset of 351 scans, the model is reported to achieve 98.6% accuracy with 0.97 precision, recall, and F-score, outperforming the CNN-BiLSTM and ViT-BiLSTM hybrids. The paper concludes that this accuracy advantage is statistically significant and that video vision transformers are a promising end-to-end framework for AD diagnosis.","pith_inferences":["A natural next test, not run in the paper, is subject-level cross-validation: splitting by patient rather than by scan, since multiple scans from one subject could inflate accuracy if they leak across train and test sets.","The same tubelet-based spatio-temporal design could transfer to other 3D medical volumes such as CT or fMRI, where inter-slice dependencies carry diagnostic information; the paper only demonstrates MRI.","The paper mentions MRI harmonization as future work; combining the model with harmonized multi-scanner data would test whether the 98.6% accuracy transfers across acquisition sites and scanning protocols.","A paired statistical test over the repeated cross-validation runs would clarify whether the accuracy gap over baselines is reliable, since the reported standard deviations of the three models overlap."],"forward_implications":["If the result holds, a video vision transformer can classify three-way Alzheimer's status (normal, MCI, AD) from a single structural MRI at 98.6% accuracy, exceeding the two hybrid baselines tested in the same study.","The end-to-end design removes the need for separate slice-level feature extraction and sequence modeling, simplifying the training pipeline for 3D medical images.","The high sensitivity for MCI, the intermediate disease stage, supports the prospect of automated early screening from routine MRI, which could prompt earlier clinical follow-up.","Because the dataset is a publicly available standard collection, other researchers can directly compare their methods against ViTranZheimer on the same data.","The reported accuracy on this dataset exceeds the published results listed in the paper's comparison table for other multiclass AD classification methods."],"supporting_citations":[{"why":"Introduces the ViViT video transformer architecture with tubelet embedding and spatio-temporal attention that ViTranZheimer is built on.","marker":"[32]"},{"why":"Describes the ADNI dataset, the source of the ADNI1 Complete 3Yr 3T MRI collection used for training and evaluation.","marker":"[29,30]"},{"why":"The authors' prior joint-transformer method, used here as the ViT-BiLSTM baseline and the non-end-to-end approach this work improves on.","marker":"[27]"},{"why":"Provides the CNN-BiLSTM hybrid baseline that ViTranZheimer is compared against.","marker":"[33]"},{"why":"Supplies SPM12, used for image registration and normalization into MNI space during preprocessing.","marker":"[31]"}],"fun_headline_variants":["ViTranZheimer: video transformer hits 98.6% on Alzheimer's MRI","3D brain MRI as video: transformer nails AD diagnosis at 98.6%","Video transformer reads 3D MRI like film, spots Alzheimer's at 98.6%","Alzheimer's MRI diagnosis: video transformer beats hybrids at 98.6%","ViTranZheimer: video transformer for AD hits 98.6% on MRI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported accuracy gap rests on the evaluation protocol being unbiased: the random 60/20/20 split and repeated 10-fold stratified cross-validation must have no data leakage and the baseline models must be trained with comparable effort, or the 1.1-2.1 point accuracy difference loses meaning.","fun_headline_variants_meta":{"raw":{"variants":["ViTranZheimer: video transformer hits 98.6% on Alzheimer's MRI","3D brain MRI as video: transformer nails AD diagnosis at 98.6%","Video transformer reads 3D MRI like film, spots Alzheimer's at 98.6%","Alzheimer's MRI diagnosis: video transformer beats hybrids at 98.6%","ViTranZheimer: video transformer for AD hits 98.6% on MRI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000737,"raw_usage":{"total_tokens":3363,"prompt_tokens":1086,"completion_tokens":2277,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":702,"completion_tokens_details":{"reasoning_tokens":2163}},"tokens_in":702,"tokens_out":2277,"duration_ms":14934,"temperature":1.0,"reasoning_tokens":2163,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:59:16.444865+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same three architectures on the same ADNI1 Complete 3Yr 3T data using subject-level cross-validation, ensuring no scan from the same subject appears in both training and test sets, and report test accuracy. If ViTranZheimer's accuracy falls to within the baseline's standard deviation, or if the original split contains scans from the same subject in both train and test, the claimed advantage collapses. A simpler check: compute a paired test or 95% confidence interval over the 10 repeat runs; if the intervals overlap, the 'statistically significant' claim is not supported by the reported numbers.","supporting_citations":[{"cited_title":"Arnab, M","cited_arxiv_id":null,"evidence_quote":"Introduces the ViViT video transformer architecture with tubelet embedding and spatio-temporal attention that ViTranZheimer is built on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The authors' prior joint-transformer method, used here as the ViT-BiLSTM baseline and the non-end-to-end approach this work improves on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CNN-BiLSTM hybrid baseline that ViTranZheimer is compared against."}],"review_version":1}