{"id":"e2847b68-2a18-4ae8-be07-22c85949c7ca","arxiv_id":"1909.01098","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A 3D siamese convolutional network on raw longitudinal brain MRIs predicts cognitive decline versus stable status with 90% validation accuracy on 247 ADNI subjects.","lead":"This paper trains a 3D siamese neural network to compare two brain MRI scans taken 12 months apart and predict whether a patient's cognition will decline. On 247 Alzheimer's Disease Neuroimaging Initiative subjects, the model reports 90% validation accuracy without using cognitive test scores or manual brain segmentation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.3 selects a subset of ADNI1 subjects and relies on MMSE-derived labels without reporting selection criteria or label validation, so the 90% accuracy may measure an easy subgroup rather than clinically meaningful cognitive decline.","rationale":"The reader's weakest assumption was that the Bhagwat et al. script labels may be noisy or incorrect. I agree that label definition is the critical vulnerability, but I sharpen it further: the manuscript itself states in Section 3.3 that a subset was selected, yet gives no selection criteria, so even accurate script labels may not support the general claim if the subset is unrepresentative or enriched for unambiguous trajectories. The paper does provide useful evidence: the architecture is clearly described, the source code is public, and the 10 repeated random subsampling runs give some protection against split-specific overfitting. However, repeated within-cohort validation cannot correct a biased target definition or a biased subject selection. The proposed test, applying the same protocol to the full eligible ADNI1 cohort and also checking an independent clinical outcome, would settle whether the 90% figure reflects true cognitive decline or an artifact of the chosen labels and subgroup. This does not overturn the reader's conditional verdict, but it explains the specific experiment that should be required before the central claim is accepted.","tokens_in":7536,"tokens_out":15429,"duration_ms":176586,"concrete_test":"Run one experiment on the full ADNI1 cohort: apply Bhagwat et al.'s script to every subject who has both a baseline and a 12-month MRI, without any additional hand-selection. Train 3D-SiameseNet with the same architecture, preprocessing, augmentation, and 10x40-subject validation protocol, and report mean validation accuracy and ROC-AUC. If feasible, also evaluate the same trained model against an independent decline label such as progression to MCI/dementia or a reliable cognitive composite. If full-cohort accuracy and independent-outcome accuracy remain near 90%, the concern is resolved; if either drops substantially, the reported accuracy is an artifact of subset selection and/or label definition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a 3D siamese CNN can classify cognitively declining versus stable patients from two whole-brain MRIs. The target itself is defined in Section 3.3: labels are assigned by following Bhagwat et al.'s script based on MMSE trajectory clustering, and the authors then \"selected a subset of these subjects\" (134 stable, 113 decline). No inclusion or exclusion criteria are reported, and no evidence is given that this subset is representative of the ADNI1 cohort or that the script's clusters correspond to clinically meaningful decline. If the selection preferentially kept subjects with extreme, easy-to-separate trajectories, or if the MMSE-based labels are noisy, then the mean validation accuracy of 90% in Table 1 can be high while the model performs poorly on the general patient population or on an independent clinical definition of decline. Because every reported accuracy is computed on this subset and no full-cohort or external evaluation is provided, the headline claim is not yet supported for 'cognitively declining VS stable patients' as a general class.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 3D-SiameseNet, a siamese convolutional network that takes a pair of whole-brain 3D MRI scans (baseline and 12-month follow-up) and classifies subjects as cognitively declining or stable. The model replaces the fully-connected branches of Bhagwat et al.'s Longitudinal Siamese Network with 3D convolution blocks and fuses the two branches by a subtraction layer. Using ADNI data, the authors select 247 subjects, follow Bhagwat et al.'s script to assign Stable/Decline labels from MMSE-based trajectory clustering, and train with data augmentation. Over 10 random 40-subject validation splits, they report a mean validation accuracy of 90%, a validation MSLE of 0.04, and a validation cross-entropy loss of 0.992. The paper also includes t-SNE visualizations of intermediate feature maps to argue that the network learns discriminative representations.","tokens_in":7812,"tokens_out":3018,"duration_ms":30791,"significance":"If the 90% accuracy held on an independent test set with clinically validated labels, the contribution would be meaningful: it would show that a relatively small 3D siamese CNN can predict cognitive decline directly from raw whole-brain longitudinal MRI, without segmentation, ROI selection, or cognitive scores as inputs. The paper provides source code, uses a public dataset, and the architecture is simple and reproducible. However, the current evaluation protocol does not support the headline claim. The accuracy is measured only on repeated validation splits rather than a held-out test set, the comparison with other models uses published numbers from a different and larger cohort, and the reported validation cross-entropy is inconsistent with the claimed accuracy. These issues need to be addressed before the central claim can be taken as established.","major_comments":[{"comment":"The reported 90% mean validation accuracy is computed over 10 random sub-sampling runs with only 40 validation subjects each, and no independent test set is ever held out. Because the same validation subjects are reused across runs and the model is trained on the remaining 207 subjects each time, the reported accuracy is a repeated random-split estimate, not an independent evaluation of a final model. This protocol does not support the abstract's claim that the network 'achieves an accuracy of 90%' on the general patient population. An independent test cohort, or at least a single fixed test split never used in any model-selection or early-stopping decision, is needed.","section":"Section 4, Table 1"},{"comment":"The comparison to LR, SVM, RF, and ANN uses accuracy values taken from Bhagwat et al. (S4 File), which were obtained on a different cohort of 1116 subjects (Section 4 states 'Bhagwat et al. did performance evaluation experiments using 1116 subjects'). This is not a head-to-head comparison: the baseline models were evaluated on a different dataset, so the statement that 'our model outperforms LR, SVM, RF, and ANN models' is not supported by the evidence presented. A fair comparison would require running the same baselines on the same 247-subject subset with the same validation protocol.","section":"Section 4, Table 2"},{"comment":"The assignment of Stable/Decline labels follows an external script based on MMSE trajectory clustering, and the authors then 'selected a subset of these subjects' (134 stable, 113 decline). No inclusion or exclusion criteria for this subset are reported, and no evidence is provided that the subset is representative of the ADNI1 cohort or that the script's clusters correspond to clinically meaningful cognitive decline. If the selection preferentially kept subjects with extreme or easy-to-separate trajectories, the 90% accuracy could be high while the model performs poorly on the broader patient population. The label-generation script and the exact selection procedure must be described, and the resulting labels should be validated against clinical outcomes.","section":"Section 3.3 and Section 4, Figure 4"},{"comment":"The reported validation cross-entropy loss of 0.992 is greater than ln(2) ≈ 0.693, which is the expected cross-entropy of a random classifier on balanced classes, yet the validation accuracy is reported as 90%. This inconsistency is unexplained. If the model is 90% accurate, a mean cross-entropy above 0.7 suggests either severely miscalibrated probabilities or a mismatch in how accuracy and loss are computed. The authors should clarify how the loss and accuracy are aggregated over the 10 runs and explain the relationship between the reported loss and the reported accuracy, or correct the metric if there is an error.","section":"Section 4, Figure 4"}],"minor_comments":[{"comment":"The phrase 'Logintudinal Siamese Network' appears to be a typo for 'Longitudinal Siamese Network'; please correct it.","section":"Section 2"},{"comment":"In the paragraph describing the penultimate Dense layer, the text says 'see Fig.5.3', but the penultimate layer corresponds to the fourth subplot (Fig.5.4); the reference should be updated.","section":"Section 5, Figure 5"},{"comment":"The abstract states that the result is obtained 'without the help of a cognitive score', but the Stable/Decline labels are derived from MMSE-based trajectory clustering (Section 3.3). Please clarify that cognitive scores are not used as model inputs, but are used to define the labels.","section":"Abstract and Section 3.3"},{"comment":"The metric MSLE (Mean Squared Logarithmic Error) is not standard for classification probabilities; please define how it is computed and why it is appropriate for this task.","section":"Section 4, Table 1"},{"comment":"The word 'tridimentional' should be 'three-dimensional'.","section":"Section 3.2"},{"comment":"The claim that average pooling is used 'to avoid a bias of the network towards edge detection' is not self-evident, because the convolution filters are learned; a brief explanation or reference would help.","section":"Section 3.1"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is interesting and the code is available, but the evaluation is not yet convincing. The comparison table borrowing accuracies from Bhagwat et al. on a different cohort is a particular concern for a journal audience, and the validation loss/accuracy inconsistency needs to be resolved. The paper would also benefit from a clearer description of the subject selection from ADNI1, as the current description is too vague to assess generalizability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid but small-scale engineering paper. The new bit is running a siamese 3D CNN directly on raw longitudinal MRI, with a subtraction fusion instead of concatenation, and showing 90% mean validation accuracy on 247 ADNI subjects. That is a real step beyond Bhagwat's 1D ROI features, and they provide code. The t-SNE visualizations are a nice qualitative check.\n\nThe main problem is that the 90% is a validation number on repeated random splits of the same 247 subjects, not on an independent cohort. Table 2 compares against LR/SVM/RF/ANN numbers taken from Bhagwat's 1116-subject study, so the 'outperforms' claim is not apples-to-apples. More importantly, the label definition in Section 3.3 is underdocumented: they followed Bhagwat's MMSE-trajectory script, then selected 247 subjects with no inclusion/exclusion criteria. If that selection is biased toward easy cases, the accuracy measures the selection, not the clinical question. This concern is real and needs addressing before the headline claim is trustworthy.\n\nI also noticed the reported validation loss of 0.992, which looks high for a 90% accuracy. It probably includes L2 regularization loss, but the paper doesn't say; worth asking.\n\nWhat the paper does well: the architecture is clearly described, the preprocessing is standard, the repeated random subsampling is better than a single split, and they are honest about memory constraints. The central design choice—subtraction to capture change—is reasonable and the embeddings give some evidence it helps separate classes.\n\nBottom line: I would not cite this for the accuracy number, but it is a legitimate contribution to the AD prediction literature and worth reading for anyone working on longitudinal MRI. If it lands in your editorial pile, send it to a referee who will insist on external validation or at least a full description of subject selection and label quality. The core is salvageable; the evaluation needs work.","headline":"A reasonable 3D siamese adaptation with a 90% validation accuracy on 247 ADNI subjects, but the lack of an independent test set and undocumented subject selection keep the headline claim unsupported.","tokens_in":8290,"tokens_out":3683,"would_cite":false,"duration_ms":34737,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 3D siamese convolutional network trained on pairs of whole-brain MRI scans taken a year apart is claimed to classify cognitively declining versus stable patients with 90% accuracy.","keywords":["3D siamese network","brain MRI","cognitive decline","longitudinal imaging","Alzheimer's disease","convolutional neural network","whole-brain classification"],"falsifier":"Re-run the same network and validation protocol on a cohort where decline is confirmed independently (for example, by expert diagnosis or documented conversion to mild cognitive impairment) using labels not derived from the trajectory script; if accuracy falls to around 75% or to chance, the central claim is not supported.","tokens_in":7391,"feed_emoji":"🧠","tokens_out":7084,"duration_ms":67096,"temperature":0.7,"pith_summary":"This paper tries to show that a 3D siamese convolutional network can predict whether a patient's cognition is likely to decline, given only two whole-brain MRI scans taken about a year apart. The network compares the two scans directly and learns the difference between them, replacing the usual pipeline of segmentation, hand-picked brain regions, and cognitive test scores. On 247 subjects from a public Alzheimer's neuroimaging database, the authors report a mean validation accuracy of 90% over ten random splits. If the result holds, longitudinal MRI alone could flag at-risk patients without expensive preprocessing or clinical scoring.","feed_headline":"Two brain scans a year apart predict decline at 90%","feed_subtitle":"A 3D siamese network reads raw whole-brain MRI, skipping segmentation and cognitive scores, to flag patients at risk.","key_machinery":"The central object is the 3D siamese architecture with difference fusion. A siamese network is a pair of identical neural-network branches that share weights, so any difference in their outputs comes from the inputs, not from different learned filters; here the inputs are the baseline and follow-up MRI volumes and the outputs are subtracted elementwise. That subtraction is the mechanism that isolates brain change over the year, and it is what lets the network avoid segmentation, ROI atlases, and cortical-thickness computation. The choice of average pooling rather than max pooling is meant to prevent the network from biasing toward edge detection, and the small 3x3x3 kernels preserve local volumetric details.","core_discovery":"The central claim is that a 3D siamese convolutional network, called 3D-SiameseNet, classifies cognitively declining versus stable patients with 90% mean validation accuracy using only pairs of raw MRI volumes. The model has two branches with shared weights, one processing the baseline scan and the other the 12-month follow-up scan; each branch stacks three blocks of 3D convolution, batch normalization, leaky ReLU, and average pooling. Instead of concatenating the branch outputs as standard siamese networks do, the authors subtract them, so the network is forced to represent morphological change between visits. The labels come from a published trajectory-modeling script, and the comparison is to four imaging-only baselines whose reported accuracies range from 75% to 77%; the paper also reports a mean squared logarithmic error of 0.04 on validation, meaning predicted probabilities sit close to the true labels.","pith_inferences":["If the label script is reliable, a natural next test is external validation on an independent cohort with clinician-confirmed progression; a drop to near the 75% baseline level would indicate the 90% is tied to the specific label assignment.","The three clusters visible in the t-SNE projection before the subtraction layer may encode anatomy such as head size; a regressor trained on those pre-subtraction features could test whether the network has learned non-diagnostic confounds.","A permutation test that swaps the two time-point images within each subject would show whether the model is truly reading temporal change; if swapped-pair accuracy stays high, the model may be exploiting visit-specific artifacts rather than progression.","Because the authors note memory limits forced downscaling, running the same architecture at native resolution with more filters is a direct test of whether the reported accuracy underestimates what the approach can do."],"forward_implications":["Imaging-only longitudinal prediction may not require segmentation, ROI selection, or cortical-thickness measures; raw whole-brain volumes are enough for the reported 90% accuracy.","Difference fusion in a siamese network is a viable way to represent disease-related change, and the t-SNE plots show the representation becomes class-separated only after the subtraction and dense layers.","The model's 90% validation accuracy exceeds the 75–77% accuracies of the logistic regression, SVM, random forest, and ANN baselines that also use only imaging, while using far fewer subjects.","Adding clinical data to the imaging branch, as the authors propose for future work, could plausibly move accuracy beyond the 90% imaging-only level toward the 94% reported for the earlier model that used both imaging and clinical scores."],"supporting_citations":[{"why":"Supplies the Stable versus Decline label script and the comparison accuracies for logistic regression, SVM, random forest, and ANN.","marker":"[2]"},{"why":"Supplies the 247 baseline and 12-month MRI image pairs used for training and validation.","marker":"[16]"},{"why":"Provides the skull-stripping preprocessing that removes non-brain tissue from each volume before training.","marker":"[17]"},{"why":"Shows that 3D convolutional models outperform 2D on MRI, motivating the whole-volume 3D input design.","marker":"[10]"},{"why":"Introduces siamese networks with shared-weight branches, the structural basis of the model.","marker":"[4]"},{"why":"Inspires the t-SNE-based layer-by-layer analysis of learned feature maps used to study discriminative abilities.","marker":"[9]"}],"fun_headline_variants":["MRI pairs across a year spot decline at 90% accuracy","3D siamese net learns decline from raw MRI pairs","Two brain scans, no segmentation, 90% decline detection","Raw MRI year apart predicts cognitive decline at 90%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The division of patients into 'stable' and 'declining' comes from a published script that labels trajectories from mental-status test scores; if those labels are noisy or wrong, the 90% accuracy may measure prediction of the script's labels rather than true cognitive decline.","fun_headline_variants_meta":{"raw":{"variants":["MRI pairs across a year spot decline at 90% accuracy","3D siamese net learns decline from raw MRI pairs","Two brain scans, no segmentation, 90% decline detection","Raw MRI year apart predicts cognitive decline at 90%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2780,"prompt_tokens":965,"completion_tokens":1815,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":581,"completion_tokens_details":{"reasoning_tokens":1746}},"tokens_in":581,"tokens_out":1815,"duration_ms":12168,"temperature":1.0,"reasoning_tokens":1746,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:26:35.064693+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same network and validation protocol on a cohort where decline is confirmed independently (for example, by expert diagnosis or documented conversion to mild cognitive impairment) using labels not derived from the trajectory script; if accuracy falls to around 75% or to chance, the central claim is not supported.","supporting_citations":[{"cited_title":"Modeling and prediction of clinical symptom trajectories in alzheimers disease using longitu- dinal data,","cited_arxiv_id":null,"evidence_quote":"Supplies the Stable versus Decline label script and the comparison accuracies for logistic regression, SVM, random forest, and ANN."},{"cited_title":"ADNI | Alzheimer’s Disease Neuroimaging Initiative","cited_arxiv_id":null,"evidence_quote":"Supplies the 247 baseline and 12-month MRI image pairs used for training and validation."},{"cited_title":"3d u-net for skull stripping in brain mri,","cited_arxiv_id":null,"evidence_quote":"Provides the skull-stripping preprocessing that removes non-brain tissue from each volume before training."},{"cited_title":"Siamese neu- ral networks for one-shot image recognition,","cited_arxiv_id":null,"evidence_quote":"Introduces siamese networks with shared-weight branches, the structural basis of the model."}],"review_version":1}