{"id":"40c5434d-608c-4c0c-9a15-7ac9788eb165","arxiv_id":"1908.06168","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Unsupervised autoencoder and next-frame prediction models trained on healthy rs-fMRI can distinguish autism patients from healthy controls with AUC around 0.70.","lead":"The paper trains deep unsupervised models on healthy resting-state fMRI scans and shows that their reconstruction errors can separate autism patients from controls with moderate accuracy. It demonstrates a label-free approach to detecting brain abnormalities from dynamic imaging data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Group mismatch on unmeasured confounds could fully explain the reported ASD-control error differences, so the central discrimination claim is not yet established.","rationale":"The paper makes a plausible and novel application of recurrent autoencoders and next-frame predictors to rs-fMRI anomaly detection, with useful architectural comparisons and a reasonable motion check. The load-bearing step is not the architecture but the inference from a between-group mean error difference to an abnormality signal. The reported AUCs are point estimates on a small held-out control group (28 subjects) with no confidence intervals, and the clinical groups are not described as matched on standard confounds. A covariate-adjusted logistic regression is the natural and decisive check. This does not invalidate the empirical observation; it determines whether the observation supports the paper's interpretation. Since the reader's verdict was already CONDITIONAL on exactly these controls, no verdict change is needed.","tokens_in":5528,"tokens_out":5358,"duration_ms":59574,"concrete_test":"On ABIDE NYU, obtain age, sex, full-scale IQ, medication status, and mean framewise displacement for the 28 control and 67 ASD test subjects. Test for group differences on each variable; then fit two logistic regression models predicting ASD: (M1) covariates only and (M2) covariates plus the subject-level anomaly score from the recurrent autoencoder. Report the likelihood-ratio test and cross-validated AUC difference between M1 and M2. If the anomaly score does not add significant discriminative value beyond the covariates, the reported AUC of 69.6 in Table 3 cannot be attributed to disease-specific abnormalities.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the assertion that higher reconstruction/prediction error in ASD versus controls reflects disease-related abnormalities, not group differences on other variables. The models are trained solely on 54 healthy controls and then applied to 28 held-out controls and 67 ASD subjects, but the two test groups are never shown to be matched on age, sex, IQ, medication, or residual motion/scan quality, and no covariate adjustment is performed in the Section 4.2 comparisons or Table 3. Because the subject-level anomaly score is a single mean squared error averaged over the whole gray-matter mask, any systematic between-group difference in signal variance, frame count after scrubbing, or preprocessing residual will masquerade as an 'abnormality' even in the absence of disease-specific dynamics. The authors' only confound check (Section 4.2) is a correlation with framewise displacement; this does not rule out age/sex/IQ/medication differences, and one frame-level correlation test is nominally significant (p=0.0251), so even the motion check is not clean. Without a covariate-adjusted analysis, the central claim that the error metric can discriminate patients from controls as an unsupervised abnormality detector is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes two unsupervised deep-learning models for resting-state fMRI: a recurrent autoencoder that reconstructs a sequence of T=20 axial slices, and a Conv-LSTM-based recurrent U-Net that predicts the next frame. The models are trained on 54 healthy controls from the NYU ABIDE site, applied to 28 held-out controls and 67 ASD patients, and a single whole-gray-matter mean squared error per subject is used as an abnormality score. The authors report that both models yield higher errors in ASD than controls, with AUCs of 69.6 (autoencoder), 62.5 (U-Net imputation), and 65.9 (recurrent U-Net imputation), and they claim this is close to supervised state-of-the-art results. They also present a regional analysis of reconstruction-error differences. The manuscript does not include a covariate-adjusted group comparison, confidence intervals, or repeated-split validation.","tokens_in":5756,"tokens_out":5958,"duration_ms":54314,"significance":"The core idea is interesting and the modeling is technically sound: training on healthy controls only and measuring reconstruction/prediction error as a label-free abnormality score avoids circularity, since the score is a function of the input alone. The architectural comparison against last-frame, spline, and non-recurrent baselines is a strength, and the predictive performance (Pearson correlation 0.999 for the recurrent U-Net) indicates the model genuinely captures rs-fMRI dynamics. If the ASD-control discrimination survived confound adjustment and replicated across splits, the approach would be a meaningful contribution to unsupervised biomarker discovery. In its current form, however, the empirical support for the central claim is weakened by unadjusted group differences and single-split statistics.","major_comments":[{"comment":"The central claim that reconstruction/prediction error can discriminate ASD from controls is not yet established because the control and ASD test groups are not shown to be matched on age, sex, IQ, medication, or residual motion/scan quality, and no covariate adjustment is performed in the group comparison. The only confound check reported is correlation with framewise displacement; one of the two frame-level correlations is nominally significant (p=0.0251), so the text's statement that there is no correlation is not supported. A covariate-adjusted analysis (e.g., regressing subject-level error on group plus these confounds) or a matched-subgroup comparison is required.","section":"Section 4.2, Table 3"},{"comment":"Because the subject-level anomaly score is a single mean squared error averaged over the whole gray-matter mask and over all retained frames, any systematic between-group difference in signal variance, number of frames retained after scrubbing, or preprocessing residual will masquerade as an abnormality even in the absence of disease-specific dynamics. The manuscript does not report these quantities separately for the two groups; please report the distributions of gray-matter signal variance, frame counts after scrubbing, and mean framewise displacement by group, and include them as covariates in the discrimination analysis.","section":"Section 3.1, Section 4.2"},{"comment":"The evaluation relies on a single random two-thirds/one-thirds split with only 28 healthy controls in the test set, and the reported p-values are from unpaired t-tests on the error metric rather than tests of the AUC. No confidence intervals, bootstrap resampling, or repeated-split/permutation analysis is provided, so the stability of the AUC values (69.6, 62.5, 65.9) is unknown. Please report bootstrap confidence intervals for the AUCs and a proper AUC significance test (e.g., DeLong or permutation), and ideally repeat the split several times.","section":"Section 3.1, Table 3"},{"comment":"The sequence length T=20 was selected after observing diminishing returns in prediction performance and then fixed for the autoencoder, so the subsequent p-values and AUCs in Table 3 do not account for this model-selection step. Please report the sensitivity of the discrimination results to T, or define a validation-based selection rule before computing the test statistics.","section":"Section 3.2, Table 2"}],"minor_comments":[{"comment":"The random split into training/validation/test is described without a seed or code release, so the reported numbers are not reproducible as stated; please specify the split procedure and seed.","section":"Section 3.1"},{"comment":"The statement that 'validation curves were monitored to ensure convergence' does not specify the early-stopping rule; please state the exact criterion used.","section":"Section 3.2"},{"comment":"The comparison of AUC values with supervised results from [1] is indirect because the studies use different sites, preprocessing, and evaluation protocols; please qualify the comparison explicitly.","section":"Discussion"},{"comment":"The regional significance map would be easier to interpret with a colorbar and an explicit statement of the FDR threshold applied; also clarify whether the t-test was two-sided.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper, and the stress-test note is right—the central discrimination claim overreaches what the current analysis can support. Trained only on healthy controls and tested on held-out controls and ASD patients from ABIDE NYU, the paper shows that a Conv-LSTM autoencoder and a recurrent next-frame predictor produce reconstruction/prediction errors that differ between groups (AUCs around 0.63–0.70, p-values about 0.001–0.005). That is a plausible and useful demonstration that unsupervised temporal-dynamics models might be used for outlier detection in rs-fMRI.\n\nWhat is genuinely good: the methods are standard but appropriately adapted; the prediction model is compared against meaningful baselines (last-frame copy, spline extrapolation/interpolation, non-recurrent U-Net) and wins on held-out healthy test data; the anatomical error map uses FDR correction and points to language-network regions consistent with the autism literature; and the anomaly score is a function of the input alone, not fitted to labels, so the core claim is not circular. The paper is also honest about training only on healthy subjects and using motion scrubbing. The citation pattern is normal for the area, and the self-cited review is not load-bearing.\n\nThe soft spot is the load-bearing one. The statistically significant group difference is established with unpaired t-tests on a single split (28 controls vs 67 ASD) with no covariate adjustment. The groups are never shown to be matched on age, sex, IQ, medication, or residual motion/scan quality, and those variables could easily produce a mean squared error difference that has nothing to do with disease-specific dynamics. The motion check is limited to framewise displacement, and one of the two frame-level correlations is nominally significant (p=0.0251), so even that check is not clean. There are also no confidence intervals, no permutation null, and T=20 was selected after observing performance, which is mild post-hoc selection. The \"state-of-the-art supervised methods\" comparison is from other papers, not from a shared evaluation pipeline, so it is weaker than the phrase suggests. No code or data are released, which hurts reproducibility.\n\nIf the authors add matching or covariate adjustment (age, sex, IQ, medication, motion), report bootstrap CIs, and run simple baselines like variance of the raw time course, the claim could be supported. As it stands, the paper is a nice proof-of-concept but not yet a validated abnormality detector. I would still send it to peer review—the application is timely and the issues are fixable—but with a clear request to address the confound problem before acceptance.","headline":"A clean, well-built application of unsupervised video anomaly detection to rs-fMRI, but the ASD-control discrimination claim is not yet established because the two test groups are not matched or adjusted for obvious confounds.","tokens_in":6254,"tokens_out":2763,"would_cite":false,"duration_ms":29269,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","92C55"],"pacs":[],"model":"deepseek-v4-flash","headline":"Unsupervised models trained only on healthy resting-state scans flag autism from reconstruction and next-frame prediction errors.","keywords":["resting-state fMRI","unsupervised learning","anomaly detection","autoencoder","next-frame prediction","convolutional LSTM","autism spectrum disorder","functional connectivity dynamics"],"falsifier":"Compute the error-based discrimination AUC in a validation sample where patients and controls are matched on age, sex, IQ, and medication status, or after explicitly regressing those covariates out of the error scores; if the AUC drops to chance, the claim that the anomaly score captures disease-specific abnormalities is refuted.","tokens_in":5362,"feed_emoji":"🧠","tokens_out":4878,"duration_ms":46100,"temperature":0.7,"pith_summary":"This paper asks whether a model trained only on healthy people's resting-state brain scans can flag abnormal brain dynamics in patients, without needing any labeled patient data. The authors train two recurrent neural networks on healthy controls: one reconstructs short clips of rs-fMRI sequences, and one predicts the next frame. They then use the mean squared error of reconstruction or prediction as an anomaly score, and show that this score is significantly higher in autism patients than in healthy controls, yielding ROC areas of about 70% for the autoencoder and 66% for the prediction model. Because the models never see patients during training, the approach offers a label-free route to abnormality detection, with performance near supervised classifiers.","feed_headline":"Prediction errors in resting-state fMRI identify autism patients","feed_subtitle":"Two unsupervised neural nets, trained only on controls, separate ASD from healthy with ROC areas near 70 percent in the ABIDE NYU sample.","key_machinery":"The load-bearing object is the anomaly score: the mean squared error between the model's output and the observed rs-fMRI frames. The models are encoder-decoder networks built on Conv-LSTM cells, meaning LSTM units extended to operate on spatial feature maps, so they propagate both spatial and temporal structure. A U-Net-style decoder with skip connections performs next-frame prediction, while a similar network without skip connections reconstructs the whole sequence; both are trained end-to-end on healthy controls only. At test time, a subject whose error is high relative to the healthy distribution is flagged as an outlier.","core_discovery":"The paper's central claim is that deviations from learned normal resting-state dynamics, measured as reconstruction or next-frame prediction error, carry diagnostic signal: in the ABIDE NYU cohort, both the recurrent autoencoder (AUC 69.6%) and the recurrent U-Net imputation model (AUC 65.9%) separate autism patients from healthy controls with statistical significance, while a non-recurrent U-Net gives weaker separation. The error maps localize significant group differences mainly to left-hemisphere temporal and frontal regions, consistent with prior reports on language-network involvement in autism.","pith_inferences":["If the approach works for autism because of atypical temporal dynamics, it should transfer to other disorders with dynamic connectivity alterations, such as schizophrenia or depression; testing on a second diagnostic group would show whether the anomaly score is a general atypicality detector or autism-specific.","Frame-level prediction errors could be used to time-stamp transient abnormalities that static connectivity averages away, offering a per-volume diagnostic marker.","The fact that reconstruction error produces a higher AUC than next-frame prediction suggests that memorizing normal dynamics may be a stronger anomaly detector than forecasting them, a hypothesis the paper does not explicitly test."],"forward_implications":["A scan can be scored without any patient labels, which matters for conditions where labeled cohorts are small.","Voxel-level error maps can point to the anatomical location of atypical dynamics, generating hypotheses about affected networks.","The lack of correlation between frame-wise displacement and error suggests the anomaly signal is not simply residual motion.","These unsupervised error features could be combined with static connectivity measures to push classification accuracy beyond either alone.","The same framework could be reused for frame imputation, to fill in volumes removed by motion scrubbing."],"supporting_citations":[{"why":"Establishes the supervised state-of-the-art ASD-vs-control classification baseline that the AUC results are compared against.","marker":"[1]"},{"why":"Supplies prior evidence localizing language-network abnormalities in autism, used to interpret the anatomical error map.","marker":"[3]"},{"why":"Defines the LSTM unit that the Conv-LSTM cells are built on.","marker":"[5]"},{"why":"Introduces future-frame prediction as an anomaly-detection strategy, the precedent for the imputation model.","marker":"[8]"},{"why":"Provides the ABIDE dataset from which the NYU training and test samples are drawn.","marker":"[9]"},{"why":"Supplies the U-Net encoder-decoder architecture that the prediction model extends.","marker":"[10]"},{"why":"Introduces Conv-LSTM cells, the mechanism that lets the model capture spatiotemporal dynamics.","marker":"[11]"},{"why":"Provides the AAL atlas used to parcellate the brain for regional error analysis.","marker":"[15]"}],"fun_headline_variants":["Prediction errors in brain scans spot autism","Unsupervised learning flags autism from fMRI dynamics","Resting-state fMRI errors separate autism from healthy","Autoencoder and U-Net detect autism in brain activity","Abnormal dynamics in resting fMRI reveal autism"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The higher error in patients reflects disease-related neural dynamics rather than unmeasured differences between the groups, such as age, sex, IQ, medication, or residual motion, because the clinical groups are not matched on these variables.","fun_headline_variants_meta":{"raw":{"variants":["Prediction errors in brain scans spot autism","Unsupervised learning flags autism from fMRI dynamics","Resting-state fMRI errors separate autism from healthy","Autoencoder and U-Net detect autism in brain activity","Abnormal dynamics in resting fMRI reveal autism"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000266,"raw_usage":{"total_tokens":1526,"prompt_tokens":776,"completion_tokens":750,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":392,"completion_tokens_details":{"reasoning_tokens":679}},"tokens_in":392,"tokens_out":750,"duration_ms":8087,"temperature":1.0,"reasoning_tokens":679,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:53:29.377827+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the error-based discrimination AUC in a validation sample where patients and controls are matched on age, sex, IQ, and medication status, or after explicitly regressing those covariates out of the error scores; if the AUC drops to chance, the claim that the anomaly score captures disease-specific abnormalities is refuted.","supporting_citations":[{"cited_title":"Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example","cited_arxiv_id":null,"evidence_quote":"Establishes the supervised state-of-the-art ASD-vs-control classification baseline that the AUC results are compared against."},{"cited_title":"A failure of left temporal cortex to specialize for language is an early emerging and fundamental property of autism","cited_arxiv_id":null,"evidence_quote":"Supplies prior evidence localizing language-network abnormalities in autism, used to interpret the anatomical error map."},{"cited_title":"Long short-term memory.Neural com- putation, 9(8):1735–1780, 1997","cited_arxiv_id":null,"evidence_quote":"Defines the LSTM unit that the Conv-LSTM cells are built on."},{"cited_title":"Future frame prediction for anomaly detection - a new baseline","cited_arxiv_id":null,"evidence_quote":"Introduces future-frame prediction as an anomaly-detection strategy, the precedent for the imputation model."},{"cited_title":"The autism brain imaging data exchange:towards a large-scale evaluation of intrinsic brain architecture in autism","cited_arxiv_id":null,"evidence_quote":"Provides the ABIDE dataset from which the NYU training and test samples are drawn."},{"cited_title":"U-net: Convolutional networks for biomedical image seg- mentation","cited_arxiv_id":null,"evidence_quote":"Supplies the U-Net encoder-decoder architecture that the prediction model extends."},{"cited_title":"Convolutional lstm network: A machine learning approach for precipitation nowcasting","cited_arxiv_id":null,"evidence_quote":"Introduces Conv-LSTM cells, the mechanism that lets the model capture spatiotemporal dynamics."},{"cited_title":"Tzourio-Mazoyer et al","cited_arxiv_id":null,"evidence_quote":"Provides the AAL atlas used to parcellate the brain for regional error analysis."}],"review_version":1}