{"id":"c772e519-72ce-4f14-a85e-5c23e431655d","arxiv_id":"2412.07783","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SwiFT, a 4D fMRI transformer, combined with group-ICA features and multi-label learning, outperforms ROI-based baselines in predicting Bayley-III scores from neonatal fMRI, with modest effect sizes.","lead":"This paper applies a 4D spatiotemporal transformer, SwiFT, to neonatal fMRI from the Developing Human Connectome Project to predict Bayley-III cognitive, language, and motor scores in toddlers. It reports that combining the transformer with group-ICA features and multi-label learning beats ROI-based baselines, though the gains are modest.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Multi-ICA's reported gains may stem from comparing models on different test cohorts: the 100 healthy ICA-defining subjects are always in training, so ICA models are evaluated on a reduced, less representative test set while baselines appear to use the full cohort.","rationale":"The reader identified the same load-bearing weakness: fixing the group-ICA estimate to 100 healthy subjects always placed in training biases the evaluation of ICA-based models. I agree with this and sharpen it: the comparison to baselines is likely confounded because the test folds for ICA models differ from those for baselines, and the ICA maps are estimated from an outcome-selected healthy subset. This directly threatens the strongest claim that Multi-ICA 'significantly and consistently outperforms' the strongest baseline. The concern is not about the architecture itself or about disagreement with consensus; it is about whether the reported p-values compare like with like. A matched re-analysis with within-fold ICA estimation and identical test folds would settle whether the claimed advantage is real. The paper is otherwise transparent about limitations, including pretraining not transferring and data imbalance, which supports a conditional rather than rejecting verdict.","tokens_in":10,"tokens_out":5038,"duration_ms":117558,"concrete_test":"Re-run the comparison under a strictly matched protocol: for each of the 5 folds, estimate group-ICA maps from training subjects only, and evaluate every model (baselines, raw SwiFT, and ICA-SwiFT) on the identical held-out test fold, with the same fold assignments for all models. Report per-fold MAE and AUC for each model on the same subjects, and apply a paired permutation test across folds with pre-specified correction for the multiple domains and metrics. If Multi-ICA no longer consistently beats the best baseline on the majority of metrics, the central claim is not supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that Multi-ICA significantly outperforms the strongest baseline (Section 3.5). This rests on Tables 1-3 and the p-values reported there. However, Section 3.4 states that 'the 100 healthy subjects used for Group ICA were always included in the training set' to avoid data leakage. That design means every ICA-experiment test fold excludes those 100 healthy subjects. If baseline experiments from Section 3.2 used 5-fold CV on the full dataset, the two sets of metrics are not computed on the same test population, so the p-values comparing them are not valid. Even if fold assignments were adjusted so that baselines also exclude those 100 subjects, the test set would still be non-representative: healthy (low-risk) subjects are under-represented, and the model is never tested on exactly the population it is intended to predict. The group-ICA maps are also estimated once from a healthy-only, outcome-selected subset and never re-estimated within each training fold, which violates a clean nested-CV protocol and means test subjects' features depend on a non-representative atlas. The statistical test itself is underspecified: no test name, no multiple-comparison correction, and the same 100 subjects appear in every training fold, which correlates folds and can deflate variance estimates. These issues jointly undermine the reported p-values and the 'consistent' gains claimed for Multi-ICA.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SwiFT, a 4D Swin Transformer, to predict Bayley-III cognitive, language, and motor composite scores from neonatal resting-state fMRI in the dHCP cohort. The authors evaluate SwiFT on raw fMRI volumes and on group-ICA-reduced features, under both single-label and multi-label prediction, comparing against ROI-based baselines (BrainNetCNN, VanillaTF, BNT, XGBoost). They report that a multi-label ICA-based variant (Multi-ICA) significantly outperforms the strongest baseline on several metrics, with p-values such as p=0.004 for cognitive MAE and p=0.002 for motor MAE in Section 3.5. They also use Integrated Gradients with Smoothgrad SQuare to identify brain regions associated with each outcome. The central empirical claim is that Multi-ICA provides significant and consistent gains over the best baseline, with the strongest evidence claimed for regression tasks.","tokens_in":12602,"tokens_out":5677,"duration_ms":51502,"significance":"If the evaluation were sound, the paper would be of interest to the neurodevelopmental-prediction community: predicting Bayley-III scores from neonatal fMRI is a clinically relevant task, the use of a 4D spatiotemporal Transformer is a reasonable architectural choice, and the inclusion of group-ICA feature extraction and IG-SQ interpretability adds value. The paper also uses a well-known public dataset (dHCP) and reports multiple baselines. However, the current evaluation protocol contains a test-cohort mismatch, outcome-based selection in the ICA feature-extraction step, and underspecified statistical testing. These issues directly affect the validity of the reported p-values and the paper's central claim that Multi-ICA significantly and consistently outperforms the baselines. The interpretability analysis is a strength but does not compensate for the statistical weaknesses in the predictive comparison.","major_comments":[{"comment":"The evaluation protocol for ICA-based models is not comparable to that for the baselines. Section 3.4 states that 'the 100 healthy subjects used for Group ICA were always included in the training set,' which means the test folds for ICA experiments always exclude those 100 subjects. Section 3.1 states that '5-fold cross-validation ensured consistent and fair splits between experiments,' suggesting that the baseline experiments in Section 3.2 use the full cohort (619 subjects with usable fMRI data). If this is the case, the Multi-ICA results in Table 3 and the baseline results in Table 1 are computed on different test populations, and the p-values in Section 3.5 comparing them are not valid. Please specify the exact subject count and fold assignment for every experiment, and either use identical test-fold compositions for all models or restrict all comparisons to the same subject subset.","section":"Section 3.4 vs. Section 3.1"},{"comment":"The group-ICA maps are estimated from 100 neonates selected because they have all Bayley-III scores above 85, i.e., an outcome-selected healthy subset. Even though ICA itself is unsupervised, the choice of the 100 subjects uses the outcome labels, and the same 100 subjects are always placed in the training set. This has two consequences: (i) the ICA atlas is estimated from a non-representative, healthy-only sample, and (ii) for test subjects, the ICA features are derived from an atlas that was built using label information and the same atlas is reused across all folds. This can bias the feature space and inflate apparent performance of ICA-based models. Please justify this design or re-estimate the ICA within each training fold on training data only, without selecting subjects by outcome.","section":"Section 3.4, 'Data Sampling'"},{"comment":"The statistical significance analysis is underspecified. No test name is given, no correction for multiple comparisons is described, and the fold structure (the same 100 subjects appear in every training fold) creates dependence across folds that standard paired tests do not handle. The p-values reported (e.g., cognitive MAE p=0.004, motor MAE p=0.002) are not tied to a specific test procedure, and it is not stated whether they compare the Multi-ICA model to the best baseline for each metric or to raw SwiFT. Given the large number of metrics, tasks, and model variants, a multiple-comparison correction is essential. Please provide the full statistical testing protocol, including the test name, the paired comparison unit, and the correction method.","section":"Section 3.5"},{"comment":"The claim that 'SwiFT yielded significant performance gains for both regression and classification tasks' in the raw-fMRI experiments is not supported by any reported significance test. The only p-values appear in Section 3.5 and concern the Multi-ICA variant. If the paper intends to claim significant gains for raw SwiFT (Table 2) over baselines (Table 1), those comparisons must be statistically tested as well, with the same cohort-matching and multiple-comparison considerations. As written, this statement overstates the evidence.","section":"Section 3.3"}],"minor_comments":[{"comment":"The sentence 'Due to the specific sequence lengths required for IC-based training (42 and 100)' is confusing: earlier in Section 3.3 the sequence lengths are given as 20, 50, and 100. Please clarify what the values 42 and 100 refer to.","section":"Section 3.4, 'Training Process'"},{"comment":"The caption says 'The strongest baseline was taken to represent baseline performance' but does not specify which baseline was used for each task and metric. Please give the exact baseline model per comparison, as the best baseline varies across metrics in Table 1.","section":"Figure 4 caption"},{"comment":"The sentence 'could be implemented into the current pipeline without major changes Finally, pretraining on adult data...' is missing a period or semicolon between 'changes' and 'Finally'. Please fix the punctuation.","section":"Section 5"},{"comment":"The abstract states that 'SwiFT significantly outperforms baseline models' but Section 3.5 reports several non-significant comparisons (e.g., cognitive classification AUC p=0.413). Please temper the abstract to reflect the specific metrics and model variants for which significance was found.","section":"Abstract"},{"comment":"The tables would be more readable if the best result in each column were bolded or marked, and if the number of subjects used in each experiment (e.g., 619 vs. 519) were stated in the table caption. This would also help the reader identify the cohort mismatch discussed above.","section":"Tables 1-3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a clinically relevant prediction problem and has a reasonable architectural motivation, but the evaluation protocol as written does not support the headline claim of significant and consistent improvement. The cohort mismatch and outcome-based ICA selection are fixable with a re-analysis using identical test folds and a properly nested feature-extraction procedure; the statistical testing also needs to be specified and corrected for multiple comparisons. I would encourage the authors to resubmit after such a re-analysis, as the underlying approach could be valuable if validated appropriately."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper’s central comparison is compromised. Multi-ICA’s reported gains over baselines (cognitive MAE p=0.004, motor MAE p=0.002) may be an artifact of different test cohorts. Section 3.4 says the 100 healthy subjects used for Group ICA were always in the training set. That means ICA models are tested only on the remaining, healthier, GA-restricted subjects, while the baseline models in Section 3.2 appear to be evaluated on the full 619-subject cohort. You can’t run a significance test between metrics computed on different test populations and call it a consistent improvement.\n\nWhat’s new: applying SwiFT to neonatal fMRI for BSID-III prediction, with multi-label learning and ICA features, is a legitimate new application. The paper is transparent: it reports means and SDs over 5-fold CV, admits pretraining didn’t transfer, and notes that ICA dimensionality (25 vs 100) made little difference. That level of honesty is welcome.\n\nThe soft spots, in order of severity: (1) the test-cohort mismatch above; (2) the group ICA maps are estimated once from outcome-selected healthy subjects and not re-estimated within each fold, which violates a clean nested-CV design and means test features depend on a non-representative atlas; (3) the statistical test is underspecified – no test name, no multiple-comparison correction, and folds are correlated because the same 100 subjects appear in every training set. These are fixable with a matched-cohort re-analysis and proper nested CV. Effect sizes are modest (AUC ~0.6), and no code is released, but those are secondary for an application paper.\n\nWho this is for: researchers working on early neurodevelopmental prediction from fMRI and on transformer applications to small medical imaging datasets. It’s a worthwhile question and the pipeline is plausible, so I’d send it to review rather than desk-reject. But the authors need to redo the evaluation before the comparative claims can be trusted.\n\nRecommendation: peer review, with major revision expected.","headline":"The application is plausible, but the central comparison is undermined by a test-cohort mismatch that likely explains the reported ICA gains.","tokens_in":13093,"tokens_out":2523,"would_cite":false,"duration_ms":24328,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SwiFT, a 4D spatiotemporal Transformer, predicts neonatal Bayley-III scores from fMRI better than connectivity-based baselines.","keywords":["fMRI Transformer","neonatal fMRI","Bayley-III","group ICA","multi-label prediction","neurodevelopmental outcomes","explainable AI"],"falsifier":"Re-run the full pipeline with ICA components estimated from all subjects, or re-estimated within each cross-validation fold, and check whether Multi-ICA still outperforms the baselines; if the advantage disappears or shrinks to noise, the reported gains are an artifact of the held-out cohort composition.","tokens_in":12111,"feed_emoji":"🧠","tokens_out":5151,"duration_ms":42165,"temperature":0.7,"pith_summary":"The paper argues that a 4D spatiotemporal Transformer called SwiFT can predict Bayley-III cognitive, language, and motor composite scores from neonatal resting-state fMRI. The central result is that combining SwiFT with group-ICA feature reduction and multi-label learning (predicting the three scores together) yields consistently better predictions than baseline models that use static functional connectivity matrices. If the claim holds, early brain scans could flag infants at risk of developmental delay, creating a wider window for intervention. The paper also shows that the model's attributions point to brain regions consistent with known functions for each outcome.","feed_headline":"Transformer predicts infant development scores from neonatal fMRI","feed_subtitle":"SwiFT with group-ICA features beats connectivity baselines for Bayley-III cognitive, motor, language.","key_machinery":"The central object is SwiFT, a Swin 4D fMRI Transformer that patches 4D fMRI volumes into spatiotemporal tokens and applies shifted-window multi-head self-attention to capture local and global dependencies across space and time. The paper reduces dimensionality with group ICA (25 or 100 components), then uses dual regression to obtain subject-specific spatial maps and functional connectivity maps as input features. A shared classification/regression head predicts the three correlated Bayley-III outcomes simultaneously. Interpretations are generated with Integrated Gradients with Smoothgrad sQuare (IG-SQ), averaged across correctly classified test subjects.","core_discovery":"On the paper's own terms, the discovery is that SwiFT with group-ICA features and multi-label learning (Multi-ICA) significantly outperforms the strongest baseline in predicting Bayley-III scores: cognitive MAE p=0.004, motor MAE p=0.002, cognitive regression MSE p=0.036, language regression MSE p=0.008, and language classification accuracy p=0.004. Both single-label and multi-label SwiFT variants exceed the baselines in balanced accuracy and regression error, while pretraining on adult fMRI datasets produced only marginal gains. The authors report that multi-label predictions align with the high correlations among cognitive, language, and motor scores, and that Integrated Gradients attributions identify medial prefrontal, Wernicke's, and motor regions for the respective delay risks.","pith_inferences":["The evaluation fixed the 100 healthy ICA-estimation subjects into the training set, so test folds contain only the remaining infants; re-estimating ICA inside each fold could change the reported gains.","Adult-data pretraining gave marginal gains, so the paper's evidence suggests cross-age pretraining is not an effective transfer path; neonatal-specific pretraining should be tested.","Significance testing across numerous metrics without correction means some reported p-values would not survive a multiple-comparison adjustment; the robust core is the regression and motor/language improvements.","The interpretation maps are averaged only over correctly classified subjects, which may make attributions look more consistent than they would be across all test cases."],"forward_implications":["Multi-label learning across correlated developmental domains improves prediction for each individual domain.","Group-ICA feature reduction cuts input dimensionality without harming predictive performance, with 25 and 100 components performing similarly.","Balanced accuracy for at-risk classification exceeds guessing levels across all three domains, suggesting the signal is usable for early screening.","Attribution maps align with established neuroscience, giving the predictions a neurobiological interpretation."],"supporting_citations":[{"why":"Supplies the SwiFT architecture, the 4D spatiotemporal attention mechanism, sequence-length conventions, pretraining scheme, and the IG-SQ interpretation method used throughout.","marker":"[15]"},{"why":"Provides the dHCP automated resting-state fMRI processing framework that produces the neonatal input data.","marker":"[2]"},{"why":"FastICA algorithm used for the spatial independent component analysis in group ICA.","marker":"[10]"},{"why":"Probabilistic ICA for fMRI (FSL MELODIC) used to decompose data into independent spatial components.","marker":"[11]"},{"why":"Dual regression method used to derive subject-specific spatial maps and connectivity features from group ICA.","marker":"[34]"},{"why":"Brain Network Transformer, a core baseline model taking functional connectivity matrices as input.","marker":"[32]"},{"why":"BrainNetCNN, a baseline convolutional model for brain networks used for neurodevelopment prediction.","marker":"[6]"},{"why":"Multi-task deep transfer learning model for early neurodevelopment prediction, providing the rationale for multi-label learning.","marker":"[23]"}],"fun_headline_variants":["SwiFT transformer beats connectivity baselines on Bayley-III scores","Neonatal fMRI Swin model predicts Bayley-III cognitive, motor, language","Swin 4D transformer improves infant development prediction from fMRI","SwiFT model outperforms in forecasting Bayley-III from newborn scans","Transformer with group-ICA features excels at neonatal outcome prediction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that always keeping the 100 healthy infants used to estimate the ICA components in the training set gives an unbiased performance estimate; the test folds then consist only of the remaining infants, and the ICA features come from an outcome-selected subset.","fun_headline_variants_meta":{"raw":{"variants":["SwiFT transformer beats connectivity baselines on Bayley-III scores","Neonatal fMRI Swin model predicts Bayley-III cognitive, motor, language","Swin 4D transformer improves infant development prediction from fMRI","SwiFT model outperforms in forecasting Bayley-III from newborn scans","Transformer with group-ICA features excels at neonatal outcome prediction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001256,"raw_usage":{"total_tokens":5123,"prompt_tokens":896,"completion_tokens":4227,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":4135}},"tokens_in":512,"tokens_out":4227,"duration_ms":29012,"temperature":1.0,"reasoning_tokens":4135,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:15:38.288136+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the full pipeline with ICA components estimated from all subjects, or re-estimated within each cross-validation fold, and check whether Multi-ICA still outperforms the baselines; if the advantage disappears or shrinks to noise, the reported gains are an artifact of the held-out cohort composition.","supporting_citations":[{"cited_title":"Fitzgibbon, Samuel J","cited_arxiv_id":null,"evidence_quote":"Provides the dHCP automated resting-state fMRI processing framework that produces the neonatal input data."},{"cited_title":"IEEE Transactions on Neural Networks 10, 626–634 (1999)","cited_arxiv_id":null,"evidence_quote":"FastICA algorithm used for the spatial independent component analysis in group ICA."},{"cited_title":"IEEE Transactions on Medical Imaging 23, 137–152 (2004)","cited_arxiv_id":null,"evidence_quote":"Probabilistic ICA for fMRI (FSL MELODIC) used to decompose data into independent spatial components."},{"cited_title":"NeuroImage 47, S148 (2009)","cited_arxiv_id":null,"evidence_quote":"Dual regression method used to derive subject-specific spatial maps and connectivity features from group ICA."},{"cited_title":"25586–25599","cited_arxiv_id":null,"evidence_quote":"Brain Network Transformer, a core baseline model taking functional connectivity matrices as input."},{"cited_title":"Brown, Steven P","cited_arxiv_id":null,"evidence_quote":"BrainNetCNN, a baseline convolutional model for brain networks used for neurodevelopment prediction."},{"cited_title":"Scientific Reports 10, 15072 (2020)","cited_arxiv_id":null,"evidence_quote":"Multi-task deep transfer learning model for early neurodevelopment prediction, providing the rationale for multi-label learning."}],"review_version":1}