{"id":"768cdb3a-a077-4351-a12d-01d48b84158d","arxiv_id":"2506.22591","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"BrainMT, a Mamba-transformer hybrid with a temporal-first scan, is claimed to outperform prior models on fMRI-based sex and intelligence prediction.","lead":"This paper proposes BrainMT, a hybrid Mamba-transformer deep network for predicting traits like sex and intelligence directly from resting-state fMRI brain scans. It reports large gains over prior models on two public datasets, but the architecture as described appears computationally infeasible and the code is not yet released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper describes global self-attention over L = T × K ≈ 44,000 tokens, which is infeasible on a 48GB L40S GPU, while also claiming linear-in-T memory; the architecture actually evaluated must differ from the text.","rationale":"The reader's weakest assumption is exactly the load-bearing concern: the architecture as written is not trainable on the stated hardware, and the claimed linear-in-T memory contradicts global attention over T×K tokens. This concern is central because the paper's headline result (SOTA on UKB and HCP) depends on the evaluated architecture matching the described architecture; if the real model uses per-volume attention or temporal pooling, the novelty claims about 'global spatial relationships' and the memory-efficiency comparison are undermined, and the experiments cannot be reproduced from the text. The paper has independent value: ablations, repeated cross-validation, and standard deviations are reported, and the Mamba component is plausible for long fMRI sequences. However, no code is provided, and the reader's earlier reviewer-style details (e.g., the abstract overstating UKB sex classification) are secondary to this architectural inconsistency. A conditional acceptance is appropriate because the discrepancy is potentially fixable by releasing code or revising the description; the SOTA claim is not internally refuted, only unverifiable as written. This stress-test does not change the reader's CONDITIONAL verdict.","tokens_in":9051,"tokens_out":6284,"duration_ms":69600,"concrete_test":"Run the released code with the reported configuration (T=200, 12 Mamba layers, 8 Transformer layers, batch size 2) on a single 48GB L40S and measure peak GPU memory and time per epoch. Separately instrument the Transformer block to log the actual input sequence length: if the transformer sees L ≈ 44,000, the run should OOM; if it sees L ≈ K (e.g., 220) or a temporally pooled length, the paper's description is inconsistent with the implementation. As an analytic cross-check, re-derive the FLOPs for Eq. 5 with L = T×K and estimate whether one training epoch is plausible within the reported wall-clock budget.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 2 (Positional embedding), the sequence is reshaped to L = T × K, with T=200 and K≈220, giving L≈44,000. The Transformer block then applies global softmax self-attention (Eq. 5) over this full sequence, with Q, K, V ∈ R^(L×Z). For L=44k, a dense attention matrix has ~1.9×10^9 entries per head; with 8 heads and fp16, the logits and weights alone exceed 30GB, before Q/K/V projections, MLPs, gradients, and activations. Training with batch size 2 on a 48GB L40S is therefore not feasible under standard PyTorch. The paper does not mention FlashAttention, sparse/factorized attention, windowed attention, or any sequence-length reduction between the Mamba and Transformer blocks. Moreover, Section 3 and Figure 1d explicitly claim BrainMT maintains 'linear complexity in T' and is 35.8% more memory-efficient than SwiFT; global attention over L = T×K would be quadratic in T, directly contradicting that claim. The most plausible resolution is that the evaluated model attends only within each time frame (L=K, applied T times) or applies temporal pooling before the Transformer, meaning the architecture evaluated is not the architecture described. Without code or a corrected description, the central SOTA claim cannot be independently reproduced or verified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BrainMT, a hybrid Mamba-transformer architecture for voxel-based fMRI phenotype prediction. The pipeline combines a 3D convolution block, a bidirectional Vision Mamba block with a temporal-first scan, and a multi-head self-attention transformer, followed by an MLP head on a classification token. The model is evaluated on sex classification and cognitive-intelligence regression using UK Biobank (6000 subjects) and HCP (1075 subjects), with repeated three-fold cross-validation against XGBoost, BrainNetCNN, BrainGNN, BrainNetTF, TFF, and SwiFT. The paper reports improved MSE/MAE/R and accuracy/AUROC relative to these baselines, plus ablations and Integrated Gradients maps. The central claim is state-of-the-art phenotype prediction while maintaining linear memory growth in the number of time frames.","tokens_in":9332,"tokens_out":6904,"duration_ms":69925,"significance":"If the empirical results hold, this is a potentially valuable contribution. The paper demonstrates that a 200-frame voxel-level input can be processed end-to-end, reports standard deviations from repeated cross-validation, includes a systematic ablation over components, layer counts, and input length, and links predictions to well-established brain networks through Integrated Gradients. The memory-efficiency comparison against SwiFT is also informative. However, the significance is conditional: the architecture as described is not computationally feasible for the stated sequence length, no code is provided to resolve the ambiguity, and the comparison with baselines may conflate architecture with input coverage. These issues prevent the current version from serving as a reproducible state-of-the-art claim.","major_comments":[{"comment":"The manuscript defines Q, K, V in R^(L×Z) with L = T × K and sets T = 200 in Section 3, but it never reports H, W, D, or K. For typical HCP/UKB volumetric resolutions, the downsampled K is in the hundreds; for example, K = 220 gives L ≈ 44,000 tokens. Under the dense softmax attention in Eq. (5), this would produce roughly 1.9 × 10^9 attention entries per head, which is not trainable on a 48GB L40S GPU at batch size 2 without sequence-length reduction or sparse/windowed attention. The text does not mention FlashAttention, factorized attention, windowed attention, temporal pooling, or any other mechanism that would make this tractable. This also contradicts the claim in Section 3 and Fig. 1d that BrainMT 'maintains linear complexity in T', because attention over L = T × K is quadratic in T. The authors must either specify the actual sequence-reduction or attention mechanism, or correct the architecture description and the linear-complexity claim.","section":"Section 2, 'Positional embedding' and 'Transformer block'; Section 3, 'Implementation details'; Fig. 1d"},{"comment":"The paper promises code at a GitHub URL, but no code, model definitions, or tensor-shape details are provided in the manuscript. Given the ambiguity about whether the transformer attends over the full L = T × K sequence or over a reduced sequence, the reader cannot determine which architecture was actually evaluated. The authors should provide at least a detailed tensor-shape table (T, H, W, D, K, L, Z after each block), a pseudocode description of the transformer block, and the exact number of attention heads; ideally, they should release the code or state precisely why the sequence length is computationally manageable.","section":"Abstract and Section 3, 'Implementation details'"},{"comment":"The paper states that baselines were run using their original implementations and hyperparameter settings, and the introduction notes that TFF and SwiFT process only 10–20 frames per forward pass, while BrainMT uses 200 frames. If the baseline results were produced with their native shorter windows, the comparison conflates architectural capacity with input temporal coverage. Please report the input length used for each baseline and include a matched-input-length comparison (e.g., SwiFT with 200 frames if feasible, or BrainMT with 20 frames) to isolate the architectural contribution to the reported gains.","section":"Section 3, 'Experimental Setup and Results'"}],"minor_comments":[{"comment":"The text claims that 'T = 200 is optimal', but the table lists only T = 100 and T = 300; the T = 200 row is missing. Add the T = 200 result or revise the statement so it is directly supported by the ablation data.","section":"Section 3, Table 3, Exp A"},{"comment":"The numeric cells are presented as concatenated values without visible plus-or-minus signs or color formatting, making it difficult to parse means and standard deviations. Please format each cell as 'mean ± std' and describe the color convention in the caption.","section":"Tables 1 and 2"},{"comment":"The memory-efficiency comparison (35.8% more memory-efficient than SwiFT) lacks measurement details: batch size, sequence length, whether activations and gradients are included, and hardware are not specified. Add these details to the caption or the implementation section.","section":"Fig. 1d and Section 3"},{"comment":"The text calls the transformer module a model of 'global spatial relationships', but the sequence is formed by concatenating spatial and temporal tokens into L = T × K. This terminology is misleading; the attention mechanism, as written, is spatiotemporal rather than purely spatial.","section":"Section 2, 'Positional embedding' and 'Transformer block'"},{"comment":"The ablation titled 'predicting functional connectivity correlations' introduces a new task without explaining the prediction target, the evaluation metric, or how SwiFT and BrainMT were adapted for it. Add a brief description of this experiment so the comparison is interpretable.","section":"Table 3, Exp E"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the internal inconsistency between the described global attention over L = T × K and the reported training feasibility and linear-complexity claim. I believe this is fixable if the authors clarify the actual sequence reduction or attention mechanism, but without code or a corrected description, the reported SOTA results cannot be independently verified. I also note that the paper promises code but does not provide it; in an area where attention implementations are often heavily optimized, this is a significant reproducibility gap."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper has a serious internal inconsistency: the architecture it describes—global self-attention over roughly 44,000 tokens—cannot be trained on a 48GB L40S with batch size 2, and it contradicts the claimed linear-in-T complexity. Unless the authors release code or clarify the actual attention mechanism, the reported state-of-the-art results can't be reproduced from the text. That is the thing to know.\n\nCredit where it's due: the idea of combining a bidirectional Mamba with a transformer for volumetric fMRI, with a temporal-first scanning order, is a reasonable extension of recent Mamba work. The experiments are thorough, with repeated three-fold cross-validation, ablations over frame counts and components, and interpretability maps. If the results hold, the gains on UKB intelligence (R from 0.04 to 0.24) and HCP (R from 0.32 to 0.41) are substantial.\n\nThe load-bearing problem is the sequence length. With T=200 and K≈220, L=44,000. The paper claims the convolution and downsampling stages \"significantly reduce the sequence length,\" but they only shrink the spatial dimensions, not time. So the linear-complexity claim is simply wrong under the described architecture. I suspect they actually apply attention per time frame or after temporal pooling, but that's not what the text says. Without code, this is a gating question, not a minor clarity issue.\n\nA second, smaller issue: the abstract overstates UKB sex classification. BrainMT essentially matches SwiFT there (97.91 vs 97.45 accuracy, 99.2 vs 99.4 AUROC), so \"outperforming existing methods by a significant margin\" only holds for HCP sex and the intelligence tasks. Also, the paper should spell out how the 70/15/15 split interacts with the repeated three-fold cross-validation reported as the main result.\n\nThe paper is aimed at researchers doing voxel-based fMRI phenotype prediction, especially those wanting to use long time series. It deserves a serious referee—the question matters, and the empirical scaffold is otherwise solid—but the referee should push hard for code and a corrected architecture description. Until then, treat the state-of-the-art claim as provisional.","headline":"A promising empirical result on long fMRI sequences, but the architecture as described cannot be trained on the stated hardware—the paper is only publishable if the authors release code or correct the attention mechanism.","tokens_in":9882,"tokens_out":4791,"would_cite":false,"duration_ms":51654,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BrainMT claims that combining a bidirectional Mamba scan over time with a global transformer over space lets fMRI models ingest 200 frames at once and predict sex and cognitive intelligence more accurately than existing voxel-based and…","keywords":["fMRI","Mamba","transformer","state-space models","phenotype prediction","long-range dependencies","sex classification","intelligence prediction"],"falsifier":"Run the released implementation with the stated settings (T = 200, K roughly 220 spatial tokens, L roughly 44,000 tokens, 8 transformer heads, batch size 2 on a 48 GB GPU) and record peak GPU memory and the attention computation's input-output shapes; if the code cannot execute at that sequence length or the reported memory is exceeded, the central claim about a trainable global transformer over 200 frames is not supported as written.","tokens_in":8816,"feed_emoji":"🧠","tokens_out":11660,"duration_ms":122416,"temperature":0.7,"pith_summary":"The paper aims to establish that a hybrid sequence model—a bidirectional Mamba block that scans fMRI volumes time-first, followed by a transformer that models global spatial structure—predicts sex and cognitive intelligence from raw resting-state fMRI volumes more accurately than existing voxel-based and connectivity-based methods. The design targets a specific bottleneck: transformers are quadratic in sequence length, so prior fMRI transformer models could only process 10–20 time frames at once and missed slow hemodynamic dynamics. With 200 frames per subject, BrainMT reports an 8.75% mean-squared-error reduction on the 1,075-subject cohort and a 6.23% reduction on the 6,000-subject cohort for intelligence, plus higher sex-classification accuracy on the smaller cohort. If these results hold, BrainMT would be the strongest published voxel-based fMRI phenotype predictor and a practical recipe for applying long-range sequence models to 4D medical images.","feed_headline":"Hybrid fMRI model uses 200 frames, beats prior predictors","feed_subtitle":"Temporal-first Mamba plus a spatial transformer improves UK Biobank and HCP phenotype predictions.","key_machinery":"The load-bearing mechanism is the temporal-first bidirectional Mamba block feeding a global transformer. In the Mamba block, a selective state-space model evolves a hidden state with input-dependent parameters, so a sequence of L = T × K tokens (200 volumes times roughly 220 spatial patches after downsampling) can be processed in linear time; arranging tokens with time as the leading dimension is what lets the scan capture long-range temporal correlations before spatial ones. The transformer block then applies Attention(Q, K, V) = Softmax(QK^T / $\\sqrt$(d_head)) V over the same token sequence to model global spatial relationships, made tractable because the preceding convolution and Mamba stages reduce the sequence length. A learnable classification token X_cls is prepended, normalized, and passed through an MLP head for the final prediction.","core_discovery":"BrainMT's central claim is that long-range spatiotemporal dependencies in fMRI can be modeled end-to-end by splitting the work: a bidirectional selective state-space (Mamba) block handles the time dimension with linear cost, and a transformer block then applies global self-attention over the downsampled volume tokens to capture spatial relations. The paper reports state-of-the-art results on both tasks: for cognitive intelligence, MSE of 0.835 on the 1,075-subject cohort and 0.932 on the 6,000-subject cohort, with Pearson R of 0.41 and 0.24, compared with best-baseline MSEs of 0.914 and 0.994; for sex classification, accuracy of 96.28% on the 1,075-subject cohort and 97.91% on the 6,000-subject cohort, outperforming all baselines on the former and matching the leading prior method on the latter. The authors also claim this is the first hybrid Mamba-transformer approach with such temporal coverage in volumetric resting-state fMRI, that BrainMT is about 35.8% more memory-efficient than the leading 4D transformer with linear complexity in the number of time frames, and that Integrated Gradients maps implicate default-mode and frontoparietal regions for intelligence and temporal/frontal regions for sex.","pith_inferences":["The paper does not decompose its gains into the contribution of the temporal-first scan versus the transformer's global spatial attention; an ablation that isolates scan direction on the 1,075-subject cohort would pin down the mechanism behind the improvement.","Because several correlation-based baselines sit at MSE near 1.0 on the 6,000-subject cohort, their errors are close to predicting the mean; reporting results on the original intelligence scale would show the real-world effect size behind the 6.23% reduction.","An untested prediction is that temporal-first scanning helps most when slow hemodynamic dynamics carry signal; stratifying participants by scan length or low-frequency power and comparing BrainMT with and without the temporal-first scan would test this."],"forward_implications":["If the reported accuracies replicate, fMRI phenotype prediction no longer needs to truncate scans to 10–20 frames; full 200-frame volumes can be fed through a single forward pass.","The reported memory saving and linear-in-time complexity mean larger cohorts, longer scans, or bigger batch sizes become feasible on a single 48 GB GPU, lowering the hardware bar for voxel-based neuroimaging models.","The hybrid recipe—linear-time temporal scan plus global spatial attention—would likely transfer to other 4D medical images such as dynamic PET or contrast-enhanced MRI, where both the time course and the full volume matter.","The Integrated Gradients maps give a concrete, model-derived list of brain regions that can be tested as biomarkers in independent datasets."],"supporting_citations":[{"why":"Supplies the selective state-space model (Mamba) formalism with input-dependent parameters that the temporal scanning block is built on.","marker":"[9]"},{"why":"Defines the bidirectional Vision Mamba block structure (forward/backward SSMs plus gating) that the paper adapts for fMRI tokens.","marker":"[34]"},{"why":"Provides the transformer-centric voxel-based fMRI baseline whose preprocessing and normalization pipeline BrainMT adopts and whose performance it must beat.","marker":"[23]"},{"why":"Provides the leading prior voxel-based transformer baseline, the frame-count finding that motivates using 200 frames, and the GPU-memory comparison target.","marker":"[16]"},{"why":"Supplies the 6,000-subject resting-state fMRI cohort and the fluid-intelligence regression targets.","marker":"[31]"},{"why":"Supplies the 1,075-subject resting-state fMRI cohort and the cognitive-function composite score target.","marker":"[30]"},{"why":"Supplies the Integrated Gradients algorithm used to generate the brain-region attribution maps.","marker":"[32]"}],"fun_headline_variants":["BrainMT: Mamba meets transformer for long-range fMRI modeling","Hybrid Mamba-Transformer for fMRI: beats baselines on two tasks","Mamba for time, transformer for space: hybrid fMRI model excels","Temporal-first Mamba with spatial transformer improves fMRI predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result rests on the transformer actually computing one global attention step over all 200 time frames and all spatial patches at once inside a 48 GB GPU; if the implementation attends only within single volumes or after pooling, the described architecture is not what produced the numbers.","fun_headline_variants_meta":{"raw":{"variants":["BrainMT: Mamba meets transformer for long-range fMRI modeling","Hybrid Mamba-Transformer for fMRI: beats baselines on two tasks","Mamba for time, transformer for space: hybrid fMRI model excels","Temporal-first Mamba with spatial transformer improves fMRI predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002304,"raw_usage":{"total_tokens":8937,"prompt_tokens":1042,"completion_tokens":7895,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":7821}},"tokens_in":658,"tokens_out":7895,"duration_ms":56735,"temperature":1.0,"reasoning_tokens":7821,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:04:00.336752+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released implementation with the stated settings (T = 200, K roughly 220 spatial tokens, L roughly 44,000 tokens, 8 transformer heads, batch size 2 on a 48 GB GPU) and record peak GPU memory and the attention computation's input-output shapes; if the code cannot execute at that sequence length or the reported memory is exceeded, the central claim about a trainable global transformer over 200 frames is not supported as written.","supporting_citations":[{"cited_title":"In: International Conference on Medical Imaging with Deep Learning","cited_arxiv_id":null,"evidence_quote":"Provides the transformer-centric voxel-based fMRI baseline whose preprocessing and normalization pipeline BrainMT adopts and whose performance it must beat."},{"cited_title":"Advances in Neural Information Processing Systems 36, 42015–42037 (2023)","cited_arxiv_id":null,"evidence_quote":"Provides the leading prior voxel-based transformer baseline, the frame-count finding that motivates using 200 frames, and the GPU-memory comparison target."},{"cited_title":"Neuroimage80, 144–168 (2013)","cited_arxiv_id":null,"evidence_quote":"Supplies the 1,075-subject resting-state fMRI cohort and the cognitive-function composite score target."}],"review_version":1}