{"id":"82077848-0db2-4af6-b684-b357f37f68df","arxiv_id":"2607.28858","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"In a controlled comparison on two brain-tumor MRI benchmarks, the Mamba-based SegMambaV2 and SegMamba models achieved the highest segmentation Dice, while 3D U-Net was fastest but least accurate.","lead":"This paper ran five deep-learning segmentation models under identical conditions on two brain-tumor MRI datasets and found that Mamba-style models scored highest on accuracy. It is a useful reference for choosing an architecture when accuracy and speed both matter.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim overreaches: the evaluated SegMamba/SegMambaV2 implementations differ from the published architectures and were trained with different hyperparameters, so the performance gap may reflect these differences rather than SSM architectural superiority.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the benchmark's premise of 'identical conditions' is violated by the modified Mamba implementations and Mamba-specific training hyperparameters. This is not a fatal flaw because the paper is transparent about these deviations and releases code, but it is exactly the kind of confound that prevents attributing the ranking to architectural design. A simple matched-training experiment would settle the issue. The conditional verdict is appropriate; I see no reason to move to ACCEPT or REJECT based on the current evidence.","tokens_in":20899,"tokens_out":2933,"duration_ms":36518,"concrete_test":"Retrain all five models with a unified protocol: use learning rate 1e-4 for all models, no Mamba-only gradient clipping, and SegMamba/SegMambaV2 implementations that include the published FUE, gated multiplication, and HS-Downsampling modules (or the exact official repository code). Recompute Tables 3 and 4. If SegMamba and SegMambaV2 still achieve the top two Dice/IoU on both datasets, the confound is cleared; if another model takes the lead or the margin shrinks, the original conclusion must be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Conclusion attributes the top scores to 'SSM architectures, SegMamba and SegMambaV2,' but the experiments do not actually evaluate the published SegMamba and SegMambaV2 under identical conditions. Section 3.2.4 states that the SegMamba implementation omits the FUE module, replaces the GSC gated multiplication with addition, and uses standard convolutional decoder blocks. Section 3.2.5 states that SegMambaV2 approximates HS-Downsampling with a single convolution and implements ToOM via axis permutation plus 1D Mamba layers. Section 3.3 then gives SegMamba and SegMambaV2 a learning rate of 1e-5 instead of 1e-4, plus gradient clipping, while the other models use 1e-4 without clipping. The central claim presupposes that observed differences are attributable to architecture, but these implementation and optimization differences are plausible alternative explanations. The paper does not provide an ablation, a control with matched hyperparameters, or evidence that the modified components preserve the original design's behavior. Therefore the claim that 'SSM architectures' outperform CNN- and Transformer-based baselines is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a unified benchmark of five 3D brain-tumor segmentation architectures—3D U-Net, SegResNet, Swin UNETR, SegMamba, and SegMambaV2—on BraTS 2023 meningioma and BraTS 2024 post-treatment glioma datasets. All models are trained from scratch with a shared MONAI-based preprocessing, augmentation, and evaluation pipeline under five-fold cross-validation, and are compared in terms of Dice, IoU, HD95, inference time, and parameter count. The main claim, stated in the Conclusion, is that the SSM architectures SegMamba and SegMambaV2 consistently achieve the best global performance on both datasets, outperforming the CNN and Transformer baselines.","tokens_in":21241,"tokens_out":3414,"duration_ms":42320,"significance":"If the central claim were supported, this would be a useful contribution: the paper addresses an important problem (heterogeneous evaluation protocols in medical image segmentation), provides code, evaluates two clinically distinct BraTS tasks, includes efficiency metrics alongside accuracy, and documents class imbalance. The five-fold protocol and public benchmark are strengths. However, the main claim is currently not supported as stated, because the evaluated SegMamba and SegMambaV2 implementations differ from the published architectures and were trained with different optimization hyperparameters from the baselines. The benchmark remains valuable as a reproducible comparison of the specific adapted implementations, provided the conclusions are reframed or the confounds are removed.","major_comments":[{"comment":"The Conclusion attributes the best results to 'SSM architectures, SegMamba and SegMambaV2,' but the experiments do not evaluate the published SegMamba and SegMambaV2. §3.2.4 states that the SegMamba implementation omits the FUE module, replaces GSC gated multiplication with addition, and uses standard convolutional decoder blocks. §3.2.5 states that SegMambaV2 approximates HS-Downsampling with a single convolution and implements ToOM via axis permutation plus 1D Mamba layers. These are material architectural differences, not cosmetic ones. The observed performance gap may reflect these modifications rather than the SSM design. To support the central claim, the authors should either evaluate the original published implementations or explicitly reframe all conclusions as applying to the adapted implementations, ideally with an ablation showing the modifications do not change relative ranki","section":"§3.2.4, §3.2.5, §5"},{"comment":"The abstract states that all architectures are trained using 'identical ... optimization strategies,' but Table 2 shows that SegMamba and SegMambaV2 use a learning rate of 1e-5 instead of 1e-4 and additionally use gradient clipping with max norm 1. This is a direct confound: the two models that win the comparison were given different optimization settings. The paper offers no control experiment with matched learning rate and clipping for all models, nor evidence that the performance gap is insensitive to these choices. At minimum, the claim of identical optimization must be removed and the results re-interpreted; ideally, the authors should run matched-hyperparameter experiments or a sensitivity analysis.","section":"§3.3, Table 2"},{"comment":"The global Dice/IoU/HD95 results are reported as five-fold averages without standard deviations, confidence intervals, or significance tests. Some of the gaps supporting the central claim are small (e.g., 0.7559 vs 0.7533 for SegMambaV2 vs SegMamba on BraTS 2024; 0.7245 vs 0.7116 on BraTS 2023). Without variability or a paired test across folds, the statement that SSM architectures 'consistently achieved' the best performance is stronger than the evidence justifies. Report per-fold results and, if possible, a paired significance test or bootstrap confidence intervals.","section":"Tables 3 and 4"}],"minor_comments":[{"comment":"The second author's email contains a duplicated domain: 'luna.hernandez102@ulpgc.es@ulpgc.es'.","section":"Author affiliation"},{"comment":"The patient identifier is given as 'BraTS-MEN-00891-000' in Figure 1 and 'BraTS-MEN-00891-00' in Figure 4; please use a consistent identifier.","section":"Figure 1 and Figure 4"},{"comment":"Use the model name consistently: 'SegMamba V2' in the table header vs 'SegMambaV2' in the text.","section":"Tables 3 and 4"},{"comment":"The phrase 'the publicly available code differs slightly' understates the listed differences (omitted FUE module, replaced gated multiplication, changed decoder). Suggest describing these as 'substantial differences' or listing them more prominently in a limitations paragraph.","section":"§3.2.4"},{"comment":"The comparison with challenge leaders is clearly labeled as indicative, which is appropriate. However, the table caption 'Best value per column among the models evaluated in this study' is confusing because the table also includes reference methods; consider restructuring the comparison.","section":"§4.5, Tables 7 and 8"}],"recommendation":"major_revision","confidential_remarks":"The benchmark has clear value as a reproducible evaluation protocol, and the code availability is a plus. However, the main architectural claim is confounded by implementation deviations and unequal optimization settings. Since the code is public, the authors could feasibly add matched-hyperparameter controls or ablate the modified components; short of that, the conclusions should be substantially narrowed. I do not think rejection is warranted, because the empirical data are presented in good detail and the confound is fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful, clearly written benchmark of five 3D segmentation models on BraTS 2023 and BraTS 2024 under a shared protocol, with code released and five-fold CV. The protocol itself is the main contribution, not the ranking. The paper does top SegMambaV2 on both datasets, but the conclusion that SSM architectures outperform CNN- and Transformer-based baselines is not supported by its own experimental setup.\n\nWhat the paper does well: same data splits for all models, standardized MONAI preprocessing, Dice/IoU/HD95 plus per-subregion results, efficiency metrics, and a fair caveat when comparing against challenge ensemblers. The authors are transparent about several limitations, which is to their credit. The comparison across two clinically distinct BraTS tasks is genuinely new and practically useful.\n\nThe soft spots are real and load-bearing. The SegMamba implementation omits the FUE module and replaces GSC gated multiplication with addition; SegMambaV2 approximates HS-Downsampling with a single convolution. So the models evaluated are not exactly the published architectures. On top of that, the Mamba models get a 10x lower learning rate and gradient clipping while the others do not. The paper calls the conditions \"identical,\" but they are not. That alone prevents attributing the score gap to architecture. The stress-test note is right: the observed differences could come from implementation details or optimization choices.\n\nAlso, there are no standard deviations or significance tests. Some gaps are small — SegMambaV2 vs SegMamba on BraTS 2024 is 0.7559 vs 0.7533, which is within plausible noise. The NETC results vary a lot across models, but the global ranking is the one the conclusion leans on.\n\nThese issues are fixable. Report per-fold metrics with confidence intervals, justify or align the hyperparameters, and either ablate the modifications or explicitly frame the benchmark as comparing available code implementations under their recommended settings, not architectural families. The paper would then be a solid reference point.\n\nWho is it for: anyone choosing a practical segmentation backbone for brain tumors, or working on benchmark methodology. It deserves a serious referee, but the conclusion needs softening and the uncertainty needs quantification before publication.","headline":"Useful benchmark with an honest protocol, but the headline claim that SSM architectures win outruns the evidence: the Mamba models were modified relative to their published designs and trained with different hyperparameters.","tokens_in":21679,"tokens_out":1749,"would_cite":true,"duration_ms":23874,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Under an identical training and evaluation protocol, the state-space models SegMamba and SegMambaV2 deliver the best global accuracy on both brain tumor segmentation benchmarks.","keywords":["brain tumor segmentation","magnetic resonance imaging","deep learning benchmark","state-space models","Mamba","3D U-Net","Swin UNETR","BraTS"],"falsifier":"Retrain SegMamba and SegMambaV2 with the published FUE module, gated multiplication, and hierarchical-scale downsampling restored, at the same 1e-4 learning rate used for the other models, and compare Dice on BraTS 2023 and BraTS 2024; if the gap narrows or reverses, the claim of architectural superiority is unsupported.","tokens_in":20807,"feed_emoji":"🧠","tokens_out":6370,"duration_ms":65984,"temperature":0.7,"pith_summary":"Five deep-learning architectures — two convolutional, one Transformer, and two state-space — were trained and evaluated under one identical protocol on two brain-tumor MRI benchmarks: meningioma segmentation and post-treatment glioma segmentation. The paper's central claim is that the two state-space models, SegMamba and SegMambaV2, achieve the best global Dice, IoU, and boundary error on both datasets, outperforming the CNN and Transformer baselines in most metrics. Because preprocessing, augmentation, optimization, and inference were identical across models, the reported differences are meant to reflect architectural design rather than experimental noise. The authors also show that each architecture family retains specific strengths, and that convolutional models remain the most efficient for hardware-constrained settings.","feed_headline":"State-space models win two-dataset brain tumor benchmark","feed_subtitle":"SegMamba and SegMambaV2 beat CNN and Transformer baselines in Dice on both BraTS datasets under one protocol.","key_machinery":"The load-bearing instrument is the protocol itself: a fixed pipeline of preprocessing, augmentation, Dice-plus-cross-entropy loss, AdamW optimization, 128-cube cropping, and sliding-window inference, applied unchanged to all five models with five-fold cross-validation. The two state-space models replace self-attention with near-linear state-space sequence modeling: SegMamba uses tri-oriented spatial Mamba blocks, and SegMambaV2 adds hierarchical-scale downsampling and ortho-oriented Mamba. What this mechanism does is isolate, as far as the protocol allows, the architectural family as the independent variable, so performance gaps can be attributed to design rather than to differences in data","core_discovery":"Under a single, deliberately homogeneous training and evaluation protocol — same data splits, preprocessing, augmentation, loss, optimizer, crop size, and sliding-window inference — the paper reports that the state-space architectures SegMamba and SegMambaV2 consistently outperform the convolutional baselines (3D U-Net, SegResNet) and the Transformer baseline (Swin UNETR) on both datasets. On BraTS 2023, SegMambaV2 reaches a global Dice of 0.7245 and SegMamba 0.7116, versus 0.6737–0.6987 for the CNN/Transformer models; on BraTS 2024, SegMambaV2 reaches 0.7559 and SegMamba 0.7533, versus 0.7142–0.7391. The SSMs also post the lowest 95th-percentile Hausdorff distances, indicating better bounda","pith_inferences":["Editorial inference: The comparison with challenge leaderboards suggests the local protocol understates what these architectures can do with full training data; an obvious extension is to rerun the exact benchmark using the complete training sets and official validation annotations, which would test whether the SSM advantage persists when more data is available.","Editorial inference: Because the released SegMamba code omits the FUE module and replaces gated multiplication with addition, and SegMambaV2 approximates its downsampling with a plain convolution, the 'SegMamba family' tested here is really a partially reimplemented variant; a faithful reimplementation could shift the reported gaps.","Editorial inference: The benchmark uses one crop size, one training budget, and a tenfold lower learning rate for the SSM models; a protocol sweep over these choices would reveal whether the ranking is stable under hyperparameter variation.","Editorial inference: The per-sub-region results imply that a hybrid architecture — an SSM backbone with a targeted attention module for sub-regions like ET or SNFH — could combine the best of both families; the paper does not test this, but the data suggest it."],"forward_implications":["If state-space architectures genuinely lead under homogeneous conditions, future 3D medical-segmentation work can treat the SegMamba family as a stronger default than Transformer or CNN baselines for accuracy-critical tasks.","The two-dataset pattern suggests the state-space advantage is not tied to one tumor type or acquisition scenario, since the ranking holds for both compact meningiomas and distorted post-treatment gliomas.","SegMambaV2 is the best high-accuracy choice where inference speed matters, while SegMamba offers a lighter-parameter alternative with nearly the same accuracy.","For hardware-constrained deployment, SegResNet remains the practical mid-point: near-sub-second inference and competitive Dice at a fraction of the parameters of the SSMs and Transformer.","Swin UNETR's per-sub-region wins indicate that Transformer attention may still be preferable when a specific tumor component is the clinical priority, even if global metrics favor SSMs."],"fun_headline_variants":["SSMs beat CNNs, Transformers on two brain tumor sets","State-space models top unified brain MRI benchmark","SegMamba edges U-Net and Swin in fair tumor test","One protocol, two datasets: SSM wins brain tumor Dice","Mamba models lead in standardized tumor segmentation"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The conclusion that state-space architectures are inherently better rests on the assumption that the SegMamba and SegMambaV2 implementations used here faithfully represent those published models and that their altered training recipe — a tenfold lower learning rate and gradient clipping — does not itself cause the performance gap.","fun_headline_variants_meta":{"raw":{"variants":["SSMs beat CNNs, Transformers on two brain tumor sets","State-space models top unified brain MRI benchmark","SegMamba edges U-Net and Swin in fair tumor test","One protocol, two datasets: SSM wins brain tumor Dice","Mamba models lead in standardized tumor segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1211,"prompt_tokens":811,"completion_tokens":400,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":319}},"tokens_in":555,"tokens_out":400,"duration_ms":5061,"temperature":1.0,"reasoning_tokens":319,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T01:27:13.369274+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain SegMamba and SegMambaV2 with the published FUE module, gated multiplication, and hierarchical-scale downsampling restored, at the same 1e-4 learning rate used for the other models, and compare Dice on BraTS 2023 and BraTS 2024; if the gap narrows or reverses, the claim of architectural superiority is unsupported.","supporting_citations":[],"review_version":1}