{"id":"7e56254b-cb6c-47f6-955c-fa95f2e3355b","arxiv_id":"2507.13384","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A benchmark of 21 patch scan orders shows that contiguous raster scans significantly outperform diagonal scans for Vision Mamba MRI segmentation, with up to 27 Dice points difference.","lead":"This paper tests whether the order in which a Vision Mamba model reads image patches changes its MRI segmentation accuracy, and finds that simple horizontal or vertical scans beat diagonal ones by large margins. It offers a cheap way to improve Mamba-based medical image models by just changing the scan path, without extra parameters or compute.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim rests on single-run Dice and a 3-block Friedman test, so the reported 27-point gap and p=0.0016 do not yet establish that scan order, rather than seed noise, drives the ranking.","rationale":"The reader's weakest assumption is exactly the load-bearing issue: single-run Dice cannot separate scan-order effects from seed noise, and the 3-block Friedman p-value is not a reliable significance statement. My read does not move the verdict because the effect sizes are large and the pattern is broadly sensible, so the appropriate outcome is not rejection but conditional acceptance subject to multi-seed re-runs and a properly calibrated significance test. I found no additional internal inconsistency in the MS2D design or the subject-level splits; the 'consistently outperform' language does overstate exceptions such as Exp20 versus Exp1, but that is a framing issue rather than a collapse of the central claim. A five-seed rerun on ISLES would directly test the decisive robustness question, and if the gap survives, the central claim is substantially strengthened.","tokens_in":10792,"tokens_out":15270,"duration_ms":179028,"concrete_test":"Rerun the 21 scan strategies on ISLES 2022 (the dataset driving the 27-point claim) with at least 5 fixed random seeds per strategy, keeping all other hyperparameters identical, and report mean and standard deviation of test Dice for each strategy. If the best-vs-worst gap (currently 0.264) is not at least two pooled seed standard deviations, or if paired bootstrap intervals across seeds overlap for the top raster and bottom diagonal strategies, then the ranking and the 'critical hyperparameter' conclusion are not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on Table II, where each of the 21 scan strategies on each of the 3 datasets is a single training run. There are no repeated seeds, confidence intervals, or paired tests across runs, so every reported number, including the headline 27-point ISLES spread (Exp3 0.815 vs Exp7 0.551), is one realization of training noise. The Friedman test in Section V-D (chi-square(20)=43.9, p=0.0016) is computed on a 3-by-21 rank matrix; with only three blocks the asymptotic chi-square approximation is not reliable and the test has no within-cell variance against which to judge the scan-order effect. Section III-D's statement that variation is 'solely' due to scan order is therefore not supported by the design. The qualitative pattern may survive a seed sweep, but the paper currently overstates certainty. The consistency claim is also weaker than stated: Exp20 (all diagonals) beats Exp1 on ISLES (0.754 vs 0.643) and nearly ties on LGG, so 'consistently outperform' is too strong for the table.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that the order in which a Vision Mamba model serializes image patches into a 1D sequence is a critical, often overlooked hyperparameter for brain MRI segmentation. The authors introduce MS2D, a drop-in replacement for VMamba's SS2D module that exposes twelve primitive scan paths while keeping the parameter count and FLOPs fixed, and use it to benchmark 21 scan strategies (twelve single-direction, six bidirectional, three four-direction mixes) on BraTS 2020, ISLES 2022, and LGG. They report Dice-score spreads of up to 27 points, a Friedman test that is claimed to show statistical significance (chi-square(20)=43.9, p=0.0016), and a qualitative finding that spatially contiguous raster scans outperform disjointed diagonal scans. The paper concludes with practical guidance to treat scan order as a free hyperparameter and to prefer contiguous scans.","tokens_in":10916,"tokens_out":3316,"duration_ms":42555,"significance":"If the central claim were fully supported, the paper would offer a simple, zero-cost design rule for Mamba-based medical image segmentation and a reusable module for exploring scan orders. The benchmark design has real strengths: a fixed architecture and fixed computational budget across all comparisons, three public datasets covering different pathologies and imaging modalities, more than 70,000 slices, and an attempt at nonparametric significance testing. The recommendation of a shortlist of scan orders is directly actionable for practitioners. However, the current evidence is not yet sufficient to establish the headline claims, because the entire comparison rests on a single run per configuration and a statistical test computed on only three blocks. The work is a useful empirical contribution, but it needs additional experimental and statistical support before the central claims can be accepted.","major_comments":[{"comment":"Every number in Table II, including the headline 27-point ISLES spread (Exp 19 at 0.820 versus Exp 7 at 0.551), comes from a single training run per configuration. There are no repeated seeds, no confidence intervals, and no measure of run-to-run variability. Since deep segmentation training is stochastic, seed noise could easily be on the order of several Dice points, which would change the ranking of the 21 strategies and the reported gaps. The manuscript needs at least three to five seeds per configuration, with mean and standard deviation reported, and paired statistical comparisons across seeds. This is load-bearing because the central claim that scan order drives performance differences depends on the ranking being reproducible.","section":"§III-D and Table II"},{"comment":"The Friedman test is computed on a 3-by-21 matrix (three datasets as blocks, 21 strategies as treatments). With only three blocks, the asymptotic chi-square approximation used to obtain p=0.0016 is not reliable, and the test has no within-cell replication with which to separate scan-order effects from training noise. An exact Friedman test or a permutation-based version should be used, and ideally the analysis should be repeated across multiple seeds so that within-condition variance can be estimated. The current p-value alone does not establish that scan order is a statistically significant factor.","section":"§V-D"},{"comment":"The paragraph at the end of Section III-D states that 'any performance variation reported in Section V can be attributed solely to the ordering of patches.' This is not supported by the design. While the training hyperparameters are held fixed, random weight initialization, data shuffling, and other sources of stochasticity mean that observed differences conflate scan order with seed effects. The word 'solely' overstates what a single-run comparison can establish; the limitation is acknowledged only in Section VI-C as a future protocol improvement, but it should be addressed before the main claim is made.","section":"§III-D"},{"comment":"The claim that spatially contiguous rasters 'consistently outperform' disjointed diagonal scans is too strong given Table II. Experiment 20, which uses all four diagonal scans, achieves 0.754 Dice on ISLES 2022, beating Experiment 1 (0.643) and several raster-based experiments, and its LGG result (0.666) is close to Experiment 1 (0.674). The qualitative pattern is plausible, but the paper should state it as a tendency with exceptions rather than a consistent ordering, or it should test the specific pairwise contrast statistically with repeated runs.","section":"§V-B and §V-C"}],"minor_comments":[{"comment":"Experiment 13 is described as a 'bidirectional horizontal scan,' but the experiment combines S1 (left-to-right) and S2 (top-to-bottom), which are a horizontal and a vertical raster, not two horizontal directions. This mislabeling should be corrected.","section":"§V-B"},{"comment":"The paper calls itself the 'first systematic study' of scan order, but it cites Zhu et al. [10], who ablated 22 scan orders on aerial imagery. The novelty claim should be qualified as the first systematic study for medical imaging or for brain MRI specifically.","section":"Abstract and §II-B"},{"comment":"The chi-square value appears as 43.9 in the abstract and as 43.86 in Section V-D. This is not a substantive issue, but the numbers should be made consistent, and the degrees of freedom and exact test variant should be stated clearly.","section":"§IV and §V-D"},{"comment":"The contribution list promises 'enabling reproducible study,' but no code repository, configuration files, or seed values are provided. Releasing the implementation and the exact training configurations would substantially strengthen the reproducibility claim.","section":"Overall"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical ablation-style benchmark, which is within scope for an applied imaging journal. I do not see a novelty-disclosure concern; the related work is cited. The main risk is that the headline statistical and performance claims are built on a single-run, three-block design; the authors should be asked to rerun with multiple seeds and a more appropriate significance test before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper asks a question worth asking: does the order in which a Vision Mamba serializes image patches matter for brain-MRI segmentation? The authors run a well-scoped benchmark of 21 scan strategies across three public datasets, introduce a clean parameter-free module (MS2D) to vary the scan, and report a wide performance spread, up to 27 Dice points on ISLES. If the effect is real, the practical guidance—prefer simple contiguous rasters over diagonal paths—is useful for a growing niche.\n\nThe novelty is genuine: Zhu et al. studied scan order on remote sensing, but not on brain MRI, and the anatomy-based motivation (directional white-matter structure) is reasonable. The paper is clearly written, and the limitations section honestly acknowledges that a single architecture and fixed hyperparameters are a constraint.\n\nThat said, the central claim outruns the evidence. Every number in Table II comes from a single training run. There are no repeated seeds, no confidence intervals, and no paired analysis, so the 27-point gap and the rank ordering of the 21 strategies could change if you rerolled the random seed. The Friedman test is computed on a 3x21 matrix; with only three blocks, the asymptotic chi-square approximation is not reliable, and the test also has no within-cell variance to compare against. The phrase \"variation can be attributed solely to the ordering\" (Section III-D) is therefore not defensible—training noise is uncontrolled.\n\nThe consistency claim is also weaker than stated. Exp20 (all diagonals) beats Exp1 (left-to-right raster) on ISLES (0.754 vs 0.643) and nearly ties on LGG, so \"diagonals consistently underperform\" is too strong. The general pattern may well hold, but the table shows a messier picture than the abstract implies. Finally, no code or data is provided, which hurts reproducibility.\n\nThese problems are fixable. Running a handful of seeds per configuration, reporting means and standard deviations, and using a more appropriate statistical test would strengthen the paper a lot. I'd expect the qualitative finding to survive such a check, but that's a guess, not what the paper currently shows.\n\nWho is this for? Researchers building Mamba-based medical segmentation models. It deserves a serious referee because the question is timely and the benchmark is well designed, but I would ask for major revision before acceptance: multi-seed experiments, an honest statistical treatment, and a release of the code/data.","headline":"Well-scoped benchmark with a plausible central claim, but single-run results and a three-block Friedman test don't yet support the strong quantitative conclusions.","tokens_in":11510,"tokens_out":2845,"would_cite":false,"duration_ms":32807,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that the order in which image patches are flattened into a 1D sequence shifts Mamba-based MRI segmentation Dice by up to 27 points, and that simple raster scans win.","keywords":["Vision Mamba","patch scan order","MRI segmentation","state-space models","spatial locality","MS2D","Friedman test","brain MRI"],"falsifier":"Run the 21 scan strategies with, say, five random seeds on the same three datasets and compare seed-averaged Dice scores; if the best-minus-worst gap shrinks below the reported range or the top-ranked scans (the horizontal raster and the orthogonal mix) are no longer consistently on top, the claim that scan order is the decisive factor would be refuted. A second check is to apply the identical protocol to a natural-image segmentation dataset: if contiguous rasters do not outperform diagonals there, the brain-MRI specificity of the claim is unsupported.","tokens_in":10538,"feed_emoji":"🧠","tokens_out":5827,"duration_ms":61313,"temperature":0.7,"pith_summary":"This paper argues that the order in which a 2D image is flattened into the 1D patch sequence fed to a Mamba-based vision model is a critical hyperparameter for brain-MRI segmentation, not a neutral implementation detail. Across 21 scan strategies on three public datasets covering more than 70,000 slices, test Dice scores varied by up to 27 points depending only on the scan path, with a Friedman test giving $\\chi^2_{20}=43.9$ and $p=0.0016$. Spatially contiguous raster scans, especially a single horizontal pass and a four-direction mix of horizontal and vertical rasters, consistently ranked highest, while disjointed diagonal scans ranked lowest. The paper introduces MS2D, a parameter-free module that exposes twelve primitive scan paths with no change in FLOPs or parameter count, and concludes that scan order should be treated as a free hyperparameter in state-space vision models.","feed_headline":"Patch order alone swings MRI Dice by 27 points","feed_subtitle":"In Mamba-based vision segmentation, choosing a simple raster over a diagonal scan is a free accuracy boost.","key_machinery":"The load-bearing object is MS2D (Multi-Scan 2D), a replacement for the standard four-direction scan unit in a VMamba-based UNet backbone. MS2D defines a bank of twelve primitive permutations of the $32\\times32$ patch grid—two row rasters, two column rasters, four main diagonals, and four serpentine traversals—and always runs exactly four parallel Mamba streams, duplicating the user-chosen scan IDs so that the FLOPs and parameter count are identical to the baseline. This makes scan order an isolated experimental variable. The statistical machinery is the Friedman test applied to the Dice-score matrix of three datasets by twenty-one scan strategies.","core_discovery":"The central claim is that the serialization order of image patches into a 1D sequence is a statistically significant and practically consequential design choice for Mamba-based MRI segmentation, and that spatially coherent rasters systematically beat disjointed diagonals. The author's evidence is the 21-experiment benchmark: the right-to-left horizontal scan (Experiment 3) and the orthogonal four-direction mix (Experiment 19) obtain the best Dice scores across datasets, with Experiment 19 reaching 0.820 on ISLES 2022 and 0.746 on LGG, while diagonal scans such as Experiment 7 fall to 0.551 on ISLES 2022. The Friedman test on the three-dataset-by-21-strategy matrix yields $\\chi^2_{20}=43.9$ with $p=0.0016$, rejecting the null that all scan orders perform equally. The paper further claims that because MS2D keeps parameter count and runtime fixed, any observed difference can be attributed solely to the scan order.","pith_inferences":["If the ranking is robust to random-seed variation, the 27-point gap implies that scan order can be cheaply exploited as a data-dependent prior, and suggests that future Mamba variants could learn per-image scan schedules.","The paper's contrast with a prior remote-sensing study, where a single left-to-right scan sufficed, implies that scan-order sensitivity is tied to directional anatomical structure; testing the same 21 strategies on natural images or on CT would directly test this domain-dependence.","A testable extension is to measure the effective receptive field or sequence-level coherence of each scan to predict which scan will win before training, rather than benchmarking all 21.","Because the paper uses a single run per configuration, the exact ranking of middle-tier scans is uncertain; cross-validation with repeated seeds could shrink the gap and reorder scans below the top tier."],"forward_implications":["Practitioners deploying Mamba-based vision models for brain MRI should screen simple horizontal and vertical raster scans before any other configuration, since they deliver near-optimal accuracy at zero extra compute.","Scan order should be added to the standard hyperparameter search for state-space vision models, on the same footing as learning rate or patch size.","Diagonal and disjointed serpentine scans should be avoided for segmentation of contiguous anatomical lesions, where they can degrade Dice by tens of points.","The MS2D module allows future work to explore adaptive or learned scan selection without changing the model's cost profile.","The preference for contiguous scans may extend to other anisotropic medical modalities and high-resolution imaging tasks, though this paper only tests brain MRI."],"supporting_citations":[{"why":"Defines the Mamba selective state-space model whose sequence processing makes scan order a design variable.","marker":"[6]"},{"why":"Introduces the four-direction selective scan (SS2D) that MS2D extends to twelve paths.","marker":"[8]"},{"why":"Provides the VM-UNet backbone into which MS2D is plugged, fixing parameter count and compute.","marker":"[19]"},{"why":"The remote-sensing scan-order ablation whose single-raster conclusion this paper challenges and extends to brain MRI.","marker":"[10]"},{"why":"Establishes Vision Mamba's bidirectional serialization of image patches, the starting point for studying scan directions.","marker":"[7]"},{"why":"ISLES 2022 dataset, one of the three benchmark sources for stroke-lesion segmentation.","marker":"[13]"},{"why":"BraTS benchmark reference covering the glioma datasets used in the evaluation.","marker":"[14]"},{"why":"TCGA-LGG dataset providing the low-grade-glioma benchmark.","marker":"[17]"}],"fun_headline_variants":["Scan order swings MRI Dice by 27 points","In Mamba vision, patch sequence order is a free accuracy lever","Forget architecture: patch order alone shifts MRI Dice 27 pts","MRI segmentation: simple rasters beat diagonal scans by 27 Dice","Study: Patch scan order is a powerful free hyperparameter for Mamba"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the single Dice score measured per scan per dataset is a reliable estimate of that scan's true performance, so that the Friedman test on the three-dataset-by-21-strategy matrix, computed with only three blocks, and the reported 27-point gap are not artefacts of training noise.","fun_headline_variants_meta":{"raw":{"variants":["Scan order swings MRI Dice by 27 points","In Mamba vision, patch sequence order is a free accuracy lever","Forget architecture: patch order alone shifts MRI Dice 27 pts","MRI segmentation: simple rasters beat diagonal scans by 27 Dice","Study: Patch scan order is a powerful free hyperparameter for Mamba"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000221,"raw_usage":{"total_tokens":1467,"prompt_tokens":976,"completion_tokens":491,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":403}},"tokens_in":592,"tokens_out":491,"duration_ms":5524,"temperature":1.0,"reasoning_tokens":403,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:02:33.039462+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the 21 scan strategies with, say, five random seeds on the same three datasets and compare seed-averaged Dice scores; if the best-minus-worst gap shrinks below the reported range or the top-ranked scans (the horizontal raster and the orthogonal mix) are no longer consistently on top, the claim that scan order is the decisive factor would be refuted. A second check is to apply the identical protocol to a natural-image segmentation dataset: if contiguous rasters do not outperform diagonals there, the brain-MRI specificity of the claim is unsupported.","supporting_citations":[{"cited_title":"Rethinking Scanning Strategies with Vision Mamba in Semantic Segmentation of Remote Sensing Imagery: An Experimental Study","cited_arxiv_id":"2405.08493","evidence_quote":"The remote-sensing scan-order ablation whose single-raster conclusion this paper challenges and extends to brain MRI."},{"cited_title":"ISLES 2022: A multi-center mri stroke-lesion segmentation dataset,","cited_arxiv_id":null,"evidence_quote":"ISLES 2022 dataset, one of the three benchmark sources for stroke-lesion segmentation."},{"cited_title":"The multimodal brain tumor image segmentation benchmark (brats),","cited_arxiv_id":null,"evidence_quote":"BraTS benchmark reference covering the glioma datasets used in the evaluation."},{"cited_title":"The cancer genome atlas low grade glioma collection (tcga-lgg) (version 3) [data set],","cited_arxiv_id":null,"evidence_quote":"TCGA-LGG dataset providing the low-grade-glioma benchmark."}],"review_version":1}