{"id":"a5f39b32-015b-4324-838e-42902257113c","arxiv_id":"2501.15423","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Adding a multi-stage cross-scale attention module to U-Net improves small stroke lesion segmentation on ATLAS v2.0, with the best ensemble achieving the highest Dice and F1 scores.","lead":"This paper applies a multi-scale attention module to U-Net models for stroke lesion segmentation from MRI scans. On the ATLAS v2.0 dataset, the module improves Dice and F1 scores for small lesions across several training schemes, though the absolute gains are modest and no statistical significance testing is reported.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Small-lesion evaluation may not be held out: the 138-image subset is drawn from the same 655-image CV dataset, and the paper never confirms each prediction came from a model trained without that image, so the central small-lesion gains could be optimistic.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing point: the held-out status of the 138-image small-lesion subset is not explicitly confirmed, and the entire central claim depends on that status. A standard 5-fold cross-validation would normally ensure each image is validated once by a model trained without it, but the paper's language of a 'dedicated testing subset' combined with the self-training scheme's description of training on the original dataset creates real ambiguity. The released code can settle this directly, so the concern is testable rather than fatal. My read does not move the reader's CONDITIONAL verdict: the paper should clarify the evaluation protocol and, if the protocol is clean, add statistical support for the word 'significantly.' The architecture's reliance on the previously published MSCSA module and the lack of modern transformer baselines are secondary; the small-lesion evaluation is the single most load-bearing element because it is the basis of the strongest claim. The code availability is a genuine positive, as it makes the required check feasible.","tokens_in":7008,"tokens_out":6454,"duration_ms":62530,"concrete_test":"Inspect the evaluation script and fold-assignment files in the released repository: for each of the 138 small-lesion IDs, verify that the Table 1 small-lesion Dice/F1 value for a given scheme uses the prediction of the model trained on the other four folds, and never a model whose training set contained that image. If any small-lesion image is scored with a model that saw its label in training, the small-lesion comparison is invalid. If the code confirms clean validation-fold predictions, compute paired 95% confidence intervals or a permutation test on the baseline-vs-MSCSA difference for this subset; if the interval includes zero, soften 'significantly outperforms' to 'outperforms'.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests entirely on the 'small lesion subset' comparison. The Dataset section says the 138 MRIs with lesions under 1,000 voxels were selected from the full ATLAS v2.0 set 'to create a dedicated testing subset,' while the Implementation section and Table 1 caption describe a 5-fold size-balanced cross-validation over the same 655-image set. The paper never states whether the small-lesion metrics are computed only from predictions on the validation fold in which each of these 138 images appears, or from models trained on the full dataset. If the small-lesion metrics come from models that saw these images during training, the reported gains (e.g., Dice +0.001 for DTK10+MSCSA, +0.019 for Ensemble+MSCSA; F1 up to +0.037) would reflect in-sample performance rather than generalization. This is load-bearing because every headline claim about small-lesion improvement is drawn from this subset. A related issue is that the caption's 'significantly outperforms' is not supported by any statistical test, confidence interval, or per-fold standard deviation in the paper.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes applying a plug-in Multi-Stage Cross-Scale Attention (MSCSA) module to U-Net-style stroke lesion segmentation networks. The module is inserted in place of standard skip connections, fusing multi-stage encoder features and using cross-scale attention with relational positional encoding. The authors evaluate MSCSA on the ATLAS v2.0 dataset under five training schemes: a default U-Net, a Top-10 CE loss variant (DTK10), Res U-Net, self-training, and an ensemble of all models. For each scheme they compare baseline versus baseline+MSCSA on the full 655-image cross-validation set and on a 138-image subset of lesions under 1,000 voxels. The reported results show that MSCSA yields small full-dataset differences (Dice changes of approximately -0.005 to +0.003) but consistent improvements on the small-lesion subset for both Dice and F1, with the largest gains for Ensemble+MSCSA (small-lesion Dice 0.458 vs. 0.439, F1 0.574 vs. 0.537). The paper claims that MSCSA significantly outperforms baselines on small lesions while remaining competitive on the full dataset.","tokens_in":7245,"tokens_out":4722,"duration_ms":42185,"significance":"If the small-lesion gains are real and reflect held-out generalization, MSCSA would be a useful plug-in component for improving small-lesion stroke segmentation across a range of U-Net training strategies. The study's strengths are its use of the public ATLAS v2.0 benchmark, evaluation across five training schemes, and release of code. The manuscript also avoids circular evaluation: the ATLAS test labels are not used to fit the models. However, the central claim rests on the small-lesion subset, and the paper does not report any statistical tests, confidence intervals, or per-fold variability, while the full-dataset differences are tiny (0.001-0.003 Dice). The significance of the work is therefore conditional on clarifying the evaluation protocol and demonstrating that the small-lesion improvements are not due to chance or inadvertent in-sample evaluation.","major_comments":[{"comment":"The small-lesion evaluation protocol is ambiguous in a load-bearing way. The Dataset section states that 138 MRIs with lesions under 1,000 voxels were selected from the 655-image ATLAS training set 'to create a dedicated testing subset,' but the Implementation details describe a 5-fold size-balanced cross-validation over the entire 655-image set. The paper never explicitly states whether the small-lesion metrics are computed exclusively from predictions on the validation fold in which each of these 138 images appears (i.e., from a model trained without that image), or whether some or all of the small-lesion predictions come from models that saw these images during training. If the latter is the case, the reported gains (e.g., Dice +0.019 and F1 +0.037 for Ensemble+MSCSA) would reflect in-sample performance rather than generalization. Please state the exact protocol for the small-lesion subset and confirm that every small-lesion prediction was made by a model whose training folds excluded that image; if this is not the case, the small-lesion experiments need to be redone with proper held-out splits.","section":null},{"comment":"The caption asserts that MSCSA 'significantly outperforms' the baselines on the small-lesion subset in all metrics, but the paper provides no statistical test, confidence interval, or per-fold standard deviation to support the word 'significantly.' Given that the full-dataset differences are only 0.001-0.003 Dice and the small-lesion gains are modest, statistical significance cannot be inferred from the point estimates alone. Please report per-fold results and a paired test across the 138 small-lesion cases (or across folds), such as a Wilcoxon signed-rank test or bootstrap confidence intervals, and temper the language if significance is not established.","section":null},{"comment":"The full-dataset comparison for Res U-Net shows that MSCSA decreases Dice from 0.638 to 0.633, yet the text in Section 3 states that 'MSCSA demonstrates competitive and comparable performance with the baselines' without noting this decrease. This is relevant to the claim that MSCSA maintains performance on large lesions without compromise. Please acknowledge this decrease explicitly and discuss whether it is within expected variability, or report per-fold results that put the difference in context.","section":null}],"minor_comments":[{"comment":"The sentence 'As shown in 1, MSCSA surpasses...' should read 'As shown in Table 1.'","section":null},{"comment":"The caption uses 'DEConv' while the text in Section 2 uses 'DWConv'; please use consistent terminology for depth-wise convolution.","section":null},{"comment":"The formulas for h2, h3, w2, w3, d2, d3 use division without explicit floor or integer-division notation; please clarify that these are integer operations, since the equations as written suggest non-integer spatial dimensions.","section":null},{"comment":"The self-training scheme uses an additional 300 unlabeled MRIs from the ATLAS test set to generate pseudo-masks. This should be disclosed as a semi-supervised/transductive element of that training scheme, because it uses data outside the 655-image training set and may affect comparability with the other schemes that do not use unlabeled test-set data.","section":null},{"comment":"The 'Top10 CE loss' (DTK10) is not defined or referenced; please provide a definition or citation so that the loss modification is reproducible.","section":null},{"comment":"The discussion mentions that MS lesions 'typically fall below 1000 mm3,' but the paper's small-lesion threshold is defined in voxels; please clarify the voxel-to-volume relationship or rephrase to avoid mixing units.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is a concise conference-style contribution with a plausible plug-in module and a focused evaluation. The main risk is the small-lesion subset protocol: the phrase 'dedicated testing subset' combined with 5-fold CV creates genuine ambiguity about whether the reported small-lesion metrics are fully held out. If the authors confirm the held-out status and add even a simple paired significance test, the contribution would be acceptable. The lack of statistical testing is common in this venue but is load-bearing here because the full-dataset gains are negligible and the headline claim is about small lesions. I would not reject, but the revision needs to address the protocol question directly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a competent, honest extension of the authors' own MSCSA module from [6] into stroke lesion segmentation. The architecture is not new, but the paper does a systematic job: it tests the plug-in across five training schemes on ATLAS v2.0, uses size-balanced cross-validation, shows per-lesion volume curves, and ships code. The consistent direction of small-lesion Dice/F1 improvements across all schemes is the real substance, and the visualization in Fig. 2 supports the claim qualitatively. Credit where due: this is a legitimate application study, not a repackaging of old results.\n\nThe soft spots are real but not fatal. The biggest one is the ambiguous status of the 138-image small-lesion subset. It is selected from the same 655-image set used for 5-fold cross-validation, and the paper never states explicitly that the small-lesion metrics come only from predictions on validation folds where those images were not seen during training. If they were included in any training fold, the gains (Dice +0.019 to +0.022, F1 up to +0.037) would be in-sample optimism. That needs to be clarified.\n\nSecond, the Table 1 caption says \"significantly outperforms\" but there are no significance tests, confidence intervals, or per-fold standard deviations anywhere. Given the full-dataset differences are 0.001-0.003 Dice, the word \"significantly\" is doing a lot of work. On the full dataset MSCSA is essentially tied with baselines, and in DTK10 it is slightly worse on F1, so the honest summary is \"competitive overall, better on small lesions.\"\n\nThird, the comparison set is entirely U-Net variants without MSCSA. No modern transformer-based segmentor or other attention plug-in is included, so the practical value relative to, say, Swin UNETR is unknown. That is a limit on the paper's scope, not a flaw in what is reported.\n\nThis paper is for researchers working on small-lesion segmentation or evaluating attention plug-ins in the U-Net family. It deserves peer review rather than desk rejection, but the review should condition acceptance on (a) explicit confirmation of the held-out status of the small-lesion subset, (b) per-fold or bootstrapped error bars, and (c) toning down or statistically supporting the word \"significantly.\" If the held-out status checks out, the small-lesion result is a modest but useful contribution.","headline":"Honest, incremental stroke-lesion paper: the existing MSCSA plug-in is shown to give consistent but small gains on small lesions across five U-Net training schemes, and the main thing to verify before trusting the headline is whether the small-lesion subset was truly held out.","tokens_in":7798,"tokens_out":1845,"would_cite":true,"duration_ms":18407,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Multi-stage attention lifts small-lesion Dice across U-Net schemes","keywords":["Stroke lesion segmentation","Multi-Stage Cross-Scale Attention","U-Net","MRI segmentation","Small lesion detection","ATLAS v2.0","Ensemble","Self-training"],"falsifier":"Re-run the 5-fold cross-validation with the 138 small-lesion images strictly excluded from all training folds; if the reported Dice and F1 advantages on that subset shrink or disappear, the small-lesion claim is an artifact of data leakage. Alternatively, evaluate the trained models on an external cohort of small stroke lesions not used in training.","tokens_in":6814,"feed_emoji":"🧠","tokens_out":4222,"duration_ms":32137,"temperature":0.7,"pith_summary":"This paper claims that inserting a Multi-Stage Cross-Scale Attention (MSCSA) module into U-Net skip connections reliably improves segmentation of small stroke lesions from MRI, while leaving performance on large lesions unchanged. Across four training schemes (standard, Top10-loss, Res U-Net, self-training), adding MSCSA raises Dice and F1 on a 138-image small-lesion subset, and the ensemble with MSCSA achieves the best overall scores on both the small subset and the full 655-image dataset. The authors argue that multi-stage feature concatenation and cross-scale attention let the network relate features across scales, which matters because small lesions are the hardest cases and the most relevant for early-stage clinical assessment. If right, the module is a drop-in upgrade for any U-Net-based segmentation pipeline.","feed_headline":"Attention module lifts small-lesion Dice across U-Net variants","feed_subtitle":"Adding cross-scale attention to skip connections improves small stroke lesion segmentation in ATLAS v2.0.","key_machinery":"The Multi-Stage Cross-Scale Attention (MSCSA) block, inserted in place of U-Net's skip connections. It takes encoder feature maps from several stages, resamples them to a common resolution, concatenates them along channels, and runs them through Cross-Scale Attention (CSA) followed by an Intra-Feed-Forward Network (Intra-FFN), with a second CSA+FFN stage. CSA uses Multi-Scale key and value Projection (MSP) to build keys/values at three scales, and Relational Positional Encoding (RPE) via depth-wise convolution to preserve local spatial structure; Intra-FFN splits the 1120-channel multi-stage map into per-stage segments to keep compute manageable. The refined maps are split back to original resolutions and fused with encoder features to feed the decoder.","core_discovery":"The central claim is that MSCSA, a plug-in module that replaces direct skip connections between encoder and decoder, makes U-Net-family models better at detecting small stroke lesions without a trade-off on larger lesions. The paper reports consistent Dice/F1 gains on the small-lesion subset (lesions under 1,000 voxels) across Default, Res U-Net, Self-Training, and Ensemble schemes, with Ensemble+MSCSA scoring highest on both the full dataset and the small subset. The mechanism is described as establishing multi-stage interactions by concatenating encoder feature maps of different resolutions and cross-scale interactions by constructing multi-scale key/value projections in attention, so the network can relate objects of different sizes.","pith_inferences":["The small-lesion subset is carved out of the same 655-image dataset that feeds the 5-fold cross-validation, so the reported gains assume the 138 images never appear in the training fold used to produce their predictions; the paper does not explicitly confirm this held-out separation.","If the held-out assumption holds, the mechanism might transfer to other small-lesion domains such as multiple sclerosis lesion segmentation, which the authors name as a planned extension.","A natural test is to measure whether the gain concentrates on the smallest lesions in the subset or is spread uniformly, since cross-scale attention should most help tiny structures.","The cost of the module (extra parameters and compute from the 1120-channel multi-stage map) is not reported; the practical case for adoption would need that trade-off quantified."],"forward_implications":["Small lesion Dice and F1 improve across all four training schemes when MSCSA is added, with the largest absolute gains on the self-training scheme.","Ensemble+MSCSA achieves the highest Dice and F1 scores on both the full ATLAS v2.0 dataset and the small-lesion subset, making it the strongest configuration tested.","Because the module works with Default, DTK10, Res U-Net, and Self-Training schemes, the benefit does not depend on one specific training recipe.","The reported gains suggest a plug-in route to better small-lesion segmentation without redesigning the whole architecture."],"supporting_citations":[{"why":"U-Net is the base architecture whose skip connections the MSCSA module modifies.","marker":"[1]"},{"why":"The original MSCSA paper supplies the module design that this work adapts to stroke lesion segmentation.","marker":"[6]"},{"why":"ATLAS v2.0 provides the 655 T1-weighted MRIs and lesion masks used for all training and evaluation.","marker":"[7]"},{"why":"MAPPING supplies the self-training scheme, the Top10 CE loss baseline, and the 5-fold size-balanced cross-validation approach.","marker":"[8]"},{"why":"nnU-Net is the self-configuring framework used to build the U-Net models and training pipeline.","marker":"[9]"},{"why":"The depth-wise convolution used as relational positional encoding in CSA is inspired by this multi-scale high-resolution vision transformer.","marker":"[10]"}],"fun_headline_variants":["Cross-scale attention lifts Dice for tiny stroke lesions","MSCSA attention sharpens small lesion boundaries in U-Net","Attention module improves small stroke lesion detection","Ensemble with cross-scale attention wins on small lesions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The gains on the small-lesion subset depend on the 138 small-lesion images never being used in the training fold that produces each prediction, and the paper does not explicitly confirm that held-out separation.","fun_headline_variants_meta":{"raw":{"variants":["Cross-scale attention lifts Dice for tiny stroke lesions","MSCSA attention sharpens small lesion boundaries in U-Net","Attention module improves small stroke lesion detection","Ensemble with cross-scale attention wins on small lesions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000172,"raw_usage":{"total_tokens":1269,"prompt_tokens":930,"completion_tokens":339,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":277}},"tokens_in":546,"tokens_out":339,"duration_ms":3729,"temperature":1.0,"reasoning_tokens":277,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:18:17.615505+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 5-fold cross-validation with the 138 small-lesion images strictly excluded from all training folds; if the reported Dice and F1 advantages on that subset shrink or disappear, the small-lesion claim is an artifact of data leakage. Alternatively, evaluate the trained models on an external cohort of small stroke lesions not used in training.","supporting_citations":[{"cited_title":"Stroke Lesion Segmentation using Multi-Stage Cross-Scale Attention","cited_arxiv_id":"2501.15423","evidence_quote":"U-Net is the base architecture whose skip connections the MSCSA module modifies."},{"cited_title":"Ethical approval was not required as con- firmed by the license attached with the open access data","cited_arxiv_id":null,"evidence_quote":"The original MSCSA paper supplies the module design that this work adapts to stroke lesion segmentation."},{"cited_title":"U-net: Convolutional networks for biomedical im- age segmentation,","cited_arxiv_id":null,"evidence_quote":"ATLAS v2.0 provides the 655 T1-weighted MRIs and lesion masks used for all training and evaluation."}],"review_version":1}