{"id":"de8b6b8c-a052-4b83-beef-df77eaaf17c8","arxiv_id":"1908.04840","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A residual-network variant with boundary weighted loss and adversarial training achieved Dice 0.881 for penumbra and 0.877 for core on the SPES-ISLES 2015 dataset, but the gains are not statistically validated.","lead":"This paper tests whether adding residual connections and a boundary-weighted loss to an adversarial segmentation network improves stroke lesion delineation on the SPES-ISLES 2015 dataset. The authors report average Dice coefficients of 0.881 for penumbra and 0.877 for core, attributing the gains to these modifications.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Boundary-loss attribution is confounded: every BD addition is paired with Lovász-Softmax, so the claimed 'boundary weighted loss' improvement is unsupported.","rationale":"Reader's verdict is CONDITIONAL and I agree with that judgement. The residual-connection comparisons are internally consistent: BL5 vs BL1 and BL7 vs BL3 show gains when only residual connections are added. However, the boundary-loss contribution has a more basic validity problem than missing significance tests: the experimental design never separates BD from LS. The improvements labelled as boundary-loss effects in the text are actually joint effects of {LS, BD}. This matters because LS is an established loss for improving Dice, so the result cannot discriminate between the proposed boundary-weighting mechanism and a standard loss change. The reader's weakest_assumption mentions 'no ablation or significance testing' but does not identify the specific confound with LS; hence partial agreement. A targeted 2x2 ablation would settle the attribution. If the ablation shows BD-only adds nothing, the paper's title claim is not supported; if BD-only does add, the concern dissolves. Either way the verdict remains conditional pending that evidence.","tokens_in":2933,"tokens_out":6522,"duration_ms":64597,"concrete_test":"On the Residual-SUMNet adversarial pipeline, train four conditions on the same SPES three-fold split: (1) CE only (reproduce BL7), (2) CE+BD only (no LS), (3) CE+LS only (no BD), and (4) CE+LS+BD (Proposed). Keep all other hyperparameters identical, including the 3-pixel boundary and weight 10. If condition (2) does not improve over BL7 while condition (3) does, the boundary weighted loss is not the source of the reported gains. Per-fold Dice and a paired test should be reported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of the paper is that residual connections and a boundary weighted loss (BD) improve ischaemic stroke segmentation. The residual half is cleanly tested (BL5 vs BL1; BL7 vs BL3). The boundary-loss half is not. In Table 1, BD is never added alone: every condition that includes BD also includes the Lovász-Softmax (LS) loss. BL2 vs BL1 adds {LS, BD}; BL4 vs BL3 adds {LS, BD}; BL6 vs BL5 adds {LS, BD}; Proposed vs BL7 adds {LS, BD}. No CE+BD-only or CE+LS-only condition is reported. Since LS is a known overlap-maximizing loss, the Dice gains attributed to BD in Section 3 ('the weighted boundary loss improves the segmentation along the boundary') could be entirely due to LS. The claim that the boundary weighting is 'significant' is therefore an untested attribution, not an experimental result. The absence of error bars or significance tests on a three-fold CV of the small SPES subset further weakens the strength of the conclusion, but the BD/LS confound is the more immediate internal flaw.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends SUMNet, an adversarially trained network for ischaemic stroke lesion segmentation, by adding residual connections in the encoder and a morphological boundary-weighted loss (BD) combined with cross-entropy (CE), Lovász-Softmax (LS), and adversarial losses. The authors report average Dice coefficients of 0.881 for penumbra and 0.877 for core on the SPES-ISLES 2015 dataset, and attribute the improvement to residual connections and the boundary-weighted loss. The evaluation uses three-fold cross-validation with eight baselines (BL1–BL7 and Proposed) that progressively add residual connections, adversarial training, and the LS+BD losses.","tokens_in":3179,"tokens_out":2967,"duration_ms":30209,"significance":"If the claimed improvements are reproducible and correctly attributed, the work would offer a practical recipe for improving stroke lesion delineation, which is clinically important for treatment decisions. The paper's ablation structure is a strength for the residual-connection component: the comparisons BL5 vs BL1 and BL7 vs BL3 isolate the effect of residual connections cleanly, and the reported core Dice gains (0.792 to 0.867 and 0.730 to 0.865) are substantial. However, the paper's central claim that the boundary-weighted loss 'significantly' improves segmentation is not supported by the experiments as reported, because BD is never added independently of LS. The absence of error bars or significance tests further weakens the strength of the conclusions. With additional targeted ablations and statistical reporting, the contribution could become solid; in its current form, the attribution of the gains to BD specifically is not established.","major_comments":[{"comment":"The boundary-weighted loss (BD) is confounded with the Lovász-Softmax (LS) loss. In every comparison that the authors cite as evidence for the BD contribution (BL2 vs BL1, BL4 vs BL3, BL6 vs BL5, and Proposed vs BL7), both LS and BD are added together. There is no condition that adds BD alone to CE, nor a condition that adds LS alone. Since LS is itself a loss designed to maximize overlap (Dice/Jaccard), the observed improvements in Table 1 could be entirely due to LS rather than the boundary weighting. The text in Section 3 ('the weighted boundary loss improves the segmentation along the boundary') is therefore an untested attribution. The authors should add, at minimum, a CE+BD-only condition and ideally a CE+LS-only condition for the same network settings to separate the two effects.","section":"Section 2 and Table 1"},{"comment":"The paper uses the word 'significantly' repeatedly (abstract, Section 3, and conclusion) to describe improvements, but reports only average Dice coefficients over three folds with no standard deviations, no confidence intervals, and no statistical test. Several of the differences are small (e.g., penumbra 0.835 to 0.838 in BL1 vs BL2 and core 0.792 to 0.802), and without variance estimates or a paired test, these differences cannot be claimed as significant. This is a load-bearing issue because the central claim rests on the magnitude and reliability of these gains. The authors should provide per-fold results, error bars, and an appropriate significance test (e.g., paired t-test or Wilcoxon signed-rank test across folds or patients).","section":"Section 3 and Table 1"},{"comment":"The evaluation is limited to the SPES subset of ISLES 2015 with internal three-fold cross-validation only. The paper does not report results on the full ISLES test set or on any external dataset, and it does not compare against previously published ISLES results. Because the reported numbers are averages over a small, single-site dataset, the generalizability of the claimed improvement to the broader population of ischaemic stroke scans remains unsubstantiated. A comparison with published state-of-the-art methods or at least an external validation split would strengthen the conclusion.","section":"Section 3 and Table 1"}],"minor_comments":[{"comment":"There are several typographical and grammatical errors: 'the the accurate' in the abstract, and 'it's effectiveness' in the conclusion should be 'its effectiveness'. These should be corrected.","section":"Abstract and Conclusion"},{"comment":"The boundary extraction parameters (3-pixel thickness, 3x3 structuring element, weight 10) are presented without justification or ablation. Since the paper's central claim concerns the boundary-weighted loss, an ablation over boundary thickness and weight would clarify the sensitivity of the results to these hyperparameters.","section":"Section 2"},{"comment":"Training details are missing: the paper does not specify the optimizer, learning rate, number of epochs, batch size, or the weighting coefficients used to combine the CE, adversarial, LS, and BD losses. Without these, the experiments are not fully reproducible.","section":"Section 2 and Table 1"},{"comment":"The qualitative figure shows only selected baseline results (BL3, BL4, BL7, Proposed). It would be helpful to show the full set of baselines, or at least BL1 and BL5, so that the claimed improvement from residual connections is visually evident.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper is an extended abstract, and the residual-connection ablation is cleanly designed. The main problem is the LS/BD confound, which is fixable by adding a few more training runs. The paper would also benefit substantially from per-fold error bars and significance testing. I do not think rejection is warranted because the residual component and the overall approach show promise, and the authors can address the confound within the scope of a revision. The self-citation to Sathish et al. is appropriate given that the method builds directly on that work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a decent ablation of residual connections for stroke lesion segmentation, but the paper's headline claim about the boundary weighted loss is not supported by the table. The residual half is tested cleanly; the boundary half is confounded with Lovász-Softmax in every condition.\n\nWhat is actually new: applying the authors' adversarial training to a residual SUMNet with a boundary-weighted CE term on the SPES-ISLES 2015 dataset. The residual ablation is well designed: BL5 vs BL1 and BL7 vs BL3 show gains, especially for core (0.792 to 0.867). That is a real, if incremental, result. The qualitative figure is helpful.\n\nThe soft spot: the boundary loss is never added alone. BL2 vs BL1 adds {LS, BD}; BL4 vs BL3, BL6 vs BL5, and Proposed vs BL7 all add both. There is no CE+BD-only or CE+LS-only condition. So when the paper says 'the weighted boundary loss improves the segmentation along the boundary,' it is guessing; the gains could be entirely from Lovász-Softmax, which is a known overlap-maximizing loss. The text sometimes treats 'LS and BD' as a package, but the conclusion singles out boundary loss. That is a confound, not a minor omission. The three-fold CV with no error bars or significance tests is also weak, and the boundary thickness (3 px) and weight (10) are not ablated. These are smaller issues for an extended abstract, but they do not help.\n\nThe citation pattern is honest; the self-citation to the authors' adversarial training is legitimate, and the other methods are standard and cited.\n\nWho is this for? Someone building on ISLES 2015 baselines might want the table, but they should not take the boundary-loss attribution at face value. For a methods claim, this needs more work. A serious editor could send it to review because the residual half is sound and the boundary half is fixable with a few extra rows. I would not desk-reject it. But I would ask for the missing ablation conditions and significance testing before accepting any of the boundary-loss language.","headline":"Residual connections get a clean test; the boundary-loss claim is confounded by Lovász-Softmax in every BD condition.","tokens_in":3718,"tokens_out":2819,"would_cite":false,"duration_ms":27912,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding residual connections and a boundary-weighted loss to adversarial training lifts ischaemic stroke lesion Dice to 0.881 penumbra and 0.877 core.","keywords":["ischaemic stroke","lesion segmentation","residual learning","boundary weighted loss","adversarial training","Dice coefficient","SPES-ISLES 2015","multi-sequence MRI"],"falsifier":"Run Residual-SUMNet with the proposed losses on the full ISLES 2015 training set or a held-out test set with the same boundary parameters; if the Dice advantage over BL7 shrinks to within fold-to-fold variability, or if reducing the boundary thickness erases the gain, the central claim that boundary weighting drives the improvement would fail.","tokens_in":2728,"feed_emoji":"🧠","tokens_out":5402,"duration_ms":46285,"temperature":0.7,"pith_summary":"Accurate delineation of the ischaemic stroke lesion's core and salvageable penumbra matters because thrombolytic treatment decisions depend on their extent. This paper argues that two targeted changes to a deep segmentation network—adding residual connections to the encoder and weighting a 3-pixel boundary band by a factor of 10 in the loss—substantially improve that delineation. On the SPES-ISLES 2015 dataset, the resulting adversarially trained Residual-SUMNet reaches average Dice coefficients of 0.881 for penumbra and 0.877 for core, beating all seven baseline configurations. The largest single gain comes from residual connections for the smaller core region, which suggests the mechanism is preservation of small-structure information through the depth of the network.","feed_headline":"Residual links and boundary loss lift stroke lesion Dice to 0.88","feed_subtitle":"Adding residual skips and a boundary-weighted loss to adversarial training sharpens core and penumbra maps on ISLES 2015.","key_machinery":"The central machinery is Residual-SUMNet: a fully convolutional segmentation network with a VGG11-like encoder modified so that a residual connection is added after each convolutional block, plus three relativistic discriminators operating on the predicted core, the predicted penumbra, and the pair. Training combines four losses: cross-entropy, Lovasz-Softmax, the adversarial loss from the discriminators, and a boundary-weighted negative log-likelihood in which a 3-pixel-thick boundary band extracted from the ground truth by subtracting the eroded label from the dilated label is weighted by factor 10. The residual connections carry multi-sequence information forward through the network depth, while the boundary term concentrates learning on the lesion edge; together they are what the paper credits for the measured Dice gains.","core_discovery":"The paper claims that residual connections and a boundary-weighted loss are not minor tweaks but decisive components of ischaemic stroke lesion segmentation. Residual-SUMNet, trained from scratch without ImageNet weights on three MRI sequences (TMax, TTP, DWI), with three relativistic discriminators and a combined loss of cross-entropy, Lovasz-Softmax, adversarial, and boundary-weighted negative log-likelihood, achieves average Dice of 0.881 for penumbra and 0.877 for core on SPES-ISLES 2015. The reported comparisons show residual connections raising core Dice from 0.792 to 0.867 between the non-residual and residual CE-only baselines, and the full proposed configuration outperforming its closest adversarial baseline, BL7, from 0.852/0.865 to 0.881/0.877. The intended conclusion is that these additions systematically improve boundary delineation and small-region segmentation.","pith_inferences":["The paper leaves implicit that a 3-pixel boundary thickness and weight 10 are optimal; varying these parameters could either enlarge or erase the reported gains, since no ablation is provided.","A single three-fold cross-validation on the SPES subset leaves open whether the method transfers to the full ISLES 2015 challenge data or to later editions with different scanners and protocols.","The large core-Dice jump from residual connections suggests that depth-driven information loss, not just class imbalance, limits small-lesion segmentation; a testable consequence is that shallower encoders would show smaller residual gains.","The same residual-plus-boundary recipe may transfer to other small-structure segmentation tasks, such as tumour or haemorrhage delineation, but that transfer is the authors' implicit hope rather than an experimentally established claim."],"forward_implications":["If the reported comparisons hold, adding residual connections after each encoder block is a cheap and general way to improve segmentation of small anatomical targets, since the largest gain appears in the small core region.","The boundary-weighted loss makes predicted lesion edges track ground-truth boundaries more closely, which is clinically relevant because thrombolytic decisions depend on penumbra extent.","Training from scratch with batch normalization on three concatenated MRI sequences can outperform the ImageNet-pretrained baseline, removing a common practical dependency.","The best result comes from combining adversarial training with both Lovasz-Softmax and boundary weighting, implying the components are complementary rather than redundant."],"supporting_citations":[{"why":"Supplies the residual connections that are added after each encoder block, the core architectural modification tested in this paper.","marker":"He et al., 2016"},{"why":"Supplies SUMNet, the fully convolutional base model that is modified into Residual-SUMNet.","marker":"Nandamuri et al., 2019"},{"why":"Supplies the adversarial training strategy with three relativistic discriminators on core, penumbra, and the pair.","marker":"Sathish et al., 2019"},{"why":"Supplies the Lovasz-Softmax loss used alongside cross-entropy and the boundary-weighted loss.","marker":"Berman et al., 2018"},{"why":"Supplies the VGG11-like encoder design that receives the residual connections.","marker":"Simonyan and Zisserman, 2014"}],"fun_headline_variants":["Residual connections and boundary loss boost stroke Dice","Residual and boundary loss sharpen stroke lesion maps","Residual learning plus boundary loss lifts stroke Dice","Boundary-weighted loss and residual nets improve stroke Dice","Residual skips and boundary loss hike stroke lesion Dice"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that improvements measured in a single three-fold cross-validation on the SPES subset, with boundary thickness and weight fixed at 3 pixels and 10, will generalize to other stroke scans and settings.","fun_headline_variants_meta":{"raw":{"variants":["Residual connections and boundary loss boost stroke Dice","Residual and boundary loss sharpen stroke lesion maps","Residual learning plus boundary loss lifts stroke Dice","Boundary-weighted loss and residual nets improve stroke Dice","Residual skips and boundary loss hike stroke lesion Dice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000609,"raw_usage":{"total_tokens":2816,"prompt_tokens":905,"completion_tokens":1911,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1837}},"tokens_in":521,"tokens_out":1911,"duration_ms":13099,"temperature":1.0,"reasoning_tokens":1837,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:30:23.375187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Residual-SUMNet with the proposed losses on the full ISLES 2015 training set or a held-out test set with the same boundary parameters; if the Dice advantage over BL7 shrinks to within fold-to-fold variability, or if reducing the boundary thickness erases the gain, the central claim that boundary weighting drives the improvement would fail.","supporting_citations":[{"cited_title":"The lov \\'a sz-softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks","cited_arxiv_id":null,"evidence_quote":"Supplies the Lovasz-Softmax loss used alongside cross-entropy and the boundary-weighted loss."}],"review_version":1}