{"id":"9abf4a96-a3da-4c30-ba14-052a0f449e01","arxiv_id":"1908.03735","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A semi-supervised pipeline with a dual-branch classification network, K-Means, and region growing segments acute ischemic stroke lesions using 460 slice-level labels and 5 pixel-level labels, achieving Dice 0.642.","lead":"This paper reports a semi-supervised method for segmenting acute ischemic stroke lesions on MRI that uses many easy slice-level labels and only five detailed pixel-level labels. It combines a weakly trained neural network, K-Means clustering, and region growing to reach a mean Dice score of 0.642 on 150 test patients.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported Dice/F1 rest on hyperparameters (K, δ) chosen by grid search on only five fine-tuning subjects; without cross-validation or error bars, the point estimate may not be stable.","rationale":"The reader's weakest assumption correctly identifies the grid-search selection of K and δ on only five fully-labeled subjects as the most load-bearing threat to the central claim. The paper's headline numerical results are a single point estimate produced after tuning on a tiny set, with no cross-validation, no confidence intervals, and no sensitivity analysis. I considered the missing same-data fully-supervised baseline as an alternative concern, since the paper's 'comparable to fully-supervised' phrasing is weakened by cross-dataset comparison to [24] and by only comparing against U-Net/FCN-8s trained with the same five labels. However, the absolute performance numbers are the foundation of the claim, and the n=5 tuning issue directly threatens those numbers; the baseline issue affects interpretation rather than the primary estimate. The proposed concrete test—repeatedly drawing pseudo-tuning sets from the 150 test subjects and measuring the spread of resulting DICE—would settle whether the reported 0.642 is stable. Until such an analysis is provided, the CONDITIONAL verdict is appropriate and no change to the reader's verdict is needed.","tokens_in":14039,"tokens_out":8196,"duration_ms":92634,"concrete_test":"Simulate the tuning protocol on the 150 fully-labeled test subjects: repeatedly (e.g., 200 times) draw 5 subjects as a pseudo-tuning set, run the same grid search over K∈{4,5,6,7} and δ∈{0.3,0.4,0.6,0.7} using DICE on those 5 subjects, then evaluate the selected configuration on the remaining 145 subjects. Report the mean and 95% interval of the resulting DICE and F1. If the interval is wide (e.g., ±0.05 or more) or the mean falls well below 0.642, the reported point estimate is not robust to the n=5 selection. If data access is unavailable, at minimum run leave-one-out cross-validation on the five fine-tuning subjects and report the variance in selected K, δ, and the resulting test-set DICE.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—mean DC 0.642 and lesion-wise F1 0.822—depends on the specific hyperparameters K=6 and δ=0.4 selected in Section 4.1 via grid search on only 5 fully-labeled subjects (Table 3). With n=5, no cross-validation, and no repeated sampling, the chosen configuration may overfit those five subjects, and the reported test numbers are a single draw from a high-variance selection process. Notably, the best configuration's DC on the fine-tuning set is 0.556, while the same configuration achieves 0.642 on the 150-subject test set, illustrating that the five tuning subjects are not representative of the test distribution. A different choice of five tuning subjects could lead to different K and δ and to a materially lower test DICE. The paper reports no confidence intervals or variance estimates, so the reader cannot tell whether 0.642 is robust or an artifact of the small tuning set. This is the most load-bearing weak point because it directly undermines the numerical value that the strongest claim, and the paper's conclusion, rest on.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semi-supervised pipeline for acute ischemic stroke (AIS) lesion segmentation on diffusion-weighted images (DWI) and apparent diffusion coefficient (ADC) maps. The method has three stages: a double-path classification network (DPC-Net) trained on 460 slice-level weakly labeled subjects to produce probability maps; K-Means clustering on DWI intensities to identify hyperintense candidate regions; and a region-growing step that combines the DPC-Net probability map with the K-Means clusters, using two parameters K (number of clusters) and delta (probability threshold) tuned on five fully labeled subjects. The authors report a Dice coefficient of 0.642 and a lesion-wise F1 of 0.822 on a 150-subject clinical test set, and compare with CAM-baseline, U-Net, and FCN-8s baselines trained on the same five fully labeled subjects. The central claim is that with very few pixel-level annotations, the proposed method approaches fully-supervised performance.","tokens_in":14267,"tokens_out":2062,"duration_ms":23545,"significance":"If the reported results are stable, the paper would be a meaningful practical contribution: it demonstrates a semi-supervised method that avoids the error-propagation of self-training, uses only slice-level weak labels plus five pixel-level annotations, and achieves Dice/F1 values in the range of fully-supervised AIS segmentation methods. The algorithmic combination of a weakly supervised CNN with an unsupervised intensity prior and a small fully labeled fine-tuning set is well motivated. The paper also reports lesion-wise metrics, which are clinically relevant given the importance of small lacunar lesions. However, the strength of this contribution rests almost entirely on the reliability of the point estimates from the 150-subject test set, since the method's two free parameters are selected on only five subjects without variance estimation.","major_comments":[{"comment":"In Table 4, the test set is split into large (60) and small (90) lesion sets, and the proposed method achieves higher DC on small lesions (0.708) than on large ones (0.543). The discussion explains that hyperintensity distribution is uneven in large lesions, but this size-dependent behavior is not analyzed with respect to the hyperparameter tuning; since 85% of AIS are lacunar, the authors should discuss whether the grid search on five subjects may be biased toward small lesions and how this affects the generalizability of the selected K and δ.","section":"Section 4.1 and Table 3"}],"minor_comments":[{"comment":"Reference [25] is an unpublished medRxiv preprint by the same group; the claim in Section 1 that 'the lesion-wise detection rate is high' for weak supervision relies on this preprint, and it would be preferable to cite a peer-reviewed source or describe the result in sufficient detail to be self-contained.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the method is clearly described, but the small-sample hyperparameter selection is a serious threat to the validity of the headline number. The authors should be asked to provide a stability analysis (e.g., leave-one-out on the five tuning subjects, or bootstrap over the 150-subject test set) and to correct the abstract discrepancy. I would also encourage the editor to check that the comparison to fully-supervised methods is formulated carefully, since the current text risks overclaiming parity with methods trained on hundreds of labels. These are fixable within the scope of a major revision, so I am not recommending rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a modest, honest engineering paper, not a methodological breakthrough. It builds a semi-supervised pipeline for acute ischemic stroke lesion segmentation from 460 slice-level labeled subjects plus five pixel-level annotated subjects, and reports a mean Dice of 0.642 and lesion-wise F1 of 0.822 on 150 test subjects. Those numbers are plausible, but they are point estimates from a private dataset with no error bars, so treat them as provisional.\n\nWhat's new: the specific pipeline—a double-path classification network (VGG-16 truncated at block 7, with a side branch at block 4) whose fused CAMs seed a K-means + region-growing refinement—is not in the cited literature. Each component is known; the combination is a reasonable engineering contribution. The paper also does several things well. It compares against U-Net and FCN-8s trained on the same five fully-labeled subjects, which is the right control for showing the value of the weakly-supervised pathway. It reports lesion-wise precision/recall/F1 in addition to Dice, and separates small (lacunar) from large lesions, which is clinically meaningful. The qualitative point that the DPC-Net probability map rejects susceptibility artifacts is supported by the examples and the CAM-baseline comparison. The writing is clear enough to reimplement the method.\n\nSoft spots. First, the abstract says 15 fully-labeled subjects while the full text consistently says 5. That mismatch must be fixed; it undercuts trust. Second, K and delta are selected by grid search on five subjects, with no cross-validation and no confidence intervals. The stress-test concern about overfitting is real in principle, but Table 3 partly defuses it: the chosen configuration (K=6, delta=0.41) also achieves the best test Dice among the four grid points (0.642 vs 0.610, 0.594, 0.322). So the selection is not visibly overfit within this small grid. Still, n=5 is thin and the 0.642 point estimate has unknown variance. Third, there is no fully-supervised baseline trained on the same data; citing a fully-supervised method's result on a different dataset is not a same-data comparison. Fourth, no code or data is provided, so independent verification means reimplementation. The citation pattern looks normal; no self-citation issues.\n\nWho this is for: researchers working on label-efficient medical image segmentation, specifically stroke. I'd take it seriously as a candidate for peer review, not a desk reject. The required revisions are concrete: fix the abstract, add error bars or a small cross-validation, and ideally evaluate on a public benchmark such as ISLES.","headline":"A plausible semi-supervised AIS segmentation pipeline with real but modest empirical claims; the headline Dice is a point estimate on a private dataset with no error bars, and the abstract/full-text subject-count mismatch needs fixing.","tokens_in":14791,"tokens_out":4129,"would_cite":true,"duration_ms":40854,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Semi-supervised stroke lesion segmentation with only five fully labeled subjects can match fully supervised accuracy.","keywords":["semi-supervised learning","acute ischemic stroke","lesion segmentation","weakly supervised learning","class activation map","K-Means clustering","region growing","diffusion-weighted imaging"],"falsifier":"Repeat the grid search on a different set of five fully labeled subjects, for example with leave-one-out over the fine-tuning set, and measure Dice on the same 150-subject test set; if the mean Dice falls well below 0.642, the reported accuracy is tied to the particular choice of five subjects rather than to the method itself.","tokens_in":13827,"feed_emoji":"🧠","tokens_out":8215,"duration_ms":72346,"temperature":0.7,"pith_summary":"This paper aims to show that acute ischemic stroke lesion segmentation can be done accurately without hundreds of pixel-level annotated scans. The proposed pipeline trains a double-path classification network on 460 subjects with only slice-level weak labels, then uses K-Means clustering and a region-growing step, fine-tuned on just five fully labeled subjects, to produce final segmentations. On 150 test subjects the method reports a mean Dice coefficient of 0.642 and a lesion-wise F1 score of 0.822, close to fully supervised methods that need many more detailed annotations. If the claim holds, it would substantially lower the annotation burden for AIS segmentation while keeping clinically useful accuracy.","feed_headline":"Five full labels give Dice 0.642 on stroke lesions","feed_subtitle":"A pipeline using 460 weak labels and 5 detailed labels outperforms supervised baselines on 150 patients.","key_machinery":"The load-bearing mechanism is a two-pathway fusion. The DPC-Net, a VGG-16 truncated before the third max-pooling layer with an extra global-average-pooling side branch at an earlier convolution block, is trained on slice-level labels and produces two class activation maps at different resolutions; thresholding the coarse map and multiplying it by the fine map yields a probability map of suspicious lesion regions. Independently, K-Means clustering on the DWI separates hyperintense pixels, using the clinical prior that AIS lesions appear bright on DWI, and connected components of the brightest cluster form candidate regions. A region-growing algorithm then starts from probability-map pixels above $\\delta$ and, when they fall inside a K-Means component, grows to fill that component, so semantic information from the network and intensity information from clustering jointly decide the final boundary. The values $K$ and $\\delta$ are selected by grid search on the five fully labeled subjects.","core_discovery":"The central claim is that combining weak semantic supervision with unsupervised intensity clustering yields lesion segmentations comparable to fully supervised deep networks. Concretely, the paper reports that its semi-supervised pipeline, trained on 460 weakly labeled subjects and fine-tuned with five fully labeled subjects, achieves a mean Dice coefficient of 0.642 and a lesion-wise F1 score of 0.822 on a 150-subject clinical test set, with lesion-wise precision 0.880. The method deliberately avoids generating fake labels for weakly labeled data, instead using fully labeled subjects only to tune two parameters: the cluster count $K$ in K-Means and the threshold $\\delta$ in region growing. The paper further claims this design is especially sensitive to small lesions, reporting a Dice of 0.708 on a 90-subject small-lesion subset.","pith_inferences":["The same weak-label-plus-clustering recipe could be tried on other hyperintense-lesion tasks in MRI, such as multiple sclerosis plaques, where the intensity prior and slice-level labels are similarly cheap.","A direct robustness test would vary which five subjects form the fine-tuning set; if $K$ and $\\delta$ shift materially, a slightly larger fine-tuning set or a learned parameter predictor would be needed.","The K-Means branch could be replaced by a learned intensity model trained on the five fully labeled slices, potentially removing the grid search while keeping the semi-supervised structure.","A head-to-head evaluation on one shared public benchmark would show whether the annotation saving transfers across scanners, field strengths, and imaging protocols."],"forward_implications":["A hospital could build an AIS segmentation system from cheap slice-level screening labels plus a handful of detailed cases, reducing expert marking time.","The reported Dice of 0.642 is close to the 0.67 of a fully supervised DWI method, suggesting weakly supervised training need not sacrifice much accuracy.","Because the method preserves small lesions better than large ones (Dice 0.708 versus 0.543), it may be useful for lacunar infarctions, which are common but hard to spot.","The high lesion-wise precision of 0.880 means few false lesion detections per patient, which matters if the output is used to flag suspected strokes."],"supporting_citations":[{"why":"Supplies the class activation map technique used to convert the weakly trained classifier into spatial probability maps.","marker":"[38]"},{"why":"Provides the clinical prior that AIS lesions are hyperintense on DWI, which motivates the K-Means branch.","marker":"[3]"},{"why":"Shows weakly supervised detection achieves high lesion-wise recall but poor pixel-level boundaries, the gap this method targets.","marker":"[25]"},{"why":"Fully supervised DWI segmentation baseline whose reported Dice of 0.67 anchors the accuracy comparison.","marker":"[24]"},{"why":"Fully supervised 3D DenseNet segmentation that required 152 fully labeled subjects, illustrating the annotation cost the method avoids.","marker":"[13]"},{"why":"Provides the VGG-16 backbone that DPC-Net truncates and extends with the side branch.","marker":"[37]"},{"why":"Trained on the five fully labeled subjects as a supervised baseline for comparison.","marker":"[39]"},{"why":"Trained on the five fully labeled subjects as a second supervised baseline for comparison.","marker":"[20]"}],"fun_headline_variants":["5 full labels drive stroke lesion Dice to 0.642","Semi-supervised stroke segmentation: Dice 0.642 with only 5 full labels","Stroke lesion Dice 0.642 from 5 labeled scans + 460 weak","5 detailed labels: stroke lesion Dice 0.642 semi-supervised","Dice 0.642 for AIS lesions with just 5 full labels"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the two parameters tuned on just five fully labeled subjects, the number of K-Means clusters and the probability threshold, keep working on the 150-patient test set rather than reflecting only those five patients.","fun_headline_variants_meta":{"raw":{"variants":["5 full labels drive stroke lesion Dice to 0.642","Semi-supervised stroke segmentation: Dice 0.642 with only 5 full labels","Stroke lesion Dice 0.642 from 5 labeled scans + 460 weak","5 detailed labels: stroke lesion Dice 0.642 semi-supervised","Dice 0.642 for AIS lesions with just 5 full labels"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001447,"raw_usage":{"total_tokens":5877,"prompt_tokens":1041,"completion_tokens":4836,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":657,"completion_tokens_details":{"reasoning_tokens":4734}},"tokens_in":657,"tokens_out":4836,"duration_ms":36320,"temperature":1.0,"reasoning_tokens":4734,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:03:36.842820+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the grid search on a different set of five fully labeled subjects, for example with leave-one-out over the fine-tuning set, and measure Dice on the same 150-subject test set; if the mean Dice falls well below 0.642, the reported accuracy is tied to the particular choice of five subjects rather than to the method itself.","supporting_citations":[{"cited_title":"& Torralba, A., (2016)","cited_arxiv_id":null,"evidence_quote":"Supplies the class activation map technique used to convert the weakly trained classifier into spatial probability maps."},{"cited_title":"W., Leung, E","cited_arxiv_id":null,"evidence_quote":"Provides the clinical prior that AIS lesions are hyperintense on DWI, which motivates the K-Means branch."},{"cited_title":"& Xia, S.,2020","cited_arxiv_id":null,"evidence_quote":"Shows weakly supervised detection achieves high lesion-wise recall but poor pixel-level boundaries, the gap this method targets."},{"cited_title":"& Rueckert, D., 2017","cited_arxiv_id":null,"evidence_quote":"Fully supervised DWI segmentation baseline whose reported Dice of 0.67 anchors the accuracy comparison."},{"cited_title":"M., Mok, V","cited_arxiv_id":null,"evidence_quote":"Fully supervised 3D DenseNet segmentation that required 152 fully labeled subjects, illustrating the annotation cost the method avoids."},{"cited_title":"& Zisserman, A., 2014","cited_arxiv_id":null,"evidence_quote":"Provides the VGG-16 backbone that DPC-Net truncates and extends with the side branch."},{"cited_title":"& Brox, T., ( 2015)","cited_arxiv_id":null,"evidence_quote":"Trained on the five fully labeled subjects as a supervised baseline for comparison."},{"cited_title":"& Darrell, T., ( 2015)","cited_arxiv_id":null,"evidence_quote":"Trained on the five fully labeled subjects as a second supervised baseline for comparison."}],"review_version":1}