{"id":"47e2132a-8ee5-4a10-a838-f8e974917542","arxiv_id":"2411.11636","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SP3 propagates scribbles through superpixels, refines pseudo-labels with a dynamic threshold, and weights supervision by superpixel-level uncertainty to improve weakly semi-supervised medical image segmentation.","lead":"This paper proposes SP3, a training method that segments medical images using only a few scribble annotations plus many unlabeled images. It reports roughly 80 percent Dice on cardiac and brain tumor MRI, suggesting a large cut in annotation effort if the results replicate.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Internal inconsistency in the main experimental evidence: Table VI reports 0.8186 Dice for ACDC 10% labeled ratio, but Table I reports 0.7937 for the same setting and 0.8186 is the ACDC-20% result; the headline SOTA claim cannot be verified from the paper as written.","rationale":"Reader's weakest assumption (superpixel class-homogeneity) is a real limitation but is mitigated by the dynamic thresholding and uncertainty weighting modules; the paper explicitly discusses bad superpixels and shows ablations where filtering helps. The deeper problem is that the numeric evidence for the central claim is self-contradictory. The identical Dice 0.8186 appears as ACDC-10% 'Ours' in Table VI, as ACDC-20% 'Ours' in Table I, and as the full-module ACDC-20% in Table V. The ASD values also mismatch: Table VI reports 2.85 for 10%, while Table I's 10% value is 3.89 and its 20% value is 2.26. This pattern strongly suggests a copy-mislabeling between the 10% and 20% conditions. Because the paper does not release code, data splits, or the EMA momentum λ, the discrepancy cannot be resolved by external verification. Consequently the strongest claim ('state-of-the-art ... approximately 80% Dice') is not reliably checkable from the manuscript. This is an internal inconsistency, not a disagreement with consensus, so it directly affects correctness. The recommendation remains CONDITIONAL: the method is plausible and the superpixel mechanism is coherent, but the experimental tables must be corrected and ideally code released before the claim can be accepted. The reader's rationale already noted this Table VI/Table I discrepancy, so we partially agree; their formal 'weakest assumption' field, however, points to the superpixel-homogeneity assumption, which we view as less decisive than the numeric contradiction.","tokens_in":19729,"tokens_out":4036,"duration_ms":32864,"concrete_test":"Re-run the thresholding ablation of Table VI on ACDC with exactly the 10% labeled ratio, fold split, and hyperparameters used for the Ours row in Table I (including the same SLIC n and EMA momentum λ). Compare the Tt(c) Dice and ASD against both tables. If Tt(c) reproduces 0.7937/3.89, Table VI's caption or numbers are wrong; if it reproduces 0.8186/2.85, Table I's Ours row is wrong. This single check determines which reported value is the actual 10% result.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SP3 achieves state-of-the-art WSSS performance at roughly 80% Dice with 3% annotation cost, supported by Tables I and II. That support is internally inconsistent. Table VI, captioned 'ACDC dataset under 10% labeled ratio', gives the full method (Tt(c)) a Dice of 0.8186 and ASD of 2.85. Table I, also ACDC at 10% labeled ratio, reports Ours as 0.7937 Dice and 3.89 ASD. The value 0.8186 is exactly the ACDC-20% result in both Table I and Table V (full modules), and 2.85 appears in Table VII, not Table I. Thus one of these tables must be mislabeled or miscomputed. Either the 10% result is 0.7937 and Table VI erroneously reports the 20% run, or the 10% result is 0.8186 and Table I understates the method. Since no code or exact split is released, the reader cannot resolve this from the manuscript. This is more load-bearing than the superpixel-homogeneity assumption because the headline claim is an empirical comparison; an internal contradiction in the numeric record makes the comparison unverifiable as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SP3, a weakly semi-supervised segmentation method that combines a small set of scribble annotations with a large set of unlabeled images. The method propagates scribble labels within SLIC superpixels to obtain dense partial supervision, refines model-generated pseudo-labels by relabeling superpixels selected with a class-specific EMA threshold, and reweights the pseudo-label loss with superpixel-level uncertainty derived from the disagreement of two decoders. Experiments on ACDC and BraTS2019 compare SP3 with weakly supervised, semi-supervised, and weakly semi-supervised baselines, and report state-of-the-art Dice scores at roughly 3% annotation cost. The paper also provides ablations of the three components, thresholding strategies, uncertainty definitions, superpixel counts, and an annotation-cost analysis.","tokens_in":19963,"tokens_out":6341,"duration_ms":57221,"significance":"If the reported numbers are correct, the paper would provide a simple combination of established ingredients (scribble expansion, pseudo-label refinement, uncertainty weighting) that works well for weakly semi-supervised medical image segmentation. The experiments cover two public benchmarks and several supervision settings, which is useful for the community, and the annotation-cost analysis is a helpful practical addition. However, the main empirical claim is currently not verifiable because of a direct conflict between the main comparison table and the ablation tables, and no code or data splits are provided. The conceptual novelty over SOUSA and existing scribble-based semi-supervised methods is incremental; the value of the paper rests almost entirely on the experimental demonstration, so the numeric inconsistencies are load-bearing.","major_comments":[{"comment":"The numeric record for the ACDC 10% labeled-ratio WSSS setting is internally inconsistent. Table I reports \"Ours\" as 0.7937 Dice and 3.89 ASD, while Table V reports the same method at 20% labeled ratio as 0.8186 Dice and 2.26 ASD. Table VI, captioned \"ACDC dataset under 10% labeled ratio,\" reports the full method Tt(c) as 0.8186 Dice and 2.85 ASD, and Table VII, also under \"ACDC ... 10% labeled ratio,\" reports \"Ours\" as 0.8186 Dice and 2.85 ASD. Thus 0.8186 appears both as the 20% result and as the 10% ablation result, while the standalone 10% result is 0.7937. At least one of these tables is mislabeled or computed under a different setting, so the headline state-of-the-art claim, the dynamic-threshold analysis, and the uncertainty ablation cannot be checked as written. The authors must correct every table, state the exact labeled-ratio definition, and release the data splits or a reproducible protocol.","section":"V-B, Tables VI and VII; IV-B, Table I"},{"comment":"The core mechanism assumes that SLIC superpixels are class-homogeneous and aligned with true anatomical boundaries. Eq. (1) relabels every pixel in a superpixel with the class of any intersecting scribble, and Eq. (5) relabels the entire superpixel with the dominant pseudo-label class; if a superpixel straddles a boundary, both operations inject structured boundary error. The paper only controls superpixel size through n and asserts that a large n excludes multi-class scribbles, but it never measures how often superpixels cross object boundaries or contain multiple tissues. Please report superpixel purity and boundary-recall statistics on the evaluation folds, and show sensitivity to SLIC compactness and to n beyond the single-fold tuning in Table VIII.","section":"III-B, Eq. (1); III-C, Eq. (5)"},{"comment":"On BraTS2019, the reported gains over SOUSA are within one standard deviation at both 5% labeled ratio (0.7506±0.1810 vs 0.7320±0.1906) and 10% labeled ratio (0.8064±0.1592 vs 0.7525±0.1629). The paper does not report paired significance tests or fold-wise results, so the claim that SP3 \"outperforms\" the comparison methods on the tumor dataset is not statistically established. Please add per-fold paired tests or report the number of runs and the fold-wise breakdown, especially given the high variance on BraTS2019.","section":"IV-B, Tables I-II"}],"minor_comments":[{"comment":"The EMA update in Eq. (7) uses max(ψ(c)) inside a batch average, but the index over which the maximum is taken and the per-sample definition of the class proportion are not explicit; also, the relationship between the scalar threshold T in Eq. (5) and the class-specific threshold Tt(c) in Eq. (7) should be stated.","section":"III-C, Eq. (7)"},{"comment":"The weighted Dice loss formula is typographically corrupted: the denominator \"P P ∗ W + P Y ∗ W\" is not a readable sum, and the numerator is ambiguous. Please rewrite Eq. (10) with explicit pixel indices and sums.","section":"III-D, Eq. (10)"},{"comment":"The notation for the supervised loss is inconsistent: Algorithm 1 uses LSUP, while Eq. (12) uses Lsup. Please unify the notation throughout.","section":"Algorithm 1 and Eq. (12)"},{"comment":"There is a dangling citation \"[ ?], [30]\" in the pseudo-label paragraph; please replace it with the intended reference.","section":"II-B"},{"comment":"The caption refers to \"red color\" to indicate Dice improvement, but the table is not rendered in color. Use boldface or another unambiguous marker instead.","section":"Table VIII caption"},{"comment":"The sentence \"Higher improvement above the upper bound and other comparative methods with lower label ratios show our method can resist noise better\" is not directly supported by the comparisons shown; please rephrase or provide the supporting analysis.","section":"IV-B.1"},{"comment":"The implementation details do not report the values of τ0 and λ used in the main experiments, even though these are free parameters of the dynamic threshold in Eq. (7). Please state them explicitly.","section":"IV-A"}],"recommendation":"major_revision","confidential_remarks":"The conflict between Tables I, V, VI, and VII is the main blocker: the ACDC 10% labeled-ratio result appears as both 0.7937 and 0.8186, and 0.8186 is also the reported 20% result. The authors must reconcile all numbers and clarify the exact experimental setting before the paper can be considered further. The absence of released code or data splits compounds the problem. The method itself is a reasonable engineering contribution, but the empirical record must be trustworthy."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper combines SLIC superpixel scribble expansion, FreeMatch-style dynamic thresholds, dual-decoder uncertainty, and superpixel pseudo-label refinement into a WSSS pipeline for cardiac and brain tumor segmentation. What is new is the combination and its application to the weak semi-supervised setting, and the main WSSS tables show consistent gains over SOUSA: ACDC 10% Dice 0.7937 versus 0.7224, BraTS 10% 0.8064 versus 0.7525. The ablations are clean, each module adds something, and the dynamic threshold analysis is informative. That is real work, and the writing is clear.\n\nI cannot accept the paper as written, though. Table VI, captioned as ACDC 10% labeled ratio, reports the full method at 0.8186 Dice and 2.85 ASD. Table I for the same setting reports 0.7937 and 3.89. The 0.8186 is exactly the ACDC-20% full result in both Table I and Table V, and the 2.85 appears again in Table VII, which also claims 10%. At least one table is mislabeled or miscalculated, and with no code or exact splits the reader cannot resolve it. That matters because the headline SOTA claim rests on these numbers; the numeric record is internally inconsistent.\n\nThe 'Dense Anno. (UB)' baseline is also called an upper bound but is trained without unlabeled data, so it is not an upper bound for a method that uses unlabeled data, and the text boasts of beating it. That comparison is misleading. Minor issues: no code release, the EMA momentum lambda and some training details are missing, and small margins in the weakly supervised comparisons are not significance-tested.\n\nThe superpixel homogeneity assumption is a real soft spot, but only in the usual sense: if superpixels cross anatomical boundaries, the expansion and relabeling inject structured boundary noise. The paper partially mitigates this with a large n and dynamic filtering, so I do not see it as fatal.\n\nThis paper is for researchers working on scribble-based medical image segmentation. The method is plausible and the experiments, apart from the inconsistency, are carefully presented. I would send it to a serious referee and require corrected tables plus code or exact splits before acceptance, but I would not desk reject it.","headline":"A useful scribble-based WSSS pipeline with real gains in the main tables, but the paper as written has an internal numeric inconsistency that blocks verification of the headline claim.","tokens_in":20571,"tokens_out":1817,"would_cite":false,"duration_ms":17691,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sparse scribbles, expanded and refined through superpixels, reach roughly 80% Dice on cardiac and brain-tumor segmentation with about 3% of the full annotation workload.","keywords":["weakly semi-supervised segmentation","scribble annotation","superpixel propagation","pseudo-label learning","superpixel-level uncertainty","dynamic thresholding","medical image segmentation"],"falsifier":"Run SP3 on a dataset with long thin structures, such as coronary vessels or nerve fibers, where SLIC superpixels frequently straddle true boundaries, and compare Dice against the same pipeline with superpixel refinement disabled; a drop to or below the no-superpixel baseline, or a monotonic decline as the fraction of class-mixed superpixels rises, would show the core assumption fails exactly where it matters.","tokens_in":19465,"feed_emoji":"🩺","tokens_out":12749,"duration_ms":105959,"temperature":0.7,"pith_summary":"SP3 is a weakly semi-supervised medical image segmentation method that trains a dual-decoder network on a small set of scribble-annotated images together with a large pool of unlabeled images. The paper's claim is that superpixel structure can compensate for the sparseness of scribbles: scribbles are expanded to entire superpixels, high-quality superpixels picked by a dynamic threshold are used to refine the network's pseudo-labels, and superpixel-level disagreement between the two decoders weights the pseudo-label loss. On ACDC cardiac MRI and BraTS2019 brain tumor data, the method reports roughly 80% Dice while using about 3% of the pixel annotation effort of full supervision, and it outperforms eight existing weakly or semi-supervised methods across the WSSS, weakly supervised, and semi-supervised settings. If this holds, segmentation models for organs and tumors could be trained from a few scribbled images instead of dense pixel-level labels.","feed_headline":"Hit 80% Dice with just 3% of pixel annotations","feed_subtitle":"SP3 expands scribbles to superpixels and beats fully supervised ACDC with 3% of the label budget.","key_machinery":"The load-bearing object is the SLIC superpixel partition, which serves as the unit for annotation propagation, pseudo-label refinement, and uncertainty estimation. In scribble expansion (Eq. (1)) a superpixel inherits the class of any scribble it contains; in refinement (Eqs. (5)-(7)) a superpixel is relabeled to its dominant predicted class when that class's proportion exceeds an EMA-based, class-specific threshold; and in uncertainty guidance (Eqs. (8)-(11)) the within-superpixel disagreement rate between the two decoder outputs produces weights $w_{ij}=e^{-u_j}$ that rescale the pseudo-label Dice loss. The argument assumes pixels inside a superpixel share texture, and therefore class, and that superpixel boundaries line up with anatomical boundaries, so these three uses turn sparse scribbles and noisy predictions into denser, more reliable supervision.","core_discovery":"The central claim is that superpixels computed offline by SLIC are a dependable structural prior that can be propagated through network training. A scribble that intersects a superpixel labels the entire superpixel by Eq. (1), converting sparse strokes into dense partial supervision. Pseudo-labels, formed by averaging the two decoder predictions, are refined by Eq. (5): any superpixel whose dominant-class pixel proportion exceeds a class-specific threshold is relabeled to that class, with the threshold updated through an exponential moving average (Eq. (7)) so that more superpixels qualify early in training and fewer once predictions become reliable. The fraction of pixels where the two decoders disagree inside a superpixel, Eq. (8), defines a superpixel-level uncertainty weight $w_{ij}=e^{-u_j}$ applied to the pseudo-label Dice loss, forcing the model to learn preferentially from regions where its own predictions agree. With these three operations, the paper reports 0.7937 and 0.8186 Dice on ACDC at 10% and 20% labeled ratios, above the fully supervised dense-annotation baseline, and 0.8064 Dice on BraTS2019 at a 10% labeled ratio, nearly matching the 0.8093 baseline.","pith_inferences":["An untested consequence of the superpixel assumption is that SP3's achievable boundary accuracy is capped by SLIC superpixel resolution: any true boundary falling inside a superpixel is erased by refinement, so objects much thinner than one superpixel should see disproportionately large Dice losses.","A stress test follows directly: on datasets with thin branching structures such as coronary vessels, the frequency of class-mixed superpixels should rise, and the advantage of SP3 over its no-superpixel pseudo-label baseline should shrink or reverse compared with ACDC.","The method treats weak labels abstractly once a scribble exists, so the same expansion-refinement-uncertainty loop could transfer to point or bounding-box annotations and to other imaging modalities whose tissue boundaries align with superpixels; the paper hints at point supervision but does not test this portability."],"forward_implications":["At a 10% labeled ratio on ACDC the method reports 0.7937 Dice and 0.6740 JI, above the scribble-only lower bound (0.5232 Dice) and above the dense-annotation upper bound (0.6824 Dice).","At a 20% labeled ratio on ACDC it reports 0.8186 Dice, 0.0607 higher than the fully supervised upper bound, indicating the superpixel and pseudo-label machinery contributes more than additional labeled pixels.","On BraTS2019 whole-tumor segmentation the method reports 0.7506 Dice at 5% labels and 0.8064 Dice at 10% labels, nearly matching the fully supervised 0.8093 Dice baseline at the higher ratio.","The reported annotation cost is about 3.5% of fully supervised pixel labeling (3.57% on ACDC, 3.49% on BraTS2019), combining the 10% labeled-sample budget with scribble sparsity.","Ablation results trace the gains to expanded-scribble supervision, superpixel refinement with dynamic class-specific thresholds, and superpixel-level uncertainty weighting: removing any of them lowers Dice and worsens boundary metrics such as ASD."],"supporting_citations":[{"why":"SLIC generates the offline superpixel partition that scribble expansion, pseudo-label refinement, and uncertainty weighting all operate on.","marker":"[64]"},{"why":"SOUSA defines the unified weakly semi-supervised scribble-learning task and is the main prior method the paper compares against and improves on.","marker":"[16]"},{"why":"URPC provides the dual-branch dynamic pseudo-label supervision approach that the paper adapts and uses as a primary WSSS and semi-supervised baseline.","marker":"[25]"},{"why":"Scribble2Label supplies the self-training pseudo-label baseline (S2L) used for comparison and grounds the pseudo-label supervision component.","marker":"[24]"},{"why":"USTM's uncertainty-aware self-ensemble is the weakly supervised baseline against which the paper's superpixel-level uncertainty design is positioned.","marker":"[27]"},{"why":"FreeMatch's EMA-based adaptive thresholding is the template for the dynamic class-specific threshold that selects high-quality superpixels for relabeling.","marker":"[56]"},{"why":"This prior superpixel pseudo-label refinement in semi-supervised brain tumor segmentation is the closest antecedent the paper extends by filtering low-quality superpixels.","marker":"[54]"},{"why":"Mutual consistency learning motivates the use of two slightly different decoders whose disagreement defines the paper's superpixel-level uncertainty.","marker":"[40]"}],"fun_headline_variants":["SP3: Superpixels turn scribbles into dense labels for weak supervision","80% Dice with 3% annotations via superpixel pseudo-labels","Superpixel propagation matches full supervision at 3% label cost","From scribbles to segmentation: SP3 hits 80% Dice with 3% labels","SP3: Pseudo-labels from superpixels beat fully supervised at 3% cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that superpixels are internally one-class regions whose borders match the true anatomical borders, so a scribble can validly color an entire superpixel and a dominant predicted class can safely overwrite the rest of a superpixel; when superpixels straddle real boundaries or mix tissues, both expansion and refinement inject structured boundary noise.","fun_headline_variants_meta":{"raw":{"variants":["SP3: Superpixels turn scribbles into dense labels for weak supervision","80% Dice with 3% annotations via superpixel pseudo-labels","Superpixel propagation matches full supervision at 3% label cost","From scribbles to segmentation: SP3 hits 80% Dice with 3% labels","SP3: Pseudo-labels from superpixels beat fully supervised at 3% cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1712,"prompt_tokens":1106,"completion_tokens":606,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":722,"completion_tokens_details":{"reasoning_tokens":500}},"tokens_in":722,"tokens_out":606,"duration_ms":6030,"temperature":1.0,"reasoning_tokens":500,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:18:11.655135+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SP3 on a dataset with long thin structures, such as coronary vessels or nerve fibers, where SLIC superpixels frequently straddle true boundaries, and compare Dice against the same pipeline with superpixel refinement disabled; a drop to or below the no-superpixel baseline, or a monotonic decline as the fraction of class-mixed superpixels rises, would show the core assumption fails exactly where it matters.","supporting_citations":[{"cited_title":"On regularized losses for weakly-supervised cnn segmen- tation,","cited_arxiv_id":null,"evidence_quote":"SLIC generates the offline superpixel partition that scribble expansion, pseudo-label refinement, and uncertainty weighting all operate on."},{"cited_title":"Segmentation only uses sparse annotations: Unified weakly and semi-supervised learning in medical images,","cited_arxiv_id":null,"evidence_quote":"SOUSA defines the unified weakly semi-supervised scribble-learning task and is the main prior method the paper compares against and improves on."},{"cited_title":"Scribble2label: Scribble-supervised cell segmentation via self-generating pseudo-labels with consistency,","cited_arxiv_id":null,"evidence_quote":"URPC provides the dual-branch dynamic pseudo-label supervision approach that the paper adapts and uses as a primary WSSS and semi-supervised baseline."},{"cited_title":"Scribble-based hierarchical weakly supervised learning for brain tumor segmentation,","cited_arxiv_id":null,"evidence_quote":"Scribble2Label supplies the self-training pseudo-label baseline (S2L) used for comparison and grounds the pseudo-label supervision component."},{"cited_title":"Cyclemix: A holistic strategy for medical image segmentation from scribble supervision,","cited_arxiv_id":null,"evidence_quote":"USTM's uncertainty-aware self-ensemble is the weakly supervised baseline against which the paper's superpixel-level uncertainty design is positioned."},{"cited_title":"Superpixel-guided iterative learning from noisy labels for medical image segmentation,","cited_arxiv_id":null,"evidence_quote":"FreeMatch's EMA-based adaptive thresholding is the template for the dynamic class-specific threshold that selects high-quality superpixels for relabeling."},{"cited_title":"Separated contrastive learning for organ-at-risk and gross-tumor-volume segmen- tation with limited annotation,","cited_arxiv_id":null,"evidence_quote":"This prior superpixel pseudo-label refinement in semi-supervised brain tumor segmentation is the closest antecedent the paper extends by filtering low-quality superpixels."},{"cited_title":"Dropout as a bayesian approximation: Representing model uncertainty in deep learning,","cited_arxiv_id":null,"evidence_quote":"Mutual consistency learning motivates the use of two slightly different decoders whose disagreement defines the paper's superpixel-level uncertainty."}],"review_version":1}