{"id":"5c45cdc2-bae5-4f67-a9ec-0a183c69ce0b","arxiv_id":"1908.04173","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"With sparse scribble labels expanded by a random walk and a closing preprocessing step, a U-Net segments bone implant microtomograms as accurately as when trained on dense expert annotations: Dice 0.751 versus 0.750.","lead":"This paper tests training a U-Net image segmentation network using only scribble annotations instead of full pixel masks, with a random walk step to densify the sparse labels. On synchrotron scans of biodegradable bone implants, the sparse method matched the dense-label Dice score while cutting annotation effort.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Preprocessing choice is selected on test folds; with n=4 the reported dense-equivalent Dice can be a selection artifact rather than a validated method.","rationale":"The paper's contribution is an empirical training-target construction: random-walk propagation of scribbles plus closing reaches dense-label U-Net quality on this implant microtomography task. For that claim to be true, the reported 0.751 must be an unbiased estimate of the pipeline's performance. The described evaluation does not ensure this. The two preprocessing options are compared directly in cross-validation, and the better one is then presented as the proposed method. Because all four volumes are used both to select the preprocessing and to report its Dice, the number is optimistic. This is the same load-bearing element the reader identified — the unexplained closing step — but the more precise failure is selection on the test folds, not merely lack of rationale. If closing had been fixed a priori (e.g., justified from random-walk leakage or chosen on a validation fold), the concern would be reduced to generalization uncertainty, which would still warrant the conditional verdict. The proposed hold-out test would settle it: if RW+closing remains at dense-level Dice when chosen without test-set access, the claim stands; if not, the headline should be weakened. The small sample size (four volumes) and absence of significance testing amplify the risk but are secondary; a nested selection protocol is the decisive check. I therefore recommend no change to the reader's conditional verdict.","tokens_in":3601,"tokens_out":6453,"duration_ms":72107,"concrete_test":"Hold out one volume as test. On the remaining three, select between RW-only and RW+closing (with fixed, stated structuring element) using an internal validation split — e.g., train on two volumes and validate on the third — without seeing the held-out volume. Apply the chosen variant to the held-out volume. Repeat for all four held-out volumes, average, and compare with dense supervision using per-fold paired differences and a sign test. If the average selected-RW Dice falls below about 0.750 or the paired differences are not consistently positive, the Table 1 equivalence is a test-set selection artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 states that two experiments, with and without binary closing, were conducted for each supervision method, and Table 1 reports both. The random-walk row that carries the paper's claim (0.751) is the closing variant; without it the same pipeline scores 0.687, 6.4 points below dense supervision (0.750). The paper provides no separate validation split or pre-registered criterion for choosing the closing. In effect, the closing is a hyperparameter selected using the leave-one-out test Dice on the same four volumes. With n=4 and per-fold standard deviations near 0.07-0.10, choosing the better of two variants on the test set can produce the reported \"match\" even if closing has no generalizable benefit. The issue is not that morphological closing is implausible — it may well remove random-walk label noise — but that the protocol does not establish that the 0.751 estimate is unbiased, and the structuring element shape and size are also unspecified. Thus the central empirical claim is not yet supported as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses the annotation bottleneck for segmenting biodegradable magnesium-based bone implants in synchrotron microtomograms. The authors propose a pipeline in which a user provides only sparse scribble annotations, a random walk algorithm propagates these to dense pixel-wise labels, an optional morphological closing is applied to the bone label, and a multi-slice 2.5D U-Net is trained on the resulting targets. Using leave-one-out cross-validation on four volumes, they report that this random-walk-based target with closing achieves a Dice coefficient of 0.751 ± 0.068, nearly identical to 0.750 ± 0.102 for dense supervision, while directly training on scribbles yields only 0.482 ± 0.106. The paper concludes that the random-walk approach with morphological post-processing matches dense annotation quality and therefore offers a practical alternative to exhaustive pixel-level labeling.","tokens_in":3798,"tokens_out":2669,"duration_ms":31520,"significance":"If the central claim were fully validated, the paper would make a useful practical contribution by showing that expensive dense expert annotations can be replaced by much cheaper scribbles for this implant-imaging task, without degradation in segmentation quality. The experimental design is simple and the comparison across supervision targets is informative, including the important negative result that direct scribble training fails badly. The paper also honestly reports per-label Dice and notes the difficulty of the corroded screw region. However, the claim of equivalence rests on a single configuration whose key preprocessing step is selected on the test folds of only four datasets, and the paper lacks any statistical analysis that would support an equivalence claim. The significance is therefore conditional on a more rigorous validation protocol.","major_comments":[{"comment":"The headline comparison is not apples-to-apples: the random walk result of 0.751 is obtained with the binary closing preprocessing, while the dense baseline of 0.750 is the variant without preprocessing. Table 1 shows that closing changes the dense result from 0.750 to 0.703 but changes the random walk result from 0.687 to 0.751. The paper gives no rationale for why this morphological operation should benefit the random walk target but harm the dense target, and the structuring element shape and size are not specified. Because the closing variant was evaluated in the same leave-one-out cross-validation and selected after seeing the test Dice, the reported 0.751 estimate is a test-set-selected maximum over two preprocessing choices. With only four volumes, this can easily produce the observed 'match' even if the closing has no generalizable benefit. The authors should pre-specify the preprocessing choice on a separate validation split, or at minimum report all variants with a clear statement of how the choice was made, and justify the asymmetry between the random walk and dense conditions.","section":"Section 2 (Experiments) and Table 1"},{"comment":"The central empirical claim is a statement of equivalence between random walk with closing (0.751 ± 0.068) and dense supervision (0.750 ± 0.102), yet no statistical test is reported. With n=4, the difference of 0.001 Dice is meaningless without a paired analysis, and the 6.4-point improvement from closing is also not shown to be significant. The paper should report per-fold results, perform a paired significance test or compute a confidence interval for the difference, and explicitly define an equivalence margin if the claim is that the methods are equivalent. Without such analysis, the reader cannot distinguish a genuine match from noise.","section":"Section 3 (Results and Discussion) and Table 1"},{"comment":"The random walk algorithm is described only as using 'the grayvalue scan for edge-preservation,' but no details are given for the edge-weight parameter (e.g., beta or conductance), the scribble annotation protocol (number of scribbles, their placement, or the fraction of annotated pixels per slice), or the number of labeled slices per volume beyond '25 slices.' These choices directly affect the random walk target and therefore the reported Dice. Without these details, the method is not reproducible and the sensitivity of the result to these hyperparameters is unknown.","section":"Section 2 (Data and Training)"}],"minor_comments":[{"comment":"The abstract and conclusion state that the random walk approach 'achieves the same Dice overlap as a dense supervision' without noting that this holds only for the variant with binary closing; the unprocessed random walk target is 6.4 percentage points worse. Please qualify the claim.","section":"Abstract and Section 4"},{"comment":"It is unclear whether '25 slices have been labeled (every 10th slice)' means 25 slices per dataset or 25 slices in total across all four datasets. Please clarify the annotation density.","section":"Section 2 (Data)"},{"comment":"The network is described as a 'multi-slice 2.5D U-Net' and later as a modified 3D U-Net, but the exact input/output configuration (eight input slices collapsed to one output slice) is only implicit. A short explicit description of the tensor shapes would improve clarity.","section":"Section 1 and Section 2"},{"comment":"There are minor typographical issues, e.g., 'synchroton' in the conclusion, and the reference list would benefit from consistent conference/journal formatting.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible extended abstract with a practically relevant question, but the central claim of dense-equivalent performance is currently supported only by a test-selected preprocessing choice over four datasets. I would like to see a revision that either provides a proper validation protocol for the closing step (e.g., a held-out split or a pre-registered decision rule), reports per-fold numbers with a paired analysis, and specifies all random-walk and closing hyperparameters, or substantially tempers the conclusion to reflect the uncertainty. If the authors can make these changes within the scope of an extended abstract, the paper could be acceptable; otherwise the claim of equivalence should be downgraded."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper reports something useful and probably true, but the key numerical equivalence is not established as robustly as the title implies. The pipeline is simple: scribbles → random walk → binary closing of the bone label → U-Net training. On leave-one-out over four synchrotron volumes it gets Dice 0.751, matching 0.750 dense supervision. That is a genuinely new empirical result for this implant task, and the direct comparison to training on raw scribbles (0.482) is a clear demonstration that naive sparse training fails.\n\nWhat I like: the experiment is honest enough to show both preprocessing rows, so you can see the closing is doing the work. The table reports all classes, not just the average. The method uses standard components and is easy to reproduce if the tuning details were shared. The connection to Can et al. is cited and the simplification is reasonable.\n\nThe soft spots are where the stress-test lands. The 0.751 number is the random-walk row with closing; without closing it is 0.687, a 6.4 point deficit to dense. The dense baseline that carries the comparison is the no-closing row (0.750) because closing hurts dense (0.703). So the headline claim is really 'best RW variant equals best dense variant.' That is a legitimate way to compare methods only if the choice of closing is justified by a validation set or a task-specific rationale. The paper gives neither. With n=4 and per-fold standard deviations around 0.07–0.10, selecting the better of two transformations on the test folds can easily produce this difference by chance. The structuring element size and the random walk parameters are also unspecified ('appropriately tuned').\n\nI would not call the paper circular — the RW targets come from the same training images, but evaluation uses independent dense ground truth, and self-training is a normal paradigm. The issue is not circularity; it is model selection bias.\n\nWho benefits: people working on scribble-supervised segmentation in 3D medical imaging, especially synchrotron/micro-CT bone imaging. The paper is an extended abstract, so readers should treat it as a preliminary result, not a definitive protocol.\n\nI would send it to peer review because the question is worth asking and the empirical setup, for all its limitations, is a fair first test. The authors should be pushed to (a) justify or validate the closing, (b) report random walk parameters and kernel size, and (c) ideally add a fifth/sixth dataset or a significance test. With those additions the claim would be credible.","headline":"A modest, plausibly useful scribble-supervision pipeline whose headline result — random walk plus closing matches dense annotation — rests on a per-method preprocessing choice evaluated on the same four volumes.","tokens_in":4335,"tokens_out":2229,"would_cite":false,"duration_ms":23004,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scribble labels match dense expert segmentation of bone implants with the help of a random-walk expansion and one closing step.","keywords":["biodegradable bone implants","synchrotron microtomography","sparse annotations","scribble supervision","random walk","U-Net","morphological closing","Dice coefficient"],"falsifier":"Run the identical random-walk-plus-closing training on an unseen synchrotron bone-implant dataset with different bone structure or contrast and compare it with a densely supervised U-Net: a gap near or above six Dice percentage points would contradict the claimed equivalence.","tokens_in":1473,"feed_emoji":"🦴","tokens_out":1682,"duration_ms":73131,"temperature":0.7,"pith_summary":"This paper claims that pixel-level expert annotations are not necessary for training a U-Net to segment biodegradable bone implants in synchrotron microtomograms. By converting user scribbles to dense targets with a random-walk algorithm and applying a binary closing to the bone label, the same network reaches a Dice coefficient of 0.751 ± 0.068 in leave-one-out cross-validation, effectively matching the 0.750 ± 0.102 achieved with dense annotations. Directly training on the scribbles alone drops to 0.482 Dice, a loss of 26.8 percentage points. If the claim holds, expanding training data to larger scan databases becomes feasible at a small fraction of the annotation cost.","feed_headline":"Scribble labels match dense expert segmentation of bone implants","feed_subtitle":"A random-walk expansion plus one closing step lifts Dice from 0.687 to 0.751, tying dense supervision.","key_machinery":"The load-bearing mechanism is a two-stage target-generation pipeline: a random-walk algorithm propagates sparse scribble labels to a dense grid, using image gray values to keep the propagated labels aligned with edges, followed by a binary morphological closing on the bone label to fill small holes and connect nearby bone regions. The random walk supplies dense supervision from a fraction of annotated pixels, and the closing step lifts total Dice from 0.687 to 0.751. The network itself is a 2.5D U-Net that takes eight neighbouring slices as input and reduces the depth to one, enabling supervision with individual 2D segmentations on partially labelled volumes.","core_discovery":"The paper's central claim is that, for this task, a U-Net trained on scribble labels propagated by random walks and then morphologically closed on the bone class produces segmentation quality equal to one trained on full expert labels. The measured total Dice is 0.751 ± 0.068 versus 0.750 ± 0.102, a difference of 0.1 percentage points, while per-class results show the closing step mainly repairs the bone and corroded-screw labels. This is what the paper argues for, with the caveat that the closing step is essential: without it the random-walk target reaches only 0.687 Dice. The corroded-screw class remains the weakest segment in every configuration, attributed to its small area and difficult gray-value contrast.","pith_inferences":["The same scribble-to-dense conversion could be tested on other synchrotron or micro-CT data where bone or biomaterial boundaries are visible in gray values; the edge-preserving random walk should carry over, while the choice of closing may need re-tuning per task.","The binary closing of the bone label acts as an implicit prior that bone is a connected structure around the implant; a principled rule for when to apply such a prior would make the pipeline more portable.","Boundary metrics or component-wise error could reveal whether the matched Dice hides a different error distribution, which matters if the goal is quantifying degradation geometry rather than only overlap.","A natural extension is to let the random walk use multiple scribbled slices jointly, which might sharpen the weak corroded-screw class without adding dense annotations."],"forward_implications":["Scribble annotation on selected slices is enough to train a U-Net that matches dense-label accuracy, so scaling training databases to more microtomogram volumes no longer requires full pixelwise tracing.","A simple fixed pipeline, random-walk propagation plus one morphological closing, replaces earlier iterative scribble-supervision setups that combine recurrent networks and conditional random fields.","The 2.5D U-Net can be trained on volumes where only every tenth slice carries a label, making partial-volume annotation a practical training protocol.","The corroded-screw class is the limiting category, so acquiring more variable training data for corrosion regions is the immediate path to further improvement.","The matching total Dice means a scribble-trained model can stand in for a dense-trained model in the segmentation step of implant degradation studies without changing the measured overlap."],"supporting_citations":[{"why":"Supplies the random-walk algorithm that propagates sparse scribble labels to a dense grid using gray values for edge preservation.","marker":"(Grady, 2006)"},{"why":"The prior scribble-supervision approach with random walks, recurrent networks, and a CRF that the paper simplifies into a non-iterative pipeline.","marker":"(Can et al., 2018)"},{"why":"Supplies the U-Net architecture that is adapted into the 2.5D multi-slice network used for segmentation.","marker":"(Ronneberger et al., 2015)"},{"why":"Proposes the multi-slice 2.5D U-Net variant that lets eight neighbouring slices be used to predict one supervised slice.","marker":"(Mehta and Sivaswamy, 2017)"},{"why":"Describes the biodegradable magnesium-based implant imaging task and the synchrotron microtomography data used in the experiments.","marker":"(Moosmann et al., 2017)"}],"fun_headline_variants":["Scribble labels with random walks match dense U-Net segmentation","Random-walk scribbles tie dense supervision for bone implant Dice","Closing step lets scribble-trained U-Net match full labels","Sparse annotation via random walks equals dense labeling for U-Net","Random-walk target with closing matches full-supervision Dice"],"cache_read_input_tokens":6528,"weakest_assumption_plain":"The load-bearing premise is that the binary closing of the bone label is a generally valid correction for this imaging task and not a tweak that happens to help these four scans; without it, the random-walk target trails dense annotation by 6.3 Dice percentage points (0.687 vs 0.750).","fun_headline_variants_meta":{"raw":{"variants":["Scribble labels with random walks match dense U-Net segmentation","Random-walk scribbles tie dense supervision for bone implant Dice","Closing step lets scribble-trained U-Net match full labels","Sparse annotation via random walks equals dense labeling for U-Net","Random-walk target with closing matches full-supervision Dice"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000515,"raw_usage":{"total_tokens":2490,"prompt_tokens":924,"completion_tokens":1566,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":1478}},"tokens_in":540,"tokens_out":1566,"duration_ms":11571,"temperature":1.0,"reasoning_tokens":1478,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:48:41.467544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical random-walk-plus-closing training on an unseen synchrotron bone-implant dataset with different bone structure or contrast and compare it with a densely supervised U-Net: a gap near or above six Dice percentage points would contradict the claimed equivalence.","supporting_citations":[{"cited_title":"Random walks for image segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the random-walk algorithm that propagates sparse scribble labels to a dense grid using gray values for edge preservation."},{"cited_title":"Learning to segment medical images with scribble-supervision alone","cited_arxiv_id":null,"evidence_quote":"The prior scribble-supervision approach with random walks, recurrent networks, and a CRF that the paper simplifies into a non-iterative pipeline."},{"cited_title":"M-net: A convolutional neural network for deep brain structure segmentation","cited_arxiv_id":null,"evidence_quote":"Proposes the multi-slice 2.5D U-Net variant that lets eight neighbouring slices be used to predict one supervised slice."},{"cited_title":"Biodegradable magnesium-based implants in bone studied by synchrotron radiation microtomography","cited_arxiv_id":null,"evidence_quote":"Describes the biodegradable magnesium-based implant imaging task and the synchrotron microtomography data used in the experiments."}],"review_version":1}