{"id":"58964f49-e58c-4c4d-bddb-89d579dcb0cb","arxiv_id":"1908.08294","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Replacing joint label fusion with a weakly supervised U-Net in a corrective learning framework speeds up quadriceps segmentation and improves accuracy on one morphologically atypical subject.","lead":"This paper swaps the slow multi-atlas registration step in a muscle segmentation pipeline for a U-Net, cutting runtime from 20 hours to 2 hours per scan. It reports better accuracy on an anatomically unusual female leg, but the evidence is just one subject and lacks error bars.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The atypical-morphology robustness claim rests on a single female test volume; with n=1 and no inter-rater variability data, the reported 0.089 DICE gain cannot be separated from label noise or random variation.","rationale":"The reader's weakest assumption identifies precisely the single-subject basis for the atypical-morphology claim. My independent read of Table 1 confirms that the only large improvement is on the female right leg (DICE 0.918 vs 0.829), while the male right and left legs show similar or slightly worse DICE and, in one case, substantially worse Hausdorff distance. This makes the n=1 concern even more central because the method's advantage is not a global trend but a single-case outlying gain. The absence of inter-rater variability data, error bars, or significance testing means the reported difference cannot be separated from manual-label noise. The paper is an extended abstract and the contribution is a practical engineering modification, so a CONDITIONAL verdict is appropriate: the claim is plausible and the speedup is real, but the robustness evidence is not yet sufficient. No additional concern beyond the reader's is needed, and the recommended verdict remains unchanged. The proposed concrete test—multi-rater repeatability plus leave-one-out cross-validation—would settle whether the female-volume gain is methodologically meaningful or a statistical artifact.","tokens_in":3062,"tokens_out":3618,"duration_ms":40196,"concrete_test":"Obtain two independent expert manual segmentations of the female right-leg volume and compute inter-rater DICE. If the inter-rater DICE is close to or smaller than the reported 0.089 DICE improvement, the U-Net gain over JLF+CL is within labeling noise. In addition, run a leave-one-out cross-validation across all seven labeled volumes (training on six, testing on the held-out volume, with the same data augmentation protocol) and report per-subject DICE, HD, and MAD for U-Net+CL and JLF+CL. The atypical-morphology robustness claim would only be credible if the female-volume improvement exceeds the cross-subject spread and inter-rater variability by a clear margin; otherwise, the single-subject result is anecdotal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of improved robustness for morphologically atypical quadriceps is supported by exactly one test volume: the female right leg, where U-Net+CL achieves DICE 0.918 versus 0.829 for JLF+CL (Table 1). This is a single observation with no error bars, no confidence interval, and no measurement of manual segmentation repeatability. The manual contours are treated as ground truth, but the paper does not report inter-rater DICE or any quality control for the female volume's labels. In musculoskeletal MRI, DICE differences of this magnitude can easily fall within manual labeling variability, especially for muscle heads with unclear boundaries. Moreover, the improvement is not consistent across all test cells: for the male right leg, U-Net+CL DICE is slightly lower (0.925 vs 0.937) and Hausdorff distance is substantially worse (30.794 mm vs 17.694 mm), so the statement 'similar or even improved accuracy' does not hold uniformly. The load-bearing assumption is therefore that this one female volume and its manual segmentation are representative of 'remarkably different morphology' and that the observed gain is attributable to the method rather than to noise in the single comparison. With n=1, no statistical inference about robustness is possible, and the paper's conclusion overstates the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a segmentation framework for the four quadriceps muscle heads in 3D T1-weighted MR images of ultra-endurance athletes, in which the computationally expensive multi-atlas segmentation with joint label fusion (JLF) inside Wang and Yushkevich's corrective learning (CL) framework is replaced by a weakly supervised U-Net. Weak labels are generated via deformable registration and random B-spline warping from seven manually segmented atlases. The test set consists of one female right thigh, one young male right thigh, and three left thigh volumes. On the female right leg, U-Net with CL achieves DICE 0.918 versus 0.829 for JLF with CL, while on the male right leg DICE declines slightly (0.925 vs 0.937) and Hausdorff distance worsens substantially (30.794 mm vs 17.694 mm). The authors report that JLF takes about 20 hours per volume on 16 CPUs, whereas each U-Net requires about 2 hours of training on a Tesla V100, and conclude that the proposed method provides similar or improved accuracy with significantly reduced computation time, especially for atypical morphologies.","tokens_in":3445,"tokens_out":3240,"duration_ms":33691,"significance":"If the accuracy claims were confirmed on a broader test set, the contribution would be practically valuable: a trainable segmentation method that avoids the per-volume cost of multi-atlas registration while maintaining or improving accuracy on morphologically unusual quadriceps would enable large-scale longitudinal studies of ultra-endurance athletes. The architectural idea of embedding a U-Net in the corrective-learning loop is reasonable and the paper makes an honest statement of the trade-offs on the male right-leg cell in Table 1. However, the evidence supporting the headline robustness claim is confined to a single female test volume, and the computation-time comparison mixes training time with per-volume inference time. The significance is therefore conditional on additional validation.","major_comments":[{"comment":"The central claim of improved robustness for 'remarkably different morphology' is supported by exactly one test volume: the female right leg, where U-Net+CL achieves DICE 0.918 versus 0.829 for JLF+CL. With n=1, no error bars, and no inter-rater variability assessment for the manual ground truth, the observed 0.089 DICE difference cannot be separated from label noise or random variation. Please report per-volume scores for all test subjects, add a repeatability analysis of the manual segmentations, or explicitly restrict the conclusion to a case study rather than a general robustness claim.","section":"§4, Table 1"},{"comment":"The runtime comparison is not apples-to-apples: 20 hours is reported as the JLF inference time for one volume on 16 CPUs, while 2 hours is the training time of each U-Net on a Tesla V100. No per-volume inference time is given for U-Net or U-Net+CL, so the statement that the approach 'reduced significantly the computation time' is not directly supported. Please report the wall-clock time to produce one test segmentation with U-Net and U-Net+CL on comparable hardware.","section":"§4, runtime comparison"},{"comment":"The manuscript does not specify how the five corrective-learning U-Nets, each trained with a different validation subject, are used to compute the U-Net+CL results in Table 1. It is unclear which network is applied to each test volume, whether the five outputs are averaged, or how the learned correction is combined with the base U-Net prediction. This omission prevents reproduction and makes the U-Net+CL results ambiguous.","section":"§3, Corrective learning step"},{"comment":"The conclusion states that the method provided 'similar or even improved accuracy', but this does not hold for all reported cells: for the male right leg, U-Net+CL DICE is lower than JLF+CL (0.925 vs 0.937) and HD is markedly worse (30.794 mm vs 17.694 mm). Please qualify the conclusion to reflect the mixed results across the male and left-leg test cases.","section":"§5, Conclusion"}],"minor_comments":[{"comment":"There is a typo in the abstract: 'corrective leaning' should read 'corrective learning'.","section":"Abstract"},{"comment":"The text says 'the Table 4 quantifies results on test set' but the table is numbered Table 1; please correct the cross-reference.","section":"§4, text before Table 1"},{"comment":"Please provide the essential architectural details of the U-Net (depth, number of channels per level, input slice size, loss function, optimizer, number of training epochs) so that the experiments are reproducible.","section":"§3, U-Net architecture"},{"comment":"The description of weak label generation is brief: please clarify whether the random B-spline warping is applied to the label maps only or to both image and labels, and how the amount of warping was chosen to avoid anatomically implausible labels.","section":"§2, Data"},{"comment":"For the left-thigh rows, the paper reports averages over 2D slices but not over the three subjects; please indicate the number of volumes and slices and report the spread (e.g., standard deviation or range) to allow the reader to judge variability.","section":"§4, Table 1"},{"comment":"The manuscript does not mention ethics approval or data availability; if the data cannot be shared, a statement to that effect would be useful for readers.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"This is a four-page extended abstract from MIDL 2019. The central idea is sensible, but the paper's own evidence does not support the general robustness claim: the atypical-morphology result is a single female volume, and the runtime claim compares JLF inference with U-Net training. I would be willing to reconsider a revised version that reports per-subject scores with variability, clarifies the CL inference procedure, and gives an honest runtime comparison. As it stands, the claims outpace the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clean, honest incremental engineering result — swapping the expensive multi-atlas JLF step for a U-Net inside Wang and Yushkevich’s corrective-learning loop, applied to quadriceps MRI from ultra-endurance athletes. The reported speedup is real and big (20 hours on 16 CPUs to 2 hours on one V100), and the paper is candid that the male-leg numbers don’t improve. What’s genuinely new is the application and the observation that a weakly supervised U-Net handles one morphologically atypical subject better than JLF. That’s a useful data point, not a proven claim.\n\nThe paper does a few things right. The method description is enough to follow the pipeline. The evaluation uses held-out test subjects, and the weak labels come from training subjects only, so the comparison isn’t circular. The authors state the tradeoff explicitly instead of spinning it. For a four-page extended abstract, that’s the right level of care.\n\nThe soft spot is the one the reader’s report puts its finger on: the headline robustness result rests on a single female volume. DICE 0.918 vs 0.829 is a large gap, but with n=1 and no inter-rater variability on the manual contours, it could be label noise or an idiosyncratic volume. The male right-leg HD also gets worse (30.8 mm vs 17.7 mm), which undercuts the “similar or even improved accuracy” phrasing. No error bars, no per-structure breakdown, no code or training hyperparameters. I’d call all of that proportionate criticism, not fatal: the paper says “for the case with remarkably different morphology,” which is narrower than the abstract’s “robustly segmenting.” The main limitation is that the conclusion generalizes one observation.\n\nI don’t buy the circularity concern. The corrective-learning nets are validated on training data and tested on held-out volumes; that’s standard. The self-referential worry doesn’t land here.\n\nWho gains: groups working on muscle segmentation in sports medicine or on cheap replacements for multi-atlas pipelines. It won’t change how we think about U-Nets. I would bring it to a reading group as an example of a quick, honest evaluation of a standard substitution rather than as a method paper. I wouldn’t cite it as evidence for robustness, but I might cite it as a practical speedup data point if I worked on quadriceps segmentation.\n\nRecommendation: as a full journal submission it’s too thin, but as an extended abstract it deserves a serious referee. Send it out; the referee should ask for more subjects, error bars, and implementation details.","headline":"Honest incremental engineering result; the robustness claim rests on a single female volume and needs more data, but the speedup and candor make it worth a proper review.","tokens_in":3875,"tokens_out":2056,"would_cite":false,"duration_ms":20174,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Replacing atlas-based joint-label fusion with a weakly supervised U-Net improves DICE on an atypical female quadriceps volume from 0.829 to 0.918 while cutting the reported 20-hour step to about 2 hours.","keywords":["quadriceps segmentation","U-Net","weakly supervised learning","joint label fusion","corrective learning","multi-atlas segmentation","magnetic resonance imaging","ultra-endurance athletes"],"falsifier":"Segment ten or more atypical right-thigh MRI volumes with both JLF+CL and U-Net+CL, using expert segmentations as ground truth; if the U-Net pipeline does not beat JLF+CL on the majority of those volumes in DICE, the paper's central robustness claim fails.","tokens_in":2834,"feed_emoji":"🦵","tokens_out":11775,"duration_ms":107166,"temperature":0.7,"pith_summary":"Segmenting the four quadriceps heads on MRI is hard because the boundaries between adjacent muscle heads are not clearly visible, and the usual atlas-based approach is both slow and brittle when a subject's thigh shape differs strongly from the atlases. The paper proposes keeping a corrective-learning post-processing stage, which learns to repair typical errors of the automatic segmentation, but replacing the expensive multi-atlas joint-label-fusion (JLF) segmenter with a weakly supervised U-Net, trained on expert segmentations plus automatically generated segmentations from deformable registration and B-spline warping. On the authors' test data, this swap raised the DICE overlap score from 0.829 to 0.918 for the morphologically atypical female right thigh, while male right and left legs were slightly lower, with male right DICE at 0.925 versus 0.937 for JLF plus corrective learning. The reported cost of the dominant segmentation step drops from roughly 20 hours per volume with JLF on 16 CPUs to about 2 hours of U-Net training on a GPU. If the result holds, automatic quadriceps segmentation becomes practical enough to track individual muscle heads over the course of an ultra-marathon.","feed_headline":"U-Net swap lifts atypical-leg DICE from 0.829 to 0.918","feed_subtitle":"Replacing 20-hour atlas matching with a 2-hour U-Net keeps accuracy on typical legs and improves it on atypical ones.","key_machinery":"The central object is the weakly supervised U-Net used as a drop-in replacement for the multi-atlas joint-label-fusion (JLF) step, a voting scheme that combines atlas labels after deformable registration. U-Net is a convolutional encoder-decoder that maps an axial MR slice to a per-pixel label map of the four quadriceps heads. The corrective-learning (CL) stage is retained: it trains a separate network on the errors of the initial automatic segmentations and applies that learned correction to the final output. Weak supervision is provided by augmenting the expert-labeled training volumes with segmentations generated through deformable registration and random B-spline warping of selected athletes, which balances morphology in the training set. The load-bearing identity is therefore the replacement itself: a fast learned segmenter that is less sensitive to large anatomical deformations, followed by the existing error-correction mechanism.","core_discovery":"The paper's claim is that a U-Net trained with expert and weakly labeled slice data can stand in for the registration-based joint-label-fusion step of a corrective-learning segmentation framework, and that for quadriceps MRI this substitution improves accuracy precisely on the case where atlas matching struggled: a subject with markedly different thigh morphology. On the female right-leg test volume, U-Net plus corrective learning gives DICE 0.918 versus 0.829 for JLF plus corrective learning, and mean absolute distance 1.638 mm versus 5.600 mm. On the typical male right and left test volumes the methods are comparable, with a slight DICE decline for U-Net on the male right leg (0.925 versus 0.937) and slight gains on the left leg. The authors also claim a practical computational advantage: the expensive step drops from about 20 hours per volume for JLF to about 2 hours of U-Net training on a GPU.","pith_inferences":["The decisive atypical-morphology comparison consists of one female volume; a multi-subject atypical test set would be needed to tell whether the improvement is a property of the method or of that particular thigh.","The reported time comparison is not end-to-end: JLF's 20 hours is a per-volume inference cost, while the U-Net figure is training time, so the true per-volume runtime of the new pipeline remains unmeasured.","The weak labels come from warping a chosen set of athletes not in the test set; the choice of those source athletes is likely to affect how well the U-Net generalizes, and a more homogeneous training set might erase the advantage."],"forward_implications":["The reported time for the expensive segmentation step drops by roughly an order of magnitude, from about 20 hours per volume with JLF on 16 CPUs to about 2 hours of U-Net training, making large athlete cohorts more feasible.","For the atypical female thigh, DICE improves from 0.829 with JLF+CL to 0.918 with U-Net+CL, and mean absolute distance improves from 5.600 mm to 1.638 mm, while typical male legs stay within a few hundredths of DICE.","Because corrective learning is retained, the method still has a mechanism for repairing boundary errors that the first-pass U-Net makes, which the authors see as the main remaining error type.","The automatic segmentation is intended to enable longitudinal monitoring of local changes in each quadriceps head along an ultra-marathon, rather than only a whole-muscle analysis."],"supporting_citations":[{"why":"Documents the lack of clear muscle-head boundaries on MRI that makes quadriceps segmentation difficult.","marker":"Prescott et al. (2011)"},{"why":"Describes the multi-atlas joint-label-fusion plus corrective-learning framework whose segmentation stage this paper replaces.","marker":"Wang and Yushkevich (2013)"},{"why":"Reports the earlier JLF-based segmentation of the same athlete dataset and supplies the workflow baselines.","marker":"Nguyen et al. (2018)"},{"why":"Introduces the U-Net architecture used as the substitute segmentation network.","marker":"Ronneberger et al. (2015)"}],"fun_headline_variants":["U-Net beats atlas on odd legs, cuts time 10x","Weak U-Net lifts atypical DICE, cuts time 90%","Atypical thigh DICE jumps 0.829 to 0.918 with U-Net","U-Net swap improves odd-leg MRI, faster than atlas","From 20h to 2h: U-Net refines quadriceps segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The robustness claim rests on a single female test volume, and if that manual segmentation is unrepresentative or noisy, the reported DICE improvement could disappear.","fun_headline_variants_meta":{"raw":{"variants":["U-Net beats atlas on odd legs, cuts time 10x","Weak U-Net lifts atypical DICE, cuts time 90%","Atypical thigh DICE jumps 0.829 to 0.918 with U-Net","U-Net swap improves odd-leg MRI, faster than atlas","From 20h to 2h: U-Net refines quadriceps segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000666,"raw_usage":{"total_tokens":2963,"prompt_tokens":795,"completion_tokens":2168,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":411,"completion_tokens_details":{"reasoning_tokens":2065}},"tokens_in":411,"tokens_out":2168,"duration_ms":14221,"temperature":1.0,"reasoning_tokens":2065,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:42:35.234667+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Segment ten or more atypical right-thigh MRI volumes with both JLF+CL and U-Net+CL, using expert segmentations as ground truth; if the U-Net pipeline does not beat JLF+CL on the majority of those volumes in DICE, the paper's central robustness claim fails.","supporting_citations":[{"cited_title":"Anatomically Anchored Template-Based Level Set Segmentation: Application to Quadriceps Muscles in MR Images from the Osteoarthritis Initiative","cited_arxiv_id":null,"evidence_quote":"Documents the lack of clear muscle-head boundaries on MRI that makes quadriceps segmentation difficult."},{"cited_title":"Multi-atlas segmentation with joint label fusion and corrective learning - an open source implementation","cited_arxiv_id":null,"evidence_quote":"Describes the multi-atlas joint-label-fusion plus corrective-learning framework whose segmentation stage this paper replaces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the earlier JLF-based segmentation of the same athlete dataset and supplies the workflow baselines."}],"review_version":1}