REVIEW 3 major objections 6 minor 14 references
Shape-Aware Complementary-Task Learning for Multi-Organ Segmentation
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Auxiliary shape tasks lift multi-organ segmentation dice to 0.9018
desk verdict Modest but believable multi-organ segmentation gain from auxiliary distance/contour tasks; likely sign typo in Eq. (2) and weak global statistics hold it back. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the complementary-task framework: given an organ segmentation label, the paper derives two additional targets—the Euclidean distance transform of each organ (a smooth map whose value at each interior pixel is the distance to the boundary) and the binary contour map (the organ edge). These targets are regressed or classified by two extra network branches that share all feature extraction with the segmentation branch. The distance map is meant to encode geometric and positional priors; the contour map is meant to focus learning on boundary accuracy. The combined loss is a plain sum of the three task losses, with no learned weighting.
What would settle it
Train the same three-branch network on a fully manually annotated multi-organ CT dataset and compare against a segmentation-only U-Net; if the dice gain over baseline shrinks to near zero or turns negative, the reported improvement depends on the specific noisy-label regime rather than on shape-prior learning itself.
Extended reading notes
Core claim
The central claim is that jointly optimizing segmentation with two label-derived auxiliary tasks—distance-map regression and contour-map detection—improves multi-organ segmentation. The network is a U-Net with three output branches sharing an encoder and decoder; the total loss sums the segmentation loss (cross-entropy plus Dice), the contour loss (same form), and mean squared error on the distance map. On 74 noisy silver-corpus CT scans used for training and 23 manually labeled gold-corpus scans for testing, the combined model reaches a mean dice of 0.9018 versus 0.8849 for the segmentation-only baseline. Organ-wise Wilcoxon tests show statistically significant improvements for several small organs, while large organs change little.
Load-bearing premise
The training labels come from a silver corpus produced by fusing multiple automatic algorithms, so they are noisy; the paper's gains are only as reliable as those labels, and on cleaner training data the shape-prior advantage could weaken or reverse.
Editorial extensions
If this is right
- Multi-organ segmentation accuracy improves without any extra manual annotation, because the auxiliary targets come from existing masks.
- Small organs benefit most, suggesting shape priors compensate for the class imbalance that makes small structures easy to miss.
- Distance-map regression alone helps slightly, while contour detection alone hurts; the combination is what yields the gain, indicating the two tasks interact.
- The same recipe transfers to any segmentation problem where a single network must handle multiple objects of varied shapes and sizes.
Reading between the lines
- One could test whether the gain persists when the auxiliary targets are computed from cleaner, expert labels; the paper's own explanation for the contour-only drop invokes label noise, so the combined gain may partly be a denoising or regularization effect rather than pure shape learning.
- The approach may extend to 3D distance transforms and contour surfaces, since whole-body CT is volumetric and slice-wise 2D training ignores through-plane geometry.
- A natural comparison would be to replace hand-defined auxiliary tasks with a learned auxiliary head that predicts signed distance or boundary probability, to see whether the specific forms matter or any structured output suffices.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-organ segmentation method based on complementary-task learning. Starting from a U-Net, the authors add two auxiliary output branches: one regresses a distance transform of the organ segmentation, and one predicts the organ contours. Both auxiliary targets are derived from the existing segmentation labels, so no extra annotation is required. The method is trained on 74 CT scans from the VISCERAL silver corpus (noisy automatic labels) and tested on 23 gold-corpus scans with manual labels. The reported overall Dice improves from 0.8849 (baseline U-Net) to 0.9018 (U-Net with distance and contour branches), with organ-wise Wilcoxon tests showing significant gains for several small organs. The paper also discusses large-organ stability and the failure of the contour-only variant.
Significance. If the reported result is reproducible, this is a practically useful contribution: it improves multi-organ segmentation without additional annotation, on a public benchmark, using simple geometric auxiliary targets. The use of a manually labeled held-out test set and per-organ paired statistical tests are strengths. The limitations are equally clear: the primary overall-Dice improvement is not tested statistically, the results come from a single split, the silver-corpus label noise affects the auxiliary targets themselves, and Eq. (2) as printed defines a distance loss with the wrong sign. The central idea remains plausible, but the stated mechanism is not currently well-defined enough to support the abstract's strong claim.
major comments (3)
- [Section 2, Eq. 2] The distance-map loss is printed as Ldist = -1/n sum_x (g(x) - p(x))^2. Minimizing this term maximizes the squared error between the predicted and ground-truth distance maps, which is the opposite of distance-map regression. Because the paper credits the Dice improvement to 'distance map regression', the loss as written does not implement the advertised mechanism. Please correct the sign if it is a typo and state exactly which loss was optimized in the experiments; the absence of released code makes this detail unverifiable and it is load-bearing for the central claim.
- [Section 3, Table 1 and Abstract] The abstract's claim of a 'significant improvement of overall dice score from 0.8849 to 0.9018' is not supported by any significance test on the overall Dice. Table 1 reports only means and standard deviations, and the Wilcoxon signed-rank results in Fig. 4 are organ-specific. With 23 test scans and a single train/test split, a paired test on per-scan overall Dice, and ideally confidence intervals or repeated splits, is needed to support the headline claim.
- [Section 3, Dataset paragraph] The training labels are described as 'automatically labeled by fusing the results of multiple algorithms, yielding noisy labels.' Since both auxiliary targets (distance map and contour map) are derived from these noisy labels, the shape prior inherits the silver-corpus label noise. The authors themselves attribute the contour-only failure to this noise, but do not discuss how the same noise affects the distance-map prior or whether the proposed gain would survive cleaner training labels. A sensitivity analysis or at least a careful discussion of this limitation is required to assess generalizability.
minor comments (6)
- [Section 2, Eq. 1] Equation (1) defines Lseg and Lcontour jointly with a single notation, but the reader has to infer which l and which sum correspond to each loss; please split the equations and define the summation indices explicitly.
- [Section 3, Table 1 and Fig. 4] Please state whether the reported Dice standard deviations are computed per volume or per slice, how many test volumes/classes are included, and how the 'overall' score averages organs.
- [Section 3, Fig. 4] The organ label 'R Rectus Abdominis' appears twice in the figure, and '1st Lum Vertebra' should be '1st Lumbar Vertebra'; also clarify whether any multiple-comparison correction was applied to the per-organ p-values.
- [Section 2, Loss Function] The paper does not state how the three losses are balanced; if all are simply summed with equal weight, this should be said explicitly, since loss weighting is a known critical factor in multi-task learning.
- [Section 3, Experiments] No code or trained model is released; given the signed-loss ambiguity, providing code would substantially improve reproducibility and confidence in the reported numbers.
- [Section 2, Learning Distance-transform] The phrase 'a boundary regularity measure of a shape with respect to an interior point' is unclear; please rephrase the motivation for the Euclidean distance transform.
Circularity Check
No circularity found: the claimed improvement is an empirical result on an independent gold test set, not a construction-level tautology.
full rationale
The paper's central claim is that adding distance-map regression and contour-map detection auxiliary tasks improves multi-organ Dice from 0.8849 to 0.9018 on the VISCERAL gold test set. The auxiliary targets are derived from the existing training segmentation labels and are incorporated as extra loss terms; no parameter is fitted to the gold test labels, no uniqueness theorem or external result is imported to force the chosen architecture, and the cited prior work supplies standard components (U-Net, distance transform, heatmap regression) rather than the load-bearing justification for the reported gain. The authors explicitly acknowledge that the silver-corpus training labels are noisy and use this to explain why contour-only training underperforms; that limitation affects external validity but does not make the experiment circular. The negative sign in Eq. (2) would make the distance loss a maximization of squared error if taken literally, but this is an internal consistency or correctness concern, not a circularity step: it does not equate the prediction with the input by construction. Since the evaluation is against manually annotated gold labels and the method does not rely on self-citation for its central claim, the appropriate circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Auxiliary-task loss weights =
1.0 each (unweighted sum)
assumptions (3)
- domain assumption Distance transform and binary contour maps derived from organ labels are useful shape priors for segmentation.
- domain assumption Silver-corpus noisy labels are accurate enough to supervise segmentation and auxiliary tasks.
- domain assumption The single VISCERAL gold test split of 23 scans is representative and the comparison is unbiased.
Cite this review
Pith. "Pith review of Shape-Aware Complementary-Task Learning for Multi-Organ Segmentation." pith.science (2026). https://pith.science/paper/7STWBAI3
@misc{pith2026190805099,
author = {Pith},
title = {Pith review of: Shape-Aware Complementary-Task Learning for Multi-Organ Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/7STWBAI3}},
note = {Machine review of arXiv:1908.05099}
}
read the original abstract
Multi-organ segmentation in whole-body computed tomography (CT) is a constant pre-processing step which finds its application in organ-specific image retrieval, radiotherapy planning, and interventional image analysis. We address this problem from an organ-specific shape-prior learning perspective. We introduce the idea of complementary-task learning to enforce shape-prior leveraging the existing target labels. We propose two complementary-tasks namely i) distance map regression and ii) contour map detection to explicitly encode the geometric properties of each organ. We evaluate the proposed solution on the public VISCERAL dataset containing CT scans of multiple organs. We report a significant improvement of overall dice score from 0.8849 to 0.9018 due to the incorporation of complementary-task learning.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:1709.05932 (2017)
Bischke, B., et al.: Multi-task learning for segmentation of building footprints with deep neural networks. arXiv preprint arXiv:1709.05932 (2017)
arXiv 2017
-
[2]
In: Proceedings of the ICML (2018)
Chen, Z., et al.: Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: Proceedings of the ICML (2018)
work page 2018
-
[3]
In: Proceedings of the ICLR (2019)
Geirhos, R., et al.: Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: Proceedings of the ICLR (2019)
work page 2019
-
[4]
In: Proceedings of the IEEE CVPR
He, K., et al.: Mask r-cnn. In: Proceedings of the IEEE CVPR. pp. 2961–2969 (2017)
work page 2017
-
[5]
In: Proceedings of the IEEE CVPR
Kendall, A., et al.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: Proceedings of the IEEE CVPR. pp. 7482–7491 (2018)
work page 2018
-
[6]
In: Proceedings of the IEEE CVPR
Misra, I., et al.: Cross-stitch networks for multi-task learning. In: Proceedings of the IEEE CVPR. pp. 3994–4003 (2016)
work page 2016
-
[7]
Medical Image Analysis 54, 207–219 (2019)
Payer, C., et al.: Integrating spatial configuration into heatmap regression based cnns for landmark localization. Medical Image Analysis 54, 207–219 (2019)
work page 2019
-
[8]
In: Proceedings of the MICCAI (2015)
Ronneberger, O., et al.: U-net: Convolutional networks for biomedical image seg- mentation. In: Proceedings of the MICCAI (2015)
work page 2015
Show all 14 references
-
[9]
IEEE TPAMI
Shelhamer, E., et al.: Fully convolutional networks for semantic segmentation. IEEE TPAMI. (2017)
2017
-
[10]
Med- ical Image Analysis (2017)
Song, Y., et al.: Progressive multi-atlas label fusion by dictionary evolution. Med- ical Image Analysis (2017)
2017
-
[11]
IEEE TMI 35(11), 2459–2475 (2016)
Jimenez-del Toro, O., et al.: Cloud-based evaluation of anatomical structure seg- mentation and landmark detection algorithms: VISCERAL anatomy benchmarks. IEEE TMI 35(11), 2459–2475 (2016)
2016
-
[12]
In: Proceedings of the MICCAI
Uslu, F., Bharath, A.A.: A multi-task network to detect junctions in retinal vas- culature. In: Proceedings of the MICCAI. pp. 92–100. Springer (2018)
2018
-
[13]
IEEE TMI (2012)
Wang, H., et al.: Multi-atlas segmentation with joint label fusion. IEEE TMI (2012)
2012
-
[14]
IEEE JBHI (2019)
Zhao, Y., et al.: Knowledge-aided convolutional neural network for small organ segmentation. IEEE JBHI (2019)
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.