REVIEW 4 major objections 5 minor 39 references
ShapeKit
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Training-free mask corrections lifted CT organ segmentation Dice by over 8 percentage points without altering network weights.
desk verdict A useful toolkit and taxonomy, but the >8% DSC claim is not backed by the table; send to review and demand proper statistics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a pipeline of fast operators built on connected-component analysis and morphological filtering, applied directly to the predicted label mask. General functions (removal of small components, suppression of non-dominant components, false-positive reassignment, left-right division, fragment merging) are combined with organ-specific rules, most notably the use of the liver's known right-side position to disambiguate the laterality of lungs and kidneys. The toolkit is optimized for speed and memory efficiency, supporting parallel batch processing and vectorized operations on standard array libraries.
What would settle it
Run ShapeKit with its default configuration on a fresh, previously unseen CT test set from a different institution without any parameter tuning, then compute the mean DSC improvement across all organs; if the mean gain is not above 8 percentage points, the central over-8% multi-organ claim does not hold as stated.
Extended reading notes
Core claim
The central discovery is that a modular set of morphological and topological corrections - removing small artifacts, suppressing non-dominant fragments, reassigning false positives, separating left-right organs using the liver as an anatomical anchor, and merging broken structures - can consistently improve segmentation quality across organs and datasets. Evaluated on predictions from the VISTA3D model on two out-of-distribution test sets, the corrections yield organ-wise DSC gains from about +2 to +8.8 points, with the largest gains on small, high-variability organs such as the gall bladder and pancreas. The paper presents this as evidence that post-processing can outperform architecture-level changes without any retraining.
Load-bearing premise
The reported gains depend on hyperparameters and organ-specific rules that the paper treats as fixed but does not validate on a separate split; if those settings were tuned on the same test sets used for evaluation, the claimed over-8% improvement overestimates what a user would get on new data.
Editorial extensions
If this is right
- Any existing segmentation model, old or new, can be upgraded with ShapeKit at zero retraining cost, making it attractive for regulated clinical deployments.
- Shape cleaning can be added as a standard preprocessing step for downstream tasks such as volumetric analysis, surgical planning, and synthetic CT generation, preventing shape errors from propagating into derived outputs.
- The reported gains, especially on low-DSC organs, suggest that a substantial part of a segmentation model's remaining error is structural and correctable by anatomical priors rather than by deeper networks.
- The error taxonomy - artifacts, false positives, redundancy, fragmentation, laterality - gives the community a shared language for auditing and improving mask quality.
Reading between the lines
- The paper's headline of over 8% multi-organ DSC improvement is not directly derivable from the per-organ table, since averaging the six reported per-organ gains yields roughly 3.9 and 4.4 percentage points for the two datasets; a pooled or averaged statistic would clarify the claim.
- If ShapeKit's thresholds were tuned after inspecting these test sets, the generalizable gain is probably smaller; a sensitivity analysis on an independent validation set would quantify this.
- The same toolbox could plausibly be adapted to other imaging modalities or to 2D slice-based corrections, and might be extended to enforce temporal consistency in longitudinal scans.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ShapeKit, a post-processing toolkit that applies anatomical shape corrections (small-component removal, false-positive reassignment, left-right separation, and merging of fragmented structures) to CT organ segmentation masks after inference. The authors claim that this training-free post-processing improves multi-organ Dice similarity coefficient (DSC) by over 8% on two out-of-distribution datasets, and that this exceeds typical gains from architecture modifications (less than 3%). The paper includes a taxonomy of shape errors, a description of the toolkit, quantitative results in Table 1, and qualitative visualizations.
Significance. If the claimed improvements are reproducible, ShapeKit would be a valuable, model-agnostic post-processing layer for clinical and research workflows, offering meaningful gains without retraining. The open-source release and the systematic taxonomy of shape errors are useful contributions. However, the current evidence does not substantiate the headline claims because the only quantitative support is a single-point per-organ comparison with no error bars, no statistical tests, and no documented validation procedure. The >10% claim in Contribution 1 is inconsistent with the reported data and is not derived from any presented result.
major comments (4)
- [Section 4.1, Table 1] The central claim in Section 1 and the abstract that ShapeKit 'can lift multi-organ DSC score by >8%' is not supported by the data presented. Table 1 reports per-organ DSC changes at a single operating point for six organs per dataset, with no sample size, no variance or confidence intervals, and no paired significance test. No multi-organ average is provided, so the number 'over 8%' cannot be read from the table; the largest single-organ gain is +8.8, and Contribution 1's '>10% DSC improvement' is not derivable from any reported value. The authors should report the multi-organ mean DSC (with standard deviation or confidence intervals), perform paired tests, and either support or retract the >10% claim.
- [Section 3.3 and Section 4] The hyperparameters of the pipeline (e.g., check_size_threshold=500, keep_top=2, organ-specific size and shape thresholds, the split axis, and the organ adjacency map) are presented as fixed, yet the paper does not describe a validation split or a sensitivity analysis. Since the test subsets are described as 'exclusively reserved for testing,' it is unclear how these parameters were selected. If they were tuned on the test data, the reported gains are overestimates and the generalization claim is invalid. The paper should describe the parameter selection procedure or report results across a range of settings.
- [Section 4.1, Table 1] The evaluation protocol is insufficiently specified. The number of subjects in each 'representative subset' is not given, and it is not stated whether the lungs are scored as separate left and right labels or as a single merged structure. The table lists 'lung L' and 'lung R' separately, but the baseline label set for VISTA3D and the exact matching procedure are not described. Without this information, the DSC values in Table 1 cannot be reproduced or interpreted.
- [Section 1 and Related Work] The paper's comparison of post-processing gains to architecture modifications ('typically <3%' [1,16]) is not supported by any experiment in this work. The comparison is citation-based and does not include any existing post-processing baseline (e.g., simple connected-component filtering as used in nnU-Net [15]) on the same datasets. To contextualize the claimed improvements, the authors should add a comparison to at least one standard post-processing method.
minor comments (5)
- [Abstract and Section 1] The abstract's 'over 8%' and Contribution 1's '>10%' are internally inconsistent; please reconcile the numbers.
- [Section 3.1] The error taxonomy is said to be 'distilled from large-scale whole-body CT datasets,' but the datasets are not named; if they include the test sets, clarify why this does not constitute selection bias.
- [Contribution 1] Contribution 1 refers to a 'widely-adopted, highly-competitive segmentation benchmark' without naming it; the actual evaluation uses subsets of JHH and AbdomenAtlasPro, which should be stated explicitly.
- [Figure 2] Figure 2 would be more informative if it included quantitative DSC values for the shown cases, rather than only qualitative overlays.
- [Table 1] Table 1 would benefit from a row reporting the average DSC across all organs and a column indicating the number of subjects per dataset.
Circularity Check
No circularity: ShapeKit's DSC gains are empirical before/after measurements, not a derivation that reduces to its inputs.
full rationale
The paper makes no first-principles derivation. Its central claim is an empirical before/after measurement: applying ShapeKit's morphological and topological rules to VISTA3D masks on two held-out subsets and computing DSC against ground truth. There is no equation in which an output is defined as the input, and no fitted parameter is renamed as a prediction: the thresholds (check_size_threshold=500, keep_top=2, organ size/shape rules) are stated as fixed design choices ('These rules are informed by the typical size and count of organs', Section 3.3), not fit to the test sets. The 'typically <3%' architecture-gain comparison is cited from external or mixed benchmarks (Touchstone, nnU-Net revisited) and is motivational context, not load-bearing for the ShapeKit measurement. The error taxonomy is descriptive and is not used to construct the evaluation metric. Therefore the evaluation is self-contained with respect to circularity. The reviewer's concern that Table 1 lacks averages, variance, and a validation split is a statistical-support and internal-consistency objection, not a demonstration that the result reduces to its inputs; per the analysis rules, that belongs in correctness risk, not circularity.
Assumptions & free parameters
free parameters (6)
- check_size_threshold =
500
- keep_top =
2
- Small component volume threshold =
user-defined
- Per-organ size/shape thresholds =
not disclosed
- Left-right split axis index =
AXIS=0
- Organ adjacency map =
hand-crafted
assumptions (6)
- domain assumption The liver is always located on the right side of the body.
- domain assumption The test subsets are out-of-distribution relative to VISTA3D training.
- standard math Connected components and morphological transformations on binary masks are reliable.
- domain assumption Dice coefficient is a sufficient metric for organ segmentation quality.
- ad hoc to paper Anatomical adjacency map is correct and complete.
- domain assumption Post-processing to enforce anatomical plausibility will generally improve agreement with ground truth.
Cite this review
Pith. "Pith review of ShapeKit." pith.science (2026). https://pith.science/paper/3CNJ3EHH
@misc{pith2026250624003,
author = {Pith},
title = {Pith review of: ShapeKit},
year = {2026},
howpublished = {\url{https://pith.science/paper/3CNJ3EHH}},
note = {Machine review of arXiv:2506.24003}
}
read the original abstract
In this paper, we present a practical approach to improve anatomical shape accuracy in whole-body medical segmentation. Our analysis shows that a shape-focused toolkit can enhance segmentation performance by over 8%, without the need for model re-training or fine-tuning. In comparison, modifications to model architecture typically lead to marginal gains of less than 3%. Motivated by this observation, we introduce ShapeKit, a flexible and easy-to-integrate toolkit designed to refine anatomical shapes. This work highlights the underappreciated value of shape-based tools and calls attention to their potential impact within the medical segmentation community.
Figures
Reference graph
Works this paper leans on
-
[15]
Nature Methods 18(2), 203–211 (2021)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2), 203–211 (2021)
work page 2021
-
[1]
Bassi, P.R., Li, W., Tang, Y., Isensee, F., Wang, Z., Chen, J., Chou, Y.C., Kirch- hoff, Y., Rokuss, M., Huang, Z., Ye, J., He, J., Wald, T., Ulrich, C., Baumgartner, M., Roy, S., Maier-Hein, K.H., Jaeger, P., Ye, Y., Xie, Y., Zhang, J., Chen, Z., Xia, ShapeKit 11 Y., Xing, Z., Zhu, L., Sadegheih, Y., Bozorgpour, A., Kumari, P., Azad, R., Mer- hof, D., Sh...
work page 2024
-
[2]
arXiv preprint arXiv:2411.02753 (2024)
Bassi, P.R., Wu, Q., Li, W., Decherchi, S., Cavalli, A., Yuille, A., Zhou, Z.: Label critic: Design data before models. arXiv preprint arXiv:2411.02753 (2024)
arXiv 2024
-
[3]
arXiv preprint arXiv:2501.04678 (2025)
Bassi, P.R., Yavuz, M.C., Wang, K., Chen, X., Li, W., Decherchi, S., Cavalli, A., Yang, Y., Yuille, A., Zhou, Z.: Radgpt: Constructing 3d image-text tumor datasets. arXiv preprint arXiv:2501.04678 (2025)
arXiv 2025
-
[4]
arXiv preprint arXiv:2211.02701 (2022)
Cardoso, M.J., Li, W., Brown, R., Ma, N., Kerfoot, E., Wang, Y., Murrey, B., Myronenko, A., Zhao, C., Yang, D., et al.: Monai: An open-source framework for deep learning in healthcare. arXiv preprint arXiv:2211.02701 (2022)
arXiv 2022
-
[5]
Chen, Q., Chen, X., Song, H., Xiong, Z., Yuille, A., Wei, C., Zhou, Z.: Towards generalizable tumor synthesis. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (2024),https://github.com/MrGiovanni/DiffTumor
work page 2024
-
[6]
Dice,L.R.:Measuresoftheamountofecologicassociationbetweenspecies.Ecology 26(3), 297–302 (1945)
1945
-
[7]
In: 2024 IEEE International Symposium on Biomed- ical Imaging (ISBI)
Du, S., Wang, X., Lu, Y., Zhou, Y., Zhang, S., Yuille, A., Li, K., Zhou, Z.: Boosting dermatoscopic lesion segmentation via diffusion models with vi- sual and textual prompts. In: 2024 IEEE International Symposium on Biomed- ical Imaging (ISBI). pp. 1–5. IEEE (2024), https://github.com/zoedsy/ BoostingLesionSegviaDiffusionModels
work page 2024
Show all 39 references
-
[8]
Scientific Data9(1), 601 (2022)
Gatidis, S., Hepp, T., Früh, M., La Fougère, C., Nikolaou, K., Pfannenberg, C., Schölkopf, B., Küstner, T., Cyran, C., Rubin, D.: A whole-body fdg-pet/ct dataset with manually annotated tumor lesions. Scientific Data9(1), 601 (2022)
2022
-
[9]
IEEE transactions on medical imaging 37(8), 1822–1834 (2018)
Gibson,E.,Giganti,F.,Hu,Y.,Bonmati,E.,Bandula,S.,Gurusamy,K.,Davidson, B., Pereira, S.P., Clarkson, M.J., Barratt, D.C.: Automatic multi-organ segmenta- tion on abdominal ct with dense v-networks. IEEE transactions on medical imaging 37(8), 1822–1834 (2018)
2018
-
[10]
arXiv preprint arXiv:2505.04522 (2025)
Guo, P., Zhao, C., Yang, D., He, Y., Nath, V., Xu, Z., Bassi, P.R., Zhou, Z., Simon, B.D.,Harmon,S.A.,etal.:Text2ct:Towards3dctvolumegenerationfromfree-text descriptions using diffusion model. arXiv preprint arXiv:2505.04522 (2025)
2025 arXiv
-
[11]
In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Guo, P., Zhao, C., Yang, D., Xu, Z., Nath, V., Tang, Y., Simon, B., Belue, M., Harmon, S., Turkbey, B., et al.: Maisi: Medical ai for synthetic imaging. In: 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 4430–4441. IEEE (2025)
2025
-
[12]
arXiv preprint arXiv:2406.05285 (2024)
He, Y., Guo, P., Tang, Y., Myronenko, A., Nath, V., Xu, Z., Yang, D., Zhao, C., Si- mon, B., Belue, M., et al.: Vista3d: Versatile imaging segmentation and annotation model for 3d computed tomography. arXiv preprint arXiv:2406.05285 (2024)
2024 arXiv
-
[13]
Medical image analysis27, 57–71 (2016)
Heinrich, M.P., Simpson, I.J., Papież, B.W., Brady, M., Schnabel, J.A.: Deformable image registration by combining uncertainty estimates from supervoxel belief prop- agation. Medical image analysis27, 57–71 (2016)
2016
-
[14]
In: IEEE/CVF Conference on Computer Vision and Pattern Recognition
Hu, Q., Chen, Y., Xiao, J., Sun, S., Chen, J., Yuille, A.L., Zhou, Z.: Label-free liver tumor segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7422–7432 (2023), https://github.com/MrGiovanni/ SyntheticTumors 12 J. Liu et al
2023
-
[16]
arXiv preprint arXiv:2404.09556 (2024)
Isensee, F., Wald, T., Ulrich, C., Baumgartner, M., Roy, S., Maier-Hein, K., Jaeger, P.F.: nnu-net revisited: A call for rigorous validation in 3d medical image segmen- tation. arXiv preprint arXiv:2404.09556 (2024)
2024 arXiv
-
[17]
arXiv preprint arXiv:2403.06459 (2024), https://github.com/MrGiovanni/Pixel2Cancer
Lai, Y., Chen, X., Wang, A., Yuille, A., Zhou, Z.: From pixel to cancer: Cellu- lar automata in computed tomography. arXiv preprint arXiv:2403.06459 (2024), https://github.com/MrGiovanni/Pixel2Cancer
2024 arXiv
-
[18]
Biomedical Engineering/Biomedizinische Technik (0) (2024)
Li, J., Zhou, Z., Yang, J., Pepe, A., Gsaxner, C., Luijten, G., Qu, C., Zhang, T., Chen, X., Li, W., et al.: Medshapenet–a large-scale dataset of 3d medical shapes for computer vision. Biomedical Engineering/Biomedizinische Technik (0) (2024)
2024
-
[19]
arXiv preprint arXiv:2501.03410 (2025)
Li, W., Bassi, P.R., Lin, T., Chou, Y.C., Zhou, X., Tang, Y., Isensee, F., Wang, K., Chen, Q., Xu, X., et al.: Scalemai: Accelerating the development of trusted datasets and ai models. arXiv preprint arXiv:2501.03410 (2025)
2025
-
[20]
Medical Image Analysis p
Li, W., Qu, C., Chen, X., Bassi, P.R., Shi, Y., Lai, Y., Yu, Q., Xue, H., Chen, Y., Lin, X., et al.: Abdomenatlas: A large-scale, detailed-annotated, & multi- center dataset for efficient transfer learning and open algorithmic benchmark- ing. Medical Image Analysis p. 103285 (...
2024
-
[21]
Li, W., Yuille, A., Zhou, Z.: How well do supervised models transfer to 3d image segmentation? In: International Conference on Learning Representations (2024), https://github.com/MrGiovanni/SuPreM
2024
-
[22]
arXiv preprint arXiv:2412.18589 (2024), https://github.com/MrGiovanni/TextoMorph
Li, X., Shuai, Y., Liu, C., Chen, Q., Wu, Q., Guo, P., Yang, D., Zhao, C., Bassi, P.R., Xu, D., et al.: Text-driven tumor synthesis. arXiv preprint arXiv:2412.18589 (2024), https://github.com/MrGiovanni/TextoMorph
2024 arXiv
-
[23]
Medical image analysis42, 60–88 (2017)
Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Medical image analysis42, 60–88 (2017)
2017
-
[24]
In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision
Liu, J., Zhang, Y., Chen, J.N., Xiao, J., Lu, Y., A Landman, B., Yuan, Y., Yuille, A., Tang, Y., Zhou, Z.: Clip-driven universal model for organ segmenta- tion and tumor detection. In: Proceedings of the IEEE/CVF International Confer- ence on Computer Vision. pp. 21152–21164 (...
2023
-
[25]
Medical Image Analysis p
Liu, J., Zhang, Y., Wang, K., Yavuz, M.C., Chen, X., Yuan, Y., Li, H., Yang, Y., Yuille, A., Tang, Y., et al.: Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomog- raphy. Medical Image Analysis p. 103226 (202...
2024
-
[26]
IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)
Ma, J., Zhang, Y., Gu, S., Zhu, C., Ge, C., Zhang, Y., An, X., Wang, C., Wang, Q., Liu, X., et al.: Abdomenct-1k: Is abdominal organ segmentation a solved problem. IEEE Transactions on Pattern Analysis and Machine Intelligence (2021)
2021
-
[27]
arXiv preprint arXiv:2504.06897 (2025)
Mao, J., Wang, Y., Tang, Y., Xu, D., Wang, K., Yang, Y., Zhou, Z., Zhou, Y.: Med- segfactory: Text-guided generation of medical image-mask pairs. arXiv preprint arXiv:2504.06897 (2025)
2025 arXiv
-
[28]
Diagnostic and interventional imaging101(1), 35–44 (2020) ShapeKit 13
Park, S., Chu, L., Fishman, E., Yuille, A., Vogelstein, B., Kinzler, K., Horton, K., Hruban, R., Zinreich, E., Fouladi, D.F., et al.: Annotated normal ct data of the ab- domen for deep learning: Challenges and strategies for implementation. Diagnostic and interventional imagin...
2020
-
[29]
In: Conference on Neural Information Processing Systems
Qu, C., Zhang, T., Qiao, H., Liu, J., Tang, Y., Yuille, A., Zhou, Z.: Abdomenatlas- 8k: Annotating 8,000 abdominal ct volumes for multi-organ segmentation in three weeks. In: Conference on Neural Information Processing Systems. vol. 21 (2023), https://github.com/MrGiovanni/Abd...
2023
-
[30]
Nature Communications12(1), 1–15 (2021)
Reinke, A., et al.: Common limitations of performance metrics in biomedical image analysis. Nature Communications12(1), 1–15 (2021)
2021
-
[31]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedi- cal image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 234–241. Springer (2015)
2015
-
[32]
Medical Image Analysis p
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J.N., Wu, Z., Ding, X.: Embracing imperfect datasets: A review of deep learning solutions for medical image segmen- tation. Medical Image Analysis p. 101693 (2020)
2020
-
[33]
In: Proceedings of the AAAI Conference on Artificial Intelligence
Wang, L., Li, D., Liu, H., Peng, J., Tian, L., Shan, Y.: Cross-dataset collaborative learning for semantic segmentation in autonomous driving. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 2487–2494 (2022)
2022
-
[34]
medRxiv (2022)
Xia, Y., Yu, Q., Chu, L., Kawamoto, S., Park, S., Liu, F., Chen, J., Zhu, Z., Li, B., Zhou, Z., et al.: The felix project: Deep networks to detect pancreatic neoplasms. medRxiv (2022)
2022
-
[35]
arXiv preprint arXiv:2506.02327 (2025)
Yang, Y., Wang, Z.Y., Liu, Q., Sun, S., Wang, K., Chellappa, R., Zhou, Z., Yuille, A., Zhu, L., Zhang, Y.D., et al.: Medical world model: Generative simulation of tumor evolution for treatment planning. arXiv preprint arXiv:2506.02327 (2025)
2025 arXiv
-
[36]
IEEE (2024), https://github.com/MrGiovanni/ContinualLearning
Zhang, T., Chen, X., Qu, C., Yuille, A., Zhou, Z.: Leveraging ai predicted and expert revised annotations in interactive segmentation: Continual tuning or full training? In: IEEE International Symposium on Biomedical Imaging. IEEE (2024), https://github.com/MrGiovanni/Continua...
2024
-
[37]
In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Zhang, Y., Li, X., Chen, H., Yuille, A.L., Liu, Y., Zhou, Z.: Continual learning for abdominal multi-organ and tumor segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 35–45. Springer (2023), https://github.com/MrGiova...
2023
-
[38]
In: Intelligent Systems in Medicine and Health, pp
Zhou, Z., Gotway, M.B., Liang, J.: Interpreting medical images. In: Intelligent Systems in Medicine and Health, pp. 343–371. Springer (2022)
2022
-
[39]
"" def reassign_left_right_based_on_liver( right_mask: np.array, left_mask: np.array, liver_mask: np.array):
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging39(6), 1856–1867 (2019),https://github.com/ MrGiovanni/UNetPlusPlus 14 J. Liu et al. Appendix: ...
2019
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.