REVIEW 3 major objections 6 minor 26 references
Contour priors on three orthogonal CT planes raise semi-automated lesion Dice from 0.67 to 0.88 on an external set of 3865 lesions.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-31 20:52 UTC pith:ESVVZ4FV
load-bearing objection Clean, large-n ablation of prior type × orthogonal planes on nnU-Net; ranking is real under their protocol but partly baked in by GT-derived contours, and the clinical-efficiency claim is still untested. the 3 major comments →
Effect of User-Prompted Priors on Semi-Automated Cancer Lesion Segmentation in Whole-Body Computed Tomography
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
More complex spatial user priors consistently improve semi-automated cancer lesion segmentation in whole-body CT. Contour priors supplied on three orthogonal planes achieve the highest performance, reaching mean Dice 0.882 on an external test of 3865 lesions compared with 0.671 for a cropped baseline given no spatial prior. Adding each further orthogonal plane produces statistically significant gains for both bounding-box and contour priors.
What carries the argument
User-prompted spatial priors (points, tight bounding boxes, or maximal-slice contours) encoded as an extra binary channel to a 3D U-Net, supplied on one, two or three orthogonal planes inside a fixed crop around each known lesion.
Load-bearing premise
The priors were simulated from perfect ground-truth masks with only small random shifts, so the claimed gains assume real operators will draw boxes and contours of similar quality.
What would settle it
Have radiologists draw real multi-plane contours and boxes on the same lesions, then compare both Dice scores and wall-clock annotation time against the simulated-prior results and against full manual 3D delineation.
If this is right
- Multi-plane contour prompts can support efficient creation of volumetric ground-truth datasets for trials and radiomics.
- Richer priors trade operator effort for higher accuracy; simpler priors may still suffice for well-defined lesions.
- Cascade pipelines that first detect lesions then segment them with orthogonal prompts become more practical.
- Total tumour volume measurement in oncology studies becomes more feasible without full slice-by-slice manual contouring.
Where Pith is reading between the lines
- Real interactive timing studies will likely favour two-plane contours over three once operator fatigue and interface friction are counted.
- The large jump from box to contour implies that boundary shape, not merely extent, is the missing cue for low-contrast metastases.
- Because the work assumes lesions are already localised, the efficiency claim for total tumour volume still depends on a separate high-recall detector.
- Orthogonal-plane prompting should generalise to other anisotropic 3D modalities where a single plane under-specifies irregular objects.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies how the type and richness of simulated user prompts affect semi-automated 3D lesion segmentation in whole-body CT. Using nnU-Net as a fixed backbone, the authors compare no-prior (crop only), centroid point, bounding-box, and contour priors encoded as extra input channels, with box/contour priors drawn from one, two, or three orthogonal planes. Training uses 3-fold cross-validation on 906 lesions (DeepLesion3D + FDG-PET-CT-Lesions); external evaluation uses the Longitudinal CT cohort (3865 lesions, 300 melanoma patients). The main findings: richer priors monotonically improve Dice (contour-3-plane 0.882 vs crop-only 0.671 on the external set); adding orthogonal planes gives statistically significant gains (Wilcoxon, Bonferroni-corrected); and a perturbation study shows performance degrades gracefully under small random translations of the prior, with contour priors degrading more than boxes. The authors conclude multi-plane contour prompts are the most effective prior for generating volumetric ground truth efficiently.
Significance. If the results hold, this is a useful, well-controlled benchmark for a practically important problem: volumetric lesion segmentation is the bottleneck for TTV-based endpoints that correlate better with survival than RECIST. Particular strengths worth crediting: (i) the nnU-Net backbone and crop are held fixed across all conditions, so differences are attributable to the prior channel alone; (ii) the evaluation uses a large external holdout (n=3865 lesions from 300 patients) disjoint from training; (iii) the authors include a perturbation study of prior misplacement (Table 2) rather than reporting only oracle-quality prompts; (iv) significance is assessed with paired tests and multiple-comparison correction. All datasets are public, so the study is in principle reproducible. The findings quantify a practically relevant trade-off between prompt complexity and accuracy, and the observation that contour priors degrade faster than boxes under misplacement is itself informative for workflow design.
major comments (3)
- [§3.2, Table 2, §5] §3.2 / Table 2 / §5 (Conclusions): The headline recommendation — that 3-plane contour priors are the most effective prompt — rests on a prior-error model that does not match how real operator priors fail. The perturbation study translates a perfect ground-truth contour by ±1–3 voxels in-plane and ±1 voxel through-plane. Real operator contours deviate from GT boundaries systematically (over/under-segmentation, irregular boundary noise; inter-observer contour Dice is typically 0.7–0.85), not by rigid sub-voxel shifts of an exact boundary. Table 2 itself shows contour priors degrade much faster than bboxes under these shifts (0.882→0.790 vs 0.791→0.763 at the largest perturbation), i.e., the contour advantage is concentrated in precisely the regime where the error model is least realistic. To be fair, the ordering does not invert even at the largest tested shift (perturbed contour 0.790 ≈ u
- [§3.2, Fig. 2, Abstract] §3.2 / Fig. 2 / Abstract: The 'No Prior' condition is not prior-free. All lesions, including the baseline, are cropped to a 6.7 cm cube centered on the GT lesion center (99th-percentile crop). A GT-centered tight crop is itself a strong localization prior — arguably stronger than a loose user-drawn bounding box. The abstract's claim of '0.882 vs 0.671 for the baseline model with no spatial prior' therefore overstates the contrast: the comparison is 'crop-only vs crop+prompt', not 'no prior vs prior'. This matters because the magnitude of the gain (0.21 Dice) is the paper's headline number. Please relabel the baseline (e.g., 'crop-only') in Fig. 2, Table 2, the abstract, and §5, and state explicitly that absolute gains would be smaller against a less localized reference.
- [§3.4, Fig. 2] §3.4 / Fig. 2: The Wilcoxon tests treat n=906 and n=3865 lesions as independent paired samples, but the test set contains 3865 lesions from only 300 patients, and the training cohort 906 lesions from ~780 patients. Lesions within a patient (shared scanner, protocol, disease biology, annotator) are correlated, so the effective sample size is smaller and the reported p-values/CIs are anti-conservative. Given the very large effect sizes (0.05–0.2 Dice between conditions) the qualitative conclusions almost certainly stand, but this should be verified or acknowledged — e.g., cluster-robust variance estimation, a patient-level bootstrap, or a mixed-effects model with patient as a random effect. Relatedly, §3.4 states the Bonferroni correction accounts for 'the number of paired samples' (n=906 / n=3865); a Bonferroni factor should be the number of comparisons, not the number of samples. If α wa
minor comments (6)
- [§3.1, §5] The 'external' test set is external in cohort but not in cancer type: FDG-PET-CT-Lesions contributes melanoma cases to training and Longitudinal CT is entirely melanoma. §5 acknowledges this obliquely ('both contain lesions from patients suffering from melanoma') but the generalizability framing in the abstract and §5 should be tempered accordingly.
- [§4, §5] The test set outperforms cross-validation for every prior (e.g., 0.882 vs 0.830). The authors' explanation (coarse PET-derived GT in part of the training data penalizes CV scores) is plausible, but it would strengthen the paper to verify it directly — e.g., report CV scores split by dataset of origin (DeepLesion3D vs FDG-PET-CT-Lesions).
- [§3.3, Eq. (1)] Point-prior evaluation uses a GT centroid click, while training samples points uniformly within the mask (Eq. 1). The centroid is the most informative single point; a brief robustness check with off-center random points at inference would show how sensitive the point-prior condition is to click placement, which is the realistic operator variable.
- [§2, §4] No comparison to contemporary promptable baselines (nnInteractive, MedSAM/SegVol, ONCOPILOT) is provided, even though these are cited in §2 and ONCOPILOT reports numbers on related data. A single baseline run of nnInteractive on the test set would substantially strengthen the claim that the proposed prior-encoding scheme is competitive; at minimum, the absence of such a comparison should be stated as a limitation.
- [Fig. 2] Fig. 2 error bars are described as 95% CIs of the per-lesion Dice distribution; given n=3865 these are vanishingly small and add little. Patient-level or fold-level variability would be more informative. Also consider reporting median Dice and a small-lesion subgroup, since mean Dice can be driven by large, easy lesions.
- [§2, §3.2, §5, Table 1] Notation/typos: 'an mean Dice' (§2); 'but that the optimal prior may, in practice, depend...' has a stray comma (§5); Table 1's voxel-size row is hard to parse (PET and CT resolutions run together). The crop side length of 6.7 cm is given without the corresponding voxel dimensions of the resampled crop, which would help reproducibility.
Circularity Check
Standard GT-simulated interactive priors; Dice ranking is empirically measured, not algebraically forced by a fit or self-definition.
specific steps
-
other
[§3.2 User-Prompted Priors; also Table 2 perturbation]
"Both the bounding box and contour priors were generated from the ground-truth lesion masks by identifying the slice with the largest lesion cross-section. ... For the bounding boxes, the smallest possible fitting rectangle over the original lesion mask was extracted. For the contour, the slice with the maximal area of the lesion mask itself was extracted along the specified axis."
The spatial priors used as model inputs are direct extractions (tight box or exact maximal-area mask slice) from the same GT masks against which Dice is scored. Richer priors therefore inject strictly more of the evaluation target as an input channel, so the monotonic ranking contour > bbox > point > none is strongly favored by construction of the independent variable. This is ordinary interactive-seg simulation, not a fitted-constant ‘prediction,’ and residual error plus perturbation drops show it is not fully tautological—but it is the only mild circularity-adjacent step.
full rationale
This is an empirical nnU-Net comparison of prompt types on held-out public CT lesion data. The central numbers (mean Dice 0.882 for 3-plane contours vs 0.671 for cropped no-prior on n=3865 external lesions; Wilcoxon gains from adding planes) are measured outputs of trained models, not rearrangements of fitted constants or self-cited uniqueness theorems. Priors are explicitly simulated from ground-truth masks (largest-slice contour / tight box, plus small translations), which is disclosed standard practice in promptable segmentation and creates expected information-content ordering, but does not make Dice equal the prior by construction: the network must still propagate 2D cues into full 3D volumes, and imperfect Dice (well below 1.0) plus larger contour degradation under perturbation show the result is not tautological. No self-citation chain, ansatz smuggling, or renamed known law carries the claim. Mild residual concern only: the independent variable is carved from the same GT used for scoring, so the experiment measures ‘how much GT slice information helps’ more than fully realistic operator error; that is a disclosed limitation, not circular derivation. Score 2.
Axiom & Free-Parameter Ledger
free parameters (3)
- crop_side_length_percentile =
99th percentile ≈ 6.7 cm
- point_prior_Poisson_lambda =
λ=0.5, offset 1
- prior_translation_ranges =
train ±3x/y, ±1z
axioms (5)
- domain assumption Lesion center/location is known a priori so fixed cropping around the lesion is valid and surrounding far anatomy can be discarded.
- ad hoc to paper Ground-truth-derived largest-slice contours/boxes plus small random translations adequately represent user-prompted priors for ranking methods.
- domain assumption Holding the nnU-Net self-configuring backbone fixed isolates the effect of prior type and plane count.
- domain assumption Dice coefficient on external manual radiologist contours is a sufficient primary metric for semi-automated ground-truth utility.
- domain assumption Manually PET-aligned, slice-interpolated FDG-PET-CT lesion labels are acceptable CT training targets despite coarser native PET spacing.
read the original abstract
In clinical oncology studies, metastatic cancer is commonly evaluated using "Response Evaluation Criteria in Solid Tumors" (RECIST), in which the diameter of up to five lesions is measured and followed over the course of treatment. However, RECIST shows limited correlation with overall survival. Total tumour volume (TTV) is a stronger predictor but typically relies on manual ground-truth segmentation of all lesions, which is time-consuming and requires expert domain knowledge. Semi-automated approaches leveraging user-prompted priors, such as bounding boxes and single-slice contours, as inputs to automated segmentation methods can facilitate the generation of ground-truth segmentations. This work investigates the impact of different user-prompted priors on semi-automated cancer lesion segmentation performance in whole-body computed tomography. Across 3-fold cross-validation and external testing, more complex spatial priors consistently improved performance, with contour priors from three orthogonal planes (axial, coronal and sagittal) achieving the best results. On the external test (n=3865 lesions), this approach achieved a mean Dice score of 0.882, compared to a mean Dice score of 0.671 for the baseline model with no spatial prior. These findings suggest that the use of multi-plane orthogonal user-prompted priors can improve semi-automated tumour lesion segmentation and support efficient generation of high-quality volumetric ground-truth data.
Figures
Reference graph
Works this paper leans on
-
[1]
World Journal of Clinical Oncology 11(2), 53–73 (Feb 2020)
Aykan, N.F., Özatlı, T.: Objective response rate assessment in oncology: Current situation and future expectations. World Journal of Clinical Oncology 11(2), 53–73 (Feb 2020). https://doi.org/10.5306/wjco.v11.i2.53
-
[2]
CA: A Cancer Journal for Clinicians 74(3), 229–263 (2024)
Bray, F., Laversanne, M., Sung, H., Ferlay, J., Siegel, R.L., Soerjomataram, I., Jemal, A.: Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: A Cancer Journal for Clinicians 74(3), 229–263 (2024). https://doi.org/10.3322/caac.21834 10 I. Stark et al
-
[3]
Medical Image Analysis 102, 103525 (May 2025)
de Grauw, M.J.J., Scholten, E.Th., Smit, E.J., Rutten, M.J.C.M., Prokop, M., van Ginneken, B., Hering, A.: The ULS23 challenge: A baseline model and benchmark dataset for 3D universal lesion segmentation in computed tomography. Medical Image Analysis 102, 103525 (May 2025). https://doi.org/10.1016/j.media.2025. 103525
-
[4]
American Journal of Cancer Research 11(4), 1121–1131 (Apr 2021)
Delgado, A., Guddati, A.K.: Clinical endpoints in oncology - a primer. American Journal of Cancer Research 11(4), 1121–1131 (Apr 2021)
2021
-
[5]
https://doi.org/10.48550/arXiv.2311
Du, Y., Bai, F., Huang, T., Zhao, B.: SegVol: Universal and Interactive Volumet- ric Medical Image Segmentation (Feb 2025). https://doi.org/10.48550/arXiv.2311. 13385
-
[6]
European Journal of Cancer 45(2), 228–247 (Jan 2009)
Eisenhauer, E.A., Therasse, P., Bogaerts, J., Schwartz, L.H., Sargent, D., Ford, R., Dancey, J., Arbuck, S., Gwyther, S., Mooney, M., Rubinstein, L., Shankar, L., Dodd, L., Kaplan, R., Lacombe, D., Verweij, J.: New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). European Journal of Cancer 45(2), 228–247 (Jan 2009). h...
-
[7]
Cancer Imaging 13(1), 140–149 (Mar 2013)
Ganeshan, B., Miles, K.A.: Quantifying tumour heterogeneity with CT. Cancer Imaging 13(1), 140–149 (Mar 2013). https://doi.org/10.1102/1470-7330.2013.0015
arXiv 2013
-
[8]
Nature Machine Intelligence 6(11), 1396–1405 (Nov 2024)
Gatidis, S., Früh, M., Fabritius, M.P., Gu, S., Nikolaou, K., Fougère, C.L., Ye, J., He, J., Peng, Y., Bi, L., Ma, J., Wang, B., Zhang, J., Huang, Y., Heiliger, L., Marinov, Z., Stiefelhagen, R., Egger, J., Kleesiek, J., Sibille, L., Xiang, L., Bendazzoli, S., Astaraki, M., Ingrisch, M., Cyran, C.C., Küstner, T.: Results from the autoPET challenge on full...
2024
-
[9]
Journal of Medical Imaging 5(1), 011005 (Jan 2018)
Huang, Q., Lu, L., Dercle, L., Lichtenstein, P., Li, Y., Yin, Q., Zong, M., Schwartz, L., Zhao, B.: Interobserver variability in tumor contouring affects the use of ra- diomics to predict mutational status. Journal of Medical Imaging 5(1), 011005 (Jan 2018). https://doi.org/10.1117/1.JMI.5.1.011005
-
[10]
Communications Medicine 3(1), 76 (May 2023)
Hudock, N.L., Mani, K., Khunsriraksakul, C., Walter, V., Nekhlyudov, L., Wang, M., Lehrer, E.J., Hudock, M.R., Liu, D.J., Spratt, D.E., Zaorsky, N.G.: Future trends in incidence and long-term survival of metastatic cancer in the United States. Communications Medicine 3(1), 76 (May 2023). https://doi.org/10.1038/ s43856-023-00304-x
2023
-
[11]
Nature Methods 18(2), 203–211 (Feb 2021)
Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU- Net: A self-configuring method for deep learning-based biomedical image seg- mentation. Nature Methods 18(2), 203–211 (Feb 2021). https://doi.org/10.1038/ s41592-020-01008-z
2021
-
[12]
https://doi.org/10.48550/arXiv.2503.08373
Isensee, F., Rokuss, M., Krämer, L., Dinkelacker, S., Ravindran, A., Stritzke, F., Hamm, B., Wald, T., Langenberg, M., Ulrich, C., Deissler, J., Floca, R., Maier- Hein, K.: nnInteractive: Redefining 3D Promptable Segmentation (Mar 2025). https://doi.org/10.48550/arXiv.2503.08373
-
[13]
https://doi.org/10.48550/arXiv.2304.02643
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., Dollár, P., Girshick, R.: Segment Anything (Apr 2023). https://doi.org/10.48550/arXiv.2304.02643
-
[14]
Korn, R.L., Crowley, J.J.: Overview: Progression-Free Survival as an Endpoint in Clinical Trials with Solid Tumors. Clinical cancer research : an official journal of the American Association for Cancer Research 19(10), 2607–2612 (May 2013). https://doi.org/10.1158/1078-0432.CCR-12-2934
-
[15]
Küstner, T., Peisen, F., Gatidis, S., Wagner, A., Megne, O., Othman, A., Sanner, A., Loßau, T., Moltz, J.H., Kohlbrandt, T., Hering, A.: Longitudinal-CT (Sep 2025). https://doi.org/10.57754/FDAT.qwsry-7t837 User-Prompted Priors for Semi-Automated Cancer Lesion Segmentation 11
-
[16]
Nature Communications 15(1), 654 (Jan 2024)
Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B.: Segment anything in medical images. Nature Communications 15(1), 654 (Jan 2024). https://doi.org/10.1038/ s41467-024-44824-z
2024
-
[17]
npj Precision Oncology 9(1), 121 (Apr 2025)
Machado, L., Alberge, L., Philippe, H., Ferreres, E., Khlaut, J., Dupuis, J., Le Floch, K., Habip Gatenyo, D., Roux, P., Grégory, J., Ronot, M., Dancette, C., Boeken, T., Tordjman, D., Manceron, P., Hérent, P.: A promptable CT foun- dation model for solid tumor evaluation. npj Precision Oncology 9(1), 121 (Apr 2025). https://doi.org/10.1038/s41698-025-00903-y
-
[18]
Clinical and Translational Oncol- ogy 27(11), 4091–4102 (Nov 2025)
Mariotti, F., Agostini, A., Borgheresi, A., Marchegiani, M., Zannotti, A., Gia- comelli, G., Pierpaoli, L., Tola, E., Galiffa, E., Giovagnoni, A.: Insights into ra- diomics: A comprehensive review for beginners. Clinical and Translational Oncol- ogy 27(11), 4091–4102 (Nov 2025). https://doi.org/10.1007/s12094-025-03939-5
-
[19]
Journal of Clinical Oncology 41(15), 2706–2712 (May 2023)
Merino, M., Kasamon, Y., Theoret, M., Pazdur, R., Kluetz, P., Gormley, N.: Ir- reconcilable Differences: The Divorce Between Response Rates, Progression-Free Survival, and Overall Survival. Journal of Clinical Oncology 41(15), 2706–2712 (May 2023). https://doi.org/10.1200/JCO.23.00225
-
[20]
eClinicalMedicine 76 (Oct 2024)
Mittal, A., Kim, M.S., Dunn, S., Wright, K., Gyawali, B.: Frequently asked questions on surrogate endpoints in oncology-opportunities, pitfalls, and the way forward. eClinicalMedicine 76 (Oct 2024). https://doi.org/10.1016/j.eclinm.2024. 102824
-
[21]
Seminars in Radiation Oncology 31(3), 186–199 (Jul 2021)
Pirasteh, A., Lovrec, P., Pedrosa, I.: Imaging and its Impact on Defining the Oligometastatic State. Seminars in Radiation Oncology 31(3), 186–199 (Jul 2021). https://doi.org/10.1016/j.semradonc.2021.03.006
-
[22]
Annals of Oncology 27(3), 373–378 (Mar 2016)
Saad, E.D., Buyse, M.: Statistical controversies in clinical research: End points other than overall survival are vital for regulatory approval of anticancer agents. Annals of Oncology 27(3), 373–378 (Mar 2016). https://doi.org/10.1093/annonc/ mdv562
doi:10.1093/annonc/ 2016
-
[23]
BMC Medical Imaging 24(1), 241 (Sep 2024)
Sun, J., Chen, K., He, Z., Ren, S., He, X., Liu, X., Peng, C.: Medical image analysis using improved SAM-Med2D: Segmentation and classification perspec- tives. BMC Medical Imaging 24(1), 241 (Sep 2024). https://doi.org/10.1186/ s12880-024-01401-6
2024
-
[24]
In: Del Bue, A., Canton, C., Pont-Tuset, J., Tommasi, T
Wang, H., Guo, S., Ye, J., Deng, Z., Cheng, J., Li, T., Chen, J., Su, Y., Huang, Z., Shen, Y., Fu, B., Zhang, S., He, J., Qiao, Y.: SAM-Med3D: Towards General- Purpose Segmentation Models for Volumetric Medical Images. In: Del Bue, A., Canton, C., Pont-Tuset, J., Tommasi, T. (eds.) Computer Vision – ECCV 2024 Workshops. pp. 51–67. Springer Nature Switzerl...
-
[25]
Journal of Medical Imaging 5(3), 036501 (Jul 2018)
Yan, K., Wang, X., Lu, L., Summers, R.M.: DeepLesion: Automated mining of large-scale lesion annotations and universal lesion detection with deep learning. Journal of Medical Imaging 5(3), 036501 (Jul 2018). https://doi.org/10.1117/1. JMI.5.3.036501
doi:10.1117/1 2018
-
[26]
Zeeuw, J.M., Kemna, R., Ali, M., van Eck, S., Wesdorp, N.J., Roor, J., van Waes- berghe, J.H.T.M., van den Bergh, J.E., Nota, I.M.G.C., Moos, S.I., van Dieren, S., van Amerongen, M.J., Bond, M.J.G., Chapelle, T., van Dam, R.M., Engel- brecht, M.R.W., Gerhards, M.F., Grunhagen, D.J., van Gulik, T.M., Hermans, J.J., de Jong, K.P., Klaase, J.M., Kok, N.F.M.,...
arXiv 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.