REVIEW 3 major objections 6 minor 4 references
Kidney and Kidney Tumor Segmentation using a Logical Ensemble of U-nets with Volumetric Validation
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A logical ensemble of three U-nets plus volumetric checks segments kidneys and tumors with average F1 0.6758.
desk verdict A clean, reproducible KiTS19 challenge write-up whose headline F1 depends on an unspecified train/test split that may leak same-patient slices into the test set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a logical ensemble of three 2D U-nets followed by volumetric validation. U-net, a fully convolutional encoder–decoder with skip connections, is used as the base segmenter; the ensemble combines two kidney-and-tumor models (one lax, one strict) with a tumor-only model using bitwise AND, so a pixel is labeled tumor only if all relevant models agree. Volumetric validation then stacks the 2D masks per patient and applies hand-set thresholds on volume, position, morphology, and sphericity to reject or relabel false regions, using the assumption that kidney and tumor anatomy is consistent across nearby slices.
What would settle it
Map predicted masks back through the inverse of the preprocessing pipeline and recompute precision, recall, and F1 against the original KiTS19 ground-truth segmentations; since the paper reports the score drops on restoration, the size of that drop directly tests whether the 0.6758 average is an artifact of evaluation space.
Extended reading notes
Core claim
The central claim is that logical ensembling of 2D U-nets with a post hoc 3D validation step yields automatic kidney and tumor segmentations whose average F1 is 0.6758 on the KiTS19 test split. After volumetric validation, the reported per-class metrics are: background precision 0.9992/recall 0.9975/F1 0.9983; kidney precision 0.8365/recall 0.8723/F1 0.8497; tumor precision 0.3964/recall 0.8623/F1 0.5019. The ensemble works by AND-ing the outputs of a lax and a strict kidney-and-tumor model, then AND-ing with a tumor-only model, and adding the masks so tumors take value 2 and kidneys value 1. Volumetric validation then applies empirical inclusion criteria—kidney volume above 19,000 normalized units, tumor volume above 350 units, sphericity above 0.29, position limits, and multi-frame consistency—to remove or relabel implausible regions. The paper argues that over-predicting tumors at the cost of precision is acceptable for cancer screening because false negatives are the more dangerous error, but it concedes the performance is below clinical deployment level.
Load-bearing premise
The headline F1 is computed on preprocessed images and ground-truth labels transformed by the same pipeline; the paper itself says restoring the original format lowers the score, so the claim assumes the transformed evaluation reflects true performance on native CT scans.
Editorial extensions
If this is right
- Volumetric validation raises average segmentation performance compared with simply stacking 2D predictions, despite the tumor class losing some recall.
- Kidney regions are located with precision and recall above 0.8, suggesting the pipeline is reliable at the organ level even where tumor boundaries are uncertain.
- The high-recall, low-precision tumor profile is appropriate for a screening pipeline in which a radiologist reviews flagged candidate regions rather than scanning the full volume.
- Further gains would have to come from better artifact removal, more training data, or a multi-class network, since the paper finds no clear volume–performance relationship to target.
Reading between the lines
- If the preprocessing-inverse drop is large, the volumetric thresholds may have been tuned to the evaluation space; testing on an external CT cohort would show whether the 0.6758 average holds on native scans.
- The authors leave the human-review cost unmeasured; a concrete next experiment would time radiologists on flagged volumes to see whether the screening workflow actually saves effort.
- The logical AND of three binary models means tumor labels require agreement across models; a single multi-class network could produce a different precision–recall trade-off and might be compared directly on the same test split.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript describes a pipeline for kidney and tumor segmentation in CT scans from the KiTS19 challenge, consisting of three binary 2D U-net classifiers (two kidney-and-tumor models and one tumor-only model), a logical ensemble rule, and heuristic volumetric validation in 3D. The authors preprocess the images with a series of computer-vision operations, train the U-nets on 256x256 frames, and post-process the stacked segmentations by rejecting kidney regions below a volume threshold or outside the central scan depth and by confirming tumor regions based on volume, morphology, consistency, and sphericity. They report precision, recall, and F1 scores for background, kidney, and tumor on a 10% test split, with a mean kidney/tumor F1 of 0.6758 after volumetric validation, and they note that this score is computed on preprocessed images and annotations rather than original-format data.
Significance. If the evaluation were sound, this would be a useful and reproducible baseline for 2D U-net ensembles with volumetric post-processing on KiTS19. The manuscript explicitly provides code, a full description of the hyperparameters, and a clear rule-based post-processing stage, which are strengths. However, the quantitative claim (mean F1 = 0.6758) currently rests on an evaluation protocol that is insufficiently specified or potentially biased: the training/test split is not stated at the patient level, the metrics are computed on preprocessed ground truth, and the volumetric validation thresholds appear to be empirically tuned on the same dataset. These issues materially affect the confidence in the reported score as a measure of generalization to new patients.
major comments (3)
- [§2.3] The description of the data split states only that "an 80:10:10 split was performed to separate data into training, validation, and test sets" for each model, without specifying whether the split unit is a patient or an individual frame. Because the dataset consists of 45,424 frames from 210 patients, a per-frame split would place highly correlated adjacent slices from the same patient into both training and test sets, and tumor-bearing frames for a given case form contiguous blocks. Such leakage would inflate the reported test-set F1 values in Table 1 and would not reflect generalization to new patients, which is the stated clinical goal and the KiTS19 benchmark setting. The authors must specify the split granularity; if the split was per-frame, they must re-evaluate with a patient-exclusive split and report the resulting scores.
- [§2.5] The evaluation is explicitly computed on "pre-processed ground truth labels from the KiTS19 dataset," and the abstract states that restoring to the original image format reduces the score. Since the preprocessing pipeline resizes, crops, and zooms images, the reported metrics do not directly quantify segmentation quality on the native KiTS19 volumes. The headline F1 score of 0.6758 is therefore not comparable to other published KiTS19 results without additional information. The authors should report the quantitative score after mapping predictions back to the original image format, or clearly state the magnitude of the reduction mentioned in the abstract.
- [§2.4] The volumetric validation parameters (kidney volume above 19,000 normalized units, center-of-mass between 20% and 80% depth, tumor volume above 350, tumor sphericity above 0.29, and axis-length thresholds) are described as derived from "basic anatomical knowledge and empirical evidence." If these thresholds were selected using the same test split on which the final F1 is computed, the reported scores incorporate tuning-on-the-test-set bias. The manuscript should describe how the thresholds were chosen (e.g., from the validation split) and confirm that they were fixed before evaluating the test set.
minor comments (6)
- [§2.3] The statement that each model was trained for "ten iterations" is ambiguous; it should be clarified as ten epochs, because deep U-net training for only ten epochs is an unusual choice that may affect reproducibility.
- [§2.3] The reported sample size of 19,479 images for the strict kidney-and-tumor model is inconsistent with the stated 4:1 ratio: combining 16,356 kidney-containing frames with one quarter of the 29,068 non-kidney frames would give approximately 23,623 images. Please reconcile the numbers or clarify the sampling scheme.
- [§2.4] The units used for volumetric and morphological thresholds (e.g., "19,000 normalized units," "350 normalized units," "major axis length > 10") are not defined. The authors should state whether these refer to voxel counts and lengths at the 256x256 resolution, and whether slice spacing is taken into account.
- [§2.5] Equations (1) and (2) contain the typo "Flase Positive" in the denominator; this should be corrected to "False Positive."
- [§4] The discussion references "Figure 5" when describing the relationship between volume and performance, but the corresponding plots appear in Figure 4 (a-c). The cross-reference should be corrected.
- [§2.5] The paper should explicitly state the definition of the "average F1 score": from Table 1, the reported 0.6758 is the mean of the kidney F1 (0.8497) and tumor F1 (0.5019). This computation should be stated in the text.
Circularity Check
No circularity: the reported F1 is an empirical benchmark measurement, not a derived quantity.
full rationale
The paper does not claim to derive segmentation performance from its own inputs or from a first-principles model. It trains three 2D U-net models on KiTS19 frames, combines their outputs with logical AND operations, applies hand-specified volumetric validation rules, and reports measured precision, recall, and F1 on a held-out split using Equations (1)-(3). No equation defines the reported F1 in terms of the training data or validation thresholds, and no fitted parameter is renamed as a prediction. The volumetric-validation thresholds in Section 2.4 are decision rules described as based on anatomical knowledge and empirical evidence, but the paper does not state that they were optimized against the test set, so no specific circular reduction can be exhibited. The use of preprocessed ground-truth labels in Section 2.5 is a clear methodological limitation, and the unspecified granularity of the 80:10:10 split in Section 2.3 is a potential external-validity concern, but neither makes the evaluation equivalent to its input by construction. There are no self-citations, no imported uniqueness theorems, no ansatz smuggled in by citation, and no renaming of a known result. Under the hard rules requiring a quoted, exhibit-able reduction, the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- Volumetric validation thresholds =
kidney volume > 19000 units, center-of-mass depth 20-80%, frames > 1; tumor volume > 350 units, major axis > 10, minor…
- Preprocessing kernel sizes =
median 9x9, mean 15x15, morphological opening 99x99, histogram equalization, Otsu thresholding, smoothing polynomial…
- Training and sampling hyperparameters =
learning rate 1e-4, beta1=0.9, beta2=0.99, batch size 12, 10 training iterations, dropout 0.5, strict model includes…
- Weighted cross-entropy class weights =
not specified
assumptions (4)
- domain assumption KiTS19 ground truth segmentations are correct and clinically meaningful.
- domain assumption Preprocessed images and transformed labels are a valid evaluation space equivalent to native CT resolution.
- ad hoc to paper The 80:10:10 split is representative and leakage-free for the volumetric validation parameters.
- domain assumption Standard U-net training on 2D slices captures 3D anatomy adequately for post-processing to fix errors.
Cite this review
Pith. "Pith review of Kidney and Kidney Tumor Segmentation using a Logical Ensemble of U-nets with Volumetric Validation." pith.science (2026). https://pith.science/paper/43F7R5F3
@misc{pith2026190802625,
author = {Pith},
title = {Pith review of: Kidney and Kidney Tumor Segmentation using a Logical Ensemble of U-nets with Volumetric Validation},
year = {2026},
howpublished = {\url{https://pith.science/paper/43F7R5F3}},
note = {Machine review of arXiv:1908.02625}
}
read the original abstract
Automated medical image segmentation is a priority research area for computational methods. In particular, detection of cancerous tumors represents a current challenge in this area with potential for real-world impact. This paper describes a method developed in response to the 2019 Kidney Tumor Segmentation Challenge (KiTS19). Axial computed tomography (CT) scans from 210 kidney cancer patients were used to develop and evaluate this automatic segmentation method based on a logical ensemble of fully-convolutional network (FCN) architectures, followed by volumetric validation. Data was pre-processed using conventional computer vision techniques, thresholding, histogram equalization, morphological operations, centering, zooming and resizing. Three binary FCN segmentation models were trained to classify kidney and tumor (2), and only tumor (1), respectively. Model output images were stacked and volumetrically validated to produce the final segmentation for each patient scan. The average F1 score from kidney and tumor pixel classifications was calculated as 0.6758 using preprocessed images and annotations; although restoring to the original image format reduced this score. It remains to be seen how this compares to other solutions.
Figures
Reference graph
Works this paper leans on
-
[1]
Heller, N., Sathianathen, N., Kalapara, A., Walczak, E., Moore, K., Kaluzniak, H., Rosenberg, J., Blake, P., Rengel, Z., Oestreich, M., Dean, J., Tradewell, M., Shah, A., Tejpaul, R., Edgerton, Z., Peterson, M., Raza, S., Regmi, S., Papanikolopoulos, N., Weight, C.: The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segme...
work page 2019
-
[2]
Ronneberger, O., Fischer, P., Brox, T.: U -net: Convolutional networks for biomedical image segmentation. In: Le cture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). pp. 234 – 241 (2015). https://doi.org/10.1007/978-3-319-24574-4_28
-
[3]
Zhixuhao: Code: unet for image segmentation, https://github.com/zhixuhao/unet
-
[4]
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U -net: Learning dense volumetric segmentation from sparse annotation. Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bio informatics). 9901 LNCS, 424–432 (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.