REVIEW 4 major objections 4 minor 13 references
Adding clicks steadily lifts PET/CT lesion segmentation
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
Combining tracer classification, organ supervision, and stochastic click sampling makes an nnU-Net model segment PET/CT lesions robustly without guidance and progressively better with clicks.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection A transparent autoPET IV methods paper with a sensible stochastic-click curriculum and honest caveats, but the interactive-guidance claim is supported only by simulated clicks on training-seen data. the 4 major comments →
autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim, stated in the conclusion, is that the pipeline 'exhibits robust performance with sparse human guidance and a proficient ability to leverage incremental human feedback to refine segmentation accuracy.' In concrete terms: with zero clicks the unified model reaches a Dice score around 0.79 and the PSMA model around 0.62, and adding one to ten clicks raises Dice monotonically while false-positive and false-negative volumes fall. The mechanism is the training curriculum, not a new network: the same residual-encoder U-Net backbone is exposed to randomly sampled amounts of guidance, so it cannot afford to ignore the image and also learns to exploit clicks when they arrive.
What carries the argument
Stochastic click sampling curriculum: during training, each sample receives a random integer k between 0 and 10, and only the first k simulated foreground and k background clicks are fed to the network as 3D Gaussian heatmaps concatenated with CT and PET. This forces one model to cover the whole interaction spectrum. Supporting machinery: an upstream tracer classifier selects FDG- or PSMA-specific weights; an auxiliary organ-segmentation head supplies tracer-invariant anatomical priors using pseudo-labels from a whole-body CT segmenter; and SUV thresholding removes low-uptake false positives at inference.
Load-bearing premise
The load-bearing premise is that the challenge's simulated clicks, generated as 3D Gaussian maps by a provided script, faithfully represent how real human users would click in clinical practice; if real clicks differ in location, timing, or count, the measured benefit of more clicks may not transfer.
What would settle it
Collect a set of PET/CT cases with real radiologist click streams (foreground and background clicks placed on actual lesions and false positives) and run the submitted pipeline with the same click counts used in the challenge. If Dice does not rise monotonically as real clicks increase, or if the zero-click baseline drops when real clicks are sparse, the claim that the pipeline leverages incremental human feedback is falsified.
If this is right
- Dense-only training is not enough: a model trained only on 10-click examples scores 0.0 Dice at 0 clicks, so any deployment that expects occasional human correction must train with sparse examples too.
- A balanced stochastic sampling distribution yields a model whose accuracy rises nearly monotonically with click count, meaning incremental human feedback can be converted directly into segmentation gains.
- Organ supervision and pre-trained anatomical priors anchor a strong zero-click baseline, so adding interactive guidance improves an already usable automatic result rather than rescuing a failed one.
- Because different models dominate in different regimes, hybrid inference—using the sparse-trained model for 0–4 clicks and the dense-trained model for 5–10—gives the best of both, a strategy that generalizes to any interactive pipeline.
Where Pith is reading between the lines
- The challenge simulates clicks as 3D Gaussian maps; real radiologists may click different locations, numbers, or orders. A natural test is to compare this curriculum against supervision with actual click streams, and to add click-location jitter during training for robustness.
- The paper evaluates on cases already seen during training, so reported gains are an upper bound; an independent test set would be needed to confirm the monotonic improvement transfers.
- Organ supervision appears to shrink false-positive volume, likely by suppressing high physiological uptake in organs; this suggests the auxiliary head could be pruned or distilled after training to save computation without losing the anatomical prior.
- The hybrid model-selection rule (choose network by click count) could be extended to a confidence-based router that learns when to switch, rather than a fixed threshold.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper describes an automated and interactive lesion-segmentation pipeline for the autoPET IV challenge, built on nnU-Net Residual Encoder. The authors integrate tracer classification using public weights, organ supervision via an auxiliary head with TotalSegmentator pseudo-labels, two ways of injecting simulated click guidance (full-guidance and stochastic click sampling), and SUV-threshold post-processing. They develop three PSMA-specific models (V0, V1, V2) and two unified multi-tracer models (V3, V4), evaluate them on 5-fold cross-validation and on interactive evaluations with 0-10 simulated clicks, and select V2 for PSMA and a hybrid V3/V4 strategy for FDG. The central claim is that the pipeline is robust with sparse human guidance and effectively uses incremental human feedback to improve segmentation accuracy.
Significance. If the central claim were fully supported, the work would be a useful system description for the autoPET IV interactive segmentation challenge, particularly the stochastic click-sampling curriculum and the hybrid deployment strategy. The authors make their code and trained weights publicly available, provide detailed training configurations, and build on publicly documented pre-trained models, which are strengths for reproducibility. However, the current evidence does not yet establish the claimed generalizable interactive capability: the interactive evaluations are performed on training-seen cases, the simulated click protocol is not characterized, and the unified-model results are based on incomplete folds without error bars. The paper is more convincingly a challenge methods report than a validated clinical human-in-the-loop study.
major comments (4)
- [Section 3.1 / Table 2] The interactive evaluation for PSMA models is performed on 100 cases 'that had already been seen by the models during training.' Table 2 therefore reports in-sample segmentation performance, not generalization. The monotonic Dice improvement with increasing clicks could reflect memorization of training cases rather than the model's ability to use human guidance on unseen data. This directly affects the conclusion in Section 5 that the pipeline 'exhibits robust performance with sparse human guidance' and 'leverages incremental human feedback.' The authors should provide results on a held-out split or the challenge test set, or explicitly reframe Tables 2-3 as development-set diagnostics without claiming generalizable interactive performance.
- [Section 2.1] The manuscript states that 10 foreground and 10 background clicks are 'simulated and modeled as 3D Gaussian maps using the provided script,' but it never describes how click positions are generated. If the script samples clicks from ground-truth lesion masks or from tracer-avid regions, the improvement with more clicks is an oracle effect rather than a demonstration of human-in-the-loop refinement. A concrete test is needed: report the click-position generation protocol, compare the simulated distribution with human click behavior, or evaluate with clicks derived from the model's current errors. Without this, the interactive claim is not transferable to clinical use.
- [Section 3.2 / Table 3] The unified-model results are based on only three folds for V3 and two folds for V4, and neither Table 2 nor Table 3 reports error bars or per-fold variability. The differences between V3 and V4 are small—for example, Dice at 4 clicks is 0.869 vs. 0.865, and at 10 clicks 0.889 vs. 0.877—so the proposed hybrid selection rule may be within fold noise. The authors should report per-fold results, standard deviations, and ideally a statistical comparison before using these numbers to choose between models in the final submission.
- [Section 3.2 / Final Submission] The hybrid-strategy description is internally inconsistent: it reads 'Model V4 would be utilized in early interactive phase (0-4 clicks), while model V4 would be chosen in densely guided steps (5-10 clicks),' which repeats V4 and cannot be the intended rule. Table 3 also does not clearly support a 0-4 threshold: at 3 clicks V4 is only 0.001 Dice above V3, and at 4 clicks V3 is above V4. Additionally, the conclusion credits 'anatomical prior and organ supervision' for the unified model's behavior, but no ablation removes organ supervision from V3/V4, so its causal role is not demonstrated. The authors should correct the typo, justify the click-count threshold with the reported data, and soften or support the attribution to organ supervision.
minor comments (4)
- [Section 2.3] There are minor typos: 'Channnel 0' should be 'Channel 0'; 'We adopted' has an inconsistent capital 'W' mid-sentence.
- [Section 3.2] The sentence describing the hybrid strategy repeats 'model V4' for both the early and dense phases; the second instance should presumably be 'model V3.'
- [Table 4] Table 4 is difficult to read as a formatted row; consider restructuring into a normal table with labeled columns instead of merged cells.
- [References] The citations are adequate for a challenge report, but the paper would benefit from citing the official autoPET IV challenge description or website to clarify the evaluation protocol and the click-simulation script.
Circularity Check
No circularity: empirical challenge report with independent external priors and measured metrics.
full rationale
This paper is an empirical challenge submission rather than a derivation, so none of its central claims reduce to its inputs by construction. The pipeline components (tracer classification, organ supervision, stochastic click sampling) are described as training strategies, and the results in Tables 1-3 are measured Dice/FPV/FNV values, not fitted quantities relabeled as predictions. The pre-trained weights used for the tracer classifier and the anatomical-prior model come from public sources by other teams ([8], [9]) and from TotalSegmentator ([11]); none of these are self-citations by the present authors. The stochastic click sampling curriculum is an input design, but the reported click-vs-performance trends are empirical outcomes on fixed validation sets, not mathematically forced by the sampling distribution. The paper also explicitly acknowledges the main limitation: Section 3.1 states that all evaluated PSMA cases 'had already been seen by the models during training, so no guarantee of final test performance can be made.' Similarly, the use of script-generated 3D Gaussian clicks is a fidelity concern about generalization to real human interaction, but it is not circular reasoning. No equation is defined in terms of a target result, no fitted parameter is renamed as a prediction, and no load-bearing claim rests on the authors' own prior work. The honest non-finding therefore applies: the derivation chain is self-contained in the sense that all reported quantities are measured outcomes with external priors, and the acknowledged limitations concern external validity rather than circularity.
Axiom & Free-Parameter Ledger
free parameters (6)
- SUV threshold (FDG) =
1.5
- SUV threshold (PSMA) =
1.0
- Dice:CE loss ratio (Model V0) =
2:1
- Click sampling distribution (Model V1) =
40% for 0 clicks, 20% for 1 click, remainder unspecified
- Click sampling distribution (Model V4) =
[0.10, 0.10, 0.10, 0.08, 0.04, 0.04, 0.04, 0.04, 0.08, 0.08, 0.30]
- Training hyperparameters (patch, batch, epochs, LR) =
patch [192,192,192], batch 2-3, epochs 250-1000, LR 1e-3/2e-4
axioms (5)
- domain assumption Simulated clicks modeled as 3D Gaussian maps approximate real human interactions
- domain assumption TotalSegmentator pseudo-labels are accurate enough for organ supervision
- domain assumption Public pre-trained weights from [8] and [9] transfer to the autoPET IV dataset
- domain assumption nnU-Net default preprocessing and data augmentation are appropriate
- domain assumption Ground truth lesion annotations in autoPET IV are correct
Cite this review
Pith. "Pith review of autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT." pith.science (2026). https://pith.science/paper/SBLKFUHW
@misc{pith2026250902402,
author = {Pith},
title = {Pith review of: autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT},
year = {2026},
howpublished = {\url{https://pith.science/paper/SBLKFUHW}},
note = {Machine review of arXiv:2509.02402}
}
read the original abstract
Lesion Segmentation in PET/CT scans is an essential part of modern oncological workflows. To address the challenges of time-intensive manual annotation and high inter-observer variability, the autoPET challenge series seeks to advance automated segmentation methods in complex multi-tracer and multi-center settings. Building on this foundation, autoPET IV introduces a human-in-the-loop scenario to efficiently utilize interactive human guidance in segmentation tasks. In this work, we incorporated tracer classification, organ supervision and simulated clicks guidance into the nnUNet Residual Encoder framework, forming an integrated pipeline that demonstrates robust performance in a fully automated (zero-guidance) context and efficiently leverages iterative interactions to progressively enhance segmentation accuracy.
Reference graph
Works this paper leans on
-
[1]
Human pathology 31(5), 578--583 (2000)
Bubendorf, L., Sch \"o pfer, A., Wagner, U., Sauter, G., Moch, H., Willi, N., Gasser, T.C., Mihatsch, M.J.: Metastatic patterns of prostate cancer: an autopsy study of 1,589 patients. Human pathology 31(5), 578--583 (2000)
work page 2000
-
[2]
Cancer 120(22), 3433--3445 (November 2014)
Farwell, M.D., Pryma, D.A., Mankoff, D.A.: Pet/ct imaging in cancer: Current applications and future directions. Cancer 120(22), 3433--3445 (November 2014). doi:10.1002/cncr.28860, epub 2014 Jun 19
-
[3]
Nature Machine Intelligence 6(11), 1396--1405 (2024)
Gatidis, S., Fr \"u h, M., Fabritius, M.P., Gu, S., Nikolaou, K., Foug \`e re, C.L., Ye, J., He, J., Peng, Y., Bi, L., et al.: Results from the autopet challenge on fully automated lesion segmentation in oncologic pet/ct imaging. Nature Machine Intelligence 6(11), 1396--1405 (2024)
work page 2024
-
[4]
https://doi.org/10.7937/gkr0-xv29 (2022)
Gatidis, S., Kuestner, T.: A whole-body fdg-pet/ct dataset with manually annotated tumor lesions (fdg-pet-ct-lesions) [dataset]. https://doi.org/10.7937/gkr0-xv29 (2022). doi:10.7937/gkr0-xv29, the Cancer Imaging Archive
-
[5]
Nature methods 18(2), 203--211 (2021)
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods 18(2), 203--211 (2021)
work page 2021
-
[6]
arXiv preprint arXiv:2404.09556 (2024)
Isensee, F., Wald, T., Ulrich, C., Baumgartner, M., Roy, S., Maier-Hein, K., Jaeger, P.F.: nnu-net revisited: A call for rigorous validation in 3d medical image segmentation. arXiv preprint arXiv:2404.09556 (2024)
Pith/arXiv arXiv 2024
-
[7]
https://doi.org/10.7937/r7ep-3x37 (2024)
Jeblick, K., et al.: A whole-body psma-pet/ct dataset with manually annotated tumor lesions (psma-pet-ct-lesions) (version 1) [dataset]. https://doi.org/10.7937/r7ep-3x37 (2024). doi:10.7937/r7ep-3x37, the Cancer Imaging Archive
-
[8]
Kalisch, H., Hörst, F., Herrmann, K., Kleesiek, J., Seibold, C.: Autopet iii challenge: Incorporating anatomical knowledge into nnunet for lesion segmentation in pet/ct (2024), https://arxiv.org/abs/2409.12155
Pith/arXiv arXiv 2024
-
[9]
Rokuss, M., Kovacs, B., Kirchhoff, Y., Xiao, S., Ulrich, C., Maier-Hein, K.H., Isensee, F.: From fdg to psma: A hitchhiker's guide to multitracer, multicenter lesion segmentation in pet/ct imaging (2024), https://arxiv.org/abs/2409.09478
Pith/arXiv arXiv 2024
-
[10]
Nature Reviews Cancer 23(7), 474--490 (July 2023)
Schwenck, J., Sonanini, D., Cotton, J.M., Rammensee, H.G., la Fougère, C., Zender, L., Pichler, B.J.: Advances in pet imaging of cancer. Nature Reviews Cancer 23(7), 474--490 (July 2023). doi:10.1038/s41568-023-00576-4, epub 2023 May 31
-
[11]
Radiology: Artificial Intelligence 5(5) (2023)
Wasserthal, J., Breit, H.C., Meyer, M.T., Pradella, M., Hinck, D., Sauter, A.W., Heye, T., Boll, D.T., Cyriac, J., Yang, S., et al.: Totalsegmentator: robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence 5(5) (2023)
work page 2023
-
[12]
, " * write output.state after.block = add.period write
ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...
-
[13]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.