Pith. sign in

REVIEW 4 major objections 4 minor 13 references

Adding clicks steadily lifts PET/CT lesion segmentation

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Combining tracer classification, organ supervision, and stochastic click sampling makes an nnU-Net model segment PET/CT lesions robustly without guidance and progressively better with clicks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A transparent autoPET IV methods paper with a sensible stochastic-click curriculum and honest caveats, but the interactive-guidance claim is supported only by simulated clicks on training-seen data. the 4 major comments →

arxiv 2509.02402 v1 pith:SBLKFUHW submitted 2025-09-02 eess.IV

autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT

classification eess.IV
keywords PET/CT lesion segmentationinteractive segmentationhuman-in-the-loopstochastic click samplingtracer classificationorgan supervisionSUV thresholdingautoPET IV
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper describes a pipeline for interactive lesion segmentation in PET/CT scans, entered in the autoPET IV challenge. It claims that training a segmentation network with a stochastic click-sampling curriculum—randomly showing 0 to 10 simulated foreground and background clicks per sample—produces a model that works well with no clicks at all and keeps improving as an expert adds more clicks. The pipeline combines tracer classification (FDG vs PSMA), separate models for the two tracers, an auxiliary organ-segmentation head for anatomical context, and SUV-threshold post-processing. The paper's headline result is that models trained only with dense guidance collapse at zero or few clicks, while models trained with balanced or sparse-leaning click distributions keep a strong baseline and use incremental guidance efficiently.

Core claim

The central claim, stated in the conclusion, is that the pipeline 'exhibits robust performance with sparse human guidance and a proficient ability to leverage incremental human feedback to refine segmentation accuracy.' In concrete terms: with zero clicks the unified model reaches a Dice score around 0.79 and the PSMA model around 0.62, and adding one to ten clicks raises Dice monotonically while false-positive and false-negative volumes fall. The mechanism is the training curriculum, not a new network: the same residual-encoder U-Net backbone is exposed to randomly sampled amounts of guidance, so it cannot afford to ignore the image and also learns to exploit clicks when they arrive.

What carries the argument

Stochastic click sampling curriculum: during training, each sample receives a random integer k between 0 and 10, and only the first k simulated foreground and k background clicks are fed to the network as 3D Gaussian heatmaps concatenated with CT and PET. This forces one model to cover the whole interaction spectrum. Supporting machinery: an upstream tracer classifier selects FDG- or PSMA-specific weights; an auxiliary organ-segmentation head supplies tracer-invariant anatomical priors using pseudo-labels from a whole-body CT segmenter; and SUV thresholding removes low-uptake false positives at inference.

Load-bearing premise

The load-bearing premise is that the challenge's simulated clicks, generated as 3D Gaussian maps by a provided script, faithfully represent how real human users would click in clinical practice; if real clicks differ in location, timing, or count, the measured benefit of more clicks may not transfer.

What would settle it

Collect a set of PET/CT cases with real radiologist click streams (foreground and background clicks placed on actual lesions and false positives) and run the submitted pipeline with the same click counts used in the challenge. If Dice does not rise monotonically as real clicks increase, or if the zero-click baseline drops when real clicks are sparse, the claim that the pipeline leverages incremental human feedback is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Dense-only training is not enough: a model trained only on 10-click examples scores 0.0 Dice at 0 clicks, so any deployment that expects occasional human correction must train with sparse examples too.
  • A balanced stochastic sampling distribution yields a model whose accuracy rises nearly monotonically with click count, meaning incremental human feedback can be converted directly into segmentation gains.
  • Organ supervision and pre-trained anatomical priors anchor a strong zero-click baseline, so adding interactive guidance improves an already usable automatic result rather than rescuing a failed one.
  • Because different models dominate in different regimes, hybrid inference—using the sparse-trained model for 0–4 clicks and the dense-trained model for 5–10—gives the best of both, a strategy that generalizes to any interactive pipeline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The challenge simulates clicks as 3D Gaussian maps; real radiologists may click different locations, numbers, or orders. A natural test is to compare this curriculum against supervision with actual click streams, and to add click-location jitter during training for robustness.
  • The paper evaluates on cases already seen during training, so reported gains are an upper bound; an independent test set would be needed to confirm the monotonic improvement transfers.
  • Organ supervision appears to shrink false-positive volume, likely by suppressing high physiological uptake in organs; this suggests the auxiliary head could be pruned or distilled after training to save computation without losing the anatomical prior.
  • The hybrid model-selection rule (choose network by click count) could be extended to a confidence-based router that learns when to switch, rather than a fixed threshold.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper describes an automated and interactive lesion-segmentation pipeline for the autoPET IV challenge, built on nnU-Net Residual Encoder. The authors integrate tracer classification using public weights, organ supervision via an auxiliary head with TotalSegmentator pseudo-labels, two ways of injecting simulated click guidance (full-guidance and stochastic click sampling), and SUV-threshold post-processing. They develop three PSMA-specific models (V0, V1, V2) and two unified multi-tracer models (V3, V4), evaluate them on 5-fold cross-validation and on interactive evaluations with 0-10 simulated clicks, and select V2 for PSMA and a hybrid V3/V4 strategy for FDG. The central claim is that the pipeline is robust with sparse human guidance and effectively uses incremental human feedback to improve segmentation accuracy.

Significance. If the central claim were fully supported, the work would be a useful system description for the autoPET IV interactive segmentation challenge, particularly the stochastic click-sampling curriculum and the hybrid deployment strategy. The authors make their code and trained weights publicly available, provide detailed training configurations, and build on publicly documented pre-trained models, which are strengths for reproducibility. However, the current evidence does not yet establish the claimed generalizable interactive capability: the interactive evaluations are performed on training-seen cases, the simulated click protocol is not characterized, and the unified-model results are based on incomplete folds without error bars. The paper is more convincingly a challenge methods report than a validated clinical human-in-the-loop study.

major comments (4)
  1. [Section 3.1 / Table 2] The interactive evaluation for PSMA models is performed on 100 cases 'that had already been seen by the models during training.' Table 2 therefore reports in-sample segmentation performance, not generalization. The monotonic Dice improvement with increasing clicks could reflect memorization of training cases rather than the model's ability to use human guidance on unseen data. This directly affects the conclusion in Section 5 that the pipeline 'exhibits robust performance with sparse human guidance' and 'leverages incremental human feedback.' The authors should provide results on a held-out split or the challenge test set, or explicitly reframe Tables 2-3 as development-set diagnostics without claiming generalizable interactive performance.
  2. [Section 2.1] The manuscript states that 10 foreground and 10 background clicks are 'simulated and modeled as 3D Gaussian maps using the provided script,' but it never describes how click positions are generated. If the script samples clicks from ground-truth lesion masks or from tracer-avid regions, the improvement with more clicks is an oracle effect rather than a demonstration of human-in-the-loop refinement. A concrete test is needed: report the click-position generation protocol, compare the simulated distribution with human click behavior, or evaluate with clicks derived from the model's current errors. Without this, the interactive claim is not transferable to clinical use.
  3. [Section 3.2 / Table 3] The unified-model results are based on only three folds for V3 and two folds for V4, and neither Table 2 nor Table 3 reports error bars or per-fold variability. The differences between V3 and V4 are small—for example, Dice at 4 clicks is 0.869 vs. 0.865, and at 10 clicks 0.889 vs. 0.877—so the proposed hybrid selection rule may be within fold noise. The authors should report per-fold results, standard deviations, and ideally a statistical comparison before using these numbers to choose between models in the final submission.
  4. [Section 3.2 / Final Submission] The hybrid-strategy description is internally inconsistent: it reads 'Model V4 would be utilized in early interactive phase (0-4 clicks), while model V4 would be chosen in densely guided steps (5-10 clicks),' which repeats V4 and cannot be the intended rule. Table 3 also does not clearly support a 0-4 threshold: at 3 clicks V4 is only 0.001 Dice above V3, and at 4 clicks V3 is above V4. Additionally, the conclusion credits 'anatomical prior and organ supervision' for the unified model's behavior, but no ablation removes organ supervision from V3/V4, so its causal role is not demonstrated. The authors should correct the typo, justify the click-count threshold with the reported data, and soften or support the attribution to organ supervision.
minor comments (4)
  1. [Section 2.3] There are minor typos: 'Channnel 0' should be 'Channel 0'; 'We adopted' has an inconsistent capital 'W' mid-sentence.
  2. [Section 3.2] The sentence describing the hybrid strategy repeats 'model V4' for both the early and dense phases; the second instance should presumably be 'model V3.'
  3. [Table 4] Table 4 is difficult to read as a formatted row; consider restructuring into a normal table with labeled columns instead of merged cells.
  4. [References] The citations are adequate for a challenge report, but the paper would benefit from citing the official autoPET IV challenge description or website to clarify the evaluation protocol and the click-simulation script.

Circularity Check

0 steps flagged

No circularity: empirical challenge report with independent external priors and measured metrics.

full rationale

This paper is an empirical challenge submission rather than a derivation, so none of its central claims reduce to its inputs by construction. The pipeline components (tracer classification, organ supervision, stochastic click sampling) are described as training strategies, and the results in Tables 1-3 are measured Dice/FPV/FNV values, not fitted quantities relabeled as predictions. The pre-trained weights used for the tracer classifier and the anatomical-prior model come from public sources by other teams ([8], [9]) and from TotalSegmentator ([11]); none of these are self-citations by the present authors. The stochastic click sampling curriculum is an input design, but the reported click-vs-performance trends are empirical outcomes on fixed validation sets, not mathematically forced by the sampling distribution. The paper also explicitly acknowledges the main limitation: Section 3.1 states that all evaluated PSMA cases 'had already been seen by the models during training, so no guarantee of final test performance can be made.' Similarly, the use of script-generated 3D Gaussian clicks is a fidelity concern about generalization to real human interaction, but it is not circular reasoning. No equation is defined in terms of a target result, no fitted parameter is renamed as a prediction, and no load-bearing claim rests on the authors' own prior work. The honest non-finding therefore applies: the derivation chain is self-contained in the sense that all reported quantities are measured outcomes with external priors, and the acknowledged limitations concern external validity rather than circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim rests on domain assumptions about the realism of simulated clicks, the quality of pseudo-organ labels from TotalSegmentator, the transferability of public pre-trained weights, and the default nnU-Net pipeline. The main hand-chosen free parameters are the SUV thresholds, the Dice:CE loss ratio, and the stochastic click sampling distributions. No new physical or conceptual entities are introduced.

free parameters (6)
  • SUV threshold (FDG) = 1.5
    Post-processing threshold from [8] to cut false positives; applied to FDG predictions.
  • SUV threshold (PSMA) = 1.0
    Post-processing threshold from [8] for PSMA.
  • Dice:CE loss ratio (Model V0) = 2:1
    Hand-set to emphasize small PSMA lesions; no ablation shown.
  • Click sampling distribution (Model V1) = 40% for 0 clicks, 20% for 1 click, remainder unspecified
    Hand-designed to improve sparse-guidance performance.
  • Click sampling distribution (Model V4) = [0.10, 0.10, 0.10, 0.08, 0.04, 0.04, 0.04, 0.04, 0.08, 0.08, 0.30]
    Hand-designed balanced distribution across 0-10 clicks.
  • Training hyperparameters (patch, batch, epochs, LR) = patch [192,192,192], batch 2-3, epochs 250-1000, LR 1e-3/2e-4
    Adopted from nnU-Net defaults or previous works; not centrally tuned in this paper.
axioms (5)
  • domain assumption Simulated clicks modeled as 3D Gaussian maps approximate real human interactions
    Section 2.1: clicks are simulated using the provided script; the paper assumes this captures interactive guidance.
  • domain assumption TotalSegmentator pseudo-labels are accurate enough for organ supervision
    Section 2.3: pseudo organ labels created by TotalSegmentator without manual verification.
  • domain assumption Public pre-trained weights from [8] and [9] transfer to the autoPET IV dataset
    Section 2.3: weights used for initialization and fine-tuning; no verification of domain shift.
  • domain assumption nnU-Net default preprocessing and data augmentation are appropriate
    Section 2.2: default planner and preprocessor used.
  • domain assumption Ground truth lesion annotations in autoPET IV are correct
    Challenge dataset assumed reliable; paper does not audit labels.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT." pith.science (2026). https://pith.science/paper/SBLKFUHW

@misc{pith2026250902402,
  author       = {Pith},
  title        = {Pith review of: autoPET IV challenge: Incorporating organ supervision and human guidance for lesion segmentation in PET/CT},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SBLKFUHW}},
  note         = {Machine review of arXiv:2509.02402}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Lesion Segmentation in PET/CT scans is an essential part of modern oncological workflows. To address the challenges of time-intensive manual annotation and high inter-observer variability, the autoPET challenge series seeks to advance automated segmentation methods in complex multi-tracer and multi-center settings. Building on this foundation, autoPET IV introduces a human-in-the-loop scenario to efficiently utilize interactive human guidance in segmentation tasks. In this work, we incorporated tracer classification, organ supervision and simulated clicks guidance into the nnUNet Residual Encoder framework, forming an integrated pipeline that demonstrates robust performance in a fully automated (zero-guidance) context and efficiently leverages iterative interactions to progressively enhance segmentation accuracy.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

  1. [1]

    Human pathology 31(5), 578--583 (2000)

    Bubendorf, L., Sch \"o pfer, A., Wagner, U., Sauter, G., Moch, H., Willi, N., Gasser, T.C., Mihatsch, M.J.: Metastatic patterns of prostate cancer: an autopsy study of 1,589 patients. Human pathology 31(5), 578--583 (2000)

  2. [2]

    Cancer 120(22), 3433--3445 (November 2014)

    Farwell, M.D., Pryma, D.A., Mankoff, D.A.: Pet/ct imaging in cancer: Current applications and future directions. Cancer 120(22), 3433--3445 (November 2014). doi:10.1002/cncr.28860, epub 2014 Jun 19

  3. [3]

    Nature Machine Intelligence 6(11), 1396--1405 (2024)

    Gatidis, S., Fr \"u h, M., Fabritius, M.P., Gu, S., Nikolaou, K., Foug \`e re, C.L., Ye, J., He, J., Peng, Y., Bi, L., et al.: Results from the autopet challenge on fully automated lesion segmentation in oncologic pet/ct imaging. Nature Machine Intelligence 6(11), 1396--1405 (2024)

  4. [4]

    https://doi.org/10.7937/gkr0-xv29 (2022)

    Gatidis, S., Kuestner, T.: A whole-body fdg-pet/ct dataset with manually annotated tumor lesions (fdg-pet-ct-lesions) [dataset]. https://doi.org/10.7937/gkr0-xv29 (2022). doi:10.7937/gkr0-xv29, the Cancer Imaging Archive

  5. [5]

    Nature methods 18(2), 203--211 (2021)

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods 18(2), 203--211 (2021)

  6. [6]

    arXiv preprint arXiv:2404.09556 (2024)

    Isensee, F., Wald, T., Ulrich, C., Baumgartner, M., Roy, S., Maier-Hein, K., Jaeger, P.F.: nnu-net revisited: A call for rigorous validation in 3d medical image segmentation. arXiv preprint arXiv:2404.09556 (2024)

  7. [7]

    https://doi.org/10.7937/r7ep-3x37 (2024)

    Jeblick, K., et al.: A whole-body psma-pet/ct dataset with manually annotated tumor lesions (psma-pet-ct-lesions) (version 1) [dataset]. https://doi.org/10.7937/r7ep-3x37 (2024). doi:10.7937/r7ep-3x37, the Cancer Imaging Archive

  8. [8]

    Kalisch, H., Hörst, F., Herrmann, K., Kleesiek, J., Seibold, C.: Autopet iii challenge: Incorporating anatomical knowledge into nnunet for lesion segmentation in pet/ct (2024), https://arxiv.org/abs/2409.12155

  9. [9]

    Rokuss, M., Kovacs, B., Kirchhoff, Y., Xiao, S., Ulrich, C., Maier-Hein, K.H., Isensee, F.: From fdg to psma: A hitchhiker's guide to multitracer, multicenter lesion segmentation in pet/ct imaging (2024), https://arxiv.org/abs/2409.09478

  10. [10]

    Nature Reviews Cancer 23(7), 474--490 (July 2023)

    Schwenck, J., Sonanini, D., Cotton, J.M., Rammensee, H.G., la Fougère, C., Zender, L., Pichler, B.J.: Advances in pet imaging of cancer. Nature Reviews Cancer 23(7), 474--490 (July 2023). doi:10.1038/s41568-023-00576-4, epub 2023 May 31

  11. [11]

    Radiology: Artificial Intelligence 5(5) (2023)

    Wasserthal, J., Breit, H.C., Meyer, M.T., Pradella, M., Hinck, D., Sauter, A.W., Heye, T., Boll, D.T., Cyriac, J., Yang, S., et al.: Totalsegmentator: robust segmentation of 104 anatomic structures in ct images. Radiology: Artificial Intelligence 5(5) (2023)

  12. [12]

    , " * write output.state after.block = add.period write

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

  13. [13]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.