Pith. sign in

REVIEW 2 major objections 5 minor 22 references

For head-and-neck nodal staging, seven geometry features from the predicted tumor mask outperform a 30-dimensional radiomics vector—and the gap widens when segmentation noise is removed.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 14:30 UTC pith:KVZP4MMW

load-bearing objection HERMES is an honest, well-scoped challenge paper; the geometry-feature idea is sensible and worth publishing, but the ground-truth-mask evidence for it carries a per-condition regularization confound that needs fixing. the 2 major comments →

arxiv 2607.26498 v1 pith:KVZP4MMW submitted 2026-07-29 cs.CV physics.med-ph

HERMES: A Hybrid Ensemble for Head-and-Neck Tumor Segmentation, TN Staging, and Recurrence-Free Survival on PET/CT

Kai Wang , Meixu Chen , Elie Nasr , Ryan Lanning , Moyed Miften This is my paper
classification cs.CV physics.med-ph
keywords head and neck cancerPET/CTtumor segmentationTNM stagingrecurrence-free survivalgeometry featuresradiomicsensemble learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether the features used to summarize a predicted tumor mask in PET/CT should be engineered to match the clinical staging rules rather than drawn from a generic radiomics dictionary. For nodal (N) staging, it finds that seven geometry features—node count, largest-node size, total nodal burden—beat a 30-dimensional intensity/texture radiomics vector in balanced accuracy (0.720 vs 0.691 on cross-validation), though the paired confidence interval includes zero. The stronger evidence comes from ground-truth masks, where geometry reaches 0.897 and radiomics 0.837, a significant gap, indicating the advantage is real and the deployed gap is a consequence of imperfect segmentation. The paper also introduces a survival loss whose numeric value tracks the concordance index during training, making early stopping interpretable, and it selects every component by out-of-fold cross-validation rather than the small held-out leaderboard.

Core claim

The central discovery is that aligning downstream features with the axes of the task's definition—size and number of involved nodes, as specified in the 7th-edition TNM rules—yields better N-stage prediction than a generic 30-dimensional radiomics descriptor. A two-by-two comparison of feature type (geometry vs radiomics) against mask source (predicted vs ground truth) isolates feature quality from mask quality: on ground-truth masks, geometry beats radiomics by a significant +0.060 balanced accuracy, while on predicted masks the gain is a favorable +0.030 trend within noise. The paper interprets this as evidence that the geometry signal is genuine and that segmentation error is the shared b

What carries the argument

The key mechanism is a seven-dimensional geometry feature block computed from connected components of the predicted lymph-node mask: node count, largest-component volume, axis-aligned extent, dominance fraction, total nodal burden, and log-volume terms, plus primary-tumor extent. This block instantiates the size and number axes of the 7th-edition radiological N-staging definition. A second mechanism is a concordance-tracking survival loss, a smoothed pairwise sigmoid over comparable pairs whose value approximates 1 minus the concordance index throughout training; it is used to train one deep expert and to guide early stopping with an interpretable scale.

Load-bearing premise

The conclusion that the predicted-mask gain reflects a genuine geometry advantage, diluted by segmentation error, assumes that mask noise does not corrupt geometry and radiomics features differently in stage-dependent ways, and that the ground-truth-mask comparison—with each feature set at its own best regularization—isolates feature quality rather than hyperparameter luck.

What would settle it

Run the same N-stage comparison with both feature sets constrained to a single shared regularization strength (or with nested cross-validation for hyperparameters) on the ground-truth masks; if the +0.060 geometry advantage collapses to within noise, the claim of intrinsic feature superiority is undermined. A stronger external test: on an independent multi-center dataset with high-quality manual contours, geometry features should reproduce a significant N-stage gain over radiomics; failure to replicate would refute the generality.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If geometry features genuinely outperform radiomics for nodal staging, then future PET/CT staging pipelines should derive features from the clinical staging criteria rather than from generic texture dictionaries.
  • Because the gap between predicted masks (0.720) and ground-truth masks (0.897) is attributed to segmentation quality, improving lymph-node Dice should translate almost directly into higher N-stage accuracy, giving a quantified target for segmentation research.
  • The concordance-tracking loss provides a trustworthy, interpretable training signal, so practitioners can stop training or select checkpoints without recomputing the concordance index.
  • The paper's preference for equal-weight ensembles and clinical-only risk models, chosen for cross-center stability, implies that the system may degrade more gracefully on unseen scanner protocols than tuned or radiomics-heavy alternatives.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's headline N-stage gain is a trend, not a confirmed effect, on predicted masks; the case rests on the ground-truth-mask comparison, where each feature set was tuned at its own best regularization. A fixed shared regularization would be a sharper test of whether geometry is intrinsically superior or merely better matched to this data.
  • The geometry features omit laterality, a defining axis of the N-staging rules; adding side-of-midline or level-aware features could push the perfect-mask ceiling above 0.897 and might close the N2b/N2c distinction.
  • The same 'align features to the task definition' principle suggests a path for T4 staging, which the paper notes is size-insensitive; features encoding proximity to at-risk anatomical structures would be the analogous design.
  • The equal-weight ensemble is a deliberate anti-overfitting choice for small data; on larger cohorts, learned weights might overtake it, so the design trade-off is worth testing as datasets grow.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. HERMES is a containerized multi-task pipeline for the HECKTOR 2026 challenge, producing GTVp/GTVn segmentation, T/N staging, and recurrence-free survival from FDG-PET/CT plus an electronic health record. Segmentation is a 10-fold ensemble of STU-Net Small models; downstream staging uses clinical variables plus either a 30-dimensional radiomics block or a proposed 7-dimensional geometry block (nodal count, size, burden, primary extent) aligned with AJCC/UICC 7th-edition staging axes. On out-of-fold cross-validation the geometry block raises N-stage balanced accuracy from 0.691 to 0.720, with a paired 95% CI that includes zero; on ground-truth masks the gap is 0.897 vs 0.837, with a CI excluding zero. For RFS, an equal-weight ensemble includes a deep expert trained with a concordance-tracking loss, which is reported to track the C-index during training. The pipeline qualifies for the HECKTOR 2026 testing phase, with validation scores reported. The paper emphasizes an honest out-of-fold selection protocol, no tuning on the public validation set, and transparent reporting of non-significance.

Significance. If the central claim holds, the paper provides a useful and clinically interpretable illustration that mask-derived features aligned with the downstream task's definition can outperform generic radiomics for nodal staging at lower dimensionality. The paper's strengths are its explicit out-of-fold protocol, release of code and the trained container, and honest reporting of confidence intervals and non-significant trends. The geometry-feature direction is well motivated by the AJCC/UICC definitions. However, the statistically significant evidence for feature superiority comes only from the ground-truth-mask comparison, and that comparison is vulnerable to per-condition hyperparameter selection; the primary predicted-mask gain is not significant. Thus the significance of the paper, if the results hold, is moderate: it is a credible design study for a challenge pipeline rather than a definitive demonstration of feature-family superiority.

major comments (2)
  1. [§3, Table 3] The ground-truth-mask comparison (geometry 0.897 vs radiomics 0.837, paired +0.060, 95% CI [+0.028,+0.093]) is the only statistically significant evidence that geometry features are superior as features. The table note says each feature set is evaluated 'at its own best regularization,' and the bootstrap CIs are computed after selecting the regularization parameter on the same out-of-fold cohort. Model selection over the C grid is therefore not accounted for in the reported interval. A 30-dimensional radiomics vector will typically require a different shrinkage level than a 7-dimensional geometry vector, so part or all of the +0.060 gap could be a tuning artifact. This is load-bearing because the predicted-mask paired gain is non-significant. Please report the selected C values for both feature sets and provide a selection-aware assessment, e.g., nested cross-validation that selects C in
  2. [§3, Fig. 3b / Discussion] The interpretation that the deployed N-stage benefit is 'diluted by imperfect segmentation' assumes that mask errors do not differentially corrupt geometry versus radiomics features. A small, discontinuous change in the predicted mask (e.g., a split or merge of a connected component) can change geometry features abruptly, while radiomics summary statistics may change more smoothly. The ground-truth-mask rows alone do not establish that the predicted-mask gap is smaller only because of shared segmentation noise. A concrete test would be to compare feature values computed on predicted versus ground-truth masks (e.g., correlation or absolute error per feature), or to add controlled mask perturbations and measure the resulting change in each feature set and in N-stage accuracy. Without such evidence, the 'dilution' claim is plausible but not directly supported.
minor comments (5)
  1. [§2.2 / §2.3] The notation '963 CT+PET patch' and '1123 patches' appears to be a formatting error for '96^3' and '112^3'. Please fix these superscripts.
  2. [Throughout] Typographical issues: 'T able 1' and 'T eam' in the abstract/table captions should be 'Table 1' and 'Team'.
  3. [§2.2] The text says the staging contribution 'replaces the radiomics block' with geometry features, but T-stage still uses a fusion of clinical, radiomics, and primary-geometry features. Please clarify that the replacement is specifically for the N-stage classical expert, not the entire staging pipeline.
  4. [§3, Table 3 note] The phrase 'two-by-two of feature type against mask source' is not actually presented as a 2x2 table in the paper. Consider either presenting the 2x2 table or rephrasing to 'comparison of feature type and mask source.'
  5. [§2.3, Eq. (1)] The equation formatting is garbled in places ('L= 1 |P| P'). Please ensure the normalized sum is rendered correctly.

Circularity Check

0 steps flagged

No circularity: the central staging ablation is an empirical OOF comparison against challenge-provided labels; the only self-citation (concordance loss) is not load-bearing.

full rationale

No specific circular step can be exhibited. The main claim is an empirical ablation: 7-dimensional geometry features versus a 30-dimensional radiomics block for N-stage balanced accuracy, evaluated on pooled out-of-fold predictions with patient-level bootstrap confidence intervals. The N-stage labels are challenge-provided AJCC/UICC 7th-edition radiological stages, not derived from the geometry features, and the paper states 'we do not define or annotate staging ourselves.' The geometry features are motivated by the staging criteria but are computed from predicted masks and compared against radiomics on held-out folds, so the result is not true by construction. The concordance-loss survival expert is a self-citation (reference [14]), but the loss is fully specified in Equation (1) and is compared in the same paper against an identically trained Cox-loss twin; the reported accuracy difference is explicitly within noise and the loss is retained for interpretability, not as a load-bearing predictive claim. The ground-truth-mask rows of Table 3 are computed 'each at its own best regularization,' which is a potential confound for the magnitude of the GT-mask gain, but it is a statistical validity concern, not a circularity: the features and labels are still separate, and the comparison is not equivalent to its inputs by construction. No self-citation chain, imported uniqueness theorem, or ansatz-by-citation is used to force the paper's choices. The paper is therefore self-contained against external benchmarks, and the honest non-finding (score 0) applies.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

No new physical entities, forces, dimensions, or conserved quantities are introduced. The geometry features are derived quantities from predicted masks and external staging rules. Free parameters are limited to standard model-selection and post-processing choices, all fit on out-of-fold data.

free parameters (6)
  • GTVp connected-component size threshold = 1000 mm^3
    Post-processing removes GTVp components below 1000 mm^3; fixed by a grid sweep on OOF predictions (§2.1, §3).
  • GTVn connected-component size threshold = 500 mm^3
    Post-processing removes GTVn components below 500 mm^3; fixed on OOF (§2.1, §3).
  • Nodal SUV gate = 2.5
    PET intensity threshold removes low-uptake nodal false positives; fixed on OOF (§2.1, §3).
  • N-stage L1 regularization C = 0.03
    Regularization strength for the geometry-based N-stage logistic regression, chosen on OOF for stability (§2.2, Table 1).
  • Concordance-loss temperature tau = 0.1
    Width of the pairwise sigmoid in Eq. (1); adopted from the authors' preceding concordance-loss paper (§2.3).
  • L1-selected N-stage feature subset = total burden, largest-node extent, dominance, plus one clinical variable
    Post-hoc L1 selection on OOF (§3); used as evidence that size/number axes, not intensity/texture, carry the nodal staging signal.
axioms (6)
  • domain assumption AJCC/UICC 7th-edition radiological N category is determined by node number, size, and laterality; T category by size (T1-T3) and invasion (T4).
    Used to justify the geometry feature design and to explain why T4 is not resolvable; §2.2, §4. If challenge labels deviate from this manual, the feature alignment is less meaningful.
  • domain assumption Predicted-mask connected components of size >= 500 mm^3 with SUV >= 2.5 correspond to true pathological nodes.
    Post-processing thresholds fixed on OOF; the geometry features depend on this mapping (§2.1, §2.2).
  • domain assumption Patient-level bootstrap CIs and pooled OOF metrics on the 782-patient training cohort transfer to unseen centers.
    All model selection and significance statements rely on OOF generalization; the hidden test set includes previously unseen centers (§3).
  • standard math The pairwise sigmoid loss in Eq. (1) with tau=0.1 has stable training dynamics and approximates 1 - C-index.
    The loss is a standard smooth surrogate; monotone tracking of validation concordance is shown empirically in Fig. 4.
  • domain assumption Ground-truth-mask feature evaluation isolates feature quality from mask quality.
    The inference that the predicted-mask gain is diluted rather than an artifact depends on this decomposition (§3, Table 3, Discussion).
  • domain assumption Equal-weight z-averaging of risk experts is a safe combination under limited sample size.
    The ensemble protects against overfitting; normalization uses training-fold statistics only to avoid test leakage (§2.3).

pith-pipeline@v1.3.0-daily-deepseek · 9089 in / 13721 out tokens · 118645 ms · 2026-08-01T14:30:02.462201+00:00 · methodology

0 comments
read the original abstract

We present HERMES (Hybrid Ensemble for Radiotherapy-target segmentation, Malignancy staging, and Event-free Survival), a single containerized algorithm for the three HECKTOR 2026 subtasks: segmentation of the primary tumor (GTVp) and pathological lymph nodes (GTVn), radiological T/N staging, and recurrence-free survival (RFS), computed from a paired FDG-PET/CT scan and an electronic health record. A 10-fold ensemble of STU-Net Small networks produces the segmentation; the predicted mask then drives two downstream tasks. Rather than pass a generic radiomics vector to the staging models, we derive from the predicted masks a compact set of geometry features aligned with the size and number axes of AJCC/UICC 7th-edition radiological N/T staging. On internal cross-validation these features raise N-stage balanced accuracy from 0.691 to 0.720 (+0.030), our largest single design gain, at lower feature dimensionality. For prognosis we combine complementary deep and clinical risk experts in an equal-weight ensemble, and train one deep expert with a concordance-tracking survival loss of our own, whose value approximates the concordance index during training. Every component was selected on honest out-of-fold predictions under a regularization-oriented protocol, with no tuning on the public validation set, and deployed as two decorrelated submissions. On the HECKTOR 2026 validation leaderboard, HERMES achieved a weighted score of 0.6454 (Mean Dice 0.641, T balanced accuracy 0.580, N balanced accuracy 0.642, RFS C-index 0.679) and qualified for the testing phase. Team: AMC_HNC.

Figures

Figures reproduced from arXiv: 2607.26498 by Elie Nasr, Kai Wang, Meixu Chen, Moyed Miften, Ryan Lanning.

Figure 1
Figure 1. Figure 1: HERMES pipeline. FDG-PET/CT and EHR feed a 10-fold STU-Net segmen￾tation; the predicted mask then drives TN staging (deep, radiomics, and geometry experts) and RFS (a deep survival ensemble with a concordance-loss expert plus a clinical Cox model). 2.1 Segmentation We use STU-Net Small (about 14 M parameters) [6], a scalable residual U-Net reported to match or exceed nnU-Net on large-scale benchmarks, whic… view at source ↗
Figure 2
Figure 2. Figure 2: Predicted (filled: red GTVp, cyan GTVn) versus ground-truth (contours: yellow GTVp, green GTVn) segmentation on the maximum-tumor axial slice for one repre￾sentative patient from each of the eight training centers (deployed-model out-of-fold predictions). that it removes low-uptake nodal false positives at no overlap cost. Predictions are visually consistent with the reference across all eight centers, inc… view at source ↗
Figure 3
Figure 3. Figure 3: (a) Predicted-mask nodal geometry (largest-node extent, node count) is mono￾tone in ground-truth N-stage. (b) N-stage balanced accuracy (out-of-fold, patient￾level bootstrap 95% CIs): clinical plus geometry (0.720) versus clinical plus radiomics (0.691). ceiling. This is our largest design effect. For T-staging, adding primary-tumor geometry to the deep and radiomics fusion moves balanced accuracy from 0.4… view at source ↗
Figure 4
Figure 4. Figure 4: The concordance loss value is an interpretable, metric-aligned training signal (representative fold, identical architecture and data). (a) Its training value stays on the scale of 1 − C-index and falls with the validation concordance throughout training. (b) Across all epochs, concordance-loss values lie along loss = 1−C-index, reading directly as an approximate mis-ranking rate, whereas the Cox loss forms… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 4 linked inside Pith

  1. [1]

    arXiv:2509.00367 (2025)

    Saeed, N., et al.: A Multimodal and Multi-centric Head and Neck Cancer Dataset for Segmentation, Diagnosis and Outcome Prediction. arXiv:2509.00367 (2025)

    arXiv 2025

  2. [2]

    arXiv:2606.20143 (2026)

    Saeed, N., et al.: HEad and neCK TumOR (HECKTOR) 2025: Benchmark of Seg- mentation, Diagnosis, and Prognosis in Multimodal PET/CT. arXiv:2606.20143 (2026)

    Pith/arXiv arXiv 2025

  3. [3]

    Medical Image Analysis 77, 102336 (2022)

    Oreiller, V., et al.: Head and neck tumor segmentation in PET/CT: The HECK- TOR challenge. Medical Image Analysis 77, 102336 (2022)

    2022

  4. [4]

    In: MICCAI, LNCS 9351, pp

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: MICCAI, LNCS 9351, pp. 234–241 (2015)

    2015

  5. [5]

    Nature Methods 18, 203–211 (2021)

    Isensee, F., et al.: nnU-Net: a self-configuring method for deep-learning-based biomedical image segmentation. Nature Methods 18, 203–211 (2021)

    2021

  6. [6]

    arXiv:2304.06716 (2023)

    Huang, Z., et al.: STU-Net: Scalable and Transferable Medical Image Segmenta- tion Models Empowered by Large-Scale Supervised Pre-training. arXiv:2304.06716 (2023)

    Pith/arXiv arXiv 2023

  7. [7]

    In: BrainLes, LNCS 12962, pp

    Hatamizadeh, A., et al.: Swin UNETR: Swin Transformers for Semantic Segmen- tation of Brain Tumors in MRI Images. In: BrainLes, LNCS 12962, pp. 272–284 (2022)

    2022

  8. [8]

    Cancer Research 77(21), e104–e107 (2017)

    van Griethuysen, J.J.M., et al.: Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Research 77(21), e104–e107 (2017)

    2017

  9. [9]

    Scientific Reports 7, 10117 (2017)

    Vallières, M., et al.: Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer. Scientific Reports 7, 10117 (2017)

    2017

  10. [10]

    In: 3D Head and Neck Tumor Segmentation in PET/CT Challenge (HECKTOR), LNCS, vol

    Wang, K., et al.: Recurrence-free survival prediction under the guidance of auto- matic gross tumor volume segmentation for head and neck cancers. In: 3D Head and Neck Tumor Segmentation in PET/CT Challenge (HECKTOR), LNCS, vol. 13209, pp. 144–153. Springer (2022)

    2022

  11. [11]

    Journal of the Royal Statistical Society B 34(2), 187–220 (1972)

    Cox, D.R.: Regression Models and Life-Tables. Journal of the Royal Statistical Society B 34(2), 187–220 (1972)

    1972

  12. [12]

    BMC Medical Research Methodology 18, 24 (2018)

    Katzman, J.L., et al.: DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Medical Research Methodology 18, 24 (2018)

    2018

  13. [13]

    In: AAAI (2018)

    Lee, C., Zame, W.R., Yoon, J., van der Schaar, M.: DeepHit: A Deep Learning Approach to Survival Analysis with Competing Risks. In: AAAI (2018)

    2018

  14. [14]

    arXiv:2607.16802 (2026).https://github.com/ Meixu-Chen/sigmoid-concordance-loss

    Chen, M., Wang, K., Wang, J.: Value-Monotonicity Matters: A Concordance Loss for Deep Survival Prediction. arXiv:2607.16802 (2026).https://github.com/ Meixu-Chen/sigmoid-concordance-loss

    Pith/arXiv arXiv 2026

  15. [15]

    Wiley-Blackwell (2009)

    Sobin, L.H., Gospodarowicz, M.K., Wittekind, C.: TNM Classification of Malig- nant Tumours, 7th edn. Wiley-Blackwell (2009)

    2009

  16. [16]

    JAMA 247(18), 2543– 2546 (1982)

    Harrell, F.E., et al.: Evaluating the yield of medical tests. JAMA 247(18), 2543– 2546 (1982)

    1982

  17. [17]

    Statistics in Medicine 30(10), 1105–1117 (2011)

    Uno, H., Cai, T., Pencina, M.J., D’Agostino, R.B., Wei, L.J.: On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Statistics in Medicine 30(10), 1105–1117 (2011)

    2011

  18. [18]

    arXiv:2211.02701 (2022)

    Cardoso, M.J., et al.: MONAI: An open-source framework for deep learning in healthcare. arXiv:2211.02701 (2022)

    Pith/arXiv arXiv 2022

  19. [19]

    In: CVPR (2016)

    He, K., et al.: Deep Residual Learning for Image Recognition. In: CVPR (2016)

    2016

  20. [20]

    Radiology 295(2), 328–338 (2020)

    Zwanenburg, A., et al.: The Image Biomarker Standardization Initiative: Stan- dardized Quantitative Radiomics for High-Throughput Image-based Phenotyping. Radiology 295(2), 328–338 (2020)

    2020

  21. [21]

    Biostatistics 8(1), 118–127 (2007)

    Johnson, W.E., Li, C., Rabinovic, A.: Adjusting batch effects in microarray ex- pression data using empirical Bayes methods. Biostatistics 8(1), 118–127 (2007)

    2007

  22. [22]

    The Lancet Oncology 17(4), 440–451 (2016)

    O’Sullivan, B., Huang, S.H., et al.: Development and validation of a staging sys- tem for HPV-related oropharyngeal cancer by the International Collaboration on Oropharyngeal cancer Network for Staging (ICON-S): a multicentre cohort study. The Lancet Oncology 17(4), 440–451 (2016)

    2016