Predicting Patient Survival with Airway Biomarkers using nn-Unet/Radiomics

Clement Chatelain; Dhruv Jain; Romain Herault; Romain Modzelewski; Sebastien Thureau; Simon Bernard; Tsiry Mayet; Zacharia Mesbah

REVIEW 4 major objections 5 minor 8 references

Predicting Patient Survival with Airway Biomarkers using nn-Unet/Radiomics

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a three-stage pipeline using nn-Unet airway segmentation, radiomic features from the trachea and the airway's bounding box, and an SVM classifier predicts 63-week survival in lung-fibrosis patients, achieving an…

desk verdict A competent AIIB 2023 competition write-up: solid segmentation, but the central survival-biomarker claim is a validation-selected 2-patient gap with no error bars. read the letter →

arxiv 2506.11677 v1 pith:P7WP4IJF submitted 2025-06-13 cs.CV cs.LG

Zacharia Mesbah , Dhruv Jain , Tsiry Mayet , Romain Modzelewski , Romain Herault , Simon Bernard , Sebastien Thureau , Clement Chatelain This is my paper · ORCID

classification cs.CVcs.LG

keywords survivalanalysissegmentationdeeplearningradiomicsairwaybiomarkerslungfibrosistracheaSVMclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that survival after a 63-week horizon in lung-fibrosis patients can be predicted from CT images by first segmenting the airways with nn-Unet, then extracting radiomic features from two regions (the trachea and a minimum bounding box around the entire airway), and finally classifying with a support vector machine. On the AIIB 2023 validation set this three-stage pipeline achieves an overall score of 0.7346 for survival classification, and the combination of trachea and bounding-box-airway features outperforms features from the full airway, trachea alone, or non-trachea regions. The authors interpret this as evidence that the trachea's shape and the overall airway dimensions carry survival-relevant information that standard whole-volume radiomics miss.

What carries the argument

The load-bearing mechanism is the trachea-isolation heuristic: in each predicted airway mask, the largest connected component in the upper third of the axial plane is retained and components in the lower third are removed, producing a trachea mask, while the complement yields the non-trachea mask. Radiomic features from the trachea mask and from a minimum bounding box around the full airway mask are then computed with an open-source radiomics package and filtered by Pearson correlation (threshold 0.20 for trachea, 0.41 for airway). These selected features are fed to a support vector machine with an RBF kernel, whose hyperparameters (C=8000, gamma=0.01) are chosen by grid search using 5-fold cross-validation. The mechanism ties together the segmentation quality, the region-choice heuristic, and the feature-selection thresholds.

What would settle it

A concrete test: on the same CT images, replace the automatic trachea mask with a same-size cylinder of lung tissue in the upper thorax and rerun the SVM; if classification accuracy does not drop, the claimed trachea-specific signal is an artifact. More directly, an independent cohort with manual expert trachea annotations could test whether the automatically isolated trachea reproduces the reported 0.7346 score, and a permutation test on the 120 training cases' labels would settle whether the combined-feature model beats chance despite the small validation set.

Watch

Extended reading notes

Core claim

The central claim is that a region-wise radiomic analysis of the segmented airway tree, rather than whole-lung or whole-airway radiomics, yields a better survival classifier for fibrotic lung disease. The paper shows that combining radiomic features extracted from the isolated trachea mask with those extracted from a minimum bounding box around the full airway mask produces the best validation-set metrics (accuracy 0.7308, F1-score 0.75, overall score 0.7346), supporting the hypothesis that the trachea's compressed or elongated appearance and the overall airway width are informative biomarkers. The claim is established through a specific pipeline: nn-Unet 3D full-resolution segmentation, a connected-component heuristic to separate the trachea, radiomic feature extraction, Pearson-correlation feature selection at thresholds of 0.20 and 0.41, and an RBF-kernel SVM with C=8000 and gamma=0.01.

Load-bearing premise

The load-bearing premise is that the largest connected component in the upper third of the axial CT slices reliably isolates the trachea, and that the trachea's appearance plus the overall airway dimensions actually carry survival-relevant information; if this region-selection heuristic fails or the radiomic signal is noise, the classifier's edge over whole-airway features collapses.

Editorial extensions

If this is right

If the central claim holds, airway radiomics centered on the trachea and the airway's minimum bounding box can serve as a reproducible, image-based biomarker for survival in fibrotic lung disease, complementing or replacing lab-dependent FVC measurements.
The pipeline can be applied automatically to a single CT scan without manual annotation, since nn-Unet supplies the masks and the trachea is isolated by a deterministic connected-component rule.
The superior performance of Combined(Trachea, Airway) over full-airway features suggests that fine-grained regional radiomics, rather than whole-structure summaries, should be the default design for similar survival-classification tasks.
The validation on 52 patients in the AIIB 2023 validation set supports pursuing larger prospective studies of tracheal radiomics as a prognostic marker.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An editorial extension: the heuristic of 'largest connected component in the upper third' is a proxy for anatomical trachea location; a direct comparison against expert trachea annotations on the same CT scans would test whether the survival signal is truly tracheal or merely a generic upper-airway effect.
Because the validation set has only 52 patients and labels were evaluated by a leaderboard, the reported 0.7346 overall score carries wide uncertainty; a permutation test or leave-one-center-out evaluation on the 120-patient training set would gauge whether the feature combination generalizes beyond the competition split.
The Pearson correlation thresholds (0.20 and 0.41) appear tuned to the validation set; a sensitivity analysis sweeping these thresholds would reveal whether the advantage of the combined feature set is stable or an artifact of the chosen cutoffs.
A testable extension: feed the same bounding-box and trachea radiomic features into a Cox proportional-hazards model on time-to-death data rather than the binarised 63-week outcome, to see whether the same features carry proportional-hazard information.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a clean write-up of an AIIB 2023 competition solution. The segmentation pipeline is solid; nn-Unet with post-processing reaching 0.86 Overall-Score is a credible result. The genuinely new bit is the attempt to isolate the trachea and test whether trachea radiomics carry survival signal. That hypothesis is worth taking seriously, and the engineering heuristic (largest connected component in the upper third of the axial plane) is simple and reproducible.

The soft spot is not the heuristic; it's the evidence that combined trachea+airway features beat other feature sets. Table 2 is computed on 52 validation scans, and the winning gap over trachea-only is 0.0385 accuracy—roughly two patients. No confidence intervals, no test. The authors also selected SVM hyperparameters (C=8000, gamma=0.01), correlation thresholds (0.20, 0.41), and the winning feature set using the same validation set they then report as the result. That produces an optimistically biased number, and with 52 samples the uncertainty blows any 0.03–0.05 gap wide open. The paper doesn't claim significance, but it does present Combined(Trachea, Airway) as 'the most favorable,' which is only true on this one small, self-selected fold.

The missing code and data don't help, though the trachea separation rule is described well enough to re-implement. I also want to flag an internal inconsistency: the abstract says 0.7346 for classification, but Table 2 shows 0.7308 accuracy for the combined feature set. That's a minor mismatch but worth fixing.

Who is this for? Anyone looking for a baseline or a sanity check in the AIIB challenge, and radiomics researchers interested in trachea-specific features. It's not yet a clinical biomarker paper. I'd send it to review only with the expectation that the authors disclose the validation-selection procedure, report test set scores if available, and add at least a paired bootstrap or a simple significance test on the gap. Without that, the central claim about combined features is speculation on two patients. If the editors want a competition-report venue, it's fine as-is; if the claim is meant to support a biological finding, it needs independent validation.

Referee Report

4 major / 5 minor

Summary. The paper describes a three-stage pipeline for the AIIB 2023 challenge Task 2 (predicting whether a lung fibrosis patient survives beyond 63 weeks). First, nn-Unet segments the airway tree in CT; second, post-processing splits the airway into a trachea mask and an airway bounding box, from which PyRadiomics features are extracted; third, an SVM with RBF kernel classifies patients. The authors report an Overall-Score of 0.8601 for the segmentation task and 0.7346 (Table 2, accuracy 0.7308) for the survival classification on the validation set, and claim that combining trachea and airway features outperforms other feature sets.

Significance. If the reported results were unbiased, the paper would offer a useful engineering contribution and a testable hypothesis about tracheal radiomics as survival biomarkers in fibrotic lung disease. The strengths are the use of a well-established segmentation tool (nn-Unet), a reproducible PyRadiomics feature extraction pipeline, and a clear three-stage description. The main value is the negative/explanatory finding that simple radiomics on a competition validation set may not support strong claims of biomarker superiority. However, the current evaluation is not sufficient to establish the central claims, because the validation set was used for model selection and the reported differences are within sampling noise.

major comments (4)

[§2.3 and §3] The validation set serves simultaneously as the model-selection set and the evaluation set: SVM hyperparameters (C=8000, gamma=0.01) were chosen by grid search, Pearson correlation thresholds (0.20 and 0.41) were set based on validation performance, and the best of several feature sets was selected from the same validation leaderboard. Consequently, the reported accuracy of 0.7308 (Overall-Score 0.7346) is an optimistically biased estimate of generalization, not an unbiased test. The paper should provide a proper evaluation protocol, e.g., nested cross-validation on the 95 labeled training patients or evaluation on an independent test set, together with confidence intervals.
[Table 2] The central claim that Combined(Trachea, Airway) outperforms other feature sets rests on a difference of 0.0385 in accuracy (0.7308 vs 0.6923) and 0.0577 in F1-score, i.e., roughly 2 out of 52 validation patients. With no significance test, no confidence intervals, and multiple feature sets compared on the same data, this difference is well within the expected sampling variability. The authors should either demonstrate that the gap is statistically reliable or temper the conclusion to a hypothesis-generating observation.
[§2.2] The trachea-isolation rule is under-specified and load-bearing for all subsequent radiomic features. The description 'retaining solely the biggest connected component within the upper third of the axial plane and omitting them within the lower third' leaves the spatial reference ambiguous (z-coordinate relative to the volume? anatomical landmarks?) and does not state how the 2D-slice operations in Figure 1 correspond to a 3D mask. The authors should specify the exact definition, provide example masks, and assess the sensitivity of the classification results to the choice of the D/3 threshold.
[§3, 'No Trachea' row] The 'No Trachea' and 'Full Airway' rows show specificity of 0.1923 and 0.1538, respectively, meaning the classifier labels nearly all patients as survivors. This suggests a strong class-imbalance or calibration problem, yet the paper does not discuss how class proportions (36 deaths vs 59 survivors in training) are handled. The authors should report the class distribution in the validation set and clarify whether any class-weight or threshold adjustment was applied.

minor comments (5)

[§2.2 heading] Typo: 'Radiomic Features Extration' should be 'Extraction'.
[Abstract] Typo: 'anOverall-Score' should be 'an Overall-Score'.
[§1.1 Evaluation] The formula for Overall-Score is missing an operator before the leakage term: it reads '(1−leakages)⋆0.3'.
[§2.3] The paper states that 5-fold cross-validation was performed on the training set, but it is not reported which fold/model was used for the validation predictions. Please clarify whether the validation predictions come from a single model retrained on all training data or from an ensemble of folds.
[§3] The paper says the results are 'computed on the validation set' but does not give the number of features retained after Pearson correlation for each mask, nor the total number of features fed to the SVM. Reporting these numbers would improve reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survival prediction pipeline is a standard train/validation setup with model selection on training folds.

full rationale

The derivation chain is a standard three-stage supervised pipeline. Stage 1 trains nn-Unet on the provided training scans and masks; Stage 2 post-processes the predicted masks to isolate the trachea and airway bounding box and extracts PyRadiomics features; Stage 3 trains an RBF-kernel SVM on the training CT radiomics and survival labels, with hyperparameters (C=8000, gamma=0.01) selected by 5-fold cross-validation on the training set, then applies the model to the 52 validation scans. The paper explicitly states that validation labels were unavailable and that scores were obtained by submission to the validation phase, so the reported 0.7346 is an external leaderboard outcome rather than a fitted quantity. The trachea/airway division is a stated biological hypothesis, not a definition that already contains the survival outcome. The references to nn-Unet and PyRadiomics are standard external tools and not self-citations. Table 2 does compare several feature sets on the validation set and report the best row, which raises a legitimate concern about selection bias or small-sample noise, but that is a statistical inference issue, not constructional circularity: the selected features are not defined in terms of the validation labels, and no equation or threshold is shown to be fitted to the validation target. Since no load-bearing reduction to its own inputs can be quoted from the text, the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim of classification performance depends on multiple hand-chosen parameters and domain assumptions, rather than a closed-form derivation.

free parameters (4)

SVM RBF kernel hyperparameters = C=8000, gamma=0.01
Chosen by grid search on the training set, affecting classification performance.
Pearson correlation thresholds for feature selection = 0.20 for trachea, 0.41 for airway
Chosen to select radiomic features prior to classification.
Centroid distance threshold for artifact removal = Not specified (predefined threshold)
Used to exclude connected components far from the main airway component.
Trachea isolation rule = z < D/3 for retained slices
Heuristic to separate trachea from lower airway, chosen by hand.

assumptions (4)

domain assumption The trachea contains survival-relevant information
Stated in Section 2.2 as a hypothesis motivating the feature extraction.
domain assumption Images closer to the trachea are more relevant to survival
Assumed in Section 2.2 to justify extracting features from the trachea and its bounding box.
domain assumption Radiomic features capture meaningful texture and shape information
Relies on the established PyRadiomics framework, cited from the literature.
domain assumption The segmentation masks are accurate enough for radiomic feature extraction
The quality of the downstream classification depends on the segmentation quality, which is reported as an Overall-Score of 0.8601.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Predicting Patient Survival with Airway Biomarkers using nn-Unet/Radiomics." pith.science (2026). https://pith.science/paper/P7WP4IJF

@misc{pith2026250611677,
  author       = {Pith},
  title        = {Pith review of: Predicting Patient Survival with Airway Biomarkers using nn-Unet/Radiomics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P7WP4IJF}},
  note         = {Machine review of arXiv:2506.11677}
}

read the original abstract

The primary objective of the AIIB 2023 competition is to evaluate the predictive significance of airway-related imaging biomarkers in determining the survival outcomes of patients with lung fibrosis.This study introduces a comprehensive three-stage approach. Initially, a segmentation network, namely nn-Unet, is employed to delineate the airway's structural boundaries. Subsequently, key features are extracted from the radiomic images centered around the trachea and an enclosing bounding box around the airway. This step is motivated by the potential presence of critical survival-related insights within the tracheal region as well as pertinent information encoded in the structure and dimensions of the airway. Lastly, radiomic features obtained from the segmented areas are integrated into an SVM classifier. We could obtain an overall-score of 0.8601 for the segmentation in Task 1 while 0.7346 for the classification in Task 2.

Figures

Figures reproduced from arXiv: 2506.11677 by the authors.

**Figure 2.** Artifacts shown in the predicted masks before post-processing [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

8 extracted references · 6 canonical work pages

[1]

Idiopathic pulmonary fibrosis (an update) and progressive pulmonary fibrosis in adults: An official ATS/ERS/JRS/ALAT clinical practice guideline

Ganesh Raghu, Martine Remy-Jardin, Luca Richeldi, Carey C Thomson, Yoshikazu Inoue, Takeshi Johkoh, Michael Kreuter, David A Lynch, Toby M Maher, Fernando J Martinez, Maria Molina-Molina, Jeffrey L Myers, Andrew G Nicholson, Christopher J Ryerson, Mary E Strek, Lauren K Troy, Marlies Wijsenbeek, Manoj J Mammen, Tanzib Hossain, Brittany D Bissell, Derrick ...

work page 2022
[2]

Nintedanib in progressive interstitial lung diseases: data from the whole INBUILD trial

Kevin R Flaherty, Athol U Wells, Vincent Cottin, Anand Devaraj, Yoshikazu Inoue, Luca Richeldi, Simon L F Walsh, Martin Kolb, Dirk Koschel, Teng Moua, Susanne Stowasser, Rainer-Georg Goeldner, Rozsa Schlenker-Herceg, Kevin K Brown, and INBUILD Trial Investigators . Nintedanib in progressive interstitial lung diseases: data from the whole INBUILD trial. Eu...

work page 2022
[3]

Human treelike tubular structure segmentation: A comprehensive review and future perspectives, 12 2022

Hao Li, Zeyu Tang, Yang Nan, and Guang Yang. Human treelike tubular structure segmentation: A comprehensive review and future perspectives, 12 2022

work page 2022
[4]

Fuzzy attention neural network to tackle discontinuity in airway segmentation

Yang Nan, Javier Del Ser, Zeyu Tang, Peng Tang, Xiaodan Xing, Yingying Fang, Francisco Herrera, Witold Pedrycz, Simon Walsh, and Guang Yang. Fuzzy attention neural network to tackle discontinuity in airway segmentation. 9 2022

work page 2022
[5]

nnU-Net : a self-configuring method for deep learning-based biomedical image segmentation

Fabian Isensee, Paul F Jaeger, Simon A A Kohl, Jens Petersen, and Klaus H Maier-Hein. nnU-Net : a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods , 18(2):203--211, February 2021

work page 2021
[6]

Van Griethuysen, Andriy Fedorov, Chintan Parmar, Ahmed Hosny, Nicole Aucoin, Vivek Narayan, Regina G.H

Joost J.M. Van Griethuysen, Andriy Fedorov, Chintan Parmar, Ahmed Hosny, Nicole Aucoin, Vivek Narayan, Regina G.H. Beets-Tan, Jean Christophe Fillion-Robin, Steve Pieper, and Hugo J.W.L. Aerts. Computational radiomics system to decode the radiographic phenotype. Cancer Research , 77:e104--e107, 11 2017

work page 2017
[7]

, " * write output.state after.block = add.period write

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...
[8]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Idiopathic pulmonary fibrosis (an update) and progressive pulmonary fibrosis in adults: An official ATS/ERS/JRS/ALAT clinical practice guideline

Ganesh Raghu, Martine Remy-Jardin, Luca Richeldi, Carey C Thomson, Yoshikazu Inoue, Takeshi Johkoh, Michael Kreuter, David A Lynch, Toby M Maher, Fernando J Martinez, Maria Molina-Molina, Jeffrey L Myers, Andrew G Nicholson, Christopher J Ryerson, Mary E Strek, Lauren K Troy, Marlies Wijsenbeek, Manoj J Mammen, Tanzib Hossain, Brittany D Bissell, Derrick ...

work page 2022

[2] [2]

Nintedanib in progressive interstitial lung diseases: data from the whole INBUILD trial

Kevin R Flaherty, Athol U Wells, Vincent Cottin, Anand Devaraj, Yoshikazu Inoue, Luca Richeldi, Simon L F Walsh, Martin Kolb, Dirk Koschel, Teng Moua, Susanne Stowasser, Rainer-Georg Goeldner, Rozsa Schlenker-Herceg, Kevin K Brown, and INBUILD Trial Investigators . Nintedanib in progressive interstitial lung diseases: data from the whole INBUILD trial. Eu...

work page 2022

[3] [3]

Human treelike tubular structure segmentation: A comprehensive review and future perspectives, 12 2022

Hao Li, Zeyu Tang, Yang Nan, and Guang Yang. Human treelike tubular structure segmentation: A comprehensive review and future perspectives, 12 2022

work page 2022

[4] [4]

Fuzzy attention neural network to tackle discontinuity in airway segmentation

Yang Nan, Javier Del Ser, Zeyu Tang, Peng Tang, Xiaodan Xing, Yingying Fang, Francisco Herrera, Witold Pedrycz, Simon Walsh, and Guang Yang. Fuzzy attention neural network to tackle discontinuity in airway segmentation. 9 2022

work page 2022

[5] [5]

nnU-Net : a self-configuring method for deep learning-based biomedical image segmentation

Fabian Isensee, Paul F Jaeger, Simon A A Kohl, Jens Petersen, and Klaus H Maier-Hein. nnU-Net : a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods , 18(2):203--211, February 2021

work page 2021

[6] [6]

Van Griethuysen, Andriy Fedorov, Chintan Parmar, Ahmed Hosny, Nicole Aucoin, Vivek Narayan, Regina G.H

Joost J.M. Van Griethuysen, Andriy Fedorov, Chintan Parmar, Ahmed Hosny, Nicole Aucoin, Vivek Narayan, Regina G.H. Beets-Tan, Jean Christophe Fillion-Robin, Steve Pieper, and Hugo J.W.L. Aerts. Computational radiomics system to decode the radiographic phenotype. Cancer Research , 77:e104--e107, 11 2017

work page 2017

[7] [7]

, " * write output.state after.block = add.period write

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

[8] [8]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize ":" * " " *...