REVIEW 4 major objections 5 minor 21 references
Advancing Lung Disease Diagnosis in 3D CT Scans
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Four-way lung CT classification reaches 0.80 Macro F1 with a 3D ResNeSt model.
desk verdict Single uncontextualized validation score with no baselines or error bars; the method is a competent composition of standard components, but 'strong performance' is not supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the combination of three components: (1) a preprocessing step that detects and removes non-lung slices from each 3D volume, cutting computation and focusing the model on lesions; (2) a 3D ResNeSt50 backbone, a residual network whose blocks include split-attention that weights interactions across feature-map groups; and (3) a weighted cross-entropy loss $L_{\text{wCE}} = -\sum_{c=1}^{C} w_c \, y_c \log(\hat{y}_c)$, where class weights $w_c$ are chosen to up-weight underrepresented classes such as squamous cell carcinoma. Together these allow the whole volume, not individual 2D slices, to drive the classification decision.
What would settle it
Re-run the same pipeline on multiple random splits of the provided validation set, or obtain the challenge test labels and compute per-class F1; if the Macro F1 drops below roughly 0.75 on any split, or the squamous cell carcinoma F1 confidence interval overlaps the majority-class baseline, the paper's 'strong performance' conclusion fails.
Extended reading notes
Core claim
On the paper's own terms, the discovery is a pipeline: after resizing each CT series to (64,256,256) and clipping intensities to [0,1], the model discards slices without lung tissue, extracts features with a 3D ResNeSt50 backbone whose split-attention blocks let feature-map groups interact, and trains with a lesion-aware weighted cross-entropy loss. The reported outcome is a Macro F1 score of 0.80 on the challenge validation set, where Macro F1 averages per-class F1 across normal, adenocarcinoma, squamous cell carcinoma, and COVID-19. The paper treats this number as showing the model's strong ability to distinguish lung conditions.
Load-bearing premise
The quantitative claim rests on the validation-set Macro F1 being stable and meaningful, but with 155 volumes, only 25 in the rarest class, and no error bars, per-class breakdown, or separate test set, the 0.80 score could shift materially with the split or the official F1 aggregation rule.
Editorial extensions
If this is right
- If the 0.80 Macro F1 holds on an independent test set, the same preprocessing and weighting recipe can be carried over to other volumetric medical classification tasks with rare classes.
- Removing non-lung slices lowers the input dimensionality, so the model can train on fewer compute resources (the paper uses four RTX 3090 GPUs) than a full-volume model.
- The weighted loss directly targets the squamous cell carcinoma class, so per-class F1 for that category should be the first number to inspect in any replication.
- Treating CT as a single 3D volume instead of a bag of 2D slices is claimed to avoid the suboptimal performance of slice-based approaches.
Reading between the lines
- The 0.80 Macro F1 is reported without error bars or a per-class breakdown; with only 25 squamous cell carcinoma validation cases, that rare-class F1 likely has a wide confidence interval, so the aggregate score may not be stable across splits.
- The non-lung slice removal is a fixed heuristic; a model that learns which slices matter from data, or one tested across scanners with different fields of view, would reveal whether the heuristic itself is essential.
- A direct comparison pairing this pipeline with the same ResNeSt50 trained on raw full volumes would isolate how much of the gain comes from preprocessing versus the backbone or the loss weighting.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a three-stage pipeline for four-class lung disease classification from 3D chest CT volumes: removal of non-lung slices, a 3D ResNeSt50 backbone, and a weighted cross-entropy loss intended to counteract class imbalance. The reported result is a Macro F1 of 0.80 on the validation split of the Fair Disease Diagnosis Challenge, and the paper states that this demonstrates strong performance. The evaluation consists of a single aggregate number, with no held-out test set, no per-class results, no confidence intervals, no baselines, and no ablations.
Significance. The proposed combination of components is plausible and the paper provides useful implementation details (volume resizing, augmentation, optimizer settings, batch size, epoch count) that would help others reproduce the pipeline. If the 0.80 Macro F1 were shown to be stable and favorably compared against alternatives, the recipe would be of practical interest for challenge-style lung-disease classification. However, the current evidence is too thin to support the stated conclusion: a single validation-set number, computed on a small cohort with a rare class, cannot by itself demonstrate 'strong performance' without error bars, per-class breakdowns, or comparators.
major comments (4)
- [§4.3 and Table 1] The central claim rests entirely on a single Macro F1 of 0.80 computed on the validation set of 155 volumes. With only 25 squamous cell carcinoma cases (13 female, 12 male) in that split, the per-class F1 for the rare class has wide sampling uncertainty, and the macro average is directly sensitive to that component. The paper reports no confidence interval, no per-class F1 values, and no number of training runs or seeds, so the reader cannot judge whether 0.80 is a stable estimate or a fluctuation of a particular split and initialization.
- [§4.3] The word 'strong' presumes a comparison that is never supplied. No baseline architecture, no variant without slice removal, no variant without weighted loss, and no reference to other challenge entries or published benchmarks are provided. Without a comparator, 0.80 is an uncontextualized number; adding even a small set of ablations (e.g., plain 3D ResNet, unweighted cross-entropy, or no slice removal) would make the claim falsifiable and interpretable.
- [§2, Eq. (1)] The weighted cross-entropy loss is presented as a contribution, but the class weights w_c are never specified. The reader cannot tell how much weight was given to squamous cell carcinoma or whether the weights were derived from the class frequencies in Table 1. Moreover, there is no ablation isolating this component, so the paper does not provide evidence that the weighting, rather than the backbone or the preprocessing, contributes to the reported 0.80.
- [§4.1 and §4.2] The slice-removal preprocessing is described only qualitatively. There is no specification of how non-lung slices are identified, how many slices are removed on average, or whether the threshold is fixed or scan-dependent. This matters because the method assumes that removed slices never contain diagnostically relevant tissue; if the rule occasionally removes apical or basal slices containing lesions, the evaluation could be biased. At minimum, the manuscript should quantify the removed slice range and validate the assumption on the training set.
minor comments (5)
- [Figure 2] The figure labels 'ResNestBlock' and 'ResNest' use a different capitalization from the text's 'ResNeSt50'; please make the notation consistent throughout.
- [Eq. (1)] The notation y, ŷ, y_c, and ŷ_c is not defined in detail; please state explicitly that y_c is the one-hot ground-truth component and ŷ_c is the predicted probability for class c.
- [Table 1] The caption uses abbreviations 'A', 'Covid', and 'G' without explanation; the table would be clearer if the caption defined that A stands for adenocarcinoma, G for squamous cell carcinoma, and Covid for COVID-19.
- [§4.2] The augmentation description 'random cropping on the vertical section to 64' is ambiguous given that volumes are already resized to depth 64 in §4.1; please clarify whether the crop is applied to the original depth before resizing or is a depth-preserving random crop.
- [General] The paper does not state whether code or trained models will be made available; adding a code-availability statement would improve reproducibility.
Circularity Check
No circularity: the reported 0.80 Macro F1 is an empirical evaluation metric on held-out validation data, and no claimed prediction reduces to a fitted input or self-citation by construction.
full rationale
The paper contains no derivation chain in which a predicted quantity is equivalent to a fitted input by construction. The central result is a Macro F1 score on the validation set of the Fair Disease Diagnosis Challenge (Section 4.3), computed by calculating F1 for each category and averaging (Section 4.2); it is an external evaluation metric applied to held-out data, not an output of the training objective. The weighted cross-entropy loss (Eq. 1) uses class weights w_c to upweight underrepresented classes, but these weights are training-loss design choices and the validation Macro F1 is not a function of w_c by construction. The preprocessing step (removing non-lung slices) and the ResNeSt50 backbone are standard engineering choices, not results derived from the evaluation metric. References [19] and [20] are prior works by the same authors on COVID-19 detection, and they are cited as background, not as load-bearing evidence for the 0.80 Macro F1 claim. No uniqueness theorem, ansatz-smuggling, or renaming of a known empirical pattern is present. Concerns about the absence of per-class F1 values, confidence intervals, or comparator baselines are correctness and robustness concerns, not circularity: a single uncontextualized validation number may be weak evidence, but it is not a derivation that reduces to its own inputs. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (1)
- Class weights w_c in the weighted cross-entropy loss (Eq. 1) =
not reported; described as based on class/lesion-volume proportions
assumptions (3)
- domain assumption 3D adaptation of ResNeSt50 preserves the split-attention benefits of the 2D architecture.
- ad hoc to paper Removing slices identified as non-lung (neck, abdomen) never removes diagnostically relevant tissue.
- domain assumption The challenge validation set annotations and the Macro F1 computation are correct and match the official evaluation.
Cite this review
Pith. "Pith review of Advancing Lung Disease Diagnosis in 3D CT Scans." pith.science (2026). https://pith.science/paper/7B254OAW
@misc{pith2026250700993,
author = {Pith},
title = {Pith review of: Advancing Lung Disease Diagnosis in 3D CT Scans},
year = {2026},
howpublished = {\url{https://pith.science/paper/7B254OAW}},
note = {Machine review of arXiv:2507.00993}
}
read the original abstract
To enable more accurate diagnosis of lung disease in chest CT scans, we propose a straightforward yet effective model. Firstly, we analyze the characteristics of 3D CT scans and remove non-lung regions, which helps the model focus on lesion-related areas and reduces computational cost. We adopt ResNeSt50 as a strong feature extractor, and use a weighted cross-entropy loss to mitigate class imbalance, especially for the underrepresented squamous cell carcinoma category. Our model achieves a Macro F1 Score of 0.80 on the validation set of the Fair Disease Diagnosis Challenge, demonstrating its strong performance in distinguishing between different lung conditions.
Figures
Reference graph
Works this paper leans on
-
[1]
A large imaging database and novel deep neural ar- chitecture for covid-19 diagnosis
Anastasios Arsenos, Dimitrios Kollias, and Stefanos Kol- lias. A large imaging database and novel deep neural ar- chitecture for covid-19 diagnosis. In 2022 IEEE 14th Im- age, Video, and Multidimensional Signal Processing Work- shop (IVMSP), page 1–5. IEEE, 2022. 1
2022
-
[2]
Data-driven covid-19 detection through medical imaging
Anastasios Arsenos, Andjoli Davidhi, Dimitrios Kollias, Panos Prassopoulos, and Stefanos Kollias. Data-driven covid-19 detection through medical imaging. In 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW), page 1–5. IEEE, 2023
work page 2023
-
[3]
Demetris Gerogiannis, Anastasios Arsenos, Dimitrios Kol- lias, Dimitris Nikitopoulos, and Stefanos Kollias. Covid- 19 computer-aided diagnosis through ai-assisted ct imaging analysis: Deploying a medical ai system. In 2024 IEEE In- ternational Symposium on Biomedical Imaging (ISBI), pages 1–4. IEEE, 2024
work page 2024
-
[4]
Cmc-cov19d: Contrastive mixup classification for covid-19 diagnosis
Junlin Hou, Jilan Xu, Rui Feng, Yuejie Zhang, Fei Shan, and Weiya Shi. Cmc-cov19d: Contrastive mixup classification for covid-19 diagnosis. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 454–461, 2021
work page 2021
-
[5]
Periphery-aware covid-19 diagnosis with contrastive repre- sentation enhancement
Junlin Hou, Jilan Xu, Longquan Jiang, Shanshan Du, Rui Feng, Yuejie Zhang, Fei Shan, and Xiangyang Xue. Periphery-aware covid-19 diagnosis with contrastive repre- sentation enhancement. Pattern Recognition, 118:108005, 2021
work page 2021
-
[6]
Cmc v2: Towards more accu- rate covid-19 detection with discriminative video priors
Junlin Hou, Jilan Xu, Nan Zhang, Yi Wang, Yuejie Zhang, Xiaobo Zhang, and Rui Feng. Cmc v2: Towards more accu- rate covid-19 detection with discriminative video priors. In European Conference on Computer Vision , pages 485–499. Springer, 2022
work page 2022
-
[7]
Boosting covid-19 severity detec- tion with infection-aware contrastive mixup classification
Junlin Hou, Jilan Xu, Nan Zhang, Yuejie Zhang, Xiaobo Zhang, and Rui Feng. Boosting covid-19 severity detec- tion with infection-aware contrastive mixup classification. In European Conference on Computer Vision , pages 537–551. Springer, 2022
work page 2022
-
[8]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,
Show all 21 references
-
[9]
Deep neural archi- tectures for prediction in healthcare
Dimitrios Kollias, Athanasios Tagaris, Andreas Stafylopatis, Stefanos Kollias, and Georgios Tagaris. Deep neural archi- tectures for prediction in healthcare. Complex & Intelligent Systems, 4(2):119–131, 2018
2018
-
[10]
Deep transparent prediction through latent represen- tation analysis
Dimitrios Kollias, N Bouas, Y Vlaxos, V Brillakis, M Se- feris, Ilianna Kollia, Levon Sukissian, James Wingate, and S Kollias. Deep transparent prediction through latent represen- tation analysis. arXiv preprint arXiv:2009.07044, 2020
2009 arXiv
-
[11]
Transpar- ent adaptation in deep medical image diagnosis
Dimitris Kollias, Y Vlaxos, M Seferis, Ilianna Kollia, Levon Sukissian, James Wingate, and Stefanos D Kollias. Transpar- ent adaptation in deep medical image diagnosis. In TAILOR, page 251–267, 2020
2020
-
[12]
Mia-cov19d: Covid-19 detection through 3-d chest ct image analysis
Dimitrios Kollias, Anastasios Arsenos, Levon Soukissian, and Stefanos Kollias. Mia-cov19d: Covid-19 detection through 3-d chest ct image analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision , page 537–544, 2021
2021
-
[13]
Ai-mia: Covid-19 detection and severity analysis through medical imaging
Dimitrios Kollias, Anastasios Arsenos, and Stefanos Kollias. Ai-mia: Covid-19 detection and severity analysis through medical imaging. In European Conference on Computer Vi- sion, page 677–690. Springer, 2022
2022
-
[14]
Ai-enabled analysis of 3-d ct scans for diagnosis of covid-19 & its severity
Dimitrios Kollias, Anastasios Arsenos, and Stefanos Kollias. Ai-enabled analysis of 3-d ct scans for diagnosis of covid-19 & its severity. In 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICAS- SPW), page 1–5. IEEE, 2023
2023
-
[15]
A deep neural architecture for harmonizing 3-d input data analysis and decision making in medical imaging
Dimitrios Kollias, Anastasios Arsenos, and Stefanos Kollias. A deep neural architecture for harmonizing 3-d input data analysis and decision making in medical imaging. Neuro- computing, 542:126244, 2023
2023
-
[16]
Domain adaptation explainability & fairness in ai for medical image analysis: Diagnosis of covid-19 based on 3-d chest ct-scans
Dimitrios Kollias, Anastasios Arsenos, and Stefanos Kol- lias. Domain adaptation explainability & fairness in ai for medical image analysis: Diagnosis of covid-19 based on 3-d chest ct-scans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ...
2024
-
[17]
Sam2clip2sam: Vision language model for segmentation of 3d ct scans for covid-19 detection
Dimitrios Kollias, Anastasios Arsenos, James Wingate, and Stefanos Kollias. Sam2clip2sam: Vision language model for segmentation of 3d ct scans for covid-19 detection. arXiv preprint arXiv:2407.15728, 2024
2024 arXiv
-
[18]
Out-of-distribution general- ization via risk extrapolation (rex)
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution general- ization via risk extrapolation (rex). In International confer- ence on machine learning , pages 5815–5826. PMLR, 2021
2021
-
[19]
Advancing covid-19 de- tection in 3d ct scans
Qingqiu Li, Runtian Yuan, Junlin Hou, Jilan Xu, Yuejie Zhang, Rui Feng, and Hao Chen. Advancing covid-19 de- tection in 3d ct scans. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition , pages 5149–5156, 2024
2024
-
[20]
Domain adaptation us- ing pseudo labels for covid-19 detection
Runtian Yuan, Qingqiu Li, Junlin Hou, Jilan Xu, Yuejie Zhang, Rui Feng, and Hao Chen. Domain adaptation us- ing pseudo labels for covid-19 detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5141–5148, 2024. 1
2024
-
[21]
Resnest: Split-attention networks
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Haibin Lin, Zhi Zhang, Yue Sun, Tong He, Jonas Mueller, R Manmatha, et al. Resnest: Split-attention networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 2736–2746, 2022. 1 3
2022
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.