Pith. sign in

REVIEW 4 major objections 6 minor 12 references

Chest Disease Detection In X-Ray Images Using Deep Learning Classification Method

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned CNNs classify chest X-rays into four classes at 96–98% test accuracy, and Grad-CAM heatmaps show which image regions drive each decision.

desk verdict Table 2's Xception row is mathematically impossible, so the paper's headline metric claims don't hold as printed, despite an honestly reported dataset and Grad-CAM caveats. read the letter →

arxiv 2505.22609 v1 pith:5EPAJHR4 submitted 2025-05-28 eess.IV cs.CVcs.LG

Alanna Hazlett , Naomi Ohashi , Timothy Rodriguez , Sodiq Adewole This is my paper
classification eess.IVcs.CVcs.LG
keywords chestX-rayclassificationtransferlearningconvolutionalneuralnetworksCOVID-19detectiontuberculosispneumoniaGrad-CAMmodelinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that four fine-tuned convolutional neural networks can classify chest X-ray images into COVID-19, normal, pneumonia, and tuberculosis classes with roughly 96 to 98 percent test accuracy on a combined dataset of 57,111 images, and that the ResNet50 variant is the best on validation and test. The authors argue this is the largest four-class dataset used for this task and that transfer learning from ImageNet-pretrained backbones is enough to reach competitive performance without training from scratch. They further claim that Grad-CAM heatmaps provide useful transparency, in one tuberculosis case overlapping with a clinician's annotations, and that this combination of accuracy and explainability moves the models closer to use in low-resource clinical settings. The paper also reports that Grad-CAM exposed dataset artifacts—PII boxes, arm tissue, and a pediatric normal class—that the models were using, which the authors take as a caution about generalization.

What carries the argument

The central machinery is transfer learning: ImageNet-pretrained CNN backbones (ResNet50, VGG16, Xception, EfficientNetV2B0) with their final layers fine-tuned on the aggregated X-ray dataset, plus hyperparameter-tuned dense heads. The companion mechanism is Grad-CAM, which computes a class-discriminative saliency map from the gradient of the class score with respect to the last convolutional feature maps, producing the heatmaps that show where each model looks. Transfer learning carries the accuracy claim; Grad-CAM carries the explainability claim.

What would settle it

Compute perceptual hashes of all 57,111 images and check train/test duplicates: near-duplicate images crossing the split, or a patient-disjoint re-test with accuracy dropping materially below 96%, would falsify the generalization claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that fine-tuned transfer-learning CNN classifiers—ResNet50, VGG16, Xception, and EfficientNetV2B0—all exceed 96% test accuracy on a four-class chest X-ray task, with EfficientNetV2B0 highest at 98.24% and ResNet50 chosen as the best overall for stable training and strong generalization. The paper further reports that Grad-CAM visualizations can highlight lung regions a medical student also flagged as diagnostic, while also revealing that some classifications rest on class-correlated artifacts outside the lungs, meaning the models are not purely learning disease signal. The authors conclude it is possible to improve the explainability of high-performing models, with the remaining work being removal of dataset idiosyncrasies before clinical deployment.

Load-bearing premise

The accuracy claim rests on the assumption that the random split into train, validation, and test sets is fair—no duplicate or same-patient images cross the boundary and the test set represents the disease population—so if that split leaks, all reported numbers are inflated.

Editorial extensions

If this is right

  • ImageNet-pretrained backbones plus a small amount of fine-tuning can produce four-class chest X-ray classifiers at 96% or better test accuracy, so a heavy custom architecture is not required.
  • Grad-CAM provides a practical audit signal: when heatmaps fall outside the lungs, the model is likely exploiting dataset artifacts rather than disease signal.
  • ResNet50 with only its last ten layers trainable offers a strong balance of accuracy and stability, supporting deployment where compute is limited.
  • Before deployment, models need preprocessing to remove PII masks and rebalancing of patient age across classes, because those cues currently leak into classification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the random image-level split is not patient-disjoint or duplicate-free, the reported 96–98% accuracy is an upper bound; a patient-exclusive split would be needed to confirm generalization.
  • Because the normal class skews pediatric and several classes contain PII boxes, a model trained on this dataset is likely to transfer poorly to hospital X-rays without those artifacts; cropping and age-balancing are testable fixes.
  • The Grad-CAM agreement with a single clinician on a single TB image is proof-of-concept; computing a quantitative overlap score over hundreds of images would turn this into a measurable claim.
  • Public dataset aggregation of this kind likely contains near-duplicate images across sources; a deduplication pass before splitting would be the cheapest way to test how much of the accuracy is memorization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This manuscript reports a transfer-learning study for chest X-ray classification into four classes (COVID-19, normal, pneumonia, tuberculosis) using a combined dataset of 57,111 images. Four pre-trained CNNs (ResNet50, VGG16, Xception, EfficientNetV2B0) are fine-tuned and evaluated, and Grad-CAM is used for interpretability. The paper claims state-of-the-art accuracy, AUC, F1-score, recall, and precision on this dataset, and argues that the work improves the explainability and generalizability of high-performing models. A single medical student (SME) comparison on one tuberculosis image is presented to support the interpretability claim, and the paper candidly reports that Grad-CAM revealed class-correlated dataset artifacts such as PII boxes and arm tissue.

Significance. If the reported results were fully supported, the manuscript would provide a large multi-source benchmark for four-class chest X-ray classification, a practical comparison of four transfer-learning backbones, and a useful demonstration of Grad-CAM as a tool for surfacing dataset artifacts. The authors' honest discussion of spurious correlations in Section 6.8 is a genuine strength. However, the central empirical claims are not currently established: Table 2 contains a mathematically impossible metric combination, Section 3.1 does not describe how the train/test split was generated or whether patients were deduplicated, and the state-of-the-art claim is asserted without any head-to-head comparison to prior models on a shared benchmark. The interpretability claim rests on a single anecdotal image. The significance is therefore conditional on substantial revision.

major comments (4)
  1. [Table 2] Table 2, Xception row: the reported F1-score of 97.02% is mutually inconsistent with precision 69.80% and recall 99.51%. Under the standard definition F1 = 2PR/(P+R), substituting P = 0.6980 and R = 0.9951 gives F1 ≈ 0.820, not 0.9702; even a macro-average over four classes cannot reach 0.9702 in this situation unless most classes have F1 = 1.0, which conflicts with the row's aggregated precision. At least one cell in this row is numerically impossible. Because Section 6 uses these metrics to assert state-of-the-art performance and Section 7 repeats the claim, the main empirical evidence cannot be accepted as reported. Please correct the metrics, provide the confusion matrices, and state the exact averaging scheme used.
  2. [Section 3.1] Section 3.1 reports only folder counts for train, validation, and test splits and does not describe how the split was generated, whether duplicate images were removed across the three source datasets, or whether images from the same patient can appear in more than one split. Since the dataset combines two Kaggle sources plus TB Portal data, patient-level leakage is a real risk. If duplicate or same-patient images cross the train/test boundary, the reported test accuracies are inflated. Please describe the split procedure (random image-level vs patient-level), any deduplication steps, and how the authors verified that no individual appears in both training and test sets. This is load-bearing for the generalization claims in Sections 6 and 7.
  3. [Section 6 and Section 7] The claim of 'state of the art accuracy, AUC, F1-score, recall, and precision' is not supported by the experiments as reported. No prior model is evaluated on the same test split, and the literature reviewed in Section 4 uses different datasets, class sets, and evaluation protocols. The statement in Section 6 that the dataset is the largest for this four-class task does not by itself establish state-of-the-art performance. To support the claim, the paper should either include direct head-to-head comparisons on a shared benchmark or substantially scope the claim down to 'high accuracy on this aggregated dataset.' Without such a comparison, the central novelty claim is unsubstantiated.
  4. [Sections 6.3 and 6.8] The interpretability claim is supported by a single anecdotal comparison of one tuberculosis image in Section 6.3. Section 6.8 itself reports that Grad-CAM frequently highlighted non-diagnostic features such as PII boxes and arm tissue, and the authors note instability in heatmap generation. With one image, no quantitative overlap metric, and no systematic evaluation across classes, the conclusion in Section 7 that the work improves the explainability of high-performing models is not established. A quantitative evaluation, e.g., Grad-CAM overlap with expert annotations on multiple images per class or a localization benchmark, would be needed to support this claim.
minor comments (6)
  1. [Abstract and Section 3.1] The manuscript contains grammatical errors and typos, e.g., 'on a labeled medical x-ray images' in the Abstract and 'to a be significant challenge' in Section 3.1. The paper should be carefully proofread.
  2. [Figure 3] Figure 3 shows an elephant image from the Keras Grad-CAM example, which is irrelevant to chest X-ray analysis. Replace it with a chest X-ray example or remove it.
  3. [Section 6.4.1 and 6.7.1] There are typos such as 'cov2' (should be 'conv2') and 'Notablly' (should be 'Notably'). These should be corrected.
  4. [Section 6.2] Section 6.2 selects ResNet50 as the best model based on validation performance, but Table 2 shows EfficientNetV2B0 has higher test accuracy (98.24% vs 97.99%). The selection rule should be stated more explicitly and consistently.
  5. [Section 6.8] Section 6.8 notes that Grad-CAM heatmaps differed depending on whether a single image or a group of images was displayed. This is a reproducibility concern; please clarify the technical cause and how results were generated consistently, or move this to a clearly labeled limitation.
  6. [Section 5.1] The exact pretrained model versions and deep learning framework (e.g., Keras/TensorFlow) are not specified. Please provide this information for reproducibility, including any preprocessing normalize details for each architecture.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: test metrics are measured on a held-out split; the Xception metric inconsistency and SOTA overclaim are correctness issues, not circularity.

full rationale

The paper is an empirical transfer-learning study, not a derivation. The central results in Section 6 and Table 2 (accuracy, AUC, F1-score, recall, precision) are measurements on a held-out test folder, and the models are standard pre-trained CNNs fine-tuned on a separate training folder. There is no fitted parameter renamed as a prediction and no equation that reduces a claimed output to its own input. The Grad-CAM analysis is qualitative and does not constitute a derivation. The phrase 'state of the art accuracy ... using our bespoke dataset' is an overclaim because no external benchmark or shared comparison set is used, but that is a validity or framing problem, not circularity. Likewise, the internally inconsistent Xception row (F1 = 97.02% with precision 69.80% and recall 99.51% is arithmetically impossible under standard micro-averaging and implausible under macro-averaging) is a correctness risk in the reported evidence, not a circularity of reasoning. No load-bearing self-citation appears: the cited prior works are external, and the paper's own empirical loop is standard supervised learning with an independent test split.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or model entities are proposed. The central claim rests on standard transfer-learning assumptions, unverified source labels, an assumed leak-free split, validation-chosen hyperparameters, and an inference from dataset size to generalization that the paper's own Grad-CAM findings contradict. The reported numbers are measurements, not derivations, so the circularity burden is low apart from the unsupported state-of-the-art self-assessment.

free parameters (4)
  • Validation-selected hyperparameters (dense layer size, activation, learning rate, trainable layer count) = Not fully reported; EfficientNetV2B0 example: 225 neurons, 0.4 dropout, learning rate 1e-4 then 5e-5
    These were chosen via hyperparameter tuning and influence the reported accuracies; without the chosen values for ResNet50, VGG16, and Xception, the results cannot be reproduced.
  • Fixed training duration and early stopping behavior = 10 epochs; early stopping and ReduceLROnPlateau used without stated patience
    Training length and early-stopping settings are design choices that alter final weights and test metrics.
  • Dataset split assignment = Train 41,219, validation 9,549, test 6,343 images
    The split is an author choice; no random seed or splitting script is given, and no patient-level separation is demonstrated.
  • Added TB samples and class composition = 10,374 TB images added; class shares COVID 15.0%, Normal 28.3%, Pneumonia 32.7%, TB 24.0%
    The rebalancing decision changes class priors and therefore accuracy, precision, and recall.
assumptions (5)
  • domain assumption ImageNet-pretrained weights transfer useful visual features to chest X-rays.
    The whole transfer-learning strategy in Section 5.1 depends on this; it is plausible but not validated for this data.
  • domain assumption Source dataset labels (Kaggle and TB Portal) are correct ground truth.
    No radiologist re-review or label validation is described; mislabeled training images would bias all metrics.
  • domain assumption The train/validation/test folders are independent and free of patient overlap.
    Section 3.1 gives only counts; no deduplication at patient level and no check across Kaggle and TB Portal sources is reported.
  • standard math Gradient descent on these architectures reaches a sufficiently good optimum within 10 epochs.
    Standard deep learning behavior is assumed without convergence analysis or repeated runs.
  • ad hoc to paper A larger dataset than comparable prior work implies stronger generalization.
    Section 6 asserts confidence in generalization from dataset size, but the paper's own artifact findings contradict this inference.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Chest Disease Detection In X-Ray Images Using Deep Learning Classification Method." pith.science (2026). https://pith.science/paper/5EPAJHR4

@misc{pith2026250522609,
  author       = {Pith},
  title        = {Pith review of: Chest Disease Detection In X-Ray Images Using Deep Learning Classification Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5EPAJHR4}},
  note         = {Machine review of arXiv:2505.22609}
}
read the original abstract

In this work, we investigate the performance across multiple classification models to classify chest X-ray images into four categories of COVID-19, pneumonia, tuberculosis (TB), and normal cases. We leveraged transfer learning techniques with state-of-the-art pre-trained Convolutional Neural Networks (CNNs) models. We fine-tuned these pre-trained architectures on a labeled medical x-ray images. The initial results are promising with high accuracy and strong performance in key classification metrics such as precision, recall, and F1 score. We applied Gradient-weighted Class Activation Mapping (Grad-CAM) for model interpretability to provide visual explanations for classification decisions, improving trust and transparency in clinical applications.

Figures

Figures reproduced from arXiv: 2505.22609 by the authors.

Figure 1
Figure 1. Chest X-ray images 4 Literature Survey We conducted an extensive literature survey, selected the six most relevant articles, and reviewed each model’s key approaches, methodology, and challenges. All the models achieved over 90% accuracy using convolutional neural network design compared to their benchmarks. Aravinda et al. [2021] highlighted their model’s explainability, which is essential for building trust in cli… view at source ↗
Figure 2
Figure 2. CNN Architecture Convolutional neural networks architecture with convolution, polling, and fully connected layerGéron [2019] 5.2 Visualization Method We applied Gradient-weighted Class Activation Map, known as Grad-CAM, for our visualization method to obtain a class activation heatmap for an image classification model. The Grad-CAM heatmap indicates warmer colors (red/yellow) highlight areas that a model predicts wi… view at source ↗
Figure 3
Figure 3. Grad-CAM Heatmap Application 6 Results Our experiments resulted in the achievement of state of the art accuracy, AUC, F1-score, recall, and precision using our bespoke dataset of 57,111 X-ray images across our four classes of COVID-19, Normal, Pneumonia, and TB. As seen in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (11 more)
Figure 5
Figure 5. Figure 5: Training Loss over 10 Epochs [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Validation Accuracy over 10 Epochs [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 8
Figure 8. Figure 8: SME Evaluation of TB X-ray (left) vs Grad-CAM with VGG16 (right) [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]
Figure 9
Figure 9. Figure 9: Confusion Matrix, ResNet50 [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: First 4 test images with prediction and Grad-CAM overlay, ResNet50 [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Confusion Matrix, VGG16 True: COVID-19 Pred: COVID-19 (1.00) True: Normal Pred: Normal (1.00) True: Pneumonia Pred: Pneumonia (1.00) True: Tuberculosis Pred: Tuberculosis (1.00) [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: First 4 test images with prediction and Grad-CAM overlay, VGG16 [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 13
Figure 13. Figure 13: Confusion Matrix, Xception [PITH_FULL_IMAGE:figures/full_fig_p009_13.png]
Figure 14
Figure 14. Figure 14: First 4 test images with prediction and Grad-CAM overlay, Xception [PITH_FULL_IMAGE:figures/full_fig_p009_14.png]
Figure 15
Figure 15. Figure 15: Confusion Matrix EfficientNetB0 [PITH_FULL_IMAGE:figures/full_fig_p010_15.png]
Figure 16
Figure 16. Figure 16: First 4 test images with prediction and Grad-CAM overlay, EfficientNetB0 [PITH_FULL_IMAGE:figures/full_fig_p010_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 8 canonical work pages

  1. [1]

    The TB portals: an open-access, web-based platform for global drug-resistant-tuberculosis data sharing and analysis

    Alex Rosenthal, Andrei Gabrielian, Eric Engle, Darrell E Hurt, Sofia Alexandru, Valeriu Crudu, Eugene Sergueev, Valery Kirichenko, Vladzimir Lapitskii, Eduard Snezhko, Vassili Kovalev, Andrei Astrovko, Alena Skrahina, Jessica Taaffe, Michael Harris, Alyssa Long, Kurt Wollenberg, Irada Akhundova, Sharafat Ismayilova, Aliaksandr Skrahin, Elcan Mammadbayov, ...

    work page 2017

  2. [2]

    Aravinda, Meng Lin, K.R

    C.V. Aravinda, Meng Lin, K.R. Udaya Kumar Reddy, and G. Amar Prabhu. A demystifying convolutional neural networks using grad- CAM for prediction of coronavirus disease ( COVID -19) on x-ray images. In Data Science for COVID -19 , pages 429--450. Elsevier, 2021. ISBN 9780128245361. doi:10.1016/B978-0-12-824536-1.00037-X. URL https://linkinghub.elsevier.com...

    work page doi:10.1016/b978-0-12-824536-1.00037-x 2021

  3. [3]

    A comprehensive analysis of deep learning-based approaches for prediction and prognosis of infectious diseases

    Kavita Thakur, Manjot Kaur, and Yogesh Kumar. A comprehensive analysis of deep learning-based approaches for prediction and prognosis of infectious diseases. Archives of computational methods in engineering : state of the art reviews, pages 1--21, jun 2023. doi:10.1007/s11831-023-09952-7. URL http://dx.doi.org/10.1007/s11831-023-09952-7

    work page doi:10.1007/s11831-023-09952-7 2023

  4. [4]

    Inception-v4, inception- ResNet and the impact of residual connections on learning

    Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. Inception-v4, inception- ResNet and the impact of residual connections on learning. arXiv, 2016. doi:10.48550/arxiv.1602.07261. URL https://arxiv.org/abs/1602.07261

    arXiv doi:10.48550/arxiv.1602.07261 2016

  5. [5]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. arXiv, 2015. doi:10.48550/arxiv.1512.03385. URL https://arxiv.org/abs/1512.03385

    arXiv doi:10.48550/arxiv.1512.03385 2015

  6. [6]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv, 2014. doi:10.48550/arxiv.1409.1556. URL https://arxiv.org/abs/1409.1556

    arXiv doi:10.48550/arxiv.1409.1556 2014

  7. [7]

    COVID -net: a tailored deep convolutional neural network design for detection of COVID -19 cases from chest x-ray images

    Linda Wang, Zhong Qiu Lin, and Alexander Wong. COVID -net: a tailored deep convolutional neural network design for detection of COVID -19 cases from chest x-ray images. Scientific Reports, 10 0 (1): 0 19549, nov 2020. doi:10.1038/s41598-020-76550-z. URL http://dx.doi.org/10.1038/s41598-020-76550-z

    work page doi:10.1038/s41598-020-76550-z 2020

  8. [8]

    CDC\_Net : multi-classification convolutional neural network model for detection of COVID -19, pneumothorax, pneumonia, lung cancer, and tuberculosis using chest x-rays

    Hassaan Malik, Tayyaba Anees, Muizzud Din, and Ahmad Naeem. CDC\_Net : multi-classification convolutional neural network model for detection of COVID -19, pneumothorax, pneumonia, lung cancer, and tuberculosis using chest x-rays. Multimedia tools and applications, 82 0 (9): 0 13855--13880, 2023. doi:10.1007/s11042-022-13843-7. URL http://dx.doi.org/10.100...

    work page doi:10.1007/s11042-022-13843-7 2023

Show all 12 references
  1. [9]

    COVID ‑19 detection from chest x-ray images using transfer learning

    Enas M F El Houby. COVID ‑19 detection from chest x-ray images using transfer learning. Scientific Reports, 14 0 (1): 0 11639, may 2024. doi:10.1038/s41598-024-61693-0. URL http://dx.doi.org/10.1038/s41598-024-61693-0

    2024 doi

  2. [10]

    Detection of COVID -19 based on chest x-rays using deep learning

    Walaa Gouda, Maram Almurafeh, Mamoona Humayun, and Noor Zaman Jhanjhi. Detection of COVID -19 based on chest x-rays using deep learning. Healthcare (Basel), 10 0 (2), feb 2022. doi:10.3390/healthcare10020343. URL http://dx.doi.org/10.3390/healthcare10020343

    2022 doi

  3. [11]

    Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow : Concepts, Tools, and Techniques to Build Intelligent Systems

    Aurélien Géron. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow : Concepts, Tools, and Techniques to Build Intelligent Systems . O'Reilly Media, Beijing [China], 2 edition, nov 2019. ISBN 9781492032649

    2019

  4. [12]

    Mingxing Tan and Quoc V. Le. EfficientNetV2 : Smaller models and faster training. arXiv, 2021. doi:10.48550/arxiv.2104.00298. URL https://arxiv.org/abs/2104.00298

    2021 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.