Pith. sign in

REVIEW 5 major objections 6 minor 18 references

Deep Learning-Powered Classification of Thoracic Diseases in Chest X-Rays

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper sets out to show that fine-tuning ImageNet-pretrained networks with focal loss materially improves multi-label thoracic disease classification on chest X-rays, reporting InceptionV3's AUC rising from 0.500 to 0.6390 on a 10% NIH…

desk verdict Routine transfer-learning benchmark on a 10% NIH ChestX-ray14 subset, and the abstract's headline F1 claim is contradicted by the paper's own tables; not worth refereeing. read the letter →

arxiv 2501.14279 v1 pith:ZKO6ZGD7 submitted 2025-01-24 eess.IV cs.CV

Yiming Lei , Michael Nguyen , Tzu Chia Liu , Hyounkyun Oh This is my paper · ORCID
classification eess.IVcs.CV
keywords ChestX-rayclassificationTransferlearningFocallossGrad-CAMClassimbalanceMulti-labelNIHX-ray14Thoracicdiseases
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a standard transfer-learning recipe—ImageNet pre-trained convolutional networks fine-tuned on a small chest X-ray sample with focal loss—can improve automated detection of thoracic diseases, and that Grad-CAM heatmaps show the models look at clinically relevant lung regions. The authors evaluate AlexNet, ResNet152, and InceptionV3 on a 10% curated subset of the NIH ChestX-ray14 dataset and report that fine-tuning raised every model's AUC, with InceptionV3 rising from 0.500 to 0.6390. The abstract also claims a 15% increase in InceptionV3's F1-score, although Tables 1 and 2 show F1 falling from 0.287 to 0.1560. If the AUC gains are real, they would suggest that relatively cheap transfer learning can produce useful disease-ranking behavior even with severe class imbalance and limited data. The authors frame the contribution as feasibility evidence, not state-of-the-art performance.

What carries the argument

The mechanism is transfer learning on ImageNet-pretrained backbones—AlexNet, ResNet152, and InceptionV3—combined with focal loss and Grad-CAM visualization. Focal loss is a modified cross-entropy that down-weights well-classified examples, redirecting gradient toward the underrepresented disease classes that dominate chest X-ray datasets. Transfer learning supplies low-level feature detectors learned from ImageNet, which the authors argue helps with small medical datasets, while Grad-CAM generates class-specific heatmaps from the final convolutional layer to show which image regions drive each prediction. The claimed result depends on these three components working together: pretrained features, imbalance-correcting loss, and localization-based interpretability.

What would settle it

A reader could compute the class-frequency and patient-overlap statistics of the 10% subset against the full NIH ChestX-ray14 dataset, then rerun the same fine-tuning on the full training split and evaluate on the full test split; if the AUC gain shrinks toward baseline or the F1 decline persists, the mini-dataset results do not generalize. A simpler check: identify the exact patient identifiers in the mini train and test sets; any patient appearing in both would inflate every reported metric.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning ImageNet-pretrained architectures with focal loss yields significant performance gains for multi-label chest X-ray classification on the NIH ChestX-ray14 dataset. On the 10% mini-dataset, InceptionV3's AUC improved from 0.500 at baseline to 0.639 after fine-tuning, a 28% relative improvement, and AlexNet's AUC rose from 0.469 to 0.6397. The abstract additionally reports a 15% F1-score increase for InceptionV3, but the tables list F1 falling from 0.287 to 0.1560, so that particular claim is not supported by the reported numbers. The paper presents Grad-CAM visualizations for positive atelectasis and effusion as evidence that the models attend to diagnostically relevant areas, and explicitly concludes that the models remain short of state-of-the-art performance.

Load-bearing premise

The load-bearing premise is that the 10% mini-dataset faithfully represents the full NIH ChestX-ray14 dataset, preserving class frequencies, image-quality distribution, and the original 80/20 patient split, so metrics measured on this subset transfer to the full dataset.

Editorial extensions

If this is right

  • If the reported AUC gains are correct, transfer learning with focal loss can take small chest X-ray datasets from near-random ranking to moderately useful screening performance without training from scratch.
  • Grad-CAM alignments with atelectasis and effusion regions, if consistent across cases, would indicate the fine-tuned models are using anatomical evidence rather than dataset artifacts.
  • Because the models improved AUC more than F1, their practical value would lie in ranking and triage rather than in balanced precision/recall diagnosis.
  • The mini-dataset protocol could serve as a fast experimental testbed for the full NIH ChestX-ray14 dataset, since the same label mapping and train/test split are retained.
  • The authors' own conclusion implies the immediate contribution is feasibility evidence, not a clinically deployable system.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 28% AUC gain is a relative improvement from a baseline of 0.500, which is random guessing; starting from a stronger baseline would likely show smaller relative gains, so the headline number should be read in context.
  • The F1-Score decline between Table 1 and Table 2 is not consistent with the abstract's '15% increase' claim; treating that claim as unverified until a matching evaluation appears would be prudent.
  • Because the 10% subset's class frequencies and patient-level separation are not characterized, the reported metrics could shift materially on the full dataset; a direct full-dataset evaluation is the natural next test.
  • Grad-CAM images are suggestive evidence of clinical alignment, but without quantitative comparison to radiologist annotations or segmentation masks, the interpretability claim remains to be demonstrated.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper applies transfer learning with AlexNet, ResNet152, and InceptionV3 to a 10% subset of the NIH ChestX-ray14 dataset for multi-label thoracic disease classification. It reports baseline (Table 1) and post-fine-tuning (Table 2) values of BCE loss, F1-score, and AUC, and uses Grad-CAM to visualize model attention. The central claim is that fine-tuning with focal loss yields significant improvements, with InceptionV3 achieving a 28% AUC improvement and a 15% F1-score increase. The paper concludes that the approach is promising but not yet state-of-the-art.

Significance. If the reported improvements were reliable, the paper would provide a straightforward demonstration that standard transfer learning helps on a small chest X-ray sample, with interpretability as a secondary contribution. The use of a public dataset and three standard architectures is sensible, and the Grad-CAM figures illustrate an interpretability pipeline that could be useful for teaching or prototyping. However, the experimental evidence as presented is not trustworthy: the headline F1 claim is directly contradicted by the paper's own tables, the loss-function attribution is inconsistent with the training configuration, and there is no validation protocol, error analysis, or comparison to established benchmarks. Because the central quantitative claim fails on internal evidence and the remaining results are modest point estimates without statistical support, the paper does not currently meet the bar for a serious journal publication.

major comments (5)
  1. [Abstract and §4.2/§4.4, Tables 1 and 2] The abstract's claim that InceptionV3 achieved a 15% increase in F1-Score is contradicted by the paper's own tables. Table 1 reports InceptionV3 baseline F1 = 0.287, while Table 2 reports post-fine-tuning InceptionNet F1 = 0.156. This is a relative decrease of approximately 45.6%, not a 15% increase. The AUC claim (0.500 to 0.639, about 27.8% relative improvement) is consistent with the tables, but the F1 claim is a load-bearing part of the abstract and is false on the manuscript's own evidence. The paper must either correct this claim or provide the missing experimental record that supports it.
  2. [§3.2, §4.3, and §4.4.1] The paper attributes performance improvements to focal loss for class imbalance, but the training configuration in §4.3 lists BCEWithLogitsLoss as the loss function and gives no focal-loss hyperparameters (alpha, gamma). Table 2 reports a column labeled 'F Loss', yet the text never specifies how focal loss was implemented, what hyperparameters were used, or how it interacts with the BCE loss. The causal statement in §4.4.1 that improvements highlight 'the importance of Focal Loss' is therefore unsupported by the described experimental setup.
  3. [§4.3 and §4.4] No validation protocol is described. The paper states hyperparameters such as learning rate schedule, batch size, number of epochs, and fine-tuning layer cuts, but it does not state how these were selected or whether a held-out validation set was used. Without such a protocol, the reported test-set numbers may reflect hyperparameter selection on the test set, and the absence of error bars or confidence intervals means that the differences between models in Table 2 (e.g., F1 0.1401 vs 0.1596) cannot be assessed statistically. This is a load-bearing omission for any claim of 'significant performance improvements.'
  4. [§2.2 and §4.2] The 10% mini-dataset is described only as 'curated' while preserving the original 80/20 split, but the paper gives no evidence that class frequencies, patient-level separation, or image-quality distributions are preserved. NIH ChestX-ray14 contains multiple images per patient, and if the mini-dataset split does not respect patient boundaries, the reported AUC and F1 values can be optimistically biased by patient leakage. The representativeness assumption is load-bearing for transferring the conclusions to the full dataset, and it is not demonstrated.
  5. [§4.4 and §6] Even taken at face value, the post-fine-tuning results are weak: the best AUC in Table 2 is 0.6397 (AlexNet), barely above random for a multi-label chest X-ray task, and the F1 values decrease relative to baseline for every model. The conclusion acknowledges the models are not state-of-the-art, but the paper does not compare against any existing benchmark or per-class results, making it impossible to judge whether the reported values are meaningful for the 14 disease classes. The central narrative of 'significant performance improvements' is not supported by the reported metrics.
minor comments (6)
  1. [§2.1] The sentence 'Acquired from Kaggle's NIH ChestX-ray14 dataset' is repeated almost verbatim later in the same paragraph; one occurrence should be deleted.
  2. [§2.3] The description of data augmentation (random cropping and horizontal flipping) is repeated several times in the same subsection; the duplication should be removed.
  3. [§4.4.2] There is a typo in 'This indicatesthe models were nearing their optimal performance'; a space is missing.
  4. [Figure 4 caption] The caption contains 'middle figuree' and incorrectly labels the third image as both 'Final Convolution Layer' and 'middle figure'; the caption should be corrected and made consistent with the figure panels.
  5. [§4.4] The claim that ResNet showed a 20% reduction in BCE loss is not consistent with Table 2: ResNet152 BCE loss decreases from 0.928 to 0.7945, which is about 14.4%, not 20%. The sentence should either be corrected or the relevant numbers should be reported explicitly.
  6. [References] Reference [14] is incomplete (missing authors' full names and correct title) and reference [16] appears to include an implausible author list; the bibliography should be checked against the original sources.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the results are empirical measurements on external data with externally pretrained models, not a derivation that assumes its own conclusion.

full rationale

The paper makes no formal derivation chain that could reduce to its own inputs. It reports an empirical comparison of three ImageNet-pretrained architectures (AlexNet, ResNet152, InceptionV3) fine-tuned on a 10% subset of the NIH ChestX-ray14 dataset. The reported AUC, F1, and BCE values in Tables 1 and 2 are measured on a held-out split, and the pretrained weights come from the external ImageNet benchmark. No parameter is fitted to the target metrics and then renamed as a prediction, and no claimed result is defined in terms of the quantity it purports to predict. The main weaknesses are internal consistency issues: the abstract's claim of a 15% F1 increase for InceptionV3 is contradicted by Tables 1 and 2 (F1 drops from 0.287 to 0.156), and the text credits focal loss while Section 4.3 states BCEWithLogitsLoss was used. These are correctness or reporting problems, not circular reasoning. There is also no load-bearing self-citation: the references cited for the dataset, architectures, and interpretability methods are external prior work, and none of them is invoked to define or justify the paper's own quantitative conclusions. Because the central claims are measured outcomes rather than derived consequences of the assumptions, the circularity score is 0.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The central claim depends on hand-chosen design choices (subset size, epochs, batch size, augmentation, loss configuration) and on assumptions about the dataset split and label quality. No new entities are introduced.

free parameters (8)
  • Mini-dataset fraction = 10% of NIH ChestX-ray14
    Authors chose 10% to speed experimentation; all results depend on this reduced training set.
  • Number of fine-tuning epochs = 20
    Training was stopped at 20 epochs; no early stopping or validation-based selection is reported.
  • Batch sizes = 64 train, 32 test
    Chosen by authors without sensitivity analysis; affects optimization and gradient noise.
  • Learning rate schedule = reduced by factor 10 every 5 epochs
    Manual schedule; initial learning rate is not stated.
  • Initial learning rate
    Not reported; required to reproduce the optimizer behavior.
  • Focal loss hyperparameters alpha and gamma
    Not reported; focal loss is claimed but the training section specifies BCEWithLogitsLoss.
  • Fine-tuning layer cuts = later layers unfrozen
    The exact layers unfrozen for each architecture are not specified; affects transfer-learning results.
  • Augmentation parameters = random crop, horizontal flip
    Crop sizes and flip probabilities are not specified.
assumptions (5)
  • standard math PyTorch autograd and the Adam optimizer correctly implement backpropagation and update the networks.
    Assumed by Sections 2.3 and 4.3; not proved in the paper.
  • domain assumption ImageNet-pretrained features transfer to grayscale chest X-rays.
    The entire transfer-learning approach in Section 3.1 relies on this; no comparison to training from scratch is shown.
  • domain assumption The 10% mini-dataset preserves the class distribution and patient-level structure of the original NIH split.
    Section 2.2 says the subset was curated and the same 80/20 split was kept, but no evidence is given that class frequencies or patient separation are preserved.
  • domain assumption The labels in NIH ChestX-ray14 are sufficiently accurate for training and evaluation.
    Section 2.1 notes label noise but does not correct for it or measure its effect.
  • domain assumption Focal loss can be applied to multi-label classification in the form used here.
    Section 3.2 states focal loss is employed, but the training protocol in Section 4.3 uses BCEWithLogitsLoss; the actual loss used is unclear.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Learning-Powered Classification of Thoracic Diseases in Chest X-Rays." pith.science (2026). https://pith.science/paper/ZKO6ZGD7

@misc{pith2026250114279,
  author       = {Pith},
  title        = {Pith review of: Deep Learning-Powered Classification of Thoracic Diseases in Chest X-Rays},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZKO6ZGD7}},
  note         = {Machine review of arXiv:2501.14279}
}
read the original abstract

Chest X-rays play a pivotal role in diagnosing respiratory diseases such as pneumonia, tuberculosis, and COVID-19, which are prevalent and present unique diagnostic challenges due to overlapping visual features and variability in image quality. Severe class imbalance and the complexity of medical images hinder automated analysis. This study leverages deep learning techniques, including transfer learning on pre-trained models (AlexNet, ResNet, and InceptionNet), to enhance disease detection and classification. By fine-tuning these models and incorporating focal loss to address class imbalance, significant performance improvements were achieved. Grad-CAM visualizations further enhance model interpretability, providing insights into clinically relevant regions influencing predictions. The InceptionV3 model, for instance, achieved a 28% improvement in AUC and a 15% increase in F1-Score. These findings highlight the potential of deep learning to improve diagnostic workflows and support clinical decision-making.

Figures

Figures reproduced from arXiv: 2501.14279 by the authors.

Figure 1
Figure 1. Baseline performance metrics of the models. • F1-Score: A balanced metric that considers both precision and recall, providing a better under￾standing of how well the models classify each label. • AUC (Area Under the Curve): A metric that measures the model’s ability to rank positive exam￾ples higher than negative ones, indicating the model’s ability to distinguish between the classes. 4.2. Baseline Evaluation Result… view at source ↗
Figure 2
Figure 2. Evaluation results of models with pre-trained weights and improved generalization across diverse thoracic disease classes, enabling the models to focus more effectively on clinically relevant features. Finally, the trained model weights were saved for future experimentation or potential deployment in real-world applications. 4.4.1 Training Loss Analysis After fine-tuning, the models showed improved performance, as s… view at source ↗
Figure 3
Figure 3. illustrates the training loss trends for AlexNet, ResNet152, and InceptionNet over the 20 epochs [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Activations of the different convolution layers: Early Convolution Layer (left), Middle Convolution [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Grad-CAM for AlexNet - Positive Atelectasis approach. However, the models have not yet achieved state-of-the-art performance, underscoring the need for further optimization. Advancements in data preprocessing, the design of loss functions, and the adoption of more effe…
Figure 6
Figure 6. Figure 6: Grad-CAM for AlexNet - Positive Effusion like EfficientNet to reduce computational costs and experimenting with domain-specific pre-training to improve feature extraction[15]. Additionally, advanced data augmentation strategies, such as brightness and rotation adjustme…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 14 canonical work pages

  1. [1]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  2. [2]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Juno Ko, Yong Yu, Iulia Ciurea-Ilies, David Ding, Julia Kuo, Bhavik Patel, David Yeh, Andy Wang, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. arXiv preprint arXiv:1901.07031, 2019

    arXiv 1901

  3. [3]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems (NeurIPS), 25, 2012

    work page 2012

  4. [4]

    Girshick, Kaiming He, and Piotr Doll ´ar

    Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Doll ´ar. Focal loss for dense object detection. arXiv preprint arXiv:1708.02002, 2017

    arXiv 2017

  5. [5]

    van der Laak, Bram van Ginneken, and Clara I

    Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A.W.M. van der Laak, Bram van Ginneken, and Clara I. S ´anchez. A survey on deep learning in medical image analysis. Medical Image Analysis, 42:60–88, 2017. 14

    work page 2017

  6. [6]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems (NeurIPS), 30:4768–4777, 2017

    work page 2017

  7. [7]

    Large-scale chest x-ray datasets and techniques: A review

    Yingzi Peng, Wei Chu, Qing Wang, Peng Li, Linyan Wang, and Yongsheng Sun. Large-scale chest x-ray datasets and techniques: A review. IEEE Journal of Biomedical and Health Informatics , 25(9):3097–3112, 2021

    work page 2021

  8. [8]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI) , 9351:234–241, 2015

    work page 2015

Show all 18 references
  1. [9]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. Computer Vision and Pattern Recognition (CVPR), v4, 2017

    2017

  2. [10]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam++: Improved visual explanations for deep convolutional networks. IEEE Winter Conference on Applications of Computer Vision (WACV), 2019

    2019

  3. [11]

    Roth, Zhiyun Gao, Liyue Lu, Ziyue Xu, Isaac Nogues, Jianhua Yao, Daniel Mollura, and Ronald M

    Hoo-Chang Shin, Holger R. Roth, Zhiyun Gao, Liyue Lu, Ziyue Xu, Isaac Nogues, Jianhua Yao, Daniel Mollura, and Ronald M. Summers. Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics, and transfer learning. IEEE Transactio...

    2016

  4. [12]

    Khoshgoftaar

    Connor Shorten and Taghi M. Khoshgoftaar. A survey on image data augmentation for deep learning.Journal of Big Data, 6(1):1–48, 2019

    2019

  5. [13]

    Smoothgrad: removing noise by adding noise

    Daniel Smilkov, Namit Thorat, Been Kim, Fernanda Vi´egas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. In International Conference on Machine Learning (ICML), 2017

    2017

  6. [14]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Wojciech Z. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    2016

  7. [15]

    Mingxing Tan and Quoc V . Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (ICML), pages 6105–6114, 2019

    2019

  8. [16]

    Learning to diagnose from scratch by exploiting dependencies among labels

    Jianxin Yao, Yufei Zhang, Kaiming He, Xian Wang, Jiwen Sun, Xue Shi, Wei Li, Hanyun Zhang, and Wei Liu. Learning to diagnose from scratch by exploiting dependencies among labels. arXiv preprint arXiv:1710.10501, 2017. 15

    2017 arXiv

  9. [17]

    A unified framework for generalized zero-shot medical image classification

    Weiguang Zhou, Junzhou Shi, Yiyu Gao, and Zongxi Su. A unified framework for generalized zero-shot medical image classification. Medical Image Analysis, 2018

    2018

  10. [18]

    Cubuk, Golnaz Ghiasi, Tsung-Yi Lin, Jon Shlens, and Quoc V

    Barret Zoph, Ekin D. Cubuk, Golnaz Ghiasi, Tsung-Yi Lin, Jon Shlens, and Quoc V . Le. Learning data augmentation strategies for object detection. In European Conference on Computer Vision (ECCV), pages 566–583, 2020. 16

    2020

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.