Pith. sign in

REVIEW 4 major objections 5 minor 11 references

Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A cross-attention vision transformer reaches 88.25% accuracy and 99.42% recall for pneumonia detection on a 600-image pediatric chest X-ray test set, outperforming CNNs and other ViTs in this study.

desk verdict Honest benchmark of ViTs vs CNNs on a public pneumonia dataset, but the pooled train/test split likely creates patient-level leakage that undermines the CrossViT superiority claim. read the letter →

arxiv 2507.10589 v1 pith:RVUOHOML submitted 2025-07-11 eess.IV cs.AIcs.CVcs.NE

Gaurav Singh This is my paper
classification eess.IVcs.AIcs.CVcs.NE
keywords pneumoniadetectionchestX-rayVisionTransformerCrossViTDenseNet-121classimbalancemedicalimageclassificationdeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper compares unsupervised and supervised machine learning, CNNs, and vision transformers for detecting pneumonia in 5,856 pediatric chest X-rays. It claims that a cross-attention vision transformer (CrossViT) achieves the best balance of metrics, with 88.25% accuracy and 99.42% recall on a 600-image test set, outperforming DenseNet-121 and all other models evaluated. The paper argues that architecture choice matters more than raw parameter count, and that high recall is the clinically important target because false negatives are dangerous. A sympathetic reading would say the paper is trying to establish that vision transformers, even trained from scratch on a small imbalanced dataset, are a promising direction for automated pneumonia screening.

What carries the argument

The load-bearing component is CrossViT's cross-attention mechanism: two transformer branches process the same image at different scales (upscaled 384x384 and downscaled 192x192), and every few layers the branches exchange information through cross-attention, letting fine and coarse features reinforce each other. This dual-scale exchange is what the paper credits for outperforming single-scale ViTs and CNNs on a small dataset.

What would settle it

Compare the filenames or patient IDs in the 600 training images and 600 test images; if any patient appears in both sets, the central comparison is compromised. Alternatively, rerun the same experiment enforcing patient-level separation and see whether CrossViT still leads.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that CrossViT—a two-branch transformer processing the image at 384x384 and 192x192 resolutions and letting the branches cross-attend—reaches 88.25% test accuracy and 99.42% recall, beating DenseNet-121 (83.05%), Compact Convolutional Transformer (84.08%), Deep-ViT (80.60%), and Modified LeNet (77.40%). The paper also finds that all deep learning models reach near-perfect recall (99.23–100%) with precision between 73.44% and 84.02%, and that CrossViT's 75M parameters outperform Deep-ViT's roughly 127M parameters, supporting the claim that architecture, not size, drives performance. These results are presented as evidence that ViTs can work without large-scale pretraining on medical imaging, though the paper notes class imbalance limits precision and suggests pretraining as a next step.

Load-bearing premise

The 600-image test set was split by image rather than by patient, so chest X-rays from the same child may appear in both training and testing; if so, the reported accuracy and model ranking are inflated.

Editorial extensions

If this is right

  • If CrossViT's result holds, vision transformers can be competitive on medical image classification even without ImageNet pretraining, which widens their use to domains where pretrained weights are unavailable.
  • At 99.42% recall, an automated screening tool built on this model would miss roughly 1 in 170 actual pneumonia cases, a clinically favorable trade-off if precision can be improved.
  • The parameter-count finding suggests that model selection for medical imaging should focus on architecture design and inductive bias rather than simply scaling up parameters.
  • The near-perfect recall across all deep models indicates that the main remaining bottleneck is false positives, so future work should target precision, class weighting, or better threshold calibration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I read the paper's split as image-level stratified sampling; if patient identity had been enforced between training and test sets, the ranking between models could change, so external validation on a patient-disjoint dataset would settle the clinical relevance.
  • A direct extension of the cross-attention idea would be to apply the same dual-scale architecture to other small medical imaging datasets where pretrained ViTs are scarce and data are imbalanced.
  • The reported 99.42% recall at 84.02% precision suggests that operating-point selection, not architecture alone, determines clinical utility; adjusting the decision threshold or using loss functions that penalize false negatives could change the trade-off substantially.
  • Because the test set is only 600 images, the five-percentage-point gap between CrossViT and DenseNet-121 carries real uncertainty; a larger or multi-center test set would be needed to confirm the ordering.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a comparative evaluation of traditional machine learning (PCA-based clustering, logistic regression, SVC) and deep learning models (Modified LeNet, DenseNet-121, Deep-ViT, CCT, Cross-ViT) for pneumonia detection from chest X-rays. Using a re-split 600-image training set and 600-image test set from the Kermany/Mooney pediatric CXR dataset, it reports CrossViT as the best performer with 88.25% accuracy and 99.42% recall, and concludes that architectural choice matters more than model size. The manuscript provides clear result tables, runtime comparisons, and a link to implementation code, but the evaluation rests on a single split with no confidence intervals or patient-level separation details.

Significance. If the reported results hold, the paper provides a useful benchmark comparison of ViT variants and CNNs on a small medical image dataset, including the noteworthy observation that a 75M-parameter CrossViT can outperform a larger Deep-ViT and a pretrained DenseNet-121 when trained from scratch. The availability of code and the explicit reporting of training and test times are strengths. However, the central superiority claim is not yet established because the data split may violate patient independence, metrics are single-run point estimates with no uncertainty quantification, and class imbalance is not formally accounted for in the evaluation. The contribution is primarily empirical rather than methodological.

major comments (4)
  1. [2 Data] The description of the stratified re-split is incomplete: the text states only that the data were "redistributed using Stratified sampling" to obtain 600 images for train and test, without clarifying whether the original train/test/validation folders were pooled, whether patient-level separation was enforced, or whether duplicate images for the same patient were removed. In the Kaggle 'Chest X-Ray Images (Pneumonia)' dataset, patient IDs are not provided and multiple images per patient are a known issue; if the same patient appears in both training and test sets, the reported 88.25% accuracy and the ranking over CCT and DenseNet-121 are inflated. The authors must describe the exact splitting procedure, run a duplicate-image hash check, and either enforce patient-level grouping or report results on a split that guarantees patient independence.
  2. [8 Results and discussion, Tables 1 and 2] All metrics in Tables 1 and 2 are point estimates from a single train/test split, with no confidence intervals, error bars, or significance tests. The differences among CrossViT (88.25%), CCT (84.08%), and DenseNet-121 (83.05%) are small relative to the expected variance on a 600-image test set and could arise from chance. The authors should run multiple seeds and report mean and standard deviation, and apply paired statistical tests (e.g., McNemar's test on the paired predictions) to support the claim that CrossViT is superior.
  3. [8 Results and discussion] The class imbalance (approximately 75% pneumonia-positive images, as shown in Fig. 2) makes accuracy and recall misleading; a trivial classifier that always predicts pneumonia would achieve roughly 75% accuracy and 100% recall. Several models in Table 2 report recall of 1.0 or near 1.0 while precision ranges from 73% to 84%, indicating that the decision threshold favors the positive class. The authors should report specificity, F1-score, Matthews correlation coefficient, and confusion matrices for all models, and discuss the threshold/class-weighting choices so that the claimed superiority of CrossViT is not an artifact of imbalanced evaluation.
  4. [7 Hyperparameters, Initialisation and Training strategy] The model selection protocol for the deep learning methods is not described. The text says hyperparameters for the ML approaches (logistic regression, SVC) were found by grid search, but it does not state whether a held-out validation set was used to select learning rates, patch sizes, augmentation parameters, or the number of epochs for the ViT and CNN models. If the 600-image test set was used for any of these choices, the reported test metrics are optimistically biased. The authors should clarify the validation strategy and, if none was used, state explicitly that all deep-learning configurations were fixed before any test-set evaluation.
minor comments (5)
  1. [Tables 1 and 2] The tables contain typographical errors: "Precsion" should be "Precision" in Table 1, and "DneseNet-121" should be "DenseNet-121" in Table 2.
  2. [8 Results and discussion] The sentence "ViTs did well as compared to CNN models" overgeneralizes: Table 2 shows Deep-ViT accuracy (80.60%) is lower than DenseNet-121 accuracy (83.05%), so the conclusion applies to CrossViT and possibly CCT, but not to all ViT variants.
  3. [2 Data and 7 Hyperparameters] Section 2 says images are rescaled to 256×256, while Section 7 says deep learning images are rescaled to 224×224; the manuscript should clarify which resolution was used for which model category to avoid ambiguity.
  4. [References] Several references are incomplete: [2] and [3] lack full author or organization details, [5] and [6] would benefit from access dates or version identifiers, and the code repository in [1] should be versioned or contain a commit hash to ensure reproducibility.
  5. [Section 5.1 and Section 8] Minor text issues include "architectue" for "architecture", "Intern Covariate Shift" for "Internal Covariate Shift", and "decomposiion" for "decomposition"; these should be corrected in revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the reported results are direct empirical measurements from training and testing on an evaluation split, with no derivation that reduces to its own inputs.

full rationale

The paper makes no pretense of deriving its accuracy, precision, or recall numbers from first principles; each reported metric is obtained by training models on a 600-image stratified split and evaluating them on the corresponding test images. The architecture choices (Deep-ViT, CCT, Cross-ViT) are cited from external prior work, and the dataset is attributed to Kermany et al. via Kaggle. The only self-reference is the author's own GitHub code link [1], which is a pointer to implementation code and is not a load-bearing mathematical premise. Hyperparameters for the ML baselines are selected by grid search, and the DL models are trained with stated schedulers and data augmentation; the final numbers are measured on a test split, not fitted to the test labels. No equation is defined in terms of a target result, and no cited uniqueness theorem or prior author result is used to force a conclusion. Therefore no step reduces to its own input by construction. Concerns about whether the 600/600 stratified split preserves patient independence, or whether grid-search hyperparameters risk tuning to the test set, are validity and generalization risks rather than circularity; they do not make the reported comparison equivalent to its assumptions.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central claim is an empirical benchmark; it introduces no new entities, mechanisms, or conserved quantities. The main load-bearing assumptions are about data labeling, split integrity, metric stability, and the absence of test-set leakage during tuning.

free parameters (7)
  • Logistic regression regularization coefficient = 0.001
    Chosen by grid search in Section 7; directly affects the logistic regression baseline accuracy.
  • SVC soft-margin parameter C = 1
    Chosen by grid search in Section 7; balances margin width and misclassification penalty.
  • Number of principal components retained = 1,648 (98% variance)
    Set in Section 3 for faster computation; changes input dimensionality for PCA clustering, logistic regression, and SVC.
  • DeepViT patch size = 32
    Set in Section 6.1; determines the number of tokens and affects model accuracy.
  • CrossViT image scales = 384x384 and 192x192
    Set in Section 6.3; the two-scale design is central to the cross-attention mechanism.
  • Learning rate for deep learning models = 1e-4
    Used for all ViT and DenseNet training in Section 7; strongly affects convergence.
  • Data augmentation shift, rotation, and probability = shift 0.1, rotation 5 degrees, probability 0.5
    Chosen in Section 7 to regularize training on the small dataset.
assumptions (5)
  • domain assumption The Kermany et al. labels are correct ground truth for pneumonia.
    Section 2 takes the dataset from Kermany et al. without radiologist re-adjudication.
  • domain assumption The stratified 600/600 split keeps patients independent between train and test.
    Section 2 does not describe patient-level separation, yet all reported metrics depend on this split being valid.
  • domain assumption Single-run metrics are stable enough to rank models.
    Tables 1 and 2 report one run per model with no standard deviation or significance test.
  • domain assumption Grid search did not use the test set.
    Section 7 says best hyperparameters were found by grid search but does not state which split was used.
  • domain assumption The test set's approximately 75% positive class ratio matches the intended deployment setting.
    All accuracy and recall values are computed under this imbalance, which affects interpretation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays." pith.science (2026). https://pith.science/paper/RVUOHOML

@misc{pith2026250710589,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Vision Transformers and Traditional Deep Learning Approaches for Automated Pneumonia Detection in Chest X-Rays},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RVUOHOML}},
  note         = {Machine review of arXiv:2507.10589}
}
read the original abstract

Pneumonia, particularly when induced by diseases like COVID-19, remains a critical global health challenge requiring rapid and accurate diagnosis. This study presents a comprehensive comparison of traditional machine learning and state-of-the-art deep learning approaches for automated pneumonia detection using chest X-rays (CXRs). We evaluate multiple methodologies, ranging from conventional machine learning techniques (PCA-based clustering, Logistic Regression, and Support Vector Classification) to advanced deep learning architectures including Convolutional Neural Networks (Modified LeNet, DenseNet-121) and various Vision Transformer (ViT) implementations (Deep-ViT, Compact Convolutional Transformer, and Cross-ViT). Using a dataset of 5,856 pediatric CXR images, we demonstrate that Vision Transformers, particularly the Cross-ViT architecture, achieve superior performance with 88.25% accuracy and 99.42% recall, surpassing traditional CNN approaches. Our analysis reveals that architectural choices impact performance more significantly than model size, with Cross-ViT's 75M parameters outperforming larger models. The study also addresses practical considerations including computational efficiency, training requirements, and the critical balance between precision and recall in medical diagnostics. Our findings suggest that Vision Transformers offer a promising direction for automated pneumonia detection, potentially enabling more rapid and accurate diagnosis during health crises.

Figures

Figures reproduced from arXiv: 2507.10589 by the authors.

Figure 1
Figure 1. Normal vs Pneumonia infected CXR 2 Data The data used for this project is based on data originally provided by Kermany et al [4] which is available as open source on Kaggle [5]. There are a total of 5,856 CXR(anterior-posterior) images from pediatric patients of one to five years old. The raw data is organized into three - train, test and validation - folders. Each folder is further having images into two sub-folder… view at source ↗
Figure 2
Figure 2. Class Distribution 3 Unsupervised Machine Learning For the start, I investigated the two classed and is their any significant difference in the data distribution that can be segregated into two groups. For this a relevant work is already done for Brain tumor segmentation using Principal components based clustering [6]. I took inspiration from this paper and used SVD decomposition instead of EM based PCA to find prin… view at source ↗
Figure 3
Figure 3. t-SNE plot of train images 4.2 Support Vector Classifier Logistic regression decision boundary is linear and it’s shown in [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: (a) LeNet-5 (b) DenseNet-121 6 Vision Transformers (ViT) CNNs are gradually being replaced by the State-of-the-art Vision Transformers proposed by Dosovit￾skiy et al [8] because of their overall computation efficiency, easy GPU parallelisation, ViTs don’t require large…
Figure 5
Figure 5. Figure 5: (a) Deep ViT (b) CCT (c) Cross ViT 7 Hyperparameters, Initialisation and Training strategy Unsupervised methods and supervised machine learning approaches are trained on a 16 core CPU machine whereas the Deep learning approaches i.e. CNNs and ViTs are trained and teste…
Figure 6
Figure 6. Figure 6: (a) PCA+Clustering (b) Logistic Regression (c) SVC [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: (a) Modified LeNet-5 (b) DenseNet-121 (c) No. of trainable parameters [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: (a) Deep-ViT (b) CCT (c) Cross-ViT References [1] Gaurav Singh. Pneumonia Detection. https : / / github . com / gauravSingh30 / DL - PneumoniaDetection. 28-Nov-2022. [2] World Health Organisation. Pneumonia in children. https://www.who.int/news-room/ fact-sheets/detail…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

11 extracted references · 7 canonical work pages

  1. [1]

    Pneumonia Detection

    Gaurav Singh. Pneumonia Detection . https : / / github . com / gauravSingh30 / DL - PneumoniaDetection. 28-Nov-2022

    work page 2022

  2. [2]

    Pneumonia in children

    World Health Organisation. Pneumonia in children. https://www.who.int/news-room/ fact-sheets/detail/pneumonia. 11-Nov-2022

    work page 2022

  3. [3]

    Number of deaths involving coronavirus disease

    Statista. Number of deaths involving coronavirus disease. https://www.statista.com/ statistics / 1113051 / number - reported - deaths - from - covid - pneumonia - and - flu-us/. 25-Nov-2022

    work page 2022

  4. [4]

    Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification)

    Kermany et al. Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for Classification). https://data.mendeley.com/datasets/rscbjbr9sj/2. 2018

    work page 2018

  5. [5]

    Chest X-Ray Images (Pneumonia)

    Paul Mooney. Chest X-Ray Images (Pneumonia). https://www.kaggle.com/datasets/ paultimothymooney/chest-xray-pneumonia . 2017

    work page 2017

  6. [6]

    PCA based clustering for brain tumor segmentation of T1w MRI images

    Irem Ersöz Kaya et al. “PCA based clustering for brain tumor segmentation of T1w MRI images”. In: Computer methods and programs in biomedicine 140 (2017), pp. 19–28

    work page 2017

  7. [7]

    Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning

    Pranav Rajpurkar et al. “Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning”. In: arXiv preprint arXiv:1711.05225 (2017). 6

    arXiv 2017

  8. [8]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy et al. “An image is worth 16x16 words: Transformers for image recognition at scale”. In: arXiv preprint arXiv:2010.11929 (2020)

    arXiv 2020

Show all 11 references
  1. [9]

    Deepvit: Towards deeper vision transformer

    Daquan Zhou et al. “Deepvit: Towards deeper vision transformer”. In: arXiv preprint arXiv:2103.11886 (2021)

    2021 arXiv

  2. [10]

    Escaping the big data paradigm with compact transformers

    Ali Hassani et al. “Escaping the big data paradigm with compact transformers”. In: arXiv preprint arXiv:2104.05704 (2021)

    2021 arXiv

  3. [11]

    Crossvit: Cross-attention multi- scale vision transformer for image classification

    Chun-Fu Richard Chen, Quanfu Fan, and Rameswar Panda. “Crossvit: Cross-attention multi- scale vision transformer for image classification”. In: Proceedings of the IEEE/CVF interna- tional conference on computer vision. 2021, pp. 357–366. 7

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.