Pith. sign in

REVIEW 5 major objections 5 minor 46 references

Lung Disease Detection with Vision Transformers: A Comparative Study of Machine Learning Methods

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A fine-tuned Vision Transformer on full chest X-ray images is claimed to reach 97.83% accuracy on three disease labels and 94.54% AUC on eight, beating CNN baselines and a lung-segmented variant.

desk verdict The abstract's headline accuracies (97.83%, 96.58%) never appear in the results section; the central claim is unsupported by the paper's own experiments, though the SAM2 segmentation comparison is the only mildly novel element. read the letter →

arxiv 2411.11376 v1 pith:5U7PENJN submitted 2024-11-18 eess.IV cs.CV

Baljinnyam Dayan This is my paper · ORCID
classification eess.IVcs.CV
keywords chestX-rayclassificationVisionTransformerlungdiseasedetectionself-attentionmedicalimagetransferlearningsegmentationcomparativestudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a Vision Transformer fine-tuned for chest X-ray classification can outperform CNN-based models, and that feeding it the full X-ray works better than feeding it lung-segmented regions. The reported evidence is that the full-image ViT reaches up to 97.83% accuracy on three disease labels and 94.54% AUC when the label set expands to eight, while the lung-segmented variant reaches 96.58% accuracy. The author argues that the self-attention mechanism captures global context across the whole image, so explicit lung segmentation is neither necessary nor beneficial. If true, this would simplify preprocessing pipelines for automated chest X-ray diagnosis while improving classification accuracy.

What carries the argument

The argument runs on the Vision Transformer's self-attention mechanism over image patches, defined as Attention(Q,K,V)=softmax(QK^T/$\sqrt$(d_k))V, which lets the model weight relationships between distant patches in a 224x224 chest X-ray. The paper contrasts this global-context capacity with the local receptive fields of CNNs, and uses the comparison between full-image and lung-segmented inputs to test whether focusing on the lung region concentrates or degrades the classification signal. The fine-tuned base model is a patch-16 ViT with 12 transformer layers and 597,511 trainable parameters.

What would settle it

Take the same images, group them by patient ID from the source dataset, and redo the train/test split so every patient's images are in only one set. If accuracy on the full-image ViT drops substantially below 97.83% or the full-image advantage over the segmented model disappears, the paper's central claim is unsupported. This can be checked without any new imaging data.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a standard Vision Transformer with 16x16 patches, fine-tuned on full chest X-ray images, classifies lung diseases more accurately than CNN-based systems and more accurately than the same ViT fine-tuned on lung-segmented images. On the three-label setting the full-image model reaches up to 97.83% accuracy and the segmented model reaches 96.58%; with eight labels the full-image model maintains a 94.54% ROC AUC. The author takes this as evidence that global self-attention can extract disease-relevant features from chest X-rays without explicit lung region segmentation.

Load-bearing premise

The entire comparison rests on the unstated assumption that no patient appears in both the training and test sets; if the 12,897 training images and 2,975 test images are split without grouping by patient, then the reported accuracy and AUC are inflated and the full-image versus segmented comparison is undermined.

Editorial extensions

If this is right

  • If the ViT result holds, explicit lung segmentation is unnecessary for accurate chest X-ray classification, simplifying the preprocessing pipeline.
  • A single fine-tuned ViT can replace CNN backbones while improving accuracy, precision, recall, F1, and AUC in the settings tested.
  • The eight-label result at 94.54% AUC indicates the method generalizes beyond the three-label setting without architectural changes.
  • The region-focused variant's lower scores suggest that localization masks do not add signal for a globally attentive model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the 97.83% and 96.58% accuracy figures are likely computed on different label sets, so they should not be read as a direct head-to-head; the paper should state the exact class configuration for each number.
  • My inference: re-running the same fine-tuning with a patient-exclusive train/test split would show whether the ViT advantage survives when no patient contributes images to both sets.
  • My inference: extending the full-image ViT to the complete 14-label set in the source data would test whether the eight-label AUC of 94.54% carries over to a harder multi-label task.
  • My inference: using the SAM2 masks directly may have injected segmentation boundary noise; a refined mask or attention-based region selection could change the comparison between the two approaches.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper claims to compare two Vision Transformer (ViT) approaches, one trained on full chest X-ray images and one on lung-segmented images, and states in the abstract and conclusion that the full-image ViT reaches 97.83% accuracy, the lung-segmented ViT reaches 96.58% accuracy, and that an AUC of 94.54% is achieved when label counts are increased to eight. The experimental section, however, reports only per-epoch loss, ROC AUC, and MCC for both models, with no accuracy, precision, recall, or F1 values in the tables. The single numerical accuracy stated in the text, 74.52%, is attributed to Table IV but appears to match Table III. The paper does not include any CNN baseline trained under the same protocol, and the dataset split is not described at patient level. The conclusion repeats the unsupported accuracy figures.

Significance. If the headline accuracy figures were supported by reproducible experiments, the claim that a standard ViT fine-tuned on full chest X-rays outperforms both CNN baselines and a lung-segmentation pipeline would be of moderate interest to the medical imaging community. The manuscript, however, does not substantiate these claims with its own reported results, does not provide error bars or statistical significance, and omits a CNN comparison. The central quantitative claims therefore currently rest on no evidentiary basis within the paper. The study also uses the public NIH ChestX-ray8 and COVID-19 datasets, which are widely studied, but the experimental reporting is too incomplete to allow replication or fair comparison.

major comments (5)
  1. [Abstract and Section VI] The abstract and conclusion claim that the full-image ViT achieves 97.83% accuracy and the lung-segmented ViT achieves 96.58% accuracy, but these numbers appear nowhere in Section V. Tables III and IV report only Loss, ROC AUC, and MCC per epoch; no accuracy column is present. The only accuracy mentioned in Section V-A is 74.52%, and that is attributed to Table IV while the surrounding AUC and MCC values (94.42%, 0.6908) are the final row of Table III, not Table IV. The central claim of the paper is therefore unsupported by the manuscript's own results.
  2. [Section V-A] The text in Section V-A conflates the two tables: it states 'As shown in Table IV' and then gives validation accuracy of 74.52%, AUROC of 94.42%, and MCC of 0.6908, but Table IV (full-image model) has 10 rows with final ROC AUC 94.41% and MCC 0.7041, while Table III (region-focused model) has 8 rows with final ROC AUC 94.42% and MCC 0.6908. Neither table contains an accuracy value. This internal inconsistency makes it impossible to determine which model achieved the stated 74.52% accuracy, and no definition is given for how validation accuracy was computed.
  3. [Section III-B] The model parameter count is internally inconsistent and does not match a ViT-base architecture. The text reports 597,511 total parameters, with a patch embedding layer of 590,592 parameters and a classification head of 5,383 parameters. These two components sum to 595,975, not 597,511, and neither sum includes the parameters of the transformer encoder, which for google/vit-base-patch16-224-in21k contains approximately 85 million parameters. The reported numbers thus omit the dominant part of the model, calling into question the accuracy of the architectural description.
  4. [Section III-A] The train/test split is not described at the patient level. The NIH ChestX-ray8 dataset contains multiple images per patient, and Section III-A only lists counts of training and test images (12,897 and 2,975). If the split is by image rather than by patient, the same patient's X-rays can appear in both training and test sets, which would inflate accuracy and AUC and invalidate the comparison between the two ViT approaches. The paper should state explicitly whether any patient-level deduplication was performed.
  5. [Sections II and IV-C] The abstract and Section IV-C claim that 'both methods surpass the performance of traditional CNN-based models,' but no CNN baseline is trained or evaluated in this study. The only CNN results cited are CheXNet and AutoThorax-Net from other papers using different datasets and evaluation protocols. Without a same-protocol CNN baseline, the claim of superiority over CNNs is not supported by the experiments.
minor comments (5)
  1. [Abstract] The abstract says 'classifcation of diseases on three label' but the dataset described in Section III-A contains seven categories, and the conclusion mentions 'eight' labels. The number of classes should be stated consistently.
  2. [Figure 5] Figure 5 is captioned 'Accuracy, Precision, Recall, and F1 Scores' but is never described in the text, and no numeric values for these metrics are given in any table. Either remove the figure or report the corresponding numbers and explain how they were obtained.
  3. [Section IV-C] The sentence 'Both approaches outperformed some of the traditional CNN-based models in this study' is unclear, as no CNN models are part of this study.
  4. [References] References [10] and [11] are the same ChestX-ray8 paper by Wang et al., listed with different page ranges and venue spellings; one should be removed or the citations should be merged.
  5. [General] There are several typographical errors, including 'classifcation' in the abstract and Section II, 'Comparision' in a table header (if present), and inconsistent capitalization of 'Vision Transformer' and 'AUROC'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical fine-tuning study with no derivation chain, no fitted-parameter-as-prediction, and no load-bearing self-citation; the headline accuracy figures are unsupported by the reported tables, which is a reporting/correctness issue rather than circularity.

full rationale

No circular step can be identified with the required specificity. The manuscript contains no mathematical derivation that reduces to its own inputs, no fitted parameter that is later relabeled as a prediction, and no load-bearing self-citation: the references are standard external sources (ViT, CheXNet, CheXpert, ChestX-ray8, SAM2) and none is authored by the present author. The abstract claims a full-image ViT accuracy of up to 97.83% and a lung-segmented ViT accuracy of 96.58%, but these numbers never appear in the results section; the only stated validation accuracy is 74.52% in Section V-A, and Table IV reports ROC AUC and MCC but not accuracy. The claimed AUC of 94.54% when 'label numbers are increased to eight' is actually the epoch-8 ROC AUC of the seven-class full-image model, so the eight-label claim is also unsupported by the reported experiments. These are evidence-quality, statistical-reporting, and internal-consistency problems, not circularity: the numbers are not obtained by defining a quantity in terms of itself, nor by fitting a parameter to a target and then predicting that same target. At most, the abstract selects the best epoch AUC from the validation curve, which is a form of validation-set selection rather than a constructional equivalence. Because no quoted passage exhibits a specific reduction of a claimed result to its own inputs, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on hand-chosen hyperparameters and unverified dataset assumptions. The most consequential assumption is the patient-level train/test split, which is not described and can strongly bias accuracy in multi-image-per-patient medical datasets. No new entities are introduced.

free parameters (5)
  • Learning rate = 1e-4
    Chosen by hand; no learning rate sweep is reported, and convergence depends on it.
  • Weight decay = 0.01
    AdamW weight decay is set without justification or sweep.
  • Number of training epochs = 10 (full-image), 8 (region-focused)
    The best epoch is selected for reporting AUC, so the effective stopping point is a hand-chosen value that maximizes validation performance.
  • Train/test split = 12,897 train / 2,975 test images
    The split is not described; since the NIH dataset has multiple images per patient, the exact split procedure directly affects the reported accuracy.
  • Choice of backbone = google/vit-base-patch16-224-in21k
    The paper claims 597,511 total parameters, but this model has about 86 million parameters, so the actual architecture used is unclear.
assumptions (4)
  • domain assumption The selected subset of NIH ChestX-ray8 and COVID-19 datasets has accurate labels and representative class distributions
    The paper treats all labels as ground truth; NIH labels are known to be noisy and the paper does not audit them.
  • domain assumption SAM2 zero-shot segmentation produces lung masks accurate enough for classification
    The region-focused approach depends on segmentation quality; the paper itself notes suboptimal segmentation could hinder performance (Section V-A).
  • domain assumption ImageNet pre-training transfers to chest X-ray classification
    The model is initialized with ImageNet-21k weights, but no evidence is provided that this transfer is appropriate for medical images.
  • domain assumption The train/test split separates patients rather than images
    The split is not described; if patient overlap exists, the reported metrics are inflated. This is load-bearing for the central comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lung Disease Detection with Vision Transformers: A Comparative Study of Machine Learning Methods." pith.science (2026). https://pith.science/paper/5U7PENJN

@misc{pith2026241111376,
  author       = {Pith},
  title        = {Pith review of: Lung Disease Detection with Vision Transformers: A Comparative Study of Machine Learning Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5U7PENJN}},
  note         = {Machine review of arXiv:2411.11376}
}
read the original abstract

Recent advancements in medical image analysis have predominantly relied on Convolutional Neural Networks (CNNs), achieving impressive performance in chest X-ray classification tasks, such as the 92% AUC reported by AutoThorax-Net and the 88% AUC achieved by ChexNet in classifcation tasks. However, in the medical field, even small improvements in accuracy can have significant clinical implications. This study explores the application of Vision Transformers (ViT), a state-of-the-art architecture in machine learning, to chest X-ray analysis, aiming to push the boundaries of diagnostic accuracy. I present a comparative analysis of two ViT-based approaches: one utilizing full chest X-ray images and another focusing on segmented lung regions. Experiments demonstrate that both methods surpass the performance of traditional CNN-based models, with the full-image ViT achieving up to 97.83% accuracy and the lung-segmented ViT reaching 96.58% accuracy in classifcation of diseases on three label and AUC of 94.54% when label numbers are increased to eight. Notably, the full-image approach showed superior performance across all metrics, including precision, recall, F1 score, and AUC-ROC. These findings suggest that Vision Transformers can effectively capture relevant features from chest X-rays without the need for explicit lung segmentation, potentially simplifying the preprocessing pipeline while maintaining high accuracy. This research contributes to the growing body of evidence supporting the efficacy of transformer-based architectures in medical image analysis and highlights their potential to enhance diagnostic precision in clinical settings.

Figures

Figures reproduced from arXiv: 2411.11376 by the authors.

Figure 2
Figure 2. Distribution of Images medical image analysis. It divides the input image into patches and processes them as sequences. Specifically, the model divides input images into patches of 16x16 pixels, which are then processed as a sequence. These patches are embedded into a lower-dimensional space, followed by position embeddings to retain spatial information, and subsequently passed through multiple layers of self-attent… view at source ↗
Figure 1
Figure 1. Sample pictures selected to ensure a balanced and representative sample of lung diseases. The training set includes a total of 12,897 images across seven categories, including 1,266 images for ”Normal” cases, 1,598 for ”Effusion,” 2,000 for ”Infiltration,” 1,471 for ”Nodule,” 3,418 for ”Pneumonia,” 1,684 for ”Atelectasis,” and 460 for ”COVID-19.” Similarly, the test set contains 2,975 images, with 317 images in the … view at source ↗
Figure 3
Figure 3. Segmented lungs 2) Accuracy, Precision, Recall, and F1 Score: Accuracy was used to provide a general sense of the model’s overall performance, capturing the proportion of correctly classified instances. Precision and recall were included to offer insights into the model’s handling of imbalanced data, where false positives or false negatives could have significant clinical consequences. The F1 score was calculated as… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Confusion matrix [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Accuracy, Precision, Recall, and F1 Scores [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 22 canonical work pages

  1. [1]

    Covid-19 image data collection

    Joseph Paul Cohen, Paul Morrison, and Lan Dao. Covid-19 image data collection. arXiv 2003.11597 , 2020

    arXiv 2003

  2. [2]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    2021

  3. [3]

    Mong, Safwan S

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, and Andrew Y. Ng. Chexpert: A large chest radiogra...

    work page 2019

  4. [4]

    Bejnordi, Arnaud A

    Geert Litjens, Thijs Kooi, Babak E. Bejnordi, Arnaud A. A. Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen A. W. M. van der Laak, Bram van Ginneken, and Clara I. Sánchez. A survey on deep learning in medical image analysis. Medical Image Analysis , 42:60--88, 2017

    work page 2017

  5. [5]

    Decoupled weight decay regularization, 2019

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, 2019

    2019

  6. [6]

    Lungren, and Andrew Y

    Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Ding, Aarti Bagul, Curtis Langlotz, Katie Shpanskaya, Matthew P. Lungren, and Andrew Y. Ng. Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning, 2017

    work page 2017

  7. [7]

    Sam 2: Segment anything in images and videos, 2024

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam 2: Segment anything in images and videos, 2024

    2024

  8. [8]

    Sze-To, A

    A. Sze-To, A. Riasatian, and H.R. Tizhoosh. Searching for pneumothorax in x-ray images using autoencoded deep features. Scientific Reports , 11:9817, 2021

    work page 2021

Show all 46 references
  1. [9]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023

    2023

  2. [10]

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision...

    2017

  3. [11]

    Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE Conference on Co...

    2017

  4. [12]

    Language models are few-shot learners,

    T. B. Brown et al., "Language models are few-shot learners," in Advances in Neural Information Processing Systems, vol. 33, pp. 1877-1901, 2020

    1901

  5. [13]

    BERT: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin et al., "BERT: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018

    2018 arXiv

  6. [14]

    Energy and policy considerations for deep learning in NLP,

    E. Strubell, A. Ganesh, and A. McCallum, "Energy and policy considerations for deep learning in NLP," arXiv preprint arXiv:1906.02243, 2019

    1906 arXiv

  7. [15]

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity,

    W. Fedus, B. Zoph, and N. Shazeer, "Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity," arXiv preprint arXiv:2101.03961, 2021

    2021 arXiv

  8. [16]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference,

    B. Jacob et al., "Quantization and training of neural networks for efficient integer-arithmetic-only inference," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2704-2713, 2018

    2018

  9. [17]

    Efficient processing of deep neural networks: A tutorial and survey,

    V. Sze et al., "Efficient processing of deep neural networks: A tutorial and survey," Proceedings of the IEEE, vol. 105, no. 12, pp. 2295-2329, 2017

    2017

  10. [18]

    Emergent abilities of large language models,

    J. Wei et al., "Emergent abilities of large language models," Transactions on Machine Learning Research, 2022

    2022

  11. [19]

    MobileBERT: a compact task-agnostic BERT for resource-limited devices,

    Y. Xu et al., "MobileBERT: a compact task-agnostic BERT for resource-limited devices," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 2158-2170, 2021

    2021

  12. [20]

    Internet of Things (IoT) connected devices installed base worldwide from 2015 to 2025,

    Statista Research Department, "Internet of Things (IoT) connected devices installed base worldwide from 2015 to 2025," Statista, 2023

    2015

  13. [21]

    Edge intelligence: Paving the last mile of artificial intelligence with edge computing,

    Z. Zhou et al., "Edge intelligence: Paving the last mile of artificial intelligence with edge computing," Proceedings of the IEEE, vol. 107, no. 8, pp. 1738-1762, 2019

    2019

  14. [22]

    Deep learning with edge computing: A review,

    J. Chen and X. Ran, "Deep learning with edge computing: A review," Proceedings of the IEEE, vol. 107, no. 8, pp. 1655-1674, 2019

    2019

  15. [23]

    Cloud-based or on-device: An empirical study of mobile deep inference,

    T. Guo, "Cloud-based or on-device: An empirical study of mobile deep inference," in 2018 IEEE International Conference on Cloud Engineering (IC2E), pp. 184-190, IEEE, 2018

    2018

  16. [24]

    Once for all: Train one network and specialize it for efficient deployment,

    H. Cai, C. Gan, and S. Han, "Once for all: Train one network and specialize it for efficient deployment," arXiv preprint arXiv:1908.09791, 2019

    1908 arXiv

  17. [25]

    SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size,

    F. N. Iandola et al., "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size," arXiv preprint arXiv:1602.07360, 2016

    2016 arXiv

  18. [26]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,

    S. Han, H. Mao, and W. J. Dally, "Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding," arXiv preprint arXiv:1510.00149, 2015

    2015 arXiv

  19. [27]

    BitNet: Scaling 1-bit Transformers for Large Language Models,

    K. Wang et al., "BitNet: Scaling 1-bit Transformers for Large Language Models," arXiv preprint arXiv:2310.11454, 2023

    2023 arXiv

  20. [28]

    Movement pruning: Adaptive sparsity by fine-tuning,

    V. Sanh et al., "Movement pruning: Adaptive sparsity by fine-tuning," arXiv preprint arXiv:2005.07683, 2020

    2005 arXiv

  21. [29]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard et al., "Mobilenets: Efficient convolutional neural networks for mobile vision applications," arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  22. [30]

    DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,

    V. Sanh et al., "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter," arXiv preprint arXiv:1910.01108, 2019

    1910 arXiv

  23. [31]

    Optimizing deep learning inference on edge devices,

    J. Wu et al., "Optimizing deep learning inference on edge devices," arXiv preprint arXiv:2009.13172, 2020

    2009 arXiv

  24. [32]

    EdgeBERT: An Efficient Transformer for On-Device Sequence-to-Sequence Generation,

    T. Tambe et al., "EdgeBERT: An Efficient Transformer for On-Device Sequence-to-Sequence Generation," arXiv preprint arXiv:2104.08036, 2021

    2021 arXiv

  25. [33]

    Quantizing deep convolutional networks for efficient inference: A whitepaper,

    R. Krishnamoorthi, "Quantizing deep convolutional networks for efficient inference: A whitepaper," arXiv preprint arXiv:1806.08342, 2018

    2018 arXiv

  26. [34]

    Block pruning for faster transformers,

    F. Lagunas et al., "Block pruning for faster transformers," arXiv preprint arXiv:2109.04838, 2021

    2021 arXiv

  27. [35]

    MnasNet: Platform-Aware Neural Architecture Search for Mobile,

    M. Tan et al., "MnasNet: Platform-Aware Neural Architecture Search for Mobile," arXiv preprint arXiv:1807.11626, 2019

    2019 arXiv

  28. [36]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick et al., "Overcoming catastrophic forgetting in neural networks," Proceedings of the National Academy of Sciences, vol. 114, no. 13, pp. 3521-3526, 2017

    2017

  29. [37]

    GLUE: A multi-task benchmark and analysis platform for natural language understanding,

    A. Wang et al., "GLUE: A multi-task benchmark and analysis platform for natural language understanding," arXiv preprint arXiv:1804.07461, 2018

    2018 arXiv

  30. [38]

    Neural architecture search: A survey,

    T. Elsken, J. H. Metzen, and F. Hutter, "Neural architecture search: A survey," The Journal of Machine Learning Research, vol. 20, no. 1, pp. 1997-2017, 2019

    1997

  31. [39]

    Federated learning: Challenges, methods, and future directions,

    T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, "Federated learning: Challenges, methods, and future directions," IEEE Signal Processing Magazine, vol. 37, no

  32. [40]

    MobileBERT: a compact task-agnostic BERT for resource-limited devices,

    Y. Xu et al., "MobileBERT: a compact task-agnostic BERT for resource-limited devices," in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 2158-2170

    2020

  33. [41]

    Apple's Neural Engine: 16-core architecture, delivering high performance for machine learning tasks,

    Apple Inc., "Apple's Neural Engine: 16-core architecture, delivering high performance for machine learning tasks," 2023

    2023

  34. [42]

    A survey on the optimization of neural network accelerators for edge computing,

    C. Zhang et al., "A survey on the optimization of neural network accelerators for edge computing," Proceedings of the IEEE, vol. 108, no. 12, pp. 2046-2063, 2020

    2020

  35. [43]

    Federated learning with non-iid data,

    Y. Zhao et al., "Federated learning with non-iid data," arXiv preprint arXiv:1806.00582, 2018

    2018 arXiv

  36. [44]

    Privacy and robustness in federated learning: Attacks and defenses,

    L. Lyu et al., "Privacy and robustness in federated learning: Attacks and defenses," arXiv preprint arXiv:2011.02255, 2020

    2011 arXiv

  37. [45]

    Scaling language models: Methods, analysis & insights from training Gopher,

    J. W. Rae et al., "Scaling language models: Methods, analysis & insights from training Gopher," arXiv preprint arXiv:2112.11446, 2021

    2021 arXiv

  38. [46]

    Ԅ; ˏ 9"j8 6 tzyy l6@ kp' 7l6 d2X Q q Gr P m6o&EQo>qD0 dY ٗ_ ڵk (Y ` ^ݻw ޽ xSSS:j|Ǐ|YUnKh0 `` 歷2 ap8 4M˲o 9r xUUj H / Q eCP4 T < _ \ Ȋ p8l0 X t:L&F oQw 7 h a&6c(ʲl27 D

    11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEco...

    arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.