CNNtention: Can CNNs do better with Attention?

Julian Glattki; Nikhil Kapila; Tejas Rathi

REVIEW 4 major objections 6 minor 31 references

CNNtention: Can CNNs do better with Attention?

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that inserting self-attention, multi-head attention, or CBAM blocks between the feature-extractor groups of a ResNet-20 improves image-classification accuracy on CIFAR-10 and MNIST, with self-attention giving the largest…

desk verdict A well-written course project whose central claim that attention helps CNNs is not supported by the evidence because the comparisons are single-run, per-model tuned, and the gains are within noise. read the letter →

arxiv 2412.11657 v3 pith:PITCLEQ5 submitted 2024-12-16 cs.CV cs.LG

Nikhil Kapila , Julian Glattki , Tejas Rathi This is my paper · ORCID

classification cs.CVcs.LG

keywords attentionmechanismsResNet-20CIFAR-10MNISTself-attentionmulti-headCBAMimageclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a convolutional network learns better when attention modules are added between, rather than inside, its feature-extraction stages. The authors build a ResNet-20 baseline for CIFAR-10 and MNIST, insert three attention variants—self-attention, eight-head multi-head attention, and CBAM—after each of three feature-extractor groups, and measure test accuracy, convergence speed, and where the models look. On CIFAR-10, self-attention reaches 91.44% versus the baseline's 90.96%; multi-head attention reaches 91.06% and CBAM 91.32%, with CBAM converging fastest. The paper concludes that CNNs indeed learn better when attention is added, while noting each mechanism carries a trade-off in computation and in how it models global context.

What carries the argument

The mechanism is sparse interleaving of attention between feature extractors: the ResNet-20 is split into three sequential groups of residual blocks, and after each group one attention block (SelfAtt, MHA, or CBAM) is inserted with a residual connection $R(x)=F(x)+x$, plus an optional learned weighting $W(x)=w\cdot F(x)+x$. SelfAtt projects feature maps with 1x1 convolutions into queries, keys, and values and applies $\mathrm{softmax}(QK^\top)V$ without the $\sqrt{d_k}$ scaling, because channel counts are small; MHA is the same idea with eight heads and the standard scaling; CBAM applies channel attention via a shared MLP on pooled descriptors followed by spatial attention via a 7x7 convolution. The attention blocks are placed only three times in the network, not after every convolution, which the authors argue keeps compute overhead low while letting the feature extractors build meaningful, non-noisy features first. Residual connections stabilize training, and the learned weight lets the network dynamically favor feature-extractor output early in training.

What would settle it

Train the baseline, SelfAtt, MHA, and CBAM versions of ResNet-20 on CIFAR-10 with identical hyperparameters, several random seeds, and multiple runs per seed; if the 0.48 percentage-point gap between self-attention and the baseline falls within run-to-run variability, the central conclusion that attention improves CNNs is not supported.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is an empirical one: a ResNet-20 whose three feature-extractor groups are separated by attention blocks consistently matches or beats the same network without attention. Self-attention and multi-head attention outperform the baseline on both datasets, and self-attention improves CIFAR-10 accuracy from 90.96% to 91.44% while also training faster (2.9h vs 3.7h). On MNIST, self-attention reaches 99.30% vs 98.52%. CBAM trails the other attention variants in final accuracy but converges fastest and is the cheapest to train. The authors also show that attention blocks need residual connections to train stably, and that a learned scalar weight on the attention branch speeds convergence; GradCAM visualizations are used to argue that self-attention and multi-head attention capture longer-range dependencies than the baseline, with CBAM mainly sharpening the baseline's existing focus.

Load-bearing premise

The comparison assumes that the small accuracy differences between models come from the attention mechanism itself, not from the fact that each model was tuned with different optimizers, learning rates, and regularization and trained only once without error bars.

Editorial extensions

If this is right

If attention is added between feature-extractor groups, ResNet-20 accuracy rises on CIFAR-10 (SelfAtt 91.44% vs 90.96% baseline) and on MNIST (99.30% vs 98.52%).
Self-attention and eight-head multi-head attention both beat CBAM in final accuracy, while CBAM converges fastest, so the choice of attention mechanism is a compute-vs-generalization trade-off.
Attention blocks must be wrapped in residual connections; without them training is unstable, so the residual path is part of the mechanism, not an optional extra.
A learned scalar weight on the attention branch improves validation accuracy (90.01% vs 88.00% for the unweighted variant) and speeds convergence.
Sparse placement of attention (three blocks total, not after every convolution) is sufficient to observe gains, which matters for deployment on resource-constrained devices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Left implicit: the sparse-placement recipe suggests attention can be a retrofit rather than a redesign—a practitioner could insert self-attention at a few intermediate depths of an existing CNN and get a small accuracy gain plus faster convergence, a much cheaper experiment than training a vision transformer.
Left implicit: the finding that eight MHA heads do not beat a single SelfAtt head on CIFAR-10 could be stress-tested across more heads and larger datasets; on ImageNet-scale tasks, the omitted $\sqrt{d_k}$ scaling may matter.
Left implicit: the weighted-residual design ($w\cdot F(x)+x$) is only sketched in the paper; ablating the learned weight over time and testing it on larger architectures would show whether dynamic feature selection generalizes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a cleanly written course project that reimplements ResNet-20 and slots three attention modules between feature-extractor groups. The writing is clear and the authors are transparent about scope. I want to give credit where it is due: the baseline reimplementation checks out against the original paper (9.04% vs 8.75% test error), which is good practice; the CBAM reduction-ratio ablation is a small but honest exploration; and the observation that attention blocks without residual connections train unstably, with a weighted residual helping, is a useful practical detail. Code and MLFlow logs are provided.

The soft spot is the experimental protocol, and it is load-bearing. Section 5.1 explicitly says different experiments were ran for each model, that learning rates were tuned per model, and that regularization was varied, with EarlyStopping. Each configuration appears to have been run once. The largest reported CIFAR-10 gain is 0.48 percentage points (SelfAtt 91.44% vs baseline 90.96%); the MNIST gain is 0.78 points. Typical run-to-run variance for ResNet-20 on CIFAR-10 is on that order, so these differences are indistinguishable from training noise. Per-model tuning makes it worse: the comparison is not a controlled ablation, and hyperparameter choices could explain the differences just as plausibly as the attention modules.

There is also an internal inconsistency: Section 4.2 says MNIST was included only for GradCAM evaluation, yet Section 5.2.1 uses MNIST accuracy as quantitative evidence that attention is effective. That undermines the second-dataset confirmation.

The conclusion in Section 5.3 overstates. 'CNNs indeed learn better when attention is added' is too strong for single runs with sub-percentage-point gains. A measured version would say the attention-augmented models achieved comparable or slightly higher accuracy in these specific configurations, with variance estimates still needed.

The novelty is minor - placing attention after feature-extractor groups and omitting positional encodings are design choices, not new mechanisms. That is fine for a course project, but it means the paper does not open a new direction. The literature review is adequate and the related-work citations are appropriate.

Overall, this is a solid course-project report. It would be a good class submission or workshop preprint. As a research paper, it needs repeated runs with error bars, a shared training protocol, and a more measured conclusion. There is no misconduct and no fatal flaw in the reasoning, just an under-powered empirical design. For peer review at a serious venue, I would not send it out; the central comparison is not reliable enough to justify referee time. But it could serve a reading group well as an example of common experimental pitfalls in deep learning papers.

Referee Report

4 major / 6 minor

Summary. The paper reimplements ResNet-20 for CIFAR-10 and inserts three attention mechanisms (self-attention, multi-head self-attention, and CBAM) between the three residual feature-extractor stages, comparing accuracy, training time, and GradCAM maps on CIFAR-10 and MNIST. It reports small accuracy gains for SelfAtt and MHA over the baseline (e.g., 91.44% vs 90.96% on CIFAR-10) and concludes that CNNs learn better when attention is added.

Significance. If the result were statistically established, it would provide a modest empirical data point about placing attention between residual stages rather than inside blocks, together with a useful computational trade-off discussion and a verified baseline implementation (the reimplementation achieves a 9.04% test error versus 8.75% reported by He et al.). The paper's strengths include a public GitHub repository, MLFlow experiment tracking, and an explicit attempt to stabilize attention blocks with residual and weighted-residual connections. However, the central comparison is not statistically supported: every model configuration appears to be run once, the reported accuracy gaps are within typical run-to-run variation for ResNet-20 on CIFAR-10, and the authors themselves state that hyperparameters were tuned per model and that MNIST was included only for GradCAM evaluation. These issues undermine the causal claim that attention improves CNNs.

major comments (4)

[Section 5.3, Table 3] The conclusion that 'CNNs indeed learn better when attention is added' rests on single-run accuracy differences of 0.48 percentage points on CIFAR-10 and 0.78 percentage points on MNIST. No error bars, seeds, or significance tests are reported, and for ResNet-20 on CIFAR-10 run-to-run variation is typically on the order of 0.2-0.5 percentage points. The reported gains are therefore indistinguishable from training noise, and the claim is not supported as stated. Please provide multiple seeds with means and standard deviations, or paired significance tests, before making a quantitative conclusion.
[Section 5.1] The paper states that 'different experiments were ran for each model,' that learning rates were tuned per model, and that regularization was varied with EarlyStopping. This means Table 3 is not a controlled ablation in which only the attention module differs; any observed accuracy change could be caused by hyperparameter choices rather than by the attention mechanism. Please fix a shared training protocol (or report and justify each configuration's hyperparameters) and ensure that the only deliberate difference is the attention insertion.
[Sections 4.2 and 5.2.1] Section 4.2 says MNIST is used 'only to ease our GradCAM evaluations,' and Section 5 says CIFAR-10 hyperparameters were extended to MNIST without independent tuning. Yet Section 5.2.1 uses MNIST accuracies as quantitative evidence that attention mechanisms are effective. Moreover, CBAM is worse than the baseline on MNIST (98.00% vs 98.52%), so the statement that 'SelfAtt/MHA consistently outperform the baseline' is not true for CBAM, and the blanket conclusion in Section 5.3 needs to be qualified.
[Section 7] Section 7 contains the instruction 'Please add a section on the delegation of work among team members at the end of the report, in the form of a table and paragraph description.' This is an assignment prompt left in the manuscript and must be removed or replaced with a proper author-contribution or acknowledgments section before the paper can be considered publishable.

minor comments (6)

[Section 5.1] The phrase 'different experiments were ran for each model' should be 'were run'; also, 'we play with regularization' is too vague and should be replaced with a precise description of the searched hyperparameters.
[Section 4.3.2] The claim that omitting the sqrt(d_k) scaling factor 'aids faster convergence as it would lead to stronger gradients' is asserted without evidence; at minimum, provide a learning-curve comparison or temper the claim.
[Figures 5 and 6] Figures 5 and 6 would benefit from axis labels and legends that explicitly identify 'with/without residual connections' and 'weighted/unweighted,' as the current captions are ambiguous.
[Table 2] Table 2 compares CBAM reduction ratios without error bars; the differences among 90.66%, 90.32%, and 89.94% are small and should be reported with variance before concluding they are 'minuscule.'
[References] References [20] and [23] are GitHub repositories; please cite the underlying papers or document the specific repository versions used.
[Section 5.2.1] The sentence 'CBAM's final performance trails behind the other attention mechanisms and stabilizes to a higher test error compared to indicating less generalizable performance' is incomplete and should be rewritten.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark comparing measured accuracies, with no derivation step that reduces to its own inputs.

full rationale

This is an empirical study whose central claim ('CNNs indeed learn better when attention is added', Section 5.3) is supported by measured test accuracies in Table 3, not by a formula that assumes the conclusion. The baseline is an independent re-implementation of ResNet-20 validated against the original paper (9.04% test error vs. 8.75%), and the attention blocks are implemented from published external mechanisms (Vaswani et al., Zhang et al., Woo et al.). The CBAM reduction ratio is tuned on a validation split in Table 2 and the chosen r=16 model is then evaluated on the test set; this is standard model selection, not a fitted parameter being renamed as a prediction. The paper's own description in Section 5.1 that 'different experiments were ran for each model' and that regularization and learning rates were tuned per model raises legitimate statistical concerns about whether the small observed gaps are causal or due to training noise, but that is a validity/robustness issue, not circularity. The only self-citation is to the authors' GitHub repository [25], which is used to document code and MLFlow runs; it is not invoked as evidence for any architectural claim or as a uniqueness theorem. No step in the paper's argument is equivalent by construction to its inputs, so no circular step is found.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim is an empirical comparison, so no new theoretical entities or physical constants are introduced. The main free choices are hyperparameters like the CBAM reduction ratio and MHA head count, plus the architectural placement of attention, which is a design decision rather than a fitted parameter.

free parameters (2)

CBAM reduction ratio r = 16
Tuned over {8, 16, 32} on validation performance; CBAM-16 is used in the main comparison.
Number of MHA heads = 8
Chosen by hand with no ablation study; the paper assumes 8 heads improve distributed representation.

assumptions (3)

domain assumption CIFAR-10 and MNIST are representative benchmarks for evaluating CNN attention mechanisms.
All quantitative conclusions are drawn from these two small image datasets, and the paper generalizes to 'CNNs' broadly in Section 5.3.
domain assumption GradCAM visualizations faithfully reflect what the models attend to.
The qualitative claims about global dependency capture rely entirely on GradCAM heatmaps in Section 5.2.1 and Figures 9-11.
domain assumption The reimplemented ResNet-20 is an accurate reproduction of the original architecture of He et al.
The paper reports a 9.04% test error versus 8.75% in the original paper; the small gap is treated as acceptable for making comparisons.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CNNtention: Can CNNs do better with Attention?." pith.science (2026). https://pith.science/paper/PITCLEQ5

@misc{pith2026241211657,
  author       = {Pith},
  title        = {Pith review of: CNNtention: Can CNNs do better with Attention?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PITCLEQ5}},
  note         = {Machine review of arXiv:2412.11657}
}

read the original abstract

Convolutional Neural Networks (CNNs) have been the standard for image classification tasks for a long time, but more recently attention-based mechanisms have gained traction. This project aims to compare traditional CNNs with attention-augmented CNNs across an image classification task. By evaluating and comparing their performance, accuracy and computational efficiency, the project will highlight benefits and trade-off of the localized feature extraction of traditional CNNs and the global context capture in attention-augmented CNNs. By doing this, we can reveal further insights into their respective strengths and weaknesses, guide the selection of models based on specific application needs and ultimately, enhance understanding of these architectures in the deep learning community. This was our final project for CS7643 Deep Learning course at Georgia Tech.

Figures

Figures reproduced from arXiv: 2412.11657 by the authors.

**Figure 2.** Overall architecture with attention added, the Feature Extractor layers are sequential blocks of convolutions that [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Self Attention module introduced in Self [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: CBAM module introduced in [15]. CBAM block placement: Our implementation differs from the original paper [15] by adding the CBAM block after every residual block instead of within the residual block. 5. Experiments and Results All our experimentation are recorded on M…

**Figure 5.** Figure 5: Stable training with (right) and without (left) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 8.** Figure 8: Train and test error on different models on [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

**Figure 9.** Figure 9: GradCAM results on CIFAR-10 and MNIST. line model can extract better features. More such examples can be seen in [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]

**Figure 10.** Figure 10: More GradCAM results on CIFAR-10 images. [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]

**Figure 11.** Figure 11: More GradCAM results on MNIST images. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 26 canonical work pages

[1]

Deep convolutional neu- ral networks for image classification: A comprehensive re- view

Waseem Rawat and Zenghui Wang. Deep convolutional neu- ral networks for image classification: A comprehensive re- view. Neural Computation, 29(9):2352–2449, 09 2017. 1

work page 2017
[2]

A review of deep learn- ing in image recognition

Myeongsuk Pak and Sanghoon Kim. A review of deep learn- ing in image recognition. In 2017 4th International Confer- ence on Computer Applications and Information Processing Technology (CAIPT), pages 1–3, 2017. 1

work page 2017
[3]

Cuevas-Tello, Jose Nunez-Varela, Cesar Puente, and Alejandra G

Gabriela Rangel, Juan C. Cuevas-Tello, Jose Nunez-Varela, Cesar Puente, and Alejandra G. Silva-Trujillo. A survey on convolutional neural networks and their performance lim- itations in image recognition tasks. Journal of Sensors , 2024(1):2797320, 2024. 1

work page 2024
[4]

Contextual convolutional neural net- works, 2021

Ionut Cosmin Duta, Mariana Iuliana Georgescu, and Radu Tudor Ionescu. Contextual convolutional neural net- works, 2021. 1

work page 2021
[5]

Noisynn: Exploring the impact of information entropy change in learning systems, 2024

Xiaowei Yu, Zhe Huang, Minheng Chen, Yao Xue, Tianming Liu, and Dajiang Zhu. Noisynn: Exploring the impact of information entropy change in learning systems, 2024. 1

work page 2024
[6]

An image is worth 16x16 words: Transformers for image recognition at scale, 2021

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. 1

2021
[7]

Do vision trans- formers see like convolutional neural networks?, 2022

Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks?, 2022. 1

work page 2022
[8]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. CoRR, abs/2010.11929, 2020. 1, 5

arXiv 2010

Show all 31 references

[9]

Squeeze-and-excitation networks, 2019

Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. Squeeze-and-excitation networks, 2019. 1

2019
[10]

Spatial transformer networks, 2016

Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks, 2016. 1

2016
[11]

Attention u-net: Learning where to look for the pancreas, 2018

Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, Ben Glocker, and Daniel Rueckert. Attention u-net: Learning where to look for the pancreas, 2018. 1

2018
[12]

Csanet: Channel spatial attention network for robust 3d face alignment and reconstruction, 2024

Yilin Liu, Xuezhou Guo, Xinqi Wang, and Fangzhou Du. Csanet: Channel spatial attention network for robust 3d face alignment and reconstruction, 2024. 1

2024
[13]

Ca-net: Comprehensive attention con- volutional neural networks for explainable medical image segmentation

Ran Gu, Guotai Wang, Tao Song, Rui Huang, Michael Aert- sen, Jan Deprest, Sebastien Ourselin, Tom Vercauteren, and Shaoting Zhang. Ca-net: Comprehensive attention con- volutional neural networks for explainable medical image segmentation. IEEE Transactions on Medical Imaging ,...

2021
[14]

Ela: Efficient local attention for deep convolutional neural networks, 2024

Wei Xu and Yi Wan. Ela: Efficient local attention for deep convolutional neural networks, 2024. 1

2024
[15]

Cbam: Convolutional block attention module, 2018

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module, 2018. 1, 3, 4, 10

2018
[16]

Cifar- 10 and cifar-100 (canadian institute for advance research)

Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar- 10 and cifar-100 (canadian institute for advance research)
[17]

Antonio Torralba, Rob Fergus, and William T. Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pat- tern Analysis and Machine Intelligence , 30(11):1958–1970,

1958
[18]

Learning multiple layers of features from tiny images

Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 2

2009
[19]

Deep residual learning for image recognition, 2015

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. 2, 10

2015
[20]

Deep residual learning for image recognition: Cifar-10, pytorch implementation

Martyn A. Deep residual learning for image recognition: Cifar-10, pytorch implementation. https://github. com/a-martyn/resnet. Accessed: 2024-12-05. 2

2024
[21]

MNIST handwritten digit database

Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. 2

2010
[22]

M ¨uller and Karla Markert

Nicolas M. M ¨uller and Karla Markert. Identifying misla- beled instances in classification datasets. pages 1–8, 2019. 2

2019
[23]

Proper ResNet implementation for CI- FAR10/CIFAR100 in PyTorch.https://github.com/ akamaster/pytorch_resnet_cifar10

Yerlan Idelbayev. Proper ResNet implementation for CI- FAR10/CIFAR100 in PyTorch.https://github.com/ akamaster/pytorch_resnet_cifar10. Accessed: 2024-12-05. 2

2024
[24]

Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Ba- tra

Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Ba- tra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR, abs/1610.02391, 2016. 2

arXiv
[25]

CNNten- tion Github Repository

Tejas Rathi Julian Glattki, Nikhil Kapila. CNNten- tion Github Repository. https://github.com/ AttentionSeekers/CNNtention, 2024. 2, 4

2024
[26]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. CoRR, abs/1706.03762, 2017. 3, 4, 5

2017 arXiv
[27]

Irwan Bello, Barret Zoph, Ashish Vaswani, Jonathon Shlens, and Quoc V . Le. Attention augmented convolutional net- works, 2020. 3

2020
[28]

Self-attention generative adversarial networks,

Han Zhang, Ian Goodfellow, Dimitris Metaxas, and Augus- tus Odena. Self-attention generative adversarial networks,
[29]

Mlflow: An open source platform for the ma- chine learning lifecycle

Databricks. Mlflow: An open source platform for the ma- chine learning lifecycle. 4
[30]

Weighted residuals for very deep networks

Falong Shen and Gang Zeng. Weighted residuals for very deep networks. CoRR, abs/1605.08831, 2016. 5

2016 arXiv
[31]

Going deeper with convolutions, 2014

Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions, 2014. 6 11

2014

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Deep convolutional neu- ral networks for image classification: A comprehensive re- view

Waseem Rawat and Zenghui Wang. Deep convolutional neu- ral networks for image classification: A comprehensive re- view. Neural Computation, 29(9):2352–2449, 09 2017. 1

work page 2017

[2] [2]

A review of deep learn- ing in image recognition

Myeongsuk Pak and Sanghoon Kim. A review of deep learn- ing in image recognition. In 2017 4th International Confer- ence on Computer Applications and Information Processing Technology (CAIPT), pages 1–3, 2017. 1

work page 2017

[3] [3]

Cuevas-Tello, Jose Nunez-Varela, Cesar Puente, and Alejandra G

Gabriela Rangel, Juan C. Cuevas-Tello, Jose Nunez-Varela, Cesar Puente, and Alejandra G. Silva-Trujillo. A survey on convolutional neural networks and their performance lim- itations in image recognition tasks. Journal of Sensors , 2024(1):2797320, 2024. 1

work page 2024

[4] [4]

Contextual convolutional neural net- works, 2021

Ionut Cosmin Duta, Mariana Iuliana Georgescu, and Radu Tudor Ionescu. Contextual convolutional neural net- works, 2021. 1

work page 2021

[5] [5]

Noisynn: Exploring the impact of information entropy change in learning systems, 2024

Xiaowei Yu, Zhe Huang, Minheng Chen, Yao Xue, Tianming Liu, and Dajiang Zhu. Noisynn: Exploring the impact of information entropy change in learning systems, 2024. 1

work page 2024

[6] [6]

An image is worth 16x16 words: Transformers for image recognition at scale, 2021

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. 1

2021

[7] [7]

Do vision trans- formers see like convolutional neural networks?, 2022

Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks?, 2022. 1

work page 2022

[8] [8]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. CoRR, abs/2010.11929, 2020. 1, 5

arXiv 2010