Pith. sign in

REVIEW 3 major objections 6 minor 35 references

SpikeAtConv: An Integrated Spiking-Convolutional Attention Architecture for Energy-Efficient Neuromorphic Vision Processing

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A directly trained spiking network with multi-threshold LIF neuron blocks reports 81.23% top-1 accuracy on ImageNet-1K, which the authors describe as a state-of-the-art result for SNNs.

desk verdict The architecture idea is real but the headline SOTA claim is untraceable: the core grid-attention module is never specified, and the paper never says whether the ImageNet models even use the non-degenerate attention variant. read the letter →

arxiv 2411.17439 v1 pith:2E3ZCSSL submitted 2024-11-26 cs.NE

Wangdan Liao , Weidong Wang This is my paper · ORCID
classification cs.NE
keywords spikingneuralnetworksneuromorphicvisionLIFneuronmulti-branchparallelspike-drivenattentionImageNet-1KdirecttrainingMaxViT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SpikeAtConv aims to show that a spiking neural network, not converted from a trained artificial network, can compete with conventional vision models on large-scale image classification. The authors build a MaxViT-style hybrid of convolution and attention, replace activations with spiking neuron blocks, and report 81.23% top-1 accuracy on ImageNet-1K for their largest model, which they call a state-of-the-art result among SNNs. The main design lesson is that running several LIF neurons with different thresholds in parallel, rather than a single spiking neuron, preserves more information and raises accuracy. The authors also report that a simplified 'binary direct-spike' attention variant, which skips computing query, key, and value, did not significantly degrade performance, suggesting attention complexity can be cut. If these results hold, directly trained SNNs are closer to ANN accuracy on hard vision tasks than earlier results suggested.

What carries the argument

The load-bearing object is the Leaky Integrate-and-Fire (LIF) neuron and its variants, particularly the Multi-Branch Parallel LIF (MBPL) block: several LIF neurons with different thresholds process the same features in parallel, and their outputs are combined before a ConvNeXt module, compensating for information lost when a single neuron converts continuous activations into binary spikes. The second mechanism is the Attention SpikeMerge Block, with two modes: Spike-Integrated Self-Attention (SISA), which puts spiking blocks around the Q/K/V computation and after the attention map, and Binary Direct-Spike Attention (BDSA), which skips Q/K/V entirely and multiplies identical binary spike vectors. The paper also names Spike-Driven Grid Attention as the module enabling global spatial interactions, but supplies no equations or pseudocode for it; its role in the argument is to inherit MaxViT's multi-axis attention design.

What would settle it

Train the Large SpikeAtConv on ImageNet-1K with the Spike-Driven Grid Attention module removed and replaced by an identity, keeping every other hyperparameter fixed; if top-1 accuracy stays near 80.5% or higher, the attention module is not load-bearing, while a clear drop below the Base model level would confirm its contribution.

Watch

Extended reading notes

Core claim

The paper's central claim is that a directly trained SNN can reach 81.23% top-1 on ImageNet-1K, exceeding the comparison SNN models Meta-SpikeFormer (79.1%) and SpikFormer (74.8%) and setting what the authors call a state-of-the-art result for SNNs. The claim is carried by SpikeAtConv, a four-stage spiking network whose blocks combine ConvNeXt-style convolutions with a Spike-Driven Grid Attention module. A second claim is that the choice of spiking neuron configuration matters: the Multi-Branch Parallel LIF block, with four parallel LIF neurons at thresholds 0.2, 1, 2, and 4, outperforms single, residual, hidden-split, and dual-convolution blocks on both CIFAR-100 and ImageNet-1K. A third claim is that a degenerate attention mode that treats Q, K, and V as identical spikes does not significantly hurt accuracy, which the authors read as evidence that their spiking blocks carry most of the representational load.

Load-bearing premise

The load-bearing premise is that the spike-form grid attention module preserves or improves MaxViT's representational capacity; the paper provides no equations, pseudocode, or ablation isolating that module, so if the premise fails, the reported accuracy cannot be attributed to the claimed attention mechanism.

Editorial extensions

If this is right

  • If the 81.23% result reproduces, directly trained SNNs can exceed 81% top-1 on ImageNet-1K with 224x224 input, a level previously associated with ANN-style performance.
  • The MBPL configuration becomes a default building block for spike-based vision backbones, since it beat single-neuron and residual variants on both CIFAR-100 and ImageNet-1K.
  • The BDSA result implies that attention-weight computation can be dropped or drastically simplified in spike-form transformers, reducing multiply-accumulate operations without proportional accuracy loss.
  • The authors' observation that LIF-based modules lose information suggests architectures should route redundant features around spiking activations, not just replace ReLU with spiking neurons.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the near-tie between the spike-free model (81.13%) and the best spiking model (80.53%) suggests the spike encoding itself costs roughly 0.6 top-1 points on ImageNet-1K; closing that gap is the real target for future SNN designs.
  • Beyond the paper: because Spike-Driven Grid Attention is never ablated, the most economical reading is that accuracy comes mainly from the SPK blocks and ConvNeXt backbone, not from attention; replacing the grid-attention block with an identity should be tested.
  • Beyond the paper: the energy-efficiency advantage of SNNs remains asserted rather than measured; an energy simulation or neuromorphic hardware study would be needed to turn the accuracy result into an efficiency claim.
  • Beyond the paper: applying MBPL blocks to spike-driven transformers for detection or segmentation would test whether multi-threshold parallel neurons help dense prediction tasks, not just classification.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes SpikeAtConv, a spiking neural network architecture that combines MaxViT-inspired multi-axis grid attention with convolutional blocks and several custom LIF-based SPK blocks. The authors report CIFAR-100 experiments that compare SPK block variants and ImageNet-1K experiments for Tiny, Base, and Large models, claiming a state-of-the-art 81.23% top-1 accuracy among directly trained SNNs. The central claim is that the proposed Spike-Driven Grid Attention and MBPL spiking neurons enable this result. However, the attention mechanism is never formally specified, the paper does not state which of the two proposed attention variants is used in the ImageNet experiments, and the reported spike-free baseline nearly matches the claimed SOTA, so the central claim is not currently verifiable.

Significance. If the 81.23% ImageNet-1K result were fully specified and reproducible, it would be a notable advance for directly trained SNNs and would strengthen the case for spike-based attention and multi-threshold spiking neurons. The paper deserves credit for conducting systematic comparisons of SPK block configurations and for reporting a spike-free baseline, which is a useful control. However, the significance is strongly conditional: the core attention module is undefined, the models are not identified by attention variant, parameter counts are missing from Table 4, no error bars or seeds are reported, and no code or checkpoints are provided. As written, the contribution cannot be verified or attributed to the proposed architecture.

major comments (3)
  1. [§3.1, §3.3, Table 4] The central contribution, Spike-Driven Grid Attention, is never specified. Section 3.1 names the module, but no equations, pseudocode, or architectural diagram define how Q, K, and V are formed or how grid attention is spikified. Section 3.3 introduces two variants, SISA and BDSA, but neither the experimental section nor Table 4 states which variant is used in the reported ImageNet models; BDSA explicitly bypasses the computation of Q, K, and V and treats them as identical, so if the Large model uses BDSA, the claimed grid-attention mechanism is absent from the SOTA result. The conclusion's admission that a degenerate self-attention mechanism did not degrade performance significantly reinforces this concern. The 81.23% top-1 result therefore cannot be attributed to the proposed attention architecture, and the experiment is not independently reproducible from the text.
  2. [Table 3, Table 4, §4.3] The evidence does not support the claim that the spiking components improve accuracy. In Table 3, the spike-free baseline reaches 81.13% top-1, which exceeds the spiking Base model (80.53%) and is within 0.10 points of the Large model (81.23%); no error bars, seeds, or significance tests are reported, so these small differences are not established as meaningful. Additionally, Table 4 is announced in §4.3 as including the number of parameters for each model, but the parameter row is missing, so the comparison to Meta-SpikeFormer and SpikFormer is not apples-to-apples in model scale or compute.
  3. [§4.1, §4.2] The hyperparameter selection procedure creates a model-selection risk: MBPL branch counts, voltage thresholds, time steps, and surrogate functions are tuned on CIFAR-100 in Tables 1 and 2 and then transferred to ImageNet without any correction for selection bias or evaluation of variance. Because the ImageNet numbers in Tables 3 and 4 are single runs, it is unclear whether the reported accuracies reflect the architecture's average behavior or a favorable configuration selected through the CIFAR-100 loop.
minor comments (6)
  1. [Abstract, §1, §3] The abstract and introduction use the term 'pulse modules' while the rest of the paper refers to 'SPK Blocks'; please unify the terminology.
  2. [§1, Contribution 4] Contribution 4 contains a typo: 'eperiments' should be 'experiments'.
  3. [§4.3, after Table 3] The discussion of Table 3 refers to an 'SSL' module, but the defined module is 'HSL' (Hidden Split LIF); this typo should be corrected.
  4. [Figure 4] The caption for Figure 4 says the zoomed panel covers the first 20 epochs, but the axis is labeled 1 through 10; align the caption with the axis range.
  5. [§3.2, DCL Block] The DCL Block description says each branch halves the channel dimension and the outputs are concatenated, but it is not stated how the 3x3 and 5x5 branches handle odd channel counts; please clarify the dimension arithmetic.
  6. [Overall] The paper does not state whether code or trained checkpoints will be released; providing them would substantially improve the reproducibility of the reported ImageNet results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the reported ImageNet results are empirical benchmarks, and the CIFAR-100 hyperparameter selection is model selection rather than a fitted-input prediction.

full rationale

This paper does not present an equation-level derivation chain from assumptions to predictions; it reports measured accuracies on CIFAR-100 and ImageNet-1K. The central claim, 81.23% top-1 accuracy for Large SpikeAtConv, is an empirical result obtained by directly training the model, and the paper does not attempt to derive that number from any fitted parameter. The SPK Block hyperparameters were explored and selected on CIFAR-100 and then used for ImageNet experiments; this is standard model selection, and the ImageNet numbers are new measurements rather than predictions statistically forced by the CIFAR-100 fits. There are no load-bearing self-citations: the cited works (MaxViT, ConvNeXt, Meta-SpikeFormer, SpikFormer, etc.) are external references, and no uniqueness theorem or ansatz from the authors' own prior work is invoked. The paper has serious clarity issues: Spike-Driven Grid Attention is named in Section 3.1 but never formally defined, Section 3.3 defines two Attention SpikeMerge Blocks (SISA and BDSA) without stating which one is used in the ImageNet models, and Table 4 omits parameter counts. These are reproducibility and attribution concerns, not circularity: the reported measurements are not equivalent to their inputs by construction, and no equation or fitted parameter is renamed as a prediction. Therefore, under the specified criteria, the appropriate finding is no significant circularity, with score 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim depends on several hand-chosen hyperparameters and on unstated assumptions about the viability of spike-form attention and surrogate-gradient training. The paper reports no independent energy or latency measurements.

free parameters (6)
  • LIF voltage thresholds in MBPL block = 0.2, 1, 2, 4
    Chosen from CIFAR-100 sweeps in Tables 1 and 2 and used in ImageNet experiments; the reported accuracy depends on these thresholds.
  • Membrane time constant tau = 2.0
    Set for LIF neurons in all main experiments; selected by validation accuracy in small-scale tests.
  • Simulation time steps T = 1 for ImageNet, 2 for variant
    T=1 is used for Base, Tiny, and Large models; a T=2 variant is also reported. Longer windows gave mixed results on CIFAR-100.
  • Surrogate gradient function = ATan
    ATan outperformed Sigmoid in CIFAR-100 ablations and is used in the main ImageNet experiments.
  • Number of MBPL branches = 4
    Four branches was the best-performing count on CIFAR-100; eight branches collapsed to 59.0% top-1 accuracy.
  • Stage depths for Large model = 2, 6, 16, 2
    Hand-chosen spindle-shaped depth distribution across the four encoding stages; the Large model's reported accuracy depends on this scale configuration.
assumptions (4)
  • domain assumption The LIF dynamics in Eq. 1, combined with a surrogate gradient, are a valid model for training the network.
    Section 3.2 introduces the LIF equation and Section 4.1 the surrogate functions, but no analysis shows the discretized spike derivative is unbiased or that training converges for deep networks.
  • ad hoc to paper MaxViT's multi-axis grid attention can be converted to spike-domain Q, K, V without losing the global mixing property.
    The introduction claims global spatial interactions within a single block for Spike-Driven Grid Attention, but Section 3.3 provides no equations for the grid attention, only two attention variants.
  • domain assumption Direct training on ImageNet for 200 epochs with the stated augmentation recipes is sufficient to support the SOTA comparison.
    The paper compares only two SNN baselines in Table 4 and does not report compute, seeds, or convergence criteria.
  • domain assumption The spike-free baseline in Table 3 is a valid control for the value of the spiking modules.
    The paper lists spike-free 81.13 in Table 3 but never explains what this baseline is or why the spiking model's near-tie is still an improvement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SpikeAtConv: An Integrated Spiking-Convolutional Attention Architecture for Energy-Efficient Neuromorphic Vision Processing." pith.science (2026). https://pith.science/paper/2E3ZCSSL

@misc{pith2026241117439,
  author       = {Pith},
  title        = {Pith review of: SpikeAtConv: An Integrated Spiking-Convolutional Attention Architecture for Energy-Efficient Neuromorphic Vision Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2E3ZCSSL}},
  note         = {Machine review of arXiv:2411.17439}
}
read the original abstract

Spiking Neural Networks (SNNs) offer a biologically inspired alternative to conventional artificial neural networks, with potential advantages in power efficiency due to their event-driven computation. Despite their promise, SNNs have yet to achieve competitive performance on complex visual tasks, such as image classification. This study introduces a novel SNN architecture designed to enhance computational efficacy and task accuracy. The architecture features optimized pulse modules that facilitate the processing of spatio-temporal patterns in visual data, aiming to reconcile the computational demands of high-level vision tasks with the energy-efficient processing of SNNs. Our evaluations on standard image classification benchmarks indicate that the proposed architecture narrows the performance gap with traditional neural networks, providing insights into the design of more efficient and capable neuromorphic computing systems.

Figures

Figures reproduced from arXiv: 2411.17439 by the authors.

Figure 1
Figure 1. Inspired by MaxViT, we propose a novel spike-driven transformer module named Spike-Driven Grid [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 1
Figure 1. The overview of SpikeAtConv. The model is primarily composed of three components: the Feature Extraction Layer, the Feature Encoding Layer, and the Decision Layer. Initially, the input image is subjected to preliminary processing within the Feature Extraction Layer, where essential characteristics are identified. Subsequently, the Feature Encoding Layer performs a comprehensive analysis to distill salient features f… view at source ↗
Figure 2
Figure 2. SPKBlock. Based on LIF neurons, we designed multiple SPK blocks to explore the impact of various hyperparameters and different combinations of multiple neurons on network performance. For example, the MBPL Block consists of multiple parallel neurons with different thresholds, while the DCL Block is composed of two parallel branches, each including a convolutional layer and a LIF neuron. τm dV dt = −(V − Vrest) + RI(… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: Attention SpikeMerge Block. This represents two different computational approaches. In the SISA Block, after computing the Q, K, and V, we add the SPK Blcok separately to obtain the spike form of Q, K, and V. Subsequently, we use Q and K to calculate the attention scor…
Figure 4
Figure 4. Figure 4: Comparison of Loss Between MaxViT and SpikeAtConv. We present the training and validation loss trajectories of our SpikeAtConv and MaxViT models. In the figure below, we emphasize the loss variations during the first 10 epochs. It is evident that SpikeAtConv experience…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 12 canonical work pages

  1. [1]

    Networks of spiking neurons: the third generation of neural network models

    Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural networks, 10(9):1659–1671, 1997

    1997

  2. [2]

    Towards spike-based machine intelligence with neuromorphic computing

    Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575(7784):607–617, 2019

    2019

  3. [3]

    Brain-inspired computing: A systematic survey and future trends

    Guoqi Li, Lei Deng, Huajin Tang, Gang Pan, Yonghong Tian, Kaushik Roy, and Wolfgang Maass. Brain-inspired computing: A systematic survey and future trends. Proceedings of the IEEE, 2024

    work page 2024

  4. [4]

    Bottom-up and top-down approaches for the design of neuromorphic processing systems: tradeoffs and synergies between natural and artificial intelligence

    Charlotte Frenkel, David Bol, and Giacomo Indiveri. Bottom-up and top-down approaches for the design of neuromorphic processing systems: tradeoffs and synergies between natural and artificial intelligence. Proceedings of the IEEE, 111(6):623–652, 2023

    work page 2023

  5. [5]

    A million spiking-neuron integrated circuit with a scalable communication network and interface

    Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668–673, 2014

    2014

  6. [6]

    Loihi: A neuromorphic manycore processor with on-chip learning

    Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, et al. Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro, 38(1):82–99, 2018

    2018

  7. [7]

    Towards artificial general intelligence with hybrid tianjic chip architecture

    Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei He, et al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 572(7767):106– 111, 2019

    2019

  8. [8]

    Backpropagation applied to handwritten zip code recognition

    Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541– 551, 1989

    work page 1989

Show all 35 references
  1. [9]

    Progressive tandem learning for pattern recognition with deep spiking neural networks

    Jibin Wu, Chenglin Xu, Xiao Han, Daquan Zhou, Malu Zhang, Haizhou Li, and Kay Chen Tan. Progressive tandem learning for pattern recognition with deep spiking neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(11):7824–7840, 2021

    2021

  2. [10]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  3. [11]

    Spiking transformers for event-based single object tracking

    Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Felix Heide, Baocai Yin, and Xin Yang. Spiking transformers for event-based single object tracking. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 8801–8810, 2022

    2022

  4. [12]

    Spik- former: When spiking neural network meets transformer

    Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng YAN, Yonghong Tian, and Li Yuan. Spik- former: When spiking neural network meets transformer. In The Eleventh International Conference on Learning Representations, 2023

    2023

  5. [13]

    Spike-driven transformer

    Man Yao, Jiakui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo Xu, and Guoqi Li. Spike-driven transformer. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 64043–64058, 2023

    2023

  6. [14]

    Learning representations by back-propagating errors

    David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986

    1986

  7. [15]

    Optimal conversion of conventional artificial neural networks to spiking neural networks

    Shikuang Deng and Shi Gu. Optimal conversion of conventional artificial neural networks to spiking neural networks. In International Conference on Learning Representations, 2021

    2021

  8. [16]

    Advancing spiking neural networks toward deep residual learning

    Yifan Hu, Lei Deng, Yujie Wu, Man Yao, and Guoqi Li. Advancing spiking neural networks toward deep residual learning. IEEE Transactions on Neural Networks and Learning Systems, 2024

    2024

  9. [17]

    Spatio-temporal backpropagation for training high- performance spiking neural networks

    Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high- performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018. 10

    2018

  10. [18]

    Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks

    Emre O Neftci, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36(6):51–63, 2019

    2019

  11. [19]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  12. [20]

    Going deeper with directly-trained larger spiking neural networks

    Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly-trained larger spiking neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 11062–11070, 2021

    2021

  13. [21]

    Deep residual learning in spiking neural networks

    Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 34:21056–21069, 2021

    2021

  14. [22]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  15. [23]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision, pa...

    2021

  16. [24]

    Mlp-mixer: An all-mlp architecture for vision

    Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:242...

    2021

  17. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  18. [26]

    Twins: Revisiting the design of spatial attention in vision transformers

    Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. Advances in neural information processing systems, 34:9355–9366, 2021

    2021

  19. [27]

    Efficientnet: Rethinking model scaling for convolutional neural networks

    Mingxing Tan. Efficientnet: Rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946, 2019

    1905 arXiv

  20. [28]

    Convnext v2: Co-designing and scaling convnets with masked autoencoders

    Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, In So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16133–16142, 2023

    2023

  21. [29]

    Maxvit: Multi-axis vision transformer

    Zhengzhong Tu, Hossein Talebi, Han Zhang, Feng Yang, Peyman Milanfar, Alan Bovik, and Yinxiao Li. Maxvit: Multi-axis vision transformer. In European conference on computer vision, pages 459–479. Springer, 2022

    2022

  22. [30]

    Coatnet: Marrying convolution and attention for all data sizes

    Zihang Dai, Hanxiao Liu, Quoc V Le, and Mingxing Tan. Coatnet: Marrying convolution and attention for all data sizes. Advances in neural information processing systems, 34:3965–3977, 2021

    2021

  23. [31]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  24. [32]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  25. [33]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009

    2009

  26. [34]

    How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.04819, 2020

    Linjun Zhang, Zhun Deng, Kenji Kawaguchi, Amirata Ghorbani, and James Zou. How does mixup help with robustness and generalization? arXiv preprint arXiv:2010.04819, 2020

    2010 arXiv

  27. [35]

    Cutmix: Regularization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023–6032, 2019. 11

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.