pith. sign in

arxiv: 2607.00774 · v1 · pith:4COECRUGnew · submitted 2026-07-01 · 💻 cs.CV · cs.LG

Soft Mixture-of-Recursions: Going Deeper with Recursive Vision Transformers

Sang In Lee , Jihun Park This is my paper

Pith reviewed 2026-07-02 14:12 UTC · model grok-4.3

classification 💻 cs.CV cs.LG
keywords recursive vision transformerssoft mixture of recursionsvision transformersparameter efficient deep learningimage classificationmixture weightsrecursion depth
0
0 comments X

The pith

A soft mixture of recursion outputs allows Vision Transformers to benefit from greater depth with little added cost.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Recursive Vision Transformers have not improved reliably when made deeper because they discard intermediate computation steps. Soft Mixture-of-Recursions addresses this by learning token-specific weights that combine the results from every recursion level. This lets SR-ViT raise accuracy on ImageNet-1K from 79.83 percent to 82.48 percent when recursion depth goes from one to four, adding only 1.7 million parameters. The same pattern holds on other vision tasks.

Core claim

SoftMoR learns token-wise mixture weights to softly combine outputs from all recursion steps, allowing intermediate representations to be utilized in a learnable and flexible way. This enables SR-ViT to improve performance consistently as recursion depth increases with minimal parameter overhead.

What carries the argument

Token-wise mixture weights that softly combine the outputs from every recursion step.

If this is right

  • Increasing recursion depth from 1 to 4 raises SR-ViT-S top-1 accuracy on ImageNet-1K from 79.83% to 82.48% with 1.7M extra parameters.
  • SR-ViT-S at depth 4 outperforms the larger DeiT-B while using roughly 27% of its parameters.
  • The gains hold across diverse vision tasks with only minimal added parameters.
  • Recursion becomes a practical route to deeper Vision Transformers rather than a parameter-saving trick alone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This mixing strategy may apply to recursive models in other domains like language or audio.
  • Future work could explore whether the mixture weights reveal which depths matter most for different tokens or tasks.
  • Combining this with other efficiency techniques might push performance further without proportional size growth.

Load-bearing premise

That learning per-token weights to mix recursion outputs will turn added recursion depth into reliable gains in representational power without training instability or overfitting.

What would settle it

A controlled experiment on ImageNet-1K showing that SR-ViT accuracy plateaus or declines when recursion depth increases past 4 under fixed training conditions would disprove the central claim.

Figures

Figures reproduced from arXiv: 2607.00774 by Jihun Park, Sang In Lee.

Figure 1
Figure 1. Figure 1: SoftMoR makes Vision Transformers effectively deeper [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of Soft Recursive Vision Transformer (SR-ViT) with Soft Mixture-of-Recursions (SoftMoR). (a) SR-ViT orga￾nizes intermediate ViT layers into SoftMoR units, where shared Transformer blocks are recursively reused across computation steps. (b) SoftMoR learns token-wise mixture weights over recursion steps using the Soft Router and aggregates outputs from all recursion steps through a Soft Mixture. (c)… view at source ↗
Figure 3
Figure 3. Figure 3: Conceptual illustrations of different recursive unit organizations. Blue boxes [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of learned token-wise mixture weights [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Token-level linear CKA across depth for DeiT-S and recursive ViT variants with different recursive unit organizations. DeiT-S [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mean token-level linear CKA according to layer distance [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional visualizations of learned mixture weights on COCO. We visualize token-wise mixture weights from the final recursive [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Additional visualizations of learned mixture weights on ADE20K. We visualize token-wise mixture weights from the final [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
read the original abstract

Recent recursive Transformer studies have primarily reused shared parameters across computation steps to construct compact, parameter-efficient models. In this work, we leverage recursion to build effectively deeper Transformers with stronger representational capacity. However, in Vision Transformers, simply increasing recursion depth does not reliably improve performance, as existing recursive approaches do not fully utilize the intermediate representations produced throughout recursive computation. We propose Soft Mixture-of-Recursions (SoftMoR) and its Vision Transformer instantiation, Soft Recursive Vision Transformer (SR-ViT). SoftMoR learns token-wise mixture weights to softly combine outputs from all recursion steps, allowing intermediate representations to be utilized in a learnable and flexible way. Across diverse vision tasks, SR-ViT consistently improves as recursion depth increases with minimal parameter overhead. On ImageNet-1K, increasing recursion depth from 1 to 4 improves SR-ViT-S top-1 accuracy from 79.83% to 82.48% with only 1.7M additional parameters, outperforming the substantially larger DeiT-B while using approximately 27% of its parameters. These results demonstrate that SoftMoR provides a parameter-efficient path to deeper and stronger Vision Transformers through recursion.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes Soft Mixture-of-Recursions (SoftMoR) and its instantiation SR-ViT to address performance plateaus in recursive Vision Transformers. It claims that learning token-wise mixture weights to softly combine outputs from all recursion steps enables effective use of intermediate representations, allowing consistent accuracy gains with increased recursion depth and minimal added parameters. On ImageNet-1K, SR-ViT-S improves from 79.83% top-1 at depth 1 to 82.48% at depth 4 using only 1.7M extra parameters while outperforming the larger DeiT-B (using ~27% of its parameters). Similar benefits are reported across diverse vision tasks.

Significance. If the soft mixture mechanism is shown to meaningfully incorporate intermediate recursion outputs rather than collapsing, the work offers a parameter-efficient route to deeper Vision Transformers. This could influence scaling strategies by demonstrating that recursion depth can be converted into representational capacity without proportional parameter growth.

major comments (2)
  1. [Abstract] Abstract: The central claim attributes accuracy gains to the learned token-wise mixture utilizing all recursion steps. However, no statistics on mixture weight distributions, entropy, or per-step mass are referenced, leaving open the possibility that weights collapse to the final step and that gains arise from repeated forward passes of the shared block rather than the proposed SoftMoR mechanism.
  2. [Experiments] Experiments (performance tables): The reported depth-scaling results (e.g., 79.83% o 82.48%) are presented without ablations that disable early recursion contributions or isolate the mixer, and without training curves or significance tests. This weakens verification that the soft combination, rather than extra computation or hyperparameter effects, drives the improvements.
minor comments (1)
  1. [Method] Clarify the exact formulation of the token-wise mixture weights with an explicit equation in the method section to aid reproducibility.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. We address the two major comments below and will revise the manuscript to include additional analyses that directly support the claims regarding the SoftMoR mechanism.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim attributes accuracy gains to the learned token-wise mixture utilizing all recursion steps. However, no statistics on mixture weight distributions, entropy, or per-step mass are referenced, leaving open the possibility that weights collapse to the final step and that gains arise from repeated forward passes of the shared block rather than the proposed SoftMoR mechanism.

    Authors: We agree that explicit statistics on the mixture weights would provide stronger evidence against collapse. In the revision we will add quantitative analysis of the learned token-wise weights, including per-step mass distributions, average entropy, and visualizations of weight allocation across recursion depths. These additions will demonstrate that early recursion steps receive non-negligible mass on average and that the mechanism does not trivially reduce to using only the final step. revision: yes

  2. Referee: [Experiments] Experiments (performance tables): The reported depth-scaling results (e.g., 79.83% o 82.48%) are presented without ablations that disable early recursion contributions or isolate the mixer, and without training curves or significance tests. This weakens verification that the soft combination, rather than extra computation or hyperparameter effects, drives the improvements.

    Authors: We acknowledge the value of targeted ablations. The revised version will include (i) an ablation that replaces the soft mixer with a hard selection of only the final recursion output, (ii) training curves comparing different depths, and (iii) standard deviations over multiple random seeds where feasible. These experiments will isolate the contribution of the learned mixture from mere repeated application of the shared block. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical results from new architecture, not derived from inputs

full rationale

The paper proposes SoftMoR as a novel mechanism for token-wise mixing of recursion outputs in recursive ViTs and reports direct empirical accuracy measurements on ImageNet and other tasks. No equations, predictions, or central claims reduce the reported gains (e.g., 79.83% to 82.48%) to quantities defined by the authors' own fitted parameters, prior self-citations, or ansatzes. The derivation chain consists of an architectural description followed by experimental validation, which is self-contained and externally falsifiable via replication on standard benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The work is an empirical neural-architecture proposal. No theoretical free parameters are introduced beyond the standard learned weights of the transformer; the mixture weights themselves are ordinary model parameters trained by gradient descent.

axioms (1)
  • domain assumption Gradient-based optimization on the mixture weights will converge to a useful blending of recursion outputs
    Implicit in any learned-mixer architecture; required for the performance claim to hold.
invented entities (1)
  • Soft Mixture-of-Recursions (SoftMoR) mixer no independent evidence
    purpose: Token-wise soft combination of all recursion-step outputs
    New architectural component introduced to solve the intermediate-representation utilization problem stated in the abstract.

pith-pipeline@v0.9.1-grok · 5741 in / 1405 out tokens · 22975 ms · 2026-07-02T14:12:37.924796+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 45 canonical work pages

  1. [1]

    Relaxed recursive trans- formers: Effective parameter sharing with layer-wise lora

    Sangmin Bae, Adam Fisch, Hrayr Harutyunyan, Ziwei Ji, Seungyeon Kim, and Tal Schuster. Relaxed recursive trans- formers: Effective parameter sharing with layer-wise lora. InInternational Conference on Learning Representations,

    work page

  2. [2]

    Mixture-of-recursions: Learn- ing dynamic recursive depths for adaptive token-level com- putation.Advances in Neural Information Processing Sys- tems, 38:96572–96617, 2026

    Sangmin Bae, Yujin Kim, Reza Bayat, Sungnyun Kim, Jiy- oun Ha, Tal Schuster, Adam Fisch, Hrayr Harutyunyan, Zi- wei Ji, Aaron Courville, et al. Mixture-of-recursions: Learn- ing dynamic recursive depths for adaptive token-level com- putation.Advances in Neural Information Processing Sys- tems, 38:96572–96617, 2026. 1, 3, 5

    work page 2026

  3. [3]

    Deep equi- librium models.Advances in neural information processing systems, 32, 2019

    Shaojie Bai, J Zico Kolter, and Vladlen Koltun. Deep equi- librium models.Advances in neural information processing systems, 32, 2019. 3

    work page 2019

  4. [4]

    MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.https : / / github

    MMSegmentation Contributors. MMSegmentation: Openmmlab semantic segmentation toolbox and benchmark.https : / / github . com / open - mmlab/mmsegmentation, 2020. 6, 11

    work page 2020

  5. [5]

    Randaugment: Practical automated data augmen- tation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmen- tation with a reduced search space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. 10

    work page 2020

  6. [6]

    Universal transformers

    Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. InIn- ternational Conference on Learning Representations, 2019. 3

    work page 2019

  7. [7]

    Scaling vision transformers to 22 billion pa- rameters

    Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdul- mohsin, et al. Scaling vision transformers to 22 billion pa- rameters. InInternational conference on machine learning, pages 7480–7512. PMLR, 2023. 1

    work page 2023

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5, 9

    work page 2009

  9. [9]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. InInternational Con- ference on Learning Representations, 2021. 1

    work page 2021

  10. [10]

    Adaptive token sampling for efficient vision transformers

    Mohsen Fayyaz, Soroush Abbasi Koohpayegani, Farnoush Rezaei Jafari, Sunando Sengupta, Hamid Reza Vaezi Joze, Eric Sommerlade, Hamed Pirsiavash, and J¨urgen Gall. Adaptive token sampling for efficient vision transformers. InEuropean conference on computer vision, pages 396–414. Springer, 2022. 3

    work page 2022

  11. [11]

    Sim- ple copy-paste is a strong data augmentation method for in- stance segmentation

    Golnaz Ghiasi, Yin Cui, Aravind Srinivas, Rui Qian, Tsung- Yi Lin, Ekin D Cubuk, Quoc V Le, and Barret Zoph. Sim- ple copy-paste is a strong data augmentation method for in- stance segmentation. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 2918–2928, 2021. 6, 11

    work page 2021

  12. [12]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. InProceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 6, 11

    work page 2017

  13. [13]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 1

    work page 2022

  14. [14]

    Vision-mor: Scal- ing vision transformer via patch-level mixture-of-recursions

    Yunhong He, Zhengqing Yuan, Weixiang Sun, Yiyang Li, Yixin Liu, Yanfang Ye, and Lichao Sun. Vision-mor: Scal- ing vision transformer via patch-level mixture-of-recursions. InProceedings of the AAAI Conference on Artificial Intelli- gence, pages 4699–4707, 2026. 1, 3

    work page 2026

  15. [15]

    Augment your batch: Improving generalization through instance repetition

    Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry. Augment your batch: Improving generalization through instance repetition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8129–8138, 2020. 10

    work page 2020

  16. [16]

    Loop- former: Elastic-depth looped transformers for latent reason- ing via shortcut modulation

    Ahmadreza Jeddi, Marco Ciccone, and Babak Taati. Loop- former: Elastic-depth looped transformers for latent reason- ing via shortcut modulation. InInternational Conference on Learning Representations, 2026. 3

    work page 2026

  17. [17]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 1

    work page 2023

  18. [18]

    Similarity of neural network represen- tations revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network represen- tations revisited. InInternational conference on machine learning, pages 3519–3529. PMlR, 2019. 9

    work page 2019

  19. [19]

    Albert: A lite bert for self-supervised learning of language representations

    Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. InInternational Conference on Learning Representations,

    work page

  20. [20]

    Exploring plain vision transformer backbones for object de- tection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. InEuropean conference on computer vision, pages 280–296. Springer, 2022. 1, 6, 11

    work page 2022

  21. [21]

    Edge- recvit: Efficient vision transformer via semantic-refined dy- namic recursion

    YiZhou Li, Jinyi Xu, Mingyu Yin, and Xianyi Zhao. Edge- recvit: Efficient vision transformer via semantic-refined dy- namic recursion. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 12987–12996, 2026. 1, 3, 5

    work page 2026

  22. [22]

    Not all patches are what you need: Expediting vision transformers via token reorganiza- tions

    Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie. Not all patches are what you need: Expediting vision transformers via token reorganiza- tions. InInternational Conference on Learning Representa- tions, 2022. 3

    work page 2022

  23. [23]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 6, 11

    work page 2014

  24. [24]

    Revisiting token pruning for 15 object detection and instance segmentation

    Yifei Liu, Mathias Gehrig, Nico Messikommer, Marco Can- nici, and Davide Scaramuzza. Revisiting token pruning for 15 object detection and instance segmentation. InProceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2658–2668, 2024. 2

    work page 2024

  25. [25]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 11

    work page 2021

  26. [26]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 11976–11986,

    work page

  27. [27]

    Mobilevit: Light- weight, general-purpose, and mobile-friendly vision trans- former

    Sachin Mehta and Mohammad Rastegari. Mobilevit: Light- weight, general-purpose, and mobile-friendly vision trans- former. InInternational Conference on Learning Represen- tations, 2022. 1

    work page 2022

  28. [28]

    When does label smoothing help?Advances in neural in- formation processing systems, 32, 2019

    Rafael M ¨uller, Simon Kornblith, and Geoffrey E Hinton. When does label smoothing help?Advances in neural in- formation processing systems, 32, 2019. 10

    work page 2019

  29. [29]

    Designing network design spaces

    Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Doll ´ar. Designing network design spaces. InProceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, pages 10428–10436,

    work page

  30. [30]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification.Advances in neural information processing systems, 34:13937–13949,

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynamicvit: Efficient vision transformers with dynamic token sparsification.Advances in neural information processing systems, 34:13937–13949,

    work page

  31. [31]

    Sliced recursive transformer

    Zhiqiang Shen, Zechun Liu, and Eric Xing. Sliced recursive transformer. InEuropean Conference on Computer Vision, pages 727–744. Springer, 2022. 1, 3, 5

    work page 2022

  32. [32]

    Training data-efficient image transformers & distillation through at- tention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herv ´e J´egou. Training data-efficient image transformers & distillation through at- tention. InInternational conference on machine learning, pages 10347–10357. PMLR, 2021. 1, 2, 5, 7, 9, 10, 11

    work page 2021

  33. [33]

    Going deeper with im- age transformers

    Hugo Touvron, Matthieu Cord, Alexandre Sablayrolles, Gabriel Synnaeve, and Herv´e J´egou. Going deeper with im- age transformers. InProceedings of the IEEE/CVF interna- tional conference on computer vision, pages 32–42, 2021. 3

    work page 2021

  34. [34]

    Detectron2.https://github

    Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2.https://github. com/facebookresearch/detectron2, 2019. 6

    work page 2019

  35. [35]

    Unified perceptual parsing for scene understand- ing

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understand- ing. InEuropean Conference on Computer Vision. Springer,

    work page

  36. [36]

    Looping back to move forward: Recursive trans- formers for efficient and flexible large multimodal models,

    Ruihan Xu, Yuting Gao, Lan Wang, Jianing Li, Weihao Chen, Qingpei Guo, Ming Yang, and Shiliang Zhang. Loop- ing back to move forward: Recursive transformers for effi- cient and flexible large multimodal models.arXiv preprint arXiv:2602.09080, 2026. 1

    work page arXiv 2026

  37. [37]

    A-vit: Adaptive to- kens for efficient vision transformer

    Hongxu Yin, Arash Vahdat, Jose M Alvarez, Arun Mallya, Jan Kautz, and Pavlo Molchanov. A-vit: Adaptive to- kens for efficient vision transformer. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10809–10818, 2022. 3

    work page 2022

  38. [38]

    Mesh: Memory-as-state-highways for re- cursive transformers

    Chengting Yu, Xiaobo Shu, Yadao Wang, Yizhen Zhang, Haoyi Wu, Jiaang Li, Rujiao Long, Ziheng Chen, Yuchi Xu, Wenbo Su, et al. Mesh: Memory-as-state-highways for re- cursive transformers. InInternational Conference on Learn- ing Representations, 2026. 1, 3, 5

    work page 2026

  39. [39]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. InProceedings of the IEEE/CVF international con- ference on computer vision, pages 6023–6032, 2019. 10

    work page 2019

  40. [40]

    Scaling vision transformers

    Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lu- cas Beyer. Scaling vision transformers. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12104–12113, 2022. 1

    work page 2022

  41. [41]

    mixup: Beyond empirical risk minimiza- tion

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. InInternational Conference on Learning Representa- tions, 2018. 10

    work page 2018

  42. [42]

    Random erasing data augmentation

    Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. InProceed- ings of the AAAI conference on artificial intelligence, pages 13001–13008, 2020. 10

    work page 2020

  43. [43]

    Scene parsing through ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ade20k dataset. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 633–641,

    work page

  44. [44]

    arXiv preprint arXiv:2103.11886 , year =

    Daquan Zhou, Bingyi Kang, Xiaojie Jin, Linjie Yang, Xi- aochen Lian, Zihang Jiang, Qibin Hou, and Jiashi Feng. Deepvit: Towards deeper vision transformer.arXiv preprint arXiv:2103.11886, 2021. 3

    work page arXiv 2021

  45. [45]

    Vision mamba: Efficient visual representation learning with bidirectional state space model

    Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xinggang Wang. Vision mamba: Efficient visual representation learning with bidirectional state space model. InInternational Conference on Machine Learning, pages 62429–62442. PMLR, 2024. 11 16

    work page 2024