Pith. sign in

REVIEW 4 major objections 5 minor 60 references

DeepTraverse: A Depth-First Search Inspired Network for Algorithmic Visual Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DeepTraverse embeds depth-first search into a vision backbone and reports top accuracy per parameter.

desk verdict The ImageNet-1k claim is a mirage: Table 3 compares a 100-class subset against full 1000-class baselines, and the rest of the paper is a known architecture combo with no ablations. read the letter →

arxiv 2506.10084 v1 pith:AHBABNAU submitted 2025-06-11 cs.CV

Bin Guo , John H.L. Hansen This is my paper
classification cs.CV
keywords depth-firstsearchvisionbackboneefficientneuralnetworksrecursiveexplorationchannelrecalibrationimageclassificationparameter-efficientarchitecturealgorithmicpriors
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DeepTraverse, a vision backbone that organizes feature extraction the way depth-first search organizes a tree traversal: it repeatedly applies a shared transformation to deepen the representation along a path, then recalibrates channel importance using global context, and repeats this exploration/backtrack cycle in stacked blocks. The authors' central claim is that this algorithmic prior yields better accuracy per parameter than conventional and efficient backbones, including ResNet, GhostNet, MobileNetV3, ShuffleNetV2, EfficientNet, FasterNet, and StarNet across CIFAR-10, CIFAR-100, ImageNet-64, and a 100-class ImageNet-1k subset. Their headline numbers are 83.16% top-1 and 96.54% top-5 accuracy on that ImageNet subset with 5.04M parameters, 73.84% on CIFAR-100 with 0.26M parameters, and 71.50% on ImageNet-64 with 0.59M parameters. If the claim holds, it would show that structuring a network around a classical search metaphor is a viable efficiency principle rather than a mere analogy.

What carries the argument

The load-bearing object is the DFSBlock, built from two named components. DFS-EB (Exploration Block) performs recursive residual refinement, $F_i = F_{i-1} + \Phi_{\text{recursive}}(F_{i-1})$, with $\Phi_{\text{recursive}}$ sharing parameters across $R$ iterations so extra depth costs almost no extra parameters. DFS-BB (Backtrack Block) is a channel recalibrator: global average pooling yields a channel descriptor $z$, a two-layer bottleneck maps it to a per-channel weight $s = \sigma(W_2 \delta(W_1 z))$, and $F' = F \odot s$ rescales the features. Stacked with residual shortcuts, these blocks instantiate the paper's DFS analogy: iterative deepening during exploration and context-driven reweighting during backtracking.

What would settle it

Train DeepTraverse and its listed baselines on the same full ImageNet-1k training set with the same epochs and resolution, then compare top-1 accuracy on the official 1,000-class validation set; if DeepTraverse no longer outperforms StarNet or FasterNet at comparable parameter counts, the headline accuracy advantage is an artifact of the 100-class subset rather than a property of the architecture.

Watch

Extended reading notes

Core claim

DeepTraverse's core claim is that visual representation learning can be structured as an explicit depth-first search over feature space. The DFS Exploration Block (DFS-EB) starts from a feature map produced by an extraction layer and applies the same recursive block $R$ times, each time adding a learned residual; because the recursive weights are shared, the effective depth grows without proportional parameter growth. The DFS Backtrack Block (DFS-BB) then pools global spatial information, computes a channel-wise attention vector through a bottleneck, and rescales the features before the next block, mimicking a backtracking step that prunes or reinforces what was found. The paper argues that the composition of these two mechanisms produces features that are progressively refined and contextually reweighted, and reports that the resulting network reaches the best accuracy-per-parameter among the models compared on the benchmarks tested.

Load-bearing premise

The headline ImageNet-1k results were measured on a 100-class subset of ImageNet, while the baselines' published numbers are for the full 1,000-class task, so the comparison assumes the subset is no easier than the full benchmark.

Editorial extensions

If this is right

  • Deepening a block by increasing $R$ costs shared-parameter compute rather than new weights, so accuracy can in principle be traded against speed without changing model size.
  • The DFS-BB's channel reweighting gives the architecture an explicit, inspectable locus where global context modifies local features, which supports the paper's interpretability claim.
  • Because the blocks are built from depthwise separable convolutions, batch norm, and pointwise mixing, the design can be dropped into existing training pipelines without new operators.
  • On the reported benchmarks, the per-parameter advantage implies that efficient backbones may have more headroom than their layer-count suggests.
  • Training DeepTraverse at wider scales on CIFAR-100 (82.20% with 14.26M parameters) suggests the DFS structure scales beyond lightweight regimes without a linear parameter penalty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is full ImageNet-1k training; the paper's ImageNet numbers come from a 100-class subset, so the architecture's real-world advantage over baselines is still open.
  • If the recursive-depth effect is real, intermediate stages $F_1,\ldots,F_R$ should show progressively finer or more class-specific features, which could be tested with probing classifiers or attention visualizations at each recursion depth.
  • The parameter-shared recursion makes DeepTraverse a natural candidate for adaptive computation, where the network selects how many recursions to run per input to trade accuracy for latency at inference time.
  • The design invites comparison with other weight-tied or recurrent architectures; if the DFS metaphor carries explanatory weight, breadth-first or best-first search could be operationalized similarly.
  • A concrete interpretability study would measure whether DFS-BB's channel weights align with human-annotated object parts; the paper does not report such a study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes DeepTraverse, a convolutional backbone whose core building block (DFSBlock) combines a recursive 'exploration' module (DFS-EB) with a channel-recalibration 'backtracking' module (DFS-BB), inspired by depth-first search. The authors claim state-of-the-art accuracy-per-parameter on CIFAR-10/100, ImageNet-64, and ImageNet-1K, with a headline 83.16% top-1 accuracy on a 100-class ImageNet-1K subset using 5.04M parameters. The architecture is described in Eqs. (2)-(12), and experiments are reported in Tables 1-3.

Significance. If the performance claims were valid, the paper would present a parameter-efficient architecture with an appealing algorithmic interpretation and a clear potential for mobile and resource-constrained vision. The paper provides no code, no hyperparameter configuration, and no ablations, so the only evidence is the tables. The central performance comparison for ImageNet-1K is invalid because Table 3 mixes a 100-class subset of ImageNet with full 1000-class references, so the headline 'state-of-the-art' result cannot be taken at face value. The contribution of the DFS-inspired components is also not isolated, so the explanatory claim is unsupported as submitted.

major comments (4)
  1. [Table 3, Section 4.3] The headline claim of 83.16% top-1 accuracy on ImageNet-1K is based on a 100-class subset of ImageNet, while every baseline number in Table 3 (ResNet50 78.76, GhostNet 80.34, etc.) is a canonical full 1000-class ImageNet validation accuracy. The text itself acknowledges this mismatch when it remarks that StarNet is 'not as well-suited for this particular 100-class ImageNet subset.' A 100-class subset has a different label space and chance baseline than the full 1000-class benchmark, so any accuracy comparison between the two is invalid. This invalidates the abstract's 'state-of-the-art' claim, the Section 4.3 analysis, and the conclusion's performance summary.
  2. [Section 4, Sections 3.2-3.4] The architecture's key hyperparameters—recursive depth R, bottleneck reduction ratio r, number of stacked DFSBlocks, and channel widths—are never specified in the main text; the text defers to 'supplemental materials' that are not provided. No ablations are reported that remove or vary the recursive parameter sharing (Eqs. (4)-(5)) or the recalibration module (Eqs. (6)-(8)). Without these, the observed accuracy differences cannot be attributed to the DFS-inspired mechanism rather than to depthwise separable convolutions, network width, or the unspecified training recipe, so the central explanatory claim is unsupported.
  3. [Section 4, 'Datasets and Experiment setting'] The training recipe is described only as 'provided by previous work on the same datasets' with no citation to that work and no details on augmentations, optimizer, weight decay, or learning-rate schedule beyond '100 epochs' and 'initial learning rate 0.1.' No code is released. Combined with the missing hyperparameter values, the reported results in Tables 1-3 are not reproducible, which is a load-bearing issue for an empirical architecture paper.
  4. [Tables 1 and 2] Several baseline entries appear inconsistent with standard published configurations (e.g., DenseNet with 0.60M parameters on CIFAR-100, EfficientNet with 0.12 GFLOPs on CIFAR-100 but 0.34 GFLOPs on CIFAR-10 for the same model family). The paper does not state which variant of each baseline was used or how parameter and FLOP counts were measured, making the claimed Pareto dominance in Sections 4.1 and 4.2 difficult to verify.
minor comments (5)
  1. [Section 3.3, Eqs. (6)-(8)] The DFS-BB module is structurally identical to the Squeeze-and-Excitation block (Hu et al., CVPR 2018), including adaptive average pooling, a two-layer bottleneck with reduction ratio r, sigmoid, and channel-wise multiplication; the authors should cite that work and clarify what is new in the 'backtracking' formulation.
  2. [References] Reference [20] is cited for DenseNet, but the cited paper is CondenseNet; the bibliography entry should be corrected to the original DenseNet paper.
  3. [Section 4.2] The term 'ImageNet64' is used without specifying whether it is Tiny ImageNet or a downsampled ImageNet-1k; the experimental setting mentions 'Tiny ImageNet with resolution of 64x64,' so the dataset should be defined consistently across the text and table.
  4. [Figures 1 and 2] The figures contain small, hand-drawn-style annotations; labels such as 'Iter = 1', 'Iter = n', and 'Backtrack Focus' are hard to read and should be enlarged or typeset.
  5. [Section 2, reference [9]] Reference [9] is a specialized paper on using DFS to minimize connected components; a standard algorithms textbook or survey would be a more appropriate citation for the definition of depth-first search.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DeepTraverse's results come from external benchmarks and its architecture is defined explicitly rather than derived from its own outputs.

full rationale

Walking the paper's derivation chain, there is no step in which a prediction is equivalent to its inputs by construction. The methodology defines DFS-EB through Eqs. (2)-(5) as an explicit residual recursion with shared weights, and DFS-BB through Eqs. (6)-(8) as an average-pooling plus two-layer bottleneck recalibration; these are architectural definitions, not fitted quantities relabeled as predictions. Reported accuracies in Tables 1-3 come from training on external datasets and evaluating on held-out test splits, so the empirical claim is independent of the architecture's own description. No load-bearing self-citation appears: references such as [19], [18], and [21] are standard external works, and no uniqueness theorem or prior result by the same authors is invoked to force a design choice. The DFS terminology is a narrative analogy rather than a mathematical constraint, so it cannot make the derivation circular. One correctness concern is flagged for the record: Section 4.3 states DeepTraverse was evaluated on a '100-class ImageNet subset' while the baselines in Table 3 are full ImageNet-1k results, which undermines the comparability of the headline 83.16% figure; this is a benchmark-validity problem, not a circularity, because the number is not constructed from its own inputs. The absence of ablations isolating the DFS components weakens the explanatory claim but does not constitute circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claim depends on unstated architecture hyperparameters (R, r, block/width layout) and on a benchmark comparability assumption that Section 4.3 itself contradicts. No new physical entities are proposed; the DFS metaphor is asserted rather than formalized.

free parameters (3)
  • Recursive depth R in DFS-EB = not reported in main text
    Controls how many times the shared recursive block is applied; no value or ablation is given in the main text.
  • Bottleneck reduction ratio r in DFS-BB = not reported in main text
    Controls capacity of the channel recalibration module (Eq. 7); no value or ablation is given in the main text.
  • Overall depth and width (number of DFSBlocks, channels) = not reported
    The backbone layout determines parameter counts and accuracy; only aggregate parameter/FLOP numbers are reported.
assumptions (4)
  • domain assumption Benchmark comparability: accuracy on a 100-class ImageNet subset is comparable to full 1000-class ImageNet-1k baselines.
    Table 3 compares DeepTraverse on a 100-class subset to published full ImageNet-1k results; Section 4.3 explicitly names the subset.
  • ad hoc to paper The DFS metaphor is a faithful computational model of recursive feature refinement.
    No formal mapping between DFS graph traversal and fixed-depth weight-tied residual refinement is given (Eqs. 4-5).
  • ad hoc to paper Backtracking in DFS corresponds to channel recalibration.
    SE-style global pooling plus reweighting (Eqs. 6-8) is asserted to model backtracking, without analysis.
  • domain assumption The training recipe inherited from previous work is sufficient for fair comparison.
    Section 4 says training follows the recipe provided by previous work without specifying it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepTraverse: A Depth-First Search Inspired Network for Algorithmic Visual Understanding." pith.science (2026). https://pith.science/paper/AHBABNAU

@misc{pith2026250610084,
  author       = {Pith},
  title        = {Pith review of: DeepTraverse: A Depth-First Search Inspired Network for Algorithmic Visual Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHBABNAU}},
  note         = {Machine review of arXiv:2506.10084}
}
read the original abstract

Conventional vision backbones, despite their success, often construct features through a largely uniform cascade of operations, offering limited explicit pathways for adaptive, iterative refinement. This raises a compelling question: can principles from classical search algorithms instill a more algorithmic, structured, and logical processing flow within these networks, leading to representations built through more interpretable, perhaps reasoning-like decision processes? We introduce DeepTraverse, a novel vision architecture directly inspired by algorithmic search strategies, enabling it to learn features through a process of systematic elucidation and adaptive refinement distinct from conventional approaches. DeepTraverse operationalizes this via two key synergistic components: recursive exploration modules that methodically deepen feature analysis along promising representational paths with parameter sharing for efficiency, and adaptive calibration modules that dynamically adjust feature salience based on evolving global context. The resulting algorithmic interplay allows DeepTraverse to intelligently construct and refine feature patterns. Comprehensive evaluations across a diverse suite of image classification benchmarks show that DeepTraverse achieves highly competitive classification accuracy and robust feature discrimination, often outperforming conventional models with similar or larger parameter counts. Our work demonstrates that integrating such algorithmic priors provides a principled and effective strategy for building more efficient, performant, and structured vision backbones.

Figures

Figures reproduced from arXiv: 2506.10084 by the authors.

Figure 1
Figure 1. DeepTraverse reasoning process: Illustrating how the architecture employs depth-first fea [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Structure of DeepTraverse, illustrating the integration of DFS-EB and DFS-BB modules [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 33 canonical work pages

  1. [1]

    Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution,

    Y . Chen, H. Fan, B. Xu, Z. Yan, Y . Kalantidis, M. Rohrbach, Y . Shuicheng, and J. Feng, “Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution,” in2019 IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 3434–3443

    work page 2019

  2. [2]

    Xception: Deep learning with depthwise separable convolutions,

    F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258

    2017

  3. [3]

    Tackling multipath and biased training data for imu-assisted ble proximity detection,

    T. He, J. Tan, W. Zhuo, M. Printz, and S.-H. G. Chan, “Tackling multipath and biased training data for imu-assisted ble proximity detection,” inIEEE INFOCOM 2022-IEEE Conference on Computer Communications. IEEE, 2022, pp. 1259–1268

    work page 2022

  4. [4]

    Micronet: Improving image recognition with extremely low flops,

    Y . Li, Y . Chen, X. Dai, D. Chen, M. Liu, L. Yuan, Z. Liu, L. Zhang, and N. Vasconcelos, “Micronet: Improving image recognition with extremely low flops,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 468–477

    work page 2021

  5. [5]

    Rigid-Motion Scattering for Texture Classification

    L. Sifre and S. Mallat, “Rigid-motion scattering for texture classification,”arXiv preprint arXiv:1403.1687, 2014

    work page Pith review arXiv 2014

  6. [6]

    Hetconv: Heterogeneous kernel-based convolutions for deep cnns,

    P. Singh, V . K. Verma, P. Rai, and V . P. Namboodiri, “Hetconv: Heterogeneous kernel-based convolutions for deep cnns,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 4835–4844

    work page 2019

  7. [7]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Min- derer, G. Heigold, S. Gelly,et al., “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  8. [8]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jegou, “Training data-efficient image transformers & distillation through attention,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 10 347–10 357

    work page 2021

Show all 60 references
  1. [9]

    A depth-first search-based algorithm for the minimization of the largest connected component in networks,

    Y . Aygül, O. Ugurlu, and U. Nuriyev, “A depth-first search-based algorithm for the minimization of the largest connected component in networks,” in2023 5th International Conference on Problems of Cybernetics and Informatics (PCI), 2023, pp. 1–4

    2023

  2. [10]

    Interleaved group convolutions,

    T. Zhang, G.-J. Qi, B. Xiao, and J. Wang, “Interleaved group convolutions,” inProceedings of the IEEE international conference on computer vision, 2017, pp. 4373–4382

    2017

  3. [11]

    Semi- supervised learning with network embedding on ambient rf signals for geofencing services,

    W. Zhuo, K. H. Chiu, J. Chen, J. Tan, E. Sumpena, S.-H. G. Chan, S. Ha, and C.-H. Lee, “Semi- supervised learning with network embedding on ambient rf signals for geofencing services,”arXiv preprint arXiv:2210.07889, 2022

    2022 arXiv

  4. [12]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,”Advances in neural information processing systems, vol. 25, 2012

    2012

  5. [13]

    Searching for mobilenetv3,

    A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y . Zhu, R. Pang, V . Vasudevan, et al., “Searching for mobilenetv3,” inProceedings of the IEEE/CVF international conference on computer vision, 2019, pp. 1314–1324

    2019

  6. [14]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,”arXiv preprint arXiv:1704.04861, 2017

    2017 arXiv

  7. [15]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510–4520

    2018

  8. [16]

    Shufflenet v2: Practical guidelines for efficient cnn architecture design,

    N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 116–131

    2018

  9. [17]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices,

    X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848–6856

    2018

  10. [18]

    Ghostnet: More features from cheap operations,

    K. Han, Y . Wang, Q. Tian, J. Guo, C. Xu, and C. Xu, “Ghostnet: More features from cheap operations,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 1580–1589. 10

    2020

  11. [19]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778

    2016

  12. [20]

    Condensenet: An efficient densenet using learned group convolutions,

    G. Huang, S. Liu, L. Van der Maaten, and K. Q. Weinberger, “Condensenet: An efficient densenet using learned group convolutions,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 2752–2761

    2018

  13. [21]

    Efficientnetv2: Smaller models and faster training,

    M. Tan and Q. Le, “Efficientnetv2: Smaller models and faster training,” inInternational conference on machine learning. PMLR, 2021, pp. 10 096–10 106

    2021

  14. [22]

    Run, don’t walk: Chasing higher FLOPS for faster neural networks,

    J. Chen, S.-h. Kao, H. He, W. Zhuo, S. Wen, C.-H. Lee, and S.-H. G. Chan, “Run, don’t walk: Chasing higher FLOPS for faster neural networks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023, pp. 12 021–12 031

    2023

  15. [23]

    Rewrite the stars,

    X. Ma, X. Dai, Y . Bai, Y . Wang, and Y . Fu, “Rewrite the stars,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, pp. 5694–5703

    2024

  16. [24]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  17. [25]

    Social ode: Multi-agent trajectory forecasting with neural ordinary differential equations,

    S. Wen, H. Wang, and D. Metaxas, “Social ode: Multi-agent trajectory forecasting with neural ordinary differential equations,” inComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII. Springer, 2022, pp. 217–233

    2022

  18. [26]

    How to train your vit? data, augmentation, and regularization in vision transformers,

    A. Steiner, A. Kolesnikov, X. Zhai, R. Wightman, J. Uszkoreit, and L. Beyer, “How to train your vit? data, augmentation, and regularization in vision transformers,”arXiv preprint arXiv:2106.10270, 2021

    2021 arXiv

  19. [27]

    Deit iii: Revenge of the vit,

    H. Touvron, M. Cord, and H. Jegou, “Deit iii: Revenge of the vit,”arXiv preprint arXiv:2204.07118, 2022

    2022 arXiv

  20. [28]

    Levit: a vision transformer in convnet’s clothing for faster inference,

    B. Graham, A. El-Nouby, H. Touvron, P. Stock, A. Joulin, H. Jégou, and M. Douze, “Levit: a vision transformer in convnet’s clothing for faster inference,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 12 259–12 269

    2021

  21. [29]

    Swin transformer v2: Scaling up capacity and resolution,

    Z. Liu, H. Hu, Y . Lin, Z. Yao, Z. Xie, Y . Wei, J. Ning, Y . Cao, Z. Zhang, L. Dong,et al., “Swin transformer v2: Scaling up capacity and resolution,”Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12 009–12 019, 2022

    2022

  22. [30]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 10 012–10 022

    2021

  23. [31]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” inProceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 568–578

    2021

  24. [32]

    A tree-based structure-aware transformer decoder for image-to-markup generation,

    S. Zhong, S. Song, G. Li, and S.-H. G. Chan, “A tree-based structure-aware transformer decoder for image-to-markup generation,” inProceedings of the 30th ACM International Conference on Multimedia, 2022, pp. 5751–5760

    2022

  25. [33]

    Xcit: Cross-covariance image transformers,

    A. Ali, H. Touvron, M. Caron, P. Bojanowski, M. Douze, A. Joulin, I. Laptev, N. Neverova, G. Synnaeve, J. Verbeek,et al., “Xcit: Cross-covariance image transformers,”Advances in neural information processing systems, vol. 34, pp. 20 014–20 027, 2021

    2021

  26. [34]

    Lightvit: Towards light-weight convolution-free vision transformers,

    T. Huang, L. Huang, S. You, F. Wang, C. Qian, and C. Xu, “Lightvit: Towards light-weight convolution-free vision transformers,”arXiv preprint arXiv:2207.05557, 2022

    2022 arXiv

  27. [35]

    Soft: Softmax-free transformer with linear complexity,

    J. Lu, J. Yao, J. Zhang, X. Zhu, H. Xu, W. Gao, C. Xu, T. Xiang, and L. Zhang, “Soft: Softmax-free transformer with linear complexity,”Advances in Neural Information Processing Systems, vol. 34, pp. 21 297–21 309, 2021

    2021

  28. [36]

    Quadtree attention for vision transformers,

    S. Tang, J. Zhang, S. Zhu, and P. Tan, “Quadtree attention for vision transformers,”arXiv preprint arXiv:2201.02767, 2022

    2022 arXiv

  29. [37]

    Scaling local self-attention for parameter efficient visual backbones,

    A. Vaswani, P. Ramachandran, A. Srinivas, N. Parmar, B. Hechtman, and J. Shlens, “Scaling local self-attention for parameter efficient visual backbones,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 12 894–12 904

    2021

  30. [38]

    Mobileformer: Bridging mobilenet and transformer,

    Y . Chen, X. Dai, D. Chen, M. Liu, X. Dong, L. Yuan, and Z. Liu, “Mobileformer: Bridging mobilenet and transformer,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5270–5279. 11

    2022

  31. [39]

    Coatnet: Marrying convolution and attention for all data sizes,

    Z. Dai, H. Liu, Q. V . Le, and M. Tan, “Coatnet: Marrying convolution and attention for all data sizes,” Advances in Neural Information Processing Systems, vol. 34, pp. 3965–3977, 2021

    2021

  32. [40]

    Bottleneck transformers for visual recognition,

    A. Srinivas, T.-Y . Lin, N. Parmar, J. Shlens, P. Abbeel, and A. Vaswani, “Bottleneck transformers for visual recognition,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16 519–16 529

    2021

  33. [41]

    Efficientvit: Enhanced linear attention for high-resolution low-computation visual recognition,

    H. Cai, C. Gan, and S. Han, “Efficientvit: Enhanced linear attention for high-resolution low-computation visual recognition,”arXiv preprint arXiv:2205.14756, 2022

    2022 arXiv

  34. [42]

    Efficientformer: Vision transformers at mobilenet speed,

    Y . Li, G. Yuan, Y . Wen, E. Hu, G. Evangelidis, S. Tulyakov, Y . Wang, and J. Ren, “Efficientformer: Vision transformers at mobilenet speed,”arXiv preprint arXiv:2206.01191, 2022

    2022 arXiv

  35. [43]

    Separable self-attention for mobile vision transformers,

    S. Mehta and M. Rastegari, “Separable self-attention for mobile vision transformers,”arXiv preprint arXiv:2206.02680, 2022

    2022 arXiv

  36. [44]

    Edgevits: Com- peting light-weight cnns on mobile devices with vision transformers,

    J. Pan, A. Bulat, F. Tan, X. Zhu, L. Dudziak, H. Li, G. Tzimiropoulos, and B. Martinez, “Edgevits: Com- peting light-weight cnns on mobile devices with vision transformers,”arXiv preprint arXiv:2205.03436, pp. 1–6, 2022

    2022 arXiv

  37. [45]

    Cyclemlp: A mlp-like architecture for dense prediction,

    S. Chen, E. Xie, C. Ge, D. Liang, and P. Luo, “Cyclemlp: A mlp-like architecture for dense prediction,” arXiv preprint arXiv:2107.10224, 2021

    2021 arXiv

  38. [46]

    As-mlp: An axial shifted mlp architecture for vision,

    D. Lian, Z. Yu, X. Sun, and S. Gao, “As-mlp: An axial shifted mlp architecture for vision,”arXiv preprint arXiv:2107.08391, 2021

    2021 arXiv

  39. [47]

    Mlp-mixer: An all-mlp architecture for vision,

    I. O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit,et al., “Mlp-mixer: An all-mlp architecture for vision,”Advances in Neural Information Processing Systems, vol. 34, pp. 24 261–24 272, 2021

    2021

  40. [48]

    Are we ready for a new paradigm shift? a survey on visual deep mlp,

    R. Liu, Y . Li, L. Tao, D. Liang, and H.-T. Zheng, “Are we ready for a new paradigm shift? a survey on visual deep mlp,”Patterns, vol. 3, no. 7, p. 100520, 2022

    2022

  41. [49]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 11 976– 11 986

    2022

  42. [50]

    When shift operation meets vision transformer: An extremely simple alternative to attention mechanism,

    G. Wang, Y . Zhao, C. Tang, C. Luo, and W. Zeng, “When shift operation meets vision transformer: An extremely simple alternative to attention mechanism,”arXiv preprint arXiv:2201.10801, 2022

    2022 arXiv

  43. [51]

    Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer,

    S. Mehta and M. Rastegari, “Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer,”arXiv preprint arXiv:2110.02178, 2021

    2021 arXiv

  44. [52]

    Relational inductive biases, deep learning, and graph networks,

    P. W. Battaglia, J. B. Hamrick, V . Bapst, A. Sanchez-Gonzalez, V . Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner,et al., “Relational inductive biases, deep learning, and graph networks,” arXiv preprint arXiv:1806.01261, 2018

    2018 arXiv

  45. [53]

    Learning fast approximations of sparse coding,

    K. Gregor and Y . LeCun, “Learning fast approximations of sparse coding,” inProceedings of the 27th international conference on international conference on machine learning, 2010, pp. 399–406

    2010

  46. [54]

    Neural execution of graph algorithms,

    P. Veliˇckovi´c, R. Ying, M. Padovano, R. Hadsell, and C. Blundell, “Neural execution of graph algorithms,” inInternational Conference on Learning Representations, 2019

    2019

  47. [55]

    Compositional attention networks for machine reasoning,

    D. A. Hudson and C. D. Manning, “Compositional attention networks for machine reasoning,” inInterna- tional Conference on Learning Representations, 2018

    2018

  48. [56]

    Chain of thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,”arXiv preprint arXiv:2201.11903, 2022

    2022 arXiv

  49. [57]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y . Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,”arXiv preprint arXiv:2305.10601, 2023

    2023 arXiv

  50. [58]

    This looks like that: deep learning for interpretable image recognition,

    C. Chen, O. Li, D. Tao, A. Barnett, C. Rudin, and J. K. Su, “This looks like that: deep learning for interpretable image recognition,” inAdvances in neural information processing systems, vol. 32, 2019

    2019

  51. [59]

    On the performance of the depth first search algorithm in supercritical random graphs,

    S. Diskin and M. Krivelevich, “On the performance of the depth first search algorithm in supercritical random graphs,” 2022. [Online]. Available: https://arxiv.org/abs/2111.07345

    2022 arXiv

  52. [60]

    Wide residual networks,

    S. Zagoruyko and N. Komodakis, “Wide residual networks,” inProceedings of the British Machine Vision Conference (BMVC). BMV A Press, 2016, pp. 87.1–87.12. [Online]. Available: https://dx.doi.org/10.5244/C.30.87 12

    2016 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.