Pith. sign in

REVIEW 4 major objections 7 minor 66 references

AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AnchorFormer shows that a vision transformer can keep global self-attention while cutting compute from quadratic to near-linear in the number of image patches, by routing attention through a small set of differentiable anchor tokens.

desk verdict Known Nyström-style low-rank attention, re-packaged as a Markov process, with implausibly large accuracy gains and no code or closest-baseline comparisons. read the letter →

arxiv 2505.16463 v3 pith:TDQ3FTOI submitted 2025-05-22 cs.CV cs.LG

Jiquan Shan , Junxiao Wang , Lifeng Zhao , Liang Cai , Hongyuan Zhang , Ioannis Liritzis This is my paper
classification cs.CVcs.LG
keywords visiontransformerefficientattentionanchortokensbipartiteMarkovprocesslinearcomplexityimageclassificationsemanticsegmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AnchorFormer is a vision transformer designed to keep the global attention of a standard ViT while escaping its quadratic cost. The idea is to route attention through a small set of $m$ anchor tokens: build a bipartite attention matrix between tokens and anchors, then reconstruct the full token-to-token attention as the one-step Markov transition through the anchors. The resulting attention $H = A\Delta^{-1}A^T V$ is never materialized as an $n \times n$ matrix; rearranging the multiplication order brings complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$. The authors report 86.62% top-1 accuracy on ImageNet at 9.2 GFLOPs and 45.75 mAP on COCO at 2.8 GFLOPs, alongside improvements on semantic segmentation. If correct, this gives a drop-in attention module that scales to high-resolution images and small patch sizes without giving up a global receptive field.

What carries the argument

The central object is the differentiable anchor-attention block: a small set of anchor tokens, implemented as neuron weights, that act as a learned bottleneck for global attention. It computes a bipartite token-anchor attention matrix $A$, builds the Markov transition matrix $F = D^{-1}G$ on the token-anchor graph, and reads off the approximate global attention $S_t = A\Delta^{-1}A^T$ from $F^2$. The identity carrying the argument is $H = A\Delta^{-1}A^T V$, evaluated as $A(\Delta^{-1}(A^T V))$ so that complexity stays at $\mathcal{O}(mn)$ instead of $\mathcal{O}(n^2)$. The Markov property supplies the probabilistic interpretation: $p(v_j|v_i) = \sum_{l=1}^m p(v_j|u_l)p(u_l|v_i)$, meaning the anchors summarize the full attention distribution.

What would settle it

Take images where informative content is spread across the whole frame, such as dense small-object scenes, and compare a trained AnchorFormer against the same model with full softmax attention: if the gap between $A\Delta^{-1}A^T$ and the exact attention matrix is large while accuracy holds, the approximation's sufficiency would be questioned; if accuracy drops sharply whenever the exact attention matrix has high rank, the linear-complexity claim would be refuted for those inputs.

Watch

Extended reading notes

Core claim

AnchorFormer's central claim is that global self-attention can be approximated accurately by a two-step path through $m$ learnable anchors: tokens attend to anchors, and anchors attend back to tokens. Writing $A \in \mathbb{R}^{n \times m}$ for the token-anchor attention matrix and $\Delta$ for the diagonal matrix of anchor degrees, the approximate token-to-token attention is $S_t = A\Delta^{-1}A^T$, normalized so its rows sum to one, and the output is $H = A\Delta^{-1}A^T V$. The anchors are represented as rows of a learnable weight matrix in a neural layer, so they are fitted by gradient descent rather than chosen by a separate non-differentiable selection step. Evaluating the product as $A(\Delta^{-1}(A^T V))$ avoids ever forming an $n \times n$ matrix, giving $\mathcal{O}(mn)$ complexity. The paper reports that this construction matches or surpasses efficient-attention baselines on classification, detection, and segmentation while cutting FLOPs.

Load-bearing premise

The load-bearing premise is that a small set of anchor tokens is a sufficient summary of the global attention distribution, so the one-step Markov transition through the anchors loses no information the downstream task needs; the paper assumes this rather than proving it and gives no error bound for the approximation.

Editorial extensions

If this is right

  • Global self-attention can be computed in $\mathcal{O}(mn)$ without forming an explicit $n \times n$ attention matrix, so ViTs can be applied to higher-resolution images or smaller patch sizes at roughly linear cost in the number of patches.
  • Because the anchors are learned by gradient descent, the model concentrates computation on informative regions without a separate region-selection step, which the paper argues is why it outperforms sparse-attention baselines such as PVT.
  • The approximation can be dropped into existing backbones: the paper demonstrates gains on DeiT and LeViT for ImageNet classification, on ESNet for COCO detection, and on Semantic-FPN and UperNet for ADE20K segmentation.
  • Anchor count is a real hyperparameter with a sweet spot near 30 anchors; too few anchors discard features, while too many add redundant information, according to the ablations.
  • The construction is independent per attention head, so the efficiency gain carries over to multi-head self-attention and hence to whole transformer stacks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: since $A\Delta^{-1}A^T$ has rank at most $m$, the approximation's ceiling is the effective rank of the true attention matrix; measuring that rank on real images would predict where the anchor bottleneck starts to hurt.
  • Editorial inference: the paper states AnchorFormer is incompatible with causal attention, but a masked variant of $A$ that forbids future tokens would likely restore causality at the same $\mathcal{O}(mn)$ cost, opening an unstated path to language-model use.
  • Editorial inference: the paper's own framing of anchor attention as a noisy approximation suggests deliberately calibrating that noise, and a testable extension would compare deterministic anchors against anchors trained to inject controlled beneficial noise.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper introduces AnchorFormer, a vision transformer variant that replaces global self-attention with an anchor-based attention computed as H = A Δ^{-1} A^T V (Eq. 11), where A is an n×m token-anchor attention matrix. The complexity is reduced from O(n^2) to O(mn) by reordering multiplications as B^T V → Δ^{-1}(B^T V) → B(Δ^{-1}B^T V). Anchors are represented as learnable neurons in a neural layer, making the selection differentiable. The authors evaluate AnchorFormer on ImageNet classification, COCO detection, and ADE20K segmentation, reporting accuracy/flops trade-offs that are often better than the listed baselines. The paper also claims that the Markov process derivation justifies that Eq. (11) accurately approximates global self-attention.

Significance. If the method delivers the reported trade-offs, it would be a practically useful efficient attention module, particularly for high-resolution vision tasks. The differentiable anchor formulation and the O(mn) complexity are appealing, and the ablation study (§4.6) shows that the differentiable anchor mechanism contributes substantially to the gains. However, the central theoretical claim—that Eq. (11) 'accurately' approximates global self-attention via a Markov process—is not supported by the derivation, which is a low-rank (Nyström-style) bottleneck rather than an exact reduction. The empirical sections are extensive but do not include comparisons to established linear/low-rank attention methods, and some headline numbers in the abstract are computed under non-comparable FLOPs budgets. The significance is therefore conditional on reframing the contribution as a learned low-rank attention with strong empirical results, rather than as a proved approximation of global attention.

major comments (4)
  1. [§3.2, Eq. (8)–(11)] The derivation of Eq. (11) relies on the assumption p(v_j|v_i) = Σ_l p(v_j|u_l)p(u_l|v_i), i.e., that the anchor u_l makes token pairs conditionally independent. This is a modeling choice, not a consequence of the softmax in Eq. (2) or of any data property. No bound is provided for ||A Δ^{-1} A^T V − softmax(QK^T/√d)V||, and for arbitrary attention matrices the relative error can be O(1). The paper should either supply such a bound or explicitly state that AnchorFormer is a learned low-rank approximation of attention, not an 'accurate' global self-attention as claimed in the abstract and Section 3.2.
  2. [Abstract and §4.3–4.4] The headline gains are not consistently computed under comparable conditions. The abstract's '81.3% higher mAP' corresponds to comparing AnchorFormer-B (2.8 GFLOPs) with YOLOX-Nano (0.54 GFLOPs), which are not comparable FLOPs; under comparable FLOPs, e.g., EfficientDet-512 at 2.5 GFLOPs versus AnchorFormer-B, the gain is about 35%. Similarly, the '9.0% higher accuracy' appears to come from the apple-to-apple DeiT replacement in Fig. 5a, not from the cross-architecture comparisons in Table 1. The authors should state the exact comparison pairs and compute percentage gains under matched FLOPs budgets.
  3. [§4.3, Table 1] The text states 'BiFormer-B takes 15.5G FLOPs to achieve 84.42% accuracy', but Table 1 lists BiFormer-B at 9.8 GFLOPs. The claimed FLOPs reduction of 40.6% for AnchorFormer-B versus BiFormer-B is therefore incorrect; the actual reduction is about 6.1% (9.2 vs. 9.8 GFLOPs). This quantitative claim needs to be corrected, and the comparison should be re-evaluated accordingly.
  4. [§4.3–4.5, Tables 1–3] The empirical evaluation does not include established linear/low-rank attention methods such as Nyströmformer, Perceiver, or Linformer. Since Eq. (11) is structurally a low-rank approximation, comparisons to these methods are needed to determine whether the anchor mechanism offers advantages over a generic learnable low-rank projection, or whether the reported gains are due to other components of the architecture. Without such baselines, the claim of superiority over 'current baselines' is overstated.
minor comments (7)
  1. [§3.2, Eq. (2)] The softmax indices appear to be transposed: in vanilla self-attention one normally has p(v_j|v_i) = exp(q_i k_j^T / √d) / Σ_j exp(q_i k_j^T / √d). Please clarify the notation.
  2. [§3, first paragraph] The sentence 'where m is the number of tokens' should read 'where m is the number of anchors'.
  3. [§3.2, Eq. (4)] The equality p(u_j|v_i) = p(v_i|u_j) is not generally true for conditional probabilities. While the algebra leading to Eq. (11) uses the row-stochastic matrices rather than this equality, the statement is misleading and should be rephrased or removed.
  4. [§3.2, Eq. (5)] The closed-form expression for u treats p(u|v_i) as a constant when differentiating the objective in Eq. (3), but p(u|v_i) depends on u through Eq. (4). This is acceptable as a motivation, but the paper should note that Eq. (5) is only an approximate stationary point, since the actual anchors are learned via gradient descent.
  5. [§4.6, Table 5] The table caption and the text say 'DeiT-T and DeiT-T are employed', but the rows clearly refer to DeiT-T and DeiT-S. Please correct the caption.
  6. [§4.5, Table 3] The dataset is referred to as 'ADE210K'; the correct name is ADE20K.
  7. [§5, Conclusion] The references [61]–[66] on positive-incentive noise appear unrelated to the main contribution of this paper; if they are not directly used in the argument, please remove them or integrate them into the related work with a clear connection.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the anchor attention formula is a low-rank modeling choice trained and evaluated on held-out data, not a fitted prediction or a self-citation-derived result.

full rationale

The derivation is self-contained. Eq. (8) is an explicit Markov/independence assumption that defines the anchor-mediated approximation; it is not derived from, nor equivalent to, the vanilla softmax of Eq. (2), and the paper does not rename a fitted quantity as a prediction. Eqs. (9)-(11) follow by matrix algebra from this assumption, so the final O(mn) attention H = A Δ^{-1} A^T V is a genuine low-rank construction rather than a circular restatement of its inputs. Anchors are trained by gradient descent on the downstream task and accuracy is measured on held-out ImageNet/COCO/ADE20K data, so no fitted parameter is masquerading as a predicted result. The self-citations [12, 13, 61-66] appear only as motivation or future-work pointers, not as load-bearing justification, and no uniqueness theorem is imported. The lack of an error bound for Eq. (8) and the absence of Nyströmformer/Perceiver comparisons are validity and novelty concerns, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the two-step Markov approximation and on the hand-chosen anchor count m. No new physical entities are introduced; anchors are learnable vectors. The Markov matrix F is a standard construction from A and Δ.

free parameters (1)
  • Anchor count m = 30
    Chosen by hand from ablation on {10,30,50,100} in Section 4.6 Table 5; the approximation quality and O(mn) complexity depend on m.
assumptions (4)
  • domain assumption Global self-attention can be expressed as a two-step Markov transition through anchors: p(v_j|v_i) = sum_l p(v_j|u_l) p(u_l|v_i).
    Eq. (8) in Section 3.2 is the core approximation and is not derived from the softmax structure of vanilla attention.
  • domain assumption The pivotal information in an input image is concentrated in a few randomly located regions, so a small anchor set can represent informative tokens.
    Stated in Section 1 and Section 3.1; it motivates why m << n anchors suffice.
  • domain assumption Anchors can be learned by gradient descent when represented as neural-layer weights w_j^S with softmax normalization.
    Section 3.3 Eq. (13); assumes the softmax over learnable anchor vectors is differentiable and optimizable to a useful distribution.
  • domain assumption The diagonal matrix Δ is invertible, i.e., every anchor has at least one token assigned so column sums of A are positive.
    Eq. (7) uses Δ^{-1}; if an anchor column of A sums to zero, the transition matrix is undefined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer." pith.science (2026). https://pith.science/paper/TDQ3FTOI

@misc{pith2026250516463,
  author       = {Pith},
  title        = {Pith review of: AnchorFormer: Differentiable Anchor Attention for Efficient Vision Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TDQ3FTOI}},
  note         = {Machine review of arXiv:2505.16463}
}
abstract

Recently, vision transformers (ViTs) have achieved excellent performance on vision tasks by measuring the global self-attention among the image patches. Given $n$ patches, they will have quadratic complexity such as $\mathcal{O}(n^2)$ and the time cost is high when splitting the input image with a small granularity. Meanwhile, the pivotal information is often randomly gathered in a few regions of an input image, some tokens may not be helpful for the downstream tasks. To handle this problem, we introduce an anchor-based efficient vision transformer (AnchorFormer), which employs the anchor tokens to learn the pivotal information and accelerate the inference. Firstly, by estimating the bipartite attention between the anchors and tokens, the complexity will be reduced from $\mathcal{O}(n^2)$ to $\mathcal{O}(mn)$, where $m$ is an anchor number and $m < n$. Notably, by representing the anchors with the neurons in a neural layer, we can differentiably learn these anchors and approximate global self-attention through the Markov process. It avoids the burden caused by non-differentiable operations and further speeds up the approximate attention. Moreover, we extend the proposed model to three downstream tasks including classification, detection, and segmentation. Extensive experiments show the effectiveness of our AnchorFormer, e.g., achieving up to a 9.0% higher accuracy or 46.7% FLOPs reduction on ImageNet classification, 81.3% higher mAP on COCO detection under comparable FLOPs, as compared to the current baselines.

Figures

Figures reproduced from arXiv: 2505.16463 by the authors.

Figure 1
Figure 1. Comparison of the proposed and other efficient attention strategies. Fig. 1a is the vanilla self-attention in ViTs. Fig. 1b is the sparse-based atten￾tion which mainly preserves the specific queries, keys, and values. Fig. 1c is the window attention which calculates the local attention within the windows. Fig. 1d is the proposed model which focuses on highly informative regions and differentiable learning the pivota… view at source ↗
Figure 2
Figure 2. The proposed AnchorFormer and all baselines. Fig. [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The architecture of the proposed AnchorFormer. AnchorFormer has [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Visual explanations generated by different models on the ImageNet validation dataset [34]. From top to down: input image, visual explanation maps of the Swin-B [3], ViT-B, DeiT-B [37] and AnchorFormer-S, respectively. (a) Comparison on DeiT (b) Comparison on LeViT [PI…
Figure 5
Figure 5. Figure 5: AnchorFormer vs. baseline on ImageNet. Fig. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 54 canonical work pages

  1. [1]

    Vaswani, Attention is all you need, arXiv preprint arXiv:1706.03762 (2017)

    A. Vaswani, Attention is all you need, arXiv preprint arXiv:1706.03762 (2017). 1

    arXiv 2017

  2. [2]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Un- terthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al., An im- age is worth 16x16 words: Transformers for image recognition at scale, arXiv preprint arXiv:2010.11929 (2020). 1

    arXiv 2020

  3. [3]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the IEEE/CVF international conference on computer vi- sion, 2021, pp. 10012–10022. 1, 2, 3, 4, 5

    work page 2021

  4. [4]

    J. Wang, H. Zhang, Y . Yuan, Adv-cpg: A customized portrait genera- tion framework with facial adversarial attacks, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,

    work page

  5. [5]

    Z. Jiao, H. Zhang, X. Li, Cnn2gnn: How to bridge cnn with gnn, arXiv preprint arXiv:2404.14822 (2024). 1

    work page Pith review arXiv 2024

  6. [6]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision, in: Proceedings of the 38th International Conference on Machine Learn- ing, ICML 2021, 18-24 July 2021, Virtual Event, V ol. 139 of Proceedings of Mac...

    work page 2021

  7. [7]

    X. Li, H. Zhang, R. Wang, F. Nie, Multiview clustering: A scalable and parameter-free bipartite graph fusion method, IEEE transactions on pat- tern analysis and machine intelligence 44 (1) (2020) 330–344. 1

    work page 2020

  8. [8]

    X. Pan, T. Ye, Z. Xia, S. Song, G. Huang, Slide-transformer: Hierarchi- cal vision transformer with local self-attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 2082–2091. 1, 4

    work page 2023

Show all 66 references
  1. [9]

    L. Zhu, X. Wang, Z. Ke, W. Zhang, R. W. Lau, Biformer: Vision trans- former with bi-level routing attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 10323– 10333. 1, 4

    2023

  2. [10]

    W. Wang, E. Xie, X. Li, D.-P. Fan, K. Song, D. Liang, T. Lu, P. Luo, L. Shao, Pyramid vision transformer: A versatile backbone for dense pre- diction without convolutions, in: Proceedings of the IEEE/CVF interna- tional conference on computer vision, 2021, pp. 568–578. 2, 3, 4

    2021

  3. [11]

    X. Dong, J. Bao, D. Chen, W. Zhang, N. Yu, L. Yuan, D. Chen, B. Guo, Cswin transformer: A general vision transformer backbone with cross- shaped windows, in: Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 2022, pp. 12124–12134. 2, 3, 4

    2022

  4. [12]

    Zhang, J

    H. Zhang, J. Shi, R. Zhang, X. Li, Non-graph data clustering via O(n) bipartite graph convolution, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (7) (2023) 8729–8742.doi:10.1109/TPAMI. 2022.3231470. 2, 3

    2023

  5. [13]

    Zhang, Y

    H. Zhang, Y . Zhu, X. Li, Decouple graph neural networks: Train multiple simple gnns simultaneously instead of one, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024). 2

    2024

  6. [14]

    Redmon, A

    J. Redmon, A. Farhadi, Yolov3: An incremental improvement, arXiv preprint arXiv:1804.02767 (2018). 2

    2018 arXiv

  7. [15]

    Michel, O

    P. Michel, O. Levy, G. Neubig, Are sixteen heads really better than one?, Advances in neural information processing systems 32 (2019). 2

    2019

  8. [16]

    A. Fan, E. Grave, A. Joulin, Reducing transformer depth on demand with structured dropout, arXiv preprint arXiv:1909.11556 (2019). 2

    2019 arXiv

  9. [17]

    L. Hou, Z. Huang, L. Shang, X. Jiang, X. Chen, Q. Liu, Dynabert: Dy- namic bert with adaptive width and depth, Advances in Neural Informa- tion Processing Systems 33 (2020) 9782–9793. 2

    2020

  10. [18]

    Z. Wang, J. Wohlwend, T. Lei, Structured pruning of large language mod- els, arXiv preprint arXiv:1910.04732 (2019). 2

    2019 arXiv

  11. [19]

    Z. Wang, H. Nie, W. Zheng, Y . Wang, X. Li, A novel tensor learning model for joint relational triplet extraction, IEEE Transactions on Cyber- netics 54 (4) (2023) 2483–2494. 2

    2023

  12. [20]

    D. Hou, C. Gao, Z. Wang, X. Li, Fgssi: a feature-enhanced framework with transferability for sequential source identification, IEEE Transac- tions on Dependable and Secure Computing (2025). 2

    2025

  13. [21]

    Cheng, P

    L. Cheng, P. Zhu, K. Tang, C. Gao, Z. Wang, Gin-sd: source detection in graphs with incomplete nodes via positional encoding and attentive fu- sion, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 38, 2024, pp. 55–63. 2

    2024

  14. [22]

    Mukherjee, A

    S. Mukherjee, A. Awadallah, Xtremedistil: Multi-stage distillation for massive multilingual models, arXiv preprint arXiv:2004.05686 (2020). 3

    2020 arXiv

  15. [23]

    D. Jia, K. Han, Y . Wang, Y . Tang, J. Guo, C. Zhang, D. Tao, Ef- ficient vision transformers via fine-grained manifold distillation. corr abs/2107.01378 (2021). 3

    2021 arXiv

  16. [24]

    Bhandare, V

    A. Bhandare, V . Sripathi, D. Karkada, V . Menon, S. Choi, K. Datta, V . Saletore, Efficient 8-bit quantization of transformer neural machine language translation model, arXiv preprint arXiv:1906.00532 (2019). 3

    2019 arXiv

  17. [25]

    Z. Liu, Y . Wang, K. Han, W. Zhang, S. Ma, W. Gao, Post-training quanti- zation for vision transformer, Advances in Neural Information Processing 7 Systems 34 (2021) 28092–28103. 3

    2021

  18. [26]

    Shridhar, H

    K. Shridhar, H. Jain, A. Agarwal, D. Kleyko, End to end binarized neural networks for text classification, in: Proceedings of SustaiNLP: Workshop on Simple and Efficient Natural Language Processing, 2020, pp. 29–34. 3

    2020

  19. [27]

    Prato, E

    G. Prato, E. Charlaix, M. Rezagholizadeh, Fully quantized transformer for machine translation, arXiv preprint arXiv:1910.10485 (2019). 3

    2019 arXiv

  20. [28]

    C. Gao, H. Liu, J. Huang, Z. Wang, X. Li, X. Li, Regularized spatial– temporal graph convolutional networks for metro passenger flow predic- tion, IEEE Transactions on Intelligent Transportation Systems (2024). 3

    2024

  21. [29]

    Y . Guo, Y . Zheng, M. Tan, Q. Chen, J. Chen, P. Zhao, J. Huang, Nat: Neural architecture transformer for accurate and compact architectures, Advances in Neural Information Processing Systems 32 (2019). 3

    2019

  22. [30]

    D. So, Q. Le, C. Liang, The evolved transformer, in: International confer- ence on machine learning, PMLR, 2019, pp. 5877–5886. 3

    2019

  23. [31]

    D. A. Spielman, S.-H. Teng, Spectral sparsification of graphs, SIAM Jour- nal on Computing 40 (4) (2011) 981–1025. 3

    2011

  24. [32]

    Zaheer, G

    M. Zaheer, G. Guruganesh, K. A. Dubey, J. Ainslie, C. Alberti, S. On- tanon, P. Pham, A. Ravula, Q. Wang, L. Yang, et al., Big bird: Trans- formers for longer sequences, Advances in neural information processing systems 33 (2020) 17283–17297. 3

    2020

  25. [33]

    H. You, Y . Xiong, X. Dai, B. Wu, P. Zhang, H. Fan, P. Vajda, Y . C. Lin, Castling-vit: Compressing self-attention via switching towards linear- angular attention at vision transformer inference, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni...

    2023

  26. [34]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255. 4, 5

    2009

  27. [35]

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ar, C. L. Zitnick, Microsoft coco: Common objects in context, in: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, Springer, 2014, ...

    2014

  28. [36]

    B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, A. Torralba, Scene parsing through ade20k dataset, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 633–641. 4

    2017

  29. [37]

    Touvron, M

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, H. J ´egou, Training data-efficient image transformers & distillation through atten- tion, in: International conference on machine learning, PMLR, 2021, pp. 10347–10357. 4, 5

    2021

  30. [38]

    M. Tan, Q. Le, Efficientnet: Rethinking model scaling for convolu- tional neural networks, in: International conference on machine learning, PMLR, 2019, pp. 6105–6114. 4

    2019

  31. [39]

    Graham, A

    B. Graham, A. El-Nouby, H. Touvron, P. Stock, A. Joulin, H. J ´egou, M. Douze, Levit: a vision transformer in convnet’s clothing for faster inference, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 12259–12269. 4

    2021

  32. [40]

    M. Chen, H. Peng, J. Fu, H. Ling, Autoformer: Searching transformers for visual recognition, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 12270–12280. 4

    2021

  33. [41]

    Li, C.-Y

    Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, C. Feichten- hofer, Mvitv2: Improved multiscale vision transformers for classification and detection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4804–4814. 4

    2022

  34. [42]

    C. Wei, B. Duke, R. Jiang, P. Aarabi, G. W. Taylor, F. Shkurti, Spar- sifiner: Learning sparse instance-dependent attention for efficient vision transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 22680–22689. 4

    2023

  35. [43]

    H. Ren, H. Dai, Z. Dai, M. Yang, J. Leskovec, D. Schuurmans, B. Dai, Combiner: Full attention transformer with sparse computation cost, Ad- vances in Neural Information Processing Systems 34 (2021) 22470– 22482. 4

    2021

  36. [44]

    N. Wang, G. Gan, P. Zhang, S. Zhang, J. Wei, Q. Liu, X. Jiang, Cluster- former: Neural clustering attention for efficient and effective transformer, in: Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics (V olume 1: Long Papers), 2022, pp...

    2022

  37. [45]

    Z. Ge, S. Liu, F. Wang, Z. Li, J. Sun, Yolox: Exceeding yolo series in 2021, arXiv preprint arXiv:2107.08430 (2021). 4

    2021 arXiv

  38. [46]

    M. Tan, R. Pang, Q. V . Le, Efficientdet: Scalable and efficient object de- tection, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10781–10790. 4

    2020

  39. [47]

    Xiong, H

    Y . Xiong, H. Liu, S. Gupta, B. Akin, G. Bender, Y . Wang, P.-J. Kin- dermans, M. Tan, V . Singh, B. Chen, Mobiledets: Searching for object detection architectures for mobile accelerators, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2...

    2021

  40. [48]

    B. Wu, C. Li, H. Zhang, X. Dai, P. Zhang, M. Yu, J. Wang, Y . Lin, P. Va- jda, Fbnetv5: Neural architecture search for multiple tasks in one run, arXiv preprint arXiv:2111.10007 (2021). 4

    2021 arXiv

  41. [49]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recog- nition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. 4

    2016

  42. [50]

    Z. Xia, X. Pan, S. Song, L. E. Li, G. Huang, Vision transformer with deformable attention, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4794–4803. 4

    2022

  43. [51]

    J. Yang, C. Li, P. Zhang, X. Dai, B. Xiao, L. Yuan, J. Gao, Focal self-attention for local-global interactions in vision transformers, arXiv preprint arXiv:2107.00641 (2021). 4

    2021 arXiv

  44. [52]

    Q. Fan, H. Huang, M. Chen, H. Liu, R. He, Rmt: Retentive networks meet vision transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 5641–5651. 4

    2024

  45. [53]

    J. Park, K. Kahatapitiya, D. Kim, S. Sudalairaj, Q. Fan, M. S. Ryoo, Grafting vision transformers, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2024, pp. 1145–1154. 4

    2024

  46. [54]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based lo- calization, in: Proceedings of the IEEE international conference on com- puter vision, 2017, pp. 618–626. 4

    2017

  47. [55]

    G. Yu, Q. Chang, W. Lv, C. Xu, C. Cui, W. Ji, Q. Dang, K. Deng, G. Wang, Y . Du, et al., Pp-picodet: A better real-time object detector on mobile devices. arxiv 2021, arXiv preprint arXiv:2111.00902. 6

    2021 arXiv

  48. [56]

    Kirillov, R

    A. Kirillov, R. Girshick, K. He, P. Doll ´ar, Panoptic feature pyramid net- works, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6399–6408. 6

    2019

  49. [57]

    T. Xiao, Y . Liu, B. Zhou, Y . Jiang, J. Sun, Unified perceptual parsing for scene understanding, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 418–434. 6

    2018

  50. [58]

    philike hetaereia

    I. Andronache, I. Liritzis, H. F. Jelinek, Fractal algorithms and rgb image processing in scribal and ink identification on an 1819 secret initiation manuscript to the “philike hetaereia”, Scientific Reports 13 (1) (2023)

    2023

  51. [59]

    Liritzis, I

    I. Liritzis, I. Andronache, C. Stevenson, A novel approach to document- ing water diffusion in ancient obsidian artifacts via the complexity analy- sis of microscope images, Journal of Archaeological Science 161 (2024) 105896. 7

    2024

  52. [60]

    Liritzis, K

    I. Liritzis, K. Mainzer, Z. Lavicza, K. Fenyvesi, V . Dinescu, S. Orlandi, Y . Hui, L. Teodorescou-Ciocanea, M. I. Elias, M. Cosmopoulos, et al., Easa expert group: Science, technology, engineering, mathematics in arts and culture (stemac), Proceedings of the European Academy ...

    2024

  53. [61]

    Li, Positive-incentive noise, IEEE Transactions on Neural Networks and Learning Systems (2022)

    X. Li, Positive-incentive noise, IEEE Transactions on Neural Networks and Learning Systems (2022). 7

    2022

  54. [62]

    Zhang, S

    H. Zhang, S. Huang, Y . Guo, X. Li, Variational positive-incentive noise: How noise benefits models, IEEE Transactions on Pattern Analysis and Machine Intelligence (2024). 7

    2024

  55. [63]

    Zhang, Y

    H. Zhang, Y . Xu, S. Huang, X. Li, Data augmentation of con- trastive learning is estimating positive-incentive noise, arXiv preprint arXiv:2408.09929 (2024). 7

    2024 arXiv

  56. [64]

    Huang, H

    S. Huang, H. Zhang, X. Li, Enhance vision-language alignment with noise, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39, 2025, pp. 17449–17457. 7

    2025

  57. [65]

    Y . Xu, S. Huang, H. Zhang, X. Li, Why does dropping edges usually outperform adding edges in graph contrastive learning?, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 39, 2025, pp. 21824–21832. 7

    2025

  58. [66]

    Huang, Y

    S. Huang, Y . Xu, H. Zhang, X. Li, Learn beneficial noise as graph aug- mentation, arXiv preprint arXiv:2505.19024 (2025). 7 8

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.