Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

A 2D Semantic-Aware Position Encoding for Vision Transformers

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that replacing coordinate-based position encodings with content-gated semantic position counts improves vision transformer accuracy, reaching 93.98% top-1 on CIFAR-10.

desk verdict A plausible semantic-aware PE idea undermined by a mismatch between the claimed relative-position formula and the implemented suffix-sum code. read the letter →

arxiv 2505.09466 v1 pith:LNECD6SG submitted 2025-05-14 cs.CV cs.AI

Xi Chen , Shiyang Zhou , Muqi Huang , Jiaxu Feng , Yun Xiong , Kun Zhou , Biao Yang , Yuhui Zhang
show 4 more authors
Huishuai Bao Sijia Peng Chuan Li Feng Shi
This is my paper · ORCID
classification cs.CVcs.AI
keywords positionencodingvisiontransformersemantic-awareattentionrelativecontextual2Ddecompositionimageclassificationbias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the position encodings vision transformers inherit from language models—absolute coordinates or fixed relative offsets—ignore what patches actually contain, and that this hurts accuracy on images with repeated textures, scattered objects, or shifted subjects. It proposes SaPE2, a 2D position encoding that measures a patch's position by summing content-dependent gate values along its row and column, converts those counts into learned embeddings, and turns the distance between embedding vectors into an attention bias. On CIFAR-10 and CIFAR-100 with a ViT-Small backbone, the method reports 93.98% and 72.23% top-1 accuracy, the best among the compared position encodings including 2D RoPE and CoPE+APE. If this holds, position encoding can be made sensitive to perceptual similarity rather than fixed grid coordinates, with direct consequences for translation robustness and resolution scaling.

What carries the argument

The load-bearing object is the gated suffix-position sum along each image axis. In Eq. (10), the position of patch $m$ relative to patch $i$ along the x-axis is defined as $p^x_{im}=\sum_{j\in I_{im}} g_{ij}$ with $I_{im}=\{j \mid y_i=y_j=y_m,\ x_m\le x_j\}$, where $g_{ij}=\sigma(q_i^\top k_j)$; a continuous position value is then mapped through interpolation of learnable integer embeddings, and the attention bias is the Euclidean distance between the resulting axis-wise position vectors. This construction is what carries the semantic-awareness claim: instead of fixed coordinates, the gate values decide how much each trailing patch counts toward the position, so visually related content can cluster in position space. The same mechanism is applied independently to the y-axis, and the two biases are added into the attention logits.

What would settle it

Train the same ViT-Small configuration with Eq. (10) replaced by a true pairwise relative position, e.g., $p^x_{im}=\sum_{j=i}^{m} g_{ij}$ for $m\ge i$ with a symmetric form for $m<i$, keeping everything else fixed. If accuracy does not drop, the suffix-sum definition is not contributing as a relative position; if accuracy drops, the boundary-relative count itself carries the gain. Additionally, with all gates set to 1, Eq. (10) reduces to $p^x_{im}=W-x_m+1$, independent of the query patch $i$, so checking whether the bias changes when $i$ moves while $m$ stays fixed directly tests whether a relative position is being measured.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that semantic-aware, content-dependent position representations improve vision transformer accuracy over coordinate-only encodings. SaPE2 decomposes the image into x- and y-axes; for each axis it computes a gate value $\sigma(q_i^\top k_j)$ for patch pairs sharing that coordinate, defines a position value $p^x_{im}$ as the sum of gates over the suffix from patch $m$ to the row end, interpolates learnable integer position embeddings at that continuous value, and forms an axis-specific position vector by stacking the query's dot products with those embeddings. The Euclidean distance between two patches' axis vectors provides the relative position bias added into attention, and adding the absolute position encoding (APE) on top gives the best results: 93.98% top-1 on CIFAR-10 and 72.23% on CIFAR-100. The authors interpret the attention maps as evidence that patches with similar content receive similar position biases, so the model groups foreground and background consistently.

Load-bearing premise

The method assumes that the suffix sum in Eq. (10)—the accumulated gates from the target patch to the end of its row or column—is a valid measure of the relative position between two patches; if that boundary-relative count is not a true relative position, the semantic-aware mechanism the paper describes is not what the experiments actually measure.

Editorial extensions

If this is right

  • On both CIFAR-10 and CIFAR-100, SaPE2+APE beats the vanilla ViT APE baseline (93.98 vs 87.41 and 72.23 vs 66.54), so content-based position bias is a direct accuracy lever under this setup.
  • Applying SaPE2 to the key side (SaPE2(K)+APE) outperforms the query side (93.98% vs 90.17% on CIFAR-10), indicating that enriching key representations with semantic position information is the more effective integration point.
  • Because the bias is computed as a Euclidean distance between axis-wise position vectors, arbitrary patch pairs across different rows and columns can receive a position bias, not just patches on the same row or column.
  • The complexity analysis places the bottleneck at O(N^2) pairwise bias storage and computation per axis, so scaling the method to larger images requires reducing that quadratic cost.
  • The qualitative attention visualization shows smaller SaPE2 distances among patches of the same semantic region (sky, subject, ground), supporting the claim that the encoding groups perceptually similar patches.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test implied but not run by the paper: replace the suffix-sum definition with a true positional sum between the query and target patches; if the accuracy gain persists, the boundary-relative count is not the active ingredient, and if it disappears, the paper's interpretation of Eq. (10) as a relative position needs revision.
  • Because $p^x_{im}$ does not depend on the distance between $i$ and $m$ except through the query's gate values, the encoding may be better described as learning a semantic 'distance to the row or column edge' per query rather than a pairwise relative position; this reframing could change how the method is extended to variable image widths.
  • The reported gains are on 32x32 images; the paper's stated goals of resolution and scale generalization would be testable on ImageNet at multiple input sizes, which the paper does not include.
  • If the gate-based position is recomputed per query, the position bias is asymmetric and content-dependent, so the method may also serve as a learned attention-prior mechanism independent of whether it is ultimately a position encoding.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SaPE2, a 2D semantic-aware position encoding for vision transformers. The method decomposes the 2D position bias into two 1D encodings along the x- and y-axes. For each axis, gate values are computed as sigmoid(q_i^T k_j) between patches in the same row or column, and a position value is obtained by summing these gates; the position value is converted to a continuous embedding by interpolating learnable integer position embeddings, and the final attention bias is the Euclidean distance between per-axis position vectors of two patches. Experiments on CIFAR-10 and CIFAR-100 with a ViT-Small backbone report that SaPE2+APE outperforms the compared baselines. The authors claim that the method improves translation equivariance, generalization across resolutions, and aggregation of semantically similar but spatially distant patches.

Significance. If the mechanism were as described, content-dependent 2D position biases would be a useful extension of contextual position encoding to vision. The paper includes a complexity analysis, a core implementation listing in Appendix D, and a case-study visualization, which are helpful for assessing the method. However, the central implementation does not match the stated relative-position mechanism, so the current empirical gains in Table 1, even if reproducible, do not yet support the paper's main claim. The idea may be salvageable after reimplementation of the position computation, but as written the contribution and the experimental evidence need substantial revision.

major comments (4)
  1. [Section 3, Eq. (10) and Appendix D] The quantity p^x_im is not a relative position between patches i and m. As defined, p^x_im = Σ_{j: y_i=y_j=y_m, x_m ≤ x_j} σ(q_i^T k_j), and the code line pos = gates.flip(-1).cumsum(dim=-1).flip(-1) computes exactly this suffix sum to the row end. For a fixed query row, p^x_im depends on the target column m only through the sum of gate values from column m to the row boundary, and it contains no term that counts or measures the interval between x_i and x_m. Two targets at different distances from i can produce identical position values whenever their suffix gate sums coincide, so the construction does not answer the question stated in the text, 'what is the relative position of the m-th patch from the i-th patch's perspective?' Consequently SaPE2_x[i] and the bias b^x_in in Eq. (15) are boundary-relative semantic quantities, not relative displacements. This is a load-bearing mismatch between the claimed mechanism and the implemented one; it affects the y-axis in the same way. Please replace the suffix sum by a sum over the interval between x_i and x_m (in Eq. (10), Eq. (12), and Appendix D) and rerun the experiments, or revise the conceptual claims throughout to describe a boundary-relative gated position encoding rather than a relative-position encoding.
  2. [Section 1, Section 6, and Eq. (15)] The claims of improved translation equivariance and resolution generalization are not supported by the implemented construction. Because p^x_im is anchored at the row end, translating the image content changes the suffix sums both at object boundaries and at the image boundary; the Euclidean distance between two suffix-count vectors is therefore not a translation-invariant or resolution-invariant quantity. The abstract and introduction promise 'generalization across varying image resolutions and scales' and 'improved translation equivariance,' but no experiment directly tests either property. Please provide targeted evaluations, such as shifted-crop classification and training-at-one-resolution-then-testing-at-another, or remove these claims from the contribution list.
  3. [Section 3, Eq. (9)] The gate values that define the position are computed from the same q_i^T k_j dot products that the positional bias subsequently modifies. This self-referential design makes it difficult to attribute the observed accuracy gain to positional information rather than to an additional content-dependent attention pathway that partly duplicates the original attention computation. This concern applies to CoPE as well, but since the paper proposes SaPE2 as a distinct method, please include an ablation that decorrelates the positional signal from the attention signal, for example by computing the gates with stop-gradient on q and k or by using a separate small network for the gates, and report whether the Table 1 gain survives.
  4. [Section 5.2, Table 1] The headline performance comparison is based on single training runs with no reported number of seeds, no standard deviations, and no error bars. On CIFAR-100 the difference between SaPE2+APE and CoPE+APE is only 0.13 percentage points, which is within the typical run-to-run variation for this type of training setup. Please report mean and standard deviation over at least three independent seeds, and state the full training configuration (optimizer hyperparameters, learning-rate schedule, weight decay, augmentation, and number of attention heads/layers) so that the comparison is reproducible.
minor comments (5)
  1. [Section 3, Eq. (12)] The notation m1, m2, ..., mW is introduced only after the equation; please define that m_r denotes the patch in the same row with x-coordinate r, i.e., x_{m_r} = r.
  2. [Appendix D] In the SaPE.forward code, the x-axis result is reshaped as (B, H, heads, W, H), which is only valid when W = H. The experiments use 32x32 inputs with patch size 4, so W = H = 8 and the issue is hidden, but the implementation does not support the claimed general 2D case. Please reshape to (B, H, heads, W, W) or otherwise correct the rectangular-image handling.
  3. [Section 5.2] There is a typo in the dataset name: 'CIFAR1100' should be 'CIFAR100'.
  4. [Figure 2] The figure uses inconsistent labels: 'SaPE2_y' appears alongside 'RePE2_y', and the notation b^x_in is written with a superscript x while the text defines b^x_in; please unify the symbols.
  5. [Section 3, Eq. (16)] The attention score formula uses the index m for the key patch while Eq. (15) defines the bias for patch n; using two different indices in the same derivation is confusing and should be made consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SaPE2 is an empirically validated architecture that builds on the external CoPE baseline, and the self-referential use of q^T k gates is a design choice rather than a fitted-input or self-citation loop.

full rationale

I find no circular step that meets the hard-evidence bar. The paper does not fit a parameter to a subset of the reported test data and then present that subset as a prediction: Table 1 reports standard CIFAR-10/CIFAR-100 train/test splits, and the learnable position embeddings are trained end-to-end, not solved from the test labels. The reference list contains no load-bearing self-citations: CoPE and 2D RoPE are external baselines, and the method explicitly builds on them rather than importing an author-specific uniqueness theorem. The fact that the gate values in Eq. (9) are computed from q^T k, the same dot product that appears in the attention logits of Eq. (16), makes the position bias content-dependent by design, but it is not a logical tautology: the final bias is a separately parameterized, learned nonlinear function of those logits, and the paper's contribution is an empirical architecture, not a theorem that reduces to its own assumptions. The potential mismatch between Eq. (10)'s stated interval-based relative position and the suffix cumsum shown in Appendix D is a correctness and mechanism-interpretation concern (the implementation computes a boundary-relative count rather than an interval-relative position), but it is not a circularity of the kind enumerated in the review criteria, because it does not involve fitting a target quantity to itself or importing an unverified self-citation. I therefore rate the paper as having no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three unproven modeling choices: q^T k as a semantic similarity measure, the suffix-sum position definition in Eq. (10), and the Euclidean combination of axis vectors. The learnable position embeddings and the max position index are additional fitted components. No new physical or conceptual entities are introduced.

free parameters (2)
  • learnable interpolated position embeddings e_x and e_y
    The integer position embeddings e[p] for each axis are learned end-to-end (Eq. 11) and directly determine the position bias; no pretraining or theoretical constraint is provided.
  • maximum position index M = not reported
    M bounds the interpolation table and is a hyperparameter of the method; the paper does not state the value used in the CIFAR experiments.
assumptions (4)
  • standard math Self-attention is permutation-invariant, so position encoding is necessary.
    Section 2 states this standard property of attention without proof.
  • domain assumption The dot product q_i^T k_j is a valid measure of semantic similarity between patches.
    Introduced implicitly in Eq. (9) and used throughout; the entire semantic-awareness claim rests on this being true before any position bias is applied.
  • ad hoc to paper The suffix-sum count in Eq. (10) encodes the relative position of the m-th patch from the i-th patch's perspective.
    Eq. (10) defines I_im as x_m <= x_j, i.e., the suffix to the row end, which is not an interval between i and m; the paper provides no derivation that this is a valid relative position.
  • ad hoc to paper Euclidean distance between axis-wise position vectors yields a meaningful 2D positional bias.
    Eq. (15) uses L2 distance between SaPE2 vectors; no theoretical or empirical justification beyond the accuracy tables.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A 2D Semantic-Aware Position Encoding for Vision Transformers." pith.science (2026). https://pith.science/paper/LNECD6SG

@misc{pith2026250509466,
  author       = {Pith},
  title        = {Pith review of: A 2D Semantic-Aware Position Encoding for Vision Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNECD6SG}},
  note         = {Machine review of arXiv:2505.09466}
}
abstract

Vision transformers have demonstrated significant advantages in computer vision tasks due to their ability to capture long-range dependencies and contextual relationships through self-attention. However, existing position encoding techniques, which are largely borrowed from natural language processing, fail to effectively capture semantic-aware positional relationships between image patches. Traditional approaches like absolute position encoding and relative position encoding primarily focus on 1D linear position relationship, often neglecting the semantic similarity between distant yet contextually related patches. These limitations hinder model generalization, translation equivariance, and the ability to effectively handle repetitive or structured patterns in images. In this paper, we propose 2-Dimensional Semantic-Aware Position Encoding ($\text{SaPE}^2$), a novel position encoding method with semantic awareness that dynamically adapts position representations by leveraging local content instead of fixed linear position relationship or spatial coordinates. Our method enhances the model's ability to generalize across varying image resolutions and scales, improves translation equivariance, and better aggregates features for visually similar but spatially distant patches. By integrating $\text{SaPE}^2$ into vision transformers, we bridge the gap between position encoding and perceptual similarity, thereby improving performance on computer vision tasks.

Figures

Figures reproduced from arXiv: 2505.09466 by the authors.

Figure 1
Figure 1. The overall pipeline of the vision model and a comparison between the absolute PE, relative [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of calculating the relative positional relationship between two patches which [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Self-attention mechanism with SaPE2 , the position encoding can be applied to either key or query. For simplicity, we ignore the channel dimension here. The dimensions and symbols outside the parentheses indicate operations on the x-axis, while those inside indicate operations on the y-axis. bias calculation to arbitrary pairs of patches. The n-th patch is not constrained to share the same y coordinate with the i-th… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Visualization of the attention bias calculated based on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Out-of-Length Scene Text Recognition: A Two-Axis Diagnosis and a Training-Free Fix

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A training-free chunk-and-stitch procedure turns ordinary word-level scene text recognizers into near-state-of-the-art long-text readers, after the paper shows the encoder-width axis, not decoder length, dominates the...

Reference graph

Works this paper leans on

29 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, NIPS’17, page 6000–6010, Red Hook, NY , USA, 2017. Curran Associates Inc

    work page 2017

  2. [2]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  3. [3]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021

    2021

  4. [4]

    An introduction to convolutional neural networks

    Keiron O’shea and Ryan Nash. An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458, 2015

    arXiv 2015

  5. [5]

    A survey of convolutional neural networks: analysis, applications, and prospects

    Zewen Li, Fan Liu, Wenjie Yang, Shouheng Peng, and Jun Zhou. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems, 33(12):6999–7019, 2021

    work page 2021

  6. [6]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In European conference on computer vision, pages 213–229. Springer, 2020

    2020

  7. [7]

    Convolutional sequence to sequence learning

    Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. Convolutional sequence to sequence learning. In International conference on machine learning, pages 1243–1252. PMLR, 2017

    work page 2017

  8. [8]

    Self-attention with relative position representations

    Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155, 2018

    arXiv 2018

Show all 29 references
  1. [9]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020

    2020

  2. [10]

    Roformer: Enhanced transformer with rotary position embedding

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024

    2024

  3. [11]

    Contextual position encoding: Learning to count what’s important

    Olga Golovneva, Tianlu Wang, Jason Weston, and Sainbayar Sukhbaatar. Contextual position encoding: Learning to count what’s important. arXiv preprint arXiv:2405.18719, 2024

    2024 arXiv

  4. [12]

    Rotary position embedding for vision transformer

    Byeongho Heo, Song Park, Dongyoon Han, and Sangdoo Yun. Rotary position embedding for vision transformer. In ECCV, pages 289–305. Springer, 2024

    2024

  5. [13]

    Tokens-to-token vit: Training vision transformers from scratch on imagenet

    Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Zi-Hang Jiang, Francis EH Tay, Jiashi Feng, and Shuicheng Yan. Tokens-to-token vit: Training vision transformers from scratch on imagenet. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5...

    2021

  6. [14]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347–10357. PMLR, 2021

    2021

  7. [15]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. In Proceedings of the IEEE/CVF international conference on computer vision, pa...

    2021

  8. [16]

    Rethinking spatial dimensions of vision transformers

    Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, and Seong Joon Oh. Rethinking spatial dimensions of vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11936–11945, 2021

    2021

  9. [17]

    Cvt: Introducing convolutions to vision transformers

    Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 22–31, 2021

    2021

  10. [18]

    Incorporating convolution designs into visual transformers

    Kun Yuan, Shaopeng Guo, Ziwei Liu, Aojun Zhou, Fengwei Yu, and Wei Wu. Incorporating convolution designs into visual transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 579–588, 2021

    2021

  11. [19]

    Deformable detr: Deformable transformers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv:2010.04159, 2020

    2010 arXiv

  12. [20]

    End-to-end object detection with adaptive clustering transformer

    Minghang Zheng, Peng Gao, Renrui Zhang, Kunchang Li, Xiaogang Wang, Hongsheng Li, and Hao Dong. End-to-end object detection with adaptive clustering transformer. arXiv preprint arXiv:2011.09315, 2020

    2011 arXiv

  13. [21]

    Fast convergence of detr with spatially modulated co-attention

    Peng Gao, Minghang Zheng, Xiaogang Wang, Jifeng Dai, and Hongsheng Li. Fast convergence of detr with spatially modulated co-attention. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3621–3630, 2021. 10

    2021

  14. [22]

    Exploring plain vision transformer backbones for object detection

    Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In European conference on computer vision, pages 280–296. Springer, 2022

    2022

  15. [23]

    Vision transformer adapter for dense predictions

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In The Eleventh International Conference on Learning Representations, 2022

    2022

  16. [24]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers

    Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In Proceedings of the IEEE/CVF conference on ...

    2021

  17. [25]

    Segmenter: Transformer for semantic segmentation

    Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmentation. In Proceedings of the IEEE/CVF international conference on computer vision , pages 7262–7272, 2021

    2021

  18. [26]

    Segformer: Simple and efficient design for semantic segmentation with transformers

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. Advances in neural information processing systems, 34:12077–12090, 2021

    2021

  19. [27]

    Rethinking and improving relative position encoding for vision transformer

    Kan Wu, Houwen Peng, Minghao Chen, Jianlong Fu, and Hongyang Chao. Rethinking and improving relative position encoding for vision transformer. InProceedings of the IEEE/CVF international conference on computer vision, pages 10033–10041, 2021

    2021

  20. [28]

    Lape: Layer-adaptive position embedding for vision transformers with independent layer normalization

    Runyi Yu, Zhennan Wang, Yinhuai Wang, Kehan Li, Chang Liu, Haoyi Duan, Xiangyang Ji, and Jie Chen. Lape: Layer-adaptive position embedding for vision transformers with independent layer normalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, ...

    2023

  21. [29]

    employed a hierarchical encoder to improve segmentation performance. A.3 Position Encodings in CV Previous transformer-based vision models often adopted absolute position encoding, relative position encoding, or rotary position encoding, similar to methods used in natural lang...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.