RECORD
Separable Self and Mixed Attention Transformers for Efficient Object Tracking
Computer Vision and Pattern Recognition · Submitted 7 Sep 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Simulations show that our Separable Self and Mixed Attention-based Tracker, SMAT, surpasses the performance of related lightweight trackers on GOT10k, TrackingNet, LaSOT, NfS30, UAV123, and AVisT datasets, while running at 37 fps on CPU, 158 fps on GPU, and having 3.8M parameters.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
The deployment of transformers for visual object tracking has shown state-of-the-art results on several benchmarks. However, the transformer-based models are under-utilized for Siamese lightweight tracking due to the computational complexity of their attention blocks. This paper proposes an efficient self and mixed attention transformer-based architecture for lightweight tracking. The proposed backbone utilizes the separable mixed attention transformers to fuse the template and search regions during feature extraction to generate superior feature encoding. Our prediction head performs global contextual modeling of the encoded features by leveraging efficient self-attention blocks for robust target state estimation. With these contributions, the proposed lightweight tracker deploys a transformer-based backbone and head module concurrently for the first time. Our ablation study testifies to the effectiveness of the proposed combination of backbone and head modules. Simulations show that our Separable Self and Mixed Attention-based Tracker, SMAT, surpasses the performance of related lightweight trackers on GOT10k, TrackingNet, LaSOT, NfS30, UAV123, and AVisT datasets, while running at 37 fps on CPU, 158 fps on GPU, and having 3.8M parameters. For example, it significantly surpasses the closely related trackers E.T.Track and MixFormerV2-S on GOT10k-test by a margin of 7.9% and 5.8%, respectively, in the AO metric. The tracker code and model is available at https://github.com/goutamyg/SMAT
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | The deployment of transformers for visual object tracking has shown state-of-the-art results on several benchmarks. | background 1.00 | 0.07 | 0.03 | asserted 1.00 |
| s1 | However, the transformer-based models are under-utilized for Siamese lightweight tracking due to the computational complexity of their attention blocks. | gap 0.70 | 0.10 | 0.02 | asserted 1.00 |
| s2 | This paper proposes an efficient self and mixed attention transformer-based architecture for lightweight tracking. | contribution 0.99 | 0.84 | 0.10 | asserted 0.99 |
| s3 | The proposed backbone utilizes the separable mixed attention transformers to fuse the template and search regions during feature extraction to generate superior feature encoding. | method 0.97 | 0.83 | 0.06 | asserted 1.00 |
| s4 | Our prediction head performs global contextual modeling of the encoded features by leveraging efficient self-attention blocks for robust target state estimation. | method 0.97 | 0.85 | 0.05 | asserted 1.00 |
| s5 | With these contributions, the proposed lightweight tracker deploys a transformer-based backbone and head module concurrently for the first time. | contribution 0.99 | 0.83 | 0.10 | asserted 1.00 |
| s6 | Our ablation study testifies to the effectiveness of the proposed combination of backbone and head modules. | result 0.67 | 0.95 | 0.08 | asserted 1.00 |
| s7 | Simulations show that our Separable Self and Mixed Attention-based Tracker, SMAT, surpasses the performance of related lightweight trackers on GOT10k, TrackingNet, LaSOT, NfS30, UAV123, and AVisT datasets, while running at 37 fps on CPU, 158 fps on GPU, and having 3.8M parameters. | result 1.00 | 0.97 | 0.74 | asserted 1.00 |
| s8 | For example, it significantly surpasses the closely related trackers E.T.Track and MixFormerV2-S on GOT10k-test by a margin of 7.9% and 5.8%, respectively, in the AO metric. | result 1.00 | 0.95 | 0.16 | asserted 1.00 |
| s9 | The tracker code and model is available at https://github.com/goutamyg/SMAT | contribution 0.73 | 0.54 | 0.03 | asserted 0.96 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
The deployment of transformers for visual object tracking has shown state-of-the-art results on several benchmarks. However, the transformer-based models are under-utilized for Siamese lightweight tracking due to the computational complexity of their attention blocks. This paper proposes an efficient self and mixed attention transformer-based architecture for lightweight tracking. The proposed backbone utilizes the separable mixed attention transformers to fuse the template and search regions during feature extraction to generate superior feature encoding. Our prediction head performs global contextual modeling of the encoded features by leveraging efficient self-attention blocks for robust target state estimation. With these contributions, the proposed lightweight tracker deploys a transformer-based backbone and head module concurrently for the first time. Our ablation study testifies to the effectiveness of the proposed combination of backbone and head modules. Simulations show that our Separable Self and Mixed Attention-based Tracker, SMAT, surpasses the performance of related lightweight trackers on GOT10k, TrackingNet, LaSOT, NfS30, UAV123, and AVisT datasets, while running at 37 fps on CPU, 158 fps on GPU, and having 3.8M parameters. For example, it significantly surpasses the closely related trackers E.T.Track and MixFormerV2-S on GOT10k-test by a margin of 7.9% and 5.8%, respectively, in the AO metric. The tracker code and model is available at https://github.com/goutamyg/SMAT
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 4 sentences without a confident label
Compares with 1 paper
-
We compare the results of the proposed SMAT against the related lightweight trackers: LightTrack [ref] , Stark-Lightning [ref] , FEAR-XS [ref] , HCAT [ref] , E.T.Track [ref] , and MixFormerV2-S [ref] , evaluated using the pretrained models provided by their authors.
MixFormerV2: Efficient Fully Transformer Tracking · Comparison to the Related Work · 0.97
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.