SEMA combines window attention with global token averaging, motivated by a dispersion theorem for generalized attention, and reports 0.2 to 0.7 percent top-1 accuracy gains over comparable vision Mamba and MILA models.
Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile. Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism. Code is available at https://github.com/lironui/Linear-Attention-Mechanism.
citation-role summary
background 1
citation-polarity summary
fields
cs.CV 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging
SEMA combines window attention with global token averaging, motivated by a dispersion theorem for generalized attention, and reports 0.2 to 0.7 percent top-1 accuracy gains over comparable vision Mamba and MILA models.