Pith. sign in

REVIEW 9 cited by

Visual Attention Network

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2202.09741 v5 pith:L3IVC6Z7 submitted 2022-02-20 cs.CV

Meng-Hao Guo , Cheng-Ze Lu , Zheng-Ning Liu , Ming-Ming Cheng , Shi-Min Hu This is my paper · ORCID
classification cs.CV
keywords attentionsegmentationimagesnetworkself-attentionvisionwhileadaptability
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision. (1) Treating images as 1D sequences neglects their 2D structures. (2) The quadratic complexity is too expensive for high-resolution images. (3) It only captures spatial adaptability but ignores channel adaptability. In this paper, we propose a novel linear attention named large kernel attention (LKA) to enable self-adaptive and long-range correlations in self-attention while avoiding its shortcomings. Furthermore, we present a neural network based on LKA, namely Visual Attention Network (VAN). While extremely simple, VAN surpasses similar size vision transformers(ViTs) and convolutional neural networks(CNNs) in various tasks, including image classification, object detection, semantic segmentation, panoptic segmentation, pose estimation, etc. For example, VAN-B6 achieves 87.8% accuracy on ImageNet benchmark and set new state-of-the-art performance (58.2 PQ) for panoptic segmentation. Besides, VAN-B2 surpasses Swin-T 4% mIoU (50.1 vs. 46.1) for semantic segmentation on ADE20K benchmark, 2.6% AP (48.8 vs. 46.2) for object detection on COCO dataset. It provides a novel method and a simple yet strong baseline for the community. Code is available at https://github.com/Visual-Attention-Network.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ClawEnvKit: Automatic Environment Generation for Claw-Like Agents

    cs.AI 2026-04 unverdicted novelty 6.0 of 10

    EVT improves the RMT backbone by using Euclidean-distance attention decay and 1D token grouping, achieving 86.6% top-1 on ImageNet-1K at 384×384 resolution.

  2. Rectifying Magnitude Neglect in Linear Attention

    cs.CV 2025-07 conditional novelty 6.0 of 10

    MALA adds a scale-sensitive offset to linear attention so attention sharpens with query magnitude, improving accuracy across several tasks.

  3. Norm$\times$Direction: Restoring the Missing Query Norm in Vision Linear Attention

    cs.LG 2025-06 conditional novelty 6.0 of 10

    NaLaFormer restores query-norm sensitivity in linear attention with a norm-aware power feature map and a cosine direction similarity that keeps attention scores non-negative.

  4. Text-to-Image GAN with Pretrained Representations

    cs.CV 2024-12 conditional novelty 6.0 of 10

    TIGER, a text-to-image GAN that stacks CLIP and DINO representations in the discriminator, reports FID 5.48 on COCO and 9.38 on CUB, surpassing prior GAN and some diffusion baselines.

  5. Multi-Token Enhancing for Vision Representation Learning

    cs.CV 2024-11 conditional novelty 6.0 of 10

    Training a vision transformer with multiple auxiliary tokens and distilling them into a single global token improves self-supervised representation quality with no additional inference cost.

  6. Lightweight Joint Audio-Visual Deepfake Detection via Single-Stream Multi-Modal Learning Framework

    cs.SD 2025-06 conditional novelty 5.0 of 10

    A 0.48M-parameter single-stream network with iterative audio-visual fusion outperforms larger two-stream baselines on DF-TIMIT, FakeAVCeleb, and DFDC deepfake detection benchmarks.

  7. A feature refinement module for light-weight semantic segmentation network

    cs.CV 2024-12 conditional novelty 4.0 of 10

    A feature refinement module that aggregates multi-stage features with non-local attention improves light-weight semantic segmentation, reaching 80.4% mIoU on Cityscapes at 214.82 GFLOPs.

  8. Heuristical Comparison of Vision Transformers Against Convolutional Neural Networks for Semantic Segmentation on Remote Sensing Imagery

    cs.CV 2024-11 reject novelty 4.0 of 10

    A 42.9M-parameter UNet with a weighted IoU-Dice-cross-entropy loss is compared against a 200M-parameter MaskFormer on iSAID aerial segmentation, reporting mIoU 73.4 and 82.48 respectively.

  9. MVFormer: Diversifying Feature Normalization and Token Mixing for Efficient Vision Transformers

    cs.CV 2024-11 conditional novelty 3.0 of 10

    MVFormer combines a weighted blend of three normalizations with a three-branch multiscale convolutional token mixer, achieving modest top-1 accuracy gains on ImageNet-1K and downstream vision tasks.

Pith tools