Pith. sign in

REVIEW 10 cited by

Dilated Neighborhood Attention Transformer

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2209.15001 v3 pith:CVYYHVVP submitted 2022-09-29 cs.CV cs.AIcs.LG

Ali Hassani , Humphrey Shi This is my paper
classification cs.CVcs.AIcs.LG
keywords attentionsegmentationade20kneighborhoodtransformercocodilatedglobal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Transformers are quickly becoming one of the most heavily applied deep learning architectures across modalities, domains, and tasks. In vision, on top of ongoing efforts into plain transformers, hierarchical transformers have also gained significant attention, thanks to their performance and easy integration into existing frameworks. These models typically employ localized attention mechanisms, such as the sliding-window Neighborhood Attention (NA) or Swin Transformer's Shifted Window Self Attention. While effective at reducing self attention's quadratic complexity, local attention weakens two of the most desirable properties of self attention: long range inter-dependency modeling, and global receptive field. In this paper, we introduce Dilated Neighborhood Attention (DiNA), a natural, flexible and efficient extension to NA that can capture more global context and expand receptive fields exponentially at no additional cost. NA's local attention and DiNA's sparse global attention complement each other, and therefore we introduce Dilated Neighborhood Attention Transformer (DiNAT), a new hierarchical vision transformer built upon both. DiNAT variants enjoy significant improvements over strong baselines such as NAT, Swin, and ConvNeXt. Our large model is faster and ahead of its Swin counterpart by 1.6% box AP in COCO object detection, 1.4% mask AP in COCO instance segmentation, and 1.4% mIoU in ADE20K semantic segmentation. Paired with new frameworks, our large variant is the new state of the art panoptic segmentation model on COCO (58.5 PQ) and ADE20K (49.4 PQ), and instance segmentation model on Cityscapes (45.1 AP) and ADE20K (35.4 AP) (no extra data). It also matches the state of the art specialized semantic segmentation models on ADE20K (58.1 mIoU), and ranks second on Cityscapes (84.5 mIoU) (no extra data).

Discussion (0). Sign in to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SurGe: Improved Surface Geometry in Point Maps

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    SurGe improves local surface geometry in feedforward point maps via gradient matching loss and Neighborhood Attention Decoder, topping average rank on eight zero-shot monocular geometry benchmarks for global AbsRel wh...

  2. Concurrence of Symmetry Breaking and Nonlocality Phase Transitions in Diffusion Models

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    Symmetry breaking and nonlocality phase transitions occur nearly simultaneously during diffusion model generation in modern transformers.

  3. iFAN: Inference-Aware Learning for Plain Mask Transformers

    cs.CV 2026-08 conditional novelty 6.0 of 10

    iFAN improves query-based mask transformers by training a mask-quality head for score ranking and distilling better intermediate-layer predictions into the final layer, yielding consistent gains at no inference-time cost.

  4. RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

    cs.LG 2026-02 conditional novelty 6.0 of 10

    A single dense-pretrained RAT+ model can be adapted with 1B tokens to run at dilation sizes D=2..128, matching dense accuracy at D=16 and losing only 1-3 points at D=64 on reasoning and long-context benchmarks, though...

  5. RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

    cs.LG 2026-02 conditional novelty 6.0 of 10

    RAT+ pretrains a single dense recurrent-augmented attention model that supports flexible dilated sparse inference after short adaptation, matching dense accuracy at moderate dilation and losing only 1-3 points at high...

  6. RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference

    cs.LG 2026-02 unverdicted novelty 6.0 of 10

    RAT+ pretrains a dense recurrent-augmented attention model once and enables flexible switching to dilated or hybrid sparse attention at inference after short adaptation, with small accuracy loss at high dilation factors.

  7. Gaussian Belief Propagation Network for Depth Completion

    cs.CV 2026-01 conditional novelty 6.0 of 10

    Depth completion via a learned Markov random field solved with Gaussian belief propagation reports state-of-the-art RMSE on NYUv2 and best iRMSE on KITTI, plus better robustness at extreme sparsity.

  8. Exploring Mutual Cross-Modal Attention for Context-Aware Human Affordance Generation

    cs.CV 2025-02 unverdicted novelty 6.0 of 10

    A disentangled pipeline using mutual cross-attention and multiple VAEs improves context-aware human pose prediction in 2D scenes over prior baselines.

  9. Flex Attention: A Programming Model for Generating Optimized Attention Kernels

    cs.LG 2024-12 unverdicted novelty 6.0 of 10

    FlexAttention supplies a compiler-driven interface that expresses common attention variants in a few lines of PyTorch and emits optimized kernels whose speed matches hand-written implementations.

  10. ScratNet: A Swin-Based Multi-Scale Dilated Network with Precision Refinement for Semiconductor Scratch Segmentation

    cs.CV 2026-07 conditional novelty 5.5 of 10

    A modified Swin-Base backbone with MDA+SIM+PR decoder modules yields higher IoU/Dice and tighter boundaries than FCN/FPN/UPerNet on semiconductor scratch segmentation.

Pith tools